Digest, Issue 2: Spring 2017

Promoting Research That Is Both Rigorous and Relevant

Given the persistence of the rigor versus relevance dichotomy, researchers engaged in highly relevant research bemoan the lack of rewards for their work in the university setting.1Bogenschneider and Corbett put it aptly:

Presidents and provosts in the academy issue hortatory statements encouraging faculty to engage in public service. University administrators assert that organizing, synthesizing, and transmitting knowledge for the public good is a legitimate academic endeavor and should be so rewarded. At the end of the day, no faculty member at a research university is fooled. At best, public service might well rank even lower than undergraduate teaching in the hierarchy of valued activities at top research universities. You can do it if it is done with the effortlessness of Gods, but don’t spend too much time at it or intimate that you actually enjoy the experience. If you do, your dedication to scholarship will be questioned.

Most often, universities define research contributions narrowly and attribute work with real-world benefits to “service” rather than “research.” As a result, much of the best applied social science is generated by private or nonprofit research organizations rather than by universities. However, universities stand much to gain from enhancing the practical relevance of their scholarship. Responsiveness to public needs is an essential element in the mission of most state universities, and many private universities are increasingly challenged to be more responsive to the communities in which they reside. Recently David Leebron, the president of Rice University, an esteemed private institution in Houston, spoke at the founding of the National Network of Education Research-Practice Partnerships, an organization based at Rice University that supports collaboration between research institutions and state and local education agencies. By his presence, President Leebron emphasized the importance of this highly relevant mission, and, by his words, this university leader emphasized the value of partnerships between scholars and educators in applying research findings to school improvement. His commitment to rigor and relevance has had a tangible payoff, as the Kinder Foundation has just awarded a grant of $10.7 million to support the Houston Education Research Consortium, a research-practice partnership housed at the Kinder Institute for Urban Research at Rice University.

Universities can rethink how they evaluate the research contributions of faculty who engage in research with practical value by assessing impact differently than the usual way.

To encourage more relevant research, universities will need to broaden their criteria for judging the significance of faculty research contributions. At the same time, researchers will need to maintain rigor in order to attain value within the university reward structure. Policy and practice leaders such as school district officials and child welfare officers can help researchers develop a relevant research agenda and identify important questions, but researchers will need to bring to bear the ideas and tools of the social sciences to respond to those questions. For the research to be rigorous and impactful, and thus to merit esteem in the university setting, it must be:

  • theoretical: informed by social science ideas that provide a framework for responding to the research question;
  • systematic: involving data collection and analysis that meet methodological standards and are well aligned with the question; able to offer the potential for generalizability beyond the local setting, either through comparative designs, scaling up, or contributions to theory that can be tested elsewhere; and able to contribute to practical solutions by building, testing, improving, or furthering understanding of approaches to improving outcomes for youth.

Universities can rethink how they evaluate the research contributions of faculty who engage in research with practical value by assessing impact differently than the usual way. Typically, research impact is assessed with quantitative measures such as citation counts in academic publications, and with qualitative measures such as statements from research experts who can recount the contributions of the work. For research intended to impact policy or practice, universities may wish to assess impact by considering the use of research in those contexts. The use of research may be conceptual, shaping ideas and the way problems or solutions are defined; or it can be instrumental, influencing particular policy or practice decisions. Oftentimes research does not lead to an immediate change in policy or practice, but it has nevertheless been considered and weighed in the decision process (evidence-informed decision making). Like contributions to a research literature, contributions to practice or policy settings can be measured quantitatively, for example, by counting mentions of the research in legislative hearings or news media, or qualitatively, by seeking letters from impartial experts who can speak to its impact in policy, practice, or broader public discourse. Thus, contributions to local, state, and national debates over policy decisions may be regarded as demonstrations of impact, complementary to the appearance of research findings in academic journals or books. These considerations speak not just to the ways university faculty members provide service to their communities, but to the importance and value of the research they conduct.

Professional associations often develop policies for evaluating research in their respective fields, and they could play a leadership role in helping universities understand how to evaluate research that is relevant as well as rigorous.

While some universities have already moved in this direction, an impetus for further advancement could come from professional associations within the social sciences, such as the American Psychological Association, the American Sociological Association, the Society for Research in Child Development, the American Educational Research Association, and the like. These organizations often develop policies for evaluating research in their respective fields, and they could play a leadership role in helping universities understand how to evaluate research that is relevant as well as rigorous. For example, Michael Burawoy, in his presidential address to the American Sociological Association, called for “public sociology,” arguing that sociology was compelled to engage with a broader public beyond the academy about the social issues that form the core content of sociology such as inequality and power. Burawoy distinguished between public sociology and “professional sociology,” the ordinary activities of the academic discipline, accepting that each has its value, and stating that public sociology must rest on a strong foundation of professional sociology. Burawoy did not, however, offer suggestions for how universities could strengthen their support for public sociology. An early critique of the notion of public sociology argued that it may fail in part because professional sociologists lacked incentives to pursue it and, unlike professional sociology, public sociology lacked “tangible measures of success.”

In another example, the American Educational Research Association has considered the question of how “engaged scholarship” should be evaluated as contributions to research in postsecondary education, and has dedicated its centennial anniversary conference to “public scholarship.” Engaged scholarship rests on principles of reciprocity, respect for mutual expertise, and research for the public good; it “is shaped by theoretical and practical understandings of community-based problems, and responds to problems of educational policy and practice.” AERA challenges departments in schools of education to identify what constitutes engaged scholarship and how to identify its quality of impact—an essential first step toward fostering the production of relevant research.

Recently, our Foundation launched the Institutional Challenge Grant, an annual award that will provide funding for a research institution to strengthen its engagement with policy and practice.

Universities, professional associations, and research funders have the opportunity to encourage and facilitate efforts to bring rigor to relevant questions. Recently, our Foundation launched the Institutional Challenge Grant, an annual award that will provide funding for a research institution to strengthen its engagement with policy and practice. Through this grant program, we hope to support leaders who will boldly tackle the challenge of creating research cultures that will bring the highest levels of rigor to tackle the most important social problems in our communities. The grant encourages research institutions to partner with state and local government agencies and local youth-serving non-profit organizations to work together to construct meaningful research agendas. It challenges leaders at research institutions to think more about how to incentivize researchers to engage in this work, how to evaluate the work in terms of both rigor and relevance, and how to build the capacities of agency partners to use research evidence.


Many researchers perceive they will need to sacrifice rigor if their research is to take on greater relevance. Our view is that it is not a zero sum game; rigor and relevance can be complementary. Indeed, one might argue that there is a moral imperative to bring as much rigor as we can muster to the most pressing social problems. If policymakers and practitioners are advised to heed rigorous research, then researchers must rise to the challenge of providing credible findings on policy and practice questions. It would be a shame if those who make decisions about public services are left to rely on research that is too often either of questionable credibility or of little relevance to their work. They should not have to choose between rigorous versus relevant research.

We do not have all the answers to the questions we have raised, but as a research funder, we also have a role to play. Over the years, the William T. Grant Foundation has frequently emphasized the importance of linking research, policy, and practice. For years, we’ve focused research funding on understanding and improving the use of research evidence; we’ve also supported a vibrant community of research-practice partnerships, and have now launched a program to encourage research institutions to create the incentives and structures to support rigorous and relevant research. We hope that leading research institutions as well as professional associations will join us in taking up these challenges. We recognize that no single effort will be transformative, but we hope that our collective efforts as researchers, research funders, universities, and professional associations can support research that, over the long term, improves the lives of young people. 

  1. For full citations and a complete reference list, download the PDF of this essay.

In this issue

As we approach policy changes that present important opportunities for researchers, I would like to share some “meditations” about language, as well as the worlds of education policy and practice, which I have repeated to myself over the ...
Meditations on Language and Learning: ESSA in Policy and Practice
How can research to improve the lives of youth in the United States also benefit the other 95 percent of the world’s youth? Questions like this highlight the urgency of bringing the highest levels of rigor to relevant research.
Integrating Rigor and Relevance in Research: Three Questions
Digest, Issue 2: Spring 2017
Strengthening practice and policy in state and local education systems will be an essential part of reducing inequality for young people in the coming years. And a fundamental piece of the effort to raise student outcomes will be building and ...
Toward New Research and New Data
By moving accountability systems from the federal to the state level and highlighting the importance of English learner progress in accountability, ESSA provides an opportunity to develop state accountability systems that better support the learning of ELs. Assessments and data ...
Learning English in the Era of the Every Student Succeeds Act
Digest, Issue 2: Spring 2017
Put simply, the dichotomy of rigor versus relevance is false. There is no inevitable trade-off between producing rigorous research and producing research with relevance for the real world. Researchers who want their work to matter in policy and practice should ...
Bringing Rigor to Relevant Questions

More Digest Issues

The Digest

Issue 9: Winter 2023-24

The ninth issue of the Digest features an update on the Institutional Challenge Grant program five years after its launch, with insights from a Foundation stock-taking effort and reflections from two grantee teams. We also share new thinking on the use of research evidence, including ways for researchers to leverage existing findings to bolster studies of effective strategies to improve research use.
The Digest

Issue 8: Winter 2022-23

The eighth issue of the Digest features insights about where new research might help identify ways to reduce inequality in youth outcomes after COVID-19, as well as how the fields of implementation science and research on research use might learn from each other in ways that yield more transformative research in years ahead.
The Digest

Issue 7: Winter 2021-22

The seventh issue of the Digest includes fresh insights from program staff and grantees on how researchers can successfully confront the challenges of studying causal mechanisms for reducing inequality, as well as how the global community of researchers and funders focused on improving the use of research evidence can continue to break new ground in the years ahead.
The Digest

Issue 6: Winter 2020-21

Essays in this issue of the Digest focus on the importance of looking beyond individual action and customary metrics in research on reducing inequality, as well as how strengths-based, race-conscious research can be produced and used to uplift communities of color.
The Digest

Issue 5: Winter 2019-20

The fifth issue of the William T. Grant Foundation Digest features insights from program staff and grantees on the importance of looking beneath the surface to consider the underlying factors that create and shape challenges that social scientists seek to address, whether they be related to reducing inequality in youth outcomes or improving the use of research evidence in policy and practice.
The Digest

Issue 4: Winter 2018/19

The fourth issue of the William T. Grant Foundation Digest features insights that may point the way toward a more nuanced understanding of evidence use and inspire new and more wide-ranging examinations of ways to reduce inequality in youth outcomes.
The Digest

Issue 3: Winter 2017/18

The third issue of the William T. Grant Foundation Digest features insights on how research on ability tracking can inform studies to improve the outcomes of English learners, as well as how researchers and school districts can partner to build learning systems based on research evidence.
The Digest

Issue 1: Summer 2016

The introductory issue of the William T. Grant Foundation Digest features essays and commentary on the value of qualitative and mixed-methods research in reducing inequality and the potential for researcher access to big data to yield useful research evidence.

Subscribe for Updates