Digest, Issue 2: Spring 2017

Integrating Rigor and Relevance in Research: Three Questions

Read Vivian Tseng And Adam Gamoran’s “how Social Science Research Can Improve Youth Outcomes In The Real World“

By calling for researchers to bring highest level of scientific rigor to research questions that have the most importance in policy and practice, Vivian Tseng and Adam Gamoran lay out a bold action agenda for researchers and institutions alike.1

There are several points in this agenda that I find particularly resonant. First, the authors argue that theory is not the sole province of researchers. Practitioners and policymakers, in fact, have their own theories, which should have a place alongside scientific theories as bases for relevant research. I would also add that youth have their own valid theories about their development, which we should also consider as we form research questions. For instance, how could we not base theories regarding the development of understudied populations on their own meaning making and experiences? Second, Tseng and Gamoran remind us that theory is not the same as theoretical talk. The style of theoretical discourse can affect the communication and impact of research on practice and policy; overly abstract jargon can make both research much harder to communicate to practitioners and policymakers. A final point that Tseng and Gamoran highlight is that rigor is not limited only to experimental causal research or quantitative research. Understanding the developmental patterns and contexts of young people requires a much larger set of research methods than even those that currently exist in the social, health, educational, evaluation, environmental, and biological sciences.

Understanding the developmental patterns and contexts of young people requires a much larger set of research methods than even those that currently exist…

Building off of these points, I’d like to suggest three questions that we should keep in mind when considering how to bring rigor to relevant questions: To whom is the research relevant? How does relevance shape the planning and conduct of the research? And, although the William T. Grant Foundation’s focus is on youth in the United States, how might rigorous, relevant research contribute to improved outcomes for the world’s youth as a whole?

Relevance for Whom?

Not all practitioners, policymakers, or youth-serving organizations enjoy equal standing: Some have more resources and influence, and some are more marginalized. If we intend to produce relevant work that is also impactful, we should pay particular attention to those decision makers and organizations that have fewer advantages. The tradition of research in culture and human development (e.g. Whiting & Whiting, 1975) has often focused on marginalized or non-Western populations, as well as those without a prominent voice in policy and practice.

What might relevant, rigorous research look like for those shut out of powerful policy or practitioner positions? In recent years, for example, my colleagues and I have conducted research on unauthorized populations in the United States; refugee youth; and research on sexual-minority youth of color. Such work can benefit from communication and dissemination to organizations devoted to serving these populations—organizations that sometimes struggle for funding due to their marginal status in their sectors.

Rigor and Relevance: Thinking Ahead in Research Planning

A great deal of advance planning is required to achieve a combination of relevance and rigor in developmental science. Small-scale intervention research, for example, may need to engage the perspectives of at-scale organizations and stakeholders from the beginning. Dodge argues that even initial pilot implementation research can benefit from integration of stakeholders who may potentially engage in larger-scale implementation. The concerns and puzzles that these stakeholders experience can productively influence the initial implementation of both interventions and associated research.

Basic research on youth development, similarly, may need to engage practitioners early on to understand how the research might be applicable in real-world settings. That is, the planning of basic research can engage the opinions of those studied (e.g., youth and their families); those with whom those youth come in contact in their daily routines (e.g., peers, teachers, service providers, etc.); and the larger contexts within which youth reside (e.g., neighborhood and policy actors). Such dialogue may lead to revision of basic research questions to consider aspects of cultural diversity, context, or political or power relations.

Research of Relevance for Global Youth

Research on youth conducted in the United States rarely addresses the needs of youth outside the U.S. Yet, in the long run, the outcomes of youth in the United States are completely and inextricably intertwined with those of the other 95 percent of the world’s youth. This fact is most powerfully demonstrated by the 2015-2030 global goals for sustainable development (the Sustainable Development Goals). These 17 goals and their associated targets, though stated as a policy agenda, also constitute an implied research agenda commensurate to the needs of 100 percent of youth on the planet, including those in the United States. The universality of the Sustainable Development Goals (unlike their predecessor, the Millennium Development Goals, which focused on the world’s most vulnerable) incorporates attention to the developmental potential of all U.S. youth. All youth researchers, whether in the U.S. or elsewhere, should be familiar with these goals, which represent a global consensus ratified by all U.N. member nations with input from researchers from across the globe.

The vast majority of the world’s research in prestigious research journals comes from the United States and other rich countries, and addresses the needs of youth from those countries. How are we to leverage the power of rigorous research to inform the developmental potential of the rest of the world’s youth? The rights to thrive and contribute to society of the 250 million children currently at risk due to stunting or extreme poverty (Black et al., 2016)? The educational prospects of the 57 million school-age children currently lacking access to education (half due to conflict and displacement)? Understanding the relevance of the “other 95%” to research on the 5% will expand the scope of rigorous research for the benefit of all.


Without adequate understanding of and investment in the developmental potential of the world’s youth, our survival as a species is at risk. The next several decades will be critical for the ability of the next generation to safeguard our planet’s future. This is roughly the time span of 8-10 five-year research grants—a sobering time frame, but one that highlights the urgency of bringing the highest levels of rigor to questions of relevance.

Footnotes
  1. For full citations and a complete reference list, download the PDF of this essay.
Downloads

In this issue

As we approach policy changes that present important opportunities for researchers, I would like to share some “meditations” about language, as well as the worlds of education policy and practice, which I have repeated to myself over the ...
Meditations on Language and Learning: ESSA in Policy and Practice
Digest, Issue 2: Spring 2017
Strengthening practice and policy in state and local education systems will be an essential part of reducing inequality for young people in the coming years. And a fundamental piece of the effort to raise student outcomes will be building and ...
Toward New Research and New Data
By moving accountability systems from the federal to the state level and highlighting the importance of English learner progress in accountability, ESSA provides an opportunity to develop state accountability systems that better support the learning of ELs. Assessments and data ...
Learning English in the Era of the Every Student Succeeds Act
We recognize that no single effort will be transformative, but we hope that our collective efforts as researchers, research funders, universities, and professional associations can support research that, over the long term, improves the lives of young people.
Promoting Research That Is Both Rigorous and Relevant
Digest, Issue 2: Spring 2017
Put simply, the dichotomy of rigor versus relevance is false. There is no inevitable trade-off between producing rigorous research and producing research with relevance for the real world. Researchers who want their work to matter in policy and practice should ...
Bringing Rigor to Relevant Questions

More Digest Issues

The Digest

Issue 9: Winter 2023-24

The ninth issue of the Digest features an update on the Institutional Challenge Grant program five years after its launch, with insights from a Foundation stock-taking effort and reflections from two grantee teams. We also share new thinking on the use of research evidence, including ways for researchers to leverage existing findings to bolster studies of effective strategies to improve research use.
The Digest

Issue 8: Winter 2022-23

The eighth issue of the Digest features insights about where new research might help identify ways to reduce inequality in youth outcomes after COVID-19, as well as how the fields of implementation science and research on research use might learn from each other in ways that yield more transformative research in years ahead.
The Digest

Issue 7: Winter 2021-22

The seventh issue of the Digest includes fresh insights from program staff and grantees on how researchers can successfully confront the challenges of studying causal mechanisms for reducing inequality, as well as how the global community of researchers and funders focused on improving the use of research evidence can continue to break new ground in the years ahead.
The Digest

Issue 6: Winter 2020-21

Essays in this issue of the Digest focus on the importance of looking beyond individual action and customary metrics in research on reducing inequality, as well as how strengths-based, race-conscious research can be produced and used to uplift communities of color.
The Digest

Issue 5: Winter 2019-20

The fifth issue of the William T. Grant Foundation Digest features insights from program staff and grantees on the importance of looking beneath the surface to consider the underlying factors that create and shape challenges that social scientists seek to address, whether they be related to reducing inequality in youth outcomes or improving the use of research evidence in policy and practice.
The Digest

Issue 4: Winter 2018/19

The fourth issue of the William T. Grant Foundation Digest features insights that may point the way toward a more nuanced understanding of evidence use and inspire new and more wide-ranging examinations of ways to reduce inequality in youth outcomes.
The Digest

Issue 3: Winter 2017/18

The third issue of the William T. Grant Foundation Digest features insights on how research on ability tracking can inform studies to improve the outcomes of English learners, as well as how researchers and school districts can partner to build learning systems based on research evidence.
The Digest

Issue 1: Summer 2016

The introductory issue of the William T. Grant Foundation Digest features essays and commentary on the value of qualitative and mixed-methods research in reducing inequality and the potential for researcher access to big data to yield useful research evidence.

Subscribe for Updates