Digest, Issue 7: Winter 2021-22

Contending with Complexity: How Two Grantees are Approaching Studies of Mechanisms

Since 2014, the William T. Grant Foundation has called for research to reduce inequality in youth outcomes. While taking stock of grants funded during the first five years of this initiative, staff noted that information on mechanisms through which inequality is produced and mitigated is often lacking. In a new essay for the Foundation, Margarita Alegría and Isabel Shaheen O’Malley explore some of the reasons that studies of causal mechanisms are rare, and offer approaches that may allow researchers to explore mechanisms more fully, despite the constraints.1

One key insight from their essay is that the mechanisms that cause inequality need not be the same as the mechanisms through which inequality is reduced. This means that studies that seek to understand the sources of inequality may not suffice to identify ways to reduce inequality, which is the Foundation’s primary interest. Consequently, at least from the standpoint of the Foundation, a focus on mechanisms to reduce inequality is essential.

Alegría and O’Malley explain that studies of mechanisms can be difficult “because of the conceptual issues involved, the complex role of the environment in shaping outcomes, and the confounding factors present in complex systems.” What’s more, the high cost of sufficiently powering studies of causal mechanisms presents its own set of challenges for researchers when they seek funding for their work.

In light of these obstacles, the authors outline strategies to help researchers pursue studies of mechanisms with confidence. While much research on inequality is incremental, the extent to which a study is innovative and can move the needle is an “increasingly important consideration for funders,” they write, and can be an especially apt strategy for exploratory or developmental studies. Similarly, incorporating rigorous qualitative and mixed methods can be advantageous in “identifying potential causal factors and processes, generating and refining conceptual models and hypotheses, and explaining relationships among factors observed in quantitative studies.”

Here, I talk to two grantees whose work illustrates these respective strategies in action. Former William T. Grant Scholar Mesmin Destin is an associate professor at Northwestern University in the Department of Psychology and the School of Education & Social Policy. Rob Smith is a research grantee and professor of sociology, immigration studies, and public affairs at the School of Public Affairs at Baruch College and the CUNY Graduate Center. Our conversation not only shows how researchers are confronting the challenges of studying mechanisms on the ground, it also reflects the Foundation’s openness to a variety of methodological approaches to understanding mechanisms. Destin uses quantitative methods to test hypotheses and Smith deploys qualitative methods toward interpretive ends, but both have delivered rich insights about mechanisms for reducing inequality in their work.

In their essay, Alegría and O’Malley write that “the mechanisms that contribute to a specific disparity are not necessarily the same mechanisms that are capable of, or best suited for, addressing it.” Does this hold true for your work?

Mesmin Destin: Absolutely. I do research related to educational and health disparities, but mostly education with implications for health. The idea of differing mechanisms for what is the cause versus what might reduce inequality is certainly the case. A way of thinking about mechanisms that’s been important for my work is just acknowledging that we are studying outcomes that are certainly multiply determined. Complex paths have led to disparities, including historical determinants, policies, unequal access, and so on. It’s also important to how we understand manifestations of individual, institutional, structural discrimination—all these different forms of exclusion and marginalization that are feeding upon each other over time.
Given how complex this reality is, we often start with how people experience and navigate inequalities. And starting there has been valuable for us, but we can’t end there. It’s important to address the proximal manifestations in addition to at least acknowledging and sometimes addressing those deep-rooted core determinants.

Rob Smith: I think part of the problem is that, as social scientists, we’re so trained to look for the causes of things, but we’re not trained to figure out how to fix them. A lot of times we can end up concluding that without major structural change we won’t be able to fix anything. But that’s not true in most cases. Any analysis about inequality probably illuminates ways to undo that inequality or disrupt it. It doesn’t have to fix everything, but to disrupt it and reduce it is very important.

The authors note that studies assessing the full causal chain, from intervention to mechanisms to improved outcomes, are often impossible because of feasibility issues or individual agencies’ funding limits. Given the complexity of the problems you’re studying, how do you navigate these constraints?

MD: For me it’s been compiling resources from multiple sources when possible and just growing piece by piece. I’ve found that the studies build upon each other, and each study informs the next. The reason it contributes to a more cohesive picture is because I try to have some sort of deliberative process in moving from one study to the next or in pitching duplication of implications of what is the next best step to take. Some things work well, and some things don’t really pan out. You keep following where you’re learning most and adding to the picture over time.

RS: A good example is my Foundation-supported study on the Deferred Action for Childhood Arrivals (DACA) executive order. This kind of long-term, fieldwork based, ethnographic research takes time. Developing cases and relationships to see what mechanisms might be causing inequality to increase or decrease takes time. When we received our grant in 2018, we had already been in the field for a few years, supported by two other funders, looking at how DACA would affect school versus work decisions and intra-family processes. With our WTG grant, we drew on this prior research to investigate the effects of causal mechanisms that had emerged in the initial findings.

Rob, as a qualitative researcher, in what ways would you say that qualitative methods lend themselves to research on causal mechanisms, particularly the long-term, accumulative kind of work you’ve mentioned?

RS: In case-oriented research you can see the intermediate steps—that there’s more than one mechanism. In quantitative work, there’s usually a treatment and you are trying to estimate the effect. Here, you are tracing processes through cases, and you can also trace the processes across cases. In the course of the research, you discover mechanisms that are not the primary mechanisms, but they are the proximal, and sometimes, the most powerful mechanisms. Building your study over the long term can be a feasible way to help you gain analytical leverage for understanding these mechanisms. For instance, we now have data on some participants dating from 2015, a time span that includes pre- and post-DACA data on how legal status affects their lives.

I get deeply involved in these cases, which also gives me understanding of how these processes work. I can show that in places that are more inclusive of immigrants, where potential DACA applicants knew more about DACA, they got more help from the local school in applying, as well as more legal help, and they were also less likely to be racially profiled when driving. But in more exclusionary places, drivers that “look Mexican” were more likely to get pulled over and, if they did not have a license, get sent to ICE for deportation. All for a traffic ticket, even if they had children. In more exclusionary places, there was also less help in applying for DACA, so more eligible people did not apply and therefore could not get driver’s licenses. This was before 2019, when New York passed the Greenlight law. Hence, DACA and secondary impacts of it, like getting a driver’s license, reduced inequality more in inclusionary places than exclusionary places. This suggests that how local police and institutions act toward immigrants matters a lot in the extent to which interventions to reduce inequality can actually reduce it.

Good case-oriented research always looks at all the available data. This includes census data, public data, social media postings—whatever is relevant to analyzing the structures and context within which the processes people are telling you about are happening. I think, ultimately, you can see more of the causal chain in qualitative research, you just can’t generalize from your findings as much.

Mesmin, your work on social mobility is innovative in that it draws from disciplines outside of your own and offers a novel framework linking identity and health. How do you begin to incorporate outside concepts in your research, particularly as it relates to causal mechanisms?

MD: There are abundant incentives for narrow thinking and narrow studies. But what helped me to think a little more broadly and to think about different levels of mechanisms were, first, devoting time to read more broadly in a deep way to understand some of the complex and innovative ideas in a field outside of mine, like psychoneuroimmunology. Second, I felt really lucky to collaborate and talk with people with deep expertise that expands in these types of ways. I’ve been inspired for years by social psychology—I’ve always had a tension toward the interplay between psychological and biological processes. Having the opportunity to engage with colleagues with deep expertise in this area has been really helpful.

I’d say to adopt and really hold tightly on to a learner orientation as a researcher rather than presuming you will always be best situated to help or even fully understand a complex process. You can learn from so many different sources beyond the existing theories of people you are working with. And this can add so much more to studies of mechanisms.

Wrapping up, how do you situate your research on causal mechanisms in the larger context of reducing inequality and improving youth outcomes? What advice could you offer others looking to pursue similar work?

MD: One thing I’d say is to be thoughtful and even critical of the idea of jumping to intervention as the ultimate goal. Having identified through smaller studies which mechanisms matter, a gratifying next step is to forge and deepen relationships with partners who are situated to continually shift what’s happening at an institutional or other structural level, rather than only trying to identify what is the intervention where we come in and say everybody must do this thing to respond to this situated form of inequality.

RS: For studying mechanisms and for bringing research to bear on real-world efforts to reduce inequality, the development of longer-term relationships with stakeholders in the community is really important. It takes time, but it’s worth it.

MD: There are so many parties invested in reducing inequality—so many different pieces of evidence and so many experiences—research that identifies causal mechanisms can really help cut across perspectives and unify parties that all want the same thing, but aren’t sure exactly what to emphasize or invest in. This research can help you learn something that will have this translatable implication in ways that aren’t just about doing one particular program, but how you approach whatever program you’re implementing. How do you make sure that each approach is addressing that key mechanism? How is that approach engaging with or translating messages to young people? You may be offering this wonderful resource, but are you doing it in a way that young people can actually internalize and capture the meaning behind it? That is what translates it into changes in outcomes.

Footnotes
  1. For full citations and a complete reference list, download the PDF of this essay.
Downloads

In this issue

Research can play a vital role in pointing policymakers, civil society, and communities toward a stronger, more sustainable, and just world. But getting there means building on what we know about what it takes for research to be useful, used, ...
Research on Research Use: Building Theory, Empirical Evidence, and a Global Field
Although many of today’s disparities have their roots in policies or actions initiated generations ago, there is much we can do today. And new research on causal mechanisms ...
Opening the Black Box: Overcoming Obstacles to Studying Causal Mechanisms in Research on Reducing Inequality
Introducing an open, centralized resource to help strengthen methods and enable greater collaboration among researchers studying the use of research evidence in policy and practice.
Strengthening Methods and Improving Collaboration: The Use of Research Evidence Methods Repository

More Digest Issues

The Digest

Issue 9: Winter 2023-24

The ninth issue of the Digest features an update on the Institutional Challenge Grant program five years after its launch, with insights from a Foundation stock-taking effort and reflections from two grantee teams. We also share new thinking on the use of research evidence, including ways for researchers to leverage existing findings to bolster studies of effective strategies to improve research use.
The Digest

Issue 8: Winter 2022-23

The eighth issue of the Digest features insights about where new research might help identify ways to reduce inequality in youth outcomes after COVID-19, as well as how the fields of implementation science and research on research use might learn from each other in ways that yield more transformative research in years ahead.
The Digest

Issue 6: Winter 2020-21

Essays in this issue of the Digest focus on the importance of looking beyond individual action and customary metrics in research on reducing inequality, as well as how strengths-based, race-conscious research can be produced and used to uplift communities of color.
The Digest

Issue 5: Winter 2019-20

The fifth issue of the William T. Grant Foundation Digest features insights from program staff and grantees on the importance of looking beneath the surface to consider the underlying factors that create and shape challenges that social scientists seek to address, whether they be related to reducing inequality in youth outcomes or improving the use of research evidence in policy and practice.
The Digest

Issue 4: Winter 2018/19

The fourth issue of the William T. Grant Foundation Digest features insights that may point the way toward a more nuanced understanding of evidence use and inspire new and more wide-ranging examinations of ways to reduce inequality in youth outcomes.
The Digest

Issue 3: Winter 2017/18

The third issue of the William T. Grant Foundation Digest features insights on how research on ability tracking can inform studies to improve the outcomes of English learners, as well as how researchers and school districts can partner to build learning systems based on research evidence.
The Digest

Issue 2: Spring 2017

The second issue of the William T. Grant Foundation Digest features writing on research rigor and relevance, as well as the potential for a new research agenda for improving the outcomes of English learners under the Every Student Succeeds Act.
The Digest

Issue 1: Summer 2016

The introductory issue of the William T. Grant Foundation Digest features essays and commentary on the value of qualitative and mixed-methods research in reducing inequality and the potential for researcher access to big data to yield useful research evidence.

Subscribe for Updates