Many of the researchers in research-practice partnerships (RPPs) are drawn to them because of frustrations that their work—while esteemed by academic colleagues—too infrequently influences education policies or practices. Many district partners in RPPs also have backgrounds as researchers, and they too want to see research address stubborn teaching and learning challenges. District leaders, administrators, and other partners are often frustrated that research is not adequately relevant to their contexts, or is not timely or accessible enough to meet their needs. Put simply, researchers want to do more impactful research, and agency staff want more useful research that can inform their work to improve student outcomes.
Given these motivations, it is no surprise that RPPs have largely focused on identifying ways to improve the production of relevant, timely, and actionable research for local and state education agencies. Partnerships across the country have sought to make fundamental shifts in how research is produced: they no longer want research agendas that are defined solely by researchers or practitioners; and they are not satisfied with one-off projects that do not culminate in meaningful change. Instead, research and agency partners seek to jointly define a long-term agenda that addresses important problems of practice. Moreover, they have invested enormous time and energy to build joint research infrastructures—including large-scale data sharing agreements, usable and secure data archives, and staff with nontraditional skill sets—that can support their collaborations (Turley & Stevens, 2015).
For all the focus on producing more useful research, too little attention has focused on supporting the use of research.
But, for all the focus on producing more useful research, too little attention has focused on supporting the use of research. Creating and communicating research that is more relevant, timely, and actionable is necessary but not sufficient to foster an education system that is able to learn from and incorporate research findings into policy changes, professional development, curricula, and teaching and learning.
Knowing how to promote the use of research more often and more effectively remains a looming challenge for all of us, and the struggle to direct as much attention to research use as to research production is not limited to research-practice partnerships. Developing an infrastructure to support meaningful and routine use of evidence is a more difficult challenge, and the path forward remains less clear and lacks political leadership. At the federal policy level, a bipartisan commission recently produced The Promise of Evidence-Based Policymaking, a much heralded set of recommendations for advancing evidence in policy (Committee on Evidence-Based Policymaking, 2017). That report, like many initiatives, focuses largely on ways to build evidence rather than support its use in policy. We at the Foundation have also failed at times to adequately envision the link between producing relevant research and fostering its use in policy and practice: Our RPP resource website, for example, includes a section providing guidance on “communicating and using research,” which provides extensive resources and tips on communicating research, but far fewer on using research. This, of course, is despite the fact that one of our two primary focus areas is improving the use of research evidence!
A theory of action can serve as a touchstone for partners as they design their partnership activities and allocate resources to support those activities.
RPPs and RPP funders are positioned to move education into the next frontier, but to do so they will need to complement their attention to evidence production with an equally robust focus on evidence use. In this essay, I argue that one way forward is for RPPs to articulate theories of action for research use, empirically test them, and then iteratively improve their work and refine their theories.
Developing a Theory of Action
A theory of action can serve as a touchstone for partners as they design their partnership activities and allocate resources to support those activities. In education, jointly crafting a working theory would allow research and agency partners respectively to articulate what they think it takes for research to influence policies, programs, or practices that would benefit students. I suspect that the discussion itself might be fruitful for increasing research partners’ understanding of policymaking and for surfacing each party’s underlying assumptions about how the partnership research will be used.
Going through this exercise before research is designed may even lead to new insights on how to make the research more usable. It is a conversation that is likely best informed by agency leaders and administrators, and in the accompanying essay, we asked three district leaders to share their lessons learned on fostering research use. In addition to their insights, my recommendations below draw upon the growing body of research on the use of research evidence.
Identify how you envision the research being used
At the outset, it is less important to get the working theory “right” than to be explicit about partners’ best guesses and assumptions about how the research will be used. One way to begin is by asking partners to articulate what it would look like if the partnership research were used, and then map backward to delineate the ways the partners can bring about desired outcomes.
Articulate what it would look like if the partnership research were used, and then map backward to delineate the ways the partners can bring about desired outcomes.
Partners might consider whether the research is intended for instrumental, conceptual, or political, process, or imposed use—or some combination thereof (see Nutley, Walter, & Davies, 2007; Weiss, 1977). Instrumental use refers to situations when research directly influences a particular policy or practice decision; for example, when information on the effectiveness of a math curriculum drives a decision about whether to continue it. Conceptual use occurs when research influences how issues and problems are defined and orients people toward certain types of solutions. Political use occurs when someone already has a position on an issue, and leverages research to support that position. While partners may plan for instrumental or conceptual uses, a policy window may open that brings the research into the political limelight. Understanding the political context around the topic being studied enables partners to plan how they will position their research findings and who they will engage. Process use refers to what practitioners learn from participating in research production, for example, about framing empirical questions, designing rigorous studies, or interpreting research findings. Lastly, imposed use denotes instances when policy requires the use of research, such as when funding is tied to the adoption of evidence-based programs.
While many of us gravitate toward instrumental uses of research, partnerships should not underestimate the value of conceptual uses (Penuel & Allen, 2017). Farrell and Coburn (2016a) describe the varied benefits that can arise from conceptual uses: for instance, engaging with research can introduce district leaders to new concepts about teaching and learning, shed light on a persistent problem that has been hard to address, shift attention to different types of solutions than have been considered in the past, and provide conceptual frameworks to guide policy or program actions. In these ways, the enlightenment function of research can have more radiating impacts on a range of policy and practice decisions down the road (Weiss, 2005).
Farrell and Coburn (2016b) also have proposed a framework that identifies the aspects of district capacity and the qualities of external partners that are likely to foster the use of research evidence. While we await findings from their empirical tests of this framework, partnerships might nevertheless find the concepts valuable as they develop their working theories of action. For example, Farrell and Coburn’s preliminary findings indicate that it is useful to understand the variation in capacity that exists across departments within a district, suggesting that partnerships may need to tailor their strategies to the capacity of a particular department (e.g., math vs. school support areas) rather than relying on an aggregate assessment of district capacity.
Be realistic about what it takes to create change
In developing their working theories, partners should avoid becoming too enamored with a rational, linear model for decision making (Nutley, Walter, & Davies, 2007). While it’s tempting to think (or hope) that providing better research will yield more evidence-based decisions, the policy process is always more complicated. It is rare that research provides the missing piece of information that resolves a dilemma and determines a policy decision (Weiss, 1977). Proponents of evidence-based policy, for example, often assert that impact evaluations should be used to decide whether a program continues to be implemented and funded. And yet, the vast majority of impact evaluations yield null or mixed effects. More often than not, evaluation findings raise a chain of other questions about whether there are ways to strengthen the program or its implementation.
Partners will need to specify realistic models of how decisions are made within their local context and then design the partnership’s work to optimize research influence. As my colleague Sandra Nutley and I (2014) have written, it is valuable to recognize that research use—like policy decision making—rarely boils down to a single moment. Nor is research use a simple process whereby research “facts” are passed from researchers to practitioners and then applied in a linear decision making process. Instead, research use often involves people individually and collectively engaging with research over time, bringing their own and their organization’s goals, motivations, routines, and political contexts with them.
Research use is not simple process whereby research “facts” are passed from researchers to practitioners and then applied in a linear decision making process.
Thus a theory of action should reflect a keen understanding of what it takes to create change: what do partners hope will happen after a piece of research is produced and communicated? What is needed to support policy or practice change? Answering these questions requires anticipating the process by which research would come to be used and the conditions that would support its use. To promote instrumental uses of research, a theory of action might include an understanding of who participates in decision making related to the targeted policy, how those decisions are made, why they are made, and when they are made. Understanding the policy process enables partners to strategize about the myriad activities that need to occur after research findings are produced. For example, does the change require coordination between departments? If so, how will the coordination happen and are there ways the research can support that coordination? Does the change require new resources, and if so, how will the research be brought to bear on budget or staffing decisions? Who will champion the changes?
Some partnerships have multiple streams of work focused on issues as wide-ranging as addressing chronic absenteeism, measuring school climate, and reducing dropout, whereas other partnerships are focused on a complex, singular problem such as improving middle-school math outcomes across an entire system. When partnerships have more than one stream of work, it may be useful to articulate a theory of action for a specific line of work, because different people, processes, and politics are involved in different areas. For example, supporting principals to use on-track data to identify high school students at risk of dropping out involves different actors and strategies than collaborating with district administrators and teachers to redesign and implement a new elementary science curriculum.
Of course, even the best laid plans can be upset in the maelstrom of policymaking, and thus my goal for encouraging RPPs to develop theories of action is not to demand perfection. Rather, I suspect that having a clear-eyed view of the decision making context, including the processes and conditions that support research use, as well as a long time horizon can help RPPs increase their impact. As articulated by Faith Connelly, Executive Director of the Baltimore Education Research Consortium, “the current politics may mean that some of this won’t ‘take’ in the moment…some of this is about waiting…months down the line for the right opening to emerge.”
Studying RPP Theories of Action
While a theory of action can provide direction for partnership work, empirical studies that test these theories are essential for learning what is working well, what is not, and how to change things going forward. Individual RPPs seek empirical work that can help them understand whether they are achieving their goals and identify ways to improve their work. Funders need to learn whether their investments in individual RPPs, or a portfolio of them, are making a difference, as well as identify ways to improve their funding and other supports for grantees. Both individual RPPs and funders need to understand the conditions under which partnerships are likely to be successful, the effectiveness of different partnership strategies, and whether certain types of partnerships are best suited to address particular types of problems or meet particular goals.
At the William T. Grant Foundation, we were initially drawn to studies of RPPs because we thought the findings would increase our understanding of what it took to improve the use of research evidence in practice and policy (Tseng, 2009). Thus far, we have funded two studies of partnerships, led by Cynthia Coburn and Joshua Glazer, respectively. Both bring an organizational lens to their studies. Glazer is examining how different partnership models and strategies influence district capacity to use research. Coburn’s team, in addition to examining district capacity, is zeroing in on how research partners’ capacity matters for research use. Both are education researchers, and thus it is not surprising that they also apply theories of organizational and adult learning to understand the process of research use and the need for learning opportunities. In contrast, child welfare researcher Larry Palinkas draws on his anthropology roots in conceptualizing the interactions between researchers and practitioners as “cultural exchanges” between professional groups in Research-Practice-Policy Partnerships for Implementation of Evidence-Based Practices in Child Welfare and Child Mental Health.In addition to conceptual sophistication, advances in measurement will be necessary to assess whether, how, and under what conditions RPPs influence the use of research. The Institute of Education Science made measurement of research use a cornerstone for the two Knowledge Utilization Centers that the agency funded in 2014 and 2015. Publications from the first of these centers are beginning to emerge (Penuel et al., 2017), and, as these measures are released and further tested, it will be useful to incorporate them into future studies of research-practice partnerships.
Empirical studies that test theories of action are essential for learning what is working well, what is not, and how to change things going forward.
Another promising direction is to empirically study the framework for RPP effectiveness developed by Erin Henrick and colleagues (2017). In their paper, the authors propose five dimensions of RPP effectiveness: 1) building trusting relationships, 2) conducting rigorous research to inform action, 3) supporting practice change, 4) informing education efforts outside the partnership, and 5) building capacity among partners. The authors have proposed an initial set of indicators for gauging progress on each dimension, which future research efforts can build on to develop formal assessment tools. Further validation work also is needed to test the dimensions and the extent to which they predict desired outcomes.
Lastly, as research-practice partnerships lay out their theories of action and seek to test them, they might themselves reach out to research partners who can bring conceptual rigor and methodological sophistication to the studies. When you are in the throes of the day-to-day work, it can be hard to gain perspective; a third-party researcher can help surface underlying assumptions, collect data with less demand characteristics, and be a thought partner in interpreting what the findings mean for improving the partnership work.
Conclusion
For those interested in studying research-practice partnerships—or other promising efforts to improve the use or usefulness of research—I hope you will consider applying for a research grant under focus area on improving the use of research evidence.
