Improving the use of research evidence in ways that benefit youth requires clarity not only about what counts as quality evidence, but also what counts as quality use. Surprisingly, the question of what it means to use research evidence well remains largely unexplored, even amid wide-ranging international efforts to strengthen the role of research in social change.
For example, there is a well-developed literature around understanding the quality of different kinds of evidence, but little in the way of an equivalent for understanding the quality of different kinds of use. Similarly, much of the debate around evidence-based policy and practice has focused on the limitations and strengths of different types of evidence, with considerably less attention given to the characteristics of different kinds of evidence use, let alone questions of how to use evidence better.
This situation is problematic because it fails to challenge the tendency for evidence use improvement efforts to focus more on the communication and synthesis of research findings and less on supporting the uptake and application of such evidence (as noted in a recent report on the UK’s What Works Centres).
Monash Q Project
Against this backdrop, we share some early ideas about how quality evidence use might be conceptualized in relation to education. The ideas presented are based on findings from the first phase of the Monash Q Project, a five-year study seeking to understand and improve the use of research evidence in Australian schools.
This first phase involved a systematic review and narrative synthesis of 112 relevant publications from health, social care, policy and education. Our aim was to explore if and how quality of evidence use had been defined and described within each of these sectors, in order to inform the development of a quality use framework for educators.
Our analysis found a definite lack of explicit definitions or descriptions of quality of evidence use across all sectors. However, there were lines of thinking in all four sectors that touched on quality use indirectly, such as discussions about the importance of practitioner expertise in using evidence and the systemic nature of evidence use improvement.
Quality Use of Research Evidence (QURE) Framework
Drawing on these ideas, we have developed the Quality Use of Research Evidence (QURE) Framework. It characterizes high-quality use of research evidence as thoughtful engagement with and implementation of appropriate research evidence, supported by a blend of individual and organizational enabling components within a complex system.
As shown in Figure 1, this definition sees quality evidence use as:
- comprising two core components (appropriate research evidence and thoughtful engagement and implementation);
- being supported by three individual enabling components (skillsets, mindsets, relationships) and three organizational enabling components (leadership, culture, infrastructure); and
- being shaped by wider system-level influences.
Figure 1
Core Components
This framing works from the starting point that quality evidence use needs to encompass the nature of the evidence and the nature of the use. At its core therefore are two inter-connected aspirations: for the research evidence to be appropriate, and for the engagement and implementation to be thoughtful.
Appropriate research evidence is about the context-specific nature of quality research evidence. From a use perspective, quality research evidence needs to be not only methodologically rigorous, but also appropriate for the issue under consideration, appropriate for the context of use, and appropriate for the use to which it is going to be put. As Sandra Nutley and colleagues argue: “Evidence quality depends on what we want to know, why we want to know it and how we envisage that evidence being used.”
Thoughtful engagement and implementation, meanwhile, reflects how (in Lorna Earl’s words) “using evidence is a thinking process” which demands a depth of engagement between the user, the research evidence and the way it is used. High quality evidence use, then, involves critical engagement with evidence, shared deliberation about its meaning, relevance and applicability, careful integration of aspects of the evidence into particular parts of the practice, and critical reflection upon the impacts of its use.
Enabling Components
The Q Project’s framework also builds on the idea that quality evidence use needs to be supported by a range of individual, organizational and system-level factors. That is, there is a need for:
- education professionals with not only the knowledge and skills to understand research evidence (skillsets), but also the values and dispositions to be open to its meaning (mindsets) and the relational sensitivity and capacity to work with others to figure out how to use it in context (relationships);
- education organizations with not only the structures and processes to enable staff to engage with evidence (infrastructure), but also the ethos and values to make evidence use a cultural norm (culture) and the leadership and commitment to demonstrate and promote its significance (leadership); and
- education systems that support quality evidence use not only within specific individuals, institutions or contexts but through coordinated interventions across multiple levels and with varied stakeholders.
So What?
The ideas presented here have implications for anyone who is interested in strengthening the role of research within school and system improvement. This includes teachers, school leaders, system leaders, teacher educators, policy-makers, funders, researchers and research brokers. Most fundamentally, the idea of quality use highlights the need for system-level developments that not only create access to valid and reliable evidence but also support intelligent use of that evidence. More specifically, the QURE Framework can be seen as an invitation to reflect honestly on our current approaches to:
- talking about evidence use – How motivated are we to shift our focus from “whether we use evidence” to “how well we use evidence?”
- enhancing evidence use – How committed are we to improving not just the “quality of evidence” but also the “quality of use?”
- practicing evidence use – How curious are we about the “appropriateness” of our evidence, and the “thoughtfulness” of our engagement and implementation of that evidence?
Our work on this topic is at an early stage and there is much more to come as we move into working closely with schools across Australia to empirically test our framework and investigate what using research evidence well looks like for teachers and students in varied contexts. We will be probing into the behaviors that are associated with different aspects of using research evidence well, and then designing and trialling interventions to build educators’ capacity to enact and embed such practices. The ultimate goal is to improve the use of research evidence in Australian schools and school systems.
