Digest, Issue 7: Winter 2021-22

Strengthening Methods and Improving Collaboration: The Use of Research Evidence Methods Repository

A few years ago, the William T. Grant Foundation provided us with the opportunity to write a monograph that reviewed the research methods used to study the use of research evidence in policy and practice (URE). After reviewing hundreds of studies, we were struck by the creative and highly varied deployment of methodological approaches used to study URE in complex and nuanced social contexts.1

One of the details that struck us most was that while many of the protocols being used were designed for the specific context of the individual study, multiple researchers were often investigating similar patterns of problems within and across the URE space. Seeing this, we realized the need for an open, centralized resource that would help strengthen methods used in the field and enable greater collaboration among researchers using similar approaches.

In response, we have developed the URE Methods Repository, an open access platform that uses the Open Science Framework (OSF) to collect high-quality measures for researchers. We are excited to extend an invitation to the URE community to make use of and contribute to the Repository.

What is the URE Methods Repository?

The Repository is an open resource for sharing research methods used to understand and improve the use of evidence in policy and practice. Community members can explore the work of others by searching for particular researchers, research methods, questions, domains (e.g., education, health policy, youth outcomes), and contexts (e.g., policy bodies, service providers).

Often, details about a particular research method are not available in published work due to constraints in the publication process. Protocols included in the Repository Collection have much more flexibility and can include artifacts such as complete surveys, interview protocols, and interpretive coding structures. It also has the ability to integrate with GitHub and other open-source code repositories; DataVerse and other open data repositories; Zotero and other reference management services; and a number of other plugins with services that contribute to specific areas of the research process. The Repository project may even link to researchers’ own sites and other relevant methods resources.

Users can contribute to the Repository either by submitting their own protocols or by adapting the work of others for their own research needs. It is designed to ensure that the lineage of any adapted work is preserved and that the contributions of researchers are recognized. For example, consider the common practice of adapting a survey that you co-developed and validated with a colleague and then each applied with some minor item modifications in two different study contexts: Through the Repository, the platform preserves the links between the original validation study and any future applications. All protocols are owned and controlled by the researchers themselves and can always be modified and updated as needed.

Who can use the URE Methods Repository?

We envision the Repository being useful for researchers within and interested in the URE field, funders who are interested in evaluating the work they have supported, and practitioners who may be personally interested in the application of research in their field. At the same time, people in many different positions may simply want to explore the work of researchers who study URE. For example, they may want to see how people studying URE in different fields have developed research methods. Or they may want to learn more about the application of particular methods that they have not yet pursued in their own work. As all Repository projects are open access, there are no pay walls or complicated institutional hoops to jump through for non-researchers to gain access.

URE researchers can also share research and methods at different stages of development. For published work, researchers can provide more detail about methods than might be possible within the length and appendix constraints of other formats. They may also tie together a particular methods approach across multiple publications, highlighting the development of the research and the methods and how they apply it in different contexts.

For research efforts that are in progress, an OSF project is highly collaborative and easily supports the iterative research process. As the platform integrates with many other services, it can be a useful dashboard for an overall study and can be used to seek comments and feedback from collaborators, other colleagues, practitioners, or funders as their work develops.

What resources and tools are available for users of the Repository?

In addition to developing the Repository itself, we have launched a companion site that we created to provide a number of supports for its use. The site offers information and guidance in the form of written materials, video tutorials, and resources related to URE research methods for researchers looking to build protocols. Another prominent support found here is a library of templates intended to provide structured but flexible guidance for designing one’s protocol for inclusion in the Repository. The templates each highlight key features of particular methods used to study URE (e.g., survey, interview, case study, document analysis, etc.) and provide an organizational framework that includes fields representing common features of a method’s description.

Another available tool on the site is an importer plugin that can take a Microsoft Word document or Google document detailing a given protocol and automatically format its contents into an existing OSF project. Using this, even researchers who are new to OSF’s platform can easily develop a protocol by downloading one of the aforementioned structure templates, filling out all relevant fields, and importing the finished product into OSF wikis in a user-friendly, streamlined process.

An invitation to participate

Again, we invite you to visit the Repository and companion site. If you are interested in contributing to the Collection by submitting or adapting a protocol, please feel free to start today. Our Team is available to provide any assistance we can to make this work as simple as possible, and we welcome your feedback and questions on any aspect of the Repository. You can contact us at uremethods [at] gmail.com.

Footnotes
  1. For full citations and a complete reference list, download the PDF of this essay.
Downloads

In this issue

Research can play a vital role in pointing policymakers, civil society, and communities toward a stronger, more sustainable, and just world. But getting there means building on what we know about what it takes for research to be useful, used, ...
Research on Research Use: Building Theory, Empirical Evidence, and a Global Field
Former William T. Grant Scholar Mesmin Destin is an associate professor at Northwestern University in the Department of Psychology and the School of Education & Social Policy. Rob Smith is a research grantee and professor of sociology, immigration studies, and ...
Contending with Complexity: How Two Grantees are Approaching Studies of Mechanisms
Although many of today’s disparities have their roots in policies or actions initiated generations ago, there is much we can do today. And new research on causal mechanisms ...
Opening the Black Box: Overcoming Obstacles to Studying Causal Mechanisms in Research on Reducing Inequality

More Digest Issues

The Digest

Issue 9: Winter 2023-24

The ninth issue of the Digest features an update on the Institutional Challenge Grant program five years after its launch, with insights from a Foundation stock-taking effort and reflections from two grantee teams. We also share new thinking on the use of research evidence, including ways for researchers to leverage existing findings to bolster studies of effective strategies to improve research use.
The Digest

Issue 8: Winter 2022-23

The eighth issue of the Digest features insights about where new research might help identify ways to reduce inequality in youth outcomes after COVID-19, as well as how the fields of implementation science and research on research use might learn from each other in ways that yield more transformative research in years ahead.
The Digest

Issue 6: Winter 2020-21

Essays in this issue of the Digest focus on the importance of looking beyond individual action and customary metrics in research on reducing inequality, as well as how strengths-based, race-conscious research can be produced and used to uplift communities of color.
The Digest

Issue 5: Winter 2019-20

The fifth issue of the William T. Grant Foundation Digest features insights from program staff and grantees on the importance of looking beneath the surface to consider the underlying factors that create and shape challenges that social scientists seek to address, whether they be related to reducing inequality in youth outcomes or improving the use of research evidence in policy and practice.
The Digest

Issue 4: Winter 2018/19

The fourth issue of the William T. Grant Foundation Digest features insights that may point the way toward a more nuanced understanding of evidence use and inspire new and more wide-ranging examinations of ways to reduce inequality in youth outcomes.
The Digest

Issue 3: Winter 2017/18

The third issue of the William T. Grant Foundation Digest features insights on how research on ability tracking can inform studies to improve the outcomes of English learners, as well as how researchers and school districts can partner to build learning systems based on research evidence.
The Digest

Issue 2: Spring 2017

The second issue of the William T. Grant Foundation Digest features writing on research rigor and relevance, as well as the potential for a new research agenda for improving the outcomes of English learners under the Every Student Succeeds Act.
The Digest

Issue 1: Summer 2016

The introductory issue of the William T. Grant Foundation Digest features essays and commentary on the value of qualitative and mixed-methods research in reducing inequality and the potential for researcher access to big data to yield useful research evidence.