Here, two distinguished researchers and past grantees of the Foundation, describe why they chose to study the use of research evidence (URE) and how that work has benefitted from and contributed to their respective fields of study. Although they come from two very distinct disciplines—Asen has a background in communication arts and McDonnell is a political scientist—both found tremendous meaning studying the use of research evidence and have written influential pieces on the subject, including peer-reviewed articles and books as well as chapters of books that resulted from their Foundation-funded grants.
Robert Asen
Robert Asen conducts research and teaches in the areas of public policy debate, public sphere studies, and rhetoric and critical theory in the Department of Communication Arts, University of Wisconsin–Madison. Asen focuses on the ways that political, economic, and cultural inequalities interact with relations of power to shape public discourse. In 2009, Rob Asen and Deb Gurke received a research grant to study how school boards understand, weigh, and incorporate various types of research evidence into their policy deliberations and decision-making. This was Asen’s first research project studying the use of research evidence (URE). Findings from the project were expanded into the book, Democracy, Deliberation, and Education (Penn State Press, 2015).
What is your academic background and what were your research interests prior to studying the use of research evidence?
My disciplinary background is communication, and within that I have a specialty in rhetoric and argumentation. A key concept in argument is the idea of evidence. Presumably in good practices of argument people articulate their preferences clearly. Whether a policy preference or something else, they offer compelling reasons to support those positions and back them up with evidence. Thus, evidence was always an element I looked at in my research prior to my Foundation-funded projects but it was not the primary focus.
Both prior to my interest in URE and today, my research has been about public policy debate and democratic deliberation. The theoretical perspective that I bring is an interest in the public sphere, that is, how public opinion in Western democracies emerges through deliberative processes. In these processes, individuals and groups come together to consider potential solutions to shared problems. I wrote a dissertation at Northwestern University that looked at welfare policy debates in the United States in the 1980s and ’90s, primarily at the Congressional level. That became the subject of my first book. My second book looked at debates about social security—principally debates in the 2000s about President Bush’s proposal to privatize social security. A consistent theme throughout my research has been how lawmakers in policy settings and ordinary citizens get together to discuss and debate issues of common concern and figure out ways to move forward on them.
Why did you decide to study URE?
About 10 years ago, William T. Grant Foundation President Bob Granger came to the University of Wisconsin–Madison campus to make a presentation about the Foundation’s new interest in studying URE and the motivation behind it. He talked about how much great research there is on political, economic, and other issues but how there was relatively little understanding in terms of how that research works in the policy process.
My interest in deliberation had a strong component that was policy-driven. My motivation for researching and writing about the welfare debates in the ’80s and ’90s and social security in the 2000s was that I was troubled by the content of some of those debates. For example, the misrepresentations of poor women that had been dominating the welfare debates suggested a false set of problems. These peripheral issues were just a distraction and I believed that we could do a better job of debating policies. Bob’s talk about URE, particularly its role in policy, struck me as a way to extend my interests and allowed me to think of another way of having an impact on policy, i.e., what could we say about the evidence that policymakers use when they debate and what role can researchers play in trying to make an intervention?
How has studying URE expanded your perspective?
Prior to thinking about URE and working with the Foundation, my focus had mostly been on the kind of deliberation that happens in legislative hearings. In studying URE, my focus has expanded to include the wider context of policymaking. Studying research use has allowed me to see this process from a range of different perspectives for two interrelated reasons: 1) the work of the Foundation in cultivating this interdisciplinary community of scholars; and 2) the work of the Foundation in making policy connections for grantees. Working in this interdisciplinary setting has allowed me to better understand the motivations and contributions of different actors.
For example, at a Foundation-funded interdisciplinary meeting on URE, those of us working on education issues went to the Department of Education (DOE) and to Capitol Hill where we met with DOE and legislative staff. This gave me a much different understanding of the public face of policymaking and the role that evidence plays in it, as well as perspective in terms of the executive agencies. I understood better how legislative hearings, as the public face of public policy, worked in the larger process. I had a better appreciation for the roles of officials at the DOE, including how they think about policy and about research evidence and its role in the policymaking process. Not only has this given me a different perspective on the process but it has even given me a better sense of the things that I was writing about before I started working on URE because I have these complementary perspectives.
What have you found most challenging or rewarding in your URE work?
One of the biggest challenges early on was recognizing that the interdisciplinary nature of this work is its strength. I did not have to make my work conform with the contributions of researchers from other disciplines. But I did not realize this right away. When my co-primary investigator Deb Gurke and I attended our first Foundation meeting following our award, we met with grantees from other disciplines and methodologies. I was a qualitative researcher who came from a humanistic tradition, and I wondered whether the Foundation really wanted a quantitative researcher who could do an evaluation study. Would the Foundation think they had made a mistake? Perhaps we had misrepresented ourselves in our proposal? But I learned that this is an interdisciplinary area of inquiry and the Foundation’s interest in our grant was because it provided a distinct, complementary, and important perspective on these questions. It is through the intersection of these different perspectives that we learn important things about URE, and each one of us contributes by providing our distinct perspectives. It’s in the interaction of those perspectives that some real insight is produced. But engaging across disciplines also means that we cannot just rely on familiar terms and assume that everyone will understand. We have to be reflective and purposeful in explaining our contributions across disciplinary lines.
Being part of a diverse network of researchers has helped to shape my work, even as I write about issues not related to URE. I also think that there were some methodological rewards for me in that being around this interdisciplinary community encouraged me to think about different methodological approaches. It brought interviewing into my repertoire and that has played a key role in my research. For example, in the project that Deb and I worked on, in addition to our observations of school board deliberations we conducted interviews with school board members, superintendents, and a key community blogger.
What is your advice for other researchers moving into or considering the study of URE? What should they know?
Researchers interested in studying URE should be willing to move outside of their comfort zones. Attending a URE meeting is not like attending a disciplinary meeting; at first, someone might feel unsure of their role. If it is true that we learn by encountering new and unfamiliar experiences, and I think it is, then studying URE can teach someone a lot. I would add, too, that studying URE is an important way of connecting research and public policy. And, in the world we live in today, we certainly can use more of this.
Lorraine McDonnell
Lorraine McDonnell is on the political science faculty of the University of California, Santa Barbara (UCSB). Prior to joining UCSB, McDonnell was a senior political scientist at RAND. Her research focuses on the politics and institutional effects of elementary and secondary education policies. In recent studies, she has examined the politics of student testing, particularly its historical role in federal policy, the policy feedback it generates among political interest groups, and how it acts as a lens for understanding the role of research and other evidence in policy decisions. She served on the National Research Council’s Board on Testing and Assessment for seven years and was a member of its advisory committee for the Division of the Behavioral and Social Sciences and Education. She is a past president of the American Educational Research Association and an elected member of the National Academy of Education.
What were your research interests prior to receiving a grant from the William T. Grant Foundation?
My prior work focused almost exclusively on the politics of education policy, by which I mean who gets what—and why. What are the factors that explain why certain kinds of policies emerge? What are the politics involved in the preferences and incentives of the different actors? In the wake of the 1983 landmark report, A Nation at Risk, for example, which documented how schools in the United States were failing, states began to develop accountability systems. Testing was at the heart of those systems and I was looking at the implementation of testing policies in three states: California, Kentucky, and North Carolina. Politics at the time was so divisive that Pete Wilson, then Governor of California, vetoed the legislation reauthorizing state assessment. Kentucky experienced similar values-based protests while North Carolina did not. This provided me with fertile ground for investigating what factors explain why states had different experiences with these accountability systems. It was an example of looking at the politics to understand what happened, i.e., was it about interest groups, ideology, leadership? These are the factors in politics; research evidence didn’t enter into it much but other kinds of evidence did.
How has studying the use of research evidence expanded your perspective?
I still adhere to the theoretical perspective of political science and policy analysis, but now I take my lens and look specifically at evidence as one part of the policy process. The model that we are using asks what explains the use of different types of evidence. In political science, shining a light on the kinds of evidence that gets used had been a secondary interest, not the main concern. In a classic study published in the early ’80s and still read by students of political science today, John Kingdom looked at the factors that influence whether something gets on the policy agenda. Research evidence plays a very small part in his theory and he devotes just part of a chapter to where research comes in. Stephen Weatherford (my co-primary investigator on the Foundation-funded grant) and I wanted to look at all the different types of evidence. We maintained that asking about research evidence use is important, but we needed to expand the lens. It’s really about evidence and where research fits into it, and where evidence fits into the policy process. That’s something we are still learning.
Did that perspective lead to any new lines of inquiry, tools, or ideas?
Stephen Weatherford and I are writing a book that grew directly out of our URE study. We are looking at the use of evidence in policymaking: the what, the who, and the how. The what is the kinds of evidence, the who are the different policy actors—not just the decision makers—and the producers of evidence. The how is the stages of the policy process. We are saying that the kinds of evidence that gets used in the policy process depends on the incentives of the users and the suppliers and the stages of the policy process. For example, research-based evidence is typically used more in the early stages of the process when a policy problem is being defined while other types of evidence, such as ideological arguments and personal experience, often become more salient during the enactment stage.
What would be your advice to political scientists looking to move into the space?
Many people who look at URE look at it apolitically; it’s about rationality. But there are big political incentives that are going to shape whether and how that research gets used. Theory and studies on URE would benefit from consideration and examination of politics. This means thinking beyond the framing and material presented in the Foundation’s call for proposals. It means applicants need to resist the temptation to shoehorn what they are doing into the Foundation’s specific call for proposals. When coming to a place like the Foundation, I recommend that investigators incorporate rather than shy away from their academic traditions and disciplines and represent these perspectives in their work, so that URE studies can also contribute to theoretical advances in the broader social science disciplines.
