The question of how to improve the use of research (URE) in policy and practice can be approached from diverse disciplinary and methodological angles. For many who study research use, attention to the challenge of URE grows naturally from an already established line of research. Other times, first-hand experience with the obstacles that prevent greater uses of research in decision-making can spur an interest in URE. For grantee Bruce Chorpita, Professor of Psychology and Professor of Psychiatry and Biobehavioral Sciences at the University of California, Los Angeles, it was a mix of both. Chorpita is the PI for the Reaching Families multisite trial (Kimberly Becker, Co-PI), an investigation of strategies to improve the use of evidence in supervision and clinical decision making and to improve family engagement in mental health services in low income communities. Here, we talk about how what sparked his career-long interest in URE, what still excites him about the field, and why we’re at the dawn of the next 50-year paradigm of how people interact with information.
How does a clinical psychologist end up studying ways to improve the use of research?
I don’t see clinical research and my current work on the use of research evidence as different things. My lens regarding how we should alleviate human suffering and address mental health challenges is primarily the use of research evidence. I spend most of my time thinking about how we develop a knowledge architecture that gives people who want to help children the very best ideas that science has to offer.
When I was in graduate school, there was already a lot going on in mental health in terms of managed care, and the movement toward evidence-based treatments was starting to take off. I was of the mindset that we had to get folks to understand and apply evidence-based treatments.
how we develop a knowledge architecture that gives people who want to help children the very best ideas that science has to offer
My first job after graduate school was in Hawaii. The whole mental health system there was under a federal consent decree because the school based mental health system was not performing up to standards. Within a year of being in Hawaii, the chief of children’s mental health appointed me to be the acting clinical director for the state. She said, You have a lot of ideas, go ahead and try to fix this system. And I was too young to know how wrong it was to say yes to that. I thought, We have all these evidence-based interventions and I have this list of programs, and we just need to connect the dots.
But what sparked a shift for me was realizing, almost immediately, that that was not going to be a feasible solution for a large state-wide system. At the time we were dealing with maybe 100 programs to pick from, and we would have to strategically select 8 or 9 at scale to serve about 70 percent of the kids in the mental health system. But we were never going to be able to afford that. And even if we could, there would still be kids that would not benefit from those programs. I was facing parent advocates and community folks who were saying that everyone should be able to have treatments informed by science. It made me realize that there had to be different ways of thinking about this.
How did “thinking differently” about research use initially play out? In what ways has your thinking evolved?
I was wondering back then, If you knew everything that was in the research, what would you do? If everybody in the system who came in contact with children with mental health needs knew everything in the research, how would they behave? But getting thousands of studies to all these people is obviously a huge problem. One early step was to form a committee of 20-30 of us that met monthly to read all of these trials and try to extract and synthesize a general message about good ideas and bad ideas, etc. That committee is actually still meeting monthly, and by 2003 had doubled the effect size by the average outcome for all children in the public mental health system in the state of Hawaii.
My current work has to do with engaging children in their mental health services. There may be 50 or so trials out there on this issue, but somebody who needs to know what to do today can’t consume all of that information. So, we built a series of tools that funnels the research to the specific problem you have in front of you. It’s called the Coordinated Knowledge System (CKS). One of the core ideas of a system like that is that it filters and then synthesizes. It’s marks a shift from a product-centric view of an evidence base to a knowledge-centric view. That is, behavior isn’t centered around using a specific product, but rather around how we let the entire evidence base inform us of what to do.
With the Coordinated Knowledge System, behavior isn’t centered around using a specific product, but rather around how we let the entire evidence base inform us of what to do.
CKS embeds research evidence into a coordinated sequence of actions for defined roles. The idea is that we will see much greater application of that evidence relative to traditional practice guidelines, which separate evidence delivery from the planning and action that follow. The CKS model focuses on supervisor and supervisee interactions, to help them more readily access, understand, and apply research evidence to help them identify engagement problems within youth treatment plans, select appropriate research-informed responses, monitor progress, and adapt treatment to promote youth engagement. People often think of CKS as a treatment package. But it’s a tool to use all of the treatments; a kind of aggregator. In that way, there’s really a direct line from those early questions about how to synthesize what is useful out of the entire universe of research.
What are some of the challenges of studying the use of research evidence? What still excites you about the field?
What’s challenging is that nearly every institution that aims to build better lives for children and families is built around a paradigm that is inconsistent with what we know about how research evidence is used. Both granting agencies and graduate schools are built around the paradigm that was developed in the mid-90s that says we need to credential and validate specific programs. Our implementation science has become a science of how to get those programs installed in places. That is not the only way, and quite possibly not even the best way, to use research evidence. Changing institutional culture worldwide is very slow and difficult.
What’s exciting is that we are reaching the limits of the current paradigm and we are recognizing that there are going to be better newer ways of interacting with the evidence base. There’s a paper from a few years back that says in health care there’s 75 articles published every day and 11 systematic review papers of trials every day. What is happening is that we are finally figuring out how to use computers to help us make sense of enormous databases to get answers in a way that is both inexpensive and scalable. To me, we are on the dawn of the next 50-year paradigm. The way humans interact with computers around their health care and mental health and well-being is going to change in the next 10 years in a way that’s going to be as radical as when we invented the printing press.
Broadly speaking, what are the implications for social science in general? Should social scientists across disciplines and policy areas be thinking about research use?
URE is the future. Evidence is a continually accumulating resource. The idea that the evidence base today is going to be the evidence base next year is false; evidence is always evolving. So how do we create that connection to what the latest research says.
If you think of knowledge as water and you have thirsty people everywhere and we have this massive reservoir of water, there are people who say we need more water (research). But that’s not getting the water to people, it’s just making the reservoir deeper. When the evidence-based treatment movement came along, we said let’s develop these buckets that are a way to take some of the evidence base to people. What we really need is the plumbing, the pipes—because the research is simply too large for any one of us to know, to carry to someone else. So how are we going to develop the technologies to allow us to aggregate, filter, synthesize, summarize that evidence base in real time and inform and answer people’s questions?
It may sound like sci-fi, but it is an achievable goal. I’m working with a group now at the National Academies on how to make the evidence from all of our research extractable and actionable through automated reasoning—using machines to help us pull the research and get us the facts. People really don’t want to know what is the best program, they want to know what the best program tells them they should do. That’s going to be achievable if we can develop strategies to make the entire evidence base actionable, and that’s not limited to mental health or education, it involves all practice and policy domains.
We said “let’s develop these buckets that are a way to take some of the evidence base to people.” What we really need is the plumbing.
I believe we all want to build better lives for people. As a social scientist, you don’t just want to be professionally successful, you want to have an impact on the world and move the ball down the field a bit.
One of my mentors told me a story about a study he did in a children’s hospital, where in one unit there was a token economy. Kids earned points, and at the end of the week they could cash them in at the store and get a toy or some goodie. And in the other unit, they had the same token economy and the same points for the same behavior, but at end of week there was no store you just reset and started earning points for the next week. He found that the children’s behavior responded equally in both settings. The message was that we will work hard for tokens regardless of whether they can be cashed in at the store. He then asked us to stop and reflect on what is in your store. Is it helping others—your family and loved ones? Is it helping children? What makes you feel good?
We have to be aware that the institutions in which we are embedded are token economies, and some of those tokens may feel rewarding but they are not leading us to that reward we really want, which might be really improving the human condition and really leveraging our enlightened selves to say we can live better lives because we have done science that tells us how to do it. The caution of my mentor’s tale is to be aware of what things are tokens and what things are really advancing your bigger-picture goals.
If you really want to be part of the way people interact with evidence and facts, if that’s a big-picture goal for you, this is the time to do it. I don’t see how you can say no to that.
