A Cost-Utility Framework to Facilitate Evidence-Based Decision-Making in Schools

Increasingly, policy requires that school district administrators’ decisions about programs and interventions are based on evidence of program effectiveness in improving student outcomes. However, policymakers and administrators must also consider their local conditions, available resources and capacity, stakeholder values, and district priorities. An important challenge is how to systematically and transparently integrate these many considerations—as well as research evidence—in decision making. Hollands and Levin will build on Multiple Decision Making Criteria (MCDM) methods that have been widely applied in fields outside of education. MCDM methods accommodate the participation of multiple stakeholders in decision making and encourage users to make explicit the objective and subjective criteria that will influence their decision. The team will use these methods and cost utility analysis to create a decision-making framework that informs the development of an online tool, which will list criteria to guide further information gathering, provide methods and instruments to gather that information, and evaluate the quality of information for each criterion. In the first year, the team will work collaboratively with districts, a state department, and school leaders to identify problem areas, establish criteria for consideration of local needs and resources and program requirements, collect relevant evidence, and assess how well each criterion is satisfied by the alternative sources of evidence. In year two, the team will develop and implement an interactive online tool based on the framework developed in year one. Personnel at each of the participating sites also will be interviewed at the outset of the project and again in the second year to assess how decision makers use research and other sources of information, and to evaluate use of the tool on site, the role it played in decision making processes, and its feasibility.