Recently Added Resources
-
It’s No Longer All about the Mean: Using Multi-site Trials to Learn About and From Impact Variation
Raudenbush and Bloom outline key features of an ambitious project project that will bring together prominent university-based methodologists and the three research firms (MDRC, Mathematica Policy Research, and Abt Associates, Inc.) that have conducted the most large multi-site trials in education, youth development, and related fields. This paper describes the project's statistical foundation, and identifies its anticipated benefits. The project is organized in two parts: 1) developing and applying methods for learning about impact variation and 2) developing and applying methods for learning from impact variation.
Research MethodsAnalyzing Multilevel Trials -
Designing and Analyzing Studies That Randomize Schools to Estimate Intervention Effects on Student Academic Outcomes without Classroom-Level Information
This MDRC publication provides practical guidance for researchers who are designing and analyzing studies that randomize schools—which comprise three levels of clustering (students in classrooms in schools)—to measure intervention effects on student academic outcomes when information on the middle level (classrooms) is missing.
Research MethodsEducation -
Detecting Intervention Effects Across Context: An Examination of the Power of Cluster Randomized Trials
This abstract discusses how to calculate power for three types of treatment heterogeneity including 1) the variability in treatment effects across sites, 2) site-specific treatment effects, and 3) moderator effects at the cluster or student level. The sample includes the studies in the first wave of CRTs funded by IES, or those funded between 2002 and 2006 by NCER and NCEE. These studies represent a range of CRTs on various topics and with different research designs and sample sizes.
Research MethodsGroup Randomized Trials -
Improving the Quality of Classroom Interactions
The Foundation has funded several innovative classroom interventions designed to alter settings in ways that result in positive outcomes for the youth within them. In order to change a setting for the better, we need to first understand how that setting works. To that end, we have developed a theoretical framework to guide our intervention research and funded a number of studies on the development of reliable and valid measures of classroom processes. Strong theory and good measures are critical components in the development of a thorough understanding of settings.
Research MethodsEducation -
An Empirical Investigation of Design Parameters for Planning Cluster Randomized Trials of Science Achievement
This article presents empirical estimates of design parameters that can be used to appropriately power CRTs in science education and compares them to estimates using mathematics and reading.
Research MethodsEducation -
Intraclass Correlation Values for Planning Group Randomized Trials in Education
Experiments that assign intact groups to treatment conditions are increasingly common in social research. In educational research, the groups assigned are often schools. The design of group randomized experiments requires knowledge of the intraclass correlation structure to compute statistical power and sample sizes required to achieve adequate power. This paper provides a compilation of intraclass correlation values of academic achievement and related covariate effects that could be used for planning group randomized experiments in education. It also provides variance component information that is useful in planning experiments involving covariates, and illustrates the use of these values to compute statistical power of group randomized experiments.
Research MethodsGroup Randomized Trials -
A Conceptual Framework for Studying the Sources of Variation in Program Effects
"Evaluations of public programs in many fields reveal that (1) different types of programs (or different versions of the same program) vary in their effectiveness, (2) a program that is effective for one group of people might not be effective for other groups of people, and (3) a program that is effective in one set of circumstances may not be effective in other circumstances. This paper presents a conceptual framework for research on such variation in program effects and the sources of this variation. The framework is intended to help researchers — both those who focus mainly on studying program implementation and those who focus mainly on estimating program effects — see how their respective pieces fit together in a way that helps to identify factors that explain variation in program effects and thereby support more systematic data collection on these factors. The ultimate goal of the framework is to enable researchers to offer better guidance to policymakers and program operators on the conditions and practices that are associated with larger and more positive effects."
Research MethodsAnalyzing Multilevel Trials -
MET Database at the Institute for Social Research
This site enables users to apply for access to quantitative data and classroom videos created by the Measures of Effective Teaching (MET) project, funded by the Bill & Melinda Gates Foundation.
Research MethodsGroup Randomized Trials -
Optimal Design with Empirical Information (OD+)
This software includes a series of empirical estimates of plausible parameter values for determining the minimum effect size that can be detected by a given number, size, and treatment/group mix of randomized groups.
Research MethodsGroup Randomized Trials -
When Is the Story in the Subgroups? Strategies for Interpreting and Reporting Intervention Effects for Subgroups
“This revised working paper examines strategies for interpreting and reporting estimates of intervention effects for subgroups of a study sample. The paper considers why and how subgroup findings are important for applied research, alternative ways to define subgroups, different research questions that motivate subgroup analyses, and the importance of prespecifying subgroups before analyses are conducted. […]
Research MethodsAnalyzing Multilevel Trials
Categories
- Reducing Inequality (14)
- Research Methods (16)
- Use of Research Evidence (27)
- Youth Social Settings (9)
Tags
- After-school Programs (4)
- Analyzing Multilevel Trials (3)
- Child Mental Health (3)
- Child Welfare (5)
- Directions for Research: Reducing Inequality (10)
- Directions for Research: the Use of Research Evidence (4)
- Education (20)
- Evidence-Based Policy (9)
- Group Randomized Trials (10)
- Higher Education (1)
- Immigration (1)
- Justice System (1)
- Research-Practice Partnerships (8)
- Tools (7)