Digest, Issue 2: Spring 2017

Toward New Research and New Data

Toward New Research Questions

At a recent meeting of researchers, policy experts and practitioners focusing on English learners, a subgroup on accountability identified four potential areas for research under ESSA: 1) building state-level capacity, 2) evaluating the effects of the ESSA Title 1 spotlight on English language proficiency, 3) examining variability in ESSA implementation, and 4) investigating the impact of changes in ELP assessments and developing assessments well-suited to improving instruction for ELs. Below are examples of research questions highlighted within each area that fit our Foundation’s interest in improving outcomes for English learners through a program, policy or practice.1

Building State Level Capacity

What is needed to build and sustain the capacity of states to assemble useful data and use it in ways that improve outcomes for ELs? The emphasis here should be on data that will allow leaders to make better decisions and to better measure the progress of ELs. Capacity of state educational agencies can be conceptualized as shared problem solving among different EL stakeholders, such as accountability and EL policymakers, administrators, and data analysis consultants. How can different stakeholders work together to use the data effectively in order to improve EL outcomes through a program, policy or practice?

Evaluating the Effects of the ESSA Title I Spotlight on EL Proficiency

Does holding individual schools accountable for student progress toward and attainment of English language proficiency lead to better EL high school graduation rates, later academic achievement, or improved socio-emotional outcomes? If so, in what ways and under what conditions? Is there variation in what accountability means across different state and local contexts, and in what ways does this affect youth outcomes? Researchers might also investigate the potential for unintended negative consequences under ESSA, as these can occur with any policy. For instance, there might be increased segregation and tracking, fewer bilingual programs, or more teacher turnover—any or all of which could dampen EL outcomes.

Examining Variability in ESSA Implementation

What characteristics of state accountability policies and practices support improvement of EL outcomes in schools and districts? One recommendation from some scholars is that accountability policies should include more measures than only assessment, because measures of socio-emotional learning and school climate are also crucial to ELs’ outcomes. Thus, researchers might undertake broader inquiries that focus on these measures as well.

Investigating the Impact of Changes in ELP Assessment

What are the impacts of changes in ELP assessments on various experiences of ELs, and do these improve their outcomes? These can include opportunities to learn content, access to rigorous language instruction, and appropriate initial classification and reclassification decisions in the context of the services provided at the local level.

Developing Assessments Suited to Improving Instruction for ELs

How can assessments be crafted to ensure that ELs receive instruction better tailored to their needs? When the federal and state governments first began to push for accountability in the 1990s, there was no organized push for teachers of ELs and content-area teachers to engage in and sustain collaborations in order to build a shared understanding of how best to serve ELs. The unfortunate result was a missed opportunity to deepen practical knowledge of how schools could ensure that ELs received better instruction tailored to their needs.

Toward the Data We Need

To arrive at the better outcomes we wish to see for ELs, we need high-quality data that allow states to know who they are serving and the progress they are making, and that also allow for the kinds of research questions we’ve posed here. While efforts to create these kinds of data are beyond the scope of our funding, we offer them for consideration by other agencies.

For example, expanding the definition of the EL category is critical to creating a more accurate picture of EL progress and, relatedly, to developing inquiries that can improve EL outcomes. The use of binary labels in the datasets, such as EL versus non- or never-EL, is a limitation. The binary labels that are used to disaggregate the EL category, such as long-term English learner versus non-long term English learner, are similarly limited. More useful would be labels across a continuum, each associated with specific language and academic competencies. This is because ELs are diverse in their language and academic competencies, not only in English, but also in their home language. Indeed, longitudinal data could also help capture the potential advantages of bilingualism, such as a student’s ability to view a situation from another person’s perspective, or enhanced academic performance. A 2016 meeting of EL research experts convened by our Foundation raised the idea that this kind of data would also provide teachers with useful information about how to instruct to the student’s strengths and weaknesses.

Research that contributes to the development of stronger assessments and broadens the data about these students can potentially go a long way toward building much-needed capacity, evaluating the effects of legislation and the ways that provisions are implemented, and, ultimately, improving the academic outcomes of a diverse student body.

Another limitation has been having data only on currently classified ELs, as this precludes learning from the long-term experiences and outcomes of former ELs who have been reclassified. With reclassification, these students effectively exit the category for data purposes. ESSA offers a step forward in this regard by allowing states to include former ELs who are still within four years of being reclassified in their academic assessments. Yet even this expanded window might not be enough. Linquanti and Hakuta are among many EL researchers who call for a “dynamic subgroup membership,” which would involve “defining and reporting the total ‘ever-EL population,’ including current ELs and reclassified former ELs.”

Given the developmental nature of language acquisition, we would learn a great deal from broadening the scope of data to include the ever-EL population. As noted earlier, reclassification has not always resulted in ELs joining mainstream classrooms and flourishing on par with those peers who have never been designated as ELs. Rachel Slama found that sizable proportions of reclassified students in Massachusetts who had started kindergarten in 2002 as ELs were able to keep pace in mainstream classrooms in the early elementary school years, but encountered difficulties in middle and high school. Some ended up having to repeat a grade. This should not surprise us, since the complexity of language use increases from the earlier grades to the later ones.

Similarly, educational settings for ELs differ between elementary school and high school. Given tracking at the high-school level, ELs and their reclassified peers have substantially different academic experiences. This means that “even reclassified former ELs may need opportunities and support to continue developing their academic language proficiency and content area knowledge and skills.” At the successful end of the continuum, we also need more and better data. For example, the absence of data on reclassified middle-schoolers who later graduate from high school college-ready hampers efforts to understand the kinds of practices that might have helped them succeed.

Overall, with neither an ever-EL category nor a longitudinal orientation toward collecting and analyzing EL data, we risk skewing the picture of EL achievement gaps and of our success or challenges in addressing them. Depending on how we measure the EL category, for instance, at a set of discrete time points or over a time span (ever EL), our overall picture of EL progress changes. That is why more comprehensive data on ELs is critically important.

Conclusion

Strengthening practice and policy in state and local education systems will be an essential part of reducing inequality for young people in the coming years. And a fundamental piece of the effort to raise student outcomes will be building and testing approaches that states and districts can adopt to effectively meet the needs of English learners. Research that contributes to the development of stronger assessments and broadens the data about these students can potentially go a long way toward building much-needed capacity, evaluating the effects of legislation and the ways that provisions are implemented, and, ultimately, improving the academic outcomes of a diverse student body.

Footnotes
  1. For full citations and a complete reference list, download the PDF of this essay.
Downloads

In this issue

As we approach policy changes that present important opportunities for researchers, I would like to share some “meditations” about language, as well as the worlds of education policy and practice, which I have repeated to myself over the ...
Meditations on Language and Learning: ESSA in Policy and Practice
How can research to improve the lives of youth in the United States also benefit the other 95 percent of the world’s youth? Questions like this highlight the urgency of bringing the highest levels of rigor to relevant research.
Integrating Rigor and Relevance in Research: Three Questions
By moving accountability systems from the federal to the state level and highlighting the importance of English learner progress in accountability, ESSA provides an opportunity to develop state accountability systems that better support the learning of ELs. Assessments and data ...
Learning English in the Era of the Every Student Succeeds Act
We recognize that no single effort will be transformative, but we hope that our collective efforts as researchers, research funders, universities, and professional associations can support research that, over the long term, improves the lives of young people.
Promoting Research That Is Both Rigorous and Relevant
Digest, Issue 2: Spring 2017
Put simply, the dichotomy of rigor versus relevance is false. There is no inevitable trade-off between producing rigorous research and producing research with relevance for the real world. Researchers who want their work to matter in policy and practice should ...
Bringing Rigor to Relevant Questions

More Digest Issues

The Digest

Issue 9: Winter 2023-24

The ninth issue of the Digest features an update on the Institutional Challenge Grant program five years after its launch, with insights from a Foundation stock-taking effort and reflections from two grantee teams. We also share new thinking on the use of research evidence, including ways for researchers to leverage existing findings to bolster studies of effective strategies to improve research use.
The Digest

Issue 8: Winter 2022-23

The eighth issue of the Digest features insights about where new research might help identify ways to reduce inequality in youth outcomes after COVID-19, as well as how the fields of implementation science and research on research use might learn from each other in ways that yield more transformative research in years ahead.
The Digest

Issue 7: Winter 2021-22

The seventh issue of the Digest includes fresh insights from program staff and grantees on how researchers can successfully confront the challenges of studying causal mechanisms for reducing inequality, as well as how the global community of researchers and funders focused on improving the use of research evidence can continue to break new ground in the years ahead.
The Digest

Issue 6: Winter 2020-21

Essays in this issue of the Digest focus on the importance of looking beyond individual action and customary metrics in research on reducing inequality, as well as how strengths-based, race-conscious research can be produced and used to uplift communities of color.
The Digest

Issue 5: Winter 2019-20

The fifth issue of the William T. Grant Foundation Digest features insights from program staff and grantees on the importance of looking beneath the surface to consider the underlying factors that create and shape challenges that social scientists seek to address, whether they be related to reducing inequality in youth outcomes or improving the use of research evidence in policy and practice.
The Digest

Issue 4: Winter 2018/19

The fourth issue of the William T. Grant Foundation Digest features insights that may point the way toward a more nuanced understanding of evidence use and inspire new and more wide-ranging examinations of ways to reduce inequality in youth outcomes.
The Digest

Issue 3: Winter 2017/18

The third issue of the William T. Grant Foundation Digest features insights on how research on ability tracking can inform studies to improve the outcomes of English learners, as well as how researchers and school districts can partner to build learning systems based on research evidence.
The Digest

Issue 1: Summer 2016

The introductory issue of the William T. Grant Foundation Digest features essays and commentary on the value of qualitative and mixed-methods research in reducing inequality and the potential for researcher access to big data to yield useful research evidence.