Learning by Doing Together: Improving Education Policy in Puerto Rico through a Research-Practice Partnership

Puerto Rico’s public school system was facing a daunting process of reconstruction even before the COVID-19 pandemic. In September 2017, the devastating force of Hurricane María destabilized the entirety of the island’s service infrastructure, public education included. Puerto Rico’s public school system had been historically underperforming relative to its peers in the mainland US and, amidst an ongoing economic crisis, was strained like never before. Compelled to respond to these needs, scholars at the University of Toronto (UofT) and other academic institutions established a research-practice partnership with the Puerto Rico Department of Education (PRDE), which was made possible through funding and support provided by the Foundation’s Institutional Challenge Grant.

Equitably addressing students’ educational needs in a surprisingly heterogeneous school district—the fifth largest in the U.S.—is complex. Students and educators across the territory face many challenges; some schools have lasting damage from Hurricane Maria and earthquakes that occurred in January 2020, while others face pronounced teacher shortages, unreliable internet access, and limited supply of devices that would enable online and hybrid learning. One-size-fits-all solutions were simply not reasonable alternatives; supporting all schools and meeting their needs equitably required placing the tools to tailor solutions in the hands of local leaders. We therefore began our research efforts by working in close collaboration with individuals in the education system who had both experiential knowledge and big picture insights into how the PRDE could best support its school principals, teachers, and students.

Education Management, Innovation, and Learning Initiative

This focus resulted in the development of an initiative aiming to improve academic leadership and management practices at both the school and district level. Specifically, the PRDE wished to establish a systematic approach to learn from the experiences, strategies, and practices of high-performing schools and school systems and adopt these in underperforming schools. These initiatives intend to reduce the alarming gap in academic achievement between students in Puerto Rico and the mainland US. Based on the National Assessment of Education Progress (NAEP) 2022 assessment, 94 percent of eighth graders in PR public schools do not demonstrate basic proficiency in mathematics, as compared to 40 percent in the nation or 55 percent in the worst performing state (National Center for Education Statistics, 2023). Our efforts intended to generate their greatest impacts among underperforming public schools and disadvantaged students’ educational outcomes. This is how the UofT-PRDE Education Management, Innovation, and Learning Initiative came to be.

Key to our theory of change for improving academic outcomes and reducing inequality in this underperforming school system is the notion that principals and other leaders at the school level are experts in their communities’ experiences and needs. No amount of top-down programming will produce lasting improvement in a school environment if not implemented by personnel who intimately understand and appreciate the school, students, and community’s circumstances.

EDUGESPRO Principals’ Academy

The initiative’s flagship project is the Academia de Desarrollo Profesional de la Educación para la Gestión de Liderazgo y la Profesionalización (EDUGESPRO in Spanish, and Principals´ Academy for short). Through this intervention study, we seek to identify how strengthening school principals’ academic leadership and management competencies at scale can create opportunities for school improvement. School principals participated in an intensive training program (154 contact hours) involving a two week-long session throughout the summer, in addition to monthly workshops and one-on-one coaching support during the academic year.¹ Participants learned to develop long-term strategies and plans based on the schools’ primary needs, and to implement state-of-the-art techniques to enhance school operations, including monitoring, target-setting, and the management of personnel in their own contexts.

Through this intervention study, we seek to identify how strengthening school principals’ academic leadership and management competencies at scale can create opportunities for school improvement.

In addition to co-designing the program, we jointly developed an impact evaluation of this at-scale intervention to measure program impacts on school-level academic leadership practices and on students’ academic and learning outcomes. The evaluation is based on a phased-in randomized control trial (RCT) study design, wherein all 854 PRDE schools were randomly assigned to participate in the program in one of three separate academic year cohorts (during Academic Years 2019-20, 2020-21, and 2021-22).² We collected detailed longitudinal data on academic leadership practices using the well-established Development World Management Survey (D-WMS) instrument (Bloom et al. 2015; Lemos and Scur 2017) at baseline and at the end of each academic year over the course of the study.^{3, 4} We are also studying the program’s medium-term consequences for the academic achievement of students in the PRDE system using standardized test scores and other detailed administrative data.

Interim findings point to EDUGESPRO playing a critical role in enhancing leadership practices and the general learning environment. Schools with principals in the first cohort scored significantly higher on overall management and leadership practices (as measured by D-WMS scores) than schools in the comparison group. The measured improvements represent approximately one quarter of the baseline gap in leadership competencies between PRDE schools and those in the continental US. We identified improvements in all competencies—target-setting, personnel management, monitoring strategies, and operations—closing between 17 and 75 percent of the baseline gap in the above scores for these competencies. Given our understanding of the relationship between these leadership competencies and students’ learning outcomes (Bloom et al. 2015), we expect the Principals’ Academy to be an effective strategy to improve the academic achievement of students across Puerto Rico.

Key to our theory of change for improving academic outcomes and reducing inequality in this underperforming school system is the notion that principals and other leaders at the school level are experts in their communities’ experiences and needs.

The third cohort will begin participating in the Principals’ Academy this coming school year, whilst data analysis of the first cohorts is ongoing. We expect this analysis to offer valuable insight as to how management improvement can help schools recover from crisis and raise the bar for schools in the post-pandemic “new normal.”

###

The Principals’ Academy reaffirms that achieving educational equity requires an approach that empowers leaders to tailor solutions to their communities, and that providing such training is an effective way to empower these local experts so that they can have maximum impact on their students’ educational experiences. Understanding and adapting to the lived experiences of a school community is key to producing long-term and sustainable change—as is stability and consistency. The challenges that schools in Puerto Rico continue to face are admittedly intimidating. Empowering principals and other personnel who understand their schools’ and students’ needs will be an invaluable tool in the effort to rise to these challenges.

Learn more about the Institutional Challenge Grant >>

NOTES:

¹A group of PRDE managers at each school region also participated in the program and received an additional 30 hours of training tailored to their specialized roles.

²The programming of the intervention during Academic Year 2020-21 was delayed substantially because of the COVID-19 pandemic, taking place during the period April 2021 – February 2022. The mode of delivery was hybrid synchronous and asynchronous online.

³The D-WMS is an internationally validated survey instrument designed to measure managerial practices across educational institutions. Based on participants’ responses to open-ended interview questions, the D-WMS scores the operations, monitoring, target-setting and personnel management practices of participant institutions. A sample of 400 PRDE schools yielded an average management score of 2.3 (on a scale from 0 to 4.0) at baseline, as compared to an average score of 2.8 for public schools in the mainland U.S.

⁴In an effort to understand the pathways through which measured changes in practices evolved, we also conducted semi-structured qualitative interviews with 35 participants whom we followed over the course of the multi-year program administration.