metadata: authors: 0: author: Haimovich Paz,Francisco
1: author: Vazquez,Emmanuel Jose
2: author: Adelman,Melissa Ann
count: Guatemala
docna: 0: docna: Scalable Early Warning Systems for School
            Dropout Prevention : Evidence from a 4.000-School Randomized
            Controlled Trial
docty: Policy Research Working Paper
subtopic: Educational Sciences,Educational Institutions & Facilities,Effective Schools and Teachers,Gender and Development,Hazard Risk Management,Disaster Management,Social Risk Management,Inequality
teratopic: Education,Gender,Urban Development,Conflict and Development,Social Development,Poverty Reduction
abstracts: cdata!: Across many low- and middle-income
            countries, a sizable share of young people drop out of
            school before completing a full course of basic education.
            Early warning systems that accurately identify students at
            risk of dropout and support them with targeted interventions
            have shown results and are in widespread use in high-income
            contexts. This paper presents impact evaluation results from
            an early warning system pilot program in Guatemala, a
            middle-income country where nearly 40 percent of sixth
            graders drop out before completing ninth grade. The pilot
            program, which was implemented in 17 percent of Guatemala’s
            primary schools and largely leveraging existing government
            resources, reduced the dropout rate in the transition from
            primary to lower secondary school by 4 percent (1.3
            percentage points) among schools assigned to the program,
            and by 9 percent (3 percentage points) among program
            compliers. Although the effect size is relatively modest,
            the low cost of the program (estimated at less than US$3 per
            student) and successful implementation at scale make this a
            promising and cost-effective approach for reducing dropout
            in resource-constrained contexts like Guatemala.
geo_reg_key: 119228,545133,517191
repnme: repnme: Scalable Early Warning Systems for School
            Dropout Prevention : Evidence from a 4.000-School Randomized
            Controlled Trial
display_title: Scalable Early Warning Systems for
            School Dropout Prevention : Evidence from a 4.000-School
            Randomized Controlled Trial
listing_relative_url: /research/2021/06/33130172/scalable-early-warning-systems-school-dropout-prevention-evidence-4000-school-randomized-controlled-trial
topicv3: Teachers Management m1326119 1523,Poverty m1326687 695,Education m1327048 470,Early Warning Systems m1327419 570,Gender m1327621 615
publishtoextweb_dt: 2021-06-01T00:00:00Z
historic_topic: Education,Gender,Urban Development,Conflict and Development,Social Development,Poverty Reduction
origu: Education LCR (HLCED)
no_of_pages: 42
author_keys: 0: author_key: 926496
1: author_key: 1381681
2: author_key: 1225054
model_extractions: Policy Research Working Paper 9685

# Scalable Early Warning Systems for School Dropout Prevention

Evidence from a 4.000-School Randomized Controlled Trial _Francisco Haimovich Emmanuel Vazquez Melissa Adelman_ Education Global Practice June 2021
Policy Research Working Paper 9685

# **Abstract**

Across many low- and middle-income countries, a sizable share of young people drop out of school before completing a full course of basic education. Early warning systems that accurately identify students at risk of dropout and support them with targeted interventions have shown results and are in widespread use in high-income contexts. This paper presents impact evaluation results from an early warning system pilot program in Guatemala, a middle-income country where nearly 40 percent of sixth graders drop out before completing ninth grade. The pilot program, which was implemented in 17 percent of Guatemala’s primary schools and largely leveraging existing government resources, reduced the dropout rate in the transition from primary to lower secondary school by 4 percent (1.3 percentage points) among schools assigned to the program, and by 9 percent (3 percentage points) among program compliers. Although the effect size is relatively modest, the low cost of the program (estimated at less than US$3 per student) and successful implementation at scale make this a promising and cost-effective approach for reducing dropout in resource-constrained contexts like Guatemala.

This paper is a product of the Education Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at fhaimovich@worldbank.org, evazquez@cedlas.org, and madelman@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
Scalable Early Warning Systems for School Dropout prevention: Evidence from a 4.000-School Randomized Controlled Trial<sup>*</sup> AEA RCT ID: <mark>AEARCTR-0004091</mark> .

**Key words** : Dropout prevention, Early Warning, Impact Evaluation, School Management, Guatemala

**JEL Classification** : I2, I3, J24

> * This study was possible thanks to the support of the Ministry of Education of Guatemala (Ministerio de Educación –MINEDUC). The authors want to also thank participants during seminars and presentations at World Bank and MINEDUC. The views expressed here are those of the authors alone and do not necessarily reflect the opinions of the World Bank. All errors are our own.
# Contents

|Introduction............................................................................................................................. 3|
|---|
|2. Context................................................................................................................................ 6|
|Intervention and study design................................................................................................... 7|
|The ENTRE program ........................................................................................................... 7|
|Experimental design and data................................................................................................. 13|
|Eligibility .......................................................................................................................... 13|
|Experimental design and research questions........................................................................ 13|
|Data .................................................................................................................................. 14|
|Descriptive statistics and balancing tests ............................................................................. 18|
|Results .................................................................................................................................. 20|
|Program effects (Intent-to-Treat Estimates)......................................................................... 20|
|Placebo tests ...................................................................................................................... 24|
|Mechanisms ...................................................................................................................... 27|
|Qualitative results .............................................................................................................. 30|
|Program participation......................................................................................................... 31|
|LATE ................................................................................................................................ 33|
|Final Remarks ....................................................................................................................... 36|
|References ............................................................................................................................ 38|
|ANNEX ................................................................................................................................ 40|
# 1. Introduction

Across many low- and middle- income countries, young people’s educational attainment continues to fall short of aspirations, as high rates of enrollment in early grades quickly decline due to students dropping out before completing a full course of basic education. In Guatemala, the country of focus in this paper, education is _de jure_ compulsory through ninth grade, but nearly 40% of sixth graders drop out before getting there. For youth who drop out prematurely, global evidence suggests that, on average, they will earn less and experience more social and economic challenges than their peers with more years of completed education (Patrinos and Psacharopoulos 2004; Oreopoulos and Salvanes 2011; Bentaouet-Kattan and Székely 2015; Cardenas, De Hoyos, and Székely 2015).

Dropping out of school is often not a discrete decision, but the culmination of an ongoing process of disengagement. A growing body of research in education, psychology, and behavioral economics points to the importance of understanding the early origins of dropout and young people’s underlying decision-making processes (O’Donoghue and Rabin 2001; Hammond et al 2007; Mullainathan and Shafir 2013; Frazelle and Nagel 2015). These features suggest that the most effective way to prevent dropout from occurring in the long-term is to build human capital from early childhood onwards. At the same time, there is a pressing need to directly intervene among youth already at high risk of dropping out. Many interventions have shown some promise, depending on the country context and level of education targeted – including information provision on the value of education, remedial tutoring, socioemotional skills coaching, and scholarships (Dearden et al 2009; Di et al 2013; Chappell et al 2015; and Avitabile et al 2019 among many others). These interventions vary significantly in their targeting approaches and level of intensity, resulting in important differences in the resources (human, financial, and otherwise) required to implement them successfully. In countries like Guatemala with high rates of dropout and constrained resources, there is strong demand for accurate targeting and low-cost, contextappropriate interventions.

In this paper, we report the outcomes of an early warning system pilot program in Guatemala aimed at reducing dropout in the transition from primary to lower secondary school. Early warning systems – which vary in their structure but usually include monitoring routinely collected 3
indicators of student risk and taking preventative actions tailored to students’ needs – are in place in at least 30 U.S. states and the majority of European countries (European Commission 2013; O’Cummings and Therriault 2015). As more low and middle-income countries invest in developing reliable education management information systems, student data that forms the foundation of an early warning system is becoming increasingly available. However, little evidence exists on whether and how such systems can effectively reduce dropout in countries with limited capacity to respond to students’ needs. In Guatemala, the pilot program, named ENTRE ( _Estrategia Nacional para la Transición Exitosa_ – National Strategy for Successful Transition), was designed to utilize recently developed data systems to identify students at risk and activate preventative, evidence-based measures at the school level. ENTRE focuses on three main factors identified as both particularly important for causing dropout in the primary to secondary transition in Guatemala and addressable through relatively low-cost interventions: a lack of knowledge among key primary school actors about effective measures to help students stay in school; a lack of knowledge among these same actors about which students are most at risk of dropping out; and a lack of prioritization of dropout as a problem to be addressed, as school actors have myriad responsibilities and dropout in the transition to lower secondary occurs after students have left their primary schools.

To assess the effectiveness of this early warning system approach, implementation of ENTRE was designed as a four-arm randomized experiment (three treatment arms and one control arm) among one-fifth of all the public primary schools in the country. In the first treatment arm, school principals and sixth grade teachers receive a user-friendly guidance manual and half-day training on specific, evidence-based methods to help students make the transition to lower secondary school – in essence, providing them more information on _how_ to help students stay in school. A letter signed by the Minister of Education accompanied the guidance manual, emphasizing the importance of every Guatemalan child completing at least nine years of basic education (primary and lower secondary) and the responsibility of schools to support them in doing so. The second treatment arm adds to the first a list of sixth-grade students at high risk of dropping out based on a prediction model using administrative data– providing them more accurate information on _who_ needs the most help. The third treatment arm adds to the first two small behavioral nudges to try and keep dropout prevention top-of-mind for principals and teachers. A fourth, control arm included schools that received no intervention or information about ENTRE.

4
We find that assignment to the program significantly reduces dropout by 1.3 percentage points (3.6 percent of the baseline dropout rate) in the subsequent school year. The effect size is similar across the three treatment arms, ranging from 1.1 to 1.5 percentage points, and we cannot reject the hypothesis that the impacts are equal across treatment arms. This suggests that the core intervention of providing information on _how_ dropout can be prevented is driving the results. Impacts are heterogeneous, with dropout reduction concentrated among higher-risk students, students attending larger primary schools (and therefore likely facing fewer supply constraints), and male students. Furthermore, imperfect compliance, both in terms of incomplete take-up of treatments and contamination of the control group, suggest that intent-to-treat (ITT) estimates are likely a lower bound of the effect of the program on the treated. We therefore also estimate impacts on program compliers and find that the local average treatment effect is a 3.1 percentage point reduction in dropout (about 8 percent of the baseline dropout rate), roughly double the ITT estimate but much less precisely estimated.

To further understand the mechanisms driving the results, we collected data on several intermediate outcomes and conducted a series of focus groups with program actors. Across treatment arms, school principals report an increase in their own expectations of the returns to education for their students, in their beliefs about their own ability to change students’ outcomes, their self-reported prioritization of dropout prevention among their responsibilities, and a large increase in targeting their efforts to at-risk students.

While the magnitude of the estimated impacts is modest, the program leverages the Ministry of Education’s existing administrative data structures, management systems, and personnel, with limited additional costs. Specifically, total costs per student are estimated at USD 2.91, 36% of which corresponds to the design of the program— i.e., the cost of implementing the program amounts to USD 1.85 per student— and 45% to the opportunity cost of principals and teachers. Estimates of the monetary benefits accruing to students who do not drop out from higher labor market earnings suggest an internal rate of return of 28% and a benefit-cost ratio of almost 19-to1, suggesting that ENTRE is a highly cost-effective, sustainable, and scalable approach to addressing short-run drivers of dropout. Moreover, ENTRE may be a platform program on which additional interventions to reduce dropout can be developed and rolled out, presenting a promising 5
opportunity for Guatemala and other countries interested in leveraging the basic preconditions to identify and support students at risk of dropout.

The remainder of the paper is organized as follows. Section 2 describes the education context in Guatemala. Section 3 describes the ENTRE program, while Section 4 describes the experimental design and data. Section 5 presents the impact evaluation results and evidence on likely mechanisms. Section 6 concludes.

# 2. Context

Over the past two decades, Guatemala has continued to make important progress in expanding access to primary and secondary education, as well as in improving student achievement. Primary and lower secondary school completion rates have increased by 43% and 112%, respectively, since 2000. At the same time, average learning outcomes have also improved – Guatemalan sixth graders’ performance in both reading and math improved much more between 2006 and 2013 than the LAC region overall (World Bank, 2019). Increasing average learning outcomes while expanding access is a significant accomplishment, yet much remains to be done.

Despite this progress, early school dropout remains an important challenge to increasing educational attainment in Guatemala. School dropout occurs at all ages, but becomes widespread starting in early adolescence. Whereas net enrollment is over 85 percent in primary school, it falls to only 45 percent in lower secondary and 25 percent in upper secondary (UNESCO, 2020). Dropout is particularly concentrated in the transitions from primary to lower secondary (6th to 7th grade) and from lower to upper secondary (9th to 10th grade), where approximately one out of three students drops out in each of these transitions (Adelman et al, 2018).

These high dropout rates reflect systemic issues that require medium to long term approaches and significant investment to comprehensively address. In Guatemala, children’s needs are immense. Over half the population is under the national poverty line, and the country has the world’s sixth highest rate of chronic malnutrition (World Bank, 2020). As a result, many children come to the school system suffering from multiple dimensions of deprivation, yet public education services are inadequate. The Government of Guatemala spends less than 3% of GDP on education, well below regional peers, primarily due to low revenues overall, as the government does devote over 20% of its budget to education. As a result, both the quantity and quality of education supply are 6
insufficient to meet existing needs. For example, preschool starts relatively late, typically around age 6, only lasts for one year, and covers less than half of the population. In addition, the supply of secondary education is also insufficient and skewed towards urban and more prosperous areas of the country. In the departments with the highest poverty rates, there is only one secondary school for every 400-500 secondary-school age youth, while in lower poverty departments, this ratio is closer to one school for every 100 youth.

Yet there is a pressing need for targeted and scalable action in the short term among the large pool of youth already at high risk of dropping out. Evidence from across Latin America shows that greater shares of youth who are out of school and out of work contribute to a range of negative social and economic outcomes, including higher rates of crime, long-term reductions in labor productivity and growth, and increases in inequality (De Hoyos, Rogers, and Székely 2016).

# 3. Intervention and study design

The ENTRE program In 2017, motivated by the challenges and constraints described above, the Government of Guatemala, with support from the World Bank, designed the ENTRE program as a scalable and low-cost early warning system. The program is aimed at reducing dropout in the transition from primary to lower secondary school, where almost a third of students stop their formal education.

Like most early warning systems, the key goals of ENTRE are to provide targeted support to students at-risk and to take timely, preventive actions. At a minimum, this requires that school principals and teachers have a basic knowledge of effective dropout prevention strategies, are able to identify students at risk, and implement these strategies in a timely manner. However, quantitative and qualitative data—including focus groups with teachers and school principals, as well as interviews with public officials conducted to inform the design of ENTRE — showed important gaps in all these dimensions.

First, school actors have limited knowledge of, or support on, dropout prevention strategies, and school principals—who play a leading role in these initiatives— are usually regular teachers 7
without any specific leadership or management preparation. Second, school actors have limited information on _who_ is at risk of dropping out. Data on mandatory “dropout risk flags” reported by school principals and teachers show that they identify on average fewer than 5% of the students who end up dropping out in the transition from primary to lower secondary.<sup>1</sup> Third, there are several parallel demands and multiple tasks that compete for the attention of principals and teachers, which may prevent dropout prevention from being top-of-mind, particularly since dropout in the transition from primary to lower secondary occurs after students have left their primary schools and are no longer under their official responsibility. For instance, surveys of school principals show that reducing dropout is not a top priority for them, as they rank it third on average out of five options (below teaching/curriculum and internal administrative tasks).

Considering these gaps, the government designed ENTRE to combine three key components: a brief training on actionable, evidence-based strategies to support students to stay in school; information on sixth-grade students at high risk of dropping out; and small behavioral nudges to try and keep dropout prevention top-of-mind for principals and teachers.

# <u>Component 1: Short training on actionable strategies to prevent dropout</u>

The first component consists of a half-day training for school principals and sixth-grade teachers on specific, evidence-based methods to help students make the transition from primary to lower secondary – in essence, providing them more information on _how_ to help students stay in school. The training was complemented by a short and user-friendly guidance manual (13 pages of text and informative illustrations in the main manual, plus 13 pages of additional resources in annex) that summarizes the training content, a copy of which was given to each participant at the beginning of the half-day training. Along with the guide, the school principals received a personalized letter from the Minister of Education that emphasized the importance of every Guatemalan child completing at least nine years of basic education and urged the school principal to lead her or his staff in implementing what they learned through the ENTRE program.

> 1Part of this could be explained by perverse incentives to under-report students at risks. The data mentioned is collected by a program that asked school actors to identify students at risk. After identifying students at risk, several administrative tasks were triggered for each student, creating significant additional work for principals and teachers.

8
The training covered the objectives of ENTRE, some of the key barriers to staying in school (lack of motivation, financial constraints, lack of family support, poor academic performance, and inertia), and several methods and resources to address these barriers. For instance, to address the lack of motivation to continue studying, in line with international evidence on cost-effective strategies for this barrier, the training presented information on economic returns to education in Guatemala (which are among the largest in Latin America) and recommended a set of activities to effectively communicate these returns to students. The participants, divided into small groups, then had the opportunity to practice the approach with role-play exercises, and to brainstorm about the logistics of implementation. The sixth-grade teacher and the principal of the same school were intentionally placed together so they could start discussing implementation in their own school. To address financial constraints, the participants were informed about means and merit-based scholarships available through the Ministry of Education (MINEDUC) and provided with instructions on how to apply to them (including where to locate the required forms). As highlighted by empirical evidence (see Bettinger et al, 2018), the participants were also informed about the importance of helping disadvantaged families to fill out these forms, since many times despite qualifying for this type of benefit, low-income families do not apply to them.

The training also included a section on the importance of focusing efforts on students at-risk of dropout. Participants were encouraged to use their own judgement to assess this risk, but they were also informed that some schools would receive complementary information on students at-risk as part of MINEDUC’s efforts to identify the most effective content for ENTRE (component 2).

# <u>Component 2: Information on students at high risk of dropping out</u>

The second component of ENTRE is a list of sixth-grade students at high risk of dropping out in the transition to lower secondary, providing school actors with more information on _who_ needs the most help. The list was estimated using routinely collected administrative panel data on students and combining linear regression models with a simple algorithm. These models are able to correctly identify 82% of the sixth-grade students who will drop out within the next year, performing better than other commonly used targeting approaches, and as well as models used in the U.S. (For more details on the methodology, please see Adelman et al., 2018).

9
The list was specific for each school, providing the name and identification number of each student, and was addressed to the school principal. To reduce any risk of negative labeling, both the letter from the Minister and the content of the training emphasized the fundamental importance of avoiding any type of stigmatization of students at risk of dropping out. The main objective of this component was to help school actors target their support to students most in need of it, given limited time and resources.<sup>2</sup>

# <u>Component 3: Behavioral nudges</u>

The third component consisted of five monthly reminders sent to school principals in order to keep dropout top-of-mind and motivate them to act. The messages were sent through SIRE, an online portal that school principals use regularly to conduct different administrative tasks and exchange information with MINEDUC.<sup>3</sup> Each month, between June 20 and October 1, these messages appeared on the homepage of the portal as soon as school principals logged in. The reminder messages utilize various behavioral insights to motivate action, namely:

- **Social recognition (message 1)** : Social incentives can exert a powerful effect on behavior given the human desire for status and recognition. Social rewards, such as status and recognition, can motivate people to exert effort and can even substitute for monetary rewards in some situations. The first message of SIRE was inspired by this theory, informing school principals about an official certificate to recognize the schools that made the most progress in terms of reducing dropout and the publication of the list of recognized schools in a popular newspaper in Guatemala. This information can act as an extrinsic incentive for school teams to act.

- **Salience (messages 2-4):** Research shows that people are more likely to respond to stimuli that are timely and accessible, and are more likely to do something that their attention is drawn towards. Even though school principals are being provided with concrete ideas,

> 2 In addition, this information could have also created accountability incentives, given that the schools were informed in advance about the specific students at risk and their dropout status can be monitored easily through MINEDUC’s online portal, SIRE.

> 3 About 85% of school principals reported in the baseline questionnaire that they use SIRE either “much” or “to a certain extent”, while only 15% of them reported using it either “a little” or “never” (14.3% and 0.25%, respectively).

10
<!-- Start of picture text --> Sistema de Registros Educativos<br>oeOWeeVeg@ cere’ @ itea: TGUigeec esNee! hiaie2a0 ae<br>hn. eee‘i- %S- af ‘wes cv rere ao BS i<br>BIENVENIDO AL SISTEMA NACIONAL DE REGISTRO EDUCATIVO:<br>ALERTAS:<br>nowene an DARIOREGULAR “Mobiiaro<br>ppecreew tna y Equipo<br>mere -_ ae {Ganeun Reconocimiento! |Aumente la inscripcioe n Primero Basico!<br><!-- End of picture text -->
<!-- Start of picture text --> Sistema de Registros Educativos<br>Inicio de Sesién: 07:20<br>Tiempo~ ‘ de Sesién: @0 :ee4 : 31 . > gen ow ae ? Daa<br>4 areQa iSSt Nagsa"as ol “1a. ‘a : WateWES &- ¥ i yo+) meats.exerprereePtenvettttesheatpo 4ASif Pray omy4<br>Seen me ots es od<br>INICIO CICLO LECTIVO DOCENTES ESTUDIANTES REGISTROS REPORTES INFRAESTRUCTURA Estrategia ENTRE<br>Su establecimiento puede ser distinguido con el “Reconocimiento Honorifico por su Accién<br>Efectiva para prevenir el abandono escolar”. Esta distincion se otorgara a quienes logren reducir<br>mas el abandono escolar de sexto grado e incrementar la inscripcién en primero basico. El<br>reconocimiento consiste en un diploma para su escuela extendido por el Ministro de Educacién,<br>el cual podra ser visible a la comunidad educativa y el nombre de los establecimientos que<br>obtengan mejores resultados seran publicados en uno de los principales diarios de Guatemala.<br>Ministerio de Educacién, Copyright© 2,015<br><!-- End of picture text -->
supervisors about the presence of a new reminder in SIRE and were invited to enter the online portal each time. The platform was also used to capture baseline and follow-up information through online questionnaires, which complemented baseline administrative information available at the beginning of the school year and a follow-up survey implemented on paper after the school year came to an end.

# 4. Experimental design and data

## Eligibility

As mentioned above, there are areas in Guatemala where children just do not have reasonable access to secondary schools. Given its features, ENTRE is targeted at schools and students where the supply-side constraints are less likely to be binding. Therefore, to minimize the role of these constraints, the universe of eligible schools was restricted to public schools in the top 30% of municipalities in terms of supply of secondary schools per student. In addition, within those municipalities, the very small schools likely to be located in remote rural areas (the schools belonging to the bottom 15% in terms of school size) were excluded from eligibility, as these schools tend to face binding local secondary school supply limitations. Finally, to target schools that could benefit more from the type of information provided by the program, we dropped schools that—according to our predictive model—had zero students at-risk of dropping out. As a result, the universe of eligible schools totals 6,037 public primary schools.<sup>6</sup>

## Experimental design and research questions

The program was designed as a multi-arm randomized experiment. Randomization was used to create four experimental groups of 1,000 schools each (three treatment arms and one control arm). Namely, from the 6,037 eligible schools, a simple random sample of 4,000 was drawn and then each was randomly assigned to one of the four experimental groups. The schools in the first treatment group receive component 1 (the half-day training and the user-friendly guidance manual, as well as a letter from the Minister of Education) which is aimed at improving knowledge on simple evidence-based methods to help students make the transition to lower secondary school –

> 6 In total, there are approximately 15,930 public and 2,699 private primary schools in Guatemala.

13
in essence, providing teachers and principals with more information on _how_ to help students stay in school. The second treatment arm adds to the first arm the list of sixth-grade students at high risk of dropping out (component 2), providing school actors more accurate information on _who_ needs the most help. The third treatment arm adds to the first two arms small behavioral nudges (component 3) to try and keep dropout prevention top-of-mind for principals and teachers. A fourth, control arm included schools that received no intervention. This multi-arm design, summarized in Diagram 1, allows us to disentangle the impact of each component on dropout.

**Diagram 1** : four-experimental groups

<!-- Start of picture text --> TREATMENT 1<br>Guide/training<br>1000 schools<br><!-- End of picture text --> <!-- Start of picture text --> TREATMENT 2<br>Guide/training<br>+ List<br>1000 schools<br><!-- End of picture text --> <!-- Start of picture text --> TREATMENT 3<br>Guide/training<br>+ List<br>+ Reminders<br>1000 schools<br><!-- End of picture text --> <!-- Start of picture text --> CONTROL<br>No intervention<br>1000 schools<br><!-- End of picture text -->

# Data

Our main data source is the administrative school data set compiled by SIRE. This data set includes unique student identifiers that has enabled longitudinal tracking of student academic records since 2011. The academic records include basic demographic information and detailed grades earned by subject and year, as well as information on enrollment status, including whether the student dropped out and whether they passed the grade. This data was the main input for estimating the dropout prediction model, as well as the data source for the main outcome of interest (student level dropout).

We also collected data on the quality of implementation of the program and several intermediate (self-reported) outcomes through a series of paper-based and online questionnaires integrated into the SIRE platform. All schools participating in the experiment were requested to complete an baseline questionnaire online, two follow-up online surveys, and a final, paper-based follow-up survey. In addition, the 3,000 treatment schools were also requested to complete an online 14
comprehension test for the guidance manual and paper-based questionnaires at the end of the halfday training. Table 1 summarizes the main questionnaires used to collect data.

Table 1. Summary of data sources

|**Data source/**<br>**Questionnaire**|**Date**|**Sample**|**Type**|**Key information**<br>**collected**|**Response**<br>**rate (%)**|
|---|---|---|---|---|---|
|**SIRE**<br>**administrative**<br>**data**|2016<br>(placebo),<br>2017<br>(placebo /<br>prediction)<br>; 2018<br>(baseline),<br>2019<br>(follow-<br>up)|All<br>schools<br>(4,000)|Online|Key administrative<br>student-level data<br>(including Students’ IDs,<br>detailed grades, age,<br>gender, and dropout<br>status)|100%|
|**Complementary**<br>**baseline data**|May 7<sup>th</sup>-<br>May 29<sup>th</sup>|All<br>schools<br>(4,000)|Online|Perceptions on school<br>dropout, returns to<br>education and influences<br>on the dropout decision,<br>as well as targeted<br>support and tasks<br>prioritized by principals.|70%|
|**Training**<br>**feedback**|May 29<sup>th</sup>-<br>June 6<sup>th</sup>|All<br>treatment<br>schools<br>(3,000)|Paper-<br>based|Feedback on the content<br>and quality of the<br>training received and<br>about the trainer<br>assigned. Confirmation<br>of reception of materials<br>(guide/list), and self-<br>reported level of<br>understanding and|91%|

15
|||||satisfaction with the<br>training.||
|---|---|---|---|---|---|
|**Comprehension**<br>**test**|July 16<sup>th</sup>|All<br>treatment<br>schools<br>(3,000)|Online|Understanding of the<br>content in the Guide<br>captured by 12 multiple<br>choice questions.|45%|
|**First-follow up**|August<br>25<sup>th</sup>|All<br>schools<br>(4,000)|Online|Perceptions on school<br>dropout, returns to<br>education and influence<br>on dropout decision, as<br>well as tasks prioritized<br>by principals.|34%|
|**Second follow-**<br>**up-follow**|September<br>25<sup>th</sup>|All<br>schools<br>(4,000)|Online|Guide and list take-up,<br>implementation details<br>and degree of targeted<br>support.|24%|
|**Third follow-up**|April 15<sup>th</sup>-<br>May15<sup>th</sup><br>(next year)|All<br>schools<br>(4,000)|Paper-<br>based|Perceptions on school<br>dropout, returns to<br>education and influence<br>on dropout decision, as<br>well as targeted support<br>and tasks prioritized by<br>principals + Guide and<br>list take-up and<br>implementation details.|85%|

Below, we summarize the main data sources for different parts of the analysis, including program participation, outcomes, and potential mechanisms.

# **Program Participation**

To assess the take-up of the program we utilize multiple data sources. For treatments 1 and 2, we use school principals’ self-reports on the reception of the dropout prevention guide (which is also 16
a proxy of participating in the training) and the list of students at risk, respectively. This information was collected in the last follow-up questionnaire, which was the only paper-based follow-up questionnaire and the one with the highest response rate. The proportion of schools with data on the reception of the guide was about 85 %, with no significant differences across control and treatment arms. For reception of the list of students at risk, the response rate was lower, with significant differences between the treatment and the control groups (the proportion of schools with data on this question was 77% and 55%, respectively). The difference may be driven by school principals in the control group not understanding the question (as they likely never heard about the list of students at-risk). The data on the take-up of treatment 3 comes directly from SIRE administrative records and is proxied by a dummy capturing whether the school principals logged into SIRE at least once in the month that followed the activation of the reminders on the platform.

# **Outcome data**

The main outcome indicator is a dummy variable capturing whether individual students who were enrolled in primary school in April 2018 had dropped out by April 2019. To construct this variable, we exploit the panel structure of our data—using the students’ identification numbers to track their academic trajectories through different years—with a variable reporting the enrollment status of the students during any given academic year (i.e., currently enrolled or dropped out). If a student is not found in the next year’s panel, we consider that the student has dropped out. In addition, potential errors in the linkage of students’ administrative records across years would also be interpreted as dropouts in our dataset. We do not expect these challenges would affect our identification strategy for two main reasons. First, even with these limitations, the SIRE data replicates well the dropout patterns observed in Guatemalan household surveys (Adelman et al, 2018). Second, there is no reason to believe that this potential measurement error would be different across randomly selected control and treatment groups. In short, we expect relatively small and random measurement error in dropout, creating (if anything) a small downward bias to our estimates.

# **Data on potential mechanisms**

The data on mechanisms was mostly collected via online questionnaires. There were some challenges with the response rates of the online questionnaires capturing the potential mechanisms, 17
so we consider analysis of this data as exploratory. In particular, 26% (22%) of the treatment (control) schools missed the baseline questionnaire, 13% (11%) the final paper-based questionnaire, and 59% (63%) the short-term online follow-up. However, the key results are robust to controlling for baseline responses and to dropping T3 schools, which had higher response rates.<sup>7</sup> Finally, the attrition rate is not correlated with the baseline responses, nor with student dropout rates at baseline or follow-up.

## **Qualitative analysis**

We also performed a series of focus groups (12 in total) to gather qualitative information on the challenges of program implementation. We interviewed almost 100 people involved in the program, including 63 teachers and principals from across treatment arms and 34 trainers in 4 different locations.<sup>8</sup> Short paper-based questionnaires were also completed during these focus group sessions.

# Descriptive statistics and balancing tests

Table 2 reports the average baseline characteristics for the students and their schools according to the administrative data available.<sup>9</sup> The table reports the averages for the control arm (column 1), the pooled treatment arms (column 2), and each treatment arm separately (columns 3-5). The administrative data includes detailed academic grades by subject (math, language, natural sciences, etc.), age, gender, school dropout rates, the average dropout probability for the cohort, and regional variables.

Overall treatment and control groups are relatively balanced. Most of the differences in the variables considered are statistically insignificant. The chi squared test after running seemingly unrelated regressions of each of these baseline variables on treatment assignment shows that the

> 7 These schools had higher response rates in online questionnaires likely because the reminders made schools more prone to see the survey link; furthermore, they explain all the difference on the response rates between treatment and control schools in the online survey used.

> 8 The teachers and principals come from various departments, including Quetzaltenango, Quiché, Totonicapán, Chimaltenango, Sololá, Sacatepéquez, Jalapa, and Jutiapa.

> 9 We did not include in this analysis the complementary online data given that the take-up was roughly 70%.

18
differences in all the pre-treatment administrative variables are jointly insignificant (the lowest p- value is 0.22).

However, the treatment groups perform worse than the control group in two key dimensions, namely dropout (36% for the pooled treatment vs. 34%), and school size (268 vs. 296, respectively), suggesting that the treatment schools are relatively more vulnerable than the control schools. In particular, the 2-percentage points gap in dropout rates is an issue because it creates a significant downward bias in the impact of the program (particularly taking into account that we are analyzing a very low-cost intervention).

To correct for this bias, as will be discussed in the next section, we will include school-fixed effects in our estimations (exploiting the availability of repeated cross-sections from the administrative data).

Table 2. Baseline average characteristics of the students in control and treatment groups.

|_Variable_|_Control_|_ENTRE_|_ENTRE 1_|_ENTRE 2_|_ENTRE 3_|
|---|---|---|---|---|---|
|Age|1291|1294|1292|1295*|1294|
|1 = Male|.<br>0.50|.<br>0.51|.<br>0.51|.<br>0.52|.<br>0.51|
|1 = Repeater|0.01|0.01|0.01|0.01|0.01|
|Grade in math (previous year)|7.25|7.25|7.25|7.25|7.25|
|Grade in language 1 (previous year)|7.42|7.41|7.39|7.44|7.41|
|<br>Grade in language 2 (previous year)|7.45|7.43|7.42|7.45|7.43|
|Grade in language 3 (previous year)|7.48|7.43*|7.42|7.45|7.41*|
|Grade in natural science (previous year)<br>|7.47<br>|7.44<br>|7.44<br>|7.44<br>|7.45<br>|
|Grade in social science (previous year)|7.42|7.42|7.41|7.43|7.41|
|Grade in artistic expression (previous year)|7.80|7.78|7.77|7.80|7.78|
|Grade in physical education (previous year)|8.07|8.04|8.03|8.05|8.04|
|Grade in citizenship education (previous year)|7.68|7.67|7.66|7.68|7.65|
|Grade in productivity and development (previous year)|7.84|7.82|7.81|7.84|7.81|
|Risk|0.34|0.36**<br>|0.37*<br>|0.36|0.36|
|Dropout rate in the transition from 6th to 7th grade in the school that the student attends|0.33|0.36**|0.36*|0.36|0.36|
|Average age in 6th grade in the school that the student attends|12.91|12.94|12.92|12.95*|12.94|
|Proportion of male students in 6th grade in the school that the student attends|0.50|0.51|0.51|0.52|0.51|
|Proportion of repeaters in 6th grade in the school that the student attends|0.01|0.01|0.01|0.01|0.01|
|Number of students at risk in the school that the student attends|11.47|12.58|12.82|13.05|11.86|
|Size of the school that the student attends|296.37|268.39*|268.77|261.83*|274.61|
|Average score in math in 6th grade (previous year) in the school that the student attends|7.30|7.31|7.30|7.27|7.36*|
|Average score in language 1 in 6th grade (previous year) in the school that the student attends|7.47|7.49|7.48|7.47|7.52|
|Average score in language 2 in 6th grade (previous year) in the school that the student attends|7.53|7.50|7.50|7.50|7.51|
|Average score in language 3 in 6th grade (previous year) in the school that the student attends|7.52|7.50|7.49|7.51|7.51|
|Average score in natural sience in 6th grade (previous year) in the school that the student attends|7.51|7.52|7.53|7.51|7.54|
|<br>Average score in social science in 6th grade (previous year) in the school that the student attends|7.46|7.50|7.49|7.47|7.54**|
|Average score in artistic expression in 6th grade (previous year) in the school that the student attends|7.85|7.85|7.85|7.83|7.87|
|Average score in physical education in 6th grade (previous year) in the school that the student attends|8.10|8.07|8.04|8.08|8.08|
|Average score in citizenship education in 6th grade (previous year) in the school that the student|7.75|7.75|7.74|7.74|7.76|
|<br>Average score in productivity and development in 6th grade (previous year) in the sch. the stud. attends|7.91|7.91|7.89|7.93|7.92|
|% of students in schools with bilingual modality<br>|0.39<br>|0.41<br>|0.41<br>|0.41<br>|0.41<br>|
|% of students in municipalities with SINAE|0.21|0.21|0.21|0.19|0.22|
|% of students in municipalities prioritized by poverty|0.53|0.54|0.53|0.56|0.53|
|% of students in Guatemala|0.18|0.19|0.19|0.17|0.20|
|% of students in El Progreso<br>|0.01<br>|0.01<br>|0.01<br>|0.01<br>|0**<br>|
|% of students in Sacatepequez|0.01|0.01|0.01|0.01|0.01|
|% of students in Chimaltenango|0.05|0.05|0.05|0.04|0.05|
|% of students in Escuintla<br>|0.06|0.06|0.05<br>|0.06|0.06|
|% of students in Santa rosa|0.03|0.01|0.01*|0.02|0.01|
|% of students in Solola|0.04|0.03|0.03|0.05|0.02*|
|% of students in Totonicapan|0.04|0.04|0.04|0.04|0.04|
|% of students in Quetzaltenango<br>|0.03<br>|0.04<br>|0.05*<br>|0.04<br>|0.03<br>|
|% of students in Suchitepequez|0.03|0.03|0.03|0.03|0.03|
|% of students in Retalhuleu|0.02|0.02|0.03|0.02|0.02|
|% of students in San marcos<br>|0.09<br>|0.09<br>|0.09<br>|0.08<br>*|0.09<br>|
|% of students in Huehuetenango<br>% of students in Quiche|0.04<br>0.07|0.04<br>0.07|0.04<br>0.07|0.06<br>0.08|0.04<br>0.07|
|% of students in Baja Verapaz|003|003|002|003|003|
|<br>% of students in Alta Verapaz|.<br>0.10|.<br>0.10|.<br>0.09|.<br>0.10|.<br>0.11|
|<br>% of students in Peten|0.05|0.04|0.05|0.03**|0.06|
|% of students in Izabal|0.04|0.04|0.04|0.04|0.05|
|% of students in Zacapa|0.00|0.00|0.00|0.00|0.00|
|<br>% of students in Chiquimula|0.03|0.04|0.03|0.04|0.04|
|% of students in Jalapa|0.05|0.04*|0.03*|0.04|0.03*|
|% of students in Jutiapa|0.03|0.02|0.03|0.03|0.02|
|_Test of joint significance(p-value)_||0.90|0.98|0.22|0.45|
|_Number of observations_|19940|57235|18974|19175|19086|

Notes: (1) * Statistically significant difference with respect to the control group at 10%. ** Statistically significant difference with respect to the control group at 5%. *** Statistically significant difference with respect to the control group at 1%. (2) Standard errors are clustered at school level.

19
# 5. Results

Program effects (Intent-to-Treat Estimates) We start by analyzing the intent-to-treat estimates on dropout. We first estimate the following “pooled” model –which maximizes our power to assess the impact—using a linear probability model (estimated with OLS): Where s is the dropout status of student _s_ in school _j_ between year _t_ and year _t+1, Entre_ is a dummy capturing whether the school was (randomly) assigned to the program,ss P _is a_ dummy yy indicating whether the program is already being implemented (i.e., whether t=school year 2018-ss PPEE 19), represent the students characteristics, and and are year and school fixed-effects.<sup>10</sup> Standard errors are clustered by school and year. ss ss XX αα αα In addition, we estimate the following model to capture the impact of the (random) assignment to the three alternative treatment arms: 3 s .(E E s .(P )) s s (2) ss kk=1 kk ss ss ss ss ss Table 3 reports the estimated models. In this table, columns 1-3 show the results for three different yy = αα+<sup>∑</sup> ββ EE(kk) PPEE + θθ. XX + αα + αα + εε specifications: (i) no control variables, (ii) basic student-level characteristics, and (iii) studentlevel characteristics and school-fixed effects (the last one is our preferred specification and corresponds to equation (1)).

The results for the pooled model are reported in the first panel of Table 3. The magnitude of the effect is quite similar in all three specifications, but we gain more precise estimates as we add more

> 10 Alternative specifications are considered in the robustness section.

20
controls.<sup>11</sup> According to our preferred specification (column 3), attending a school that was assigned to the program significantly reduces dropout by 1.3 percentage points (about 4 percent of the baseline dropout rate) and this effect is significant at the 1 percent level. In the second panel, we report the estimated ITT estimates for each arm. The results for our preferred specification (column 3) indicate that the effect of the treatment is similar across the different arms, ranging from 1.1 to 1.5 percentage points, with all the coefficients statically significant at the 5 or 10 percent level (depending on the arm considered). In fact, we cannot reject the hypothesis that the impact is the same across the three treatment arms, suggesting that the basic intervention on _how_ dropout can be prevented is mostly driving the results.

Table 3. Impact of the program on the probability of student dropout

|||ITT||
|---|---|---|---|
||(1)|(2)|(3)|
|**_Panel (i): Pooled Model_**||||
|_ENTRE j_|-0.010|-0.013**|-0.013***|
||(0.018)|(0.006)|(0.004)|
|Observations|152,673|145,628|145,628|
|R-squared|0.000|0.301|0.331|
|**_Panel (ii): Treatment arms_**||||
|_ENTRE 1 j_|-0.013|-0.016**|-0.015***|
||(0.022)|(0.008)|(0.005)|
|_ENTRE 2 j_|-0.011|-0.010|-0.011**|
||(0.022)|(0.007)|(0.005)|
|_ENTRE 3 j_|-0.007|-0.013*|-0.013**|
||(0.023)|(0.007)|(0.005)|
|Observations|152,673|145,628|145,628|
|R-squared|0.000|0.301|0.331|
|Controls at student level|NO|YES|YES|
|School fixed effects|NO|NO|YES|

Notes: 1) Standard errors clustered at school-year level in parenthesis. 2) *** p<0.01, ** p<0.05, * p<0.1. 3) The dependent variable is a dummy variable that is equal to 1 if student _s_ in school _j_ dropped out school between year _t_ and year _t+1_ ; 4) _ENTREj_ represents the interaction _Entrejt_ x _Postt_ where _Entrejt_ : (1= Student _s_ attends a school _j_ in year _t_ that is in the treatment group) and _Postt_ : (1 = year of the intervention; 0 = year before the intervention); _ENTRE ij_ with _i_ =1,2,3 captures the same interaction as _ENTREj_ but using assignment to the treatment arm _i_ instead of assignment to the general treatment group. 5) All specifications in panel (i) include the corresponding _Postt_ and _Entrejt_ dummies, while those in panel (ii) include the _Postt_ and _Entre ijt_ dummies; 6) Student characteristics used as control: age dummies, gender and risk (probability of dropout estimated in the Early Warning System).

11 This is indeed a general pattern that we observe throughout the analysis conducted in this paper. For simplification purposes, in the rest of the paper we mostly report the equivalent to column (iii) –our preferred specification —but more detailed tables are available upon request. We also conduct additional checks in the robustness and placebo sections.

21
In Table 4, we report the same models but include interactions with variables to capture the presence of heterogeneous effects, namely: (i) the ex-ante probability of dropout of the students (based on the predictive model described above); (ii) the school size; and (iii) the gender of the student. As in the previous table, the first panel reports the pooled model, and the second panel the corresponding coefficients for the three arms.<sup>12</sup> Table 4. Heterogeneity in the impact of the program on the probability of student dropout.

|||Interaction:<br>||
|---|---|---|---|
||(i) Pre-treatment<br>dropout probability|(ii) School size<br>(=1if large school)|(iii) Gender<br>(=1if boy)|
|**_Panel (i): Pooled Model_**||||
|ENTRE_j_+ (ENTRE_j_) x (_Interaction j_)|-0.029**|-0.020***|-0.019***|
||(0.012)|(0.006)|(0.007)|
|_ENTRE j_|0.001|-0.005|-0.007|
||(0.006)|(0.006)|(0.007)|
|Observations|145,628|145,628|145,628|
|R-squared|0.333|0.331|0.331|
|**_Panel (ii): Treatment arms_**||||
|ENTRE 1_j_ +(ENTRE 1_j_) x (_Interaction j_)|-0.032**|-0.026***|-0.032***|
||(0.015)|(0.008)|(0.009)|
|ENTRE 2_j_ +(ENTRE 2_j_) x (_Interaction j_)|-0.036**|-0.014**|-0.011|
||(0.014)|(0.007)|(0.008)|
|ENTRE 3_j_ +(ENTRE 3_j_) x (_Interaction j_)|-0.020|-0.021***|-0.013|
||(0.015)|(0.007)|(0.009)|
|_ENTRE 1 j_|0.000|-0.003|0.003|
||(0.008)|(0.008)|(0.009)|
|_ENTRE 2 j_|0.007|-0.007|-0.011|
||(0.007)|(0.008)|(0.009)|
|_ENTRE 3 j_|-0.005|-0.004|-0.013|
||(0.007)|(0.008)|(0.009)|
|Observations|145,628|145,628|145,628|
|R-squared|0.333|0.331|0.332|
|Controls at student level|YES|YES|YES|
|School fixed effects|YES|YES|YES|

Notes: 1) Standard errors clustered at school-year level in parenthesis; 2) *** p<0.01, ** p<0.05, * p<0.1. 3) The dependent variable is a dummy variable that is equal to 1 if student _s_ in school _j_ dropped out school between year t and year t+1. 4) _ENTREj_ represents the interaction _Entrejt_ x _Postt_ where _Entrejt_ : (1= Student _s_ attends a school _j_ in year _t_ that is in the treatment group) and _Postt_ : (1 = year of the intervention; 0 = year before the intervention); _ENTRE ij_ with i=1,2,3 captures the same interaction as _ENTREj_ but using assignment to the treatment arm _i_ instead of assignment to the general treatment group. 5) _Interaction j_ in column (i) is given by the variable _Risk sjt_ (Probability of dropout for student _s_ that attends school _j_ in year _t_ estimated in the Early Warning System); in column (ii) by _Large school sjt_ : (1=Large school: size in 2017 > median), and in column (iii) by _Man sjt_ : (1=Man). 6) All specifications in panel (i) include the corresponding _Postt_ , _Entrejt_ and _Interaction j_ dummies, while those in panel (ii) include the _Postt_ and _Entre i jt_ dummies; 6) Student characteristics used as control: age dummies, gender and risk (probability of dropout estimated in the Early Warning System).

12 In all these cases, to avoid any potential bias, we included interactions not only with the variable capturing the impact of the assignment to the program ((Entre)x(Post)) but also with both the program (when relevant) and year dummy, and we included the variable of interest as a direct control as well.

22
First, we analyze whether the impact of the program varies according to students’ ex-ante probability of dropout. Targeting support to at-risk students is one of the cornerstones of the program’s design, and we would expect a larger impact for those identified as at-risk because they are the population whose outcomes may change due to intervention (versus low-risk students, who are likely to transition to secondary school regardless). Indeed, we find that the impact of the program is almost entirely driven by the students with higher risk of dropout. For the overall program effect, as shown in column 1 of table 4, while the impact of the program was negligible for students with a very low risk of dropout (0.1 percentage points), the program reduced dropout by almost 3 percentage points for the students with the highest risk. This pattern is similar when analyzing the different treatment arms (column 1, second panel), with the largest point estimate corresponding to treatment 2 with a 3.6 percentage point gap. For treatment 3, the pattern is quite similar, but the estimated impact is not statistically significant.

Second, we analyze heterogeneity based on school size (second column of table 4). Students in larger schools could benefit more from the program for at least two reasons: first, the fact that supply constrains are less likely to be binding in the areas where they are located (as usually there is more supply of secondary education around larger schools), and second, targeting support is probably more relevant in larger schools, where teachers and principals may have less time for each student. As shown in the first panel, the impact is 1.5 percentage points bigger in large schools (schools above the median school size), adding to 2 percentage points of reduction in dropout. In small schools, the results suggest a (statistically insignificant) reduction in dropout of 0.5 percentage points. The pattern is quite similar across treatment arms (second panel), with the largest point estimate corresponding to treatments 1 and 3.

Third, we analyze heterogeneity by gender (third column of table 4). Results could vary by gender for several reasons, including whether the treatment itself is better suited for boys or girls, or whether teachers or principals have biased perceptions on who needs the treatment more. The first panel show that the overall impact of the program tended to favor boys. The reduction in dropout was 1.9 percentage points for boys and only 0.7 percentages points for girls (and this coefficient is not statistically significant). However, this result masks important heterogeneity across treatment groups. In treatment 1—the only group that did not receive “objective” information on the dropout risk by student – basically all the effect of the program is explained by boys (3.2 percentage points), while girls did not benefit from the program at all (the coefficient is indeed 23
~~Ac~~
April 2018 and April 2019 (i.e., before and after the pilot was implemented), in our placebo experiments we compare them between April 2017 and April 2018 (i.e., one year before the implementation of the ENTRE pilot). If our main experiment is just picking up pre-existing trends, these trends should be also captured by our placebo experiments. Tables 5 and 6 show that the coefficients of the placebo regressions are all quite small in magnitude and statistically insignificant, providing strong evidence against pre-existing trends. This is further illustrated in figures 3 and 4.

Table 5. Placebo test of the impact of the program on the probability of student dropout

||<br>ITT<br>|
|---|---|
||Experiment<br>Placebo|
|**_Panel (i): Pooled Model_**||
|_ENTRE j_|-0.000<br>-0.013***|
||(0.000)<br>(0.004)|
|Observations|147,977<br>145,628|
|R-squared|0.351<br>0.331|
|**_Panel (ii): Treatment arms_**||
|_ENTRE 1 j_|-0.000<br>-0.015***|
||(0.000)<br>(0.005)|
|_ENTRE 2 j_|-0.000<br>-0.011**|
||(0.000)<br>(0.005)|
|_ENTRE 3 j_|0.000<br>-0.013**|
||(0.000)<br>(0.005)|
|Observations|147,977<br>145,628|
|R-squared|0.351<br>0.331|
|Controls at student level|YES<br>YES|
|School fixed effects|YES<br>YES|

Notes: 1) Placebo: estimates using data from 2016 and 2017, when the ENTRE program did not exist. Experiment: estimates using data from 2017 and 2018 (year of the intervention) and the preferred specification in Table 3. 2) Standard errors clustered at schoolyear level in parenthesis; 3) *** p<0.01, ** p<0.05, * p<0.1. 4) The dependent variable is a dummy variable that is equal to 1 if student _s_ in school _j_ dropped out school between year t and year t+1. 5) _ENTREj_ represents the interaction _Entrejt_ x _Postt_ where _Entrejt_ : (1= Student _s_ attends a school _j_ in year _t_ that is in the treatment group) and _Postt_ : (1 = year 2017 in Placebo; = year 2018 in Experiment; 0 otherwise); _ENTRE ij_ with i=1,2,3 captures the same interaction as _ENTREj_ but using assignment to the treatment arm _i_ instead of assignment to the general treatment group. 6) All specifications in panel (i) include the corresponding _Postt_ and _Entrejt_ dummies, while those in panel (ii) include the _Postt_ and _Entre ijt_ dummies; 7) Student characteristics used as control: age dummies, gender and risk (probability of dropout estimated in the Early Warning System).

25
Figure 3. Impact of ENTRE on students’ dropout rate in the experiment and the placebo Notes: The graph shows the ITT estimates of the impact of ENTRE and each of its treatment arms (named Entre 1, Entre 2 and Entre 3) on students’ dropout rate in the real experiment and in the placebo. The latter replicates the estimation using data from 2016 and 2017 (instead of 2017 and 2018), when the program did not exist.

Table 6. Placebo test of the heterogeneity in the impact of ENTRE on students’ dropout rate

||||Inter|action:|||
|---|---|---|---|---|---|---|
||(i)Pre-treatment|dropoutprobability|(ii)Gende|r(=1 if boy)|(iii)School size|(=1 if large school)|
||Placebo|Experiment|Placebo|Experiment|Placebo|Experiment|
|**_Panel (i): Pooled Model_**|||||||
|ENTRE_j_+ (ENTRE_j_) x (_Interaction j_)|0.002|-0.029**|0.002|-0.019***|-0.000|-0.020***|
||(0.005)|(0.012)|(0.006)|(0.007)|(0.000)|(0.006)|
|_ENTRE j_|-0.002|0.001|-0.002|-0.007|-0.000|-0.005|
||(0.003)|(0.006)|(0.006)|(0.007)|(0.000)|(0.006)|
|Observations|147,977|145,628|147,977|145,628|147,977|145,628|
|R-squared|0.351|0.333|0.351|0.331|0.351|0.331|
|**_Panel (ii): Treatment arms_**|||||||
|ENTRE 1_j_ +(ENTRE 1_j_) x (_Interaction j_)|-0.000|-0.032**|0.004|-0.032***|-0.000|-0.026***|
||(0.006)|(0.015)|(0.007)|(0.009)|(0.000)|(0.008)|
|ENTRE 2_j_ +(ENTRE 2_j_) x (_Interaction j_)|0.007|-0.036**|-0.006|-0.011|0.000|-0.014**|
||(0.007)|(0.014)|(0.007)|(0.008)|(0.000)|(0.007)|
|ENTRE 3_j_ +(ENTRE 3_j_) x (_Interaction j_)|-0.000|-0.020|0.007|-0.013|0.000|-0.021***|
||(0.006)|(0.015)|(0.007)|(0.009)|(0.000)|(0.007)|
|_ENTRE 1 j_|-0.001|0.000|-0.005|0.003|0.000|-0.003|
||(0.003)|(0.008)|(0.007)|(0.009)|(0.000)|(0.008)|
|_ENTRE 2 j_|-0.005|0.007|0.007|-0.011|-0.000|-0.007|
||(0.004)|(0.007)|(0.007)|(0.009)|(0.000)|(0.008)|
|_ENTRE 3 j_|-0.000|-0.005|-0.007|-0.013|-0.000|-0.004|
||(0.003)|(0.007)|(0.007)|(0.009)|(0.000)|(0.008)|
|Observations|147,977|145,628|147,977|145,628|147,977|145,628|
|R-squared|0.351|0.333|0.351|0.332|0.351|0.331|
|Controls at student level|YES|YES|YES|YES|YES|YES|
|School fixed effects|YES|YES|YES|YES|YES|YES|

Notes: 1 <mark>)</mark> Placebo: estimates usin <mark>g</mark> data from 2016 and 2017 <mark>,</mark> when the ENTRE <mark>p</mark> ro <mark>g</mark> ram did not exist. Ex <mark>p</mark> eriment: estimates usin <mark>g</mark> data from 2017 and 2018 (year of the intervention) and the preferred specification in Table 4. 2) Standard errors clustered at schoolyear level in parenthesis; 3) *** p<0.01, ** p<0.05, * p<0.1. 4) The dependent variable is a dummy variable that is equal to 1 if student _s_ in school _j_ dropped out school between year t and year t+1. 5) _ENTREj_ represents the interaction _Entrejt_ x _Postt_ where _Entrejt_ : (1= Student _s_ attends a school _j_ in year _t_ that is in the treatment group) and _Postt_ : (1 = year 2017 in Placebo; = year 2018 in Experiment; 0 otherwise); _ENTRE ij_ with i=1,2,3 captures the same interaction as _ENTREj_ but using assignment to the treatment 26
<!-- Start of picture text -->
-——0.1% —0.1% aLarge school —Small School —<br>|. —s 0.0% 0.0% Ly<br>High-fisk Low-risk Bo Girl<br>-0.5%<br>-0.7%<br>-2.1% -1.9%<br>-2.9% (***)<br>(“*)<br>)<br>OPlacebo Real Experiment OPlacebo Real Experiment Placebo Real Experiment<br>By pre-treatment dropout risk By school size By gender<br><!-- End of picture text -->
In the first panel of Table 7 we show the coefficient capturing the (pooled) impact of ENTRE on different self-reported indicators (each coefficient corresponds to a different regression). The second panel reports the number of observations (schools) corresponding to each regression. The outcome variables are expressed as changes between the baseline and follow-up surveys. We measure the impact at two different moments of follow-up: short-term (about 2 months after the training was implemented and about a month before the enrollment for the next academic year begins) and mid-term (in the middle of the next academic year, about 8 months after the training was conducted).

Table 7. Impact of the program on different self-reported indicators at school level

|_ΔY j: Change between baseline and follow-up in:_|(1)Coe|fficient|(2)Obse|rvations|
|---|---|---|---|---|
||Short-term|Medium-term|Short-term|Medium-term|
|_Dropout expectations (standardized)_|-0.22***<br>(0.08)|0.03<br>(0.06)|1,005|2,385|
|_Expected Returns to education (standardized)_|0.25**<br>(0.10)|0.02<br>(0.07)|1,005|2,255|
|_Prioritized tasks (dropout)_|0.45***<br>(0.13)|0.27***<br>(0.08)|1,005|2,385|
|_Prioritized tasks (Authorities' requests)_|-0.22**<br>(0.11)|-0.16**<br>(0.07)|1,005|2,373|
|_Beliefs about principal's influence on the dropout decision (standardized index)_|0.03<br>(0.08)|0.07<br>(0.06)|1,005|2,383|
|_Beliefs about main influence on the dropout decision (Principal)_|-0.01<br>(0.01)|0.02*<br>(0.01)|1,005|2,381|
|_Belief that all students should complete secondary education (standardized index)_|0.01<br>(0.10)|-0.00<br>(0.06)|1,005|2,400|
|_Growth mindset proxy (principal's perception)_|0.03<br>(0.02)|0.03*<br>(0.02)|1,005|2,352|
|_Knowledge of students' families (standardized index)_|-0.09<br>(0.08)|0.01<br>(0.05)|1,005|2,394|
|_Frequency of meetings (standardized index)_|0.10<br>(0.09)|0.04<br>(0.06)|1,005|2,396|
|_Targeted support (standardized)_|0.61***<br>(0.08)|0.12**<br>(0.05)|970|2,387|

Notes: (1) Each coefficient in panel (1) corresponds to a different regression of treatment assignment on the change between baseline and follow-up in the outcome variable mentioned in the name of each raw. (2) Short-term refers to the change between the baseline and the first follow-up and Medium-term corresponds to the change between the baseline and the third follow-up. (3) Observations in panel (2) corresponds to the number of schools without missing data who were therefore used in the regressions. (4) All regressions control for the average characteristics of the students in the school (proportion of men, average risk of dropout and proportion of students of age 11, 12, 13, 14, 15, 16, 17, 18, and 19 & more). (5) Standard errors in parenthesis. (6) *** p<0.01, ** p<0.05, * p<0.1. (7) Standardized variables were transformed to have a mean of 0 and a standard deviation of 1.

The analysis shows some interesting findings. First, being assigned to ENTRE reduced principals’ dropout expectations in the short-term by 0.22 standard deviations and increased the perceived returns to education (one of the topics most highlighted in the ENTRE training) by 0.25 standard 28
deviations. However, both effects vanish by the next academic year. This may be capturing a shortterm (but timely) impact on the knowledge/motivation of school principals in the few months that followed the training, right before the enrollment process for the next academic year started.

We also analyze the impact of the program on a set of variables capturing principals’ perceptions of their role in preventing dropout and of the growth mindset of students. We find some evidence of a positive impact in these dimensions (e.g. the analysis suggests that the program increases the growth mindset mentality of students – as reported by the principal— by 3 percentage points), but the impacts are rather small and only significant at the 10 percent level.

In addition, we do not find any impact on principals’ self-reports related to students’ families. The relevance of regular communication with families was one of the aspects highlighted in both the training and the guide. However, according to self-reported data, neither the frequency of meetings nor principals’ perceptions of knowledge of families was affected by ENTRE. Overall, the coefficients (mostly) have the expected signs, but they are statistically insignificant.

Next, we look at the impact of the program on targeted support to students at-risk of dropping out. This is at the heart of the ENTRE program and highlighted in a special section of the training for all schools across the three treatment arms. The results show a strong effect in this dimension. According to our estimation, the program improved the provision of targeted support by more than 60% of a standard deviation. While the impact decreased with time, it remained statistically significant in the next academic year.

Finally, we looked at the impact of ENTRE on principals’ self-reported priorities. One of the channels through which ENTRE was expected to work was through the higher prioritization of dropout prevention among competing tasks (a challenge identified in the focus groups that informed the design of the program). School principals were asked to rank priorities from 1 to 5, with 1 being the least important, and 5 the most important. Our estimation indicates that being assigned to ENTRE significantly improved the ranking of dropout prevention among competing tasks in both the short and medium-term. Two months after the training was conducted, the ranking of dropout prevention improved by almost 0.5 (about 34% of a standard deviation). In the next academic year, the effect was cut by almost half (but it was still statistically significant).

29
# Qualitative results

Results from the focus groups with teachers, principals and trainers are mostly in line with the main quantitative outcomes described above. In particular, the dropout prevention guide was the element in the intervention most frequently exalted, with an excellent reception among teachers and principals, which highlights the importance of the core intervention on _how_ dropout can be prevented. School actors also declared an increase in their targeted efforts regardless of the treatment arm to which they belonged. Moreover, principals in the third treatment arm reported that the automated reminders were too frequent and confusing, possibly explaining why the third treatment arm was indistinguishable in its impact.

The pattern of heterogeneous effects by dropout probability, school size, and gender is further confirmed by the anecdotical evidence provided in the interviews. Besides the above-mentioned general claim of a greater emphasis on those students who needed more support, principals and teachers in multi-grade schools<sup>13</sup> reported having no time for ENTRE activities, which is consistent with the program’s lack of effect on small schools. In addition, when asked about proposals to improve the program, many school actors suggested that the guide should include differentiated strategies for boys and girls.

Self-reported data from open questions in the focus groups also provide information on mechanisms and compliance. Teachers and principals claimed to have focused their strategies on motivating students to pursue secondary. At the time the interviews were made (about 4 months after the training was implemented), school actors showed a strong confidence on the positive results from the program, consistent with the short-term impact on dropout expectations described in the subsection above. Finally, trainers reported some problems with the logistic during the delivery of the guide and the list of students at risk, which suggests that compliance was not perfect and justifies the analysis in the next subsection.

> 13 Multi-grade schools are schools where students from two or more grades are put together in a classroom. They are typically prevalent among semi-urban and rural areas, and their size is significantly lower than the average school size.

30
# Program participation

The previous section reports the intent-to-treat effects (ITT), which capture the impacts of being “offered” a treatment regardless of the take-up rate of the components of the program. Another parameter of interest is the Local Average Treatment Effect (LATE), which measures the impact of a program for those individuals who are offered the program and do participate (i.e., the _compliers_ ). When there is not full compliance, the ITT tends to underestimate the LATE. As we will discuss in this section, this is likely the case here. The available data indicates that the RCT faced compliance challenges both in terms of partial take-up of the program and contamination of the control group, and hence the ITT estimates are likely a lower-bound of the effect of the program on the treated.

Table 8 and Figure 5 describe the take-up of the components by the different treatment arms and the control group. For component 1—which provided information on _what_ to do to prevent dropout—we use as a proxy of participation whether the principals report receiving the dropout prevention guide that was delivered during the training session. As shown in columns (1) and (2) of Table 8, about 80 percent of the schools selected to participate in the ENTRE pilot report receiving the guide, and this percentage was roughly the same across treatment arms. In addition, this data also shows that there was some contamination in this dimension. About one-third of the schools in the control group also report receiving the guide. This could have happened either because they participated in the training or because they managed to get the materials afterwards.

We then assess take-up of component 2, which provided information on _who_ is at risk of dropping out. According to the design of the program, schools in treatment groups T2 and T3 should have received an envelope containing both the guide and the list during the training. There are different reasons why take-up of the list was incomplete, including school actors missing their training session, or receiving the wrong envelope (i.e., an incomplete envelope or the envelope for another school). We proxy the take-up of this component with a dummy capturing if the school principals report receiving the list of students at-risk. Columns (3) and (4) of the table shows the take-up rate of the list for each treatment arm. Roughly two-thirds of the schools that should have received the 31
<!-- Start of picture text --> 81% go0% 81% 79%<br>65%<br>62%<br>32%<br>25%<br>11%<br>a 0% 0% 0%<br>Guide List Reminders<br>MCONTROL M®Treatment1 MTreatment2 Treatment 3<br><!-- End of picture text -->
Finally, we assess the take-up of the reminders sent through SIRE. These reminders were activated in a sample of schools by an experienced technical team of the government using a specific software used to send alerts to subsamples of schools. The team had access to three pieces of information regarding the implementation of this component. First, using a guest id, the team verified that the reminders were posted according to the planned calendar. Second, the team had access to the list of schools that according to the SIRE team received the reminders. This list perfectly matched the treatment assignment. Third, the team had access to schools’ SIRE login activity for the month that followed the activation of the reminders. We combined these last two data sets to proxy the (timely) take-up of the component. Specifically, we approximate the takeup of the reminders with a dummy that equals one if the school logged into SIRE within that month and the SIRE team reports that the school received the reminders. Based on this, almost 80 percent of the schools in T3 were exposed to the reminders in a timely manner, and there was no contamination to other arms.<sup>15</sup> LATE As shown in the previous section, the take-up and contamination rates differ across treatment arms. In this context, it is not straightforward to compute the LATE (Behaghel et al, 2014). To address this, we do a rough estimation of the LATE for the pooled model using participation in the training as a proxy for take-up (as described above). There are three main reasons to select this variable to capture overall participation in the program. First, the training/guide was probably the most relevant activity in terms of both budget and substance (as suggested also by the ITT estimates). Second, other key elements of the program were distributed in the training, namely the Minister’s letter and the list of students at-risk. Third, the quality of the data capturing participation in the training is more reliable than the data capturing participation in other components.<sup>16</sup>

> 15 As mentioned above, it is important to note that the treatment receiving the reminders (T3) ended up having larger response rates in all the online follow-up questionnaires (but not the paper-based or administrative data sets). For instance, the response rate for a “knowledge test” was 40% larger for this group in comparison to T1 or T3. The reason for this was that the link to the questionnaires was placed next to the text of the reminders. This probably created an extra burden for this group of schools that might have had a negative impact on the perception of the program.

> 16 On the one hand, the response rate for the variable capturing participation in the training/guide is considerably larger than the one corresponding to the list of students at risk (and the response rate does not differ across treatment and control groups). On the other hand, the “take-up” of reminders was reported directly by SIRE officials (we do not have direct data from school principals).

33
Table 9 reports the LATE estimates for the pooled model. The results indicate that ENTRE reduces dropout by 3.1 percentage points for students in complier schools (column 3 of Table 9), roughly doubling the ITT estimates. As expected, the LATE estimations are significantly more imprecise, with the standard errors three times larger than those corresponding to the ITT. Table 10 shows a similar pattern of heterogeneous effects by dropout probability, school size, and gender. The results show that participating in the program reduces dropout by about 6 percentage points for the students with the highest risk, 4.5 percentage points for students in large schools, and approximately 4 percentage points for boys.

Table 9. Impact of the program on the probability of student dropout: LATE estimates.

|||LATE||
|---|---|---|---|
||(1)|(2)|(3)|
|_ENTRE j_|-0.025|-0.030**|-0.031**|
||(0.040)|(0.013)|(0.013)|
|Observations|130,762|124,626|124,626|
|R-squared|0.000|0.294|0.325|
|Controls at student level|NO|YES|YES|
|School fixed effects|NO|NO|YES|

Notes: (1) Local Average Treatment effects were estimated by instrumenting treatment (proxied as the effective reception of the ENTRE guide according to self-reported data in follow-up 3) with treatment assignment. (2) Standard errors clustered at schoolyear level in parenthesis. (3) *** p<0.01, ** p<0.05, * p<0.1. (4) The dependent variable is a dummy variable that is equal to 1 if student _s_ in school _j_ dropped out school between year _t_ and year _t+1_ . (5) _ENTREj_ represents the interaction _Entrejt_ x _Postt_ where in this case _Entrejt_ : (1= Student _s_ attends a school _j_ in year _t_ that was treated) and _Postt_ : (1 = year of the intervention; 0 = year before the intervention). (6) All specifications include the corresponding _Postt_ and _Entrejt_ dummies. (7) Student characteristics used as control: age dummies, gender and risk (probability of dropout estimated in the Early Warning System).

34
Table 10. Heterogeneity in the impact of the program on the probability of student dropout: LATE estimates.

|||Interaction:<br>||
|---|---|---|---|
||(i) Pre-treatment<br>dropout probability|(ii) School size<br>(=1 if large school)|(iii) Gender<br>(=1 if boy)|
|ENTRE_j_+ (ENTRE_j_) x (_Interaction j_)|-0.061*|-0.046***|-0.044***|
||(0.032)|(0.018)|(0.017)|
|_ENTRE j_|-0.007|-0.013|-0.018|
||(0.017)|(0.020)|(0.018)|
|Observations|124,626|124,626|124,626|
|R-squared|0.327|0.325|0.325|
|Controls at student level|YES|YES|YES|
|School fixed effects|YES|YES|YES|

Notes: (1) Local Average Treatment effects were estimated by instrumenting treatment (proxied as the effective reception of the ENTRE guide according to self-reported data in follow-up 3) with treatment assignment. (2) Standard errors clustered at schoolyear level in parenthesis. (3) *** p<0.01, ** p<0.05, * p<0.1. (4) The dependent variable is a dummy variable that is equal to 1 if student _s_ in school _j_ dropped out school between year t and year t+1. (5) _ENTREj_ represents the interaction _Entrejt_ x _Postt_ where _Entrejt_ : (1= Student _s_ attends a school _j_ in year _t_ that was treated) and _Postt_ : (1 = year of the intervention; 0 = year before the intervention). (6) _Interaction j_ in column (i) is given by the variable _Risk sjt_ (Probability of dropout for student _s_ that attends school _j_ in year _t_ estimated in the Early Warning System); in column (ii) by _Large school sjt_ : (1=Large school: size in 2017 > median), and in column (iii) by _Man sjt_ : (1=Man). (7) All specifications include the corresponding _Postt_ , _Entrejt_ and _Interaction j_ dummies. (8) Student characteristics used as control: age dummies, gender and risk (probability of dropout estimated in the Early Warning System).

# - <u>Scalability and cost benefit analysis</u>

As the size of the RCT suggests, the program was designed to be implemented at scale by relying on existing MINEDUC staff, including trainers, teachers, and school principals, and at a very low cost per student benefited by the program (roughly USD 2 dollars). The training and overall implementation of the program was chiefly in the hands of MINEDUC, with minor support from the World Bank. Expanding ENTRE to additional grades would reduce the costs per student by approximately 76%, given that the training and materials for principals (which represent roughly half of the budget for an average school) are a “fixed cost” for the school, there are many multiplegrade schools and the number of teachers per student is substantially larger in the sixth grade than in the first five years of primary education.

The cost-benefit analysis was conducted using the J-PAL tool and following its cost guidelines.<sup>17</sup> As previously mentioned, the intervention is low-cost, with a total cost per student of USD 2.91, 36% of which corresponds to the design of the program and 45% to the opportunity cost of

> 17 See https://www.povertyactionlab.org/resource/conducting-cost-effectiveness-analysis-cea

35
principals and teachers involved in it. Setting aside the one-time-only expenditures in the design of the intervention, the cost of implementing the program amounts to USD 1.85 per student. Estimates of the monetary benefits of ENTRE suggest an internal rate of return of 28% and a benefit-cost ratio of almost 19 with a 5% discount rate over a 40-years span. Another interesting cost-effectiveness measure can be computed by multiplying the main ITT estimate of the program (0.013) by 100 and dividing it by the implementation cost per student (1.85). This measure of additional years of student participation per USD 100 spent allows us to compare ENTRE with other type of interventions. For example, in a recent review from J-PAL, the impact of all the scholarships, conditional cash transfers, and subsides analyzed range between 0.01 and 0.17 additional years of education per USD 100 spent (including the well-known PROGRESA in Mexico with an impact of 0.01 additional years).<sup>18</sup> ENTRE is much more cost-effective based on this metric, since it provides 0.70 additional years of student participation per USD 100 spent. Moreover, and perhaps most importantly for scale and sustainability, for Guatemala where perstudent government spending at the primary level was over USD 530 in 2018, the costs of a program like ENTRE are feasible to absorb into national public budgets.<sup>19</sup>

# 6. Final Remarks

This paper presents experimental evidence that a low-cost and scalable early warning system can significantly reduce school dropout rates in the transition from primary to lower secondary, in a middle-income country with high average dropout. Using a 4,000-school multi-arm randomized experiment, we show that the program reduced the probability of student dropout in treated schools by 1.3 percentage points, and we estimate that this effect grows to 3.1 percentage points when we look at the impact on compliers (roughly 3.6% and 8.5% of the baseline dropout rate). The low implementation cost of the program—about 2 USD per student—makes the intervention highly cost-effective. The (conservative) rate of return of the program is estimated to be almost 30%, with a cost-benefit ratio of approximately USD 19 per each 1 USD invested. The evaluation was embedded in a large-scale implementation including almost 17% of all primary schools in Guatemala, making the results more informative for understanding the impacts of the program in ongoing, “real world” conditions.

> 18 See Figure 4 in J-PAL Policy Bulletin (2017) for a cost-effectiveness comparison with these and other programs.

> 19 Data on public education spending from the World Development Indicators.

36
The impact of ENTRE is heterogeneous, with significantly larger effects for students at higher risk of dropout, students in large schools, and boys. In particular, ITT (LATE) estimates show that the program reduced dropout by about 2.9 (6.1) percentage points for the students with the highest risk, 2.1 (4.6) percentage points for students in large schools, and approximately 1.9 (4.4) percentage points for boys.

Our multi-arm design sheds some light on the potential mechanisms driving the results. The results indicate that the bulk of the impact seems to be explained by the component providing information on _how_ to prevent dropout (as opposed to providing information on _who_ is at-risk of dropping out or small behavioral nudges to try and keep dropout prevention top-of-mind for principals). In a setting like Guatemala, where school actors have limited knowledge of and support on dropout prevention strategies, and school principals—who play a leading role in these initiatives— are usually regular teachers without any specific leadership or management preparation, the training and guidelines on simple strategies to prevent dropout appear to be highly impactful. This component was also the most intensive in terms of time and resources (representing approximately 92% of the cost of the program).

Self-reported data from school principals also helps to further unpack the mechanisms driving the results. This data shows that about two months after the training, principals in treated schools perceived higher returns to schooling, ranked dropout as a higher priority, and provided more targeted support to at-risk students. The impact on targeted support and prioritization are not only large in quantitative terms but also persist in the next academic year (with a smaller but still economically and statistically significant impact).

Finally, the results suggest that school actors have gender-biased perceptions of who is at risk of dropout (significantly underestimating the relative likelihood of dropout for girls) and that providing objective information on these probabilities seems to have corrected these biases (and distributed the benefits of the program more evenly across boys and girls). This is an important suggestive result for the dropout prevention literature that calls for further research.

37