# **4.3 Possible Threats from the New Situation**

Notwithstanding sizable and significant effects on a sample that is representative of the population of interest, the success of the mentoring intervention at scale is not guaranteed. The difference between the implementation protocol in our research setting—where we had an active role in guaranteeing the smooth progress of the field experiment—and the government operations can translate into contrasting aftermaths of the program. For example, the criteria for closing schools were very different in the two situations. Although the official enrollment threshold for closing schools is six students, schools in the second experiment were allowed to remain open if they had at least three enrolled students in either of the two school years when the experiment took place. In addition, children in schools with more than 29 enrolled students were required to transfer to schools in the regular public school system. As a result, only two schools closed in the sample of 230 schools in the second experiment (see Section 2.2).<sup>14</sup> Hence, a threat to the success of the program in the new policy situation may come from the possible school closures as a result of the mentoring intervention under the government implementation. If instead the presence of a mentor increases the probability that schools remain open, this mechanism could represent an opportunity to learn something about the mechanisms behind the scalability of the program in our context. Figure 1 shows the relationship between school closures, as measured by the year-to-year presence in the school census two years after the second experiment, and school size, as measured by the number of enrolled students in the school year before the second experiment. The green (lighter) line shows how the probability of school closures varies by school size for schools that did not receive the program during the government intervention. The probability of school closure is bimodal, and its two modes are positioned around the two critical enrollment thresholds.

> 14While school closures are obviously a “non-negotiable” aspect (List, 2022) for the success of the API program, there may be other “negotiable” differences in the program implementation across the experimental and the policy settings that we cannot directly study due to a lack of monitoring data outside of the experimental sample/period. First, to avoid refusal of the assigned mentor among the communities of the evaluation schools, each mentor in the experimental sample was provided with two baskets of food, throughout the school year, as donations to the community leaders as well as for personal consumption. Second, as a way to attenuate the potentially detrimental consequences of mentors’ dropping out of the program during the evaluation period, the government delegates in Chiapas arranged for a replacement within two weeks from the day of a mentor’s departure from a community. If the dropout was part of the _Plus_ group, the replacement would receive an additional three-day training session that would make up for the content covered during the extra week of the initial training session. Third, there might be differences between the experiment and the policy rollout in the implementation of the training module of the _Plus_ modality, such as the number of extra days and the content of the curriculum of the initial training as well as the frequency of the peer-to-peer sessions.

24