the highest indices of a big group of students off-task (Annex table A23). Although we found significant impacts on SPAECE math scores for schools in the middle of the distribution, they are not significantly different from the rest of the distribution.

We also explored whether teachers’ age made a difference, thinking that new teachers might have the most to gain from this program. As seen in Table 31, the youngest quartile of teachers (age 22-31) did not register statistically significant learning gains in their classrooms, but teachers in the second quartile (age 31-35) did achieve big gains in ENEM scores and SPAECE math scores (this heterogeneous effect is significant for the ENEM scores). This is consistent with evidence from other studies that teacher effectiveness improves dramatically after the first five years of experience, suggesting that in this case that teachers who are still young but have accumulated a critical level of experience may be best positioned to benefit from this type of program.

Table 31: Heterogeneous Impact of Program on Teachers, by Age (math and Portuguese teachers only)

|<br>Dependent Variable|<br>(Quartile 1)<br>22.00-|<br>(Quartile 2)<br>31.33-|<br>(Quartile 3)<br>35.67-|<br>(Quartile 4)<br>40.67-|<br>p-value|
|---|---|---|---|---|---|
||31.00|35.50|40.50|64.00||
|**SPAECE** **1**<sup>**st**</sup> **Grade**||||||
|Spaece - Mathematics|0.055|0.098*|0.087*|0.080|0.924|
||[0.052]|[0.051]|[0.047]|[0.053]||
|Spaece-Portuguese|0.034|0.043|0.055|0.056|0968|
||[0.034]|[0.038]|[0.040]|[0.040]|.|
|N|10699|10204|8441|7045||
|**ENEM** **3**<sup>**rd**</sup> **Grade** **(controli**|**ng** **for** **ENEM** **t-1)**|||||
|ENEM-Portuuese|0.032|0.161***|0.011|0.020|0023|
|g|[0.040]|[0.040]|[0.043]|[0.042]|.|
|ENEM-Mathematics|0.035|0.154***|-0.017|-0.009|002|
||[0.048]|[0.044]|[0.043]|[0.045]|.|
|N<br>Note: All specifications inclu<br>transfer and previous achiev<br>Quartile 1=Quartile 2=Quart|7815<br>de student control<br>ment. Standard er<br>ile 3=Quartile 4. St|6049<br>s for age, sex, rac<br>rors in brakets, c<br>atistical significa|5828<br>e, parental educa<br>lustered at the sc<br>nce levels * p<0.1|4654<br>tion, paved stree<br>hool level . P-valu<br> ** p<0.05 *** p|t, cash<br>e tests if<br><0.01.|

# _d. Compliance Effects_

As discussed in Section 3.f, implementation of the teacher feedback and coaching intervention depended importantly on the engagement of the pedagogical coordinator in each school, as these were interface between teachers and the ELOS expert coaches. To generate a quantitative measure of school engagement, the coordinators and school directors maintained an online registry on the number of Skype conference calls, teacher observations, and feedback meetings held each week. For a qualitative measure, the ELOS team administered a final certification test to all coordinators who had participated in at least 80% of all online and face-to-face activities.

We use these qualitative measures to estimate the impact of program compliance on student learning outcomes, parallel to our methodology for estimating its impact on classroom dynamics. Models (4) and (5) (section 3.f) use three benchmarks as a proxy measure of the intensity of pedagogical coordinators’ engagement with the program and likely capacity to support teachers in mastering the new teaching techniques.

(i) _certification -_ the pedagogical coordinator achieved the minimal score on the final test necessary for certification (e.g., adequate (regular), good or excellent) – 138 coordinators achieved this (ii) _good or excellent -_ 108 coordinators achieved this; and (iii) _excellent_ – 49 coordinators achieved this Results are shown on Table 32<sup>22</sup> . As expected, we find higher magnitudes on the 2SLS when compared to ITT analysis. The results suggest strongly that coordinators’ mastery of the content of the program enhances their ability to support teachers in adopting the new practices and that these practices, if adopted faithfully, can have significant impact on student learning. On both SPAECE and ENEM, the size of the positive impact on student scores rises in parallel with coordinator qualifications. For the 49 schools whose coordinators achieved excellence, the learning impacts are quite strong – 0.232 SD on SPAECE math, 0.134 on SPAECE Portuguese, and 0.175 on ENEM Portuguese. The only result that is not statistically significant is on ENEM math.

Table 32: 2SLS estimates of the effect on student learning (292 school sample)

> 22 1st stage estimates are shown in Annex table A24.

35