|Certification by <br>|ELOS<br>||
|---|---|---|
|Certifield|Total|156 Sample|
|No|21|18|
|(%)|12.07%|11.54%|
|Yes|153|138|
|(%)|87.93%|88.46%|
|Total|174|156|
|(%)|100%|100%|
|Grade for Certif<br>|ication byEL<br>|OS<br>|
|Grade|~~Total~~|~~156 Sample~~|
|Bad|21|18|
|(%)|12.07%|11.54%|
|Regular|39|30|
|(%)|22.41%|19.23%|
|Good|63|59|
|(%)|36.21%|37.82%|
|Excelent|51|49|
|(%)|29.31%|31.41%|
|Total|174|156|
||100%|100%|

To assess the degree to which partial compliance affected program results, we estimated the effects of the program on the schools of different pedagogical coordinators using an Instrumental Variables model. This estimate tells us the impact of the program on those schools that received the complete intervention (e.g., their pedagogical coordinator acquired the key skills imparted by the training) as compared with our Intentto-Treat estimates (Section 4), which show the average impacts of the program on all 174 schools that were offered participation. The IV estimation uses the randomized assignment into the program (e.g., offered participation) as an instrument to predict the expected degree of full engagement in the program.

We use three measures of full engagement: (i) the pedagogical coordinator achieved certification (he or she scored adequate (regular), good or excellent on the final test); (ii) the pedagogical coordinator achieved a score of good or excellent on the final test; and (iii) the pedagogical coordinator scored excellent on the final test. We expect the magnitude of the results to increase from specification (i) to (ii) and (iii).

The IV estimate is conducted in a two-stage least-squares (2SLS) setup as initially used to adjust partial compliance in experiments by Angrist et al. (1996)<sup>11</sup> . In the first stage regression, we predict the degree of full engagement in the program from the random assignment. In the second stage, we regress our outcome variables on the predicted full engagement that we found in the first stage. The assumption is that a pedagogical coordinator receiving certification satisfies the exclusion restriction in an instrumental variables (IV) setup. This leads to the 2SLS estimation of the equation: �� = �� + ����,��� + �′��� + ���� + �� (4) where ci is a dummy for being certified, and Xi is the vector of covariates. The associated first-stage relationship using Zi as an instrument is �� = �′��� + ��� + �� (5) The estimates use the 4<sup>th</sup> specification of the ITT analysis (including all controls). The estimate of � is statistically significant for all three measures of full engagement: 0.89, 0.67 and 0.4 for the measures (i), (ii) and (iii), respectively (Annex Table A.14).

Table 11 confirms that the effects of the intervention are consistent with the regression estimates presented in section 3. The program had a significant and positive impact on the share of class time teachers devoted to instruction, increasing in the treatment schools by 5.8% to 15.8% (0.279 to 0.754 SD) as we go from specification (i) to (iii). The program helped teachers reduce the time spent on classroom management by 3.1% to 8.5% (0.188 to 0.510 SD), and time off-task from 2.8% to 7.5% (0.216 to 0.584 SD). The reduction on the percentage of time a big group of students is off task ranged from 3.4% to 9.2% (0.121 to 0.327 SD). The only variable that was not significantly affected was “time on instruction with all students engaged”. These results are quite dramatic. They suggest that pedagogical coordinators played a key role in the implementation of this program, and hint at the importance they may have more broadly for school quality. Improvements in _all_ dimensions of teachers’ classroom practice were consistently larger in schools where the pedagogical coordinator had stronger mastery of the concepts and techniques presented in the _Teach Like a Champion/Aula Nota 10_ book and by the ELOS coaches. Program impacts in the 49 schools where pedagogical coordinators scored at the highest level (excellent) are very large. Instruction averaged 86% of class time – higher than the 85% Stallings good practice benchmark. Time on classroom management was

> 11 Angrist et al (2002) is an example of using IV models to adjust for partial compliance in an RCT of a voucher program in Colombia.

20