Data from a questionnaire applied to principals at baseline and endline provide mixed evidence on the possibility that some control schools became aware of the program. Principals were asked to identify the single most important of six possible strategies (including the option of doing nothing) to improve teachers’ effectiveness (Table 15). On the one hand, the share of principals in treatment schools that identified “feedback based on classroom observation” as most important rose from 11% to 22%, compared to an increase among control school principals of only 3.5 percentage points. However, the increase in the number of control school principals who named “coaching of teachers” as the most important strategy was as large as the increase among treatment school principals. As discussed in the next section on contamination, there was in fact another teacher coaching program implemented in Ceará in 2014 and 2015, which we believe is a more likely explanation for this result.

<u>Table 15: Principals’ Survey: Which of these instruments are most important for raising teacher quality?</u>

||Spoken<br>guidance|Written<br>guidance|Feedback on<br>lesson<br>planning|Feedback<br>based on<br>classroom<br>observation|Coaching of<br>teachers|Nothing|Total|
|---|---|---|---|---|---|---|---|
|Baseline||||||||
|Control|84|11|23|17|0|2|136|
|Treatment|104|11|18|19|3|1|156|
|Total|188|22|41|36|3|2|292|
|Endline||||||||
|Control|69|10|20|22|15|0|136|
|Treatment|76|13|15|35|16|1|156|
|Total|145|23|35|57|31|1|292|

A final source of potential spillover is the fact that 8% of teachers in the control group and 10% in the treatment group work in more than one school. As most secondary schools run morning and afternoon and sometimes evening shifts, teachers may work the different shifts in two different schools. If a teacher in a treatment school also works in a control school, it would be natural to share information about the program, including teaching practices recommended by the coaches, with the control school’s pedagogical coordinator.<sup>12</sup> To investigate the scope for this, we verified the school assignments of teachers in our sample. Only 3% of our control schools and 1.7% of our treatment schools have teachers that work in both treatment and control schools.

Table 16: Possibility of spillover from teachers working in more than one school Number of schools where teachers work

||1|2|3|4|Total|
|---|---|---|---|---|---|
|Control|107|21|7|1|136|
|(%)|30.66|6.02|2.01|0.29|38.97|
|Treatment|119|30|7|0|156|
|(%)|34.1|8.6|2.01|0|44.7|
|Not Observed|42|10|5|0|57|
||12.03|2.87|1.43|0|16.33|
|Total|268|61|19|1|349|
||76.79|17.48|5.44|0.29|100|

We implemented two strategies to check the robustness of our impact estimates in the context of likely spillover. First, we assumed that the municipalities with the largest number of classrooms observed, whether from treatment or control schools, are more susceptible to spillover. The larger the absolute number of treatment teachers participating in the program, the higher the chance that some of these may know teachers in control schools in the same municipality. They might understandably think these colleagues could also benefit from the coaching advice and training materials imparted by the program, such as the _Aula Nota 10_ ( _Teach Like a Champion_ ) book. In fact, in focus group discussions at the end of the program, several control school principals confirmed that some of their teachers had heard about the _Aula Nota 10_ book and purchased it on their own initiative. We were unable to get systematic data on these anecdotes, but we tested the potential impact of between-school spillovers by including in our regressions a variable on the number of classrooms in the municipality and a variable on the number of treated classrooms in the same municipality.<sup>13</sup> The results in table 17 suggest that the additional variables we used to test for spillovers were not significant. The intervention effect was greater than the mean effect size in all specifications of the model. Except for time on instruction with all students engaged, all the other four variables are significant at the 1% level. The effect of the intervention is 5,79% on time spent on instructional activities; -3,24% on classroom management activities; -2,55% on teacher time off task is -2,55; and -4,85% on big group of students off-

> 12 Regarding the pedagogical coordinators, they are usually only assigned to work in one school. 13 We adapted this strategy based on the work of Miguel and Kremer (2004).

24