to classroom observations in over 15,000 classrooms across 7 Latin American countries, Chang et al. (2014) who use video recording of 200 8<sup>th</sup> grade mathematics lessons in 200 classrooms with ex-post expert analysis in Indonesia, Seidman et al. (2018) who develop and apply the Teacher Instructional Practices and Processes System (TIPPS) instrument in 197 secondary schools in Uganda, Wolf et al (2018) who apply TIPSS in pre-primary classrooms in Ghana, the SDI surveys mentioned above which to-date have recorded observations for over 69,000 teachers in 10 Sub-Saharan African countries (www.sdindicators.org), and finally the development and application of the _Teach_ instrument in Mozambique, Pakistan, the Philippines, and Uruguay (Molina et al. 2018).<sup>3</sup> Fewer still are studies that explicitly compare how different observation instruments perform, with Bruns, De Gregorio and Taut (2016) who compare Stallings and CLASS for 51 teachers in the Santiago Metropolitan region and two adjacent regions in Chile being a rare exception.

# **3.  Sample, study design, and data**

# **_3.1 Setting, sample, and study design_**

The study setting is Tanzania where 106 schools were reached for this Extra Teacher Observation Study (ETOS). The sample was purposively chosen from the nationally representative sample of 400 schools that had been selected for the 2014 and 2016 rounds of the Tanzania SDI survey (Martin and Wane 2016).<sup>4</sup> The ETOS schools are located in 11 regions across the country. Data were collected across two rounds: the first round was carried out between August and November 2016 (with almost 65% of the sample in August itself), the second round was carried out in November 2016. Eighty-eight of the schools are located in rural areas, 9 are in Dar es Salaam, and 9 are in other (semi-)urban areas (in this analysis we group these last two and designate them as “urban”).<sup>5</sup> The ETOS schools have on average 14 teachers and average class size of 58 enrolled students (Table 1A). In this sample (and during the first round of the survey), 14 percent of teachers were absent from the school on the day of an unannounced visit, and 39 percent of the teachers were not in the classroom (either

> 3 Aslam and Kingdon (2011) use self-reported, rather than observed, practices to study the same issues.

> 4 The schools were selected in a way that would minimize the additional costs (of training and enumerator visits in Round 1 and Round 2 of data collection) that would arise from adding the activities from this task to those of the 2016 round of the SDI. The resulting sample is not nationally representative and is clustered in four parts of the country: Dar es Salaam (8 schools): Western (Rukwa, Tabora, Shinyanga, Simiyu; 35 schools), Southern (Iringa, Ruvuma, Lindi, Mtwara; 36 schools), and Northern (Manyara, Kilimanjaro; 25 schools). The goal was to be roughly proportional to the number of schools in each region.

> 5 The schools selected for ETOS reflect a broad geographic range, and a broad set of socio-economic conditions relevant for Tanzania.  Statistical representativeness was not sought since the goal was to investigate the properties of the various instruments (internal properties, in relation to each other, and in relation to student test scores). As discussed in the text, the characteristics of schools in the sample are nevertheless similar, on average, to those in the national sample—especially within rural and urban areas.

5