teachers were observed engaging at least once during the lesson in what could be described as good pedagogical practices, such as asking questions that require students to apply information, use their creativity, or demonstrate understanding, but for some indicators the share is very low (for example only 21 percent of teachers summarized it at the end of the class). About two-thirds of teachers provided encouraging or “correcting” feedback (which might be considered good practice) to students. At the same time, classroom observations find few teachers providing “scolding” feedback (which might be considered bad practice) to the students. However, very few teachers assigned or reviewed homework, and few used local language for instructions. While Table 1 suggests that conditions in rural and urban schools were quite different, Table 3 suggests that teachers behaved similarly towards students in the two settings.<sup>8</sup>

# **_3.3 Classroom observation instruments_**

Four classroom observation instruments were used in ETOS; two were administered in classrooms themselves and two were used afterwards by coding videos of the lessons. During the first and second rounds of data collection, teachers were observed in classrooms using the SDI and Stallings instruments (Stallings 1976, Stallings, Knight, and Markham 2014, and World Bank 2015).<sup>9</sup> In addition to the inclassroom enumerators, cameras were set up in selected classrooms to video the lessons. The videos ran for the entire length of the lesson and were subsequently divided into two or three clips of around 20 minutes each. These clips were subsequently coded using both the Classroom Assessment Scoring System (CLASS; Hamre et al. 2007, Pianta et al. 2012)<sup>10</sup> as well as the “Teach” scoring system (Molina et al. 2018; Molina et al. 2019).<sup>11</sup> Details of the instruments are described in detail below, but the key features of each is summarized in the Box 1. Three of the instruments include an explicit measurement of time-on-task (CLASS is the exception). The observation systems have different requirements in terms of enumerator or rater knowledge. The SDI and Stallings approaches are “low-inference” in the sense that they do not require enumerators to interpret much of what they are observing—rather, for these instruments, enumerators are mostly in the position of recording what they see in the form of a checklist of pre-populated categories. The CLASS and _Teach_ approaches are “high-inference” since they require raters to map specific observed

> 8 Appendix 1 Table 6 reports the same means, but for the national sample.  The means are generally similar and within about 10 to 15 percent of each other.  The largest differences are for teacher summarizes the lesson at the end of class, assigns homework, reviews homework, teacher uses local language for instruction which are roughly 50 percent lower in the ETOS sample.

> 9 The SDI instrument is is available at https://microdata.worldbank.org/index.php/catalog/2748/download/39237. <u>The Stallings instrument and manual are available at https://openknowledge.worldbank.org/handle/10986/22401.</u> 10 Instrument and manual are in Pianta et al. (2012). Coflan, Hasan, and Raggatz (2018) also provide information on the description and indicators related to each dimension.

> 11 Instrument and manual are available at <u><mark>http://documents.worldbank.org/curated/en/949541542659103528/TeachObserver-Manual.</mark></u>

7