Stallings, or video for these (11 schools). At the same time, two schools that had not been reached in Round 1 could now be reached and are included in the sample.

# **_3.4 Student tests_**

In addition to the classroom observations, students of the Grade 4 teachers who were observed also sat for a test in each round. The test had three sections, covering mathematics, Kiswahili, and English. The test was curriculum-based—pegged to the Grade 3 and early Grade 4 curriculum and developed by Tanzanian education academics in collaboration with the research team. The mathematics test consisted of tasks ranging from identifying and sequencing numbers, addition of one to three-digit numbers, one- and two-digit subtraction, and single digit multiplication and divisions. The Language tests consisted of several different tasks ranging from a simple task testing knowledge of the alphabet, to word recognition, to a more challenging reading comprehension test. The test in Round 1 included 15 or 16 questions for each subject.<sup>19</sup> Basic classical item analysis and IRT analysis suggests that the test generally worked well, although for this analysis we removed two items (2 from mathematics and 1 from Kiswahili) because of poor item functioning.<sup>20</sup> The Cronbach alphas for the three subjects was 0.73, 0.83, and 0.63 for mathematics, Kiswahili, and English respectively.<sup>21</sup> Despite having been validated against the curriculum, students generally did poorly on the test. In Round 1, students answered on average 38 percent of questions correctly in mathematics (i.e. about 5 questions), 52 percent correctly in Kiswahili (just over 7 questions), and 26 percent correctly in English (just over 4 questions).<sup>22</sup> While the goal was originally to administer the tests in a way that would allow estimation of valueadded models (i.e. by comparing growth in performance from Round 1 to Round 2), this is likely not advisable. The main reason is that the time between the two rounds was compressed due to delays in the implementation of Round 1. For some students, the gap between rounds was as little as just over 1 month— meaning that value added estimates would be largely meaningless. We nevertheless exploit the two rounds of data by averaging student scores across both rounds—thereby increasing the signal-to-noise ratio.

# **4. Basic descriptive statistics**

> 19 Round 2 included 3 additional questions for mathematics, which we ignore in this analysis.

> 20 - Section 2 of Appendix 2 (available online via a link on https://sites.google.com/site/decrgdeonfilmer/working <u>papers) details the psychometric analysis conducted on the student test score data. We make the determination of</u> “poor item functioning” based on analyzing the results of correlation between item responses, poor properties of item characteristic curves and item information function curves.

> 21 Properties are similar in Round 2: the Cronbach alphas for the three subjects was 0.85, 0.71, and 0.63 for mathematics, Kiswahili, and English respectively.

> 22 Further details on the test questions and the item analysis are available in Appendix 2 (available online via a link on https://sites.google.com/site/decrgdeonfilmer/working-papers).

14