all levels of knowledge.<sup>1</sup> All non-absent children in Grade 3 (12,110 children) in the sample schools were tested in the three subjects at baseline; in the follow-up round, we tracked all children in the roster from the previous year and tested them if present at any school in the village in addition to any new kids enrolled in the tested grade (see Appendix I.D. for further details). The same tests were administered by our team across all schools and test materials were not shared after testing to ensure impartial and comparable test circumstances. In order to facilitate comparisons of the test over time and to better relate the test to underlying student knowledge, we use item response theory to score and equate the test appropriately adjusting for the difficulty of each question.<sup>2</sup> The scores thus derived are standardized for the first year (with mean 0 and standard deviation 1), but the test-equating methodology imposes no further restrictions across years, so that the 2<sup>nd</sup> year scores are standard deviation changes normed by the first year distribution.

# iv. Child Surveys

This survey is administered to randomly selected children in the tested grade and gathers information on child educational history, household composition, household assets, and child anthropometrics. In every school, we survey 10 children from the tested grade; in schools with less than 10 children in the tested grade, all children are surveyed. This exercise gives us information on over 6,000 children.

# _<u>Household-based Surveys</u>_

This survey is administered to over 1,800 households. These households are selected based on a census conducted in the sample villages in 2003, which listed over 80,000 households. We stratify by enrollment status and over-sample households with children who might be eligible by age for (tested) Grade 3.

1 Criterion-referenced tests help identify whether students meet a criterion but can be less informative for those below or above the critical level. Norm-referenced tests seek to maximize variation by estimating the learning level of a particular student. To design the test, an extensive pilot was carried out to identify lower and upper limits of learning in the population and analyze the validity and reliability of the instrument used. The data from this phase was used to refine the final test. As a result, all three tests (English, mathematics, and Urdu) start from simple problems and gradually increase in difficulty. Andrabi et al. (2002) detail the psychometric properties of the test, including tests of reliability.

2 In addition to the appropriate weighting of test score items by their difficulty, item response allows us to determine the precision of the test instrument at different points along the children’s knowledge distribution. Our analysis shows that scores around the middle of the distribution are more precisely estimated than at the ends of the distribution; this is a standard issue with all tests, since items designed to provide information at the extremes of the distribution also add to information for the mean, but not necessarily the other way around (Andrabi et al., 2002). We equate test scores across years using item response scaled scores, where identification is based on a set of rotating questions that were repeated across years.