# **4. Evaluation: Data Collection, Baseline Results, and Empirical Approach**

# **4.1 Data collection**

The evaluation took place in 280 villages in Jaunpur district in the state of Uttar Pradesh (UP), India. The state of UP is the most populous state in India with a larger population than Mexico (166 million inhabitants according to the 2001 census). It is also among the five worst performing states in terms of basic literacy. Jaunpur district was chosen in part because it is close to the state average in terms of literacy, and in part because it was one of the districts where Pratham was not already working. Districts in India are divided into administrative blocks. Four of these blocks were randomly selected to participate in the study, and the study villages were then randomly selected within each block. The survey and the study are thus representative of Jaunpur district (and its 3.9 million population) as a whole.

In both treatment and comparison villages, a baseline survey was conducted in March and April 2005, and the endline survey took place in March and April 2006. The evaluation instruments included a detailed data set on 10 randomly selected households per village; a dataset on reading and math outcomes for all children aged 7 to 14 (at the time of the baseline) in 30 randomly selected households in the villages; a dataset of headmasters of government primary schools and all VEC members; data on school infrastructure and functioning; and an average of 6.7 observations for each school on teacher presence, obtained during random, unannounced visits during school hours. Every survey was conducted at least twice; hence, each dataset has a panel structure. The data on learning outcomes were collected using a simple tool developed by Pratham, and used in the ASER survey. For reading, children were presented with a simple paragraph and asked to read it. If they could not read it the tester quickly switched to a panel with a few single words on it, and if they could read it, the tester moved up to a longer and more complex story. If the child struggled to read the words on the panel, the tester showed a panel with several letters and asked the child to identify them. A similar approach is used for math—with the levels in the math case ranging from easy to hard in addition, subtraction, multiplication, and division.<sup>11</sup> The final sample for the baseline survey consists of 2,800 households, 316 schools, 17,533 children (ages 7-14) tested in reading and math, and 1,029 VEC member interviews from the 280 villages. In the endline 17,419 children were tested. There was very little attrition from the baseline survey: The endline sample includes all but 716 of the children in the baseline (the attrition is evenly spread across the various treatment and control groups). An additional group of 2,131 newly-turned seven-year-olds were added to the sample at endline (although these are not included in most of the analysis that follows since we have no baseline for them).

# **4.2 Baseline results**

Baseline data broken down by treatment group are available from the authors. They confirm that there are no systematic differences between treatment and comparison groups or between the different treatment groups: the

> 11 The survey instruments and data collection are all available on the J-PAL web site (www.povertyactionlab.org).

12