metadata: authors: 0: author: Das, Jishnu
1: author: Dercon, Stefan
2: author: Habyarimana, James
3: author: Krishnan, Pramila
4: author: Muralidharan, Karthik
5: author: Sundararaman, Venkatesh
count: World
docna: 0: docna: School inputs, household substitution, and
            test scores
docty: Policy Research Working Paper
subtopic: Labor Markets,Educational Sciences,Educational Institutions & Facilities,Educational Populations,Education for Development (superceded),Educational Policy and Planning - Textbook,Effective Schools and Teachers,Education For All,Crime and Society,Rural Labor Markets
teratopic: Social Protections and Labor,Education,Finance and Financial Sector Development
abstracts: cdata!: Empirical studies of the relationship
            between school inputs and test scores typically do not
            account for the fact that households will respond to changes
            in school inputs. This paper presents a dynamic household
            optimization model relating test scores to school and
            household inputs, and tests its predictions in two very
            different low-income country settings -- Zambia and India.
            The authors measure household spending changes and student
            test score gains in response to unanticipated as well as
            anticipated changes in school funding. Consistent with the
            optimization model, they find in both settings that
            households offset anticipated grants more than unanticipated
            grants. They also find that unanticipated school grants lead
            to significant improvements in student test scores but
            anticipated grants have no impact on test scores. The
            results suggest that naïve estimates of public education
            spending on learning outcomes that do not account for
            optimal household responses are likely to be considerably
            biased if used to estimate parameters of an education
            production function.
repnme: repnme: School inputs, household substitution, and
            test scores
display_title: School inputs, household
            substitution, and test scores
listing_relative_url: /research/2011/04/14028504/school-inputs-household-substitution-test-scores
topicv3: Teachers Management m1326119 1523,Labor Markets m1326488 855,Textbooks m1326802 2129,Criminal Justice m1326993 1014,Education m1327048 470
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Social Protections and Labor,Education,Finance and Financial Sector Development
no_of_pages: 44
model_extractions: WPS5629 Policy Research Working Paper 5629 Impact Evaluation Series No. 49 School Inputs, Household Substitution, and Test Scores _Jishnu Das Stefan Dercon James Habyarimana Pramila Krishnan Karthik Muralidharan Venkatesh Sundararaman_ The World Bank Development Research Group Human Development and Public Services Team April 2011
Policy Research Working Paper 5629

# **Abstract**

Empirical studies of the relationship between school inputs and test scores typically do not account for the fact that households will respond to changes in school inputs. This paper presents a dynamic household optimization model relating test scores to school and household inputs, and tests its predictions in two very different low-income country settings—Zambia and India. The authors measure household spending changes and student test score gains in response to _unanticipated_ as well as _anticipated_ changes in school funding. Consistent with the optimization model, they find in both settings that households offset anticipated grants more than unanticipated grants. They also find that unanticipated school grants lead to significant improvements in student test scores but anticipated grants have _no impact_ on test scores. The results suggest that naïve estimates of public education spending on learning outcomes that do not account for optimal household responses are likely to be considerably biased if used to estimate parameters of an education production function.

This paper is a product of the Human Development and Public Services Team, Development Research Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at jdas1@worldbank.org.

_The Impact Evaluation Series has been established in recognition of the importance of impact evaluation studies for World Bank operations and for development in general. The series serves as a vehicle for the dissemination of findings of those studies. Papers in this series are part of the Bank’s Policy Research Working Paper Series. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **School Inputs, Household Substitution, and Test Scores**

Jishnu Das<sup>*</sup> Stefan Dercon James Habyarimana Pramila Krishnan Karthik Muralidharan Venkatesh Sundararaman _JEL Classification_ : H52, I21, O15 _Keywords_ : school grants, school inputs, household substitution, education in developing countries, randomized experiment, India, Zambia, Africa, education production function

* Jishnu Das (World Bank and Center for Policy Research, Delhi: jdas1@worldbank.org ) Stefan Dercon (Oxford University, BREAD, and CEPR: stefan.dercon@economics.ox.ac.uk ) James Habyarimana (Georgetown University, IZA, and Center for Global Development: jph35@georgetown.edu ) Pramila Krishnan (Cambridge University and CEPR: pk237@cam.ac.uk ) Karthik Muralidharan (UC San Diego, NBER, BREAD, and J-PAL: kamurali@ucsd.edu) Venkatesh Sundararaman (World Bank: vsundararaman@worldbank.org)

We thank Julie Cullen, Gordon Dahl, Roger Gordon, Gordon Hanson, Hanan Jacoby and several seminar participants for comments. The World Bank and the UK Department for International Development (DFID) provided financial support for both the Zambia and India components of this paper. The experiment in India is part of a larger project known as the Andhra Pradesh Randomized Evaluation Study (AP RESt), which is a partnership between the Government of Andhra Pradesh, the Azim Premji Foundation, and the World Bank. We thank Dileep Ranjekar, Amit Dar, Samuel C. Carlson, and officials of the Department of School Education in Andhra Pradesh for their continuous support. We are especially grateful to DD Karopady, M Srinivasa Rao, and staff of the Azim Premji Foundation for their leadership in implementing the project in Andhra Pradesh. Vinayak Alladi provided excellent research assistance. The findings, interpretations, and conclusions expressed in this paper are those of the authors and do not necessarily represent the views of the World Bank, its Executive Directors, or the governments they represent.
# **1. Introduction**

The relationship between school inputs and education outcomes is of fundamental importance for education policy and has been the subject of hundreds of empirical studies around the world (see Hanushek 2002, and Hanushek and Luque 2003 for reviews of US and international evidence respectively). However, while the empirical public finance literature has traditionally paid careful attention to the behavioral responses of agents to public programs<sup>1</sup> , the empirical literature estimating education production functions has rarely accounted for household re-optimization in response to public spending. This is a critical gap because (a) household responses to education policies will mediate the extent to which different types of education spending translate into learning outcomes, and (b) parameters of education production functions are typically not identified if household inputs respond to changes in school-level inputs (see Urquiola and Verhoogen 2009 for one such example in the context of class-size).

We develop a dynamic model of household optimization that clarifies how increases in school-provided inputs translate into learning outcomes. We then test the main predictions of the model in two very different countries – Zambia and India – using unique matched data sets of school and household spending, and panel data on student achievement. A key contribution of this paper is our ability to measure household spending changes and student test-score gains in response to both _unanticipated_ as well as _anticipated_ changes in school funding. The former measures the production function effect of increased school funding (a partial derivative holding other inputs constant), while the latter measures the policy effect (a total derivative that accounts for re-optimization by agents).

The theoretical framework of a dynamic forward-looking model provides a useful guide to the key issues. In this framework, households' optimal spending decisions will take into account all information available at the time of decision making. The impact of school inputs on test scores depends then on (a) whether such inputs are anticipated or not and (b) the extent of substitutability between household and school inputs in the education production function. The model predicts that if household and school inputs are technical substitutes, an anticipated increase in school inputs in the next period will decrease household contributions that period. Unanticipated increases in school inputs limit the scope for household responses, leaving

> 1Illustrative examples include Meyer (1990) on unemployment insurance, Cutler and Gruber (1996) on health insurance, Eissa and Leibman (1996) on the EITC, Autor and Duggan (2003) on disability insurance.  See Moffitt (2002) for an overview on labor supply responses to welfare programs.

1
household contributions unchanged in the short run. These differences lead to a testable prediction: If household and school inputs are (technical) substitutes, unanticipated inputs will have a larger impact on test scores than anticipated inputs. We test this using data on educational spending for largely substitutable school inputs, such as books and writing materials in both Zambia and India.

Our data from Zambia allow us to distinguish between two different types of school spending: a predictable and fixed rule-based school block grant and an unpredictable districtlevel source of funds that varied widely across schools. The cross-sectional variation in the _perstudent_ rule-based grant comes from variation in school enrollment, which is instrumented for with the size of the catchment area (Case and Deaton 1999, and Urquiola 2006 use a similar instrumental variable strategy). We find that household spending substantially offsets variations in predicted per-student school grants. Evaluated at the mean, for each dollar spent on schools via the predictable grants, household spending on education reduces by a similar amount. In contrast, unpredictable grants have no impact on household spending. We also find that student test scores respond positively to the unanticipated sources of funds (test scores in schools receiving these funds are 0.10 standard deviations (SD) higher for both the English and mathematics tests for a mean transfer of just under $3 per pupil), but that they do not vary with variations in anticipated funds. This evidence is strongly suggestive that the two main predictions of the model are correct and is robust to several checks. However, we cannot fully rule out all identification concerns, and therefore test the model again using experimental variation induced by a randomly-assigned school grant program in the Indian state of Andhra Pradesh.

The Andhra Pradesh (AP) school block grant experiment was conducted across a representative sample of 200 government-run schools in rural AP with 100 schools selected by lottery to receive a school grant (also around $3 per pupil) over and above their regular allocation of teacher and non-teacher inputs. The conditions of the grant specified that the funds were to be spent on inputs used directly by students and not on any infrastructure or construction projects. The program was implemented for two years. In the first year, the grant was exogenously assigned and a surprise for recipient schools, while in the second year, the grant continued to be exogenous (relative to the comparison schools), but was now anticipated by the parents and teachers of program schools.

2
We find that household education spending in program schools is significantly lower in the second year than in the first year of the program suggesting that households offset the anticipated grant significantly more than they offset the unanticipated grant (just like in Zambia). Evaluated at the mean, the point estimates suggest that for each dollar spent in the form of the anticipated grant in the treatment group, household spending declines by 0.85 dollars (and we cannot reject that the grant is completely offset by the household). Further, students in program schools perform significantly better than those in comparison schools at the end of the first year of the (unanticipated) school grant program, scoring 0.08 and 0.09 SD more in language and mathematics tests respectively for a transfer of about $3 per pupil. In the second year of the program, there is no significant effect of the (anticipated) school grant on test scores. These findings are again consistent with the two main predictions of the model and are virtually identical to those from Zambia. The two sets of results complement each other and provide greater external validity to our findings. The Zambia case offers an analysis of two sources of funding (rule-based and discretionary), but relies on cross-sectional data and instrument quality. The AP case offers experimental variation in one source of funding, which changes from unanticipated to anticipated over time.

The policy implications of our results, which are discussed in the concluding section, follow from the insight that the impact of any school input on test-scores will depend on the degree of substitutability between the school input and what households can provide. The impact of anticipated school grants in both settings is low or zero, not because the money did not reach the schools (it did) or because it was not spent well (there is no evidence to support this), but because households realigned their own spending patterns optimally. The replication of the findings in two very different settings<sup>2</sup> , with two different implementing agencies (the government in Zambia and a leading non-profit organization in AP), and in representative population-based samples suggests that the impact of school grant programs is likely to be highly attenuated by household responses. Further, we find no heterogeneity in household responses

> 2 The two settings are similar in some ways including having high primary school enrollment but low student test scores and having limited funding for recurrent non-salary expenditures (Pratham 2010, Kanyika et al. 2005). However, at the time of the study, Zambia experienced severe declines in per-capita government education expenditure and a stagnant labor market, while Andhra Pradesh has been one of the fastest growing states in India with large increases in government spending in education over the last decade.  Our finding very similar results in a dynamic, growing economy and in another that was, at best, stagnant at the time of our study suggests that the results generalize across very different labor market conditions and the priority given to education in the government's budgetary framework.

3
across asset-poor and asset-rich households suggesting that school grants for learning materials may largely be viewed as pure income transfers to households, and that their long-term impact on learning is unlikely to be higher than the income elasticity of test scores. This has direct implications for thinking about the effectiveness of many such programs across several developing countries.<sup>3</sup> The distinction between anticipated and unanticipated inputs and the differential ability of households to substitute across various inputs could account for the wide variation in estimated coefficients of school inputs on test scores (Glewwe 2002, Hanushek 2003, or Kreuger 2003), and our results highlight the empirical importance of distinguishing between policy effects and production function parameters (Todd and Wolpin 2003, and Glewwe and Kremer 2005 make this point theoretically). A failure to reject the null hypothesis in studies that use the production function approach could arise either because the effect of school inputs on test scores through the production function is zero or because households (or teachers or schools) substitute their own resources for such inputs. While in our case the substitution takes the form of textbooks or writing materials, in a more general setting it may include parental time<sup>4</sup> , private tuition and other inputs.<sup>5</sup> Our results show that the policy effect of school inputs is different from the production function parameters with consequences both for estimation techniques and for policy.

The remainder of the paper is structured as follows. Section 2 describes the theoretical framework and develops the dynamic model which motivates our estimating equations. Section 3 presents results from Zambia using cross-sectional variation in anticipated and unanticipated school funding, while section 4 presents results from the school grant experiment in India. Section 5 discusses robustness to alternative interpretations and section 6 concludes.

# **2. Model**

The aim of this section is to offer an analytical framework to organize the empirical investigation and to understand the results. Becker and Tomes (1976) provide a classic model of the role of parents in spending on educational inputs, but do not model the interaction of school

> 3Examples include school grants under the Sarva Shiksha Abhiyan (SSA) program in India, the Bantuan Operasional Sekolah (BOS) grants in Indonesia, and several similar school grant programs in African countries (see Reinikka and Svensson 2004 for descriptions of school grant programs in Uganda, Tanzania, and Ghana).

> 4 Houtenville and Conway (2008) estimate an achievement production function that includes measures of parental effort and find that parental effort is negatively correlated with school resources.

> 5 Of course, not all school inputs are substitutes.  As we show in Section 2, these predictions do not hold for school inputs that are complementary to household inputs.

4
and household inputs. Todd and Wolpin (2003) allow for the possible substitutability of household and school inputs, but do not offer an explicit optimization model to derive empirical predictions. The contribution of our model is to specify the household's dynamic optimization problem, solve it subject to both budget and production function constraints, and to derive the Euler equation that shows the optimal growth path of test scores (based on an appropriate shadow price of the cost of investing in educational inputs in each period).<sup>6</sup> We use this solution to discuss the differential impact of anticipated and unanticipated school inputs on test-score improvements and show how this varies based on whether school and household spending are substitutes or complements.

A household derives (instantaneous) utility from the test scores of a child, _TS_ , and the consumption of other goods, _X_ . The household maximizes an inter-temporal utility function _U(.)_ , additive over time and states of the world with discount rate _β_ (<1), subject to an inter-temporal budget constraint. Finally, test scores are determined by a production function relating current achievement _TSt_ to past achievement _TSt-1_ , household educational inputs _zt_ , school inputs<sup>_w_</sup> _t_<sup>, non</sup> time-varying child characteristics _μ_ and non time-varying school characteristics _η_ . We assume that household utility is additively separable, increasing and concave in test scores and other goods [A1]; and that the production function for test scores is given by _TSt_  _F(TSt_  _1,wt z,t ,_  _,_  _)_ where _F_ (.) is concave in its arguments [A2]. Under [A1] and [A2] the household problem is Here _u_ and _v_ are concave in each of their arguments. The inter-temporal budget constraint, Equation (2), links asset levels<sup>_A_</sup> _t_  1 with initial assets _At_ , private spending on educational inputs _zt_ , income _yt_ and the consumption of other goods, _Xt_ . The price of educational inputs is the numéraire, the price of other consumption goods is _Pt_ and _r_ is the interest rate. The production function constraint, Equation (3) dictates how inputs are converted to educational outcomes, and

> 6This relates closely to the discussion on durable goods and inter-temporal household optimization; see Deaton and Muellbauer (1980), Jacoby and Skoufias (1997), Foster (1995) and Dercon and Krishnan (2000).

5
the boundary condition, Equation (4) requires that at _t=T_ , the household disposes of all remaining assets so that all loans are paid back and there is no bequest motive.

We treat test scores as the observable measure of human capital. The latter is what parents care about, while the former is what they observe and optimize with respect to. The formulation can be seen as a short-cut for an alternative set up in which parents derive future utility from the flow of returns to the child‘s stock of human capital, as in a more standard human capital investment model. As we are mainly interested in deriving the optimal dynamic path for reaching the desired stock of human capital considering the costs and benefits of boosting test scores in current and future periods, the insights gained from using a human capital investment model are going to be similar, given the other assumptions made, especially the concavity of the period-byperiod production function.<sup>7</sup> In this formulation, credit markets are perfect so that there are no bounds on<sup>_A_</sup> _t_  1 apart from Equation (4).<sup>8</sup> Moreover, households choose only the levels of _Xt_ and _zt_ so that school inputs,<sup>_w_</sup> _t_ are beyond its control. In the contexts studied here, this is a reasonable assumption since school resources are allocated at state or federal levels and are not tied to a local property tax that residents may choose (unlike in the US). At the time the household makes its decision, it knows the underlying stochastic process governing<sup>_w_</sup> _t_<sup>but not the actual level; we assume that school</sup> inputs are a source of uncertainty in the model—for simplicity, the only source.

Maximization of Equation (1) subject to Equations (2) and (3) provides a decision rule related to _TSt_ , characterizing the demand for test scores. Since test scores are a stock, we define a per-period price for test scores as the user-cost of increasing the stock in one period by one unit, i.e., the relevant (shadow) price in each period for the household. As in the durable goods literature (Deaton and Muellbauer 1980), the user cost, evaluated at period _t_ is (see Das et al. (2004) for its derivation): Here, the first term measures the cost of taking resources at _t_ and transforming them into one

> 7 Further, we assume that households care about the level of educational achievement, a stock. Results are unaffected if households care about the (instantaneous) flow from educational outcomes, provided that the flow is linear in the stock.

> 8It is straightforward to incorporate imperfect credit markets in this framework (see Das et al. 2004).

6
extra unit of test scores. When implemented through a production function, the cost of buying an extra unit is the inverse of the marginal product of spending, _F_<sup>_z_</sup> _t_ (.) . However, since _TS_ is durable, increasing _TS_ in period _t_ , reduces the cost of acquiring _TS_ in period _t+1_ proportional to _FTSt_ (.) and the second term thus measures the present value of this reduction in cost in the next period, expressed in monetary terms.<sup>9</sup> Given the user cost, the first-order Euler condition determines the optimal path of educational outcomes between period _t-1_ and _t_ as: This is a standard Euler equation stating that along the optimal path, test scores will be smooth, so that the marginal utilities of educational outcomes will be equal in expectations, appropriately discounted and priced. Finally, the concavity of the production function _in each time period_ will limit the willingness of households to boost education fast since the cost is increasing in household inputs.<sup>10</sup> Starting from low levels in childhood, the optimal path will be characterized by a gradual increase in educational achievement over time.

Under the further assumptions that household utility is additively separable and of the CRRA form, and that marginal utility is defined as _TSt_  , (ρ the coefficient of relative risk aversion), Equation (6) can be rewritten as: Where _e_ t is an expectation error, uncorrelated with information at _t-1_ . Taking logs and expressed for child _i_ , we obtain the optimal growth path: 9 In the durable goods literature, the user cost per period is derived by assuming that the good is sold in the second period. Though there is no ―second hand market‖ for test scores, the shadow price for consuming a unit of test scores derived above is similar to those derived in the durable goods literature (see Foster 1995 for a similar derivation of the rental-equivalent price of boosting nutritional status in one period).

10 The "per-period" concavity of the education production function can be motivated in several ways, the most intuitive of which is the existence of limits to how much a student can learn in a given period of time. While the unit of time is not specified in the model (as in the consumption smoothing literature in general), it is natural to consider the unit to be one year in the context of education, since decisions regarding education are typically made prior to the start of the school year. If an additional school grant arrives after this initial spending and is spent on learning materials, households are unlikely to be able to sell materials already purchased and we assume that they will only re-optimize at the start of the next school year.

7
which is determined by the path of user-costs, and a term capturing surprises.

In this paper, we do not aim to use the structural dynamic model to estimate an impulseresponse function over time to an unexpected change in inputs (the data requirements for that exercise are beyond any education data set we know of). However, we can use this theoretical model to derive an empirical model that nests some key predictions on how anticipated and unanticipated inputs affect the path of test scores. To derive these, assume that school resources are not known with certainty until households make decisions regarding their own inputs. Let _a u wt_ (<sup>_w_</sup> _t_ ) be inputs at time t that were anticipated (unanticipated) at _t-1_ . For _unanticipated_ increases in school inputs, households are unable to respond till the next time period and are therefore pushed off the optimal path (see footnote 10). The increase in educational achievement in period _t_ is given by _Fwt_<sup>_dw_</sup> _tu_ , and the change in the growth path is given by _ln( TSt_<sup></sup> _wtu Fw )_ which is strictly positive.

In the case of anticipated increases, the effect on the path of outcomes will depend on the impact on the user-cost of educational achievement at _t_ , since there is no _direct_ impact on the budget constraint at _t_ (all information related to the anticipated inputs, including the budget constraint, will have been incorporated into the programming problem at _t-1_ ). Using the implicit function theorem with Equation (5) and assuming _TSt_  1(  ) _TSt_  1  _F_ ( _wt_ , _zt_ ,  ,  ) where the Hessian of _F_ (.) is negative semi-definite, The change in the optimal growth path is given by If household and school inputs are technical substitutes so that<sup>_F_</sup> _zt_<sup>_w_</sup> _t_  0 , anticipated increases in school inputs at _t_ increase the relative user-cost of boosting _TS_ at _t_ , resulting in lower 8
growth of test scores, ceteris paribus, between _t_ and _t-1_ .<sup>11</sup> Households have (price) incentives to shift resources for educational spending to _t-1_ , boosting educational achievement at _t-1_ in anticipation of the higher resources at _t_ , and also to take advantage of the higher overall resources for educational inputs that allow them to spend relatively less on educational inputs compared to other commodities. Thus, the effect of an unanticipated change is higher than that of an anticipated change: household spending on educational inputs at _t_ is unchanged, as households cannot move some of their spending to _t-1_ , or to other commodities, as they could with anticipated increases of government spending.<sup>12,13</sup> Assuming identical risk preferences, an empirical specification consistent with (8) is: Here, _wita_ and _witu_ are anticipated and unanticipated changes in school inputs, measured in this paper by the flows of funds, while  _X t_ reflects all other sources of changes in the user cost between _t_ and _t-1_ . The core prediction is that the marginal effect of anticipated funds is lower than that of unanticipated funds when household and school inputs are substitutes. Finally, it is easy to see that if a portion of what the econometrician regards as unanticipated was anticipated by the household (or was substitutable even after the 'surprise' arrival of the school grant), then

> 11 In other words, if<sup>_F_</sup> _zt_<sup>_w_</sup> _t_  0 , an increase in<sup>_w_</sup> _t_<sup>will decrease the marginal product of</sup><sup>_z_</sup> _t_<sup>(and therefore increase</sup>

the price of boosting TS by increasing<sup>_z_</sup> _t_<sup>). We do not model the schools' choice of inputs to spend on, but if their</sup> objective function is to maximize _TS_ , they should optimally allocate cash grants across different inputs and therefore account for the degree of substitution with households. One way to interpret these results is that schools are constrained in what they can do and are hence unable to spend this funding on inputs that could not be easily substituted for by parental resources. These constraints could arise either due to thin markets, explicit restrictions on the use of the grants (to hire teachers for instance), an inability to exploit scale economies (for instance, to improve infrastructure), or parental preferences expressed through school committees to spend on substitutable items). 12 If school and households inputs are technical complements, increasing school inputs at _t_ will increase the marginal productivity of household inputs at t, and through the decline in user-costs lead to higher growth in test scores along the optimal path between t and t-1. _Anticipated_ lower user costs for educational inputs at t relative to t-1 create incentives to shift resources from t-1 to t, leading to a higher growth of test scores between t and t-1. Whether this reduces spending and therefore test scores at t-1 depends on preferences, as households have incentives to keep the optimal path of test scores smooth, while taking advantage of the additional government spending at t to spend more on other commodities.

13 The model above is written as if there is only one type of school and household inputs. It is straightforward to allow for multiple inputs, taking _w_ and _z_ as _vectors_ of educational inputs in the model. Different inputs could have different cross-derivatives, implying different degrees of technical substitutability, so that the extent to which the household may substitute for school spending on particular inputs may differ. We return to this issue in section 5.

9
the estimate of α2 will be a lower bound of the true production function effect (see section 5.4).

# **3. Zambia**

# **3.1 Background and Context**

The educational system in Zambia is based on public schools (less than 2 percent of all schools are privately run) and the country has a history of high primary enrollment rates. Teacher salaries are paid directly by the central government, and account for the majority of spending on school-level resources; schools receive few other resources from government. Districts receive some discretionary funding for non-salary purposes from the central government and aid programs. However, since the 1990s, these sources were highly unreliable and unpredictable, partly due to the operation of a "cash budget" in view of the poor macroeconomic situation, and partly due to the irregularity of much of the aid flows to the education sector (Dinh, et al. 2002). In 2002, the year of our survey, less than 24 percent of all schools received such discretionary grants and conditional on receipt, there was considerable variation with some schools receiving 30 times as much as others. Few resources were distributed in kind to schools during the year of the survey (see Das et. al 2003). Overall, the share of discretionary resources was only about a tenth of the share of the teacher salary bill.

Parental involvement in schools is high with parents traditionally expected to contribute considerably to the finances of the school via fees paid through the Parent Teacher Association (PTA). Limited direct government funding for non-salary purposes during economic decline put pressure on parents to provide for inputs more usually provided by government expenditure. This customary arrangement regarding PTA fees changed in 2001; following an agenda of free education, all institutionalized parental contributions to schools, including formal PTA fees were banned in April 2001. At the same time, this put further pressure to complement school finances by further direct private parental spending on education.

In 2001 the year preceding our survey, a rule-based cash grant through the government's Basic Education Sub-Sector Investment Program (BESSIP) was provided to every school to reverse some of the pressure on school finances arising from a persistent economic decline. These grants were fixed at $600 per school ($650 in the case of schools with Grades 8 and 9) irrespective of school enrollment to exclude any discretion by the administration. The grant was managed via a separate funding stream from any other financial flows, and directly delivered to 10
the school, via the headmaster. Spending decisions were made at the Annual General Meeting, before the start of the school year. The share of this funding in overall school funding was considerable: for 76% of schools it was the only public funding for non-salary inputs, while its average share in total school resources was 86%. The scheme also attracted much publicity, increasing its transparency; combined with the simplicity of the allocation rule, this ensured that the grants reached their intended recipients. Disbursement was fast and reliable and 95 percent of all schools had received the stipulated amounts by the time of the survey and the remainder within 1 month of survey completion (Das et al. 2003).<sup>14</sup> Therefore, we expect that in the year of the survey the fixed cash grants would be anticipated by households making their educational investment decisions for the year, contrary to discretionary sources, which had become highly unpredictable and therefore unanticipated. Furthermore, because the grants were fixed in size, there was considerable variation across schools in per-student terms due to underlying differences in enrollment. We use the variation in per-student amounts to examine the crowding-out of household expenditures, a strategy discussed further below.

# **3.2 Sampling and Data**

We collected data in 2002 from 172<sup>15</sup> schools in 4 provinces of Zambia (covering 58 percent of the population), where the schools were sampled to ensure that every enrolled child had an equal probability of inclusion. The results are therefore externally valid within the 4 provinces of the study. The school surveys provide basic information on school materials and funding as well as test scores for mathematics and English for a sample of 20 students in grade 5 in every school, who were tested in 2001 as part of an independent study and were then retested in 2002 to form a panel.

To supplement these data, we also collected information for 540 households matched to a sub-sample of 34 schools identified as "remote" using GIS mapping tools (defined as schools where the closest neighboring school was at least 5 kilometers away). From these schools, the closest village was chosen and 15 households were randomly chosen from households with at least one child of school-going age. The restriction of the household survey sample to 34 remote

> 14 This contrasts with the early experience in Uganda (Reinnika and Svensson 2004).

> 15 The initial sample contained 182 schools, although 2 yielded only incomplete information, 5 were private schools not covered in this paper and 3 could not be matched to the test scores data from the Examination Council of Zambia.

11
schools allows us to match household and school inputs in an environment where complications arising from school choice are eliminated. We use the entire sample of 172 schools to estimate the relationship between test scores and cash grants to schools (rule-based and discretionary). We use the sub-sample of 34 schools matched to 540 households to estimate the relationship between rule-based cash grants to schools and household expenditures on education.

Table 1 presents summary statistics separately for rural and urban schools, as well as for schools that are in our "remote" sample and matched to households. As might be expected, there are significant differences between rural and urban areas, with the latter having a better-off student body, but not necessarily better school supplies per student. Our sample of "remote" schools is not significantly different from rural schools on most measures, but they attract poorer students than the other rural schools, and have relatively more books and desks per student (though each desk or textbook is still shared by two students). Per student funding from the predictable rule-based grant increases as we go from urban to rural to remote schools, which is consistent with a fixed rule-based grant being distributed among fewer students in rural and remote areas.

Substantial parts of school spending are suitable for substitution by parents. On average 54% is spent on books, chalk (for slates), stationary and other school materials by the school while 23% is spent on sports materials and equipment. About 19% is spent on utilities, maintenance and infrastructure, and only 3% is spent on allowances and other costs linked to teachers.<sup>16</sup>

# **3.3 Empirical Methodology**

We first test whether there is crowding out of household educational spending in response to anticipated grants. We estimate a cross-section demand model for the 1195 children (from 540 households) matched to 34 schools in which household spending on school-related inputs is regressed on anticipated and unanticipated grants with and without a set of controls for child, household and school-level variables: respectively anticipated (rule-based) and unanticipated (discretionary) grants per student in

> 16 Looking at average spending shares by households, 27% is on books and stationary, and other materials for school while 19% is spent on cash contributions of various forms (although PTA fees were formally abolished) and other direct cash payments to the school. The remainder, 54% of household expenditure, is on school and sport uniforms and shoes, and for sports activities at school.

12
school _j_ that matches to child _i_ , and _X i_ are other characteristics of child _i_ including assets owned by the household. We test  2  3  0 , i.e., households respond negatively to the preannounced, anticipated rule-based grants at the school level by cutting back their own funding, but are unable to respond to cash grants that are unanticipated.

To address the concern that _waj_ captures unobserved components of household demand operating through an enrollment channel, we use the size of the eligible cohort in the catchment area as an instrument for school enrollment and therefore the level of per-student cash grants. This instrumentation strategy is similar to Case and Deaton (1999), Urquiola (2006) in the case of class-size and more recently by Boonperm et al. (2009) and Kaboski and Townsend (2008) in the context of large fixed grants to villages in Thailand. Using the size of the eligible cohort as an instrument for enrollment is especially credible in this context since we use only a sample of _remote_ schools and can abstract away from issues of school choice. We also confirm that there is no correlation between the instrument and _X i_ .

We explore the impact of different spending types using Equation (13), based on (11), modeling changes in standardized test-scores _TS_ between _t_ and _t-1_ regressed on anticipated and unanticipated spending, and a set of controls at _t-1_ capturing sources of heterogeneity and differences in user costs.

_a u_  _TSit_   _o_   _1 ln wit_  _2 ln wit_   _3 X t_  _1_   _it_ (13) The prediction is that α1 < α2: unanticipated spending will have a larger effect on test scores than anticipated spending.<sup>17</sup>

# **3.4 Results**

# **3.4.1 Household Spending**

The results of estimating (12) are presented in Table 2, showing results without and with controls, and using the size of the eligible cohort in the catchment area as an instrument.<sup>18</sup> The results are consistent with the predictions from our model: across all specifications, the estimated elasticity of substitution for anticipated grants ( ) is always negative and significant and ranges

> 17 In one specification shown, Xt-1 will include the lagged dependent variable TSit-1 as a further control for heterogeneity in the path of test-scores over time.

> 18 We can reject the hypothesis that the instrument is weak: the F-statistic of the first stage regression is above 10. The impact of an extra child in the catchment area on enrollment is 0.68 – which is close to the actual enrollment of about 80% in the sample.

13
from -0.72 to -1.12 while the coefficient of unanticipated grants ( ) is small and insignificant.<sup>19</sup> Evaluated at the mean we cannot reject the hypothesis that for each dollar spent on the rule-based grant per student, households reduce school expenditure by one dollar, while there is no substitution of discretionary, unanticipated spending.

One concern may be that households in larger villages (which have smaller per capita anticipated funding) have a different overall demand for education. We address this concern by comparing household expenditure across schools with different levels of rule-based grants. We divide schools into two categories - those receiving less than the median per-child rule-based grant ("low rule-based grant schools) and those receiving more than the median ("high rule-based grant schools) - and Table 3 shows school and household expenditure for these two types of schools. As expected, we find that the per-student grant is significantly lower in the "low rulebased grant" schools. However, household spending on education is significantly higher in these schools. Most importantly, there is _no significant difference_ in total expenditure per child across these two school types. This suggests that overall demand for education is similar across the households in the sample, and that they compensate/offset for lower/higher spending at the school level.<sup>20</sup>

# **3.4.2 Test Scores**

Tables 4A and 4B show the results for English and Mathematics for different specifications where all estimations are at the school level, based on equation (13). The high variability in discretionary funding, with less than a quarter of the school sample receiving any fund, and other schools receiving very high levels, encourages us to explore two specifications for discretionary funding. Table 4A shows the results, expressing discretionary funding as dummy variable, while in Table 4B, we introduce both the level and the square of discretionary funding. In each table, we show two specifications for test-score results for English and for Mathematics. In a first specification, we only include some geographical characteristics (rural/urban and province dummies). The second is our key result, and includes also changes in other school level characteristics that change over time in the data (changes in head teacher, changes in chair of the

> 19 Only 4 schools (or 12%, with about 150 students in total) in this sample received discretionary funding, possibly weakening this test.

> 20 In a parent‘s succinct summary: " _The school had no textbooks this year, so we had to buy our own_ ".

14
Parent-Teaching Association, and changes in fees for this association).<sup>21</sup> Other school-level controls did not affect the results.

For all specifications, the coefficient on anticipated grants is small and insignificant: there is no improvement in test-scores from these rule-based grants. Adding higher order terms for anticipated grants does not make any difference. For English, there is a consistent impact of discretionary funds received by the school. In a specification with level and square terms (Table 4B), the overall effect is significant at 10% (in column 2).<sup>22</sup> When added as a dummy in Table 4A, the effect is significant at 5% (in column 2). For Mathematics, the effect of discretionary funds is significant only when added as a dummy (at 10%, see column 3 in Table 4A). Nonparametric investigation of the relationship between levels of discretionary funds and test score gains suggested a highly non-linear relationship for both English and Mathematics (not shown). Consistent with Table 4A, a positive relationship with discretionary funds exists for both subjects, but Table 4B suggests that a simple (quadratic) parametric formulation is only sufficient to capture this relationship for English. Focusing on the results in Table 4A (columns 2 and 4), we find that on average, receiving discretionary funds adds 0.10 of a SD of test-scores, in both English and Mathematics; in contrast, and consistent with the predictions of the model, there is no impact from rule-based, anticipated school grants.

One key threat to identification in the results above is the possibility that the discretionary/unanticipated grants may have been targeted to areas with the most potential improvement in test scores. Alternatively, parents and communities that care enough to obtain these funds for their schools may also be motivated to increase test scores in other ways. We address this concern by comparing the observed characteristics of schools that do and do not receive these discretionary funds and find that there is no significant difference between these types of schools (Table 5). In this table, as column [3] shows, we find no difference in initial levels of test scores, other school performance indicators, location or wealth characteristics between these two types of schools. Column [4] shows OLS results when using these characteristics to try to explain whether discretionary funding was received, and we reject the joint significance of these characteristics. At least on the basis of observables, there is no evidence of differences between these two types of schools that are correlated with the trajectory

> 21 These controls could be thought of as potentially changing the benefits of spending on schooling by parents (i.e. the user costs).

> 22 Although the squared term is negative, for all observed values in the sample, the overall effect is still positive.

15
of test score of gains.

# **3.4.3 Limitations**

These results are strongly suggestive of the processes outlined in the theory: household and school-level funds are technical substitutes in the production function of test scores and when school-level funding increases, it crowds out private spending within the household. Consequently, such grants have little (if any) impact on the path of test-scores. We are also able to show that the lack of a relationship between test-scores and school-level funding is _not_ because such funds have no effect through the production function—when households are ―surprised‖ and cannot adjust their own expenditures, test-scores increase with school funding.

However, there are a few caveats to the estimates presented from Zambia. First, our household substitution results are only valid for the ―remote‖ rural sample and while we can show that household spending offsets variation in anticipated funding, our test of the hypothesis that it does not respond to unanticipated funding is based on a small sample (only 4 of the 34 linked schools in the remote sample reported any unanticipated funding at all). Second, while standard in the literature, we cannot rule out that the size of the catchment area (used as an instrument for school-level enrollment) could be correlated to returns in the labor market or historical levels of education in the population. These in turn may be directly correlated to educational investments thus biasing downwards our estimates of crowding-out. Third, unanticipated funds could have been targeted in unobservable ways to schools where parental substitution would be less and where test-scores were more likely to increase even in the absence of funding.

While the patterns of observed characteristics and the stability of the results to the use of credible instrumental variables suggest that these are not serious concerns, we cannot fully rule out these alternate explanations. We therefore present results from a field experiment in the Indian state of Andhra Pradesh designed to specifically determine the pattern of crowd-out and we show that the results obtained are virtually identical across these different contexts. The randomized school grants address the second and third caveats above, while the first limitation is addressed by collecting spending data from a large sample of households at multiple points in time (both when the grants were a surprise, and later when they were expected), which allows us to test differential household responses to anticipated and unanticipated funds across a large representative sample of schools.

16
# **4 Andhra Pradesh, India**

# **4.1   Background and Context**

Andhra Pradesh (AP) is the 5<sup>th</sup> largest state in India, with a population of over 80 million, 73% of whom live in rural areas. AP is close to the all-India average on various measures of human development such as gross enrollment in primary school, literacy, and infant mortality, as well as on measures of service delivery such as teacher absence. There are a total of over 60,000 government primary schools in AP and over 70% of children in rural AP attend government-run schools (Pratham 2010).

The average rural primary school is quite small, with total enrollment of around 80 to 100 students and an average of 3 teachers across grades one through five.<sup>23</sup> Teachers are well paid, with the average salary of regular civil-service teachers being over Rs. 8,000/month and total compensation including benefits being over Rs. 10,000/month (per capita income in AP is around Rs. 2,000/month). Regular teachers' salaries and benefits comprise over 90% of noncapital expenditure on primary education in AP, leaving relatively little funds for recurring nonteacher expenses.<sup>24</sup> Some of these funds are used to provide schools with an annual grant of Rs. 2,000 for discretionary expenditures on school improvement and to provide each teacher with an annual grant of Rs. 500 for the purchase of classroom materials of the teachers‘ choice. The government also provides children with free text books through the school. However, compared to the annual spending on teacher salaries of over Rs. 300,000 per primary school (three teachers per school on average) the amount spent on learning materials is very small. It has been suggested therefore that the marginal returns to spending on learning materials used directly by children may be higher than more spending on teachers (Pritchett and Filmer 1999). The AP School Block Grant experiment was designed to evaluate the impact of providing schools with grants for learning materials, and the continuation of the experiment over two years (with the provision of a grant each year) allows us to test the differences between unanticipated and anticipated sources of school funds.

> 23 This is a consequence of the priority placed on providing all children with access to a primary school within a distance of 1 kilometer from their homes.

> 24 Funds for capital expenditure (school construction and maintenance) come from a different part of the budget. Note that all figures correspond to the years 2005 - 07, which is the time of the study, unless stated otherwise.

17
# **4.2  Sampling, Randomization, and Program Description**

The school block grant (BG) program was evaluated as part of a larger education research initiative (across 500 schools) known as the Andhra Pradesh Randomized Evaluation Studies (AP RESt), with 100 schools being randomly assigned to each of four treatment and one control groups.<sup>25</sup> We sampled 5 districts across each of the 3 socio-cultural regions of AP in proportion to population. In each of the 5 districts, we randomly selected one administrative division and then randomly sampled 10 mandals (the lowest administrative tier) in the selected division. In each of the 50 mandals, we randomly sampled 10 schools using probability proportional to enrollment. Thus, the universe of 500 schools in the study was representative of the schooling conditions of the typical child attending a government-run primary school in rural AP. Experimental results in this sample can therefore be credibly extrapolated to the full state of Andhra Pradesh.

The school year in AP starts in mid June, and baseline tests were conducted in the 500 sampled schools during late June and early July, 2005.<sup>26</sup> After the baseline tests were evaluated, 2 out of the 10 project schools in each mandal were randomly allocated to one of 5 cells (four treatments and one control). Since 50 mandals were chosen across 5 districts, there were a total of 100 schools (spread out across the state) in each cell. The geographic stratification allows us to estimate the treatment impact with mandal-level fixed effects and thereby net out any common factors at the lowest administrative level of government, and also improve the efficiency of the estimates of program impact.

Since no school received more than one treatment, we can analyze the impact of each program independently with respect to the control schools without worrying about any confounding interactions. The analysis in this paper is based on the 200 schools that comprise the 100 schools randomly chosen for the school block grant program and the 100 that were randomly assigned to the comparison group. Table 6 shows summary statistics of baseline

> 25 AP RESt is a partnership between the government of AP, the Azim Premji Foundation (a leading non-profit organization in India), and the World Bank. The World Bank and the UK Department for International Development (DFID) provided financial support. The Azim Premji Foundation (APF) was the main implementing agency for the study.  See Muralidharan and Sundararaman (2010, 2011) for details of other interventions.

> 26 The selected schools were informed by the government that an external assessment of learning would take place in this period, but there was no communication to any school about any of the treatments at this time.

18
school and student characteristics for both treatment and comparison schools and the null of equality across treatment groups cannot be rejected for any of the variables.<sup>27</sup> As mentioned earlier, the block grant intervention targeted non-teacher and noninfrastructure inputs directly used by students. The block grant amount was set at Rs. 125 per student per year (around $3) so that the average additional spending per school was the same across all four programs evaluated under the AP RESt.<sup>28</sup> After the randomization was conducted, project staff from the Azim Premji Foundation (APF) personally went to selected schools to communicate the details of the school block grant program (in August 2005). The schools had the freedom to decide how to spend the block grant, subject to guidelines that required the money to be spent on inputs directly used by children. Schools receiving the block grant were given a few weeks to make a list of items they would like to procure. The list was approved by the project manager from APF, and the materials were jointly procured by the teachers and the APF field coordinators and provided to the schools by September, 2005. This method of grant disbursal ensured that corruption was limited and that the materials reached the schools and children.

APF field coordinators also informed the schools that the program was likely to continue for a second year subject to government approval. Thus, while program continuation was not guaranteed, the expectation was that it was likely to continue for a second year. Schools were told early in the second year (June 2006) that they would continue being eligible for the school grant program and the same procedure was followed for disbursal of materials (no money was handed over to schools or teachers, and procurement was conducted jointly).

Table 7 shows that the majority of the grant money was spent on student stationary such as notebooks, and writing materials (over 40%), classroom materials such as charts (around 25%), and practice materials such as workbooks and exercise books (around 20%). A small amount (under 10%) of the grant was spent in the first year on student durable items like school bags, and plates/cups/spoons for the school mid-day meal program. This amount seems to have been transferred to stationary and writing materials in the second year. We also see that the overall

> 27 Table 6 shows sample balance between the comparison schools and those that received the block grant, which is the focus of the analysis in this paper.   The randomization was done jointly across all treatments and the sample was also balanced on observables across the other treatments.

> 28 The block grant was set on the basis of the number of students who took the baseline tests as opposed to the number of students enrolled (except for the first grade where there was no baseline test).  This ensured that schools that inflated enrollment (which is not uncommon in India) were not rewarded with a larger grant.

19
spending pattern at the school level is quite stable across the first and second year of the grant. Many of these items could be provided directly by parents for their children, suggesting a high potential for substitution.

# **4.3  Data**

Data on household expenditure on education was collected from a household survey that attempted to cover every household with a child in a treatment or comparison school and administer a short questionnaire on education expenditures on the concerned child during the previous school year.<sup>29</sup> Data on household spending was collected at three points in time – alongside the baseline tests for spending incurred in the pre-baseline year (Y0), during the second year of the program about spending during the first year (Y1), and after two full years of the program about spending during the second year (Y2). Data on household education spending was collected retrospectively to ensure that this reflected all spending during the school year.<sup>30</sup> The outcome data used in this paper comprise of independent learning assessments in math and language (Telugu) conducted at the beginning of the study, and at the end of each of the two years of the experiment. The baseline test (June-July, 2005) covered competencies up to that of the previous school year. At the end of the school year (March-April, 2006), schools had two rounds of tests with a gap of two weeks between them. The first test covered competencies up to that of the previous school year, while the second test covered materials from the current school year's syllabus. The same procedure was repeated at the end of the second year, with two rounds of testing. Doing two rounds of testing at the end of each year allows for the inclusion of more overlapping materials across years of testing, reduces the impact of measurement errors specific to the day of testing by having multiple tests around two weeks apart, and also reduces sample attrition due to student absence on the day of the test.

For the rest of this paper, Year 0 (Y0) refers to the baseline tests in June-July 2005; Year 1 (Y1) refers to the mean score across both rounds of tests conducted at the end of the first year of the program in March-April, 2006; and Year 2 (Y2) refers to the mean score across both rounds of tests conducted at the end of the second year of the program in March-April, 2007. All analysis is carried out with normalized test scores, where individual test scores are converted to

> 29 The data was collected from a short survey that was only based on the ―main‖ child who was being covered in the school assessments and not for other siblings or other components of household spending.

> 30 We obtained spending data from a total of 8,612 households for Y0 (no data was collected for retrospective spending on children in grade 1, because it was their first year in school), 13,572 households for Y1, and 10,189 households for Y2.

20
z-scores by normalizing them with respect to the distribution of scores in the control schools on the same test.<sup>31</sup>

# **4.4   Results**

# **4.4.1  Household Spending**

We estimate: ln _zijkt_   0  _Y_ 0  1  _Y_ 1  2  _Y_ 2  3  _BG_  _Y_ 0  4  _BG_  _Y_ 1  5  _BG_  _Y_ 2  _Zm_   _ijk_ (14) where _ln zijkt_ is the expenditure incurred by the household on education of child _i,_ at time _t_ ( _j, k,_ denote the grade, and school), _Yn_ is the project year, _BG_ is an indicator for whether or not the child was in a ―block grant‖ school, and standard errors are clustered at the school level. The parameters of interest are  3 , which should equal zero if the randomization was valid (no differential spending by program households in the year prior to the intervention);  4 , which measures the extent to which household spending adjusted to an unanticipated increase in school resources (since the block grant program was a surprise in the first year of the project), and  5 , which measures the response of household spending to an anticipated increase in school resources (since the grant was mostly anticipated in the second year).<sup>32</sup> All regressions include a set of mandal-level dummies ( _Zm_ ) to account for stratification and to increase efficiency.

Table 8 confirms that that  3 and  4 are not significantly different from zero while  5 is significantly negative. We report the results both with and without a full set of household controls, and the results are unchanged. The findings are fully consistent with the predictions of the model: in Y1, households did not adjust to the unexpected grant, while in Y2, household spending was adjusted in anticipation of provision of materials by the school (using the grant).<sup>33</sup> The estimated elasticity of -0.25 to -0.27 suggests that at the mean household expenditure for the

> 31 Since all analysis is done with normalized test scores (relative to the control school distribution), a student can be absent on one testing day and still be included in the analysis without bias because the included score is normalized relative to the control school distribution for the same test that the student took.

> 32 We say ―mostly anticipated‖ because it was not guaranteed that the program would be continued to the second year, but field reports suggest that the perception of the likelihood of continuation was high enough that households waited to see the materials provided by the schools before doing their own spending.

> 33 This was further corroborated by field reports after the program was withdrawn, which suggest that most parents did not buy the materials that they thought would be provided by the school.

21
comparison group (Rs 411 in Y2), the per-child grant of Rs. 125 would be almost entirely offset, and we cannot reject that the substitution is 100% (the estimated offset is around 85%).<sup>34</sup>

# **4.4.2  Student Test Scores**

Our default specification for studying the impact of the school block grant, consistent with equation (11) uses the form: The main dependent variable of interest is  _Tijkm_ , which is the change in the normalized test score on the specific test (normalized with respect to the score distribution of the comparison schools), where _i, j, k, m_ denote the student, grade, school, and mandal respectively.<sup>_Y_</sup> 0<sup>indicates</sup> the baseline tests, while<sup>_Y_</sup> _n_<sup>indicates a test at the end of</sup><sup>_n_years of the program. Including the</sup> normalized baseline test score improves efficiency due to the autocorrelation between test-scores across multiple periods.<sup>35</sup> These regressions also include a set of mandal-level dummies ( _Zm_ ) and the standard errors are clustered at the school level. We also run the regressions with and without controls for household and school variables. They will allow us to capture any sources of changes in user costs, as in (11).

The _BG_ variable is a dummy at the school level indicating if it was selected to receive the school block grant (BG) program, and the parameter of interest is _δ_ , which is the effect on the normalized test scores of being in a school that received the grant. The random assignment of treatment ensures that the _BG_ variable in the equation above is not correlated with the error term, and the estimate of the one-year and two-year treatment effects are therefore unbiased.<sup>36</sup>

> 34 As in the Zambia case, we used a logarithmic specification; estimating a linear model in levels of spending we found identical results, including that we could not reject total substitution by households of the school grant in Y2. 35 The inclusion of the baseline test score also allows us to control also for individual heterogeneity correlated with baseline test-scores. In the case of Zambia, we explored adding this to the specification in table 4A and 4B, but, this creates endogeneity problems for inference related to the variables of interest (spending). In the AP case, the randomization ensures that the BG variable is uncorrelated with the error term.  Since grade 1 children did not have a baseline test, we set the normalized baseline score to zero for these children (similarly for children in grade 2 at the end of two years of the treatment).

> 36 We also check for differential post-treatment attrition of teachers and students and find that there is no differential attrition or turnover of teachers between "block grant" and "control" schools.  However, there is a small amount of differential student participation in the test at the end of the first year of the program (with attrition from the baseline test-taking sample of 5.4% and 8.2% in the treatment and control groups respectively). As weaker students may drop out of the testing sample, this may bias our estimate of the first-year treatment effect downwards, but since the magnitude of differential attrition is small (2.8%), this bias is likely to be quite small, especially since baseline scores are controlled for.  In the second year, however, there is no differential attendance on the end of year tests.

22
Note that specification in (15) can be used to consistently estimate the one-year and two-year effect of the program, but not the second year effect alone (with the second-year gains as the dependent variable controlling for Y1 scores) because Y1 scores are a post-treatment outcome that are correlated with the treatment. Thus, specifications with second-year gains as the dependent variable controlling for Y1 scores will not provide consistent estimates of _δ_ for the second year of the program. We show these results for illustrative reasons, and to help understand the mechanism for the differing results on 1-year and 2-year impacts of the program.

Columns 1 and 4 of Table 9 show that students in schools that received the block grant scored 0.09 standard deviations (SD) higher than those in comparison schools at the end of the first year of the program for mathematics, and 0.08 SD higher for Telugu. Test scores were 0.04 SD and 0.07 SD higher for mathematics and Telugu at the end of the second year (Table 9 – columns 3 and 6). The difference at the end of year one is significant for each subject, but not so at the end of two years. The addition of school and household controls does not significantly change the estimated value of _δ_ , again confirming the validity of the randomization (tables available on request).

It is striking that after two years of block grants, there is _no_ significant effect on test scores, despite the gains after the first year. The size of gains after two years (with point estimates below the point estimates after Y1) suggest that the second year of block grants did not add much to learning outcomes, while depreciation of earlier gains may explain that average gains (in terms of point estimates) after Y2 are smaller than achieved after Y1, and not significant.<sup>37</sup> These findings are entirely consistent with the predictions of the model, and confirm the considerable substitution in terms of household spending on education in response to the program when anticipated.

Finally, we tested for heterogeneity of the block grant (BG) program effect across student and school characteristics by adding a set of characteristics and their interaction with the BG variable in (15). The main result is the _lack_ of heterogeneous treatment effects by several household and child-level characteristics.<sup>38</sup> For example, if we expect poor households to be

> 37 Columns (2) and (5) of Table 9 shows the results of estimating equation (15) with the _second-year gains_ on the left hand side.   Recall that this estimate is biased as discussed above, but it suggests that the effect of the block grant program in the second year alone was close to zero in mathematics and 0.05SD in Telugu (both of which are not significantly different from zero).

> 38 We tested the interaction of the program with school size, proximity to urban centers, school infrastructure, household affluence, parental literacy, caste, sex and baseline test score.

23
more credit constrained and to be within their desired ̳optimal‘ amount of spending on education, then we would expect that they would offset less of the value of the grant and that the grant would have a larger impact on learning outcomes of poorer households. This suggests that even poor households were spending enough on education so as to almost completely substitute away the value of the school grant from their own spending.

# **5 Robustness and Interpretation**

We find strong suggestive evidence from two different low-income countries for a model in which households respond to anticipated school funding. The crowding out of private spending is sufficiently substantial to lead to no impact on test scores from anticipated school grants, while unanticipated changes positively impact the growth of test scores of children. In this section, we discuss the robustness of our results and its interpretation.

# **5.1 What Are the Components of Spending?**

One possible concern regarding our interpretation of the results is that the lack of responsiveness by households to unanticipated funds arises because schools spend these funds on different inputs with different parameters of (technical) substitution between household and school funding. Simply put, it is possible that all the unanticipated grants were spent on hiring teachers (who households cannot substitute for) and all the anticipated grants on textbooks (which they can). We compare patterns of spending across various spending categories, and show that this is not the case. In AP, the pattern of spending across various categories is almost identical between the first and second years of the project (Table 7), and it seems clear that the funds were spent on the same type of inputs both when they were unanticipated (first year) and anticipated (second year). In Zambia, we cannot attribute school spending to specific sources of funding (discretionary vs. rule-based). However, the total shares spent on those items most suitable for substitution (books, chalks, and stationary) add up to 57% and 47% respectively for schools without and with discretionary funding, suggesting that in both cases, substantial and similar spending occurs on items that could be substituted by households.

This also helps rule out explanations based on diminishing returns to the items procured or the durable nature of school materials. In both countries, the majority of the grant is spent on material that is used up during the school year (stationery, notebooks, practice books, etc). In the AP experiment, it is possible that some of the classroom materials purchased may be durable, 24
and the results reflect diminishing returns to durables in the second year. However, we see that the exact same fraction of the grant was spent on classroom materials in both years, suggesting that even these materials needed to be replenished. We also explicitly record spending on durables (bags, uniforms, plates, etc.) and find that these accounted for less than 10% of spending in the first year, and under 1% in the second year.

# **5.2 Are the Unanticipated Grants “True” Surprises?**

A further possible concern with our approach may be that the distinction between anticipated and unanticipated funding is artificial, and households can similarly anticipate both sources after all. For the AP program, this is hard to sustain: as mentioned earlier, the schools had no reason whatsoever to expect the program in the first year, while the grant was eagerly anticipated by schools in the second year. Also, as suggested earlier, most household spending on education occurs at the start of the school year when the school typically provides parents with a list of items to procure for their child for the school year. In the first year of the experiment, the announcement of the grant program was made around one and a half months into the school year and materials were typically procured a few weeks after that. Thus, it is highly likely that materials bought with the grant supplemented the initial household spending and that the firstyear program effect represents the "production function" effect of additional spending on school materials. In the second year of the program, field reports suggest that in many cases, treatment schools reduced the list of what they expected parents to buy expecting to use the grant to buy some of these items. Thus, the difference in the degree of anticipation of funds in the first and second year is quite clear.

Similarly, in Zambia the uncertainty related to the cash-budget meant that actual spending and budgets were far apart. The typical arrival of these funds at varying points during the school year suggest that households were unlikely to be able to respond to these (as suggested by the positive test score gains in these schools in Table 4A and 4B, and the findings in Table 2). In any case, we see clearly in Table 3 that households do respond substantially to variations in the rule-based grants and that they spend much more/less in schools with lower/higher per-student rule-based funding.

# **5.3 Budgetary Offsets**

A third possibility is that there are correlations between the two different types of funds that may be confounding our results. In Zambia, we find a _positive_ but insignificant relationship 25
between rule-based funding and discretionary funding [p-value=0.22]. In AP, the concern would be if anticipated funds are offset by a reduction of other transfers to the program schools. We measure the total grants received by the schools from all other sources and we verify that there is no difference in year to year receipts of funds in either treatment or control schools. There is also no significant difference between the amounts received in treatment and control schools in any year, or a significant difference between any of these differences across the years (tables available on request).

# **5.4 Storage and Smoothing**

In interpreting our results, a question that arises is whether households or schools could have smoothed the unexpected grant by either saving some of the funds or storing some materials for use in later years (if the materials had already been bought). We argue that this does not seem to have taken place because the households don‘t appear to reduce their expenditure in response to the unanticipated grant in either AP or in Zambia. On the school side, the program design in AP did not provide schools the option of saving funds. They could have saved materials, but they spend on the same sets of materials in both years suggesting that storage was limited, and that the grant led to a near one for one increase in learning materials in the first year. In Zambia, the cash budget system in government spending would have given little scope for smoothing spending, though some of the funds did get used for durable infrastructure.

But even if some smoothing via savings, storage or durable goods spending by the school may have been possible, the coefficient on the unexpected grant is a _lower bound_ on the production function parameter (because in this case, the full value of the grant will not have been spent in the same time period) and our results show that the production function effect of the school grant is positive – which would not have been apparent if the relationship between school grants and test scores were to have been estimated using anticipated grants.

# **5.5 What Did Households Substitute Spending Toward?**

One striking implication of our results is that while it was possible in production function terms to obtain a significant increase in student test scores in both contexts for a relatively inexpensive intervention (spending $3/student to raise test scores by 0.1 SD compares very favorably with the cost effectiveness of other education interventions in developing countries), parents chose to not make that investment in the next period when they could have continued making it (in AP), and seem to offset rule-based grants completely in Zambia. While this result 26
suggests that the households had a low income elasticity of test scores, it is impossible to make any further conclusions without further information on what the household spent the extra cash on. Specifically, given the declining marginal benefit of spending on education in each particular period, the household may still have found it better to save this money for spending on future educational materials, rather than just spend it on goods and services this year. As we do not have information on what the households did with the extra cash available, we cannot explore this further.<sup>39</sup> We summarize this section by noting that while having further disaggregated data on both school and household expenditures would allow for an even more precise understanding of the mechanism for our test score results, the combination of the household spending results and the test score results are most parsimoniously explained by the theoretical framework laid out in this paper. We consider but end up rejecting several alternative explanations for these results and finding the same results in two contexts as varied as Zambia and India makes us confident that our results present evidence of differential household responses to anticipated and unanticipated school grants.

# **6 Conclusion**

Data on test-scores and household expenditures in the context of school grant programs in Zambia and Andhra Pradesh in India suggest that grants anticipated by households crowd-out private educational spending. Consequently, school grants that are fully anticipated have no impact on test-scores. Unanticipated grants elicit no household responses and do have positive impacts on learning.

These results have implications for common estimation techniques in the educational literature. The dominant technique for estimating the effect of school inputs on test scores is based on the production function approach, where achievement (or changes in achievement) is regressed on school inputs. Following Todd and Wolpin (2003), these estimates represent the policy effect of school inputs that combines both the effect of inputs on test scores through the production function, as well as household responses to such inputs. Our use of unanticipated

> 39 What we _do_ know from our evidence is that households did not spend it on other (non-education) inputs that may raise child test scores directly, such as child nutrition; if so, we could have had substantial crowding out, but still a positive impact on test-scores, which is rejected by our evidence.

27
inputs allows the estimation of both effects separately, thus shedding more light on the process through which school inputs may or may not affect educational attainments.

This distinction between anticipated and unanticipated inputs could account for the wide variation in estimated coefficients of school inputs on test scores (Glewwe 2002, Hanushek 2003, or Kreuger 2003). The production function framework does not separate anticipated from unanticipated inputs and so the regressor is a combination of these two different variables. The estimated coefficient is bounded below by the policy effect and above by the production function parameter; the distance from either bound depends on the extent to which the schooling inputs were anticipated or not. While experimental evaluations of education interventions typically overcome selection and omitted variable concerns, the distinction highlighted in this paper is relevant even for experiments, since the interpretation of experimental coefficients depends on the time horizon of the evaluation and whether this was long enough for other agents (especially households) to re-optimize their own inputs.

Although we find evidence of high crowding out of anticipated inputs, our results do not suggest an educational policy where inputs are provided unexpectedly. Although test scores in the current period increase with unanticipated inputs, the additional consumption will push households off the optimal path. In subsequent periods, therefore, they will readjust expenditures until the first-order conditions are valid again – unanticipated inputs in the current period will not have persistent effects in the future (except due to the durable nature of the good). The policy framework that is suggested under this approach involves a deeper understanding of the relationship between public and private spending, acknowledging that this may vary across different components of public spending. Our key policy implication is that schooling inputs that are less likely to be substituted away by households are better candidates for government provision.<sup>40</sup> What might such inputs be? One important example may be teaching inputs, whereby the combination of economies of scale in production (relative to private tuition), difficulty of substituting for teacher time by poorly educated parents, or the generic non-availability of trained personnel in every village could make public provision more efficient (see Andrabi et al., 2009).

> 40 An alternative could be to give very large grants to school. The anticipated grant in both countries was relatively small. For example, in AP only 12% of households were spending less than the per pupil school grant. If a grant larger than household spending had been given, then crowding out of household spending would have been bounded, and the additional school grant may have had a positive impact on test-scores as total spending by schools and households would have been increased.

28
In a parallel experiment on the provision of an extra teacher to randomly-selected schools in Andhra Pradesh, Muralidharan and Sundararaman (2010) find that the impact of the extra teacher was identical in both the first and second year of the project – suggesting that teacher inputs were less likely to be substituted away. Similarly, inputs like school infrastructure that retain some aspects of public-goods and would thus be under-provided by non-coordinating households are a good candidate for government provision.

The approach followed here of treating test scores as a household maximization problem with the production function acting as a constraint explicitly recognizes the centrality of households in the domain of child learning. This has important implications for both estimation and policy and further research could potentially separate inputs with high/low degrees of substitutability with regard to private expenditures. One hurdle for such studies is the lack of matched school and household data and the identification of "surprises" in the provision of inputs; long-term data on schooling inputs and panel data on student learning would allow for a deeper understanding in varied contexts based on deviations from means, as is standard in the consumption literature (following Hall 1978). Investments in such data collection will provide the necessary infrastructure for evaluation of short, medium, and long-run impacts of education policy innovations and should be a high priority for education policy makers and funders of education research.

29