metadata: authors: 0: author: Buttenheim, Alison
1: author: Alderman, Harold
2: author: Friedman, Jed
count: Lao People's Democratic Republic
docna: 0: docna: Impact evaluation of school feeding programs
            in Lao PDR
docty: Policy Research Working Paper
subtopic: Health Care Services Industry,Health Service Management and Delivery,Nutrition,Educational Sciences,Food Security
teratopic: Industry,Governance,Health, Nutrition and Population,Education
abstracts: cdata!: Despite the popularity and widespread
            implementation of school feeding programs, evidence on the
            impact of school feeding on school participation and
            nutritional status is mixed. This study evaluates school
            feeding programs in three northern districts of the Lao
            People's Democratic Republic (Lao PDR). Feeding
            modalities included on-site feeding, take-home rations, and
            a combination. District-level implementation of the
            intervention sites and selective take-up present
            considerable evaluation challenges. To address these
            limitations, the authors use difference-in-difference
            estimators with propensity-score weighting to construct two
            plausible counterfactuals. They find minimal evidence that
            the school feeding schemes increased enrollment or improved
            childrens nutritional status. Several robustness checks and
            possible explanations for 0 findings are presented.
geo_reg_key: 80475,80451,80471
repnme: repnme: Impact evaluation of school feeding
            programs in Lao PDR
display_title: Impact evaluation of school feeding
            programs in Lao PDR
listing_relative_url: /research/2011/01/13310349/impact-evaluation-school-feeding-programs-lao-pdr
topicv3: Food Security m1326439 199,Nutrition m1326779 644,Education m1327048 470,School Feeding m1504302 3655,Health m1504661 3480
publishtoextweb_dt: 2011-01-01T00:00:00Z
historic_topic: Industry,Governance,Health, Nutrition and Population,Education
no_of_pages: 59
model_extractions: WPS5518 Policy Research Working Paper 5518 Impact Evaluation of School Feeding Programs in Lao PDR _Alison Buttenheim Harold Alderman Jed Friedman_ The World Bank Development Research Group Human Development and Public Services Team January 2011
Policy Research Working Paper 5518

# **Abstract**

Despite the popularity and widespread implementation of school feeding programs, evidence on the impact of school feeding on school participation and nutritional status is mixed. This study evaluates school feeding programs in three northern districts of the Lao People’s Democratic Republic (Lao PDR). Feeding modalities included on-site feeding, take-home rations, and a combination. District-level implementation of the intervention sites and selective take-up present considerable evaluation challenges. To address these limitations, the authors use difference-in-difference estimators with propensity-score weighting to construct two plausible counterfactuals. They find minimal evidence that the school feeding schemes increased enrollment or improved children’s nutritional status. Several robustness checks and possible explanations for null findings are presented.

This paper is a product of the Human Development and Public Services Team, Development Research Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at abutt@wharton.upenn.edu, Halderman@worldbank.org, jfriedman@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
Impact evaluation of school feeding programs in Lao PDR<sup>*</sup> Alison Buttenheim University of Pennsylvania Harold Alderman The World Bank Jed Friedman The World Bank

* The authors thank the World Food Programme and the World Bank Research Committee for financial support. We acknowledge the Government of Lao PDR, and the Ministry of Education for their support of this research. particularly thank Ms. Yangxia Lee of the Ministry of Education . The Department of Statistics provided crucial fieldwork and data management services, under the expert guidance of Dr. Samayachan Boupha,  Mr. Thipsavanh Intharack, and Mr. Khamphanh Chaleunphonh . The World Food Programme Country Office in Vientiane provided support and technical expertise throughout the study. We are also indebted to our Survey Director, Robert McLaughlin. All findings, interpretation, and conclusions expressed here are those of the authors, and do not represent the view of the World Bank, its Executive Directors, or the countries they represent, nor do they reflect the view of the World Food Programme. Contact: Alison Buttenheim, Robert Wood Johnson Foundation Health & Society Scholars Program, University of Pennsylvania, 3641 Locust Walk, Philadelphia PA 19104. Phone: (215) 898-5456. Email: abutt@wharton.upenn.edu.
# 1 Introduction

Millennium Development Goal 2 calls for increased primary school enrollment and reductions in the gender gap in school enrollment. Many policy solutions have been proposed to achieve this goal, from teacher quality initiatives to transfer programs. School feeding programs, a form of conditional transfer, have been an important and prominent part of this policy portfolio in recent decades, and have been implemented widely in developing countries and for low-income populations in developed countries. In exchange for regular attendance, children are provided a meal or snack during the school day and/or food rations to take home. This transfer is hypothesized to shift parental preferences for children’s educational participation by reducing the costs and increasing the benefits of time spent at school. If successful, the transfers associated with school feeding can improve educational participation (enrollment, attendance, and age at school entry), achievement and cognition (test scores, grade progression), and nutritional status (height and weight-for-age and micronutrient status).

Despite the attention and resources devoted to school feedings programs, little rigorous evidence exists to support these investments. Prior research on school feeding has been hindered by school-based rather than household-based samples, cross-sectional data, and non-randomized designs (Adelman, Gilligan & Lehrer, 2008). In 2005, the World Food Programme initiated a three-country study of school feeding programs jointly with the World Bank that was designed to address these shortcomings. Impact evaluations were launched in Burkina Faso, Uganda, and Lao PDR. This paper reports the results from the study in Lao PDR, which took place in two northern provinces of the country from 2006-2008. The intervention region is notable for its ethnic diversity, low school participation, poor household food security, and lack of 2
transportation infrastructure. The study included a baseline survey in 2006, followed by the rollout of the school feeding interventions, and a follow-up survey in 2008.

The present study contributes to the literature on school feeding through a quasiexperimental longitudinal comparison of different school feeding modalities (on-site feeding vs. take-home rations). Our household-based sample avoids the problem of looking only at children currently in school. The study also contributes to the impact evaluation literature by explicitly addressing evaluation challenges through the construction of two counterfactuals, using difference-in-difference estimation with propensity-score weighting.

To preview our results, we find very little conclusive evidence that school feeding affected enrollment or nutritional status in this population. The evaluation presents several methodological challenges which are discussed in detail below. In an attempt to confirm our null findings, we undertake several additional analyses related to sample attrition, spillover effects, and program take-up and implementation. The study raises some concerns about the potential for school feeding programs where school capacity is limited.

The paper proceeds as follows: we first discuss the theory and prior evidence of school feeding programs, then introduce our evaluation strategy. Details on the Lao PDR setting and the school feeding intervention are provided, along with a description of the sample and measures. Results and discussion follow.

# 2 Food‐for‐education and child development: Theory and evidence

Providing food to school children, either during the school day in the form of a snack or in the form of rations to take home, has several goals. First, the transfer is intended to decrease the net cost of schooling and thereby shift parental demand for children’s educational participation, 3
leading to improvements in enrollment, attendance, and age at school entry. A second goal is to alleviate short-term hunger during the school day to improve children’s concentration and cognitive functioning, leading to better learning and higher achievement. A third goal is to improve children’s long-run macro- and micronutritional status through the provision of additional calories and fortified foods, reducing malnutrition and its attendant negative impact on future health and productivity (Adelman, Gilligan, et al., 2008).

Previous empirical work has found mixed evidence for the impact of school feeding (for comprehensive reviews, see Kristjansson, Petticrew, MacDonald, et al., 2007; Adelman, Gilligan, et al., 2008; Bundy, Burbano, Grosh, et al., 2009). Adelman, Gilligan, et al., (2008) point out that relatively few of the studies in the literature measuring enrollment impacts use a randomized design, perhps reflecting the popularity of the intervetnion and the political obstacles to randomization. Thus, their review covers a wider range of approaches to identifying causal impacts including studies that use quasi-experimental methods such as natural or administrative experiments that identify impacts by exploiting a quasi-random component of program eligibility. Studies that investigate impacts on nutrient consumption are more likely to have a randomized design, but fewer studies that look at anthropometric outcomes employ this approach and virtually no randomized studies look at outcomes of younger family members.

Results are most compelling for school enrollment and attendance, particularly where initial rates of participation are low (E. Jacoby, Cueto & Pollitt, 1996; Ahmed, 2004) . For example, Ahmed and Del Ninno (2002) provide evidence of an 8% increase in enrollment and a 12% increase in attendance in a program targeted to poor households. The effect of school feeding programs on age at first schooling is also of interest given prior work on the importance of timely school entry for future school and labor market success. However, the effect of school 4
feeding on school entry age has not been demonstrated empirically and is identified in reviews as an important evidence gap (Adelman, Gilligan, et al., 2008).

Evidence of impact of school feeding on learning achievement and cognitive function is also hard to find. Studies have shown significant impact in one but not multiple domains, e.g., increased math but not language scores or vice versa (Tan, Lane & Lassibille, 1999; Ahmed, 2004). Similar to school participation findings, the impact of school feeding on achievement and cognition appears to depend on initial nutrition status, with undernourished children realizing the biggest gains (Simeon & Grantham-McGregor, 1989; Whaley, Sigman, Neumann, et al., 2003). The mechanisms linking school feeding to achievement are more complex than the school feeding-participation relationship, and are therefore less straightforward to evaluate. School feeding is hypothesized to improve educational achievement through at least three routes: First, school feeding should lead to more time in school, providing more opportunities for learning. Unintended consequences must be factored in, however. For example, if enrollment increases in response to school feeding programs but no additional teachers are hired, classroom crowding may impede effective teaching. For on-site feeding in particular, the provision of a meal during the school day may take away from teaching time (Grantham-McGregor, Chang & Walker, 1998).

A second mechanism is improved cognitive functioning and attention span associated with the alleviation of hunger during the school day. A third mechanism is improved long-term health associated with better nutrition and resistance to infection, which in turn reduces illnessrelated absences and thereby improves performance. The second and third mechanisms are wellgrounded empirically. Previous work has confirmed that children suffering from micronutrient deficiencies do poorly in school. Anemia in particular is a widespread problem with clear health 5
and educational attainment implications (Beard & Connor, 2003; WHO/CDC, 2004): Providing iron supplementation to children to reduce anemia has been shown to improve cognitive development (McCann & Ames, 2007) and increase school participation (Bobonis, Miguel & Puri-Sharma, 2006). It has also been demonstrated that a healthy breakfast improves same-day and long-term performance in undernourished populations; and that breakfast provided at school improves attendance and reduces tardiness.

While micronutrient deficiencies and short-term hunger are fairly easy to address through school feeding, supporting children’s long-term growth trajectory may not be. Existing evidence suggests that where there is an effect of school feeding on child growth, it is most likely modest (Powell, Walker, Chang, et al., 1998; van Stuijvenberg, Kvalsvig, Faber, et al., 1999; Grillenberger, Neumann, Murphy, et al., 2003). Anthropometric studies have confirmed that school-age children may be too old to experience catch-up growth or recover from growth faltering (Martorell, Khan & Schroeder, 1994; Martorell, 1995; Behrman, Alderman & Hoddinott, 2004; World Bank, 2006). However, where school entry is delayed and grade repetition is high, nutritionally-vulnerable older children may benefit from exposure to school feeding during pre-pubertal growth spurts.

How and whether food transfers reach the intended beneficiaries has important implications for effectiveness. School feeding formats can include on-site feeding (OSF), takehome rations (THR), or both. The impact of the alternative formats could differ, even if the value of the transfer is comparable. OSF and THR differ in the timing of receipt of the food transfer (daily vs. periodic), conditionality (daily attendance vs. average attendance), and content (OSF may include dairy productions, for example). THR do not provide short-term nutritional benefits that can boost concentration and learning effectiveness.

6
THR also limits the ability to target the transfer to the intended beneficiary, as households may allocate the food rations to any household member. Jacoby (2002) introduced the notion of the intrahousehold flypaper effect to assess how much of the food benefit actually ‘sticks” to the intended recipients. Evidence from Bangladesh and the Philippines suggest that the transfers are quite sticky (H. G. Jacoby, 2002; Ahmed, 2004); if they are not, then the nutritional impact of school feeding hinges on the degree to which rations provided to the child’s household are consumed by younger children.

The lack of targeting within schools also compromises efficiency. While school feeding programs are often placed purposively in particularly vulnerable regions or schools, they are rarely targeted to vulnerable children within schools. This means that well-nourished children who would attend school in the absence of the school feeding program still receive food. The cost of school feeding programs (ranging from $10-$60 per student per year, with the lower costs generally pertaining to snacks) might therefore be better spent on other potentially more sustainable and higher-impact interventions such as deworming, uniform and book purchasing, or teacher training (Tan, et al., 1999; Miguel & Kremer, 2004; Evans, Kremer & Ngatia, 2008).

This discussion suggests many outstanding evaluation questions related to school feeding. Many of the studies reviewed above have methodological flaws that limit causal inference and external validity. We address some of these shortcomings here through the use of a household-based sample that includes non-enrolled children; a longitudinal quasi-experimental design comparing multiple modalities, and the use of propensity-score weighting techniques to construct plausible counterfactuals.

7
# 3 Evaluation strategy

The goal of the study was to assess the impact of alternative school feeding formats on children’s health and educational outcomes. Ideally, the school feeding formats would have been randomly assigned to schools, so that any identified differences in outcomes could be attributed to the school feeding programs (Heckman & Smith, 1995). The design of the present study, however, offers several challenges to the identification of a suitable counterfactual. First, political and logistical circumstances dictated that school feeding formats (on-site feeding, takehome rations, or both) could be randomized only at the district level. Second, the control district (receiving no intervention) had to be selected from a neighboring province, as all other districts in the intervention province were already participating in school feeding. Key socioeconomic and demographic characteristics used to select the control district are shown in Table 1; the control district appears fairly well-matched. However, tests for equality of means of key baseline child, household, school and village characteristics across the four districts (Table 2) reveal significant differences across the districts, particularly in ethnic composition and geography, characteristics which we might expect to be associated with health and nutrition outcomes of children.

A third evaluative challenge is selective take up of the intervention. Within the three intervention districts, villages chose whether to participate in the offered school feeding program. Comparisons of the health and education outcomes in “takeup” vs. “non-takeup” villages across the three intervention districts suggest that takeup villages had better outcomes at baseline for most outcomes of interest (Figure 1-3). Preliminary analyses also suggested that the villagelevel predictors of take-up varied substantially across districts (see Table 3). For example, while the proportion of children enrolled in school is positively correlated with take-up when the three 8
intervention districts are pooled (Table 3, first column), the association is clearly strongest in Phongsaly district compared to Khua or Nhot districts.<sup>1</sup> Given these challenges, and the availability of pre- and post-intervention data, we used two distinct difference-in-difference (DiD) estimators to construct two plausible counterfactuals. First, the within-district DiD compares the changes in outcomes from baseline to follow-up in takeup villages vs. non-takeup villages in the same district: where Y0 and Y1 denote outcomes at baseline and follow-up respectively, and T and C denote within-district treatment (takeup) and control (non-takeup). At the child level, this is an intent-totreat analysis (ITT), as we do not account for whether individual children received transfers of food at school or via rations, but only whether they lived in a take-up village or not. Formally, our DiD specification is where Y _ivt_ is the outcome for child _i_ in village _v_ at time _t_ , Takeup is the treatment indicator, and Round2 is the follow-up survey, X represents household characteristics and V stands for the set of dummy variables for each village. εit is the error term which is composed of individual and family unobserved fixed characteristics as well as a stochastic disturbance term,  it:

> 1 These multivariate results are consistent with the bivariate associations shown in Figure 1. Odds ratios for takeup regressed on current enrollment in 2006 are 4.32 for Phongsaly (p <.001); 1.29 for Khua (p = 0.456); and 2.22 for Nhot Ou (p = .001).

9
The village fixed effects account for any time invariant community level unobservables including any fixed factors associated with schools. The interaction term of Takeup*Round2 reflects the difference-in-difference and is the coefficient of interest for the impact evaluation. We estimate (2) separately for each of the three intervention districts.

However, due to the selective take-up issue mentioned above, these estimates are possibly biased. To adjust for the bias, we use a propensity-score matching technique to trim and weight the observations. We first model the decision to take up the program as a function of baseline child, household, school and village characteristics. These takeup models allow us to calculate within-district propensity scores, or the probability that a village would participate in the school feeding program. Based on exploratory analyses of take-up, a separate propensity score model was estimated for each of three intervention districts using the same set of baseline covariates (Table 3). Village-level propensity scores were assigned to all children in the sample. We then use the propensity scores to weight observations in the DiD analysis, such that take-up villages are given a weight of 1, and non-take up villages are given a weight of (p/1-p), where p is the propensity score (Hirano, Imbens & Ridder, 2003; Chen, Mu & Ravallion, 2009). We also trim the top and bottom five percentiles of propensity score values. The weighting and trimming serve to balance the observations between take-up and non-takeup villages along observable dimensions.<sup>2</sup> This subsequent matched DiD analysis would yield estimates of the causal impact of school feeding if the matching adequately captures the village-level determinative factors of the take-up decision.

However, we must interpret the results of this within-district propensity-matched if there are unobservable factors which are also likely to affect take-up and affords an alternative

> 2 The findings are virtually unchanged if we instead trim the top and bottom percentile.

10
estimate of program impact. The second DiD specification therefore exploits the existence of a control district in our sample. In this specification, we include only takeup villages from the intervention districts, and all of the villages in the control district (Ngoi). Propensity-score weights are recalculated for these district-specific samples, and samples are again trimmed. This specification allows us to compare the intervention villages to the villages in the control district that are most similar but were not eligible for the school feeding program.

While this evaluation approach has obvious drawbacks relative to a “gold-standard” randomized design with complete take-up, it also offers advantages over other recent school feeding evaluations. First, the sample of children is based on a household rather than a school sampling frame, meaning that children who are not enrolled in school are included in the analysis. Second, we include children from age 3 to age 14, capturing potential spillover effects for older and younger siblings. Third, the longitudinal design allows for the DiD analysis. Finally, the design includes three treatment arms (OSF only, THR only, and OSF plus THR) and also includes a control district. In addition to the core analyses, several robustness checks and supplemental analyses were also undertaken, the results of which are described below.

# 4 Setting

## **_4.1 School feeding in Lao PDR_**

This study is set in four districts in northern Lao People’s Democratic Republic (Lao PDR). Lao PDR (population 5.6 million) is classified as a “Least Developed Country” by the UN and has a headcount poverty ratio of 33%. Three-quarters of the population is rural, with average household size of six (World Food Programme, 2007). Half of all children are stunted and one11
third are underweight (Unicef, 2008). Primary school net enrollment rate is 84%. Many villages, particularly in remote, mountainous areas, have no schools or have schools with only one or two primary grades. Parents who want their children to continue schooling must send the children to a neighboring village. If the travel distance is too far to allow for a daily commute, children may board at their school. Boarding usually entails living during the school week in simple dormitories constructed by parents. These “informal boarders” are responsible for preparing their own meals and are considered nutritionally vulnerable.

In this context, the World Food Programme has been operating school feeding programs in the Lao PDR since 2002. The program initially targeted 12 districts in three northern provinces. Much of the population is these districts lives in remote mountainous areas with limited access to roads. Enrollment rates, particularly for girls, are low in these areas and household food insecurity is prevalent. The WFP program originally provided a daily snack made from corn-soya blend at school, and additional take-home rations of canned fish and rice for girls and for informal boarders to encourage their enrollment and continued attendance. Takehome rations are meant to be conditional on attending school at least 80% of the time. To participate in the school feeding program, villages were required to convene a school feeding committee, build food storage facilities, provide labor for food preparation, and, in some cases, travel to WFP food distribution points to pick up food allocations.

In 2006, WFP expanded the school feeding program to the remaining 7 districts in the targeted provinces. This roll-out provided the opportunity to undertake a longitudinal, evaluation of school feeding impact, and to compare the different school feeding modalities. Three of the new districts in Phongsaly province were selected as intervention sites, and a neighboring district in Luang Prabang Province was selected as a control site. Due to concerns about possible 12
spillover effects and perceived equity, the World Food Programme and the Lao Ministry of Education determined that implementation of the interventions should be done at the district level and type-of-program assignation given randomly across the three districts in Phongsaly province. Interventions were assigned as follows: Phongsaly District (Phongsaly Province): On-site feeding Khua District (Phongsaly Province): On-site feeding and take-home rations Nhot Ou District (Phongsaly Province): Take-home rations Ngoi (Luang Prabang District): Control district The design of the take-home rations in the evaluation study differed from that used in other districts in northern Lao PDR. Following WFP guidelines, take-home rations in other districts were targeted at girl students and at informal boarders. For the impact evaluation study, it was determined that take-home rations would be provided to both girls and boys, with a separate additional ration also provided to informal boarders.

# **_4.2 Sample_**

Data for the study come from a longitudinal survey of approximately 4,500 households with school-aged children in rural villages in the four sampled districts of northern Lao PDR. The region is ethnically very diverse, with over 50 distinct ethnic groups. At standard levels of significance (.05) and power (.8), the study size is sufficient to detect a difference in enrollment of 5 percentage points post-intervention based on baseline attendance data, as well as sufficient enough to detect a change in anthropometric z-scores of approximately 0.3 standard deviations. Eligible households (those with at least one school-aged child, defined as children aged 6-10) 13
were randomly selected using a multiple stage probability sampling scheme. In the first stage, 75 primary sampling units were randomly selected from each district with probability proportional to the population in each village (as listed by the 2005 census). For the most part, primary sampling units were villages. Some large villages comprised two or more PSUs. At the second stage, enumerators and the village head drew up a complete household listing and identified eligible households based on the village head’s knowledge of child ages. Fifteen eligible households were randomly selected from each PSU. In cases where the total number of eligible households was fewer than fifteen, all eligible households were sampled. From a target of 4,500 households, successful interviews were conducted with 4,169 households in 263 villages, a 93% response rate.

After the baseline survey, eligible villages were informed of the rollout of the school feeding program and invited to participate. Villages that wanted to participate had to meet minimum participation requirements as described above. Participation in sampled villages with existing schools in 2006 was 35/58 (60%) in Phongsaly, 47/63 in Khua (75%), and 34/59 (58%) in Nhot Ou. There was also considerable variation in the start date of the school feeding program, with some villages beginning in Fall 2006 and others beginning more than a year later. Therefore, children’s exposure to the school feeding program is likely correlated with other village characteristics that are associated with education and health outcomes of interest. We address some of these analytic challenges below.

In Fall 2008, a follow-up survey was fielded. Where possible, the sampled villages and households from 2006 were located and re-interviewed. A total of 11 villages had moved from their 2006 location (due to the government’s village relocation policies), had merged with another village, or could otherwise not be re-interviewed, leading to 119 households lost to 14
follow up. Within recontacted villages, an additional 412 households had left the village or could not otherwise be located and re-interviewed. When possible, replacement households were randomly drawn from a listing of eligible households in the villages with children aged 6-10 who had not been sampled in 2006. A total of 286 replacement households were added to the sample in 2008. Recontact rates therefore are 96% at the village level and 87% at the household level.

An extensive household questionnaire was used to collect information on household composition, assets, livestock, agricultural, shocks, food security, diet diversity, and social capital. The household questionnaire also included detailed education histories and daily activities for children age 6-14 (6-16 in 2008), and diet diversity, anthropometry, and hemoglobin assessments for children age 3-10 (3-12 in 2008). To the extent possible, the household survey modules were adapted either from the 2002-03 Lao Expenditure and Consumption Survey (LECS3, a Living Standards Measurement Survey) or from survey modules used in similar school feeding program evaluations. Food security and diet diversity questions were drawn from the FANTA Diet Diversity and Household Food Security scales (Hoddinott & Yohannes, 2002; Swindale & Bilinsky, 2005). In 2008, modules were added on child health and morbidity, parental perceptions of the school feeding programs.

In both 2006 and 2008, the initial English versions of the questionnaires were translated into Lao by the Lao National Statistics Centre staff and pretested. Revisions were made to the questionnaires based on pretesting. After final revisions of the English and Lao versions, a blind back-translation from Lao to English was completed and checked against the English version for consistency. Enumerators were recruited and trained by the National Statistics Centre and provincial and district officials, and included men and women fluent in at least one local language in addition to Lao. Two nutritionists with extensive training and fieldwork experience 15
were recruited to assist with anthropometry and hemoglobin testing. The HemoCue Hb201+ photometer was used for hemoglobin testing. Locally-made height-boards and locally-sourced digital scales were used for anthropometry. During fieldwork, the nutritional assessment was conducted at a central location in each sampled village on the final day of interviewing. For complete details on the survey and fieldwork, see Buttenheim & McLaughlin (2006).

For analyses presented here, we restrict the sample in each year to children ages 3-14 living in villages with schools<sup>3</sup> . The analytic sample includes 10,748 children in 2006 and 9,810 children in 2008. Impact of school feeding is assessed along multiple dimensions: School participation is captured by a dummy variable for currently enrolled in school. Nutritional status is measured by height-for-age and weight-for-age standardized z-scores, calculated from measured height and weight and reported age using the WHO Child Growth Standards (WHO Multicentre Growth Reference Study Group, 2006); and by an indicator for whether the child is anemic, based on altitude-adjusted measured hemoglobin. (Nestel, 2002). The cutoff for anemia was 11.0 g/dL for children under 5 years, and 11.5 g/dL for children 5-10 years old.

# 5 Results

## **_5.1 Current enrollment_**

Current enrollment for children 6-14 by district, survey year and school feeding takeup is shown in Figure 1, with three important results. First, enrollment at baseline is higher in takeup villages than in non-takeup villages. Second, enrollment increases in all districts from 2006-2008,

> 3 Nineteen villages were excluded based on having no school. An additional village in Khua was excluded because no hemoglobin measures were taken there. The analytic sample includes 58 villages in Phongsaly, 63 villages in Khua, 59 villages in Nhot Ou, and 64 villages in Ngoi.

16
including in the control district of Ngoi. Third, enrollment increases do not appear any larger in takeup villages compared to non-takeup villages within the same district. Estimates from the difference-in-difference models (Table 4) confirm this. The first three columns present results from within-district estimates (comparing takeup to non-takeup villages) and the next three columns compare takeup villages in intervention districts to all villages in the control district (Ngoi). The top panel shows all children, followed by age- and sex-stratified analyses. The coefficient of interest is the interaction of takeup village * 2008, the follow-up survey round. We find weak evidence for any impact of school feeding programs on children’s enrollment status. There is a marginally significant effect in take-up villages in Phongsaly district (which provided onsite feeding) of 5 percent when compared with villages in Ngoi. In Nhot Ou district, which provided take-home rations, a significant enrollment boost of 7 percent is observed. The stratified models point to younger girls (ages 6-10) driving this result in Phongsaly (with a 15% increase). In Nhot Ou, however, increases are seen for younger boys and older girls (ages 11-14). However, this finding is not supported by a comparable increase in enrollment in Khua District, which offered both on-site meals and take-home rations, and saw only a marginally significant increase in enrollment of 10% only among younger boys. In addition, school feeding takeup is associated with a significant decline in enrollment among older girls.

It is clear from the results in Table 4 that enrollment increased across the board in 2008 relative to 2006 (note the large and positive coefficients on the 2008 term in most models, particularly for younger children). This finding echoes reports by the World Food Programme in Lao PDR that enrollment has been increasing in districts with school feeding interventions (World Food Programme, 2008, 2009a). It is not clear from the present analyses, however, that these increases can be attributed to school feeding. It should also be noted that the baseline 17
survey was fielded in spring 2006 while the follow-up survey was fielded in fall 2008 at the beginning of the school year.

As an additional assessment of the impact of school feeding on enrollment of neverschooled children, we also estimated a single-difference model of current enrollment, restricting the sample only to those children who were of school age at baseline (2006) but were not currently enrolled. This model is estimated only for 2008. Results (Table 5, model 5A) show 9 percent higher enrollment in takeup villages in Phongsaly but no significant difference for Ngoi or Nhot Ou.

Another possible effect of school feeding on enrollment is that children enter school at an earlier age. School feeding programs could encourage timely school entry by changing parental perceptions about the costs and benefits of schooling for young children around the school entry age. Previous research has shown that starting school at the recommended school entry age is associated with future school success. In our sample, the most common reason offered by parents to explain non-enrollment of children was that the child was “too young” or “too small.” The availability of school meals may shift parent preferences toward sending a child perceived as too young or small to school. If so, we should observe a drop in the age at school entry. On the other hand, if older children who have never attended school also enroll for the first time, the age at school entry may go up. We investigate this in another single-difference estimate of the impact of school feeding on age at school entry (Table 5, model 5B). Four of six specifications show earlier school entry for children in take-up villages, with results most compelling for Phongsaly. . Stratified models (not shown) also show a large and significant negative coefficient (younger age at entry) in Phongsaly for younger children in particular.

18
A final assessment of the impact of school feeding on enrollment uses a continuous rather than dichotomous measure of exposure to school feeding. We first predict the number of days that onsite feeding was provided as a proportion of the total possible feeding days from 20062008, using the same set of predictors used earlier to model takeup. We then use this predicted index (range 0-1) of OSF as our exposure variable in the same set of two difference-in-difference models in the two districts that received OSF (Phongsaly and Khua). Results (Table 5, model 5C) indicate significantly _lower_ enrollment in 2008 in Khua villages with more OSF days compared to villages with fewer days. A comparable measure of intensity of take-home rations is not available in the dataset.

Taken together, the models presented for school participation do not demonstrate a robust impact of school feeding on school participation across the three intervention districts. It is clear that villages that took up the programs had higher enrollment at baseline, and that the entire region experienced a secular increase in enrollment over the two-year period, consistent with other education-related interventions such as the UNICEF-supported Child Friendly Schools program. The results suggest increased enrollment and earlier age at school entry in Phongsaly and Nhot Ou takeup villages (which offered one school feeding format) compared to Ngoi control villages. However, in Khua district, where both programs were offered, the results point to null findings. .

# **_5.2 Nutritional status_**

As discussed above, school meals can alleviate short-term hunger, boost micronutrient status, and perhaps provide longer-term nutrition to support child growth. In this study we assess nutritional status through weight-for-age, which captures both short- and long-term nutritional status; height-for-age, which primarily reflects the longer-term nutritional trajectory of children; 19
and anemia, a measure of micronutrient deficiency. Results for weight-for-age (Table 6) do not provide a consistent picture of the impact of school feeding on child weight. For all children combined, there is a significant positive impact of take-home rations in Nhot Ou (compared to Ngoi controls), on the magnitude of a 0.22 standard deviation in weight-for-age. This is the same district that appeared to have an increase in enrollment associated with the rations. Khua and Phongsaly also show marginally significant increases in weight-for-age (in different specifications). Stratified analyses complicated the picture somewhat: The largest effect size (0.84) is for boys ages 3-5 in the between-district analysis. For the same group, however, the within-district analysis for Nhot Ou shows a decline of 0.45 standard deviations. Coefficients on the year terms indicate that young boys in non takeup villages in Nhot Ou improved their nutritional status from 2006 to 2008, but boys in takeup villages did not realize the same gains in weight-for-age by 2008. In the between-district analysis, the opposite is true: Boys in Nhot Ou takeup villages made larger gains by 2008.

Height-for-age analyses are also inconclusive (Table 7). Nhot Ou once again demonstrates a significant increase of 0.29 standard deviations in between-district analyses, driven by large and significant gains for both younger boys and younger girls. The same phenomenon that was observed in the weight-for-age analysis emerges here for young boys: the within-district analysis shows a large negative coefficient, driven by large secular gain for nontakeup villages. A similar picture emerged in within-district analyses for girls ages 6-10 in Phongsaly. Results for anemia (Table 8) do not paint a clear picture either. Significant results suggesting reductions in anemia prevalence by 2008 emerge only in stratified analyses: for younger children in Nhot Ou, and for older girls in Khua and Nhot Ou 20
Other recent analyses of school feeding programs in Burkina Faso have found evidence of nutritional spillover effects for younger siblings (Kazianga, de Walque & Alderman, 2010). Spillover effects would show up in our analysis as a gain in weight-for-age or height-for-age or reductions in anemia for younger children (age 3-5). While we do observe significant increases in height-for-age for boys and girls ages 3-5 in Nhot Ou who were exposed to take-home rations in between-district analyses, this appears to be driven more by the declines in height-for-age among children in the control villages. Similarly, the marginally significant reduction in anemia for younger children in Nhot Ou takeup villages is relative to the increase in anemia prevalence in non-takeup villages in the within-district analysis. Collectively, the nutritional analyses fail to find evidence of positive effect of school feeding on children’s nutritional status.

# **_5.3 Additional analyses_**

Our two difference-in-difference specifications do not yield compelling or consistent evidence of the positive impact of school feeding in this study context. We undertake a set of additional analyses to explore possible sources of null findings.

# 5.3.1 Selective village and household attrition

Eleven villages in the 2006 sample had relocated by 2008 and could not be re-interviewed. Attrition from the village sample by 2008 is significantly associated with the number of households in the village, with larger villages less likely to later relocate (results not shown). This is consistent with the Government of Laos policy to relocate or combine smaller villages to achieve economies of scale in service provision and agricultural development (Evrard & Goudineau, 2004; Baird & Shoemaker, 2005; World Food Programme, 2005). Six of the 11 relocated villages were in Phongsaly district. Given that these villages were also less likely to 21
take up school feeding (given our estimates of takeup propensity), it is not likely that their attrition is biasing estimates of school feeding downward (in fact, the opposite is probably true). At the household level, almost 400 sampled households had relocated or could not otherwise be interviewed in 2008. Households in Khua and Nhot Ou were less likely to attrit relative to Ngoi (results not shown). In addition, households in larger villages, households with more children, and households with higher levels of per capita expenditures were less likely to attrit. Replacement households had significantly higher per capita expenditures in 2008 than panel households, but not significantly different child nutritional status or odds of current enrollment. The precise implications of this selection bias are not clear. If children with fewer resources benefitted more from school feeding, then our models may slightly underestimate program effects. Some of the relocated children may have benefitted from school feeding programs before they left, or may have moved to villages that also had school feeding programs.

# 5.3.2 Geographic  spillover effects

One possible explanation for our findings of minimal effects of school feeding is that students in non-takeup villages started to attend school in adjacent villages that did have school feeding programs. This would attenuate any observed differences between children in take-up vs. nontakeup villages, as some children in non-takeup villages might experience improvement in enrollment and nutritional status associated with the takeup in adjacent villages. One piece of evidence in support of this spillover effect would be a widening gap in the proportion of “informal” boarders (students who stay at school or travel more than an hour each way to school) in non-takeup vs. takeup villages from 2006 to 2008. We find no evidence of such a gap (results available from the authors).

22
# 5.3.3 Magnitude and intensity of food transfers in intervention districts

Effects of school feeding may depend on the magnitude or intensity of the food transfer provided. The World Food Programme reports that the OSF ration is intended to provide 100 grams of corn-soya blend and 12.5 grams of sugar each school day, with a target of 83 feedings per term. In the follow-up survey, villages reported the frequency of meals provided in the current term and the number of days that meals had been provided in previous terms (Table 9). Intensity of OSF varied in Phongsaly vs. Khua. While 34/35 (97%) of take-up schools in Phongsaly reported providing OSF meals every day in the current term, only 27/47 (57%) take up schools in Khua reported providing a meal every day. Indeed, 9 participating schools report never providing a meal this term. A second measure of OSF intensity, calculated as the number of days since Fall 2006 that the school reports providing a meal, is also higher in Phongsaly (median = 280) compared to Khua (median = 235), even though schools in Khua enrolled in the school feeding program earlier than Phongsaly schools. Overall, Phongsaly schools that participated in school feeding reported providing meals on 58% of all possible schools days since Fall 2006, compared to 49% of Khua schools. Together, these results suggest that program implementation difficulties in Khua may have compromised the effectiveness of the program.

Take-home rations for boys and girls in this intervention included 15 kilograms of rice upon enrollment in September, 30 kilograms of rice at the end of the school year in May if attendance was 80%, and one can of fish each month if attendance was 80% for the month. Informal boarder rations provided an additional four kilograms of rice, two cans of fish and one bag of salt each month if attendance was 80% for the month. Participating villages reported generally similar ration amounts, although the timing of distribution differed from the WFP schedule in many cases.

23
# 5.3.4 Conditionality of OSF and THR

In order for OSF and THR program to positively affect school participation, the transfers should be conditional on student attendance. There is some evidence that the OSF snack was provided to non-enrolled children (results not shown). In Phongsaly, 10/35 (29%) of takeup schools reported providing the snack to non-enrolled children. In Khua, the figure was 20/43 (47%). At the child level, 11% of non-enrolled school-aged children and preschoolers in Phongsaly and 19% of the same population of children in Khua were reported by a parent to have consumed a WFP school snack in the past 24 hours. Given that 7% of children in Ngoi were reported to have had a WFP snack as well (even though there is no WFP school feeding scheme there), these are likely somewhat noisy overestimates, but again may point to program implementation issues, particularly in Khua.

It is difficult with these survey data to calculate a comparable figure for take home rations. Households did report sources of food transfers, but only for months in the past year in which they also report a rice insufficiency. Among households with no enrolled children in the two THR districts, only five reported receiving WFP rations specifically through the SFP. These data cannot identify households with some enrolled and some non-enrolled children who may receive rations for each child regardless of enrollment.

# 5.3.5 Other supply side determinants of enrollment

The introduction of several other development and education programs in both the intervention and control districts during the study period presents additional challenges to making causal inferences about school feeding impact. Specifically, other programs that improved the availability or quality of schooling and therefore increased enrollment may have been 24
implemented differentially in the study districts. We estimated a model of current enrollment with village fixed effects including several time-varying measures of school quality. The most robust finding was that children in villages with schools that gained toilets between 2006 and 2008 were more likely to enroll (results not shown). There was no apparent effect of the school’s participation in the Unicef “blue box” hygiene improvement program (which provides interactive games, story cards, songs, posters and other materials with key hygiene messages), but receipt of the blue box was negatively associated with children’s WAZ and HAZ, indicating that this program is likely targeted to low-resource schools.

# 6 Discussion

The goal of this study was to provide a rigorous evaluation of the impact of school feeding on children’s human capital formation in Lao PDR. The evaluation presented several programmatic and methodological challenges, which we have attempted to address through the construction of multiple counterfactuals and robustness checks. In particular, we explicitly tackle the problems of district-level (rather than village-level) randomization of interventions; selective takeup of the intervention within district; and inconsistent implementation of the program in terms of intensity and conditionality. We find no consistent effect of school feeding on either enrollment or nutritional status. The significant effect on enrollment and age at school entry of take-home rations (in Nhot Ou) and onsite feeding (in Phongsaly) is not observed in Khua, where both OSF and THR were delivered. This makes it difficult to ascribe these improvements to a specific school feeding modality.

Previous research has suggested that school feeding programs are most effective in areas with low enrollment and household resource constraints. In our sample, larger and less remote 25
villages with higher baseline enrollment selected into the school feeding programs. We might have seen greater effects if the program had 100% takeup or had been targeted to relatively disadvantaged villages. More consistent implementation might also have produced more compelling effects.

Non-takeup villages in our sample had the opportunity to enumerate reasons for nonparticipation. The top four reasons elicited in each district among non-takeup villages are presented in Table 10. In all districts and for both feeding modalities, the most common response was that the WFP food delivery point was too far away; another frequent response was lack of access to a road. These hurdles will not be easily overcome in the field as WFP continues its school feeding program expansion in Lao PDR. Problems were also cited with the necessity to build the food storage warehouse and to recruit sufficient village volunteers to run the program. This suggests a threshold level of social capital and social efficacy that is required for villages to participate in school feeding, which may discourage the villages that have the most to gain from participating. Finding a solution to these takeup problems should be a priority for the WFP in Lao PDR and in other settings where program implementation is challenging.

The present study was one of three country studies in school feeding impact assessment undertaken by the World Food Programme and the World Bank between 2004 and 2008. The Burkina Faso study site (Kazianga, et al., 2010) finds a large and significant positive effect on child weight-for-age for younger siblings of eligible children, with benefits accruing primarily for young boys. THR and OSF provided roughly similar benefits. Results indicated that the benefits accruing to young children would have cost 9 times the value of the food transfer had it been provided as a direct transfer. The study argues that failure to account for spillover effects may lead to underestimates of benefits associated with school feeding programs.

26
The Uganda school feeding study took place in Internally Displaced Persons (IDP) camps in Northern Uganda (Adelman, Alderman, Gilligan & Konde-Lule, 2008; Adelman, Alderman, Gilligan & Lehrer, 2008; Alderman, Gilligan & Lehrer, 2008). Sampled households lived in the camps at the time of the baseline survey in 2005 but had resettled out of the camps (either returning home or moving to smaller camps closer to their original homes) by the time of the follow-up survey in 2007. The follow-up survey located and re-interviewed 81% of the baseline sample.

Nutritional effects of the Uganda school feeding program were consistent with the Burkina Faso study. Both on-site and rations programs reduced anemia prevalence among older girls (10-13). Preschool children whose siblings were exposed to the on-site feeding had significant increased in height-for-age and reduction in anemia prevalence, with no similar finding for preschoolers exposed to rations.

Recent discussions of school feeding programs globally (Bundy, et al., 2009) and locally in Lao PDR (World Food Programme, 2009b) have emphasized the need to think strategically about the role that school feeding programs should play in broader social and educational policies. The unusually high transport costs in rural Lao PDR make school feeding very expensive relative to other interventions and to school feeding programs in comparable countries. This threatens future sustainability of the program. While this evaluation has not been able to provide unqualified support for school feeding impact, it has pointed out some relevant implementation challenges that need attention. Results should be incorporated into ongoing planning efforts within country, and be added to the growing impact evaluation literature on school feeding worldwide.

27