metadata: authors: 0: author: Galasso, Emanuela
1: author: Yau, Jeffrey
count: Madagascar
docna: 0: docna: Learning through monitoring : lessons from a
            large-scale nutrition program in Madagascar
docty: Policy Research Working Paper
subtopic: Nutrition,Educational Sciences,Hydrology,Inequality
teratopic: Rural Development,Health, Nutrition and Population,Education,Poverty Reduction
abstracts: cdata!: Monitoring data are generally collected
            as a by-product of the process of monitoring program
            implementation. Yet this rich source of data have not been
            exploited to assess the effectiveness of the program. In
            this paper the authors use detailed administered data from a
            large-scale, community-based nutrition program in Madagascar
            to argue that this data can be used to estimate the
            differential effect of increased exposure to the program and
            study how these returns to exposure evolve over time. They
            find that the returns to exposure are positive: communities
            exposed for an additional one or two years display on
            average lower malnutrition rates of around 7-9 percentage
            points. And they find that the returns decrease as time and
            duration increase, although they do not dissipate to zero.
            These results are consistent with the hypothesis that the
            returns to the program reflect learning effects from the
            intervention. Finally, the results show higher differential
            returns to the program in poorer areas and areas more
            vulnerable to diseases. These findings have important
            implications for how such programs should be scaled-up
            within a country.
geo_reg_key: 80475,80458,80473
repnme: repnme: Learning through monitoring : lessons from
            a large-scale nutrition program in Madagascar
display_title: Learning through monitoring :
            lessons from a large-scale nutrition program in Madagascar
listing_relative_url: /research/2006/11/7186255/learning-through-monitoring-lessons-large-scale-nutrition-program-madagascar
topicv3: Water m1326522 137,Poverty m1326687 695,Nutrition m1326779 644,Education m1327048 470,Malnutrition m1627088 4737
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Rural Development,Health, Nutrition and Population,Education,Poverty Reduction
no_of_pages: 44
model_extractions: WPS4058

# Learning through Monitoring:

Lessons from a Large Scale Nutrition Program in Madagascar<sup>†</sup> Emanuela Galasso and Jeffrey Yau _Development Research Group Bureau of Economics World Bank Federal Trade Commission*_

## Abstract:

Monitoring data is generally collected as a by-product of the process of monitoring program implementation. Yet this rich source of data has not been exploited to assess the effectiveness of the program. In this paper we use detailed administered data from a large-scale, community-based nutrition program in Madagascar, to argue that this data can be used to estimate the _differential_ effect of increased exposure to the program and study how these returns to exposure evolve over time. We find that the returns to exposure are positive: communities exposed for additional one (or two) years display on average lower malnutrition rates of around 7-9 percentage points. Moreover, we find that the returns are decreasing as time and duration increase, though they do not dissipate to zero. These results are consistent with the hypothesis that the returns to the program reflect learning effects from the intervention. Finally, the results show higher differential returns to the program in poorer areas and areas more vulnerable to diseases. These findings have important implications for how such programs should be scaled-up within a country.

_Keywords_ : impact evaluation, duration, nutrition intervention, community-based program, large-scale programs, propensity-score, matching _JEL classification_ : I12, C14, C31 World Bank Policy Research Working Paper 4058, November 2006 _The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the view of the World Bank, its Executive Directors, or the countries they represent. Policy Research Working Papers are available online at http://econ.worldbank.org._ † This research has been sponsored by the World Bank Research Grant “Community Nutrition: Evaluation of Impacts in Africa”, a comparative meta-analysis on the effectiveness of community-based nutrition programs in Madagascar and Senegal. We thank Claudia Rokx, the project task manager for her enthusiastic support to the evaluation effort, Harold Alderman for useful discussions throughout the project, Rinku Murgai, Martin Ravallion and seminar participants at the NEUDC at Brown University, the World Bank and Virginia Tech. We are extremely grateful to the Seecaline project management unit, to Malalanirina Ratomaharo in particular, and to all the provincial directors for providing the monitoring data and sharing their vast knowledge of the program. We thank Bart Minten for sharing the data on the Commune Census. We wish to thank Quinghua Zhao for providing technical help in the data entry. The findings, interpretations and conclusions of this paper are those of the authors and should not be attributed to the World Bank, its Executive Directors, or countries they represent, the Federal Trade Commission or any individual commissioner.

Correspondence by emails to <u>egalasso@worldbank.org</u> or yyau@ftc.gov 1
# **Introduction**

Monitoring and supervision are intrinsic aspects of the management of any social programs. Monitoring over time how the program performs in the field allows administrators to understand whether the project is implemented according to its design. An essential data tool underlying the monitoring activities is usually referred to as ‘monitoring data’, or MIS (monitoring and information system), which is a subset of the administrative data that is specifically linked to activities of a specific program. The data contained in the monitoring system generally produces simple descriptive statistics that, at different level of aggregation, provide feedback to the implementing agents to address eventual problems. Administrative data have been increasingly used as a tool to evaluate program performance of public assistance programs in developed countries, but there is still very limited evidence to date that uses administrative data to evaluate social programs in developing countries and hardly any that uses monitoring data for this purpose.<sup>1</sup> Why is it the case? The main reason constraining the use monitoring data for evaluation purposes is that, by design, it only collects information on participants, therefore making it impossible to estimate what would have happened in the absence of the program. Evaluating a public program requires data from which to construct participant groups and comparisons groups. Simple _reflexive_ comparisons (before-after) using only data on participants would generally impose a very strong identification assumption (Heckman, Lalonde, Smith 1999): they implicitly assign all the observed change in the outcomes of interest to the program, and assume away the need to construct a counterfactual change in the absence of the program<sup>2</sup> . As such, evaluations using monitoring data would not allow quantifying the parameters of interest generally used in the evaluation literature, such as the average treatment effect or the average effect of treatment on the treated, both of which require information on nonparticipants.

Yet, monitoring data have inherent (but un-explored) features that make them an attractive tool for analytical and evaluation work. First, it covers the entire universe of the participating population, allowing the results based

> 1 A notable exception is Ravallion (2000), who uses administrative data on the geographical allocation of expenditures of a workfare program in Argentina matched with a poverty map to implement a methodology to measure the latent differences in the average allocation of the program between poor and non-poor.

> 2 An interesting variation to using reflexive comparisons is proposed by Piehl et al. (2003), who show that sufficiently long time-series data on participants can be used to test for structural breaks and identifying impacts.

2
on this data to generalize to the entire target population and to study how the same program might have different outcomes in different geographic locations depending on their initial conditions. Second, it is created as a natural by-product of program implementation, making it a readily-available, low-cost source of data. Finally, it is collected since program inception and therefore has a time dimension that would be worth exploring when assessing the effectiveness of a program over time.

In this paper we argue that monitoring data can be fruitfully analyzed in the context of assessing program impact to address a different set of questions, questions that are highly relevant for programs where the treatment impact crucially depends on duration of exposure to the program. Specifically, we examine the relationship between the duration of program exposure and nutrition outcomes. The returns to exposure is important from a policy perspective in programs where gains are cumulative over time and where (as in most cases) the program impacts are expected to be heterogeneous across socio-economic groups or areas in across the country. If the returns to exposure are positive and the program is gradually phased in within a country, it might pay to target sequentially groups/areas with the highest marginal return to exposure first, in order to maximize the total gains from the investment in the program.

When estimating the differential returns to program duration on child nutritional outcomes, we use a community-based nutrition intervention in Madagascar. The program (SEECALINE)<sup>3</sup> was initiated under World Bank support in 1999 and has been gradually scaled-up to cover about one-third of the total targeted population. Its gradual expansion provides substantial variation in program exposure. Its large scale feature allows us to exploit the richness of the monitoring data that was collected over time during its implementation.

We quantify the marginal effect of increasing duration to the program by comparing at any given point in time the nutritional outcomes of the “comparable” communities that experience different durations of program exposure. We call this parameter of interest _differential treatment effect (DTE)._ Figure 1, which plots the outcome trends for sites joining the program at different points in time, illustrates this scenario. The vertical distance between these curves is the observed difference in outcomes between the longer and the shorter exposure sites. These vertical

> 3 Seecaline stands for " _<u>Surveillance et Education des Ecoles et des Communautés en matière d’ALImentation et de Nutrition Elargie</u>_ " (Surveillance and Education of Schools and Communities on Food and General Nutrition) and is among the few examples of African countries to embark in a nationwide, large-scale community-based nutrition program.

3
differences represent the parameters to be estimated in our empirical work at any given point in time. Indeed, there are very few papers that take into account the _duration in program implementation_ as an important parameter of interest, with the exception being Behrman, Cheng and Todd (2004), Armecin et al (2005), and Gertler (2004). Based on household survey data, these papers suggest that the marginal impact of increased program exposure is positive _at a given point in time_ . Obviously, a simple comparison among communities with different lengths of exposure is likely to produce biased estimates to the extent that the phase-in is not made at random. In order to account for this potential selection into different durations, we adopt an empirical strategy that was first proposed by Behrman, Cheng and Todd (2004) that only require the use of program participants. The attractiveness of applying these methods to the monitoring data is that the differential effect of exposure can be estimated _at different points in time_ . As time progresses, the pattern of the differential effect can provide suggestive evidence on how the returns evolve over time.

Why should duration matter in the context of this type of program, or more generally in the context of social programs? Duration might simply reflect _composition effects_ : as in any other program that is gradually phased-in, it takes time for the program to fully cover the target population. At the program inception in the community only a subset of the target population will have benefited from it. Over time, a larger fraction of life for these children is naturally going to be exposed to the program. Assuming a positive impact of the program, communities that are exposed for longer duration should therefore have on average better outcomes relative to areas where children have exposed for less. Under this assumption, the returns to exposure should be positive and decreasing over time. At the limit, when both sets of communities have their children fully exposed to the program, the differential effect should be converging to zero.

A complementary, though not exclusive, explanation depends on the nature of the intervention. Communitybased nutrition programs target at children early in life (generally from zero to three years of age), and promote behavioral change in the context of existing resources, acting through improvement in the knowledge of child care and feeding practices<sup>4</sup> . In this context, duration might matter per se because of _learning effects_ . These effects may

> 4 Mothers are taught the importance of exclusive breastfeeding (at least for infants), the timing and composition of complementary feeding, and hygiene and nutrition practices, especially when the child is ill. The basic package of growth monitoring and promotion is usually accompanied by nutrition IEC and nutrition education, micro-nutrient supplementation to

4
arise from improved mothers’ knowledge on feeding and general nutritional practices that results from their participation to the program (direct effect). They may also work as an indirect effect of increased overall learning by all mothers absorbing nutrition education messages as well as by community workers who become more efficient in their program implementation. In this setting, the differential effect of exposure should be positive, but not necessarily be eliminated over time. Whether these returns are increasing/constant/or decreasing over time will depend on the specific technological features of these learning effects in the children’s health production.

Our first set of results indicate that communities with longer program exposure show significantly better nutritional outcomes, measured in terms of lower malnutrition rates. These results echo the findings from the existing literature that returns to program exposure are positive at a given point in time. For one extra year of exposure, the differential effect is declining with age, and it is statistically significant only for the 0-6 months old children but not for older children (7-12 and 13-36 months old). For two extra year of exposure, this “age-ranking” phenomenon disappears, and the returns to exposure are significant for all age groups. More specifically, communities exposed to the program for two extra years, where most of the target population has been covered by the program, display on average lower malnutrition rates by 7-9 percentage points for all age groups. The results on the age-ranking suggest that the program impact takes time to materialize, and the younger children stand to gain more because a larger fraction of their lives has been affected by the program.

Second, the differential effects of exposure are decreasing over time for all age groups, even accounting for seasonal effects. It is important to highlight the fact that the differential effects of exposure, however, do not disappear as duration increases. We can ignore the composition effect when focusing on the cohort of 0-6 months old, who are rapidly fully exposed in both sets of communities (short and long) after six months of observation. Even when fully exposed to the program, we find that the returns to exposure for this age group decrease over time but do not converge to zero. We interpret this finding as supportive evidence towards learning effects of the program.

After examining the overall effect of differential length of exposure, we investigate the extent to which these effects vary along several key environmental factors that affect children’s health in the intervention areas. These children and pregnant women. One important variation across countries is whether or not supplementary feeding of malnourished children is included.

5
factors are (i) whether or not an intervention area has been hit by cyclone; (ii) poverty rate; (iii) the length of the lean season; and (iv) accessibility to safe water. We exploit the large-scale aspect of the program and the large sample size of our monitoring data (representing the universe of participants’ communities) to re-estimate the returns to exposure for different subgroups of the targeted population and analyze whether the return differs across the subgroups.

Our results show that the returns are higher in poorer areas, in areas that have been exposed to cyclones, and in areas that have a longer lean season. The fact that the results vary significantly according to these key characteristics of the program sites has an important implication for the potential expansion of the program within the current districts of intervention as well as to the new areas in the rest of the country.

The rest of the paper is structured as follow: next section reviews the relevant literature, and section 3 describes the program and the monitoring data in details, highlighting the phase-in aspect observed in the data and the large variation in both initial nutritional outcome and economic characteristics. After fully describing the program features and the data structure, we will discuss the empirical method that we employ to estimate the return to the program duration, and our results are presented in section 6. Section 7 concludes the paper.

# **2.  Literature Review**

There exist only a few papers that focus on the _duration in program implementation_ as the primary parameter of interest. Behrman, Cheng and Todd (2004) studies the impact of a pre-school program in Bolivia and finds that the impact on early childhood development outcomes is observed among children who have been exposed to the program for more than one year, compares to those exposed for shorter durations. They develop empirical methods to estimate the differential effect of duration both using a survey sample of participants and nonparticipants as well as using a sample of participating children only. An analogous approach is followed by Armecin et al (2005), who find outcomes for young children that are increasing with duration of exposure using household data from an early childhood development program in the Philippines. Finally, Gertler (2004) shows that the improvements in terms of the incidence of illness of children in the Mexican anti-poverty program _PROGRESA_ is significant only for those villages with two years of program exposure, relatively to those with less than one year.

6
These papers consistently suggest that the marginal impact of increased program exposure is positive _at a given point in time_ . Our empirical analysis builds on this existing evidence by providing estimates on how the returns of exposure evolve as time and duration into the program increase.

In addition to the small number of papers that study the impact of program duration, there also exists very little empirical evidence on how effective in terms of reducing malnutrition these types of interventions are for _largescale_ programs (Gillespie, Mason, Martorell 1996, Heaver 2002a).<sup>5</sup> The quantitative evidences available on the impact of community-based, nutrition intervention and other direct nutritional programs mainly come from either case studies or small-scale programs.<sup>6</sup> The only large-scale program that includes a nutrition component and that has been rigorously evaluated is _PROGRESA_ (now _Oportunidades_ ) implemented in Mexico.<sup>7</sup> Even in the context of a randomized design, Attanasio et al (2003) have documented sizable differences in both the distribution of background characteristics as well as the returns on those characteristics across poorest and non-poorest states within the same country. They show that household observable characteristics do not provide sufficient information to extrapolate the effects of the intervention from one set of regions to another. Therefore, results from case studies

> 5 Many evaluation exercises of large-scale nutrition programs have generally not been planned ex-ante in the sense of placing particular attention to carefully construct a set of potential comparable, control communities or populations. This is the case for the evaluation of the Tamil Nadu Integrated Nutrition project (TINP), which shows a decrease in underweight prevalence of 1.5 per cent per year in participating districts (Heaver 2002b).  Comparing differential trends over time in intervention areas and non-intervention areas may over/under-estimate the true changes attributed to the intervention if the selection bias due to the purposive targeting is not adequately addressed (Pitt, Rosenzweig, Gibbons 1993). Another notable example is the famous growth monitoring and promotion program in Honduras (Atencion Integral a la Ninez, AIN, Plowman et al 2004): AIN communities were chosen to be relatively more disadvantaged.  The evaluation approach employed was such that the matched comparison communities ended up being exposed by the program, so that the evaluation focused on _reflexive_ comparisons (before-after).

> 6 The first large-scale, community-based nutrition programs were implemented in South Asia (Tamil Nadu Integrated Nutrition Project (TINP) and the Bangladesh Integrated Nutrition Project (BINP)), but they suffered from weak evaluation design that failed to disentangle the effect of the program itself from the effects of other programs and of the general welfare improvements arising from economic growth. Notable examples include the Iringa project in Tanzania (Gillespie et al 2003), the Indonesian Nutrition Development Program and the Weaning Program in Indonesia (Favin Griffiths 1999), the ANEP program in the Dominican Republic (USAID 1988), the Nutrition Communication Project in Mali (Ross 1997)

> 7 The program, however, does not focus exclusively on nutrition. It offers a conditional cash transfer, conditional the households being able to fulfill some behavioral requirements which are designed to improve, among other things, prenatal care, immunization, nutrition monitoring, and supplementation and participation in health and nutrition education. With a randomized design during the phase-in, the program has shown significant improvements (among other outcomes) in children exposure to illnesses, anemia and height (Gertler 2004). The extrapolation of the results to other settings, however, is difficult for various reasons.  On one hand, its design of the program makes it hard to disentangle the effects that are attributed to the cash component and that are attributed to the behavioral change induced by the cash transfer. Community based programs do not have a cash-transfer component and use voluntary participation to change behavior. On the other hand, the estimated impacts may not generalize to poorer contexts outside Latin America, especially to South Asia and Africa, where the bulk of the number of malnourished children and the malnutrition rates are higher, and where the problems of food security are much more severe.

7
or small-scale programs, even if they are well-designed randomized field trials, can hardly be generalized to a much larger population. Our paper will exploit the large scale dimension of the monitoring data to document the extent to which the specific parameter of program impact (in our instance, returns to exposure) varies along some key socioeconomic characteristics in the context of a large scale, nutrition intervention.

# **3. The Program Seecaline: Program Design and Phase-in**

The program SEECALINE is a community-based nutrition program that started in 1999 and is gradually scaled up to cover more than half of the country’s districts. The objective of the program is to improve the nutritional status of children under the age of three and of pregnant and lactating mothers in the targeted project areas. To maximize geographical coverage as well as to provide quality services on a large-scale, the program is contracted out to local NGOs for implementation (management, delivery, operations research and supervision) at the local level<sup>8</sup> where the services are delivered locally by a community nutrition worker (ACN), who is usually an elected woman from the targeted community. The ACN's activities are in turn supervised by the NGO<sup>9</sup> .

_<u>The preventive approach</u>_ <u>: Although conventional wisdom suggests that the</u> _quantity of food_ is the key to prevent growth faltering and retardation, recent studies have argued that nutrition practices, feeding methods, and the protection from diseases are equally important (Allen, Gillespie 2001, Cebu Study Team 1992, WHO 2003). Seecaline adopts a preventative approach to prevent growth from faltering before it becomes too severe. As a community-based nutrition program, Seecaline revolves around a monthly growth monitoring activity as a focal point. The participating communities are mobilized towards raising the awareness of the problem of malnutrition and are taught and encouraged to improve hygiene, child care and nutrition practices. All children under the age of three and the pregnant and lactating women in a community are eligible to participate in all of the program activities. On a monthly basis, the ACN weighs all of the children under the age of three and provides counseling to the mothers regarding the nutritional status of their children indicated by the growth chart. The ACN may also visit a participating family if its child’s growth chart shows no progress or if he/she misses a weighing sessions.

> 8 Specifically, by local level, we mean “sites,” which are defined to be geographically bound by a five kilometer radius.  This setup is designed to minimize the (direct and indirect) transport cost incurred by the mothers to participate in the site activities as well as to minimize the ACN'’s workload.  In this paper, we will use “sites” and “communities” interchangeably.

> 9 In the monthly situation meeting, the ACN and the NGO representatives discuss the problems of the participating community.

8
Besides growth monitoring, the program provides other activities. Eligible children are given micronutrient supplementation and de-worming. The promotion of behavioral change includes direct counseling to the lactating and pregnant women, nutritional education sessions and cooking demonstrations by the community nutrition worker who heavily emphasizes the proper weaning practices and prepares recipes that make use of available the food sources that are available in the local community. The community nutrition worker also carries out an annual census of all of children under the age of three and mobilizes the mothers to participate in the weighing and education sessions.

Seecaline is a large-scale, community-based nutrition program that was gradually phased-in over time. The rest of this section will describe in detail the phase-in process. This description provides a crucial guidance regarding the variables used to model whether a community belongs to a longer vs. shorter duration.

_<u>Selection and timing of participation of districts of intervention</u>_ <u>: Madagascar is administratively organized in</u> six provinces, 111 districts, around 1,400 communes and 16,000 villages. Seecaline’s activities started in 1999 in four provinces and expanded to all six provinces in 2000.<sup>10</sup> The program targets the most malnourished districts: based on a nationally representative anthropometric survey collected in 1997/98, only the districts (46 out of 111) that had an average malnutrition rates above the national average were selected for the intervention. The total number of sites to be opened in each district was chosen to reach a coverage rate of 50% of all children below three years of age. The 50% target was to be achieved gradually over time, with 10% coverage per year in the first two years of operation and 15% coverage per year in the third and forth years. In addition, ten rural districts affected by droughts and cyclones in the year 2000 were selected to participate.<sup>11</sup> The program expansion stopped at the end of 2001/beginning of 2002. The intervention currently includes about 3,600 project sites.<sup>12</sup> _<u>Selection and timing of participation of communities within the intervention districts</u>_ <u>: The program expansion</u> followed a sequential contractual engagement of NGOs across districts. NGO were selected based on a bidding

> 10 CNII is an extension of an earlier Bank pilot project originally targeted to two provinces (Antananarivo and Toliary).

> 11 Finally six additional urban districts that cover the province capitals were added in 2002, in the aftermath of a political crisis. Emergency sites were opened to alleviate the cost of the crisis on urban populations, and were subsequently made permanent. Our data is not available for such urban areas.

> 12 Note that we are using communities and program sites interchangeably. A site is identified by its geographical delimitation (of a radius of five kilometers, within the commune boundaries). A site generally comprises 1 to 3 villages.

9
workshop organized every year around August/September<sup>13</sup> . The selected NGOs were then assigned a given number of sites to be open in any given district, based on an estimate of the target population in the districts.<sup>14</sup> The program in each province was initially advertised to all majors of the communes included within the district. Mayors would then organize a meeting with all the communities to make them aware of the program and encourage them to participate. A community, to be eligible to open a project site, had to accessible for most part of the year (by auto/motorbike, chariot or pirogue). Accessibility in Madagascar is a major constraint for service delivery, and the phasing in of the program reflected this constraint. The NGO, and more specifically the animator who is in charge of supervising and providing support to the community nutrition worker, need to able to reach the sites regularly by motorbike. Given the logistical constraints, the NGOs, in partnership with the communes, started on average with the most accessible communities first, and then expanded to the least accessible sites over time.

# **4. The Data**

The monitoring data of the program consists of monthly detailed information at the site level. This information is originated from the registry and is kept by the community nutrition worker. Each registry contains information on the program activities, health outcomes, and the time when the site joins the program.

The monitoring data covers about 3,600 sites in the whole country (as of January 2003). The information spans from 1999 to the end of 2002, with information varying depending on the date of entry to the program. We will confine the analysis to the biggest four main provinces (Antananarivo, Fianarantsoa, Tulear and Tamatave), which represent 85% of the total target population, and for which there is substantial variation in the phase-in of the program. We will make use of the quarterly data in order to smooth out noises in the outcome indicators.

The registry for each site contains information about the monthly participation of registered mothers, the total number of children weighed in each month, and the number per age group who are classified as severely or moderately malnourished (as indicated by their position in the growth chart).<sup>15</sup> Moreover, for each community we

> 13  Eligible NGOs had to have had at least one year of experience in the district of intervention.

> 14 Each NGO could be assigned a maximum of 20 sites and had to hire a supervisor ( _animateur_ ) to be assigned to each site. 15 The registry contains also information on food supplementation (total quantity received, and the share of children receiving any food supplementation by age group), vitamin A supplementation (by age group) and on participation rates of mothers to nutritional education sessions and cooking demonstrations.

10
have information of the distance of the site to the center of the commune, as well as basic information on each partner NGO that is in charge of the site (such as size/personnel, whether it is local or national, their experience and sector of activity before joining Seecaline).

We complement the monitoring data with two additional sources of data. The first one is the Commune Census data, conducted in 2001 under a joint collaboration between Cornell University, the National Statistical Institute (INSTAT) and the agricultural research institute within the Ministry of Scientific Research (FOFIFA). The census contains detailed information on demographic and socio-economic characteristics of all communes in the country<sup>16</sup> . The most important aspect of this data is that it includes some key variables that capture the site selection process such as remoteness, main economic activities, and access to infrastructure. Second, we use commune level estimates<sup>17</sup> of poverty at the commune from the poverty map developed by Mistiaen et al (2002) by combining the 1993 household survey with the 1993 population census. The technique allows estimating consumption-based measures of poverty and inequality at very low level of geographic disaggregation. Mistiaen et al (2002) document a considerable degree of spatial heterogeneity in poverty across administrative units within provinces in particular across districts.

# **4.1. The Definition of Treatment**

As described earlier, the program is gradually phased-in each year to the intervention districts. In August and September each year, the provincial directors (of Seecaline) operate a bidding session in which partner NGOs are selected. These partner NGOs are responsible for opening new sites for each district. There exists a grace period between the time when the contract (between the Seecaline and the NGOs) is signed and the time when a site is actually opened and operated because it usually takes two to four months to select sites, elect a community nutrition worker, and train the NGO personnel and the community nutrition worker.

> 16 The Census covers 1385 (out of a total of 1394) communes in the country in 2001. The missing communes could not be reached because of the local security reasons. The questionnaire was administered to a focus group composed of residents of the commune.

> 17 The Communes were introduces in 1995, replacing Firaisanas as the smallest administrative unit. In order to create communes, some of the Firaisanas were subsequently split or changed some of the boundaries. Our unit of analysis is the community: we are able to assign all communities to old Firaisanas/new Communes with the help of a geographic mapping provided to us by the Statistical Institute.

11
Because we focus on quantifying the impact of different program participation durations on the malnutrition rate of young children, we first define the length of exposure to the program. A site is observed as soon as it begins its operation. We take the date of the first weighing session at a site as the first period in which the site begins its operation, _t_ 0 , and we will call this the entry date. At any given point in time, _t_ , after the site is open, the length of exposure to the program for each site _s_ is defined as _d s t_ ≡ _t_ − _t s_ 0 . As shown in Figure 2, the date of entry is clustered around the first and forth quarters throughout the entire program period in which we have observations (i.e. between 1999 and 2002).

This gradual phase-in feature of the program at a roughly one-year interval allows us to construct two alternative definitions of treatment:

1. Being exposed to the program for _<u>one extra period (year</u>_ <u>):</u><sup>_d_</sup> _t_ + 1 versus<sup>_d_</sup> _t_<sup>;18</sup>

2. Being exposed to the program for _<u>two extra years:</u>_<sup>_d_</sup> _t_ + 2 versus<sup>_d_</sup> _t_<sup>.19</sup> In our framework, the treatment indicator compares sites participating in Seecaline for a longer duration (the “treated” sites),<sup>_d_</sup> _t_ + τ , τ = 2 , 1 , with sites participating for a shorter duration (the “untreated” or “comparison”

> sites),<sup>_d_</sup> _t_<sup>.  It is important to note that</sup><sup>_d_</sup> _t_<sup>is time-varying and is measured at different points in time, as the impact</sup>

is computed at five different quarters. As time progresses, all of the sites in our sample increase their program

> exposure over time (<sup>_d_</sup> _t_<sup>), while the absolute</sup><sup>_difference_in the length of exposure between the “treated” and</sup>

> “untreated” sites remains constant ( τ , at one or two years intervals).

When measuring the impact at a particular point in time, we are able to quantify the _return to the length of_ _exposure_ . When keeping fixed the absolute difference in the length of exposure between the treated and the

> 18. We suppress the subscript _s_ in _dst_ .   It is possible that some of the “untreated or comparison” sites (i.e. sites with shorter exposure) have not been exposed to the program (i.e. _dt=0_ ), depending on the entry date and the date when we measure the impact.

> 19 We experimented with different thresholds in separating sites into different groups, where we compare sites with _<u>at least</u>_ one (but less than two) years or at least two years difference in the exposure to the program.  The results are qualitatively consistent and are available from the authors. The definition of treatment used in the paper makes it easier to provide a more precise and straightforward interpretation.

12
comparison sites and measuring the impact at different points in time, we are able to capture the _change in the return to the length of exposure._

# **4.2. Characteristics of Sites with Different Lengths of Exposure to the Program**

Since we focus on quantifying the return to duration of participation by comparing sites that enter the Seecaline program at different points in time, it is important to ask the question: “How different are sites that joined at different points in time?” Table 3 provides the group mean characteristics of the sites in each of the treatment categories. The comparison of these mean characteristics suggests that the program was progressively phased-in to the areas that are more isolated (both in terms of communes being more distant to the nearest urban center and further away from main roads, as well as in terms of more isolated communities within communes), with fewer infrastructure (connection to electricity and presence of markets), poorer (as indicated by the headcount ratio), and with a higher average initial malnutrition rate (measured at the opening date of the site). The differences reiterate the point that the phase-in aspect of the program is not made at random, and it is invalid to take a simple mean comparison approach that measure the difference in outcomes across the two treatment groups and attribute the difference to the length of exposure.

Importantly, the data also shows a substantial heterogeneity across sites that joined the program within the same period. Figures 2 and 3 show that the dispersion around the mean at different dates of joining the program is large along different dimensions, such as the poverty rate and the initial malnutrition rate. The extent of the dispersion is advantageous for our purposes because it gives us scope for finding sites that are “comparable” across different duration of exposure to the program with respect to the observed characteristics.

These data sources (the monitoring data, commune census, and poverty map) provide us with the key predetermined characteristics to us model the selection of communities/sites into different lengths of exposure within the intervention districts. We have information on the socio-economic characteristics of the communes such as remoteness, existing infrastructure and weather shocks that hit the commune in the three years prior to the census. The shock information helps us identify those communes in districts added only in a second phase in 2000 due to droughts and cyclones severely hitting the country. As the sequential selection of sites was done in 13
partnership between the communes and the NGOs and the accessibility of the sites (measured by the distance between the site and the center of the commune) is one of the main constraints in the expansion of the program,, we include in our selection model the information on the remoteness of the sites, key socio-economic characteristics of the communes of intervention, NGO characteristics, and the interactions between remoteness and the NGO characteristics.

One important concern in modeling selection into different durations is to what extent we can capture the placement that are done sequentially in communities where most needed, i.e. poorer and more malnourished sites. In this regard, we control for both the poverty rate of the commune and the initial nutrition status of the target population. We take the malnutrition rate at the first measurement of the sites as the (pre-intervention) indicator of the initial health status of the target population. Since the initial malnutrition rate is a time-varying covariate, we de-trend it by estimating a flexible year trend.

In the next section, we will re-state the question that we address in this study and describe in details the sources of identification **.**

# **4.3. The Target Population**

The population of interest is all children living in the intervention areas who are aged between zero and three, which is the age window that has the largest risk of growth failure (Shrimpton et al 2001)<sup>20</sup> . We categorize the target population into three age groups (0-6 months, 7-12 months and 13-36 months) and use site as the level of observation. The separate analysis by age groups is motivated by the nature of the intervention because the feeding and child care practices vary greatly across these groups. For instance, one of the main messages of the program is to encourage exclusive breastfeeding for children six months old or younger, which comprises 14% of the target population. On the other hand, children who are seven to twelve months old (20% of the target population) enter the weaning phase, a period during which semi-solid complementary food and liquids are introduced. In this transition, children are vulnerable to losing weight because this is the period when they start to be exposed to foodborn illnesses. The program emphasizes the importance of the preparation and composition of food for this age

> 20 Once a child gets older than three years of age, growth retardation becomes permanent and has important long term consequences on cognitive development, schooling and productivity later in life.

14
group. Older children (1-3 years of age) account for 64% of the target population in the participating communities (table 2). During this last age window, children are vulnerable to infections. During these critical period the immune system is still not fully mature (Martorell 1999) and susceptible to illnesses. The program emphasizes the importance of hygiene practices as well as promotes child care practices in the event of illnesses.

# **4.4. The Outcome of Interest**

Our outcome of interest is the average malnutrition rate, which is measured by the nutritional status of children under three, of the communities participating to the program. We use weight-for-age<sup>21</sup> as an indicator for the nutritional status of children. If the weight (for a given age) of a particular child falls below two (three) standard deviations from the international standard (z-scores), then the child is considered as moderately (severely) malnourished.

Figure 5a presents the non-parametric density estimates of malnutrition rates over time, separated by the three age groups.. Three points are worth noting in this figure: (1) the proportion of underweight children increases with age, (2) There is a downward trend in the outcome of interest in the participating communities, as malnutrition rates decrease over time for all age groups, (3) There is seasonal variation in malnutrition rates, which slightly rebound around the lean season (in the first and last quarters of each year), especially for the 7-12 and 13-36 months old children.

In addition, we document in figure 5b the trends in malnutrition rates over time, separated by age groups, for participating communities that have different lengths of exposure. As we stress earlier in this paper, the vertical distance between these trends cannot be entirely attributed to the difference in the duration of program exposure. It is because the vertical distance captures the difference in the average malnutrition rates between sites that entered the program in two different periods without accounting for initial differences in the selective selection into different program durations.

> 21 The weight of each child attending the weighing sessions is obtained from the registry information.

15
# **5. Empirical Methodology**

We are interested in measuring the effect on malnutrition rate of different durations of exposure to the Seecaline program and the heterogeneity of the program impacts with respect to some key socio-economic characteristics of the sites. Following Behrman, Cheng and Todd (2004), we use propensity-score matching to estimate the marginal effect of being exposed to the program for different durations. An attractiveness of this methodology is that the marginal program impact can be estimated using only data on program participants. Moreover, the approach does not require a model in the selection into the program and allows for selection based on unobservables (Behrman, Cheng, and Todd (2004)). The marginal impact is estimated by comparing “comparable” participating communities that enter the Seecaline program at different points in time. Knowing that communities are selected into the program in a non-random fashion, we estimate a model of program exposure duration selection. We focus on the aggregate outcomes at the community level rather than household-level outcomes. As the socio-economic characteristics of the participants (and non-participants) in the communities are not available in the data set, we control for aggregate socio-economic variables at low level of geographical disaggregation (i.e. communities and communes). We also exploit the panel dimension of the monitoring data to compute the differential effect at <u>different points in time, as the length of exposure increases.</u>

# **5.1. Estimating the differential returns to the length of program exposure**

To formally define the parameter of interest, we need some notations. We first define the malnutrition rate for

> age group _a_ living in site _s_ and participating in the Seecaline program for _d st_ periods as of time _t_ as a random

> variable _Yst_ ( _a_ , _d st_ ) .  The differential treatment effect is defined as:

> where _DTEst_ ( _a_ , _d st_ , τ) denotes the difference in the average malnutrition rate measured at time period _t_ for the

children in age group _a_ living in the sites that have participated for _d st_ + τ years relative to the counterfactual

> situation in which the sites only participate for _d s_ ' _t_ years. In what follows, we subsume _d st_ and the subscript _s_ in

the differential treatment effect for notational brevity. τ measures the additional years of program exposure, where 16
τ = 2 ,1 in our case. In a nutshell, what we do is to take two groups of sites with different lengths of exposures at time period _t_ and compare their average malnutrition rates, controlling for all the covariates that might affect their assignments into these different program durations. Given that the outcome of interest is malnutrition rates, _DTEt_ ( _a_ , τ) < 0 should be interpreted as a marginal _improvement_ of the nutritional status of children of age group _a_ (in sites exposed for τ years more). Rearranging the above expression as: _E_ [ _Yt_ ( _a_ , _d t_ + τ |) _d t_ + τ ] − _E_ [ _Yt_ ( _a_ , _d t_ |) _d t_ + τ ] .

it becomes clear that this parameter can never be directly observed in the data, for the second term represents a counterfactual outcome<sup>22</sup> that cannot be observed<sup>23</sup> .

We employ the method of propensity-score matching to non-parametrically estimate the _DTEt_ ( _a_ , τ ) . This estimation strategy can produce a consistent estimator for the above parameter of interest, provided that the two identification assumptions for propensity-score matching hold. First, the counterfactual average malnutrition rates measured in period _t_ of the longer-exposed sites had they instead have shorter durations of exposure can be estimated using the period _t_ average malnutrition rates of the shorter-exposed sites, conditional on a vector of preprogram characteristics of the sites. The identification assumption is that the selection into different lengths of exposure is conditionally exogenous, once we adjust for differences in the exogenous background characteristics that have been used in the phase-in of the program. The second condition requires that there is sufficient overlapping support in the distribution of covariates of the two treatment groups (long and short exposure).

> Using the panel structure of our data, we can quantify _DTEt_ ( _a_ , τ ) at different points in time. Note that

_DTEt_ ( _a_ , τ ) is time-varying: as time progresses, duration<sup>_d_</sup> _t_<sup>increases for both ‘treated’ and ‘non-treated’ sites the</sup> absolute difference in τ <u>stays the same. Keeping in mind that</u> _DTEt_ ( _a_ , τ) < 0 implies a differential _improvement_ , we propose two hypotheses:

> 22 Specifically, the counterfactual is expresses as the average malnutrition rate at time _t_ for the longer exposed (<sup>_d_</sup> _t_ +τ ) sites

> had they been exposed for a shorter duration (<sup>_d_</sup> _t_<sup>).</sup>

23 We also estimated the parameter of interest using a regression framework in which all of the matching variables included in the propensity-score model are included. We found that the results are even stronger in the regression framework than in the non-parametric technique. Nonetheless, we chose the non-parametric method because we want to impose in our estimation strategy the overlapping condition (comparability) across sites with different durations and do not want to impose the linear functional form on the impact of interest.

17
_negative_ . Our data allows us to look at the cases where τ = 2and τ = .1 We cannot map out the entire function of the returns to exposure because we only have two values of τ ; nevertheless, we can draw inferences on the slope of the function.

> (2) Δ _DTEt_ − _t_ '( _a_ , τ ) = _DTEt_ ( _a_ , τ ) − _DTEt_ '( _a_ , τ ) ≥ 0 ∀ _t_ > _t_ ' , That is, the returns are _decreasing (non-_

_increasing)_ with the length of program exposure.

There are several concerns when estimating the differential impact of the length of program exposure. One important concern is to be able to disentangle _time effect_ from the _program effect_ . There might be an underlying trend for the health outcome even in the absence of the intervention due to a general trend in economic growth and welfare of the population (and therefore a downward trend in the level of malnutrition rate). Nonetheless, the existence of this trend does not automatically bias our results, insofar as the trend applies equally to all communities that participate in the program for different durations. We will assume that all participating sites experience the same trend, if any, so such a trend can be eliminated by taking the difference between sites with longer and shorter durations at any given point in time.

A related concern is to be able to separate _seasonal effect_ (therefore deviations from a general trend) from the differential exposure to the program. We address the issue by comparing the estimated differential returns across the same quarter in two different years. Should the results be driven only by seasonality, we would find that the observed differential effects to be the same.

Finally, an important caveat is that the observed outcome is obtained only from the children who attended the weighing sessions. Given that our outcome measures are aggregate in nature, we cannot observe who attended (the weighing sessions) and who did not. Bias may arise from the possibility that the weighed children represent systematically the more (or less) malnourished children in the community. In an appendix, we provide a sensitivity analysis and show that our results are not affected by the non-compliance of some of the eligible participants in the participating communities.

18
# **5.2. Heterogeneity in returns and its policy implications**

Besides quantifying the returns to the length of program exposure, we assess the effectiveness of a large-scale intervention and draw policy implications from it. Unlike small-scale interventions, which often focus on a fairly homogeneous group of participants, large-scale, but not necessary universal, programs bring about heterogeneity in characteristics (and possibly returns) across different regions and areas of a particular country. The impact estimates of pilot program cannot be generalized to a larger population or even to the same population in different areas of the country (what is commonly referred to in the evaluation literature as a problem of _external validity_ ).<sup>24</sup> In the context of our analysis, we estimate how the (differential) returns to the program vary according to some key aggregate economic and environmental factors that affect the child health production function: poverty levels, incidence of cyclones, the length of the lean season and access to drinkable water. Poverty levels proxy for the average level of economic conditions and resources/nutrients availability. Cyclones, the length of the lean season, and the lack of access to safe water are all factors that enhance the likelihood of morbidity and the likelihood of growth faltering. Promoting behavioral change in feeding and child care practices in environmental settings with low incomes and where the existing supplies of food and water are contaminated might offer sizable benefits to a child’s health. We estimate whether the marginal gains for “participating in Seecaline earlier” vary systematically along these dimensions.

In order to measure how the DTE varies according to each of these dimensions of interest, we divide the sample in two subgroups (with/without cyclones/water, with poverty or length of the lean season below/above the median) for each of the variables ( _X tk_ 0<sup>) that capture these characteristics. We then re-estimate the</sup><sup>_DTE_for each of</sup> − _k k_ these subgroups, i.e. we estimate _DTEt_ ( _a_ , τ , _X_ ; _X_ = _j_ ) for two different groups _j_ = 1,0 .

Should the returns to exposures be systematically correlated with some socio-economic characteristics of the areas of intervention, the results could provide guidance for future expansions of the program into different areas of the country.

> 24 Another important effect coming with scale are general equilibrium effects, generally defined as cases where the intervention itself on a large scale affect both demand, supply, and the equilibrium price (Moffitt 2004, Heckman, Lalonde and Smith 1999). In our case, the intervention promotes behavioral changes, and it does not affect directly a factor or product market (as in the case, for example of a labor market intervention).

19
This extrapolation to other settings and populations can be done only under clear assumptions (Heckman, Vytlacil 2006): (i) The results hold only for a given support of the distribution of environmental variables _F_ ( _X_<sup>_t_</sup> _o_ ) and can be extrapolated only to other areas that have the same support on these socio-economic characteristics. (ii) Under our identification assumption, selection into different lengths of exposure is exogenous, conditional on the observable characteristics _X_<sup>_t_</sup> 0 , ‘treatment’ is independent to unobservable. In order to extrapolate the results of the DTE to other areas, one would have to assume that the relationship between observables and unobservables are the same across the current intervention areas and the new potential target populations.

# **6. Results**

The first step to estimate the return to program duration of exposure is to estimate the probability of a site being exposed for differential durations, as a function of the key observable factors that were used in the program site selection process. The estimated marginal effects of propensity score model (presented in table 4) summarize the main features of how the program was phased-in over time. Among these features, accessibility and the characteristics of the NGOs play the most important role. Sites located in more accessible communes and in more accessible villages within communes were chosen to participate in the program earlier. These ‘early’ sites were also relatively “better-off” as indicated by the coefficient estimate on the poverty rate of the commune. The NGO characteristics and their interaction with the remoteness of the sites are jointly significant, and the sites operated by larger NGOs and the NGOs working on environment or agriculture sectors were chosen to start earlier, and relatively more so in more accessible sites. On the other hand, the NGOs with more experience in the district before their partnership with the program began with the relatively more remote areas. The weather shock variables are jointly significant, reflecting the timing of addition of new districts for cyclones and droughts in 1999-2000.

# **6.1 Returns to the Length of Program Exposure**

Table 5 reports the main results of the estimated treatment impact of the length of program exposure on the malnutrition status of children in three different age groups, where the impacts are measured at five different quarters. The returns to one year of additional exposure are positive and significant for the younger age groups.

20
Put it differently, living in sites that participate in Seecaline for one extra year, the 0-6 months old children experience an average of 8% reduction in malnutrition rate while the corresponding figure is 4% for the 7-12 months old children. When the extra year of program exposure increases to two, children in all three age groups enjoy an average of 8% reduction in malnutrition rate. With an average initial malnutrition rate of 46 percentage points, an average of eight percentage points reduction in malnutrition rates for all age groups amounts to around 20% of the initial malnutrition rate. These results are even more substantial for the younger children, whose average initial malnutrition rate is lower to begin with. In sum, this large-scale community-based nutrition program exert a positive and (statistically and economically) significant impact on all three age groups for the two lengths of program exposure.

The gains to program exposure for the 13-36 months old children materialize only after two extra years of exposure. Sites that have two additional years of exposure have impacts that are comparable across all three age groups. We interpret this pattern as a reflection of the _composition effect_ : With shorter duration of program exposure, younger children, relatively to their older counterpart, live a longer fraction of their lives in sites that are covered by the program. As sites continue their participation in the program, even the older children get exposed to the program for a longer fraction of their lives. Put it differently, it takes the 0-6 months old children six months to receive the “full dose” of treatment, defined as the situation where all children live their entire lives under the program. Therefore, when living only in sites that expose for one extra year, the majority of the children in the 1336 months old group have lived their lives without the program.

Another notable feature of the results is that the estimated program impact decreases over time. This decrease in estimated impact is confirmed by looking at the same quarter in different years, which eliminates the potential confounding effect of seasonality of malnutrition rates. A possible explanation is the “ _catching-up_ ” argument: Recall that the monitoring data only collects information on the participating sites. Although both the treated and potential comparison sites always remain one and two years of difference in the length of exposure to the Seecaline program, all of these sites increase their program participation durations as we follow them over time. As time progresses, sites with a shorter duration “ _catch up_ ” (at least partially) to a level of nutritional status closer 21
to that of sites with the longer duration. As the potential comparison sites also started to receive the program services, the nutritional status of their children will improve.

In our data, we observe that this estimated impact is decreasing over time but does not converge to zero, even for those age groups who are fully exposed. This finding is consistent with the hypothesis that the return to the length of exposure decreases as the length of exposure increases; that is, the health production function is concave in the length of exposure.

# **6.2 Impact Heterogeneity**

After investigating the overall impact of the program, we examine how the program impact varies with some of the important aggregate environmental factors underlying the health production function. These factors include poverty, incidence of cyclones, and the length of the lean season. Table 6 reports their pair-wise correlations while table 7 presents the results of the estimated DTE for the subgroups of each of the covariate of interest.

Sites with a higher poverty rate have higher returns to exposure over two years for all age groups relatively to sites with lower poverty rates (panel 7a). Panels 7b-d of table 7 present all of the estimates related to variables affecting morbidity. Sites that are attacked more by cyclones and sites that experience a longer lean season have higher DTE. The systematic higher returns are evident over durations of extra two years, especially for children aged 7-12 and 13-36, who are the age groups that are more susceptible to diseases. The results on accessibility to safe water, however, do not exhibit any significant impact, but we suspect that our measure of access to water in the data is not perfect: It applies only to the presence of public wells<sup>25</sup> only in the main village within the commune, so the actual access to piped water could be quite heterogeneous across villages within the same commune.

Taken together, these subgroup analyses show that the differential returns to exposure are higher in areas with scarcer resources and areas that are more vulnerable to diseases. The marginal benefits from having (earlier) the

> 25 More specifically, the variable access to safe drinkable water is equal to 1 if the commune has any piped water provided by the national or by a local water utility. There is also very limited information on the relative quality of water provided by the national water utility with respect to the local providers.

22
Seecaline program in the economically disadvantaged environment suggests that if a large-scale nutrition intervention is designed to phase in over time, then it pays to target the needier communities first.

# **7. Conclusion and Discussion**

This paper quantifies the impact of a large scale community-based nutritional intervention targeted at the children below the age of three, lactating women, and pregnant women implemented in Madagascar since 1999. The contribution of the study is to show how to community-level administrative data that is collected as part of the regular process of monitoring program activities can be used for the purpose of documenting the impact of the program. Such exercise can have high potential payoffs in terms of applications to other contexts, since most social programs in developing countries are carefully monitored by a monitoring system. The data produced during this process has two key attractive features that are not shared by household level data: ‘scale’ and ‘time’. Scale implies that monitoring data does represent the entire universe of participants and therefore can be potentially used to characterize the heterogeneity of impact. The time dimension enriches the analysis to be implemented at different point in times, documenting the time evolution of impact. This type of data however comes with its own limitations. First, monitoring data (and administrative data in general) do not generally have a large set of covariates. We argue that this disadvantage can be overcome by complementing these data with census-based data with socio-economic information available at low levels of geographical disaggregation. Second, by its nature monitoring data is observed only for participating communities. This does not allow the researcher to estimate the counterfactual of no participation. However, we argue that monitoring data can be used to define a new parameter of interest that measures the _differential returns to exposure to the program_ .

This parameter is estimated by comparing the malnutrition rates of communities that were exposed to the program for a longer duration with (comparable) communities that were exposed to the program for a shorter period of time. With the data at hand the comparison can be done at different points in time. We exploit the phase-in features of the program to model the selection of communities into different program durations and use the method of matching to estimate the differential effects of the length of exposure on aggregate community malnutrition rates. Our results show that on average there are positive returns to exposure: communities that were exposed to 23
the program for an extra one (or two) years displayed lower malnutrition rates than the communities that had shorter length of exposure. The reduction in malnutrition rates for different age groups takes time to materialize, and it takes up to two years for all of the three age groups (0-6 months, 7-12 months, 13-36 months) to achieve a 7- 9 percentage points reduction in malnutrition rates. With an initial malnutrition rate of around 46 percentage points, this effect on malnutrition rates amounts to 15 to 20 percentage points over a two year period. When looking at the time profile of these effects, we find that the differential returns to exposure decrease over time although they do not converge to zero as time and duration increase.

Furthermore, when examining how the effects of differential length of exposure to the program vary according to some key social and environmental conditions, we find that the differential returns are higher for poorer communities, and communities more vulnerable to cyclones and to a longer lean season.

The results do not address the question of not whether the program should be pursued or not, but rather how it should expanded once it a small scale (internally valid) evaluation has shown promising and significant effects of the program and a country has decided to pursue such an approach on a larger scale. How should the program be phased-in geographically and where are the highest marginal returns from program expansion? Our study shows that the use of monitoring data creates an opportunity for applications in the context of large-scale programs, where the socio-economic characteristics and the potential returns to the program vary greatly across different intervention areas. In this respect, the current analysis can be complemented by the evaluations that are based on household survey data. Indeed, our future research will utilize household survey data to quantify other parameters of interest and highlight how the estimated return to exposure quantified in this study is related to the estimated parameters of interest in our future studies that use the household survey data.

24