metadata: authors: 0: author: Rosas Raffo,Nina
1: author: Sabarwal,Shwetlena
count: Sierra Leone
docna: 0: docna: Can you work it ? evidence on the productive
            potential of public works from a youth employment program in
            Sierra Leone
docty: Policy Research Working Paper
subtopic: Health Care Services Industry,Labor Markets,Labor Policies,Health Service Management and Delivery,Educational Sciences,Rural Labor Markets
teratopic: Social Protections and Labor,Conflict and Development
abstracts: cdata!: This paper examines the short-term
            impacts of a labor-intensive public works program on
            household welfare and economic prospects. Using a
            community-level randomized control trial approach, the paper
            finds that the public works program targeted at youth in
            Sierra Leone successfully provided temporary employment to
            youth characterized by low educational attainment. Cash
            income among program participants increased by nearly three
            times relative to the control counterparts, and treatment
            households experienced a 29 percent rise in monthly income.
            There is also evidence of significant re-optimization of
            household labor allocation and expenditure in response to
            program participation. First, there is an overall
            crowding-in of labor force participation by household
            members beyond program participation. Second, the extra
            income is spent partly to improve the quality of life and
            partly to secure future earnings. The treated households
            raised spending on food, medicines, and assets. They also
            expanded utilization of health services. Meanwhile, the
            consumption of temptation goods was greater, albeit by a
            small amount, and the rate of absenteeism among students was
            higher. To secure future earnings, the treated households
            set up new businesses: they were nearly four times more
            likely than the control households to set up new household
            enterprises. They also boosted their participation in
            informal savings groups and their investments in their homes
            and existing businesses. These results demonstrate that
            public works interventions have considerable potential as
            productive safety nets in post-conflict settings such as
            Sierra Leone. They can provide immediate income support, but
            also open avenues for investment in the productive capacity
            of poor households.
geo_reg_key: 517191,1927320
repnme: repnme: Can you work it ? evidence on the
            productive potential of public works from a youth employment
            program in Sierra Leone
display_title: Can you work it evidence on the
            productive potential of public works from a youth employment
            program in Sierra Leone
listing_relative_url: /research/2016/02/25977554/can-work-evidence-productive-potential-public-works-youth-employment-program-sierra-leone
topicv3: Public Works m1326041 1665,Labor Markets m1326488 855,Education m1327048 470,Health Services Delivery m1327294 1305,Health m1504661 3480
publishtoextweb_dt: 2016-02-25T00:00:00Z
historic_topic: Social Protections and Labor,Conflict and Development
no_of_pages: 33
author_keys: 0: author_key: 1111986
1: author_key: 950739
model_extractions: WPS7580 Policy Research Working Paper 7580

# Can You Work it?

Evidence on the Productive Potential of Public Works from a Youth Employment Program in Sierra Leone _Nina Rosas Shwetlena Sabarwal_ Social Protection and Labor Global Practice Group February 2016
Policy Research Working Paper 7580

# **Abstract**

This paper examines the short-term impacts of a laborintensive public works program on household welfare and economic prospects. Using a community-level randomized control trial approach, the paper finds that the public works program targeted at youth in Sierra Leone successfully provided temporary employment to youth characterized by low educational attainment. Cash income among program participants increased by nearly three times relative to the control counterparts, and treatment households experienced a 29 percent rise in monthly income. There is also evidence of significant re-optimization of household labor allocation and expenditure in response to program participation. First, there is an overall crowding-in of labor force participation by household members beyond program participation. Second, the extra income is spent partly to improve the quality of life and partly to secure future earnings. The treated households raised spending on food, medicines, and assets. They also expanded utilization of health services. Meanwhile, the consumption of temptation goods was greater, albeit by a small amount, and the rate of absenteeism among students was higher. To secure future earnings, the treated households set up new businesses: they were nearly four times more likely than the control households to set up new household enterprises. They also boosted their participation in informal savings groups and their investments in their homes and existing businesses. These results demonstrate that public works interventions have considerable potential as productive safety nets in post-conflict settings such as Sierra Leone. They can provide immediate income support, but also open avenues for investment in the productive capacity of poor households.

This paper is a product of the Social Protection and Labor Global Practice Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at nrosas@worldbank.org and ssabarwal@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
**Can You Work it? Evidence on the Productive Potential of Public Works from a Youth Employment Program in Sierra Leone**

Nina Rosas<sup>1</sup> and Shwetlena Sabarwal<sup>2</sup> _JEL Classification: C93, H53, I31, I38, O15, O20_ _Keywords: Public works, safety nets, social protection, impact evaluation, youth, employment, Sierra Leone_ Acknowledgments: This work has been carried out in collaboration with Sierra Leone’s National Commission for Social Action (NaCSA) and the Ministry of Finance and Economic Development’s Integrated Project Administration Unit. We thank Statistics Sierra Leone for data collection; the field coordinators, Samantha Zaldívar and Joshua McCann, for ensuring high‐quality data; and Lucine Park for excellent research assistance. The paper benefited from the support and guidance of Suleiman Namara and John Van Dyck. We thank Kathleen Beegle, Deon Filmer, Patrick Premand, and Laura Ralston for discussions and comments that improved the paper.

> 1 Social Protection Specialist, World Bank, email: nrosas@worldbank.org.

> 2 Senior Economist, World Bank, email: ssabarwal@worldbank.org.
# **I. Introduction**

Within the antipoverty toolkit, labor‐intensive public works (PW) programs are currently in fashion. As of 2015, 94 developing countries were implementing PW programs, and many of these were fragile and post‐ conflict states.<sup>3</sup> This popularity reflects a shift in the perceived development potential of such programs as more than instruments of mere income stabilization. In a recent review, Subbarao et al. (2013) list four major ways in which these interventions are currently being used in developing countries: (1) the mitigation of covariate shocks, (2) the mitigation of idiosyncratic shocks in response to a temporary or structural jobs crisis, (3) as a bridge to more permanent employment, and (4) poverty relief. This list suggests that PW can, at least theoretically, both create resilience against shocks and generate opportunities for escaping poverty. Is this optimism justified? Can PW programs really offer both resilience and opportunity, particularly in the context of fragile and post‐conflict states?

Theoretically, the justification for using PW to provide poor households with economic opportunities is straightforward. To the extent that PW programs produce greater, more stable incomes even in the absence of adverse shocks, they can directly and positively impact household welfare. This can occur through one or more of the following channels: increased consumption, more investment in human capital, greater investment in productive assets, and expanded opportunities to engage in higher‐risk, higher‐return activities. Furthermore, PW programs often hold significant appeal among governments because of the secondary benefit of creating community infrastructure.<sup>4</sup> Likewise, PW effectively supply contributory transfers, which, given the labor component, are not perceived as handouts. This contrasts with other social protection programs such as pure unconditional cash transfers (Fiszbein et al. 2009). Some argue that there are additional benefits of PW, including low‐ cost design features such as self‐targeting or enhanced social cohesion.<sup>5</sup> However, empirical evidence on these theorized multifaceted impacts is limited. While there are a handful of studies that show the positive impacts of PW on household income and expenditures, direct evidence of the productive potential of PW is relatively scarce.<sup>6</sup> Indeed, some studies seem to suggest that PW programs are not successful in helping participants to secure long‐term employment.<sup>7</sup> Given signs of the growing adoption and ambitious scope of recent programs, it is important to understand the full extent to which these programs impact household welfare, especially because the programs are costly.<sup>8</sup> Not only does their execution require significant administrative effort, which can be a strain on government capacity, especially in developing countries, these programs also potentially have distortionary impacts on local labor markets. Furthermore, if not designed and implemented properly,

> 3 Following the food, fuel, and financial crises of the last decade, a number of governments introduced PW. See World Bank (2015) for a list of countries currently implementing PW programs.

> 4 See Subbarao et al. (2013) for a detailed discussion.

> 5 There is little rigorous evidence to support this claim. See Andrews and Kryeziu (2013); Subbarao et al. (2013).

> 6 For comprehensive reviews of evaluations of PW programs, see Alderman and Yemtsov (2012); Hagen‐Zanker and Leon Himmelstine (2013); IEG (2011); McCord and van Seventer (2004); Subbarao et al. (2013).

> 7 See Betcherman, Olivas, and Dar (2004); Card, Kluve, and Weber (2010); Martin and Grubb (2001).

> 8 One example is the Ethiopia Productive Safety Net Program, which was introduced in 2004 and successfully reached approximately 7.6 million households in five years. More recently, India's Mahatma Gandhi National Rural Employment Guarantee scheme is said to reach approximately 56 million households and is one of the largest antipoverty programs in the world (Subbarao et al. 2013). See also Azam (2012); Dutta et al. (2012) on India and Adams and Kebede (2005); Berhane et al. (2011); Gilligan, Hoddinott, and Taffesse (2008) on Ethiopia.

2
such programs may lack credibility given the potentially large scope for corruption and mismanagement. Understanding whether PW programs are a good public policy choice for fiscally strapped developing countries is therefore important. This paper contributes to this critical area of inquiry. It provides evidence on how precisely PW can impact household welfare in a number of dimensions in the short run. Specifically, it examines the short‐term causal impacts of a PW program in Sierra Leone on household welfare, with a particular focus on household consumption, access to services, and investments in productive assets.

Evaluating the causal impacts of PW in a post‐conflict setting such as Sierra Leone is another key contribution of this paper. PW programs are considered particularly well suited for post‐conflict and other fragile contexts because they can provide immediate short‐term employment to poor households that most likely face tremendous deprivation during and after conflicts. The employment generated by these programs helps address youth employment challenges and ex‐combatant reintegration, which represent pressing concerns for post‐conflict recovery. An added bonus is that PW projects can be designed to rebuild infrastructure damaged during conflicts. Because of these features, PW programs have been quickly launched and scaled up following conflicts in Guinea‐Bissau, Liberia, Nepal, Rwanda, and Sudan (Subbarao et al. 2013). However, the evidence of the effectiveness of PW programs in post‐conflict settings is extremely limited. This is an issue given that such programs are a costly gamble for countries in the middle of post‐conflict reconstruction. These programs could also overstretch the already weak institutional capacity in these countries; effective delivery is thus a crucial concern against a backdrop of poor governance, political instability, and the potential for a return to civil unrest.

Another major contribution of this paper is the use of a randomized control trial approach to measure rigorously the causal impacts of a large PW program. It has been argued that evaluations of PW have often lacked a credible identification of causal impacts (Zimmermann 2013). Most evaluations rely on nonexperimental and, in some cases, quasi‐experimental methods.<sup>9</sup> In contrast, this evaluation exploits the phased design of a national PW program to randomize targeted communities into control and treatment groups and measure the causal impacts of the program during the three or four months following program delivery in treatment communities and before program initiation in control communities.

The results indicate that, in the short run, PW programs implemented in a post‐conflict setting can have important welfare impacts on poor households. The impacts stem from increases in household economic activity, but these increases go beyond direct program participation effects. In fact, PW participation seems to alter overall household labor allocation significantly by crowding in labor market activity among nonparticipating household members, that is, those members not engaged in PW. The share of nonparticipating household members working for cash increased by 24 percent, or 5 percentage points, in treatment households. These enhanced levels of economic activity were also reflected in higher rates of female labor force participation and migration (both in‐ and out‐migration) in treated households.

> 9 One exception is the evaluation of Beegle, Galasso, and Goldberg (2014), who use village‐level and household‐level randomization strategies to analyze a PW program in Malawi.

3
This expanded economic activity translates into higher household incomes among treated households: on average, the total value of reported household cash income during the month prior to the survey rose by 51 percent. However, there appears to be a decrease in in‐kind income received by treatment households, leading to an overall increase in household income of 29 percent. Households use part of the new income on consumption and improved quality of life. They also invest part of it to secure better future earnings.

Overall, there is an increase in household spending on food and medicine. Rural households also spend more on household supplies, while urban households raise their consumption of cigarettes and other tobacco products. Households likewise boost their accumulation of selected assets, especially in rural areas. These assets include durable goods, livestock, and vehicles. The utilization of health services grows among both rural and urban households: treated households report more frequent visits to health facilities and spend more on drugs and medications. However, the utilization of education services appears to decline somewhat: student absenteeism rises in treatment households.

A key result of the paper is the findings on the impacts of PW programs in unleashing productive potential. The treated households increased their participation in informal savings groups. They also exhibited substantially greater investments in home improvements and existing enterprises. However, the biggest boost to the productive potential of poor households resulted from investments in new businesses. The treated households were nearly four times more likely than the control households to set up new household enterprises. Taken together, these results show that PW programs can successfully expand the economic opportunities of beneficiary households.

The remainder of the paper is organized as follows. Section II provides an overview of the program under evaluation. Section III describes the evaluation methodology, including a description of data sources. Section IV provides descriptive analysis. Section V presents the main results of the evaluation. Section VI offers a brief analysis of program cost‐effectiveness. Section VII concludes.

# **II. Program Design and Implementation**

In 2010, the government of Sierra Leone launched the Youth Employment Support Project with the support of the World Bank. The project included a labor‐intensive PW component known as the Cash for Work (CfW) Program, the objective of which was to provide additional income and temporary employment opportunities to vulnerable youth.

The CfW program was targeted at individuals in the 15–35 year age‐group in poor and vulnerable communities. Program beneficiaries were selected through a three‐stage process: (1) geographical targeting to identify the beneficiary communities, (2) the submission of requests by communities to receive program funds for subprojects, and (3) community‐based targeting to identify beneficiary households within the selected communities.<sup>10</sup>

> 10 The geographical targeting was undertaken based on estimates of poverty and food insecurity. The communities were given a list or positive menu of eligible subproject types, namely, (a) feeder road rehabilitation and maintenance, (b) agriculture, and (c) renewable energy and environmental mitigation. A subset of communities submitting requests were then selected based on whether the subproject requested conformed to several requirements, including presence on the positive list, the suitability of the subproject for the locality, and community endorsement. In practice, community‐based targeting relied heavily on self‐ targeting.

4
The National Commission for Social Action (NaCSA), a semiautonomous government agency, had the overall responsibility for implementation of the program. Locally, the program was implemented by independent contractors hired by NaCSA. The contractors were responsible for managing the day‐to‐day implementation of the subprojects, including the procurement of materials and other inputs, recording the attendance of workers, and making payments to beneficiaries for days worked.

To target appropriate individuals within the communities, NaCSA facilitated the establishment of community oversight committees (COCs), which were responsible for identifying the poorest households that included at least one member between 15 and 35 years of age who was willing and able to work. The COCs relied on local definitions of poverty in selecting beneficiaries. The COCs were also tasked with monitoring the progress of work and payments and resolving CfW‐related disputes.

According to the design of the project, the beneficiaries were entitled to a minimum of 50 days and a maximum of 75 days of work at a daily wage rate of Le 7,500, which was equivalent to approximately $1.80 at the time of the evaluation in 2012. The design also stipulated that the wage rate should be set lower than the market wage to discourage nonpoor applicants from participating; however, in practice, ad hoc market studies were used to determine the wage rate. Payments to beneficiaries were made by contractors on a weekly basis based on time sheets kept by the contractors.

Program rollout took place in line with a randomized phase in strategy. In early 2012, 276 communities were identified as potential recipients of the CfW intervention.<sup>11</sup> They were randomly divided into two groups: the treatment group (143 communities), which was scheduled to receive the CfW program during the evaluation period (April–August 2012), and the control group (133 communities), which did not receive the program during the evaluation period. Following the phase in design, the control group was scheduled to receive the program about four months after the treatment group. The randomization was conducted centrally by the authors. The design is summarized in Figure 1 (see Annex 1 for all figures).

One key feature of program implementation was the practice of informal rotation. It is not uncommon for PW programs in Sierra Leone and elsewhere to impose rotation. This is predicated on the principle that every eligible and willing individual in the community should have an opportunity to participate in the program. According to Subbarao et al. (2013), rotation systems are a feature of PW implementation across the world. Where the demand for employment exceeds the opportunities created, programs often opt to give the largest number of poor people a chance to work. This system plays an especially crucial role in post‐conflict settings, where considerations of fairness are central concerns within program implementation. However, rotation is often imposed in an informal and ad hoc fashion, which makes measurement and documentation difficult. Also, if not incorporated adequately into operational procedures, rotation can increase the risk of leakage.

> 11 The overall CfW program included 470 subprojects in four waves, with 108, 143, 133, and 86 subprojects in the first, second, third, and fourth waves, respectively. The randomized phase in was carried out over the second and third waves of the program; the discussion in this paper therefore refers to those two waves only.

5
# **III. Empirical Strategy and Data**

## **A. Empirical Strategy**

Given the randomized phase in of the program, this evaluation relies on a phased in randomized control trial methodology to establish the causal impacts of the CfW program. The advantage of such a phased in trial design is that a simple comparison of the outcomes among the two randomly created groups yields an unbiased estimate of the impact of the CfW program.

The overall evaluation sample is provided by 276 communities: 143 treatment communities and 133 control communities (see above). The study population includes 17,608 beneficiary households, 8,944 in the treatment group and 8,664 in the control group. However, because control communities received the program approximately four months after the treatment communities, the evaluation is able to examine the impacts of the program only in the short run, that is, over approximately three or four months.

In the analysis of program impacts, two sets of analyses were undertaken. First, because all the surveyed households had been selected to participate in the program, intent‐to‐treat (ITT) analysis has been conducted. The impacts of the program on various household‐level outcomes of interest are estimated using ordinary least squares regression (equation 1). The regression used to obtain the ITT estimates is as follows: where _Treat_ is a dummy for assignment to the treatment group, that is, it equals 1 if a household belongs to a community that was randomly assigned to the treatment group and 0 if a household was in a community randomly assigned to the control group.

Despite careful efforts to ensure adherence to randomized assignment, there is a possibility of noncompliance; in other words, households that may have initially signed up for the program may not have participated, or some of the control beneficiaries may have participated. To ensure robustness, estimates have been produced of the effect of the treatment on those households with a treatment status affected by the random assignment, that is, the compliers; this effect is known as the local average treatment effect (LATE).<sup>12</sup> These estimates are the instrumental variable (IV estimates) of _β_ in the equation above, using the dummy for random assignment as an instrument for treatment. However, the results are similar to the ITT estimates because of the low levels of noncompliance: the household surveys indicate that only around 1.8 percent of the treatment households reported they did not participate, and 7.4 percent of the control households reported they participated. Therefore, only a subset of the LATE estimates—the overall estimates—are presented in this paper.

Outlier corrections and log transformations have been conducted for variables in leone values. To reduce the influence of outliers, variables in leone values that were identified with outliers have been winsorized at the 1st and 99th percentiles.<sup>13</sup> The outlier‐corrected estimates and the log‐transformed estimates are presented in Annex 2.

> 12 See Angrist and Imbens (1995); Angrist, Imbens, and Rubin (1996).

> 13 Outliers were identified using the blocked adaptive computationally efficient outlier nominators algorithm proposed by Billor, Hadi, and Velleman (2000), using the 15th percentile of the chi‐squared distribution as a threshold to separate outliers from

6
# **B. Data**

One challenge in collecting data for the evaluation has been that PW beneficiaries within communities— people who would be working on specific CfW subprojects—were not pre‐identified. Instead, they were selected on the first day of the subproject, largely on a first‐come, first‐served basis, subject to meeting the eligibility criteria. This precluded the possibility of simultaneous baseline data collection in the control and treatment households.

Data were collected in three phases: (1) data collection for beneficiary tracking, (2) data collection through unannounced site visits at CfW projects in the treatment communities, and (3) an endline household survey. All data collection was carried out by Sierra Leone’s national statistical agency, Statistics Sierra Leone. The administrative data that were used in the community (subproject) randomization process were collected and maintained by NaCSA for operational and monitoring purposes.

To enable the documentation and follow‐up on treatment and control households, beneficiary tracker surveys were administered on the first day of implementation for CfW treatment (April 2012) and control (July 2012) subprojects. These trackers were the basis on which treatment and control households were identified and interviewed for the endline survey (July–August 2012). The tracker surveys collected basic demographic information from beneficiaries, contact details so they could be tracked during subsequent survey rounds, and information on beneficiary perceptions of the program and its processes prior to beginning work at the sites. The surveys were administered at 276 sites among a total of 17,670 potential beneficiaries.

Unannounced observational visits were fielded midway through the physical implementation of the subprojects at 141 of the 143 treatment sites (May–June 2012).<sup>14</sup> This survey was conducted in two parts that were designed to collect information on overall program implementation and specific processes, as well as the program knowledge and satisfaction among beneficiaries. Part 1 consisted of observations (including a worker roll call, checks of attendance, and other records) and an interview with a contractor representative on site; part 2 consisted of an interview of one man and one woman randomly selected among the beneficiaries. A total of 279 beneficiaries—3 percent of the 8,883 people working on the 141 subprojects—were interviewed, around half of whom were women.

The endline household survey was administered concurrently among treatment and control households: at the end of implementation for the treatment group and immediately before implementation for the control group (July–August 2012). Thus, although the households in the control subprojects had already been identified for enrollment in the program, they had not yet started working. The endline survey was administered to an average of 20 beneficiary households in each of the 275 subprojects, for a total of 5,506 beneficiary households.<sup>15</sup> The survey covered a range of topics, but focused on measuring the program effects along the following dimensions: (1) labor market outcomes and economic activity; (2) household assets, consumption, and savings; and (3) the utilization of education and health services.

> nonoutliers. For the Winzorization, values above the 99th percentile of the distribution and values below the 1st percentile were replaced with the 99th percentile and with the 1st percentile of the distribution, respectively. Food expenditure and savings were winsorized at the 3rd percentile, considering the volume of outliers.

> 14 Two treatment sites were inactive at the time of the unannounced visits.

> 15 One subproject in the Western Area was canceled during the implementation of the impact evaluation (IE).

7
# **C. Measurement Issues and Possible Threats to Identification**

The absence of baseline data across the treatment and control communities is not considered a serious limitation on causal identification. Power calculations suggest that a randomized cluster design that is clustered by communities (CfW subprojects), with 276 communities, is sufficient to ensure a balance between the treatment and control households, thereby guaranteeing the comparability of the two groups for the purpose of causal attribution. Nonetheless, the variables collected during the endline surveys that were not expected to change over the short duration of the program provide evidence— albeit limited—of pre‐intervention balance between the treatment and control groups. As shown in Table 1, the overall treatment group is statistically comparable with the control group in most of the variables tested (see Annex 2 for all tables).

Another measurement issue relates to the practice of rotation (described in section II) that was applied in an informal and largely undocumented manner among the treatment communities. This has implications for the measurement of program impacts because program impacts were measured through registered beneficiaries who had been identified on the first day of subproject implementation. If this informal rotation occurs, the impacts captured by the impact evaluation (IE) may be under‐ or overestimated. For instance, registered beneficiaries may be more well‐informed or have stronger networks than nonregistered beneficiaries. This may lead them to exhibit better outcomes even in the absence of the program, and the IE would then overestimate the program’s true effects. Conversely, nonregistered beneficiaries may be poorer and may enjoy higher returns to receiving the transfer (for instance, through lower substitution effects). In this case, the IE may underestimate the true impact of the program. Another cause of potential underestimation is the fact that, owing to informal rotation, some beneficiary households worked fewer days than intended in program design, leading to an underestimation of the welfare impacts of the program. Because of informal rotation, actual program impacts may have been diluted given that the same total transfer amount was distributed across a larger number of beneficiaries.

The data collected during the unannounced site visits indicate that, on average, 13.1 percent of beneficiaries on site at the time of the visits were not registered beneficiaries. Similarly, time sheet records show that, on average, 13.6 percent of the workers listed on the time sheets were not registered beneficiaries. On average, there were 65 registered beneficiaries per subproject and 10 nonregistered beneficiaries listed on time sheets. This is not definitive evidence of informal rotation because these nonregistered beneficiaries may have simply been alternates, that is, replacements from the same household who had been nominated to work in the event of absence by the beneficiaries. Nonetheless, at least a portion of these additional workers were likely to be from households not captured by the IE. Discussions with administrative staff confirm that this was a practice at least in some sites despite attempts to prevent it.

Program impacts have been measured through the endline survey administered simultaneously to treatment and control households shortly before the CfW was launched in control communities. At the time of the endline survey, control communities were aware that they had been approved for a CfW subproject in the coming months (see above). It is plausible that this knowledge could have influenced the survey responses among control households. Specifically, control households might have strategically underreported their incomes and assets to influence the perception that they are poor and, hence, eligible for program participation. If this is the case, the IE may have overestimated program impacts. However, 8
the likelihood of systematic underreporting of incomes and assets by control households is considered low because (1) the data collection team did not reveal its connection with the CfW program, but, rather, identified itself as a national statistical office team that was conducting a socioeconomic survey; and (2) the endline survey took place in the days immediately after the control households had been registered for the CfW program; so, there was no incentive to underreport to influence the selection process. To the extent that control households had an incentive to underreport to participate in future rounds of the CfW or other programs, the treatment communities would have been subject to the same level of underreporting as the control communities.

Another possible concern relates to anticipation effects on the part of the control households. The control households could have made certain decisions (especially in relation to intrahousehold labor allocation) or delayed certain investments (for example, in new businesses) in anticipation of the CfW program, which would have led to the over‐ or underestimation of program impacts. The possibility of anticipation effects is considered low given that there was no systematic announcement or publicity related to the CfW program well in advance of the program launch among treatment or control communities. There is, in general, also considerable unpredictability associated with the delivery of government programs in Sierra Leone so that it is unlikely households would re‐optimize in anticipation of a government program.

# **IV. Descriptive Analysis**

## **A. Subproject Characteristics**

The community‐specific infrastructure projects organized within the CfW program are referred to as subprojects. The CfW subprojects had national coverage and were spread fairly evenly across urban and rural areas. Figure 2a shows the geographical distribution of all the subprojects covered under the evaluation; Figure 2b shows the precise location of the treatment sites based on Global Positioning System data.<sup>16</sup> The most common types of subprojects are feeder road rehabilitation projects (67 percent), inland valley swamp rice projects (9 percent), and other agricultural projects (13 percent).<sup>17</sup> The average labor intensity of the subprojects—the share of subproject costs allocated toward wages—was 60 percent, but there was some variance; road subprojects were typically less labor intensive. Given that the program was put in place following the food and fuel price crisis, seasonal considerations were not detailed in the project documents (World Bank 2010); however, where possible, the program aimed to align the timing of subproject implementation with seasonality concerns. In practice, the treatment subprojects were implemented between April and July or August, which coincides with the end of the agricultural harvest period and the beginning of the lean season.

## **B. Beneficiary Characteristics**

The data from the beneficiary tracker survey indicate that the CfW program reached primarily youth with low levels of educational attainment who were working in the agricultural sector. The program also met the gender targets and did not induce negative impacts on schooling among the young participants. While in this sense the program succeeded in reaching the target population, targeting performance was weaker

> 16 Global Positioning System data were collected only from treatment sites.

> 17 Inland valley swamp rice projects refer to the establishment of a specific type of rice production system.

9
in attracting the poorest youth to participate, largely because of overreliance on self‐targeting (Figure 3).<sup>18</sup> The average age among beneficiaries was 27 years, and 92 percent of the beneficiaries were between the ages of 15 and 35, and, hence, well within the eligible age‐group of the CfW program. The program design emphasized adequate participation (a minimum of 30 percent) among women beneficiaries, and this goal seems to have been met: the average share of women among the beneficiaries was 33 percent.

The main occupation outside the CfW program of nearly half the beneficiaries (49 percent) was in the agricultural sector. Other top occupational categories outside the program included students (15 percent), street sales and related sales and services (10 percent), and building and related trades (4 percent). More than half the beneficiaries (56 percent) had not engaged in any remunerated work in the month prior to participating in the program. Among the 44 percent who had paid work outside the program, the average daily earnings were $2.50, and the majority (75 percent) were self‐employed. Almost half the beneficiaries (46 percent) reported they were engaged in unpaid family farm work.

Educational attainment was fairly low among the CfW beneficiaries: 52 percent had less than primary education, that is, incomplete primary or no schooling; 35 percent had completed primary school; and only 12 percent had completed secondary school or above. The average level of education was lower among women beneficiaries. Nearly 60 percent of the women had no education, compared with about 35 percent of the men.

A potential concern with PW programs targeted on youth is they may attract youth who are attending school, leading them to drop out. Our data suggest that, for the most part, the program did not lead youth to drop out of school to enter the program: only 3 percent of the beneficiaries who had ever attended school reported they had stopped attending to enter the program.

# **V. Results**

This section outlines the program impacts on various dimensions of household welfare, including employment, migration, income, consumption, assets, access to health and education services, and savings and investment. Data gathered during the unannounced site visits as well as discussions with administrative staff indicate that informal rotation took place in at least some CfW sites (see subsection III.C). This implies that a nontrivial share of households did not participate in the program for the entire duration and, therefore, that the estimated program impacts may represent the lower bounds.

## **A. Impacts on Economic Activity**

The CfW program had substantial impacts on overall household economic activity (Table 2). In itself, this is not remarkable given that the program provided employment to beneficiary households, and part of the increased economic activity in treatment households can thus be linked to pure participation effects. However, the impacts observed exceed the effects of mere participation and arise from the crowding in of nonparticipant household members in the labor market.<sup>19</sup>

> 18 See Rosas (2016) and Rosas and Solbes (2013) for further discussion.

> 19 Nonparticipating household members refers to those members of beneficiary households who are not participating in the PW program. Program rules restrict PW participation to only one eligible member per household, and these beneficiaries are identified during data collection in the household roster.

10
Overall, the members of treatment households were significantly more likely to be engaged in remunerated work (Table 2, column 1). The share of household members who had worked for cash in the last 12 months was about 11 percentage points higher in treatment households than the control mean of 33 percent.<sup>20</sup> The share was slightly more pronounced among rural households than urban households (12 percent vs. 10 percent).

Program participation also crowds in labor force participation among adult household members over and above the direct PW program participation. In treatment households, the employment of nonparticipating household members increased (Table 2, column 3).<sup>21</sup> The share of nonparticipating household members who had worked for cash during the last 12 months was 28 percent in the treatment group and 22 percent in the control group. The rise in the share of household members who were not participating in the program, but who were participating in paid work was larger in rural areas than in urban areas (7 percentage points vs. 4 percentage points). There was no evidence of short‐term impacts on the incidence of child labor in rural or urban areas.

Participation in the program led to a net increase in labor force participation among women in treatment households (Table 2, column 2). This is also not remarkable given that the CfW program rules included a predefined quota of at least 30 percent participation by women. About 46 percent of the women in treatment households had worked for cash in the last 12 months, compared with 35 percent in the control households. The rise in female labor force participation within households was almost double in rural areas compared with urban areas (14 percentage points vs. 8 percentage points) even though the two areas started at a similar control mean of 35 percent.

One interesting auxiliary impact of program participation was an increase in reported migration among the treated households (Table 3). A rise in both in‐ and out‐migration—individuals moving, respectively, into or out of households from another town or city—was observed, but the overall impacts were small. Among both rural and urban households, out‐migration was somewhat greater than in‐migration, although the difference was wider in rural households. Greater migration seems intuitively like a natural outcome of expanded economic potential in areas where job opportunities are scarce.

**B. Impacts on Household Income, Expenditure, Savings, and Investment**

# **_B.1 Income Effects_**

Given the large program impacts on household economic activity, the clear income effects are not surprising. The treated households had higher reported income: on average, the reported cash income during the previous month increased by 51 percent. However, the value of reported cash and in‐kind income taken together rose by only 29 percent (

> 20 In this paper, all recall periods are measured from the date of the household survey, unless indicated otherwise.

> 21 Substantial increases were observed both in households in which the beneficiary is a man and in households in which the beneficiary is a woman.

11
Table 4, columns 1–2), signaling a decline in the in‐kind payments received by treatment households, presumably from changes in the overall composition of the labor market activities undertaken by household members. In line with the results on economic activity, rural households exhibit larger income effects than their urban counterparts.

Given that the IE relied on randomization at the subproject level, it has not been possible to generate individual ITT estimates of the income effects among program participants, that is, those household members working in the CfW program. However, a rough estimate can be obtained using propensity score matching to identify comparable individuals in the control communities.<sup>22</sup> Based on this approach, the program is estimated to have nearly tripled monthly cash incomes among participants (a rise by a factor of 2.6, or an increase of Le 81,070) compared with their control counterparts.

The average increase in cash incomes among participants was less than a third of what the participants were entitled to receive under the program over the period under analysis.<sup>23</sup> This result holds at the household level: the average increase in cash incomes among treatment households was only about two‐ thirds of what these households should have received. Based on the impacts observed on household economic activity, it is clear that CfW participation crowds in employment among treated households. This discrepancy cannot therefore be attributed to intrahousehold substitution effects in terms of labor market participation, that is, a reduction in the participation in paid work among non–program participants within the household. In addition, given the low share of beneficiaries who were doing any paid work before the program was introduced, individual substitution effects (that is, forgone earnings) among CfW participants are likely to have been low.<sup>24</sup> The discrepancy is more likely to be linked to the practice of informal rotation or to leakage. Rotation is a practice whereby a subproject accepts more beneficiaries working fewer days than it is designed to do, typically because, socially, this approach is perceived to be fairer (see above). This had been a common practice in previous rounds, and, although the data do not allow ascertaining the exact extent to which rotation occurred, the data do provide an indication that it occurred. As with any program providing cash or in‐kind transfers, there is also the possibility of leakage: money reaching individuals who are not the intended beneficiaries. While the IE found no concrete evidence of payment leakage, the program implementers have acknowledged that the PW program management and payment arrangements in place during the IE represented a high risk of leakage.<sup>25</sup>

**_B.2 Impacts on the Consumption of Goods and Services_**<sup>**_26_**</sup>

B.2.1 Asset Accumulation and the Consumption of Goods The efficacy of PW programs as antipoverty instruments depends in part on how the income effects influence household spending and consumption patterns. Unfortunately, the time frame of the evaluation

> 22 The propensity score—nearest neighbor matching—is used to generate the matching estimates through a probit of beneficiary status on age, age<sup>2</sup> , gender, education, literacy, marriage status, and whether the individual is the household head.

> 23 Between 43 percent and 65 percent, depending on whether the minimum or maximum days of work are applied. Beneficiaries were entitled to earn between $90 and $135 equivalent over a three‐ to four‐month period.

> 24 These cannot be accurately measured because of rotation practices and the implications for attendance tracking.

> 25 Contractors were responsible both for recording the time worked and for making cash payments to beneficiaries.

> 26 Given the differences in the log and level consumption estimates, both are presented in Tables 5 and 6. For simplicity, only the most conservative estimate among these two is reported in the main text. For robustness, only estimates that are significant at the 10 percent level or below in both logs and level are reported.

12
prohibits us from looking at these patterns in the longer term. Nonetheless, the short‐run impacts are suggestive and can provide important insights into the potentially longer‐term dynamics.

For consumptions goods, a short consumption module was administered to surveyed households to capture program impacts on key expenditure categories, including utilities (electricity, fuel), food, children’s schooling, hygiene, home improvements, transfers by the household to individuals outside the household, and some temptation goods. The results show that the program had positive impacts on the spending of beneficiary households on food in line with the CfW design as a mechanism to support beneficiary households in meeting food needs, particularly in the face of rising food prices. Treatment households reported spending 8 percent more on food than control households in the month previous to the survey (Table 5, column 3).

There was a statistically significant increase among urban households in the consumption of cigarettes and other tobacco products.<sup>27</sup> However, the increase was small in economic terms, equivalent to $0.20, or less than 1 percent of the average monthly cash income of the control households. As expected, the impacts on household consumption patterns appear to have varied systematically across rural and urban areas. Rural households registered higher expenditure on household supplies and, to a lesser extent, clothing for girls. There was a decline in household spending on fuel, especially in rural areas. In urban areas, there were declines in interhousehold transfers (about 27 percent) and in household spending on schooling for both boys and girls (25 percent and 23 percent, respectively).

The treatment households appear to have invested in household assets (Table 7). Rural households, in particular, boosted their ownership of consumer durables, motor vehicles, livestock, and telephones. There was also a significant decline in the value of jewelry owned by households, especially urban households, which may be linked to investments in new businesses (see Table 10). The overall patterns suggest that, in the short run, higher income leads, in part, to greater spending on welfare‐enhancing goods, but also, to some extent, on temptation goods such as cigarettes and other tobacco products. B.2.2 Health Services Program participation produces positive impacts on beneficiary household utilization of health services, particularly in favor of young male children (Table 8). On average, the treated households undertook more health facility visits than the control group (by 12 percent). The proportion of boys aged 0 to 5 who were taken to a health facility when they were sick was higher by 9 percent (8 percentage points) in the treatment households. The increase was even greater (23 percent) among all boys aged 0 to 5 irrespective of their health status at the time they were taken to the doctor, indicating the program expanded routine health checkups for these children.

In addition, program participation led to greater spending on drugs and medications. The treated households reported spending 16 percent more on drugs and medications than the control group during the month previous to the survey.

The intervention did not have any design features intended to encourage more health‐seeking behavior. Nonetheless, it seems that higher income generated through PW program participation may have relieved

> 27 This contrasts with existing evidence on the impacts of cash transfers. A recent review shows that cash transfers lead to either no increase or a significant decrease in household spending on temptation goods (Evans and Popova 2014).

13
financial constraints that beneficiary households faced in accessing health services. The expansion in health‐seeking behavior was observed in both rural and urban households.

B.2.3 Education Services In contrast with the program impacts on access to health services, CfW participation had no systematic positive impact on access to education (Table 9). Indeed, student absenteeism appears to have risen among treatment households. The average number of school days missed in the four weeks previous to the survey increased by nearly 51 percent (0.2 days). Also, expenditure data show that there were declines of about 27 percent in household spending on girls and boys schooling in urban areas (see Table 6, columns 9‐10). However, at least in the short run, treated households did not report lower school enrollments among children, only lower school attendance rates.<sup>28</sup> At the same time, there was no expansion in children’s labor market participation. One possible explanation for the rise in school absenteeism is that, because of the substantial increase in labor market participation among adult household members, school‐going children may have been skipping school to help with household chores or to take care of younger siblings. While there was no observed increase in the time children spent on household chores, the IE captured a relatively narrow set of data on time use and the distribution of household responsibilities among members.

# **_B.3 Impacts on Savings and Investment_**

Part of the income gains from CfW participation were clearly being used for welfare‐enhancing consumption (food, household supplies, and consumer durables), especially among rural households (see above). However, the longer‐term poverty reduction potential of this instrument is directly linked to the extent to which CfW income is used to enhance the productive potential and overall resilience of targeted households. It is on this dimension that some of the most interesting program impacts emerge, despite the fact that the CfW program’s emphasis was on temporary employment rather than on boosting the productive capacity of poor households.

First, there were positive impacts on savings. Among treated households, participation in informal savings groups ( _osusu_ ) increased by 16 percent (4 percentage points; Table 10, columns 2). However, there were no discernable impacts on the likelihood of having a formal savings account. Furthermore, the result was driven primarily by the impact in rural areas, as there was no statistically significant change observed in urban areas. The total monetary value of reported savings also rose substantially among rural households (by about 25 percent).

Various types of investments were examined, and a pattern emerges of increased investment in homes and businesses among treated households (Table 10, columns 6‐9). Investments by treated households in home improvements rose by 33 percent.<sup>29</sup> There was also a 39 percent expansion in investments in existing businesses in rural areas. These findings validate the argument that PW programs may promote resilience against future shocks.

> 28 Higher absenteeism may signal higher drop‐out rates in the medium term given that the endline survey was conducted in July– August and the new school year starts in September. 29 This is consistent with findings from an unconditional cash transfer program in Kenya where treatment households increased their ownership of metal roofs appreciably (Haushofer and Shapiro 2013).

14
One of the starkest impacts was in terms of investments in new businesses. The treatment households were nearly four times more likely than control households to establish new enterprises: only 9 percent of the control households reported that household members had set up new enterprises during the previous three months, compared with 34 percent in the treatment group. Petty trading was the most common type of new enterprise in both urban and rural areas among both the treatment group and the control group.

# **VI. Cost‐Effectiveness**

Since the primary goal of the program was to increase access to short‐term employment opportunities by providing an income supplement for vulnerable youth through PW, cost‐effectiveness analysis was conducted on two main outcomes: (1) the net number of temporary jobs created and (2) the net increase in household incomes. The analysis broadly follows the methodology outlined in Dhaliwal et al. (2011), which seeks to measure the effect a program achieves in terms of outcomes of interest at a given cost. The ITT impact estimates used to calculate the cost‐effectiveness ratios are significant at the 1 percent level. Given that existing evaluations of PW programs are limited and that cost‐effectiveness estimates are even scarcer, particularly in the Sub‐Saharan African context, this paper does not provide comparisons with other programs. Nonetheless, these estimates are presented to aid future comparisons.

The average cost per subproject was approximately $16,000, of which an average 60 percent was allocated toward the payment of wages, while the remaining 40 percent covered contractor fees, materials to implement the works, and COC administrative costs. In addition to these costs, NaCSA incurred operational costs in the range of 10 percent to 15 percent of the subproject costs for implementation support and the monitoring of subproject design and implementation. This implies an average total cost per subproject of between $17,600 and $18,400 or an aggregate cost over the 143 treatment communities of between $2.5 million and $2.6 million. For simplicity, the cost‐effectiveness ratios are calculated based on the higher share of administrative costs, at 15 percent.

The ITT estimate of net job creation beyond the jobs directly created by the PW program yields a total of 4,343 jobs in the 143 treatment communities. Combining this with the 8,944 jobs directly created by the program in these communities implies a cost of approximately $198 per temporary job created, including all administrative costs. If account is taken only of the share of the cost designated as direct transfers to beneficiaries, the cost per temporary job created would be $103. The ITT estimate of the impact on net household income was Le 86,078 (approximately $21) or a total of Le 769,881,632 (approximately $185,000) when aggregated across the 8,944 treated households in the 143 treatment communities. Including all administrative costs, this implies that about every $14 spent on the program generated an additional $1 of household income. Excluding all administrative costs, the cost‐effectiveness ratio for net income gains would be about half this amount. Thus, each $7 designated as a direct transfer to beneficiaries generated an additional $1 of income.

These cost‐effectiveness estimates exclude the spillovers to nonregistered beneficiaries that occurred because of informal rotation practices as well as program leakage. These estimates also capture only the pure labor and economic welfare effects and exclude other important secondary impacts on welfare, particularly the impacts on social outcomes such as improved intrahousehold bargaining and greater utilization of health services. They can therefore be considered conservative estimates of the program’s 15
impacts had the program been implemented as designed. In addition, because only about 55 percent of the total costs were intended to go to beneficiaries in the form of direct transfers, the impacts could be enhanced by improving administrative efficiency.

# **VII. Conclusions**

PW programs have become a popular policy instrument for protecting the poor from income shocks. Can they also improve the overall productive capacity of beneficiary households? To answer this question, this paper examines the short‐term impacts of a PW program in Sierra Leone targeted at youth in poor and vulnerable communities. The phased implementation of the program was exploited to implement a community‐level randomized control trial to help measure the causal impacts of the program on household outcomes over three or four months.

In keeping with the design, the program succeeded in attracting young people (aged 15–35) with low educational attainment who were predominantly working in the agricultural sector. Furthermore, it was successful in impacting targeted households through the anticipated employment channels. Treatment household members were substantially more likely to become engaged in remunerated work. These employment impacts were reflected in household incomes, which rose by 29 percent among treatment households.

Beyond the pure participation effects, the program impacted household welfare in the short run in three key ways. First, there was crowding in of labor force participation among treatment households beyond the direct program participation. Second, part of the additional income was spent on greater consumption to improve the quality of life. Treated households in rural areas spent more on household supplies, clothing, durable goods, and medicines, while treated households in urban areas spent more on food and medicines. There were also positive impacts on the utilization of health services, particularly in favor of young male children. Third, part of the extra income was invested to boost household productive capacity. Participation in the CfW program increased the likelihood of enterprise creation among households by a factor of nearly four. Furthermore, program participation also increased household asset accumulation, the participation in informal savings groups, and investment in home improvements as well as businesses.

However, there were also negative program impacts. Treatment households augmented their consumption of temptation goods, notably cigarettes and other tobacco products, even if by a small degree. Within the short time frame of the study, school absenteeism was higher among the children in treatment households. On the other hand, there was no increase in the participation of children in the labor market or in time spent on household chores, suggesting that other activities not captured by the IE may have been at play.

Taken together, these results suggest that, even in fragile and post‐conflict settings, PW programs can strongly impact household welfare in the short run. In Sierra Leone, the CfW program catalyzed major impacts on nearly all key aspects of household welfare and decision making: employment, income, migration, consumption, savings, investment, health care, and education. Households appear to re‐ optimize their labor allocation and spending substantially in response to PW participation and use the additional income not only to improve the quality of life, but also to boost future earnings.

16