metadata: authors: 0: author: Bagga,Aanchal
1: author: Holmlund,Marcus Erik
2: author: Khan,Nausheen
3: author: Subha,Mani
4: author: Mvukiyehe,Eric
5: author: Premand,Patrick
count: World
docna: 0: docna: Do Public Works Programs Have Sustained
            Impacts ? A Review of Experimental Studies from LMICs
docty: Policy Research Working Paper
abstracts: cdata!: Many low- and middle-income countries
            (LMICs) have introduced public works programs that offer
            temporary cash-for-work opportunities to poor individuals.
            This paper reviews experimental evidence on the impacts of
            public works programs on participants over the short and
            medium run, providing new insights on whether they have
            sustained impacts. The findings show that public works
            mainly increase employment and earnings during the program.
            Short-term positive effects tend to fade in the medium run,
            except in a few cases in which large impacts on savings or
            investments in self-employment activities are also observed.
            Importantly, the estimated impacts on earnings are much
            lower than planned transfer amounts due to forgone earnings,
            raising questions about cost-effectiveness. There is also
            little evidence of public works programs improving food
            consumption expenditure. The review finds evidence of
            improvements in psychological well-being and
            women&#x2019;s empowerment in some cases, but not
            systematically, and with limitations in measurement. The
            paper concludes by outlining directions for future research.
display_title: Do Public Works Programs Have
            Sustained Impacts ? A Review of Experimental Studies from LMICs
listing_relative_url: /research/2023/06/34075527/public-works-programs-sustained-impacts-review-experimental-studies-lmics
topicv3: Labor Markets m1326488 855,Social Protection and Labor m1327194 697,Jobs m1503931 2670
publishtoextweb_dt: 2023-06-06T00:00:00Z
dois: 10.1596/1813-9450-10471
origu: Off of Sr VP Dev Econ/Chief Econ (DECVP)
author_keys: 0: author_key: 1949395
1: author_key: 997187
2: author_key: 1484200
3: author_key: NA
4: author_key: 972271
5: author_key: 921098
model_extractions: Policy Research Working Paper 10471

# Do Public Works Programs Have Sustained Impacts?

A Review of Experimental Studies from LMICs _Aanchal Bagga Marcus Holmlund Nausheen Khan Subha Mani Eric Mvukiyehe Patrick Premand_ Development Economics Development Impact Evaluation Group June 2023
Policy Research Working Paper 10471

# **Abstract**

Many low- and middle-income countries (LMICs) have introduced public works programs that offer temporary cash-for-work opportunities to poor individuals. This paper reviews experimental evidence on the impacts of public works programs on participants over the short and medium run, providing new insights on whether they have sustained impacts. The findings show that public works mainly increase employment and earnings during the program. Short-term positive effects tend to fade in the medium run, except in a few cases in which large impacts on savings or investments in self-employment activities are also observed. Importantly, the estimated impacts on earnings are much lower than planned transfer amounts due to forgone earnings, raising questions about cost-effectiveness. There is also little evidence of public works programs improving food consumption expenditure. The review finds evidence of improvements in psychological well-being and women’s empowerment in some cases, but not systematically, and with limitations in measurement. The paper concludes by outlining directions for future research.

This paper is a product of the Development Impact Evaluation Group, Development Economics. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at smani@fordham.edu and ppremand@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
**<mark>Do Public Works Programs Have Sustained Impacts? A Review of Experimental Studies from LMICs</mark> ***

Aanchal Bagga, Tufts University** Marcus Holmlund, DIME, The World Bank Nausheen Khan, DIME, The World Bank Subha Mani, Fordham University and IZA Eric Mvukiyehe, Duke University Patrick Premand, DIME, The World Bank Keywords: Public works programs, Experimental evidence, Low- and middle-income countries, Sustainability, Social protection, Safety nets, Employment.

JEL codes: H41, C93, O12, J22, I38 <mark>*We thank the authors of the various studies reviewed in this paper, as well as Samih Ferrah, Christelle Tchoupe, Jules Gazeaud, Oliver Sterck, Raphael Cottin, Issa Kiemtore, Brooke Hill, and Horacio Vera Cossio for their excellent research assistance during the implementation of the public works studies across countries. We specifically thank Girum Abebe, Ugo Gentilini, S</mark> amuel Kembou Nzale, <mark>Elizaveta Perova, Hrishikesh Tirumala Madabushi, Michael Weber, and an anonymous reviewer from DIME for detailed comments and suggestions on an earlier draft.</mark> Funding from the World Bank Jobs, UFGE and i2i Trust Funds is gratefully acknowledged. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the World Bank and its affiliated organizations, the Executive Directors of the World Bank, or the governments they represent. <mark>**Aanchal Bagga (e</mark> mail: <u>Aanchal.Bagga@tufts.edu</u> <u><mark>),</mark></u> <mark>Marcus Holmlund (e</mark> mail: <u><mark>mholmlund@worldbank.org),</mark></u> <mark>Nausheen Khan (e</mark> mail: <u><mark>nkhan8@worldbank.org),</mark></u> <mark>Subha Mani, Corresponding author (e</mark> mail: <u><mark>smani@fordham.edu), Eric Mvukiyehe (e</mark></u> mail: <u><mark>eric.mvukiyehe@duke.edu),</mark></u> <mark>and Patrick Premand (email: ppremand@worldbank.org).</mark>
# **1. Introduction**

There is growing emphasis on expanding social protection around the world. The United Nations Sustainable Development Goals call upon countries to “implement nationally appropriate social protection systems for all” (United Nations, 2018). Cash transfers and public works are among the most widely used and debated social protection programs in low- and middle-income countries (LMICs) (e.g., World Bank, 2018; Banerjee, 2022).<sup>1</sup> Both types of programs provide cash for consumption support and poverty alleviation in the short run and offer promise for medium-run improvements in living standards facilitated by investments in human capital and productive assets.

The experimental literature on cash transfers goes back nearly 20 years, and their effectiveness is well-established (e.g., Fiszbein and Schady, 2009; Baird et al., 2014, Millań et al., 2019; Kondylis and Loeser, 2021). While the experimental evidence on public works has been much thinner, several new studies have been completed in the past five years. This paper reviews experimental evaluations of public works programs from LMICs. As such, we extend the scope of earlier reviews by Subbarao et al. (2013) and Gehrke and Hartwig (2018), which rely mostly on quasi-experimental evidence. We examine the impacts of public works on participants’ short-term employment and earnings, and whether effects are sustained over the medium run. We also discuss effects on consumption, savings, assets, women’s empowerment, and psychological well-being.

There are various types of public works programs, and their objectives link to both social protection and employment policies. For instance, several large-scale public works programs in low- and middle-income countries, such as those in Ethiopia or India, were established to address climatic shocks.<sup>2</sup> Humanitarian organizations like the World Food Programme also routinely deploy policy instruments with features of public works programs in response to shocks or seasonality (World Food Programme, 2016). Some public works programs in Latin America were implemented in response to severe macroeconomic instability and crisis, for instance, in Argentina.<sup>3</sup> Public works programs have also been adopted in fragile settings to respond to economic instability induced by conflict and spur post-conflict recovery, such as in the Central African Republic, Comoros, Côte d’Ivoire, Democratic Republic of Congo (DRC), and Sierra Leone. In these cases, they aim to provide public goods or rebuild community infrastructure destroyed during the

> 1 For instance, around 60 low- and middle-income countries are implementing workfare programs (World Bank, 2018).

> 2 In 2005, Ethiopia introduced the Productive Safety Net Program (PSNP) to address chronic food insecurity arising from drought shocks (Tesfay et al., 2017). Similarly, in the 1990s, one state in India, Maharashtra offered India’s first public works program in response to a massive drought. Much later India introduced the world’s largest public works program as a ‘right to work’, the Mahatma Gandhi National Rural Employment Guarantee Act (MGNREGA), which guarantees 100 days of temporary employment per year to every household residing in a rural district in India (Imbert and Papp, 2015; Mani et al., 2020).

> 3 The currency crisis in the 1990s led Argentina to introduce the _Trabajar_ “To Work” program that provided short-term work to the poor (Ravallion and Jalan, 1999), but the deepening of the financial crisis in 2002 led to the expansion of the program to _Jefes De Hogar_ (Galasso and Ravallion, 2004) wherein all unemployed heads of household became eligible to participate in the workfare program. In some cases, the workers were also required or encouraged to save some of their wage earnings.

2
conflict and improve economic security and employment opportunities for individuals in conflict-affected regions.<sup>4</sup> Increasingly, the focus of public works programs is to improve medium-run employability, assets, and resilience to future shocks (Gehrke and Hartwig, 2018). Besides providing short-term employment or income support, these programs also aim to open pathways to productive employment after participants exit the program. This is often done by combining the public works experience with additional interventions, such as vocational training or micro-entrepreneurship support.<sup>5</sup> This paper reviews experimental evidence on public works programs, including a wave of recent studies, and provides directions for future research. We use a meta-analysis model to estimate the average impact of public works programs on individuals’ earnings and employment. We use data from all countries for which a published paper, working paper, or project report was either publicly available or made available by the authors as of December 2021. As such, the review is based on 11 randomized control trials across 9 countries––Comoros (Gazeaud et al., 2019), Côte d’Ivoire (Bertrand et al., 2021), DRC <mark>(Brandily-Snyers et al., 2021;</mark> Mani and Mvukiyehe, 2021), Djibouti (Devoto et al., 2017), the Arab Republic of Egypt (World Bank, DECRG; Croke et al., 2023), Ethiopia (Abebe et al., 2021), the Lao People’s Democratic Republic (World Bank, EAPGIL, 2020), Sierra Leone (Rosas and Sabharwal 2016), and Tunisia (Leight and Mvukiyehe, 2022) – see Appendix Table A1 for details.<sup>6</sup> We complement earlier work by Subbarao et al. (2013) and Gehrke and Hartwig (2018) in several important ways. First, we focus on public works experiments. Recent experimental studies provide causal evidence on the impacts of public works programs on individual participants and their households in diverse contexts, thus offering both greater internal and external validity. In contrast, existing reviews of public work programs are mostly based on quasi-experimental studies. This is a potential concern, as there are significant differences in findings generated by experimental and non-experimental evaluation methods (LaLonde, 1986; Smith and Todd, 2005; McKenzie, 2017; Card et al., 2018; Millán et al., 2019). The issue may be particularly salient for public works programs

> 4 Comoros rolled out a Social Safety Net Project (SSNP) that provided public works opportunities to poor households (Gazeaud et al., 2019). Public works programs in Sierra Leone, Côte d’Ivoire, DRC, and the Central African Republic were also introduced to facilitate post-conflict recovery (Rosas and Sabharwal, 2016; Bertrand et al., 2021; Brandily-Snyers et al., 2021; Mani and Mvukiyehe, 2021; Alik-Lagrange et al., 2023).

> 5 To achieve these broader objectives of transitioning workers from workfare to regular employment, during 1998-2000, Argentina’s Proempleo experiment provided specialized training or vouchers that entitled employers of workfare participants to a sizable wage subsidy (Galasso et al., 2004). The public works in Côte d’Ivoire included complementary micro-entrepreneurship or job search skills training (Bertrand et al., 2021). In Urban DRC, the public works program was complemented with incentivized savings and skills training ( <mark>Brandily-Snyers et al., 2021)</mark> . Similarly, the World Food Programme increasingly implements food-forassets interventions that seek to put a greater emphasis on asset creation and environmental outcomes, together with providing complementary market access or livelihood support (World Food Programme, 2016). In recent innovations, digital public works have been piloted to also foster digital skill acquisitions.

> 6 A study from Malawi (Beegle et al., 2017) was not used, as it did not report results on employment and earnings in ways that are readily comparable to the other studies. A study from the Central African Republic did not have a report available at the time of the review, although it was recently published (Alik-Lagrange et al., 2023).

3
that rely on participants’ self-selection (Bertrand et al., 2021; Banerjee et al., 2022). Second, we review evidence from studies that document impacts on outcomes at various points in time where impacts measured during program (4 studies) are classified as “short run,” and post-program impacts (11 studies) are classified as “medium run.” 4 of these 11 studies track respondents over two rounds. Overall, this allows us to assess not just the short-run effects of the public works programs, but also their medium-run effects, which remain underexplored in the literature until recently. Importantly, we pay close attention to some of the intermediary outcomes (e.g., asset accumulation, savings) that relate to mechanisms that may drive sustained program impacts over the medium run. Third, we draw lessons from programs targeting different types of public works participants (e.g., youth; urban and rural populations; women), as well as from diverse cultural and institutional contexts. Lastly, while we focus our discussion on economic outcomes, we also examine impacts on women’s empowerment and psychological well-being, which remain understudied in the literature.

There are a few caveats to this review. First, we focus on documenting the impact on participants’ outcomes. We are unable to comment on the general equilibrium effects or externalities of public works programs. This is not to suggest that these are not important, especially for programs that have a strong focus on the provision of public goods (e.g., roads, irrigation systems), the improvement of environmental outcomes (e.g., reforestation), or the generation of social externalities (e.g., social cohesion in post-conflict settings). However, only a few studies to date have been designed to identify general equilibrium effects or measure externalities (Abebe et al., 2021; Muralidharan et al., 2021; Leight and Mvukiyehe, 2022; Croke et al., 2023). Second, we are unable to directly compare the cost-effectiveness of the programs we study, as only two studies provide detailed cost data and cost-effectiveness analysis (Rosas et al., 2016; Bertrand et al., 2021). Third, while comparing across countries is helpful for external validity, the body of experimental evidence is largely concentrated in Sub-Saharan and North Africa, highlighting the need for additional evidence from other regions. Lastly, it is beyond the scope of this review to comment on the data quality or limitations of specific studies, though we provide information on attrition or take-up that is relevant to interpret some of the country-specific findings.

The review documents substantial variation in the impacts of public works between studies and over time, but also highlights some consistent patterns and policy implications. First, public works tend to significantly improve participants’ employment and earnings during their participation in the program. Second, these short-term effects fade in the medium run, except in a few cases with particularly large short-term impacts on savings or investments in self-employment activities. Third, observed impacts on earnings are much lower than planned transfer amounts due to forgone earnings, which raises questions about cost-effectiveness. Fourth, there is little evidence of public works programs improving food consumption expenditure. Lastly, there is evidence of improvements in psychological well4
being and women’s empowerment in some cases, but it is not systematic, and measurement of these dimensions remains at times rather narrow.

# **2. Conceptual framework**

This section outlines various channels through which public works may impact participants’ outcomes in the short to medium runs. While it provides an overview of a broad range of mechanisms (including general equilibrium effects and externalities), the results section will later focus on a narrower set of outcomes and impacts for participants that are measured in the studies included in our review.

Public works programs can impact individual and household-level outcomes through a combination of short-run income and substitution effects. Access to a public works program in a community is likely to attract unemployed as well as underemployed individuals, giving them an opportunity to move away from low-paid farm and off-farm work (Subbarao et al., 2013; Filmer and Fox, 2014). This, in turn, may increase both temporary employment and income for participants. The income effect may also increase households’ demand for food and non-food items (and hence consumption expenditures) in the short run (Ravi and Engler, 2015; Liu and Deininger, 2013; Azam, 2012). However, if program participants (or other members of their households) substitute other forms of employment (e.g., self-employment or low-paying wage jobs) to access public works jobs, then participation in public works could have little to no effect on overall household income and consumption due to forgone earnings from these other occupations ( <mark>Bertrand et al., 2021; Brandily-Snyers et al., 2021)</mark> . Depending on the type of labor substitution and the frequency of individuals’ earnings from each occupation, it could even have a (temporary) negative effect on participants’ household income and consumption, for instance, if public works payments are made less frequently or later than other income sources. The availability of short-term employment through public works may also allow households to smooth consumption expenditure and food security during the lean season when agricultural productivity is low (Zimmermann, 2014; Ravi and Engler, 2015; Beegle et al., 2017).

Since unemployment or underemployment is known to be positively correlated with poor mental health and unhappiness (Clark and Oswald, 1994; Farré et al., 2018), public works opportunities may improve an individual’s psychological well-being (Bertrand et al., 2021, Hussam et al., 2021). At the same time, these opportunities might also create social stigma for participants (Ravallion, 2019). Several public works programs are specifically designed to encourage women’s participation in the workforce (Afridi et al., 2016; Devoto et al., 2017; World Food Programme, 2021). The ability to earn income, which may otherwise be restricted or even unavailable to women in low-and middleincome countries, has the potential to improve economic empowerment and autonomy. This may, in turn, reduce domestic violence (Buller et al., 2018; Bhalotra et al., 2021), though, in some contexts, greater autonomy may increase gender-based violence (Hidrobo and Fernald, 2013).

5
Public works programs may also increase household income and consumption expenditure through savings, investment, and skill accumulation in the medium run, that is, after participation in the program ends. Income from public works used for savings and productive investments like the purchase of inputs (e.g., fertilizers) or assets (livestock, smartphones, etc.) may lead to subsequent improvements in household income and consumption expenditure (Bertrand et al., 2021; Mani and Mvukiyehe, 2021). On-the-job learning and soft skills accumulation facilitated by participation in public works programs could also increase employment and earnings for participants, although there is currently little empirical evidence to support this (Alik-Lagrange et al., 2017; Bertrand et al., 2021).

Public works programs may have broader externalities or general equilibrium effects that affect program participants or non-participants and can occur both in the short and medium runs. While the empirical analysis in this review does not consider these effects (as mentioned above), we briefly discuss them here. General equilibrium effects may operate via changes in private sector wages, public good creation in the community, or other social externalities. For example, public works programs may crowd out private sector employment and increase private sector wages, which in turn can increase household earnings and consumption expenditures among participants and non-participants alike (Azam, 2012; Imbert and Papp, 2015; Abebe et al., 2021; Muralidharan et al., 2021). The increase in local wage rates may also increase the overall price of goods, as such affecting the demand for food and non-food items.<sup>7</sup> Theoretically, the increase in private sector wages can also reduce the demand for labor among non-participants in the private sector.

Further, public works programs often involve the construction or maintenance of public infrastructure that can spur economic opportunities for both participant and nonparticipant households. For instance, public works programs may improve environmental outcomes and assets by planting trees or rehabilitating degraded lands (Adjognon et al., 2020). In drought-prone regions, the construction of canals and irrigation systems can increase returns on agricultural investments and activities (Subbarao et al., 2013). Similarly, the construction or rehabilitation of roads can provide access to markets and better prices for agricultural commodities (Asher and Novasad, 2020).

Lastly, public works programs may also produce social externalities by facilitating positive interactions between participants and could reduce crime and other risky behaviors through shifting time use toward public works and increasing income (Fetzer, 2020; Bertrand et al., 2021). Increased intra-group contact in ethnically diverse communities could potentially contribute to social cohesion outcomes.

# **3. Program features**

> 7 The increase in price of goods is likely to come from firms in the private sector passing on the increase in the cost of production to the consumer, though this effect can vary across market structures.

6
There is substantial variation in key design elements across the 11 public works programs included in our study. Below (and in Table 1), we summarize the following features: duration of employment, targeting, wages, maximum earnings, and type of work. Overall, the public works programs included in this review are qualitatively like those from other low- and middle-income countries described in Subbarao et al. (2013), except for those that allow for repeated participation every year, like India's Mahatma Gandhi National Rural Employment Guarantee Act (MGNREGA) or the Ethiopia rural Productive Safety Net Programme (PSNP), which are not represented here.

**Duration of employment.** Public works programs are designed to offer employment for a limited duration: the median participant is offered the equivalent of approximately 4 months of work. The maximum duration is 2.5 months in Djibouti and up to 18 months in the Egypt Community program. The programs can provide consecutive months of employment, though some spread days worked within one year or over longer periods. For instance, the two programs in Comoros and Ethiopia offer a total of 9 months of work, but for 3 months per year over 3 years. The program in Lao PDR offers employment during specific times in the agricultural cycle, spreading 75 days (~3.75 months) of employment over two full rainy seasons (~1.5 years).<sup>8</sup> The maximum duration of employment for all programs included in our review is summarized in Table 1.

**Targeting.** All programs applied some form of geographic targeting. Four programs focused on urban areas, four on rural areas, and three covered both (Table 1). All programs used self-targeting, whereby the program is advertised, and individuals are invited to apply if they are interested. Many targeted specific groups such as youth (in Côte d’Ivoire and Sierra Leone), or women (in Djibouti, the Egypt Community program, and Lao PDR).

**Wages.** Daily wages differ widely, ranging from $2.5 in Sierra Leone to $14.7 in the Egypt Community program, with a median of $9.60 (see Table 1).<sup>9</sup> Setting a wage for a public works program involves trade-offs (Bertrand et al., 2021). On the one hand, setting a low wage can facilitate self-targeting by discouraging individuals who already work and have higher earnings from leaving their occupation to join the program. On the other hand, setting a higher wage can enhance take-up as well as lead to larger impacts on earnings among individuals most in need. Five of the 11 programs set wages within 20% of the minimum wage, with 2 programs further below and 2 programs further above the minimum wage (see Table 1 and additional details in Table A2). This again shows substantial variation between programs. In practice, however, it is not always clear that the minimum wage is an appropriate benchmark, because a large share of workers in low-income

> 8 There is some variation in the number of days offered within some programs. For instance, the Sierra Leone program offered 50-75 days of work and the Egypt Infrastructure program offered 1 week to 3 months of work. In these instances, we have used the maximum days allowed in the program in the calculations, since the average time worked is not always available. In the Egypt Community program, the duration of participation was set between 12-18 months; however, on average, individuals work up to 11.7 months. 9 All wages are converted to US dollars using the 2021 PPP conversion factor from the World Bank’s International Comparison Program (https://data.worldbank.org/indicator/PA.NUS.PPP).

7
<!-- Start of picture text --> ~ 300 g Egypt Infrastructure e<br>a Tunisia Egypt Community<br>Ee<br>Ay<br>[a<br>nN<br>=)<br>&© CoteAd'Ivoire:<br>=200<br>F=<br>lo}<br>§<br>i)<br>QO<br>Bb<br>g<br>5 100 e Urban DRC, Rural DRC<br>2 e * Laos ® Ethiopia<br>S Djibouti<br>2 ®Sierra Leone<br>2 ® Comoros<br>: 0<br>m1 2 3 4 5 6 7 8 9 10 1 12 13 14 15 16 17 18<br>S Duration of Public Works (in months)<br><!-- End of picture text -->
programs explicitly tailor activities to their target population. For example, the Egypt Community program mainly targeted women and focused on activities that were not considered physically strenuous, such as cleanliness and environmental awareness campaigns, early childhood education, and literacy promotion. Similarly, in Djibouti, the program targeted only women and included light community works such as street rehabilitation or small artisanal projects.

**Take-up:** Program take-up is high in most studies, as shown in Table 1. Take-up rates were above 90 percent in Djibouti, Côte d’Ivoire, and Ethiopia. The lowest take-up rates occur in the Egypt infrastructure program (40 percent) and the Urban DRC program (56 percent).

9
# **Table 1: Program features**

||**Comoros**|**Côte**<br>**d'Ivoire**|**Djibouti**|**Urban DRC**|**Rural DRC**|**Egypt, Arab**<br>**Rep.**<br>**Community**|**Egypt, Arab**<br>**Rep.**<br>**Infrastructure**|**Ethiopia**|**Lao PDR**|**Sierra**<br>**Leone**|**Tunisia**|
|---|---|---|---|---|---|---|---|---|---|---|---|
|Duration of<br>employment|9 months|7 months|2.5 months|4 months|4 months|18 months|3 months|9 months|3.75 months|3.75 months|3 months|
|Daily wages<br>(Value in 2021<br>PPP US<br>dollars)|$5.41|$10.34|$9.64|$4.83|$4.83|$26.95|$14.75|$4.23|$21.03|$2.52|$14.50|
|Ratio of<br>program wage<br>to minimum<br>wage|0.4|0.8|NA|2.1|0.5|1.2|1.2|NA|1.1|6.3|0.9|
|Take-up rate|84%|93%|<br>95.9%|56%|73%|79%|40%|97%|n/a|n/a|73%|
|Geographic<br>targeting|Urban/<br>Rural|Urban|Urban|Urban|Rural|Rural|Urban/<br>Rural|Urban|Rural|Urban/<br>Rural|Rural|

Notes: Duration of employment presents the maximum length of program available in a year except in Lao PDR where it’s computed over 1.5 years. Further, in Comoros and Ethiopia, the program is offered for a few months every year for three consecutive years. In converting maximum days available on a public works project to months we assume that only 20 days of work is available in a month. See Table A2 for the calculation of the ratio of daily program wage to daily minimum wage.

10
# **4. Overview of experimental studies designs**

## **4.1 Experimental designs**

Table 2 summarizes the experimental design and sample for the 11 studies included in this review. Of the 11 studies, five use individual or household-level randomized designs (Djibouti, Côte d’Ivoire, Lao PDR, and the two studies in DRC), two use cluster-level randomized designs with randomization at the community level (Sierra Leone and Ethiopia<sup>11</sup> ), and four use a two-stage cluster randomized designs (Comoros,<sup>12</sup> both studies in Egypt, and Tunisia).<sup>13</sup> In the two-stage cluster RCTs, communities were first randomized into treatment (receives the public works programs) and control (does not receive the public works programs); in the second stage, within the treated communities, individuals/households were further randomized into treatment (eligible to work in the public works program) and control (ineligible to work in the public works program). The goal of the two-stage designs was to assess spillover effects, which we do not discuss here.<sup>14</sup> Two of the public works experiments included multiple treatment arms to isolate the impact of the work component from complementary interventions that were designed to facilitate post-program transition to other forms of employment that generate more sustained impacts. In Urban DRC, <mark>these complementary features included incentivized savings and skills training (</mark> Brandily-Snyers et al., 2021). In Côte d’Ivoire, random subsets of participants were offered <mark>either additional entrepreneurship training to facilitate selfemployment or training in job search skills to facilitate wage employment (B</mark> ertrand et al., 2021). Since only these two studies contain multiple treatments, we do not attempt to draw more general conclusions on complementary program design features in this paper.

# **4.2 Evaluation sample**

The median study sample size is 2,126. Individual-level randomized designs have a sample size ranging from 952 to 6,014. Cluster randomized designs have larger samples ranging from 1,017 to 19,442 with at least 30 clusters per treatment arm across studies. The median sample size per cluster is 31, comparable to the median sample size per cluster of 26 noted in Muralidharan and Niehaus (2017).<sup>15</sup>

> 11  In Ethiopia, clusters were randomized into treatment and control, and, within treatment clusters, the participants were selected by local _Ketena_ committees. 12  In Comoros the design varied the treatment intensity, 40 percent of eligible households received the <mark>intervention in treatment clusters and 20 percent of eligible households received the intervention</mark> in control clusters.

> 13  A few studies also offer public works opportunities to the control group in the post-evaluation period in a randomized phase-in design, namely, Egypt infrastructure, Sierra Leone, Ethiopia, and Djibouti. 14 In Comoros, a third level of randomization was carried out: in every treatment household, the gender of the participant was further randomly chosen within the household.

> 15 All individual level randomized experiments can detect at least a $1.50 improvement in consumption expenditure and earnings, assuming a standard deviation of $8 with 5 percent type I error rate and 80 percent power. Among the cluster RCTs, there is an improvement in consumption and earnings––the Egypt studies can detect at least a $1.40 improvement, Comoros $2.10, Ethiopia $1.70, Sierra Leone $1.00, and Tunisia $1.90. These are computed using the country specific cluster sizes noted in Table 2 and assuming a standard deviation of $8, intra-cluster correlation of 0.10, 5 percent type I error rate and 80 percent power. All these calculations also assume 100 percent take-up. In practice, however most programs did not have 100 percent take-up. Particularly low take-up rates in Urban DRC (56 percent) and Egypt infrastructure (40 percent) will reduce the power to detect significant effects substantially.

11
Follow-up survey attrition rates are generally small to modest in size. One exception is the study on the Egypt Community program, where attrition was 23 percent due to an administrative issue and not because of migration (Croke et al., 2023). Attrition in the Urban DRC (17.2 percent) and Djibouti (11.4 percent) studies was also relatively high, perhaps because of the urban contexts where mobility tends to be higher. Finally, the Tunisia study also had a high attrition rate of 19.6 percent between the first and second follow-up surveys, plausibly explained by the long-time gap between rounds (about four years). The studies in Comoros, Côte d’Ivoire, and Ethiopia reported very low attrition rates of 4.00 percent, 6.20 percent at endline (2.60 percent at midline), and 2.94 percent, respectively.

Table 3 summarizes the key demographic characteristics of study participants. The median age of the participants is 33.2 years old (programs in Sierra Leone, Côte d’Ivoire, and the Egypt Community program explicitly targeted youth). The average household size varies between 3.9 and 6.9, and the median household size is 6.1. Except in Djibouti and Lao PDR, where the program targets only women, both men and women participate in most programs, but women account for less than half of participants in several countries, such as Comoros, Rural DRC, and Sierra Leone. In nine of the ten studies that report it, the median number of participants with no education is 33 percent.

12
# **Table 2: Evaluation Design**

||**Comoros**|**Côte**<br>**d’Ivoire**|**Djibouti**|**Urban DRC**|**Rural DRC**|**Egypt, Arab**<br>**Rep.**<br>**Community**|**Egypt,**<br>**Arab Rep.**<br>**Infrastruct**<br>**ure**|**Ethiopia**|**Lao**<br>**PDR**|**Sierra**<br>**Leona**|**Tunisia**|
|---|---|---|---|---|---|---|---|---|---|---|---|
|Level of<br>randomization|Three levels:<br>community;<br>household;<br>recipient’s<br>gender|Individual|Household|Individual|Individual|Two levels:<br>community;<br>individual|Two levels:<br>village;<br>individual|Community|Individu<br>al|Community|Two levels:<br>Community;<br>individual|
|Multiple||||||||||||
|treatment arms<br>(to test<br>additional<br>features)|No|Yes|No|Yes|No|No|No|No|No|No|No|
|Total number<br>of clusters|62|n/a|n/a|n/a|n/a|196|260|90|n/a|276|80|
|Sample size|2,181|2,958 (short<br>run)|952 (short<br>run)|6,014|2043|1,017 (short<br>run)<br>340 (medium<br>run)|1,844|19,442|1099|5,486|2,126 (short run)<br>1,748 (medium<br>run)|

13
|||3,934|897|||||
|---|---|---|---|---|---|---|---|
|||(medium)|(medium|||||
|||run|run)|||||
|Attrition|4.0%|6.2%|11.4%|17.2%|n/a<br>23%<br>n/a<br>2.94%|3.0%|n/a<br>19.6%|

Notes: Attrition rates are reported for the latest survey round. Sample size noted in the table reflects the no. of observations used for estimating the ITT effect of the public works programs on employment outcomes.

14
**Table 3: Sample descriptive statistics**

||Mean|Proportion|Proportion|Mean|Proportion|
|---|---|---|---|---|---|
||Age|Women|with|Household|Living in an|
||||no|Size|Urban Area|
|||(%)|Education|||
||||(%)||(%)|
||(1)|(2)|(3)|(4)|(5)|
|Comoros|29.6|36|22|6.6|n/a|
|Côte d’Ivoire|24.7|42|49.0|6.1|93|
|Djibouti<br>|33.4|100|82.4|6.9|100|
|Urban DRC<br>|32.6|42.7|n/a|6|100|
|Rural DRC<br>|34.2|38.1|26.1|6.6|0|
|Egypt,<br>Arab<br>|27.6|82.2|1.5|n/a|n/a|
|Rep.||||||
|Community||||||
|Egypt,<br>Arab<br>|n/a|0.43|25.1|4.5|25|
|Rep.||||||
|Infrastructure||||||
|Ethiopia<br>|33|57.6|24.9|3.9|100|
|Lao PDR|33.3|100|41|5.9|n/a|
|Sierra Leone|27|33|52|n/a|49.6|
|Tunisia|41.5|49.8|60|n/a|n/a|

Notes: Most of these characteristics are obtained using the control sample at baseline (except Sierra Leone, where we use the information from program participants). Tunisia and Egypt do not have a regular baseline and hence we used data from the control group in the 1<sup>st</sup> follow-up survey. The Urban DRC study reports median household size. n/a – not available.

# **4.3 Follow-up surveys and key outcomes**

All studies administered at least one follow-up survey after the start of the intervention to collect detailed data across the treatment and control groups. Four studies (Comoros, Côte d’Ivoire, Djibouti, Lao PDR) fielded follow-up surveys to measure impacts during the public works intervention. All studies, except Comoros, conducted at least one post-intervention follow-up survey to estimate impacts after the program ended. However, there is substantial variation in the timing of the post-program follow-up surveys, ranging from 1 month after the program in the Egypt Community and Sierra Leone programs to 66 months in the Tunisia program. Only four studies (Côte d’Ivoire, Djibouti, Egypt Community, and Tunisia) fielded two follow-up surveys to examine the evolution 15
<!-- Start of picture text --> Comoros +. ————= 4<br>Céte d'Ivoire od c=<br>Laos (4)<br>Djibouti -o—1 =»<br>Sierra Leone td<br>Egypt Community —") 8<br>Tunisia =) — -<br>Egypt Infrastructure fy =f<br>Rural DRC a =“<br>Ethiopia — « .<br>meee Project<br>Urban DRC —_— =! m= Baseline Survey<br>201 2 201 4 201 6 201 g 2020 1022 m=" Ist follow-up<br>mea 2nd follow-up<br><!-- End of picture text -->
# **<mark>5. Results––Impact estimates from 11 RCTs of Public Works Programs</mark>**

We now present the intent-to-treat (ITT) estimates of public works interventions on key outcomes. ITT estimates are based on the initial assignment to treatment status, regardless of whether individuals participated in the public works program. In most cases, governments or other parties that introduce programs can make them available to their target population, but they cannot mandate take-up, hence the ITT estimate is arguably the most policy-relevant parameter as it estimates the benefit of providing _access_ to a program. <mark>The ITT estimate is computed as the difference in average outcomes in the treatment group (randomly chosen public works participants, from within the population targeted by the program) and average outcomes in the control group (randomly chosen non-participants from within the same population).</mark><sup>16</sup> <mark>We report results separately for each outcome, study, and, when applicable, followup survey. In addition, we aggregate results across studies and follow-up surveys using a</mark> random effects REML model (like the approach in McKenzie, 2021). The meta-analysis takes as inputs the point estimates and standard errors from each study. Higher weight is given to studies that have smaller standard errors.

# **5.1 Employment**

Since public works programs offer cash in exchange of work, we first present their impacts on participants’ employment. In Figure 3, we document the estimated impacts on individuals’ employment from the different studies, along with the control mean for each case. The figure displays results by the timing of the survey with respect to program completion, with short-run impacts measured before individuals exit the program at the top (4 to 0 months before program completion), followed by impacts measured after individuals have exited the program in the medium run (from 1 to 66 months after program completion). Lastly, we also present estimates from a random-effects meta-analysis regression model that captures the short run, medium run, and overall impact of public works on employment.

Short-run treatment effects during the program are positive and significant in Comoros, Côte d’Ivoire, and Djibouti. In Lao PDR, they are positive but marginally not significant. The probability of participants being employed increases by 14 to 54.5 percentage points during the program, with an average effect of 29.3 percentage points (95 percent CI:10.8, 47.8) across the four studies. These effects correspond to very large increases in relative terms in Comoros (263 percent) and in Djibouti (256 percent), where small shares of the control group are otherwise employed (11.9 percent and 21.3 percent, respectively). The relative effects are much smaller in Côte d’Ivoire (16 percent), where

> 16 All cluster randomized controlled trials included in this review, except the study on the Egypt Community program, compare randomly chosen participants in treatment clusters to non-participants in control clusters. In some cases, the sampling strategy used to sample non-participants in control clusters might differ from the sampling strategy used to sample participants in treatment clusters (e.g., Leight and Mvukiyehe, 2022).

17
85 percent of the control group has an occupation (in most cases working in informal jobs or self-employment). Even though the effect is not statistically significant in Lao PDR, the point estimate is non-negligible, suggesting a 15.5 percentage points increase, or 46 percent, relative to the control group.<sup>17</sup> Next, we examine the effects measured after individuals exited the public works program. These post-program effects remain substantial in Sierra Leone at 1 month (11.2 percentage points, or 34 percent relative to the control mean), Egypt Community at 1 month (21 percentage points, or 65 percent relative to the control mean), though not significant in Egypt Infrastructure at 4 months (1 percentage points; 1 percent relative to the control mean). Three studies continue to document positive employment effects at 12 months (Rural DRC and Tunisia) and at 25 months (Urban DRC). Four studies, however, find effects that are not statistically significant (Djibouti at 9 months, Côte d’Ivoire at 12 months, Ethiopia at 12 months, and Egypt Community at 27 months) and one study finds negative impacts (Tunisia at 66 months). Overall, the medium-run treatment effects are not significantly different from zero at 3.4 percentage points (95 percent CI: -1.2, 7.9) and decline with length of time since program completion. A fading-out of effects over time is observed across studies and is also observed within studies that include two follow-up surveys (Côte d’Ivoire, Djibouti, Egypt Community, and Tunisia).

The positive effects on employment are primarily obtained in studies with low levels of baseline employment. The largest relative effects are observed in Comoros (263 percent), Djibouti (256 percent) and Tunisia (87 percent), where employment in the control group is relatively low (11.9 percent, 21.3 percent, and 9 percent, respectively). These large relative effects can be driven by the specific groups that are targeted in these cases, such as youth or women who are more likely to be unemployed or outside the labor force. They might also be explained by differences in the structure of the labor market across settings. For instance, underemployment is more widespread in less formalized economies, where many individuals are engaged in low-paying occupations and few are formally unemployed.

The average effect on the likelihood of employment also hides substitution between occupations. In Côte d’Ivoire, a substantial share of treated youth substituted out of informal wage jobs and self-employment to take public works jobs, limiting the net effects on employment. In Ethiopia, the program increased employment in public works by 4.6 percentage points but also decreased employment in the private sector by 4.7 percentage points, resulting in an overall null result.

> 17 The authors also note that in an alternative specification there is a 16 percentage points increase in the likelihood of working in paid work. There is also an increase in being a ‘regular earner’.

18
Control Study Timing Mean Short-run effects Comoros -4m 119 Céte d'Ivoire 3m 85 Laos -2m 33.6 Djibouti Om 21.3 Heterogeneity: 1 = 326.41, 2 = 97.94%, H? = 48.66 Test of0, = 8: Q(3) = 99.77, p= 0.00 Test of0 = 0: z=3.11,p= 0.00 Medium-run effects Sierra Leone lm 33.2 Egypt Community Im 32.3 Egypt Infrastructure 4m 95 Djibouti 9m 23.7 Tunisia 12m 9.1 Rural DRC 12m 38.6 Ethiopia 12m 36.6 Céte d'Ivoire 12m 86 Urban DRC 25m 53.3 Egypt Community 27m 51.2 Tunisia 66m 18.3 Heterogeneity: 7 = 54.81, I? = 95.32%, H? = 21.38 Test of8, = 8: Q(10) = 150.33, p= 0.00 Test of6 = 0: z= 1.46, p=0.14

|Effect si<br>with95%|ze<br> CI|Weight<br>(%)|
|---|---|---|
|i<br>31.40 [27.87,|34.93]|6.87|
|a<br>14.00 [11.06,|16.94]|6.90|
|15.50 [-4.69,|35.69]|4.81|
|HE<br>54.50 [44.92,|64.08]|6.33|
|<=<br>29.29 [10.80.|47.77]||
|| |<br>11.20 [ 8.85,|13.55]|6.93|
|sos<br>21.00 [15.71,|26.29]|6.76|
|1.00 [-0.96.|2.96]|6.94|
|-3.30[-5.93.|-0.67]|6.92|
|soe<br>7.90 [ 3.00,|12.80]|6.79|
|440[ 0.11.|8.69]|6.83|
|-0.10[-2.45,|2.25]|6.93|
|1.50[-1.44.|4.44]|6.90|
|5.30[ 0.20.|10.40]|6.77|
|-6.80 f15.03,|1.43]|6.48|
|i<br>-5.20 [-9.32.|-1.08]|6.84|
|3.40[-1.16,|7.96]||

# Overall

>_> 9.85 [ 1.89, 17.81]

Heterogeneity: t = 236.91. ? = 98.72%, H? = 78.17 Test of8, = 8: Q(14) = 523.28, p = 0.00 Test of6 = 0: z=2.42. p=0.02 Test of group differences: Q,(1) = 7.10. p=0.01 20 -10 0 10 20 30 40 50 60 70 Random-effects REML model
Overall, the effects of public works on participants’ employment are mixed. On the one hand, they seem promising as most of the studies (8 out of 11) find positive and significant effects on employment at some point in time. Consequently, the overall impact of public works programs on employment is 9.8 percentage points (95 percent CI: 1.9, 17.8). On the other hand, the average employment impacts during the program are 29.3 percentage points. This can be seen as quite low: it means that, for every 100 individuals taking on jobs, 71 would have also worked in the absence of the program. As such, while public works programs do increase the share of individuals employed in the short run, their net effect on the number of participants employed is smaller than the number of positions they offer. In addition, the evidence shows that public works programs do not produce much sustained impacts on employment: the overall impact after the programs end is small (3.4 percentage points) and not significant (p=0.14), with only three of the nine point estimates beyond one-month post-intervention revealing a positive impact.

# **5.2 Earnings**

Figure 4 presents the effects of public works programs on monthly earnings for each study and follow-up survey. We find a positive and significant increase in earnings in at least one point in time in 10 of the 11 studies. Increases in earnings are observed in all cases with an increase in employment, except for the Urban DRC program. Similarly, no effect on earnings is found when there is no corresponding effect on employment, except in Côte d’Ivoire at 12 months, when a small impact on earnings is observed, and in Ethiopia, where the public works opportunity fully crowds out employment in the private sector.<sup>18</sup> Positive effects on earnings are found in all studies measuring impacts during the program, with an average increase of 51.3 USD (95 percent CI: 7.1, 95.4); this is sizable compared to the unweighted control mean of 55.5 USD. The positive effects on earnings are also found to persist up to 12 months after the program, except for Egypt Infrastructure (measured at 4 months) and Djibouti (measured at 9 months). However, positive impacts on earnings are not found for the three studies with follow-up surveys beyond 12 months post-intervention. The overall medium-run (i.e., post-program) effect is an increase of 7.8 USD (95 percent CI: -1.3, 16.8), which is significant at the 10 percent level (p=0.09). Whether a study shows significant earnings gains after the program seems associated with participants’ ability to acquire savings or productive assets for self-employment or business activities, which we will discuss further below.

> 18 In Ethiopia, and as mentioned earlier, the intervention increases public employment by 4.6 percentage points and reduced private employment by 4.7 percentage points. Hence the net effect on employment is zero.

20
|Study|Control<br>Timing Mean|Effect si<br>with95%|ze<br> CI|Weight<br>(%)|
|---|---|---|---|---|
|Short-run effects|||||
|Comoros|-4m<br>26.03|6.28[ 4.16,|8.40]|7.01|
|Céte d'Ivoire|-3m<br>177.32|HE 112.10[89.19,|135.01]|6.13|
|Laos|-2m<br>4.87||<br>61.56[56.70,|66.42]|6.97|
|Djibouti|Om<br>13.87||<br>28.79[19.21,|3837]|6.84|
|Heterogeneity:t?=19|93.91,I?=99.58%,H?=237.02|—<—_-<br>51.27[7.07,|95.47]||
|<br>Testof6;=8,:Q(3)=|<br>492.82,p =0.00||||
|<br>Testof6= 0: z=2.27,|<br> <br>p= 0.02||||
|Medium-run effects|||||
|SierraLeone|Im = 77.52|i<br>27.27[16.31,|38.23]|6.79|
|EgyptCommunity|Im<br>15.51|7.46 [ 4.64,|10.29]|7.00|
|Egypt Infrastructure|4m<br>264.91|8.85 F17.49,|35.19]|5.89|
|Djibouti|9m=:10.62|-2.11[-4.44,|0.22]|7.01|
|Tunisia|<br>12m<br>24.04|.oF<br>28.32[10.54,|46.10]|6.45|
|RuralDRC|12m<br>30.11|<br><br>11.30[<br>5.44,|17.16]|6.95|
|Ethiopia|12m 166.59|a<br>21.62[<br>7.23,|36.01]|6.64|
|Cote d'Ivoire|12m 179.97|+<br>18.05[ 2.59,|33.51]|6.58|
|UrbanDRC|25m<br>56.28|1.12 [-7.45,|9.69]|6.88|
|Egypt Community|27m<br>57.41|-9.72 F18.85,|-0.59]|6.86|
|Tunisia|66m 100.21|-<br>-32.20 £56.74,|-7.66]|6.01|
|Heterogeneity: tT? = 19|0.54, I?= 94.94%, H? = 19.78|7.75 [-1.32,|16.82]||
|Test of 0;= 8,: Q(10) =<br>Testof0= 0: z= 1.67,|88.16,p = 0.00<br><br>p= 0.09||||
|Overall||><br>19.03[3.10,|34.96]||

Overall Heterogeneity: tT? = 941.67, I? = 99.25%, H? = 133.28 Test of 0; = ,: Q(14) = 687.97, p= 0.00 Test of0 = 0: z = 2.34, p= 0.02 Test of group differences: Q,(1) = 3.57, p = 0.06

-60-40-20 0 20 40 60 80100120140

Random-effects REML model
earnings are substantially below the maximum possible value of monthly earnings during the program, which points to large forgone earnings for participants. The ratio of impacts on earnings relative to the maximum potential monthly earnings during the program is 23 percent in Comoros, 54 percent in Côte d’Ivoire, 70 percent in Lao PDR, and 36 percent in Djibouti. On average, impacts on earnings are 46 percent of maximum potential program earnings, pointing to substantial forgone earnings.<sup>19</sup> The impacts on earnings relative to the maximum potential program earnings also decline over time and are low after the program, with an average of 7 percent (significant at the 10 percent level, p=0.09). While lower impacts on earnings relative to the value of program earnings are expected after the program, the rate at which impacts decay is key for calculating cost-effectiveness, because this affects the net present value of program impacts on earnings over time. Overall, the fact that effects on earnings are much smaller than the maximum possible earnings from public works during the program and quickly fade over time raises questions about the overall cost-effectiveness of public works programs. The results also suggest variations between programs, though few studies, aside from those in Côte d’Ivoire and Sierra Leone, provide the detailed information on program costs sufficient to conduct a full cost-effectiveness analysis.

Overall, the results show that the public works programs do lead to changes in earnings in the short run, but that these effects tend to fade in the medium run. In addition, the effects on earnings are small relative to the transfer amounts during the program, raising questions about cost-effectiveness.

> 19 Note that Figure 5 shows ratios with respect to maximum potential earnings, not average earnings. Imperfect take-up and participants working less than the allowed time partly explain why the ratios are smaller than 1, though most of the difference is explained by forgone earnings. For instance, in Côte d’Ivoire, impacts on earnings during the program are 63 percent of average earnings (purely explained by forgone earnings), and 54 percent of maximum earnings (explained by both forgone earnings and imperfect take-up). Average earnings cannot be estimated for each study, which is why we report results for maximum potential earnings.

22
|Control<br>Study<br>Timing Mean|Ef<br>wit|fect si<br>h95%|ze<br> CI|Weight<br>(%)|
|---|---|---|---|---|
|Short-run effects<br><br><br>|||||
|Comoros<br>-4m<br>26.03|i<br>0.23 [|0.16,|0.31]|6.94|
|Cote d'Ivoire<br>-3m<br>177.32<br><br><br>|HH<br>i0.54[ <br><br>|0.43,<br>|0.65]<br>|6.75<br>|
|Laos<br>-2m<br>4.87<br><br><br>|Mi<br>0.70[ <br><br>|0.64,<br>|0.75]<br>|7.02<br>|
|Djibouti<br>Om<br>13.87|7<br>0.36[|0.24,|0.48]|6.65|
|Heterogeneity: t?=0.04, I?=95.64%,H?=22.92|“0.46|0.26,|0.66]||
|Test of0; = 0; Q(3)= 100.09,p= 0.00|||||
|Testof80= 0: z= 4.41,p= 0.00|||||
|Medium-run effects|||||
|Sierra Leone<br>Im<br>= 77.52|—f_<br>0.54[|0.32,|0.76]|5.85|
|EgyptCommunity<br>Im<br>15.51|0.03 [|0.01,|0.04]|7.12|
|Egypt Infrastructure<br>4m<br>264.91|0.03 [|-0.06,|0.13]|6.84|
|Djibouti<br>9m<br>~=—:10.62|-0.01 [|-0.04,|0.02]|7.09|
|Tunisia<br>12m<br>24.04||<br>0.10[|0.04,|0.16]|7.00|
|RuralDRC<br>12m<br>30.11<br><br>|0.12 [ <br><br>|-0.06,<br>|0.30]<br>|6.18<br>|
|Ethiopia<br>12m 166.59<br><br>|—h-<br>0.26[ <br><br>|0.09, <br>|0.43]<br>|6.28<br>|
|Cote d'Ivoire<br>12m 179.97|=<br>0.09[|0.01,|0.16]|6.95|
|UrbanDRC<br>25m<br>56.28|0.01 [|-0.25,|0.28]|5.36|
|Egypt<br>Community<br>27m<br>57.41|-0.03 [|-0.07,|0.00]|7.08|
|Tunisia<br>66m 100.21|i<br>-0.11 [|-0.20,|-0.03]|6.89|
|Heterogeneity: t? = 0.02, I? =96.70%, H? = 30.30|0.07 [|-0.01,|0.16]||
|Test of8, = 0; Q(10)= 60.95,p= 0.00|||||
|Test of 8= 0: z= 1.68, p= 0.09|||||
|Overall<br>Heterogeneity: t? = 0.06, I? = 98.80%, H? = 83.59<br>Test of 8;= 9;; Q(14)= 712.10, p= 0.00<br>Test of<br>0 = 0: z= 2.96,p= 0.00|><br>0.19[|0.06,|0.31]||
|Test ofgroup differences: Q,(1)= 11.68,p= 0.00<br>|-4-2<br>0<br>2<br>4<br>6<br>8||||

Random-effects REML model
Control Effect size Weight Study Timing Mean with 95% CI (%) Short-run effects Comoros -4m 95.85 -1.19[ -3.61, 1.23] 13.13 Djibouti Om = 72.34 -3.27[ -4.25, -2.29] 13.39 Heterogeneity: t? = 1.27, I? = 58.84%, H? = 2.43 4 -2.54[ -4.48, -0.59] Test of 6, = 0; Q(1) = 2.43, p= 0.12 Test of8 = 0: z=-2.55, p=0.01 Medium-run effects Sierra Leone Im 89.48 741 [ -0.31, 15.13] 10.80 Egypt Infrastructure 4m 267.98 -14.21 [-32.56, 4.14] 5.65 Djibouti om 45 0.47[ -0.12, 1.06] 13.43 Tunisia 12m 37.54 so 8.20[ 2.76, 13.64] 11.99 Rural DRC 12m 36.08 Hs 18.36[ 14.22, 22.50] 12.56 Urban DRC 25m 94.97 -2.18[ -8.57, 4.21] 11.52 Tunisia 66m 59.78 ——— 15.09[ 1.24, 28.94] 7.53 Heterogeneity: t? = 70.21, ? = 92.52%, H? = 13.37 5.72 [ -1.24, 12.69] Test of 0, = 0;; Q(6) = 87.71, p = 0.00 Test of 6=0: z= 1.61, p=0.11

# Overall

3.64 [ -2.08, 9.37] Heterogeneity: t? = 63.46, I? = 98.39%, H? = 62.30 Test of 0; = 8; Q(8) = 140.89, p = 0.00 Test of 6=0: z= 1.25, p=0.21 Test of group differences: Q,(1) = 5.01, p = 0.03

-30 -20-10 0 10 20 30

Random-effects REML model
# **5.4 Savings and assets**

In Figures 7 and 8, we present the estimated effects of public works programs on the probability of having any savings in the 12 months prior to the survey and a household asset index, respectively.<sup>22</sup> Savings and assets can be used to smooth consumption in case of a shock or to facilitate investments and increase income from agricultural and selfemployment activities. They are both important mechanisms through which the effects of the public works program could sustain in the medium run.

The observed increases in earnings 12 months after program completion (shown in Figure 4) are associated with a higher incidence of savings<sup>23</sup> (Rural DRC, Tunisia, and Côte d’Ivoire) and higher assets (Rural DRC and Tunisia). The short-run increase in earnings in Côte d’Ivoire stems from higher wages from the public works program. However, the impacts 12 months post-program are due to higher profits from selfemployment activities. Public works wages were paid into a bank account, leading to a substantial increase in savings during the program that were partly used to invest in capital for these small business activities (Bertrand, et al., 2021). Similarly, in Rural DRC, the 12month impacts are associated with an increase in self-employment (by 8.5 percentage points), number of total hours worked (increase of almost 25 percent relative to the control mean), and increased ownership of productive assets such as livestock and physical assets such as refrigerators, televisions, air conditions, and motorbikes, that increased by 50 percentage points and 43.2 percentage points, respectively (Mani and Mvukiyehe, 2021). Even though the Ethiopia study does not report any information on the likelihood of saving, households in Ethiopia also used the earnings from public works to save, with eligible households doubling their savings post-program.

Overall, the probability of having savings increases by 5.9 percentage points (95 percent CI: 0.60, 11.20), and the household asset index increases by 0.21 standard deviations (95 percent CI: 0.04, 0.37).

> 22 Note that, compared to employment and earnings, we slightly have less consistent measures on savings and assets across studies.

> 23 An increase in the amount of savings was also documented in Côte d’Ivoire and Ethiopia.

25
|Mean<br>Study<br>Timing Control|||Effect si<br>with95%|ze<br>CI|Weight<br>(%)|
|---|---|---|---|---|---|
|Short-run effects||||||
|Comoros<br>4m<br>6.6|||0.90 [-1.26,|3.06]|8.44|
|Cote d'Ivoire<br>3m<br>47|||{829.00 [26.98,|31.02]|8.46|
|Laos<br>-2m 27.1|||3.56 [-2.24,|9.36]|7.77|
|Djibouti<br>Om<br>22.1|+||6.20[ 1.69,|10.71]|8.06|
|Heterogeneity: 1?= 166.97,? =98.63%,H?= 72.82|||10.01 [-2.80,|22.83]||
|Testof0,= 0;: Q(3)=377.65, p=0.00||||||
|Testof6=0: z= 1.53,p=0.13||||||
|Medium-run effects<br>EgyptCommunity<br>Im<br>1.5|i||4.00[ 1.45,|6.55]|8.40|
|Egypt Infrastructure<br>4m<br>5.1|||0.00 [-0.02,|0.02]|8.56|
|Djibouti<br>om<br>16|||-0.70 [-1.43,|0.03]|8.55|
|Tunisia<br>12m<br>1||||2.60[ 1.23,|3.97]|8.52|
|RuralDRC<br>12m 18.7||+|20.60 [16.88,|24.32]|8.22|
|Coted'Ivoire<br>12m<br>62|;||5.00 [ 2.06,|7.94]|8.34|
|Egypt Community<br>27m<br>1.2|||1.10 [-3.02,|5.22]|8.14|
|Tunisia<br>66m<br>2.2|||-1.20 [-2.38,|-0.02]|8.53|
|Heterogeneity:t?=45.89,<br>2?=99.44%,H?=179.98|||3.80[-0.97,|8.58]||

Medium-run effects Egypt Community Im 1.5 Egypt Infrastructure 4m 5.1 Djibouti om 16 Tunisia 12m 1 Rural DRC 12m 18.7 Cote d'Ivoire 12m 62 Egypt Community 27m 1.2 Tunisia 66m 2.2 Heterogeneity: t? = 45.89, 2? = 99.44%, H? = 179.98 Test of 0, = 0;: Q(7) = 159.77, p = 0.00 Test of9= 0: z= 1.56, p= 0.12

# Overall

<!-- Start of picture text --> _> 5.90 [ 0.60, 11.20]<br>.<br>-5 0 5 1015 20 25 30<br><!-- End of picture text --> Heterogeneity: t? = 85.26, ? = 99.59%, H? = 244.67 Test of 6, = 0; Q(11) = 961.75, p = 0.00 Test of6 = 0: z=2.18, p= 0.03 Test of group differences: Q,(1) = 0.79, p = 0.37 Random-effects REML model
# Control Study Timing Mean

Short-run effects Laos -2m_ -.1837 Heterogeneity: t? = 0.00, I? = .%, H?=. Test of 8, = 8;: Q(0) = 0.00, p=. Test of 8 = 0: z= 0.54, p= 0.59 Effect size Weight with 95% CI (%) 0.03 [ -0.08, 0.14] 14.79 0.03 [ -0.08, 0.14]

## Medium-run effects

Egypt Community lm O01 EgyptInfrastructure 4m _ -.016 Tunisia 12m 263 Rural DRC 12m -.017 Egypt Community 27m -.04 Tunisia 66m 162 Heterogeneity: t? = 0.05, I? = 94.28%, H? = 17.49 Test of 8, = 8;: Q(5) = 160.72, p = 0.00 Test of0 = 0: z= 2.54, p= 0.01

# Overall

0.15 [ -0.02, 0.33] 13.55 -0.01 [ -0.05, 0.03] 15.66 —i— 0.29 [ 0.12, 0.45] 13.63 +E 0.62[ 0.53, 0.71] 15.07 0.11 [-0.01, 0.24] 14.53 —i— 0.28[ 0.07, 0.48] 12.76 << 0.24[ 0.05, 0.42] i 0.21 [ 0.04, 0.37] Heterogeneity: 1? = 0.05, I? = 94.28%, H? = 17.48 Test of 0, = 0,: Q(6) = 162.32, p = 0.00 Test of8 = 0: z= 2.45, p= 0.01 Test of group differences: Q,(1) = 3.66, p = 0.06

-0.1 0 0.10.20.30.40.50.6 0.7

Random-effects REML model
decision-making in the household.<sup>24</sup> The measures vary across countries and tend to be relatively narrow, and only three of the 9 point estimates document a positive increase (Lao PDR during the program, Egypt Community at one-month post-program, and Tunisia at 12 months). In Lao PDR, impacts were found on measures such as intra-household decisionmaking and community participation. The increase in women’s empowerment in Tunisia is driven by a large relative increase in the probability that women respondents have an income-generating activity. The Egypt Community study finds an increase in women’s control over household resources 1-month post-program, but this effect does not persist and there is a reduction (which is not statistically significant) in women’s control over household economic resources in the medium-run. In Comoros and Rural DRC, there is no evidence that the program had any effect on women’s empowerment. The Côte d’Ivoire program targeted youths, and impacts were much larger for women compared to men, but dedicated measures of empowerment were not collected. Overall, evidence shows that public works programs can improve women’s empowerment, but they do not do so systematically. While the uneven evidence may in part be due to limitations in measurement, open questions remain on how to design public works programs to achieve consistent improvements in this dimension.<sup>25</sup>

# **6.2 Psychological well-being**

Public works jobs can provide a sense of purpose and improve psychological well-being among participants, in particular for youth or otherwise underemployed populations. Eight of the 11 studies we review (or 12 of the 15 point estimates) analyze impacts on some form of psychological or subjective well-being measure. Given variations in these measures, we do not attempt to aggregate them. Still, four of the 12 point estimates, or three of the 11 studies (Tunisia, Egypt Infrastructure, and Côte d’Ivoire) find that the public works programs positively affected participants' psychological well-being. Individuals in the treatment group were significantly less likely to report distressing thoughts/memories, feel sad or depressed, or feel irritable as compared to those in the control group. Evidence from Côte d’Ivoire shows that effects on psychological well-being can persist beyond the shortrun, though there are signs of decay between short-run impacts (0.20 standard deviations) and medium-run impacts (0.11 standard deviations). Overall, results show that public works can improve psychological well-being, but these effects are not observed systematically, at least based on the measures available to date across studies.

> 24 The Rural and Urban DRC studies also measure views on women’s access to power and on gender-based violence and, in Comoros, the study measures women’s experiences of physical, emotional, or economic violence in their household. None of these studies, however, finds significant impacts on women’s empowerment. In a spin-off study in Lao PDR, no effect was found on gender-based violence (including no evidence of a backlash effect (Perova et al., 2021).

> 25 There is active research on the topic. For instance, Christian et al. (forthcoming) investigate whether public works can improve women’s empowerment when designed specifically to increase women’s access to earned income and use a much broader measure of women’s empowerment than the studies included in this paper.

28
**Table 5: Directional impacts of public works programs on women’s empowerment and psychological well-being**

|||Survey|Women’s|Psychological|
|---|---|---|---|---|
|||Timing|Empowerment|Well-being|
|||(1)|(2)|(3)|
|Comoros||-4m|0|0|
|Côte d’Ivoire||-3m|NA|+|
|Lao PDR||-2m|+|NA|
|Djibouti||0m|NA|0|
|Sierra Leone||1m|NA|NA|
|Egypt,<br>Arab|Rep.,<br>|1m|+|0|
|Community<br>|||||
|Egypt,<br>Arab|Rep.,<br>|4m|NA|+|
|Infrastructure|||||
|Djibouti||9m|0|0|
|Tunisia||12m|+|+|
|Rural DRC||12m|0|0|
|Ethiopia||12m|NA|NA|
|Côte d’Ivoire||12m|NA|+|
|Urban DRC||25m|0|0|
|Egypt,<br>Arab|Rep.,<br>|27m|0|0|
|Community|||||
|Tunisia||66m|0|0|

Notes: These results are shown to reflect the sign of estimates instead of coefficient estimates or effect sizes because of differences in measurement. Point estimates that are not statistically significant are coded as 0. Countries are ordered by survey timing with respect to program completion, i.e., we first present countries that measure impacts during the program, followed by countries that measure 1 to 66 months post-program impacts. The Egypt Community study refers to psychological well-being as subjective well-being. See Appendix Table A3 for country specific variable definitions on women’s empowerment and psychological wellbeing.

# **7. Conclusion: Lessons for policy and directions for future research**

Public works programs in low-and middle-income countries are designed to offer shortrun employment and earnings opportunities to poor households and are often introduced to facilitate recovery from shocks (e.g., drought, conflict). These programs frequently also aim to sustainably raise living standards via, for example, improvements in assets, selfemployment, savings, and community infrastructure. This paper reviews experimental evidence from 11 public works programs that together offer both internal and external validity across diverse settings. Some important conclusions and policy lessons emerge from the review.

29
First, public works programs generate short-run improvements in employment and earnings. In some cases, the magnitude of impacts on participants’ employment during the program can be large (such as in Comoros or Djibouti). Still, impacts on earnings tend to be small relative to the total program earnings, even during the program, raising questions about program cost-effectiveness. Second, increases in employment and earnings generally fade and do not persist much after individuals have exited the program. A few notable exceptions are found for programs that generate impacts on savings or productive investments linked to self-employment activities (such as in Rural DRC or Côte d’Ivoire). Third, and perhaps surprisingly, there is little evidence of public works programs increasing food consumption expenditures across the studies we review. Finally, there is some evidence of improvements in women’s empowerment and psychological well-being, though these are not found consistently, perhaps due to limitations in measurement.

Overall, the stylized message from the review is that public works programs have been relatively more successful in delivering on their short-run objectives to raise individuals’ employment and earnings, rather than in achieving medium-run impacts on participants’ economic outcomes or broader effects on household-level outcomes. These results also raise questions on how to design programs to achieve more robust impacts on outcomes related to broader policy objectives, for instance women’s empowerment or psychosocial well-being.

Despite the rapid increase in public works experiments in low-and middle-income countries, much remains to be learned. First, we need better data to accurately measure program participation (ideally from administrative information systems) and outcomes at multiple points in time. None of the existing studies measures program take-up and participation and outcomes related to employment, consumption, assets, psychological well-being, and women’s empowerment in the short-run (during the program) and after the program. Second, the duration of program participation and the level of wages vary widely across countries. It is unclear what the optimal program length or wage should be, but these parameters can affect impacts on short-run earnings, consumption, savings and investments, and those may in turn affect outcomes in the medium term. Third, it would be valuable to identify effective “public works plus” interventions that can facilitate transitions from temporary public works programs to more steady forms of employment, including in self-employment. Currently, there is very little evidence on the value-added of skills training or savings intervention in facilitating the transition from public works. In particular, given the recent success of multi-faceted economic inclusion programs (e.g., Banerjee et al., 2015, Bedoya et al., 2019; Bossuroy et al., 2022) in poor settings, it would be useful for future studies to experiment with different modalities and bundles of savings facilitation, capital transfers, or skills interventions (e.g., vocational, socioemotional, or 30
business training) as complements to the public works program.<sup>26</sup> Fourth, additional research using broader measures of women’s empowerment is also critical to better understand intra-household dynamics, such as the conditions under which improvements in women participants’ individual earnings improve household consumption and their decision-making power. Fifth, there is a need for more rigorous cost-effectiveness analysis. Only two studies in the literature offer detailed cost-effectiveness analysis (Bertrand et al., 2021 and Rosas et al., 2016). Future experiments in this area must pay close attention to collecting cost data to allow for a clear comparison between marginal benefits and marginal costs of the intervention. Lastly, while our review has not focused on general equilibrium effects, the value of public goods provided, or other social externalities from public works programs, are also first-order questions on which more evidence would be needed across settings.

> 26 <mark>In Côte d’Ivoire the basic public works program induced substantial savings, possibly facilitated by the payment of wages into bank accounts.  Some participants were also offered a complementary basic entrepreneurship training to facilitate the transition to self-employment activities after their exit from the program, and other participants were offered a wage employment sensitization training to help them prepare and apply for wage jobs. Post-program impacts on being employed, total earnings, and the number of hours worked were not statistically different across the different treatment arms. Overall, the changes in skills and practices were small in magnitude and did not generate earnings beyond those induced by the basic public works program, suggesting limited value-added of the complementary interventions. In contrast, in Urban DRC, participants that received a combined package of public works programs, incentivized savings, and hard skills training experienced large impacts on savings and investment, indicative of medium-run improvements. Thus, the results suggest that a multifaceted approach may be most effective in supporting livelihoods in Urban DRC, where participation in the public works program alone did not significantly improve economic welfare outcomes in the medium run.</mark>

31