metadata: authors: 0: author: Jeong,Dahyeon
1: author: Trako,Iva
count: World
docna: 0: docna: Cash and In-Kind Transfers in Humanitarian
            Settings : A Review of Evidence and Knowledge Gaps
docty: Policy Research Working Paper
abstracts: cdata!: Over the past decade, humanitarian
            assistance and social protection have increasingly emerged
            as a policy response tool to support crisis-affected
            populations facing conflict or natural disasters. This paper
            presents a descriptive literature review of non-contributory
            humanitarian assistance interventions in low-and-middle
            income countries. It uses evidence from twenty-one
            experimental or quasi-experimental studies to understand the
            effects on five outcome categories: (i) basic needs, (ii)
            financial outcomes, (iii) gender, (iv) human development,
            and (v) social cohesion. The findings show that gender,
            human development, and social cohesion are the least
            explored outcomes in humanitarian contexts. Moreover,
            evidence is scarce on the comparative performance of
            different modalities (for example, cash vs. in-kind),
            targeting mechanisms, cost-effectiveness of alternative
            interventions, heterogeneity analysis, and longer-term
            effects of interventions. The paper makes the case that
            there is a high dividend to be earned from conducting more
            impact evaluations in humanitarian settings.
display_title: Cash and In-Kind Transfers in
            Humanitarian Settings : A Review of Evidence and Knowledge Gaps
listing_relative_url: /research/2022/04/33804168/cash-in-kind-transfers-humanitarian-settings-review-evidence-knowledge-gaps
topicv3: Gender and Human Development m1326908 919,Food and In-kind Transfers m1326956 1609,Social Protection and Labor m1327194 697,Cash Transfers m1327437 1608,Humanitarian Aid m1503719 3039
publishtoextweb_dt: 2022-04-27T00:00:00Z
dois: 10.1596/1813-9450-10026
origu: Off of Sr VP Dev Econ/Chief Econ (DECVP)
author_keys: 0: author_key: 1919507
1: author_key: 1630031
model_extractions: Policy Research Working Paper 10026 Cash and In-Kind Transfers in Humanitarian Settings A Review of Evidence and Knowledge Gaps _Dahyeon Jeong Iva Trako_ Development Economics Development Impact Evaluation Group April 2022
Policy Research Working Paper 10026

# **Abstract**

Over the past decade, humanitarian assistance and social protection have increasingly emerged as a policy response tool to support crisis-affected populations facing conflict or natural disasters. This paper presents a descriptive literature review of non-contributory humanitarian assistance interventions in low-and-middle income countries. It uses evidence from twenty-one experimental or quasi-experimental studies to understand the effects on five outcome categories: (i) basic needs, (ii) financial outcomes, (iii) gender, (iv) human development, and (v) social cohesion. The findings show that gender, human development, and social cohesion are the least explored outcomes in humanitarian contexts. Moreover, evidence is scarce on the comparative performance of different modalities (for example, cash vs. in-kind), targeting mechanisms, cost-effectiveness of alternative interventions, heterogeneity analysis, and longer-term effects of interventions. The paper makes the case that there is a high dividend to be earned from conducting more impact evaluations in humanitarian settings.

This paper is a product of the Development Impact Evaluation Group, Development Economics. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at itrako@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Cash and In-Kind Transfers in Humanitarian Settings: A Review of Evidence and Knowledge Gaps<sup>*</sup>

Dahyeon Jeong<sup>†</sup> Iva Trako<sup>‡</sup> Originally published in the Policy Research Working Paper Series on _April 2022._ This version is updated on _September 2022._ To obtain the originally published version, please email prwp@worldbank.org.

**Keywords:** Systematic review, social protection, humanitarian settings, cash transfers, in-kind transfers, basic needs, financial outcomes, human development, gender, social cohesion

**JEL Classification:** D04, H84, I00

> * We are thankful for the support and guidance provided by Felipe Dunsch, Kristen McCollum, Jonas Heirman, and Andrea Cook from the World Food Programme’s Office of Evaluation (OEV). In addition, the team would like to thank Florence Kondylis, Marcus Holmlund, Benedetta Lerva, Patrick Premand, Vincenzo di Maro, Ugo Gentilini, Mattias Lundberg, and Margaret Grosh for their valuable comments and suggestions. We are also grateful to Eric Jospe and Nathalie Anne Kiersznowski for their research assistance. Finally, we would like to acknowledge the efforts of Emma Frankham, Meghan Quinn and OEV’s Comms Unit in providing copy-editing support. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.

> † Economist. Development Impact Evaluation (DIME). The World Bank. Email: dahyeonjeong@worldbank.org

> ‡ Economist. Development Impact Evaluation (DIME). The World Bank. Email: itrako@worldbank.org
# **1. Introduction**

A humanitarian crisis is a singular event, or series of events, that threatens the health, safety, or well-being of a community or large group of people.<sup>1</sup> According to the 2021 Global Humanitarian Overview Report, an estimated 235 million people would need humanitarian assistance and protection in 2021, an increase of more than 65 million since 2020 (UNOCHA 2020). In addition, over 91.9 million individuals were estimated to have been forcibly displaced worldwide in 2020 due to prosecution, conflict, generalized violence, or other human rights — violations , representing an increase for the eighth consecutive year (UNHCR 2020). The need for humanitarian assistance and social protection has been exacerbated by the COVID-19 pandemic, which presents an unprecedented challenge to the humanitarian system in both scale and complexity. It is, therefore, more important than ever to have reliable and rigorous evidence on both the impact of humanitarian response programs, that is, “what works,” in addressing the needs of crisis-affected populations, as well as how cost-effective the implementation of such programs is, so that donors, aid agencies, and policy makers can make informed decisions and target those in greatest need.

Over the last decade, social protection programs in humanitarian contexts have increasingly emerged as a policy response tool to address poverty and hunger (Ulrichs and Sabates-Wheeler 2018).<sup>2</sup> Moreover, as part of the commitments under Sustainable Development Goal 1, the global community has agreed to expand the coverage of social protection measures for all and to achieve substantial coverage of the poor and the vulnerable by 2030 (United Nations 2015).<sup>3</sup> This expansion also includes the scale-up of social protection programs in humanitarian contexts (fragile and conflict-affected populations), which we define in this paper as humanitarian assistance interventions.

> 1 For more information on humanitarian crises, see Humanitarian Coalition (last accessed December 2021). 2 In more stable developing contexts, a vast literature in social sciences shows that social protection programs help reduce poverty and inequality, enhance livelihoods, and have long-term positive impacts on human capital development (Baird et al. 2014; Bastagli et al. 2016; Davis et al. 2016; Handa et al. 2018; Hidrobo et al. 2018). 3 According to the 2020 Global Humanitarian Assistance Report, there is a clear upward trend in the use of social protection programs in the form of cash and vouchers. For example, the use of cash-based transfers in humanitarian crises has doubled since 2015, amounting to a total volume of US$5.6 billion or 17.9 percent of the global humanitarian assistance in 2019. However, cash and voucher transfers still account for a minor share of humanitarian assistance, with the majority of the humanitarian portfolio being provided in-kind (CaLP 2020; Overseas Development Institute 2015).

2
Social protection in development settings comprises social assistance tools (e.g., unconditional and conditional cash transfers, vouchers, social insurance, and labor market interventions) that are generally operated by governments as a response to long-term structural issues. On the other hand, humanitarian assistance tends to focus on emergency or one-off responses to address the urgent needs of crises-affected populations, and it is mainly implemented through international humanitarian actors in contexts where governments are too fragile to respond. Even though humanitarian crises are generally attributed to a single and distinct event that occurs with little or no warning (sudden onset), most humanitarian crises are complex, protracted, and chronic (slow onset), evolving from a series of events that exacerbate vulnerabilities of a population over a prolonged period (UNOCHA 2018).

The main objective of this review is to assess and summarize the state of existing knowledge on the effects of humanitarian assistance interventions and to identify evidence gaps in the literature for both future research and for evidence-based decision-making by policy makers and practitioners. In doing so, we address two research questions within the context of humanitarian settings. First, what are the impacts of humanitarian assistance programs on individual and household-level outcomes? Second, what is the relative effectiveness and efficiency of different modalities and delivery channels in achieving their objectives?

Drawing on evidence from experimental and quasi-experimental studies conducted in low- and middle-income countries and using a narrative approach, we examine the overall effects of humanitarian assistance programs on a wide range of individual and household-level outcomes, such as basic needs, financial outcomes, gender, human development, and social cohesion. We focus our review on the impact of humanitarian assistance interventions in the form of unconditional cash transfers (UCTs), conditional cash transfers (CCTs), food transfers (FT), vouchers, and public works (PW), as these are the most widely used forms of assistance and, therefore, a priority for evidence generation in the humanitarian sector.

There have been a few systematic reviews on cash and in-kind interventions in humanitarian contexts (Gentilini 2016; Doocy and Tappis 2018; Aurino and Giunti 2021).<sup>4</sup> For example, Doocy and Tappis (2018) review the impacts and value for money of cash-based

> 4 Relatedly, Puri et al. (2017) assess how rigorous impact evaluations can help inform and improve different types of interventions in humanitarian emergencies, including cash and in-kind transfers.

3
interventions on emergency-affected populations. Gentilini (2016) focuses on the question of which modality is more cost-effective, and reviews studies that compare alternative transfer modalities (e.g., cash vs. food) within a consistent evaluation framework. Aurino and Giunti’s (2021) review is the closest to our study in terms of the study inclusion criteria. They include studies that measure the impact of emergency cash, food, or other in-kind transfers, but their study differs in its focus on child development outcomes. Our review complements these existing reviews by including more recent rigorous studies that evaluate the impact of unconditional or conditional cash and in-kind transfers in humanitarian settings. We also analyze the impacts on a more comprehensive set of outcomes that have been overlooked by the existing literature, including human capital, women’s empowerment, and social cohesion. Therefore, this systematic review provides an overview of the evidence to date on humanitarian assistance programs and discusses several promising future research directions to help close the knowledge gaps, flagging particularly promising interventions for practitioners and policymakers.

Taken together, five stylized findings emerge from our review. First, despite the growing use of social assistance programs in humanitarian settings, there is relatively little rigorous research on what works, for whom, and why. We find that only twenty-one studies use experimental or quasi-experimental methods to rigorously assess the impact of humanitarian assistance programs, and only two of them discuss the cost-effectiveness/cost-benefit of such interventions.

Second, the evidence base for humanitarian assistance programs across different sectors or outcomes of interest varies significantly. On the one hand, we find that most evidence in humanitarian settings concentrates on basic needs outcomes, such as food security, food and nonfood expenditure, and coping strategies. This is followed by studies that report household financial outcomes such as assets, income, credit, and savings, where the evidence can be characterized as emerging. On the other hand, there is a dearth of studies examining human development outcomes, such as health, education, and labor, as well as gender and social cohesion. In particular, we find that women’s empowerment and gender-based violence outcomes are the least explored in the context of humanitarian assistance programs.

Third, in terms of the impact of humanitarian assistance programs, the limited evidence points to mixed and inconclusive results for outcomes that are not typically studied. While the 4
literature suggests that most humanitarian assistance programs can effectively improve basic needs outcomes, such as food security and expenditures, it is difficult to draw general conclusions about other outcomes. The small number of studies show mixed conclusions on income, credit and savings, labor, gender-based outcomes, and social cohesion. For this reason, it is important to emphasize the relevance of context-specificity for successful approaches to humanitarian assistance programs. This means that program design needs to be tailored to the specific type of crisis and to the broader context in which the crisis takes place, as these factors are some of the main determinants for the feasibility of implementation and uptake by targeted beneficiaries.

Fourth, regarding the relative effectiveness of different modalities, the decision about the most appropriate modality versus another cannot be generalized and pre-determined. The modality performance depends on the nature of the humanitarian crisis (sudden onset vs. slow onset), the objective of the program or the main outcome of interest, the profile of the targeted population, implementation costs, and local market capacity, among others. This reflection highlights the need for future research to better understand the conditions under which cash transfers are more effective than in-kind transfers.

Fifth, the evidence from our review indicates that cash transfers are more efficient to deliver than in-kind modalities, implying that they might be more cost-effective on average. In particular, studies suggest that mobile money cash transfers can be the most efficient, followed by manual cash delivery, vouchers, and food transfers being the most expensive way to deliver assistance in humanitarian settings. Given the lack of evidence on cost-effectiveness, we argue that future impact evaluations should provide a more robust analysis for cost calculations of humanitarian assistance programs.

This paper is organized as follows. Section 2 describes the methodological approach used in this systematic review. Section 3 presents the impacts of humanitarian assistance programs on the main outcomes across five outcome categories using rigorous impact evaluations. In this section, we also discuss the efficiency of these interventions in terms of implementation costs. In Section 4, we assess the relative effectiveness of different transfer modalities. Section 5 discusses the evidence gaps and proposes new areas for research. Finally, Section 6 summarizes our key findings and discusses implications for policy making.

5
# **2. Methodology**

To answer our main research questions, we conducted a systematic review of the experimental and quasi-experimental literature focused on humanitarian assistance programs. In this review, we organize and present our findings in different outcome domains at the individual and household levels. This section describes the process we followed for identifying studies to include in our analysis, which consisted of three iterative stages: (i) search and inclusion criteria, (ii) screening process, and (iii) data extraction. We also describe how we mapped study-specific outcomes onto broader outcome categories.

## **Identification of Studies and Inclusion Criteria**

We collected information on humanitarian assistance interventions through computeraided searches and reviews of reference lists in the identified studies. A primary systematic search for peer-reviewed and gray literature was conducted in several search engines and databases using a combination of predetermined keywords and vocabulary for social assistance and emergencies, which are presented in **Table 1** . In addition to the database search, the bibliographies and citations of included studies were thoroughly analyzed for further studies that met inclusion criteria. We also reviewed websites of organizations working in the humanitarian field (e.g., Oxfam, Mercy Corps) to search for relevant gray literature. Lastly, we contacted experts and researchers who have frequently published on the impacts of social safety net programs in peer-reviewed journals and asked them to indicate any other relevant published studies that we could incorporate into our review.

**Table 2** outlines the inclusion criteria. First, we sought to include studies situated in humanitarian crises. For this review, a humanitarian crisis is defined as a singular event or a series of events that threaten the health, safety, or well-being of a community or large group of people. Among policy makers, there are two recognized types of humanitarian crises: (i) sudden onset, which is a single, distinct event that occurs with little or no warning (e.g., earthquake); and (ii) slow onset or protracted crises, which are more complex humanitarian crises that demand prolonged assistance over several years (e.g., severe drought or ongoing conflict). In this review, we focus on both types of humanitarian assistance scenarios. These humanitarian crises are often further categorized into three broad categories: (i) natural disasters, (ii) conflict-related emergencies, and (iii) health outbreaks. Programs implemented before the onset of the

6
humanitarian crises and evaluated during the humanitarian crises were excluded from the review.

We limit our analysis to programs that provided humanitarian assistance to crisis-affected populations in the form of unconditional cash transfers (UCTs), conditional cash transfers (CCTs), food transfers (FT), vouchers, and public works (PW), as these are the most widely used forms of transfers and, therefore, an evidence priority in the humanitarian sector. We excluded other types of interventions in humanitarian settings such as microfinance, WASH, community-driven development programs or peacebuilding activities, and health and medical assistance. We decided to focus primarily on non-contributory humanitarian assistance interventions because many governments and humanitarian actors have recently started to use social protection tools in humanitarian settings, but very little is known about their effectiveness in such contexts.<sup>5</sup> Studies included in this review were required to employ either experimental or quasiexperimental designs that can isolate the causal impact of humanitarian assistance programs on outcomes of crisis-affected populations. We required that studies were either randomized control trials or that they estimated intervention effects using one of the following quasi-experimental methods: difference-in-difference, propensity score matching, instrumental variables, regression discontinuity design, or another causal estimation technique. We also included experimental studies that did not have a pure control group, which are particularly relevant in humanitarian settings. Studies that did not establish any counterfactual were excluded.

The search did not impose restrictions on the timing of publication, although focusing on experimental and quasi-experimental evaluation designs effectively limits our attention to studies published after 2007.

Regarding types of studies, publication in a peer-reviewed journal was a preferred requirement for inclusion. We also included studies from working paper series (e.g., World Bank Policy Research, CDG, IFPRI) and technical reports if they included a suggested formal institutional citation. This analysis is also restricted to low- and middle-income countries (as

> 5 It is important to note that the evidence from development contexts cannot be applied directly to complex humanitarian contexts since the latter are characterized by harsher conditions such as increased economic, social, institutional, and security challenges. This situation creates an important knowledge gap, particularly in terms of the design and implementation of effective social protection measures in humanitarian settings.

7
defined by the country-income groupings of the World Bank), where most humanitarian assistance programs are implemented, with no other explicit population exclusion criteria.

# **Screening Process**

The screening of studies and the application of inclusion and exclusion criteria took place in two rounds. In the first round, all studies that appeared in the search process were classified as potentially eligible or excluded by only reviewing the citation and abstract. Studies were deemed ineligible if they did not meet the following inclusion criteria: (i) not a humanitarian setting, (ii) not a social assistance program, (iii) duplicates, and (iii) of no relevance to the study question. The majority of ineligible studies were either not from a humanitarian setting or were duplicates of another report. If duplicate studies published in different formats appeared in the search process, studies published in an academic journal were included rather than identical studies published as technical reports.

In the second screening round, each potentially eligible study from the first screening was assigned two reviewers for full-text review. Each reviewer read the full text of the study and assessed study features meant to proxy for study quality in terms of methodology, including the use of an evaluation design that would generate causal impacts. For instance, to be included in our final sample, studies needed to isolate the impact of the humanitarian assistance program using some sort of comparison group and report the precision of estimated effects. Considering these criteria, each reviewer rated studies as “For Review” or “Excluded,” and the reason for exclusion was recorded. Studies for which there were any doubts or disagreements about potential eligibility were discussed by the authors to arrive at a final rating. This process resulted in twenty-one included studies that were assigned for data extraction.

# **Data Extraction and Description of Included Studies**

We extracted three types of information from each of the included studies: study, intervention (contrast), and outcomes. _Studies_ are defined as independent publications of humanitarian assistance programs (e.g., journal articles, working papers, technical reports). _Interventions (Contrasts)_ represent the different treatment arms/contrast groups of a program within a study. For example, a study might provide one group of beneficiaries with a cash transfer, a second group with a food transfer, a third group with a voucher, and a fourth group 8
with no transfer. For our analysis, these would produce four interventions/contrasts. _Outcomes_ are comparisons of treatment and comparison groups on a set of outcome measures. Given the small sample size of studies in this review, we did not have enough power to conduct a metaregression analysis that aggregates evidence across studies to estimate an average effect size for each class of outcomes. For this reason, we conducted a narrative literature review on the state of evidence regarding humanitarian assistance interventions.

# **2.3.1 Study level**

**Table 3** provides the characteristics of the studies in our sample, and shows that half of the studies are published in peer-reviewed journals, one-third are technical reports from research institutions, and the remaining 14 percent are working papers.<sup>6</sup> Less than half of the studies (43 percent) used experimental methods (RCTs), highlighting the methodological, logistical, and ethical challenges of conducting RCTs in humanitarian settings. Among those that used RCTs, 20 percent do not have a pure control group, which means they compared different modalities (e.g., one group receives conditional cash transfers while the other group receives food transfers). Around one-third of the studies also reported cost analyses of the evaluated interventions, but only four discussed the cost-efficiency of different modalities.

**Figure 1** shows the geographic distribution of the studies, demonstrating coverage across 12 countries, but with a concentration of research in Africa.<sup>7</sup> The 21 studies correspond to 17 humanitarian assistance programs across the world. On average, there are 1.2 studies per — program sometimes we reviewed more than one study on the same humanitarian assistance program that reported different outcome variables.

Most of the studies (76 percent) are from a conflict-affected setting, evenly split between refugee camps and non-camps, while the remaining five (24 percent) were in a natural disaster setting. We did not find any published studies that report humanitarian assistance impacts from a health outbreak setting, which also met our inclusion criteria. The publication year ranges from 2007 to 2020, with the number of studies growing in recent years. **Figure 2** illustrates the

> 6 At the study level, we document author name(s), publication year, country, publication type, humanitarian crisis type, regional distribution, and impact evaluation design.

> 7 The twenty-one studies consist of eight studies in Africa (Democratic Republic of Congo (DRC), Ethiopia, Mali, Niger, and Uganda), seven studies in the Middle East (Jordan, Lebanon, and Yemen), three studies in Latin America (Ecuador), and three studies in Asia (Fiji, the Philippines, and Sri Lanka).

9
increasing trend in publications, particularly since 2019. **Appendix Table 1** presents additional details of each study, including intervention components and evaluation design.

# **2.3.2 Intervention (contrast) level**

We also extracted information about the evaluated interventions, and **Table 4** summarizes the characteristics of the treatment arms of the included studies.<sup>8</sup> **Appendix Table 2** presents information about additional characteristics of the programs evaluated in each country.

The types of interventions used by studies in this review can be broadly grouped into cash transfers and food transfers. Cash is handed out manually or delivered as mobile money and/or cash vouchers. Similarly, food is distributed as an in-kind transfer (individually or through schools) or as a voucher that can be exchanged for food in shops or at organized fairs. We assigned humanitarian assistance interventions to _at least_ one of five intervention categories: (i) unconditional cash transfers (UCT), (ii) conditional cash transfers (CCT), (iii) vouchers, (iv) food transfers (FT), and (v) public works (PW). Each humanitarian assistance _program_ combines one or more of these five intervention categories, implemented mainly by international organizations (e.g., World Food Programme, UNICEF) and non-governmental organizations (e.g., Concern Worldwide, Mercy Corps, IRC, Oxfam). In fewer cases, these programs have also been implemented by governments.

We reviewed forty-two humanitarian assistance contrasts from twenty-one studies covering low- and middle-income countries. Regarding the intervention or contrast type, most studies focused on UCTs, followed by food transfers, vouchers, CCTs, and public works. There are, on average, 2.05 contrasts per study, ranging from two to four treatment arms (including the control group and studies without a pure control).<sup>9</sup> Since one of the focuses of this review is the relative effectiveness of social assistance modalities, it is important to understand the underlying characteristics of each of the five types of interventions in a humanitarian setting.

**_UCT interventions (20)._** Unconditional cash transfers are by far the most common intervention type included in our review. Twenty UCT contrasts/interventions correspond to

> 8 We identified several important impact evaluation design characteristics, such as intervention category, target population, intervention duration, sample size, recipient identity, transfer schedule, payment type (multiple or single), total payment value, and cost per beneficiary,.

> 9 Several studies have also pooled their interventions (e.g., CCT, Voucher, and FT) and compared them to the control group. These cases are not included in the intervention categories.

10
— eleven studies five in the Middle East (Lebanon and the Republic of Yemen), three in Asia (Fiji, Sri Lanka, and the Philippines), and three in Africa (the Democratic Republic of Congo and Niger). Four of these studies used random assignment without a pure control, while the remaining seven used quasi-experimental methods. Studies that used random assignment without a pure control directly compare different UCT modalities (e.g., single payment vs. multiple payments), or they compare UCTs to food transfer or voucher programs. Programs that used non-random assignment compare UCTs to the control group. This type of intervention is used more in conflict-affected settings (60 percent) than in natural disaster settings (40 percent).

**_Food transfer interventions (11)._** Food transfers are the second most common intervention type in our review. Eleven food transfer contrasts correspond to nine studies—four in Africa (Ethiopia, Mali, and Uganda), one in Asia (Sri Lanka), three in Latin America (Ecuador), and one in the Middle East (Yemen). Four of these studies used random assignment, one without a pure control (e.g., general food distribution vs. school feeding) and three with a pure control (e.g., food transfers compared to the control group). The remaining five studies used non-random assignment and contrasted food transfers with the control group. Food transfer interventions were more likely to be implemented in conflict-affected settings (67 percent), but some evidence exists in natural disaster settings.

**_Voucher interventions (5)._** There are five voucher interventions in this review that — correspond to three studies two in Africa (DRC) and one in Latin America (Ecuador). All of these studies used random assignment (two with a pure control and one without a pure control), and they were all implemented in a conflict-affected setting.

**_CCT interventions (4)._** CCT interventions are not very popular in humanitarian settings. There are four conditional cash transfer interventions in this review that correspond to two — studies one in the Middle East (Yemen) and one in Latin America (Ecuador). Both studies used random assignment with a pure control, and they were only implemented in conflict-affected settings.

**_Public works interventions (2)._** There are only two public work interventions in this review (a cash-for-work intervention and a food-for-work intervention) that correspond to two — studies one in Africa (Ethiopia) and one in the Middle East (Jordan). Neither intervention used

11
random assignment, and public works interventions were used equally in conflict-affected and natural disaster settings.

The humanitarian assistance interventions included in this review mainly targeted crisisaffected populations, such as internally displaced individuals, refugees, severely food-insecure households, and households affected by natural disasters. More than half of these humanitarian assistance interventions (60 percent) targeted the household head (both men and women), while 38 percent exclusively targeted adult women. Only one intervention targeted the primary caregiver of children aged 5–14. Most of these interventions were implemented in a rural setting, with only three being implemented in both rural and urban areas. Sample sizes varied between 252 and 11,500 households, with an average of 2,078. The duration of the intervention varied between 4 and 24 months, with most being less than 12 months.

# **2.3.3 Outcome level**

In this review, we classified all outcome measures as belonging to one of five categories: (i) basic needs (e.g., caloric intake or availability, the value of food consumed or food expenditure, dietary diversity, food insecurity, and coping strategies); (ii) financial outcomes (e.g., livestock, nonfarm productive assets, farm productive assets, land, and savings); (iii) gender (e.g., women’s empowerment and gender-based violence); (iv) human development (e.g., education outcomes, health and nutrition, and labor force participation outcomes); and (v) social cohesion outcomes (e.g., social participation and trust in institutions).

To identify evidence gaps and applied research priorities, we consider a basic metric of the level of evidence to be the number of rigorous studies per outcome category, following Gentilini (2016). Given the limitations of conducting research in humanitarian settings and the possible threats to external validity, it is important to interpret these results with caution. To assess the relative level of existing evidence, we define as “substantial” the evidence base informed by more than ten rigorous impact evaluation studies in humanitarian settings. In cases where such a number is between five and ten, the evidence can be considered “emerging,” while if there are fewer than five, it may be deemed “limited.” Where no evaluation evidence was available, evidence is clearly “absent.”

**Table 5** presents the level of evidence across the different outcome categories. Overall, we find a wide range of evidence bases for humanitarian assistance programs across different

12
settings. It can be argued that most evidence in humanitarian settings concentrates on basic needs outcomes, such as food security, food and non-food expenditure, and coping strategies. Therefore, the evidence base for basic needs outcomes can be classified as substantial relative to other categories. After basic needs, financial outcomes (which include asset ownership, income, and, credit and savings) are the second most reported outcome group. The evidence base for financial outcomes is emerging. However, less evidence exists on the impact of humanitarian assistance programs on human development, such as education, health, economic opportunities (e.g., labor), and especially social cohesion and gender-related outcomes. For these outcome categories, the evidence base is limited. In particular, women’s empowerment and gender-based violence outcomes are some of the least explored impact measures of humanitarian assistance interventions.<sup>10</sup> Given these findings, policy makers should carefully consider this unbalanced evidence base when deciding on interventions and reforms in humanitarian settings.

# **3. Main Results**

This section explores the results of the twenty-one studies included in the review using five outcome categories. The first outcome category is basic needs, which includes food security, food and non-food expenditures, and coping strategies. The second category is financial outcomes consisting of income, assets, and credit and savings. The third category explores human development outcomes, such as education, labor-force participation, health, and subjective wellbeing. The fourth category includes gender-related outcomes such as gender-based violence and women’s empowerment. Lastly, the fifth category explores social cohesion outcomes.

## **Basic Needs Outcomes**

### **Food security**

Food security is the most reported outcome in the humanitarian assistance literature. Vulnerable populations in humanitarian settings often face high levels of food insecurity, which disproportionately affect households living in poverty. Children are particularly vulnerable to

> 10 In our review, twelve studies report outcomes on non-food expenditure, ten on food security, nine on food expenditure, nine on assets, seven on coping strategies, five on credit and savings, five on health, five on education, four on labor, four on social cohesion, four on women’s empowerment, three on income, two on subjective well-being, and one on gender-based violence. Some of the studies evaluated more than one outcome, while others only focused on one particular outcome. **Appendix Table 3** indicates what type of outcomes are reported in each study included in the review.

13
food insecurity, because adequate diets and nutritious foods are crucial for child development. In our review, we found twelve studies that reported the effects of humanitarian assistance programs on household-level food security outcomes, nine of which compared the intervention to a pure control group. These studies used various ways to measure food security, ranging from its simplest form (e.g., number of meals eaten per day) to standard indicators (e.g., Household Dietary Diversity Score, Food Consumption Score, Food Insecurity Experience Scale).<sup>11</sup> Most of the studies in our review find positive and statistically significant effects of humanitarian assistance programs on food security outcomes (Gilligan and Hoddinott 2007; Tusiime et al. 2013; Hidrobo et al. 2014; Bonilla et al. 2017; Kurdi et al. 2019; Tranchant et al. 2019; Chaaban et al. 2020; Quattrochi et al. 2020). Four of these studies find improvements in Household Dietary Diversity Score (HDDS), although some of the coefficients are not statistically significant. The only exception is Lombardini and Mager (2019), who find that a cash-for-work intervention in a refugee camp in Jordan did not have any positive effect on food security.

Overall, these results suggest that most types of humanitarian assistance interventions have large impacts on food security outcomes in fragile and conflict-affected settings, not only through increased consumption but also through improved quality of diets and less severe experiences of food security. One limitation of the current evidence is that food security measures are mainly measured at the household level, which leaves a gap of knowledge regarding the intrahousehold distribution of food consumed and especially the food security situation among children.

# **Food expenditure**

While food security is typically measured by looking at consumption and dietary patterns in food groups, one can look at the dollar expenditure on food purchases as a complementary measure of food security. Food expenditure can be defined as the amount of money spent on food in any given period. This measure can be particularly useful for people who purchase most of their food rather than produce it themselves. In our review, seven studies look at the impact of humanitarian assistance programs on food expenditure, four of which compared the intervention

> 11 For detailed descriptions of different indicators, see “Data4Diets: Food Security Indicators” from Tufts University Friedman School of Nutrition Science and Policy.

14
to a pure control group (Tusiime et al. 2013; Lehmann and Masterson 2014; Kurdi et al. 2019; Tranchant et al. 2019).

For cash transfer interventions, Lehmann and Masterson (2014) find that unconditional cash transfers (UCTs) to Syrian refugees totaling US$575<sup>12</sup> over five months increased food and water expenditure by $25 per month relative to the control group.<sup>13</sup> Similarly, Kurdi et al. (2019) find that CCTs of $30/month in Yemen increased spending on non-staple food items. However, they find no impact on monthly spending.

In terms of food assistance interventions, Tranchant et al. (2019) examined the impact of a food assistance program in the form of general food distribution (GFD) and school feeding (SF) on food expenditure outcomes during conflict in northern Mali. They find that both GFD and SF increased monthly food expenditures by approximately 20 percent. In contrast, Tusiime et al. (2013) find that the GFD of a World Food Programme (WFP) intervention in northern Uganda, which provided at least 40 percent of the recommended dietary allowance, _decreased_ food expenditure by 35 percent.

Overall, these findings suggest that in settings characterized by chronic food insecurity and among conflict-affected populations, UCTs and food transfers can positively impact the food expenditure of vulnerable populations.<sup>14</sup> It also suggests that food expenditure alone is not the best measure of food security; the impact on food expenditure can be positive or negative depending on the intervention type. For example, food expenditure may decrease when direct food transfers are provided to recipients as there is less need to purchase food with their own money. At the same time, expenditure may increase if they spend more on nutritious foods rather than calorie-based staple foods. When the type of aid is cash, it is most likely to increase food expenditure if they were previously food insecure.

# **Non-food expenditure**

> 12 All dollar amounts are US dollars unless indicated.

> 13 In the winterization cash assistance program for Syrian refugees in Lebanon, both the treatment and control group also received a restricted food voucher of approximately $30 per person per month. In this review, we focus on the winterization cash transfer of $575 that was only received by the treatment group.

> 14 These effects on food expenditure are similar to those in the general literature on social protection. For example, a recent meta-analysis of social assistance programs, including 48 studies of 39 social protection programs, found that transfers increased monthly food expenditure by 17 percent on average (Hidrobo et al. 2018).

15
While food security may be the priority in social assistance, people in emergencies also need better clothing, housing, health, and education. Housing and health are especially important for those injured and whose homes are damaged by natural disasters. In our review, non-food expenditure is, in fact, the most reported outcome, with twelve studies in total. In general, most studies with a pure control find that humanitarian assistance recipients significantly increased non-food expenditure, including clothing, heating supplies, housing material, education, health expenses, school fees, agricultural inputs, and assets (Quattrochi et al. 2020; Hidrobo et al. 2014; Lehmann and Masterson 2014; Hidrobo et al. 2016; Bonilla et al. 2017; de Hoop et al. 2019; Tranchant et al. 2019; Chaaban et al. 2020). Only Tusiime et al. (2013) find no impact of food transfers on non-food expenditure in Northern Uganda in a conflict setting.

Similar to the food security outcomes, these results also suggest that various modalities of humanitarian assistance programs effectively improve non-food expenditure compared to the control group, with UCTs having the greatest impact. Since humanitarian assistance — — interventions either cash or in-kind constitute an increase in real income, some of that extra income is expected to be spent on non-food consumption whenever the local markets for these items are available in these fragile contexts.

# **Coping strategies**

Many food-insecure households employ various strategies to cope with a lack of food and income. In the humanitarian context, coping mechanisms are typically used in reference to the negative or harmful strategies individuals or households use in difficult economic situations that may produce longer-term negative consequences. Economic-related coping strategies may include skipping meals, selling productive or non-productive assets, and being forced to move, while employment-related coping strategies may include child labor, family separation, and sexual exploitation. Seven studies in our review report on the use of coping strategies in humanitarian settings, covering mostly conflict-affected populations in DRC, Lebanon, Mali, Niger, Uganda, and Yemen.

Overall, four out of seven studies with a pure control find that humanitarian assistance beneficiaries switch to better coping strategies (Tusiime et al. 2013; Lehmann and Masterson 2014; Bonilla et al. 2017; Schwab 2019), one study finds no effect (Quattrochi et al. 2020), and two studies report mixed results (Aurino et al. 2019; Kurdi et al. 2019).

16
For instance, Lehmann and Masterson (2014) examined the effects of an unconditional cash payment to support Syrian refugees living in Lebanon to support them in the winter months. They find that Syrian refugees who received the winterization cash assistance were less likely to reduce meal frequency and portion size. Notably, treatment households were 50 percent less likely to have their children work and sell their productive assets.<sup>15</sup> Bonilla et al. (2017) find that unconditional cash transfers to vulnerable displaced households in eastern DRC reduced the percentage of households skipping medical treatment due to lack of money. Similarly, Schwab (2019) finds that both food and cash transfers decrease child farm labor in the context of the Yemen civil war. Lastly, Tusiime et al. (2013) find that food aid during conflicts in Northern Uganda decreased the likelihood of selling poultry and other livestock. This study also finds that the effect is driven by male-headed households, which requires further investigation.

It is important to note that some types of humanitarian assistance may increase child labor as a coping strategy. Aurino et al. (2019) assessed the impacts of an emergency school feeding program and a general food distribution program among children in northern Mali during political and economic turmoil and violent conflict. They find that general food distribution _increased_ the likelihood of children, especially boys, participating in farm labor or housework during the conflict in Mali, while school feeding led to lower participation and time spent in work among girls.

# **Financial Outcomes**

## **Assets**

Asset ownership, an indicator of household economic status, is one of the most reported outcomes in the humanitarian response literature. Ten studies examined the impact of humanitarian assistance programs on asset outcomes (seven with a pure control group and three without a pure control group). These studies report several asset measures ranging from livestock (e.g., own cattle, poultry), non-farm productive assets (e.g., household assets), farm productive assets (e.g., tractors, knapsack sprayer, hand mill), and land ownership.

> 15 A more recent study by <mark>Özler</mark> et al. (2021) evaluates the impact of the Emergency Social Safety Net (ESSN) program in Turkey, which is the largest unconditional cash transfer program for international refugees in the world. They find that this humanitarian assistance program caused substantial increases in the food consumption score and reduced the use of negative coping strategies for Syrian refugees, especially in the short-run.

17
Although one study in our review finds a significant reduction in asset ownership (Gilligan and Hoddinott 2017), most of the studies find that humanitarian assistance programs increase asset ownership during emergencies (Lehmann and Masterson 2014; Bonilla et al. 2017; Lombardini and Mager 2019; Schwab 2019; Ivaschenko et al. 2020; Quattrochi et al. 2020;). For instance, Lehmann and Masterson (2014) find that Syrian refugees in Lebanon who received labeled cash transfers intended for “winterization” were more likely to own ovens and heaters than households in the control group. Similarly, Schwab (2019) finds that recipients of cash transfers during the Yemeni civil war increased their livestock assets by 15 percent relative to non-recipients, the equivalent of a sheep or goat. Unconditional cash transfers are effective at increasing assets not only in conflict settings, but also following natural disasters. For example, households in Fiji who received cash assistance following the 2016 Tropical Cyclone Winston were 13–26 percentage points more likely to recover from the cyclone’s damage, and the effectiveness of those transfers increased in the presence of a functioning local market (Ivaschenko et al. 2020).

There is also evidence that cash voucher interventions for non-food items and cash-forwork can increase asset ownership and household wealth. For example, Quattrochi et al. (2020) find asset vouchers in eastern DRC increased recipients’ assets by 0.16 standard deviations compared to the control group. Similarly, Lombardini and Mager (2020) find that households involved in cash-for-work activities increased their wealth significantly more than the control group.

Conversely, Gilligan and Hoddinott (2007) find that providing emergency food aid in the form of food-for-work to individuals who suffered from the 2002 severe drought in rural Ethiopia led to a negative and statistically significant effect on the growth of livestock holdings. One potential explanation for this decrease in livestock is that program participants may have increased their food consumption to meet higher food energy requirements because of participating in public works. If these food requirements were large enough, program participants might have needed to draw on their livestock assets to meet their food needs.

# **Credit and savings**

Compared to asset ownership, a smaller number of studies (six) report impacts on credit and savings (four with a pure control group and two without a pure control group). In these 18
studies, outcomes are expressed mainly in terms of whether the household had any savings, loans/debts, or the total monetary value of the savings.

Overall, this evidence is too inconclusive to be able to draw any lessons. On the one hand, cash transfer programs in conflict settings have not led to statistically significant effects on savings or loans (Lehmann and Masterson, 2014; Schwab 2019). For instance, levels of debt following the winterization cash transfer program for Syrian refugees in Lebanon ($575 transfer value) were statistically similar in treatment and control households at $500 and $513 in outstanding loans, respectively. In contrast, Quattrochi et al. (2020) find increased debt among Syrian refugees who received vouchers for non-food items. However, the authors note that the effect may indicate either greater access to credit or increased borrowing to meet daily needs. Similarly, Bonilla et al. (2017) find that unconditional cash transfers of $120 to vulnerable households in eastern DRC increased the percentage of households with any savings, increased average savings, and decreased the percentage of households with any debt yet _increased_ the average debt.

# **Income**

Measuring household income, especially in a humanitarian setting, can be very challenging since self-reported measures of total income generally suffer from recall bias, increasing the margin of measurement error. In addition, this outcome is regarded as unreliable given the wide variety of income-generating activities that vulnerable populations engage in during a humanitarian crisis. In this review, only three studies report effects on income (two with a pure control group and one without a pure control group).

Keeping in mind measurement limitations, cash-for-work (CFW) interventions can improve income in humanitarian settings. For instance, Lombardini and Mager (2020) analyzed the effects of a cash-for-work (CFW) program that provided temporary employment in public projects to Syrian refugees in a Jordanian refugee camp. They find that households with at least one member engaged in CFW in the previous 12 months reported income on average 23 percent higher than comparable households in the camp, and they were 19 percentage points less likely to engage in income-generating opportunities other than CFW. On the other hand, Quattrochi et al. (2020) assessed the impact of providing non-food vouchers ranging from $55–90 per household 19
to displaced and conflict-affected populations in eastern DRC and find that this type of intervention did not have a significant effect on household income.

# **Human Development Outcomes**

## **Education**

Across many of the world’s poorest countries, armed conflict continues to destroy not just school infrastructure, but also the hopes and ambitions of a whole generation of children. Therefore, it is critical to strengthen human capital development not only during sudden onset crises but also during slow-onset crises, in addition to ensuring basic needs like food are being met. While direct interventions targeted to improve human capital are desired, simple cash and food transfers in humanitarian settings could also affect outcomes like education and economic opportunities. We found six studies that report impacts of humanitarian assistance programs on children’s education outcomes, covering mainly conflict-affected populations in DRC, Lebanon, and Mali (all with pure control groups). No comparative impact evaluation contrasts different modalities in attaining educational goals.

Four of the six studies find that UCTs have a positive and statistically significant effect on children’s education. First, Chaaban et al. (2020) find that unconditional cash transfers over 4–22 months had a very large effect (10–30 percentage points) on the enrollment rate of Syrian refugees in Lebanon. Second, Lehmann and Masterson (2014) find that children who received the winterization cash transfer intervention in a refugee camp setting in Lebanon were six percentage points more likely to be enrolled relative to the control group. The authors argue that this positive effect was driven by an improvement in parents’ abilities to cover transport costs and school expenditures. Third, de Hoop et al. (2019) find that displaced Syrian children who received the _No Lost Generation_ cash transfer program ($20–65 monthly transfer value) were not more likely to be enrolled in school, citing the supply-side capacity constraint of schools as the main reason behind this result.<sup>16</sup> However, the study finds positive and statistically significant effects on school attendance among children who enrolled in school and benefited from the cash transfer program. They also find that the program significantly increased children’s desire to complete

> 16 The Lebanese public school system incorporated vast numbers of Syrian children in a short time, and, as a result, many schools reached full capacity. The enrollment rate for children aged 5–9 was already high at 91 percent.

20
primary and secondary schooling. Moreover, total education expenditure increased by approximately 54–70 percent. Fourth, Bonilla et al. (2017) find that internally displaced households in eastern DRC that received a $120 unconditional cash transfer were more likely to enroll their children in school, especially boys. These findings highlight the potential of UCTs to make a difference in access to education for displaced children from conflict settings, and the need to coordinate demand-side and supply-side interventions in settings of conflict displacement.

The remaining two studies that look at education outcomes did not find positive impacts. For instance, Quattrochi et al. (2020) find that providing cash vouchers for non-food items to displaced and conflict-affected populations in eastern DRC did not substantially improve the school attendance of children aged 5–18. Lastly, Aurino et al. (2019) caution that well-intended programs might have unexpected results by distorting the incentives of households. The authors examine the impact of two types of food transfer (FT) on children’s education outcomes during conflict in Mali: emergency school feeding and general food distribution (GFD). They find that, while on-site school meals increased school enrollment by ten percentage points and years of schooling by half a year, general food distribution led to a 20 percent _decline_ in school attendance over five years, especially among boys. A finding that boys spent more time on farm activities and other household work suggests there may be an important trade-off between children’s education and labor constraints faced by households.

Overall, the evidence from this review suggests that humanitarian assistance programs can effectively support children’s educational outcomes if the transfer is sufficiently large to offset — — the opportunity costs of schooling which are particularly high during crises and if there are no major supply-side constraints. Moreover, successful approaches to humanitarian assistance in terms of education outcomes need to be tailored to the specific context in which the crisis takes place. For instance, opportunity costs of schooling may be higher for adolescent boys if they are involved in farm work or if schools are perceived as targets of armed conflict, while insecurity and violence may affect girls’ access to school if households perceive that girls are more likely to be targets of violence.

# **Economic opportunities and labor**

21
In theory, the effects of social assistance on labor are ambiguous. A standard economic model predicts that an increase in unexpected, unearned income can induce an individual to work less to enjoy more leisure. On the other hand, additional assistance could incentivize individuals to work more by making them more productive and/or alleviating credit and insurance constraints. For example, by expanding their business or making riskier investments. The additional assistance could also help with job search activities (Baird et al. 2018). In our review, four studies report on the effects of humanitarian assistance programs on labor outcomes covering mainly conflict-affected populations in Ecuador, Lebanon, and Yemen.

The evidence on labor force participation is inconclusive and mixed. For instance, Lehmann and Masterson (2014) find that unconditional cash transfers reduced the number of days worked by recipient households in a refugee environment, suggesting that humanitarian assistance may decrease the incentive to work. Consistent with this finding, Chaaban et al. (2020) find that long-term unconditional cash transfers reduced employment from 53 to 36 percent among Syrian refugees, while they find an increase in the percentage of unemployed men seeking work with better employment conditions. Conversely, the long-term unconditional cash transfer gave women the option to leave the labor force and avoid low-paying jobs they would have otherwise had to take part in. Overall, the findings suggest that refugees might face undesirable or hazardous working conditions that they can afford to avoid by being more selective in job searches or dropping out of the labor market only when cash assistance is available. In contrast, Schwab (2019) finds that among households in the vicinity of the Yemen civil war, those who received food and cash transfers were more likely to do off-farm paid work.

Finally, Hidrobo et al. (2016) highlight the importance of gender dynamics when looking at the effects of humanitarian assistance transfers on labor. They evaluated equivalently valued monthly transfers of $40 (in the form of food, vouchers, or cash) targeted at women and investigated how labor participation changes for both women and their partners among Colombian refugees and poor Ecuadorian households. The authors find no impacts on participation and working hours on agriculture and non-agricultural work for both the women and their partners. The null effect of female-targeted transfers, compared to other studies, suggests that female and male beneficiaries may have different preferences over consumption and labor, and face different labor market conditions.

22
# **Health and subjective well-being**

Emergency environments tend to be chaotic and are often characterized by the collapse of public service delivery, including water, sanitation, and health care. In such environments, health or human welfare may deteriorate rapidly. Humanitarian assistance programs can be a powerful tool for mitigating the negative consequences of this type of crisis. In our review, five studies examined the impacts on health outcomes ranging from children’s anthropometric measures (e.g., height-for-age, weight-for-height, and mid-upper arm circumference), instances of diseases (e.g., malaria and diarrhea), and mental health, to behavior changes and nutrition knowledge, as well as access to water and health care.

In our review, three out of five studies find significant positive effects on most health outcomes, while the remaining two studies do not find any significant effects. For instance, Ecker et al. (2019) find that providing unconditional cash transfers (UCTs) to conflict-affected households during the civil war in Yemen significantly improved children’s anthropometric outcomes (measured as weight-for-height z-score and mid-upper arm circumference z-score). Similarly, Chaaban et al. (2020) find that an unconditional cash transfer program significantly improved a broad range of health outcomes among Syrian refugees in Lebanon.<sup>17</sup> Lastly, Kurdi et al. (2019) find that a conditional cash transfer (CCT) program during the civil conflict in Yemen significantly decreased child malnutrition.

On the other hand, evidence on the effectiveness of vouchers and food transfers is limited. Quattrochi et al. (2020) find a statistically insignificant, positive effect of non-food item vouchers on weight-for-height z-scores, and they did not find any impact on mid-upper arm circumference. Similarly, Tranchant et al. (2019) find an statistically insignificant, negative effect of food aid on the height of children aged 2–5. These findings suggest either that vouchers and food transfer

> 17 The authors find a significant increase in the access to safe drinking water by 15–30 percentage points on a base of 67 percent for the control group. Moreover, cash beneficiaries reported that their access to primary healthcare improved and their need for hospitalization decreased. Cash support presumably improved their ability to cover healthcare costs and other indirect costs such as transportation costs. Importantly, the authors find that 55 percent of those who received cash over a longer period (more than 12 months) reported better mental health (measured using the Mental Health Inventory–MHI-5), relative to 18.5 percent in the control group. However, similar effects were not detected for those who received cash over a shorter period (less than 12 months).

23
interventions might not be very effective at improving children’s anthropometric outcomes, or that detecting changes in height and weight requires a larger sample to improve statistical power.

Lastly, we found two studies that reported on subjective well-being. For example, Quatrochi et al. (2020) find large, positive effects of non-food item (NFI) vouchers on the mental health of displaced and conflict-affected adults in eastern DRC. This positive effect appears to be driven by higher levels of well-being (measured using the WHO scale) and life satisfaction (measured using the question, “All things considered, how satisfied are you with your life as a whole these days on a scale of 1 to 10?”). Similarly, Lombardini and Mager (2020) report the impact of a cash-for-work (CFW) program on the subjective well-being of Syrian refugees in Jordan. The four questions used in this study measure whether respondents felt they positively contribute to their family and whether they felt satisfied with their lives. They find three positive coefficients among the four questions, but only one of them is statistically significant. Given the lack of evidence on this type of outcome, this is an outcome area where more evidence would be useful.

# **Gender Outcomes**

Gender outcomes are less explored in humanitarian assistance. Even outside of humanitarian settings, the evidence on policies and programs that effectively reduce genderbased violence and empower women in the developing world is scarce, and there is still no consensus on theories and mechanisms. It is often assumed that improving the economic situation of women and ensuring they have an equal share of resources within their households will alleviate gender-based violence and potentially also give them more household decision-making power. However, research has shown this is not always the case. In particular, concerns have been raised that cash transfers could disadvantage women by reducing their control over assistance in the household. In this review, we found five studies that report on the impact of humanitarian assistance programs on gender outcomes. In particular, four studies report on women’s empowerment (three with a pure control and one without a pure control), while only one study reports on gender-based violence.<sup>18</sup>

> 18 In these studies, women’s empowerment is measured through indicators such as women’s decision-making power and attitudes toward gender roles.

24
Similar to the link between cash-based interventions and women’s empowerment in the broader literature, the evidence is also ambiguous when focused on humanitarian settings. For example, on the one hand, Kurdi et al. (2019) show that conditional cash transfers as part of the Yemen Emergency Crisis Response Project improved women’s empowerment. In particular, treated women were more likely to report taking their child alone to a health center and having higher aspirations for their daughters’ education. On the other hand, Lombardini and Mager (2020) find that Syrian refugee households involved in CFW activities in Jordan had a higher proportion of women engaged in income-generating activities. However, they find no evidence that cash-for-work (CFW) improved gender equality outcomes.<sup>19</sup> Relatedly, Hidrobo et al. (2016) show that refugee and poor women who received cash, vouchers, and food transfers as part of a program designed to reduce poverty and food security in northern Ecuador were more likely to spend more time engaged in domestic labor, which may be interpreted as there were fewer opportunities for women to spend time outside of the home.

Hidrobo et al. (2016) is the only study that exclusively focuses on the impacts of humanitarian assistance on gender-based violence. The authors show that equally valued transfers of cash, voucher, and food targeted to Colombian refugee women significantly decreased controlling behaviors and physical or sexual violence compared to the control group. When examined by individual modality, they find that food transfers reduced physical or sexual violence, cash reduced controlling behaviors, and vouchers reduced controlling behaviors and physical or sexual violence. However, these differences across modalities are not significantly different from each other.

# **Social Cohesion Outcomes**

There is still no consensus in the literature on the definition of either social capital or social cohesion or how to measure them. In this review, we focus on proxies for the characteristics of social relations in a certain community, including cooperation and solidarity between groups and individuals, trust in individuals, lack of discrimination, confidence in institutions, and agency.<sup>20</sup>

> 19 Lombardini and Mager (2020) show that individuals involved in CFW activities were less likely to report that men should support with care work in the home compared to individuals in the control group. However, this is not surprising because the intervention was not targeted to women, and 80 percent of the respondents were men. 20 The concept of social cohesion carries different connotations depending on the context, identity, culture, and social and political dynamics. Even though these definitions often respond to the policy needs and the particular interest of

25
We found four studies that examined the impacts of humanitarian assistance programs on social cohesion (three with a pure control and one that compared different modalities), covering mainly conflict-affected populations in DRC, Ecuador, and Lebanon.

Despite the limited evidence on social cohesion, the few studies that report this outcome suggest that humanitarian assistance programs can effectively promote social capital during a humanitarian crisis. For instance, Lehmann and Masterson (2014) find that cash transfer recipients in a refugee camp in Lebanon had significantly fewer disputes among household members. They also find that the relationship between Syrian refugees and Lebanese community members improved, as measured by social interactions in providing and receiving help. Relatedly, Quattrochi et al. (2020) measured the effect of non-food vouchers on social cohesion in DRC based on contributions to other households, contributions to the village, conflicts with other households, trust, and incidences of theft. They find a positive effect on the contributions to the village, but there were no effects on other measures of social cohesion. Similarly, Valli et al. (2019) show that equally valued transfers of cash, vouchers, and food targeted to Colombian refugees in Ecuador improved outcomes on personal agency, attitudes about diversity, confidence in institutions, and social participation. However, the program did not have any effects on social cohesion among Ecuadorian participants.<sup>21</sup> Even though the included studies in our review find positive effects on social cohesion, it — — is important to note that humanitarian assistance programs in-kind or cash might also have negative effects by increasing social tensions when certain groups, notably refugees, are provided with a cash transfer that is unavailable to host communities. To alleviate such tensions, interventions in humanitarian settings that aim to improve social cohesion outcomes could either include host communities as beneficiaries or sensitize them to the benefits of cash programs targeting refugees.

> agencies and institutions, they are generally linked to the wider scope of democratic governance, including human rights and social accountability. For instance, the Council of Europe, one of the major contributors to the conceptualization and measurement of social cohesion, defines social cohesion as “a society’s capacity to ensure the well-being of all its members by minimizing disparities and avoiding marginalization; to manage differences and divisions and to ensure the means of achieving welfare for all” (Council of Europe 2004: 3).

> 21 The authors suggest that participating in the nutrition and health training sessions, as well as the program messaging around inclusiveness, might have contributed to the observed rise in social cohesion.

26
# **Cost Analysis**

As humanitarian budgets are limited, a key question is whether one delivery method has a higher value for money than others. Methods that are typically used to assess value for money are i) cost analysis, ii) cost efficiency, iii) cost-effectiveness, iv) cost-benefit, and v) market impact analysis (Doocy and Tappis 2018).<sup>22</sup> Experimental and quasi-experimental studies have the advantage of producing causal impact estimates that can be used for cost-effectiveness, cost-benefit, and market impact analysis. However, it is often challenging to summarize the impacts of a program into an aggregated metric. For example, interventions in humanitarian contexts affect a wide range of short-term and long-term outcomes, including food security, income, expenditure, health, and education. Therefore, most studies that perform cost-effectiveness analysis choose to report only on a smaller set of outcomes. Similarly, not all outcomes can be quantified in monetary terms. For example, it is difficult to assign monetary values to improvements in women empowerment, intimate partner violence, and subjective well-being measures. For this reason, while cost-benefit analysis can be a powerful tool, it may not provide the full benefit of a program, and it may inaccurately characterize the comparative effectiveness if one modality disproportionally benefits nonquantifiable outcome measures.

Aside from effectiveness, quantifying total costs that go into delivering programs is equally challenging, especially when multiple organizations provide resources with different or no accounting systems. Identifying and valuing non-monetary items such as volunteers' time and people involved in the programs is not straightforward either. Moreover, other considerations beyond programs' effectiveness and cost must be considered. For example, design features (e.g., targeting, the size and frequencies of transfers, and duration); scalability; potential negative/positive general equilibrium effects; and risks around security, corruption, and delay may affect the effectiveness and costs of a given program differently, making value-for-money exercise more challenging (Gentilini 2016).

In our review, four out of twenty-one studies report the implementation costs of different transfer modalities with equivalent monetary values. First, Aker et al. (2011) find that the total

> 22 See Doocy and Tappis (2018) for the definitions of these different types of analysis.

27
program cost of manual cash transfers is 7 percent lower than that of mobile money transfers. One important consideration is that the comparison depends on the frequency of transfers. Since mobile money transfers require higher fixed costs upfront to open and set up the mobile money accounts, the average cost of each transfer will be lower with a higher number of transfers. Furthermore, the authors find that recipients in the mobile money transfer group benefited from reduced opportunity costs of their time which is not adequately considered when comparing the implementation cost. Overall, they conclude that the additional benefits from the mobile money transfer outweigh the small cost increase.

The other three studies compare the costs of other types of interventions (but not mobile money), and argue that the cash transfer method is the most economic intervention to implement. Sandström and Tchatchua (2010) find that cash transfer is at least 5 percent cheaper than delivering the equivalently valued basket of food transfer. Similarly, Aker (2017) finds the perrecipient implementation cost is 8 percent less for cash transfer than voucher transfer. Hidrobo et al. (2014) compare all three modalities, and find that cash transfer is 9 percent cheaper than voucher transfer and 75 percent cheaper than food transfer per recipient per transfer.

Two of the four studies also present a brief analysis of which modality might produce the most value for money. Aker et al. (2011) report a simple cost-benefit calculation based on the opportunity cost of receiving manual vs. mobile cash and the increased cultivation of a cash crop. The authors conclude that the mobile money transfer has a cost-benefit ratio greater than one relative to the manual cash transfer. Hidrobo et al. (2014) perform a cost-effectiveness analysis using five food security indicators (food consumption, caloric intake, Household Dietary Diversity Score, Dietary Diversity Index, and Food Consumption Score). They find that manual food distribution is the least cost-effective, while voucher transfer is slightly more cost-effective than cash transfers. One reason why cash transfer is not the most effective in this context may be the cost of manufacturing debit cards, as noted by the authors.

Without accurate cost data about alternative program designs and delivery modalities, we may be recommending policies and programs that are not cost-effective or that are not worth scaling up. Only by knowing the effectiveness and costs of an intervention can policy makers make informed decisions about allocating scarce resources. Unfortunately, most studies in this review do not collect detailed cost information, with the exception of Hidrobo et al. 2014, who 28
collected disaggregated cost data via WFP’s accounting ledgers and interviews, including staff costs. To improve the reporting on cost-effectiveness, the World Bank Strategic Impact Evaluation Fund’s 2019 guidelines recommend relying on administrative data (e.g., budgets and spending reports) as much as possible to use costs information that is “disaggregated, intervention-specific, and captured in real-time over the course of an intervention.” The document also discusses the need to collect non-financial information that is not typically quantified in accounting ledgers. Beneficiaries’ travel time to obtain transfers (as in Aker et al. 2011) is an example of non-financial data that can be systematically different across transfer modalities.<sup>23</sup>

# **4. Comparative Performance of Transfer Modalities**

Decisions about the types of transfers (e.g., cash, voucher, vs. food) and the delivery mechanisms (e.g., mobile money vs. cash-in-envelope, or lump sum vs. multiple transfers) involve multiple factors. First, the condition of market functioning needs to be assessed. As is the case in many humanitarian settings, missing markets can undermine the effectiveness of cash transfers if there are limited opportunities to use the cash transfer. Second, the existing physical and financial infrastructure needs to be taken into account because different modalities require delivery systems. The system may vary from staff traveling to communities distributing manual cash or goods, to payment service providers such as post offices disbursing cash over the counter, to electronic mechanisms such as transfers by mobile phone or directly into bank accounts. In particular, conflict and rapid-onset natural disasters can disrupt payment systems, damage infrastructure, and displace people and businesses. Delivering cash digitally may not be the most effective delivery method if the cost of cashing it out is high for villagers in remote areas with poor road conditions. Third, the selection of food items matters. If the in-kind food items are _inframarginal_ (the consumption quantity even without transfer is greater than the quantity provided), in-kind will have little to no distortion effects. On the other hand, if there is a household surplus of some food items, then in-kind transfers can be distortionary in the sense that it might induce

> 23 A good example of collecting costing data is the World Bank’s <u>Partnership for Economic Inclusion’s Costing Dashboard, which reports and compares costs of economic inclusion policies. This type of tool could be used as a</u> framework to collect costing data for cash and in-kind programs in humanitarian settings. Hendren and Sprung-Keyser (2020) also developed a unifying framework to compare the effectiveness of different policies and interventions, which relies mainly on understanding the long-term impact of policies as well as knowing the implementation cost.

29
households to consume more food items than they desire, or those food items may go to waste if they are not easily storable.

Recognizing that the effectiveness and efficiency of cash largely depend on contextspecific factors like the availability of financial service providers, functioning markets, and security, humanitarian actors have committed to increasing the use of cash (instead of, or in addition to, in-kind) whenever feasible (UNOCHA 2017). Reasons for cash preference include ethical motivations, as it is generally believed to respect beneficiary preference, empowering them to address their own needs by affording them more choice in local markets. On the other hand, governments may prefer in-kind transfers under the assumption that households may be short-sighted and spend money for immediate gratification (Hanna and Karlan 2017), or they may not take into account social benefits when making individual decisions. Nonetheless, donors and practitioners continue to call for additional evidence from the humanitarian space on how the benefits and impacts of cash-based transfers compare to in-kind, with an eye to outcomes regarding risk and cost-effectiveness (HLPHF 2016).

In this section, we examine impact evaluation studies that compared the relative effectiveness of alternative transfer modalities according to different dimensions of outcomes. A summary of the impacts is presented in **Table 6** , which displays the most effective transfer modality according to different outcome dimensions.

# **Basic Needs Outcomes**

We found six studies that report the most effective modality in terms of basic needs outcomes, such as food security, food expenditure, non-food expenditure, and coping strategies.

**Food security** . Even though food security is one of the most explored outcomes in humanitarian settings, the debate over the most effective modality for improving food security outcomes remains inconclusive. When comparing different modalities of transfers, the data show that, on average, cash and in-kind transfers are similarly effective in improving overall food security. Certain differences between cash and in-kind transfers are not very significant and depend on the indicator used to measure them. For instance, one of the most used indicators is food consumption. Hidrobo et al. (2014) used a randomized controlled trial to assess the impacts of cash (CCT), food vouchers, and food transfers in a refugee camp in Ecuador, and they find that

30
all three modalities significantly improved the quantity and quality of food consumed. However, while the impacts on food consumption were larger for food-receiving beneficiaries relative to both cash and voucher transfers, they were not statistically significant.<sup>24</sup> On the other hand, Aker (2017) did not find any significant differences in food consumption between food vouchers and cash, partly because voucher households could resell part of what they had purchased. However, Aker (2017) also noted that food transfers were distorting because those receiving cash spent more money on health and education items.

Another indicator that provides information on the impact on food security at the household level is caloric intake. In contrast to food consumption, food transfers tend to have a larger impact on caloric intake relative to cash in most contexts. For example, in Ecuador, Hidrobo et al. (2014) find that food transfers led to a significantly larger increase in consumed calories (relative to cash transfers) mainly due to larger increases in consumption of cereals (41 percent of households' caloric intake). A potential mechanism behind this effect is a change in diet, where cash beneficiaries shift from highly caloric foods to a diverse diet, including eggs, milk and dairy, vegetables, and meat.

Lastly, to analyze the quality of consumption patterns and diets, another indicator used in the literature is dietary diversity (e.g., dietary diversity index, food consumption scores, and Household Dietary Diversity Score). In the humanitarian literature, results are mixed. On the one hand, Hidrobo et al. (2014) find that vouchers led to significantly larger increases in dietary diversity (relative to both cash and food transfers). This effect was mainly due to larger increases in the number of days consuming vegetables, eggs, milk, and dairy. Similarly, Sandstrom and Tchatchua (2010) find that those receiving cash increased dietary diversity by consuming more meat, dairy products, and processed foods (relative to the food transfer group), but the differences are quite small. On the other hand, Aker (2017) finds that dietary diversity was similar across food voucher-receiving and cash-receiving households.<sup>25</sup>

> 24 Evidence from the development literature on the relative effectiveness of cash versus food indicates that impacts on food consumption are higher for cash than for food beneficiaries (Ahmed, Quisumbing, Nasreen, Hoddinott, and Bryan 2010; Barker, Filmer, and Rigolini 2014; Cunha 2014). In the case of Ahmed et al. (2010), who compared the relative effectiveness of cash and food transfers to the ultra-poor in Bangladesh, one potential explanation is that the size of the cash transfer was significantly higher than the food transfer.

> 25The findings in non-humanitarian settings are also consistent. For example, Skoufias et al. (2008) evaluated the randomized cash and in-kind transfers of Mexican government’s food assistance program, the _Programa de Apoyo_

31
This literature also examines different modalities of delivering cash (mobile vs. manual). For instance, Aker et al. (2011) compared the effect of mobile money cash transfers and manual cash transfers. They found that those who received mobile money transfers increased dietary diversity (as measured through HDDS) more than those who received cash manually. The authors suggested that this may be due to lower transaction costs and greater privacy.

**Food expenditure** . Two studies examined the impacts of different modalities of transfers on food expenditure. Sandstrom and Tchatchua (2010) randomly assigned beneficiaries into cash transfer and food transfer groups during WFP operations in Sri Lanka. They find that total food expenditure was similar for both groups, but cash households spent more on meat, dairy products, and processed foods and less on rice and wheat, relative to food transfer households. In contrast, in DRC, Aker (2017) finds that total weekly food expenditures were 13 percent lower for households receiving cash than those receiving food vouchers. This was largely because vouchers were commodity-based. Once again, the data shows mixed results for cash and in-kind transfers, suggesting that their effectiveness is similar on average.

**Coping strategies.** There is limited evidence on how coping strategies are differentially affected by different transfer modalities. For this outcome group, the data also show that in-kind and cash transfers have similar effectiveness. For instance, Schwab (2019) finds that cash and food transfers did not significantly affect child labor during civil unrest in Yemen. Similarly, Aker et al. (2011) did not find any differences between mobile money transfers and manual cash transfers on coping strategies measured by selling land and cutting trees. These studies do not include conventional coping strategies, including reducing the portions of meals, working as a casual laborer, selling livestock, or borrowing. Therefore, more evidence on this outcome would be particularly useful.

**Non-food expenditure:** Similar to the previous basic needs outcomes, six studies in our review compare the effectiveness of different modalities on non-food expenditure. In this case,

> _Alimentario_ (PAL), and found similar effects on food and total consumption. Using the same program, Cunha (2014) reaches the same conclusion and explains that it is because the in-kind food transfers of this program were inframarginal in terms of _total_ food consumption. However, he also notes there was a large variation in over- or underconsumption across the ten items provided (e.g., bean transfer amounts were smaller than consumption needs, while milk powder amounts were significantly greater than consumption needs). One exception to this overall literature is Hoddinott et al. (2018), who find that households randomized to receive in-kind transfers in Niger had higher increases in food consumption scores (FCS) and dietary diversity relative to the cash group.

32
there is some suggestive evidence that unconditional cash transfers increase non-food expenditure more than other types of interventions, but overall the evidence is still inconclusive (Sandström and Tchatchua 2010; Aker 2017; Schwab 2019).

The initial findings from the literature on basic needs outcomes are threefold. First, for food security, the effects are similar across modalities. Therefore, it might be useful to consider other factors when determining which type of transfer to use, such as implementation cost, presence of functioning local markets, and, importantly, beneficiary preference. Second, the specific types of food consumed differ depending on the food basket provided, restrictions placed on food vouchers, and the availability and prices of food locally. It is possible to make specific food items more accessible and affordable to increase the intake of specific micronutrients. Third, cash tends to increase non-food expenditures, such as clothing or agricultural inputs. Assuming households know how to best spend their money, this suggests that in-kind transfers could be distortionary.

# **Financial Outcomes**

Evidence from a direct comparison of cash versus in-kind transfers within the same intervention in humanitarian settings is more limited for this outcome category. In our sample, three studies assess the relative effectiveness of different modalities on financial outcomes. Two evaluated different unconditional cash transfer modalities in Niger and the Philippines (Aker et al. 2011; Mercy Corps 2015), while one compared cash transfers and vouchers in a refugee camp setting in the DRC (Aker 2017).

**Asset ownership.** Following Typhoon Haiyan in the Philippines in 2013, households who received unconditional cash transfers in a lump sum (single payment) had more productive assets compared with households that received three payments (multiple payments) of the same amount (Mercy Corps 2015). In Niger, Aker et al. (2011) find that households who received cash transfers through mobile phones had more non-durable assets than those who received manual cash, though there was no difference in durable asset ownership. The study suggests that households receiving mobile transfers are less likely to sell non-durable assets (e.g., lamps and flashlights) less frequently than those receiving manual cash transfers. Lastly, Aker (2017) finds no difference in asset ownership between households receiving cash vs. vouchers in a refugee camp setting in DRC.

33
**Savings.** Regarding the relative effectiveness of different modalities on savings, Mercy Corps (2015) evaluated several unconditional cash transfer modalities in a natural disaster setting. Following Typhoon Haiyan in the Philippines, Mercy Corps’ cash transfer program compared four UCT treatment arms: (i) a lump sum cash transfer, (ii) a multiple payment cash transfer, (iii) a multiple payment cash transfer plus a financial literacy training, and (iv) a multiple payment cash transfer plus a financial literacy training and savings encouragements through messages. First, the authors find no evidence that the lump sum and the multiple payment disbursements were different when it came to encouraging savings behavior. Second, for households that received multiple cash payments, the addition of the financial literacy training component did not have any effect on savings behavior. The authors note that one-off trainings often included in relief efforts are likely insufficient to affect financial behaviors, but that as communities move forward with recovery, ensuring that households have access to the benefits of financial products will play a role in accelerating improvement and reducing vulnerability. Third, for households that received multiple cash payments and the financial literacy training, also receiving nudges through voice messages that encouraged saving led to a statistically significant increase in the use of both informal and formal savings products among beneficiaries who reported receiving the messages. This suggests that nudges can be a powerful tool for improving savings behavior in a humanitarian setting.

On the other hand, Aker (2017) compared the effectiveness of unconditional cash transfers and vouchers in an informal camp for internally displaced persons in the DRC. The author finds that households receiving cash transfers saved slightly more during the intervention than households receiving vouchers. Similarly, 9 percent of households receiving cash had savings at endline compared to only 1 percent of households that received vouchers. Even though these results suggest that cash may be more conducive to monetary savings, while vouchers are more conducive to asset purchases, the study did not provide evidence that one modality has a greater effect on asset ownership than the other.

**Income.** Regarding the relative effectiveness of different modalities, household income is only examine by Aker (2017), who compared the effectiveness of unconditional cash transfers and vouchers in increasing access to food and essential non-food items in an informal camp for internally displaced persons in DRC. The author does not find a statistically significant difference

34
in endline household incomes between unconditional cash transfer and voucher groups, suggesting that one modality does not have a greater effect than the other.

# **Human Development Outcomes**

In terms of education, health, and labor outcomes, there is not a single evaluation of the relative effectiveness of different modalities in improving human capital outcomes in humanitarian settings, highlighting the urgent need for more research in this field.

However, a handful of studies present data on the relative effectiveness of cash and in-kind transfers in non-humanitarian settings on short and longer-term nutrition-related outcomes, from which we can extrapolate some lessons to inform humanitarian debates. For instance, Cunha (2014) measured the impact of a food assistance program in Mexico and shows that both food and cash transfers increased the intake of micronutrients (iron) among children. However, the difference is not statistically significant. Anemia prevalence was also reduced significantly in both food- and cash-receiving households.

In Uganda, Gilligan and Roy (2013) examined the impact of two transfer modalities linked to preschool enrollment: cash vs. food. They find that cash transfers decreased anemia prevalence by approximately ten percentage points among children, while food transfers had no significant impacts. The authors interpret the limited impact of food as potentially driven by factors such as (i) households sharing food rations across all household members, reducing rations targeted only for children; and (ii) households not valuing the food ration as much because it was difficult to sell in the market in exchange for cash. In another example, Baker, Filmer, and Rigolini (2014) evaluated the impact of a food-cash scholarship program in Cambodia and found that neither treatment modality had significant impacts on anthropometric indicators, possibly because of the small transfer size and short treatment exposure.

Lastly, Langendorf et al. (2014) compared several types of cash and food combinations with the aim to reduce severe malnutrition and mortality rates among children. Their findings suggest that combining cash and food transfers may reduce the incidence of malnutrition at about twice the rate compared to either cash transfers or supplementary food alone.

While the cash versus in-kind transfer debate is largely about demand-side issues, there is a much larger agenda around the supply-side services, such as health and education, especially 35
in humanitarian settings. In particular, it is important to note that transfers cannot replace services and when it comes to the effectiveness and efficiency of transfer-based interventions, whether cash or in-kind. In terms of human development outcomes, the availability and quality of health and education services is indispensable. In other words, there is little rationale to implement cash or in-kind transfers to improve health and/or education outcomes if the supply of such services is unavailable or of inadequate quality. For instance, de Hoop et al. (2019) find that cash provided to children increased school attendance among Syrian children in Lebanon, but the effects of the humanitarian response were limited due to overcrowding in schools. This evaluation highlights the importance of being mindful of supply changes when increasing the demand for services.

# **Gender Outcomes**

There is minimal literature that investigates the relative effectiveness of cash and in-kind transfers on gender-based violence, women’s empowerment, and social cohesion in humanitarian settings.

**Gender-based violence.** In the context of the Colombian refugee crisis, Hidrobo et al. (2016) studied whether equally valued transfers of cash, vouchers, and food targeted at women have different impacts on gender-based violence (GBV). Overall, the study shows that transfer interventions (CCT, FT, and vouchers) significantly decreased controlling behaviors and physical/sexual violence compared to the control group. When examined by individual modality, they find that food transfers reduced physical or sexual violence, cash reduced controlling behaviors, and vouchers reduced controlling behaviors as well as physical or sexual violence. This finding suggests that partners do not use violence to extract resources because the effects are similar for cash, which can be easily extracted, as well as for food transfers and food vouchers, which cannot be easily extracted. However, differences across modalities are not statistically significant.

**Women’s empowerment.** Aker (2017) is the only study that compared the relative effectiveness of female-targeted unconditional cash transfers and vouchers on intra-household decision-making in a humanitarian context (a refugee camp in eastern DRC). The author finds that most respondents reported making joint decisions on children’s education, inter-household sharing, and savings, which did not differ by transfer modality. Interestingly, however, the cash

36
group is less likely to discuss the use of transfers with other family members (relative to the voucher group).<sup>26</sup>

## **Social Cohesion Outcomes**

**Social cohesion.** In terms of studies that measure the relative effectiveness of different modalities in humanitarian contexts, only one study looked at impacts on social cohesion. Aker (2017) compared the relative effectiveness of unconditional cash transfers and vouchers in a refugee camp context in eastern DRC. This study measured social cohesion as a willingness to share part of the transfer with other households. The author finds that both types of program recipients shared part of their transfer, suggesting that sharing is an important coping mechanism within the refugee camp. While cash transfer households were 15 percentage points more likely to share the money received with other households, voucher households were 15 percentage points more likely to share goods purchased with the transfer.

# **5. Evidence Gaps and Areas for Future Research**

Given the relatively limited evidence on the impact of humanitarian assistance programs, and especially the lack of evidence on the relative effectiveness of modalities, it is very challenging to draw conclusions about the generalizability and the applicability of such evidence to other contexts. Most studies in our review report very little information on operational aspects and mechanisms behind the results. This limits an understanding of what makes a particular intervention successful and hinders the possibility of policy makers replicating programs in similar contexts. In this section, we propose several areas of focus for future research.

**Basic Needs.** First, we need to better understand how different modalities affect the types of food consumed and consumption at the individual level (e.g., children vs. adults or male vs. female). The impacts of cash versus in-kind transfers depend on the indicators used to measure them (e.g., caloric intake, dietary diversity) and on other factors such as the characteristics of the targeted population and the capacity of local markets. Second, the effect of food assistance on

> 26 Aker et al. (2017) also find that the mobile money transfer system, compared to manual cash transfers, resulted in an increase in the diversity of crops grown by the household. The impact was driven by an increase in two marginal cash crops that are primarily grown by women. While not an explicit measure of control over resources, this evidence suggests there may be positive impacts of the mobile money modality on women’s decision-making power beyond the cash transferred. Authors theorize that this is due to the increased privacy that mobile money affords women. However, more evidence is needed on the potentially empowering aspects of digital modalities.

37
food expenditure is unclear because food expenditure can increase or decrease. We suggest studies focus more on food consumption when aid is in-kind. Third, there is modest evidence that cash transfers increase non-food expenditure more than other types of interventions. This implies that, in general, cash transfers better reflect the preferences of beneficiaries, which should be considered when deciding the types of interventions.

Despite most evidence in humanitarian settings relating to basic needs outcomes such as food security, food/non-food expenditure, and coping strategies, the evidence base on the comparative performance of transfer modalities in this cluster is still emerging and inconclusive. While cash allows beneficiaries to spend the money in their best interests, they might not be aware of the nutritional composition of different food groups. On the other hand, food transfers have the benefit of providing the most balanced diet, but it might be difficult to identify which nutrients households are deficient in, and this may vary across households. Therefore, the existing evidence suggests that what makes one modality more effective than another depends on other factors such as the characteristics of the targeted population and the capacity of local markets, among others. Future research could provide a better understanding of the interactions between different transfer modalities, the context, and the capacity of local markets.

Lastly, one important limitation of the current evidence is that basic needs outcomes are mainly measured at the household level. This leaves a gap of knowledge regarding the intrahousehold distribution of food and non-food consumed, and highlights the need for individuallevel or gendered indicators in order to improve our knowledge of consumption in fragile and conflict-affected settings.

**Financial outcomes.** To enhance livelihoods and earning opportunities in humanitarian settings, it is important to identify key constraints faced by vulnerable populations, such as a lack of capital or information and skills. Therefore, for this cluster of outcomes, a successful intervention could provide a blend of cash and in-kind interventions, where “in-kind” could refer to physical capital, assets, materials, training, or nudges. One example of this type of intervention that has not been explored much in humanitarian settings is graduation programs or “cash plus” interventions, which include not only cash transfers, but also the provision of livestock, life-skills coaching, training on income-generating activities, or access to saving facilities and microcredit. However, such programs can be difficult to implement in humanitarian settings because many

38
fundamental services that complement the cash transfer might be destroyed during conflict or natural disasters. Despite this, even in such contexts, there are still opportunities to put together a package of support that would help people move out of vulnerability or better cope with risks and shocks. Cash transfers alone are not always the most effective modality for livelihoods, but a combination of both cash and in-kind modalities could be more effective in humanitarian contexts.<sup>27</sup>

**Human Development.** The evidence is inconclusive for the human development field, which includes education, labor market participation, and health outcomes. In addition, there is no documentation on the relative effectiveness of different modalities in improving human capital outcomes, which highlights the need for more research in this field.

In terms of education outcomes, important areas for future research remain. For instance, similar to cash transfers, variation in the size and duration of food assistance may influence educational effectiveness. Moreover, understanding to what extent complementary interventions, such as adapted materials for at-home learning, tablet-based learning, remote learning, or other ed-tech interventions, as well as supply-side educational interventions, may enhance the impacts of humanitarian assistance interventions. Another direction for future research is to examine the quality of education and learning outcomes.

Conflict-affected populations (such as forcibly displaced individuals) face multiple obstacles in humanitarian settings, such as legal restrictions, loss of assets, physical and mental health issues, skills mismatches, lack of social networks, excessive labor supply, and discrimination. Overall, the findings suggest that labor market participation outcomes depend highly on the characteristics of those forcibly displaced, the labor market conditions, and available job opportunities. Therefore, more evidence in this area will be particularly useful to generate generalized findings.

In terms of health and nutrition outcomes, future studies need to investigate the frequency and types of illness among adults and the treatment conditions of such events. For instance, cash transfers combined with interventions addressing the psychological and mental health effects of

> 27 The analysis from this review suggests that “cash plus” interventions, such as the Mercy Corps’ (2015) cash transfer program that combined cash transfers with financial literacy training and savings encouragements, might have the potential for greater impact on livelihoods and household economic outcomes than cash alone.

39
forced displacement (e.g., therapy interventions) or vouchers labeled to pay for medical costs related to mental health issues could be promising avenues to explore.

To summarize, these findings indicate a clear need for more research on human development in humanitarian settings. In this field, “cash plus” interventions (a blend of cash and in-kind interventions that provide not only cash but also education, health, skills training, early childhood development, or nutrition services) may be more effective than either cash transfers or supplementary food alone.

**Gender Outcomes** . There is an evidence gap regarding the effectiveness and efficiency of social protection programs in terms of gender outcomes, with gender-based violence being the least explored area in the humanitarian literature. Women and girls are at increased risk of various negative outcomes in conflicts and humanitarian crises, including harm to their physical and mental health and increased exposure to violence and exploitation.

The existing evidence is scarce and ambiguous when focused on humanitarian settings. Since the studies that tried to examine women’s empowerment were only based on a few questions, including the perception of gender norms, we suggest that future research include more extensive measures of women’s empowerment by having a standalone module to contribute to the literature. Moreover, since gender-based violence is the least explored outcome in humanitarian settings, it would be interesting to explore whether humanitarian assistance interventions effectively reduce gender-based violence in humanitarian settings by carefully accounting for potential mechanisms in intra-household dynamics. In addition, variations in recipient gender, marital status, and the size and duration of the transfer may impact gender outcomes differently. For example, it would be interesting to explore whether female-targeted transfers can decrease gender-based violence and improve women’s well-being in humanitarian settings.<sup>28</sup> More analysis on the comparative effectiveness and cost-efficiency of different modalities for gender objectives is also needed.

> 28 In a non-humanitarian setting, Haushofer et al. (2019) explore how giving unconditional cash transfers to both women and men in Kenya reduced different types of intimate partner violence. Transfers to women reduced physical and sexual violence, while transfers to men reduced only physical violence. The authors argue that husbands use physical violence to extract resources, but dislike it, while sexual violence is not used to extract resources, but is pleasurable.

40
Other gender outcomes worth exploring in humanitarian settings are child marriage, teenage pregnancy, sexual exploitation, and abuse, especially in conflict-affected settings. Complementary components to cash transfers could also be considered. In addition, further investigation is needed to understand the medium- and long-term impacts of social protection interventions on women’s and girls’ socioeconomic vulnerability in humanitarian settings, as these may change over time. Lastly, considerations must be made concerning program design, such as the targeting and registration process, to reduce risks to women related to the changing household dynamics caused by social protection programs.

**Social Cohesion.** Sharing and social cohesion is an area where investigating different modalities is especially interesting as some transfers, such as mobile money transfers, are much easier to hide than other types of transfers, such as food assistance. Refugees and internally displaced individuals lack social networks in host communities to help them overcome information asymmetries and integrate into the labor market. In this sense, interventions that complement social assistance with ways to build social networks for the forcibly displaced could lead to promising results. Given that there is little consensus on the exact domains and indicators necessary to measure social cohesion comprehensively, a more unified framework and consensus on the relevant components and measurement tools would contribute significantly to this literature. It would also be interesting to collect information on overall community dynamics, including spillover effects on non-recipient households in treatment communities, to assess potential negative effects on those not eligible for benefits.

# **6. Discussion and Conclusion**

This study reviewed the existing evidence on the effectiveness and efficiency of humanitarian assistance programs across five outcome categories. This systematic review complements existing evidence on the effectiveness of these programs in improving individual and household-level outcomes, and informs the debate surrounding the comparative performance of different transfer modalities. Based on the available evidence, our review suggests the following conclusions:

**_Despite the growing use of social protection programs in humanitarian settings, there is relatively little rigorous research on what works, for whom, and why._** From our review, we find that only twenty-one studies have used experimental or quasi-experimental methods to rigorously assess the impact of humanitarian assistance programs.

41
**_There is large variation in the availability of evidence for humanitarian assistance programs across different sectors._** On the one hand, the evidence base for studies that report on basic needs such as food security, food and non-food expenditures, and coping strategies is quite substantial, followed by studies that report on the household economy and financial outcomes where the evidence can be categorized as emerging. On the other hand, the evidence base for studies that report on human development outcomes, such as health, education, labor, as well as gender and social cohesion, is much more limited. In particular, we find that women’s empowerment and gender-based violence outcomes are the least explored in the context of humanitarian assistance programs. This imbalance suggests that policy decisions, especially in terms of human development, should be made with caution due to the limited generalization of the results, and also indicates the need for research to fill key evidence gaps.

**_In terms of the impact of humanitarian assistance programs, the limited evidence points to mixed and inconclusive results._** The literature suggests that most humanitarian assistance programs can effectively improve individual and household-level outcomes compared to the control group. However, the evidence is too limited to be able to draw general conclusions.

**_Regarding the relative effectiveness of different modalities, the decision about the most appropriate modality versus another cannot be generalized and pre-determined_** _._ Six out of twenty-one studies compare the relative performance of alternative transfer modalities. The existing evidence suggests that modality performance and their differences depend on a series of factors such as the nature of the humanitarian crisis (sudden onset vs. slow onset), the objective of the program or the main outcome of interest, the profile of the targeted population, implementation costs, and local market capacity, among others. All these factors must be considered when deciding between transfer modalities. Given the limited evidence on the relative effectiveness of transfer modalities in humanitarian settings, future research could provide a better understanding of the conditions for which cash transfers are more effective than in-kind transfers. In addition, it would be beneficial to explore complementarities between different interventions, such as blended interventions or “cash plus” interventions.

**_The cost to implement the equivalent value of transfers is lower for cash than in-kind transfers._** It is difficult to know which modality is the best value for money. Four studies in our review report implementation costs and suggest that cash transfers are more cost efficient than

42
in-kind transfers, but only two of them discuss the cost-benefit or cost-effectiveness of such programs. In particular, studies suggest that mobile money cash transfers are the most efficient transfer method if mobile network infrastructure is available and convenient, and beneficiaries understand how to withdraw and transfer mobile money. The second most efficient transfer method is manual cash delivery, followed by voucher transfers, with food transfers being the most expensive way to deliver assistance. Studies find that the effectiveness of equivalentlyvalued transfer modalities are similar, suggesting that cash may be more cost-effective if the per unit implementation cost is cheaper. However, as noted by Gentilini (2016), design features, scalability, general equilibrium effects, and risks all matter for evaluating cost-effectiveness more generally. While it is encouraging that at least two studies included a cost-benefit or costeffectiveness analysis, we argue that future impact evaluations need to provide a more robust analysis for cost calculations of humanitarian programs.

**_Given the lack of rigorous causal evidence on humanitarian assistance programs, there is a high dividend to be earned from conducting more impact evaluations in humanitarian settings._** Substantial evidence gaps need to be filled to better understand implementation design choices, such as which population to target, what type of modality to transfer, and the duration and frequency of transfers. In this review, we identified several cross-cutting evidence gaps. More studies are needed on: (i) the impact of humanitarian assistance on human development, gender, and social cohesion; (ii) the conditions for which cash transfers can be more effective than in-kind transfers; (iii) targeting mechanisms in humanitarian settings; and (iv) not only the immediate impacts of humanitarian assistance programs, but also the long-term effects that arise in the recovery period.

43