International Cash-Based Food Assistance: USAID Has Established Processes to Monitor Cash and Voucher Projects, but Data Limitations Impede Evaluation
Released: 2016-09-20T00:00:00

Letter

Background

In 2015, FFP provided EFSP funding for cash transfer and food voucher projects in 30 countries (see fig. 1). To deliver assistance through these projects, USAID’s implementing partners employ a variety of mechanisms. These mechanisms include (1) distributing cash manually or electronically through accounts at banks or other financial institutions and (2) distributing paper or electronic food vouchers that entitle the holder to buy goods—typically, approved items from participating vendors—or services up to the voucher’s designated cash value. The value of the cash transfers and food vouchers is generally based on a formula that attempts to bridge the gap between the beneficiaries’ food needs and their capacity to meet those needs.

Additionally, in fiscal years 2010 through 2015, USAID funded the following EFSP regional awards for cash transfer and food voucher projects: (1) the Syria Regional Award, in countries hosting Syrian refugees—Egypt, Iraq, Jordan, Lebanon, and Turkey—awarded each year since 2012; (2) the Central America Drought Award—for Guatemala, El Salvador, and Honduras—awarded in 2014; and (3) the Ebola Regional Response—for Guinea, Sierra Leone, and Liberia—awarded in 2015.

USAID EFSP Funding since 2010

In fiscal years 2010 through 2015, USAID awarded EFSP grants totaling about $3.27 billion, including $1.42 billion for cash transfer and food voucher projects. Awards for such projects grew from about $76 million in fiscal year 2010 to nearly $432 million in fiscal year 2015 (see fig. 2). In fiscal years 2010 through 2015, awards for cash transfers increased from about $42 million to approximately $119 million, while awards for food vouchers increased from about $34 million to approximately $312 million. During the same period, USAID awards of Title II funding for emergency food aid—primarily in-kind assistance—decreased from about $1.52 billion to approximately $1.07 billion.

USAID Requirements for Proposing Emergency Cash-Based Food Assistance Projects

Since 2010, USAID’s APS for International Emergency Food Assistance has outlined agency requirements for EFSP cash-based food assistance proposals. The APS, which serves as a primary source of information for prospective applicants for EFSP grants, requires applicants to explain the rationale for the assistance modality they propose—cash transfer, food vouchers, LRP, in-kind food aid, or some combination of these modalities. Since 2011, the APS has required grant applicants to provide justification for a proposed project that includes cash-based assistance or in-kind food aid, in terms of the project’s timeliness, cost-effectiveness, or appropriateness.

Timeliness. Since 2011, the APS has required grant applicants to explain whether in-kind food aid or prepositioned stocks can arrive in a sufficiently timely manner through the regular ordering process to address urgent or emergency needs.

Cost-effectiveness. Since 2011, the APS has required grant applicants to provide cost-effectiveness information that affected their choice of modality. Further, according to the APS, in certain cases the cost of cash-based food assistance may be lower than that of in-kind assistance, while in other cases the difference may be negligible.

Appropriateness. Since 2011, the APS has required grant applicants to explain why cash transfers or food vouchers, or both, may be more appropriate than in-kind food distributions. For example, potential beneficiaries may have physical access to functioning markets but lack sufficient purchasing power. (Fig. 3 shows other examples of reasons that EFSP grant applicants have cited for deeming cash transfer or food voucher projects to be most appropriate.) The 2015 APS states that, depending on market conditions, cash-based food assistance may be deemed more or less appropriate than in-kind food aid to address specific emergency food security needs. Since 2015, the APS has required applicants to justify their selection of either cash transfers or food vouchers as the preferred modality.

USAID released a draft revision of the APS in May 2016 for public comment; according to USAID officials, the revised version will be finalized in the autumn of 2016. The draft APS requires grant applicants proposing EFSP projects to provide a justification that addresses how market appropriateness, feasibility, project objectives, and cost-efficiency influenced the modality selection. According to the draft APS, when addressing how feasibility influenced the selection of the assistance modality, applicants are to consider the time-sensitive nature of the emergency, including availability of rapid-response options such as prepositioned commodities or prenegotiated cash transfer or voucher response mechanisms.

USAID Monitoring Requirements for Cash Transfer and Food Voucher Projects

Monitoring and evaluation perform two separate but interrelated functions. Monitoring is the collection of data to determine whether projects are being implemented as intended and the tracking of progress with preselected indicators throughout the life of a project. Data collected through monitoring can be used by project managers to make incremental changes and adjustments to project implementation. Evaluations consist of ad hoc or periodic studies to assess whether and how projects achieved their expected goals. An evaluation may also identify outcomes that can be attributed to the project and may assess the project’s cost- effectiveness. Evaluations may rely on a range of quantitative and qualitative measures in addition to preselected indicators, comprehensive research designs, and appropriate statistical analysis of data, including data collected through monitoring activities. Results of evaluations can provide program managers with critical evidence for future program design.

USAID’s APS contains monitoring and evaluation–related requirements for cash transfer and food voucher grant applications as well as minimum reporting requirements for data in projects’ final reports.

Requirements for grant applications. Since 2010, the APS has required grant applications to include a monitoring and evaluation plan. In addition, the 2015 APS required that the monitoring and evaluation plan include a logical framework showing the causal linkages between activities, outputs, outcomes, and goals; identify assumptions and potential risks that are critical to the success of a project; and specify key indicators with proposed targets to track the project’s performance. In 2015, the APS also began requiring grant applicants to plan and budget for a baseline survey and a final evaluation survey for projects that propose an implementation period of greater than 12 months. Under the 2015 APS, such projects that include activities to influence beneficiary behaviors must also include at least one high-level food security or nutrition outcome indicator for each project purpose.

Minimum reporting requirements for project final reports. Since 2010, the APS has required implementing partners to submit final reports containing monitoring data that meet minimum programmatic reporting requirements. The reports are to include data on the number of beneficiaries targeted and reached; the cost per beneficiary; retail price information on key staples (before, during, and after project implementation); and the actual number and value of cash transfers or food vouchers distributed and redeemed by beneficiaries. The reports are also to include information about beneficiaries’ use of resources provided through cash transfer projects. In addition, since 2011, the APS has required that final reports include the time from donor-signed agreement to first distribution; a description of how the program addressed gender needs; the planned number and value of cash transfers or food vouchers distributed to, and redeemed by, beneficiaries; and information on the types and quantities of commodities they procured with food vouchers. Further, in 2015, USAID began requiring final reports to include the average cost per beneficiary per month for each modality as well as learning on the appropriateness of selected modalities.

USAID and Implementing Partners Have Established Processes for Monitoring Cash and Voucher Projects

USAID has established a process for monitoring implementation of EFSP cash transfer and food voucher projects by assigning monitoring roles and responsibilities to headquarters and in-country mission staff, developing country monitoring plans, and developing tools to assist its field staff. USAID’s process includes actions such as visiting distribution and project sites, speaking with beneficiaries and retailers, and meeting regularly with partners’ in-country staff. Implementing partners have established processes to monitor cash and voucher projects during and after distributions of assistance. To ensure that assistance is delivered according to their procedures and to the targeted beneficiaries, implementing partners monitor distributions and interview beneficiaries about the distribution process. In addition, implementing partners conduct postdistribution surveys to gather information about the relevance, efficiency, and effectiveness of the assistance.

USAID Has Established a Process to Monitor Implementation of Cash and Voucher Projects

USAID has assigned monitoring roles and responsibilities to headquarters and in-country mission staff. According to USAID officials, FFP grant officers, in headquarters, are primarily responsible for reviewing implementing partners’ quarterly and final reports. The grant officers are additionally responsible for meeting with implementing partner headquarter officials to discuss project progress and performance. Also, FFP’s Monitoring and Evaluation Team is responsible for developing monitoring tools, standards, guidance, implementing partner reporting requirements, and providing training for FFP officers and implementing partners who are responsible for monitoring cash and voucher projects. The FFP field officers are primarily responsible for verifying information provided by implementing partners, communicating regularly with implementing partners in country, and providing grant officers with information on project progress and performance.

The seven FFP field officers we interviewed told us that they conduct various monitoring activities to verify information that partners provide, such as visiting distribution and project sites, completing site visit reports, speaking with beneficiaries and retailers, and meeting regularly with partners’ in-country staff. In addition, the FFP field officers said they discuss the results of their monitoring efforts and site visit reports with the grant officers on a regular basis. To aid FFP field officers in conducting site visits, the FFP Monitoring and Evaluation Team, in 2015, developed a monitoring tool that includes sample questions for FFP officers to consider when making site visit observations and to ask respondents during monitoring visits. The tool is organized by type of respondent (i.e., implementing partner staff, beneficiaries, service providers, and retailers or market vendors), and the information collected is used to complete trip reports. According to FFP officials, as of December 2015, the Monitoring and Evaluation Team had shared the tool with every USAID field office.

To ensure regular monitoring of all FFP programs in a country, the FFP Monitoring and Evaluation Team began training FFP field officers in developing country monitoring plans in 2014, according to USAID officials. Agency officials said the country monitoring plans use a risk- based approach to prioritize the monitoring of country projects across the FFP portfolio (including EFSP and Title II projects), to establish the number of site visits per month and year, to determine which monitoring activities to conduct, and to allocate staff resources. In addition, USAID officials said they plan to institute a requirement that FFP officers in missions with FFP programs must complete country monitoring plans and report periodically on progress in implementing objectives identified in the country monitoring plan. As of June 2016, USAID had developed monitoring plans for 26 of 28 countries. According to the FFP officers we spoke with, country monitoring plans have helped them prioritize monitoring site visits for both Title II and EFSP projects despite challenges such as limited embassy resources (e.g., motor-pool availability and staffing shortfalls), adverse operating conditions (e.g., road closures or weather), and security concerns. According to USAID officials, to mitigate security-related constraints, FFP has awarded contracts to third-party monitors in countries where USAID has limited access. For example, for Somalia, USAID contracted with a third-party monitor to verify project activities and conduct post-distribution surveys on a monthly basis for 20 percent of the project sites. According to a representative of the monitor, it uses a risk-based approach to monitoring, prioritizing site visits and rotating the sites visited on a monthly basis.

Implementing Partners Have Established Processes to Monitor Cash and Voucher Projects during and after Distribution of Assistance

To monitor projects and collect required information, implementing partners have established processes to monitor activities during and after distributions of EFSP cash transfer and food voucher projects.

Distribution monitoring. According to implementing partner representatives for projects in Kenya, Liberia, and Somalia, their monitoring processes include observing distribution processes to ensure that the cash transfers and food vouchers are delivered according to standard procedures, for the stated purposes, and to targeted beneficiaries. The representatives said that during distribution, monitors collect information on the actual number of beneficiaries served, the actual transfer value provided, the timing of the distribution, and beneficiary participation by gender and age. They also collect information about the distribution site and other aspects of the distribution process, such as queue management, waiting time, beneficiary verifications, access, security, and safety. In addition, implementing partner monitors may speak with beneficiaries about their satisfaction with the distribution process. Implementing partners also monitor financial service providers to ensure that their implementation of the cash transfer is according to plan. In addition, implementing partner representatives monitor the voucher process at the vendor or retailers’ level.

Figure 4 shows implementing partner representatives conducting distribution monitoring for a cash transfer project in Liberia.

Postdistribution monitoring. According to implementing partner representatives for programs in Kenya, Liberia, and Somalia, their processes include surveying beneficiaries and non-beneficiaries at their residences after the assistance is distributed. According to the implementing partners, postdistribution surveys are one of the main information sources for assessing the relevance, efficiency, and effectiveness of the assistance provided. During the surveys, implementing partner representatives gather information about beneficiary households, including, among others, income, expenditures, food security outcomes, coping strategies, perceived problems with the assistance, and modality preferences. In addition, implementing partner representatives said their processes also include monitoring markets after distributions by sampling traders, shops, and markets for changes in commodity prices. Further, for food voucher projects, implementing partner representatives for the project in Somalia said that they also monitor vendors or retailers to collect information on food voucher use and availability of commodities, accuracy of commodity price displays, and to determine if the food vouchers are used for intended purposes.

We observed implementing partner representatives using technology and tools that may enhance the collection and analysis of monitoring data. For example, implementing partner representatives for the project in Somalia, based in Nairobi, Kenya, were using beneficiary management systems to register beneficiaries and were remotely monitoring beneficiary purchases and redemptions, and vendor sales and activities, for an EFSP food voucher project in Somalia. In Kenya and Liberia, we observed representatives of implementing partners for projects in those countries using mobile devices with standardized forms to conduct on-site distribution monitoring, postdistribution monitoring, and market monitoring surveys (see fig. 5). Implementing partner representatives showed us the tablets and standardized forms they use for collecting monitoring information for their projects in Somalia. According to the implementing partners, using this technology enables rapid aggregation and analysis of the data collected. We also observed implementing partner representatives conducting remote phone-based surveys with beneficiaries for projects in Kenya and Somalia. The survey included a short series of questions on household food consumption and coping strategies. Further, the implementing partners established hotlines in Kenya and Somalia for beneficiaries to provide feedback, including complaints, about project implementation and the assistance provided.

Data Gaps and Indicator Weaknesses Limit USAID’s Ability to Evaluate the Performance of Cash and Voucher Projects

Incomplete reporting of data that USAID requires, as well as weaknesses in USAID’s performance indicators, limits the agency’s ability to evaluate the projects’ performance. Our review of the final reports for 14 cash transfer and food voucher projects found that most of the reports lacked some of the required data that are necessary for performance evaluation. In addition, the indicators that USAID uses to assess projects’ timeliness, cost-effectiveness, and appropriateness—three criteria that it considers in approving grant applications—have weaknesses. For example, USAID’s indicator for timeliness does not capture delays in actual implementation against what was planned. Moreover, USAID’s indicator for cost- effectiveness—cost per beneficiary—does not produce data that can be compared across projects or modalities, because it does not include a standardized unit for measuring costs. Additionally, USAID has not set a benchmark for assessing market appropriateness based on market prices. These limitations affect USAID’s ability not only to evaluate the overall performance of cash transfer and food voucher programs but also to learn from experience and make informed decisions on future projects.

Most Final Reports We Reviewed Lacked Required Data about Project Performance

Our review of the final reports submitted for the 14 cash transfer and food voucher projects found that most of the reports lacked some data about the projects’ performance that USAID’s APS required. According to USAID officials, implementing partners may have communicated the missing information through other means, such as during meetings or in e-mails or quarterly reports; however, the 2013 APS, to which the projects were subject, required partners to submit the information in their projects’ final report. The APS required the final report for each project to include the number of beneficiaries targeted and reached, disaggregated by sex and age, and to verify that the program assessed and addressed gender needs and issues. In addition, the APS required the final report to list the planned and actual number and value of food vouchers or cash transfers that implementing partners distributed and beneficiaries redeemed. Further, the APS required the final report to include information on how the beneficiaries used cash transfers as well as information on the types and quantities of commodities that beneficiaries procured with food vouchers. Finally, the APS required the final report to include data that the agency will use to measure projects’ overall performance in terms of timeliness, cost-effectiveness, and appropriateness: the time from donor- signed agreement to first distribution to beneficiaries; the cost per beneficiary; and retail price information on key staples in the area of the program before, during, and after the distribution.

While most of the 14 final reports that we reviewed included most of the required data on project beneficiaries, only 1 report, for a food voucher project in Sudan, included all 12 data elements required by USAID; the other reports lacked up to 8 of the required elements. Figure 6 shows the required data elements that were included in the 14 final reports that we reviewed.

Data on beneficiaries. All 14 final reports that we reviewed included data on the number of beneficiaries reached, although several reports were missing other required data about the beneficiaries who received the cash transfers or food vouchers. The reports indicated that cash transfer or food voucher projects reached almost 926,000 beneficiaries in Burundi, Chad, the DRC, Haiti, the Philippines, Somalia, Sudan, Yemen, and Zimbabwe. Two reports did not list the number of beneficiaries targeted by the project. In the 12 reports that listed these data, our analysis found that 9 of the 12 projects reached or exceeded their target numbers. However, 4 reports did not enumerate beneficiaries by age range and 4 reports did not describe how gender needs were assessed and addressed, as USAID requires.

Data on assistance distributed and redeemed. Half of the 14 reports we reviewed did not list the planned value of cash transfers or the number of food vouchers distributed as the APS requires. Only 5 reports—for projects in Chad, DRC, Somalia, Sudan, and Zimbabwe—included all required information about the actual number and value of cash transfers or food vouchers distributed and redeemed. Further, of the 4 final reports for cash transfer projects, only 1 report included the required information about beneficiaries’ use of the transferred cash. Of the 8 final reports for food voucher projects, only 4 listed, as required, the types and quantities of commodities that beneficiaries procured with their vouchers. Only 1 of 2 projects that comprised both cash transfer and food vouchers included the required information on the beneficiaries’ use of the cash and types and on quantities of commodities procured with the vouchers.

Data on timeliness, cost-effectiveness, and appropriateness. Most of the 14 reports we reviewed were missing required data that USAID uses to analyze the timeliness, cost-effectiveness, and appropriateness of cash transfer or food voucher projects.

Timeliness. Twelve of the 14 reports did not list the number of days from the award agreement date to the cash transfer or food voucher distribution date. In addition, 5 of the 14 reports did not list the first date of distribution, making it difficult to determine the time from the partner’s signing of the agreement with USAID to the first distribution of the cash transfers or food vouchers. We requested any data that implementing partners had submitted through other documents, such as e-mails or quarterly reports; however, USAID did not provide such data.

Cost-effectiveness. Six of the 14 final reports did not list a cost per beneficiary as required by USAID. According to a country officer, the final report for one project excluded the cost per beneficiary because the project, which was jointly funded by FFP and the Office of U.S. Foreign Disaster Assistance (OFDA), used an OFDA award mechanism that does not require this calculation.

Appropriateness. Twelve of the 14 final reports did not list retail price information on key staples in the area of the program before, during, and after the cash transfer or food voucher distribution as required by USAID. Some of the 12 reports listed commodity prices during distributions but did not list the prices 2 weeks before the program began or 2 weeks after it ended. According to USAID, country officers do not request additional price data from partners if there is no evidence of projects’ having a negative impact on the local market.

USAID’s Current Indicators for Assessing Cash and Voucher Projects’ Timeliness, Cost- Effectiveness, and Appropriateness Have Weaknesses

USAID’s indicators for measuring cash transfer and food voucher projects’ timeliness, cost-effectiveness, and appropriateness have weaknesses that limit the agency’s ability to evaluate these aspects of projects’ performance. USAID’s indicator for timeliness produces data that show how quickly the distribution occurs after the award agreement is signed; however, the indicator does not compare actual and planned distribution schedules, which would identify delays in project implementation. Moreover, USAID’s indicator for cost-effectiveness does not produce data that can be compared across projects or modalities, because it does not include a standardized unit for measuring costs. Further, USAID does not have a benchmark for the market appropriateness indicators that measure the impact of cash or voucher assistance on local markets. According to federal standards for internal control, management should use quality information, including relevant data from reliable sources, to achieve an agency’s objectives. As a result of the weaknesses in these indicators, USAID and its implementing partners have limited ability to fully evaluate cash transfer and food voucher projects’ timeliness, cost-effectiveness, and appropriateness and may miss opportunities to learn from past experiences to improve the program.

USAID’s Indicator for Timeliness Does Not Track Implementation Delays

The indicator that USAID and its implementing partners use to assess cash transfer and food voucher projects’ timeliness provides a measure of how quickly assistance is delivered after an award is signed, but it does not systematically capture any delays in project implementation. Since 2011, USAID’s APS has required that the final report for each cash transfer and food voucher project include, as an indicator of the project’s timeliness, the number of days between grant approval and distribution of assistance. According to USAID officials, the agency uses this measure to determine how quickly assistance can be provided under different modalities. USAID also told us that it considers the duration of time between award and first distribution, based on the proposed project timeline, as a general indicator of implementing partners’ initial progress in starting a project. USAID officials noted that anecdotal evidence suggests that the number of days between grant approval and first distribution of assistance has generally decreased for cash transfer and food voucher projects.

The number of days between grant approval and first distribution of assistance does not register delays in project implementation—an important aspect of timeliness, since delays in implementing cash and voucher projects can have a severe impact on beneficiaries who rely on the assistance for their livelihood. USAID does not systematically collect data that would show cash transfer and food voucher project delays, such as the planned dates of distribution for comparison with the actual dates.

Since 2011, the APS has not required partners to list in their final reports the planned date of first distribution, which appears in the implementing partner’s award proposal. Delays have been noted for many EFSP projects. For example, a recent USAID Inspector General report compared the planned and actual distribution dates for four cash transfer and food voucher projects in West Africa during the Ebola response and concluded that delays in distribution—resulting from delays in award approvals and challenges in staff recruitment and coordination—averaged 3 months. We requested and obtained the planned first-distribution dates for the 14 cash transfer and food voucher projects we reviewed and then compared the planned dates with the actual dates listed in the projects’ final reports. We found that for 7 of the 14 projects, first distribution of assistance was delayed by an average of 2 months.

USAID’s Indicator for Cost- Effectiveness Does Not Produce Comparable Data

The indicator that USAID uses to measure the cost-effectiveness of cash transfer and food voucher projects does not include a standardized unit for measuring cost. As a result, USAID is unable to use the cost data it collects to assess the relative cost-effectiveness of its EFSP cash transfer and food voucher projects or to compare the cost-effectiveness of such projects with that of projects that used other modalities. Since 2011, USAID’s APS has required that the final report for each cash or voucher project include, as an indicator of cost-effectiveness, the cost per beneficiary. This cost is calculated by dividing the aggregate cost of providing assistance by the actual number of beneficiaries of a cash transfer or food voucher project.

We found that in the 8 final reports that listed the cash transfer or food voucher project’s cost per beneficiary—USAID’s indicator for cost- effectiveness—the reported costs could not be used to compare the projects’ cost-effectiveness because the cost units were not standardized. In these 8 reports, the reported costs ranged widely, from $10 to $219 per beneficiary. According to USAID, per-beneficiary costs may range widely for a number of reasons, including variance in the size of the transfers, the frequency of the transfers, the presence of complementary services, the overall scale of the projects, and overhead costs for each project. For example, given economies of scale, a project that prints vouchers for 50,000 beneficiaries may have lower overhead costs than a project that prints vouchers for 10,000. Also, indirect costs, such as for security or transportation, may be higher for projects that require additional security or are located in remote, less accessible districts. In addition, according to an implementing partner responsible for a cash transfer project and a food voucher project in Liberia, its cost-per-beneficiary calculation for the voucher project did not include indirect costs because they were included in its calculation for the cash transfer project. USAID officials acknowledged that without a standardized cost unit, partners have applied different methods to calculate the cost per beneficiary for cash transfer and food voucher projects.

To weigh the cost-effectiveness of various food aid modalities during project design, one of USAID’s implementing partners, the United Nations World Food Programme (WFP), uses a method that compares different modalities’ costs for delivering the same numbers of calories and for delivering the same nutritional values. Using this method—known as the Omega tool analysis—to compare potential costs, WFP determined that in Burkina Faso and Niger, food transfers could deliver the same nutritional value at a lower cost than cash transfers or food vouchers. However, WFP’s analysis show that in both cases, combining modalities could deliver the same nutritional value at a lower cost than using a single modality. In Senegal, WFP determined that food vouchers or a combination of food transfers and cash transfers or food vouchers could deliver the same nutritional value at a lower cost than food transfers. According to WFP officials, as of July 2016, WFP had used this tool while designing projects in 24 countries and plans to rely on this tool to assess cost-efficiency and cost-effectiveness. WFP officials also noted that WFP’s country offices planned to use this tool to validate cost-efficiency assumptions at the close of programs.

According to a USAID official, the agency is considering revising the APS to require grant applicants to use the cost of a food ration for cash-based and in-kind food aid projects when estimating potential cost-effectiveness. The official indicated that the change is intended to standardize methods for justifying proposed projects on the basis of cost-effectiveness. The draft version of the APS that USAID released for public comment in May 2016 provides more explicit parameters for estimating cost-effectiveness that could enable the agency to compare proposed modalities during the application process.

USAID’s Indicator for Cash and Voucher Projects’ Appropriateness Lacks Associated Benchmarks for Market Impact

USAID has not required implementing partners to establish benchmarks for measuring cash transfer and food voucher projects’ impact on prices in local markets—an indicator it uses to measure the projects’ appropriateness. Since 2010, the APS has required that implementing partners’ final reports include retail price information on key staples in the area around the project before the project begins, monthly during the project, and after the project ends. However, USAID has not required the partners to establish any associated benchmarks for this indicator and as a result, USAID and implementing partners may not be able to assess whether market price fluctuations are within acceptable ranges.

Implementing partners are using different thresholds to assess market impact. For example, WFP’s policy is to reassess the value of the assistance provided if prices for staple foods fluctuate by 10 percent or more. According to WFP officials, WFP may also change the assistance modality as a result of price fluctuations. Representatives of other implementing partners told us that they reassess project implementation when prices fluctuate by 20 percent or more. Implementing partner representatives also stated that, because of a lack of guidance on this topic, they were unsure of how USAID assesses the market information they submit. As of the end of June 2016, USAID’s draft revisions to the APS included a reference to market monitoring guidance for implementing partners. According to this guidance, implementing partners are to set price thresholds as a basis for determining when price changes must be investigated and explained. The guidance also states that when prices increase or decrease beyond this set parameter, the change should be flagged and the cause should be investigated. The draft revisions to the APS also require partners to include market analysis information and any significant changes in their quarterly and final project reports. The draft revisions to the APS do not explicitly require partners to report the price thresholds in their final reports.

Studies Showed Cash, Vouchers, and Food Transfers Can Improve Food Security, and Contextual Factors May Explain Impact Variations

Our review of 14 rigorous studies of the relative impacts of cash transfers, food vouchers, and food transfers found the studies demonstrate that all three modalities can improve food security outcomes for people facing food emergencies. The modalities’ impacts on food security outcomes varied by study, with no modality consistently outperforming the others. Contextual factors, such as the severity of the food crisis and the capacity of local markets, may have contributed to this variation. Studies that compared the relative costs of the three modalities generally reported that cash transfers were least expensive, although most of the studies did not account for the full costs associated with the modalities.

Studies Demonstrated That All Three Modalities Can Improve Food Security Outcomes

Among the studies we reviewed, those comparing recipients of cash transfers, food vouchers, and food transfers with control groups of individuals who received no assistance demonstrated that all three modalities can lead to significant improvements in dietary quantity and quality. For example, a study in Ecuador found that, compared with the control group, all recipients of cash transfers, food vouchers, and food transfers experienced significantly improved outcomes in terms of the value and volume of food consumed, caloric intake, and dietary diversity. Similarly, a study in Bangladesh found that all recipients of cash transfers, food transfers, or combinations of the two modalities experienced significantly improved outcomes for value and volume of food and for caloric intake.

Eight of the 14 studies we reviewed included control groups, which allowed us to compare food security outcomes for recipients of cash transfers, food vouchers, or food transfers with outcomes for groups of individuals that did not receive any type of assistance. Among 6 studies that used control groups to examine the value and volume of food provided, 5 studies showed statistically significant improvements as a result of receiving all three types of assistance. Similarly, 3 of 6 studies that used control groups to examine caloric intake also found statistically significant improvements for all three modalities.

The studies showed that all three assistance modalities generally led to significantly improved outcomes for the most frequently assessed outcomes in the studies—value and volume of food consumed, caloric intake, and dietary quality. However, the results for some other outcomes were mixed. In particular, 3 of the 4 studies that considered nutritional status showed some modalities leading to significant improvements, while the fourth study showed no modality leading to significant improvements.

Studies Showed the Modalities Had Varying Impacts on Food Security Outcomes

Our analysis of the 14 studies comparing the impacts of cash transfers, food vouchers, and food transfers on food security outcomes found that the modalities’ performance varied by study and project. Moreover, none of the three modalities consistently outperformed the others. Contextual differences among the projects may have contributed to the variations in the modalities’ impacts. (For detailed results of our literature review, see app. II.)

Modality Performance Varied by Study, and No Modality Consistently Outperformed the Others

Our review of the 14 studies found that the impacts of the modalities on food security outcomes varied by study. For example, the study of the project in Yemen reported that recipients of the cash transfers bought a wider range and higher value of food items, showing that cash transfers provided significantly greater improvements in dietary quality than did food transfers. The Yemen study also found that food transfers provided higher levels of caloric intake than did cash transfers, which the study’s authors attributed to the relatively inexpensive staples, such as wheat and oil, included in the food transfers.

In contrast, a study of a project in Niger found that food transfers resulted in significantly greater improvements in dietary quality than did cash transfers, indicating that, in this instance, food transfers provided a more varied and higher-quality diet. This finding was attributed to the fact that cash recipients in Niger bought significantly cheaper bulk grains with their transfers than food recipients. The study authors determined that the cash beneficiaries purchased these cheap bulk grains in anticipation of seasonal price increases—essentially stocking up on supplies for the “hungry” season. As a result, the food recipients, who relied on the food transfers provided by the project, had a more varied and higher-quality diet.

In some instances, a modality significantly outperformed one or both of the other modalities for a particular outcome in some countries but not in others. For example, studies of projects in Niger, Sri Lanka, Uganda, and Yemen showed that cash transfers had a greater impact on the value and volume of food than food transfers had, but studies of projects in Bangladesh, the DRC, Ecuador, and Mexico showed that the three modalities had comparable impacts on this outcome. Moreover, a study of a project in Niger comparing the impacts of the three modalities found that food transfers provided the greatest dietary diversity, while studies of projects in Malawi, Uganda, and Yemen found that cash transfers provided greater dietary diversity than food transfers. The DRC and Ecuador studies did not find that any modality had a consistently and significantly greater impact on dietary diversity.

Differences in Modality Impacts Were Often Not Statistically Significant

In 11 of the 14 studies, some of the differences in modalities’ impacts on food security outcomes were not statistically significant and therefore may have resulted by chance. For example, the study of the Ecuador project found that cash transfers, food vouchers, and food transfers all led to improvements in beneficiaries’ dietary diversity. While statistically significant compared with the control group, the improvements that resulted from each of the three modalities were not statistically different from each other across the metrics studied. Similarly, a study of a project in Mexico noted that both cash transfers and food transfers increased the value and volume of food consumed, and levels of nutrients, with statistically insignificant differences in the two modalities’ impacts.

Contextual Factors May Help Explain Variations in Modality Impacts

While we could not determine precisely why the three modalities’ impacts varied in the studies we reviewed, contextual differences in the projects the studies examined may help explain such variation. According to researchers we spoke with, the effectiveness of cash transfers, food vouchers, and food transfers are heavily influenced by contextual factors such as the severity of the food crisis, beneficiaries’ specific needs when projects began, changes in market prices, and the projects’ designs.

While each of the 14 studies we reviewed controlled for multiple factors when comparing the modalities’ impacts on food security outcomes, we noted numerous contextual differences among the evaluated projects that may have contributed to variation in these impacts. The projects’ purposes and goals ranged from providing disaster relief to responding to seasonal food emergencies, aiding displaced persons, addressing long- standing emergencies, or helping achieve development objectives. In addition, the projects varied in factors such as the value and frequency of the transfers and whether the assistance was conditional or unconditional. For example, a study in Sri Lanka examined a 3-month project that took place in tsunami-affected regions of the country. The study found that cash transfers had a greater impact than food transfers on dietary quantity. However, its findings are difficult to apply to many other situations because of the post-tsunami conditions that prevailed, the relatively short duration of the emergency relief project, and some unevenness in the frequency of food transfers relative to cash transfers.

Changes in markets and prices can affect the modalities’ relative impacts on food security outcomes. For example, a study of a project in Ethiopia found that both food transfers and a mixture of food and cash transfers significantly outperformed cash-only transfers in reducing the periods of time when beneficiaries experienced food shortages. However, the study’s authors noted that the project took place during a period of high food-price inflation that considerably reduced the value of the cash transfers compared with that of the food transfers.

Moreover, differences in projects’ design—specifically, whether the project provided any assistance in addition to the food or cash transfers— may have contributed to variation in the modalities’ impacts. For example, a study of a project in Bangladesh concluded that aspects of project design, and in particular the use of complementary programs designed to achieve project outcomes, were associated with the greatest impacts. This study found that modalities combined with complementary programs that provided guidance and training on nutrition significantly outperformed cash transfers, food vouchers, and food transfers that were not combined with complementary programs. According to the study’s authors, these results help demonstrate that food security outcomes can be improved by including complementary programs designed to achieve those objectives and that these improvements occur when such programs are combined with any of the three modalities.

Studies Generally Reported Cash Transfers Were Least Expensive, but Fewer Than Half Considered Full Costs

Among the 14 studies we reviewed, 11 considered the relative costs of cash transfers, food vouchers, and food transfers. Seven of these 11 studies reported that cash transfers were least expensive, while 1 study reported that food transfers were least expensive. Of the 3 remaining studies that considered the modalities’ relative costs, 1 study did not identify the least expensive modality and the other 2 studies had mixed results, depending on the method used to estimate costs. The studies considered costs for a range of activities related to delivering the assistance, such as administration, staffing, banking for cash transfers, production for food vouchers, and storage and transportation for food transfers. However, only 3 of the 11 studies considered not only the activity costs related to delivering the assistance but also the costs of purchasing food.

The three studies that considered the costs of purchasing food reached varying conclusions about the modalities’ overall relative costs. One study, of a project in Niger, found that food transfers were least expensive overall. Another study, of a project in Malawi, reported mixed results, finding that cash was more cost-effective but food transfers were more cost-efficient. The third study, of a project in Bangladesh, did not clearly identify the least expensive modality.

According to researchers, food transfers may be the least expensive modality overall if market conditions allow implementing partners to purchase food at prices low enough to offset the costs of delivering it, since these costs are usually higher than the costs of delivering cash transfers or food vouchers. For example, in some situations, implementing partners’ savings from purchasing large amounts of food wholesale—that is, for considerably lower prices than beneficiaries would pay in their local markets—could offset the higher costs of delivering the food.

Conclusions

With USAID’s additional flexibility to choose among modalities of food assistance, the ability to demonstrate the performance of each modality is of increasing importance. Monitoring and evaluation are essential to assessing and demonstrating, with timely and credible evidence, the effectiveness of the various modalities employed to deliver assistance, including cash transfers and food vouchers. USAID and its implementing partners have established processes to monitor EFSP cash transfer and food voucher projects. Specifically, USAID has assigned roles and responsibilities and developed monitoring plans and tools to aid FFP field officers who conduct site-visits and meet with implementing partners. Further, implementing partners have developed processes to monitor the effectiveness of their projects through distribution and postdistribution monitoring. However, the incompleteness of required information about project performance in partners’ final reports limits USAID’s ability to assess whether EFSP cash and voucher projects met their performance goals. In addition, weaknesses in USAID’s indicators for measuring EFSP cash transfer and food voucher projects’ timeliness, cost, and appropriateness—criteria that USAID considers in approving the projects—limit the extent to which reported data can demonstrate the effectiveness of such projects and be used to evaluate the performance of cash and vouchers relative to in-kind food aid.

The 14 studies we reviewed showed that cash transfers, food vouchers, and food transfers can all improve food security. At the same time, the studies’ findings of variation in the modalities’ impact on food security suggest that when selecting modalities for emergency food assistance, USAID and other donors should carefully consider contextual factors that could influence project outcomes. By taking steps to strengthen monitoring and evaluation of its cash transfer and food voucher projects, USAID will ensure access to information about each implemented project that it can use in planning future projects—including selecting the appropriate modality—and will help USAID optimize its efforts to respond to continuing food emergencies around the world.

Recommendations for Executive Action

To strengthen USAID’s monitoring and evaluation of cash transfer and food voucher projects and help ensure improved program oversight of these projects, we recommend that the USAID Administrator take the following two actions:

Take steps to ensure that final reports submitted for cash transfer and food voucher projects comply with USAID’s minimum data requirements.

Strengthen the indicators USAID uses to measure the timeliness, cost-effectiveness, and appropriateness of EFSP cash transfer and food voucher projects.

Agency Comments and Our Evaluation

We provided a draft of this report to USAID, which provided both written and technical comments. In its written comments, reproduced in appendix IV, USAID agreed with our findings and recommendations. Regarding our first recommendation, USAID agreed to standardize data collected in final reporting by soliciting and hiring program support officers for each of its geographic teams. Regarding our second recommendation, USAID agreed to improve indicators to ensure evaluation and comparison across its emergency food assistance portfolio by updating these indictors in its forthcoming Annual Program Statement for International Emergency Food Assistance. We incorporated USAID’s technical comments as appropriate.

We are sending copies of this report to the appropriate congressional committees; the Administrator of USAID; and the Secretary of State. The report is also available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-9601 or melitot@gao.gov. Contact points for our Offices of Congressional Relations and of Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V.

Appendix I: Objectives, Scope and Methodology

In this report, we (1) examine the U.S. Agency for International Development’s (USAID) and implementing partners’ processes for monitoring cash transfer and food voucher projects; (2) analyze the extent to which monitoring data that partners reported to USAID can be used to evaluate the performance of such projects; and (3) review studies that examined the relative impacts of cash transfers, food vouchers, and food transfers on food security outcomes and the relative cost of these modalities.

To address these objectives, we analyzed data and reviewed program documents provided by USAID and its implementing partners for Emergency Food Security Program (EFSP) cash transfer and food voucher projects, including the United Nations World Food Programme (WFP) and nongovernmental organizations (NGOs). We met in Washington, D.C., with USAID officials and with implementing partner officials representing NGOs that received USAID EFSP grants, and we met in Rome, Italy, with officials from WFP and the U.S. Mission to the UN. In addition, we visited Kenya and Liberia, where we reviewed projects that were under way in Kenya, Liberia, and Somalia. We selected these projects using criteria that included the amount of EFSP funding, the type of project (cash transfer or food voucher), and the implementing partner. While in Kenya and Liberia, we met with USAID officials from the U.S. missions; representatives of implementing partners, vendors, and financial institutions; and project beneficiaries, among others.

To examine USAID’s and implementing partners’ processes for monitoring EFSP cash and voucher projects, we reviewed activities that USAID and implementing partners, including WFP and NGOs, undertook. We also reviewed relevant program documents that they provided, such as award agreements, project quarterly and final reports, market monitoring tools, site visit reports, and results of distribution surveys. We interviewed USAID officials in Washington, D.C., including award agreement officers, country backstop officers, and members of FFP’s monitoring and evaluation team, among others. We also interviewed FFP regional monitoring officers located in Dakar, Senegal, and in Nairobi, Kenya, and we interviewed FFP field officers responsible for monitoring cash and voucher projects in eight countries. In addition, we met in Washington, D.C., with implementing partner officials representing NGOs that were awarded USAID EFSP grants and in Rome, Italy, with officials from WFP and the U.S. Mission to the UN. Further, we visited Kenya and Liberia, where we reviewed EFSP-funded cash and voucher projects under way in Kenya, Liberia, and Somalia. In Kenya and Liberia, we also visited project sites, observed implementing partners conducting monitoring activities, and met with beneficiaries of projects in those countries.

To examine the extent to which reported data can be used to evaluate the performance of cash transfer and food voucher projects, we reviewed USAID’s 2013 Annual Program Statement (APS) for International Emergency Food Assistance and grant agreements for final reporting requirements. In addition, we reviewed 14 final reports that implementing partners submitted to USAID for, respectively, four cash transfer projects, eight food voucher projects, and two projects with both cash transfer and food voucher components. We selected these projects using the following criteria, intended to ensure a diverse sample of implementing partners: (1) the funding was awarded for NGO projects in fiscal years 2013 and 2014 or for WFP projects in fiscal year 2014, all of which were subject to USAID’s 2013 APS; (2) the award was at least $2 million for NGO projects or at least $3 million for WFP projects; and (3) the projects closed by October 31, 2015. Of the 36 NGO projects funded in fiscal years 2013 and 2014, 22 received awards of at least $2 million; of these, only 11 were closed by October 31, 2015, and therefore were included in our review. Of the 18 WFP projects funded in fiscal year 2014, 7 received awards of at least $3 million; of these 7 projects, only 3 were closed by October 31, 2015, and therefore were included in our review. Because the 14 projects represent a nonprobability sample, our findings may not be generalizable to all USAID EFSP programs. We reviewed the APS and grant agreements to identify minimum programmatic reporting requirements, and we reviewed the final reports for the 14 selected projects for data responding to these requirements. In addition, we reviewed the 14 final reports for data and the indicators that USAID uses to measure cash transfer and food voucher projects’ timeliness, cost- effectiveness, and appropriateness—namely, time from award to first distribution, cost per beneficiary, and food price data. We met with USAID officials in Washington, D.C., to discuss APS requirements, implementing partner reports, and the quality of indicators.

To review studies’ conclusions about the relative impacts of cash transfers, food vouchers, and food transfers on food security and the modalities’ relative costs, we took the following steps:

To identify relevant studies, we (1) considered prior reviews of rigorous evaluations related to cash-based humanitarian assistance and food security; (2) conducted our own search of literature using appropriate terms; (3) and asked six researchers who had conducted prior reviews of rigorous evaluations whether they knew of any additional evaluations. We screened the studies identified by these sources to determine whether any of the studies met the following criteria: (1) compared cash transfers, food vouchers, or food transfers with at least one other modality; (2) evaluated at least one food security outcome; and (3) considered food security outcomes by means of randomized control trials or groups, carefully selected comparison groups, or a quasi-experimental design that used statistical techniques to make precise comparisons. This process resulted in our selecting 14 rigorous studies, made public since 2006, that evaluated whether cash transfers, food vouchers, or food transfers were more successful in achieving intended impacts on food security outcomes in situations where at least two modalities were implemented and could be compared. These 14 studies examined projects in 10 countries: Bangladesh, the Democratic Republic of the Congo, Ecuador, Ethiopia, Malawi, Mexico, Niger, Sri Lanka, Yemen, and Uganda. (See app. II for a list of these studies.)

To review the 14 studies we selected, we used a data collection instrument (DCI) designed to examine the studies’ design, quality, and major findings. (App. II lists the studies we reviewed and provides a detailed summary of the results of our review.)

We developed the following eight key food security outcomes based on our analysis of the metrics in the 14 selected studies and on our discussions with researchers with relevant expertise: value and volume of food, caloric intake, dietary diversity, food consumption score, nutrient number and levels, caloric value, experiential measures, and nutritional status. We reviewed each study’s results to determine whether any modality demonstrated a statistically significant improvement in one or more of these outcomes compared with the other modality, or modalities, that the study examined. Two independent analysts used the DCI to review the studies, discussing and reconciling any differences in their initial assessments. In some instances, studies presented the results for several metrics for the same outcome, such as a household dietary diversity score and a household dietary diversity index. In those instances, our decision rule was that one modality had to demonstrate a statistically significant improvement over all other modalities examined for all metrics that the outcome comprised. If this criterion was not met, we determined that the modalities’ impacts were comparable. We did not consider results for subpopulations, such as by region, income group, gender, or age range, except when results for a particular outcome were reported only for a subpopulation. We also did not consider impacts on factors that did not qualify as costs or food security outcomes, such as beneficiaries’ income, assets, and overall consumption. When examining findings about dietary diversity, we relied on indices and metrics that had been created for this purpose; we did not assess results for individual food groupings, as these varied greatly between studies and did not lend themselves to the overall methodology that we employed. (App. II summarizes the results of our review.) The researchers who helped us identify relevant studies that met our criteria generally stated that they believed the 14 studies we selected constituted a reasonable body of evidence that no one modality significantly outperforms the other and that all modalities can lead to improvements in food security. However, one researcher reported that the evidence base does not allow for generalizable conclusions about the situations in which specific modalities are most appropriate.

We also considered whether the 14 studies examined the relative costs of the modalities, and in the studies that did, we considered which modality was found to be least expensive at delivering the same level of assistance. The studies employed a variety of data and methods to estimate relative costs. For example, some studies examined cost-efficiency, others examined cost-effectiveness, and still others examined both. In addition, some studies used a costing method adapted from the health economics field that involves identifying the costs of specific “activities” required to implement a modality, such as staffing, banking, and production. Meanwhile, three studies comparing cash and food transfers examined the costs of both delivering the modalities and purchasing the food beneficiaries consumed. If more than one method was used to estimate costs in a study, we considered the results for all methods used and reported that one modality was the least expensive only if all methods used found it so. Otherwise, we reported that the results were mixed. In one instance, a study examined four projects in one country: one project provided food transfers, another provided cash transfers, and two others provided a mix of cash and food transfers. This study examined both the costs and the cost-efficiency of the transfers. In that instance, because the results differed by project and method used, we could not determine which modality was least expensive.

Our review of the 14 studies was not intended to assess any individual donor such as USAID and thus the conclusions we analyzed cannot be inferred to assess the performance of USAID’s projects. In addition, several of the projects examined were pilots conducted to assess the comparative performance of food and cash on household food security.

The projects that the studies examined varied in terms of contextual factors such as the projects’ purposes and goals, projects’ design and outcomes examined. While the studies demonstrate that no modality consistently outperformed the others across a range of settings and situations, this variance does not allow us to determine the precise conditions in which one modality might outperform the others. To obtain context and background, we also considered additional studies on cash- based assistance that did not meet our criteria for inclusion in our systematic review but provided insights into projects that used cash transfers, food vouchers, food transfers, or a combination of these modalities. (See app. III for a list of these additional studies.)

We conducted this performance audit from June 2015 to September 2016 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Summary Results of GAO Literature Review

Appendix III: Additional Studies Examining Food Aid Modalities

In addition to reviewing 14 rigorous studies that compared the impacts of cash transfers, food vouchers, and food transfers on food security outcomes, we consulted the following 23 studies for insights into projects that included one or more of these modalities. In some cases, these studies examined large-scale cash transfer or food voucher projects or the relative costs of the modalities and were recommended by food security researchers we interviewed. 1. Attanasio, David, Erich Battistin, and Alice Mesanard. “Food and Cash Transfer: Evidence from Colombia.” The Economic Journal, vol. 122 (March 2011): 92-124. 2. Bailey, Sarah, and Paul Harvey. “State of Evidence on Humanitarian Cash Transfer.” Background note for the High Level Panel on Humanitarian Cash Transfers, Overseas Development Institute. March 2015. 3. Bailey, Sarah, and Sophie Pongracz. “Humanitarian Cash Transfers: Cost, Value for Money and Economic Impact.” Background note for the High Level Panel on Humanitarian Cash Transfers, Overseas Development Institute. March 2015. 4. Creti, Pantaleo. The Impact of Cash Transfers on Local Markets: A Case Study of Unstructured Markets in Northern Uganda. Cash Learning Partnership, April 2010. 5. Creti, Pantaleo. The Voucher Programme in the Gaza Strip: Mid- Term Review (Final Report). Commissioned by the World Food Programme and Oxfam, United Kingdom, 2011. 6. De Sardan, Jean Pierre, Hannatou Adamou, Oumarou Hamani, Younoussi Issa, Nana Issaley, and Issaka Oumarou. “Cash Transfers in Niger: The Manna, the Norms and the Suspicions.” Translation of a paper undertaken by LASDEL and originally published in French. 2013. 7. Doocy, Shannon, and Hannah Tappis. Cash-Based Approaches in Humanitarian Emergencies: A Systematic Review. Funded by the U.K.’s Department of International Development (DFID). Johns Hopkins. 2015. 8. Doocy, Shannon, Emily Lyle, and Hanna Teppis. Emergency Transfers in Northern Syria. An Economic Evaluation of GOAL Food Assistance Programs in Idleb Governorate. Johns Hopkins, Bloomberg School of Public Health, September 2015. 9. Dunn, Sophia, Mike Brewin, and Aues Scek. Cash and Voucher Monitoring Group: Final Monitoring Report of the Somalia Cash and Voucher Transfer Programme. London: Humanitarian Policy Group, Overseas Development Institute, 2014. 10. Gentilini, Ugo. Our Daily Bread: What is the Evidence on Comparing Cash versus Food Transfers? Social Protection & Labor Discussion Paper No. 1420. Washington, D.C.: World Bank Group, July 2014. 11. Gentilini, Ugo. “Revisiting the ‘Cash vs. Food’ Debate: New Evidence for an Old Puzzle.” World Bank Research Observer, vol. 31 (2016): 135-167. 12. Gentilini, Ugo. The Other Side of the Coin: The Comparative Evidence of Cash and In-Kind Transfers in Humanitarian Situations. Washington, D.C.: World Bank Group, July 2016. 13. Gilligan, Daniel, Melissa Hidrobo, John Hoddinott, Shalini Roy, and Benjamin Schwab. “Much Ado about Modalities: Multicountry Experiments on the Effects of Cash and Food Transfers on Consumption Patterns.” International Food Policy and Research Institute paper prepared for the Agriculture & Applied Economics Association Annual Meeting, Minneapolis, July 2014. 14. Hedlund, Kerren, Ben Allen, Maria Bernandez, Muriel Cala, Saul Guerrero, Chloe Milloz Baudy, Juilien Morel, Panos Navrozidis, Silkie Pietzsch, and Michael Yemene. Meta-Evaluation of ACF Fresh Food Voucher Programmes. ACF International, with funding from the Cash Learning Partnership and the European Commission on Humanitarian Aid and Civil Protection. January 2012. 15. Hoddinott, John, Daniel Gilligan, Melissa Hidrobo, Amy Margolies, Shalini Roy, Susanna Sandström, Benjamin Schwab, and Joanna Upton. Enhancing WFP’s Capacity and Experience to Design, Implement, Monitor and Evaluate Vouchers and Cash Transfer Programmes. Study Summary. International Food Policy Research Institute, 2013. 16. Husain, Arif, Jean-Martin Bauer, and Susanna Sandstrӧm.

Economic Impact Study: Direct and Indirect Impact of the WFP Food Voucher Programme in Jordan. World Food Programme, 2014. 17. Kardan, Andrew, Ian MacAuslan, and Ngoni Marimo. Evaluation of Zimbabwe’s Emergency Cash Transfer (ZECT) Program—Final Report. Oxford Policy Management, supported by WFP and Concern Worldwide, 2010. 18. Majewski, Brian, Lois Austin, Katherine George, Carol Ward, and Kurt Wilson. WFP’s 2008 Cash and Voucher Policy (2008-2014): A Policy Evaluation. Evaluation Report—Volume 1. Konterra Group paper for World Food Programme. 2014. 19. Margolies, Amy, and John Hoddinott, “Costing Alternative Transfer Modalities,” IFPRI Discussion Paper 01375. International Food Policy Research Institute, September 2014. 20. Maunder, Nick, Victoria De Bauw, Neil Dillon, Gabrielle Smith, and Sharon Truelove. “Evaluation of the Use of Different Transfer Modalities in ECHO Humanitarian Aid Actions, 2011-2014.” Analysis for Economic Decisions. Evaluation commissioned by the European Commission. 2016. 21. Michelson, Hope, Christopher Barrett, Laura Cramer, Eric Lentz, Megan McGlinchy, Mitchell Morey, and Richard Mulwa. “Cash, Food, or Vouchers? An Application of the Market Information and Food Security Response Analysis Framework in Urban and Rural Kenya.” Food Security, vol. 4 (2012): 455-469. 22. Mountfield, Ben. Unconditional Cash Transfers in Gaza: An External Review. Commissioned by Oxfam, Great Britain. 2012. 23. Poulsen, Lene, Sophia Dunn, Sado Hashi, Mohamed Adnan Ismail, Colleen McMillon, Caroline Tanner and Njoroge Thuo.

Somalia Protracted Relief and Recovery Operation: Strengthening Food and Nutrition Security and Enhancing Resilience, June 2012–December 2015. Mid-Term Evaluation Report. World Food Programme, 2015.

Appendix IV: Comments from the U.S. Agency for International Development

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Joy Labez (Assistant Director), Sushmita Srikanth (Analyst-in-Charge), David Blanding, Carol Bray, Ming Chen, Martin De Alteriis, Neil Doherty, Mark Dowling, Reid Lowe, Julia Ann Roberts, and Shannon Roe made key contributions to this report.

Related GAO Products

International Food Assistance: Cargo Preference Increases Food Aid Shipping Costs, and Benefits Are Unclear. GAO-15-666. Washington, D.C.: September 25, 2015.

International Food Assistance: USAID Should Systematically Assess the Effectiveness of Key Conditional Food Aid Activities. GAO-15-732. Washington, D.C.: September 10, 2015.

International Cash-Based Food Assistance: USAID Has Processes for Initial Project Approval but Needs to Strengthen Award Modification and Financial Oversight. GAO-15-760T. Washington, D.C.: March 26, 2015.

USAID Farmer-to-Farmer Program: Volunteers Provide Technical Assistance, but Actions Needed to Improve Screening and Monitoring. GAO-15-478. Washington, D.C.: April 30, 2015.

International Cash-Based Food Assistance: USAID Has Developed Processes for Initial Project Approval but Should Strengthen Financial Oversight. GAO-15-328. Washington, D.C.: March 26, 2015.

International Food Aid: Better Agency Collaboration Needed to Assess and Improve Emergency Food Aid Procurement System. GAO-14-22. Washington, D.C.: March 26, 2014.

International Food Aid: Prepositioning Speeds Delivery of Emergency Aid, but Additional Monitoring of Time Frames and Costs Is Needed. GAO-14-277. Washington, D.C.: March 5, 2014.

Global Food Security: USAID Is Improving Coordination but Needs to Require Systematic Assessments of Country-Level Risks. GAO-13-809. Washington, D.C.: September 17, 2013. E-supplement GAO-13-815SP.

International Food Assistance: Improved Targeting Would Help Enable USAID to Reach Vulnerable Group. GAO-12-862. Washington, D.C.: September 24, 2012.

World Food Program: Stronger Controls Needed in High-Risk Areas. GAO-12-790. Washington, D.C.: September 13, 2012.

International Food Assistance: Funding Development Projects through the Purchase, Shipment, and Sale of U.S. Commodities Is Inefficient and Can Cause Adverse Market Impacts. GAO-11-636. Washington, D.C.: June 23, 2011.

International School Feeding: USDA’s Oversight of the McGovern-Dole Food for Education Program Needs Improvement. GAO-11-544. Washington, D.C.: May 19, 2011.

International Food Assistance: Better Nutrition and Quality Control Can Further Improve U.S. Food Aid. GAO-11-491. Washington, D.C.: May 12, 2011.

International Food Assistance: A U.S. Governmentwide Strategy Could Accelerate Progress toward Global Food Security. GAO-10-212T. Washington, D.C.: October 29, 2009.

International Food Assistance: USAID Is Taking Actions to Improve Monitoring and Evaluation of Nonemergency Food Aid, but Weaknesses in Planning Could Impede Efforts. GAO-09-980. Washington, D.C.: September 28, 2009.

International Food Assistance: Local and Regional Procurement Can Enhance the Efficiency of U.S. Food Aid, but Challenges May Constrain Its Implementation. GAO-09-570. Washington, D.C.: May 29, 2009.