International Food Assistance: Improved Targeting Would Help Enable USAID to Reach Vulnerable Groups
Released: 2012-09-24T00:00:00

Letter

Background

The United States Has Stated Its Commitment to Combating World Hunger and Malnutrition

Combating world hunger and malnutrition is a stated objective of the Food for Peace Act, which authorizes international food assistance for developing countries. The United States has also stated its commitment to the Millennium Development Goal to halve world hunger by 2015, and it supports the Scaling-Up Nutrition (SUN) movement to provide assistance to country-led efforts to address maternal and child malnutrition. To support SUN, the United States and others initiated the 1,000 Days public-private partnership, which aims to improve nutrition for pregnant and lactating mothers and children under 2. Adequate nutrition in this critical period in a child’s life is widely recognized to have the greatest impact on saving lives, developing a child’s cognitive and physical capacity, and mitigating the risk of chronic disease. According to the USAID Policy Framework 2011-2015, USAID plans to ensure that the quality of U.S. government food aid is improved within 3 years to meet the nutritional requirements of vulnerable populations overseas, including by developing new blended products and formulations to support pregnant and lactating mothers and children under 2.

We previously reported that, although Title II emergency funding is intended to address short-term food needs, more than half of the funding in fiscal year 2010 was spent on multiyear emergency programs. See GAO-11-491. In 2011, the 14 countries that received U.S. emergency food assistance every year from fiscal years 2006 through 2011 were Afghanistan, Burundi, Central African Republic, Chad, Colombia, Democratic Republic of the Congo, Ethiopia, Kenya, Nepal, Rwanda, Somalia, Sudan, Tanzania, and Uganda. In addition, 23 percent of the emergency food commodities were delivered to 15 countries that received U.S. food assistance for 3 to 5 years from fiscal years 2006 through 2011. Three percent was delivered to four countries that received emergency U.S. food aid for 1 to 2 years. percent.$207 million, which accounted for about 17 percent of total Title II emergency funding. A higher percentage of the total population in Ethiopia suffers from malnutrition than in most other recipient countries, with 51 percent of children under 5 suffering from stunting.

Ethiopia, one of the four countries we visited, received about WFP is the largest provider of global food aid and procurement. The countries that received the largest amounts of specialized food products from WFP were Ethiopia, Pakistan, Kenya, Niger, and Somalia.

Specialized food products are designed to meet specific nutritional needs of vulnerable groups but are more costly than traditional food products. As a result, within a fixed budget, USAID and its implementing partners must decide whether to provide more nutritious but more costly food to fewer people, or less nutritious and less costly food to more people. In other words, they face a quality-quantity trade-off. Table 1 provides illustrative examples of cost per ration for the three different types of food assistance. See appendix III for a more detailed comparison of cost differences between traditional food products and specialized food products.

Targeting in food assistance programs is an iterative process that aims to ensure that food reaches and is consumed by people whose characteristics meet certain eligibility criteria, such as age, gender, income level, asset level, or nutritional status. Figure 3 presents a simplified schematic of the overall targeting process and its key phases— design, implementation and monitoring, and evaluation—and steps within each phase. As key stakeholders in the targeting process, USAID and its implementing partners, including WFP and NGOs, play an important role, as do host governments. In the design phase, implementing partners design food assistance programs and submit proposals to USAID. USAID reviews the proposals and decides whether to fund the programs. Feedback occurs within and across each of the phases—both in host countries and at USAID headquarters—and is crucial to maximizing targeting effectiveness, leading to steps within the process that may not be strictly sequential. For example, during the design phase, USAID and its implementing partners may conduct an assessment of needs to determine the basis for the design of a program; however, as needs may change or be clarified, they may retarget or make adjustments during the monitoring phase to address issues that may arise.

Various In-Country Factors Affect USAID and Implementing Partners’ Ability to Target Food Assistance Effectively to Vulnerable Groups

USAID and its implementing partners face a range of in-country factors that, to varying degrees, affect their ability to effectively target food assistance to vulnerable groups. These factors include (1) the quality of data used to identify and reach recipients, (2) host government policies, and (3) sharing of rations among recipients and community members. Targeting effectiveness is reduced when data quality is poor, host government policies cause distortions in program design and implementation, and sharing prevents food rations from being consumed by the intended recipients in the intended amounts. USAID and its implementing partners take steps to mitigate such challenges by, for example, employing technology to improve data quality, coordinating closely with government officials to foster better relationships, and educating recipients about proper food usage to reduce sharing. In some cases, host governments have facilitated targeting efforts by, for example, establishing national targeting guidelines that set a common standard, or national statistical offices that assist in collecting data. Nevertheless, ensuring that food assistance reaches intended recipients remains difficult.

Poor Data Quality May Hinder Implementing Partners’ Ability to Identify and Reach Intended Recipients

Lack of Reliable Population and Household Data May Hinder Targeting Effectiveness

Poor data quality—lack of timely and accurate information—may affect implementing partners’ ability to effectively identify and reach recipients. For example, in Zimbabwe, USAID and three implementing partners noted that a lack of current and reliable population data made it difficult to determine the overall number and geographic distribution of households that are in need of food assistance.partner told us that because it used inaccurate data on average In Guatemala, an implementing household size to determine the initial ration size, people who were initially identified received more food than they would have received if the data had been accurate. Although the error was later corrected, if the data had been accurate, resources could have been used more optimally to reach people in need.

USAID and implementing partners we spoke with stated that sudden natural disasters or conflicts could raise security concerns for implementing partners, hindering their ability to reach the originally targeted recipients. Furthermore, gathering reliable data on transient populations is challenging. For example, USAID and an implementing partner in Ethiopia told us that in some areas of the country, it is difficult to determine the number and location of people in need of food assistance, particularly pastoralists, who move often as a traditional way of life and to cope with drought or natural disasters. As a result, it is difficult for implementing partners to accurately assess the needs in a particular geographic area and design an appropriate food assistance program.

In addition, natural disasters or conflicts may raise security concerns, hindering ability to reach targeted recipients. We recently reported that security concerns prevented WFP from conducting field monitoring of food distribution to determine whether the food rations reached the originally targeted recipients in some high-risk areas of Ethiopia, Kenya, and Somalia. For example, WFP noted that it has been unable to access six districts in the Somali region of Ethiopia since May 2011. As a result, WFP’s ability to collect data to ensure that the intended recipients received their food assistance in these high-risk areas is limited.

USAID and implementing partners have taken some measures to improve data quality by building capacity through technology, training, and other activities. For example, USAID funds the Famine Early Warning Systems Network (FEWS NET), which is used to monitor and prepare for changes in food assistance needs. FEWS NET monitors and analyzes vulnerability information, using multiple sources such as satellite imagery and field observations. Moreover, some countries, such as Ethiopia, have established national statistical offices that can assist in collecting data for targeting food assistance. In addition, a 2011 report on food assistance stated that implementing partners are working on increasing the speed, accuracy, accessibility, and comparability of information. Implementing partners in two countries we visited told us that they are using mobile devices, such as tablets and phones, to collect recipient and distribution data. The use of technology enables the implementing partners to better identify and track recipients throughout the program and identify needs. In the aforementioned example about excess ration size in a Guatemala program, the implementing partner used tablets to collect information on recipient consumption patterns. In this way, the implementing partner ultimately discovered the ration error and corrected the ration size for each household, freeing up resources to reach more recipients as a result. Also, implementing partners in Guatemala and Sri Lanka indicated that they train their staff and community volunteers on data collection, and work with the host governments to improve the governments’ ability to collect data. In addition, some countries, such as Sri Lanka, have conducted repeated assessments of food assistance needs over several years, which can lead to improvements in the precision of the data collected.

Barrett, et al., 67. partner in Guatemala stated that data need to be improved continuously to measure outcomes and impacts of targeting, particularly for programs with a nutritional objective.

Host Government Policies May Cause Distortions, Limiting Targeting Effectiveness

Host government policies may lead to distortions, hampering targeting effectiveness, but implementing partners have made some efforts to reduce these adverse effects. We previously reported that one of the key challenges to accurately assessing the needs of vulnerable groups was a lack of coordination among key stakeholders—especially with host governments—on assessments of food assistance needs. In addition, some host country governments have been criticized for underestimating actual needs or directing implementing partners to operate only in certain geographic areas, due to political or other reasons. As a result, implementing partners may not be able to reach recipients or locations most in need of food assistance. For example, an implementing partner in Ethiopia reported to USAID that the government of Ethiopia set an artificial quota for the number of people targeted in each household that in some cases did not reflect the actual needs, and severely hampered the partner’s ability to reach vulnerable groups as a result. However, USAID and implementing partner officials in Ethiopia also told us that working with the government’s distorted figures is less challenging now than in the past, due in part to recent efforts of local and regional government officials to improve the validity and documentation of needs assessments as well as better stakeholder coordination.

In some instances, however, host government policies may facilitate targeting efforts. For example, the government of Sri Lanka has worked closely with WFP to identify vulnerable groups and has supported efforts to improve both data collection and the analysis of food needs, including by supporting the research organization that partners with WFP in conducting assessments of needs for food assistance. In another example, the government of Ethiopia has published National Targeting Guidelines that are intended to standardize and improve targeting efforts.country operate under a commonly understood set of targeting policies and practices.

This document helps all food assistance stakeholders in the To address host government policies that cause distortions, implementing partners undertake efforts to coordinate with stakeholders and verify information on food assistance needs. Implementing partners we spoke with told us they work with each other and with host governments in the initial phase of the targeting process to increase transparency, in an effort to encourage more accurate government estimates of actual needs. For example, in Ethiopia, USAID officials told us that to increase transparency, donors are working with the government to introduce software tools and technology that facilitate access to information and increase public awareness and thereby discourage government authorities from manipulating data on food assistance needs. Moreover, to help facilitate distribution of food assistance to intended recipients in Guatemala, implementing partners stated that it is essential to closely coordinate with government authorities at the beginning of the targeting process to obtain approval for the use of new products and to set up the appropriate distribution channels and protocol. In addition, in Sri Lanka, an implementing partner told us that it plans to use local organizations to conduct independent verification of the potential recipient list, which is largely selected by the government. Doing so would help the implementing partner ensure that only recipients who qualify for food assistance are included on the list, increasing the likelihood that food assistance reaches the intended recipients. Despite these efforts, implementing partners have limited ability to influence host government policies.

Sharing Reduces Targeting Effectiveness If Food Rations Are Not Consumed by the Intended Recipients in the Intended Amounts

Sharing within recipient households and among community members may result in food rations being consumed by unintended recipients or in unintended amounts, but implementing partners have taken some measures to reduce sharing. External assessments suggest that sharing of food rations is a widespread and established coping mechanism when insufficient food is available. The 2011 Food Aid Quality Review (FAQR) report and the 2011 WFP guidance on targeted food assistance programs acknowledge that sharing of specialized food products is a concern, and according to a 2011 USAID assessment of a food assistance program in the Somali region of Ethiopia, sharing of food rations is widespread. In addition, in countries we visited, USAID and its implementing partners told us that both CSB and traditional food products are routinely shared within and among households in some communities—a finding we previously reported in 2011. The 2011 USAID assessment also notes that sharing is an established coping mechanism for the recipient community when not everyone in the community receives food rations. When food rations are shared, the intended recipients may not consume the intended food products in the desired amounts, which may reduce targeting effectiveness by limiting nutritional impact, particularly for specialized food products that are intended for vulnerable groups.

Implementing partners have made efforts to reduce the likelihood of sharing, especially of specialized food products. Specifically, implementing partners have employed various strategies to teach recipients how to use specialized food products and have monitored recipient food ration consumption. For example, one implementing partner in Guatemala requires pregnant or lactating women to attend education sessions, where they learn about the benefits of the specialized food products and how to properly prepare them, before they can receive rations. Implementing partners in Guatemala also print culturally relevant instructional images on the food packages or the canvas bags given to recipients to carry the rations. The images explain how to prepare the food products and depict the type of person for whom the products are intended—such as a pregnant woman or a child under 2 years of age. One of these implementing partners reported that it had seen an improvement in recipient participation in these education sessions and expected that these sessions would reduce sharing.

In addition, implementing partners use community volunteers to monitor effectiveness or consumption of food products. For example, in Guatemala, implementing partners train “mother leaders”—mothers who are also recipients—to provide training to other recipients on how to prepare food and monitor outcomes by, for example, observing improvement in a child’s weight or overall health appearance. In Sri Lanka, another implementing partner uses health volunteers from the recipient community and coordinates with the host government to ensure that specialized food products are consumed by the children through monthly monitoring of their nutritional status at government-run clinics and weighing stations. The health volunteers also follow up with the mothers of these children, who are receiving specialized food products, if they do not bring their children to the monthly checkup. While implementing partners have taken these and other steps to address sharing, evidence of the impact of these steps has yet to be determined.

Weaknesses in the Design, Monitoring, and Evaluation Phases of USAID’s Targeting Process Hinder Targeting Effectiveness, Although Some Improvements Are Under Way

Weaknesses in the design, monitoring, and evaluation phases of USAID’s targeting process hinder targeting effectiveness, although the agency is taking actions to make improvements. In the design phase of the targeting process, USAID does not provide sufficient guidance on whether and how to target specialized food products. Specifically, USAID’s guidance on design for both emergency and development programs is neither up-to-date nor complete, and does not adequately address key benefits and risks that inform decisions on whether to target specialized food products. In both USAID’s monitoring and evaluation phases, weaknesses limit targeting effectiveness and hinder decision making. USAID currently does not require monitoring of key indicators needed to determine the level of targeting effectiveness for either emergency or development programs. Furthermore, its evaluations do not systematically address targeting effectiveness. Without adequate guidance, monitoring, and evaluations, USAID cannot ensure targeting effectiveness in its food assistance programs. USAID is taking some steps to improve both guidance and monitoring. For example, USAID has a contract with Tufts University to develop updated guidance, and the agency is taking steps to improve monitoring by planning to track indicators such as detailed age breakdowns that are key to better understanding targeting effectiveness. However, these steps do not fully address the weaknesses in USAID’s targeting process.

USAID Does Not Provide Sufficient Guidance in the Design Phase on Whether and How to Target Specialized Food Products, Although the Agency Is Starting to Make Improvements

USAID’s Guidance for Targeting Is Neither Up-to-Date Nor Complete, Hindering Decision Making

We found that USAID’s guidance for targeting is neither up-to-date nor complete for both emergency and development programs, which reduces the ability of implementing partners to make informed decisions in the design phase. USAID currently provides its implementing partners with a range of guidance and tools. Of these, the Commodities Reference Guide is USAID’s official standard reference for food assistance programs and is intended to be used by USAID and implementing partner staff in deciding how to plan, manage, control, evaluate, and use Title II- funded food products. It is available on USAID’s public website and provides information on available food products, including nutritional values, physical properties, and storage and handling guidelines. However, USAID has not updated the Commodities Reference Guide since 2006 and has not included guidance in the Commodities Reference Guide on all of the products currently used in USAID food assistance programs. The 2011 Food Aid Quality Review also noted that the Commodities Reference Guide and other USAID guidance relevant to targeting are neither up-to-date nor complete and recommended, for example, that USAID improve its guidance to enable implementing partners to better determine whether to use certain products for programs.

We found that the lack of updated and complete guidance has hindered implementing partners’ ability to make better-informed targeting decisions. One participant at our roundtable, for example, told us that his organization was unable to find all of the products it was using for a program in the outdated Commodities Reference Guide. As a result, it was not able to use these products in its program. Furthermore, USAID has recently deployed some limited quantities of various new specialized products without providing official standard guidance on how to use them. We recommended in 2011 that USAID provide clear guidance on whether and how best to use new specialized food products, including guidance to its implementing partners on targeting strategies to ensure that the products reach their intended recipients. USAID concurred with our recommendation and is taking steps to develop new guidance, but has deployed new specialized food products in the interim. USAID has purchased relatively small quantities of new specialized food products over the past 2 years, including those shown in table 2 below. For example, USAID purchased just over $6.5 million worth of these products in 2011, as compared with $502 million of traditional food products and $42 million of traditional specialized food products purchased through Title II emergency program funding.

In addition, USAID is planning to introduce nine new or reformulated products in the final part of 2012 and 2013, including new RUTFs and ready-to-use supplementary foods (RUSFs) (see app. IV). USAID has not issued fully updated or complete guidance for all of these products. However, it has issued some guidance on their use. Moreover, USAID officials told us that they are providing the products only on a limited basis to organizations such as UNICEF that have experience using them in controlled environments, such as clinics, and have issued their own guidance on the use of these products.

USAID guidance inadequately addresses key benefits and risks of using specialized food products, according to USAID and implementing partners we spoke with during our field visits and our expert roundtable. This inadequate guidance hinders decision making on whether to use these products. As discussed earlier, the benefit of specialized food products is that, while more costly, they are also more nutritious, or of higher quality, than traditional food products. However, USAID has not quantified or clearly defined the degree of benefit that specialized food products may provide. In 2011, we reported that in recent years, nutritionists have debated the appropriateness of using fortified and blended foods to prevent and treat malnutrition in young children 6 to 24 months old, who have smaller stomachs, making it more difficult for them to eat enough of the product to obtain sufficient nutrients. As a result, the benefits of some traditional specialized food products are not clear. In addition, limited information on new specialized products is available. As we previously reported, USAID and implementing partners do not know how well new specialized food products perform in promoting nutritional health indicators, such as weight gain and growth, particularly in a program setting, or how well they perform in comparison to traditional food products.which these products promote desired outcomes, is still being studied by USAID, WFP, nutritionists, and other researchers. As a result, while USAID is building knowledge about these products, it is not providing sufficient guidance on the benefits of specialized food products to implementing partners.

The efficacy of new specialized products, or the extent to USAID also lacks guidance on how to adequately address risks of using specialized food products, according to implementing partners we spoke with during our field visits and our expert roundtable. A key targeting risk is that various factors implementing partners face in-country may reduce targeting effectiveness to such a degree that the additional cost of using specialized food products outweighs the potential benefit. This trade-off becomes more significant with the higher cost of new specialized food products, for which the cost per ration can be more than triple the cost of traditional food products. Poor data quality, host government policies, and sharing may reduce implementing partners’ ability to identify and reach recipients, but USAID’s existing guidance does not adequately inform decisions on whether the reduction in targeting effectiveness is of such a degree that the use of specialized food products is no longer justified.

USAID is taking some steps to enhance guidance on whether and how to use new specialized food products, but fully up-to-date and complete guidance will not be completed until at least late 2013. In response to our 2011 recommendation on improved targeting guidance, USAID stated in its official agency response in July 2011 that it would work to address our recommendations through the second Food Aid Quality Review study now under way with Tufts University. This work is expected to include cost-effectiveness analyses on new specialized food products, adding information important to help determine whether and how to use them. In addition, according to USAID documents and officials, USAID is updating and improving the Commodities Reference Guide and other guidance related to targeting, including for new specialized food products. However, this work will not be completed until September 2013 at the earliest, according to USAID officials. USAID also plans to introduce other guidance before September 2013, including updated fact sheets for individual products. According to USAID officials, this interim guidance will be released on an as-needed basis, beginning in October 2012. In addition, USAID has existing guidance that helps inform implementing partners’ decision making, including its Annual Program Statement (APS), Food for Peace Information Bulletins, and some Food and Nutrition Technical Assistance (FANTA) guidance.

Weaknesses in USAID’s Monitoring and Evaluation Efforts Related to Targeting Could Limit Targeting Effectiveness and Hinder Decision Making, but the Agency Is Starting to Take Steps to Improve Monitoring

USAID Does Not Monitor Key Indicators of Targeting Effectiveness, but Is Initiating Improvements in This Area

USAID does not require monitoring of key indicators needed to determine the level of targeting effectiveness, although it is beginning to make improvements in this area. Information on indicators that are consistent with the goals of the program is critical to determining how effectively a program targets food assistance. Targeting effectiveness can be measured by the extent to which food assistance reaches correctly targeted recipients—that is, the percentage of intended recipients that actually receive food assistance in the intended amounts (see fig. 4). Effectively targeted programs reduce the magnitude of these errors. USAID guidance states that monitoring should be used to measure progress toward planned program results. Additionally, FANTA guidance states that monitoring efforts should allow USAID and its implementing partners to assess the extent to which targeted recipients received intended food assistance. According to a USAID official, USAID field staff do consider targeting during their routine monitoring of food assistance programs. In addition, USAID requires its implementing partners to collect some data, such as the number of intended recipients for all food assistance programs, and requires other indicators to be monitored depending on the type of program—emergency or development. However, USAID does not currently require sufficient monitoring of key indicators consistent with program goals that would allow its implementing partners to report on levels of targeting effectiveness. For example, it cannot determine the effectiveness of a program targeting children under 2 because it does not monitor the age of the actual recipients in either emergency or development programs.

USAID monitoring is inadequate for both emergency and development programs because it does not monitor key data on recipients that would allow USAID to measure whether food assistance is actually reaching the intended recipients. Specifically, for emergency programs, USAID collects the total number of intended recipients from its implementing partners, but does not collect the total number of actual recipients or indicators such as breakdowns of age and gender for intended or actual recipients. According to USAID, these types of more specific indicators may not be as important for some emergency programs that focus solely on rapid lifesaving. However, these indicators are important for emergency programs that do have specific targeting goals, such as reaching severely malnourished children. For development programs, USAID collects both the total number of intended and actual recipients from its implementing partners, but as with its monitoring of emergency programs, does not collect data on key indicators such as breakdowns of age and gender. Without monitoring full sets of data for both intended and actual recipients, including key indicators consistent with program goals, USAID has limited ability to learn about the magnitude of targeting errors or the degree to which its implementing partners are achieving their program goals.

According to USAID and implementing partner officials, it is particularly complex to gather monitoring information on indicators related to targeting effectiveness about actual recipients, due in part to cost and data quality issues. These challenges are heightened for programs using new specialized food products, which are designed to provide nutritional benefits to very specific vulnerable groups, such as malnourished children or pregnant or lactating women. Identifying and selecting recipients for such programs requires using indicators that are more complex than those used for programs designed for the general population. Some of the indicators, such as nutritional status, are costly to measure and prone to errors. For example, implementing partners we spoke with during our fieldwork in Guatemala and Sri Lanka told us that they have difficulty in collecting data for some indicators in other, non-USAID programs using new specialized food products due to resource constraints, lack of technical capacity by some local NGO staff, or problems with unreliable data.

USAID is making improvements in monitoring of some nutrition-focused development programs, for example, by planning to require implementing partners to collect data on the age of young children, a common criterion for new specialized food products. However, as mentioned earlier, indicators key to measuring targeting effectiveness are not consistently monitored across all USAID food assistance programs. According to the Standards of Internal Control in the Federal Government, program managers need to compare actual performance to planned results and analyze significant differences. Without reporting targeting effectiveness, USAID cannot compare actual targeting effectiveness to planned results. As a result, USAID may not be able to make fully informed targeting decisions for both ongoing and future food assistance programs. For example, USAID may not be able to track the performance of food assistance programs’ targeting over time or across programs and may therefore miss opportunities to identify improvements to the targeting effectiveness of these programs.

USAID’s targeting evaluations are not systematic, in part, because they are not routinely conducted. USAID requires evaluations to be completed for all of its development programs but does not require them for its emergency programs. Instead, emergency programs are required to submit Annual Results Reports, which contain many of the same types of information as evaluations, but for which no baseline assessment is conducted. According to USAID officials, the difference between these requirements is due to the fact that emergency programs are by nature typically in places where there may not be the time or resources available to do a proper baseline assessment. evaluations that discussed targeting effectiveness included information on the magnitude of inclusion or exclusion errors and the level of community satisfaction with targeting. For example, USAID’s evaluation of an emergency program in Zimbabwe discussed inclusion and exclusion error, a key measure of targeting effectiveness, within a section focused exclusively on targeting. Similarly, USAID’s evaluation of an emergency program in Ethiopia mentioned the level of community satisfaction with targeting: almost 90 percent of the respondents to a survey of community members were generally satisfied with the fairness of the program’s targeting. Some evaluations, however, contained only a brief mention of targeting in general, with no mention of targeting effectiveness. For example, an evaluation of a development program in Bolivia mentioned targeting and contained tables showing monitoring indicators for the baseline compared against the final evaluation, but did not explain how the recipients were originally targeted or how the final evaluation results were verified. Other evaluations, such as a 2011 evaluation of a development program in Guatemala, did not discuss targeting or targeting effectiveness at all.

USAID policy and guidance call on USAID and its implementing partners to use evaluations as opportunities to learn about past programs to inform decision making for new programs. USAID policy calls for evaluations to “systematically generate knowledge about the magnitude and determinants of program performance, permitting those who design and implement programs…to refine designs and introduce improvements to future efforts.” USAID guidance states that evaluations should assess the extent to which the program is meeting its stated objectives. For example, if a program is providing food assistance to a vulnerable subpopulation, effective targeting is an important program objective. However, USAID’s evaluations of its food assistance programs do not systematically address targeting effectiveness, and as a result, the agency’s ability to assess the extent to which a program is meeting its stated objectives is hindered, and it may miss opportunities for learning lessons that could be useful when designing new programs or improving ongoing ones.

Conclusions

The use of specialized food products, especially some of those most recently introduced, offers the promise of providing better nutrition to the most vulnerable. However, the increased cost of these new specialized products means that their use may likely reduce the overall number of recipients receiving food assistance under a fixed program budget—a quality-quantity trade-off. Choosing more costly specialized food products over less costly traditional food products may be the optimal policy option in certain circumstances, including areas with a high percentage of children suffering from hunger and malnutrition. However, the achievement of this policy goal requires effective targeting of food assistance so that food ultimately reaches the intended recipients. If food assistance is not targeted effectively, the program may fail to achieve its nutritional goals while simultaneously feeding fewer people.

USAID recognizes the need to update and broaden its guidance on the use of specialized food products, but this revision will not be completed until late 2013 at the earliest. Issuance of improved interim guidance related to food assistance targeting will help USAID and its implementing partners make better-informed decisions about whether and how to deploy the range of food products that are available, particularly new specialized products. Moreover, the monitoring and reporting of key indicators consistent with program objectives are necessary to ensure that specialized food products are, in fact, reaching intended recipients. Improved targeting—which takes an approach that is appropriate to the circumstances and conditions—would better ensure that valuable food resources are put to their most optimal use and that vulnerable groups receive the most effective assistance available to them.

Recommendations for Executive Action

To improve USAID’s targeting of specialized food products to vulnerable groups, such as children under 2 and pregnant women, we recommend that the Administrator of USAID take the following two actions:

As USAID continues to purchase new specialized food products without updated guidance, it should issue, as appropriate, improved interim guidance to assist implementing partners in deciding whether and how to target specialized food products.

When USAID chooses to provide specialized food products to targeted vulnerable groups, it should establish and report program- specific indicators related to each targeted group to allow USAID to assess its programs’ effectiveness in reaching these groups.

Agency Comments and Our Evaluation

We provided a draft of this report to USAID, USDA, and State for comment. In its written comments, reproduced in appendix V, USAID concurred with our recommendations. USDA and State provided no written comments. We also provided relevant excerpts of this report to WFP for comment. USAID, USDA, and WFP provided technical comments that were incorporated, as appropriate.

USAID strongly agreed with our recommendation on improving interim guidance to help implementing partners decide whether and how to target specialized food products. USAID provided examples of recent and ongoing efforts that are expected to contribute to improved guidance on new specialized food products. For example, USAID expects to publish on its website updated fact sheets on food products provided in its food assistance programs and will prioritize issuing those relating to specialized food products. Although USAID noted that some existing guidance is available for three of the new specialized food products it is introducing, such as CSB+, the agency also acknowledged that it expects to issue its own guidance on all new products and update the Commodities Reference Guide.

USAID agreed with our recommendation on establishing and reporting program-specific indicators to allow USAID to assess its programs’ effectiveness in reaching targeted groups. USAID agreed with us on the need to develop new, program-specific indicators to assess the nutrition goals of new specialized food products for its Title II emergency programs and indicated that it would engage with partners on demonstrating impact and results. To that end, USAID indicated that it is in the process of recruiting a nutritionist to ensure that products used match their intended purpose and high-value specialized products are properly targeted.

We are sending copies of this report to appropriate congressional committees, the Administrator of USAID, the Secretaries of Agriculture and State, and relevant agency heads. The report is also available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-9601 or melitot@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VI.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to (1) describe in-country factors that the U.S. Agency for International Development (USAID) and its implementing partners face in targeting vulnerable groups, and (2) examine the extent to which USAID’s targeting process supports effective targeting.

To address these objectives, we met with officials at USAID and its implementing partners, including the World Food Program (WFP) and nongovernmental organizations (NGOs). In addition, we met with officials at the U.S. Department of Agriculture (USDA) and the U.S. Department of State. We also spoke with academics, experts, and practitioners associated with research institutes, universities, and NGOs. We examined USAID program documents, including guidance and tools, related to food assistance targeting processes. Furthermore, we conducted fieldwork in four countries—Ethiopia, Guatemala, Sri Lanka, and Zimbabwe—and met with officials from U.S. missions, implementing partners, and relevant host government agencies. We also convened a roundtable of 10 practitioners and experts—including representatives from implementing partners such as NGOs and WFP, academia, and research organizations—to discuss in-country factors that affect the ability of USAID and its partners to target vulnerable groups, as well as the guidance and monitoring and evaluation tools that USAID and its implementing partners use to target food assistance activities (see app. II for the list of participating organizations in our roundtable).

To provide context and background, we analyzed data from USAID and WFP to identify trends in U.S. funding for international food assistance and procurement data on the use of traditional and specialized food products. As these data were for background purposes, we did not assess their reliability. In addition, we reviewed data that we reported on in 2011 concerning cost information for specialized food products relative to traditional food products. We then reviewed similar data to obtain updates about the costs and relative length of feeding for these products and interviewed USAID, WFP, and Tufts University about the reliability of the updated data. We used this information to create an analysis comparing the amount of time various commodities could be provided for the cost of other commodities. We found that these updated data were sufficiently reliable for the purposes of this report, in that they demonstrated the order of magnitude of the relative cost of different types of food products used in food assistance programs. Although commodity prices may fluctuate and suggested feeding lengths may vary by program or individual recipient, the data were sufficiently reliable to demonstrate that there are large differences in the cost of feeding, depending on the products used. In addition, we reviewed various literature on the targeting process, as well as USAID guidance and tools to facilitate targeting decisions.

To describe in-country factors USAID and its implementing partners face in targeting vulnerable groups, we reviewed literature on targeting and new specialized food products issued by academics, research institutes, implementing partners, USAID contractors, UN organizations involved in humanitarian assistance, and independent international organizations; spoke with in-country officials such as relevant host government officials and implementing partners; and obtained the input of our roundtable participants.

To examine the extent to which USAID’s targeting process supports effective targeting, we analyzed responses and information from the general methodologies listed above. To examine the extent to which USAID provides guidance to its implementing partners on targeting, we reviewed existing USAID guidance for targeting and USAID’s contract with Tufts University and spoke with USAID and Tufts University officials about the scope of work for this contract, including the section on updating guidance. We reviewed information from USAID about the product types, costs, and tonnage of new specialized food products purchased since fiscal year 2010. We interviewed USAID about the sources of this information and also compared it to data about these products from other sources of information. These included the requests for applications that USAID provides to its implementing partners for new specialized food products, including ready-to-use therapeutic foods and lipid nutritional supplements; USAID’s commodity price calculator; and relevant legislation authorizing the use of these products. We found that the data were sufficiently reliable for the purposes of this report, in that they showed the magnitude and trends of purchases of new specialized food products with USAID funding in recent years. To examine the extent to which USAID monitors and evaluates targeting effectiveness and other related information, we analyzed monitoring information provided to us by USAID about numbers of planned recipients and actual beneficiaries for Title II food assistance programs since 2009. We reviewed current USAID policies and procedures on monitoring and evaluation. We also reviewed guidance provided by Food and Nutrition Technical Assistance (FANTA), under a cooperative agreement with USAID. This guidance covers aspects of monitoring and evaluation, such as the performance measures to be used for food assistance programs. Through searches of USAID’s website and discussions with cognizant officials, we identified a total of 30 final evaluations of USAID programs going back to 2009. Final evaluations are conducted at the end of a program. However, USAID could not assure us that it had provided all of the evaluations conducted for its development programs, and also noted that it does not require evaluations of its emergency programs. We selected 20 of these final evaluations for review based on the following criteria: we included all 3 final evaluations for the single-year programs, and selected 17 final evaluations for the multi-year programs to ensure that we had coverage by year and geographic region. We reviewed these evaluations to examine the extent to which they had addressed targeting issues. Finally, we reviewed evaluations that WFP conducted of those of its programs that were implemented with USAID funding.

We conducted this performance audit from October 2011 to September 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Organizations That Participated in GAO Roundtable

Appendix III: Cost Differences between Traditional Food Products and Specialized Food Products

Table 3 provides a comparison of cost differences between traditional and selected specialized food products, which include both traditional specialized food products and new specialized food products. For example, to feed a child 6 to 23 months old, a traditional grain-based representative ration costs $0.02 to $0.06 per day, a CSB+ ration, $0.10 to $0.21 per day, and a ready-to-use therapeutic food (RUTF) ration, $0.42 to $0.46 per day.

The cost per ration is one aspect of the relative cost of food assistance products; the length of time a product is used in a food assistance program also affects its overall relative cost. Some experts suggest that, although an individual daily ration of a new specialized food product may be relatively expensive, it may ultimately be less costly overall because it may be fed for a shorter period of time based on its suggested length of use. However, we found that some new specialized food products with a relatively shorter suggested length of use may still cost relatively more overall. For example, as shown in the table, although an RUTF’s suggested length of use (42 to 90 days) for a child 6 to 23 months old is shorter than that of a traditional grain-based ration (90 to 120 days), the RUTF may still cost more overall ($17.77 to $41.40) than the grain-based ration ($2.07 to $7.45). Despite their higher overall costs, RUTFs may be the optimal choice in certain circumstances, such as for emergencies in areas with a high percentage of children suffering from severe acute malnutrition.

Table 4 illustrates the number of days that traditional food products (grain-based rations) or traditional specialized food products (CSB) can be provided for the cost of providing a new specialized food product based on its suggested length of use. For example, for the cost of providing a nutritional supplementary paste fortified ration for 180 to 545 days, a grain-based ration could be provided for 741 to 3,121 days and a CSB ration could be provided for 183 to 769 days.

Appendix IV: USAID New Specialized Food Products Pending Introduction

USAID has thus far deployed a relatively limited quantity of new specialized food products but plans to introduce the following products to address various needs:

Ready-to-use supplementary food (RUSF) - Nutritionally dense and highly fortified for management of moderate acute malnutrition.

Fortified vegetable oil - Fortified with vitamins A and D.

Fortified milled cereals - Reformulated and standardized to improve general rations.

CSB++ (WFP Supercereal+) - Formulated by WFP for children 6 to 24 months of age.

CSB-14 – Reformulated CSB to be prepared with vegetable oil.

Appendix V: Comments from the U.S. Agency for International Development

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgment

In addition to the person named above, Joy Labez (Assistant Director), Carol Bray, Marc Castellano, Ming Chen, Anna Chung, Debbie Chung, Martin De Alteriis, Mark Dowling, Etana Finkler, David Schneider, and Jeremy Sebest made key contributions to this report. Sada Aksartova, Vida Awumey, Teresa Heger, Erin McLaughlin, Michael Maslowski, Julia Ann Roberts, Barbara Shields, and Phillip Thomas also contributed to this report.

Related GAO Products

World Food Program: Stronger Controls Needed in High-Risk Areas. GAO-12-790. Washington, D.C.: September 13, 2012.

Farm Bill: Issues to Consider for Reauthorization. GAO-12-338SP. Washington, D.C.: April 24, 2012.

International Food Assistance: Funding Development Projects through the Purchase, Shipment, and Sale of U.S. Commodities Is Inefficient and Can Cause Adverse Market Impacts. GAO-11-636. Washington, D.C.: June 23, 2011.

International Food Assistance: Better Nutrition and Quality Control Can Further Improve U.S. Food Aid. GAO-11-491. Washington, D.C.: May 12, 2011.

International School Feeding: USDA’s Oversight of the McGovern-Dole Food for Education Program Needs Improvement. GAO-11-544. Washington, D.C.: May 19, 2011.

International Food Assistance: A U.S. Governmentwide Strategy Could Accelerate Progress toward Global Food Security. GAO-10-212T. Washington, D.C.: October 29, 2009.

International Food Assistance: Key Issues for Congressional Oversight. GAO-09-977SP. Washington, D.C.: September 30, 2009.

International Food Assistance: USAID Is Taking Actions to Improve Monitoring and Evaluation of Nonemergency Food Aid, but Weaknesses in Planning Could Impede Efforts. GAO-09-980. Washington, D.C.: September 28, 2009.

International Food Assistance: Local and Regional Procurement Provides Opportunities to Enhance U.S. Food Aid, but Challenges May Constrain Its Implementation. GAO-09-757T. Washington, D.C.: June 4, 2009.

International Food Assistance: Local and Regional Procurement Can Enhance the Efficiency of U.S. Food Aid, but Challenges May Constrain Its Implementation. GAO-09-570. Washington, D.C.: May 29, 2009.

International Food Security: Insufficient Efforts by Host Governments and Donors Threaten Progress to Halve Hunger in Sub-Saharan Africa by 2015. GAO-08-680. Washington, D.C.: May 29, 2008.

Somalia: Several Challenges Limit U.S. International Stabilization, Humanitarian, and Development Efforts. GAO-08-351. Washington, D.C.: February 19, 2008.

Foreign Assistance: Various Challenges Limit the Efficiency and Effectiveness of U.S. Food Aid. GAO-07-905T. Washington, D.C.: May 24, 2007.

Foreign Assistance: Various Challenges Impede the Efficiency and Effectiveness of U.S. Food Aid. GAO-07-560. Washington, D.C.: April 13, 2007.

Foreign Assistance: U.S. Agencies Face Challenges to Improving the Efficiency and Effectiveness of Food Aid. GAO-07-616T. Washington, D.C.: March 21, 2007.