International Food Assistance: Improved Targeting Would Help Enable USAID to Reach Vulnerable Groups
Published: 2012-09-24T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In fiscal year 2011, USAID spent approximately $1.7 billion on food assistance reaching over 46 million people in 48 countries. USAID targets food assistance so that benefits accrue selectively to only a portion of the overall population, typically the most vulnerable. Effective targeting is important to maximize the impact of limited resources, especially as USAID begins to use more nutritious but more costly specialized food products to address hunger and malnutrition among vulnerable groups. GAO was asked to (1) describe in-country factors that USAID and its implementing partners face in targeting vulnerable groups, and (2) examine the extent to which USAID's targeting process supports effective targeting. GAO analyzed program data and documents; interviewed relevant officials; convened a roundtable of food assistance experts and practitioners; and conducted fieldwork in Ethiopia, Guatemala, Sri Lanka, and Zimbabwe.
section_title: What GAO Found
paragraphs: In-country, the U.S. Agency for International Development (USAID) and its implementing partners face a range of factors that, to varying degrees, affect their ability to target food assistance effectively to vulnerable groups. These factors include (1) the quality of data used to identify and reach recipients, (2) host government policies, and (3) sharing of rations among recipients and community members. Targeting effectiveness is reduced when data quality is poor, host government policies cause distortions in program design and implementation, and sharing prevents food rations from being consumed by the intended recipients in the intended amounts. USAID and its implementing partners try to mitigate such challenges by, for example, employing technology to improve data quality, coordinating closely with government officials to foster better relationships, and educating recipients about proper food usage to reduce sharing. In some cases, host governments have facilitated targeting efforts by, for example, establishing national targeting guidelines that set a common standard, or national statistical offices that assist in collecting data. Nevertheless, ensuring that food assistance reaches intended recipients remains difficult.
Weaknesses in the design, monitoring, and evaluation phases of USAID's targeting process hinder targeting effectiveness, although the agency is taking actions to make improvements. In the design phase of the targeting process, USAID does not provide sufficient guidance on whether and how to target specialized food products. Specifically, USAID's guidance on design currently is neither up-to-date nor complete, and does not adequately address key benefits and risks that inform decisions on whether and how to target specialized food products. In USAID's monitoring and evaluation phases, weaknesses limit targeting effectiveness and hinder decision making. USAID currently does not require monitoring of key indicators needed to determine the level of targeting effectiveness. For example, during implementation USAID does not monitor actual recipients in its emergency programs. Furthermore, its evaluations do not systematically address targeting effectiveness. Without adequate guidance, monitoring, and evaluations, USAID cannot ensure targeting effectiveness in its food assistance programs. USAID is taking some steps to improve both guidance and monitoring. For example, USAID is updating guidance and plans to track indicators such as detailed age breakdowns that are key to better understanding targeting effectiveness. However, these steps do not fully address the weaknesses in USAID's targeting process.
section_title: What GAO Recommends
paragraphs: GAO recommends that the Administrator of USAID improve USAID's targeting of specialized food products to vulnerable groups by (1) issuing, as appropriate, improved interim guidance to assist implementing partners in deciding whether and how to target specialized food products; and (2) establishing and reporting program-specific indicators related to targeted vulnerable groups, to assess effectiveness in reaching such groups. USAID agreed with the recommendations and provided examples of recent efforts to address them.

section_title: Letter
paragraphs: As the largest international food assistance donor, providing about half of  global food assistance, the United States plays an important role in  addressing hunger and malnutrition among vulnerable groups around the  world. Nearly 1 billion people suffer from undernourishment, which  contributes to more than one-third of child deaths globally, according to  United Nations (UN) sources. In fiscal year 2011, with funding authorized  under the Food for Peace Act, the U.S. Agency for International  Development (USAID) reported that it spent approximately $1.7 billion on  food assistance that reached over 46 million people in 48 countries. Of  this total, USAID spent approximately $1.2 billion on emergency  programs to help alleviate hunger and malnutrition in countries affected  by natural or man-made disasters, such as drought or conflict. USAID  programs provide three types of food assistance products: (1) traditional  food products, such as wheat, corn, and sorghum; (2) traditional  specialized food products, such as corn soy blend (CSB), a nutritionally  enhanced commodity; and (3) new specialized food products, such as  ready-to-use supplementary or therapeutic foods. Both traditional and  new specialized food products are intended for vulnerable groups.
USAID’s goal is to focus its food assistance efforts on the reduction of  hunger and malnutrition in vulnerable groups through targeting. As  defined by USAID, targeting is any method by which an intervention is  designed or implemented so that benefits can accrue selectively to only a  portion of the overall population. Targeting may be categorized by  geographic area or eligibility criteria, which are usually defined by  recipients’ characteristics, such as age, gender, income level, asset level,  or nutritional status. For the purposes of this report, we define the  effectiveness of targeting as the degree to which USAID and its  implementing partners are able to (1) accurately assess needs and  identify recipients using appropriate eligibility criteria, and (2) ensure that  the food assistance provided reaches and is consumed by the targeted  recipients as defined by the eligibility criteria. While targeting may never  be perfect, targeting error indicators can be used to assess effectiveness.  These indicators include the percentage of intended recipients that did  not receive food assistance or the percentage of people who were not  eligible for assistance but still received it.
Effective targeting is important, particularly in the context of constrained  resources, continuing humanitarian and food emergencies, and  increasing focus on addressing malnutrition in vulnerable populations.  USAID’s food assistance budget has declined from approximately $2.3  billion in appropriations in fiscal year 2009 to less than $2 billion in fiscal  year 2011. The demand for international food assistance remains high as  the threat of drought and famine persists in the Horn of Africa and other  regions. Moreover, USAID has begun to introduce, for limited use, some  new but more costly specialized food products designed to improve  nutritional outcomes for vulnerable groups, such as children under 2  years of age. However, as we previously reported, providing food that is  more nutritious but also more costly, within a fixed budget, would result in  fewer recipients fed.  you asked us to (1)  As part of our work on international food assistance, describe in-country factors that USAID and its implementing partners face  in targeting vulnerable groups, and (2) examine the extent to which  USAID’s targeting process supports effective targeting.
Our current work on international food assistance includes a recently issued report,  GAO, World Food Program: Stronger Controls Needed in High-Risk Areas, GAO-12-790  (Washington, D.C.: Sept. 13, 2012). In addition, we are conducting a review of the Feed  the Future initiative, the U.S. governmentwide strategy to address global hunger and food  security. The strategy was developed pursuant to a U.S. pledge made at the G-8 Summit  in L’Aquila, Italy, to provide at least $3.5 billion for agricultural development and global  food security over 3 years.
To address these objectives, we met with officials from USAID and its  implementing partners, including the UN World Food Program (WFP) and  nongovernmental organizations (NGO). We also met with academics,  experts, and practitioners associated with research institutes and  universities, as well as officials from the U.S. Departments of Agriculture  (USDA) and State (State). In addition, we reviewed USAID’s targeting  framework, including guidance, related to the food assistance targeting  process. We analyzed data from USAID and WFP to identify trends in  food assistance funding, the use of specialized food products, and the  costs of these products as compared with traditional food products.  Furthermore, we conducted fieldwork in four countries—Ethiopia,  Guatemala, Sri Lanka, and Zimbabwe—and met with officials from U.S.  missions, implementing partners, and relevant host government agencies.  Finally, we convened a roundtable of 10 experts and practitioners— including representatives from academia, research organizations, and  implementing partners such as WFP and NGOs—to further delineate, on  the basis of our initial work, in-country factors that affect targeting  vulnerable groups and the process that USAID and its implementing  partners use to target food assistance. Appendix I provides a detailed  discussion of our objectives, scope, and methodology.
We conducted this performance audit from October 2011 to September  2012 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: The United States Has Stated Its Commitment to Combating World Hunger and Malnutrition
paragraphs: Combating world hunger and malnutrition is a stated objective of the Food  for Peace Act, which authorizes international food assistance for  developing countries. The United States has also stated its commitment  to the Millennium Development Goal to halve world hunger by 2015, and  it supports the Scaling-Up Nutrition (SUN) movement to provide  assistance to country-led efforts to address maternal and child  malnutrition. To support SUN, the United States and others initiated the  1,000 Days public-private partnership, which aims to improve nutrition for  pregnant and lactating mothers and children under 2. Adequate nutrition  in this critical period in a child’s life is widely recognized to have the  greatest impact on saving lives, developing a child’s cognitive and  physical capacity, and mitigating the risk of chronic disease. According to  the USAID Policy Framework 2011-2015, USAID plans to ensure that the  quality of U.S. government food aid is improved within 3 years to meet the  nutritional requirements of vulnerable populations overseas, including by  developing new blended products and formulations to support pregnant  and lactating mothers and children under 2.
We previously reported that, although Title II emergency funding is intended to address  short-term food needs, more than half of the funding in fiscal year 2010 was spent on  multiyear emergency programs. See GAO-11-491. In 2011, the 14 countries that received  U.S. emergency food assistance every year from fiscal years 2006 through 2011 were  Afghanistan, Burundi, Central African Republic, Chad, Colombia, Democratic Republic of  the Congo, Ethiopia, Kenya, Nepal, Rwanda, Somalia, Sudan, Tanzania, and Uganda. In  addition, 23 percent of the emergency food commodities were delivered to 15 countries  that received U.S. food assistance for 3 to 5 years from fiscal years 2006 through 2011.  Three percent was delivered to four countries that received emergency U.S. food aid for 1  to 2 years. percent.$207 million, which accounted for about 17 percent of total Title II  emergency funding. A higher percentage of the total population in  Ethiopia suffers from malnutrition than in most other recipient countries,  with 51 percent of children under 5 suffering from stunting.
Ethiopia, one of the four countries we visited, received about   WFP is the largest provider of global food aid and  procurement. The countries that received the largest amounts of  specialized food products from WFP were Ethiopia, Pakistan, Kenya,  Niger, and Somalia.
Specialized food products are designed to meet specific nutritional needs  of vulnerable groups but are more costly than traditional food products.  As a result, within a fixed budget, USAID and its implementing partners  must decide whether to provide more nutritious but more costly food to  fewer people, or less nutritious and less costly food to more people. In  other words, they face a quality-quantity trade-off. Table 1 provides  illustrative examples of cost per ration for the three different types of food  assistance. See appendix III for a more detailed comparison of cost  differences between traditional food products and specialized food  products.
Targeting in food assistance programs is an iterative process that aims to  ensure that food reaches and is consumed by people whose  characteristics meet certain eligibility criteria, such as age, gender,  income level, asset level, or nutritional status. Figure 3 presents a  simplified schematic of the overall targeting process and its key phases— design, implementation and monitoring, and evaluation—and steps within  each phase. As key stakeholders in the targeting process, USAID and its  implementing partners, including WFP and NGOs, play an important role,  as do host governments. In the design phase, implementing partners  design food assistance programs and submit proposals to USAID. USAID  reviews the proposals and decides whether to fund the programs.  Feedback occurs within and across each of the phases—both in host  countries and at USAID headquarters—and is crucial to maximizing  targeting effectiveness, leading to steps within the process that may not  be strictly sequential. For example, during the design phase, USAID and  its implementing partners may conduct an assessment of needs to  determine the basis for the design of a program; however, as needs may  change or be clarified, they may retarget or make adjustments during the  monitoring phase to address issues that may arise.
section_title: Various In-Country Factors Affect USAID and Implementing Partners’ Ability to Target Food Assistance Effectively to Vulnerable Groups
paragraphs: USAID and its implementing partners face a range of in-country factors  that, to varying degrees, affect their ability to effectively target food  assistance to vulnerable groups. These factors include (1) the quality of  data used to identify and reach recipients, (2) host government policies,  and (3) sharing of rations among recipients and community members.  Targeting effectiveness is reduced when data quality is poor, host  government policies cause distortions in program design and  implementation, and sharing prevents food rations from being consumed  by the intended recipients in the intended amounts. USAID and its  implementing partners take steps to mitigate such challenges by, for  example, employing technology to improve data quality, coordinating  closely with government officials to foster better relationships, and  educating recipients about proper food usage to reduce sharing. In some  cases, host governments have facilitated targeting efforts by, for example,  establishing national targeting guidelines that set a common standard, or  national statistical offices that assist in collecting data. Nevertheless,  ensuring that food assistance reaches intended recipients remains  difficult.
subsections: section_title: Poor Data Quality May Hinder Implementing Partners’ Ability to Identify and Reach Intended Recipients
subsections: section_title: Lack of Reliable Population and Household Data May Hinder Targeting Effectiveness
paragraphs: Poor data quality—lack of timely and accurate information—may affect  implementing partners’ ability to effectively identify and reach recipients.  For example, in Zimbabwe, USAID and three implementing partners  noted that a lack of current and reliable population data made it difficult to  determine the overall number and geographic distribution of households   that are in need of food assistance.partner told us that because it used inaccurate data on average   In Guatemala, an implementing  household size to determine the initial ration size, people who were  initially identified received more food than they would have received if the  data had been accurate. Although the error was later corrected, if the  data had been accurate, resources could have been used more optimally  to reach people in need.
USAID and implementing partners we spoke with stated that sudden  natural disasters or conflicts could raise security concerns for  implementing partners, hindering their ability to reach the originally  targeted recipients. Furthermore, gathering reliable data on transient  populations is challenging. For example, USAID and an implementing  partner in Ethiopia told us that in some areas of the country, it is difficult  to determine the number and location of people in need of food  assistance, particularly pastoralists, who move often as a traditional way  of life and to cope with drought or natural disasters. As a result, it is  difficult for implementing partners to accurately assess the needs in a  particular geographic area and design an appropriate food assistance  program.
In addition, natural disasters or conflicts may raise security concerns,  hindering ability to reach targeted recipients. We recently reported that  security concerns prevented WFP from conducting field monitoring of  food distribution to determine whether the food rations reached the  originally targeted recipients in some high-risk areas of Ethiopia, Kenya,  and Somalia. For example, WFP noted that it has been unable to  access six districts in the Somali region of Ethiopia since May 2011. As a  result, WFP’s ability to collect data to ensure that the intended recipients  received their food assistance in these high-risk areas is limited.
USAID and implementing partners have taken some measures to improve  data quality by building capacity through technology, training, and other  activities. For example, USAID funds the Famine Early Warning Systems  Network (FEWS NET), which is used to monitor and prepare for changes  in food assistance needs. FEWS NET monitors and analyzes vulnerability  information, using multiple sources such as satellite imagery and field  observations. Moreover, some countries, such as Ethiopia, have  established national statistical offices that can assist in collecting data for  targeting food assistance. In addition, a 2011 report on food assistance  stated that implementing partners are working on increasing the speed,  accuracy, accessibility, and comparability of information. Implementing  partners in two countries we visited told us that they are using mobile  devices, such as tablets and phones, to collect recipient and distribution  data. The use of technology enables the implementing partners to better  identify and track recipients throughout the program and identify needs. In  the aforementioned example about excess ration size in a Guatemala  program, the implementing partner used tablets to collect information on  recipient consumption patterns. In this way, the implementing partner  ultimately discovered the ration error and corrected the ration size for  each household, freeing up resources to reach more recipients as a  result. Also, implementing partners in Guatemala and Sri Lanka indicated  that they train their staff and community volunteers on data collection, and  work with the host governments to improve the governments’ ability to  collect data. In addition, some countries, such as Sri Lanka, have  conducted repeated assessments of food assistance needs over several  years, which can lead to improvements in the precision of the data  collected.
Barrett, et al., 67. partner in Guatemala stated that data need to be improved continuously  to measure outcomes and impacts of targeting, particularly for programs  with a nutritional objective.
section_title: Host Government Policies May Cause Distortions, Limiting Targeting Effectiveness
paragraphs: Host government policies may lead to distortions, hampering targeting  effectiveness, but implementing partners have made some efforts to  reduce these adverse effects. We previously reported that one of the key  challenges to accurately assessing the needs of vulnerable groups was a  lack of coordination among key stakeholders—especially with host  governments—on assessments of food assistance needs. In addition,  some host country governments have been criticized for underestimating  actual needs or directing implementing partners to operate only in certain  geographic areas, due to political or other reasons. As a result,  implementing partners may not be able to reach recipients or locations  most in need of food assistance. For example, an implementing partner  in Ethiopia reported to USAID that the government of Ethiopia set an  artificial quota for the number of people targeted in each household that in  some cases did not reflect the actual needs, and severely hampered the  partner’s ability to reach vulnerable groups as a result. However, USAID  and implementing partner officials in Ethiopia also told us that working  with the government’s distorted figures is less challenging now than in the  past, due in part to recent efforts of local and regional government  officials to improve the validity and documentation of needs assessments  as well as better stakeholder coordination.
In some instances, however, host government policies may facilitate  targeting efforts. For example, the government of Sri Lanka has worked  closely with WFP to identify vulnerable groups and has supported efforts  to improve both data collection and the analysis of food needs, including  by supporting the research organization that partners with WFP in  conducting assessments of needs for food assistance. In another  example, the government of Ethiopia has published National Targeting  Guidelines that are intended to standardize and improve targeting  efforts.country operate under a commonly understood set of targeting policies  and practices.
This document helps all food assistance stakeholders in the  To address host government policies that cause distortions, implementing  partners undertake efforts to coordinate with stakeholders and verify  information on food assistance needs. Implementing partners we spoke  with told us they work with each other and with host governments in the  initial phase of the targeting process to increase transparency, in an effort  to encourage more accurate government estimates of actual needs. For  example, in Ethiopia, USAID officials told us that to increase  transparency, donors are working with the government to introduce  software tools and technology that facilitate access to information and  increase public awareness and thereby discourage government  authorities from manipulating data on food assistance needs. Moreover,  to help facilitate distribution of food assistance to intended recipients in  Guatemala, implementing partners stated that it is essential to closely  coordinate with government authorities at the beginning of the targeting  process to obtain approval for the use of new products and to set up the  appropriate distribution channels and protocol. In addition, in Sri Lanka,  an implementing partner told us that it plans to use local organizations to  conduct independent verification of the potential recipient list, which is  largely selected by the government. Doing so would help the  implementing partner ensure that only recipients who qualify for food  assistance are included on the list, increasing the likelihood that food  assistance reaches the intended recipients. Despite these efforts,  implementing partners have limited ability to influence host government  policies.
section_title: Sharing Reduces Targeting Effectiveness If Food Rations Are Not Consumed by the Intended Recipients in the Intended Amounts
paragraphs: Sharing within recipient households and among community members  may result in food rations being consumed by unintended recipients or in  unintended amounts, but implementing partners have taken some  measures to reduce sharing. External assessments suggest that sharing  of food rations is a widespread and established coping mechanism when  insufficient food is available. The 2011 Food Aid Quality Review (FAQR)  report and the 2011 WFP guidance on targeted food assistance  programs acknowledge that sharing of specialized food products is a  concern, and according to a 2011 USAID assessment of a food  assistance program in the Somali region of Ethiopia, sharing of food  rations is widespread. In addition, in countries we visited, USAID and its  implementing partners told us that both CSB and traditional food products  are routinely shared within and among households in some  communities—a finding we previously reported in 2011. The 2011  USAID assessment also notes that sharing is an established coping  mechanism for the recipient community when not everyone in the  community receives food rations. When food rations are shared, the  intended recipients may not consume the intended food products in the  desired amounts, which may reduce targeting effectiveness by limiting  nutritional impact, particularly for specialized food products that are  intended for vulnerable groups.
Implementing partners have made efforts to reduce the likelihood of  sharing, especially of specialized food products. Specifically,  implementing partners have employed various strategies to teach  recipients how to use specialized food products and have monitored  recipient food ration consumption. For example, one implementing  partner in Guatemala requires pregnant or lactating women to attend  education sessions, where they learn about the benefits of the specialized  food products and how to properly prepare them, before they can receive  rations. Implementing partners in Guatemala also print culturally relevant  instructional images on the food packages or the canvas bags given to  recipients to carry the rations. The images explain how to prepare the  food products and depict the type of person for whom the products are  intended—such as a pregnant woman or a child under 2 years of age.  One of these implementing partners reported that it had seen an  improvement in recipient participation in these education sessions and  expected that these sessions would reduce sharing.
In addition, implementing partners use community volunteers to monitor  effectiveness or consumption of food products. For example, in  Guatemala, implementing partners train “mother leaders”—mothers who  are also recipients—to provide training to other recipients on how to  prepare food and monitor outcomes by, for example, observing  improvement in a child’s weight or overall health appearance. In Sri  Lanka, another implementing partner uses health volunteers from the  recipient community and coordinates with the host government to ensure  that specialized food products are consumed by the children through  monthly monitoring of their nutritional status at government-run clinics  and weighing stations. The health volunteers also follow up with the  mothers of these children, who are receiving specialized food products, if  they do not bring their children to the monthly checkup. While  implementing partners have taken these and other steps to address  sharing, evidence of the impact of these steps has yet to be determined.
section_title: Weaknesses in the Design, Monitoring, and Evaluation Phases of USAID’s Targeting Process Hinder Targeting Effectiveness, Although Some Improvements Are Under Way
paragraphs: Weaknesses in the design, monitoring, and evaluation phases of USAID’s  targeting process hinder targeting effectiveness, although the agency is  taking actions to make improvements. In the design phase of the  targeting process, USAID does not provide sufficient guidance on  whether and how to target specialized food products. Specifically,  USAID’s guidance on design for both emergency and development  programs is neither up-to-date nor complete, and does not adequately  address key benefits and risks that inform decisions on whether to target  specialized food products. In both USAID’s monitoring and evaluation  phases, weaknesses limit targeting effectiveness and hinder decision  making. USAID currently does not require monitoring of key indicators  needed to determine the level of targeting effectiveness for either  emergency or development programs. Furthermore, its evaluations do not  systematically address targeting effectiveness. Without adequate  guidance, monitoring, and evaluations, USAID cannot ensure targeting  effectiveness in its food assistance programs. USAID is taking some  steps to improve both guidance and monitoring. For example, USAID has  a contract with Tufts University to develop updated guidance, and the  agency is taking steps to improve monitoring by planning to track  indicators such as detailed age breakdowns that are key to better  understanding targeting effectiveness. However, these steps do not fully  address the weaknesses in USAID’s targeting process.
subsections: section_title: USAID Does Not Provide Sufficient Guidance in the Design Phase on Whether and How to Target Specialized Food Products, Although the Agency Is Starting to Make Improvements
subsections: section_title: USAID’s Guidance for Targeting Is Neither Up-to-Date Nor Complete, Hindering Decision Making
paragraphs: We found that USAID’s guidance for targeting is neither up-to-date nor  complete for both emergency and development programs, which reduces  the ability of implementing partners to make informed decisions in the  design phase. USAID currently provides its implementing partners with a  range of guidance and tools. Of these, the Commodities Reference  Guide is USAID’s official standard reference for food assistance  programs and is intended to be used by USAID and implementing partner  staff in deciding how to plan, manage, control, evaluate, and use Title II- funded food products. It is available on USAID’s public website and  provides information on available food products, including nutritional  values, physical properties, and storage and handling guidelines.  However, USAID has not updated the Commodities Reference Guide  since 2006 and has not included guidance in the Commodities Reference  Guide on all of the products currently used in USAID food assistance  programs. The 2011 Food Aid Quality Review also noted that the  Commodities Reference Guide and other USAID guidance relevant to  targeting are neither up-to-date nor complete and recommended, for  example, that USAID improve its guidance to enable implementing  partners to better determine whether to use certain products for  programs.
We found that the lack of updated and complete guidance has hindered  implementing partners’ ability to make better-informed targeting  decisions. One participant at our roundtable, for example, told us that his  organization was unable to find all of the products it was using for a  program in the outdated Commodities Reference Guide. As a result, it  was not able to use these products in its program. Furthermore, USAID  has recently deployed some limited quantities of various new specialized  products without providing official standard guidance on how to use them.  We recommended in 2011 that USAID provide clear guidance on whether  and how best to use new specialized food products, including guidance to  its implementing partners on targeting strategies to ensure that the  products reach their intended recipients. USAID concurred with our recommendation and is taking steps to develop new guidance, but has  deployed new specialized food products in the interim. USAID has  purchased relatively small quantities of new specialized food products  over the past 2 years, including those shown in table 2 below. For  example, USAID purchased just over $6.5 million worth of these products  in 2011, as compared with $502 million of traditional food products and  $42 million of traditional specialized food products purchased through  Title II emergency program funding.
In addition, USAID is planning to introduce nine new or reformulated  products in the final part of 2012 and 2013, including new RUTFs and  ready-to-use supplementary foods (RUSFs) (see app. IV). USAID has not  issued fully updated or complete guidance for all of these products.  However, it has issued some guidance on their use. Moreover, USAID  officials told us that they are providing the products only on a limited basis  to organizations such as UNICEF that have experience using them in  controlled environments, such as clinics, and have issued their own  guidance on the use of these products.
USAID guidance inadequately addresses key benefits and risks of using  specialized food products, according to USAID and implementing  partners we spoke with during our field visits and our expert roundtable.  This inadequate guidance hinders decision making on whether to use  these products. As discussed earlier, the benefit of specialized food  products is that, while more costly, they are also more nutritious, or of  higher quality, than traditional food products. However, USAID has not  quantified or clearly defined the degree of benefit that specialized food  products may provide. In 2011, we reported that in recent years,  nutritionists have debated the appropriateness of using fortified and  blended foods to prevent and treat malnutrition in young children 6 to 24  months old, who have smaller stomachs, making it more difficult for them  to eat enough of the product to obtain sufficient nutrients. As a result,  the benefits of some traditional specialized food products are not clear. In  addition, limited information on new specialized products is available. As  we previously reported, USAID and implementing partners do not know  how well new specialized food products perform in promoting nutritional  health indicators, such as weight gain and growth, particularly in a  program setting, or how well they perform in comparison to traditional  food products.which these products promote desired outcomes, is still being studied by  USAID, WFP, nutritionists, and other researchers. As a result, while  USAID is building knowledge about these products, it is not providing  sufficient guidance on the benefits of specialized food products to  implementing partners.
The efficacy of new specialized products, or the extent to  USAID also lacks guidance on how to adequately address risks of using  specialized food products, according to implementing partners we spoke  with during our field visits and our expert roundtable. A key targeting risk  is that various factors implementing partners face in-country may reduce  targeting effectiveness to such a degree that the additional cost of using  specialized food products outweighs the potential benefit. This trade-off  becomes more significant with the higher cost of new specialized food  products, for which the cost per ration can be more than triple the cost of  traditional food products. Poor data quality, host government policies, and  sharing may reduce implementing partners’ ability to identify and reach  recipients, but USAID’s existing guidance does not adequately inform  decisions on whether the reduction in targeting effectiveness is of such a  degree that the use of specialized food products is no longer justified.
USAID is taking some steps to enhance guidance on whether and how to  use new specialized food products, but fully up-to-date and complete  guidance will not be completed until at least late 2013. In response to our  2011 recommendation on improved targeting guidance, USAID stated in  its official agency response in July 2011 that it would work to address our  recommendations through the second Food Aid Quality Review study  now under way with Tufts University. This work is expected to include  cost-effectiveness analyses on new specialized food products, adding  information important to help determine whether and how to use them. In  addition, according to USAID documents and officials, USAID is updating  and improving the Commodities Reference Guide and other guidance  related to targeting, including for new specialized food products.  However, this work will not be completed until September 2013 at the  earliest, according to USAID officials. USAID also plans to introduce other  guidance before September 2013, including updated fact sheets for  individual products. According to USAID officials, this interim guidance  will be released on an as-needed basis, beginning in October 2012. In  addition, USAID has existing guidance that helps inform implementing  partners’ decision making, including its Annual Program Statement (APS),  Food for Peace Information Bulletins, and some Food and Nutrition  Technical Assistance (FANTA) guidance.
section_title: Weaknesses in USAID’s Monitoring and Evaluation Efforts Related to Targeting Could Limit Targeting Effectiveness and Hinder Decision Making, but the Agency Is Starting to Take Steps to Improve Monitoring
subsections: section_title: USAID Does Not Monitor Key Indicators of Targeting Effectiveness, but Is Initiating Improvements in This Area
paragraphs: USAID does not require monitoring of key indicators needed to determine  the level of targeting effectiveness, although it is beginning to make  improvements in this area. Information on indicators that are consistent  with the goals of the program is critical to determining how effectively a  program targets food assistance. Targeting effectiveness can be  measured by the extent to which food assistance reaches correctly  targeted recipients—that is, the percentage of intended recipients that  actually receive food assistance in the intended amounts (see fig. 4).  Effectively targeted programs reduce the magnitude of these errors.  USAID guidance states that monitoring should be used to measure  progress toward planned program results. Additionally, FANTA  guidance states that monitoring efforts should allow USAID and its  implementing partners to assess the extent to which targeted recipients  received intended food assistance. According to a USAID official,  USAID field staff do consider targeting during their routine monitoring of  food assistance programs. In addition, USAID requires its implementing  partners to collect some data, such as the number of intended recipients  for all food assistance programs, and requires other indicators to be  monitored depending on the type of program—emergency or  development. However, USAID does not currently require sufficient  monitoring of key indicators consistent with program goals that would  allow its implementing partners to report on levels of targeting  effectiveness. For example, it cannot determine the effectiveness of a  program targeting children under 2 because it does not monitor the age of  the actual recipients in either emergency or development programs.
USAID monitoring is inadequate for both emergency and development  programs because it does not monitor key data on recipients that would  allow USAID to measure whether food assistance is actually reaching the  intended recipients. Specifically, for emergency programs, USAID collects  the total number of intended recipients from its implementing partners, but  does not collect the total number of actual recipients or indicators such as  breakdowns of age and gender for intended or actual recipients.  According to USAID, these types of more specific indicators may not be  as important for some emergency programs that focus solely on rapid  lifesaving. However, these indicators are important for emergency  programs that do have specific targeting goals, such as reaching severely  malnourished children. For development programs, USAID collects both  the total number of intended and actual recipients from its implementing  partners, but as with its monitoring of emergency programs, does not  collect data on key indicators such as breakdowns of age and gender.  Without monitoring full sets of data for both intended and actual  recipients, including key indicators consistent with program goals, USAID  has limited ability to learn about the magnitude of targeting errors or the  degree to which its implementing partners are achieving their program  goals.
According to USAID and implementing partner officials, it is particularly  complex to gather monitoring information on indicators related to targeting  effectiveness about actual recipients, due in part to cost and data quality  issues. These challenges are heightened for programs using new  specialized food products, which are designed to provide nutritional  benefits to very specific vulnerable groups, such as malnourished children  or pregnant or lactating women. Identifying and selecting recipients for  such programs requires using indicators that are more complex than  those used for programs designed for the general population. Some of  the indicators, such as nutritional status, are costly to measure and prone  to errors. For example, implementing partners we spoke with during our  fieldwork in Guatemala and Sri Lanka told us that they have difficulty in  collecting data for some indicators in other, non-USAID programs using  new specialized food products due to resource constraints, lack of  technical capacity by some local NGO staff, or problems with unreliable  data.
USAID is making improvements in monitoring of some nutrition-focused  development programs, for example, by planning to require implementing  partners to collect data on the age of young children, a common criterion  for new specialized food products. However, as mentioned earlier,  indicators key to measuring targeting effectiveness are not consistently  monitored across all USAID food assistance programs. According to the  Standards of Internal Control in the Federal Government, program  managers need to compare actual performance to planned results and  analyze significant differences. Without reporting targeting  effectiveness, USAID cannot compare actual targeting effectiveness to  planned results. As a result, USAID may not be able to make fully  informed targeting decisions for both ongoing and future food assistance  programs. For example, USAID may not be able to track the performance  of food assistance programs’ targeting over time or across programs and  may therefore miss opportunities to identify improvements to the targeting  effectiveness of these programs.
USAID’s targeting evaluations are not systematic, in part, because they are not routinely  conducted. USAID requires evaluations to be completed for all of its development  programs but does not require them for its emergency programs. Instead, emergency  programs are required to submit Annual Results Reports, which contain many of the same  types of information as evaluations, but for which no baseline assessment is conducted.  According to USAID officials, the difference between these requirements is due to the fact  that emergency programs are by nature typically in places where there may not be the  time or resources available to do a proper baseline assessment. evaluations that discussed targeting effectiveness included information on  the magnitude of inclusion or exclusion errors and the level of community  satisfaction with targeting. For example, USAID’s evaluation of an  emergency program in Zimbabwe discussed inclusion and exclusion  error, a key measure of targeting effectiveness, within a section focused  exclusively on targeting. Similarly, USAID’s evaluation of an emergency  program in Ethiopia mentioned the level of community satisfaction with  targeting: almost 90 percent of the respondents to a survey of community  members were generally satisfied with the fairness of the program’s  targeting. Some evaluations, however, contained only a brief mention of  targeting in general, with no mention of targeting effectiveness. For  example, an evaluation of a development program in Bolivia mentioned  targeting and contained tables showing monitoring indicators for the  baseline compared against the final evaluation, but did not explain how  the recipients were originally targeted or how the final evaluation results  were verified. Other evaluations, such as a 2011 evaluation of a  development program in Guatemala, did not discuss targeting or targeting  effectiveness at all.
USAID policy and guidance call on USAID and its implementing partners  to use evaluations as opportunities to learn about past programs to inform  decision making for new programs. USAID policy calls for evaluations to  “systematically generate knowledge about the magnitude and  determinants of program performance, permitting those who design and  implement programs…to refine designs and introduce improvements to  future efforts.” USAID guidance states that evaluations should assess  the extent to which the program is meeting its stated objectives. For  example, if a program is providing food assistance to a vulnerable  subpopulation, effective targeting is an important program objective.  However, USAID’s evaluations of its food assistance programs do not  systematically address targeting effectiveness, and as a result, the  agency’s ability to assess the extent to which a program is meeting its  stated objectives is hindered, and it may miss opportunities for learning  lessons that could be useful when designing new programs or improving  ongoing ones.
section_title: Conclusions
paragraphs: The use of specialized food products, especially some of those most  recently introduced, offers the promise of providing better nutrition to the  most vulnerable. However, the increased cost of these new specialized  products means that their use may likely reduce the overall number of  recipients receiving food assistance under a fixed program budget—a  quality-quantity trade-off. Choosing more costly specialized food products  over less costly traditional food products may be the optimal policy option  in certain circumstances, including areas with a high percentage of  children suffering from hunger and malnutrition. However, the  achievement of this policy goal requires effective targeting of food  assistance so that food ultimately reaches the intended recipients. If food  assistance is not targeted effectively, the program may fail to achieve its  nutritional goals while simultaneously feeding fewer people.
USAID recognizes the need to update and broaden its guidance on the  use of specialized food products, but this revision will not be completed  until late 2013 at the earliest. Issuance of improved interim guidance  related to food assistance targeting will help USAID and its implementing  partners make better-informed decisions about whether and how to  deploy the range of food products that are available, particularly new  specialized products. Moreover, the monitoring and reporting of key  indicators consistent with program objectives are necessary to ensure  that specialized food products are, in fact, reaching intended recipients.  Improved targeting—which takes an approach that is appropriate to the  circumstances and conditions—would better ensure that valuable food  resources are put to their most optimal use and that vulnerable groups  receive the most effective assistance available to them.
section_title: Recommendations for Executive Action
paragraphs: To improve USAID’s targeting of specialized food products to vulnerable  groups, such as children under 2 and pregnant women, we recommend  that the Administrator of USAID take the following two actions:
As USAID continues to purchase new specialized food products  without updated guidance, it should issue, as appropriate, improved  interim guidance to assist implementing partners in deciding whether  and how to target specialized food products.
When USAID chooses to provide specialized food products to  targeted vulnerable groups, it should establish and report program- specific indicators related to each targeted group to allow USAID to  assess its programs’ effectiveness in reaching these groups.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to USAID, USDA, and State for  comment. In its written comments, reproduced in appendix V, USAID  concurred with our recommendations. USDA and State provided no  written comments. We also provided relevant excerpts of this report to  WFP for comment. USAID, USDA, and WFP provided technical  comments that were incorporated, as appropriate.
USAID strongly agreed with our recommendation on improving interim  guidance to help implementing partners decide whether and how to target  specialized food products. USAID provided examples of recent and  ongoing efforts that are expected to contribute to improved guidance on  new specialized food products. For example, USAID expects to publish  on its website updated fact sheets on food products provided in its food  assistance programs and will prioritize issuing those relating to  specialized food products. Although USAID noted that some existing  guidance is available for three of the new specialized food products it is  introducing, such as CSB+, the agency also acknowledged that it expects  to issue its own guidance on all new products and update the  Commodities Reference Guide.
USAID agreed with our recommendation on establishing and reporting  program-specific indicators to allow USAID to assess its programs’  effectiveness in reaching targeted groups. USAID agreed with us on the  need to develop new, program-specific indicators to assess the nutrition  goals of new specialized food products for its Title II emergency programs  and indicated that it would engage with partners on demonstrating impact  and results. To that end, USAID indicated that it is in the process of  recruiting a nutritionist to ensure that products used match their intended  purpose and high-value specialized products are properly targeted.
We are sending copies of this report to appropriate congressional  committees, the Administrator of USAID, the Secretaries of Agriculture  and State, and relevant agency heads. The report is also available at no  charge on the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-9601 or melitot@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made major contributions to this report  are listed in appendix VI.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to (1) describe in-country factors that the U.S.  Agency for International Development (USAID) and its implementing  partners face in targeting vulnerable groups, and (2) examine the extent  to which USAID’s targeting process supports effective targeting.
To address these objectives, we met with officials at USAID and its  implementing partners, including the World Food Program (WFP) and  nongovernmental organizations (NGOs). In addition, we met with officials  at the U.S. Department of Agriculture (USDA) and the U.S. Department of  State. We also spoke with academics, experts, and practitioners  associated with research institutes, universities, and NGOs. We  examined USAID program documents, including guidance and tools,  related to food assistance targeting processes. Furthermore, we  conducted fieldwork in four countries—Ethiopia, Guatemala, Sri Lanka,  and Zimbabwe—and met with officials from U.S. missions, implementing  partners, and relevant host government agencies. We also convened a  roundtable of 10 practitioners and experts—including representatives  from implementing partners such as NGOs and WFP, academia, and  research organizations—to discuss in-country factors that affect the ability  of USAID and its partners to target vulnerable groups, as well as the  guidance and monitoring and evaluation tools that USAID and its  implementing partners use to target food assistance activities (see app. II  for the list of participating organizations in our roundtable).
To provide context and background, we analyzed data from USAID and  WFP to identify trends in U.S. funding for international food assistance  and procurement data on the use of traditional and specialized food  products. As these data were for background purposes, we did not  assess their reliability. In addition, we reviewed data that we reported on  in 2011 concerning cost information for specialized food products relative  to traditional food products. We then reviewed similar data to obtain  updates about the costs and relative length of feeding for these products  and interviewed USAID, WFP, and Tufts University about the reliability of  the updated data. We used this information to create an analysis  comparing the amount of time various commodities could be provided for  the cost of other commodities. We found that these updated data were  sufficiently reliable for the purposes of this report, in that they  demonstrated the order of magnitude of the relative cost of different types  of food products used in food assistance programs. Although commodity  prices may fluctuate and suggested feeding lengths may vary by program  or individual recipient, the data were sufficiently reliable to demonstrate  that there are large differences in the cost of feeding, depending on the  products used. In addition, we reviewed various literature on the targeting  process, as well as USAID guidance and tools to facilitate targeting  decisions.
To describe in-country factors USAID and its implementing partners face  in targeting vulnerable groups, we reviewed literature on targeting and  new specialized food products issued by academics, research institutes,  implementing partners, USAID contractors, UN organizations involved in  humanitarian assistance, and independent international organizations;  spoke with in-country officials such as relevant host government officials  and implementing partners; and obtained the input of our roundtable  participants.
To examine the extent to which USAID’s targeting process supports  effective targeting, we analyzed responses and information from the  general methodologies listed above. To examine the extent to which  USAID provides guidance to its implementing partners on targeting, we  reviewed existing USAID guidance for targeting and USAID’s contract  with Tufts University and spoke with USAID and Tufts University officials  about the scope of work for this contract, including the section on  updating guidance. We reviewed information from USAID about the  product types, costs, and tonnage of new specialized food products  purchased since fiscal year 2010. We interviewed USAID about the  sources of this information and also compared it to data about these  products from other sources of information. These included the requests  for applications that USAID provides to its implementing partners for new  specialized food products, including ready-to-use therapeutic foods and  lipid nutritional supplements; USAID’s commodity price calculator; and  relevant legislation authorizing the use of these products. We found that  the data were sufficiently reliable for the purposes of this report, in that  they showed the magnitude and trends of purchases of new specialized  food products with USAID funding in recent years. To examine the extent  to which USAID monitors and evaluates targeting effectiveness and other  related information, we analyzed monitoring information provided to us by  USAID about numbers of planned recipients and actual beneficiaries for  Title II food assistance programs since 2009. We reviewed current USAID  policies and procedures on monitoring and evaluation. We also reviewed  guidance provided by Food and Nutrition Technical Assistance (FANTA),  under a cooperative agreement with USAID. This guidance covers  aspects of monitoring and evaluation, such as the performance measures  to be used for food assistance programs. Through searches of USAID’s  website and discussions with cognizant officials, we identified a total of 30  final evaluations of USAID programs going back to 2009. Final  evaluations are conducted at the end of a program. However, USAID  could not assure us that it had provided all of the evaluations conducted  for its development programs, and also noted that it does not require  evaluations of its emergency programs. We selected 20 of these final  evaluations for review based on the following criteria: we included all 3  final evaluations for the single-year programs, and selected 17 final  evaluations for the multi-year programs to ensure that we had coverage  by year and geographic region. We reviewed these evaluations to  examine the extent to which they had addressed targeting issues. Finally,  we reviewed evaluations that WFP conducted of those of its programs  that were implemented with USAID funding.
We conducted this performance audit from October 2011 to September  2012 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Organizations That Participated in GAO Roundtable
section_title: Appendix III: Cost Differences between Traditional Food Products and Specialized Food Products
paragraphs: Table 3 provides a comparison of cost differences between traditional and  selected specialized food products, which include both traditional  specialized food products and new specialized food products. For  example, to feed a child 6 to 23 months old, a traditional grain-based  representative ration costs $0.02 to $0.06 per day, a CSB+ ration, $0.10  to $0.21 per day, and a ready-to-use therapeutic food (RUTF) ration,  $0.42 to $0.46 per day.
The cost per ration is one aspect of the relative cost of food assistance  products; the length of time a product is used in a food assistance  program also affects its overall relative cost. Some experts suggest that,  although an individual daily ration of a new specialized food product may  be relatively expensive, it may ultimately be less costly overall because it  may be fed for a shorter period of time based on its suggested length of  use. However, we found that some new specialized food products with a  relatively shorter suggested length of use may still cost relatively more  overall. For example, as shown in the table, although an RUTF’s  suggested length of use (42 to 90 days) for a child 6 to 23 months old is  shorter than that of a traditional grain-based ration (90 to 120 days), the  RUTF may still cost more overall ($17.77 to $41.40) than the grain-based  ration ($2.07 to $7.45). Despite their higher overall costs, RUTFs may be  the optimal choice in certain circumstances, such as for emergencies in  areas with a high percentage of children suffering from severe acute  malnutrition.
Table 4 illustrates the number of days that traditional food products  (grain-based rations) or traditional specialized food products (CSB) can  be provided for the cost of providing a new specialized food product  based on its suggested length of use. For example, for the cost of  providing a nutritional supplementary paste fortified ration for 180 to 545  days, a grain-based ration could be provided for 741 to 3,121 days and a  CSB ration could be provided for 183 to 769 days.
section_title: Appendix IV: USAID New Specialized Food Products Pending Introduction
paragraphs: USAID has thus far deployed a relatively limited quantity of new  specialized food products but plans to introduce the following products to  address various needs:
Ready-to-use supplementary food (RUSF) - Nutritionally dense and  highly fortified for management of moderate acute malnutrition.
Fortified vegetable oil - Fortified with vitamins A and D.
Fortified milled cereals - Reformulated and standardized to improve  general rations.
CSB++ (WFP Supercereal+) - Formulated by WFP for children 6 to 24  months of age.
CSB-14 – Reformulated CSB to be prepared with vegetable oil.
section_title: Appendix V: Comments from the U.S. Agency for International Development
section_title: Appendix VI: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
subsections: section_title: Staff Acknowledgment
paragraphs: In addition to the person named above, Joy Labez (Assistant Director),  Carol Bray, Marc Castellano, Ming Chen, Anna Chung, Debbie Chung,  Martin De Alteriis, Mark Dowling, Etana Finkler, David Schneider, and  Jeremy Sebest made key contributions to this report. Sada Aksartova,  Vida Awumey, Teresa Heger, Erin McLaughlin, Michael Maslowski, Julia  Ann Roberts, Barbara Shields, and Phillip Thomas also contributed to this  report.
section_title: Related GAO Products
paragraphs: World Food Program: Stronger Controls Needed in High-Risk Areas.  GAO-12-790. Washington, D.C.: September 13, 2012.
Farm Bill: Issues to Consider for Reauthorization. GAO-12-338SP.  Washington, D.C.: April 24, 2012.
International Food Assistance: Funding Development Projects through  the Purchase, Shipment, and Sale of U.S. Commodities Is Inefficient and  Can Cause Adverse Market Impacts. GAO-11-636. Washington, D.C.:  June 23, 2011.
International Food Assistance: Better Nutrition and Quality Control Can  Further Improve U.S. Food Aid. GAO-11-491. Washington, D.C.: May 12,  2011.
International School Feeding: USDA’s Oversight of the McGovern-Dole  Food for Education Program Needs Improvement. GAO-11-544.  Washington, D.C.: May 19, 2011.
International Food Assistance: A U.S. Governmentwide Strategy Could  Accelerate Progress toward Global Food Security. GAO-10-212T.  Washington, D.C.: October 29, 2009.
International Food Assistance: Key Issues for Congressional Oversight.  GAO-09-977SP. Washington, D.C.: September 30, 2009.
International Food Assistance: USAID Is Taking Actions to Improve  Monitoring and Evaluation of Nonemergency Food Aid, but Weaknesses  in Planning Could Impede Efforts. GAO-09-980. Washington, D.C.:  September 28, 2009.
International Food Assistance: Local and Regional Procurement Provides  Opportunities to Enhance U.S. Food Aid, but Challenges May Constrain  Its Implementation. GAO-09-757T. Washington, D.C.: June 4, 2009.
International Food Assistance: Local and Regional Procurement Can  Enhance the Efficiency of U.S. Food Aid, but Challenges May Constrain  Its Implementation. GAO-09-570. Washington, D.C.: May 29, 2009.
International Food Security: Insufficient Efforts by Host Governments and  Donors Threaten Progress to Halve Hunger in Sub-Saharan Africa by  2015. GAO-08-680. Washington, D.C.: May 29, 2008.
Somalia: Several Challenges Limit U.S. International Stabilization,  Humanitarian, and Development Efforts. GAO-08-351. Washington, D.C.:  February 19, 2008.
Foreign Assistance: Various Challenges Limit the Efficiency and  Effectiveness of U.S. Food Aid. GAO-07-905T. Washington, D.C.: May  24, 2007.
Foreign Assistance: Various Challenges Impede the Efficiency and  Effectiveness of U.S. Food Aid. GAO-07-560. Washington, D.C.: April 13,  2007.
Foreign Assistance: U.S. Agencies Face Challenges to Improving the  Efficiency and Effectiveness of Food Aid. GAO-07-616T. Washington,  D.C.: March 21, 2007.