Afghanistan Development: Enhancements to Performance Management and Evaluation Efforts Could Improve USAID's Agricultural Programs
Released: 2010-07-14T00:00:00

Letter

Background

Afghanistan is one of the world’s poorest countries and ranks near the bottom of virtually every development indicator category, including life expectancy; literacy; nutrition; and infant, child, and maternal mortality. According to the most recent National Risk and Vulnerability Assessment conducted by the government of Afghanistan between 2007 and 2008, the Afghan poverty rate was 36 percent. The highest rates of poverty were among nomads and rural farmers and varied across regions and provinces. (For additional information on regional poverty in Afghanistan see GAO-10- 756SP.) The survey also found that agricultural activities provided the Afghan population’s primary livelihood; 55 percent of households were engaged in farming and 68 percent had livestock. According to the World Bank, the agricultural sector accounts for 30 percent of Afghanistan’s gross domestic product. The National Risk and Vulnerability Assessment states that agricultural productivity is hampered by water shortage, lack of credit, insufficient outreach of agricultural and veterinary extension services, and poor access to markets.

Afghanistan suffers from limited means to capture water resources, soil degradation, overgrazing, deforestation, and desertification. As shown in figure 1, Afghanistan is mountainous and much of its land is not arable. According to the National Risk and Vulnerability Assessment, household access to arable land increased between 2005 and 2007/2008, largely due to increasing access to irrigated land across urban, rural, and nomadic households. Additionally, in 2007/2008, 40 percent of households nationwide had access to irrigated land and 17 percent had access to rain- fed land. Farms in Afghanistan averaged 1.4 hectares for irrigated land and 2.8 hectares for rain-fed land. The survey also found that wheat was the most frequently cited crop produced on irrigated and rain-fed land during the summer planting season, followed by opium and potatoes on irrigated land and cotton and barley on rain-fed land; corn, sorghum, and rice were grown on irrigated land during the winter planting season. Some households also grew fruit and nut trees and grapes. (For additional information on major crops grown in Afghanistan see GAO-10-756SP.)

USAID Funds Agricultural Programs in Afghanistan

As figure 2 shows, between fiscal years 2002 and March 31, 2010, USAID awarded about $1.4 billion to 41 agricultural-assistance programs in Afghanistan, with almost two-thirds of the amount (about $900 million) disbursed.

As table 1 shows, disbursements of U.S. funds for agricultural programs represented 14 percent of all USAID assistance programs in Afghanistan from fiscal years 2002-March 31, 2010. Moreover, the percentage of USAID’s total assistance to Afghanistan disbursed to agricultural programs has increased from 6 percent in fiscal years 2002-2004 to 17 percent in fiscal years 2008 and 2009. Appendix II has more funding information on USAID’s agricultural programs, including the eight agricultural programs in our review.

The administration requested $827 million dollars in fiscal year 2010 for USAID agricultural-assistance programs.

U.S. Agricultural Strategy Changing Focus to Support Counterinsurgency Priorities

In fiscal years 2002–2003, to help address the complex humanitarian crisis in Afghanistan, the U.S. government provided emergency assistance that helped avert a famine, significantly reduced the suffering of the most vulnerable Afghans, and assisted the return of refugees. USAID provided Afghanistan with 355,270 metric tons of wheat and other emergency food assistance (valued at $206.4 million), and the U.S. Department of Agriculture provided 79,600 metric tons of surplus wheat (valued at $38.7 million). Over the 2-year period, the United States provided over 60 percent of all international food assistance received by Afghanistan. According to the World Food Program, the food assistance provided by the United States and the international community helped avert famine in Afghanistan.

As we previously reported, from 2002 through 2004, increased opium poppy cultivation spread and drug trafficking grew as a threat to Afghanistan’s security and stability. During that time, the United States supported Afghan- and United Kingdom-led counternarcotics efforts. These efforts reportedly had little effect on the illicit narcotics industry because of limited security and stability across Afghanistan. In response, the U.S. government made counternarcotics a top U.S. priority and developed a strategy in 2004 to reduce poppy cultivation, drug production, and trafficking, shifting the emphasis of the United States’ agricultural assistance programs in Afghanistan from food security programs to counternarcotics-related ADP. This part of the U.S. counternarcotics strategy was intended to offer incentives to stop opium poppy production by helping farmers and farm laborers obtain other ways to earn a living. The strategy also called for strong disincentives such as forced eradication, interdiction, and law enforcement, while at the same time spreading the Afghan government’s antinarcotics message. The United States’ efforts also were expected to build the Afghan government’s capacity to conduct counternarcotics efforts on its own. As part of its counternarcotics efforts, beginning in 2005, USAID awarded most of its new agricultural funds to alternative-development programs (ADP)—to (1) increase agricultural productivity, (2) accelerate economic growth, and (3) eliminate illicit drug cultivation. As figure 3 shows, between 2005 and 2008, $494 million, or 71 percent of new awards were for ADP.

During this time period, USAID funded ADP, as well as other large agricultural programs. Figure 4 provides a brief description of the goals and objectives of the eight programs included in this review including, beginning in 2005, ADP-Northeast, ADP-South, and ADP-East; beginning in 2006, the Accelerating Sustainable Agriculture Program; beginning in 2008, ADP-Southwest, the Afghanistan Water, Agriculture, and Technology Transfer program, and the Afghanistan Vouchers for Increased Production in Agriculture program; and beginning in 2009, the Incentives Driving Economic Alternatives-North, East and West, as a follow-on program to ADP-East. USAID identified six of the eight programs as ADP, excluding the Accelerating Sustainable Agriculture Program and the Afghanistan Vouchers for Increased Production in Agriculture. Appendix III specifies the provinces where the eight agricultural programs included in our review operated and gives examples of the range of projects that these programs implemented.

In 2009, under the direction of the President and the Special Representative for Afghanistan and Pakistan, the United States shifted the focus of its agricultural strategy in Afghanistan from counternarcotics to counterinsurgency efforts. This shift de-emphasized eradication. According to the Special Representative for Afghanistan and Pakistan, eradication unduly punished and alienated farmers for making a “rational economic decision,” while ignoring the profits gleaned by traffickers and insurgents from the sale of processed opium and heroin. Further, the Administration noted that economic growth and new job creation were critical to U.S. counterinsurgency efforts in Afghanistan because they provide licit alternatives to narcotics- and insurgent-related activities and connect people to their government. As a result, the Administration integrated agricultural programs with other U.S. efforts, including military operations, and directed more resources to the agricultural sector. For example, the Afghanistan Vouchers for Increased Production in Agriculture program, which originally operated outside of the southeastern portion of Afghanistan, was expanded to Helmand and Kandahar in 2010, where according to Department of Defense officials, the United States and Afghanistan have begun military operations to break the momentum of the insurgency. Furthermore, the Administration increased the involvement of the U.S. Department of Agriculture and aligned U.S. efforts with the current agricultural priorities of the Afghan government, as laid out in the Ministry of Agriculture, Irrigation, and Livestock’s National Agriculture Development Framework. The strategy focuses on the following four areas: 1. increasing agricultural productivity by increasing farmers’ access to quality inputs, such as improved seeds and fertilizer, and effective extension services; 2. regenerating agribusiness by increasing linkages between farmers, markets, credit, and trade corridors; 3. rehabilitating watersheds and improving irrigation structure; and 4. increasing the Ministry of Agriculture, Irrigation, and Livestock’s capacity to deliver services and promote the private sector and farmer associations through direct budget and technical assistance.

Gaps Exist in USAID’s Efforts to Plan, Monitor, and Evaluate Agricultural Programs in Afghanistan

USAID’s Automated Directives System establishes performance management and evaluation procedures that USAID expects its staff to follow in planning, monitoring, and evaluating its agricultural assistance programs in Afghanistan. USAID operationalized these procedures through the development of a Mission Performance Management Plan (PMP). Similarly, USAID requires its implementing partners to develop monitoring and evaluation plans, which are generally included in implementing partners’ program PMPs. The collection of planning, monitoring, and evaluating efforts, when taken together, enable USAID to manage the performance of its programs. While USAID has noted that Afghanistan is an insecure environment in which to implement its programs, the agency has generally required the same performance management and evaluation procedures as it does in other countries in which it operates. In October 2008, USAID approved new guidance that outlined several alternative monitoring methods—especially when site visits are difficult or not possible—in high threat environments such as Afghanistan. This guidance, however, was not disseminated to USAID staff until December 2009, and the USAID Mission to Afghanistan agricultural staff did not become aware of the guidance until June 2010. This guidance was included in GAO’s review where applicable. Figure 5 presents a summary of USAID’s Automated Directives System’s performance management and evaluation procedures it expects its staff to follow, grouped into planning, monitoring, and evaluating categories.

USAID’s Planning for Agricultural Programs in Afghanistan Generally Included Key Activities, but Lacks a Current Mission Performance Management Plan

USAID’s Automated Directives System requires USAID officials to complete a Mission PMP for each of its high-level objectives as a tool to manage its performance management and evaluation procedures. In line with this requirement, USAID’s Mission to Afghanistan developed its first PMP in 2006; the document covered the years 2006, 2007, and 2008. The Mission operated without a PMP to guide its efforts after 2008. According to USAID, the Mission is in the process of developing a new missionwide PMP that will reflect the current Administration’s priorities and strategic shift to counterinsurgency. USAID expects the new PMP to be completed by the end of fiscal year 2010. The Mission attributed the delay in creating the new PMP to the process of developing new strategies in different sectors and gaining approval from the U.S. Embassy in Afghanistan and from agency headquarters in Washington.

Overall, the 2006-2008 Mission PMP incorporated key planning activities. For example, the PMP identified indicators, established baselines and targets, planned for data quality assessments, and described the frequency of data collection for four high-level objectives for all USAID programs in Afghanistan, including its agricultural programs. The eight agricultural programs we reviewed all fell under one of these four high-level objectives identified in the Mission PMP—“developing a thriving licit economy led by the private sector.” In addition, the PMP described regular site visits, random data checks, and data quality assessments as the means to be used to verify and validate information collected. Furthermore, the Mission PMP noted that the PMP was developed to enable staff to actively and systematically assess their contributions to the Mission’s program results and take corrective action when necessary. It noted that indicators, when analyzed in combination with other information, provide data for program decision making. The Mission PMP did not include plans for evaluations and special studies for the high-level objective that the eight programs included in this review supported; but according to USAID, the agency has planned evaluations for seven of the eight agricultural programs included in this review during fiscal year 2010. In addition, USAID has planned to conduct evaluations of agricultural depots and veterinarian field units, activities included in several agricultural programs.

Similar to the Automated Directives System’s requirement that USAID develop a Mission PMP as a planning tool to manage the process of monitoring and evaluating progress—including establishing targets for each performance indicator—implementing partners are required to develop and submit monitoring and evaluation plans to USAID for approval. To keep its performance-management system cost-effective, reduce its burden, and ensure implementing partner activities and USAID plans are well-aligned, USAID requires its implementing partners to integrate performance-data collection in their performance-management activities and work plans. In fulfilling this requirement, the implementing partners submitted monitoring and evaluation plans for the eight programs included in our review to USAID for approval. The implementing partners’ plans, among other things, generally contained goals and objectives, indicators, and targets. However, we found that USAID had not always approved these plans and did not require implementing partners to set targets for each of their indicators, which are needed to assess program performance. Figure 6 shows the number of performance indicators by fiscal year with targets that the implementing partner developed and submitted to USAID for approval. The number of indicators with targets varied over time.

The three programs we reviewed that were active in 2005, identified indicators and, in some cases, targets in their monitoring and evaluation plans to track progress; however, according to implementing partners, USAID did not approve these plans in 2005. Implementing partners for the eight programs we reviewed were contractually required to submit monitoring and evaluation plans. According to implementing partners, USAID was developing a common set of indicators for all three programs to track. In 2006, all three programs were requested to revise their monitoring and evaluation plans and develop PMPs that included a common set of performance indicators. All three programs submitted revised plans in their 2006 PMPs, however, USAID subsequently approved only two out of the three program PMPs. ADP-South’s PMP was never formally approved during the life of the program. USAID officials were unable to explain or provide reasons for the lack of approval. In addition, while ADP-South and ADP-East were intended to end early in fiscal year 2009, when their contracts were extended into fiscal year 2009, these programs were not required to set targets for all of their indicators during the additional time frame. The USAID officials we spoke with were uncertain as to why their predecessors did not require this of the implementing partners. In addition, several of the other programs, such as the Afghanistan Water, Agriculture, and Technology Transfer and the Afghanistan Vouchers for Increased Production in Agriculture programs did not establish targets for all of their indicators. As a result, in fiscal year 2009, out of the seven active agricultural programs we reviewed, two had set targets for all of their indicators.

USAID Undertook Efforts to Monitor Agricultural Programs, but Did Not Fully Analyze and Interpret Performance Data

According to USAID’s Automated Directives System, monitoring efforts should include, among other things, collecting performance data, assessing data quality, identifying limitations, and taking steps to mitigate data limitations. USAID regularly collected program reports containing performance data from implementing partners for the eight programs we reviewed and assessed data quality, as well as mitigated data limitations, by conducting site visits when feasible, regularly communicating with implementing partners, and completing a data quality assessment for performance data. USAID assigned a monitoring official—known as an agreement or contracting officer’s technical representative—to oversee implementing partners’ activities for each of the eight agricultural programs we reviewed. Monitoring officials identified quarterly reports submitted by implementing partners as key documents used to collect performance data. To assess data quality and make efforts to mitigate data limitations, USAID conducted site visits and documented these efforts by completing monitoring reports, progress reports, and trip reports.

According to USAID, Afghanistan’s insecure environment limited the frequency of some site visits and monitoring officials’ ability to consistently verify reported data. As such, the frequency of site visits varied within and across programs. Moreover, also according to USAID, formal site visit reports are seldom completed. As a result of time constraints, documentation of site visits is often limited to photographic documentation combined with informal emails from staff participating in site visits. In 2009, USAID conducted site visits for two of the eight programs included in our review.

In 2008 and 2009, the USAID Mission director cited USAID’s efforts to monitor project implementation in Afghanistan as a significant deficiency, in the Mission’s Federal Management Financial Integrity Act of 1982 Annual Certification. These assessments raised concerns that designated USAID staff are “prevented from monitoring project implementation in an adequate manner with the frequency required” and noted that there is a high degree of potential for fraud, waste, and mismanagement of Mission resources. USAID further noted that the deficiency in USAID’s efforts to monitor projects will remain unresolved until the security situation in Afghanistan improves and stabilizes.

USAID identified several actions to address the limitations to monitoring project implementation, these include placement of more staff in the field to improve monitoring capacity, use of hired security services to provide protection to Mission staff traveling to project sites, use of provincial reconstruction team staff to obtain information on the progress of USAID–funded activities where the provincial reconstruction teams operate, use of more Afghan staff, who have greater mobility than expatriate staff, to monitor projects, hiring of a contractor to monitor the implementation of construction projects and conduct regular site visits, use of Google Earth geospatial mapping to substitute for site visits, frequent and regular communication with implementing partners, collection of implementing partner videos or photographs—including spot checks of implementing partner records or files, and feedback from Afghan ministries and local officials.

USAID Conducted Data Quality Assessments

USAID performance management procedures require that Mission performance data reported to Washington for Government Performance and Results Act (GPRA) reporting purposes or for reporting externally on Agency performance must have had a data quality assessment at some time within the 3 years before submission. USAID established 10 Mission agricultural indicators that it reports to the joint Department of State- USAID’s Foreign Assistance Coordination and Tracking System. As required, USAID completed a data quality assessment for all 10 Mission agricultural indicators in November 2008. As table 3 shows, USAID’s data quality assessments generally provided high, medium, or low rankings of quality for the data collected.

USAID’s assessments also identified actions to mitigate weaknesses in data quality.

Suggestions for improving data quality included clarifying definitions and qualifying activities, increasing the frequency of direct monitoring, and increasing the effort to understand the impact of activities.

USAID Did Not Consistently Analyze and Interpret Performance Data

In addition to collecting performance data and assessing the data’s quality, USAID’s Automated Directives System also includes the monitoring activities of analyzing and interpreting performance data in order to make program adjustments, inform higher-level decision making and resource allocation. We found that while USAID collects implementing partner performance data, or information on targets and results, the agency did not fully analyze and interpret this performance data for the eight programs in our review. Some USAID officials in Afghanistan told us that they reviewed the information reported in implementing partners’ quarterly reports in efforts to analyze and interpret a program’s performance for the eight programs in our review, although they could not provide any documentation of their efforts to analyze and interpret program performance. Some USAID officials also said that they did not have the time to fully review the reports. As a result, it is unclear the extent to which USAID uses performance data to make program adjustments, inform higher-level decision making and resource allocation.

As previously noted, efforts to monitor program performance as outlined in USAID’s Automated Directives System should include and document decisions on how performance data will be used to guide higher-level decision making and resource allocation. Additionally, USAID is required to report results to advance organizational learning and demonstrate USAID’s contribution to overall U.S. government foreign assistance goals. While USAID did not fully analyze and interpret program data, the Mission does meet semiannually to examine and document strategic issues and determine whether the results of USAID-supported agricultural activities are contributing to progress toward high-level objectives. With respect to reporting of results, the Mission reported aggregate results in the Foreign Assistance Coordination and Tracking System—discussed earlier.

Limited Evaluations Completed for Agricultural Programs GAO Reviewed

USAID’s Automated Directives System requires USAID to undertake at least one evaluation for each of its high-level objectives; to disseminate the findings of evaluations; and to use evaluation findings to further institutional learning, inform current programs, and shape future planning.

As noted earlier, each of the eight agricultural programs included in this review support the high-level objective of “developing a thriving licit economy led by the private sector.” In May 2007, USAID initiated an evaluation covering three of the eight agricultural programs included in our review—ADP-Northeast, ADP-East, and ADP-South. This evaluation was intended to assess the progress of the alternative-development initiatives toward achieving program objectives and offer recommendations for the coming years. The evaluators found insufficient data to evaluate whether the programs were meeting objectives and targets and, thus, shifted their methodology to a qualitative review based on interviews and discussions with key individuals. As required under USAID’s evaluation requirements, USAID posted the evaluation to its Web site for dissemination. In addition, as noted earlier, USAID is planning to conduct evaluations in fiscal year 2010 for all but one of the agricultural programs included in this review.

We are uncertain of the extent to which USAID used the 2007 evaluation to adapt current programs and plan future programs. Few staff were able to discuss the evaluation’s findings and recommendations and most noted that they were not present when the evaluation of the three ADP programs was completed and, therefore, were not aware of the extent to which changes were made to the programs. With regard to using lessons learned to plan future programs, USAID officials told us that, in planning for the Afghanistan Vouchers for Increased Production in Agriculture program, key donors met with USAID staff and the Afghan government to share ideas and lessons learned from other programs. However, the officials could not provide documentation of this discussion or examples of how programs were modified as a result of the discussion.

Several Agricultural Programs Did Not Achieve or Establish Targets

Based on our assessment of USAID implementing partner data, we found that six of the eight agricultural programs we reviewed fell short of achieving their established targets for several of their performance indicators. Additionally, although USAID requires implementing partners to submit information on indicators, targets, and results, as previously noted, not all indicators had established targets to allow for performance assessments. As figure 8 shows, six of the eight programs we reviewed did not meet their performance targets in the most recent year for which information was reported on performance targets. For the two programs that met all their targets, we found, as previously discussed, that they did not establish targets for several indicators and, thus, we could not fully assess performance for those indicators. We also found that the three longest-running programs in our review showed declines in performance over time. We measured performance for the eight agricultural programs in our review by comparing annual results against annual targets reported by USAID’s implementing partners. We assessed the extent to which targets were fully met. We decided that this measure of performance was appropriate because implementing partners are allowed to adjust and revise target levels to better reflect available information in the field. Our analysis is detailed in appendix IV.

With respect to the three longest-running agricultural programs in our review—ADP-Northeast, ADP-South, and ADP-East—we found that the number of indicators that met or exceeded annual targets generally declined from 2006 to 2008. For example, ADP-Northeast met 33 percent of its targets in fiscal year 2006 and 29 percent of its targets in fiscal year 2008. While ADP-Northeast showed improvements in the percentage of indicators that met targets between fiscal years 2006 and 2007, the percentage declined in 2008. Similarly, ADP-South met 79 percent of its targets in fiscal year 2006, but 50 percent of its targets in fiscal year 2008. Although ADP-South showed substantial improvements in fiscal year 2009, the performance assessment was based on 5 out of 25 indicators that had set targets (see fig. 6 earlier in the report); the remaining 20 indicators showed results, but did not have annual targets. The Mission noted that these declines coincided with declines in the security environment; however, the Mission acknowledged that it had not conducted any analysis to confirm that the security environment was the reason for the declines in performance. From 2006 to 2008, the percentage of targets met declined for indicators, such as the number full-time equivalent jobs; Afghans trained in business skills; and hectares of improved irrigation as a result of infrastructure works. Appendix V has details on the indicators, targets, and results for the latest year where performance data was available for each of the eight programs in our review. Based on our assessment, on average, the percentage of targets met declined from 2006 to 2008 across these three programs.

The longest-running program in our review that is currently active, the Accelerating Sustainable Agricultural Program, showed improvements in fiscal year 2008, but declined in fiscal year 2009 in the number of targets met. For example, the program met 0 percent of its targets in fiscal year 2007, 100 percent of its targets in fiscal year 2008, and 67 percent of its targets in fiscal year 2009.

As shown in figure 8, trends in targets met could not be determined for the most recent programs—Afghanistan Water, Agriculture, and Technology Transfer, ADP-Southwest, Afghanistan Vouchers for Increased Production in Agriculture, and Incentives Driving Economic Alternatives-North, East and West—because sufficient data was not available to establish trends. In addition, as noted earlier, most of these programs failed to establish targets for all of their indicators and, thus, we could not assess performance for all indicators. For example, even though recent performance data show that Afghanistan Water, Agriculture, and Technology Transfer and Afghanistan Vouchers for Increased Production in Agriculture programs met all targets for fiscal year 2009, both programs did not set targets for all indicators. In fiscal year 2009, the Afghanistan Water, Agriculture, and Technology Transfer program set targets for 3 out of 5 indicators, while Afghanistan Vouchers for Increased Production in Agriculture set targets for 2 out of 10 indicators. All indicators with targets met or exceeded their annual target.

USAID Faces Several Challenges in Implementing and Overseeing Agricultural Programs in Afghanistan

The security situation, the Afghan government’s lack of capacity, and USAID’s difficulties in providing management and staff continuity challenge the implementation of agricultural programs in Afghanistan. The security situation hinders USAID’s ability to reach key areas of the country and monitor programs. Additionally, while the Afghan government’s capacity to carry out its core functions has improved, key ministries, including the Ministry of Agriculture, Irrigation, and Livestock—which works to restore Afghanistan’s licit agricultural economy through increasing production and productivity, natural resource management, improved physical infrastructure, and market development—lack the ability to implement their missions effectively. Finally, USAID’s ability to maintain institutional knowledge has been hampered by high staff turnover.

Afghanistan’s Security Environment Challenges USAID’s Ability to Oversee and Implement Agricultural Programs

USAID noted difficulties in program oversight and implementation caused by the challenging security environment in Afghanistan. In November 2009, we reported that while U.S. and international development projects in Afghanistan had made some progress, deteriorating security complicated such efforts to stabilize and rebuild the country. And as we reported in May 2010, the lack of a secure environment has continued to challenge reconstruction and development efforts. Specifically, USAID has cited the security environment in Afghanistan as a severe impediment to its ability to monitor projects. For example, USAID noted that solely traveling by road to visit alternative development, food assistance, and environmental projects in rural areas of northern and eastern Afghanistan is normally not allowed due to security constraints, and must, consequently, be combined with some air travel. However, air service in much of the north and east is limited during the winter months, which has complicated oversight efforts. Similarly, USAID officials are required to travel with armored vehicles and armed escorts to visit projects in much of the country. Consequently, as USAID officials stated, their ability to arrange project visits can become restricted if military forces cannot provide the necessary vehicles or escorts because of heightened fighting or other priorities. We experienced similar restrictions to travel beyond the embassy compound when we visited Afghanistan in July 2009. For example, we were initially scheduled to visit agricultural sites in Jalalabad, but could not due to security threats. Instead, implementing partners traveled to Kabul to meet with us. According to USAID, limited monitoring has heightened the risk of fraud, waste, and mismanagement of USAID resources.

In addition to increasing challenges in overseeing programs, the security environment has also challenged USAID’s ability to implement programs, increasing implementation times and costs for projects in nonsecure areas. In particular, U.S. officials cited poor security as having caused delays, disruptions, and even abandonment of certain reconstruction projects. For example, according to implementing partner officials, in ADP-Southwest, some 15 to 20 illegal security checkpoints run by the Taliban and criminals near major trade centers have increased costs to and endangered the lives of farmers they support.

USAID predicated the success of its agricultural programs on a stable or improving security environment. In preparing its 2005-2010 strategic plan, USAID assumed that security conditions would remain stable enough to continue reconstruction and development activities. Likewise, several implementing partner documents included this assumption, and USAID officials affirmed that this assumption remains true today. Furthermore, the commander of the North Atlantic Treaty Organization-led International Security Assistance Force and U.S. forces in Afghanistan testified in his June 2009 confirmation hearing that improved security was a prerequisite for the development of local governance and economic growth in Afghanistan. However, as figure 9 illustrates, while attack levels continue to fluctuate seasonally, the annual attack “peak” (high point) and “trough” (low point) for each year since September 2005 have surpassed the peak and trough, respectively, for the preceding year.

Limited Afghanistan Ministerial Capacity

USAID has increasingly included and emphasized capacity building among its programs to address the government of Afghanistan’s lack of capacity to sustain and maintain many of the programs and projects put in place by donors. In 2009, USAID rated the capability of 14 of 19 Afghan ministries and institutions it works with as 1 or 2 on a scale of 5, with 1 representing the need for substantial assistance across all areas and 5 representing the ability to perform without assistance. For example, the Ministry of Agriculture, Irrigation, and Livestock was given a rating of 2—needing technical assistance to perform all but routine functions—while the Ministry for Rural Rehabilitation and Development was given a rating of 4—needing little technical assistance. Although USAID has noted overall improvement among the ministries and institutions in recent years, none was given a rating of 5. USAID officials noted that a key Afghan official was recently moved from the Ministry for Rural Rehabilitation and Development to enhance the Ministry of Agriculture, Irrigation, and Livestock’s capacity. As a result, USAID officials also said that they have recently begun to work more closely with Ministry of Agriculture, Irrigation, and Livestock.

According to the Afghanistan National Development Strategy, Afghanistan’s capacity problems are exacerbated by government corruption, a significant and growing problem in the country. Transparency International’s 2009 Corruption Perception Index ranked the country 179 out of 180. Similarly, in April 2009, USAID published an independent report, Assessment of Corruption in Afghanistan, that found that corruption, defined as “the abuse of public position for private gain,” is a significant and growing problem across Afghanistan that undermines security, development, and democracy-building objectives. According to USAID’s assessment, pervasive, entrenched, and systemic corruption is now at an unprecedented scope in the country’s history. The causes of corruption in Afghan public administration, according to the Afghanistan National Development Strategy, can be attributed to a lack of institutional capacity in public administration, weak legislative and regulatory frameworks, limited enforcement of laws and regulations, poor and nonmerit-based qualifications of public officials, low salaries of public servants, and a dysfunctional justice sector. Furthermore, the sudden influx of donor money into a system already suffering from poorly regulated procurement practices increases the risk of corruption and waste of resources. However, the assessment also noted that Afghanistan has or is developing most of the institutions needed to combat corruption, but these institutions, like the rest of the government, are limited by a lack of capacity, rivalries, and poor integration. The assessment also noted that the Afghan government’s apparent unwillingness to pursue and prosecute high-level corruption was particularly problematic.

USAID moved to address this lack of capacity and growing corruption by including a capacity-building component in its more recent contracts. For example, the Afghanistan Water, Agriculture, and Technology Transfer program was designed to, among other things, improve the capabilities of Afghan ministries and universities by partnering with them on research- based decision making and outreach projects, and to identify water and land-use policies and institutional frameworks that encourage individuals and local, provincial, and the national governments to increase sustainable economic development. Likewise, the Assessment of Corruption in Afghanistan report noted that “substantial USAID assistance already designed to strengthen transparency, accountability, and effectiveness— prime routes to combat corruption—in the most critical functions of national and subnational government.” For example, the assessment points to alternative-development and agricultural efforts to create incentives to not grow poppy, but also notes that these efforts should be coordinated with enforcement efforts supported by the Departments of Defense, Justice, and State.

The Administration has further emphasized capacity building by pursuing a policy of Afghan-led development, or “Afghanization,” to ensure that Afghans lead efforts to secure and develop their country. At the national level, the United States plans to channel more of its assistance through the Afghan government’s core budget. At the field level, the U.S.-assistance plan is to shift assistance to smaller, more flexible, and faster contract and grant mechanisms to increase decentralized decision making in the field. The new U.S. government agricultural strategy, linked to the U.S. effort to counter insurgency, stresses the importance of increasing the Ministry of Agriculture, Irrigation, and Livestock’s capacity to deliver services and promote the private sector and farmers’ associations through direct budget and technical assistance. However, USAID also recognized that, with the move toward direct assistance to government ministries, USAID’s vulnerability to waste and corruption is anticipated to increase. According to USAID officials, direct budget assistance to the Ministry of Agriculture, Irrigation, and Livestock is dependent on the ability of the ministry to demonstrate the capacity to handle the assistance. These officials noted that an assessment of the Ministry of Agriculture, Irrigation, and Livestock’s ability to manage direct funding was being completed. The U.S. Embassy has plans under way to establish a unit at the embassy to receive and program funds on behalf of the Ministry while building the Ministry’s capacity to manage the funds on its own.

USAID Faces Challenges to Maintaining Institutional Knowledge

USAID has not taken steps to mitigate challenges to maintaining institutional knowledge, a problem exacerbated by high staff turnover. As we noted earlier, USAID did not consistently document decisions and staff could not always respond to our questions about changes that had taken place over the life of the programs, often noting that they were not present at the time of the changes. For example, when we inquired about changes in the results and performance data reported, USAID officials in Afghanistan were not able to comment on the performance data or why changes were made to the data and noted that they were either not present when the changes took place or were too recently staffed to comment on performance data reported. Likewise, the Special Representative for Afghanistan and Pakistan’s staff responsible for drafting the current agricultural strategy for the United States stated that they could not effectively discuss USAID program implementation over the last several years because they were not there and lacked institutional knowledge of the programs.

We previously reported that USAID and other agencies in Afghanistan lack enough acquisition and oversight personnel with experience working in contingency operations. The USAID Mission to Afghanistan has experienced high staff turnover—USAID personnel are assigned 1-year assignments with an option to extend their assignment for an additional year—which USAID acknowledged hampered program design and implementation. In addition, the State Department Office of Inspector General noted in its recent inspection of the entire embassy and its staff, including USAID, that 1-year assignments coupled with multiple rest-and- recuperation breaks limited the development of expertise, contributed to a lack of continuity, and required a higher number of officers to achieve strategic goals. For example, the USAID monitoring officials for the eight programs we examined were in place on average 7.5 months (see table 4). Moreover, the length of time that a monitoring official is in place has declined. As of September 2009, the two most recently initiated programs, the Afghanistan Vouchers for Increased Production in Agriculture program and the Incentives Driving Economic Alternatives-North, East, and West program, have had monitoring officials in place for an average of only 3 months each.

USAID has not addressed the need to ensure the preservation of institutional knowledge. USAID officials noted that the effectiveness of passing key information from one monitoring official to another, is dependent on how well the current official has maintained his or her files and what guidance, if any, is left for their successor. USAID officials noted that a lack of documentation and knowledge transfer may have contributed to the loss of institutional knowledge.

Conclusions

Agricultural development is a key element of U.S. counterinsurgency efforts in Afghanistan. The United States considers agricultural assistance a key contribution to Afghanistan’s reconstruction and stabilization. Since 2002, the United States has awarded about $1.4 billion toward agricultural programs in Afghanistan and plans to invest hundreds of millions of dollars more.

As such, ensuring sufficient oversight and accountability for development efforts in Afghanistan takes on particular importance. In addition to relying on implementing partners to execute its programs, a key part of the U.S. oversight and accountability efforts involves a reliance on the collection and analysis of implementing partner data. These implementing partners are expected to and have reported routinely on the performance of USAID’s agricultural programs. However, USAID has not always approved performance indicators established by its implementing partners, allowing one program to operate for almost 5 years without approved performance indicators. Additionally, USAID did not ensure that its implementing partners had established targets for each performance indicator, and USAID did not consistently analyze and interpret implementing partner performance data, which is vital to making program adjustments, higher- level decisions, and resource allocation. Without a set of agreed-upon indicators and targets, and analysis and interpretation of reported performance data, it becomes more difficult to accurately assess the performance of USAID agricultural programs. It is also unclear whether or how USAID has used evaluations to further institutional learning, inform current programs, and shape future planning. Best management practices have demonstrated that routine evaluations enable program managers to identify program vulnerabilities, implement lessons learned, help program managers understand program weaknesses and make needed improvements. Moreover, a lack of documentation of key programmatic decisions and an insufficient method to transfer knowledge to successors have contributed to the loss of institutional knowledge and the ability of the U.S. government and others to build on lessons learned. This makes it more difficult for USAID officials responsible for programmatic decisions, most of whom are in place for less than a year, to make informed decisions about shaping current and future programs.

Recommendations for Executive Action

To enhance the performance management of USAID’s agricultural programs in Afghanistan, we recommend the Administrator of USAID take steps to ensure the approval of implementing partner performance indicators; ensure that implementing partners establish targets for all performance consistently analyze and interpret program data, such as determining the extent to which annual targets are met; make use of results from evaluations of its agricultural programs; and address preservation of institutional knowledge.

Agency Comments and Our Evaluation

USAID provided written comments on a draft of this report. The comments are reprinted in appendix VI. USAID generally agreed with the report’s findings, conclusions, and recommendations and described several initiatives that address elements of the recommendations. In further discussions with USAID to clarify its response, USAID officials stressed the challenges involved in working in Afghanistan as a result of the security environment and working conditions. They submitted additional documentation, including new guidance on monitoring in high-threat environments, which was disseminated in December 2009. USAID also provided technical comments, which we have included throughout this report as appropriate. We also provided drafts of this report to the Departments of Agriculture, Defense, and State, all of which declined to comment.

We are sending copies of this report to interested congressional committees, USAID, and the Departments of Agriculture, Defense, and State. In addition, the report will be available at no charge on GAO's Web site at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-7331 or johnsoncm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix VII.

Objectives, Scope, and Methodology

This review assesses (1) how the United States has changed the focus of its agricultural efforts in Afghanistan since 2002, (2) USAID’s performance management and evaluation efforts of agricultural programs in Afghanistan, (3) the extent to which USAID’s agricultural programs in Afghanistan met targets, and (4) USAID’s efforts to mitigate challenges in implementing agricultural programs in Afghanistan. In addition, we analyzed financial information on USAID’s programs in Afghanistan and reported on the financial status of its agricultural programs.

To assess USAID’s agricultural programs in Afghanistan, including changes in focus, we met with officials from USAID, the Departments of Agriculture, Defense, and State—including the Special Representative for Afghanistan and Pakistan’s office—and implementing partners in Washington, D.C., and Kabul, Afghanistan. In Kabul, we also met with officials from the United Nations, the governments of Afghanistan and the United Kingdom, and a local research group to discuss agricultural efforts. We traveled to the provinces of Badakhshan and Farah to meet with U.S. and Afghan officials and discussed various U.S.–funded projects. For example, in Farah, we met with the local Afghan officials and beneficiaries of U.S. assistance to discuss the progress of USAID’s agricultural projects, visited a project site, and met with U.S. contractors implementing the projects. In Badakhshan, we also met with local officials and beneficiaries to discuss USAID agricultural efforts and how U.S. assistance was being used. In addition, we reviewed past GAO work and reports from other agencies in the U.S. accountability community and nongovernmental organizations on Afghanistan’s current situation and the challenges it faces. We reviewed U.S. government documents concerning the U.S. agricultural strategy and efforts in Afghanistan, as well as USAID funding data. Beginning in fiscal year 2005, USAID began financing agricultural projects as part of the U.S. government’s counternarcotics strategy. These were initially referred to as alternative-livelihood programs, but were later called alternative-development programs (ADP). To track this strategy over time, we reported the share of annual obligated and disbursed funds for ADP and for other agriculture activities. We focused our review on the eight USAID agricultural programs that were active between 2007 and 2009 and had total awards of more than $15 million; however, our analysis of financial information included all USAID agricultural programs. None of the agricultural programs included in GAO’s review were Office of Transition Initiatives (OTI) and the Office of U.S. Foreign Disaster Assistance (OFDA) programs. We did not address agriculture-related projects carried out independently by other U.S. government agencies, such the Department of Defense’s Commander’s Emergency Response Program, or those carried out by multilateral institutions to which the United States contributes, such as the United Nations Development Programme or World Food Program.

To assess USAID’s performance management and evaluation efforts, we reviewed the Government Performance and Results Act of 1993 and pertinent GAO evaluations of performance management practices to identify best practices. In addition, we examined USAID’s Automated Directives System and USAID performance management and evaluation documents to identify the agency’s procedures, requirements, and guidance. The Government Performance and Results Act and USAID’s Automated Directives System establish requirements or provide guidance at a level higher than the program level; the former operates at the agency level and the latter directs most of its performance management and evaluations procedures to the bureau or mission level. Nevertheless, effective planning for results at the agency, mission, and bureau level is a function of effective planning for results at the program level. We view results-oriented management tools, including the setting of indicators and targets, as important at all levels of an agency, including the program level. Consequently, we determined that the Government Performance and Results Act criteria, which is operationalized for USAID through its Automated Directives System, is applicable at this level as well. We reviewed USAID and implementing partner planning, funding, and reporting documents for their agricultural programs in Afghanistan, as well as those addressing evaluations. Our review of these documents provided us with information regarding the programs’ performance management structure, goals, objectives, indicators, and targets. We examined these and other documents to determine the extent to which the Mission followed requirements, guidance, and best practices.

To assess the extent to which performance was achieved, we reviewed all quarterly and annual reports, implementing partner PMPs, and annual work plans for the eight agricultural programs under review. The data were primarily compiled from implementing partner quarterly reports from April 2005 though September 2009. When data were not available, we used the PMPs and other documents to fill in gaps. USAID could not provide all the needed documents; we, therefore, requested missing documents from the implementing partners. To determine the validity and reliability of the data reported in the quarterly reports, we requested USAID’s completed data quality assessments. We received one USAID data quality assessment completed for all agricultural programs from November 2008, and one program data quality assessment completed by USAID monitoring officials, also from November 2008. We also checked the data for inconsistencies and questioned USAID officials and implementing partners about any inconsistencies. We found the data in the quarterly reports to be sufficiently reliable for our purposes.

The data collected were organized in a spreadsheet on a quarterly basis for the eight programs. To track program performance over time, we collected all reported quantitative data on indicators, targets, and results. For each reported indicator, we measured performance achieved as the ratio of results against established targets. We found that implementing partners were inconsistent in reporting on targets, by either not setting targets, or in two cases, retroactively setting or revising targets. In addition, two programs shifted reporting time frames between fiscal and calendar years. Some of the changes resulted from implementing partner audits or a USAID Regional Inspector General audit of the data collected and reported to USAID. Although USAID encourages and permits changes to targets over the life of a program in response to new information, these factors complicated attempts to determine performance. Furthermore, in some cases, in addition to targets, the results were also updated retroactively. We captured changes in target levels and reported results by inserting additional lines in a spreadsheet. This process allowed us to determine changes in targets, results, and performance achieved over time.

In general, data reported in quarterly reports were presented cumulatively; however, we found this presentation masked performance achieved in a specific year. Therefore, once all cumulative data were entered into the spreadsheet, we calculated the numbers to show annual targets, results, and performance. The performance data collected were categorized into eight categories: (1) met or exceeded target, (2) achieved 76 to 99 percent of target set, (3) achieved 51 to 75 percent of target set, (4) achieved 26 to 50 percent of target set, (5) achieved 1 to 25 percent of target set, (6) achieved zero progress toward target set, (7) number of indicators used to assess performance, and (8) no target set. Based on the categorical assessment, we were able to determine the number of indicators reported annually and over the life of the program in each of the categories noted above. We are reporting program performance achievements on the annual percentage of indicators that met or exceeded the target. For example, if there were 15 indicators and 9 indicators had met or exceeded the target, than annual program performance was 60 percent (9/15). We decided this measure of performance was appropriate because implementing partners are allowed to adjust and revise target levels to better reflect available information in the field. Further, we found that the percentage of indicators meeting their targets could increase or decrease for a variety of reasons, including changes in measures, the types of measures or the targets set, as well as changes in actual underlying performance. A review of all those factors was beyond the scope of this report.

To examine the challenges faced by agricultural efforts in Afghanistan, we reviewed U.S. strategy documents and USAID documents addressing the status of and challenges faced by U.S. efforts in Afghanistan, including security, Afghan capacity and corruption, and USAID staffing and workspace concerns. We also reviewed Department of Defense documents on counterinsurgency strategy and joint diplomatic-military plans. We updated attack data on which we had previously reported. We assessed the data and found them to be sufficiently reliable for our purposes. We reviewed the Afghan government and nongovernmental organization reports regarding capacity and corruption in Afghanistan. To compile a list of USAID monitoring officials, we reviewed the names listed in data USAID provided.

To report financial information on USAID agriculture programs and individual projects, we used financial information from the “Pipeline Report” generated from USAID’s Phoenix financial management information system provided to us by the Office of Financial Management at the USAID Mission to Afghanistan. This report contains cumulative financial information for individual projects that may be funded through a contract, cooperative agreement, or grant. We received pipeline reports for all USAID projects in Afghanistan as of the end of fiscal years 2004-2009 and March 31, 2010, and for selected agriculture projects, we received quarterly pipeline reports from fiscal years 2005-2009. The March 31, 2010, pipeline report contains financial information on USAID projects in Afghanistan from fiscal year 2002 to March 31, 2010, for 254 projects of which 41 were agriculture projects. We also checked the data for inconsistencies and questioned USAID officials about any inconsistencies. To describe the financial status of the USAID agriculture program, we used three financing concepts: award, unliquidated obligation, and disbursement. These are related to, but not exactly the same as budget concepts. Award refers to the dollar amount of the award in a signed contract, cooperative agreement, or grant. The signed document indicates the period of time over which the project is expected to be implemented. The amount of the award, the time frame, and other elements of the contractual agreement may be changed through a formal amendment process. Unliquidated obligations represent the current amount of obligations remaining to be disbursed. Disbursements are those funds that have been released from the U.S. Treasury. Cumulative obligations is the total of unliquidated obligations plus disbursements.

The e-supplement presents information on Afghanistan for select indicators, including poverty rates by province and region, and the number of households that produced crops during the summer and winter seasons during 2007/2008. The information comes from supplementary tables to a Government of Afghanistan household survey, National Risk and Vulnerability Assessment 2007/8, http://nrva.cso.gov.af/index.html, released online in January 2010. The survey covered the period from September 2007 through August 2008, and was conducted with the financial and technical assistance of the European Commission and several other organizations.

The report groups contiguous provinces into 8 regions. For each of the 34 provinces and 8 regions we report the population and compute the share of population that has access to land, has access to safe drinking water, is literate (ages 15 years and over), is urban, has access to electricity, has access to public health facilities (1 hour or less by foot), and owns livestock. The poverty rate is the share of population living on less than a minimum level of food and nonfood (for example, shelter) consumption. In the map, provinces are categorized into 5 groups from high to low ranges of poverty rates. These are from a World Bank analysis of the assessment’s data. These are ranges of poverty rates and not confidence intervals. The poverty rate is only presented for the regions, a national average for the year and for seasons of the year.

For a list of 25 crops, including other crops, the report lists the number of households that produce each crop during the summer and winter seasons on irrigated land. Since more than one crop may be grown each season by a farmer, the assessment reports the number of households that report each crop as its primary, second, or third crop. For example, during the summer season, 1,349,200 households cultivate a primary crop, 889,200 households (66 percent) cultivate a second crop, and 448,700 households (33 percent) produce a third crop. For each crop, we computed the number of households that reported producing that crop, regardless whether as a primary, second, or third crop, for each season. We also provide tables showing the number of households that report producing crops as the primary, second, or third crop, for each season. There is no information on the number of households that produce different crops on rain-fed land. The assessment reports that 591,000 households have access to rain-fed land, but some of these households may also have access to irrigated land.

We reviewed the survey methodology of National Risk and Vulnerability Assessment 2007/8 report and found it to be sufficiently sound, particularly given the challenging environment in which the data were collected and the potentially sensitive nature of the questionnaire topics. That said, we were not able to ascertain information on some aspects of the survey, which would have helped shed light on its quality. A survey’s design can be judged by its success or failure in minimizing the following types of errors.

Sampling error. The report provides no information on the precision of its estimates. Usually, this is expressed in a confidence interval. We cannot, therefore, judge the reliability of the point estimates.

Nonresponse error. The report mentions a process of reserve replacement sampled households in the event of a noncontact. However, no data is kept on how frequently these were used, so it is not possible to calculate a response rate. Officials reported that this replacement rate was low.

Coverage error. Sixty-eight of the 2441 primary sampling units were replaced, mostly due to security concerns. Such coverage errors could lead to a coverage bias if those covered are categorically different from those not covered with respect to variables of interest.

Measurement error. While most of the questionnaire is not of a sensitive nature, we have to be aware that farmers might not be completely honest with a government interviewer when it comes to the cultivation of illicit crops. As such, our assessment was based only on information that was made available about the survey methodology.

We conducted this performance audit from March 2009 through July 2010 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Funding Breakout of USAID Agricultural Programs in Afghanistan, as of the End of Fiscal Year 2009

Summary financial information about the eight agriculture programs discussed in our report is reported in table 5. Because we selected relatively large programs, the total amount of funds that USAID planned to spend on these eight programs over $1 billion or 75 percent of the total awards for all 41 agriculture programs. One program, the Afghanistan Vouchers for Increased Production in Agriculture program, accounts for 26 percent of all agricultural awards. As of the end of fiscal year 2009, these eight programs accounted for 69 percent of total USAID agricultural assistance disbursements, with one program accounting for 18 percent of all disbursements.

USAID Agricultural Program, by Province, and Examples of Activities

The following figure shows the provinces in which the eight agricultural programs we reviewed were active and an example of the types of activities they undertook.

GAO Analysis of Implementing Partner Performance Data

The following tables provide information on the annual number of indicators that fell into one of the eight percentage categories. Performance is measured by comparing results against targets. We assessed annual performance—the number of indicators that met or exceeded their target—for each of the eight programs to highlight improvements and declines in performance that took place in a given fiscal year. As shown below, the data collected were organized into eight categories (1) met or exceeded target, (2) achieved 76 to 99 percent of target set, (3) achieved 51 to 75 percent of target set, (4) achieved 26 to 50 percent of target set, (5) achieved 1 to 25 percent of target set, (6) achieved zero progress toward target set, (7) number of indicators used to assess performance, and (8) no target set. Based on the categorical assessment, we were able to determine the number of indicators that fell into one of the eight categories, the number of indicators with target levels reported, and the total number of indicators reported. For each fiscal year, the first column represents the number of indicators whose performance fell within an indicated range. The second column is a percentage of total number of indicators (with and without targets) that fell within an indicated range. Please note this percentage is based on total number of indicators tracked, and may differ from those in figure 8, which is based on total number of indicators with targets.

Eight USAID Agricultural Programs, Annual Targets, Results, and Percentage of Targets Met, for the Latest Year

The tables below provide information on annual targets, results, and percentage of each target met for the eight programs we reviewed. The data provided is based on the latest year where performance data was available.

Comments from the U.S. Agency for International Development

The following are GAO’s comments on the U.S. Agency for International Development’s (USAID) letter dated July 1, 2010.

GAO Comments

1. GAO modified the report to reflect the new Automated Directives System Guidance on Monitoring in High Threat Environments. However, GAO would note that the Mission to Afghanistan that GAO was directed to for all inquiries was not aware of the December 2009 guidance until June 2010. In addition our application of the Automated Directives System criteria was consistent with the new guidance and required only minor technical revisions. 2. None of the agricultural programs included in GAO’s review were Office of Transition Initiatives (OTI) or Office of U.S. Foreign Disaster Assistance (OFDA) programs. 3. USAID’s Automated Directives System generally requires the same procedures for a conflict zone that it requires elsewhere or else documentation specifically describing when those procedures are not followed. The new guidance does not provide any exemptions with regard to approval of monitoring and evaluation plans and the establishment of indicators and targets, which USAID did not consistently approve. 4. Without approved indicator targets, it is unclear how performance can be reviewed or assessed. 5. GAO acknowledges that USAID is currently developing a PMP on page 14 of the report. 6. At the time of our review, USAID had completed a one midterm evaluation covering three of the eight programs we reviewed. USAID staff, however, were unable to indicate how the findings of the evaluation were used to inform the design of subsequent programs. Additionally, the midterm evaluation included recommendations for improving the three programs, but USAID staff were unable to comment on how the recommendations of the evaluation were implemented. 7. GAO removed its mention of the semiannual reviews from the recommendation based on additional information provided.

GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, the following staff contributed to the report: Hynek Kalkus, Assistant Director; Thomas Costa; Farahnaaz Khakoo; Bruce Kutnick; Sara Olds; Steven Banovac; Joseph Carney; Elizabeth Curda; Mark Dowling; Gena Evans; Etana Finkler; Justin Fisher; Cindy Gilbert; Gloria Mahnad; Kara Marshall; Jackie Nowicki; Sheila Rajabiun; and Jena Sinkfield.