U.S. Agency for International Development: Status of Achieving Key Outcomes and Addressing Major Management Challenges
Published: 2001-08-17T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: This report reviews the U.S. Agency for International Development's (USAID) fiscal year 2000 performance report and fiscal year 2002 performance plan to assess the agency's progress in achieving selected key outcomes that are important to the agency's mission.
section_title: What GAO Found
paragraphs: GAO found that although USAID reported progress toward achieving the selected outcomes, the extent of the progress is unclear because the agency based its support on disaggregated and, in some cases, out-of-date and selective data. Unlike past years when USAID issued separate performance reports and performance plans, the agency issued a performance overview supplemented by more detailed data in the fiscal year 2000 budget justification to Congress, both of which incorporated elements of performance reporting and planning. In the fiscal year 2000 Performance Overview, USAID based its statements of progress on self-reported fiscal year 1999 performance data provided by individual USAID missions. In addition, USAID reported progress toward achieving agency goals and objectives by relying on selected information about an individual country's missions' performance. Although USAID reported detailed fiscal year 2000 performance data at the operating unit level in its budget justification, those data were not aggregated to summarize progress toward agency objectives.

section_title: Letter
paragraphs: As you requested, we reviewed the U.S. Agency for International Development’s (USAID) fiscal year 2000 performance report and fiscal year 2002 performance plan required by the Government Performance and Results Act of 1993 (GPRA) (P. L. 103-62, 107 Stat. 285) to assess the agency’s progress in achieving selected key outcomes that you identified as important mission areas for the agency. These are the same outcomes we addressed in our June 2000 review of the agency’s fiscal year 1999 performance report and fiscal year 2001 performance plan to provide a baseline by which to measure the agency’s performance from year to year.These selected key outcomes are   economic growth and development, the spread of Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome (HIV/AIDS), and   natural and man-made disasters.
As agreed, using the selected key outcomes for USAID as a framework, we (1) assessed the progress USAID has made in achieving these outcomes and the strategies the agency has in place to achieve them and (2) compared USAID’s fiscal year 2000 performance report and fiscal year 2002 performance plan with the agency’s prior year performance report and plan for these outcomes. Additionally, we agreed to analyze how USAID addressed its major management challenges, including the governmentwide high-risk areas of strategic human capital management and information security, that we and USAID’s Inspector General identified. Appendix I provides detailed information on how USAID addressed these challenges.
subsections: section_title: Background
paragraphs: Purposes of the Government Performance and Results Act - GPRA is intended to shift the focus of government decisionmaking, management, and accountability from activities and processes to the results and outcomes achieved by federal programs. New and valuable information on the plans, goals, and strategies of federal agencies has been provided since federal agencies began implementing GPRA. Under GPRA, annual performance plans are intended to clearly inform the Congress and the public of (1) the annual performance goals for agencies’ major programs and activities, (2) the measures that will be used to gauge performance, (3) the strategies and resources required to achieve the performance goals, and (4) the procedures that will be used to verify and validate performance information. These annual plans, issued soon after transmittal of the President’s budget, provide a direct linkage between an agency’s longer-term goals and mission and day-to-day activities. Annual performance reports are to subsequently report on the degree to which performance goals were met. The issuance of the agencies’ performance reports, due by March 31, represents a new and potentially more substantive phase in the implementation of GPRA —the opportunity to assess federal agencies’ actual performance for the prior fiscal year and to consider what steps are needed to improve performance and reduce costs in the future.
USAID Mission, Operating Environment, and Strategic Plan - USAID, an independent federal government agency, is responsible for implementing U.S. foreign assistance programs. USAID’s Administrator receives overall foreign policy guidance from the Secretary of State on implementing U.S. foreign assistance. USAID operates programs in about 125 countries, with resident staff in 75 countries in 4 regions of the world: sub-Saharan Africa, Asia and the Near East, Latin America and the Caribbean, and Europe and Eurasia. USAID promotes a wide range of assistance objectives related to economic and agricultural growth; population, health, and nutrition; the environment; democracy, governance, and the rule of law; education and training; and humanitarian assistance. USAID staff frequently work in difficult environments and under evolving demands, and programs often require years of effort to achieve desired outcomes.
At USAID, decisions about programs and operating resources are based on the settings in which the agency pursues its goals. These settings can vary significantly across countries and regions, as well as over time. Therefore, USAID relies on local participation in planning and implementing programs since mission personnel are more familiar with the needs of a specific locale. According to USAID, the agency develops a new strategic plan and objectives for operating units to accomplish, or modifies existing ones, as appropriate. As part of this effort, operating units develop plans for performance monitoring that include baseline data and performance targets. Operating units annually report progress against performance targets and, based on the results, request resources for the following fiscal year. USAID headquarters uses these results both to allocate funding and to make “sector-wide assessments of the effectiveness of various objectives and approaches” for use in its annual performance report.
In 1997, USAID defined a limited set of performance goals and indicators for each of the agency’s strategic goals. According to the agency, the goals, indicators, and targets are broad and at a country level. They share common attributes, such as “they are commonly available across countries; they are for the most part independently collected and available from published sources; and they permit performance reporting using a fairly compact set of tables that can be readily summarized and aggregated.” However, despite these attributes, USAID acknowledges that one cannot attribute overall country progress toward specific outcome areas solely to USAID programs. According to the USAID strategic plan, despite supporting and contributing to the achievement of performance goals, actual achievement is at best only weakly linked to USAID programs and resources. With this in mind, USAID revised its strategic plan in October 2000, and beginning this year, the agency will use the strategic objectives and indicators of operating units, which include country missions, regional offices, and bureaus, as the agency’s performance goals for its annual performance plan and annual performance report. USAID felt that at this level, it could more clearly demonstrate its contributions.
Further, the agency changed the manner in which it will present performance plans and results by issuing two annual documents replacing previous years’ Annual Performance Plan and Annual Performance Report, starting with results reporting for fiscal year 2000 and planning for 2002.
Both new documents will incorporate elements of past year results and future year plans, with a general description of both provided in a Performance Overview and more detailed performance data and targets provided in the annual Budget Justification to the Congress.
section_title: Assessment of USAID’s Progress and Strategies in Achieving Selected Key Outcomes
paragraphs: This section discusses our analysis of USAID’s performance in achieving its selected key outcomes and the strategies the agency has in place.Although USAID reported it made progress toward achieving the selected outcomes, the extent of the progress is unclear because the agency based its support on disaggregated and, in some cases, out-of-date and selective data. Actual progress toward achieving outcomes, detailed in the FY 2000 Performance Overview, is not clear because USAID did not base its assessment of progress on fiscal year 2000 performance data. Instead, the agency relied on selective performance data the operating units submitted for fiscal year 1999. Data presented in the Fiscal Year 2002 Budget Justification to the Congress represent 2000 performance; however, USAID provided no discussion of the data’s validity and reliability. In addition, limiting performance data to the disaggregated level of the operating units’ strategic objectives makes determining progress toward achieving agency goals difficult.  Finally, the agency provided a discussion of strategic human capital management issues and strategies related to its efforts for addressing natural and man-made disasters, but it did not discuss human capital issues and strategies in the context of its economic development and HIV/AIDS activities.  The agency also did not discuss information technology issues and strategies in the context of accomplishing the three key outcomes.
subsections: section_title: Economic Growth and Development
paragraphs: USAID reported it made progress toward achieving its outcome of increasing economic growth during 2000. However, the Performance Overview does not clearly articulate the agency’s actual progress and relies on 1999 performance results and mission self-assessments. Moreover, fiscal year 2000 data reported in the Budget Justification were not aggregated to summarize regional or agency level performance.
To achieve this outcome USAID undertakes programs under three broad objectives:   expanding and strengthening critical private markets,   enhancing agricultural development and encouraging food security, and   expanding and creating more equitable access to economic opportunity for the rural and urban poor.
Seventy-five missions, regional offices, and bureaus (operating units) carry out 152 programs, also referred to as strategic objectives, under one or more of the three objectives. For example, more than two-thirds of the operating units supporting the economic growth outcome have programs with a focus on strengthening markets. The agency reported in the Performance Overview that overall it met or exceeded performance expectations for encouraging economic growth and agricultural development for 88 percent of the 152 strategic objectives. USAID based this determination on operating unit self-assessments but does not explain how these self-assessments were conducted.
In addition, USAID did not assess agency progress against the performance goals contained in the Annual Performance Plan for Fiscal Year 2000. For example, the agency did not report 2000 performance for the percentage of countries with annual growth of Gross Domestic Product per capita of at least 1 percent. Instead, the performance overview includes a discussion of economic growth trends in USAID recipient countries by region drawing on these broad country-level indicators.
USAID’s fiscal year 2002 Budget Justification, which USAID completed after it submitted the Performance Overview, does contain fiscal year 2000 performance data. These data described the progress missions and other units made relative to agreed upon targets. However, the data are presented for individual operating units and are not aggregated to summarize regional or agencywide performance in either this document or the Performance Overview.
USAID reported that 13 percent of the operating units’ economic growth and agricultural development strategic objectives did not meet expectations. For example, the overview states that several USAID programs under the strengthening markets objective failed to meet their goals. The Performance Overview listed operating units in the Europe and Eurasia region, including Albania, Armenia, Croatia, Kyrgyzstan, Moldova, Tajikistan, Turkmenistan, and Uzbekistan, as having difficulties. USAID further reported that programs in other regions, including Angola, Colombia, and Haiti, also fell short of expectations. USAID attributed these difficulties to external factors, especially to the lack of host government commitment to reforms and, in some cases, to political instability stemming from government transition. For example, USAID reported that in Croatia the agency suspended assistance under its financial program in 1998 because the government showed no willingness to engage in any meaningful economic reform.
USAID’s explanation for not meeting some of its performance goals is consistent with its reporting of the prior year’s performance. USAID acknowledged that external factors, such as international financial conditions, might seriously affect desired program outcomes. Although, the Performance Overview provides examples of some programs that did not meet expectations in 1999, it does not provide details on all programs that did not meet expectations. Nor does it discuss strategies for addressing unmet goals in either 2000 or 2001. Although the Budget Justification contains detailed fiscal year 2000 performance information for individual operating units, explanations for why missions failed to meet performance targets for specific strategic objectives are not always provided. For example, one target of USAID’s program for reconstruction and expanded economic opportunity in Lebanon was “7,800 informal sector loans delivered.” The actual number of loans provided in 2000 was 5,880. The Budget Justification did not discuss the significance of, or provide a reason for, this shortfall.
The Performance Overview does not assess the impact of its fiscal year 2000 performance on the likelihood of achieving its fiscal year 2001 goals for economic growth and agricultural development. The overview contains a general statement concluding that prospects for further progress are good, but it does not state what data this assertion is based on. It adds, however, that actual progress will continue to depend mainly on the self- help efforts of governments and countries.
The Budget Justification discusses fiscal year 2000 performance and fiscal year 2001 prospects for each strategic objective described for the individual countries and regions. For example, one strategic objective presented in the Budget Justification for the Philippines is to “accelerate the economic transformation of Mindanao.” USAID cited progress in some of its efforts in fiscal year 2000, such as the number of marginal farm and fishing families initiating commercial-level production of more high-value products. The agency reported that performance in 2001 will be affected by the general poor performance of the national economy and the degree to which progress is achieved in managing the continued conflict in parts of Mindanao.
The Performance Overview includes a general discussion of planned activities at a few operating units for achieving unmet performance goals (based on fiscal year 1999 data), but it does not include specific strategies or time frames. For example, in commenting on efforts in Haiti, where programs did not meet expectations, USAID reported that it would continue to work in collaboration with other donors to consolidate gains and continue a reform process. USAID’s overall strategies for achieving its performance goals, such as expanding trade through policy, legal, and regulatory reform; increasing production of agricultural commodities; and providing small loans and business training for the rural poor, are reasonable. The overview provides a general discussion of USAID’s plans for 2002, stating that the basic framework that guides operating choices will remain the same. The Budget Justification contains more detailed information by country and region.
USAID recognized the contribution of other multilateral and bilateral donor organizations, host-country governments, and U.S. government agencies. The Performance Overview included a general statement acknowledging that its economic growth and agricultural development programs do not operate in a vacuum. The overview also lists major multilateral and bilateral donors. The Fiscal Year 2002 Budget Justification provided more details on USAID’s specific contributions and the contributions of other donor programs by country and strategic objective. In addition, the overview discussed using studies and evaluations to confirm its broad-based economic approach to development, but it did not state whether these tools were employed when developing plans for 2002. In addition, the overview did not discuss issues related to strategic human capital management and information security as strategies to achieve increased economic growth and agricultural development. The overview includes a separate section that discusses USAID’s agencywide strategic goal to achieve USAID goals efficiently and effectively. This goal includes objectives that address strategic human capital management and information technology issues.
section_title: The Spread of HIV/AIDS
paragraphs: It is difficult to assess the progress USAID made in 2000 toward achieving this outcome. The agency did not systematically report on its sole indicator—condom usage—detailed in the Annual Performance Plan for 2000. Moreover, the Performance Overview relies on 1999 performance data of selected missions, and fiscal year 2000 data reported in the Budget Justification were not aggregated to summarize regional or agency level performance.
Strategies for addressing this selected outcome of reducing the transmission and impact of HIV/AIDS are diverse and include efforts to change sexually risky behavior and cultural norms; develop, test, and promote prevention and provide care; treat and prevent sexually transmitted infections; enhance nongovernmental, public- and private- sector organizations’ capacities to prevent the spread of HIV/AIDS and support persons affected by the disease; and improve the availability, quality, and use of evaluations and surveillance information. USAID works closely with the international community on HIV/AIDS efforts, including host governments, other national and multilateral assistance agencies, nongovernmental organizations, and the private sector. In addition, USAID partners with the U.S. Bureau of the Census to maintain and update the HIV/AIDS International Surveillance Database and provides funding for HIV/AIDS research to the Centers for Disease Control.
The performance goal for USAID’s HIV/AIDS activities is to reduce HIV infections by 10 percent among 15- to 24-year-olds between 1998 and 2007.  Although this goal is objective and quantifiable, progress toward achieving it cannot be assessed since USAID did not systematically report prevalence data for 2000 either at the regional or country level.  In addition, USAID also did not systematically report on its sole proxy indicator for reducing HIV/AIDS prevalence—condom usage—identified in the Annual Performance Plan for 2000 but did not cite a reason. However, the agency reported on other generally accepted proxy indicators for reducing HIV/AIDS transmissions not prescribed in the plan for 2000, such as condom sales, but data reported in the Performance Overview were mostly from 1999, while 2000 data reported in the Budget Justification were not sufficiently aggregated to summarize agency performance.
In a March 2001 report, we concluded, in part, that USAID’s ability to measure the impact of its activities on reducing the transmission of HIV/AIDS is limited by a “lack of routine reporting of results to headquarters.” While the agency has begun implementing a new monitoring and evaluation system, planning for the system did not specify to whom data would be reported or how it would be used. We also noted in the report that in 1998 USAID established a data repository for collecting and tracking performance data from HIV/AIDS programs. However, the agency did not require that missions actually report information to that data facility. The lack of a reporting requirement to a centralized unit—whether at the regional or headquarters level—affected the agency’s ability to generalize about progress toward the selected outcome and to make management and resource decisions based upon the data. It also inhibited sharing best practices because the agency cannot compare activities across countries. We concluded that failure to address these issues not only inhibits USAID’s ability to measure performance, but also hinders the agency’s decisionmaking regarding allocation of resources among missions and regional offices and limits its efforts to identify best practices.
USAID’s reporting of 2000 results and its planning for 2002 performance underscore our conclusion. USAID did not provide information on indicators that summarized worldwide or regional levels. Further, in addition to a reliance on fiscal year 1999 data, information provided in the Performance Overview to support overall statements of accomplishments is selective and at a country level, and may or may not be representative of all countries where similar data are collected. For example, in the Performance Overview USAID addressed progress toward the reduction of HIV/AIDS prevalence by discussing only three countries, saying its assistance resulted in a 50-percent decline in prevalence rates among young urban women in Uganda, a reported reduction in rates for 15- to 19- year-olds in Zambia, and the maintenance of low prevalence rates in Senegal.  Despite citing no data for the latter two countries, USAID officials said Uganda, Senegal, and Zambia represent significant accomplishments and show the donor community can have impact on stemming the spread of HIV/AIDS. Although it is important to show where USAID and its partners in the international community have made strides against the pandemic, these three countries represent just 7 percent of the countries where USAID operates HIV/AIDS programs.   Moreover, the citation of three interventions generally considered successes does not provide enough information to assess overall progress toward achieving the desired outcome.
USAID’s 2002 Budget Justification provided data on various HIV/AIDS indicators for each unit operating HIV programs. As with the Performance Overview, the indicators were generally accepted proxies for HIV/AIDS prevalence. These indicators were diverse—almost 60 distinct measures used by 24 missions and regional offices in the Africa region alone (see table 1 for examples from Zambia and South Africa). In addition, many missions used output- rather than outcome-oriented performance measures, reflecting their primary use as management tools for specific program activities. A few operating units, such as the missions in Uganda and Ghana, reported prevalence-based indicators, while other country missions, such as in Rwanda, Ethiopia, and Mozambique, reported on condom usage. Although the data contained in the Budget Justification may be useful for program management within a specific mission, or for individuals interested in assessing effects of activities in a specific country, the lack of consistent indicators limits the usefulness of this document for determining agencywide progress toward achieving the key outcome.
In 2001, USAID began expanding its response to the HIV/AIDS pandemic by developing new strategies and activities designed to prevent new transmissions of HIV and identify and treat those with the disease. In the Performance Overview, USAID stated it would support both high- prevalence countries and low-prevalence, high-risk countries by building capacity to provide care and support for affected individuals and families; reduce the social and economic impact of key vulnerable groups, such as health, education, and government professionals; and develop and introduce new prevention and care approaches and technologies, such as microbicides, lower-cost testing with quicker results, and behavior change activities.
The Performance Overview provided no specific set of overall performance targets for its efforts in 2002, but it discussed the accomplishments USAID expects to make by 2007 in unspecified “selected countries,” which include reducing HIV prevalence rates among 15- to 24-year-olds by 50 percent in high-prevalence countries and maintaining prevalence rates below 1 percent among 15- to 49-year-olds in low-prevalence, high-risk countries;   providing access to interventions aimed at reducing mother-to-child transmissions of HIV to 25 to 50 percent of infected mothers in high- prevalence countries;   building host-country capacity to provide basic care and psychosocial services to at least 25 to 50 percent of children affected by AIDS, including those who have lost parents to the disease; and lessening the impact of HIV/AIDS on the key labor sectors, such as education, health, and government professionals.
The above measures are objective and measurable, and USAID stated that the targets would be regularly reviewed and revised in light of the evolving pandemic and ongoing analyses of the cost-effectiveness of the programs. To do so, USAID, in collaboration with the Centers for Disease Control, the World Health Organization, and the Joint United Nations Programme on HIV/AIDS, will improve HIV sentinel surveillance systems in selected countries to report annually on HIV prevalence among 15- to 24-year-olds. However, USAID did not state the degree by which information will be aggregated so that decisionmakers in USAID’s regions and headquarters, as well as the Congress, can deliberate on resource allocations based on progress toward reducing the spread of the HIV/AIDS pandemic.
As with the performance data for 2000, USAID’s 2002 Budget Justification provided more detailed indicators of planned future performance than the Performance Overview. However, the indicators are again disaggregated to the mission level. The lack of meaningful data aggregation—whether to the regional or agency level—of HIV/AIDS performance indicators reported in the Performance Overview and Budget Justification limits their value for determining progress toward this selected outcome. We understand that missions must operate HIV/AIDS programs as they fit with the needs of the host nation and that some USAID activities are not conducive to quantitative measurement. However, there are methods of data aggregation, such as showing the number of country missions that achieved planned targets for any particular indicator common among missions, that provide more meaningful assessments of whether progress is being made toward the agency’s goals and objectives.
section_title: Natural and Man-Made Disaster
paragraphs: USAID reported it made progress toward achieving its outcome of saving lives and reducing suffering associated with natural or man-made disasters during 2000. However, the Performance Overview does not clearly articulate the agency’s actual progress and relies on 1999 performance results and mission self-assessments. Moreover, fiscal year 2000 data reported in the Budget Justification were not aggregated to summarize regional or agency level performance.
USAID supports this outcome around the world through a total of 31 strategic objectives in 24 operating units. USAID stated performance expectations were met or exceeded for 87 percent of operating units’ strategic objectives. However, the agency based this determination on 1999 operating unit self-assessments, without explaining how these self- assessments were conducted. The overview also discussed USAID’s humanitarian assistance trends by regions, using indicators such as crude mortality rate; prevalence of acute malnutrition in children under age 5; number of refugees and internally displaced persons; and changes in the freedom, civil liberties, and political rights classification of post-conflict countries.  Data for these indicators also represented 1999 performance results.
USAID used the operating units’ strategic objectives and local performance indicators to report progress in the Budget Justification. The performance indicators are measurable and provide a reasonable assessment of USAID’s specific efforts. For example, a strategic goal of USAID’s program in El Salvador is to reduce the vulnerability of the rural poor to natural disasters in target areas. One of the performance measures is the number of community members trained in disaster preparedness. USAID’s target for 2000 was to train 750 community members. The actual number for fiscal year 2000 was 2,313.
The Performance Overview does not assess the impact of its fiscal year 2000 performance on the likelihood of achieving its fiscal year 2001 goals. It does include a discussion of future plans that describes a number of recommended actions that, based on its past experience, the agency intends to implement to improve its capacity to respond to disasters. In addition, the Budget Justification provides specific details for individual countries’ strategic objectives. For example, for USAID’s program in the Dominican Republic, the Budget Justification states that its Hurricane Georges Reconstruction objective is meeting expectations and should achieve all objectives by December 31, 2001.
In March 2001, we testified that USAID’s disaster recovery assistance program in the wake of hurricanes Mitch and Georges has made progress.After some initial start-up problems, most activities are scheduled for completion on or before December 31, 2001, as USAID and congressional staff had informally agreed. In its most recent quarterly report for the period ending June 30, 2001, USAID reported that it and the other U.S. agencies involved in the hurricane assistance had spent about $413 million, or 67 percent, of the disaster recovery funding. With some variance by country, USAID and the other U.S. departments and agencies involved have used the disaster recovery assistance to bring about economic recovery, improve public health and access to education, provide permanent housing for displaced families, and improve disaster mitigation and preparedness. To achieve these broad objectives, USAID funded infrastructure construction and repair, technical assistance and training, loans for farmers and small businesses, and some commodities.
The Performance Overview provides a general discussion of USAID’s plans for 2002, stating that the agency’s disaster relief, based on the 1999 self-assessments of performance from the operating units, are on track and that no major changes to the strategic framework are expected. USAID further reports that it anticipates responding to the same levels of crises in the next year but that it may be called upon to provide increasing levels of assistance. The Performance Overview includes a general discussion of the participation of other U.S. and international agencies. It specifically mentions USAID’s relationship with United Nations agencies, particularly the World Health Organization and the United Nations Administrative Coordinating Committee/Subcommittee on Nutrition. The overview states that the committee/subcommittee is a coordinating mechanism for exchanging information and technical guidance on nutrition and that it assists USAID in monitoring the performance goals on nutrition status of children under age 5 and crude mortality rate. The Budget Justification also discusses USAID’s collaboration with other agencies for specific strategic objectives.
USAID reports that during fiscal year 2000, it began an evaluation of internal responses to emergencies and transitions with a special focus on complex emergency transitions, but it did not provide specific details. USAID reported sharing lessons learned across the agency and with implementing partners. The overview addresses some human capital issues in its discussion of recommended actions needed to strengthen USAID’s capacity to respond to complex emergencies. For example, USAID plans to develop a reserve corps of nondirect-hire workers to supplement direct-hire humanitarian workers and establish a career ladder for its humanitarian response professionals.
section_title: Comparison of USAID’s Fiscal Year 2000 Performance Report and Fiscal Year 2002 Performance Plan With the Prior Year Report and Plan for Selected Key Outcomes
paragraphs: USAID’s revised approach to performance planning and reporting, which uses the Performance Overview for discussing general performance and future expected results, and the Budget Justification for presenting detailed data on operating unit performance, have advantages and disadvantages. Reporting progress against the operating units’ strategic objectives gives a clearer picture of how the agency’s activities contribute to performance goals. However, USAID acknowledges that the main drawback to the reporting change is that there are many different objectives and performance indicators and that the measures and objectives are based on programs that are tailored to local needs. Another problem is that the Performance Overview for 2000 contains limited and out-of-date performance data, making it difficult to assess USAID’s progress. Moreover, the overview does not consistently provide explanations for unmet goals or strategies for achieving them in the future. The Budget Justification provides more detailed fiscal year 2000 performance information and fiscal year 2002 plans for individual countries and regional offices and bureaus.
USAID’s decision to use actual operating units’ strategic objectives as the agency’s performance goals responds to past criticisms of its use of broad development performance indicators. For example, in June 2000, we reported that indicators of economic growth could not realistically serve as measures of USAID’s specific efforts. We further stated, and USAID acknowledged, that although its assistance can contribute to economic growth, many other factors determine the actual level of growth in a country. However, by disaggregating performance planning and reporting data to the operating unit level, USAID has made it more difficult to assess progress toward achieving agency objectives and goals.
section_title: USAID’s Efforts to Address Its Major Management Challenges Identified by GAO
paragraphs: We have identified two governmentwide high-risk areas: human capital and information security. Regarding human capital, we found that USAID’s Performance Overview discussed performance results of efforts in 2000 to resolve strategic human resource issues. The indicators for this area were objective and measurable. The Performance Overview also discussed two performance goals for 2002 regarding strategic human capital management—strengthening human capital resources planning capabilities and addressing skill shortages. These goals and their associated indicators are objective and measurable. USAID also reported meeting two key milestones to support information security challenges management and technology it achieved in 2000, but the agency did not state whether these were all the performance targets for the year. USAID also listed two performance indicators related to information security for 2002—the security and integrity of the legacy core financial management systems enhanced and federal certification of the general control environment and system security for half of the agency’s missions obtained. Both indicators are objective and measurable. While the Performance Overview discussed achievements and planned performance for strategic human capital and information security on an agency level, it generally did not discuss them within the selected key outcomes.
In January 2001, we reported on one additional management challenge specific to the agency. USAID faces challenges in developing reliable performance measures and accurately reporting the results of its programs. Our past work, and that of the USAID Inspector General, has identified a number of problems with the annual results data that USAID’s operating units have been reporting. For example, in commenting on USAID’s fiscal year 2000 performance plan, we noted that the plan provided limited confidence that the agency had the capacity to obtain credible, results-oriented program performance information. In addition, a 1999 Inspector General review of data prepared by 18 missions for 1996 showed problems with the results reported for 83 percent, or 252, of the 302 indicators. USAID has acknowledged that performance measurement is a challenge. In the Performance Overview, the agency reported the results of a number of efforts it undertook in 2000 to improve the quality and availability of performance data, most of which were objective and measurable. USAID also provided three performance goals and indicators for 2002 to continue addressing this challenge. However, the goals and indicators were general in scope, not readily measurable, and not well explained.
In June 2000, we reported on two additional challenges related to financial management: USAID had not implemented an integrated financial management system, and the agency’s financial information was unreliable. In the Performance Overview, USAID stated it initiated a comprehensive study of its financial management operations worldwide, begun transitioning to a new accounting system, and begun configuring a commercial financial software package for the agency. In 2002, USAID plans to install a fully operational, secure, core financial system compliant with federal requirements and that will integrate worldwide financial operations.
section_title: Conclusions
paragraphs: USAID has made an attempt to address past criticisms of its performance indicators. For example, many indicators in past years were too broad to demonstrate the agency’s actual contribution toward meeting performance goals. This year’s approach sought to tie the agency’s efforts more closely to the agency’s desired outcomes by reporting performance data at the operating unit level. By using this approach, however, USAID did not adequately demonstrate progress toward the agency’s selected key outcomes in its performance reporting documents. Although USAID provided aggregate data of the number of strategic objectives met under an outcome area, additional information summarizing results at the unit level and linking it to agency goals is needed to help assess overall progress. For example, under economic development activities, it would have been useful for USAID to provide information on the number of units meeting performance targets related to the agency objective of expanding and strengthening critical private markets and associated activities, such as developing private enterprises.
section_title: Recommendation
paragraphs: We recommend that the Administrator of USAID take actions to provide clearer evidence of progress toward achieving agency outcomes. The actions should include reporting aggregated performance results of agency objectives and the activities under them. This aggregation should be sufficient to provide a clear linkage between overall statements of progress found in the Performance Overview and unit level data presented in the Budget Justification.
section_title: Scope and Methodology
paragraphs: As agreed, our evaluation was generally based on the requirements of GPRA, the Reports Consolidation Act of 2000, guidance to agencies from the Office of Management and Budget (OMB) for developing performance plans and reports (OMB Circular A-11, part 2), previous reports and evaluations by us and others, our knowledge of USAID’s operations and programs, our identification of best practices concerning performance planning and reporting, and our observations on USAID’s other related GPRA efforts. We also discussed our review with agency officials at USAID and OMB, and with the USAID Office of Inspector General. The agency outcomes that were used as the basis for our review were identified by the Ranking Minority Member of the Senate Committee on Governmental Affairs as important mission areas for the agency and do not necessarily reflect the outcomes for all of USAID’s programs or activities. We did not independently verify the information contained in the Performance Overview or Budget Justification, although we did draw from our other work in assessing the validity, reliability, and timeliness of USAID’s performance data. We conducted our review from April 2001 through July 2001 in accordance with generally accepted government auditing standards.
section_title: Agency Comments
paragraphs: We received oral comments on a draft of this report from the Director, Center for Development Information and Evaluation and other USAID officials who agreed with our general recommendation that the agency take actions to provide clearer evidence of progress toward achieving outcomes.  In addition, the officials agreed in principle that aggregation of performance data would allow a better understanding of agency progress. However, the officials noted that the complexity of the issues USAID addresses, the long-term sustained efforts needed to affect positive change, and the difficult environments where the agency works can lead to differences in programs between countries.  As a result, they were concerned that many outcome and some output measures were not conducive to aggregation. We modified our recommendation to reflect the agency’s concerns.
In response to our statement that USAID failed to report on HIV/AIDS prevalence and condom sales, USAID officials stated that these data have become available since the issuance of the Performance Overview and Budget Justification.  In addition, they noted that under monitoring and evaluation activities for USAID’s expanded response to HIV/AIDS, the agency is developing a template for annual HIV/AIDS performance reports to help guide the field missions. This template will show not only the topics and order of discussion in the report, but also the data needs and sources to inform the discussion. USAID officials said this monitoring and evaluation activity could become a model for agencywide performance planning and reporting.  USAID officials provided technical comments that we incorporated as appropriate.
As arranged with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after the date of this letter. At that time, we will send copies to appropriate congressional committees; the Administrator, U.S. Agency for International Development; and the Director, Office of Management and Budget. Copies will also be made available to others upon request.
If you or your staff have any questions, please call me at (202) 512-4128. Key contributors to this report are listed in appendix II.
section_title: Appendix I: Observations on the U.S. Agency for International Development’s Efforts to Address Its Major Management Challenges
paragraphs: The following table identifies the major management challenges confronting the U.S. Agency for International Development (USAID), which include the governmentwide high-risk areas of strategic human capital management and information security. The first column lists the challenges identified by our office, USAID’s Inspector General, and the agency itself. The second column discusses progress USAID made in resolving its challenges, based on its FY 2000 Performance Overview. The third column discusses the extent to which USAID’s fiscal year 2002 performance plan includes performance goals and measures to address the challenges identified by us and USAID’s Inspector General. We found that USAID reported progress during 2000 in resolving most of its challenges except for three: (1) the reconciliation of financial management information, (2) the agency’s broad and changing mandate, and (3) strengthening of collaboration with partners and stakeholders. Of the agency’s 12 major management challenges, its performance plan had (1) goals and measures directly related to 7 of the challenges; (2) no goals and measures related to 2 of the challenges but discussed strategies to address them; and (3) no goals, measures, or strategies to address 3 of the challenges.
section_title: Appendix II: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to Mr. Suda, Shawket Ahmed, David G. Bernet, Janey Cohen, Kirk Daubenspeck, Patrick Dugan, Audrey Solis, Laverne Tharpes and Tom Zingale made key contributions to this report.