Public Financial Management: Improvements Needed in USAID's and Treasury's Monitoring and Evaluation Efforts
Released: 2012-09-13T00:00:00

Letter

Background

PFM Focuses on a Country’s Budget Cycle

The broad objectives of PFM are to achieve overall fiscal discipline, allocation of resources to priority needs, and efficient and effective allocation of public services, according to OECD. While donors may use different definitions of PFM, most definitions focus on a country’s budget cycle, the process used to manage public resources. The budget cycle centers around four main phases: (1) budget formulation, (2) budget execution, (3) accounting and reporting, and (4) external oversight (see fig. 1). OECD states that PFM includes all components of a country’s budget cycle.

A budget cycle starts with the budget formulation process, in which the government, often with legislative oversight, plans for the use of the coming year’s resources in accordance with policy priorities. After the government approves the budget and the new fiscal year begins, programming agencies and the ministry of finance, or appropriate entity, are responsible for executing the budget. They use the resources allocated to them for items such as salaries for public servants, operating costs for their offices, and goods and services delivered to their beneficiaries. The ministry of finance, or equivalent, manages the flow of funds and monitors and makes in-year adjustments to help ensure compliance with the budget and PFM rules. Throughout the fiscal year, each programming agency is to account for and record its expenditures. The ministry of finance centrally consolidates these accounts. At the end of the fiscal year, the ministry of finance, or equivalent, issues an accounting report that demonstrates how the budget was implemented. External or independent entities, such as a country’s supreme audit institution, may review this report. The audit institution reviews the government’s revenue collection and spending and issues its own statement on the execution of the budget and the strength of the PFM systems. In many countries, the institution presents this audit report to an appropriate government entity for further scrutiny and action.

PFM processes involve a number of governmental entities. While the ministry of finance is generally the focus of a country’s PFM system, PFM extends to all public ministries that are charged with delivering services or have spending authority. Civil society, donors, and oversight institutions, such as a country’s supreme audit institution, also help to ensure the proper management of public funds through external scrutiny and review.

Assessment Tools Highlight Both Improvements and Weaknesses in Developing Countries’ PFM Systems

Broad PFM assessments conducted by international organizations report improvements in countries’ PFM systems, as well as continued weaknesses. Several international organizations, including the Public Expenditure and Financial Accountability (PEFA) program, World Bank, International Budget Partnership, and Transparency International, have developed assessment tools to assess broad aspects of PFM systems.These assessments have highlighted varying levels of progress in improving PFM systems around the world, and the last two have identified relatively low levels of transparency in many countries’ budgets and high levels of corruption, respectively. More specifically:

PEFA: A 2010 monitoring report on the PEFA program, released by the PEFA Secretariat, found that while PFM systems were improving overall, progress varied among elements of countries’ systems. The PEFA secretariat’s analysis, based on a comparison of 33 repeat PEFA assessments from 2005 through 2010, showed that more countries had a higher number of improved versus worsened scores, indicating a broad trend of PFM improvement across the countries surveyed. According to the analysis, country actions taken to strengthen PFM features in the earlier stages of the budget cycle, such as planning, are more likely to improve or maintain a high score than actions taken later in the budget cycle, such as control and oversight of actual spending.

World Bank: According to our analysis of the World Bank’s quality of budgetary and financial management indicator scores from 2005 through 2010, slightly more than one-third of the countries assessed showed improvements in the quality of their PFM systems, while one- quarter of the countries showed a worsening in the quality of their PFM systems.

International Budget Partnership: In its 2010 open budget survey of 94 countries, the International Budget Partnership concluded that the state of budget transparency was poor. Only about 21 percent of the countries surveyed had open budgets while 44 percent of countries provided limited to no budget information. Nonetheless, the survey found the trend toward open budgets was favorable based on substantial improvements in budget transparency, especially among countries that had provided little information in the past. Some of these governments achieved improvements by simply making budget documents available on their websites.

Transparency International: The organization’s Corruption Perceptions Index showed that nearly three quarters of the 178 countries in the index scored below 5, on a scale of 10 (very clean) to 0 (highly corrupt), indicating what it categorizes as a serious corruption problem.

For further information on each organization’s assessment tools and selected results, including the percentile rankings of our six case study countries in selected PFM diagnostic tools, see appendix II.

International Attention to Strong and Effective PFM Systems Has Increased since 2003

Donors and recipient governments have increased their attention to strengthening PFM systems, recognizing that strong and effective PFM systems underpin fiscal and macroeconomic stability, guide the allocation of public resources to national priorities, support the efficient delivery of services for poverty reduction and economic development, and make possible the transparency and scrutiny of public funds. In 2003, OECD’s Development Assistance Committee established a Working Party on Aid Effectiveness, which has played a critical role in establishing an initial international donor coordination framework and setting goals for donors and aid recipient countries to strengthen PFM. The working party has sponsored four global development forums since 2003. The second forum, held in Paris in 2005, resulted in the Paris Declaration on Aid Effectiveness which included broad commitments by recipient governments and donors to strengthen PFM systems and use those systems as appropriate. More than 100 countries and aid agencies, including the United States, endorsed the declaration. By signing the declaration, recipient governments made clear commitments to strengthen their systems to the maximum extent possible, and donor governments made clear commitments to use those systems wherever possible. These commitments were renewed and refined in the subsequent forum in Accra, Ghana, in 2008. The most recent forum was held in Bussan, South Korea, in 2011. (See table 1.)

USAID and Treasury are the two main U.S. agencies providing PFM- related assistance. Combined, the two agencies have PFM-related projects in 70 countries in all regions of the world, as shown in figure 2. State also conducts some PFM-related activities, although it has not funded programs. See appendix III for details on State’s PFM activities.

USAID provides PFM capacity-building activities through its development programs and is seeking to provide more of its assistance through recipient countries’ financial systems. Capacity-building activities to strengthen PFM systems are typically part of broader democracy and governance (DG) or economic growth (EG) programs. PFM activities that are included as components of DG programs typically address the areas of legislative function and processes, public sector executive function, local government decentralization, and anticorruption reforms. PFM activities included as components of EG programs typically address fiscal and monetary policy issues. USAID has identified DG and EG programs with PFM components in over 60 countries since 2007. However, according to a USAID official, USAID cannot determine the total funding for PFM activities because it does not collect data at a sufficiently detailed level to precisely identify PFM activities. Therefore, the official reported that USAID is unable to separate PFM assistance from other assistance in larger DG and EG development programs. In 2011, total funding for DG programs was $1.7 billion, and for EG programs, $4.2 billion.appendix IV.

For illustrative examples of USAID PFM-related projects, see In addition to PFM capacity-building programs, USAID is seeking to increase the use of recipient country PFM systems to deliver assistance. As part of its implementation and procurement reforms under its institutional reform agenda, known as USAID Forward, USAID has announced an agency target of obligating 30 percent of its annual assistance through local systems, including both partner country PFM systems and local not-for-profit and for-profit organizations, by fiscal year 2015.

Treasury’s OTA provides technical assistance through advisors who work in-country within the finance ministry, the central bank, or other government entities. OTA’s program consists of five core areas, as follows:

Budget and Financial Accountability helps foreign governments reform and strengthen their PFM systems in order to promote control, accountability, and transparency over resources, and to improve a country’s overall financial condition.

Banking and Financial Services promotes strong financial sectors in which institutions are well regulated, stable, and accessible; serve as efficient intermediaries between savers and investors; and are resistant to criminal activity.

Government Debt Issuance and Management helps host countries implement sound public debt management practices and develop markets through which the government can finance itself.

Economic Crimes helps counterpart governments build their capacity to prevent, detect, investigate, and prosecute complex financial crimes.

Revenue Policy and Administration provides assistance to ministries of finance and other relevant organizations that strengthens their ability to serve the country and its people through the efficient and responsible collection of revenues.

OTA provides both long-term resident and intermittent advisors. Long- term resident advisors provide advice and training to ministers of finance, central bank governors, and other government officials. Country engagement typically lasts between 2 and 10 years, according to an OTA official. Intermittent advisors provide highly specialized technical assistance in support of long-term projects, projects requiring several different specialties, and projects of short duration. According to an OTA official, they currently have about 70 advisors in roughly 50 countries. In 2011, OTA’s total funding was $44.5 million, with $25.5 million in direct OTA appropriations and $19 million in transfers from other agencies, including State, USAID, and the Millennium Challenge Corporation. For illustrative examples of OTA technical assistance projects, see appendix IV.

USAID’s and Treasury’s Processes for Developing PFM Programs Involve Consultation with Stakeholders and Rely on Assessments of Host-Country Systems

USAID’s Processes

USAID is implementing new processes for developing programs that reflect new agency reform priorities to increase the use of country systems to deliver U.S. assistance. USAID’s work in PFM has traditionally involved capacity building under broader programs designed to improve fiscal management and promote good governance. According to agency officials, USAID has increased its attention to PFM issues. USAID’s new country strategy development and project design processes include various analyses and assessments that may identify opportunities to strengthen and use countries’ PFM systems, as prioritized by USAID’s reform agenda. In these new processes, USAID may identify the need for PFM assistance during countrywide development assessments or through other required assessments. Furthermore, according to USAID guidance, country offices are to develop these assistance programs in collaboration with country stakeholders throughout the program development process. USAID’s new processes are similar in structure and approach to prior processes, but, according to USAID, aim to incorporate more analytical rigor at all stages of the strategic planning framework so that USAID’s efforts are better aligned with the recipient country’s development efforts.

USAID may identify the need for PFM assistance during the country strategy development process. According to USAID, most country offices are required to develop a Country Development Cooperation Strategy, a 5-year strategy document, by the end of fiscal year 2013. USAID’s latest draft of the guidance for developing the country strategy, released in September 2011, states that the strategy should demonstrate how it is integrating the goals of USAID’s assistance reform effort, USAID Forward, such as working through host-country systems and developing the capacity of civil society and private sector partners. In developing the country strategy document, the USAID country office is to consult with various country stakeholders and conduct several assessments to understand the development context and develop goals and objectives. According to the guidance, the country office is required to develop the strategy document with a focus on maximizing the impact of USAID resources and build the capacity of specific institutions and related governance systems at the national, regional, and local levels. For example, if a USAID country office determines that a nontransparent and inefficient financial system is a key obstacle to economic growth, the country office is to work with the host government to improve its capacity for sound financial management and equitable allocation of resources. According to USAID officials, one indicator of need for a PFM project would be an assessment that the country has a major fiscal imbalance that needs to be corrected by a combination of increased revenue mobilization or reduction of budget expenditures. Another indicator would be a determination, arrived at through repeated reporting on corruption in the media, or internal and external publications, and supported by stakeholder interviews, that anticorruption programs would improve PFM.

The goals and objectives outlined in the country strategy are to provide the basis for project design, monitoring, and evaluation. As of June 2012, USAID stated it had approved 15 country strategies, and 73 USAID country offices are scheduled to complete a strategy by October 2013. See figure 3 for highlights of the elements of USAID’s new strategy and project design processes that could identify the need for PFM capacity building.

According to the new USAID guidance on project design, after USAID approves the strategy document and identifies the need for PFM-related assistance, USAID country offices are to begin the project design phase. This phase is to begin with the formation of a design team that is responsible for the project’s development from planning to implementation. Overall, the project design phase is to comprise three stages, as follows:

Conceptual stage. During the conceptual stage, the project design team is to conduct stakeholder outreach, assessments, analysis, and implementation planning. This stage results in a concept paper, which provides a summary of a proposed project that country-office management can review to assess how the project aligns with the country strategy, the likelihood of success, and the assumptions underlying project success. After USAID management approves the concept paper, the design team transitions to the analytical stage.

Analytical stage. During the analytical stage, the design team seeks to understand the problem or constraints identified during the conceptual stage, and identify and assess critical assumptions. The team is to conduct a series of targeted assessments, including required gender, environmental, and sustainability analyses and other social, political, and institutional analyses. Of these analyses, we identified two that pertain directly to PFM systems, as follows:

Sustainability analysis. The sustainability analysis assesses the partner government’s ability to manage the continuation of the project after the project has concluded. According to USAID guidance, to build sustainability into a project, the design should consider how the country office will increase the skills and capacity of local stakeholders involved in maintaining development gains after the project ends—as well as how USAID will ensure that relevant activities or services are gradually tied to sustainable financing models through private sector participation or through sustainable, publicly managed arrangements and government processes. Institutional analysis. An institutional analysis is an in-depth assessment of the local institutions and systems most critical to the implementation of the project’s development interventions, including an assessment of the quality of their leadership, structure, and staff; and identification of their administrative and financial management strengths and weaknesses. Using the analysis, USAID is to develop a plan for project activities that are necessary and sufficient to bring these institutions up to the level of performance or engagement as partners appropriate for their roles in the project’s implementation and their eligibility for direct USAID funding.

According to USAID guidance, the analytical stage results in a project authorization document that will be the basis for project implementation, adaptation, and evaluation, which includes a summary of the analyses that underlie the rationale for the project design.

Approval stage. A successful project design results in an approved project authorization, which enables a project to move from the planning stage to implementation. The project authorization sets out the purpose and duration of the project, defines fundamental terms and conditions of the assistance when a partner country agreement is anticipated, and approves an overall total budget level, subject to the availability of funds.

In addition to the above analyses, USAID has developed a PFM risk- assessment framework (PFMRAF) to measure the fiduciary risk, or the risk of funds being misspent or mismanaged, when USAID plans to provide aid through the country’s finance systems. USAID guidance commits country offices to offer, if appropriate, a USAID assessment of partner country PFM systems with the goal of providing funding for project implementation through the use of those systems. If the offer is accepted, the assessment must be carried out using the PFMRAF. Whenever possible, USAID should begin conducting the PFMRAF during the conceptual stage of the project design process. The PFMRAF consists of the following five stages: 1. The Rapid Appraisal provides a high-level snapshot of country-level fiduciary risks associated with the use of partner country PFM systems and helps inform decisions on whether to undertake a more rigorous, formal risk assessment. 2. The Risk Assessment, Analysis, Mitigation, and the Approval for Use of Partner Country Systems establishes the baseline level of risk corresponding to contemplated funding levels and identifies vulnerabilities of the partner country PFM sector in which USAID is considering use of the system for project implementation. USAID determines whether any systemic risk can be reasonably mitigated and, if so, what kind of mitigating measures might be introduced to reduce the risk. USAID establishes an accountability framework with a set of conditions that would, if complied with, constitute formal approval for the use of a partner country PFM system. 3. The Project Design, Approval, Designation of Responsibilities, and Selection of the Funding Mechanism incorporates the approval of use of country systems into the project design, includes risk mitigation measures, such as capacity-building technical assistance, concurrent audits, and disbursements in tranches, as appropriate, and the appropriate funding mechanism. 4. Negotiating and Preparing the Bilateral Project Agreement with the Partner Country Government involves the preparation of the bilateral agreement in accordance with the risk mitigation measures outlined in the approval for use of partner country systems and in collaboration with the partner country government. 5. Implementation, Monitoring, and Evaluation occurs once both countries agree to the bilateral agreement and as outlined in the project design.

According to USAID, the country office is to incorporate recommendations on actions to mitigate identified risks to providing assistance directly to the host government into various stages of the project design. According to one USAID official, the country office’s decision to implement the recommended steps for mitigating risk may depend on the availability of funds. According to USAID, if risks cannot be mitigated, USAID is not to deliver assistance through the partner country’s financial systems. Mitigation steps may include adding a PFM component to a project, such as technical assistance to improve an aspect of the financial system. As of June 2012, USAID had completed the rapid assessment in 20 countries and was in the process of finalizing the report for an additional 12. For 4 of the countries for which USAID had completed the rapid assessment, USAID had decided to either delay or not to proceed to stage 2 for various reasons, including funding cuts and the political situation in the country. For the second stage of the PFMRAF, USAID had completed the risk assessment for 3 countries and was in the process of conducting the assessment for 11 others. USAID had completed four stages of the PFMRAF for 3 countries. Table 2 contains a summary of the status of countries where USAID has begun the PFMRAF process.

Stage 2 on hold or no plan to continue to stage 2 in four of these countries.

Treasury’s Processes

Treasury OTA’s processes for developing PFM programs involve collaboration with country government officials whom Treasury has determined have a strong commitment to reform and are seeking to develop strong PFM, as well as OTA’s assessment of the country system. OTA receives requests for assistance from foreign governments, other U.S. government agencies, U.S. embassies, international organizations, and OTA advisors and other donors already working in countries on other projects. According to OTA officials, OTA receives two to three requests for assistance per month, but due to the agency’s limited financial resources and the financial commitment required for each project, it selectively chooses its projects. OTA’s principles for providing assistance are as follows: work with partners committed to ownership of their reform; advocate self-reliance by providing countries with knowledge and skills required to generate and manage their own resources and reduce dependence on international aid; and work with ministry staff daily to build capacity through mentoring and on-the-job training on PFM practices.

After receiving the request, OTA begins a preliminary assessment process to identify weaknesses and areas for assistance, according to OTA officials. The preliminary assessment is primarily a review of available information on the country’s PFM system, which may include information from the U.S. embassy, as well as relevant country reports and assessments prepared by international financial institutions, other bilateral assistance providers, and nongovernmental organizations. After reviewing documents, if it decides to proceed with the assessment, OTA completes a more in-depth, in-country assessment of PFM, which consists of meetings with U.S. embassy staff, other donors, and relevant ministry officials, including the minister of finance or the head of the central bank, as well as high-level policy staff and mid-level supervisors. According to OTA officials, the purpose of the in-country assessment is to review the structural issues related to the country system and determine whether the country has dedicated staff committed to working toward their reform efforts. At any point during the assessment process, OTA may determine that OTA assistance is not suitable for the country’s needs. OTA officials told us that in one such case, government officials requested that OTA manage their budget for them, which is against OTA’s principles because doing so would not promote self-reliance or help the country develop the capacity to generate and better manage its own revenues. Finally, OTA officials told us that their decision on whether to provide assistance depends on whether sufficient funding is available.

The in-country assessment typically results in a draft of official terms of reference, which identifies weaknesses to be addressed during OTA’s engagement. According to OTA officials, the terms of reference describes the broad goals of the project and represents agreement between Treasury and the host-country counterpart, the ministry staff with whom the advisor will work on a daily basis. For example, the 2010 terms of reference for a budget project in Honduras identified the following four agreed-upon areas for assistance: improve operational efficiency and enhance accountability by strengthening the organization of the ministry of finance; move toward compliance with International Public Sector Accounting enhance capacity to conduct fiscal analysis and produce more reliable macroeconomic estimates and revenue and expenditure projections; and conduct workshops on basic finance methods and terminology.

The terms of reference also identifies the advisor’s host-country counterparts and agreements regarding each party’s responsibilities. According to OTA officials, after OTA fully vets a project and secures funding, the parties finalize and sign the terms of reference. However, OTA and its counterparts may seek to revise the terms of reference if the central purpose of the technical assistance project changes or during times of transition, such as when a finance minister changes. In Honduras, the 2010 terms of reference replaced an existing OTA project following a gap in OTA assistance during a political crisis in the Honduran government in 2009.

The OTA advisor assigned to the country uses the broad goals expressed in the terms of reference to develop a work plan. The work plan contains specific objectives, milestones, and planned completion dates designed to work toward goals agreed upon in the terms of reference, and the plan must be developed within 60 days of the advisor arriving in-country. Each work plan covers a 1-year period and is the primary basis for regular monthly progress reports to OTA headquarters. OTA considers work plans, and the monthly reports on which they are based, to be dynamic documents that reflect the project’s progress in real time, and advisors can and should change or modify the work plan during the course of a project as appropriate and in consultation with the ministry staff and OTA management. Moreover, according to OTA officials, a 1-week in-country assessment may not provide all of the information needed and may require that the advisor rewrite the plan after a few months in the country. While the work plan is considered a management tool to monitor projects, OTA officials told us that in recent years it has also become a joint document between the OTA advisor and ministry staff as a shared tool to monitor and discuss progress.

USAID and Treasury Use Certain Processes to Monitor and Evaluate PFM-Related Programs, but Implementation Has Some Weaknesses

USAID’s Processes

USAID Uses Multiple Tools to Monitor Its PFM Assistance but Lacks a Process for Tracking Use of Country Systems

To monitor PFM-related programs, USAID develops performance management plans, reviews periodic progress reports, and conducts site visits, among other activities, but reviews of USAID programs have identified agency-wide weaknesses in implementation, including using unreliable baseline data and inaccurate reporting of results. USAID develops the monitoring and evaluation framework during the project design, and it generally is to include the following:

A performance management plan: This is a tool to plan and manage the process of monitoring, evaluating, and reporting progress toward achieving the project’s development objectives. The performance management plan includes performance indicators and targets that link to the project objectives.

A work plan: This specifies activities to be undertaken and the proposed schedule for these activities during the life of the project.

Periodic progress reports: USAID assistance agreements with nongovernmental organizations generally also require implementing partners to submit periodic progress reports, the frequency of which vary depending on the assistance agreement, but these reports may not be required more frequently than quarterly or less frequently than annually. The progress reports should generally contain a comparison of actual accomplishments with the goals and objectives established for the period and reasons why established goals were not met, if appropriate. Implementing partners should immediately notify USAID of developments that have a significant impact on the award- supported activities.

Site visits: USAID guidance states that site visits are conducted as needed by the technical officers, who are responsible for monitoring.

USAID works with the implementing partner to produce and approve the work plan and the performance management plan. USAID’s technical officer approves the plans. USAID guidance also requires staff to conduct at least one portfolio review each year that covers all activities included in their various programs. These reviews determine, among other things, whether the activities are achieving the desired results.

Prior reviews of USAID programs have identified challenges in the agency’s implementation of its monitoring processes across many types of programs, including DG and EG programs. In its fiscal year 2011 memo on management and performance challenges, USAID’s Inspector General identified specific monitoring-related weaknesses as one of USAID’s most serious management and performance challenges. The USAID Inspector General reported problems with assistance planning in 25 out of 80 performance audits conducted in fiscal year 2011. Assistance planning is important because it provides the means for program implementers to track progress toward program objectives and helps to ensure that USAID assistance programs achieve planned results. In addition, 37 of the 80 audit reports the Office of the Inspector General issued in fiscal year 2011 identified cases in which USAID operating units or their implementing partners reported misstated, unsupported, or unvalidated results. In recent reports, we also identified deficiencies in USAID’s monitoring practices, including the lack of an integrated plan for monitoring and evaluating nonemergency food aid, monitoring practices that do not correspond to agency performance guidelines, difficulties in developing meaningful outcome indicators related to building trade capacity, undocumented site visits for assistance programs in Burma, and lack of performance targets and baseline data for indicators related to PFM efforts in Afghanistan.

Moreover, in our review of the fiscal year 2011 and 2012 Inspector General’s audits of USAID’s DG and EG programs, we found monitoring weaknesses cited in 20 of the 32 audit reports. monitoring weaknesses, including unreliable or nonexistent baseline data; performance data weaknesses, such as results that were not reported, lack of data verification, or inaccurate reporting of data; lack of a current performance management plan; and inadequate monitoring of program activities, including lack of regular submission of progress reports.

With regard to our three case study countries, we found that the country offices generally applied USAID’s monitoring processes in all three PFM- related programs; however, we did find some conditions that could make monitoring more difficult, as described below. Our review was not intended to be comprehensive or applicable to all USAID PFM-related programs.

USAID’s DG and EG offices are the primary offices providing PFM-related assistance.

Performance management plans: All three programs were clearly defined in terms of overall objectives, project objectives, tasks, expected results or outcomes, and required plans and reports. All three programs—the Kosovo Growth and Fiscal Stability Initiative, the Peru ProDecentralization Project, and Liberia’s Governance and Economic Management Support Program—had performance management plans containing objectives and indicators. Two of the three performance management plans also included targets. However, the targets for Liberia’s program had not been determined 9 months into the program.

Work plans: All three programs also had work plans that specified tasks and timelines. However, Liberia’s Governance and Economic Management Support Program initially operated without an approved work plan, although it did establish an action plan to guide its activities.

Periodic progress reports: Assistance agreements for all three USAID programs required quarterly progress reports, and the implementing partners submitted all reports according to agreed-upon time frames. In our review of the three programs, we found that the language used to describe the objectives and results in the quarterly progress reports did not correspond to the language in the work plans, which could make it difficult to track progress against the work plans’ objectives. For example, the quarterly reports for the Kosovo Growth and Fiscal Stability Initiative Program describe progress on five separate lines of effort under the objective to improve fiscal stewardship, while the statement of work and work plan discuss only three lines of effort under the same objective.

Site visits: USAID officials responsible for monitoring progress on all three projects said that they do not rely solely on progress reports to track progress, and that additional monitoring tools included weekly progress meetings, frequent site visits, and telephone and e-mail communications with implementers. For example, the USAID official in Kosovo reported that he works on a daily basis with the implementing partners who produce the quarterly and annual reports and that they exchange e-mails and phone conversations daily.

See appendix IV for descriptions of USAID PFM-related assistance programs from our case study countries.

In addition to USAID’s reported program monitoring weaknesses, we found that the agency does not have a process to fully identify and measure its use of country systems. USAID has set an agency target of obligating 30 percent of the agency’s annual assistance through local systems, including both partner country PFM systems and local not-for- profit and for-profit organizations, by fiscal year 2015. currently cannot track progress toward its 30 percent target because its accounting system cannot identify the full range of such assistance, which includes a variety of implementing mechanisms from host-country contracting to direct cash transfers. The project data in USAID’s accounting system does not provide sufficiently detailed information, such as the location of the organization receiving assistance, to identify qualifying assistance. According to a senior USAID official, the agency is working on a system whereby it will tag each entity receiving assistance with an identifier, such as government or not-for-profit organization, as well as a vendor location signifying U.S.-based or non-U.S. based. According to the official, this tagging process will facilitate identifying use of country systems. However, USAID’s financial system currently cannot distinguish whether non-U.S. based not-for-profit and for-profit organizations are located in the host country or in a third-party country.

USAID’s process to assess the effectiveness of its PFM-related programs involves independent evaluations, but weaknesses in the agency’s overall evaluation practices have been reported. USAID may use independent external evaluations in the middle or at the end of a program based on the need to inform program decisions. USAID adopted a new evaluation policy in January 2011 and updated its guidance in February 2012 to reflect it. The new evaluation policy requires that all large projects have at least one evaluation and that evaluations use methods that generate the highest-quality and most credible evidence, including experimental The policy also states that 3 percent of program budgets methods.should be devoted to external evaluation and classifies two types of evaluations: performance evaluations, which focus on descriptive and normative questions and other questions pertinent to program design, management, and operational decision making; and impact evaluations, which define a counterfactual (what would have happened had the program not been implemented) to control for external factors and measure the change in a development outcome that is attributable to a defined intervention.

USAID reported that experimental methods generate the strongest evidence for impact evaluations; however, experimental methods and impact evaluations may be difficult to apply to PFM capacity-building efforts. For example, USAID evaluations of PFM programs have highlighted difficulties associated with conducting impact evaluations, including lack of data or resources, and too short a time period to identify impact. A February 2012 USAID report on the first year of implementation of the new evaluation policy noted that USAID had not yet completed an evaluation under the new policy.

USAID adopted the new policy to address a number of weaknesses it had identified in its evaluation practices. USAID reported that the number of evaluations submitted to USAID’s Development Experience Clearinghouse—its main repository for agency evaluations—decreased from nearly 500 in 1994 to approximately 170 in 2009 despite an almost three-fold increase in program dollars managed. Furthermore, according to USAID, the majority of evaluations in recent years relied heavily on anecdotal information and expert opinion, rather than systematically collected evidence. We have also reported weaknesses in USAID evaluation practices in several areas in recent years, including not planning for evaluations or not using evaluation results. Furthermore, the weaknesses in performance indicators that the USAID IG identified in DG and EG programs could create difficulties for evaluating PFM programs. For example, weaknesses in baseline data or performance indicators would make it difficult to use quantitative measures to evaluate these programs.

These evaluations were not subject to USAID’s new evaluation policy, which was issued in January 2011. evaluation of the Economic Management for Stability and Growth program and found that the program improved the government’s institutional capacity in fiscal and monetary policy.did not document its evaluation methodology.

Treasury’s Processes

Treasury’s OTA Uses Various Processes to Monitor PFM- Related Programs

Treasury’s OTA uses various processes to monitor PFM-related programs, including narrative monthly and trip reports, annual quantitative performance measures, and customer feedback surveys, but weaknesses exist in some of its evaluation processes. The work plan is the key document OTA uses to monitor a project. OTA uses monthly reports to document and monitor progress toward the milestones identified in the annual work plan, including changes to milestones or timelines, and to facilitate communication between the resident advisors and headquarters. OTA advisors may also share monthly reports with host-country counterpart institutions, U.S. embassy staff, other Treasury bureaus and offices, and other interested partners such as USAID, the Millennium Challenge Corporation, and relevant international financial institutions to monitor progress and coordinate donor assistance as relevant. In addition to updates on work-plan progress, the monthly reports may also include other information such as activities completed; significant meetings held; and important political, social, and economic developments. OTA also uses monthly reports to track changes made to the work plan agreed to by the advisor and his or her counterpart. For example, an OTA budget project in Cambodia listed implementation of program budgeting as one its primary objectives and improving budget classification as one its secondary objectives. In the monthly reports, the resident advisor documented the need to refocus the reform efforts on budget classification to better align with the Cambodian government’s ongoing PFM reform efforts. Further, in the case of OTA’s 2011 Honduras budget project to implement international public sector accounting standards, the objectives remained unchanged, but milestones were changed in the monthly reports. According to the advisor, some milestones had to be extended due to overly ambitious initial expectations that were purposely established to encourage the ministry staff to undertake reforms. See appendix IV for descriptions of OTA-funded PFM projects in our case study countries.

OTA headquarters also maintains regular communication with its advisors. Senior OTA officials monitor project performance through regular contact with advisors in the field by e-mail, telephone, and site visits. One advisor told us that their director would already be aware of any issues raised in the monthly reports, given the frequency of their communications. Resident advisors also told us that senior Treasury staff visited Cambodia and Honduras to review OTA projects and talk with counterparts. Following site visits, OTA officials are required to prepare trip reports. For example, in a trip report from November 2011, a senior OTA official reported that he met with host-country counterparts to discuss the four current OTA engagements in Cambodia, including the budget project. The visit confirmed the reported difficulties in implementing the government financial management information system as a result of a procurement delay, but also affirmed OTA’s role in its eventual implementation. OTA also uses voluntary customer surveys to collect feedback on OTA projects from the host-country counterpart. For example, a Honduran government official who completed the voluntary customer feedback survey for the budget project indicated that the project met expectations and made a significant contribution by strengthening the host-country staff’s technical capacity through the training sessions that OTA staff offered.

We found that OTA applied its monitoring processes to all three of our case study country projects, but monthly reports lacked some details on progress. We reviewed OTA’s monitoring activities in three PFM-related projects to examine how OTA applied its monitoring process to these projects. Our review was not intended to be comprehensive or applicable to all OTA projects. We found the projects to be clearly defined with specific objectives, milestones, proposed completion dates, and regularly submitted monthly reports. In addition, in all three cases, the resident advisor reported frequent communication on project progress with headquarters. However, in two of the case study countries, Cambodia and South Africa, work plans had major changes in objectives, but only Cambodia documented its change in a monthly report. Additionally, our review found that the level of detail of the monthly report narratives on progress varied, with a few reports having little detail, which could make tracking progress for these specific projects difficult without other communication between the advisor and OTA management. However, as noted above, OTA headquarters maintained frequent communication with advisors in the field.

OTA’s quantitative performance measures have been a useful project- level indicator of performance but have not been a useful indicator of OTA-wide performance because of conceptual problems and errors introduced by OTA when aggregating the performance data. OTA advisors and management score technical assistance projects annually based on project-specific indicators under four main categories. The projects are scored on a scale of one (lowest) to five (highest) for “traction,” or the degree of engagement with host-country counterparts, and “impact,” or the results of project activities that bring about change in counterpart law, systems, processes, and procedures. OTA officials told us that the traction and impact measures also reflect the language and values of OTA, and are useful to management, advisors, and host- country counterparts. The primary purpose of the quantitative performance scores was to respond to an Office of Management and Budget reporting requirement for evaluation data. While OTA reported that it met or exceeded targets for traction and impact set in 2008, in every year from 2009 through 2011, OTA officials acknowledged that the aggregate values associated with its annual goals were of limited value due to lack of comparability across programs and over time. In previous work on the Office of Management and Budget, we highlighted the difficulty of representing program performance with a single rating. Using a single rating can force agencies to simplify more nuanced and complex performance results, a circumstance similar to OTA’s aggregation of traction and impact scores. A senior official told us that OTA, in complying with the OMB requirement, designed the measures to be as useful as possible at the project level. For example, OTA uses the project-level traction and impact measures in setting expectations and discussing progress with both the host-country counterpart and the advisor. In addition, the official reported that OTA is continually looking for additional ways to use the traction and impact scores. One example of how researchers have used PFM performance data is by developing analytical approaches to identify the determinants of strong PFM systems and elements of successful PFM reform efforts, which can help to control for important differences across countries.

Our analysis of OTA performance data found several errors that OTA introduced when aggregating project-level data; these errors collectively raised questions about the reliability of the instrument used to aggregate OTA’s quantitative performance measures and, thus, suggested there may be limitations in its ability to provide insight into performance across OTA projects. OTA uses this instrument to calculate the annual performance averages which are compared against annual targets and provided to OMB and to Congress in annual reports. Our analysis of the spreadsheet containing these data, and limited spot checking of some underlying data for our three case-study countries, identified a number of errors, including those introduced when transcribing data to the spreadsheet. In one instance, OTA listed a performance score under the wrong country, which resulted in inaccurate information for two countries’ projects.

OTA has not yet fully implemented its requirement to conduct independent evaluations—that is, evaluations conducted by someone other than the resident advisor—of its completed technical assistance. OTA guidance for project reporting and documentation includes requirements for end-of-tour reports when a resident advisor leaves and end-of-project reports when a technical assistance project is completed. The end-of-project report is a postproject evaluation whose purpose is to (1) compare accomplishments with the initial objectives of the project and (2) improve the planning and execution of future projects. A senior OTA official said that the requirement for the end-of-tour report was fully implemented and that OTA intended to fully implement the end-of-project report requirement by the end of 2012. According to the guidance, the evaluation must be conducted independently; the report is the responsibility of the associate director and can be delegated to a senior advisor, but it should not be prepared by an advisor who prepared an end-of-tour report. Both OECD guidance and USAID guidance also highlight the importance of independence in evaluation. In contrast to the guidance, OTA provided three end-of-project reports that were conducted by OTA staff who had been involved in providing some of the technical assistance being evaluated. While no comprehensive and fully independent end-of-project evaluation has been conducted, OTA officials described relevant insights from a postproject trip to one country that was undertaken by a senior OTA official in part to better understand the longer-term impact of OTA assistance. During the trip, that official identified some factors that significantly limited the impact of this project, notably a lack of counterparty commitment at the policy level to implement and sustain reforms in key areas identified in the terms of reference.

Conclusions

As more donors, including the United States, seek to provide additional assistance through countries’ systems, the need for strong PFM systems and accountability in recipient countries become even more important to help lessen corruption and ensure countries effectively manage resources. Given USAID’s stated goal of obligating 30 percent of annual assistance through local systems, including partner country PFM systems, by 2015, it must ensure that those systems are functioning properly and transparently. Efforts to strengthen PFM systems can lower the risks that assistance delivered through country systems will be misspent, while also increasing the capacity of the recipient country to effectively manage its own resources. Given concerns about transparency and corruption in many aid-recipient countries, achieving the expected benefits of using recipient countries’ PFM systems may be difficult without concerted efforts by the donors and countries to strengthen these systems. In addition, the risks to using these systems are increased without efforts to strengthen them. To help strengthen these systems, USAID provides developing countries with PFM capacity- building assistance. In light of this assistance, and given the difficulties USAID has experienced in the past with implementing monitoring and evaluation practices, the importance of developing efforts to strengthen PFM systems and effectively monitoring and evaluating these efforts has increased significantly. Previous difficulties USAID has experienced with regard to monitoring, including poor quality of reported data on projects, and the lack of reliable baseline data, could affect USAID’s ability to conduct effective evaluations of these projects, as evaluators will not have access to reliable data. Also, while USAID’s new evaluation policy places greater emphasis on impact evaluations and experimental methods, it may be difficult for USAID to evaluate its PFM capacity- building efforts using these approaches. Finally, for USAID to achieve its target for use of local systems, by fiscal year 2015, it must be able to identify and measure the full extent of assistance that qualifies.

Treasury OTA advisors provide technical assistance targeted at weaknesses in countries’ PFM systems. Although OTA has taken numerous steps to monitor this assistance, errors in its aggregated performance data and the lack of comprehensive postproject evaluations limit OTA’s ability to effectively evaluate its assistance. OTA has adapted a challenging mandate from OMB to create a useful measure of its efforts’ performance for management, advisors, and host-country counterparts; however, conceptual problems with and errors in the aggregated measures undermine the measures’ reliability. With greater confidence in the quality of the data, opportunities exist to better identify patterns in performance across OTA programs, such as economic or institutional factors that influence program performance. These patterns could help OTA better understand the strengths and weaknesses of its assistance programs and make appropriate changes. OTA could also use the quantitative performance measurement system it has developed to experiment and document the results of new approaches. Finally, although OTA guidance recognizes the importance of postproject evaluations by including in its guidance the requirements for an end-of- project evaluation, the agency has yet to enforce this requirement. Without a postproject evaluation OTA may not fully understand results of its technical assistance or be able to apply lessons learned to new projects.

Recommendations for Executive Action

To monitor progress toward USAID’s target to obligate 30 percent of its annual assistance through local systems by 2015, we recommend that the Administrator of USAID direct the appropriate offices to develop a process to reliably identify and track the agency’s use of local systems in all countries receiving assistance.

To help ensure that USAID conducts effective evaluations of PFM-related programs under its new policy, we recommend that the Administrator of USAID direct the appropriate offices to ensure that they are establishing adequate monitoring practices for PFM-related programs. Such practices may include selecting proper indicators, collecting reliable baseline data, and ensuring the reliability of reported results.

To improve the effectiveness of OTA’s technical assistance, we recommend that the Secretary of the Treasury direct OTA to take the following two actions to improve monitoring and evaluation: implement additional controls to improve the process for computing OTA-wide annual performance measures, and fully implement OTA’s existing requirement for end-of-project evaluations and, consistent with its existing guidance, have an independent party conduct the evaluations.

Agency Comments and Our Evaluation

We provided a draft of this report to USAID, Treasury, and State for their review and comment. Both USAID and Treasury concurred with our recommendations in their written comments, which are reproduced in appendix V and appendix VI respectively. These agencies along with State provided technical comments and updated information, which we have incorporated throughout this report, as appropriate.

In concurring with our two recommendations, USAID reported that it is in the process of refining definitions that will identify and help measure the assistance that qualifies to meet the agency’s target of obligating 30 percent of its annual assistance through local systems by 2015. USAID also reported that it has implemented two complementary reforms that will help ensure effective evaluations and adequate monitoring of its PFM assistance. The first reform involves USAID planning for the monitoring and evaluation of assistance during the early stages of project design, including defining indicators and collecting baseline data. The second reform requires USAID to plan final monitoring and evaluation schedules during project design.

In concurring with our two recommendations, Treasury reported that OTA has corrected several errors in the 2011 annual performance measures, and has taken steps to strengthen data controls, including conducting additional reviews and increasing staff resources dedicated to computing the performance measures. In addition, OTA has begun to implement its requirement for independent end-of-project evaluations of its technical assistance and intends to fully implement the requirement by the end of 2012.

We are sending copies of this report to appropriate congressional committees. We are also sending copies of this report to the Administrator of USAID and the Secretaries of the Treasury and State. In addition, this report is available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-9601 or melitot@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this reported are listed in appendix VII.

Appendix I: Scope and Methodology

To examine the process the U.S. Agency for International Development (USAID) uses to develop programs to strengthen public financial management (PFM) systems, we reviewed USAID’s official policy and procedures, as contained in the Automated Directives System, as well as USAID’s new guidance documents for developing country strategies and designing projects, including the Country Development Cooperation Strategy and Project Design Guidance. We also interviewed USAID officials, including officials from the democracy and governance (DG) and economic growth (EG) offices. Most USAID country offices are required to have an approved country strategy by the end of fiscal year 2013 and to follow the new project design guidance starting July 2012. Because USAID was still in the process of transitioning to these new processes during the course of our audit, we did not review USAID’s implementation of these processes.

To examine the process the U.S. Department of Treasury’s Office of Technical Assistance (OTA) uses to develop programs to strengthen PFM systems, we interviewed senior OTA officials regarding OTA’s policies and procedures. We interviewed resident advisors regarding how OTA assessed and developed projects in our three OTA case study countries. We also reviewed OTA’s official policy guidance document and project- specific documents, including assessment reports, signed terms of reference, work plans, and monthly reports.

To examine and assess the processes USAID uses to monitor and evaluate its PFM-related programs, we reviewed USAID program development guidance; monitoring and evaluation policies and procedures; USAID reports; and program documents, including assistance agreements, monitoring and evaluation plans, and progress reports; USAID external project evaluations; past GAO reports; and USAID Inspector General reports. We used the review of the documents for our three USAID country case studies to illustrate the implementation of USAID’s monitoring processes for its PFM-related programs. We also interviewed agency officials in Washington, D.C., and conducted telephone interviews and had e-mail communications with key country- office staff for each of our case study countries.

To examine and assess OTA’s processes for monitoring and evaluating PFM-related programs, we reviewed OTA project reporting and documentation instructions, project work plans, and monthly reports, end- of-tour and end-of-project reports, annual quantitative performance data, Organization for Economic Cooperation and Development guidance on evaluation, and conducted interviews with OTA headquarters staff. We used the review of the documents for our three OTA case studies to illustrate the implementation of OTA’s monitoring processes. We supplemented the document review with interviews with current or former advisors for each of the case study countries. In addition, we assessed the reliability of the instrument OTA uses to aggregate quantitative performance measures across projects. We examined spreadsheets provided by OTA for consistency, examined data for outliers and missing values, and spot-checked the transcription of data to the spreadsheet for our case study countries. Due to a number of errors in the OTA data, we could not determine if the aggregated performance data were sufficiently reliable for identifying patterns in performance across projects or over time.

In selecting our case study countries, we focused on countries in which OTA or USAID had relevant ongoing or recently completed projects designed to strengthen PFM systems. We selected six countries: Cambodia, Honduras, and South Africa for OTA, and Kosovo, Liberia, and Peru for USAID. In selecting these countries, we considered the following two factors:

Geographic diversity: For each agency, we selected countries from three different geographical regions.

Country income group diversity: For each agency, we chose a country that the World Bank has listed as (1) lower income, (2) lower-middle income, and (3) upper-middle income in order to report examples from different income levels, which may also be associated with different institutional characteristics.

In cases where more than one country would be acceptable under our decision criteria, we considered additional criteria, such as the availability of other broad-based indicators. For OTA, we focused our selection on countries receiving technical assistance from OTA’s Budget and Financial Accountability team, given its focus on traditional PFM aspects. For USAID, we selected our countries from a list of countries with significant PFM-related programs that USAID provided. USAID excluded some countries with PFM-related programs from the list because staff were not available to discuss their programs with us. Our review of USAID and OTA case study countries was not intended to be comprehensive or applicable to all their respective programs and projects or generalizable to all countries.

To describe recent trends in country PFM systems, we reviewed the data and publications of five international organizations that conduct broad assessments of country PFM systems. These broad assessment tools include the Public Expenditure and Financial Accountability Program, the World Bank’s Country Policy and Institutional Assessment’s quality of budgetary and financial management indicator, International Budget Partnership’s Open Budget Survey, Transparency International’s Corruption Perceptions Index, and the International Monetary Fund’s fiscal transparency Reports on the Observance of Standards and Codes. We converted our six case study country scores for the three PFM diagnostic tools for which scores were available (Open Budget Survey, Corruption Perceptions Index, and quality of budget indicator) into percentile rankings to illustrate each country’s performance as measured by the three PFM diagnostic tools. We also interviewed officials at the World Bank and International Monetary Fund to discuss their PFM-related diagnostic tools.

To describe the Department of State’s PFM-related efforts, we conducted interviews with agency officials in Washington, D.C. We reviewed State documents, including agency guidance, waiver packages, and program documents. We also reviewed relevant appropriations laws to identify the requirements for State’s fiscal transparency reviews.

We conducted this performance audit from October 2011 through September 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Several International Organizations Reported Results of Their Public Financial Management Diagnostic Tools

Five international organizations — the Public Expenditure and Financial Accountability Program, the World Bank, International Budget Partnership, Transparency International, and the International Monetary fund — have developed tools to assess various aspects of countries’ public financial management (PFM) systems, and some have published recent findings or results. We illustrate the percentile ranking for each of our case study countries in three broad PFM diagnostic tools – the Open Budget Survey, the Corruption Perceptions Index, and the World Bank’s quality of budgetary and financial management indicator.

Public Expenditure and Financial Accountability Program’s PFM Performance Measurement Framework

In December 2001, a multiagency partnership founded the Public Expenditure and Financial Accountability Program (PEFA) to strengthen the ability of aid recipients and donors to assess and improve country public expenditure, procurement, and financial accountability systems. In 2005, the program developed the PFM Performance Measurement Framework, known as the PEFA Framework, to provide a measure of the strengths and weaknesses of a country’s PFM system. The PEFA Framework incorporates a PFM performance report, which includes an assessment of the evaluated country’s PFM performance along six core dimensions of PFM. The six dimensions are the following:

Credibility of the budget: The budget is realistic and is implemented as intended.

Comprehensiveness and transparency: The budget and the fiscal risk oversight are comprehensive, and fiscal and budget information is accessible to the public.

Policy-based budgeting: The budget is prepared with due regard to government policy.

Predictability and control in budget execution: The budget is implemented in an orderly and predictable manner, and arrangements for the exercise of control and stewardship in the use of public funds exist.

Accounting, recording, and reporting: Adequate records and information are produced, maintained, and disseminated to meet decision-making control, management, and reporting purposes.

External scrutiny and audit: Arrangements for scrutiny of public finances and follow up by executive are operating.

The six core dimensions include 28 high-level indicators, each of which is assigned a letter score from A to D. The initial assessment helps establish performance baselines, while repeat assessments help in monitoring performance progress over time.

In 2011, the PEFA secretariat released the 2010 PEFA Monitoring Report. Based on a comparison of 33 repeat PEFA assessments over the 2005-2010 period, the analysis shows that more countries had a higher number of highest or improved scores (23 countries) than lowest or worsened scores (8 countries), indicating a broad trend of PFM improvement across the countries surveyed. According to the analysis, while PFM systems are improving overall, systems features vary significantly. Formal PFM features where progress can be achieved through adopting a new law, regulation, or technical tool, or focusing on no more than a few agencies, or at an early stage in the budget cycle are more likely to improve or maintain a high score than functional PFM features where progress requires actually implementing a new law or regulation, or coordinating the work of many agencies, or working later in the budget cycle.

Starting July 2012, the PEFA partners are extending the PEFA program by 5 years and conducting a comprehensive review of the PEFA Framework, the first since it was launched in 2005. One of the objectives is to improve confidence in PEFA assessments through an endorsement process that provides an incentive to ensure adherence to PEFA good practices in undertaking an assessment.

World Bank’s Quality of Budgetary and Financial Management Indicator

The World Bank undertakes annual Country Policy and Institutional Assessments to assess the quality of a country’s present policy and institutional framework. One of the 16 indicators the World Bank uses to assess the performance of a country’s current policies and institutions is the quality of budgetary and financial management. This criterion assesses the extent to which the country has a comprehensive and credible budget linked to policy priorities; effective financial management systems to ensure that the budget is implemented as intended in a controlled and predictable way; and timely and accurate accounting and fiscal reporting. Over the 5-year period from 2005 through 2010, 26 out of 73 countries, or slightly more than one-third, showed improvements in the quality of their PFM systems, while 19 countries’ scores worsened. Most countries, 62 percent in 2010, are clustered in the mid-range.

In addition to the quality of budgetary and financial management indicator, the World Bank uses other broad PFM diagnostic tools, including the Country Financial Accountability Assessments and the Public Expenditure Reviews. The objective of the Country Financial Accountability Assessments is to support the World Bank’s development objectives by identifying strengths and weaknesses in country PFM systems. The assessments are intended to help identify priorities and inform the design and implementation of capacity-building programs. The assessments also describe and analyze financial management and expenditure controls, including expenditure monitoring, accounting and financial reporting, internal controls, internal and external auditing, and legislative review. Information obtained from the assessments, taken together with that obtained from other World Bank diagnostic products and other sources, supports the preparation of an integrated fiduciary assessment. The results of these assessments inform the preparation of the World Bank’s Country Assistance Strategy, particularly the sections dealing with the size of the support program, the sectors to be supported, selection of lending instruments, and approaches to risk management. The assessments are particularly important where World Bank resources are managed by the country’s own PFM system, as in the case of budget support.

The Public Expenditure Review’s objectives are to strengthen budget analysis and processes to achieve a better focus on growth and poverty reduction, and to assess public expenditure policies and programs to provide governments with an external review of their policies. Public Expenditure Reviews may also address the incentives and institutions needed to improve the efficacy of public spending in major sectors such as health and education.

International Budget Partnership’s Open Budget Survey

The International Budget Partnership’s Open Budget Survey assesses the availability in each country of eight key budget documents, as well as the comprehensiveness of the data contained in these documents. The survey also examines the extent of effective oversight provided by legislatures and supreme audit institutions, as well as the opportunities available to the public to participate in national budget decision-making processes.

Appendix III: State Department Conducts Fiscal Transparency Reviews and Administers the Domestic Finance for Development Initiative

State Department Conducts Two Public Financial Management- Related Activities

The State Department (State) does not directly fund public financial management (PFM) programs, but its Office of Monetary Affairs (OMA) is responsible for two PFM-related activities. First, since 2008, U.S. appropriations laws have required State to evaluate the fiscal transparency of foreign governments receiving U.S. assistance. When State determines it is important to the United States’ national interest, it may approve waivers for countries that are deemed to be nontransparent that allow U.S. agencies to provide assistance to these countries in accordance with the legal prohibition against providing such assistance without waivers. State processed 28 such waivers in 2011.

Second, State supports the presidential initiative called Domestic Finance for Development (DF4D), which President Obama announced in 2011. Under this initiative, State is to help countries use their own resources and leverage other donor resources to meet development goals. State is piloting this initiative in five countries: El Salvador, Honduras, Kyrgyzstan, Tunisia, and Zambia. The goal of DF4D is to strengthen the commitment to reform within partner countries; provide technical assistance in partner countries, such as taxation expertise, including through innovative public- private partnerships; and elevate the importance and interrelation of domestic resource mobilization, fiscal transparency, and anticorruption efforts in public finance as key components for sustainable economic development. Because State has not funded programs in the past for this initiative, State has reached out to other organizations operating in these countries, including international and bilateral organizations, to leverage their programs and resources.

State Processes for PFM Activities Derive from Legal Requirements and Presidential Initiatives

State has developed processes for carrying out its PFM-related activities required by legislative mandate and presidential initiative. Under the Fiscal Transparency Review Process (FTRP), OMA reviews central governments expected to receive bilateral economic assistance and international security assistance for several dimensions of transparency: publicly disclosed budget, budget breakdown by ministry, standards for awarding natural resource contracts, and timely and accurate documents. Each year, OMA reviews all countries it deemed nontransparent in the prior year, and countries where recent events may have affected fiscal transparency to evaluate whether they meet the threshold of fiscal transparency. State uses the IMF’s definition of fiscal transparency as a guideline for the FTRP. During this process, OMA officials said they obtain information about the level of transparency of each country by collecting information from U.S. embassy staff working in those countries. In addition, OMA staff review publicly available reports published by international organizations and civil society representatives, such as the World Bank, International Monetary Fund, International Budget Partnership, and the World Economic Forum. Because these organizations do not always prepare country reports on an annual basis, officials told us they use these reports as a check on their internally generated information, rather than relying on them as primary sources of information. For countries that State finds to be nontransparent, State can issue a waiver that allows the country to receive otherwise restricted assistance. As part of the process of requesting a waiver, embassy staff in country develop action plans to assist the country in improving the level of transparency in its budget process. The action plan should address specific fiscal transparency issues identified in the transparency review process and should include recommendations of short- and long-term steps that the countries can take to improve budget transparency. Embassies work with host governments to encourage implementation of action plans, which may include activities such as continued daily engagement with country officials working on budget reforms, providing training and coordinating training on issues related to the budget process, and funding local nongovernmental organizations to perform budget oversight functions.

For OMA’s DF4D initiative, State officials, working with other agencies and organizations, helped develop programs based on needs expressed by country government officials that built on existing reform efforts and development priorities. As an interagency effort, OMA helps identify and leverage existing programs and resources, such as those of multilateral organizations and other donors, such as USAID’s Innovation Fund and the Global Health Initiative. For example, USAID is implementing a $7.6 million program to advance El Salvador’s fiscal reform agenda by building capacity and improving systems for public expenditure and management and tax revenue mobilization, promoting private sector engagement, and creating a $2 million Revenue Challenge Fund to support improved tax collection at the municipal level. With five countries piloting DF4D, State plans to proceed by selecting additional partner countries based on performance against quantitative DF4D-related measures, consultations with posts, and expressions of interest from ministers of finance and other economic leaders. Moreover, State plans to use action plans developed for FTRP for countries to be considered for DF4D. State encourages posts to report on PFM issues and opportunities to mobilize domestic resources, raise the issues with relevant stakeholders in the public and private sectors, and work with OMA staff to further the objectives of the DF4D initiative.

State Is Taking Steps to Monitor Progress of Nontransparent Aid Recipients

State is beginning to conduct additional monitoring as part of its fiscal transparency evaluations. Starting in 2012, State is requiring benchmarks in its action plans for nontransparent countries so that it can compare progress annually. According to State officials, some country action plans had benchmarks but the quality of the benchmarks varied. The action plan attempts to capture all the steps necessary to improve a country’s fiscal transparency and includes more than just State actions. Lastly, appropriations law for 2012 requires State to evaluate whether or not the country has made progress toward improved fiscal transparency included in any waiver request submitted. State guidance reports that progress toward implementation of embassy action plans will factor into its decision of whether to renew waivers.

Appendix IV: Examples of Public Financial Management-related Projects in Six Countries

To provide illustrative examples of U.S. projects to strengthen public financial management (PFM) systems, we chose six case study countries. For the U.S. Agency for International Development (USAID), we selected projects in Kosovo, Liberia, and Peru and for the U.S. Department of the Treasury (Treasury), we selected projects in Cambodia, Honduras, and South Africa. We used the following criteria to select the three countries for each agency:

The agency has relevant ongoing or recently completed projects focused on strengthening PFM systems.

The countries represent different geographic regions.

The countries have different income levels, which may be associated with different levels of government capabilities. For each U.S. agency, we selected a country that the World Bank has classified as lower income (Cambodia and Liberia), lower-middle income (Honduras and Kosovo), and upper-middle income (Peru and South Africa).

For one PFM-related project in each country we summarized the project’s background, and, for selected objectives, we summarized some of USAID’s expected results and some of Treasury’s activities. These examples are meant to be illustrative and not generalizable.

Selected USAID PFM– Related Projects

Tables 3 through 5 summarize selected USAID PFM-related projects in Kosovo, Liberia, and Peru.

Kosovo Growth and Fiscal Stability Initiative Time period: July 2010-July 2013 Award amount: $1,051,208 Description: Since 1999, USAID’s Kosovo economic policy and institutional strengthening programs have focused on establishing key central economic institutions and an enabling environment for private sector growth. USAID is to adjust the focus of technical assistance as the Growth and Fiscal Stability Initiative builds upon the experience and lessons learned from the creation of reliable financial institutions in the central government to address the fiscal stewardship challenges faced by subnational governments. The initiative is to work with municipalities in areas that are directly linked with the Ministry of Finance and Economy, such as budget, treasury, property tax, and public private partnerships.

Liberia: Governance and Economic Management Support Program (GEMS)

Time period: July 2011-June 2016 Contract amount: $44,902,679 Description: USAID-GEMS is to address significant governance challenges remaining after USAID completed its previous capacity- building program in 2010. The program is to strengthen human and institutional capacity within selected ministries, agencies, and commissions. USAID-GEMS is to develop and maintain systems that increase transparency and accountability, increase efficiency, reduce expenditures, increase revenue, and limit corruption.

Peru: USAID ProDecentralization 2 Time period: August 2008-July 2012 Award amount: $10,644,432 Description: The USAID/Peru ProDecentralization project is in the second phase. The first phase targeted national, regional, and municipal institutions responsible for implementing the decentralization process. The second phase seeks to continue to improve the policy framework at the national level and strengthen the institutional capabilities of regional and municipal governments. The national project aims to improve the legal and policy framework for decentralization, including fiscal policies that support a more equitable distribution of public resources. Regional and local-level activities are to target technical assistance and training to the diverse needs of Peru’s regional and municipal governments in effectively administering resources and responding to citizens’ increasing expectations. Under this project, existing services offered to subnational governments in planning, budgeting, accountability, and institutional strengthening are supplemented by new training and technical assistance programs.

Tables 6 through 8 summarize selected Treasury Office of Technology Assessment (OTA) PFM-related projects in Cambodia, Honduras, and South Africa.

Cambodia: Technical Advisory Services on Budgeting Time period: October 2009–September 2010 Description: The Cambodian government has been implementing a plan to create a credible budget and improve accountability. In the current phase of the plan, OTA and Cambodian government actions are focused on decentralizing the reform effort to the line ministries and subnational levels. These actions include expanding the use of strategic and program budgeting, implementing a PFM information system, improving macroeconomic forecasting, enhancing the linkage between the capital and operating budgets, and improving overall financial accountability. The overall aim is to improve the ability of the budget to be an instrument for policy delivery and to support effective and efficient service delivery.

Honduras: Technical Advisory Services on Budgeting Time Period: January 2011-December 2013 Description: OTA’s technical assistance focuses on implementing international public sector accounting standards.

South Africa: Technical Advisory Services on Budgeting Time Period: February 2007—January 2009 Description: Treasury OTA has conducted technical assistance projects in South Africa in the areas of budget formulation, intergovernmental finance, infrastructure budgeting, public finance training, and others since 1997. In 2006, the National Treasury of South Africa requested a new OTA budget project to focus on the organization of an expenditure program performance evaluation unit, performance information and cost analysis, and support for the Collaborative Africa Budget Reform Initiative. The central mission of the resident advisor was to establish an expenditure program performance review system.

Appendix V: Comments from the U.S. Agency for International Development

Appendix VI: Comments from the Department of the Treasury

Appendix VII: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

Cheryl Goodman (Assistant Director), Michael Maslowski, Shirley Min, RG Steinman, Michael Hoffman, Debbie Chung, Grace Lui, David Dayton, and Etana Finkler made key contributions to this report.