Afghanistan Governance: Performance-Data Gaps Hinder Overall Assessment of U.S. Efforts to Build Financial Management Capacity
Released: 2011-09-20T00:00:00

Letter

Background

Since 2002, the United States has allocated more than $72 billion for development, governance, and security in Afghanistan. See appendix II for a breakdown by year of U.S. allocation of funds for Afghanistan from fiscal year 2002 to fiscal year 2011. After almost a decade of donor-led efforts in Afghanistan, the United States and international donors have increased their focus on transitioning leadership to the Afghan government. The government of Afghanistan and the United Nations sponsored international conferences in London (January 2010) and Kabul (July 2010), which were attended by senior officials representing about 70 major donors, including the United States. Participants committed to supporting Afghan government leadership and ownership and agreed to increase the percentage of development aid delivered through the Afghan government to 50 percent by 2012, if the Afghan government showed progress in areas such as strengthening its public financial-management systems and reducing corruption. In November 2010, representatives from 48 countries contributing to the United Nations–mandated International Security Assistance Force agreed to a plan to begin transitioning security responsibilities to the Afghan government in 2011 and to complete this transition by 2014. In June 2011, the U.S. President reiterated that the process of security transition in Afghanistan will be complete by 2014. According to donors and the Afghan government, ensuring that public funds are used in a transparent and responsible manner is necessary for effective governance. Key elements of Afghanistan’s PFM include developing a national budget that represents the country’s priorities (budget formulation), spending the approved budget in the intended time frame (budget execution), and ensuring that funds are used as intended (through audits).

Public Financial Management Roadmap to Improve Afghan Government Capacity

In July 2010, the Afghan government published a plan, called the Public Financial Management Roadmap, to strengthen the Afghan government’s performance in three key areas at the national and provincial level: budget formulation, budget execution, and accountability and transparency of financial management. Additionally, a cross-cutting area is to increase the capacity of Afghan ministries. MOF led the development of the Roadmap, with input from the World Bank and International Monetary Fund, as well as partner governments such as the United States and United Kingdom. Table 1 provides an overview of the Roadmap’s components and their key areas. Additionally, appendix III provides information on the Afghan government’s budget process.

U.S. embassy guidance issued since the release of the Roadmap states that U.S. governance activities are to increase focus on developing the capacity of Afghan civilian agencies in budget prioritization and execution to achieve progress towards Afghan self-governance in anticipation of the security transition.

Multiple Donors Provide PFM Capacity-Building Assistance

Several international donors are helping the Afghan government improve its PFM capacity. The World Bank, the United Kingdom, and the United States are key donors providing assistance to build PFM capacity of Afghan civilian agencies. Figure 1 shows donor PFM assistance to Afghan government entities at the national and provincial levels.

Through the Combined Security Transition Command–Afghanistan (CSTC-A), DOD helps build the PFM capacity of Afghanistan’s security ministries, the Ministries of Defense (MOD) and Interior (MOI). CSTC-A is primarily U.S.-funded and staffed. In April 2009, the United States and its North Atlantic Treaty Organization (NATO) allies agreed to establish NATO Training Mission–Afghanistan (NTM-A) to oversee institutional training and development of the Afghan National Security Forces. NTM- A/CSTC-A now operates as an integrated NATO and U.S. command.

International donors provide a significant share of the total funding for Afghanistan in the form of funds channeled through the Afghan government budget as well as “off-budget” assistance that does not use the Afghan government’s budget system. According to our preliminary analysis, the total estimated budget for Afghanistan for March 21, 2010– March 20, 2011 was $10.2 billion; donors were expected to fund about two-thirds of the Afghan government budget of $4.4 billion and the entire reported off-budget assistance of $5.8 billion. Figure 2 shows the total funding to Afghanistan through the Afghan government budget and off- budget donor assistance for March 2010 to March 2011.

Various U.S. Agency Efforts Support Afghanistan’s PFM Capacity-Building Goals

USAID, Treasury, and DOD support the Roadmap goals through various activities such as (1) USAID projects that provide technical assistance and training to Afghan civil servants, (2) Treasury advisers’ assistance to MOF, and (3) DOD’s mentoring and coaching assistance through CSTC- A to MOD and MOI. USAID provides training and technical assistance mainly through two contractor-implemented projects. Treasury provides technical assistance through 6 advisers in MOF, who work with senior officials on issues such as budget execution. Through CSTC-A, DOD has 22 advisers at MOD and MOI, who advise officials on developing their budgets and strengthening the payroll system to improve accuracy.

USAID Provides Technical Assistance and Training to Afghan Civilian Ministries

USAID is funding several projects that provide training, mentoring and coaching, and technical assistance that address Roadmap goals. USAID has two primary PFM capacity-building projects—the Economic Growth and Governance Initiative (EGGI) project that has a contract value of approximately $92 million over 5 years, and the Afghanistan Civil Service Support (ACSS) project that has a contract value of approximately $84 million over 1 year. Both these projects are implemented by Deloitte Consulting, which has hired international contractors and local Afghans. EGGI supports all four PFM goals of strengthening budget formulation, improving budget execution, increasing accountability and transparency, and improving ministry capacity. For example, according to USAID, EGGI contractors provided program-budget technical assistance to 37 Afghan central ministries and agencies so that these entities can formulate annual budgets that conform to MOF guidance. Examples of EGGI’s technical assistance include contractors helping MOF’s fiscal policy unit develop a fiscal forecasting model and providing policy assistance to the Ministers of Finance and Economy. ACSS similarly supports most Roadmap goals through training it provides in financial management and procurement to Afghan civil servants in Kabul and in the provinces. Table 2 provides information on EGGI and ACSS, as well as other USAID projects that, while not exclusively focused on PFM capacity building, provide some PFM capacity-building assistance.

A July 2009 Treasury assessment, which was part of a proposal for enhanced civilian assistance to Afghanistan, identified the need for advisers to assist with public expenditure management in MOF’s offices of treasury, budget, and internal audit. Treasury assigned six advisers from its Office of Technical Assistance to assist various MOF offices. These advisers are embedded in MOF offices and provide assistance that supports Roadmap goals by advising senior Afghan MOF officials, analyzing and monitoring budget expenditures and controls, and developing accounting and administrative manuals. Table 3 indicates location of Treasury advisers within MOF, type of assistance provided, and the PFM goals support by this assistance.

As of April 2011, DOD, through CSTC-A, and other coalition partners had 587 advisers and mentors working with their Afghan counterparts to build the capacity of MOD and MOI in about 30 functional areas, such as intelligence, personnel management, logistics, and finance and budget. Of the 587 advisers and mentors, CSTC-A had 7 advisers at MOD and 15 at MOI who provided advice in the finance and budget area. CSTC-A is working to ensure that MOD and MOI are capable of operating without coalition assistance by 2014.

DOD employees or military personnel under CSTC-A provide leadership for capacity building in the finance and budget area; they are partnered with specific Afghan officials in MOI and MOD units and oversee the work of U.S. contractors. CSTC-A has developed comprehensive Ministerial Development Plans to guide its capacity-building efforts at MOI and MOD, including plans for finance and budget (see app. IV for a listing of all focus areas). Table 4 shows the specific goals in the finance and budget plans for MOD and MOI for 2011.

According to CSTC-A officials, advisers at MOD focus on building Afghan capacity and generally do not carry out the work of MOD officials. At MOI, advisers meet with finance and budget partners daily to help formulate budget and develop pay procedures, such as the electronic payroll system. As shown in table 5, the advisers provide advice that supports the Roadmap goals of improving budget formulation and execution, and increasing accountability and transparency. According to DOD officials, the goal of their capacity-building efforts is to ensure that MOD and MOI are fully capable of carrying out key functions such as finance and budget without coalition assistance.

According to the official terms of reference for CSTC-A advisers, they advise, facilitate, and collaborate with their MOI and MOD counterparts. The guidance states that on rare occasions, a crisis will occur and the adviser will have to perform the relevant functions. According to the CSTC-A officer directing finance capacity building at MOD, the advisers should allow their Afghan counterparts to make mistakes and learn from them, unless the mistake is critical.

Results of U.S. Efforts to Improve Afghanistan’s PFM Capacity Cannot Be Fully Determined

The overall results of U.S. efforts cannot be fully determined because (1) U.S. agencies providing PFM capacity assistance to the Afghan government have reported mixed results of their efforts, and (2) weaknesses in USAID’s performance management plans and frameworks, such as lack of performance targets and data, prevent reliable assessments of USAID’s results. USAID’s evaluations of its two primary PFM projects indicate that some activities were successfully completed, while others were terminated because their usefulness was questionable. Treasury advisers assessed that although their assistance at MOF had a positive effect, the results fell short of what they were trying to accomplish. Additionally, CSTC-A reported that while MOD has progressed to being able to perform critical financial-management functions with minimal coalition support, MOI continues to rely on coalition support for these functions. Moreover, CSTC-A has extended transition milestones for security ministries. Regarding USAID’s performance management, target and performance data have not been approved for PFM efforts at the USAID Mission level as well as for PFM-focused projects.

U.S. Agencies Report Mixed Results of PFM Capacity-Building Activities

USAID, Treasury, and DOD reported mixed results related to U.S. efforts to build Afghan PFM capacity.

USAID has reported output and some outcome information for its EGGI and ACSS projects, and conducted project evaluations that indicate positive and negative results. In 2010, the EGGI project trained approximately 800 government employees on how to develop a program budget, which allows budget units to request funding based on expected outputs and outcomes of specific programs. In addition, EGGI provided technical assistance to 37 budget units to help them prepare annual budgets for Afghan fiscal year March 2011 to March 2012. According to USAID, this training was effective because all budget units prepared program budgets and submitted them in a timely manner. USAID also reported that 10 ministries “graduated” from budget training as these units can prepare their own budgets with little or no assistance from the EGGI team. According to USAID evaluators, program budgeting represented a major programmatic accomplishment for the EGGI project.

However, other EGGI activities were not as successful, in part because USAID and the project’s implementing partner did not adequately consult with relevant stakeholders, such as other international donors conducting related activities. For example, EGGI developed a Revenue Reconciliation Database to report revenue collections to the Afghanistan Revenue Department in real time. According to USAID evaluators, the EGGI revenue database does not directly connect with the Afghan government’s accounting system, as it was intended to do, or the revenue-collection system being developed by DFID; therefore, the database does not provide information that is useful for planning or analysis. USAID has since terminated this activity. However, USAID officials noted that because of delays in the installation of the DFID system, the USAID database is the only functional system in use by MOF’s Afghanistan Revenue Department to collect and report revenue data. Additionally, USAID’s evaluators noted that USAID and DFID did not adequately coordinate efforts related to creating a medium taxpayer office in Herat.

According to USAID, ACSS trained approximately 16,000 civil servants in Kabul (5,759 participants) and in 26 provinces (10,121 participants) from February 2010 to March 2011. It provided training in five core subjects, including financial management and procurement. According to evaluations by USAID and the Afghan government’s Independent Administrative Reform and Civil Service Commission, ACSS training contributed to improvements in civil servants’ performance. However, USAID evaluators noted that a lack of baseline data makes it impossible to measure the extent of these improvements.

In addition, USAID’s evaluation of ACSS cites several challenges resulting from the project’s Afghan-led approach, which involved Afghan officials in making decisions related to project implementation. Some Afghan officials viewed the Afghan-led approach to mean that they did not need to ensure accountability for some inventory. As a result, in some cases, USAID inventory stickers were reportedly removed from assets before the items were transferred to Afghan control, causing accountability issues, according to USAID’s evaluation. In addition, some Afghan managers interpreted the concept of Afghan-led to mean that they could prevent the project’s monitoring and evaluation team from obtaining data needed to assess project results, according to USAID’s evaluation. For example, when ACSS staff requested monitoring and evaluation data, some Afghan managers responded that they were not required to provide data and that such requests represented an inappropriate desire to exert control over their operations. As a result, the quality of monitoring and evaluation of the project suffered.

To assess the effect of Treasury advisers’ technical assistance to MOF, Treasury’s Office of Technical Assistance requires each adviser to submit monthly reports and an annual evaluation. For the period October 2009 to September 2010, Treasury advisers assessed that although their assistance related to budget and financial management at MOF had a positive effect, the results were less than what they were trying to accomplish. For instance, the advisers gave a low score for results related to their efforts to design management reports to improve communication of financial information and enhanced budget control. However, the advisers assigned a high rating for results related to developing training materials to support capacity building. The advisers also assessed the level of commitment and involvement of their Afghan counterparts as being above average, with room for improvement.

CSTC-A has established a process to assess progress on key objectives and rate the capacity of MOD and MOI, in areas including finance and budget functions, on a quarterly basis. According to CSTC-A’s assessments, MOD has progressed from significant reliance on coalition support in 2008 to being capable of executing core functions related to finance and budget with minimal coalition support by 2011. MOI has progressed from not being able to accomplish finance and budget functions on its own in 2008 to being able to carry out core functions with significant coalition assistance by 2011. Figure 3 shows actual and projected ratings for MOD and MOI’s capability in finance and budget operations. Progress at MOI has been slow, in part because CSTC-A’s capacity-building efforts at MOI started in 2006, several years after its efforts began at MOD in 2002. CSTC-A has extended the time frames to meet interim and final capacity-building goals at MOD and MOI due to various challenges. See appendix IV for a description of CSTC-A’s capacity assessment process.

According to CSTC-A assessments, MOD transitioned from needing significant coalition support to accomplish finance and budget functions to requiring some coalition assistance in December 2008. Although CSTC- A’s rating of MOD has not changed since then, CSTC-A officials documented progress at MOD. For example, CSTC-A’s quarterly assessment of MOD for January through March 2011 reported that MOD  executed critical functions, such as paying of salaries to soldiers, with minimum coalition support; implemented a policy to transition pay operations from U.S. embedded teams to Afghan army finance officers; streamlined pay systems and expanded electronic funds transfer capability so that salaries of 97 percent of Afghan soldiers are deposited directly into their bank accounts;  executed its budget for salaries and other operational expenditures at over 99 percent for Afghan fiscal year 2010-2011; and implemented an integrated program budget-formulation process for this fiscal year, such that the budget request was tied to the ministry’s strategic goals and performance measures.

However, MOD has not progressed to needing only coalition oversight in carrying out finance and budget functions because of a lack of sufficient delegation of budget authority from MOD’s central finance office to subordinate units and commands of the Afghan army. Although CSTC- A officials worked with MOD to clarify roles and responsibilities of variou offices, highly centralized budget authority prevented MOD from developing and executing the budget with input from the commands of the Afghan National Army. Therefore, CSTC-A advisers are still involved to ensure that MOD develops integrated program budgets.

Although MOI progressed from an inability to accomplish its finance and budget functions in March 2008 to being able to accomplish these functions with significant coalition support by March 2009, MOI’s capacity is far behind MOD’s. CSTC-A officials stated that their capacity-building efforts at MOI started in 2006, several years after CSTC-A’s capacity- building efforts began at MOD in 2002. CSTC-A’s quarterly assessment of MOI for January through March 2011 rates MOI at the same level as in 2009, citing factors such as lack of consolidation of MOI’s personnel databases due to delays in contracts and lack of telecommunications network expansion. As a result, CSTC-A or Afghan officials are not certain that all salary payments are being made to legitimate Afghan National Police personnel rather than “ghost” employees. Additionally, current MOI finance office employees do not have formal training in properly executing the budget and salary functions. Moreover, according to CSTC-A assessment, MOI has a top-down organizational culture in which officials tend to delegate key decisions to the minister. For example, MOI’s Program Budget Advisory Committee, which is responsible for reviewing expenditures, is reluctant to make decisions affecting budget execution and pushes these up to the Minister of Interior. This reportedly resulted in necessary actions not being taken in a timely fashion.

CSTC-A’s goal is for MOD and MOI to achieve self-sustainability prior to 2014, when coalition forces are scheduled to transition security responsibilities to the Afghan government. However, CSTC-A has extended the time frame for MOD and MOI finance offices to meet interim and final goals due to various challenges. For example, in early 2010, MOD’s finance office was expected to reach the interim goals of operating autonomously with only coalition oversight by March 2011 and the final goal of autonomous operations by January 2012. However, in March 2011 the projected date for meeting the interim goal was extended by 3 months to June 2011 and for meeting the final goal was extended by 6 months to June 2012. Similarly, we have previously reported that in several instances DOD pushed out completion dates related to training of Afghan army and police forces.

The time frames for MOI to reach interim and final goals have also been revised. For example, in early 2010, CSTC-A expected MOI’s finance office to operate with some coalition assistance by March 2011 and become fully autonomous by March 2012. However, in March 2011, the time frames were revised so that the interim and final goals were expected to take 8 additional months each, and projected to be accomplished by November 2011 and November 2012, respectively. The delay was attributed to problems in implementing the electronic payroll system. Despite these delays, CSTC-A officials noted that they expect MOI to become fully autonomous by identifying and addressing high-risk areas, such as electronic payroll.

USAID’s Performance Management Frameworks Lack Performance Targets and Data

USAID’s Automated Directives System (ADS) establishes performance management and evaluation procedures USAID is expected to follow with respect to planning, monitoring, and evaluating its programs. While USAID has noted that Afghanistan is an insecure environment in which to implement its programs, the agency has generally maintained the same performance management and evaluation procedures as it does in other countries in which it operates. For PFM capacity building in Afghanistan, we found a lack of compliance with USAID guidance at the mission level and at the implementing-partner level. Additionally, USAID evaluations also note weaknesses in the performance management of PFM projects. Appendix V presents a summary of the planning, monitoring, and evaluating requirements that make up USAID’s performance management and evaluation procedures.

At the mission level, ADS requires USAID officials to complete a Mission performance management plan for each of its high-level objectives as a tool to manage its performance management and evaluation procedures. The guidance also requires that USAID establish performance targets for each of the indicators. We previously reported in July 2010 that USAID has operated without a required Mission performance management plan for Afghanistan since the end of 2008. Subsequently, USAID issued a new performance management plan that realigned objectives based on U.S. strategies adopted in 2009 as well as agreements made at the London Conference in January 2010 and the Kabul Conference in July 2010. This plan represents the U.S. Mission’s tool to plan and manage the process of assessing and reporting progress towards assistance and foreign policy objectives in Afghanistan. As figure 4 shows, the plan contains a results framework that includes two PFM-related objectives and several related indicators, but lacks performance targets for each indicator as required.

The plan noted that baselines and targets for each indicator would be established in the first and second quarters of fiscal year 2011. Additionally, according to the plan, many indicators do not have baseline data or targets because some indicators are either new or were in the previous plan but data were never collected for them. This is contrary to ADS, which requires that targets be established for each performance indicator. In addition, some indicators are for proposed activities and need to be finalized. For example, one of the indicators, “Percent increase in standardized Public Financial Management assessment scores,” relies on baseline data for Afghan civilian ministries that is not yet available. In April 2011, donors initiated joint assessments of 14 Afghan civilian ministries, which account for 90 percent of the development budget, to establish baselines of their PFM capacity. These assessments are due by 2012, with preliminary information only available for MOF.

ADS documents USAID’s performance management and monitoring procedures. Project implementers must follow these requirements outlined in USAID award documents. For example, project implementers are required to identify performance indicators, define their project’s “starting point” by establishing baselines, and define changes that signal success by establishing performance targets for each project year. In addition, project implementers are required to regularly collect, analyze, and interpret performance data in order to improve their ability to make project adjustments in a timely manner. The performance management frameworks for PFM capacity-building projects did not meet USAID guidance because of deficiencies such as a lack of baselines, targets, and performance data.

The performance indicators for EGGI and ACSS related to PFM capacity- building activities do not consistently provide baselines, performance targets, or actual performance data for each indicator, as required. For example, as shown in figure 5, for fiscal year 2010, EGGI’s implementing partner did not establish performance targets or provide actual data on a quarterly or annual basis for training activities conducted by EGGI. Additionally, EGGI’s implementing partner did not report target or actual data for the indicator called “Annual Tax Revenues Collected in Priority Revenue Mustofiats” because MOF did not release revenue data during the project’s performance year. Subsequently, USAID provided fiscal year 2010 quarterly and annual data for this indicator to GAO in the fourth quarter of fiscal year 2011. While these data may provide information that is useful for project evaluation, since they were provided retroactively, they were unavailable for USAID to monitor the progress of the activity during the performance year. Similarly, ACSS’s implementing partner did not provide evidence of baseline, quarterly performance targets, or actual quarterly data for any of the project’s PFM training indicators. This lack of targets and actual performance data rendered associated indicators ineffective for either tracking the progress of associated activities or assessing the extent to which USAID’s implementing partners met interim goals. Appendix VI provides the available data for fiscal years 2010 and 2011 for EGGI and ACSS.

For fiscal year 2011, although USAID-funded evaluations note that EGGI improved its performance measures and data collection, our analysis indicates that both EGGI and ACSS did not establish performance targets for each indicator, or report actual data for completed quarters, as required. As shown in figure 6, EGGI did not establish quarterly or annual targets for its training activities that include key areas, such as program budgeting and tax administration, noted in EGGI’s workplan. Similarly, ACSS did not establish baselines for any of its PFM indicators, which would make it difficult to assess the project’s training accomplishments.

In addition to the lack of compliance with USAID guidance noted above, USAID-funded evaluations of EGGI and ACSS have noted other weaknesses in the projects’ performance management frameworks. For example, the evaluator of the EGGI project was critical of the large number of performance indicators for the project’s first year that do not demonstrate a direct link between project efforts and improved capacity of the Afghan government. More specifically, the evaluator noted that only 4 of 7 indicators related to EGGI’s PFM capacity-building component are directly affected by the project’s work. The other indicators—national domestic revenues, national tax revenues, and national nontax revenues—are indirectly affected by the project’s work. Another EGGI evaluation noted that since MOF was not publishing data for the indicator related to annual taxes in priority provinces, the implementer should replace it with a different indicator that can be used to monitor activities.

The evaluator of the ACSS project noted that the project’s work plan, performance management plan, and operations manual were not sufficiently rigorous or comprehensive and, given the scope and scale of the project, the evaluator expected higher-quality project documentation. The evaluator also noted that the project’s performance monitoring framework lacks results-based information about the effectiveness of the project’s technical advisers.

In addition to weaknesses in the performance management frameworks of USAID’s PFM capacity projects, we have previously noted similar deficiencies in other USAID projects in Afghanistan. For example: In July 2008, we reported that, among other things, limitations in USAID’s data collection and performance evaluation frameworks impeded the agency’s ability to evaluate the effects of its roads projects. In July 2010, we reported that USAID did not assure that all indicators had targets for the eight GAO-reviewed agriculture projects. In November 2010, we reported that four of the six implementers of GAO-reviewed water projects did not always establish targets for performance indicators.

We are following up on USAID’s progress on these recommendations. Additionally, the USAID Administrator committed to tracking resources against outcomes as effectively as possible at a congressional hearing in July 2010.

Conclusions

The United States and other international donors have begun to focus on building the Afghan government’s capacity for a successful transition of leadership for security and governance to Afghanistan. Improving the Afghan government’s ability to manage its public finances is an important part of this transition effort. USAID, Treasury, and DOD, along with other international donors, have undertaken various efforts to address the Afghan government’s PFM capacity. While DOD and Treasury have assessed and reported mixed results based on their efforts, the overall effect of U.S. efforts is not known because USAID, which has a key role in building Afghan civilian ministries’ PFM capacity, has not consistently established baselines and targets, or reported actual performance data. We have previously reported on deficiencies in USAID’s performance management efforts in Afghanistan and made recommendations to improve the assessment of USAID program performance and the efforts of USAID’s implementing partners. During a congressional hearing in July 2010, USAID’s Administrator identified defining, tracking, observing, and reporting on results of USAID projects as a priority, noting that it is important to determine how USAID efforts contribute to the U.S. strategy for Afghanistan. He also committed to tracking resources against outcomes as effectively as possible. The lack of approved mission-level and implementing partner-level performance targets calls into question USAID’s efforts to live up to the Administrator’s commitment. Given the importance of USAID efforts to improving Afghanistan’s PFM capacity and the need for reliable performance results data to base future development assistance and funding decisions on, it is vital that USAID take steps to ensure its performance management efforts are consistently implemented.

Recommendations for Executive Action

We recommend that for public financial management (PFM) efforts, the USAID Administrator take the following three actions: (1) establish targets, as required, for each PFM-related performance indicator in its Mission Performance Management Plan for Afghanistan, (2) take steps to ensure that the USAID-approved performance management plan for each implementing partner includes baseline data and targets for each indicator, and (3) ensure that implementing partners report performance data at the frequency established in the performance management plan.

Agency Comments and Our Evaluation

We provided a draft of this report to DOD, State, Treasury, USAID, and the World Bank for comment and review. USAID provided written comments, which are reprinted in appendix VII, as well as technical comments, which we have incorporated as appropriate. DOD, State, Treasury, and the World Bank had no comments.

USAID concurred with all three recommendations and noted that it has started taking steps to address these. Specifically, USAID stated that it had started a review of the Mission performance management plan to determine if adequate PFM-related performance indicators, including baseline data and targets, are included. Additionally, USAID noted that it has commenced a comprehensive review of all awards to USAID implementing partners working on projects and activities in the PFM sector to ensure USAID-approved PMPs are in place for each award and each implementing partner’s performance management plan includes sufficient baseline data and targets for each indicator. As part of this review, USAID noted that it is also examining the reporting requirements set forth in relevant awards, including performance requirements, and would take corrective action, as needed.

We are sending copies of this report to the appropriate congressional committees, the Secretaries of Defense, State, and Treasury, as well as the Administrator of USAID, and other interested parties. The report also is available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at (202) 512-7331 or johnsoncm@gao.gov. Contact points for our Offices of Public Affairs and Congressional Relations may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VIII.

Appendix I: Objectives, Scope, and Methodology

This report examines (1) U.S. efforts to improve the Afghan government’s public financial management (PFM) capacity, including the extent to which these efforts aligned with the PFM goals identified by the Afghan government and the international community; and (2) the extent to which U.S. efforts have helped to improve the Afghan government’s PFM capacity.

To address these objectives, we focused on U.S. agencies’ PFM capacity-building activities since 2009, when President Obama concluded the administration’s review of U.S. efforts in Afghanistan and announced a strategy for Afghanistan and Pakistan. We reviewed documents and records from the U.S. Departments of Defense (DOD), State (State), and the Treasury (Treasury), and the U.S. Agency for International Development (USAID). We selected these agencies because they provide guidance or assistance related to U.S. PFM capacity building for the Afghan government. We also reviewed documents from the Afghan government and other donors, such as the World Bank. We interviewed officials from DOD, State, Treasury, and USAID in Washington, D.C., and in Kabul, Afghanistan. Additionally, in Kabul, we interviewed officials from other international donors providing PFM assistance, such as the World Bank and the United Kingdom, as well as Afghan government officials from organizations including the Ministry of Finance (MOF), Ministry of Defense (MOD), Ministry of Public Health, and the Independent Administrative Reform and Civil Service Commission.

To inventory and describe U.S. government efforts to build the Afghan government’s PFM capacity, we interviewed officials from U.S. agencies, including USAID, Treasury, and DOD, and reviewed relevant documents. We also compared U.S. agency efforts with the key components of the Public Financial Management Roadmap (Roadmap), which represents goals agreed upon by donors and the Afghan government to improve the Afghan government’s PFM capacity.  For USAID PFM capacity-building work we reviewed documents— including base contracts, contract modifications and scope of work— that describe USAID-funded PFM capacity building work in Afghanistan. To identify USAID PFM capacity-building projects since 2009, we interviewed officials from USAID’s Office of Democracy and Governance, and Office of Economic Growth. Some USAID projects are divided into major areas of focus called components. Two USAID projects—Economic Growth and Governance Initiative (EGGI) and Afghanistan Civil Service Support (ACSS)—have at least one component related to building the PFM capacity of Afghan government entities at the national and provincial levels. We selected these projects for a more detailed review. Some projects did not have a component focused on building the Afghan government’s capacity in budget formulation, budget execution, or transparency and accountability. However, we included them in our inventory because USAID officials noted that these projects contribute to increasing the Afghan government’s PFM capacity. We decided to focus on projects that primarily build PFM capacity at the national and provincial levels because the Roadmap prioritizes capacity building at these levels.  To describe the PFM efforts of Treasury officials, we reviewed documents including Treasury’s 2009 assessment of the need for advisers to assist the Afghan government with public-expenditure management and a summary of the roles of advisers currently at MOF. In Kabul, we also interviewed the Treasury advisers and met with them at MOF.  To describe DOD’s PFM capacity-building work, we reviewed documents including organizational charts and staffing levels, policy guidance and operating procedures, and various implementation, development, and evaluation plans for both MOD and Ministry of Interior (MOI). For information on the role of mentors and advisers in building capacity at MOD and MOI, we interviewed senior DOD officials at the Combined Security Transition Command–Afghanistan (CSTC-A), who were providing the services to both MOI and MOD, as well as MOD Finance officials who were receiving these services. In addition, we reviewed MOD and MOI training manuals, course materials, and adviser guides developed specifically for mentoring and capacity building in these ministries. We also reviewed the contracts providing mentoring and training services to MOD and MOI.  To assess alignment of U.S. efforts and goals with the PFM goals of international donors and the Afghan government, we compared efforts undertaken by USAID, Treasury, and DOD with the key components of the Roadmap. We also reviewed U.S. plans and guidance issued since 2009, including the U.S. Foreign Assistance for Afghanistan Post Performance Management Plan (2011-2015) and U.S. Mission guidance to U.S. agencies on PFM issues in Afghanistan. For U.S. military efforts we reviewed goals stated in CSTC-A Ministerial Development Plans for finance and budget functions for MOD and MOI. We also interviewed U.S. officials from USAID, Treasury, and CSTC-A to corroborate the extent to which the Roadmap guided their efforts in working with Afghan ministries. Additionally, we obtained relevant documentation from MOF officials, including the implementation plan and a 6-month progress report related to the Roadmap.  To assess U.S. efforts within the broader context of other donor efforts, we interviewed donors including the World Bank and United Kingdom’s Department for International Development (DFID) in Kabul. We also obtained documents, such as the Roadmap Implementation Plan and Technical Assistance Summary Reports, and interviewed officials from the World Bank and MOF.

To assess the results of U.S. PFM efforts, we analyzed relevant documentation from U.S. agencies.  For USAID, we analyzed the performance plans for the Mission as well as for PFM-specific projects. Additionally we analyzed the results reported in project evaluations as well as monthly, quarterly, and annual reports for the two main PFM-focused projects. USAID evaluations were based on document reviews as well as interviews with contractors, USAID officials, and Afghan government officials. Additionally, the evaluation for EGGI included surveys of and interviews with program beneficiaries (e.g., Afghan officials in budget units). We found these evaluations to be generally reliable to provide information on the results of specific activities conducted under EGGI and ACSS for the period for which the evaluations were conducted.  From the Mission Performance Management Plan (PMP), we identified objectives and associated indicators that pertain to PFM. We then reviewed the associated data provided by USAID that complemented the results framework noted in the Mission PMP. We identified whether USAID data included fiscal year 2011 targets for the PFM indicators we had identified in the PMP. In analyzing USAID project-performance indicators, we reviewed the most current performance data from USAID and its implementing partners. Target and performance data were not available for the first quarter of fiscal year 2010 for ACSS and EGGI because ACSS began operations in the second quarter of fiscal year 2010; although EGGI started in the first quarter of fiscal year 2010, USAID officials reported that they did not expect performance targets and data for this quarter because the project was being set up. As of the date of this report’s publication, no data were available for the fourth quarter of fiscal year 2011, so our analysis does not include data from that quarter. As part of our analysis of USAID project-performance indicators, we reviewed the most current performance data provided by USAID and its implementing partners. We also interviewed USAID officials and the contractor to obtain clarifying information about performance data. Due to unexplained changes and gaps in some target and performance data, we could not verify the reliability of all performance data reported by USAID. Additionally, we obtained documentation and corroborated the effectiveness of U.S. efforts during meetings with Afghan officials from relevant governmental organizations, such as the Independent Administrative Reform and Civil Service Commission. At the commission’s training institute, we met with contractors who provide training to Afghan civil servants as part of the ACSS project.  To assess the results of the efforts of Treasury advisers, we reviewed a sample of adviser monthly reports as well as advisers’ assessment for October 2009 to September 2010, the latest available report for a complete fiscal year. We also interviewed officials from Treasury’s Office of Technical Assistance to obtain information on the adviser assessment and reporting process. Moreover, in Afghanistan we met with MOF officials and contractors in MOF’s budget, treasury, and internal audit offices, and we obtained their input regarding USAID’s and Treasury’s efforts and results. We assessed data obtained for our analysis to be sufficiently reliable to provide an overall assessment of the extent to which Treasury advisers’ assistance has contributed to MOF’s PFM capacity.  We did not independently assess the capability of MOD and MOI finance offices; rather, we relied on CSTC-A’s capability milestone ratings, which are used to measure the capability level of a specific area or department. We spoke with cognizant CSTC-A officials about the reliability of these ratings and also reviewed documentation about the ratings formulation process. In addition, CSTC-A officials provided documentation that showed modifications in their rating process to make it more rigorous. For example, in 2010, CSTC-A introduced interim ratings, which it believed provided greater detail and a more gradual transition from requiring significant coalition assistance to being capable of fully autonomous operations. We believe that the capability milestone ratings are sufficiently reliable to measure the extent of progress in finance and budget functions at MOD and MOI. Additionally, we analyzed finance- and budget-related development and evaluation plans for MOD and MOI, quarterly assessment briefings, Ministerial Development Board presentations, internal control finance reports, and other adviser reports. We compared quarterly assessments and projected capability milestone ratings over time to identify the extent to which DOD was meeting its targets. We also visited MOD’s Finance office in Kabul and interviewed Afghan officials to obtain their input into U.S. efforts to build their ministry’s capacity in finance and budget.

Appendix II: U.S. Allocations of Funds to Afghanistan from Fiscal Year 2002 to Fiscal Year 2011

Since 2002, the United States has allocated more than $72 billion for security, governance, and development to Afghanistan (see table 6).

Appendix III: Afghan Government Budget Formulation, Budget Execution, and Audit Process

According to U.S. agency officials and documents, the budget process in Afghanistan is highly centralized, with the Afghan national government responsible for developing and executing the country’s budget. According to the Department of the Treasury (Treasury), none of the 34 Afghan provinces has the authority to raise revenues or spend public funds. Budgets are developed and executed through central ministries and their provincial offices, called provincial directorates. The provincial governor, appointed by the Afghan President, and the Provincial Council, whose members are elected, can influence the national budget and indicate priorities through the Provincial Development Committee, which also includes representatives from provincial directorates, according to Treasury officials. The ministries then formulate specific budgets and convey them to Ministry of Finance (MOF), which develops the overall national budget. The Afghan parliament, called the National Assembly, has the authority to approve or reject the national budget in its entirety, but cannot make changes to it during the final review process, according to documentation from the U.S. Agency for International Development (USAID). Figure 7 provides an illustration of Afghan budget formulation, execution, and audits at the national and provincial levels.

According to Treasury officials, once the National Assembly approves the budget, ministries prepare and submit allotment requests for the operating budget. MOF establishes quarterly allotments for the operating budget for each ministry. For the development budget, each ministry can submit a budget allotment request only if there is an approved contract (or legal binding agreement) for the project, according to the Treasury adviser. MOF must approve the allotment request before the budget allotment is established. The external budget is not subjected to the budget allotment process. The central ministries and their provincial offices implement programs and execute funds. However, MOF and its provincial offices, called mustofiats, issue payments in response to requests from ministries. Audit of expenditures to ensure proper use of public funds is the responsibility of the Control and Audit Office— Afghanistan’s supreme audit institution—and the audit offices of central ministries, according to Treasury officials.

Appendix IV: CSTC-A Process to Assess Ministry Capacity

The Combined Security Transition Command–Afghanistan (CSTC-A) has established a process to regularly assess and rate the capability of Ministries of Defense (MOD) and Interior (MOI) in various areas including finance and budget. The CSTC-A assessment process uses the objectives in the Ministerial Development Plan as criteria to assess progress in capacity building at the ministries. For example, CSTC-A advisers at MOD provide quarterly assessments, through Narrative Assessment Worksheets, in which they assign a rating and provide a description of progress and challenges associated with each objective. Additionally, MOD advisers complete an online survey to provide information about the scope and quality of their interaction with their Afghan counterparts on a quarterly basis. According to CSTC-A officials, the MOI assessment and rating is based on interviews with advisers and Afghan officials, internal quarterly assessments, and other officials’ reporting on a quarterly basis. Figure 8 shows CSTC-A’s assessment process for MOD.

Senior CSTC-A officials review input from the advisers and agree on an overall rating. This rating and associated details are presented in a quarterly assessment that includes highlights of progress towards the objectives, key objectives to focus on in the next quarter, and an analysis of the ministry’s strengths, weaknesses, opportunities, and threats. These assessments, which are available in English and Dari, are presented to senior coalition forces and Afghan government officials as part of the Ministerial Development Board process. This review process has three key goals: (1) to institutionalize the formal review and oversight of the separate Ministerial Development Plans for various critical ministry functions (see table 7 for areas of focus for MOD and MOI capacity building), (2) to assess progress toward achieving fully autonomous operations including addressing key impediments, and (3) to reinforce and set the current and following quarter’s objectives and initiatives. Although CSTC-A does not currently use adviser surveys for MOI, it is in the process of developing and implementing these, according to CSTC-A officials.

Appendix V: U.S. Agency for International Development (USAID) Requirements for Performance Management and Evaluation

Appendix VI: Fiscal Years 2010 and 2011 Data for USAID PFM Projects

The figures below show fiscal years 2010 and 2011 performance data for U.S. Agency for International Development (USAID) projects that focus significantly on public financial management (PFM) capacity building at the national and provincial level.

Appendix VII: Comments from the U.S. Agency for International Development

Appendix VIII: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

Major contributors to this report were Tetsuo Miyabara, Assistant Director; Mary Koenen; Bruce Kutnick; Mona Sehgal; and Eddie Uyekawa. Technical assistance was provided by Ashley Alley, Pedro Almoguera, Emily Gupta, Jeffrey Isaacs, Gergana Danailova-Trainor, Karen Deans, Denise Fantone, Etana Finkler, Jacqueline Nowicki, Esther Toledo, and Pierre Toureille.