Afghanistan Governance: Performance-Data Gaps Hinder Overall Assessment of U.S. Efforts to Build Financial Management Capacity
Published: 2011-09-20T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The United States has allocated over $72 billion to Afghanistan since 2002. With other international donors, it is focused on transitioning leadership to the Afghan government and has pledged to provide at least 50 percent of its development aid through the Afghan government budget. Improving Afghanistan's public financial management capacity is critical to this transition. In 2010, the Afghan government, consulting with donors, issued a Public Financial Management Roadmap (Roadmap), which outlines goals to improve Afghanistan's capacity to develop a national budget and expend funds. GAO reviewed (1) U.S. efforts to improve the Afghan government's public financial management capacity, including the extent to which they support Roadmap goals, and (2) the extent to which U.S. efforts have improved the government's capacity. GAO reviewed documents and interviewed officials from the U.S. Agency for International Development (USAID); Departments of State, Defense (DOD), and the Treasury (Treasury); World Bank; and Afghan government in Washington, D.C., and Kabul, Afghanistan.
section_title: What GAO Found
paragraphs: USAID, Treasury, and DOD support the Public Financial Management Roadmap (Roadmap) goals through various activities such as (1) USAID projects that provide technical assistance and training to Afghan civil servants, (2) Treasury advisers' assistance to the Ministry of Finance (MOF), and (3) DOD's Combined Security Transition Command-Afghanistan (CSTC-A) that provides support to the Ministries of Defense (MOD) and Interior (MOI). GAO found that these efforts are aligned with the Roadmap goals. USAID provides training and technical assistance mainly through two contractor-implemented projects. One USAID project provides technical assistance to 37 civilian ministries to develop their annual budgets, while another USAID project provides training in areas such as financial management and procurement to Afghan civil servants. Treasury provides technical assistance through 6 advisers in MOF, who work with senior officials on issues such as budget execution. Through CSTC-A, DOD has 22 advisers at MOD and MOI, who advise officials on developing their budgets and strengthening the payroll system to improve accuracy. The overall extent to which U.S. efforts have improved the public financial management capacity of the Afghan government cannot be fully determined because (1) U.S. agencies have reported mixed results, and (2) weaknesses in USAID's performance management frameworks, such as lack of performance targets and data, prevent reliable assessments of its results. USAID's evaluations of its two public financial management projects indicate that some activities were successfully completed, while others were terminated because these activities were not deemed useful. Treasury advisers assessed that although their assistance at MOF had a positive effect, some results had limitations. For example, advisers assessed that their efforts to design reports for improved communication of financial information were not as successful as they had expected. Additionally, CSTC-A assessed that while MOD has made progress since 2008 and can perform critical financial management functions with minimal international support, MOI still needs significant international support for such operations. In early 2010, CSTC-A projected that MOD would transition to needing no coalition support for finance and budget functions by January 2012, and MOI would reach a similar goal by March 2012. However, in early 2011, CSTC-A extended time frames for meeting its benchmarks for MOD and MOI to March 2012 and November 2012, respectively. Regarding deficiencies in USAID's performance management framework, both the USAID Mission performance management plan and project-specific plans lack performance targets as required for each indicator related to public financial management. Additionally, implementing partners, such as contractors, have not consistently reported performance data for all indicators. Moreover, baselines for public financial management capacity of civilian ministries have not yet been established. In the absence of baselines, performance targets, and data, it is difficult to assess the extent to which USAID efforts have increased the public financial management capacity of Afghan ministries.
section_title: What GAO Recommends
paragraphs: GAO recommends that the USAID Administrator take steps to (1) establish performance targets in its Mission Performance Management Plan (PMP); (2) ensure implementing partners' PMPs include baselines and approved targets; and (3) ensure implementing partners routinely report performance data. USAID concurred with GAO recommendations and is taking steps to address them.

section_title: Letter
paragraphs: The United States has allocated more than $72 billion to stabilize and  build Afghanistan since 2002 and, along with the international community,  has begun focusing on transitioning leadership for security, governance,  and economic growth to the government of Afghanistan. In January 2010,  the United States joined other donors by pledging that within the next 2  years it would provide 50 percent or more of its Afghan development aid  through the Afghan government’s national budget. Such direct assistance  is intended to help develop the capacity of Afghan government ministries.  However, increase in direct assistance from donors is conditioned on the  Afghan government’s progress in areas such as reducing corruption and  strengthening its public financial-management (PFM) systems. In July  2010, the Afghan government, in consultation with donors, issued a plan  called the Public Financial Management Roadmap (Roadmap) to improve  its capacity to develop a national budget, expend funds, and increase  accountability and transparency.
Due to broad congressional interest in U.S. efforts to improve the Afghan  government’s governance capacity, we performed this review under the  Comptroller General’s authority to conduct work on his own initiative. We  examined (1) U.S. efforts to improve the Afghan government’s PFM  capacity, including the extent to which these efforts support and are  aligned with the PFM goals identified by the Afghan government and the  international community; and (2) the extent to which U.S. efforts have  helped to improve the Afghan government’s PFM capacity.
To address these objectives, we focused on U.S. agencies’ PFM  capacity-building activities since 2009, after President Obama announced  the U.S. strategy for Afghanistan and Pakistan. We reviewed documents  and records from the U.S. Departments of State (State), the Treasury  (Treasury), and Defense (DOD), and the U.S. Agency for International  Development (USAID). Additionally, we reviewed documents from the  Afghan government and other donors, such as the World Bank. We  interviewed officials from DOD, State, Treasury, and USAID in  Washington, D.C., and in Kabul, Afghanistan. Additionally, in Kabul, we  interviewed other international donors, such as the World Bank and the  United Kingdom’s Department for International Development (DFID), as  well as Afghan government officials from organizations including the  Ministry of Finance (MOF), Ministry of Defense (MOD), and the  Independent Administrative Reform and Civil Service Commission.
To identify U.S. efforts to improve the Afghan government’s PFM  capacity, we obtained documentation on relevant plans, projects, and  activities, and interviewed officials from U.S. agencies including USAID,  Treasury, and DOD. To identify the extent to which U.S. efforts address  the Roadmap goals, we identified the U.S. PFM capacity-building efforts  and compared these efforts with the Roadmap. We also obtained a  description of PFM-related activities funded by other donors, including the  World Bank and the United Kingdom. To assess the results of U.S. PFM  efforts, we reviewed U.S. agencies’ assessments of their efforts and  analyzed performance plans and reported results from USAID, Treasury,  and DOD. Additionally, we obtained documentation and attempted to  corroborate the effectiveness of U.S. efforts during meetings with Afghan  officials from relevant ministries, such as MOF and MOD. We assessed  data obtained for our analysis to be sufficiently reliable for the purposes  for which they were used in this report.
We conducted this performance audit from November 2010 through  September 2011 in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives. Details on our  objectives, scope, and methodology are contained in appendix I.
subsections: section_title: Background
paragraphs: Since 2002, the United States has allocated more than $72 billion for  development, governance, and security in Afghanistan. See appendix II  for a breakdown by year of U.S. allocation of funds for Afghanistan from  fiscal year 2002 to fiscal year 2011. After almost a decade of donor-led  efforts in Afghanistan, the United States and international donors have  increased their focus on transitioning leadership to the Afghan  government. The government of Afghanistan and the United Nations  sponsored international conferences in London (January 2010) and Kabul  (July 2010), which were attended by senior officials representing about 70  major donors, including the United States. Participants committed to  supporting Afghan government leadership and ownership and agreed to  increase the percentage of development aid delivered through the Afghan  government to 50 percent by 2012, if the Afghan government showed  progress in areas such as strengthening its public financial-management  systems and reducing corruption. In November 2010, representatives  from 48 countries contributing to the United Nations–mandated  International Security Assistance Force agreed to a plan to begin  transitioning security responsibilities to the Afghan government in 2011  and to complete this transition by 2014. In June 2011, the U.S. President  reiterated that the process of security transition in Afghanistan will be  complete by 2014. According to donors and the Afghan government,  ensuring that public funds are used in a transparent and responsible  manner is necessary for effective governance. Key elements of  Afghanistan’s PFM include developing a national budget that represents  the country’s priorities (budget formulation), spending the approved  budget in the intended time frame (budget execution), and ensuring that  funds are used as intended (through audits).
subsections: section_title: Public Financial Management Roadmap to Improve Afghan Government Capacity
paragraphs: In July 2010, the Afghan government published a plan, called the Public  Financial Management Roadmap, to strengthen the Afghan government’s  performance in three key areas at the national and provincial level:  budget formulation, budget execution, and accountability and  transparency of financial management. Additionally, a cross-cutting area  is to increase the capacity of Afghan ministries. MOF led the development  of the Roadmap, with input from the World Bank and International  Monetary Fund, as well as partner governments such as the United  States and United Kingdom. Table 1 provides an overview of the  Roadmap’s components and their key areas. Additionally, appendix III  provides information on the Afghan government’s budget process.
U.S. embassy guidance issued since the release of the Roadmap states  that U.S. governance activities are to increase focus on developing the  capacity of Afghan civilian agencies in budget prioritization and execution  to achieve progress towards Afghan self-governance in anticipation of the  security transition.
section_title: Multiple Donors Provide PFM Capacity-Building Assistance
paragraphs: Several international donors are helping the Afghan government improve  its PFM capacity. The World Bank, the United Kingdom, and the United  States are key donors providing assistance to build PFM capacity of  Afghan civilian agencies. Figure 1 shows donor PFM assistance to  Afghan government entities at the national and provincial levels.
Through the Combined Security Transition Command–Afghanistan  (CSTC-A), DOD helps build the PFM capacity of Afghanistan’s security  ministries, the Ministries of Defense (MOD) and Interior (MOI). CSTC-A  is primarily U.S.-funded and staffed. In April 2009, the United States and  its North Atlantic Treaty Organization (NATO) allies agreed to establish  NATO Training Mission–Afghanistan (NTM-A) to oversee institutional  training and development of the Afghan National Security Forces. NTM- A/CSTC-A now operates as an integrated NATO and U.S. command.
International donors provide a significant share of the total funding for  Afghanistan in the form of funds channeled through the Afghan  government budget as well as “off-budget” assistance that does not use  the Afghan government’s budget system. According to our preliminary  analysis, the total estimated budget for Afghanistan for March 21, 2010– March 20, 2011 was $10.2 billion; donors were expected to fund about  two-thirds of the Afghan government budget of $4.4 billion and the entire  reported off-budget assistance of $5.8 billion. Figure 2 shows the total  funding to Afghanistan through the Afghan government budget and off- budget donor assistance for March 2010 to March 2011.
section_title: Various U.S. Agency Efforts Support Afghanistan’s PFM Capacity-Building Goals
paragraphs: USAID, Treasury, and DOD support the Roadmap goals through various  activities such as (1) USAID projects that provide technical assistance  and training to Afghan civil servants, (2) Treasury advisers’ assistance to  MOF, and (3) DOD’s mentoring and coaching assistance through CSTC- A to MOD and MOI. USAID provides training and technical assistance  mainly through two contractor-implemented projects. Treasury provides  technical assistance through 6 advisers in MOF, who work with senior  officials on issues such as budget execution. Through CSTC-A, DOD has  22 advisers at MOD and MOI, who advise officials on developing their  budgets and strengthening the payroll system to improve accuracy.
subsections: section_title: USAID Provides Technical Assistance and Training to Afghan Civilian Ministries
paragraphs: USAID is funding several projects that provide training, mentoring and  coaching, and technical assistance that address Roadmap goals. USAID  has two primary PFM capacity-building projects—the Economic Growth  and Governance Initiative (EGGI) project that has a contract value of  approximately $92 million over 5 years, and the Afghanistan Civil Service  Support (ACSS) project that has a contract value of approximately $84  million over 1 year. Both these projects are implemented by Deloitte  Consulting, which has hired international contractors and local Afghans.  EGGI supports all four PFM goals of strengthening budget formulation,  improving budget execution, increasing accountability and transparency,  and improving ministry capacity. For example, according to USAID, EGGI  contractors provided program-budget technical assistance to 37 Afghan  central ministries and agencies so that these entities can formulate  annual budgets that conform to MOF guidance. Examples of EGGI’s  technical assistance include contractors helping MOF’s fiscal policy unit  develop a fiscal forecasting model and providing policy assistance to the  Ministers of Finance and Economy. ACSS similarly supports most  Roadmap goals through training it provides in financial management and  procurement to Afghan civil servants in Kabul and in the provinces. Table  2 provides information on EGGI and ACSS, as well as other USAID  projects that, while not exclusively focused on PFM capacity building,  provide some PFM capacity-building assistance.
A July 2009 Treasury assessment, which was part of a proposal for  enhanced civilian assistance to Afghanistan, identified the need for  advisers to assist with public expenditure management in MOF’s offices  of treasury, budget, and internal audit. Treasury assigned six advisers  from its Office of Technical Assistance to assist various MOF offices.  These advisers are embedded in MOF offices and provide assistance that  supports Roadmap goals by advising senior Afghan MOF officials,  analyzing and monitoring budget expenditures and controls, and  developing accounting and administrative manuals. Table 3 indicates  location of Treasury advisers within MOF, type of assistance provided,  and the PFM goals support by this assistance.
As of April 2011, DOD, through CSTC-A, and other coalition partners had  587 advisers and mentors working with their Afghan counterparts to build  the capacity of MOD and MOI in about 30 functional areas, such as  intelligence, personnel management, logistics, and finance and budget.  Of the 587 advisers and mentors, CSTC-A had 7 advisers at MOD and 15  at MOI who provided advice in the finance and budget area. CSTC-A is  working to ensure that MOD and MOI are capable of operating without  coalition assistance by 2014.
DOD employees or military personnel under CSTC-A provide leadership  for capacity building in the finance and budget area; they are partnered  with specific Afghan officials in MOI and MOD units and oversee the work  of U.S. contractors. CSTC-A has developed comprehensive Ministerial  Development Plans to guide its capacity-building efforts at MOI and MOD,  including plans for finance and budget (see app. IV for a listing of all focus  areas). Table 4 shows the specific goals in the finance and budget plans  for MOD and MOI for 2011.
According to CSTC-A officials, advisers at MOD focus on building Afghan  capacity and generally do not carry out the work of MOD officials. At MOI,  advisers meet with finance and budget partners daily to help formulate  budget and develop pay procedures, such as the electronic payroll  system. As shown in table 5, the advisers provide advice that supports  the Roadmap goals of improving budget formulation and execution, and  increasing accountability and transparency. According to DOD officials,  the goal of their capacity-building efforts is to ensure that MOD and MOI  are fully capable of carrying out key functions such as finance and budget  without coalition assistance.
According to the official terms of reference for CSTC-A advisers, they  advise, facilitate, and collaborate with their MOI and MOD counterparts.  The guidance states that on rare occasions, a crisis will occur and the  adviser will have to perform the relevant functions. According to the  CSTC-A officer directing finance capacity building at MOD, the advisers  should allow their Afghan counterparts to make mistakes and learn from  them, unless the mistake is critical.
section_title: Results of U.S. Efforts to Improve Afghanistan’s PFM Capacity Cannot Be Fully Determined
paragraphs: The overall results of U.S. efforts cannot be fully determined because (1)  U.S. agencies providing PFM capacity assistance to the Afghan  government have reported mixed results of their efforts, and (2)  weaknesses in USAID’s performance management plans and  frameworks, such as lack of performance targets and data, prevent  reliable assessments of USAID’s results. USAID’s evaluations of its two  primary PFM projects indicate that some activities were successfully  completed, while others were terminated because their usefulness was  questionable. Treasury advisers assessed that although their assistance  at MOF had a positive effect, the results fell short of what they were trying  to accomplish. Additionally, CSTC-A reported that while MOD has  progressed to being able to perform critical financial-management  functions with minimal coalition support, MOI continues to rely on coalition  support for these functions. Moreover, CSTC-A has extended transition  milestones for security ministries. Regarding USAID’s performance  management, target and performance data have not been approved for  PFM efforts at the USAID Mission level as well as for PFM-focused  projects.
subsections: section_title: U.S. Agencies Report Mixed Results of PFM Capacity-Building Activities
paragraphs: USAID, Treasury, and DOD reported mixed results related to U.S. efforts  to build Afghan PFM capacity.
USAID has reported output and some outcome information for its EGGI  and ACSS projects, and conducted project evaluations that indicate  positive and negative results. In 2010, the EGGI project trained  approximately 800 government employees on how to develop a program  budget, which allows budget units to request funding based on expected  outputs and outcomes of specific programs. In addition, EGGI provided  technical assistance to 37 budget units to help them prepare annual  budgets for Afghan fiscal year March 2011 to March 2012. According to  USAID, this training was effective because all budget units prepared  program budgets and submitted them in a timely manner. USAID also  reported that 10 ministries “graduated” from budget training as these units  can prepare their own budgets with little or no assistance from the EGGI  team. According to USAID evaluators, program budgeting represented a  major programmatic accomplishment for the EGGI project.
However, other EGGI activities were not as successful, in part because  USAID and the project’s implementing partner did not adequately consult  with relevant stakeholders, such as other international donors conducting  related activities. For example, EGGI developed a Revenue  Reconciliation Database to report revenue collections to the Afghanistan  Revenue Department in real time. According to USAID evaluators, the  EGGI revenue database does not directly connect with the Afghan  government’s accounting system, as it was intended to do, or the  revenue-collection system being developed by DFID; therefore, the  database does not provide information that is useful for planning or  analysis. USAID has since terminated this activity. However, USAID  officials noted that because of delays in the installation of the DFID  system, the USAID database is the only functional system in use by  MOF’s Afghanistan Revenue Department to collect and report revenue  data. Additionally, USAID’s evaluators noted that USAID and DFID did  not adequately coordinate efforts related to creating a medium taxpayer  office in Herat.
According to USAID, ACSS trained approximately 16,000 civil servants in  Kabul (5,759 participants) and in 26 provinces (10,121 participants) from  February 2010 to March 2011. It provided training in five core subjects,  including financial management and procurement. According to  evaluations by USAID and the Afghan government’s Independent  Administrative Reform and Civil Service Commission, ACSS training  contributed to improvements in civil servants’ performance. However,  USAID evaluators noted that a lack of baseline data makes it impossible  to measure the extent of these improvements.
In addition, USAID’s evaluation of ACSS cites several challenges  resulting from the project’s Afghan-led approach, which involved Afghan  officials in making decisions related to project implementation. Some  Afghan officials viewed the Afghan-led approach to mean that they did not  need to ensure accountability for some inventory. As a result, in some  cases, USAID inventory stickers were reportedly removed from assets  before the items were transferred to Afghan control, causing  accountability issues, according to USAID’s evaluation. In addition, some  Afghan managers interpreted the concept of Afghan-led to mean that they  could prevent the project’s monitoring and evaluation team from obtaining  data needed to assess project results, according to USAID’s evaluation.  For example, when ACSS staff requested monitoring and evaluation data,  some Afghan managers responded that they were not required to provide  data and that such requests represented an inappropriate desire to exert  control over their operations. As a result, the quality of monitoring and  evaluation of the project suffered.
To assess the effect of Treasury advisers’ technical assistance to MOF,  Treasury’s Office of Technical Assistance requires each adviser to submit  monthly reports and an annual evaluation. For the period October 2009  to September 2010, Treasury advisers assessed that although their  assistance related to budget and financial management at MOF had a  positive effect, the results were less than what they were trying to  accomplish. For instance, the advisers gave a low score for results  related to their efforts to design management reports to improve  communication of financial information and enhanced budget control.  However, the advisers assigned a high rating for results related to  developing training materials to support capacity building. The advisers  also assessed the level of commitment and involvement of their Afghan  counterparts as being above average, with room for improvement.
CSTC-A has established a process to assess progress on key objectives  and rate the capacity of MOD and MOI, in areas including finance and  budget functions, on a quarterly basis. According to CSTC-A’s  assessments, MOD has progressed from significant reliance on coalition  support in 2008 to being capable of executing core functions related to  finance and budget with minimal coalition support by 2011. MOI has  progressed from not being able to accomplish finance and budget  functions on its own in 2008 to being able to carry out core functions with  significant coalition assistance by 2011. Figure 3 shows actual and  projected ratings for MOD and MOI’s capability in finance and budget  operations. Progress at MOI has been slow, in part because CSTC-A’s  capacity-building efforts at MOI started in 2006, several years after its  efforts began at MOD in 2002. CSTC-A has extended the time frames to  meet interim and final capacity-building goals at MOD and MOI due to  various challenges. See appendix IV for a description of CSTC-A’s  capacity assessment process.
According to CSTC-A assessments, MOD transitioned from needing  significant coalition support to accomplish finance and budget functions to  requiring some coalition assistance in December 2008. Although CSTC- A’s rating of MOD has not changed since then, CSTC-A officials  documented progress at MOD. For example, CSTC-A’s quarterly  assessment of MOD for January through March 2011 reported that MOD    executed critical functions, such as paying of salaries to soldiers, with  minimum coalition support;  implemented a policy to transition pay operations from U.S.  embedded teams to Afghan army finance officers;  streamlined pay systems and expanded electronic funds transfer  capability so that salaries of 97 percent of Afghan soldiers are  deposited directly into their bank accounts;    executed its budget for salaries and other operational expenditures at  over 99 percent for Afghan fiscal year 2010-2011; and  implemented an integrated program budget-formulation process for  this fiscal year, such that the budget request was tied to the ministry’s  strategic goals and performance measures.
However, MOD has not progressed to needing only coalition oversight in  carrying out finance and budget functions because of a lack of sufficient  delegation of budget authority from MOD’s central finance office to  subordinate units and commands of the Afghan army. Although CSTC- A officials worked with MOD to clarify roles and responsibilities of variou offices, highly centralized budget authority prevented MOD from  developing and executing the budget with input from the commands of  the Afghan National Army. Therefore, CSTC-A advisers are still involved  to ensure that MOD develops integrated program budgets.
Although MOI progressed from an inability to accomplish its finance and  budget functions in March 2008 to being able to accomplish these  functions with significant coalition support by March 2009, MOI’s capacity  is far behind MOD’s. CSTC-A officials stated that their capacity-building  efforts at MOI started in 2006, several years after CSTC-A’s capacity- building efforts began at MOD in 2002. CSTC-A’s quarterly assessment  of MOI for January through March 2011 rates MOI at the same level as in  2009, citing factors such as lack of consolidation of MOI’s personnel  databases due to delays in contracts and lack of telecommunications  network expansion. As a result, CSTC-A or Afghan officials are not  certain that all salary payments are being made to legitimate Afghan  National Police personnel rather than “ghost” employees. Additionally,  current MOI finance office employees do not have formal training in  properly executing the budget and salary functions. Moreover, according  to CSTC-A assessment, MOI has a top-down organizational culture in  which officials tend to delegate key decisions to the minister. For  example, MOI’s Program Budget Advisory Committee, which is  responsible for reviewing expenditures, is reluctant to make decisions  affecting budget execution and pushes these up to the Minister of Interior.  This reportedly resulted in necessary actions not being taken in a timely  fashion.
CSTC-A’s goal is for MOD and MOI to achieve self-sustainability prior to  2014, when coalition forces are scheduled to transition security  responsibilities to the Afghan government. However, CSTC-A has  extended the time frame for MOD and MOI finance offices to meet interim  and final goals due to various challenges. For example, in early 2010,  MOD’s finance office was expected to reach the interim goals of operating  autonomously with only coalition oversight by March 2011 and the final  goal of autonomous operations by January 2012. However, in March  2011 the projected date for meeting the interim goal was extended by 3  months to June 2011 and for meeting the final goal was extended by 6  months to June 2012. Similarly, we have previously reported that in  several instances DOD pushed out completion dates related to training of  Afghan army and police forces.
The time frames for MOI to reach interim and final goals have also been  revised. For example, in early 2010, CSTC-A expected MOI’s finance  office to operate with some coalition assistance by March 2011 and  become fully autonomous by March 2012. However, in March 2011, the  time frames were revised so that the interim and final goals were  expected to take 8 additional months each, and projected to be  accomplished by November 2011 and November 2012, respectively. The  delay was attributed to problems in implementing the electronic payroll  system. Despite these delays, CSTC-A officials noted that they expect  MOI to become fully autonomous by identifying and addressing high-risk  areas, such as electronic payroll.
section_title: USAID’s Performance Management Frameworks Lack Performance Targets and Data
paragraphs: USAID’s Automated Directives System (ADS) establishes performance  management and evaluation procedures USAID is expected to follow with  respect to planning, monitoring, and evaluating its programs. While  USAID has noted that Afghanistan is an insecure environment in which to  implement its programs, the agency has generally maintained the same  performance management and evaluation procedures as it does in other  countries in which it operates. For PFM capacity building in Afghanistan,  we found a lack of compliance with USAID guidance at the mission level  and at the implementing-partner level. Additionally, USAID evaluations  also note weaknesses in the performance management of PFM projects.  Appendix V presents a summary of the planning, monitoring, and  evaluating requirements that make up USAID’s performance  management and evaluation procedures.
At the mission level, ADS requires USAID officials to complete a Mission  performance management plan for each of its high-level objectives as a  tool to manage its performance management and evaluation procedures.  The guidance also requires that USAID establish performance targets for  each of the indicators. We previously reported in July 2010 that USAID  has operated without a required Mission performance management plan  for Afghanistan since the end of 2008. Subsequently, USAID issued a  new performance management plan that realigned objectives based on  U.S. strategies adopted in 2009 as well as agreements made at the  London Conference in January 2010 and the Kabul Conference in July  2010. This plan represents the U.S. Mission’s tool to plan and manage  the process of assessing and reporting progress towards assistance and  foreign policy objectives in Afghanistan. As figure 4 shows, the plan  contains a results framework that includes two PFM-related objectives  and several related indicators, but lacks performance targets for each  indicator as required.
The plan noted that baselines and targets for each indicator would be  established in the first and second quarters of fiscal year 2011.  Additionally, according to the plan, many indicators do not have baseline  data or targets because some indicators are either new or were in the  previous plan but data were never collected for them. This is contrary to  ADS, which requires that targets be established for each performance  indicator. In addition, some indicators are for proposed activities and need  to be finalized. For example, one of the indicators, “Percent increase in  standardized Public Financial Management assessment scores,” relies on  baseline data for Afghan civilian ministries that is not yet available. In  April 2011, donors initiated joint assessments of 14 Afghan civilian  ministries, which account for 90 percent of the development budget, to  establish baselines of their PFM capacity. These assessments are due by  2012, with preliminary information only available for MOF.
ADS documents USAID’s performance management and monitoring  procedures. Project implementers must follow these requirements  outlined in USAID award documents. For example, project implementers  are required to identify performance indicators, define their project’s  “starting point” by establishing baselines, and define changes that signal  success by establishing performance targets for each project year. In  addition, project implementers are required to regularly collect, analyze,  and interpret performance data in order to improve their ability to make  project adjustments in a timely manner. The performance management  frameworks for PFM capacity-building projects did not meet USAID  guidance because of deficiencies such as a lack of baselines, targets,  and performance data.
The performance indicators for EGGI and ACSS related to PFM capacity- building activities do not consistently provide baselines, performance  targets, or actual performance data for each indicator, as required. For  example, as shown in figure 5, for fiscal year 2010, EGGI’s implementing  partner did not establish performance targets or provide actual data on a  quarterly or annual basis for training activities conducted by EGGI.  Additionally, EGGI’s implementing partner did not report target or actual  data for the indicator called “Annual Tax Revenues Collected in Priority  Revenue Mustofiats” because MOF did not release revenue data during  the project’s performance year. Subsequently, USAID provided fiscal year  2010 quarterly and annual data for this indicator to GAO in the fourth  quarter of fiscal year 2011. While these data may provide information that  is useful for project evaluation, since they were provided retroactively,  they were unavailable for USAID to monitor the progress of the activity  during the performance year. Similarly, ACSS’s implementing partner did  not provide evidence of baseline, quarterly performance targets, or actual  quarterly data for any of the project’s PFM training indicators. This lack of  targets and actual performance data rendered associated indicators  ineffective for either tracking the progress of associated activities or  assessing the extent to which USAID’s implementing partners met interim  goals. Appendix VI provides the available data for fiscal years 2010 and  2011 for EGGI and ACSS.
For fiscal year 2011, although USAID-funded evaluations note that EGGI  improved its performance measures and data collection, our analysis  indicates that both EGGI and ACSS did not establish performance targets  for each indicator, or report actual data for completed quarters, as  required. As shown in figure 6, EGGI did not establish quarterly or annual  targets for its training activities that include key areas, such as program  budgeting and tax administration, noted in EGGI’s workplan. Similarly,  ACSS did not establish baselines for any of its PFM indicators, which  would make it difficult to assess the project’s training accomplishments.
In addition to the lack of compliance with USAID guidance noted above,  USAID-funded evaluations of EGGI and ACSS have noted other  weaknesses in the projects’ performance management frameworks. For  example, the evaluator of the EGGI project was critical of the large  number of performance indicators for the project’s first year that do not  demonstrate a direct link between project efforts and improved capacity of  the Afghan government. More specifically, the evaluator noted that only 4  of 7 indicators related to EGGI’s PFM capacity-building component are  directly affected by the project’s work. The other indicators—national  domestic revenues, national tax revenues, and national nontax  revenues—are indirectly affected by the project’s work. Another EGGI  evaluation noted that since MOF was not publishing data for the indicator  related to annual taxes in priority provinces, the implementer should  replace it with a different indicator that can be used to monitor activities.
The evaluator of the ACSS project noted that the project’s work plan,  performance management plan, and operations manual were not  sufficiently rigorous or comprehensive and, given the scope and scale of  the project, the evaluator expected higher-quality project documentation.  The evaluator also noted that the project’s performance monitoring  framework lacks results-based information about the effectiveness of the  project’s technical advisers.
In addition to weaknesses in the performance management frameworks  of USAID’s PFM capacity projects, we have previously noted similar  deficiencies in other USAID projects in Afghanistan. For example:  In July 2008, we reported that, among other things, limitations in  USAID’s data collection and performance evaluation frameworks  impeded the agency’s ability to evaluate the effects of its roads  projects.  In July 2010, we reported that USAID did not assure that all indicators  had targets for the eight GAO-reviewed agriculture projects.  In November 2010, we reported that four of the six implementers of  GAO-reviewed water projects did not always establish targets for  performance indicators.
We are following up on USAID’s progress on these recommendations.  Additionally, the USAID Administrator committed to tracking resources  against outcomes as effectively as possible at a congressional hearing in  July 2010.
section_title: Conclusions
paragraphs: The United States and other international donors have begun to focus on  building the Afghan government’s capacity for a successful transition of  leadership for security and governance to Afghanistan. Improving the  Afghan government’s ability to manage its public finances is an important  part of this transition effort. USAID, Treasury, and DOD, along with other  international donors, have undertaken various efforts to address the  Afghan government’s PFM capacity. While DOD and Treasury have  assessed and reported mixed results based on their efforts, the overall  effect of U.S. efforts is not known because USAID, which has a key role  in building Afghan civilian ministries’ PFM capacity, has not consistently  established baselines and targets, or reported actual performance data.  We have previously reported on deficiencies in USAID’s performance  management efforts in Afghanistan and made recommendations to  improve the assessment of USAID program performance and the efforts  of USAID’s implementing partners. During a congressional hearing in July  2010, USAID’s Administrator identified defining, tracking, observing, and  reporting on results of USAID projects as a priority, noting that it is  important to determine how USAID efforts contribute to the U.S. strategy  for Afghanistan. He also committed to tracking resources against  outcomes as effectively as possible. The lack of approved mission-level  and implementing partner-level performance targets calls into question  USAID’s efforts to live up to the Administrator’s commitment. Given the  importance of USAID efforts to improving Afghanistan’s PFM capacity  and the need for reliable performance results data to base future  development assistance and funding decisions on, it is vital that USAID  take steps to ensure its performance management efforts are consistently  implemented.
section_title: Recommendations for Executive Action
paragraphs: We recommend that for public financial management (PFM) efforts, the  USAID Administrator take the following three actions: (1) establish  targets, as required, for each PFM-related performance indicator in its  Mission Performance Management Plan for Afghanistan, (2) take steps to  ensure that the USAID-approved performance management plan for each  implementing partner includes baseline data and targets for each  indicator, and (3) ensure that implementing partners report performance  data at the frequency established in the performance management plan.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DOD, State, Treasury, USAID, and  the World Bank for comment and review. USAID provided written  comments, which are reprinted in appendix VII, as well as technical  comments, which we have incorporated as appropriate. DOD, State,  Treasury, and the World Bank had no comments.
USAID concurred with all three recommendations and noted that it has  started taking steps to address these. Specifically, USAID stated that it  had started a review of the Mission performance management plan to  determine if adequate PFM-related performance indicators, including  baseline data and targets, are included. Additionally, USAID noted that it  has commenced a comprehensive review of all awards to USAID  implementing partners working on projects and activities in the PFM  sector to ensure USAID-approved PMPs are in place for each award and  each implementing partner’s performance management plan includes  sufficient baseline data and targets for each indicator. As part of this  review, USAID noted that it is also examining the reporting requirements  set forth in relevant awards, including performance requirements, and  would take corrective action, as needed.
We are sending copies of this report to the appropriate congressional  committees, the Secretaries of Defense, State, and Treasury, as well as  the Administrator of USAID, and other interested parties. The report also  is available at no charge on the GAO Web site at http://www.gao.gov.
If you or your staff members have any questions about this report, please  contact me at (202) 512-7331 or johnsoncm@gao.gov. Contact points for  our Offices of Public Affairs and Congressional Relations may be found  on the last page of this report. GAO staff who made key contributions to  this report are listed in appendix VIII.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This report examines (1) U.S. efforts to improve the Afghan government’s  public financial management (PFM) capacity, including the extent to  which these efforts aligned with the PFM goals identified by the Afghan  government and the international community; and (2) the extent to which  U.S. efforts have helped to improve the Afghan government’s PFM  capacity.
To address these objectives, we focused on U.S. agencies’ PFM  capacity-building activities since 2009, when President Obama concluded  the administration’s review of U.S. efforts in Afghanistan and announced  a strategy for Afghanistan and Pakistan. We reviewed documents and  records from the U.S. Departments of Defense (DOD), State (State), and  the Treasury (Treasury), and the U.S. Agency for International  Development (USAID). We selected these agencies because they  provide guidance or assistance related to U.S. PFM capacity building for  the Afghan government. We also reviewed documents from the Afghan  government and other donors, such as the World Bank. We interviewed  officials from DOD, State, Treasury, and USAID in Washington, D.C., and  in Kabul, Afghanistan. Additionally, in Kabul, we interviewed officials from  other international donors providing PFM assistance, such as the World  Bank and the United Kingdom, as well as Afghan government officials  from organizations including the Ministry of Finance (MOF), Ministry of  Defense (MOD), Ministry of Public Health, and the Independent  Administrative Reform and Civil Service Commission.
To inventory and describe U.S. government efforts to build the Afghan  government’s PFM capacity, we interviewed officials from U.S. agencies,  including USAID, Treasury, and DOD, and reviewed relevant documents.  We also compared U.S. agency efforts with the key components of the  Public Financial Management Roadmap (Roadmap), which represents  goals agreed upon by donors and the Afghan government to improve the  Afghan government’s PFM capacity.    For USAID PFM capacity-building work we reviewed documents— including base contracts, contract modifications and scope of work— that describe USAID-funded PFM capacity building work in  Afghanistan. To identify USAID PFM capacity-building projects since  2009, we interviewed officials from USAID’s Office of Democracy and  Governance, and Office of Economic Growth. Some USAID projects  are divided into major areas of focus called components. Two USAID  projects—Economic Growth and Governance Initiative (EGGI) and  Afghanistan Civil Service Support (ACSS)—have at least one  component related to building the PFM capacity of Afghan  government entities at the national and provincial levels. We selected  these projects for a more detailed review. Some projects did not have  a component focused on building the Afghan government’s capacity in  budget formulation, budget execution, or transparency and  accountability. However, we included them in our inventory because  USAID officials noted that these projects contribute to increasing the  Afghan government’s PFM capacity. We decided to focus on projects  that primarily build PFM capacity at the national and provincial levels  because the Roadmap prioritizes capacity building at these levels.    To describe the PFM efforts of Treasury officials, we reviewed  documents including Treasury’s 2009 assessment of the need for  advisers to assist the Afghan government with public-expenditure  management and a summary of the roles of advisers currently at  MOF. In Kabul, we also interviewed the Treasury advisers and met  with them at MOF.    To describe DOD’s PFM capacity-building work, we reviewed  documents including organizational charts and staffing levels, policy  guidance and operating procedures, and various implementation,  development, and evaluation plans for both MOD and Ministry of  Interior (MOI). For information on the role of mentors and advisers in  building capacity at MOD and MOI, we interviewed senior DOD  officials at the Combined Security Transition Command–Afghanistan  (CSTC-A), who were providing the services to both MOI and MOD, as  well as MOD Finance officials who were receiving these services. In  addition, we reviewed MOD and MOI training manuals, course  materials, and adviser guides developed specifically for mentoring  and capacity building in these ministries. We also reviewed the  contracts providing mentoring and training services to MOD and MOI.    To assess alignment of U.S. efforts and goals with the PFM goals of  international donors and the Afghan government, we compared efforts  undertaken by USAID, Treasury, and DOD with the key components  of the Roadmap. We also reviewed U.S. plans and guidance issued  since 2009, including the U.S. Foreign Assistance for Afghanistan  Post Performance Management Plan (2011-2015) and U.S. Mission  guidance to U.S. agencies on PFM issues in Afghanistan. For U.S.  military efforts we reviewed goals stated in CSTC-A Ministerial  Development Plans for finance and budget functions for MOD and  MOI. We also interviewed U.S. officials from USAID, Treasury, and  CSTC-A to corroborate the extent to which the Roadmap guided their  efforts in working with Afghan ministries. Additionally, we obtained  relevant documentation from MOF officials, including the  implementation plan and a 6-month progress report related to the  Roadmap.    To assess U.S. efforts within the broader context of other donor  efforts, we interviewed donors including the World Bank and United  Kingdom’s Department for International Development (DFID) in Kabul.  We also obtained documents, such as the Roadmap Implementation  Plan and Technical Assistance Summary Reports, and interviewed  officials from the World Bank and MOF.
To assess the results of U.S. PFM efforts, we analyzed relevant  documentation from U.S. agencies.    For USAID, we analyzed the performance plans for the Mission as  well as for PFM-specific projects. Additionally we analyzed the results  reported in project evaluations as well as monthly, quarterly, and  annual reports for the two main PFM-focused projects. USAID  evaluations were based on document reviews as well as interviews  with contractors, USAID officials, and Afghan government officials.  Additionally, the evaluation for EGGI included surveys of and  interviews with program beneficiaries (e.g., Afghan officials in budget  units). We found these evaluations to be generally reliable to provide  information on the results of specific activities conducted under EGGI  and ACSS for the period for which the evaluations were conducted.    From the Mission Performance Management Plan (PMP), we  identified objectives and associated indicators that pertain to PFM.  We then reviewed the associated data provided by USAID that  complemented the results framework noted in the Mission PMP. We  identified whether USAID data included fiscal year 2011 targets for  the PFM indicators we had identified in the PMP.  In analyzing USAID project-performance indicators, we reviewed the  most current performance data from USAID and its implementing  partners. Target and performance data were not available for the first  quarter of fiscal year 2010 for ACSS and EGGI because ACSS began  operations in the second quarter of fiscal year 2010; although EGGI  started in the first quarter of fiscal year 2010, USAID officials reported  that they did not expect performance targets and data for this quarter  because the project was being set up. As of the date of this report’s  publication, no data were available for the fourth quarter of fiscal year  2011, so our analysis does not include data from that quarter. As part  of our analysis of USAID project-performance indicators, we reviewed  the most current performance data provided by USAID and its  implementing partners. We also interviewed USAID officials and the  contractor to obtain clarifying information about performance data.  Due to unexplained changes and gaps in some target and  performance data, we could not verify the reliability of all performance  data reported by USAID. Additionally, we obtained documentation and  corroborated the effectiveness of U.S. efforts during meetings with  Afghan officials from relevant governmental organizations, such as  the Independent Administrative Reform and Civil Service  Commission. At the commission’s training institute, we met with  contractors who provide training to Afghan civil servants as part of the  ACSS project.    To assess the results of the efforts of Treasury advisers, we reviewed  a sample of adviser monthly reports as well as advisers’ assessment  for October 2009 to September 2010, the latest available report for a  complete fiscal year. We also interviewed officials from Treasury’s  Office of Technical Assistance to obtain information on the adviser  assessment and reporting process. Moreover, in Afghanistan we met  with MOF officials and contractors in MOF’s budget, treasury, and  internal audit offices, and we obtained their input regarding USAID’s  and Treasury’s efforts and results. We assessed data obtained for our  analysis to be sufficiently reliable to provide an overall assessment of  the extent to which Treasury advisers’ assistance has contributed to  MOF’s PFM capacity.    We did not independently assess the capability of MOD and MOI  finance offices; rather, we relied on CSTC-A’s capability milestone  ratings, which are used to measure the capability level of a specific  area or department. We spoke with cognizant CSTC-A officials about  the reliability of these ratings and also reviewed documentation about  the ratings formulation process. In addition, CSTC-A officials provided  documentation that showed modifications in their rating process to  make it more rigorous. For example, in 2010, CSTC-A introduced  interim ratings, which it believed provided greater detail and a more  gradual transition from requiring significant coalition assistance to  being capable of fully autonomous operations. We believe that the  capability milestone ratings are sufficiently reliable to measure the  extent of progress in finance and budget functions at MOD and MOI.  Additionally, we analyzed finance- and budget-related development  and evaluation plans for MOD and MOI, quarterly assessment  briefings, Ministerial Development Board presentations, internal  control finance reports, and other adviser reports. We compared  quarterly assessments and projected capability milestone ratings over  time to identify the extent to which DOD was meeting its targets. We  also visited MOD’s Finance office in Kabul and interviewed Afghan  officials to obtain their input into U.S. efforts to build their ministry’s  capacity in finance and budget.
section_title: Appendix II: U.S. Allocations of Funds to Afghanistan from Fiscal Year 2002 to Fiscal Year 2011
paragraphs: Since 2002, the United States has allocated more than $72 billion for  security, governance, and development to Afghanistan (see table 6).
section_title: Appendix III: Afghan Government Budget Formulation, Budget Execution, and Audit Process
paragraphs: According to U.S. agency officials and documents, the budget process in  Afghanistan is highly centralized, with the Afghan national government  responsible for developing and executing the country’s budget. According  to the Department of the Treasury (Treasury), none of the 34 Afghan  provinces has the authority to raise revenues or spend public funds.  Budgets are developed and executed through central ministries and their  provincial offices, called provincial directorates. The provincial governor,  appointed by the Afghan President, and the Provincial Council, whose  members are elected, can influence the national budget and indicate  priorities through the Provincial Development Committee, which also  includes representatives from provincial directorates, according to  Treasury officials. The ministries then formulate specific budgets and  convey them to Ministry of Finance (MOF), which develops the overall  national budget. The Afghan parliament, called the National Assembly,  has the authority to approve or reject the national budget in its entirety,  but cannot make changes to it during the final review process, according  to documentation from the U.S. Agency for International Development  (USAID). Figure 7 provides an illustration of Afghan budget formulation,  execution, and audits at the national and provincial levels.
According to Treasury officials, once the National Assembly approves the  budget, ministries prepare and submit allotment requests for the  operating budget. MOF establishes quarterly allotments for the operating  budget for each ministry. For the development budget, each ministry can  submit a budget allotment request only if there is an approved contract (or  legal binding agreement) for the project, according to the Treasury  adviser. MOF must approve the allotment request before the budget  allotment is established. The external budget is not subjected to the  budget allotment process. The central ministries and their provincial  offices implement programs and execute funds. However, MOF and its  provincial offices, called mustofiats, issue payments in response to  requests from ministries. Audit of expenditures to ensure proper use of  public funds is the responsibility of the Control and Audit Office— Afghanistan’s supreme audit institution—and the audit offices of central  ministries, according to Treasury officials.
section_title: Appendix IV: CSTC-A Process to Assess Ministry Capacity
paragraphs: The Combined Security Transition Command–Afghanistan (CSTC-A) has  established a process to regularly assess and rate the capability of  Ministries of Defense (MOD) and Interior (MOI) in various areas including  finance and budget. The CSTC-A assessment process uses the  objectives in the Ministerial Development Plan as criteria to assess  progress in capacity building at the ministries. For example, CSTC-A  advisers at MOD provide quarterly assessments, through Narrative  Assessment Worksheets, in which they assign a rating and provide a  description of progress and challenges associated with each objective.  Additionally, MOD advisers complete an online survey to provide  information about the scope and quality of their interaction with their  Afghan counterparts on a quarterly basis. According to CSTC-A officials,  the MOI assessment and rating is based on interviews with advisers and  Afghan officials, internal quarterly assessments, and other officials’  reporting on a quarterly basis. Figure 8 shows CSTC-A’s assessment  process for MOD.
Senior CSTC-A officials review input from the advisers and agree on an  overall rating. This rating and associated details are presented in a  quarterly assessment that includes highlights of progress towards the  objectives, key objectives to focus on in the next quarter, and an analysis  of the ministry’s strengths, weaknesses, opportunities, and threats. These  assessments, which are available in English and Dari, are presented to  senior coalition forces and Afghan government officials as part of the  Ministerial Development Board process. This review process has three  key goals: (1) to institutionalize the formal review and oversight of the  separate Ministerial Development Plans for various critical ministry  functions (see table 7 for areas of focus for MOD and MOI capacity  building), (2) to assess progress toward achieving fully autonomous  operations including addressing key impediments, and (3) to reinforce  and set the current and following quarter’s objectives and initiatives.  Although CSTC-A does not currently use adviser surveys for MOI, it is in  the process of developing and implementing these, according to CSTC-A  officials.
section_title: Appendix V: U.S. Agency for International Development (USAID) Requirements for Performance Management and Evaluation
section_title: Appendix VI: Fiscal Years 2010 and 2011 Data for USAID PFM Projects
paragraphs: The figures below show fiscal years 2010 and 2011 performance data for  U.S. Agency for International Development (USAID) projects that focus  significantly on public financial management (PFM) capacity building at  the national and provincial level.
section_title: Appendix VII: Comments from the U.S. Agency for International Development
section_title: Appendix VIII: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: Major contributors to this report were Tetsuo Miyabara, Assistant Director;  Mary Koenen; Bruce Kutnick; Mona Sehgal; and Eddie Uyekawa.  Technical assistance was provided by Ashley Alley, Pedro Almoguera,  Emily Gupta, Jeffrey Isaacs, Gergana Danailova-Trainor, Karen Deans,  Denise Fantone, Etana Finkler, Jacqueline Nowicki, Esther Toledo, and  Pierre Toureille.