Combating Terrorism: State Department's Antiterrorism Program Needs Improved Guidance and More Systematic Assessments of Outcomes
Published: 2008-02-29T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of State's (State) Antiterrorism Assistance (ATA) program's objectives are to provide partner nations with counterterrorism training and equipment, improve bilateral ties, and increase respect for human rights. State's Office of the Coordinator for Counterterrorism (S/CT) provides policy guidance and its Bureau of Diplomatic Security, Office of Antiterrorism Assistance, (DS/T/ATA) manages program operations. GAO assessed (1) State's guidance for determining ATA priorities, (2) how State coordinates ATA with other counterterrorism programs, (3) the extent State established ATA program goals and measures, and (4) State's reporting on U.S. international counterterrorism assistance. To address these objectives, GAO reviewed State documents and met with cognizant officials in Washington, D.C., and four ATA program partner nations.
section_title: What GAO Found
paragraphs: S/CT provides minimal guidance to help prioritize ATA program recipients, and S/CT and DS/T/ATA do not systematically align ATA assistance with U.S. assessments of foreign partner counterterrorism needs. S/CT provides policy guidance to DS/T/ATA through quarterly meetings and a tiered list of priority countries, but the list does not provide guidance on country counterterrorism related program goals, objectives, or training priorities. S/CT and DS/T/ATA also did not consistently use country-specific needs assessments and program reviews to plan assistance. S/CT has established mechanisms to coordinate the ATA program with other U.S. international efforts to combat terrorism. S/CT holds interagency meetings with representatives from the Departments of State, Defense, Justice, and Treasury and other agencies as well as ambassador-level regional strategic coordinating meetings. GAO did not find any significant duplication or overlap among the various U.S. international counterterrorism efforts. State has made progress in establishing goals and intended outcomes for the ATA program, but S/CT and DS/T/ATA do not systematically assess the outcomes and, as a result, cannot determine the effectiveness of program assistance. For example, although sustainability is a principal focus, S/CT and DS/T/ATA have not set clear measures of sustainability or integrated sustainability into program planning. State reporting on U.S. counterterrorism assistance abroad has been incomplete and inaccurate. S/CT has not provided a congressionally mandated annual report to Congress on U.S. government-wide assistance related to combating international terrorism since 1996. After 1996, S/CT has only submitted to Congress annual reports on the ATA program. However, these reports contained inaccurate program information, such as the number of students trained and courses offered. Additionally, the reports lacked comprehensive information on the results of program assistance that would be useful to Congress.

section_title: Letter
paragraphs: Combating terrorism has become the nation’s top national security goal  and the highest strategic objective at U.S. embassies worldwide. The U.S.  Department of State (State) is the lead federal agency responsible for  coordinating and supervising U.S. international counterterrorism efforts,  and directly supports the objectives set forth in the U.S. government’s  National Strategy for Combating Terrorism Abroad. This mission involves  collaborating with various other U.S. government agencies working  overseas, such as the Departments of Defense, Homeland Security, and  Justice, as well as with foreign partners and allies facing terrorist threats.
State’s Antiterrorism Assistance (ATA) has assumed an increasingly  important role in U.S. counterterrorism efforts, and is a key mechanism in  State’s efforts to help foreign nations improve their ability to combat  terrorism. Funding for ATA has increased over fourfold in recent years— from $38 million in fiscal year 2001 to $175 million in fiscal year 2007.  ATA’s legislatively mandated  objectives are to (1) enhance the  antiterrorism skills of friendly countries by providing counterterrorism  training and equipment; (2) improve bilateral ties with partner nations by  offering assistance; and (3) increase respect for human rights by sharing  modern, humane and effective antiterrorism techniques with foreign civil  authorities. Within State, the Office of the Coordinator for  Counterterrorism (S/CT) and the Bureau of Diplomatic Security, Office of  Antiterrorism Assistance (DS/T/ATA) oversee and manage ATA. Specific  roles and responsibilities for S/CT and DS/T/ATA are described in a 1991  internal policy guidance memorandum and summarized in State’s Foreign  Affairs Manual. As outlined in these documents, S/CT is responsible for  providing written policy guidance, including suggested regional and  country training priorities, for ATA. DS/T/ATA is responsible for  implementing and managing program operations, including developing  annual, country-specific plans that designate the type of assistance to be  provided, consistent with country program goals and objectives.
In response to your request, we reviewed State’s implementation,  coordination, assessment, and reporting on ATA. Specifically, we assessed  (1) State’s guidance for determining country recipients and types of  program assistance to provide; (2) how State coordinates ATA with other  U.S. government counterterrorism programs; (3) the extent to which State  establishes clear ATA goals, and measures sustainability of program  outcomes; and (4) State’s reporting on U.S. international counterterrorism  assistance.
To address these objectives, we reviewed and analyzed State planning,  funding, and reporting documents concerning ATA. Our work focused  primarily on the time period from fiscal year 2001 to fiscal year 2007,  although we included more recent data where available and appropriate.  We interviewed officials from S/CT and DS/T/ATA, including ATA program  managers responsible for each of the six in-country programs— Afghanistan, Colombia, Indonesia, Kenya, Pakistan, and the Philippines— and visited four of the six in-country programs. In these countries, we  reviewed country-specific program documents, and interviewed ATA in- country program managers, course instructors, and other contractors; U.S  embassy officials responsible for managing counterterrorism assistance  and activities; and partner nation government officials. We also observed  various types of ATA training and equipment that were provided to partner  nation security units. We found funding and program data DS/T/ATA  provided to us, with the exception of certain program data provided in  DS/T/ATA annual reports to Congress, sufficiently reliable for our  purposes. We conducted this performance audit from November 2006  through January 2008 in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives. (See app. I for a  more complete description of our scope and methodology.)
subsections: section_title: Background
paragraphs: Congress authorized State’s ATA program in 1983 through the Foreign  Assistance Act. According to the legislation, and as noted above, the  purpose of ATA is “(1) to enhance the antiterrorism skills of friendly  countries by providing training and equipment to deter and counter  terrorism; (2) to strengthen the bilateral ties of the United States with  friendly governments by offering concrete assistance in this area of great  mutual concern; and (3) to increase respect for human rights by sharing  with foreign civil authorities modern, humane, and effective antiterrorism  techniques.”
subsections: section_title: ATA Program Assistance
paragraphs: ATA offers a wide range of counterterrorism assistance to partner nations,  but most assistance consists of (1) training courses on tactical and  strategic counterterrorism issues and (2) grants of counterterrorism  equipment, such as small arms, bomb detection equipment, vehicles, and  computers. DS/T/ATA also provides specialized consultations to partner  nations on specific counterterrorism issues on an as-needed basis. ATA  curricula and training focus on enhancing critical counterterrorism  capabilities, which cover issues such as crisis management and response,  cyberterrorism, dignitary protection, bomb detection, airport security,  border control, kidnap intervention and hostage negotiation and rescue,  response to incidents involving weapons of mass destruction, countering  terrorist finance, and interdiction of terrorist organizations. According to  DS/T/ATA, all of its courses emphasize law enforcement under the rule of  law and sound human rights practices.
DS/T/ATA provides training primarily through contract employees and  interagency agreements with other U.S. law enforcement agencies.  DS/T/ATA selects, oversees, and evaluates all contracted instructors.  According to DS/T/ATA, most instructors are retired law enforcement or  military personnel who have expertise specific to the ATA curricula.
DS/T/ATA provides training both onsite in the partner nation and at  facilities in the United States, depending on the nature of the course and  the availability of special equipment and necessary facilities. However, in  fiscal year 2007, DS/T/ATA delivered nearly 90 percent of all training  overseas due, in part, to the lack of domestic facilities in the United States  during a transition in contracting for U.S.-based facilities. ATA has  provided increasingly more assistance overseas over the past several  years. An S/CT official noted that the trend reflects a recognition that  training is generally more effectively delivered in the partner nation.
DS/T/ATA has provided most overseas assistance by sending instructors to  the partner nation to conduct a specific course. The partner nation and the  U.S. embassy provide support in designating a facility or training site and  assisting DS/T/ATA headquarters staff with other logistical issues.  DS/T/ATA has established an in-country training presence through  bilateral arrangements with six priority partner nations: Afghanistan,  Colombia, Indonesia, Kenya, Pakistan, and the Philippines. These  countries were the largest recipients of program assistance from fiscal  year 2002 through fiscal year 2007. In general, these programs included  permanent training facilities such as classrooms, computer labs, and  shooting and demolition ranges, which DS/T/ATA used to provide training  on an ongoing basis. Each of the in-country programs has a permanently  posted in-country ATA program manager, along with other ATA staff at the  U.S. post in the host nation—in some cases, in-country staff included  trainers and course instructors. (See fig. 1.)
section_title: ATA Program Funding
paragraphs: ATA is State’s largest counterterrorism program, and receives  appropriations under the Nonproliferation, Anti-Terrorism, Demining, and  Related Programs account. Fiscal year 2002 appropriations for ATA  increased to about $158 million— over six times the level of funding  appropriated in fiscal year 2000. Appropriations for the program have  fluctuated since fiscal year 2002, and increased to over $175 million in  fiscal year 2007, including supplemental appropriations. (See fig. 2.)
From fiscal years 2002 to 2007, program assistance for the top 10  recipients of ATA allocations ranged from about $11 million to about      $78 million. The top 10 recipients represented about 57 percent of ATA  funding allocated for training and training-related activities over the 6-year  period. ATA funding for the other 89 partner nations that received  assistance during this period ranged from $9,000 to about $10.7 million.  (See app. II for additional information on ATA funding for specific partner  nations.)
section_title: ATA Program Management
paragraphs: The Coordinator for Counterterrorism, the head of S/CT, is statutorily  charged with the overall supervision (including policy oversight of  resources) and coordination of the U.S. government’s counterterrorism  activities. The broadly mandated role of the Assistant Secretary for  Diplomatic Security, the head of the Bureau of Diplomatic Security,  includes implementing security programs to protect diplomatic personnel  and advise chiefs of mission on security matters. Specific roles and  responsibilities for S/CT and DS/T/ATA regarding ATA are described in a  1991 internal policy guidance memorandum, the Omnibus Diplomatic  Security Act of 1986, and incorporated into State’s Foreign Affairs  Manual. Table 1 provides a summary of key responsibilities described in  the guidance.
As shown in table 1, S/CT is responsible for leading the initial assessment  of a partner nation’s counterterrorism needs, and DS/T/ATA is responsible  for developing annual, country-specific plans. Under current program  operations, DS/T/ATA conducts an initial assessment of a new participant  nation’s counterterrorism capabilities, and conducts subsequent  assessments—referred to as program reviews— every 2 to 3 years  thereafter. In general, the needs assessments include input from the  embassy teams, but the assessments themselves are conducted by  technical experts contracted by DS/T/ATA. According to DS/T/ATA, the  purpose of the needs assessment and program review process is to  determine the forms of assistance for a partner nation to detect, deter,  deny, and defeat terrorism; and to evaluate program effectiveness.
section_title: State’s Implementation of ATA Lacks Guidance and Use of Country Needs Assessments
paragraphs: ATA lacks guidance beyond a tiered list of priority countries and  assistance is not systematically aligned with counterterrorism needs. S/CT  provides minimal policy guidance to help determine ATA priorities and  ensure that assistance provided supports broader U.S. policy goals. In  addition, S/CT and DS/T/ATA did not systematically use country-specific  needs assessments and program reviews to plan what types of assistance  to provide partner nations in accordance with State policy guidance. The  assessments we reviewed had weaknesses and inconsistencies.
subsections: section_title: S/CT Provides a Tiered Country List, but Little Additional Policy Guidance
paragraphs: In accordance with the 1991 State policy guidance memorandum, S/CT  prepares a tiered list of countries to help prioritize and determine where to  provide ATA assistance. However, S/CT provides little additional guidance  to DS/T/ATA regarding program priorities and how to allocate program  funding. Additionally, other factors besides those reflected in the tiered  list influence which countries receive assistance.
According to State officials, S/CT places countries on the tiered list in one  of four priority categories based on criteria that address several factors,  including country-specific threats and the level and depth of diplomatic  and political engagement in a country. State officials indicated that other  factors also may be considered in determining whether and where a  country is placed on the list, such as the presence of a U.S. military base or  a planned international sporting or cultural event with U.S. participation.  Since 2006, S/CT has reviewed and discussed the tiered list—including  changes, additions, or deletions—with DS/T/ATA during quarterly  meetings. DS/T/ATA officials stated that DS/T/ATA was able to provide  more substantial input and suggestions for the latest version of the tiered  list because S/CT provided a draft list to DS/T/ATA for comment for the  first time prior to the August 2007 meeting. As of August 2007, over 70  countries were on the list, with 12 to 24 countries in each of the four  categories. However, countries were not ranked or prioritized within each  category.
In addition to the quarterly meetings, S/CT told us that they had  established a series of regional roundtable discussions in 2006 between  S/CT regional subject experts and DS/T/ATA counterparts. According to  the S/CT official, the roundtables are intended as a means of identifying  priority countries and their counterterrorism needs for purposes of  developing budget requests.
S/CT provides little guidance to DS/T/ATA beyond the tiered list, although  the 1991 State policy guidance memorandum states that S/CT’s written  policy guidance for the program should include suggested country training  priorities. State’s Office of Inspector General previously reported that  earlier versions of S/CT’s tiered list included additional guidance, such as  the rationale for support, and suggested areas for training. However, S/CT  began providing increasingly abbreviated guidance as its responsibilities  beyond ATA grew after September 11, 2001. While S/CT provides some  additional guidance to DS/T/ATA during quarterly meetings and on other  occasions, DS/T/ATA officials in headquarters and the field stated they  received little or no guidance from S/CT beyond the tiered list. Officials  responsible for the ATA in-country program in Colombia stated they had  minimal interaction with S/CT. As a result, neither S/CT nor DS/T/ATA can  ensure that program assistance provided to specific countries supports  broader U.S. antiterrorism policy goals.
Other factors beyond S/CT’s tiered list of countries, such as unforeseen  events or new governmental initiatives, also influence which countries  receive program assistance. We found that 10 countries on the tiered list  did not receive ATA assistance in fiscal year 2007, while 13 countries not  on the tiered list received approximately $3.2 million. S/CT and DS/T/ATA  officials stated that assistance does not always align with the tiered list  because U.S. foreign policy objectives sometimes cause State, in  consultation with the President’s National Security Council, to provide  assistance to a non-tiered-list country.
section_title: S/CT and DS/T/ATA Do Not Systematically Align Program Assistance with Counterterrorism Needs
paragraphs: According to the 1991 State policy guidance memorandum and DS/T/ATA  standard operations procedures, ATA country-specific needs assessments  and program reviews are intended to guide program management and  planning. However, S/CT and DS/T/ATA did not systematically use the  assessments to determine what types of assistance to provide to partner  nations or develop ATA country-specific plans. In addition, the  assessments we reviewed had several weaknesses and inconsistencies.  Although the 1991 State policy memorandum states that S/CT should lead  the assessment efforts, a senior S/CT official stated that S/CT lacks the  capacity to do so. As a result, DS/T/ATA has led interagency assessment  teams in recent years, but the assessments and recommendations for types  of assistance to be provided may not fully reflect S/CT policy guidance  concerning overall U.S. counterterrorism priorities.
subsections: section_title: DS/T/ATA Does Not Consistently Use Country Needs Assessments
paragraphs: DS/T/ATA officials responsible for five of the top six recipients of ATA  support—Colombia, Kenya, Indonesia, Pakistan, and the Philippines—did  not consistently use ATA country needs assessments and program reviews  in making program decisions or to create annual country assistance plans.  DS/T/ATA officials responsible for the in-country programs in four of these  countries had not seen the latest assessments for their respective  countries. While some officials responsible for three of these five in- country programs stated they had reviewed at least one of the assessments  conducted for their countries since 2000, the officials said that the  assessments were either not useful or that they were used for  informational purposes only.
The Regional Security Officer, Deputy Regional Security Officer, and  DS/T/ATA Program Manager for Kenya had not seen any of the  assessments that had been conducted for the country since 2000. Although  the in-country program manager for Kenya was familiar with the  assessments from her work in a previous position with DS/T/ATA, she  stated that in general, the assessments were not very useful for  determining what type of assistance to provide. She said that the initial  needs assessment for Kenya failed to adequately consider local needs and  capacity.
The Regional Security Officer and Assistant Regional Security Officer for  Indonesia stated they had not seen the latest assessment for the country.  The DS/T/ATA program manager for Indonesia said that he recalled using  one of the assessments as a “frame of reference” in making program and  resource decisions. The in-country program manager also recalled seeing  one of the assessments, but stated that he did not find the assessment  useful given the changing terrorist landscape; therefore, he did not share it  with his staff.
The DS/T/ATA Program Manager for Pakistan stated that decisions on the  types of assistance to provide in Pakistan were based primarily on the  knowledge and experience of in-country staff regarding partner nation  needs, rather than the needs assessments or program reviews. He added  that he did not find the assessments useful, as the issues identified in the  latest (2004) assessment for the country were already outdated.
section_title: Needs Assessments Generally Lacked Prioritized Recommendations and Were Inconsistent
paragraphs: We reviewed 12 of the 21 ATA country-specific needs assessments and  program reviews that, according to ATA annual reports, DS/T/ATA  conducted between 2000 and 2007 for five of the six in-country programs.  The assessments and reviews generally included a range of  recommendations for counterterrorism assistance, but did not prioritize  assistance to be provided or include specific timeframes for  implementation. Consequently, the assessments do not consistently  provide a basis for targeting program assistance to the areas of a partner  nation’s greatest counterterrorism assistance need. Only two of the  assessments—a 2000 needs assessment for Indonesia and a 2003  assessment for Kenya— prioritized the recommendations, although a 2004  assessment for Pakistan and a 2005 assessment for the Philippines listed  one or two recommendations as priority ATA efforts. In addition, the  information included in the assessments was not consistent and varied in  linking recommendations to capabilities. Of the 12 assessments we  reviewed:    Nine included narrative on a range of counterterrorism capabilities, such  as border security and explosives detection, but the number of capabilities  assessed ranged from 5 to 25. The 2001 needs assessment for Colombia  included narrative on the government’s antikidnapping capability and  equipment needs, but did not assess any counterterrorism capabilities. The  2002 assessment for Indonesia provided narrative on ATA assistance  provided, but did not include an assessment of any counterterrorism  capabilities.
Only four of the assessments that assessed more than one capability  linked recommendations provided to the relevant capabilities. Most of the  recommendations in the assessments we reviewed were for ATA  assistance, although some recommended host government actions to  improve counterterrorism capability, or other U.S. government assistance.
Six included capability ratings, but the types of ratings used varied. A 2003  assessment for Colombia rated eight capabilities, rating them 1 through 5  with definitions for each rating level; the 2004 assessment for Colombia  rated 24 capabilities, rating them as poor, low, fair, or good, without any  definitions.
Two used a format that DS/T/ATA began implementing in 2001. The  assessments following the new format generally included consistent types  of information and clearly linked recommendations provided to an  assessment of 25 counterterrorism capabilities. However, they did not  prioritize recommendations or include specific timeframes for  implementing the recommendations.
section_title: Country Assistance Plans Were Not Used or Were Not Linked to Needs Assessments
paragraphs: Although the 1991 State policy memorandum states that DS/T/ATA should  create annual country assistance plans that specify training objectives and  assistance to be provided based upon the needs assessments and program  reviews, we found that S/CT and DS/T/ATA did not systematically use the  assessments to create annual plans for the five in-country programs.  DS/T/ATA officials we interviewed regarding the five in-country programs  stated that in lieu of relying on the assessments or country assistance  plans, program and resource decisions were primarily made by DS/T/ATA  officials in the field based on their knowledge and experience regarding  partner nation needs. Some DS/T/ATA officials said they did not find the  country assistance plans useful. The program manager for Pakistan stated  that he used the country assistance plan as a guide, but found that it did  not respond to changing needs in the country. The ATA program manager  for Kenya said that he had not seen a country assistance plan for that  country.
We requested ATA country assistance plans conducted during fiscal years  2000-2006 for the five in-country programs included in our review, but  S/CT and DS/T/ATA only provided three plans completed for three of the  five countries. Specifically, S/CT and DS/T/ATA provided a 2006 ATA  country assistance plan for Colombia, a 2007 plan for Pakistan, and a plan  covering fiscal years 2006-2008 for the Philippines. DS/T/ATA officials  stated that they were able to locate only draft and informal planning  documents for Indonesia and Kenya, and that S/CT and DS/T/ATA did not  develop plans for any programs prior to 2006.
Of the three ATA country assistance plans DS/T/ATA provided, we found  that the plans did not link planned activities to recommendations provided  in the needs assessments and program reviews. The current plan for the  Philippines included a brief reference to a 2005 needs assessment, but the  plan did not identify which recommendations from the 2005 assessment  were intended to be addressed by current or planned efforts. The plan for  Pakistan did not mention any of the assessments conducted for that  country.
section_title: S/CT Has Established Various Mechanisms to Coordinate Program Assistance
paragraphs: As a part of its responsibility, S/CT has established mechanisms to  coordinate the ATA program with other U.S. government international  counterterrorism training assistance and to help avoid duplication of  efforts. S/CT chairs biweekly interagency working group meetings of the  Counterterrorism Security Group’s Training Assistance Subgroup to  provide a forum for high-level information sharing and discussion among  U.S. agencies implementing international counterterrorism efforts. The  Training Assistance Subgroup includes representatives from the  Departments of State, Defense, Justice, Homeland Security, Treasury, and  other agencies. S/CT also established the Regional Strategic Initiative in  2006 to coordinate regional counterterrorism efforts and strategy. S/CT  described the Regional Strategic Initiative as a series of regionally based,  interagency meetings hosted by U.S. embassies to identify key regional  counterterrorism issues and develop a strategic approach to addressing  them, among other goals. A senior S/CT official stated that meetings have  generated new regional training priorities for ATA. As of November 2007,  Regional Strategic Initiative meetings have been held for the East Africa,  Eastern Mediterranean, Iraq and Neighbors, Latin America, Southeast Asia,  South Asia, Trans-Sahara, and Western Mediterranean regions.
Based on our review of program documents, interviews, and meetings  with officials in the four countries we visited, we did not find any  significant duplication or overlap among U.S. agencies’ country-specific  training programs aimed at combating terrorism. Officials we met with in  each of these countries noted that they participated in various embassy  working group meetings, such as Counterterrorism Working Group and  Law Enforcement Working Group meetings, during which relevant  agencies shared information regarding operations and activities at post.  DS/T/ATA officials also coordinated ATA with other counterterrorism  efforts through daily informal communication among cognizant officials in  the countries we visited.
section_title: State Has Made Progress in Establishing ATA Goals, but S/CT and DS/T/ATA Do Not Assess Sustainability
paragraphs: In response to concerns that ATA lacked elements of adequate strategic  planning and performance measurement, State recently took action to  define goals and measures related to the program’s mandated objectives.  S/CT and DS/T/ATA, however, do not systematically assess  sustainability—that is, the extent to which assistance has enabled partner  nations to achieve and sustain advanced counterterrorism capabilities.  S/CT and DS/T/ATA lack clear measures and processes for assessing  sustainability, and program managers do not consistently include  sustainability in ATA planning.
subsections: section_title: State Has Recently Established ATA Goals and Measures, and Emphasizes Sustaining Partner Nations’ Counterterrorism Capabilities
paragraphs: State did not have measurable performance goals and outcomes related to  the mandated objectives for ATA prior to fiscal year 2003, but has recently  made some progress to address the deficiency. State’s Office of Inspector  General recommended in 2001, 2005, and 2006 reports that S/CT and  DS/T/ATA take steps to establish measurable long-term goals and  evaluations of program performance. Similarly, State responded to issues  raised in a 2003 Office of Management and Budget assessment of ATA by  developing specific goals and measures for each of the program’s  mandated objectives.
Since fiscal year 2006, State planning documents, including department  and bureau-level performance plans, have listed enabling partner nations  to achieve advanced and sustainable counterterrorism capabilities as a key  program outcome. S/CT and DS/T/ATA officials further confirmed that  sustainability is the principal intended outcome and focus of program  assistance. In support of these efforts, DS/T/ATA appointed a Sustainment  Manager in November 2006. The Sustainment Manager’s broadly defined  responsibilities include coordinating with other DS/T/ATA divisions to  develop recommendations and plans to assist partner nations in  developing sustainable counterterrorism capabilities.
section_title: S/CT and DS/T/ATA Do Not Assess Sustainability of Capabilities
paragraphs: Despite progress towards establishing goals and intended outcomes, State  has not developed clear measures and a process for assessing  sustainability and has not integrated the concept into program planning.  The Government Performance and Results Act of 1993 (GPRA) requires  agencies in charge of U.S. government programs and activities to identify  goals and report on the degree to which goals are met. S/CT and DS/T/ATA  officials noted the difficulty in developing direct quantitative measures of  ATA outcomes related to partner nations’ counterterrorism capabilities.  However, GPRA and best practices cited by the Office of Management and  Budget, us, and others provide flexible guidelines for agency and program  managers to develop adequate measures of program effectiveness. Our  past work also has stressed the importance of establishing program goals,  objectives, priorities, milestones, and measures to use in monitoring  performance and assessing outcomes as critical elements of program  management and effective resource allocation.
We found that the measure for ATA’s principal intended program outcome  of sustainability is not clear. In its fiscal year 2007 Joint Performance  Summary, State reported results and future year targets for the number of  countries that had achieved an advanced, sustainable level of  counterterrorism capability. According to the document, partner nations  that achieve a sustainable level of counterterrorism would graduate from  the program and no longer receive program assistance. However, program  officials in S/CT and DS/T/ATA directly responsible for overseeing ATA  were not aware that the Joint Performance Summary listed numerical  targets and past results for the number of partner nations that had  achieved sustainability, and could not provide an explanation of how State  assessed the results. DS/T/ATA’s Sustainment Manager also could not  explain how State established and assessed the numerical targets in the  reports. The Sustainment Manager further noted that, to his knowledge,  S/CT and DS/T/ATA had not yet developed systematic measures of  sustainability.
DS/T/ATA’s current mechanism for evaluating partner nation capabilities  does not include guidance or specific measures to assess sustainability.  According to program guidance and DS/T/ATA officials, needs  assessments and program reviews are intended to establish a baseline of a  partner nation’s counterterrorism capabilities and quantify progress  through subsequent reviews. DS/T/ATA officials also asserted that the  process is intended to measure the results of program assistance.  However, the process does not explicitly address sustainability, and  provides no specific information or instruction regarding how reviewers  are to assess sustainability. Moreover, the process focuses on assessing a  partner nation’s overall counterterrorism capabilities, but does not  specifically measure the results of program assistance. The assessment  and review process also does not provide S/CT and DS/T/ATA a means for  determining whether a partner nation’s capabilities changed because of  program assistance, the country’s own efforts, or through assistance  provided by other U.S. agencies or third countries. The head of  DS/T/ATA’s Assessment, Review, and Evaluations Unit told us that he had  not received guidance to assess progress toward sustainability, and had  only limited interaction with the Sustainment Manager on integrating  sustainability into the assessment and review process.
DS/T/ATA has not systematically integrated sustainability into country- specific assistance plans, and we found a lack of consensus among  program officials about how to address the issue. In-country program  managers, embassy officials, instructors, and partner nation officials we  interviewed held disparate views on how to define sustainability across all  ATA participant countries, and many were not aware that sustainability  was the intended outcome for the program. Several program officials  stated that graduating a country and withdrawing or significantly reducing  program assistance could result in a rapid decline in the partner nation’s  counterterrorism capabilities, and could undermine achieving other  program objectives, such as improving bilateral relations. Further,  although State has listed sustainability in State-level planning documents  since 2006, S/CT and DS/T/ATA have not issued guidance on incorporating  sustainability into country-specific planning, and none of the country  assistance plans we reviewed consistently addressed the outcome. As a  result, the plans did not include measurable annual objectives or planned  activities targeted at enabling the partner nation to achieve sustainability.  For example, Colombia’s assistance plan listed transferring responsibility  for the antikidnapping training to the Colombian government and  described planned activities to achieve that goal. However, the plan did  not include measurable objectives to determine whether activities achieve  intended results. Although the plan for the Philippines stated that the  country program goal for fiscal year 2007 was to “maximize sustainment,”  it did not include measures of sustainability or describe how planned  activities would contribute to the intended outcome.
section_title: State Reporting on U.S. Counterterrorism Assistance Abroad Has Been Incomplete and Inaccurate
paragraphs: Since 1996, State has not complied with a congressional mandate to  report to Congress on U.S. international counterterrorism assistance.  Additionally, State’s annual reports on ATA have contained inaccurate  data regarding basic program information, do not provide systematic  assessments of program results, and lack other information necessary to  evaluate program effectiveness.
subsections: section_title: S/CT Has Not Prepared a Mandated Report
paragraphs: The Foreign Assistance Act requires the Secretary of State to report  annually on the amount and nature of all assistance provided by the U.S.  government related to international terrorism. Since 1996, State has  submitted ATA annual reports rather than the report required by the  statute.
The legislation that authorized ATA in 1983 required annual presentations  to Congress of aggregate information on all countries that received  program assistance. In 1985, Congress added a new, broader reporting  obligation, requiring the Secretary of State to report on all assistance  related to international terrorism provided by the U.S. government during  the preceding fiscal year. Although the original ATA-specific 1983  reporting provision was repealed in 1996, the requirement for the broader  report remains.
S/CT is responsible for preparing the reports on U.S. international  counterterrorism assistance. The S/CT official directly responsible for ATA  told us that he only recently became aware of the reporting requirement  and noted confusion within State over what the statute required. He also  asserted that the ATA annual report, which is prepared by DS/T/ATA, and  State’s annual “Patterns of Global Terrorism” report were sufficiently  responsive to congressional needs. He further noted that, in his view, it  would be extremely difficult for State to compile and report on all U.S.  government terrorism assistance activities, especially given the significant  growth of agencies’ programs since 2001. Officials in State’s Bureau of  Legislative Affairs indicated that, to their knowledge, they had never  received an inquiry from congressional staff about the missing reports.  DS/T/ATA officials told us DS/T/ATA has continued to produce the ATA  annual report to Congress even after the reporting requirement was  removed in 1996. However, State has not issued DS/T/ATA’s annual report  to Congress on ATA for fiscal year 2006 that was planned for release in  2007. DS/T/ATA officials noted that they did, however, complete and  circulate the final report within State.
section_title: ATA Annual Reports Contain Inaccuracies
paragraphs: Recent ATA annual reports have contained inaccurate data relating to  basic program information on numbers of students trained and courses  offered. For example, DS/T/ATA reported inaccurate data on program  operations in ATA’s two top-funded partner nations—Afghanistan and  Pakistan.
Afghanistan. ATA annual reports for fiscal years 2002 to 2005 contain  narrative passages describing various ATA training and training-related  assistance activities for the Afghan in-country ATA program. According to  these reports, 15 students were trained as part of a single training event  over the 4-year period. DS/T/ATA subsequently provided us data for fiscal  year 2005 training activity in Afghanistan, which corrected the  participation total in that year from 15 participants in 1 training event to  1,516 participants in 12 training events. DS/T/ATA officials acknowledged  the report disparities.
Pakistan. According to the fiscal year 2005 ATA annual report, ATA  delivered 17 courses to 335 participants in Pakistan that year. Supporting  tables in the same report listed 13 courses provided to 283 participants.  Further, a summary report provided to us from the DS/T/ATA internal  database produced a third set of numbers describing 13 courses provided  to 250 course participants during fiscal year 2005. DS/T/ATA officials  acknowledged this inconsistency, but they were unable to identify which  set of figures was correct.
DS/T/ATA officials noted that similar inaccuracies could be presumed for  prior years and for other partner nations. Significantly, the officials  indicated that inaccuracies and omissions in reports of the training  participants and events were due to a lack of internal policies and  procedures for recording and reporting program data. In the absence of  documented policies and procedures, staff developed various individual  processes for collecting the information that resulted in flawed data  reporting. Additionally, DS/T/ATA officials told us that its inadequate  information management system and a lack of consistent data collection  procedures also contributed to inaccurate reporting.
section_title: ATA Annual Reports Lack Performance and Other Useful Program Information
paragraphs: DS/T/ATA’s annual reports to Congress on ATA from fiscal year 1997 to  2005 did not contain systematic assessments of program results. Further,  the reports did not consistently include information on key aspects of the  program, such as program activities, spending, and management initiatives  that would be helpful to Congress and State in evaluating ATA.
GPRA, Office of Management and Budget guidance, and our previous  work provide a basis and rationale for the types of information that are  useful in assessing program performance. According to this guidance, key  elements of program reporting include clearly defined objectives and  goals, comparisons of actual and projected performance that include at  least 4 years of annual data, explanations and plans for addressing unmet  goals, and reliable information on the program’s activities and financial  activity.
We reviewed ATA annual reports for fiscal years 1997 through 2005, and  found that the reports varied widely in terms of content, scope, and  format. Moreover, the annual reports did not contain systematic  assessments of program performance or consistent information on  program activity, such as number and type of courses delivered, types of  equipment provided, and budget activity associated with program  operations. In general, the reports contained varying levels of detail on  program activity, and provided only anecdotal examples of program  successes, from a variety of sources, including U.S. embassy officials, ATA  instructors, and partner nation officials. DS/T/ATA program officials  charged with compiling the annual reports for the past 3 fiscal years noted  that DS/T/ATA does not have guidance on the scope, content, or format for  the reports.
section_title: Conclusions
paragraphs: Although ATA plays a central role in State’s broader effort to fight  international terrorism, deficiencies in how the program is guided,  managed, implemented, and assessed could limit the program’s  effectiveness. Specifically, minimal guidance from S/CT makes it difficult  to determine the extent to which program assistance directly supports  broader U.S. counterterrorism policy goals. Additionally, deficiencies with  DS/T/ATA’s needs assessments and program reviews may limit their utility  as a tool for planning assistance and prioritizing among several partner  nations’ counterterrorism needs. As a result, the assessments and reviews  are not systematically linked to resource allocation decisions, which may  limit the program’s ability to improve partner nation’s counterterrorism  capabilities.
Although State has made some progress in attempting to evaluate and  quantitatively measure program performance, ATA still lacks a clearly  defined, systematic assessment and reporting of outcomes, which makes it  difficult to determine the overall effectiveness of the program. This  deficiency, along with State’s noncompliance with mandated reporting  requirements, has resulted in Congress having limited and incomplete  information on U.S. international counterterrorism assistance and ATA  efforts. Such information is necessary to determine the most effective  types of assistance the U.S. government can provide to partner nations in  support of the U.S. national security goal of countering terrorism abroad.
section_title: Matter for Congressional Consideration
paragraphs: Congress should reconsider the requirement that the Secretary of State  provide an annual report on the nature and amount of U.S. government  counterterrorism assistance provided abroad given the broad changes in  the scope and nature of U.S. counterterrorism assistance abroad, in  conjunction with the fact that the report has not been submitted since  1996.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretary of State take the following four actions:  1.  Revisit and revise internal guidance (the 1991 State policy  memorandum and Foreign Affairs Manual, in particular) to ensure that  the roles and responsibilities for S/CT and DS/T/ATA are still relevant  and better enable State to determine which countries should receive  assistance and what type, and allocate limited ATA resources.  2.  Ensure that needs assessments and program reviews are both useful  and linked to ATA resource decisions and development of country- specific assistance plans.  3.  Establish clearer measures of sustainability, and refocus the process  for assessing the sustainability of partner nations’ counterterrorism  capabilities. The revised evaluation process should include not only an  overall assessment of partner nation counterterrorism capabilities, but  also provide guidance for assessing the specific outcomes of ATA.  4.  Comply with the congressional mandate to report to Congress on U.S.  international counterterrorism assistance.
section_title: Agency Comments and Our Evaluation
paragraphs: State provided us oral and written comments (see app. III) on a draft of  this report. State also provided technical comments which we have  incorporated throughout the report, as appropriate.
Overall, State agreed with our principal findings and recommendations to  improve its ATA program guidance, the needs assessment and program  review process, and its assessments of ATA program outcomes. State  noted that the report highlights the difficulties in assessing the benefits of  developing and improving long-term antiterrorism and law enforcement  relationships with foreign governments. State also outlined a number of  ongoing and planned initiatives to address our recommendations. Some of  these initiatives were underway during the course of our review and we  refer to them in the report. We will follow up with State to ensure that  these initiatives have been completed, as planned.
However, although State supported the matter we suggest for  congressional consideration, it did not specifically address our  recommendation that it comply with the congressional mandate to report  on U.S. counterterrorism assistance.
As agreed with your office, unless you publicly announce the contents of  the report earlier, we plan no further distribution until 30 days after the  report date. At that time, we will send copies of the report to interested  congressional committees and to the Secretary of State. We will also make  copies available to others upon request. In addition, the report will be  available at no charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  Charles Michael Johnson, Jr. (202) 512-7331, e-mail johnsoncm@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. Other GAO contact  and staff acknowledgments are listed in appendix IV.
section_title: Appendix I: Scope and Methodology
paragraphs: To assess State’s guidance for determining country recipients, aligning  program assistance with partner nation needs, and coordinating  Antiterrorism Assistance (ATA) with other U.S. government  counterterrorism programs, we  Interviewed cognizant officials from the Office of Coordinator for  Counterterrorism (S/CT) and the Bureau of Diplomatic Security, Office of  Antiterrorism Assistance (DS/T/ATA) in Washington, D.C., including senior  officials responsible for overseeing and managing ATA and ATA program  managers responsible for each of the six in-country programs:  Afghanistan, Colombia, Indonesia, Kenya, Pakistan, and the Philippines.
Reviewed and analyzed State planning, funding, and reporting documents  concerning ATA, including relevant reports from State’s Office of  Inspector General on the management and implementation of ATA; S/CT’s  fiscal year 2007 tiered lists of priority countries for ATA assistance and  S/CT criteria for establishing the tier list; DS/T/ATA budget information for  fiscal years 2000 to 2008; a 1991 State policy memorandum delineating  S/CT’s and DS/T/ATA’s roles and responsibilities for ATA; relevant  sections of State’s Foreign Affairs Manual summarizing roles and  responsibilities for ATA; DS/T/ATA internal policy and procedure  documents, including DS/T/ATA’s Assessment, Review and Evaluations  Unit’s most current (2004) standard operations procedures; State  documents and U.S. embassy cables regarding the Regional Strategic  Initiative; and DS/T/ATA’s Annual Reports to Congress on the ATA for  fiscal years 1997 to 2005.
Reviewed and analyzed available country-specific program documents for  five of the in-country programs—Colombia, Indonesia, Kenya, Pakistan,  and the Philippines—including country-specific needs assessments  conducted for each of these partner nations; country assistance plans; data  on the number of ATA courses provided and personnel trained in these  countries; and memoranda of intent between the U.S. government and  host country governments regarding ATA in these countries for fiscal  years 2000 to 2007. These five countries were among the largest six  recipients of program assistance for fiscal years 2002 to 2007 and each  country received a range of ATA training and other assistance during the  period we reviewed. DS/T/ATA was unable to provide four of the needs  assessments that, according to annual reports, were conducted for two of  these countries in that time, and was only able to provide three ATA  country assistance plans that were completed for three of the five  countries for fiscal years 2006 to 2008.
Conducted fieldwork between July and September 2007 in four countries  where ATA provides a range of assistance through an in-country presence:  Colombia, Indonesia, Kenya, and the Philippines. These four programs  represented about 55 percent of ATA allocations for training and training- related activities in fiscal year 2006, and about 43 percent of funding in  fiscal year 2007. As this was not a generalizeable sample, our observations  in these four countries may not be representative of all programs. In these  countries, we interviewed ATA in-country program managers, course  instructors, and other contractors; U.S. embassy officials responsible for  managing counterterrorism assistance and activities; and partner nation  government officials. We also observed various types of ATA training and  examined equipment that was provided to partner nation security units.  Additionally, to assess the extent to which State establishes clear ATA  goals and measures sustainability of program outcomes, and State’s  reporting on U.S. international counterterrorism assistance, we  Interviewed cognizant officials from S/CT and DS/T/ATA in Washington,  D.C., including senior officials responsible for overseeing and managing  ATA and ATA program managers responsible for each of the six in-country  programs: Afghanistan, Colombia, Indonesia, Kenya, Pakistan, and the  Philippines. Additionally, we interviewed cognizant officials in DS/T/ATA’s  Assessment, Review, and Evaluations Unit, Training Curriculum Division,  Training Delivery Division, and Training Management Division, including  the Sustainment Manager.
Reviewed and analyzed State strategic planning and performance  reporting documents related to ATA for fiscal years 2001 to 2007, including  State budget justifications, State Performance Plans; State Performance  Summaries; Bureau Performance Plans; Mission Performance Plans for  Afghanistan, Colombia, Indonesia, Kenya, Pakistan, and the Philippines;  and DS/T/ATA annual reports to Congress on ATA as noted above. We also  reviewed Office of Management and Budget’s fiscal year 2003 review of  ATA and relevant State Office of Inspector General reports relating to  performance measurement issues for ATA. Additionally, we reviewed all  available S/CT and DS/T/ATA guidance related to assessing program  performance, including internal standard operating procedure documents  and course evaluation instruments, as well as ATA authorizing legislation  and related revisions. To further assess State’s reporting on international  counterterrorism assistance, we reviewed DS/T/ATA’s annual reports on  ATA for consistency and accuracy. As noted earlier, we found some errors  with these reports, and have concerns about the data on training and  nontraining activities. Although we describe the errors, we did not use  these data in our analyses. To assess the reliability of the data on funding  to recipient countries, we interviewed ATA officials and performed some  cross-checks with other sources. We determined the data on funding were  sufficiently reliable for the purposes of this report.
section_title: Appendix II: Top Recipients of ATA Funding, Fiscal Years 2002 to 2007
paragraphs: As shown in table 2, program assistance for the top 10 recipients of ATA  funding from fiscal years 2002 to 2007 ranged from about $11 million to  about $78 million. The top 10 funding recipients received about 57 percent  of ATA funding allocated for training and training related activities over  the 6-year-period. ATA has established an in-country presence in each of  the top six partner nations, including in-country program staff and  permanent training facilities such as classrooms, computer labs, and  shooting and demolition ranges. Afghanistan received the most funding  over the six-year-period. According to DS/T/ATA officials, the scope of the  in-country program in Afghanistan is more narrowly defined than other  ATA programs; it focuses principally on training and monitoring a  Presidential Protective Service.
section_title: Appendix III: Comments from the Department of State
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: In addition to the individual named above, Albert H. Huntington, III, and  David C. Maurer, Assistant Directors; Karen A. Deans; Matthew E. Helm;  Elisabeth R. Helmer; Grace Lui; and Emily T. Rachman made key  contributions to this report.