Combating Terrorism: State Department's Antiterrorism Program Needs Improved Guidance and More Systematic Assessments of Outcomes
Released: 2008-03-31T00:00:00

Letter

Background

Congress authorized State’s ATA program in 1983 through the Foreign Assistance Act. According to the legislation, and as noted above, the purpose of ATA is “(1) to enhance the antiterrorism skills of friendly countries by providing training and equipment to deter and counter terrorism; (2) to strengthen the bilateral ties of the United States with friendly governments by offering concrete assistance in this area of great mutual concern; and (3) to increase respect for human rights by sharing with foreign civil authorities modern, humane, and effective antiterrorism techniques.”

ATA Program Assistance

ATA offers a wide range of counterterrorism assistance to partner nations, but most assistance consists of (1) training courses on tactical and strategic counterterrorism issues and (2) grants of counterterrorism equipment, such as small arms, bomb detection equipment, vehicles, and computers. DS/T/ATA also provides specialized consultations to partner nations on specific counterterrorism issues on an as-needed basis. ATA curricula and training focus on enhancing critical counterterrorism capabilities, which cover issues such as crisis management and response, cyberterrorism, dignitary protection, bomb detection, airport security, border control, kidnap intervention and hostage negotiation and rescue, response to incidents involving weapons of mass destruction, countering terrorist finance, and interdiction of terrorist organizations. According to DS/T/ATA, all of its courses emphasize law enforcement under the rule of law and sound human rights practices.

DS/T/ATA provides training primarily through contract employees and interagency agreements with other U.S. law enforcement agencies. DS/T/ATA selects, oversees, and evaluates all contracted instructors. According to DS/T/ATA, most instructors are retired law enforcement or military personnel who have expertise specific to the ATA curricula.

DS/T/ATA provides training both onsite in the partner nation and at facilities in the United States, depending on the nature of the course and the availability of special equipment and necessary facilities. However, in fiscal year 2007, DS/T/ATA delivered nearly 90 percent of all training overseas due, in part, to the lack of domestic facilities in the United States during a transition in contracting for U.S.-based facilities. ATA has provided increasingly more assistance overseas over the past several years. An S/CT official noted that the trend reflects a recognition that training is generally more effectively delivered in the partner nation.

DS/T/ATA has provided most overseas assistance by sending instructors to the partner nation to conduct a specific course. The partner nation and the U.S. embassy provide support in designating a facility or training site and assisting DS/T/ATA headquarters staff with other logistical issues. DS/T/ATA has established an in-country training presence through bilateral arrangements with six priority partner nations: Afghanistan, Colombia, Indonesia, Kenya, Pakistan, and the Philippines. These countries were the largest recipients of program assistance from fiscal year 2002 through fiscal year 2007. In general, these programs included permanent training facilities such as classrooms, computer labs, and shooting and demolition ranges, which DS/T/ATA used to provide training on an ongoing basis. Each of the in-country programs has a permanently posted in-country ATA program manager, along with other ATA staff at the U.S. post in the host nation—in some cases, in-country staff included trainers and course instructors. (See fig. 1.)

ATA Program Funding

ATA is State’s largest counterterrorism program, and receives appropriations under the Nonproliferation, Anti-Terrorism, Demining, and Related Programs account. Fiscal year 2002 appropriations for ATA increased to about $158 million— over six times the level of funding appropriated in fiscal year 2000. Appropriations for the program have fluctuated since fiscal year 2002, and increased to over $175 million in fiscal year 2007, including supplemental appropriations. (See fig. 2.)

From fiscal years 2002 to 2007, program assistance for the top 10 recipients of ATA allocations ranged from about $11 million to about $78 million. The top 10 recipients represented about 57 percent of ATA funding allocated for training and training-related activities over the 6-year period. ATA funding for the other 89 partner nations that received assistance during this period ranged from $9,000 to about $10.7 million. (See app. II for additional information on ATA funding for specific partner nations.)

ATA Program Management

The Coordinator for Counterterrorism, the head of S/CT, is statutorily charged with the overall supervision (including policy oversight of resources) and coordination of the U.S. government’s counterterrorism activities. The broadly mandated role of the Assistant Secretary for Diplomatic Security, the head of the Bureau of Diplomatic Security, includes implementing security programs to protect diplomatic personnel and advise chiefs of mission on security matters. Specific roles and responsibilities for S/CT and DS/T/ATA regarding ATA are described in a 1991 internal policy guidance memorandum, the Omnibus Diplomatic Security Act of 1986, and incorporated into State’s Foreign Affairs Manual. Table 1 provides a summary of key responsibilities described in the guidance.

As shown in table 1, S/CT is responsible for leading the initial assessment of a partner nation’s counterterrorism needs, and DS/T/ATA is responsible for developing annual, country-specific plans. Under current program operations, DS/T/ATA conducts an initial assessment of a new participant nation’s counterterrorism capabilities, and conducts subsequent assessments—referred to as program reviews— every 2 to 3 years thereafter. In general, the needs assessments include input from the embassy teams, but the assessments themselves are conducted by technical experts contracted by DS/T/ATA. According to DS/T/ATA, the purpose of the needs assessment and program review process is to determine the forms of assistance for a partner nation to detect, deter, deny, and defeat terrorism; and to evaluate program effectiveness.

State’s Implementation of ATA Lacks Guidance and Use of Country Needs Assessments

ATA lacks guidance beyond a tiered list of priority countries and assistance is not systematically aligned with counterterrorism needs. S/CT provides minimal policy guidance to help determine ATA priorities and ensure that assistance provided supports broader U.S. policy goals. In addition, S/CT and DS/T/ATA did not systematically use country-specific needs assessments and program reviews to plan what types of assistance to provide partner nations in accordance with State policy guidance. The assessments we reviewed had weaknesses and inconsistencies.

S/CT Provides a Tiered Country List, but Little Additional Policy Guidance

In accordance with the 1991 State policy guidance memorandum, S/CT prepares a tiered list of countries to help prioritize and determine where to provide ATA assistance. However, S/CT provides little additional guidance to DS/T/ATA regarding program priorities and how to allocate program funding. Additionally, other factors besides those reflected in the tiered list influence which countries receive assistance.

According to State officials, S/CT places countries on the tiered list in one of four priority categories based on criteria that address several factors, including country-specific threats and the level and depth of diplomatic and political engagement in a country. State officials indicated that other factors also may be considered in determining whether and where a country is placed on the list, such as the presence of a U.S. military base or a planned international sporting or cultural event with U.S. participation. Since 2006, S/CT has reviewed and discussed the tiered list—including changes, additions, or deletions—with DS/T/ATA during quarterly meetings. DS/T/ATA officials stated that DS/T/ATA was able to provide more substantial input and suggestions for the latest version of the tiered list because S/CT provided a draft list to DS/T/ATA for comment for the first time prior to the August 2007 meeting. As of August 2007, over 70 countries were on the list, with 12 to 24 countries in each of the four categories. However, countries were not ranked or prioritized within each category.

In addition to the quarterly meetings, S/CT told us that they had established a series of regional roundtable discussions in 2006 between S/CT regional subject experts and DS/T/ATA counterparts. According to the S/CT official, the roundtables are intended as a means of identifying priority countries and their counterterrorism needs for purposes of developing budget requests.

S/CT provides little guidance to DS/T/ATA beyond the tiered list, although the 1991 State policy guidance memorandum states that S/CT’s written policy guidance for the program should include suggested country training priorities. State’s Office of Inspector General previously reported that earlier versions of S/CT’s tiered list included additional guidance, such as the rationale for support, and suggested areas for training. However, S/CT began providing increasingly abbreviated guidance as its responsibilities beyond ATA grew after September 11, 2001. While S/CT provides some additional guidance to DS/T/ATA during quarterly meetings and on other occasions, DS/T/ATA officials in headquarters and the field stated they received little or no guidance from S/CT beyond the tiered list. Officials responsible for the ATA in-country program in Colombia stated they had minimal interaction with S/CT. As a result, neither S/CT nor DS/T/ATA can ensure that program assistance provided to specific countries supports broader U.S. antiterrorism policy goals.

Other factors beyond S/CT’s tiered list of countries, such as unforeseen events or new governmental initiatives, also influence which countries receive program assistance. We found that 10 countries on the tiered list did not receive ATA assistance in fiscal year 2007, while 13 countries not on the tiered list received approximately $3.2 million. S/CT and DS/T/ATA officials stated that assistance does not always align with the tiered list because U.S. foreign policy objectives sometimes cause State, in consultation with the President’s National Security Council, to provide assistance to a non-tiered-list country.

S/CT and DS/T/ATA Do Not Systematically Align Program Assistance with Counterterrorism Needs

According to the 1991 State policy guidance memorandum and DS/T/ATA standard operations procedures, ATA country-specific needs assessments and program reviews are intended to guide program management and planning. However, S/CT and DS/T/ATA did not systematically use the assessments to determine what types of assistance to provide to partner nations or develop ATA country-specific plans. In addition, the assessments we reviewed had several weaknesses and inconsistencies. Although the 1991 State policy memorandum states that S/CT should lead the assessment efforts, a senior S/CT official stated that S/CT lacks the capacity to do so. As a result, DS/T/ATA has led interagency assessment teams in recent years, but the assessments and recommendations for types of assistance to be provided may not fully reflect S/CT policy guidance concerning overall U.S. counterterrorism priorities.

DS/T/ATA Does Not Consistently Use Country Needs Assessments

DS/T/ATA officials responsible for five of the top six recipients of ATA support—Colombia, Kenya, Indonesia, Pakistan, and the Philippines—did not consistently use ATA country needs assessments and program reviews in making program decisions or to create annual country assistance plans. DS/T/ATA officials responsible for the in-country programs in four of these countries had not seen the latest assessments for their respective countries. While some officials responsible for three of these five in- country programs stated they had reviewed at least one of the assessments conducted for their countries since 2000, the officials said that the assessments were either not useful or that they were used for informational purposes only.

The Regional Security Officer, Deputy Regional Security Officer, and DS/T/ATA Program Manager for Kenya had not seen any of the assessments that had been conducted for the country since 2000. Although the in-country program manager for Kenya was familiar with the assessments from her work in a previous position with DS/T/ATA, she stated that in general, the assessments were not very useful for determining what type of assistance to provide. She said that the initial needs assessment for Kenya failed to adequately consider local needs and capacity.

The Regional Security Officer and Assistant Regional Security Officer for Indonesia stated they had not seen the latest assessment for the country. The DS/T/ATA program manager for Indonesia said that he recalled using one of the assessments as a “frame of reference” in making program and resource decisions. The in-country program manager also recalled seeing one of the assessments, but stated that he did not find the assessment useful given the changing terrorist landscape; therefore, he did not share it with his staff.

The DS/T/ATA Program Manager for Pakistan stated that decisions on the types of assistance to provide in Pakistan were based primarily on the knowledge and experience of in-country staff regarding partner nation needs, rather than the needs assessments or program reviews. He added that he did not find the assessments useful, as the issues identified in the latest (2004) assessment for the country were already outdated.

Needs Assessments Generally Lacked Prioritized Recommendations and Were Inconsistent

We reviewed 12 of the 21 ATA country-specific needs assessments and program reviews that, according to ATA annual reports, DS/T/ATA conducted between 2000 and 2007 for five of the six in-country programs. The assessments and reviews generally included a range of recommendations for counterterrorism assistance, but did not prioritize assistance to be provided or include specific timeframes for implementation. Consequently, the assessments do not consistently provide a basis for targeting program assistance to the areas of a partner nation’s greatest counterterrorism assistance need. Only two of the assessments—a 2000 needs assessment for Indonesia and a 2003 assessment for Kenya— prioritized the recommendations, although a 2004 assessment for Pakistan and a 2005 assessment for the Philippines listed one or two recommendations as priority ATA efforts. In addition, the information included in the assessments was not consistent and varied in linking recommendations to capabilities. Of the 12 assessments we reviewed: Nine included narrative on a range of counterterrorism capabilities, such as border security and explosives detection, but the number of capabilities assessed ranged from 5 to 25. The 2001 needs assessment for Colombia included narrative on the government’s antikidnapping capability and equipment needs, but did not assess any counterterrorism capabilities. The 2002 assessment for Indonesia provided narrative on ATA assistance provided, but did not include an assessment of any counterterrorism capabilities.

Only four of the assessments that assessed more than one capability linked recommendations provided to the relevant capabilities. Most of the recommendations in the assessments we reviewed were for ATA assistance, although some recommended host government actions to improve counterterrorism capability, or other U.S. government assistance.

Six included capability ratings, but the types of ratings used varied. A 2003 assessment for Colombia rated eight capabilities, rating them 1 through 5 with definitions for each rating level; the 2004 assessment for Colombia rated 24 capabilities, rating them as poor, low, fair, or good, without any definitions.

Two used a format that DS/T/ATA began implementing in 2001. The assessments following the new format generally included consistent types of information and clearly linked recommendations provided to an assessment of 25 counterterrorism capabilities. However, they did not prioritize recommendations or include specific timeframes for implementing the recommendations.

Country Assistance Plans Were Not Used or Were Not Linked to Needs Assessments

Although the 1991 State policy memorandum states that DS/T/ATA should create annual country assistance plans that specify training objectives and assistance to be provided based upon the needs assessments and program reviews, we found that S/CT and DS/T/ATA did not systematically use the assessments to create annual plans for the five in-country programs. DS/T/ATA officials we interviewed regarding the five in-country programs stated that in lieu of relying on the assessments or country assistance plans, program and resource decisions were primarily made by DS/T/ATA officials in the field based on their knowledge and experience regarding partner nation needs. Some DS/T/ATA officials said they did not find the country assistance plans useful. The program manager for Pakistan stated that he used the country assistance plan as a guide, but found that it did not respond to changing needs in the country. The ATA program manager for Kenya said that he had not seen a country assistance plan for that country.

We requested ATA country assistance plans conducted during fiscal years 2000-2006 for the five in-country programs included in our review, but S/CT and DS/T/ATA only provided three plans completed for three of the five countries. Specifically, S/CT and DS/T/ATA provided a 2006 ATA country assistance plan for Colombia, a 2007 plan for Pakistan, and a plan covering fiscal years 2006-2008 for the Philippines. DS/T/ATA officials stated that they were able to locate only draft and informal planning documents for Indonesia and Kenya, and that S/CT and DS/T/ATA did not develop plans for any programs prior to 2006.

Of the three ATA country assistance plans DS/T/ATA provided, we found that the plans did not link planned activities to recommendations provided in the needs assessments and program reviews. The current plan for the Philippines included a brief reference to a 2005 needs assessment, but the plan did not identify which recommendations from the 2005 assessment were intended to be addressed by current or planned efforts. The plan for Pakistan did not mention any of the assessments conducted for that country.

S/CT Has Established Various Mechanisms to Coordinate Program Assistance

As a part of its responsibility, S/CT has established mechanisms to coordinate the ATA program with other U.S. government international counterterrorism training assistance and to help avoid duplication of efforts. S/CT chairs biweekly interagency working group meetings of the Counterterrorism Security Group’s Training Assistance Subgroup to provide a forum for high-level information sharing and discussion among U.S. agencies implementing international counterterrorism efforts. The Training Assistance Subgroup includes representatives from the Departments of State, Defense, Justice, Homeland Security, Treasury, and other agencies. S/CT also established the Regional Strategic Initiative in 2006 to coordinate regional counterterrorism efforts and strategy. S/CT described the Regional Strategic Initiative as a series of regionally based, interagency meetings hosted by U.S. embassies to identify key regional counterterrorism issues and develop a strategic approach to addressing them, among other goals. A senior S/CT official stated that meetings have generated new regional training priorities for ATA. As of November 2007, Regional Strategic Initiative meetings have been held for the East Africa, Eastern Mediterranean, Iraq and Neighbors, Latin America, Southeast Asia, South Asia, Trans-Sahara, and Western Mediterranean regions.

Based on our review of program documents, interviews, and meetings with officials in the four countries we visited, we did not find any significant duplication or overlap among U.S. agencies’ country-specific training programs aimed at combating terrorism. Officials we met with in each of these countries noted that they participated in various embassy working group meetings, such as Counterterrorism Working Group and Law Enforcement Working Group meetings, during which relevant agencies shared information regarding operations and activities at post. DS/T/ATA officials also coordinated ATA with other counterterrorism efforts through daily informal communication among cognizant officials in the countries we visited.

State Has Made Progress in Establishing ATA Goals, but S/CT and DS/T/ATA Do Not Assess Sustainability

In response to concerns that ATA lacked elements of adequate strategic planning and performance measurement, State recently took action to define goals and measures related to the program’s mandated objectives. S/CT and DS/T/ATA, however, do not systematically assess sustainability—that is, the extent to which assistance has enabled partner nations to achieve and sustain advanced counterterrorism capabilities. S/CT and DS/T/ATA lack clear measures and processes for assessing sustainability, and program managers do not consistently include sustainability in ATA planning.

State Has Recently Established ATA Goals and Measures, and Emphasizes Sustaining Partner Nations’ Counterterrorism Capabilities

State did not have measurable performance goals and outcomes related to the mandated objectives for ATA prior to fiscal year 2003, but has recently made some progress to address the deficiency. State’s Office of Inspector General recommended in 2001, 2005, and 2006 reports that S/CT and DS/T/ATA take steps to establish measurable long-term goals and evaluations of program performance. Similarly, State responded to issues raised in a 2003 Office of Management and Budget assessment of ATA by developing specific goals and measures for each of the program’s mandated objectives.

Since fiscal year 2006, State planning documents, including department and bureau-level performance plans, have listed enabling partner nations to achieve advanced and sustainable counterterrorism capabilities as a key program outcome. S/CT and DS/T/ATA officials further confirmed that sustainability is the principal intended outcome and focus of program assistance. In support of these efforts, DS/T/ATA appointed a Sustainment Manager in November 2006. The Sustainment Manager’s broadly defined responsibilities include coordinating with other DS/T/ATA divisions to develop recommendations and plans to assist partner nations in developing sustainable counterterrorism capabilities.

S/CT and DS/T/ATA Do Not Assess Sustainability of Capabilities

Despite progress towards establishing goals and intended outcomes, State has not developed clear measures and a process for assessing sustainability and has not integrated the concept into program planning. The Government Performance and Results Act of 1993 (GPRA) requires agencies in charge of U.S. government programs and activities to identify goals and report on the degree to which goals are met. S/CT and DS/T/ATA officials noted the difficulty in developing direct quantitative measures of ATA outcomes related to partner nations’ counterterrorism capabilities. However, GPRA and best practices cited by the Office of Management and Budget, us, and others provide flexible guidelines for agency and program managers to develop adequate measures of program effectiveness. Our past work also has stressed the importance of establishing program goals, objectives, priorities, milestones, and measures to use in monitoring performance and assessing outcomes as critical elements of program management and effective resource allocation.

We found that the measure for ATA’s principal intended program outcome of sustainability is not clear. In its fiscal year 2007 Joint Performance Summary, State reported results and future year targets for the number of countries that had achieved an advanced, sustainable level of counterterrorism capability. According to the document, partner nations that achieve a sustainable level of counterterrorism would graduate from the program and no longer receive program assistance. However, program officials in S/CT and DS/T/ATA directly responsible for overseeing ATA were not aware that the Joint Performance Summary listed numerical targets and past results for the number of partner nations that had achieved sustainability, and could not provide an explanation of how State assessed the results. DS/T/ATA’s Sustainment Manager also could not explain how State established and assessed the numerical targets in the reports. The Sustainment Manager further noted that, to his knowledge, S/CT and DS/T/ATA had not yet developed systematic measures of sustainability.

DS/T/ATA’s current mechanism for evaluating partner nation capabilities does not include guidance or specific measures to assess sustainability. According to program guidance and DS/T/ATA officials, needs assessments and program reviews are intended to establish a baseline of a partner nation’s counterterrorism capabilities and quantify progress through subsequent reviews. DS/T/ATA officials also asserted that the process is intended to measure the results of program assistance. However, the process does not explicitly address sustainability, and provides no specific information or instruction regarding how reviewers are to assess sustainability. Moreover, the process focuses on assessing a partner nation’s overall counterterrorism capabilities, but does not specifically measure the results of program assistance. The assessment and review process also does not provide S/CT and DS/T/ATA a means for determining whether a partner nation’s capabilities changed because of program assistance, the country’s own efforts, or through assistance provided by other U.S. agencies or third countries. The head of DS/T/ATA’s Assessment, Review, and Evaluations Unit told us that he had not received guidance to assess progress toward sustainability, and had only limited interaction with the Sustainment Manager on integrating sustainability into the assessment and review process.

DS/T/ATA has not systematically integrated sustainability into country- specific assistance plans, and we found a lack of consensus among program officials about how to address the issue. In-country program managers, embassy officials, instructors, and partner nation officials we interviewed held disparate views on how to define sustainability across all ATA participant countries, and many were not aware that sustainability was the intended outcome for the program. Several program officials stated that graduating a country and withdrawing or significantly reducing program assistance could result in a rapid decline in the partner nation’s counterterrorism capabilities, and could undermine achieving other program objectives, such as improving bilateral relations. Further, although State has listed sustainability in State-level planning documents since 2006, S/CT and DS/T/ATA have not issued guidance on incorporating sustainability into country-specific planning, and none of the country assistance plans we reviewed consistently addressed the outcome. As a result, the plans did not include measurable annual objectives or planned activities targeted at enabling the partner nation to achieve sustainability. For example, Colombia’s assistance plan listed transferring responsibility for the antikidnapping training to the Colombian government and described planned activities to achieve that goal. However, the plan did not include measurable objectives to determine whether activities achieve intended results. Although the plan for the Philippines stated that the country program goal for fiscal year 2007 was to “maximize sustainment,” it did not include measures of sustainability or describe how planned activities would contribute to the intended outcome.

State Reporting on U.S. Counterterrorism Assistance Abroad Has Been Incomplete and Inaccurate

Since 1996, State has not complied with a congressional mandate to report to Congress on U.S. international counterterrorism assistance. Additionally, State’s annual reports on ATA have contained inaccurate data regarding basic program information, do not provide systematic assessments of program results, and lack other information necessary to evaluate program effectiveness.

S/CT Has Not Prepared a Mandated Report

The Foreign Assistance Act requires the Secretary of State to report annually on the amount and nature of all assistance provided by the U.S. government related to international terrorism. Since 1996, State has submitted ATA annual reports rather than the report required by the statute.

The legislation that authorized ATA in 1983 required annual presentations to Congress of aggregate information on all countries that received program assistance. In 1985, Congress added a new, broader reporting obligation, requiring the Secretary of State to report on all assistance related to international terrorism provided by the U.S. government during the preceding fiscal year. Although the original ATA-specific 1983 reporting provision was repealed in 1996, the requirement for the broader report remains.

S/CT is responsible for preparing the reports on U.S. international counterterrorism assistance. The S/CT official directly responsible for ATA told us that he only recently became aware of the reporting requirement and noted confusion within State over what the statute required. He also asserted that the ATA annual report, which is prepared by DS/T/ATA, and State’s annual “Patterns of Global Terrorism” report were sufficiently responsive to congressional needs. He further noted that, in his view, it would be extremely difficult for State to compile and report on all U.S. government terrorism assistance activities, especially given the significant growth of agencies’ programs since 2001. Officials in State’s Bureau of Legislative Affairs indicated that, to their knowledge, they had never received an inquiry from congressional staff about the missing reports. DS/T/ATA officials told us DS/T/ATA has continued to produce the ATA annual report to Congress even after the reporting requirement was removed in 1996. However, State has not issued DS/T/ATA’s annual report to Congress on ATA for fiscal year 2006 that was planned for release in 2007. DS/T/ATA officials noted that they did, however, complete and circulate the final report within State.

ATA Annual Reports Contain Inaccuracies

Recent ATA annual reports have contained inaccurate data relating to basic program information on numbers of students trained and courses offered. For example, DS/T/ATA reported inaccurate data on program operations in ATA’s two top-funded partner nations—Afghanistan and Pakistan.

Afghanistan. ATA annual reports for fiscal years 2002 to 2005 contain narrative passages describing various ATA training and training-related assistance activities for the Afghan in-country ATA program. According to these reports, 15 students were trained as part of a single training event over the 4-year period. DS/T/ATA subsequently provided us data for fiscal year 2005 training activity in Afghanistan, which corrected the participation total in that year from 15 participants in 1 training event to 1,516 participants in 12 training events. DS/T/ATA officials acknowledged the report disparities.

Pakistan. According to the fiscal year 2005 ATA annual report, ATA delivered 17 courses to 335 participants in Pakistan that year. Supporting tables in the same report listed 13 courses provided to 283 participants. Further, a summary report provided to us from the DS/T/ATA internal database produced a third set of numbers describing 13 courses provided to 250 course participants during fiscal year 2005. DS/T/ATA officials acknowledged this inconsistency, but they were unable to identify which set of figures was correct.

DS/T/ATA officials noted that similar inaccuracies could be presumed for prior years and for other partner nations. Significantly, the officials indicated that inaccuracies and omissions in reports of the training participants and events were due to a lack of internal policies and procedures for recording and reporting program data. In the absence of documented policies and procedures, staff developed various individual processes for collecting the information that resulted in flawed data reporting. Additionally, DS/T/ATA officials told us that its inadequate information management system and a lack of consistent data collection procedures also contributed to inaccurate reporting.

ATA Annual Reports Lack Performance and Other Useful Program Information

DS/T/ATA’s annual reports to Congress on ATA from fiscal year 1997 to 2005 did not contain systematic assessments of program results. Further, the reports did not consistently include information on key aspects of the program, such as program activities, spending, and management initiatives that would be helpful to Congress and State in evaluating ATA.

GPRA, Office of Management and Budget guidance, and our previous work provide a basis and rationale for the types of information that are useful in assessing program performance. According to this guidance, key elements of program reporting include clearly defined objectives and goals, comparisons of actual and projected performance that include at least 4 years of annual data, explanations and plans for addressing unmet goals, and reliable information on the program’s activities and financial activity.

We reviewed ATA annual reports for fiscal years 1997 through 2005, and found that the reports varied widely in terms of content, scope, and format. Moreover, the annual reports did not contain systematic assessments of program performance or consistent information on program activity, such as number and type of courses delivered, types of equipment provided, and budget activity associated with program operations. In general, the reports contained varying levels of detail on program activity, and provided only anecdotal examples of program successes, from a variety of sources, including U.S. embassy officials, ATA instructors, and partner nation officials. DS/T/ATA program officials charged with compiling the annual reports for the past 3 fiscal years noted that DS/T/ATA does not have guidance on the scope, content, or format for the reports.

Conclusions

Although ATA plays a central role in State’s broader effort to fight international terrorism, deficiencies in how the program is guided, managed, implemented, and assessed could limit the program’s effectiveness. Specifically, minimal guidance from S/CT makes it difficult to determine the extent to which program assistance directly supports broader U.S. counterterrorism policy goals. Additionally, deficiencies with DS/T/ATA’s needs assessments and program reviews may limit their utility as a tool for planning assistance and prioritizing among several partner nations’ counterterrorism needs. As a result, the assessments and reviews are not systematically linked to resource allocation decisions, which may limit the program’s ability to improve partner nation’s counterterrorism capabilities.

Although State has made some progress in attempting to evaluate and quantitatively measure program performance, ATA still lacks a clearly defined, systematic assessment and reporting of outcomes, which makes it difficult to determine the overall effectiveness of the program. This deficiency, along with State’s noncompliance with mandated reporting requirements, has resulted in Congress having limited and incomplete information on U.S. international counterterrorism assistance and ATA efforts. Such information is necessary to determine the most effective types of assistance the U.S. government can provide to partner nations in support of the U.S. national security goal of countering terrorism abroad.

Matter for Congressional Consideration

Congress should reconsider the requirement that the Secretary of State provide an annual report on the nature and amount of U.S. government counterterrorism assistance provided abroad given the broad changes in the scope and nature of U.S. counterterrorism assistance abroad, in conjunction with the fact that the report has not been submitted since 1996.

Recommendations for Executive Action

We recommend that the Secretary of State take the following four actions: 1. Revisit and revise internal guidance (the 1991 State policy memorandum and Foreign Affairs Manual, in particular) to ensure that the roles and responsibilities for S/CT and DS/T/ATA are still relevant and better enable State to determine which countries should receive assistance and what type, and allocate limited ATA resources. 2. Ensure that needs assessments and program reviews are both useful and linked to ATA resource decisions and development of country- specific assistance plans. 3. Establish clearer measures of sustainability, and refocus the process for assessing the sustainability of partner nations’ counterterrorism capabilities. The revised evaluation process should include not only an overall assessment of partner nation counterterrorism capabilities, but also provide guidance for assessing the specific outcomes of ATA. 4. Comply with the congressional mandate to report to Congress on U.S. international counterterrorism assistance.

Agency Comments and Our Evaluation

State provided us oral and written comments (see app. III) on a draft of this report. State also provided technical comments which we have incorporated throughout the report, as appropriate.

Overall, State agreed with our principal findings and recommendations to improve its ATA program guidance, the needs assessment and program review process, and its assessments of ATA program outcomes. State noted that the report highlights the difficulties in assessing the benefits of developing and improving long-term antiterrorism and law enforcement relationships with foreign governments. State also outlined a number of ongoing and planned initiatives to address our recommendations. Some of these initiatives were underway during the course of our review and we refer to them in the report. We will follow up with State to ensure that these initiatives have been completed, as planned.

However, although State supported the matter we suggest for congressional consideration, it did not specifically address our recommendation that it comply with the congressional mandate to report on U.S. counterterrorism assistance.

As agreed with your office, unless you publicly announce the contents of the report earlier, we plan no further distribution until 30 days after the report date. At that time, we will send copies of the report to interested congressional committees and to the Secretary of State. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions about this report, please contact Charles Michael Johnson, Jr. (202) 512-7331, e-mail johnsoncm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Other GAO contact and staff acknowledgments are listed in appendix IV.

Appendix I: Scope and Methodology

To assess State’s guidance for determining country recipients, aligning program assistance with partner nation needs, and coordinating Antiterrorism Assistance (ATA) with other U.S. government counterterrorism programs, we Interviewed cognizant officials from the Office of Coordinator for Counterterrorism (S/CT) and the Bureau of Diplomatic Security, Office of Antiterrorism Assistance (DS/T/ATA) in Washington, D.C., including senior officials responsible for overseeing and managing ATA and ATA program managers responsible for each of the six in-country programs: Afghanistan, Colombia, Indonesia, Kenya, Pakistan, and the Philippines.

Reviewed and analyzed State planning, funding, and reporting documents concerning ATA, including relevant reports from State’s Office of Inspector General on the management and implementation of ATA; S/CT’s fiscal year 2007 tiered lists of priority countries for ATA assistance and S/CT criteria for establishing the tier list; DS/T/ATA budget information for fiscal years 2000 to 2008; a 1991 State policy memorandum delineating S/CT’s and DS/T/ATA’s roles and responsibilities for ATA; relevant sections of State’s Foreign Affairs Manual summarizing roles and responsibilities for ATA; DS/T/ATA internal policy and procedure documents, including DS/T/ATA’s Assessment, Review and Evaluations Unit’s most current (2004) standard operations procedures; State documents and U.S. embassy cables regarding the Regional Strategic Initiative; and DS/T/ATA’s Annual Reports to Congress on the ATA for fiscal years 1997 to 2005.

Reviewed and analyzed available country-specific program documents for five of the in-country programs—Colombia, Indonesia, Kenya, Pakistan, and the Philippines—including country-specific needs assessments conducted for each of these partner nations; country assistance plans; data on the number of ATA courses provided and personnel trained in these countries; and memoranda of intent between the U.S. government and host country governments regarding ATA in these countries for fiscal years 2000 to 2007. These five countries were among the largest six recipients of program assistance for fiscal years 2002 to 2007 and each country received a range of ATA training and other assistance during the period we reviewed. DS/T/ATA was unable to provide four of the needs assessments that, according to annual reports, were conducted for two of these countries in that time, and was only able to provide three ATA country assistance plans that were completed for three of the five countries for fiscal years 2006 to 2008.

Conducted fieldwork between July and September 2007 in four countries where ATA provides a range of assistance through an in-country presence: Colombia, Indonesia, Kenya, and the Philippines. These four programs represented about 55 percent of ATA allocations for training and training- related activities in fiscal year 2006, and about 43 percent of funding in fiscal year 2007. As this was not a generalizeable sample, our observations in these four countries may not be representative of all programs. In these countries, we interviewed ATA in-country program managers, course instructors, and other contractors; U.S. embassy officials responsible for managing counterterrorism assistance and activities; and partner nation government officials. We also observed various types of ATA training and examined equipment that was provided to partner nation security units. Additionally, to assess the extent to which State establishes clear ATA goals and measures sustainability of program outcomes, and State’s reporting on U.S. international counterterrorism assistance, we Interviewed cognizant officials from S/CT and DS/T/ATA in Washington, D.C., including senior officials responsible for overseeing and managing ATA and ATA program managers responsible for each of the six in-country programs: Afghanistan, Colombia, Indonesia, Kenya, Pakistan, and the Philippines. Additionally, we interviewed cognizant officials in DS/T/ATA’s Assessment, Review, and Evaluations Unit, Training Curriculum Division, Training Delivery Division, and Training Management Division, including the Sustainment Manager.

Reviewed and analyzed State strategic planning and performance reporting documents related to ATA for fiscal years 2001 to 2007, including State budget justifications, State Performance Plans; State Performance Summaries; Bureau Performance Plans; Mission Performance Plans for Afghanistan, Colombia, Indonesia, Kenya, Pakistan, and the Philippines; and DS/T/ATA annual reports to Congress on ATA as noted above. We also reviewed Office of Management and Budget’s fiscal year 2003 review of ATA and relevant State Office of Inspector General reports relating to performance measurement issues for ATA. Additionally, we reviewed all available S/CT and DS/T/ATA guidance related to assessing program performance, including internal standard operating procedure documents and course evaluation instruments, as well as ATA authorizing legislation and related revisions. To further assess State’s reporting on international counterterrorism assistance, we reviewed DS/T/ATA’s annual reports on ATA for consistency and accuracy. As noted earlier, we found some errors with these reports, and have concerns about the data on training and nontraining activities. Although we describe the errors, we did not use these data in our analyses. To assess the reliability of the data on funding to recipient countries, we interviewed ATA officials and performed some cross-checks with other sources. We determined the data on funding were sufficiently reliable for the purposes of this report.

Appendix II: Top Recipients of ATA Funding, Fiscal Years 2002 to 2007

As shown in table 2, program assistance for the top 10 recipients of ATA funding from fiscal years 2002 to 2007 ranged from about $11 million to about $78 million. The top 10 funding recipients received about 57 percent of ATA funding allocated for training and training related activities over the 6-year-period. ATA has established an in-country presence in each of the top six partner nations, including in-country program staff and permanent training facilities such as classrooms, computer labs, and shooting and demolition ranges. Afghanistan received the most funding over the six-year-period. According to DS/T/ATA officials, the scope of the in-country program in Afghanistan is more narrowly defined than other ATA programs; it focuses principally on training and monitoring a Presidential Protective Service.

Appendix III: Comments from the Department of State

Appendix IV: GAO Contact and Staff Acknowledgments

Acknowledgments

In addition to the individual named above, Albert H. Huntington, III, and David C. Maurer, Assistant Directors; Karen A. Deans; Matthew E. Helm; Elisabeth R. Helmer; Grace Lui; and Emily T. Rachman made key contributions to this report.