Security Assistance: Evaluations Needed to Determine Effectiveness of U.S. Aid to Lebanon's Security Forces
Published: 2013-03-19T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Following Syrias withdrawal from Lebanon in 2005 and war between Israel and Hezbollah in 2006, U.S. agencies increased their allocations of security assistance for Lebanon from $3 million in 2005 to about $28 million in 2006. This assistance included training and equipment funded and implemented by State or DOD for the Lebanese Armed Forces and Internal Security Forces of Lebanon. However, questions remain regarding the effectiveness of security assistance as a tool of U.S. policy in Lebanon, including concerns about the influence of foreign actors, primarily Syria and Iran, and extremist militant groups operating in Lebanon.
GAO was asked to review U.S. security assistance to Lebanon. GAOs review, covering fiscal years 2007 through 2012, assessed the extent to which the U.S. government (1) adjusted its strategic goals and security assistance programs in Lebanon, (2) funded assistance programs for Lebanese security forces, and (3) evaluated the effectiveness of security assistance programs in Lebanon. GAO reviewed budgetary data and planning documents and interviewed U.S. and Lebanese government officials in Washington, D.C.; Tampa, Florida; and Beirut, Lebanon.
section_title: What GAO Found
paragraphs: The United States has kept strategic goals for Lebanon constant since 2007 and adjusted security assistance in response to political and security conditions. Since 2007, U.S. strategic goals for Lebanon have been to support the nation as a stable, secure, and independent democracy. According to U.S. officials, U.S. policy priorities include supporting the Government of Lebanon in establishing stability and security against internal threats from militant extremists and the influence of Iran and Syria. U.S. programs to help achieve these priorities include Foreign Military Financing, International Military Education and Training (IMET), International Narcotics Control and Law Enforcement (INCLE), Antiterrorism Assistance, Counterterrorism Financing, Export Control and Related Border Security, and Section 1206 and 1207 authorities. While strategic goals have not changed, program implementation has changed to meet conditions on the ground, according to U.S. officials. For example, the Department of State (State) delayed committing Foreign Military Financing funds to Lebanon for 3 months in 2010, following an exchange of fire between the Lebanese Armed Forces and Israeli forces.
U.S. agencies allocated over $925 million for security assistance programs for Lebanon from fiscal years 2007 through 2012; State has disbursed and the Department of Defense (DOD) has committed the majority of the funds.
To date, State has evaluated only one of its security assistance programs for Lebanon, the INCLE program; neither State nor DOD has completed plans or established time frames to evaluate the other programs. State's evaluation policy requires that certain programs be evaluated periodically. Without such evaluations, State and DOD have little objective evidence to show that the programs have been effective or what the proper mix of programs should be. Evaluations can be facilitated through appropriate performance measurement. However, GAO and other agencies have previously reported deficiencies in how agencies measure program performance. For example, GAO found in 2011 that the IMET program evaluation efforts had few of the elements commonly accepted as appropriate for measuring performance. State and DOD are undertaking efforts to develop better performance indicators.
section_title: What GAO Recommends
paragraphs: State and DOD should complete plans with milestone dates to evaluate security assistance programs in Lebanon and develop better performance indicators to facilitate evaluation. State and DOD concurred.

section_title: Letter
paragraphs: Lebanon is a religiously diverse country transitioning toward  independence and democratic consolidation after a 15-year civil war and  subsequent occupation by Syrian and Israeli forces. Lebanon is home to  the militant group Hezbollah, which is also a member of the Lebanese  government. Following Syrian withdrawal from Lebanon in 2005 and the  war between Israel and Hezbollah in the summer of 2006, the United  States significantly increased assistance for Lebanon, for the Lebanese  Armed Forces (LAF) and Internal Security Forces (ISF). In 2005, U.S.  agencies allocated about $3 million in U.S. security assistance for  Lebanon but the next year allocated almost $28 million in security  assistance. In 2010, the LAF fired on Israeli Defense Forces along  Lebanon’s and Israel’s shared border. A member of Congress raised  concerns about U.S. assistance to Lebanon in the wake of this clash.
In response to your request that we review U.S. security assistance to  Lebanon, we assessed the extent to which the U.S. government, from  fiscal years 2007 through 2012, (1) adjusted its strategic goals and  security assistance programs in Lebanon, (2) funded assistance  programs for Lebanese security forces, and (3) evaluated the  effectiveness of security assistance programs in Lebanon. To address  these objectives, we analyzed Department of State (State) and  Department of Defense (DOD) documents, such as the Mission Strategic  and Resource Plans for Lebanon, program plans and guidance,  congressional budget justifications, and other relevant documents. We  obtained funding data from State and DOD on programs that provide  security assistance to Lebanon. We also interviewed State and DOD  officials in Washington, D.C., and Beirut, Lebanon, and DOD officials in  Tampa, Florida. Appendix I provides a detailed discussion of our scope  and methodology.
We conducted this performance audit from June 2012 to March 2013 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Lebanon is a small, religiously diverse country on the Mediterranean Sea  that borders Israel and Syria. (See fig. 1.)
Religious tensions among Lebanon’s Maronite Christians, Sunni Muslims,  and Shiite Muslims, among others, along with an influx of Palestinian  refugees into Lebanon, have fueled Lebanon’s internal strife and conflicts  with its neighbors. During the civil war between 1975 and 1990, both  Syrian and Israeli forces occupied the country. In the midst of the civil war  and Israel’s continued occupation of southern Lebanon, Hezbollah  emerged in Lebanon as a powerful Islamic militant group. Throughout the  1990s, Hezbollah, funded by Iran and designated by the United States  and Israel as a terrorist organization, pursued its military campaign  against Israeli forces occupying Lebanon while also participating in  Lebanon’s political system. In 2000, Israeli forces withdrew from southern  Lebanon. In 2005, with pressure from the international community, Syrian  forces withdrew from Lebanon following the assassination of Lebanon’s  prime minister. The subsequent parliamentary elections that year resulted  in a member of Hezbollah holding a cabinet position for the first time. In  the summer of 2006, Hezbollah and Israel entered into a month-long  conflict that ended with the adoption of United Nations Security Council  Resolution 1701 by both the Israeli and Lebanese governments. The  resolution called for Israeli withdrawal from southern Lebanon in parallel  with the deployment of Lebanese and United Nations forces and the  disarmament of all armed groups in Lebanon, among other things. Since  2011, instability in neighboring Syria may have exacerbated sectarian  conflict within Lebanon. See figure 2 for a timeline of selected political  events in Lebanon.
The United States has provided Lebanon with assistance, such as  emergency humanitarian aid during the civil war and training for military  forces under the International Military Education and Training (IMET)  program. The United States and Lebanon have historically enjoyed a  good relationship in part because of cultural and religious ties, a large  Lebanese-American community in the United States, and the pro- Western orientation of Lebanon, particularly during the Cold War.  Following the Syrian withdrawal in 2005 and the 2006 Israeli-Hezbollah  war, the United States increased its security assistance to Lebanon. The  United States provided assistance to the LAF, which is generally  responsible for providing border security, counterterrorism, and national  defense, and to the ISF, or police force, which is generally responsible for  maintaining law and order in Lebanon. The fiscal year 2006  appropriations marked the first time since 1984 that U.S. agencies  allocated Foreign Military Financing grants to help modernize and equip  the LAF. Since then, the United States has provided security assistance  to Lebanon through the Foreign Military Financing program; IMET;  International Narcotics Control and Law Enforcement (INCLE) program;  the Nonproliferation, Antiterrorism, Demining, and Related (NADR)  programs; and Section 1206 and Section 1207 authorities for training and  equipping foreign militaries and security forces and for reconstruction,  stabilization, and security activities in foreign countries, respectively. The  NADR security assistance for Lebanon has been provided through three  programs: Antiterrorism Assistance, Counterterrorism Financing, and  Export Control and Related Border Security (EXBS). Table 1 describes  these security assistance programs. In addition, to these security  assistance programs, U.S. Special Forces units have provided  specialized training to LAF Special Forces units, according to agency  officials.
section_title: The United States Has Kept Strategic Goals for Lebanon Constant Since 2007 and Adjusted Security Assistance in Response to Political and Security Conditions
paragraphs: The United States has kept strategic goals for Lebanon constant since  2007. These goals include supporting the Government of Lebanon in  establishing stability and security against internal threats from militant  extremists and countering destabilizing influences. U.S. agencies have  adjusted security assistance in response to Lebanon’s political and  security conditions. For example, State and DOD have delayed releasing  funds and limited the types of equipment provided. Both agencies have  also implemented additional assistance programs since fiscal year 2007.
subsections: section_title: U.S. Strategic Goals Are to Support a Stable, Secure, and Independent Lebanese Government
paragraphs: Since 2007, U.S strategic goals for Lebanon have been to support the  nation as a stable, secure, and independent democracy. According to  DOD and State officials, the overarching priorities for Lebanon remain  focused on supporting Lebanese sovereignty and stability and countering  the influence of Syria and Iran. Security-related goals for Lebanon focus  on counterterrorism and regional stability or internal security. Programs  activities seek to support development of the LAF and the ISF as the only  legitimate providers of Lebanon’s security. The goals and objectives of  the individual security assistance programs are intended to support the  U.S. strategic goals and overarching priorities, as the following examples  illustrate:
Goals for the Foreign Military Financing, IMET, and 1206 programs in  Lebanon since the departure of Syrian forces are to bolster the  capability of the LAF, nurture the bilateral military relationship  between the United States and the LAF, and continue encouraging  establishment of a stable, legal, and pro-U.S. civil government.
Goals for the INCLE police training program are (1) to build Lebanon’s  operational capacity to combat crime, and prevent and respond to  terror attacks; and (2) to assist Lebanon in developing the ISF into a  competent, professional, and democratic police force with the  necessary training, equipment, and institutional capacity to enforce  the rule of law in Lebanon, cement sovereign Lebanese government  control over its territory, and protect the Lebanese people.
Goals for the Antiterrorism Assistance program in Lebanon are to  develop and build the Lebanese government’s capacities in border  security, mid- and senior-level leadership development, and  counterterrorism investigations.
Goals for the Counterterrorism Financing Program are to deny  terrorists access to money, resources, and support.
Goals for EXBS in Lebanon focus on strengthening the capability of  Lebanese enforcement agencies to effectively control cross-border  trade in strategic goods.
Goals for the 1207 program were to strengthen Lebanon’s internal  security forces after armed conflicts in 2006 and 2007.
section_title: State and DOD Have Adjusted Implementation of Security Assistance Programs to Meet Conditions on the Ground
paragraphs: The goals and objectives of U.S. security assistance to Lebanon have  continued to focus on supporting Lebanese sovereignty and stability and  countering the influences of Syria and Iran. However, according to State  and DOD officials, the agencies have changed how they implement  programs based on changes in the political and security situation, for  example, by delaying the release of funds or limiting the types of  equipment provided. In one instance, State delayed committing fiscal year  2010 Foreign Military Financing funds as a result of an incident on August  3, 2010, in which the LAF opened fire on an Israeli Defense Force unit  engaged in routine maintenance along the Blue Line, alleging that it had  crossed into Lebanese territory. Two Lebanese soldiers, a journalist, and  an Israeli officer were killed. According to State, in response to concerns  raised by a member of Congress, State delayed committing $100 million  of fiscal year 2010 Foreign Military Financing funds for Lebanon, citing  the need to determine whether equipment that the United States provided  to the LAF was used against Israel. According to a State official, State  committed the funds in November 2010 after consulting with the member  of Congress.
In addition, according to State officials, State and DOD decided to place a  temporary hold on lethal assistance to the LAF in January 2011, after the  collapse of Prime Minister Saad Hariri’s government. In March 2012, the  agencies decided to lift the hold on lethal assistance based on the Mikati  government’s adherence to key international obligations to the Special  Tribunal for Lebanon and United Nations Security Council Resolution.
The U.S. government has also implemented new security assistance  programs in Lebanon since fiscal year 2007 in response to changing  security assistance needs, according to State officials. For example, State  began implementing the EXBS program in Lebanon in fiscal year 2009— after State performed an assessment to identify deficiencies in the  country’s ability to detect and interdict weapons of mass destruction and  advanced conventional weapons, according to officials of State’s Bureau  of International Security and Nonproliferation. State determined that  Lebanon did not have a comprehensive system to regulate trade in  strategic goods and technologies for the purpose of preventing the  proliferation of such weapons, and lacked the necessary legal and  institutional elements to manage strategic trade consistent with  international standards. In addition, in fiscal year 2010, State’s Bureau of  Counterterrorism began implementing its Counterterrorism Financing  program in Lebanon. According to bureau officials, the goals of the  program are to build the foreign capacity of the Government of Lebanon  to develop laws and regulations to deny terrorists access to funds.
State has also changed the focus of some of the security assistance  programs. For example, the INCLE police training program is no longer  providing basic training to cadets, according to the Bureau of International  Narcotics and Law Enforcement Affairs. The bureau turned the basic  training over to the ISF as of July 2012 and is providing leadership and  management training. The bureau is working with the Federal Bureau of  Investigation and the Drug Enforcement Administration to develop  specialized training to increase the ISF’s investigatory skills, according to  U.S. embassy officials. The bureau is also expanding its community  policing program, according to bureau officials. In addition, State’s Bureau  for Counterterrorism has changed the focus of its antiterrorism training,  according to officials of State’s Bureau of Diplomatic Security whom we  interviewed in Lebanon. They explained that when the program began in  2007, it focused on training Lebanese security details to protect national  leaders. The trained protection details have demonstrated a mastery of  the skills for which they were trained. As a result, the Bureau for  Counterterrorism and the Bureau of Diplomatic Security consider the  protection mission to be complete and since 2010 has shifted its focus to  border security, management and leadership, and counterterrorism  investigations, according to the Diplomatic Security officials. Finally, while  there have been no major changes to the EXBS program in Lebanon,  U.S. embassy officials stated that the United States has increased the  program’s emphasis on convincing the government to change its export  control laws.
section_title: U.S. Agencies Allocated Over $925 Million in Security Assistance for Lebanon during Fiscal Years 2007-2012 and Have Disbursed or Committed Most of the Funds
paragraphs: U.S. agencies have allocated over $925 million in security assistance for  Lebanon from fiscal years 2007 through 2012, with funds varying by year  and program. The majority of funds—69 percent—was from the Foreign  Military Financing program, though State and DOD also utilized seven  other programs. State has disbursed the majority of funds it allocated for  each fiscal year, while DOD has committed the majority of funds for  Foreign Military Financing.
subsections: section_title: Security Assistance Allocated for Lebanon Varied by Year
paragraphs: In order to help achieve U.S. strategic goals, State and DOD have  allocated more than $925 million in security assistance for Lebanon’s LAF  and ISF from fiscal years 2007 through 2012. This funding peaked in  fiscal year 2007 at about $323 million, declined in fiscal year 2008 to  about $32 million, and has fluctuated from fiscal years 2009 through  2012. (See fig. 3.)
In fiscal year 2007, the $325 million allocated for security assistance for  Lebanon followed the 2006 war between Hezbollah and Israel and  assisted Lebanon in fulfilling its obligations under the United Nations  Security Council Resolution 1701. This funding level represented a  significant increase over fiscal year 2006, when U.S. security assistance  for Lebanon totaled only about $28 million. See appendix II for  information on allocation, obligation, and disbursement or commitment of  security assistance for Lebanon by program and year.
section_title: U.S. Agencies Allocated the Majority of Security Assistance through the Foreign Military Financing Program and Used Seven Other Programs
paragraphs: U.S. agencies allocated the majority (approximately 69 percent, or $641  million) of security assistance for Lebanon from fiscal years 2007 through  2012 through the Foreign Military Financing program, which provides  grants and loans to the Lebanese Armed Forces to purchase U.S.  equipment, services, and training. Figure 4 shows the distribution of U.S.  security assistance allocated for Lebanon by program.
The Foreign Military Financing program funded the purchase of various  equipment and services. For example, Lebanon received trucks, truck  tractors, trailers, ambulances, cargo-troop carriers, and armament  vehicles through the Foreign Military Financing program. Other equipment  purchased through the Foreign Military Financing program included  helicopters, ships, radios, and spare parts. Appendix III presents selected  equipment and services provided to Lebanon by the U.S. government and  by other governments from fiscal years 2007 through 2012.
section_title: U.S. Agencies Disbursed or Committed the Majority of Funds
paragraphs: Generally, State disbursed the majority of funds allocated during the  same fiscal year for security assistance programs providing funds for  Lebanon, and DOD committed the majority of Foreign Military Financing  funds for Lebanon. See appendix II for additional information on  allocation, obligation, and disbursement or commitment of security  assistance for Lebanon by program and year. Specifically, State  disbursed about 78 percent of total allocations from fiscal years 2007  through 2012 for the IMET, INCLE, Antiterrorism Assistance,  Counterterrorism Financing, and EXBS programs and the Section 1206  and 1207 authorities. Although State has disbursed the majority of these  allocated funds, State has not disbursed some funds allocated for the  INCLE and Antiterrorism Assistance programs for fiscal years 2007  through 2011. According to State officials, some obligated funds are  waiting to be disbursed and all of the unobligated funds are no longer  available for obligation. DOD has committed 87 percent of Foreign  Military Financing funds for Lebanon for fiscal years 2007 through 2012.
section_title: U.S. Agencies Have Done Little Evaluation of Security Assistance Programs in Lebanon
paragraphs: As of February 2013, U.S. agencies had evaluated only the INCLE police  training program for Lebanon and did not have firm plans to evaluate the  other six ongoing security assistance programs. State’s evaluation policy  requires periodic evaluations of certain programs, consistent with  standards established in U.S. law. Evaluations can be facilitated by  collecting data on appropriate performance indicators. However, we and  other audit agencies have previously reported deficiencies in how State  and DOD measure program performance. For example, we found in 2011  that the IMET program evaluation efforts had few of the elements  commonly accepted as appropriate for measuring progress and did not  objectively measure how IMET contributes to long-term desired  outcomes. We have also reported deficiencies in how DOD defined  performance measures for the 1206 program. In response to those  reports, State and DOD concurred with our recommendations and  described agency efforts to develop better performance measures.
subsections: section_title: State Policy Requires Periodic Evaluations of Certain Programs
paragraphs: State’s evaluation policy, which is partly based on the Government  Performance and Results Modernization Act of 2010, requires that all  large programs, projects, and activities be evaluated at least once in their  lifetime or every 5 years, whichever is less. According to State, the 2010  act strengthened the mandate to evaluate programs, requiring agencies  to include a discussion of evaluations in their strategic plans and  performance reports. State established its evaluation policy in February  2012 in part to comply with the requirements of this act. In addition,  according to State, the policy supports State’s goal of connecting  evaluation to its investments in diplomacy and development to ensure  that they align with the agency’s overarching strategic goals and  objectives. State’s evaluation guidance requires each bureau to evaluate  two to four projects, programs, or activities over a 24-month period  beginning with fiscal year 2012, depending on the size, scope, and  complexity of the programs being evaluated and the availability of  funding. State’s evaluation policy also requires all bureaus to complete a  bureau evaluation plan and to update it annually. State requires the  bureaus’ plans to align with evaluation policy guidance and to assist each  bureau in assessing the extent to which its efforts contribute to achieving  its intermediate objectives and, by extension, its longer-term goals.
section_title: Program Evaluations Can Be Facilitated by Performance Measures
paragraphs: Program evaluations are individual systematic studies conducted  periodically or on an ad hoc basis to assess how well a program is  working. As a key component of effective program management,  evaluations assess how well a program is working and help managers  make informed decisions about current and future programming. An  evaluation provides an overall assessment of whether a program works  and identifies adjustments that may improve its results. Depending on  their focus, evaluations may examine aspects of program operations  (such as in a process evaluation) or factors in the program environment  that may impede or contribute to its success. Program evaluations may  systematically compare the effectiveness of alternative programs aimed  at the same objective. Types of evaluation include process (or  implementation), outcome, and impact evaluations, as well as cost-benefit  and cost-effectiveness analyses.
Performance measurement, by contrast, is the ongoing monitoring and  reporting of program accomplishments, particularly progress toward pre- established goals. Performance measurement is typically conducted by  program or agency officials and focuses on whether a program has  achieved its objectives, expressed as measurable performance  indicators. As indicators of progress toward goals, performance measures  can inform overall program evaluations. Program evaluations typically  examine a broader range of information on program performance and its  context than is feasible to monitor on an ongoing basis. Both forms of  assessment—program evaluation and performance measurement—aim  to support resource allocation and other policy decisions to improve  service delivery and program effectiveness. But performance  measurement, because of its ongoing nature, can serve as an early  warning system to management and as a vehicle for improving  accountability to the public.
section_title: State Has Evaluated One Security Assistance Program in Lebanon
paragraphs: As of February 2013, U.S. agencies had evaluated only one of their  security assistance programs in Lebanon. The Bureau of International  Narcotics and Law Enforcement Affairs contracted with an external  organization to conduct an evaluation of the INCLE police training  program for the ISF. The evaluation was conducted between November  2010 and May 2011. The bureau commissioned the evaluation to  establish the relevance, effectiveness, impact, and sustainability of the  program from 2008 through 2010. The evaluation covered five training  courses, and the evaluation team collected data by means of focus  groups, semistructured interviews, structured observation, learning tests,  field cluster research, and document review.
The INCLE police training evaluation demonstrated the importance of  conducting a program evaluation. According to the evaluation, the training  was effective, with clear evidence that the ISF trainees had both learned  and been able to retain the core knowledge taught in all five courses. The  program also succeeded in reaching ISF personnel from throughout the  country. The evaluation further determined that the training program  made a relevant contribution to international donor assistance to the ISF.  However, the evaluation concluded that the program did not reach its  principal stated objective in the program period. The evaluation found no  evidence to suggest that the performance of the ISF had systematically  improved as a result of the training program. The evaluation also  concluded that, although trainees learned and retained knowledge  successfully, they were generally not able to apply the skills taught by the  program in their daily duties. The evaluation identified the failure to apply  skills as the key impediment to program effectiveness and attributed the  failure to a flaw in the program design, which failed to sufficiently engage  the ISF in the inception stage to clarify objectives and secure broad  consensus on program goals. Moreover, the design of the training was  not informed by a systematic assessment of the ISF’s training needs.
The final evaluation report included five recommendations to the Bureau  of International Narcotics and Law Enforcement Affairs, including a joint  recommendation with the ISF, and two additional recommendations for  the ISF. The recommendations included a number of subcomponents.  The report recommended that the bureau take the following actions:  continue the training based on renewed program consensus;  closely coordinate program transition with ISF on the operational  level; expand senior leadership training; strengthen internal capacity for program design and management;  and  in addition to quantitative performance measures, design meaningful  qualitative performance indicators in cooperation with the ISF to  provide relevant and timely progress information throughout the  program period.
The evaluation report also outlined steps to transition the training program  to full ISF control. Bureau officials stated that the bureau has adopted  many of the report’s recommendations, including the transition of one  course to full ISF control in July 2012, the creation of a training  coordination process for INL and other donors to maximize synergies and  avoid duplication of effort, and the creation of a strategic planning cell  within the ISF to coordinate directly with the bureau and other donors.
section_title: DOD Has Not Evaluated the Effectiveness of the Section 1206 Program in Lebanon
paragraphs: Although the Section 1206 program in Lebanon has not been evaluated,  DOD—under the direction of the Under Secretary of Defense for Policy—  is implementing a new assessment process for the Section 1206  program. The objectives of the assessment process are to measure implementation of Section 1206 programs to build partner assess quality and timeliness of program implementation, measure the impact of programs, and estimate return on investment.
DOD is implementing this new process in part to respond to our 2010  recommendation that DOD develop and implement specific plans to  monitor, evaluate, and report routinely on Section 1206 project outcomes  and their impact on U.S. strategic objectives. DOD conducted a pilot test  of its assessment process of counterterrorism-oriented Section 1206  programs in the Philippines and stability operations-oriented Section 1206  programs in Georgia in 2012. DOD also assessed Section 1206 programs  in Djibouti, Tunisia, and Poland from March through June 2012. DOD  officials stated in December 2012 that they were not able to assess the  program in Lebanon because of the security situation, but they plan to  include Lebanon as soon as the security situation permits.
Although DOD has not evaluated the effectiveness of the 1206 program  or any other security assistance programs in Lebanon in which it  participates, it conducts various types of assessments of the LAF. For  example, since 2010, the U.S. Central Command and the LAF have  participated in reviews—known as joint capabilities reviews—to assess  the progress of the LAF based on eight broad critical capabilities. The  critical capabilities—for example, Land and Border Defense and  Security—are linked to the broad strategic goals described in State’s  Mission Strategic and Resource Plans for Lebanon (now Mission  Resource Requests), according to DOD Central Command officials. The  joint capabilities reviews include milestone dates from 2013 through 2015.  The U.S. Central Command also conducts annual assessments linked to  the Theater Campaign Plan for Lebanon. These assessments grade the  LAF on desired outcomes and make recommendations for course  corrections. According to the DOD Central Command officials, the  assessments are organized by lines of effort, such as defeating violent  extremist organizations and building partner capacity. These lines of effort  are not linked to specific security assistance programs, however.
section_title: State Has Not Completed Plans or Established Time Frames to Evaluate the Effectiveness of the Remaining Security Assistance Programs in Lebanon
paragraphs: Although bureau evaluation plans were due in May 2012, State bureaus  responsible for security assistance programs were at varying stages of  implementing State’s evaluation policy at the time of this report. For  example, State’s Bureau of Political-Military Affairs, which is responsible  for the Foreign Military Financing and IMET programs, created an  evaluation plan for fiscal year 2013 which does not include an evaluation  of the programs in Lebanon, according to a bureau official.
State’s Bureau of Counterterrorism, which manages the Counterterrorism  Financing program and—along with the Bureau of Diplomatic Security—is  responsible for the Antiterrorism Assistance program, created an  evaluation plan for fiscal year 2013, according to a Bureau of  Counterterrorism official. This plan does not include an evaluation of the  programs in Lebanon. The Bureau of Counterterrorism will evaluate the  Antiterrorism Assistance programs for two other partner nations,  Bangladesh and Morocco. According to a Bureau of Counterterrorism  official, the bureau will consider the Antiterrorism Assistance Program in  Lebanon for the next tranche of evaluations beginning in fiscal year 2014.
State’s Bureau of International Security and Nonproliferation, which  implements the EXBS program, submitted its bureau evaluation plan on  April 10, 2012. The plan, which covers fiscal years 2012 through 2015,  does not include an evaluation of the EXBS program. The bureau does  periodically assess the legal, regulatory, and institutional components of a  country’s strategic trade control systems using a 419-point assessment  methodology. The bureau conducted such an assessment for Lebanon in  2010, according to bureau officials. However, according to the bureau’s  evaluation plan, the assessments do not assess enforcement capacity to  the extent preferred and may not fully satisfy the intent of State’s  evaluation policy. The bureau is working with the Office of U.S. Foreign  Assistance Resources to determine how to integrate the assessment  methodology into program evaluations that would meet State’s evaluation  policy. The bureau may evaluate EXBS within the fiscal years 2013 to  2016 time frame.
Although State bureaus are at varying stages of developing their  evaluation plans, State has developed department-wide guidance for  standardizing the way it measures the effectiveness of its programs and  has entered into contracts with five contractors for monitoring and  evaluation services, according to State officials. State bureaus can use  these contracts to evaluate their programs.
section_title: Past Deficiencies Reported in Agencies’ Performance Measurement May Inhibit Program Evaluation
paragraphs: We have reported previous deficiencies in performance measurement of  State’s and DOD’s security assistance programs, and these deficiencies  may inhibit agencies’ ability to conduct program evaluations. As we have  previously reported, performance measurement of State and DOD  security assistance programs exhibited several deficiencies, including a  lack of specific, measurable, and outcome-oriented performance  indicators.
In 2008, we reported that State did not systematically assess the  outcomes of the Antiterrorism Assistance program and, as a result,  could not determine the effectiveness of this program. More  recently, a 2012 State Office of Inspector General assessment of the  antiterrorism programs in certain countries reported that the Bureau of  Diplomatic Security’s Office of Antiterrorism Assistance could not  determine the Antiterrorism Assistance program’s effectiveness in part  because they had not developed specific, measurable, and outcome- oriented program objectives.
In 2011, we reported that State’s and DOD’s ability to assess IMET’s  effectiveness was limited by several weaknesses in program  monitoring and evaluation, including the lack of a performance plan  for IMET that explained how the program was expected to achieve its  goals and how progress could be assessed through performance  measures and targets.
In 2010, we reported that DOD and State had incorporated little  monitoring and evaluation into the Section 1206 program and had not  consistently defined performance measures for Section 1206  projects.
section_title: Although Facing Challenges, State and DOD Are Continuing Efforts to Improve Performance Measurement
paragraphs: According to State and DOD officials, assessing the impacts of security  assistance programs is challenging, but they continue to seek  improvements in performance measurement. For example, officials of the  Bureaus of Counterterrorism and Diplomatic Security stated that it is an  enormous task to come up with concrete, meaningful performance  indicators for the Antiterrorism Assistance program, which addresses  aviation security, investigations, and response techniques. In addition,  officials at the U.S. Embassy in Beirut stated that State and DOD are  struggling to develop specific, measurable performance indicators and  still have a long way to go. DOD officials stated that developing metrics to  measure the results of security assistance programs in Lebanon is  difficult. The officials stated that some indicators may be subjective and  difficult to quantify—for example, U.S. influence in building relationships  or the willingness of the LAF to fight. Furthermore, they said that it is  difficult to quantify what would have happened in the absence of U.S.  assistance.
Because the agencies do not have specific, measurable, and outcome- oriented performance indicators, U.S. officials have cited anecdotal  evidence that they believe demonstrates the effectiveness of U.S.  security assistance programs in Lebanon. For example, some officials  have cited the LAF’s success in taking control of a Palestinian refugee  camp from Al-Qaeda-inspired militants as an indication of the impact of  the assistance. The United States provided ammunition and other  supplies to assist the LAF during the 3-month engagement. As other  indications of effectiveness, U.S. officials also mentioned the LAF’s  actions in arresting members of a Shiite clan involved in kidnappings, the  professionalism demonstrated by LAF Special Forces units in training  exercises, and increases in drug seizures. However, the use of only  positive anecdotal evidence does not provide sufficient context and scope  to help officials judge the effectiveness of these programs.
State and DOD officials described agency efforts to improve performance  measurement. For example, the Bureau of International Narcotics and  Law Enforcement Affairs is shifting from reporting output measures, such  as numbers of students trained, to outcome measures, according to  bureau officials. The bureau plans to collect more accurate performance  metrics that focus on measuring the impact of the INCLE police training  program in Lebanon. The bureau also plans to consider such factors as  the quality of the officers and the extent to which women are integrated  into the ISF, according to bureau officials. Moreover, the Bureau of  International Narcotics and Law Enforcement Affairs is working with the  ISF to improve the performance indicators in the ISF’s strategic plan,  according to officials at the U.S. Embassy in Beirut. The current plan  includes some output measures, such as number of traffic stops made by  police. Embassy officials also stated that they are working with the LAF to  develop a military cooperation plan that will include performance  measures. In addition, according to DOD Central Command officials,  efforts are under way to develop a new framework for its annual  assessments of the LAF that will include improved performance  measures. The new framework will examine the effects of DOD efforts in  Lebanon.
section_title: Conclusions
paragraphs: Since 2007, U.S. agencies have allocated almost $1 billion in security  assistance for Lebanon, supporting its efforts to build a stable, secure,  and independent democracy following the withdrawal of Syrian forces and  the Israeli-Hezbollah war. U.S. security assistance for Lebanon has  presented certain risks because of the influence wielded by the militant  group Hezbollah, which is now a member of the Lebanese government.  Ensuring that U.S. security assistance for Lebanon is effective in  achieving U.S. strategic goals is now more important than ever—to help  Lebanon resist the influence of Iran, which funds Hezbollah, and to  withstand the potential spillover of conflict in Syria.
Program evaluation and performance measurement are key management  tools to ensure that U.S. security assistance is effective in achieving U.S.  strategic goals. However, the U.S government has evaluated the  effectiveness of only one security assistance program in Lebanon, a  program to which it allocated about 14 percent of the nearly $1 billion in  security assistance allocations since 2007. Neither State nor DOD has  completed plans or time frames to evaluate the remaining six ongoing  U.S. security assistance programs in Lebanon. Without evaluations of all  of its security assistance programs in Lebanon, the U.S. government  cannot show that the programs have been effective in achieving their  specific objectives or that they constitute the best mix of security  assistance for Lebanon to support U.S. strategic goals for the country.  Moreover, because State and DOD currently measure program  effectiveness using performance indicators that are not specific,  measurable, and outcome-oriented, program measurement cannot  facilitate evaluations. While State and DOD officials described general  agency efforts to develop better performance indicators, they have not yet  produced plans with specific time frames for completing their efforts.
section_title: Recommendations for Executive Action
paragraphs: To enhance the U.S. government’s ability to determine if security  assistance programs in Lebanon have been effective in achieving their  specific objectives and that they constitute the best mix of security  assistance to support U.S. strategic goals for the country, and to help  State and DOD track progress toward established goals for Lebanon, we  recommend that  1.  the Secretary of State, in consultation with the Secretary of Defense,  complete plans to evaluate the effectiveness of security assistance  programs in Lebanon, including milestone dates for implementing the  plans;  2.  the Secretary of State develop performance indicators for State’s  security assistance programs for Lebanon that are specific,  measurable, and outcome-oriented; and  3.  the Secretary of Defense develop performance indicators for DOD’s  security assistance programs for Lebanon that are specific,  measurable, and outcome-oriented.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to State and DOD for comment. State  and DOD provided written comments which are reprinted in appendixes  IV and V, respectively.  State also provided technical comments, which  we have incorporated into the report, as appropriate. In their comments,  State and DOD generally concurred with the report’s findings and  recommendations.
In its written comments, State said that it recognized that a robust,  coordinated, and targeted evaluation function is essential to its ability to  measure and monitor program performance; make decisions for  programmatic adjustments and changes; document program impact;  identify best practices and lessons learned; help assess return on  investment; provide inputs for policy, planning, and budget decisions; and  assure accountability to the American people.  State said that Lebanon is  a good example of a security assistance program that involves complex  program goals and therefore requires a carefully designed evaluation  framework.  State noted, however, that the qualitative versus quantitative  nature of security assistance makes formal evaluation of such programs a  unique challenge. State said that, short of formal program evaluation  relative to Lebanon, it relies on feedback from other sources, such as  periodic Joint Capabilities Reviews involving the LAF and the U.S.  government; Mission Resource Requests from the U.S. Embassy in  Beirut; and capability assessments from the U.S. Central Command.  We  agree that these reviews may provide useful input but are limited because  they are not linked to specific security assistance programs and do not  substitute for formal evaluations, which State agreed would improve its  ability to determine the effectiveness of U.S. security assistance.
In its written comments, DOD stated that it would coordinate with State to  evaluate the effectiveness of Lebanon’s security forces with clear metrics  that can be evaluated and communicated back to Congress.  DOD also  agreed to improve assessment and evaluation metrics established to  measure the results of assistance provided under Section 1206 and  Section 1207 authority. DOD commented that the draft report did not  consider current DOD evaluating programs that utilize key baseline  documents, such as the Joint Capabilities Review. The draft report,  however, acknowledged that DOD assesses the capabilities of the LAF in  other types of assessments but stated that, while these are useful, they  do not provide the program-specific evaluation that is needed to ensure  that U.S. security assistance is effective in achieving U.S. strategic goals.  DOD's 2012 implementation guidance for assessing Section 1206  programs also recognizes this limitation of other types of assessments.  This guidance states that DOD must be able to demonstrate the return on  investment that Section 1206 training and equipment provide to DOD and  to the U.S. government, and that the assessments are a means for DOD  leadership to determine which types of programs are more successful  and which partner nations make demonstrable progress toward the  objectives of the Section 1206 programs.
DOD also stated that, in the future, it requests that GAO extend it the  courtesy of a formal out brief prior to releasing the draft report to the  Congress. We acknowledge the importance of holding exit meetings with  agency officials. As such, our protocol is to offer an exit meeting with  agency officials after our data collection and analysis are complete. We  requested an exit with DOD on January 28, 2013.  However, due to the  limited availability of the designated DOD policy official with oversight of  U.S. efforts in Lebanon and a subsequent change in the designated  official, we were not able to hold the exit meeting until February 15, 2013.   DOD was provided a draft of the report in advance of the exit meeting.   DOD’s technical comments provided at this meeting were incorporated in  the report, as appropriate.
We are sending copies of this report to the appropriate congressional  committees and the Secretaries of State and Defense. In addition, the   report is available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-7331 or johnsoncm@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix VI.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: We assessed the extent to which the U.S. government, from fiscal years  2007 through 2012, (1) adjusted its strategic goals and security  assistance programs in Lebanon, (2) funded assistance programs for  Lebanese security forces, and (3) evaluated the effectiveness of security  assistance programs in Lebanon.
To assess the extent to which the U.S. government has adjusted its  strategic goals and security assistance programs in Lebanon, we  reviewed documents from the Department of State (State) and the  Department of Defense (DOD) for fiscal years 2007 through 2012,  including Mission Strategic and Resource Plans for Lebanon,  congressional budget justifications, country plans, and other relevant  documents. We also interviewed knowledgeable officials from State and  DOD. At State headquarters in Washington, D.C., we spoke with officials  from the Bureaus of Counterterrorism, Diplomatic Security, International  Narcotics and Law Enforcement Affairs, International Security and  Nonproliferation, and Political-Military Affairs, and from the Office of U.S.  Foreign Assistance Resources. Within DOD, we met with officials from  the Office of the Secretary of Defense and the Defense Security  Cooperation Agency in the Washington, D.C., area, as well as officials of  the U.S. Central Command and U.S. Special Operations Command in  Tampa, Florida. In addition, we met with U.S. officials and officials of the  Lebanese Armed Forces and the Lebanese Internal Security Forces at  the U.S. Embassy in Beirut, Lebanon.
To assess the extent to which the U.S. government funded security  assistance programs for Lebanon Armed Forces and Lebanese Internal  Security Forces from fiscal years 2007 through 2012, we analyzed budget  and funding data from State and DOD. Recognizing that different  agencies and bureaus may use slightly different accounting terms, we  provided State with the definitions from GAO’s A Glossary of Terms Used  in the Federal Budget Process (GAO-05-734SP) and requested that it  provide the relevant data according to those definitions. State provided  data on the status of allocations, obligations, unobligated balances, and  disbursements as of September 30, 2012, for funding accounts that  supported security assistance in Lebanon: International Narcotics Control  and Law Enforcement; International Military Education and Training;  Nonproliferation, Antiterrorism, Demining, and Related Programs; and the  Section 1206 and Section 1207 authorities. State collected the data  directly from each bureau if it was a State-implemented account.  However, because Foreign Military Financing funds are budgeted and  tracked in a different way than other foreign assistance accounts, DOD  provided us with data on allocations and commitments. All data pertain to  overt activities and are nominal numbers that have not been adjusted for  inflation. We also discussed the types and amounts of assistance  provided with various officials of the Lebanese Armed Forces and the  Lebanese Internal Security Forces. To assess the reliability of the data  provided, we requested and reviewed information from officials from each  agency regarding the agency’s underlying financial data system or  systems and the checks, controls, and reviews used to ensure the  accuracy and reliability of the data provided. We determined that the data  provided were sufficiently reliable for the purposes of this report.
To assess the extent to which the U.S. government has evaluated the  effectiveness of its security assistance programs in Lebanon, we  reviewed relevant State and DOD documents, including an independent  evaluation of the Bureau of International Narcotics and Law Enforcement  Affairs’ police training program in Lebanon, DOD’s Section 1206  Assessment Handbook, and various State and DOD program and country  assessments. We also examined State evaluation guidelines, Bureau of  International Narcotics and Law Enforcement Affairs evaluation guidance,  various bureau evaluation plans, and other documents. Furthermore, we  reviewed relevant GAO reports, including those that discussed how  agencies measure program performance. We also reviewed special GAO  publications on performance measurement and evaluation. In addition,  we interviewed officials from State and DOD in Washington, D.C., and at  the U.S. Embassy in Beirut, Lebanon, and DOD officials at the U.S.  Central Command and U.S. Special Operations Command in Tampa,  Florida.
We conducted this performance audit from June 2012 to March 2013 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Status of Funds for U.S. Security Assistance Programs for Lebanon
paragraphs: This appendix provides additional information on Department of State  (State) and Department of Defense (DOD) programs used to provide  security assistance to Lebanon from fiscal years 2007 through 2012.  State and DOD utilized eight programs to provide more than $925 million  in security assistance to Lebanon from fiscal years 2007 through 2012.  The majority of security assistance (95 percent) was provided through  three programs: Foreign Military Financing, International Narcotics  Control and Law Enforcement (INCLE), and the Section 1206 authority.  Specifically, this appendix provides additional information on the Foreign  Military Financing, INCLE, and the Section 1206 authority such as  selected equipment and training provided and the status of funds  including allocation, obligation, and commitment or disbursement of funds  for the other five programs.
subsections: section_title: Foreign Military Financing Background
subsections: section_title: State administers Foreign Military Financing funding, with significant input from embassies, DOD, and the regional military commands. DOD’s Defense Security Cooperation Agency executes the program. Foreign Military Financing provides grants and loans to foreign governments and international organizations for the acquisition of U.S. defense equipment, services, and training.
paragraphs: From fiscal years 2007 through 2012, State allocated a total of about $641  million in Foreign Military Financing for Lebanon. DOD reported that for fiscal  years 2007 through 2012, the majority of allocated funds had been committed  (see fig. 5). For Lebanon, Foreign Military Financing has provided basic  equipment such as tactical radios, ammunition, rifles, helmets, body armor,  and trucks. Lebanon has also received night vision goggles, missiles, and  helicopters.
section_title: INCLE Background
subsections: section_title: State administers the INCLE program and uses it to assist the Government of Lebanon in developing its law enforcement and judicial capacity. These funds are generally available for obligation for 2 years. Once the period of availability for new obligations expires, the funds are available for an additional 5 years to liquidate the obligations. The period of availability for obligations of INCLE funds may be extended for an additional 4 years in certain cases.
paragraphs: From fiscal years 2007 through 2012, State allocated a total of $130 million in  INCLE funds for Lebanon. While most of the allocated funds have been  obligated, about 40 percent of total allocated funds from fiscal years 2007  through 2012 have been disbursed (see fig. 6). According to State, fiscal year  2007 marked the first allocation of INCLE funds for Lebanon. In Lebanon, the  INCLE program has funded the construction of a police station building and  the purchase of riot helmets and batons, pistols, and police vehicles, among  other equipment. In addition, the INCLE program has funded training and  technical assistance to the Lebanese Internal Security Forces, including, for  example, courses in community policing.
section_title: Section 1206 Background
subsections: section_title: The National Defense Authorization Act for Fiscal Year 2006, as amended, established the authority for Section 1206 programs to train and equip foreign national military and nonmilitary maritime forces, such as coast guards, to conduct counterterrorist operations or to support military and stability operations in which the U.S. armed forces are a participant.
paragraphs: From fiscal years 2007 through 2012, DOD allocated a total of $111 million  through the Section 1206 program for Lebanon (see fig. 7). State reported  that all allocated funds from fiscal years 2007 through 2012 have been  obligated and disbursed.
For Lebanon, Section 1206 funding has provided vehicle spare parts,  ammunition, and other basic supplies to the Lebanese Armed Forces (LAF).  In particular, equipment provided under Section 1206 was used to restock the  LAF arsenal with basic ammunition after the 2007 siege at Nahr al-Bared  Palestinian refugee camp and to begin to build the LAF’s first secure  communications system.
State and DOD jointly administer  Section 1206 assistance. Funded  from the DOD operations and  maintenance accounts, Section  1206 funds remain available for 1  year for obligation. Once the period  of availability for new obligations  expires, the funds are available for  an additional 5 years to liquidate  the obligations.
In addition to the Foreign Military  Financing, INCLE, and Section  1206 programs, State and DOD  provided security assistance to  Lebanon through five other  programs: Antiterrorism  Assistance, International Military  Education and Training, Section  1207 Train and Equip, Export  Control and Related Border  Security, and Counterterrorism  Financing. See tables 2 through 6  for details on allocation,  obligation, and disbursement of  funds by these programs for  security assistance for Lebanon  from fiscal years 2007 through  2012.
Table 7 presents selected security equipment and services provided by  the United States to the Lebanese Armed Forces (LAF) through the  Foreign Military Financing and Section 1206 programs from fiscal years  2007 through 2012. According to data provided by the Defense Security  Cooperation Agency, 78 cases were approved for Lebanon during this  time frame and ranged in value from $18,000 to $38.2 million. The  defense articles or services are listed in descending order based on the  total estimated case value as reported by the Defense Security  Cooperation Agency. Table 8 presents selected security equipment or  funding provided by countries other than the United States to the LAF, as  reported by the LAF. Countries are listed in descending order based on  the estimated value of security assistance provided.
section_title: Appendix IV: Comments from the U.S. Department of State
section_title: Appendix V: Comments from the Department of Defense
section_title: Appendix VI: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Jeff Phillips (Assistant Director);  Jenna Beveridge; Teakoe S. Coleman; David Dayton; and La Verne  Tharpes made key contributions to this report. Martin de Alteriis, Grace  Lui, and Jeremy Sebest provided additional technical assistance.