Security Assistance: Evaluations Needed to Determine Effectiveness of U.S. Aid to Lebanon's Security Forces
Released: 2013-03-19T00:00:00

Letter

Background

Lebanon is a small, religiously diverse country on the Mediterranean Sea that borders Israel and Syria. (See fig. 1.)

Religious tensions among Lebanon’s Maronite Christians, Sunni Muslims, and Shiite Muslims, among others, along with an influx of Palestinian refugees into Lebanon, have fueled Lebanon’s internal strife and conflicts with its neighbors. During the civil war between 1975 and 1990, both Syrian and Israeli forces occupied the country. In the midst of the civil war and Israel’s continued occupation of southern Lebanon, Hezbollah emerged in Lebanon as a powerful Islamic militant group. Throughout the 1990s, Hezbollah, funded by Iran and designated by the United States and Israel as a terrorist organization, pursued its military campaign against Israeli forces occupying Lebanon while also participating in Lebanon’s political system. In 2000, Israeli forces withdrew from southern Lebanon. In 2005, with pressure from the international community, Syrian forces withdrew from Lebanon following the assassination of Lebanon’s prime minister. The subsequent parliamentary elections that year resulted in a member of Hezbollah holding a cabinet position for the first time. In the summer of 2006, Hezbollah and Israel entered into a month-long conflict that ended with the adoption of United Nations Security Council Resolution 1701 by both the Israeli and Lebanese governments. The resolution called for Israeli withdrawal from southern Lebanon in parallel with the deployment of Lebanese and United Nations forces and the disarmament of all armed groups in Lebanon, among other things. Since 2011, instability in neighboring Syria may have exacerbated sectarian conflict within Lebanon. See figure 2 for a timeline of selected political events in Lebanon.

The United States has provided Lebanon with assistance, such as emergency humanitarian aid during the civil war and training for military forces under the International Military Education and Training (IMET) program. The United States and Lebanon have historically enjoyed a good relationship in part because of cultural and religious ties, a large Lebanese-American community in the United States, and the pro- Western orientation of Lebanon, particularly during the Cold War. Following the Syrian withdrawal in 2005 and the 2006 Israeli-Hezbollah war, the United States increased its security assistance to Lebanon. The United States provided assistance to the LAF, which is generally responsible for providing border security, counterterrorism, and national defense, and to the ISF, or police force, which is generally responsible for maintaining law and order in Lebanon. The fiscal year 2006 appropriations marked the first time since 1984 that U.S. agencies allocated Foreign Military Financing grants to help modernize and equip the LAF. Since then, the United States has provided security assistance to Lebanon through the Foreign Military Financing program; IMET; International Narcotics Control and Law Enforcement (INCLE) program; the Nonproliferation, Antiterrorism, Demining, and Related (NADR) programs; and Section 1206 and Section 1207 authorities for training and equipping foreign militaries and security forces and for reconstruction, stabilization, and security activities in foreign countries, respectively. The NADR security assistance for Lebanon has been provided through three programs: Antiterrorism Assistance, Counterterrorism Financing, and Export Control and Related Border Security (EXBS). Table 1 describes these security assistance programs. In addition, to these security assistance programs, U.S. Special Forces units have provided specialized training to LAF Special Forces units, according to agency officials.

The United States Has Kept Strategic Goals for Lebanon Constant Since 2007 and Adjusted Security Assistance in Response to Political and Security Conditions

The United States has kept strategic goals for Lebanon constant since 2007. These goals include supporting the Government of Lebanon in establishing stability and security against internal threats from militant extremists and countering destabilizing influences. U.S. agencies have adjusted security assistance in response to Lebanon’s political and security conditions. For example, State and DOD have delayed releasing funds and limited the types of equipment provided. Both agencies have also implemented additional assistance programs since fiscal year 2007.

U.S. Strategic Goals Are to Support a Stable, Secure, and Independent Lebanese Government

Since 2007, U.S strategic goals for Lebanon have been to support the nation as a stable, secure, and independent democracy. According to DOD and State officials, the overarching priorities for Lebanon remain focused on supporting Lebanese sovereignty and stability and countering the influence of Syria and Iran. Security-related goals for Lebanon focus on counterterrorism and regional stability or internal security. Programs activities seek to support development of the LAF and the ISF as the only legitimate providers of Lebanon’s security. The goals and objectives of the individual security assistance programs are intended to support the U.S. strategic goals and overarching priorities, as the following examples illustrate:

Goals for the Foreign Military Financing, IMET, and 1206 programs in Lebanon since the departure of Syrian forces are to bolster the capability of the LAF, nurture the bilateral military relationship between the United States and the LAF, and continue encouraging establishment of a stable, legal, and pro-U.S. civil government.

Goals for the INCLE police training program are (1) to build Lebanon’s operational capacity to combat crime, and prevent and respond to terror attacks; and (2) to assist Lebanon in developing the ISF into a competent, professional, and democratic police force with the necessary training, equipment, and institutional capacity to enforce the rule of law in Lebanon, cement sovereign Lebanese government control over its territory, and protect the Lebanese people.

Goals for the Antiterrorism Assistance program in Lebanon are to develop and build the Lebanese government’s capacities in border security, mid- and senior-level leadership development, and counterterrorism investigations.

Goals for the Counterterrorism Financing Program are to deny terrorists access to money, resources, and support.

Goals for EXBS in Lebanon focus on strengthening the capability of Lebanese enforcement agencies to effectively control cross-border trade in strategic goods.

Goals for the 1207 program were to strengthen Lebanon’s internal security forces after armed conflicts in 2006 and 2007.

State and DOD Have Adjusted Implementation of Security Assistance Programs to Meet Conditions on the Ground

The goals and objectives of U.S. security assistance to Lebanon have continued to focus on supporting Lebanese sovereignty and stability and countering the influences of Syria and Iran. However, according to State and DOD officials, the agencies have changed how they implement programs based on changes in the political and security situation, for example, by delaying the release of funds or limiting the types of equipment provided. In one instance, State delayed committing fiscal year 2010 Foreign Military Financing funds as a result of an incident on August 3, 2010, in which the LAF opened fire on an Israeli Defense Force unit engaged in routine maintenance along the Blue Line, alleging that it had crossed into Lebanese territory. Two Lebanese soldiers, a journalist, and an Israeli officer were killed. According to State, in response to concerns raised by a member of Congress, State delayed committing $100 million of fiscal year 2010 Foreign Military Financing funds for Lebanon, citing the need to determine whether equipment that the United States provided to the LAF was used against Israel. According to a State official, State committed the funds in November 2010 after consulting with the member of Congress.

In addition, according to State officials, State and DOD decided to place a temporary hold on lethal assistance to the LAF in January 2011, after the collapse of Prime Minister Saad Hariri’s government. In March 2012, the agencies decided to lift the hold on lethal assistance based on the Mikati government’s adherence to key international obligations to the Special Tribunal for Lebanon and United Nations Security Council Resolution.

The U.S. government has also implemented new security assistance programs in Lebanon since fiscal year 2007 in response to changing security assistance needs, according to State officials. For example, State began implementing the EXBS program in Lebanon in fiscal year 2009— after State performed an assessment to identify deficiencies in the country’s ability to detect and interdict weapons of mass destruction and advanced conventional weapons, according to officials of State’s Bureau of International Security and Nonproliferation. State determined that Lebanon did not have a comprehensive system to regulate trade in strategic goods and technologies for the purpose of preventing the proliferation of such weapons, and lacked the necessary legal and institutional elements to manage strategic trade consistent with international standards. In addition, in fiscal year 2010, State’s Bureau of Counterterrorism began implementing its Counterterrorism Financing program in Lebanon. According to bureau officials, the goals of the program are to build the foreign capacity of the Government of Lebanon to develop laws and regulations to deny terrorists access to funds.

State has also changed the focus of some of the security assistance programs. For example, the INCLE police training program is no longer providing basic training to cadets, according to the Bureau of International Narcotics and Law Enforcement Affairs. The bureau turned the basic training over to the ISF as of July 2012 and is providing leadership and management training. The bureau is working with the Federal Bureau of Investigation and the Drug Enforcement Administration to develop specialized training to increase the ISF’s investigatory skills, according to U.S. embassy officials. The bureau is also expanding its community policing program, according to bureau officials. In addition, State’s Bureau for Counterterrorism has changed the focus of its antiterrorism training, according to officials of State’s Bureau of Diplomatic Security whom we interviewed in Lebanon. They explained that when the program began in 2007, it focused on training Lebanese security details to protect national leaders. The trained protection details have demonstrated a mastery of the skills for which they were trained. As a result, the Bureau for Counterterrorism and the Bureau of Diplomatic Security consider the protection mission to be complete and since 2010 has shifted its focus to border security, management and leadership, and counterterrorism investigations, according to the Diplomatic Security officials. Finally, while there have been no major changes to the EXBS program in Lebanon, U.S. embassy officials stated that the United States has increased the program’s emphasis on convincing the government to change its export control laws.

U.S. Agencies Allocated Over $925 Million in Security Assistance for Lebanon during Fiscal Years 2007-2012 and Have Disbursed or Committed Most of the Funds

U.S. agencies have allocated over $925 million in security assistance for Lebanon from fiscal years 2007 through 2012, with funds varying by year and program. The majority of funds—69 percent—was from the Foreign Military Financing program, though State and DOD also utilized seven other programs. State has disbursed the majority of funds it allocated for each fiscal year, while DOD has committed the majority of funds for Foreign Military Financing.

Security Assistance Allocated for Lebanon Varied by Year

In order to help achieve U.S. strategic goals, State and DOD have allocated more than $925 million in security assistance for Lebanon’s LAF and ISF from fiscal years 2007 through 2012. This funding peaked in fiscal year 2007 at about $323 million, declined in fiscal year 2008 to about $32 million, and has fluctuated from fiscal years 2009 through 2012. (See fig. 3.)

In fiscal year 2007, the $325 million allocated for security assistance for Lebanon followed the 2006 war between Hezbollah and Israel and assisted Lebanon in fulfilling its obligations under the United Nations Security Council Resolution 1701. This funding level represented a significant increase over fiscal year 2006, when U.S. security assistance for Lebanon totaled only about $28 million. See appendix II for information on allocation, obligation, and disbursement or commitment of security assistance for Lebanon by program and year.

U.S. Agencies Allocated the Majority of Security Assistance through the Foreign Military Financing Program and Used Seven Other Programs

U.S. agencies allocated the majority (approximately 69 percent, or $641 million) of security assistance for Lebanon from fiscal years 2007 through 2012 through the Foreign Military Financing program, which provides grants and loans to the Lebanese Armed Forces to purchase U.S. equipment, services, and training. Figure 4 shows the distribution of U.S. security assistance allocated for Lebanon by program.

The Foreign Military Financing program funded the purchase of various equipment and services. For example, Lebanon received trucks, truck tractors, trailers, ambulances, cargo-troop carriers, and armament vehicles through the Foreign Military Financing program. Other equipment purchased through the Foreign Military Financing program included helicopters, ships, radios, and spare parts. Appendix III presents selected equipment and services provided to Lebanon by the U.S. government and by other governments from fiscal years 2007 through 2012.

U.S. Agencies Disbursed or Committed the Majority of Funds

Generally, State disbursed the majority of funds allocated during the same fiscal year for security assistance programs providing funds for Lebanon, and DOD committed the majority of Foreign Military Financing funds for Lebanon. See appendix II for additional information on allocation, obligation, and disbursement or commitment of security assistance for Lebanon by program and year. Specifically, State disbursed about 78 percent of total allocations from fiscal years 2007 through 2012 for the IMET, INCLE, Antiterrorism Assistance, Counterterrorism Financing, and EXBS programs and the Section 1206 and 1207 authorities. Although State has disbursed the majority of these allocated funds, State has not disbursed some funds allocated for the INCLE and Antiterrorism Assistance programs for fiscal years 2007 through 2011. According to State officials, some obligated funds are waiting to be disbursed and all of the unobligated funds are no longer available for obligation. DOD has committed 87 percent of Foreign Military Financing funds for Lebanon for fiscal years 2007 through 2012.

U.S. Agencies Have Done Little Evaluation of Security Assistance Programs in Lebanon

As of February 2013, U.S. agencies had evaluated only the INCLE police training program for Lebanon and did not have firm plans to evaluate the other six ongoing security assistance programs. State’s evaluation policy requires periodic evaluations of certain programs, consistent with standards established in U.S. law. Evaluations can be facilitated by collecting data on appropriate performance indicators. However, we and other audit agencies have previously reported deficiencies in how State and DOD measure program performance. For example, we found in 2011 that the IMET program evaluation efforts had few of the elements commonly accepted as appropriate for measuring progress and did not objectively measure how IMET contributes to long-term desired outcomes. We have also reported deficiencies in how DOD defined performance measures for the 1206 program. In response to those reports, State and DOD concurred with our recommendations and described agency efforts to develop better performance measures.

State Policy Requires Periodic Evaluations of Certain Programs

State’s evaluation policy, which is partly based on the Government Performance and Results Modernization Act of 2010, requires that all large programs, projects, and activities be evaluated at least once in their lifetime or every 5 years, whichever is less. According to State, the 2010 act strengthened the mandate to evaluate programs, requiring agencies to include a discussion of evaluations in their strategic plans and performance reports. State established its evaluation policy in February 2012 in part to comply with the requirements of this act. In addition, according to State, the policy supports State’s goal of connecting evaluation to its investments in diplomacy and development to ensure that they align with the agency’s overarching strategic goals and objectives. State’s evaluation guidance requires each bureau to evaluate two to four projects, programs, or activities over a 24-month period beginning with fiscal year 2012, depending on the size, scope, and complexity of the programs being evaluated and the availability of funding. State’s evaluation policy also requires all bureaus to complete a bureau evaluation plan and to update it annually. State requires the bureaus’ plans to align with evaluation policy guidance and to assist each bureau in assessing the extent to which its efforts contribute to achieving its intermediate objectives and, by extension, its longer-term goals.

Program Evaluations Can Be Facilitated by Performance Measures

Program evaluations are individual systematic studies conducted periodically or on an ad hoc basis to assess how well a program is working. As a key component of effective program management, evaluations assess how well a program is working and help managers make informed decisions about current and future programming. An evaluation provides an overall assessment of whether a program works and identifies adjustments that may improve its results. Depending on their focus, evaluations may examine aspects of program operations (such as in a process evaluation) or factors in the program environment that may impede or contribute to its success. Program evaluations may systematically compare the effectiveness of alternative programs aimed at the same objective. Types of evaluation include process (or implementation), outcome, and impact evaluations, as well as cost-benefit and cost-effectiveness analyses.

Performance measurement, by contrast, is the ongoing monitoring and reporting of program accomplishments, particularly progress toward pre- established goals. Performance measurement is typically conducted by program or agency officials and focuses on whether a program has achieved its objectives, expressed as measurable performance indicators. As indicators of progress toward goals, performance measures can inform overall program evaluations. Program evaluations typically examine a broader range of information on program performance and its context than is feasible to monitor on an ongoing basis. Both forms of assessment—program evaluation and performance measurement—aim to support resource allocation and other policy decisions to improve service delivery and program effectiveness. But performance measurement, because of its ongoing nature, can serve as an early warning system to management and as a vehicle for improving accountability to the public.

State Has Evaluated One Security Assistance Program in Lebanon

As of February 2013, U.S. agencies had evaluated only one of their security assistance programs in Lebanon. The Bureau of International Narcotics and Law Enforcement Affairs contracted with an external organization to conduct an evaluation of the INCLE police training program for the ISF. The evaluation was conducted between November 2010 and May 2011. The bureau commissioned the evaluation to establish the relevance, effectiveness, impact, and sustainability of the program from 2008 through 2010. The evaluation covered five training courses, and the evaluation team collected data by means of focus groups, semistructured interviews, structured observation, learning tests, field cluster research, and document review.

The INCLE police training evaluation demonstrated the importance of conducting a program evaluation. According to the evaluation, the training was effective, with clear evidence that the ISF trainees had both learned and been able to retain the core knowledge taught in all five courses. The program also succeeded in reaching ISF personnel from throughout the country. The evaluation further determined that the training program made a relevant contribution to international donor assistance to the ISF. However, the evaluation concluded that the program did not reach its principal stated objective in the program period. The evaluation found no evidence to suggest that the performance of the ISF had systematically improved as a result of the training program. The evaluation also concluded that, although trainees learned and retained knowledge successfully, they were generally not able to apply the skills taught by the program in their daily duties. The evaluation identified the failure to apply skills as the key impediment to program effectiveness and attributed the failure to a flaw in the program design, which failed to sufficiently engage the ISF in the inception stage to clarify objectives and secure broad consensus on program goals. Moreover, the design of the training was not informed by a systematic assessment of the ISF’s training needs.

The final evaluation report included five recommendations to the Bureau of International Narcotics and Law Enforcement Affairs, including a joint recommendation with the ISF, and two additional recommendations for the ISF. The recommendations included a number of subcomponents. The report recommended that the bureau take the following actions: continue the training based on renewed program consensus; closely coordinate program transition with ISF on the operational level; expand senior leadership training; strengthen internal capacity for program design and management; and in addition to quantitative performance measures, design meaningful qualitative performance indicators in cooperation with the ISF to provide relevant and timely progress information throughout the program period.

The evaluation report also outlined steps to transition the training program to full ISF control. Bureau officials stated that the bureau has adopted many of the report’s recommendations, including the transition of one course to full ISF control in July 2012, the creation of a training coordination process for INL and other donors to maximize synergies and avoid duplication of effort, and the creation of a strategic planning cell within the ISF to coordinate directly with the bureau and other donors.

DOD Has Not Evaluated the Effectiveness of the Section 1206 Program in Lebanon

Although the Section 1206 program in Lebanon has not been evaluated, DOD—under the direction of the Under Secretary of Defense for Policy— is implementing a new assessment process for the Section 1206 program. The objectives of the assessment process are to measure implementation of Section 1206 programs to build partner assess quality and timeliness of program implementation, measure the impact of programs, and estimate return on investment.

DOD is implementing this new process in part to respond to our 2010 recommendation that DOD develop and implement specific plans to monitor, evaluate, and report routinely on Section 1206 project outcomes and their impact on U.S. strategic objectives. DOD conducted a pilot test of its assessment process of counterterrorism-oriented Section 1206 programs in the Philippines and stability operations-oriented Section 1206 programs in Georgia in 2012. DOD also assessed Section 1206 programs in Djibouti, Tunisia, and Poland from March through June 2012. DOD officials stated in December 2012 that they were not able to assess the program in Lebanon because of the security situation, but they plan to include Lebanon as soon as the security situation permits.

Although DOD has not evaluated the effectiveness of the 1206 program or any other security assistance programs in Lebanon in which it participates, it conducts various types of assessments of the LAF. For example, since 2010, the U.S. Central Command and the LAF have participated in reviews—known as joint capabilities reviews—to assess the progress of the LAF based on eight broad critical capabilities. The critical capabilities—for example, Land and Border Defense and Security—are linked to the broad strategic goals described in State’s Mission Strategic and Resource Plans for Lebanon (now Mission Resource Requests), according to DOD Central Command officials. The joint capabilities reviews include milestone dates from 2013 through 2015. The U.S. Central Command also conducts annual assessments linked to the Theater Campaign Plan for Lebanon. These assessments grade the LAF on desired outcomes and make recommendations for course corrections. According to the DOD Central Command officials, the assessments are organized by lines of effort, such as defeating violent extremist organizations and building partner capacity. These lines of effort are not linked to specific security assistance programs, however.

State Has Not Completed Plans or Established Time Frames to Evaluate the Effectiveness of the Remaining Security Assistance Programs in Lebanon

Although bureau evaluation plans were due in May 2012, State bureaus responsible for security assistance programs were at varying stages of implementing State’s evaluation policy at the time of this report. For example, State’s Bureau of Political-Military Affairs, which is responsible for the Foreign Military Financing and IMET programs, created an evaluation plan for fiscal year 2013 which does not include an evaluation of the programs in Lebanon, according to a bureau official.

State’s Bureau of Counterterrorism, which manages the Counterterrorism Financing program and—along with the Bureau of Diplomatic Security—is responsible for the Antiterrorism Assistance program, created an evaluation plan for fiscal year 2013, according to a Bureau of Counterterrorism official. This plan does not include an evaluation of the programs in Lebanon. The Bureau of Counterterrorism will evaluate the Antiterrorism Assistance programs for two other partner nations, Bangladesh and Morocco. According to a Bureau of Counterterrorism official, the bureau will consider the Antiterrorism Assistance Program in Lebanon for the next tranche of evaluations beginning in fiscal year 2014.

State’s Bureau of International Security and Nonproliferation, which implements the EXBS program, submitted its bureau evaluation plan on April 10, 2012. The plan, which covers fiscal years 2012 through 2015, does not include an evaluation of the EXBS program. The bureau does periodically assess the legal, regulatory, and institutional components of a country’s strategic trade control systems using a 419-point assessment methodology. The bureau conducted such an assessment for Lebanon in 2010, according to bureau officials. However, according to the bureau’s evaluation plan, the assessments do not assess enforcement capacity to the extent preferred and may not fully satisfy the intent of State’s evaluation policy. The bureau is working with the Office of U.S. Foreign Assistance Resources to determine how to integrate the assessment methodology into program evaluations that would meet State’s evaluation policy. The bureau may evaluate EXBS within the fiscal years 2013 to 2016 time frame.

Although State bureaus are at varying stages of developing their evaluation plans, State has developed department-wide guidance for standardizing the way it measures the effectiveness of its programs and has entered into contracts with five contractors for monitoring and evaluation services, according to State officials. State bureaus can use these contracts to evaluate their programs.

Past Deficiencies Reported in Agencies’ Performance Measurement May Inhibit Program Evaluation

We have reported previous deficiencies in performance measurement of State’s and DOD’s security assistance programs, and these deficiencies may inhibit agencies’ ability to conduct program evaluations. As we have previously reported, performance measurement of State and DOD security assistance programs exhibited several deficiencies, including a lack of specific, measurable, and outcome-oriented performance indicators.

In 2008, we reported that State did not systematically assess the outcomes of the Antiterrorism Assistance program and, as a result, could not determine the effectiveness of this program. More recently, a 2012 State Office of Inspector General assessment of the antiterrorism programs in certain countries reported that the Bureau of Diplomatic Security’s Office of Antiterrorism Assistance could not determine the Antiterrorism Assistance program’s effectiveness in part because they had not developed specific, measurable, and outcome- oriented program objectives.

In 2011, we reported that State’s and DOD’s ability to assess IMET’s effectiveness was limited by several weaknesses in program monitoring and evaluation, including the lack of a performance plan for IMET that explained how the program was expected to achieve its goals and how progress could be assessed through performance measures and targets.

In 2010, we reported that DOD and State had incorporated little monitoring and evaluation into the Section 1206 program and had not consistently defined performance measures for Section 1206 projects.

Although Facing Challenges, State and DOD Are Continuing Efforts to Improve Performance Measurement

According to State and DOD officials, assessing the impacts of security assistance programs is challenging, but they continue to seek improvements in performance measurement. For example, officials of the Bureaus of Counterterrorism and Diplomatic Security stated that it is an enormous task to come up with concrete, meaningful performance indicators for the Antiterrorism Assistance program, which addresses aviation security, investigations, and response techniques. In addition, officials at the U.S. Embassy in Beirut stated that State and DOD are struggling to develop specific, measurable performance indicators and still have a long way to go. DOD officials stated that developing metrics to measure the results of security assistance programs in Lebanon is difficult. The officials stated that some indicators may be subjective and difficult to quantify—for example, U.S. influence in building relationships or the willingness of the LAF to fight. Furthermore, they said that it is difficult to quantify what would have happened in the absence of U.S. assistance.

Because the agencies do not have specific, measurable, and outcome- oriented performance indicators, U.S. officials have cited anecdotal evidence that they believe demonstrates the effectiveness of U.S. security assistance programs in Lebanon. For example, some officials have cited the LAF’s success in taking control of a Palestinian refugee camp from Al-Qaeda-inspired militants as an indication of the impact of the assistance. The United States provided ammunition and other supplies to assist the LAF during the 3-month engagement. As other indications of effectiveness, U.S. officials also mentioned the LAF’s actions in arresting members of a Shiite clan involved in kidnappings, the professionalism demonstrated by LAF Special Forces units in training exercises, and increases in drug seizures. However, the use of only positive anecdotal evidence does not provide sufficient context and scope to help officials judge the effectiveness of these programs.

State and DOD officials described agency efforts to improve performance measurement. For example, the Bureau of International Narcotics and Law Enforcement Affairs is shifting from reporting output measures, such as numbers of students trained, to outcome measures, according to bureau officials. The bureau plans to collect more accurate performance metrics that focus on measuring the impact of the INCLE police training program in Lebanon. The bureau also plans to consider such factors as the quality of the officers and the extent to which women are integrated into the ISF, according to bureau officials. Moreover, the Bureau of International Narcotics and Law Enforcement Affairs is working with the ISF to improve the performance indicators in the ISF’s strategic plan, according to officials at the U.S. Embassy in Beirut. The current plan includes some output measures, such as number of traffic stops made by police. Embassy officials also stated that they are working with the LAF to develop a military cooperation plan that will include performance measures. In addition, according to DOD Central Command officials, efforts are under way to develop a new framework for its annual assessments of the LAF that will include improved performance measures. The new framework will examine the effects of DOD efforts in Lebanon.

Conclusions

Since 2007, U.S. agencies have allocated almost $1 billion in security assistance for Lebanon, supporting its efforts to build a stable, secure, and independent democracy following the withdrawal of Syrian forces and the Israeli-Hezbollah war. U.S. security assistance for Lebanon has presented certain risks because of the influence wielded by the militant group Hezbollah, which is now a member of the Lebanese government. Ensuring that U.S. security assistance for Lebanon is effective in achieving U.S. strategic goals is now more important than ever—to help Lebanon resist the influence of Iran, which funds Hezbollah, and to withstand the potential spillover of conflict in Syria.

Program evaluation and performance measurement are key management tools to ensure that U.S. security assistance is effective in achieving U.S. strategic goals. However, the U.S government has evaluated the effectiveness of only one security assistance program in Lebanon, a program to which it allocated about 14 percent of the nearly $1 billion in security assistance allocations since 2007. Neither State nor DOD has completed plans or time frames to evaluate the remaining six ongoing U.S. security assistance programs in Lebanon. Without evaluations of all of its security assistance programs in Lebanon, the U.S. government cannot show that the programs have been effective in achieving their specific objectives or that they constitute the best mix of security assistance for Lebanon to support U.S. strategic goals for the country. Moreover, because State and DOD currently measure program effectiveness using performance indicators that are not specific, measurable, and outcome-oriented, program measurement cannot facilitate evaluations. While State and DOD officials described general agency efforts to develop better performance indicators, they have not yet produced plans with specific time frames for completing their efforts.

Recommendations for Executive Action

To enhance the U.S. government’s ability to determine if security assistance programs in Lebanon have been effective in achieving their specific objectives and that they constitute the best mix of security assistance to support U.S. strategic goals for the country, and to help State and DOD track progress toward established goals for Lebanon, we recommend that 1. the Secretary of State, in consultation with the Secretary of Defense, complete plans to evaluate the effectiveness of security assistance programs in Lebanon, including milestone dates for implementing the plans; 2. the Secretary of State develop performance indicators for State’s security assistance programs for Lebanon that are specific, measurable, and outcome-oriented; and 3. the Secretary of Defense develop performance indicators for DOD’s security assistance programs for Lebanon that are specific, measurable, and outcome-oriented.

Agency Comments and Our Evaluation

We provided a draft of this report to State and DOD for comment. State and DOD provided written comments which are reprinted in appendixes IV and V, respectively. State also provided technical comments, which we have incorporated into the report, as appropriate. In their comments, State and DOD generally concurred with the report’s findings and recommendations.

In its written comments, State said that it recognized that a robust, coordinated, and targeted evaluation function is essential to its ability to measure and monitor program performance; make decisions for programmatic adjustments and changes; document program impact; identify best practices and lessons learned; help assess return on investment; provide inputs for policy, planning, and budget decisions; and assure accountability to the American people. State said that Lebanon is a good example of a security assistance program that involves complex program goals and therefore requires a carefully designed evaluation framework. State noted, however, that the qualitative versus quantitative nature of security assistance makes formal evaluation of such programs a unique challenge. State said that, short of formal program evaluation relative to Lebanon, it relies on feedback from other sources, such as periodic Joint Capabilities Reviews involving the LAF and the U.S. government; Mission Resource Requests from the U.S. Embassy in Beirut; and capability assessments from the U.S. Central Command. We agree that these reviews may provide useful input but are limited because they are not linked to specific security assistance programs and do not substitute for formal evaluations, which State agreed would improve its ability to determine the effectiveness of U.S. security assistance.

In its written comments, DOD stated that it would coordinate with State to evaluate the effectiveness of Lebanon’s security forces with clear metrics that can be evaluated and communicated back to Congress. DOD also agreed to improve assessment and evaluation metrics established to measure the results of assistance provided under Section 1206 and Section 1207 authority. DOD commented that the draft report did not consider current DOD evaluating programs that utilize key baseline documents, such as the Joint Capabilities Review. The draft report, however, acknowledged that DOD assesses the capabilities of the LAF in other types of assessments but stated that, while these are useful, they do not provide the program-specific evaluation that is needed to ensure that U.S. security assistance is effective in achieving U.S. strategic goals. DOD's 2012 implementation guidance for assessing Section 1206 programs also recognizes this limitation of other types of assessments. This guidance states that DOD must be able to demonstrate the return on investment that Section 1206 training and equipment provide to DOD and to the U.S. government, and that the assessments are a means for DOD leadership to determine which types of programs are more successful and which partner nations make demonstrable progress toward the objectives of the Section 1206 programs.

DOD also stated that, in the future, it requests that GAO extend it the courtesy of a formal out brief prior to releasing the draft report to the Congress. We acknowledge the importance of holding exit meetings with agency officials. As such, our protocol is to offer an exit meeting with agency officials after our data collection and analysis are complete. We requested an exit with DOD on January 28, 2013. However, due to the limited availability of the designated DOD policy official with oversight of U.S. efforts in Lebanon and a subsequent change in the designated official, we were not able to hold the exit meeting until February 15, 2013. DOD was provided a draft of the report in advance of the exit meeting. DOD’s technical comments provided at this meeting were incorporated in the report, as appropriate.

We are sending copies of this report to the appropriate congressional committees and the Secretaries of State and Defense. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-7331 or johnsoncm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VI.

Appendix I: Objectives, Scope, and Methodology

We assessed the extent to which the U.S. government, from fiscal years 2007 through 2012, (1) adjusted its strategic goals and security assistance programs in Lebanon, (2) funded assistance programs for Lebanese security forces, and (3) evaluated the effectiveness of security assistance programs in Lebanon.

To assess the extent to which the U.S. government has adjusted its strategic goals and security assistance programs in Lebanon, we reviewed documents from the Department of State (State) and the Department of Defense (DOD) for fiscal years 2007 through 2012, including Mission Strategic and Resource Plans for Lebanon, congressional budget justifications, country plans, and other relevant documents. We also interviewed knowledgeable officials from State and DOD. At State headquarters in Washington, D.C., we spoke with officials from the Bureaus of Counterterrorism, Diplomatic Security, International Narcotics and Law Enforcement Affairs, International Security and Nonproliferation, and Political-Military Affairs, and from the Office of U.S. Foreign Assistance Resources. Within DOD, we met with officials from the Office of the Secretary of Defense and the Defense Security Cooperation Agency in the Washington, D.C., area, as well as officials of the U.S. Central Command and U.S. Special Operations Command in Tampa, Florida. In addition, we met with U.S. officials and officials of the Lebanese Armed Forces and the Lebanese Internal Security Forces at the U.S. Embassy in Beirut, Lebanon.

To assess the extent to which the U.S. government funded security assistance programs for Lebanon Armed Forces and Lebanese Internal Security Forces from fiscal years 2007 through 2012, we analyzed budget and funding data from State and DOD. Recognizing that different agencies and bureaus may use slightly different accounting terms, we provided State with the definitions from GAO’s A Glossary of Terms Used in the Federal Budget Process (GAO-05-734SP) and requested that it provide the relevant data according to those definitions. State provided data on the status of allocations, obligations, unobligated balances, and disbursements as of September 30, 2012, for funding accounts that supported security assistance in Lebanon: International Narcotics Control and Law Enforcement; International Military Education and Training; Nonproliferation, Antiterrorism, Demining, and Related Programs; and the Section 1206 and Section 1207 authorities. State collected the data directly from each bureau if it was a State-implemented account. However, because Foreign Military Financing funds are budgeted and tracked in a different way than other foreign assistance accounts, DOD provided us with data on allocations and commitments. All data pertain to overt activities and are nominal numbers that have not been adjusted for inflation. We also discussed the types and amounts of assistance provided with various officials of the Lebanese Armed Forces and the Lebanese Internal Security Forces. To assess the reliability of the data provided, we requested and reviewed information from officials from each agency regarding the agency’s underlying financial data system or systems and the checks, controls, and reviews used to ensure the accuracy and reliability of the data provided. We determined that the data provided were sufficiently reliable for the purposes of this report.

To assess the extent to which the U.S. government has evaluated the effectiveness of its security assistance programs in Lebanon, we reviewed relevant State and DOD documents, including an independent evaluation of the Bureau of International Narcotics and Law Enforcement Affairs’ police training program in Lebanon, DOD’s Section 1206 Assessment Handbook, and various State and DOD program and country assessments. We also examined State evaluation guidelines, Bureau of International Narcotics and Law Enforcement Affairs evaluation guidance, various bureau evaluation plans, and other documents. Furthermore, we reviewed relevant GAO reports, including those that discussed how agencies measure program performance. We also reviewed special GAO publications on performance measurement and evaluation. In addition, we interviewed officials from State and DOD in Washington, D.C., and at the U.S. Embassy in Beirut, Lebanon, and DOD officials at the U.S. Central Command and U.S. Special Operations Command in Tampa, Florida.

We conducted this performance audit from June 2012 to March 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Status of Funds for U.S. Security Assistance Programs for Lebanon

This appendix provides additional information on Department of State (State) and Department of Defense (DOD) programs used to provide security assistance to Lebanon from fiscal years 2007 through 2012. State and DOD utilized eight programs to provide more than $925 million in security assistance to Lebanon from fiscal years 2007 through 2012. The majority of security assistance (95 percent) was provided through three programs: Foreign Military Financing, International Narcotics Control and Law Enforcement (INCLE), and the Section 1206 authority. Specifically, this appendix provides additional information on the Foreign Military Financing, INCLE, and the Section 1206 authority such as selected equipment and training provided and the status of funds including allocation, obligation, and commitment or disbursement of funds for the other five programs.

Foreign Military Financing Background

State administers Foreign Military Financing funding, with significant input from embassies, DOD, and the regional military commands. DOD’s Defense Security Cooperation Agency executes the program. Foreign Military Financing provides grants and loans to foreign governments and international organizations for the acquisition of U.S. defense equipment, services, and training.

From fiscal years 2007 through 2012, State allocated a total of about $641 million in Foreign Military Financing for Lebanon. DOD reported that for fiscal years 2007 through 2012, the majority of allocated funds had been committed (see fig. 5). For Lebanon, Foreign Military Financing has provided basic equipment such as tactical radios, ammunition, rifles, helmets, body armor, and trucks. Lebanon has also received night vision goggles, missiles, and helicopters.

INCLE Background

State administers the INCLE program and uses it to assist the Government of Lebanon in developing its law enforcement and judicial capacity. These funds are generally available for obligation for 2 years. Once the period of availability for new obligations expires, the funds are available for an additional 5 years to liquidate the obligations. The period of availability for obligations of INCLE funds may be extended for an additional 4 years in certain cases.

From fiscal years 2007 through 2012, State allocated a total of $130 million in INCLE funds for Lebanon. While most of the allocated funds have been obligated, about 40 percent of total allocated funds from fiscal years 2007 through 2012 have been disbursed (see fig. 6). According to State, fiscal year 2007 marked the first allocation of INCLE funds for Lebanon. In Lebanon, the INCLE program has funded the construction of a police station building and the purchase of riot helmets and batons, pistols, and police vehicles, among other equipment. In addition, the INCLE program has funded training and technical assistance to the Lebanese Internal Security Forces, including, for example, courses in community policing.

Section 1206 Background

The National Defense Authorization Act for Fiscal Year 2006, as amended, established the authority for Section 1206 programs to train and equip foreign national military and nonmilitary maritime forces, such as coast guards, to conduct counterterrorist operations or to support military and stability operations in which the U.S. armed forces are a participant.

From fiscal years 2007 through 2012, DOD allocated a total of $111 million through the Section 1206 program for Lebanon (see fig. 7). State reported that all allocated funds from fiscal years 2007 through 2012 have been obligated and disbursed.

For Lebanon, Section 1206 funding has provided vehicle spare parts, ammunition, and other basic supplies to the Lebanese Armed Forces (LAF). In particular, equipment provided under Section 1206 was used to restock the LAF arsenal with basic ammunition after the 2007 siege at Nahr al-Bared Palestinian refugee camp and to begin to build the LAF’s first secure communications system.

State and DOD jointly administer Section 1206 assistance. Funded from the DOD operations and maintenance accounts, Section 1206 funds remain available for 1 year for obligation. Once the period of availability for new obligations expires, the funds are available for an additional 5 years to liquidate the obligations.

In addition to the Foreign Military Financing, INCLE, and Section 1206 programs, State and DOD provided security assistance to Lebanon through five other programs: Antiterrorism Assistance, International Military Education and Training, Section 1207 Train and Equip, Export Control and Related Border Security, and Counterterrorism Financing. See tables 2 through 6 for details on allocation, obligation, and disbursement of funds by these programs for security assistance for Lebanon from fiscal years 2007 through 2012.

Table 7 presents selected security equipment and services provided by the United States to the Lebanese Armed Forces (LAF) through the Foreign Military Financing and Section 1206 programs from fiscal years 2007 through 2012. According to data provided by the Defense Security Cooperation Agency, 78 cases were approved for Lebanon during this time frame and ranged in value from $18,000 to $38.2 million. The defense articles or services are listed in descending order based on the total estimated case value as reported by the Defense Security Cooperation Agency. Table 8 presents selected security equipment or funding provided by countries other than the United States to the LAF, as reported by the LAF. Countries are listed in descending order based on the estimated value of security assistance provided.

Appendix IV: Comments from the U.S. Department of State

Appendix V: Comments from the Department of Defense

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Jeff Phillips (Assistant Director); Jenna Beveridge; Teakoe S. Coleman; David Dayton; and La Verne Tharpes made key contributions to this report. Martin de Alteriis, Grace Lui, and Jeremy Sebest provided additional technical assistance.