Drug Control: DOD Needs to Improve Its Performance Measurement System to Better Manage and Oversee Its Counternarcotics Activities
Published: 2010-07-21T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Defense (DOD) leads detection and monitoring of aerial and maritime transit of illegal drugs into the United States in support of law enforcement agencies. DOD reported resources of more than $1.5 billion for fiscal year 2010 to support its counternarcotics activities. Congress mandated GAO report on DOD's counternarcotics performance measurement system. Specifically, this report addresses the extent to which (1) DOD's counternarcotics performance measurement system enables DOD to track progress and (2) DOD uses performance information from its counternarcotics performance measurement system to manage its activities. GAO analyzed relevant DOD performance and budget documents, and discussed these efforts with officials from DOD and the Office of National Drug Control Policy (ONDCP).
section_title: What GAO Found
paragraphs: DOD does not have an effective performance measurement system to track the progress of its counternarcotics activities; however, it continues efforts to improve the system. GAO has previously reported that measuring performance provides managers a basis for making fact-based decisions. DOD has established performance measures for its counternarcotics activities and a database to collect performance information, including measures, targets, and results. However, these measures lack a number of attributes, such as being clearly stated and objective, which GAO considers key to successful performance measures. In May 2010, DOD issued new guidance for its counternarcotics performance measurement system. However, DOD officials noted the department will face challenges implementing the guidance. These challenges include creating performance measures that assess program outcomes and ensuring adequate resources, such as expertise in performance management, are available to develop measures. DOD rarely uses the information in its performance measurement system to manage its counternarcotics activities and has applied few practices to facilitate its use. GAO has found that the full benefit of collecting performance information is realized only when managers use it to inform key decisions. However, DOD officials responsible for counternarcotics activities throughout the department told us they rarely use data submitted to the system to manage activities. Rather, they tend to manage programs using data not submitted to the system, such as information obtained in weekly program meetings regarding the cost and timeliness of projects. Moreover, officials responsible for oversight of DOD's activities stated they use the system to develop reports for ONDCP, but not to allocate resources. While DOD has applied some practices to facilitate the use of the performance information in its system, it does not utilize certain key practices identified by GAO, such as frequently and effectively communicating performance information. Absent an effective performance management system, DOD lacks critical information to use to improve the management and oversight of its counternarcotics activities.
section_title: What GAO Recommends
paragraphs: GAO recommends that the Secretary of Defense take steps to improve DOD's counternarcotics performance measurement system by (1) revising its performance measures and (2) applying practices to better facilitate the use of performance data to manage its counternarcotics activities. DOD concurred with GAO's recommendations.

section_title: Letter
paragraphs: The global drug trade threatens U.S. national security by weakening the  rule of law in affected countries, financing the activities of global and  regional terrorists, and contributing to dangers such as weapons  trafficking. The Department of Defense (DOD) leads detection and  monitoring of aerial and maritime transit of illegal drugs into the United  States in support of law enforcement agencies. Additionally, DOD’s  counternarcotics activities include sharing information with U.S. and  foreign agencies, as well as helping foreign countries build their  counternarcotics capacity. DOD reported resources of more than $1.5  billion for fiscal year 2010 in support of these activities.
The National Defense Authorization Act for Fiscal Year 2010 mandated  that GAO report on the performance measurement system used by DOD to  assess its counternarcotics activities. We have previously reported that  performance measurement systems used by results-oriented agencies  include steps to measure performance to gauge progress and use the  information obtained to make key management decisions. In April 2010  we briefed congressional staff from the defense committees on our  preliminary observations regarding DOD’s counternarcotics performance  measurement system. This report contains the final results of our  evaluation. Specifically, we address the extent to which (1) DOD’s  counternarcotics performance measurement system enables DOD to track  progress and (2) DOD uses performance information from its  counternarcotics performance measurement system to manage its  activities.
To address these objectives, we analyzed DOD strategy, budget, and  performance documents, as well as DOD and Office of National Drug  Control Policy (ONDCP) guidance on performance measures. Further, we  discussed DOD’s counternarcotics performance measurement system and  its use of performance information with officials from ONDCP and DOD  components including the Office of the Deputy Assistant Secretary of  Defense for Counternarcotics and Global Threats (DASD-CN>), U.S.  Africa Command (AFRICOM), U.S. Central Command (CENTCOM), U.S.  European Command (EUCOM), U.S. Northern Command (NORTHCOM),  U.S. Southern Command (SOUTHCOM), the Joint Interagency Task Force- South (JIATF-S), the Joint Interagency Task Force-West (JIATF-W), and  the DOD Inspector General (DOD-IG). We evaluated a generalizable  random sample of DOD’s fiscal year 2009 counternarcotics performance  measures (115 of 239 measures) to assess the extent to which these  measures adhered to GAO criteria on the key attributes of successful  performance measures. We also analyzed the extent to which DOD applies  key management practices identified by GAO to facilitate the use of  performance information from its counternarcotics performance  measurement system. Moreover, we visited CENTCOM, SOUTHCOM, and  JIATF-S to examine DOD’s use of performance data to support its  counternarcotics mission. (See appendix I for a complete description of  our scope and methodology.)
We conducted this performance audit from December 2009 to July 2010 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Background
subsections: section_title: DOD Counternarcotics Strategy and Activities
paragraphs: According to DOD’s Counternarcotics Strategy developed in fiscal year  2009, the department seeks to disrupt the market for illegal drugs by  helping local, state, federal, and foreign government agencies address the  drug trade and narcotics-related terrorism.  DOD achieves this mission  through three goals—detecting and monitoring drug trafficking, sharing  information on illegal drugs with U.S. and foreign government agencies,  and building the counternarcotics capacity of U.S. and foreign partners.
DASD-CN>, with oversight from the Under Secretary of Defense for  Policy, exercises management and oversight of DOD’s counternarcotics  activities and performance measurement system. DASD-CN>’s  responsibilities include ensuring DOD develops and implements a  counternarcotics program with clear priorities and measured results.  Programs, Resources, and Assessments, a division within DASD-CN>,  is the lead office for the development of counternarcotics resources and  plans. Among other activities, this office directs and manages the planning,  programming, and budgeting system of the DOD counternarcotics  program and is responsible for updating and disseminating guidance on  DOD’s counternarcotics performance measurement system.
DOD’s counternarcotics activities are implemented through DOD’s  combatant commands, military departments, and defense agencies.  According to DOD, these organizations provide assets, such as aircraft and  patrol ships, military personnel, and other assistance, to support U.S. law  enforcement agencies and foreign security forces in countering narcotics  trafficking.
In support of DOD’s counternarcotics activities, DOD reported resources  totaling approximately $7.7 billion from fiscal year 2005 to fiscal year 2010,  including more than $6.1 billion appropriated to its Counternarcotics  Central Transfer Account and more than $1.5 billion in supplemental  appropriations (see table 1).
Of these resources, DOD estimated that approximately $4.2 billion were in  support of its international counternarcotics activities from fiscal years  2005-2010.
section_title: Previous GAO Reporting and Legislation Related to DOD’s Counternarcotics Performance Measures
paragraphs: DOD efforts to develop performance measures for its counternarcotics  activities are long-standing. We reported in December 1999 that DOD had  not developed a set of performance measures to assess the impact of its  counternarcotics operations, but had undertaken initial steps to develop  such measures. In January 2002 and November 2005, we found that DOD  was in the process of developing performance measures focused on its  role of detecting and monitoring the trafficking of illegal drugs into the  United States. In November 2005 we recommended that DOD, in  conjunction with other agencies performing counternarcotics activities,  develop and coordinate counternarcotics performance measures.
In December 2006 Congress directed ONDCP—the organization that  establishes U.S. counternarcotics goals and coordinates the federal budget  to combat drugs—to produce an annual report describing the national  drug control performance measurement system that identifies the  activities of national drug control program agencies, including DOD. In  May 2007 ONDCP issued guidance requiring DOD and other national drug  control program agencies to annually submit to the Director of ONDCP a  performance summary report including performance measures, targets,  and results. In addition, ONDCP officials stated that they have  recommended improvements to DOD’s performance measures, both in  correspondence and in meetings with DOD staff.
section_title: DOD Has Not Developed a System to Effectively Track the Progress of Its Counternarcotics Activities, but Continues to Work to Improve Its Efforts
paragraphs: DOD does not have an effective system for tracking the progress of its  counternarcotics activities; however, it continues efforts to improve the  system. We have found that measuring performance provides managers a  basis for making fact-based decisions. DOD has established performance  measures for its counternarcotics activities and a database to collect  performance information. However, these measures lack a number of  attributes which we consider key to successful performance measures  and, therefore, do not provide a clear indication of DOD’s progress toward  its counternarcotics goals. Recognizing the need to update and improve its  measures, in May 2010, DOD issued new guidance for its counternarcotics  performance measurement system. However, DOD officials noted the  department will faces challenges implementing the guidance.
subsections: section_title: DOD Has Developed Performance Measures and a Database for Its Counternarcotics Activities
paragraphs: We have previously reported that effective performance measurement  systems include steps to measure performance, such as establishing  performance measures and collecting data. In response to ONDCP’s 2007  guidance, DOD developed performance measures for its fiscal year 2007  counternarcotics activities and established a centralized database within  its performance measurement system to collect data on those  performance measures. The counternarcotics performance measurement  system database, maintained by DASD-CN>, requires DOD components  to submit performance information at specified intervals during the fiscal  year, such as results for performance measures, the mechanisms used to  collect results data, and future performance targets. For fiscal year 2009,  DOD guidance required that all projects funded by its Counternarcotics  Central Transfer Account have a performance measure. As a result, DOD  reported it had 285 performance measures for its fiscal year 2009  counternarcotics activities. Of those, 239 were performance measures  related to DOD’s mission of supporting U.S. agencies and foreign partners  in countering narcotics trafficking. (See table 2 for examples of DOD’s  counternarcotics performance measures.)
section_title: DOD’s Fiscal Year 2009 Counternarcotics Performance Measures Exhibit Some, but Not All, Key Attributes of Successful Performance Measures
paragraphs: DOD’s current set of counternarcotics performance measures varies in the  degree to which it exhibits key attributes of successful performance  measures. Prior GAO work has identified nine attributes of successful  performance measures. Table 3 shows the nine attributes, their  definitions, and the potentially adverse consequences of not having the  attributes.
Our analysis found that DOD’s counternarcotics performance measures  lack several of the key attributes of successful performance measures.  Based on our analysis of a generalizable sample of DOD’s fiscal year 2009  performance measures, we found the attributes of core program  activities and linkage were generally present, but other attributes such as  balance and limited overlap were missing, and attributes including  governmentwide priorities, reliability, objectivity, clarity, and measurable  targets were present in varying degrees.
We found that the attribute of core program activities was identified in the  set of measures, while balance and limited overlap did not appear to be  present.
Core program activities. We estimate that all of DOD’s counternarcotics  performance measures cover the department’s core program activities. We  have previously reported that core program activities are the activities that  an entity is expected to perform to support the intent of the program, and  that performance measures should be scoped to evaluate those activities.  For the measures we reviewed, DOD divides its core counternarcotics  activities across its 3 goals and 13 objectives (see table 2). In our analysis,  we found at least one performance measure covering each of DOD’s  counternarcotics objectives. Therefore, we determined that DOD’s core  program activities were covered.
Balance. DOD’s set of performance measures lack balance. We have  previously reported that balance exists when a set of measures ensures  that an organization’s various priorities are covered. According to DOD,  performance measures best cover its priorities when five measurable  aspects of performance, as defined by DOD—input, process, output,  outcome, and impact—are present in its performance measures. As an  example, “number of attendees to basic counterdrug intelligence course”  is, in our determination, a measure of output, as it measures the services  provided by DOD. We estimate 93 percent of DOD’s fiscal year 2009  performance measures are input, process, or output measures, while 6  percent are outcome measures and 0 percent are impact measures.  Therefore, given that DOD’s set of measures is highly skewed towards  input, process, and output measures and contains no impact measures, we  determined that the set is not balanced by DOD’s criteria. Performance  measurement efforts that lack balance overemphasize certain aspects of  performance at the expense of others, and may keep DOD from  understanding the effectiveness of its overall mission and goals.
Limited overlap. We determined there to be overlap among DOD’s  performance measures. We found instances where the measures and their  results appeared to overlap with other measures and results. When we  spoke with DASD-CN> officials concerning this, they stated that the set  of measures could be conveyed using fewer, more accurate measures. We  have reported that each performance measure in a set should provide  additional information beyond that provided by other measures. When an  agency has overlapping measures, it can create unnecessary or duplicate  information, which does not benefit program management.
Of the remaining six attributes of successful performance measures, only  one attribute—linkage—was present in almost all of the measures, while  the other five attributes—governmentwide priorities, reliability,  objectivity, clarity, and measurable targets—appeared in varying degrees  (see figure 1).
DOD’s counternarcotics performance measures demonstrate linkage. We  estimate that 99 percent of DOD’s measures are linked to agencywide  goals and mission. DOD’s counternarcotics performance measurement  system database requires that for each performance measure entered into  the database, a goal and related objective of DOD’s counternarcotics  mission be identified. Our analysis found that in all but one instance,  linkage between DOD’s goals and performance measures is easily  identified.
However, DOD’s counternarcotics performance measures did not fully  satisfy five attributes.
Governmentwide priorities. We estimate that 41 percent of the measures  we analyzed cover a broader governmentwide priority, such as quality,  timeliness, efficiency, cost of service, or outcome. We determined, for  example, that the governmentwide priority of “quality” was reflected in the  measure “number of sensors integrated and providing reliable and  dependable radar data to JIATF-S and/or host nations,” because it  measures the reliability and dependability of detection services. In the  majority of the instances, however, measures did not address a  governmentwide priority. For example, the measure “number of trained  military working dog teams trained” was determined not to cover a  governmentwide priority because it does not measure the quality or  efficiency of training provided. When measures fail to cover  governmentwide priorities managers may not be able to balance priorities  to ensure the overall success of the program.
Reliability. We estimate that 46 percent of DOD’s performance measures  have data collection methods indicated in the database that generally  appear reliable. Reliability refers to whether a measure is designed to  collect data or calculate results such that the measure would be likely to  produce the same results if applied repeatedly to the same situation. For  each entry in the database, users are directed to enter, among other  information, one performance measure and its associated methodology,  target, and result. However, in numerous instances the system contained  multiple performance measures entered into fields that should contain  only one measure. Such entries could result in errors of collecting,  maintaining, processing, or reporting the data. Additionally, some  measures did not provide enough information on data collection methods  or performance targets to assure reliability. For example, a measure in the  database states “continuous U.S. Navy ship presence in the SOUTHCOM  area of responsibility.” The performance target listed for this measure is  “3.5,” but to what 3.5 refers—such as days, number of ships, or percentage  points—is not explained. Moreover, the methodology in the database for  this measure is entered as “not applicable.” Therefore, the measure’s  methodology does not provide insight into how DOD could measure  whether or not it reached its target of 3.5. As a result, we determined that  this measure did not have data collection methods to gather reliable  results. We have previously reported that if errors occur in the collection  of data or the calculation of their results, it may affect conclusions about  the extent to which performance goals have been achieved.
Objectivity. We estimate that 59 percent of DOD’s performance measures  for its counternarcotics activities are objective. We have previously  reported that to be objective, measures should indicate specifically what is  to be observed, in which population or conditions, and in what time frame,  and be free of opinion and judgment. We estimate that 41 percent of DOD’s  measures are not objective and could therefore face issues of bias or  manipulation. For example, a measure in the database is, “percent of  inland waterways controlled by Colombian Marine Corps forces.” For this  measure, no criteria for “controlled” is provided and it is not clear how the  Colombian government reports the percentage of waterways under its  control and over what time frame this control will occur.
Clarity. We estimate that 65 percent of DOD’s performance measures  exhibit the attribute of clarity. A measure achieves clarity when it is  clearly stated and the name and definition are consistent with the  methodology used for calculating the measure. However, we estimate that  35 percent of DOD’s measures are not clearly stated. For example, one of  DOD’s measures linked to the objective of sharing information with U.S.  and partner nations is “identify and establish methodology for  implementation.” For this measure, no associated methodology is  identified, and it is unclear what is being implemented. We have previously  reported that a measure that is not clearly stated can confuse users and  cause managers or other stakeholders to think that performance was  better or worse than it actually was.
Measurable target. We estimate that 66 percent of DOD’s measures have  measurable targets. Where appropriate, performance goals and measures  should have quantifiable, numerical targets or other measurable values.  Some of DOD’s measures, however, lacked such targets. For example, one  performance measure identified its target as “targets developed by the  local commander.” As it is not quantifiable, this target does not allow  officials to easily assess whether goals were achieved because  comparisons cannot be made between projected performance and actual  results.
section_title: DOD Is Working To Improve Its Counternarcotics Performance Measures, but Implementation Challenges Exist
paragraphs: DOD officials have acknowledged that weaknesses exist in the  department’s current set of counternarcotics performance measures. In  May 2010 DOD issued revised guidance for its counternarcotics  performance measurement system to guide users in establishing  performance measures that more accurately capture the quantitative and  qualitative achievements of DOD’s activities. To do this, the guidance  states that performance measures should be, among other attributes,  useful for management and clearly stated. The guidance describes  different types of performance measures that can be used to monitor  DOD’s contribution to its strategic counternarcotics goals, such as those  that measure DOD’s efficiency, capability, and effectiveness at performing  its activities. Additionally, according to the guidance, DOD components  should provide evidence of the quality and reliability of the data used to  measure performance.
However, DOD officials noted four specific challenges that the department  faces in developing performance measures consistent with its revised  guidance.
Creating performance measures that assess program outcomes. Some  DOD officials noted that, because DOD acts as a support agency to partner  nations and other law enforcement entities—and the actual interdiction of  drugs is conducted by other entities—measuring the outcome of DOD’s  performance is difficult. While developing outcome measures can be  challenging, we have found that an agency’s performance measures should  reflect a range of priorities, including outcomes. Moreover, we have found  that methods to measure program outcomes do exist. For example,  agencies have applied a range of strategies to develop outcome measures  for their program, such as developing measures of satisfaction based upon  surveys of customers. In addition, officials from EUCOM, AFRICOM, and  JIATF-S stated that while developing outcome performance measures can  be difficult, developing such measures for support activities is possible  and is done at other federal agencies. For example, EUCOM indicated it  could track the outcome of the support it provides to partner nations by  tracking the annual percentage increase in interdictions and arrests  related to illicit trafficking. Additionally, JIATF-W indicated that it  conducts quarterly command assessments of current programs, which  focus on aligning resources provided by JIATF-W to the outcomes of its  law enforcement partners.
Implementing revisions in a timely manner. DOD officials noted that  implementing revisions to the department’s performance measures in a  timely fashion will be difficult given that such revisions are resource and  time intensive. Further, while including dates for submission, DOD’s  revised guidance does not clearly specify a time frame by which DOD  components should revise the counternarcotics performance measures  that are to be submitted to the database. We have previously reported that  establishing timetables for the development of performance measures can  create a sense of urgency that assists in the effort being taken more  seriously. DASD-CN> officials noted that time frames by which DOD’s  measures would be revised are being discussed. However, these officials  do not expect new performance measures to be established in fiscal year  2010, and said that fiscal year 2011 would be the earliest year of full  implementation of the guidance.
Ensuring adequate resources are available. DOD officials noted that  ensuring adequate resources—such as expertise and training in  performance management—are available to develop performance  measures at both DASD-CN> and the combatant commands will be a  challenge. These officials noted that DOD employees tasked with  developing performance measures and tracking the progress towards  achieving goals are not sufficiently trained to design and monitor outcome  performance measures. We have previously reported that access to trained  staff assists agencies in their development of performance measures.
Ensuring reliable data. DOD officials noted that ensuring data used to  measure DOD performance are reliable is challenging. To measure the  performance of its counternarcotics activities DOD officials told us they  rely heavily on external sources of data, such as U.S. law enforcement  agencies and foreign government officials. This challenge can pose issues  for DOD regarding data verification and ensuring proper information is  recorded for performance measures.
section_title: DOD Rarely Uses the Performance Information Contained in Its Performance Measurement System to Manage Its Counternarcotics Activities and Has Applied Few Practices to Facilitate Its Use
paragraphs: DOD makes limited use of its performance measurement system to  manage its counternarcotics activities and has applied few practices to  facilitate its use. We have found that the full benefit of collecting  performance information is realized only when managers use the  information to inform key decisions. While DOD has applied some  practices to facilitate the use of the performance information in its system,  it does not utilize certain key practices, such as frequently and effectively  communicating performance information. Absent an effective  performance management system, DOD lacks critical information to use to  improve the management and oversight of its counternarcotics activities.
subsections: section_title: Agencies Can Use Performance Information to Manage for Results
paragraphs: We have previously reported that, in addition to measuring performance,  effective performance measurement systems include steps to use  information obtained from performance measures to make decisions that  improve programs and results. We identified several ways in which  agencies can use performance information to manage for results, including  using data to (1) identify problems and take corrective actions, (2) develop  strategy and allocate resources, and (3) identify and share effective  approaches.
section_title: DOD Submits Performance Reports to ONDCP, But Makes Limited Use of the Information in Its Performance Measurement System to Manage and Oversee Its Counternarcotics Activities
paragraphs: DOD officials representing DASD-CN>, AFRICOM, CENTCOM,  EUCOM, NORTHCOM, SOUTCOM, JIATF-S, and JIATF-W told us they  rarely use information from DOD’s counternarcotics performance  measurement system to manage counternarcotics activities. Specifically,  they rarely use the system to:  Identify problems and take corrective actions. Agencies can use  performance information to identify problems or weaknesses in programs,  to try to identify factors causing the problems, and to modify a service or  process to try to address problems. DOD officials representing DASD- CN> and SOUTHCOM told us that they currently make limited use of  the performance information in DOD’s performance measurement system  to manage counternarcotics activities. Officials from DASD-CN> stated  that they use data from the performance measurement system to produce  reports for ONDCP, which may include information identifying problems  in the implementation of DOD’s counternarcotics activities. However, in  reviewing these documents, we found that the reports did not include a  clear assessment of DOD’s overall progress toward its counternarcotics  goals. For instance, the report submitted to ONDCP for fiscal year 2009  contained detailed information on 6 of DOD’s 285 counternarcotics  performance measures, but did not clearly explain why the results of these  6 measures would be critical to the success of DOD’s counternarcotics  program. Moreover, according to ONDCP, DOD’s reports for fiscal years  2007, 2008, and 2009 did not fulfill the requirements of ONDCP’s guidance  because the reports were not authenticated by the DOD-IG.
Further, officials from AFRICOM, CENTCOM, EUCOM, NORTHCOM,  JIATF-S, and JIATF-W told us they do not use the DOD’s performance  measurement system to manage counternarcotics activities. While these  officials indicated that they submitted performance information to the  system’s database as required by DOD guidance, they stated they tend to  manage programs using information not submitted to the system (see table  4). For example, CENTCOM officials told us information obtained in  weekly program meetings regarding the timeliness and cost of  counternarcotics projects, not data sent to the system’s database, is most  often used to help them identify problems and make program adjustments.
Recognizing the need improve the information in the system’s database,  officials from DASD-CN> told us that for fiscal year 2011 they are  working with DOD components to integrate performance information into  the system’s database that can be more useful for decision making.  Officials from several combatant commands stated they could integrate  performance information obtained from outside sources into the  counternarcotics performance measurement system. Officials from JIATF- S, for example, told us they collect and analyze a variety of data on  counternarcotics activities that they do not input into DOD’s  counternarcotics performance measurement system. On a daily basis,  JIATF-S collects information on “cases”—that is, boats or planes  suspected of illegal trafficking. In addition to tracking the number of  cases, JIATF-S compiles information as to whether or not a particular case  was targeted, detected, or monitored, and whether or not those actions  resulted in interdictions or seizures of illegal drugs. By compiling this  information, officials at JIATF-S told us they can better identify program  outcomes, areas in which their efforts are successful, and ways to take  corrective actions.
Develop strategy and allocate resources. Agencies can use performance  information to make decisions that affect future strategies, planning, and  budgeting, and allocating resources. DASD-CN>’s role includes both  defining the strategic goals and managing the budgeting system of the  DOD counternarcotics program. DOD’s counternarcotics guidance states  that information from the counternarcotics performance measurement  system will inform strategic counternarcotics plans, but it does not clearly  state how the system will be used to inform decisions to allocate  resources. Moreover, officials from DASD-CN> told us that the office  does not currently link performance information from the  counternarcotics performance measurement system’s database directly to  budget allocation decisions. In addition, our analysis of DOD’s fiscal year  2011 Drug Interdiction and Counterdrug Activities Budget Estimates— which provides details on DOD’s fiscal year 2011 budget request for its  counternarcotics activities—identified no clear link between budget  allocation decisions and performance information in the system’s  database. DOD officials told us they plan to incorporate performance  information from the counternarcotics performance measurement system  into future budget requests provided to Congress.
Identify and share effective approaches. We have reported that high- performing organizations can use performance information to identify and  increase the use of program approaches that are working well. According  to DOD’s counternarcotics performance measurement system guidance,  DASD-CN> will use performance information submitted to the system’s  database to compile reports for ONDCP, which DASD-CN> has done.  However, DASD-CN> officials told us they do not currently use the  system to produce reports for DOD components, which could assist in  identifying and sharing effective approaches between DOD’s components.  While indicating performance reports could be a useful tool, officials from  several DOD components told us they had not received such reports from  DASD-CN>. DOD’s May 2010 guidance does not state whether the  system will be used to produce such reports in the future.
section_title: DOD Has Applied Few Practices to Facilitate the Use of Its Counternarcotics Performance Measurement System
paragraphs: We have found that agencies can adopt practices that can facilitate the use  of performance data. These include (1) demonstrating management  commitment to results-oriented management; (2) aligning agencywide  goals, objectives, and measures; (3) improving the usefulness of  performance data to better meet management’s needs; (4) developing  agency capacity to effectively use performance information; and (5)  communicating performance information within the agency frequently and  effectively.
As part of its role overseeing DOD’s counternarcotics activities, DASD- CN> manages the DOD counternarcotics performance measurement  system. DASD-CN> applies some practices to facilitate the use of its  counternarcotics performance measurement system. For example, DASD- CN> has recently taken steps to demonstrate management  commitment by issuing revised guidance emphasizing the development of  improved performance measures and, according to DASD-CN>  officials, conducting working groups with some DOD components to  assist them in revising performance measures. Moreover, DASD-CN>  officials told us they are taking steps to increase staffing to better oversee  the performance measurement system. We have found that the  commitment of agency managers to result-oriented management is critical  to increased use of performance information for policy and program  decisions. Further, DASD-CN> has created a results framework that  aligns agencywide goals, objectives, and performance measures for its  counternarcotics activities. As we have previously reported, such an  alignment increases the usefulness of the performance information  collected by decision makers at each level, and reinforces the connection  between strategic goals and the day-to-day activities of managers and staff.
However, DASD-CN> has not applied certain key practices to facilitate  the use of data, such as improving the usefulness of performance  information in its performance measurement system, developing agency  capacity to use performance information, and communicating  performance information frequently and effectively. Furthermore, DOD  officials told us they face challenges using DOD’s performance  measurement system to manage their activities due to (1) the limited  utility of the performance measures and data currently in DOD’s  counternarcotics database, (2) insufficient capacity to collect and use  performance information, and (3) infrequent communication from DASD- CN> regarding performance information submitted to the database.  For instance, DOD’s guidance emphasizes the development of  performance measures that are, among other attributes, useful for  management and supported by credible data. However, DOD officials from  several combatant commands told us that the performance measures and  targets currently in the system are of limited utility and will need to be  revised. Moreover, officials from several DOD components emphasized the  need to build additional capacity to use performance data, such as  receiving training on how to revise performance standards and measures.  We have found that the practice of building analytical capacity to use  performance information—both in terms of staff trained to do analysis and  availability of research and evaluation resources—is critical to an agency  using performance information in a meaningful way. Finally, DOD  components told us that they received little feedback or direction from  DASD-CN> regarding performance information they submitted to the  system. We have previously reported that improving the communication of  performance information among staff and stakeholders can facilitate the  use of performance information in key management activities. For more  information see table 5.
section_title: Conclusions
paragraphs: DOD reported more than $1.5 billion in fiscal year 2010 for its  counternarcotics activities, but has not yet developed an effective  performance measurement system to readily inform progress toward the  achievement of its counternarcotics goals. We have previously reported  that performance measurement systems include steps to measure  performance to gauge progress and use the information obtained to make  key management decisions. DOD acknowledges weaknesses in its  performance measurement system and has taken steps to improve the  system, such as revising its guidance for the development of performance  measures and holding working groups with DOD components. However,  its current set of measures lack key attributes of successful performance  measures, such as balance, objectivity, and reliability. Moreover, DOD  infrequently uses the information presently in its counternarcotics  performance measurement system and has yet to fully apply key practices  to facilitate its use. Absent an effective performance measurement system,  DOD lacks critical performance information to use to improve its  management decisions, eliminate wasteful or unproductive efforts, and  conduct oversight of its activities.
section_title: Recommendations for Executive Action
paragraphs: To improve DOD’s performance measurement system to manage and  oversee its counternarcotics activities, we recommend that the Secretary  of Defense take the following two actions: 1.  To address weaknesses identified in DOD’s counternarcotics  performance measurement system, we recommend that the Secretary  of Defense direct the Deputy Assistant Secretary for Counternarcotics  and Global Threats to review the department’s performance measures  for counternarcotics activities and revise the measures, as appropriate,  to include the key attributes of successful performance measures  previously identified by GAO. 2.  To address factors associated with the limited use of DOD’s  counternarcotics performance measurement system, we recommend  that the Secretary of Defense direct the Deputy Assistant Secretary for  Counternarcotics and Global Threats to apply practices that GAO has  identified to facilitate the use of performance data.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DOD and ONDCP for their review and  comment. We received written comments from DOD, which are reprinted  in appendix II. DOD concurred with our recommendations, and stated it  has developed and begun to implement a plan to improve the quality and  usefulness of its counternarcotics performance measurement system.  ONDCP did not provide written comments.
We received technical comments from DOD and ONDCP, which we have  incorporated where appropriate.
We are sending copies of this report to interested congressional  committees, the Secretary of Defense, and the Director of the Office of  National Drug Control Policy. In addition, the report will be available at no  charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-4268 or fordj@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made major contributions to this report are  listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Section 1016 of the National Defense Authorization Act for Fiscal Year  2010 directed GAO to report on the Department of Defense’s (DOD)  performance measurement system used to assess its counternarcotics  activities. In response to this mandate, we examined the extent to which  (1) DOD’s counternarcotics performance measurement system enables  DOD to track progress and (2) DOD uses performance information from  its counternarcotics performance measurement system to manage its  activities.
Our work focused on the efforts of DOD to develop an effective  counternarcotics performance measurement system. Within DOD, we  spoke with officials from several relevant components involved in the  management, oversight, and implementation of DOD’s counternarcotics  activities, including the Office of the Deputy Assistant Secretary of  Defense for Counternarcotics and Global Threats (DASD-CN>), U.S.  Africa Command (AFRICOM), U.S. Central Command (CENTCOM), U.S.  European Command (EUCOM), U.S. Northern Command (NORTHCOM),  U.S. Southern Command (SOUTHCOM), the Joint Interagency Task Force- South (JIATF-S), the Joint Interagency Task Force-West (JIATF-W), and  the DOD Inspector General (DOD-IG). We also discussed DOD efforts with  officials from the Office of National Drug Control Policy (ONDCP), the  organization that establishes U.S. counternarcotics goals and coordinates  the federal budget to combat drugs.
To examine the extent to which DOD’s counternarcotics performance  measurement system enables the department to track its progress we  analyzed DOD strategy, budget, and performance documents, such as  DOD’s Counternarcotics Strategy, Drug Interdiction and Counterdrug  Activities Budget Estimates, and Performance Summary Reports. We  reviewed relevant DOD and ONDCP guidance on performance measures,  such as DOD’s Standard Operating Procedures for the Counternarcotics  Performance Metrics System and ONDCP’s Drug Control Accounting  circular. Further, we evaluated a generalizable random sample of DOD’s  fiscal year 2009 counternarcotics performance measures (115 of 239  measures) to assess the extent to which these measures adhered to GAO  criteria on the key attributes of successful performance measures.  Because we followed a probability procedure based on random selections,  our sample is only one of a large number of samples that we might have  drawn. Since each sample could have provided different estimates, we  express our confidence in the precision of our particular sample’s results  at a 95 percent confidence interval (e.g., plus or minus 6 percentage  points). This is the interval that would contain the actual population value  for 95 percent of the samples we could have drawn. To evaluate the  sample, two analysts independently assessed each of the performance  measures against nine attributes of successful performance measures  identified by GAO. Those analysts then met to discuss and resolve any  differences in the results of their analyses. A supervisor then reviewed and  approved the final results of the analysis. In conducting this analysis, we  analyzed information contained in DOD’s counternarcotics performance  measurement system database and spoke with DOD officials responsible  for managing counternarcotics activities and entering information into the  database. We did not, however, review supporting documentation  referenced but not included in the system’s database, nor did we assess  other databases that might exist at the DOD component level. We also  discussed DOD’s performance measures with cognizant officials from  ONDCP and several DOD components, including DASD-CN>,  AFRICOM, CENTCOM, EUCOM, NORTHCOM, SOUTHCOM, JIATF-S,  JIATF-W, and the DOD-IG.
To evaluate the extent to which DOD uses performance information from  its counternarcotics performance measurement system to support its  mission, we held discussions with officials from DOD components— including DASD-CN>, AFRICOM, CENTCOM, EUCOM, NORTHCOM,  SOUTHCOM, JIATF-S, and JIATF-W—to determine the ways in which  these components use information from DOD’s system, as well as other  sources of performance information. We also examined DOD’s  Performance Summary Reports and fiscal year 2011 Drug Interdiction  and Counterdrug Activities Budget Estimates to assess the extent to  which these materials reported that DOD used performance information  from its counternarcotics performance measurement system database.  Further, we analyzed the extent to which DOD applies key management  practices previously identified by GAO to facilitate the use of  performance information from its counternarcotics performance  measurement system. We also traveled to Tampa, Miami, and Key West,  Florida where we visited CENTCOM, SOUTHCOM, and JIATF-S. In these  visits, we met with DOD officials responsible for management and  implementation of counternarcotics activities to discuss DOD’s use of  performance data to support its counternarcotics mission.
To determine the completeness and consistency of DOD funding data, we  compiled and compared data from DOD with information from cognizant  U.S. agency officials in Washington, D.C. We also compared the funding  data with budget summary reports from the ONDCP to corroborate their  accuracy. Although we did not audit the funding data and are not  expressing an opinion on them, based on our examination of the  documents received and our discussions with cognizant agency officials,  we concluded that the funding data we obtained were sufficiently reliable  for the purposes of this report.
We conducted this performance audit from December 2009 to July 2010 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: Contact and Staff Acknowledgments
subsections: section_title: Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Juan Gobel, Assistant Director;  Elizabeth Curda; Martin de Alteriis; Karen Deans; Mark Dowling; Justin  Fisher; Richard Geiger; Eileen Larence; Marie Mak; Christopher Mulkins;  John Pendleton; Elizabeth Repko; and Mark Speight made key  contributions to this report.