Drug Control: DOD Needs to Improve Its Performance Measurement System to Better Manage and Oversee Its Counternarcotics Activities
Released: 2010-07-21T00:00:00

Letter

Background

DOD Counternarcotics Strategy and Activities

According to DOD’s Counternarcotics Strategy developed in fiscal year 2009, the department seeks to disrupt the market for illegal drugs by helping local, state, federal, and foreign government agencies address the drug trade and narcotics-related terrorism. DOD achieves this mission through three goals—detecting and monitoring drug trafficking, sharing information on illegal drugs with U.S. and foreign government agencies, and building the counternarcotics capacity of U.S. and foreign partners.

DASD-CN>, with oversight from the Under Secretary of Defense for Policy, exercises management and oversight of DOD’s counternarcotics activities and performance measurement system. DASD-CN>’s responsibilities include ensuring DOD develops and implements a counternarcotics program with clear priorities and measured results. Programs, Resources, and Assessments, a division within DASD-CN>, is the lead office for the development of counternarcotics resources and plans. Among other activities, this office directs and manages the planning, programming, and budgeting system of the DOD counternarcotics program and is responsible for updating and disseminating guidance on DOD’s counternarcotics performance measurement system.

DOD’s counternarcotics activities are implemented through DOD’s combatant commands, military departments, and defense agencies. According to DOD, these organizations provide assets, such as aircraft and patrol ships, military personnel, and other assistance, to support U.S. law enforcement agencies and foreign security forces in countering narcotics trafficking.

In support of DOD’s counternarcotics activities, DOD reported resources totaling approximately $7.7 billion from fiscal year 2005 to fiscal year 2010, including more than $6.1 billion appropriated to its Counternarcotics Central Transfer Account and more than $1.5 billion in supplemental appropriations (see table 1).

Of these resources, DOD estimated that approximately $4.2 billion were in support of its international counternarcotics activities from fiscal years 2005-2010.

Previous GAO Reporting and Legislation Related to DOD’s Counternarcotics Performance Measures

DOD efforts to develop performance measures for its counternarcotics activities are long-standing. We reported in December 1999 that DOD had not developed a set of performance measures to assess the impact of its counternarcotics operations, but had undertaken initial steps to develop such measures. In January 2002 and November 2005, we found that DOD was in the process of developing performance measures focused on its role of detecting and monitoring the trafficking of illegal drugs into the United States. In November 2005 we recommended that DOD, in conjunction with other agencies performing counternarcotics activities, develop and coordinate counternarcotics performance measures.

In December 2006 Congress directed ONDCP—the organization that establishes U.S. counternarcotics goals and coordinates the federal budget to combat drugs—to produce an annual report describing the national drug control performance measurement system that identifies the activities of national drug control program agencies, including DOD. In May 2007 ONDCP issued guidance requiring DOD and other national drug control program agencies to annually submit to the Director of ONDCP a performance summary report including performance measures, targets, and results. In addition, ONDCP officials stated that they have recommended improvements to DOD’s performance measures, both in correspondence and in meetings with DOD staff.

DOD Has Not Developed a System to Effectively Track the Progress of Its Counternarcotics Activities, but Continues to Work to Improve Its Efforts

DOD does not have an effective system for tracking the progress of its counternarcotics activities; however, it continues efforts to improve the system. We have found that measuring performance provides managers a basis for making fact-based decisions. DOD has established performance measures for its counternarcotics activities and a database to collect performance information. However, these measures lack a number of attributes which we consider key to successful performance measures and, therefore, do not provide a clear indication of DOD’s progress toward its counternarcotics goals. Recognizing the need to update and improve its measures, in May 2010, DOD issued new guidance for its counternarcotics performance measurement system. However, DOD officials noted the department will faces challenges implementing the guidance.

DOD Has Developed Performance Measures and a Database for Its Counternarcotics Activities

We have previously reported that effective performance measurement systems include steps to measure performance, such as establishing performance measures and collecting data. In response to ONDCP’s 2007 guidance, DOD developed performance measures for its fiscal year 2007 counternarcotics activities and established a centralized database within its performance measurement system to collect data on those performance measures. The counternarcotics performance measurement system database, maintained by DASD-CN>, requires DOD components to submit performance information at specified intervals during the fiscal year, such as results for performance measures, the mechanisms used to collect results data, and future performance targets. For fiscal year 2009, DOD guidance required that all projects funded by its Counternarcotics Central Transfer Account have a performance measure. As a result, DOD reported it had 285 performance measures for its fiscal year 2009 counternarcotics activities. Of those, 239 were performance measures related to DOD’s mission of supporting U.S. agencies and foreign partners in countering narcotics trafficking. (See table 2 for examples of DOD’s counternarcotics performance measures.)

DOD’s Fiscal Year 2009 Counternarcotics Performance Measures Exhibit Some, but Not All, Key Attributes of Successful Performance Measures

DOD’s current set of counternarcotics performance measures varies in the degree to which it exhibits key attributes of successful performance measures. Prior GAO work has identified nine attributes of successful performance measures. Table 3 shows the nine attributes, their definitions, and the potentially adverse consequences of not having the attributes.

Our analysis found that DOD’s counternarcotics performance measures lack several of the key attributes of successful performance measures. Based on our analysis of a generalizable sample of DOD’s fiscal year 2009 performance measures, we found the attributes of core program activities and linkage were generally present, but other attributes such as balance and limited overlap were missing, and attributes including governmentwide priorities, reliability, objectivity, clarity, and measurable targets were present in varying degrees.

We found that the attribute of core program activities was identified in the set of measures, while balance and limited overlap did not appear to be present.

Core program activities. We estimate that all of DOD’s counternarcotics performance measures cover the department’s core program activities. We have previously reported that core program activities are the activities that an entity is expected to perform to support the intent of the program, and that performance measures should be scoped to evaluate those activities. For the measures we reviewed, DOD divides its core counternarcotics activities across its 3 goals and 13 objectives (see table 2). In our analysis, we found at least one performance measure covering each of DOD’s counternarcotics objectives. Therefore, we determined that DOD’s core program activities were covered.

Balance. DOD’s set of performance measures lack balance. We have previously reported that balance exists when a set of measures ensures that an organization’s various priorities are covered. According to DOD, performance measures best cover its priorities when five measurable aspects of performance, as defined by DOD—input, process, output, outcome, and impact—are present in its performance measures. As an example, “number of attendees to basic counterdrug intelligence course” is, in our determination, a measure of output, as it measures the services provided by DOD. We estimate 93 percent of DOD’s fiscal year 2009 performance measures are input, process, or output measures, while 6 percent are outcome measures and 0 percent are impact measures. Therefore, given that DOD’s set of measures is highly skewed towards input, process, and output measures and contains no impact measures, we determined that the set is not balanced by DOD’s criteria. Performance measurement efforts that lack balance overemphasize certain aspects of performance at the expense of others, and may keep DOD from understanding the effectiveness of its overall mission and goals.

Limited overlap. We determined there to be overlap among DOD’s performance measures. We found instances where the measures and their results appeared to overlap with other measures and results. When we spoke with DASD-CN> officials concerning this, they stated that the set of measures could be conveyed using fewer, more accurate measures. We have reported that each performance measure in a set should provide additional information beyond that provided by other measures. When an agency has overlapping measures, it can create unnecessary or duplicate information, which does not benefit program management.

Of the remaining six attributes of successful performance measures, only one attribute—linkage—was present in almost all of the measures, while the other five attributes—governmentwide priorities, reliability, objectivity, clarity, and measurable targets—appeared in varying degrees (see figure 1).

DOD’s counternarcotics performance measures demonstrate linkage. We estimate that 99 percent of DOD’s measures are linked to agencywide goals and mission. DOD’s counternarcotics performance measurement system database requires that for each performance measure entered into the database, a goal and related objective of DOD’s counternarcotics mission be identified. Our analysis found that in all but one instance, linkage between DOD’s goals and performance measures is easily identified.

However, DOD’s counternarcotics performance measures did not fully satisfy five attributes.

Governmentwide priorities. We estimate that 41 percent of the measures we analyzed cover a broader governmentwide priority, such as quality, timeliness, efficiency, cost of service, or outcome. We determined, for example, that the governmentwide priority of “quality” was reflected in the measure “number of sensors integrated and providing reliable and dependable radar data to JIATF-S and/or host nations,” because it measures the reliability and dependability of detection services. In the majority of the instances, however, measures did not address a governmentwide priority. For example, the measure “number of trained military working dog teams trained” was determined not to cover a governmentwide priority because it does not measure the quality or efficiency of training provided. When measures fail to cover governmentwide priorities managers may not be able to balance priorities to ensure the overall success of the program.

Reliability. We estimate that 46 percent of DOD’s performance measures have data collection methods indicated in the database that generally appear reliable. Reliability refers to whether a measure is designed to collect data or calculate results such that the measure would be likely to produce the same results if applied repeatedly to the same situation. For each entry in the database, users are directed to enter, among other information, one performance measure and its associated methodology, target, and result. However, in numerous instances the system contained multiple performance measures entered into fields that should contain only one measure. Such entries could result in errors of collecting, maintaining, processing, or reporting the data. Additionally, some measures did not provide enough information on data collection methods or performance targets to assure reliability. For example, a measure in the database states “continuous U.S. Navy ship presence in the SOUTHCOM area of responsibility.” The performance target listed for this measure is “3.5,” but to what 3.5 refers—such as days, number of ships, or percentage points—is not explained. Moreover, the methodology in the database for this measure is entered as “not applicable.” Therefore, the measure’s methodology does not provide insight into how DOD could measure whether or not it reached its target of 3.5. As a result, we determined that this measure did not have data collection methods to gather reliable results. We have previously reported that if errors occur in the collection of data or the calculation of their results, it may affect conclusions about the extent to which performance goals have been achieved.

Objectivity. We estimate that 59 percent of DOD’s performance measures for its counternarcotics activities are objective. We have previously reported that to be objective, measures should indicate specifically what is to be observed, in which population or conditions, and in what time frame, and be free of opinion and judgment. We estimate that 41 percent of DOD’s measures are not objective and could therefore face issues of bias or manipulation. For example, a measure in the database is, “percent of inland waterways controlled by Colombian Marine Corps forces.” For this measure, no criteria for “controlled” is provided and it is not clear how the Colombian government reports the percentage of waterways under its control and over what time frame this control will occur.

Clarity. We estimate that 65 percent of DOD’s performance measures exhibit the attribute of clarity. A measure achieves clarity when it is clearly stated and the name and definition are consistent with the methodology used for calculating the measure. However, we estimate that 35 percent of DOD’s measures are not clearly stated. For example, one of DOD’s measures linked to the objective of sharing information with U.S. and partner nations is “identify and establish methodology for implementation.” For this measure, no associated methodology is identified, and it is unclear what is being implemented. We have previously reported that a measure that is not clearly stated can confuse users and cause managers or other stakeholders to think that performance was better or worse than it actually was.

Measurable target. We estimate that 66 percent of DOD’s measures have measurable targets. Where appropriate, performance goals and measures should have quantifiable, numerical targets or other measurable values. Some of DOD’s measures, however, lacked such targets. For example, one performance measure identified its target as “targets developed by the local commander.” As it is not quantifiable, this target does not allow officials to easily assess whether goals were achieved because comparisons cannot be made between projected performance and actual results.

DOD Is Working To Improve Its Counternarcotics Performance Measures, but Implementation Challenges Exist

DOD officials have acknowledged that weaknesses exist in the department’s current set of counternarcotics performance measures. In May 2010 DOD issued revised guidance for its counternarcotics performance measurement system to guide users in establishing performance measures that more accurately capture the quantitative and qualitative achievements of DOD’s activities. To do this, the guidance states that performance measures should be, among other attributes, useful for management and clearly stated. The guidance describes different types of performance measures that can be used to monitor DOD’s contribution to its strategic counternarcotics goals, such as those that measure DOD’s efficiency, capability, and effectiveness at performing its activities. Additionally, according to the guidance, DOD components should provide evidence of the quality and reliability of the data used to measure performance.

However, DOD officials noted four specific challenges that the department faces in developing performance measures consistent with its revised guidance.

Creating performance measures that assess program outcomes. Some DOD officials noted that, because DOD acts as a support agency to partner nations and other law enforcement entities—and the actual interdiction of drugs is conducted by other entities—measuring the outcome of DOD’s performance is difficult. While developing outcome measures can be challenging, we have found that an agency’s performance measures should reflect a range of priorities, including outcomes. Moreover, we have found that methods to measure program outcomes do exist. For example, agencies have applied a range of strategies to develop outcome measures for their program, such as developing measures of satisfaction based upon surveys of customers. In addition, officials from EUCOM, AFRICOM, and JIATF-S stated that while developing outcome performance measures can be difficult, developing such measures for support activities is possible and is done at other federal agencies. For example, EUCOM indicated it could track the outcome of the support it provides to partner nations by tracking the annual percentage increase in interdictions and arrests related to illicit trafficking. Additionally, JIATF-W indicated that it conducts quarterly command assessments of current programs, which focus on aligning resources provided by JIATF-W to the outcomes of its law enforcement partners.

Implementing revisions in a timely manner. DOD officials noted that implementing revisions to the department’s performance measures in a timely fashion will be difficult given that such revisions are resource and time intensive. Further, while including dates for submission, DOD’s revised guidance does not clearly specify a time frame by which DOD components should revise the counternarcotics performance measures that are to be submitted to the database. We have previously reported that establishing timetables for the development of performance measures can create a sense of urgency that assists in the effort being taken more seriously. DASD-CN> officials noted that time frames by which DOD’s measures would be revised are being discussed. However, these officials do not expect new performance measures to be established in fiscal year 2010, and said that fiscal year 2011 would be the earliest year of full implementation of the guidance.

Ensuring adequate resources are available. DOD officials noted that ensuring adequate resources—such as expertise and training in performance management—are available to develop performance measures at both DASD-CN> and the combatant commands will be a challenge. These officials noted that DOD employees tasked with developing performance measures and tracking the progress towards achieving goals are not sufficiently trained to design and monitor outcome performance measures. We have previously reported that access to trained staff assists agencies in their development of performance measures.

Ensuring reliable data. DOD officials noted that ensuring data used to measure DOD performance are reliable is challenging. To measure the performance of its counternarcotics activities DOD officials told us they rely heavily on external sources of data, such as U.S. law enforcement agencies and foreign government officials. This challenge can pose issues for DOD regarding data verification and ensuring proper information is recorded for performance measures.

DOD Rarely Uses the Performance Information Contained in Its Performance Measurement System to Manage Its Counternarcotics Activities and Has Applied Few Practices to Facilitate Its Use

DOD makes limited use of its performance measurement system to manage its counternarcotics activities and has applied few practices to facilitate its use. We have found that the full benefit of collecting performance information is realized only when managers use the information to inform key decisions. While DOD has applied some practices to facilitate the use of the performance information in its system, it does not utilize certain key practices, such as frequently and effectively communicating performance information. Absent an effective performance management system, DOD lacks critical information to use to improve the management and oversight of its counternarcotics activities.

Agencies Can Use Performance Information to Manage for Results

We have previously reported that, in addition to measuring performance, effective performance measurement systems include steps to use information obtained from performance measures to make decisions that improve programs and results. We identified several ways in which agencies can use performance information to manage for results, including using data to (1) identify problems and take corrective actions, (2) develop strategy and allocate resources, and (3) identify and share effective approaches.

DOD Submits Performance Reports to ONDCP, But Makes Limited Use of the Information in Its Performance Measurement System to Manage and Oversee Its Counternarcotics Activities

DOD officials representing DASD-CN>, AFRICOM, CENTCOM, EUCOM, NORTHCOM, SOUTCOM, JIATF-S, and JIATF-W told us they rarely use information from DOD’s counternarcotics performance measurement system to manage counternarcotics activities. Specifically, they rarely use the system to: Identify problems and take corrective actions. Agencies can use performance information to identify problems or weaknesses in programs, to try to identify factors causing the problems, and to modify a service or process to try to address problems. DOD officials representing DASD- CN> and SOUTHCOM told us that they currently make limited use of the performance information in DOD’s performance measurement system to manage counternarcotics activities. Officials from DASD-CN> stated that they use data from the performance measurement system to produce reports for ONDCP, which may include information identifying problems in the implementation of DOD’s counternarcotics activities. However, in reviewing these documents, we found that the reports did not include a clear assessment of DOD’s overall progress toward its counternarcotics goals. For instance, the report submitted to ONDCP for fiscal year 2009 contained detailed information on 6 of DOD’s 285 counternarcotics performance measures, but did not clearly explain why the results of these 6 measures would be critical to the success of DOD’s counternarcotics program. Moreover, according to ONDCP, DOD’s reports for fiscal years 2007, 2008, and 2009 did not fulfill the requirements of ONDCP’s guidance because the reports were not authenticated by the DOD-IG.

Further, officials from AFRICOM, CENTCOM, EUCOM, NORTHCOM, JIATF-S, and JIATF-W told us they do not use the DOD’s performance measurement system to manage counternarcotics activities. While these officials indicated that they submitted performance information to the system’s database as required by DOD guidance, they stated they tend to manage programs using information not submitted to the system (see table 4). For example, CENTCOM officials told us information obtained in weekly program meetings regarding the timeliness and cost of counternarcotics projects, not data sent to the system’s database, is most often used to help them identify problems and make program adjustments.

Recognizing the need improve the information in the system’s database, officials from DASD-CN> told us that for fiscal year 2011 they are working with DOD components to integrate performance information into the system’s database that can be more useful for decision making. Officials from several combatant commands stated they could integrate performance information obtained from outside sources into the counternarcotics performance measurement system. Officials from JIATF- S, for example, told us they collect and analyze a variety of data on counternarcotics activities that they do not input into DOD’s counternarcotics performance measurement system. On a daily basis, JIATF-S collects information on “cases”—that is, boats or planes suspected of illegal trafficking. In addition to tracking the number of cases, JIATF-S compiles information as to whether or not a particular case was targeted, detected, or monitored, and whether or not those actions resulted in interdictions or seizures of illegal drugs. By compiling this information, officials at JIATF-S told us they can better identify program outcomes, areas in which their efforts are successful, and ways to take corrective actions.

Develop strategy and allocate resources. Agencies can use performance information to make decisions that affect future strategies, planning, and budgeting, and allocating resources. DASD-CN>’s role includes both defining the strategic goals and managing the budgeting system of the DOD counternarcotics program. DOD’s counternarcotics guidance states that information from the counternarcotics performance measurement system will inform strategic counternarcotics plans, but it does not clearly state how the system will be used to inform decisions to allocate resources. Moreover, officials from DASD-CN> told us that the office does not currently link performance information from the counternarcotics performance measurement system’s database directly to budget allocation decisions. In addition, our analysis of DOD’s fiscal year 2011 Drug Interdiction and Counterdrug Activities Budget Estimates— which provides details on DOD’s fiscal year 2011 budget request for its counternarcotics activities—identified no clear link between budget allocation decisions and performance information in the system’s database. DOD officials told us they plan to incorporate performance information from the counternarcotics performance measurement system into future budget requests provided to Congress.

Identify and share effective approaches. We have reported that high- performing organizations can use performance information to identify and increase the use of program approaches that are working well. According to DOD’s counternarcotics performance measurement system guidance, DASD-CN> will use performance information submitted to the system’s database to compile reports for ONDCP, which DASD-CN> has done. However, DASD-CN> officials told us they do not currently use the system to produce reports for DOD components, which could assist in identifying and sharing effective approaches between DOD’s components. While indicating performance reports could be a useful tool, officials from several DOD components told us they had not received such reports from DASD-CN>. DOD’s May 2010 guidance does not state whether the system will be used to produce such reports in the future.

DOD Has Applied Few Practices to Facilitate the Use of Its Counternarcotics Performance Measurement System

We have found that agencies can adopt practices that can facilitate the use of performance data. These include (1) demonstrating management commitment to results-oriented management; (2) aligning agencywide goals, objectives, and measures; (3) improving the usefulness of performance data to better meet management’s needs; (4) developing agency capacity to effectively use performance information; and (5) communicating performance information within the agency frequently and effectively.

As part of its role overseeing DOD’s counternarcotics activities, DASD- CN> manages the DOD counternarcotics performance measurement system. DASD-CN> applies some practices to facilitate the use of its counternarcotics performance measurement system. For example, DASD- CN> has recently taken steps to demonstrate management commitment by issuing revised guidance emphasizing the development of improved performance measures and, according to DASD-CN> officials, conducting working groups with some DOD components to assist them in revising performance measures. Moreover, DASD-CN> officials told us they are taking steps to increase staffing to better oversee the performance measurement system. We have found that the commitment of agency managers to result-oriented management is critical to increased use of performance information for policy and program decisions. Further, DASD-CN> has created a results framework that aligns agencywide goals, objectives, and performance measures for its counternarcotics activities. As we have previously reported, such an alignment increases the usefulness of the performance information collected by decision makers at each level, and reinforces the connection between strategic goals and the day-to-day activities of managers and staff.

However, DASD-CN> has not applied certain key practices to facilitate the use of data, such as improving the usefulness of performance information in its performance measurement system, developing agency capacity to use performance information, and communicating performance information frequently and effectively. Furthermore, DOD officials told us they face challenges using DOD’s performance measurement system to manage their activities due to (1) the limited utility of the performance measures and data currently in DOD’s counternarcotics database, (2) insufficient capacity to collect and use performance information, and (3) infrequent communication from DASD- CN> regarding performance information submitted to the database. For instance, DOD’s guidance emphasizes the development of performance measures that are, among other attributes, useful for management and supported by credible data. However, DOD officials from several combatant commands told us that the performance measures and targets currently in the system are of limited utility and will need to be revised. Moreover, officials from several DOD components emphasized the need to build additional capacity to use performance data, such as receiving training on how to revise performance standards and measures. We have found that the practice of building analytical capacity to use performance information—both in terms of staff trained to do analysis and availability of research and evaluation resources—is critical to an agency using performance information in a meaningful way. Finally, DOD components told us that they received little feedback or direction from DASD-CN> regarding performance information they submitted to the system. We have previously reported that improving the communication of performance information among staff and stakeholders can facilitate the use of performance information in key management activities. For more information see table 5.

Conclusions

DOD reported more than $1.5 billion in fiscal year 2010 for its counternarcotics activities, but has not yet developed an effective performance measurement system to readily inform progress toward the achievement of its counternarcotics goals. We have previously reported that performance measurement systems include steps to measure performance to gauge progress and use the information obtained to make key management decisions. DOD acknowledges weaknesses in its performance measurement system and has taken steps to improve the system, such as revising its guidance for the development of performance measures and holding working groups with DOD components. However, its current set of measures lack key attributes of successful performance measures, such as balance, objectivity, and reliability. Moreover, DOD infrequently uses the information presently in its counternarcotics performance measurement system and has yet to fully apply key practices to facilitate its use. Absent an effective performance measurement system, DOD lacks critical performance information to use to improve its management decisions, eliminate wasteful or unproductive efforts, and conduct oversight of its activities.

Recommendations for Executive Action

To improve DOD’s performance measurement system to manage and oversee its counternarcotics activities, we recommend that the Secretary of Defense take the following two actions: 1. To address weaknesses identified in DOD’s counternarcotics performance measurement system, we recommend that the Secretary of Defense direct the Deputy Assistant Secretary for Counternarcotics and Global Threats to review the department’s performance measures for counternarcotics activities and revise the measures, as appropriate, to include the key attributes of successful performance measures previously identified by GAO. 2. To address factors associated with the limited use of DOD’s counternarcotics performance measurement system, we recommend that the Secretary of Defense direct the Deputy Assistant Secretary for Counternarcotics and Global Threats to apply practices that GAO has identified to facilitate the use of performance data.

Agency Comments and Our Evaluation

We provided a draft of this report to DOD and ONDCP for their review and comment. We received written comments from DOD, which are reprinted in appendix II. DOD concurred with our recommendations, and stated it has developed and begun to implement a plan to improve the quality and usefulness of its counternarcotics performance measurement system. ONDCP did not provide written comments.

We received technical comments from DOD and ONDCP, which we have incorporated where appropriate.

We are sending copies of this report to interested congressional committees, the Secretary of Defense, and the Director of the Office of National Drug Control Policy. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-4268 or fordj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

Section 1016 of the National Defense Authorization Act for Fiscal Year 2010 directed GAO to report on the Department of Defense’s (DOD) performance measurement system used to assess its counternarcotics activities. In response to this mandate, we examined the extent to which (1) DOD’s counternarcotics performance measurement system enables DOD to track progress and (2) DOD uses performance information from its counternarcotics performance measurement system to manage its activities.

Our work focused on the efforts of DOD to develop an effective counternarcotics performance measurement system. Within DOD, we spoke with officials from several relevant components involved in the management, oversight, and implementation of DOD’s counternarcotics activities, including the Office of the Deputy Assistant Secretary of Defense for Counternarcotics and Global Threats (DASD-CN>), U.S. Africa Command (AFRICOM), U.S. Central Command (CENTCOM), U.S. European Command (EUCOM), U.S. Northern Command (NORTHCOM), U.S. Southern Command (SOUTHCOM), the Joint Interagency Task Force- South (JIATF-S), the Joint Interagency Task Force-West (JIATF-W), and the DOD Inspector General (DOD-IG). We also discussed DOD efforts with officials from the Office of National Drug Control Policy (ONDCP), the organization that establishes U.S. counternarcotics goals and coordinates the federal budget to combat drugs.

To examine the extent to which DOD’s counternarcotics performance measurement system enables the department to track its progress we analyzed DOD strategy, budget, and performance documents, such as DOD’s Counternarcotics Strategy, Drug Interdiction and Counterdrug Activities Budget Estimates, and Performance Summary Reports. We reviewed relevant DOD and ONDCP guidance on performance measures, such as DOD’s Standard Operating Procedures for the Counternarcotics Performance Metrics System and ONDCP’s Drug Control Accounting circular. Further, we evaluated a generalizable random sample of DOD’s fiscal year 2009 counternarcotics performance measures (115 of 239 measures) to assess the extent to which these measures adhered to GAO criteria on the key attributes of successful performance measures. Because we followed a probability procedure based on random selections, our sample is only one of a large number of samples that we might have drawn. Since each sample could have provided different estimates, we express our confidence in the precision of our particular sample’s results at a 95 percent confidence interval (e.g., plus or minus 6 percentage points). This is the interval that would contain the actual population value for 95 percent of the samples we could have drawn. To evaluate the sample, two analysts independently assessed each of the performance measures against nine attributes of successful performance measures identified by GAO. Those analysts then met to discuss and resolve any differences in the results of their analyses. A supervisor then reviewed and approved the final results of the analysis. In conducting this analysis, we analyzed information contained in DOD’s counternarcotics performance measurement system database and spoke with DOD officials responsible for managing counternarcotics activities and entering information into the database. We did not, however, review supporting documentation referenced but not included in the system’s database, nor did we assess other databases that might exist at the DOD component level. We also discussed DOD’s performance measures with cognizant officials from ONDCP and several DOD components, including DASD-CN>, AFRICOM, CENTCOM, EUCOM, NORTHCOM, SOUTHCOM, JIATF-S, JIATF-W, and the DOD-IG.

To evaluate the extent to which DOD uses performance information from its counternarcotics performance measurement system to support its mission, we held discussions with officials from DOD components— including DASD-CN>, AFRICOM, CENTCOM, EUCOM, NORTHCOM, SOUTHCOM, JIATF-S, and JIATF-W—to determine the ways in which these components use information from DOD’s system, as well as other sources of performance information. We also examined DOD’s Performance Summary Reports and fiscal year 2011 Drug Interdiction and Counterdrug Activities Budget Estimates to assess the extent to which these materials reported that DOD used performance information from its counternarcotics performance measurement system database. Further, we analyzed the extent to which DOD applies key management practices previously identified by GAO to facilitate the use of performance information from its counternarcotics performance measurement system. We also traveled to Tampa, Miami, and Key West, Florida where we visited CENTCOM, SOUTHCOM, and JIATF-S. In these visits, we met with DOD officials responsible for management and implementation of counternarcotics activities to discuss DOD’s use of performance data to support its counternarcotics mission.

To determine the completeness and consistency of DOD funding data, we compiled and compared data from DOD with information from cognizant U.S. agency officials in Washington, D.C. We also compared the funding data with budget summary reports from the ONDCP to corroborate their accuracy. Although we did not audit the funding data and are not expressing an opinion on them, based on our examination of the documents received and our discussions with cognizant agency officials, we concluded that the funding data we obtained were sufficiently reliable for the purposes of this report.

We conducted this performance audit from December 2009 to July 2010 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Defense

Appendix III: Contact and Staff Acknowledgments

Contact

Staff Acknowledgments

In addition to the individual named above, Juan Gobel, Assistant Director; Elizabeth Curda; Martin de Alteriis; Karen Deans; Mark Dowling; Justin Fisher; Richard Geiger; Eileen Larence; Marie Mak; Christopher Mulkins; John Pendleton; Elizabeth Repko; and Mark Speight made key contributions to this report.