Office of National Drug Control Policy: Office Could Better Identify Opportunities to Increase Program Coordination
Released: 2013-04-25T00:00:00

Letter

Background

Office of National Drug Control Policy

ONDCP was established by the Anti-Drug Abuse Act of 1988 to, among other things, enhance national drug control planning and coordination and represent the drug policies of the executive branch before Congress. In this role, the office is responsible for (1) developing a national drug control policy, (2) developing and applying specific goals and performance measurements to evaluate the effectiveness of national drug control policy and National Drug Control Program agencies’ programs, (3) overseeing and coordinating the implementation of the national drug control policy, and (4) assessing and certifying the adequacy of the budget for National Drug Control Programs.

The 2010 Strategy is the inaugural strategy guiding drug policy under President Obama’s administration. According to ONDCP officials, it sought a comprehensive approach to drug policy, including an emphasis on drug abuse prevention and treatment efforts and the use of evidence- based practices or interventions—approaches to prevention or treatment that are based in theory and have undergone scientific evaluation. For the 2010 Strategy, ONDCP changed its approach and moved from publishing a 1-year Strategy to publishing a 5-year Strategy, which ONDCP is to update annually. The annual updates, which ONDCP has issued for 2011 and 2012, are to provide an implementation progress report as well as an opportunity to make adjustments to reflect policy changes.

ONDCP established two overarching policy goals in the 2010 Strategy for (1) curtailing illicit drug consumption and (2) improving public health by reducing the consequences of drug abuse, and seven subgoals under them that delineate specific quantitative outcomes to be achieved by 2015, such as reducing drug-induced deaths by 15 percent. the achievement of these two policy goals and seven subgoals (collectively referred to as goals), the Strategy and annual updates include seven strategic objectives. These objectives are to be achieved by implementing 112 action items, with lead and participating agencies designated for each action item.

See figure 1 for additional details.

When developing the Strategy, ONDCP identified data sources for each of the seven subgoals, such as SAMHSA’s National Survey on Drug Use and Health.

Thirty-day prevalence is defined as having used within the past 30 days. Lifetime prevalence is defined as having ever used. Morbidity refers to incidence of disease.

We previously reported that ONDCP developed the objectives and action items in the Strategy through a consultative process with drug control agencies and other stakeholders because these agencies have primary responsibility for implementing them. The objectives represent broad policy areas, such as prevention, treatment, and domestic law enforcement, and collectively contribute to all of the Strategy’s goals. The action items under each objective represent the activities needed to accomplish the objective and may highlight specific drug control program activities or call for agencies to perform a specific task, such as preparing a report on the health risks of youth drug abuse.

Drug Abuse Prevention and Treatment Programs

The Strategy’s drug abuse prevention and treatment objectives are supported by action items implemented through programs across multiple agencies. Specifically, the 15 federal agencies we selected to include in our review collectively allocated about $4.5 billion in fiscal year 2012 to programs that provide drug abuse prevention and treatment services (drug abuse prevention and treatment programs) as shown in figure 2.

In 2010, Congress directed us to conduct routine investigations to identify programs, agencies, offices, and initiatives with duplicative goals and activities within departments and government wide and report annually to Congress. In March 2011 and February 2012, we issued our first two annual reports to Congress in response to this requirement.basis of the framework established in these reports, we used the following definitions for assessing drug abuse prevention and treatment programs:

Fragmentation occurs when more than one federal agency (or more than one organization within an agency) is involved in the same broad area of national interest.

Overlap occurs when fragmented agencies or programs have similar goals, engage in similar activities or strategies to achieve them, or target similar beneficiaries.

Duplication occurs when two or more agencies or programs are engaged in the same activities or provide the same services to the same beneficiaries.

Results Show Lack of Progress on Strategy Goals, although Implementation of Most Action Items Is Complete or On Track; ONDCP Established a New Mechanism to Monitor Progress

Our analysis indicates that ONDCP and federal agencies have not made progress toward achieving four of five goals established in the 2010 Strategy for which primary data are available. However, ONDCP reported strong progress in implementing most of the action items in the Strategy intended to support achievement of these goals. ONDCP officials stated that implementing these action items is necessary but may not be sufficient to achieve Strategy goals. ONDCP’s Performance Reporting System (PRS) is a new monitoring mechanism intended to provide more specific, routine information on progress and help identify causal factors for performance gaps and options for improvement. According to ONDCP officials, the office plans to report on PRS results for the first time in 2013 and assess the system’s reliability and effectiveness, which could help increase accountability for improving results and identify ways to bridge the gap between the lack of progress toward Strategy goals and the strong progress on implementing actions in the Strategy.

Results Show Lack of Progress toward Achieving Strategy Goals

ONDCP officials responsible for overseeing the development of the 2010 Strategy stated that they established the Strategy’s seven quantitative goals based in part on the availability and quality of data sources to measure the effects of drug control policy and what they considered could realistically be achieved within designated time frames, among other things. Nonetheless, these officials said that the goals are aggressive given that drug use trends for some of the measures were increasing prior to their establishment. As of March 2013, our analysis indicates that of the five goals for which data on results were available, one shows progress and four show either no change or movement away from the 2015 goals—for example, drug-induced deaths have increased, instead of decreased, from the 2009 baseline, as shown in table 1.

ONDCP and federal drug control agencies have made progress toward achieving the goal for decreasing the lifetime prevalence of eighth graders who have used drugs, alcohol, or tobacco, which consists of three separate measures. In addition, while results from the primary data source for the goal to reduce drugged driving are not available, results from ONDCP’s secondary data source indicate progress toward achieving this goal. However, progress has not been made on the other four goals for which primary data are available. For example, the 30-day prevalence of reported drug use among 12- to 17-year-olds has not changed from the 2009 baseline to 2011. According to the data source for this measure—SAMHSA’s National Survey on Drug Use and Health—this is due primarily to an increase in the rate of reported marijuana use, offset by decreases in the rates of reported non-medical use of prescription drugs and inhalants. The survey indicates that marijuana accounts for almost 80 percent of illicit drug use among 12- to 17-year-olds. In addition, drug-related morbidity, as measured by the number of emergency room visits for drug misuse and abuse and HIV infections attributable to drug use, increased from 2009 to 2010 and is higher than the established goals for these measures. The data source for the number of emergency room visits for drug misuse and abuse— SAMHSA’s Drug Abuse Warning Network—indicates that misuse and abuse of prescription drugs accounted for the majority of these visits in 2010 (58 percent).

ONDCP Monitors Progress on Implementing Strategy Action Items and Reported That Most Are Complete or on Track

In August 2010, ONDCP instituted a process to track the implementation status of action items established to achieve the objectives, and ultimately the goals, of the Strategy. Agencies that have lead responsibility for implementing action items submit progress reports to ONDCP on an annual basis. The office reviews these updates and information from other sources, such as interagency meetings and agency budget submissions, and classifies action items into five categories—complete, on track, delayed but progressing, facing budget issues, and at risk. Despite the lack of progress we found toward meeting the Strategy’s goals, ONDCP reported in its most recent publication of results in July 2012 that 96 percent (107 of 112) of the action items are complete or on track, as shown in figure 3.

We previously reported in July 2012 that ONDCP monitors and reports on the implementation status of action items to, among other things, help hold agencies accountable for implementing the action items; give credit to agencies for progress made; and motivate them, if needed, to take steps to address delays. Officials we interviewed from all seven agencies we selected stated that Strategy action items reflect their already existing priorities or activities. According to ONDCP officials, this alignment helps facilitate implementation. ONDCP officials stated that the implementation of Strategy action items is necessary to achieve Strategy goals but may not be sufficient. According to these officials, a variety of factors could affect achievement of these goals, such as worsening economic conditions, changing demographics, or changing social or political environments; the passage of state laws that decriminalize marijuana use or allow its use for medical purposes; failure to obtain sufficient resources to address drug control problems; insufficient commitment from agency partners; and the need for new action items that include initiatives or activities beyond those that are under way or ONDCP officials stated that the PRS is to provide more planned.specific information about where the Strategy is on or off track and prompt diagnostic reviews to identify causal factors contributing to any problems identified, as discussed below.

ONDCP Has a New System in Place to Monitor Progress toward Goals and Plans to Define How the System Will Be Used to Assess Progress and Review Its Effectiveness

In April 2012, ONDCP established the PRS to monitor and assess progress toward meeting Strategy goals and objectives, and the office issued a report (the PRS report) describing the system with the 2012 Strategy update. The PRS includes interagency performance measures For example, one of the six and targets under each Strategy objective.performance measures under the objective to strengthen efforts to prevent drug use in our communities is the average age of initiation for all illicit drug use, which has a 2009 baseline of 17.6 years of age and a 2015 target of 19.5 years of age. Similarly, one of the four performance measures under the objective to seek early intervention opportunities in health care is the percentage of respondents, aged 12 to 17, who have used prescription-type drugs nonmedically in the past year. This measure has a 2009 baseline of 7.7 percent and a 2015 target of 6.5 percent. According to ONDCP officials, the PRS augments ONDCP’s current method of examining performance by allowing the office to assess the effectiveness of interagency efforts to achieve the Strategy’s objectives.ONDCP officials stated that they plan to release the results of the PRS measures for the first time in 2013. According to the PRS report, system information is to be used to inform budget formulation and resource allocation, Strategy implementation, and policy making, among other things.

We assessed PRS measures and found them to be generally consistent with attributes of effective performance management identified in our prior work as important for ensuring performance measures demonstrate For example, the PRS results and are useful for decision making. measures are clear, have limited overlap, and have measurable targets. Specifically, the measures are clearly stated, with descriptions included in the 2012 PRS report, and all 26 of them have or are to have numerical targets. In addition, the measures were developed with input from stakeholders through an interagency working group process, which included participants from Education, DOJ, and HHS, among others.According to the PRS report, the working groups brainstormed candidate performance measures for each Strategy objective and assessed them based on criteria such as being clear, quantifiable, and valid indicators for the objective. The groups also identified data sources, such as national surveys on drug use and treatment services, to use to report on these measures and evaluated the data sources to determine if they were unbiased and collected data routinely, among other things.

The PRS report states that the system is to provide early warning about progress toward achieving Strategy objectives. However, it does not clearly define how progress toward 2015 targets for PRS measures is to be assessed and what actions are to be taken if intermediate results are not on track to meet these targets. ONDCP officials stated that they plan to provide information about the process to assess intermediate performance results in the PRS report that is to be issued in 2013. These officials said that the report is still in development, but that for each PRS measure, they plan to compare the linear trend line from the baseline to the 2015 target with results to date; examine auxiliary data sources, such as other relevant national surveys, to refine the assessment; and work with interagency subject area experts to arrive at a final assessment. ONDCP officials stated that when results are determined to not be on track to meet 2015 targets, the PRS is to serve as a trigger for an interagency review of potential causes of performance gaps and options for improvement. These proposed actions are consistent with leading practices that we have previously identified regarding using intermediate goals and measures to show progress or contributions to intended results and clearly documenting significant events to help ensure that management directives are carried out.

According to ONDCP officials, information collected through the PRS is to provide valuable insights to help identify where the Strategy is on track and when further problem solving and evaluation are needed. However, the system is still in its early stages and, as of March 2013, operational information is not available to evaluate its effectiveness. ONDCP officials stated that as part of the annual process to assess PRS results, they plan to review the measures to determine the extent to which they reliably capture agency performance and whether there is a need to modify them. Further, the 2012 PRS report states that ONDCP plans to continue to improve measures as better data sources become available. Officials said that they plan to address any modification of the measures in the 2013 PRS report. According to these officials, ONDCP plans to assess the effectiveness of the PRS more comprehensively to determine how well it is working and whether any adjustments need to be made after the system has been operational for a longer period of time. Such actions are consistent with Standards for Internal Control in the Federal Government which calls for reviewing established performance measures to validate their propriety and integrity.

ONDCP’s plans to clearly define and document its performance assessment process and evaluate how well the PRS and its measures are working should help increase accountability for improving results and enhance the system’s effectiveness as a mechanism to monitor progress toward Strategy goals and objectives and assess where further action is needed to improve progress. These actions could also help ONDCP identify ways to bridge the gap between the lack of progress toward Strategy goals and the reported strong progress agencies are making in implementing action items intended to facilitate the achievement of those goals.

Drug Abuse Programs Are Fragmented and Overlapping; Many Reported That They Did Not Coordinate, and ONDCP Has Not Assessed Extent of Overlap and Coordination Opportunities

Drug abuse prevention and treatment programs are fragmented across 15 federal agencies that funded or administered 76 programs in fiscal year 2011, and we identified overlap in 59 of these programs. Agency officials who administer programs in two areas that we reviewed in detail—that is, programs for youth and programs for offenders—reported making various efforts to coordinate program activities, which, according to our previous work, can minimize the risk of duplication. However, 29 of 76 (about 40 percent) surveyed programs reported no coordination with other federal agencies on drug abuse prevention or treatment activities. ONDCP has not assessed all drug abuse prevention or treatment programs to identify the extent of overlap and potential duplication and opportunities for coordination.

Drug Abuse Programs Are Fragmented across 15 Federal Agencies, and Many Are Overlapping, Which May Result in Inefficiencies

Federal drug abuse prevention and treatment programs are fragmented across 15 federal agencies that administered 76 programs in fiscal year 2011. Agencies reported that they administered four types of programs, specifically:

22 programs were drug abuse prevention programs;

21 programs were drug abuse treatment programs;

13 programs were both drug abuse prevention and treatment

20 programs were neither drug abuse prevention nor treatment programs, but programs that may provide or fund drug abuse prevention or treatment services to support other program objectives.

Table 2 provides the number of these types of programs within the 15 federal agencies, and appendix III provides additional information on the programs each agency administered in fiscal year 2011.

Furthermore, we identified overlap among the prevention and treatment programs we reviewed. As we have previously reported, overlapping programs can lead to target populations being eligible for similar services through multiple programs. However, overlapping programs are not necessarily duplicative, if the services provided and the populations Of the programs we reviewed, 59 of served differ in meaningful ways.the 76 programs (nearly 80 percent) are overlapping because they can provide or fund at least one drug abuse prevention or treatment service that at least one other program can provide or fund, either to similar population groups or to reach similar program goals. For example, six programs reported that they can provide or fund drug abuse prevention services for students and youth in order to support program goals of preventing drug use and abuse among young people. All six of the programs reported that they can provide or fund services to conduct outreach and educate youth on drug use. In addition, all of these programs also reported that they can provide or fund other similar services—such as public advertising and media campaigns or workplace education and training—with at least one other program. (See fig. 4.)

In addition, 15 programs reported that they can provide or fund many of the same prevention and treatment services provided to the offender population, that is, those individuals involved in the criminal justice system. Specifically, these programs reported that they can provide or fund many of the same treatment services in order to support similar program goals of identifying and meeting the treatment needs of offenders and providing services to reduce recidivism and facilitate reentry. For example, 12 of the 15 programs can provide or fund medical evaluations and different forms of therapy, including individual and family therapy, as shown in figure 5.

In addition, programs with the goals of (1) expanding state and community capacity to prevent drug abuse and (2) addressing underage drinking also demonstrated overlap. For example, 9 programs reported that they can provide or fund drug abuse prevention and treatment services to almost all population groups listed in our survey in support of program goals to expand the capacities of state- and community-level entities to prevent drug abuse. Prevention services that could be provided by these programs included youth education, family education and support services, and public outreach activities. Furthermore, 4 programs with the goal of reducing underage drinking also reported providing similar types of prevention services, including youth education and public outreach activities.

Fragmentation and overlap in some programs have resulted from agency efforts to meet specific areas of national need, among other things. For example, SAMSHA officials told us that DOJ’s and states’ expansion of funding for drug courts in the 1980s resulted in a strain on the number of community substance abuse treatment services available to drug courts. SAMHSA recognized the need for additional funding in this area and began to fund the expansion and enhancement of substance abuse treatment services specifically available to drug courts. In addition, having multiple programs that can provide similar services to similar beneficiaries can help to fill gaps in services for beneficiaries. However, this fragmentation and overlap could also make it challenging to develop a coordinated federal approach to providing drug abuse prevention and treatment services because of the number of different federal agencies involved, and could result in potential inefficiencies among programs providing similar services. Specifically, our body of work on fragmentation and overlap has found that agencies often can realize a range of benefits, such as improved customer service and decreased administrative burdens and cost savings from addressing issues related to fragmentation and overlap, as we discuss below.

Many Programs Reported That They Do Not Coordinate, and ONDCP Has Not Assessed the Extent of Overlap and Identified Coordination Opportunities to Help Improve Efficiencies

Coordination Efforts in Two Areas with Overlap Helped Minimize Risk of Duplication, but Not All Programs Reported Such Efforts

Agency officials who administer programs in two areas that we reviewed in more detail—specifically, programs for youth and programs for offenders—reported making various efforts to coordinate overlapping program activities or services, which can serve to minimize the risk of duplication. However we found that not all programs surveyed reported coordinating with other federal agencies on drug abuse prevention or treatment activities in the year prior to our survey. For those areas that we reviewed in more detail, we found the following:

Prevention services for student and youth: While 6 programs that provide services to youth are overlapping, the risk of potential duplication among these programs may be low because of interagency coordination efforts. Officials overseeing these programs from Education, ONDCP, and SAMHSA reported that the agencies coordinate to improve program efficiencies. For example, using an interagency agreement, Education jointly administers the Safe Schools/Healthy Students program with DOJ and HHS to provide complementary educational, mental health, and law enforcement services to help prevent youth violence and drug use. Similarly, SAMHSA and ONDCP maintain an interagency agreement to jointly administer the Drug Free Communities Support program. Officials from SAMHSA said that the agreement defines the roles and responsibilities of the two agencies, and establishes agreed-upon standard operating procedures.

In addition, officials from Education, ONDCP, and SAMHSA reported that some programs and the services they can provide or fund are distinct because they target specific subgroups among students and youth, or they differ in scope. For example, the 21st Century Community Learning Center program allows for additional uses of funds that are not related to drug abuse prevention, like tutoring and mentoring, and does not require that grantees include drug abuse prevention as a program component. Officials from Education said that this indicates a difference in scope from the Safe Schools/Healthy Students program, which requires grantees to include drug abuse prevention services as a main program component. These officials also reported taking steps to identify opportunities for increasing efficiencies. For example, in its fiscal year 2013 budget justification, Education proposed consolidating several existing programs that seek to help schools provide programs and activities including alcohol, drug, and violence prevention. According to Education officials, the consolidation would more effectively target resources and address the needs of grantees.

Prevention and treatment services for offenders: Officials from the four agencies overseeing the programs that can provide or fund treatment and prevention services to the offender population also said that the agencies conduct coordination efforts to help ensure that programs provide complementary services to this population, which can minimize the risk for potential duplication. OJP and SAMHSA officials reported that both agencies coordinate to ensure that these programs provide funding for different program elements. For example, according to OJP and SAMHSA officials, SAMHSA funding for drug courts is used for treatment services, while OJP funding for drug courts is used for administrative or case management purposes. While OJP is not restricted from funding the same treatment services SAMHSA can fund, officials from both agencies said that they use multiple coordination mechanisms to help minimize the risk of potential duplication. For example, OJP and SAMHSA jointly administer two drug court programs. For the agencies’ other programs serving offenders that are not jointly administered, SAMHSA officials told us they share requests for grant applications and information on potential awardees with OJP officials to ensure that grants are not awarded to the same grantee for the same purpose. We also previously reported that program overlap is minimal and the risk of potential duplication is low among OJP’s and SAMHSA’s offender programs that assist former inmates or inmates preparing for release from federal, state, and local correctional facilities.

Additionally, officials from OJP and SAMHSA reported that their programs specifically serve offenders in the state and local justice systems, while the Bureau of Prisons and the Administrative Office of the United States Court’s programs specifically serve offenders who are or were incarcerated in federal prisons, which results in these programs having a low risk for duplication. In addition, these officials reported that the two agencies regularly share information and coordinate on prerelease planning for inmates in federal prisons and on transitioning inmates from prison to court-ordered drug testing and treatment after release, or vice versa.

Coordination efforts among the programs included in the two areas we reviewed in detail were consistent with practices that we have previously reported federal agencies use to implement collaborative efforts. These efforts could address crosscutting issues that may reduce potentially duplicative, overlapping, or fragmented efforts. However, 29 of the 76 programs (about 40 percent) surveyed reported no coordination with other federal agencies on drug abuse prevention or treatment activities. As we have previously reported, because fragmentation across agencies can create an environment in which programs are not delivered as efficiently and effectively as possible, coordination across government is essential.interagency coordination efforts among the programs that did not report Therefore, there may be additional opportunities to implement any of these efforts, to identify potential efficiencies that better leverage available resources and minimize overlap and potential duplication.

While ONDCP has identified activities related to the development and implementation of the Strategy and Drug Control Budget that promote coordination, it has not systematically assessed all drug abuse prevention or treatment programs to examine the extent of overlap and potential for duplication and identified opportunities for coordination among programs to more efficiently use limited resources. According to ONDCP officials, the office’s current processes are sufficient to determine the extent of overlap and potential for duplication. Specifically, officials from ONDCP and other agencies with whom we spoke reported that they have made efforts to promote coordination through the Strategy, which emphasizes the importance of agencies collaborating on the Strategy’s goals. In addition, ONDCP officials stated that as part of the office’s annual process for developing the National Drug Control Budget submission, they review prevention and treatment programs for which funding is requested to verify that the programs serve unique needs and populations. ONDCP officials stated that their familiarity with the programs and information obtained from outreach at the programmatic level and interagency meetings, among other things, help in this review.

However, the purpose of the interagency meetings and other efforts to facilitate coordination is to develop and implement the Strategy and not to identify overlap or potential duplication. Furthermore, the purpose of the budget process is to develop a consolidated funding request to implement the Strategy and help ensure that the Strategy is adequately resourced rather than to identify overlap or potential duplication across all programs. Accordingly, ONDCP has not conducted a systematic assessment of all federal drug abuse prevention and treatment programs, including those not captured in the budget, and the services they are allowed to provide to determine the extent to which they overlap and where opportunities exist to pursue coordination strategies in order to more efficiently use limited resources. Further, ONDCP officials stated that there is no overlap among these programs, but our analysis of prevention and treatment services identified overlap in 59 of them. Standards for Internal Control in the Federal Government highlights the importance of having access to operational and other data to determine whether programs are meeting goals for accountability and efficient use of resources. Additionally, The Standard for Project Management states that to ensure related projects are managed to achieve more benefits than could be achieved with stand-alone efforts, management should coordinate common activities or programs and the efficient use of resources across activities.activities across organizations are to achieve desired benefits.

This can include such efforts as mapping out how various Further, our previous work on characteristics of effective national strategies states that such strategies should include mechanisms for coordinating agency implementation efforts, which could entail the identification of specific processes for coordination. As discussed earlier, officials from 29 of the 76 programs (about 40 percent) surveyed reported that they did not participate in any coordination efforts over the past year. Additionally, there may be other opportunities for improved efficiencies among agencies that have some coordination efforts in place. For example, while SAMHSA and OJP officials have made efforts to provide complementary drug court services, a SAMHSA official stated that a drug court awardee using SAMHSA funds to provide treatment services and OJP funds to administer the drug court still must meet complex, separate reporting requirements to each agency for such things as budget and performance management information. Our previous work has shown that fragmented services with separate agency requirements can be burdensome, difficult, and costly for service providers. ONDCP is uniquely situated to conduct an assessment across the 76 programs engaged in drug abuse prevention and treatment activities to identify the extent of overlap and potential for duplication and prospective processes for coordination. ONDCP could use the results of our analysis as a starting point for conducting this assessment. Such actions would better position ONDCP to help ensure that federal agencies undertaking similar prevention and treatment efforts identify opportunities for increased efficiencies, such as using coordination mechanisms to mitigate the risk of duplication and reducing administrative burdens on grantees, and better leverage available resources. These mechanisms could include, for example, joint program administration established through interagency agreements, sharing requests for grant applications, or joint performance measurement reporting among programs serving the same grant or cooperative agreement awardees.

Agencies We Reviewed Have Completed Few Program Evaluations, with Three Looking at Program Effectiveness, but Have Taken Other Steps to Ensure Programs Are Effective

While agencies in our review had completed few program evaluations since 2007, in part because of funding constraints, more are under way or planned. ONDCP officials stated that because few programs have completed program evaluations, they rely on a range of available performance information, including agency performance metric data, to inform development and implementation of the Strategy. In addition, agency officials said that they have taken other steps to help ensure that programs supporting the Strategy are effective, such as requiring or encouraging the use of interventions—drug abuse prevention or treatment practices—in their programs for which there is research or evidence that the interventions work.

Agencies in Our Review Have Completed Few Program Evaluations, but More Are Under Way or Planned

Program evaluations allow for comprehensive assessments of whether programs are achieving desired results to help allocate scarce resources to effective interventions, among other things. Program evaluation is closely related to performance measurement and reporting; evaluations have been used to supplement performance reporting by measuring results that are too difficult or expensive to assess annually or by exploring why program goals were not met. Program evaluations can tell agencies how well their programs are working, suggest options for improving program performance, and assist in program management. However, the drug control agencies in our review had completed few program evaluations since 2007, although more are under way or planned. Specifically, we found that 3 of the 15 agencies had completed evaluations of 6 programs since 2007, which accounts for about 8 percent of the 76 drug abuse prevention and treatment programs included in our review, and 8 agencies had started or planned 22 additional program evaluations.one or more of their programs.

Conclusions

The public health, social, and economic consequences of illicit drug use, coupled with the constrained fiscal environment of recent years, highlight the need to ensure that federal programs efficiently and effectively use their resources to address this problem. ONDCP has developed a 5-year Strategy to reduce illicit drug use and its consequences, but our analysis shows lack of progress toward achieving four of the Strategy’s five goals for which primary data are available. ONDCP established the PRS in April 2012 to monitor progress toward the Strategy’s goals and objectives and provide information to guide efforts to meet these goals. While this is promising, ONDCP does not plan to report on results until later in 2013, and until then, operational information is not available to evaluate its effectiveness.

A wide range of federal drug abuse prevention and treatment programs support the achievement of the Strategy’s goals. While some fragmentation and overlap of program services may be beneficial, they also entail potential inefficiencies that programs may not be able to afford in this era of resource constraints. Because of its responsibility for coordinating implementation of national drug control policy across the federal government, ONDCP is uniquely situated to conduct an assessment to identify overlap and potential duplication among prevention and treatment programs and coordination opportunities. Our work has identified specific areas of fragmentation and overlap of services among these programs that could be used by ONDCP, along with other information such as results from agencies’ program evaluation efforts, to help agencies begin or improve coordination efforts to help reduce or eliminate potential inefficiencies. For example, ONDCP could identify the causes for the lack of coordination among the 29 agencies that reported conducting no coordination and further examine the overlap we identified in programs focused on capacity building and underage alcohol use, as a starting point for a broader assessment across federal agencies. Such an assessment would better position ONDCP to identify opportunities for increased efficiencies, such as using coordination mechanisms to mitigate the risk of duplication, and help ensure that federal agencies undertaking similar prevention and treatment efforts better leverage available resources to achieve Strategy goals.

Recommendation for Executive Action

To identify opportunities to increase efficiencies and therefore better leverage agency prevention and treatment resources, we recommend that the Director of ONDCP assess the extent of overlap and potential for duplication across federal programs engaged in drug abuse prevention and treatment activities and identify opportunities for increased coordination. ONDCP could use our work as a starting point for this assessment.

Agency Comments and Our Evaluation

We provided a draft of this report to ONDCP; the Departments of Health and Human Services, Justice, Education, Defense, Housing and Urban Development, Labor, Transportation, Veterans Affairs, and Homeland Security; and the Federal Judiciary for review and comment. ONDCP provided written comments, which are reprinted in appendix V and summarized below. ONDCP concurred with our recommendation, and also provided perspectives on overlap among drug abuse prevention and treatment programs and progress toward achieving Strategy goals.

In its comments, ONDCP reiterated that we reported finding overlap but not actual instances of duplication among the drug prevention and treatment programs we reviewed. The office also made the points, with examples, that some overlapping programs (1) may not serve identical populations and may target different specific subgroups of a large population category, such as different types of youth age groups, and (2) may provide distinct services. We acknowledged these factors in our report, and maintain that this is why it is important to systematically review the extent of overlap among prevention and treatment programs, taking into account targeted subgroups and allowable services, to help ensure that they efficiently use limited resources to deliver these important services. ONDCP also reiterated, as we stated, that overlapping programs may provide positive benefits, such as reinforcing key prevention messages. Further, the office agreed that coordination efforts among programs can help avoid duplication and maximize program effectiveness. This is consistent with our report, which noted that overlap and fragmentation may not necessarily lead to duplication, but can create an environment in which programs are not delivered as efficiently and effectively as possible, and that coordination among programs helps to reduce the risk of duplication and increase efficiencies.

ONDCP stated that while extensive coordination of prevention and treatment programs is already taking place, there is always room for improvement, and that it will work with agencies administering these programs to further enhance coordination.

With regard to progress toward achieving Strategy goals, ONDCP stated that it is important to analyze trends for each drug category separately and noted that we recognized this in our discussion of ONDCP’s goal to reduce drug use among 12- to 17-year-olds by 15 percent by 2015. As the report states, marijuana accounts for almost 80 percent of drug use in this age group, and the lack of progress on this goal is due primarily to an increase in the rate of reported marijuana use, offset by decreases in the rates of other drug use. We also state that misuse and abuse of prescription drugs is the primary factor for one of the measures for the goal to reduce drug-related morbidity by 15 percent, which also shows lack of progress. Finally, ONDCP noted, as we did in the report, that while results from the primary data source for the goal to reduce drugged driving by 10 percent are not available, results from ONDCP’s secondary data source indicate progress.

ONDCP and the Departments of Health and Human Services, Justice, Education, Defense, Transportation, and Housing and Urban Development provided technical comments on this report that we incorporated as appropriate.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to selected congressional committees; the Director of ONDCP; Secretaries of Health and Human Services, Education, Defense, Housing and Urban Development, Labor, Transportation, Veterans Affairs, and Homeland Security; Attorney General; Director of the Administrative Office of United States Courts; and other interested parties. In addition, the report will be available at no charge on GAO’s website at http://www.gao.gov. If you or your staff have any questions about this report, please contact Eileen R. Larence at (202) 512-8777 or larencee@gao.gov or Linda T. Kohn at (202) 512-7114 or kohnl@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix VI.

Appendix I: Relatively Few Studies Have Assessed the Effect of Societal Factors on Youth Marijuana Use, and These Studies Had Limitations

Prior studies have found that a number of factors may affect marijuana use among youth, that is, individuals 17 years old and under. For example, we reported in 1993 that several risk factors were associated with marijuana use based on our review of a national longitudinal survey. Specifically, we found that young people who had reported that they engaged in delinquent activities, such as running away from home, truancy, fighting, and theft, were more likely to have used marijuana; however, we also noted that underlying factors may be associated with both drug use and delinquent activities. More recently, in 2000, a meta- analysis of 101 longitudinal studies from around the world identified a number of risk factors that predicted future marijuana use, including the availability of illicit drugs and guns, family history of antisocial behavior, positive attitudes toward substance use, low perceived risk of drug use, and rebelliousness.

Review of Studies on the Effect of Societal Factors on Youth Marijuana Use

Other factors, including state laws and changing attitudes and social norms regarding drugs, may also affect drug use. We examined studies on three of these other factors, which we refer to as societal factors, which may affect youth marijuana use. Specifically, we examined studies that addressed: (1) the passage of laws allowing for the use of marijuana for medical purposes (medical marijuana laws); (2) the reduction in penalties for marijuana possession (decriminalization); and (3) the favorable portrayal of drugs in the media (prodrug media), such as movies, television, music, and music videos. For instance, a number of states have passed varying degrees of laws related to the production, possession, or use of marijuana for varying purposes. In addition, separate from laws that may allow for the use of marijuana for certain medical purposes, some states and local governments have taken actions to reduce or eliminate certain criminal penalties for the possession of small amounts of the drug. Finally, some speculate whether media depictions of illicit drugs influence youth drug use. For example, according to one study, frequent viewing of media messages may increase the likelihood of the adoption of ideas or beliefs reflected in the media portrayals, which may influence subsequent behaviors, such as the use of illicit substances.

Through searches of databases, Internet websites, and other sources available as of June 2012, we identified 476 documents that addressed possible factors relating to youth drug use, including the societal factors described above, as well as others such as alcohol use and mental illness. We reviewed these documents to determine if the studies met our literature review criteria, which required that the study (1) address one of the three societal factors included in our review, specifically medical marijuana laws, decriminalization, or prodrug media; (2) report results on U.S. youth (age 17 and under); (3) have a national or multiple state scope; and (4) assess the effect of the factor on youth marijuana use through the use of methods such as comparison groups or statistical analyses. We identified 10 studies that assessed the effect of medical marijuana laws or decriminalization. We also identified 10 studies on prodrug media that focused on U.S. youth; however, 4 of the latter 10 studies did not have a national or multiple state scope, and none of these 10 studies assessed the effect of media on youth marijuana use. Specifically, 4 assessed the association between media and youth marijuana use, and 6 provide descriptive statistics on, for example, the prevalence of drugs and prodrug representations in various forms of media. While these studies may not assess the effect of media on youth marijuana use or include a national or a multiple state scope, we included them because they provide useful context regarding the portrayal of drugs in the media. Two social scientists and, as applicable, a statistician reviewed each of the 20 studies to determine whether the design, implementation, and analyses of the study were sufficiently sound to support the study’s results and conclusions based on generally accepted social science principals. On the basis of these reviews, we excluded 3 studies on medical marijuana laws and 1 study on decriminalization. Below, we provide information on 2 medical marijuana studies, 4 decriminalization studies, and the 10 media studies from our review.selected these studies on medical marijuana laws and marijuana decriminalization to include in this report based on the sufficiency of their methodologies; therefore, our results cannot be generalized to all research about the potential effect of these factors on youth drug use.

The studies that assessed the effect of medical marijuana laws that met our review criteria found mixed results on effects of the laws on youth marijuana use. These studies are described in table 4 below. One potential limitation to these studies may be the variation in implementation of medical marijuana laws across states. We discuss this and other potential limitations, as well as the ways that some studies addressed this issue, later in this appendix.

Three of the four studies that assessed the effect of marijuana decriminalization that met our review criteria found little to no effect of the laws on youth marijuana use. These studies, as well as one study that found an increase, are described in table 5. One potential limitation to these studies is that all four studies contained data on marijuana use that had been collected in 1990 or earlier, which may mean that their findings are not applicable to current youth drug use patterns; however, we included them in our review because they provide useful information regarding the previous effects of decriminalization. We discuss this and other potential limitations later in this appendix.

None of the studies we identified on prodrug media assessed whether exposure to prodrug media caused changes in marijuana or other drug use. Of the 10 studies we identified that address prodrug media, 4 assessed the association between such media and youth marijuana use (see table 6). Three of the four studies found that exposure to various forms of media (e.g., rap music, music, and media depicting the positive aspects of marijuana use) were associated with higher rates of marijuana or other drug use. The fourth study found that information obtained from the Internet changed reported drug-using behavior, but did not determine whether marijuana or other drug use increased or decreased as a result.

Six of the 10 studies that addressed prodrug media provided descriptive statistics on, for example, the prevalence of drugs and prodrug representations in various forms of media. Five of these studies described the nature and frequency of illicit drugs in various forms of media, including movies, television, music videos, and music. The media content from these studies ranges from 1996 through 2005, and may not reflect the current content of media. Some of the findings from these descriptive studies are described in table 7.

The sixth study assessed the perceived value of “the media” and other sources as credible providers of drug information, based on anonymous interviews with 223 high school students across two states. This study found that adolescents reported listening to their parents and teachers the most about drugs and alcohol, as compared with movie, television, music, or sports celebrities, but the results of this study cannot be generalized to a larger population.

We identified several challenges related to inherent difficulties in analyzing this subject matter or with the study designs that may limit the generalizability of these studies. For example, a study’s design may have been sufficiently sound to support the study’s results and conclusions, but it could have used old data, which may mean that the study’s findings are not applicable to current youth drug use patterns. We describe these limitations below:

Variation in implementation of medical marijuana laws: Office of National Drug Control Policy (ONDCP) officials stated that states implement medical marijuana laws differently and that these differences may influence the effect of the laws on youth marijuana use. For example, according to these officials, some states enacted laws where medical marijuana has had a very visible presence throughout the state, while other states’ laws have low visibility or have not been implemented yet. Further, ONDCP officials stated that some studies incorrectly looked at the changes in marijuana use before and after medical marijuana laws are passed instead of before and after such laws are implemented. The officials noted that this is an important distinction because the implementation of medical marijuana laws after passage can be a lengthy process. ONDCP officials also noted that the states that eventually passed medical marijuana laws tended to have higher rates of youth marijuana use before such laws were passed. However, some of the studies we reviewed attempted to account for these issues in their study designs. For example, one study used a model that controlled for unmeasured state characteristics that do not change over time, such as differences in social norms, and that might affect marijuana use and the likelihood of a state passing a medical marijuana law.

Self-reported data: As we have previously reported, research about drug use among youth is limited, in part, because most studies rely on self-reported data, and low estimates are likely because drug use is illegal. For example, youth may be unlikely to report their drug use in the survey for fear of negative repercussions, regardless of whether the results of the survey are shared with law enforcement. For our analysis, all of the 6 studies on the effects of medical marijuana laws or decriminalization on youth marijuana use as well as the 4 studies on the association between media and youth marijuana use employ self-reported data.

Use of historical data: While 4 of the 10 studies on the effect of or association between societal factors and youth marijuana use data collected since 2000 (2 medical marijuana and 2 media studies), the remainder of the studies used older data (the oldest being collected in 1975). In particular, all 4 studies on decriminalization, as well as 2 of the media studies, contained data on marijuana use that had been collected before 2000, which may mean that their findings are not applicable to current youth drug use patterns.

Appendix II: Objectives, Scope, and Methodology

This report addresses the following objectives: 1. To what extent has progress has been made toward achieving Strategy goals and does ONDCP have mechanisms in place to monitor progress? 2. To what extent does fragmentation, overlap, and duplication exist across drug abuse prevention and treatment programs, and do ONDCP and federal agencies coordinate efforts to reduce the potential for unnecessary overlap or duplication? 3. To what extent do federal agencies that have drug abuse prevention and treatment programs conduct evaluations of these programs, including assessments of program effectiveness?

In addition, we provide information on what the available research suggests about the potential effect of societal factors, such as state laws allowing the use of marijuana for medical purposes, on youth drug use.

Assessing Progress toward Strategy Goals and Mechanisms to Monitor Progress

To assess progress toward Strategy goals and the extent to which ONDCP has mechanisms in place to monitor progress, we analyzed the 2010 Strategy and 2011 and 2012 annual updates, the National Drug Control Budget, and ONDCP documentation about its Performance Reporting System and associated performance measures. We also analyzed ONDCP documents on the implementation status of Strategy action items and implementation plans and reports from selected federal drug control agencies. In addition, we analyzed agency budget justifications and strategic plans. We also reviewed information provided by ONDCP and publicly available data sources, such as the National Survey on Drug Use and Health, to determine progress toward Strategy goals.

On the basis of such factors as the number of Strategy action items for which agencies are responsible, the size of agency drug control budgets, and inclusion of a balance of drug prevention, treatment, and supply reduction missions, we selected the following seven agencies to focus on in our review: within the Department of Health and Human Services (HHS), the Substance Abuse and Mental Health Services Administration, National Institutes of Health—specifically, the National Institute on Drug Abuse and the National Institute on Alcohol Abuse and Alcoholism—and Centers for Disease Control and Prevention; within the Department of Justice (DOJ), the Office of Justice Programs (OJP)— specifically, the Bureau of Justice Assistance, Bureau of Justice Statistics, National Institute of Justice, and Office of Juvenile Justice and Delinquency Prevention—and Drug Enforcement Administration; within the Department of Homeland Security, U.S. Customs and Border Protection; and the Department of Education (Education).

While these seven agencies are not representative of all federal drug control agencies, they are responsible for implementing a majority of activities in the Strategy and provide a range of perspectives about Strategy implementation and progress. Specifically, our selected agencies and ONDCP have lead implementation responsibilities for approximately 70 percent of the 112 action items in the Strategy. We also interviewed officials from ONDCP and these agencies to obtain information about progress on the Strategy’s goals, the Performance Reporting System, how ONDCP monitors progress on Strategy objectives and action items, the implementation status of these action items, and the effect of the Strategy on agencies’ drug control activities. We compared the information we obtained from documents and agency officials about the Strategy and ONDCP’s monitoring mechanisms with criteria for performance management identified in our prior work. We also reviewed ONDCP’s categorizations for a nonprobability sample of 24 out of 82 action items led by ONDCP and our seven selected agencies to validate ONDCP’s characterization of their implementation statuses. Although we cannot generalize the results to all action items led by these agencies, this allowed us to determine the extent to which we agreed with ONDCP’s categorizations of the action items we reviewed. Finally, we interviewed seven drug policy experts to discuss their perspectives on the Strategy and performance management. We selected these experts based on our review of drug policy literature, their expertise in drug policy issues and work conducted in this area, and recommendations from these and other researchers. The information from these seven policy experts cannot be generalized to other experts; however, they provided us with a range of views about the Strategy and ONDCP’s monitoring mechanisms.

Assessing Fragmentation, Overlap, and Duplication across Drug Abuse Prevention and Treatment Programs and Agency Coordination Efforts

Identifying Programs

To identify federal drug abuse prevention and drug abuse treatment programs, and programs that include drug abuse prevention or treatment activities, we developed a preliminary list based on the agencies included in our previous work in this area and a review of ONDCP and agency information, such as the fiscal year 2013 National Drug Control Budget, the 2010 National Drug Control Strategy, and publically available information on agency websites. We also reviewed the Catalogue of Federal Domestic Assistance. On the basis of ONDCP guidance, we defined a drug abuse prevention program as a federal program that provides services, allocates funding, or allows for activities focused on discouraging the first-time use of controlled substances—specifically illicit drugs and the problematic use of alcohol—and encouraging those who have begun to use controlled substances to cease their use. We defined a drug abuse treatment program as a federal program that provides services, allocates funding, or allows for activities focused on identifying and assisting users of controlled substances—specifically illicit drugs and the problematic use of alcohol—to become drug-free and remain drug- free.

We focused our review on programs that directly administer or fund drug abuse prevention or treatment programs, or programs that include these activities, and therefore excluded programs that, for example, exclusively focus on law enforcement or policy, conduct research, or fund overhead costs. In addition, we also excluded programs that reimbursed drug abuse treatment services as part of a health benefit plan, such as HHS’s Medicare and Medicaid programs and the Department of Defense’s Defense Health Program, which includes military health benefit plans like TRICARE. Our initial review determined that 3 of 19 agencies included in the National Drug Control Budget for funding drug abuse prevention and treatment programs did not administer programs that met our scope of review. As a result, we excluded these agencies from our review. In total, our initial search identified 91 potential programs that 16 federal agencies administered.

Next, we shared our preliminary list of programs with agency officials and asked them to verify that the included programs met our criteria and if any should be added or deleted. During initial meetings with agency officials, we asked them to clarify program names, descriptions, and services offered. We used the information obtained to make additional refinements to the list of agencies and programs included in our review. For example, some programs were removed from the list because they were no longer in operation in fiscal year 2012 or were part of a larger program already listed, and other programs were added. We also excluded one additional agency from our review. Specifically, we excluded the Court Services and Offender Supervision Agency for the District of Columbia because its programs are primarily for residents of the District of Columbia. Thus the number of agencies in the scope of this study decreased from 16 to 15. After completing these refinements our list identified a total of 95 programs.

To obtain program-specific information, we sent a total of 95 web-based questionnaires to the 15 agencies included in our review.questionnaires are described in detail below. We included several screening questions in the questionnaire to further verify that programs met our definition of a drug abuse prevention program, drug abuse treatment program, or a program that includes these activities, and were in operation in fiscal year 2012. Fourteen programs did not pass our screening questions and therefore were excluded from our analysis. We also removed 4 programs from our analysis after reviewing the responses and determining that the program did not meet one or more of our criteria These for inclusion, and consolidated the responses of 1 program at the request of agency officials. In total, 76 programs were included in our final analysis of questionnaire responses.

We developed a web-based questionnaire to collect detailed information on federal drug abuse prevention and drug abuse treatment programs, and programs that include these activities, for fiscal year 2011. The questionnaire included questions on program objectives, target groups served, services provided, and coordination activities with other federal agencies. We also collected data on program obligations—defined as definite commitments that create a legal liability of the government for the payment of goods and services ordered or received—as available, for fiscal year 2011. Specifically, we asked programs to provide the amount of federal funds obligated specifically for their drug abuse prevention or treatment activities, and the total amount of federal funds obligated for all program activities in fiscal year 2011. In many cases, programs were not able to provide data on funds obligated specifically for their drug abuse prevention or treatment activities because agency officials told us that they did not report budgetary data at this level, among other reasons.

To minimize errors arising from differences in how questions may be interpreted and to reduce variability in responses, we conducted pretests with four agencies and 6 programs from July to August 2012. To ensure that we obtained a variety of perspectives on our questionnaire, we selected programs that differed in program scope, services provided, and target groups served. We included budget staff as well as program officials in the pretest to ensure budget-related terms in the survey were understandable. An independent reviewer also reviewed a draft of the questionnaire prior to its administration. After completing the pretests, we administered the survey to 95 programs. We received completed surveys for 95 programs, for a 100 percent response rate. telephone calls to officials and sent them follow-up e-mail messages, as necessary, to clarify their responses or obtain additional information.

We consolidated questionnaire responses for SAMHSA’s Substance Abuse and Mental Health block grant program at the request of agency officials. As a result, we ultimately received responses for 94 unique programs.

We used standard descriptive statistics to analyze responses to the questionnaire. Because this was not a sample survey, there were no sampling errors. To minimize other types of errors, commonly referred to as nonsampling errors, and to enhance data quality, we employed survey design practices in the development of the survey and in the collection, processing, and analysis of the survey data. For instance, as previously mentioned, we pretested the questionnaire with federal officials to minimize errors arising from differences in how questions might be interpreted and to reduce variability in responses. We further reviewed the questionnaire to ensure the ordering of sections was appropriate and that the questions within each section were clearly stated and easy to comprehend. To reduce nonresponse, another source of nonsampling error, we sent out e-mail reminder messages to encourage officials to complete the questionnaire. In reviewing the questionnaire data, we performed automated checks to identify inappropriate answers. We further reviewed the data for missing or ambiguous responses and followed up with agency officials when necessary to clarify their responses. To assess the reliability of obligations data, we incorporated questions about the reliability of the programs’ data systems and if there were any limitations to reporting the data. On the basis of our application of recognized survey design practices and follow-up procedures, we determined that the data were of sufficient quality for our purposes. All data analysis programs were also independently verified by a data analyst for accuracy.

To determine the extent of fragmentation, overlap, and duplication, we compared data from the 76 drug abuse prevention and treatment programs to review the types of services offered across the federal government and used the following definitions from our prior work:

Fragmentation occurs when more than one federal agency (or more than one organization within an agency) is involved in the same broad area of national interest.

Overlap occurs when fragmented agencies or programs have similar goals, engage in similar activities or strategies to achieve them, or target similar beneficiaries.

Duplication occurs when two or more agencies or programs are engaged in the same activities or provide the same services to the same beneficiaries.

To determine the extent of fragmentation, we used the questionnaire data to identify the number of agencies that deliver similar types of drug abuse prevention and treatment programs. To determine the potential for overlap, we identified the number of programs with similar goals that provided similar services to similar populations. We also conducted an in- depth analysis of two areas—prevention services for students and youths and treatment services for the offender populations—to further assess the potential for overlap among these programs. We selected these areas because of the number of programs and extent of overlap in these areas. For those areas in which we completed our in-depth review, we also determined the potential for duplication by reviewing programmatic information and holding meetings with program staff to determine whether the programs were providing the same services to the same beneficiaries.

To assess coordination efforts to reduce the potential for unnecessary overlap or duplication, we analyzed questionnaire responses from the 76 programs regarding agency efforts to coordinate drug abuse prevention and treatment program activities. We also analyzed the 2010 Strategy and interviewed ONDCP and agency officials about actions taken to coordinate activities. We compared these reported actions with criteria for coordinating interagency efforts, internal controls, and desirable characteristics of effective national strategies identified in our prior work.

Assessing Evaluations of Drug Abuse Prevention and Treatment Programs

To identify which of the 76 drug abuse prevention and treatment programs at the 15 agencies included in our review had program evaluations, including assessments of effectiveness, we first conducted Internet and library searches to identify any reports or other documents that described evaluations that had been completed or for which interim reports had been issued since 2007. We then interviewed or received written responses from agency officials responsible for administering the 76 programs, in which they confirmed the evaluations that we found through our searches and identified additional evaluations of the programs that had been completed, were under way or had interim reports completed, or were planned since 2007. We selected 2007 as the starting point for our review in order to provide a long enough time frame to include evaluations that may take multiple years to complete. We obtained copies of all completed program evaluations and reviewed each to determine whether these evaluations included assessments of effectiveness (i.e., determining the extent to which a program is achieving its objectives) as described in our Designing Evaluations: 2012 Revision, which provides guidance on evaluation methodologies. For each program evaluation, we reviewed its objectives, scope, and any description of how its findings were used by the agency. Furthermore, we interviewed agency officials about factors affecting the lack of program evaluation. We also reviewed agency documents and interviewed agency officials to identify whether the agencies took other steps to help ensure that their programs are effective, such as requiring programs or their grantees to use evidence-based interventions, and whether the agencies prescribed specific evidence-based interventions that must be used in their programs. We also interviewed ONDCP officials to determine how program evaluations were used to inform policy and resource allocation decisions.

We conducted this performance audit from July 2012 to March 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix III: Fiscal Year 2011 Obligations for Drug Abuse Prevention and Treatment Programs in Our Review

Agency or subagency Program name Department of Defense (DOD) DOD civilian agencies

Department of Justice Bureau of Prisons

Agency or subagency Program name Department of Transportation Federal Aviation Administration

Department of Education

Agency or subagency Program name Indian Health Service

Agency or subagency Program name

Targeted Capacity Expansion General—Recovery Oriented Systems of Care Targeted Capacity Expansion General—Technology Assisted Care Treatment Drug Courts—Adults (SAMHSA only)

Treatment Drug Courts—Juvenile (SAMHSA only)

Treatment Drug Courts—Adult (joint with the Bureau of Justice Assistance)

Treatment Drug Courts—Juvenile (joint with the Office of Juvenile Justice and Delinquency Prevention)

Agency or subagency Program name Department of Housing and Urban Development

For the purpose of our review, we identified a drug abuse prevention program as a federal program that provides services, allocates funding, or allows for activities focused on discouraging the first-time use of controlled substances—specifically illicit drugs and the problematic use of alcohol—and encouraging those who have begun to use controlled substances to cease their use. We defined a drug abuse treatment program as a federal program that provides services, allocates funding, or allows for activities focused on identifying and assisting users of controlled substances—specifically illicit drugs and the problematic use of alcohol—to become drug-free and remain drug-free.The Drug Demand Reduction program funds, at least in part, the National Guard Bureau Prevention, Treatment, and Outreach program; the Air Force Drug Demand Reduction program; the Army Substance Abuse Program; the Marine Corps Substance Abuse Program; and the Navy Alcohol and Drug Abuse Prevention program, as well as drug testing for the department’s civilian employees. The military services use Drug Demand Reduction program funds to provide drug abuse prevention services, including drug testing, education, and outreach. The Drug Demand Reduction Program does not fund drug abuse treatment services or services related to the prevention or treatment of alcohol abuse. However, the military services may use other funding sources to provide those services. For example, the U.S. Army uses funds from its Operations and Maintenance Account to provide some drug abuse treatment services. We requested that surveyed programs provide the total amount of federal funds obligated specifically for drug abuse prevention and treatment activities in fiscal year 2011. For those agencies that were unable to provide this information, we reported that this information was not available. Program officials reported that they were unable to provide the total amount of federal funds obligated specifically for their program’s drug abuse prevention or treatment activities in fiscal year 2011 for a variety of reasons, such as that the programs do not collect this type of budgetary data. The Bureau of Prisons reported obligations for its drug abuse prevention and treatment programs in total, but was not able to report obligations for individual programs.

Under 20 U.S.C. § 7164, funds may not be used for medical services, drug treatment, or rehabilitation, except for pupil services or referral to treatment for students who are victims of, or witnesses to, crime or who illegally use drugs with regard to Safe and Drug-Free Schools and Communities. Under 20 U.S.C. § 7175, with regard to 21st Century Community Learning Centers, each eligible entity that receives an award may use the award funds to carry out a broad array of before and after school activities that advance student academic achievement that are listed in the statute.

For the purpose of our review, we assessed the activities of Education’s Safe and Supportive Schools and Safe Schools/Healthy Students programs separately; according to officials from Education, they are considered to be activities within a single program–the Safe and Drug-Free Schools and Communities National Activities.

Appendix IV: Information about Completed Program Evaluations Conducted by Agencies in Our Review

We found that the agencies included in our review have completed six evaluations of drug abuse prevention and treatment programs since 2007, three of which examined effectiveness and three of which examined program implementation. The completed program evaluations that included assessments of effectiveness examined the following:

Office of Justice Program’s (OJP) Drug Courts Program.examined 23 adult drug courts across the United States and compared them with 6 sites in the same geographic areas. The evaluation focused on four main issues: (1) whether drug courts reduce drug use and other problems associated with drug abuse, (2) how drug courts work, (3) how offender attitudes and behaviors change when they are exposed to drug courts and how these changes help explain the effectiveness of drug courts, and (4) whether drug courts generate cost savings. OJP officials told us that as a result of the study, the major findings of the evaluation were translated into seven evidence-based design features that OJP is including in drug court grant solicitations. When making awards, OJP gives priority to applicants that include the seven design features in their proposals, as OJP considers these design features to be indicators of an effective program. The seven design features are (1) screening and assessment, (2) target population (3) procedural and distributive justice, (4) judicial interaction, (5) monitoring, (6) treatment and other services, and (7) relapse prevention, aftercare, and community integration.

The Department of Veterans Affairs’ (VA) Substance Use Disorder Outpatient Program and Substance Use Disorder Residential Program. The study looked at the quality of VA substance abuse and mental health care for veterans diagnosed with substance use disorder (SUD) or one or more other mental health conditions. The evaluation asked the following questions: (1) To what extent is VA achieving the program outcomes for veterans with SUD or other mental health conditions? (2) How do the outcomes for VA patients compare with outcomes for comparable veterans treated in non-VA- funded care? (3) How does the availability of care compare across Veterans Health Administration medical centers? (4) When there is a diagnosis of SUD and a mental health condition, are both conditions being managed? (5) What factors influence the use of VA specialty mental health services by veterans whose condition is a result of or aggravated by their service? (6) How widespread is the use of the strongest evidence-based models of care for SUD and the other mental health diagnoses? The study identified areas for further research and recommended strategies to improve SUD and mental health care in VA.

Substance Abuse and Mental Health Services Administration’s (SAMHSA) Substance Abuse Prevention and Treatment Block Grant Program. The study assessed the extent to which this program is effective, functioning as intended, and achieving desired outcomes. The program provides funds to 50 states, nine territories (including the District of Columbia), and one Indian tribe for activities to prevent and treat substance abuse. The evaluation examined (1) the processes and activities by which states implement the legislative and policy requirements of the Substance Abuse Prevention and Treatment Block Grant program, (2) activities associated with the federal administration of the program and how they support program implementation and accountability, (3) state system processes and capacity for the collection and submission of data on block grant- funded activities, (4) specified outcomes associated with states’ treatment and prevention services, and (5) the ways in which states used and leveraged program funds. The study identified program successes, as well as challenges and areas for improvement.

The other completed program evaluations examined:

SAMHSA’s Underage Drinking Prevention Education Initiative.evaluations examined implementation and perceived outcomes of 2008 and 2010 Town Hall meetings intended to raise public awareness of underage drinking and engage communities in its prevention. The evaluation also reviewed the barriers and challenges in planning and convening the Town Hall meetings. The evaluations reviewed, among other things, changes over time in the numbers of Town Hall meetings held and participation by community-based organizations. The evaluations also examined Town Hall attendees and presenters and the settings, formats, and contents of the Town Halls. Officials from SAMHSA, which administers the program, said they would use the evaluation results to shape future program proposals and inform and improve technical assistance and training activities. SAHMSA plans to share the results with participants and use the results in presentations on best practices.

SAMHSA’s Assertive Adolescent and Family Treatment Program.The study focused on the third cohort of Assertive Adolescent and Family Treatment Program grant recipients and had two overarching goals: (1) to document the program implementation process, and (2) to explore how implementation supports, such as training and technical assistance for the clinical and assessment components of the project, guide how the funded programs evolve. The report findings addressed the characteristics of agency, staff, and clients and their relationship with program implementation, grantees’ use of implementation activities, implementation barriers, strategies used and lessons learned, and the overall impact of the program on program sites and their clients.

Appendix V: Comments from the Office of National Drug Control Policy

Appendix VI: GAO Contacts and Staff Acknowledgments

GAO Contacts

Staff Acknowledgments

In addition to the contacts named above, Karen Doran, Assistant Director; Mary Catherine Hult, Assistant Director; David Alexander; Willie Commons III; Elizabeth Curda; Susan Czachor; Lorraine Ettaro; Michele Fejfar; Jill Lacey; Kelly Liptan; Emily Loriso; Linda Miller; Erin O’Brien; Dae Park; Emily Ryan; C. Jenna Sondhelm; and Johanna Wong made significant contributions to the work.