Independent Evaluation of the Substance Abuse 

Prevention and Treatment Block Grant Program 

Final Evaluation Report Executive Summary 

U.S. Department of Health and Human Services 

Substance Abuse and Mental Health Services Administration 

Center for Substance Abuse Treatment 

Center for Substance Abuse Prevention 

http://www.samhsa.gov
Independent Evaluation of the Substance Abuse Prevention and Treatment Block Grant Program 2

Executive Summary

The Substance Abuse Prevention and Treatment Block Grant (SAPT BG) Program provides funds to 

States, Territories, the Pacific Jurisdictions, and one Native American Tribe to plan, carry out, and 

evaluate activities to prevent and treat substance abuse. The SAPT BG Program, legislated by Congress 

in 1981, is administered by the Center for Substance Abuse Prevention (CSAP) and Center for 

Substance Abuse Treatment (CSAT) of the Substance Abuse and Mental Health Services Administration 

(SAMHSA) and represents the largest source of Federal funding to States for the prevention and 

treatment of substance use disorders. It constitutes a substantial amount of all States’ budgets for 

substance abuse programming and serves an average of 2 million individuals each year. States have 

flexibility in determining how funds should be allocated to address local needs; however, to receive 

funding, States must meet specific set-aside and maintenance of effort (MOE) requirements and conduct 

activities designed to achieve the 17 legislative goals of the Program.

In 2005, CSAT and CSAP contracted with Altarum Institute under Task Order Number 280-03-3501 to 

conduct the first independent evaluation of the SAPT BG Program. The purpose of the evaluation is to 

assess the extent to which the SAPT BG is effective, functioning as intended, and achieving desired 

outcomes. 

The evaluation has five main objectives:

 To examine the processes and activities by which States implement the legislative and policy 

requirements (e.g., 17 goals) of the SAPT BG Program 

 To assess activities associated with the Federal administration of the SAPT BG Program and 

how they support Program implementation and accountability

 To examine State system processes and capacity regarding the collection and submission of data 

on BG-funded activities

 To assess specified outcomes associated with States’ treatment and prevention services 

 To explore unique ways in which States use and leverage SAPT BG funds.

To examine the effectiveness and impact of the SAPT BG Program, the evaluation employed a 

multimethod design that incorporated process and outcome evaluation strategies. Quantitative and 

qualitative data collection, analysis, and triangulation were necessary to obtain a complete picture of 

Program strengths, effects, areas for improvement, and client- and State system-level outcomes. Specific 

data collection methods included review of Program documents, secondary analysis of data collected for 

the Program’s National Outcome Measures (NOMs), and the development and administration of 

interview protocols with State and Federal staff and Web-based surveys for Program reviewers. 

The independent evaluation of the SAPT BG Program resulted in six key findings about the outcomes 

and effects of the Program, pointing to Program successes as well as to areas for improvement in 

processes and implementation. 
Independent Evaluation of the Substance Abuse Prevention and Treatment Block Grant Program 3

Executive Summary

Key Finding 1: The SAPT BG Program has demonstrated a positive effect 

on the health and lives of individuals with substance use disorders.

Secondary analysis of data collected for the NOMs demonstrated positive client outcomes in all six 

treatment domains: alcohol and drug abstinence, employment/school participation, stable housing, social 

connectedness, criminal justice involvement, and retention in treatment. 

Key Finding 2: The SAPT BG Program has acted as a major impetus for 

improving State prevention and treatment systems’ infrastructure and 

capacity.

SAPT BG Program emphasis on demonstrating the effectiveness of BG-funded programs and services to 

reduce substance abuse and to improve the lives of those affected by it has driven State system 

infrastructure development and capacity improvements and resulted in the following outcomes:

 Increased availability of services for diverse and underserved populations

 Increased development and implementation of evidence-based practices (EBPs)

 Improved development and collection of specific outcome measures

 Increased development and maintenance of State data management systems.

Key Finding 3: States have leveraged SAPT BG Program requirements, 

resources, and Federal guidance to sustain and improve their State systems.

States have used the BG requirements and funding to go beyond the intended and expected outcomes of 

the Program, leveraging BG resources to sustain and improve State substance abuse prevention and 

treatment systems. Although not a requirement for BG funding, State leveraging of SAPT BG Program 

requirements, resources, and Federal guidance demonstrates the importance of the BG in the 

development of State systems. States leveraged BG funds to:

 Prevent harm to the service system resulting from State legislature reductions in funds for 

prevention and treatment and advocate for additional State funding. In some states, prevention 

activities would not exist without SAPT BG support.

 Provide BG funds as seed money for new programs that other public and private organizations 

have subsequently funded

 Set State policies and priorities based on Federal leadership and development of national policies 

and priorities.

Key Finding 4: Through a standard system of communication, monitoring, 

and reporting, CSAT, CSAP, and the States effectively and efficiently 

manage the SAPT BG Program. 

CSAP and CSAT have developed several successful management strategies to steer States as they work 

toward the 17 legislative goals. Management involves the following crucial activities:

 Communicating Program goals and activities 

 Monitoring and oversight to facilitate open communication and ensure compliance
Independent Evaluation of the Substance Abuse Prevention and Treatment Block Grant Program 4

Executive Summary

 Leading complex data collection and reporting activities

 Providing technical assistance (TA) and training to aid States in meeting their goals.

Key Finding 5: The SAPT BG Program has contributed to the development 

and maintenance of successful State collaborations with other agencies and 

stakeholders concerned with preventing substance abuse and treating 

substance use disorders.

State substance abuse agencies have increased the development and maintenance of collaborative 

working relationships with a variety of other Federal, State, and local agencies and providers. States 

fostered many of these partnerships as they worked to accomplish the Program’s 17 legislative goals. 

These State collaborations served five critical roles:

 To increase achievement of Synar Program goals and objectives

 To improve the coordination of prevention services

 To improve the coordination of treatment services with public and private health insurers

 To expand services and programs available through joint funding initiatives

 To increase the ability to address statewide critical public health or safety issues.

Key Finding 6: Although baseline data support the need for prevention 

services and activities, the use of national survey State estimates data alone 

to assess the NOMs limits CSAP’s ability to attribute changes in the NOMs 

to SAPT BG-funded prevention services and activities. 

To reduce the data collection burden for State and local prevention agencies, CSAP uses data from the 

National Survey on Drug Use and Health (NSDUH) to fulfill NOMs data requirements, including 30-

day substance use, perceived risk or harm from use, age of first use, perception of disapproval/attitude 

towards substance use, and perception of workplace policy. Two significant difficulties are inherent in 

this strategy: conclusions about NOMs changes as a result of BG-funded prevention services and 

activities cannot be made based primarily on the results of national survey State estimates that do not 

identify individuals or groups who may have been affected by BG-funded activities, and the NSDUH is 

limited by small sample sizes in many States, which leads to under coverage of some populations. 

Additional data are needed to link changes in NOMs measures to interaction with BG-funded prevention

services and activities.
Independent Evaluation of the Substance Abuse Prevention and Treatment Block Grant Program 5

Executive Summary

Challenges and Recommendations for Program Improvement

In addition to its strengths and accomplishments, the large and complex SAPT BG Program experiences 

a variety of challenges. By addressing and resolving these challenges, CSAT and CSAP can improve 

Program effectiveness and efficiency and increase the quality of Program services provided to 

individuals and communities.

Challenge: Need for improved communication and a consistent message from Federal to State 

staff about some Program goals and requirements. States commented about the lack of unified, 

consistent messages from CSAT and CSAP in some specific areas, including:

 NOMs data definitions and standard data collection processes

 The reporting of financial information and MOE compliance 

 Expectations for BG applications 

Acceptable fulfillment of the 17 legislative goals.

Challenge: Need for clarification about the roles and responsibilities of Federal, State, and 

contractor staff related to CSAT Core Technical Reviews and Technical Assistance (TA). Although 

CSAT State Project Officers (SPOs) work with States to refine their BG applications and to address the 

17 goals, CSAT SPOs are not involved in substantive aspects of State Core Technical Reviews. To 

ensure an objective assessment of compliance, CSAT SPOs typically attend the Core Technical Review

but remain in the background, which is confusing for States. In addition, States perceive that decisionmaking authority for CSAT TA requests lies in the hands of the TA contractor, the TA contract 

Government Project Officer (GPO), and the Branch Chief, rather than with the CSAT SPOs, who 

typically are most informed about individual State issues and TA needs. 

Recommendations for Program Improvement:

Improve Program Communication and Guidance

 Clarify Program data definitions and requirements, including “what counts” for achievement of the 17 

goals, MOE and other financial calculations, and the NOMs data elements

 Develop unified Federal guidance about Program requirements and expectations and a data dictionary 

with uniform and realistic definitions 

 Continue to seek State input and develop better definitions for required outcome data elements

 Provide opportunities for internal communication within CSAT and CSAP, training and mentoring 

staff to ensure that consistent guidance is provided to States

 Strengthen ongoing communication between State Project Officers and their assigned states via 

devoted resources for knowledge management
Independent Evaluation of the Substance Abuse Prevention and Treatment Block Grant Program 6 

Executive Summary 

Recommendations for Program Improvement:

Clarify Roles and Responsibilities

 Clarify the role of the CSAT SPO to avoid confusion among State and Federal staff

 Assign CSAT SPOs a more substantive role in Program monitoring and TA provision to take 

advantage of SPO State-specific expertise and to improve State satisfaction with monitoring and 

support 

Challenge: High level of burden on States to provide information for the Program. Each year, 

States are required to produce an SAPT BG application that describes the activities conducted to achieve 

the 17 legislative goals. The length of the report reflects the collection of information for reports to 

Congress. The majority of States spend 6 to 9 months each year gathering information for and 

developing the BG application, using staff resources that States argue could be better spent on TA for 

providers and other BG subrecipients. States also are asked to provide a large amount of background 

information in preparation for Program monitoring reviews – information that States say can be obtained 

from their applications and online resources. 

Challenge: Limited utility of Program monitoring reports and recommendations for some States.

The time lag associated with the finalization of monitoring reports (6 to 12 months) makes some 

recommendations obsolete by the time they reach the States. Delays are most pronounced when the 

review contractor, SPO, and States disagree about review findings, which must be resolved before a 

report can be finalized. In addition, some report recommendations are too general and do not account for 

the unique combination of political, social, and economic forces that affect the State’s prevention and 

treatment system. Some States report that TA recommendations in the treatment review reports seem 

motivated by contractor interests and skills rather than what is in the best interest of the State. 

Recommendations for Program Improvement:

Reduce State Administrative Burden

 Implement a multiple-year application cycle (every 3 to 5 years) that would require States to submit a 

multiple-year plan and provide annual progress reports based on plan objectives

 Revisit the primary purpose of the BG application and eliminate questions or areas that do not address 

it

 Encourage Program monitors and reviewers to obtain information from WebBGAS or online resources 

to reduce State administrative burden
Independent Evaluation of the Substance Abuse Prevention and Treatment Block Grant Program 7 

Executive Summary 

Recommendations for Program Improvement:

Improve Utility of Program Monitoring Reports and Recommendations

 Expedite the report review process by instituting a process through which States are able to contest 

review findings without delaying the finalization of the review report

 Set and enforce deadlines for submission of report comments and report revisions so that reports are 

not delayed by any one individual 

 Select and train reviewers to ensure that they possess a comprehensive understanding of the State 

systems that they may be assigned to review 

 Further involve CSAT SPOs in reviewing TA recommendations to ensure that they address State 

concerns and are not motivated by contractor interests

Challenge: Unmet TA and training needs. Many States do not know how to make formal requests for 

Federal TA and training and are not aware of the potential areas in which TA is available. In addition, 

the treatment and prevention system reviews produce some useful TA recommendations, but the TA is 

either not approved or not conducted to the satisfaction of the State. Finally, there is a disconnect 

between Federal TA providers and CSAT SPOs; several States reported receiving support for a TA 

request from their CSAT SPO only to have it denied by the CSAT TA contract GPO or Branch Chief. 

Challenge: Limited ability to demonstrate some individual-level outcomes and system-level 

outcomes. NSDUH data used for individual-level prevention NOMs do not measure individual attitudes 

and behaviors as a result of participation in BG-funded prevention activities. Thus, Federal and State 

staff cannot claim that BG-funded prevention activities contribute directly to improvements in attitudes 

and behaviors of those served with BG funds. Use of the Treatment Episode Data Set (TEDS) for the 

treatment NOMs presents the following difficulties: Not all States participate in the TEDS initiative and 

Recommendations for Program Improvement:

Improve and Expand Provision of TA

 Clarify the TA and training request process and regularly inform States about the process 

 Increase efforts to market TA and training to the States so that they understand what is available

 Expand the scope of Federal TA provided to States to include additional TA designed to identify and 

meet the needs of diverse populations, address EBP implementation challenges, and assist with State 

infrastructure enhancements

 Clarify the role of the CSAT SPO in TA provision and encourage direct communication between 

States and CSAT TA decisionmakers in order to improve responsiveness to States; promote a team 

approach 

 To improve TA access and satisfaction, provide additional resources for the following TA and training 

formats:

Guidelines and support for train-the-trainer models, regular training cycles, and a helpdesk

Distance learning, Web-based trainings, and online tutorials

Peer-to-peer TA at workshops, conferences, and regional meetings

Wider dissemination of “off-the-shelf” TA tools and materials
Independent Evaluation of the Substance Abuse Prevention and Treatment Block Grant Program 8 

Executive Summary 

questions remain about the consistency of data collection and data quality procedures across States. 

Finally, the Program needs some system-level outcome measures related to infrastructure development, 

collaboration with other State agencies and organizations, and effectiveness of TA and other Program 

support activities. 

Recommendations for Program Improvement:

Improve Data Collection Strategies and Processes

 Develop prevention outcome measures that assess attitudes and behaviors pre- and post-interaction 

with BG-funded services and activities when the prevention strategy supports this evaluation design.

 Compare NSDUH results on the NOMs for respondents who did and did not report experience with 

prevention services and messages to determine whether exposure to prevention services or messages 

can be associated with more desirable NOMs results.

 Strongly encourage States to participate in TEDS and to use TEDS data definitions to improve the 

reliability of NOMs data 

 Develop materials, host Web-based and in-person trainings, and provide onsite TA to States and 

subrecipients to ensure that data definitions are being interpreted correctly and consistently

 Continue close collaboration with State substance abuse and other appropriate State agencies (e.g., 

data and statistics, corrections) to develop more valid and effective outcome measures 

 Develop 2 to 4 system-level indicators to demonstrate Program effects on State systems development 

and enhancement

Challenge: Need for additional resources to further improve State data infrastructures. Many 

States are still struggling to develop data collection, information, and monitoring systems that will 

enable them to track outcomes effectively. States have received modest funding to create new or 

overhaul existing data collection and reporting systems; however, they need additional resources to 

continue improvements and to foster data literacy. States also cited a need for improved networking 

opportunities among States so that they can learn from each other and not have to “reinvent the wheel.”

Center for Substance Abuse Treatment and Center for Substance Abuse Prevention. (2009). Independent Evaluation of the Substance Abuse 

Prevention and Treatment Block Grant Program: Final Evaluation Report Executive Summary. (Task Order No. 280-03-3501). Rockville, MD: 

Substance Abuse and Mental Health Services Administration. Also available online: http://www.tie.samhsa.gov/. 

Recommendations for Program Improvement:

Invest Additional Resources to Improve State Data Infrastructures

 Invest additional resources to help States complete the development and maintenance of data 

collection systems that will increase their ability to demonstrate SAPT BG Program outcomes and 

make data-driven decisions 

 Create more opportunities for State-to-State TA regarding the development and maintenance of State 

data collection and reporting systems. Additional opportunities for State-to-State TA, regional 

trainings, and networking at conferences will help more States develop and maintain effective data 

systems