3C SESSION

SAMHSA’s 988 Crisis Systems Response Training and 

Technical Assistance Center: Crisis Community 

Collaboration

March 26, 2024
Crisis Systems Response TTAC

The Substance Abuse and Mental Health Services Administration (SAMHSA) 

has selected Altarum to provide training and technical assistance support 

to states, territories, tribal organizations, and community partners across the 988 

Suicide and Crisis Lifeline and crisis continuum of care. Along with our partners, 

W2 Consulting Corporation and Change Matrix, LLC, who have extensive 

experience with crisis services, technical assistance, and health equity, the 

Crisis Systems Response Training and Technical Assistance Center (TTAC) was 

formed to support the continued growth of 988 Lifeline and build a more robust 

crisis care system.
Agenda

• Welcome

• Introductions

• Updates from the 988 & BHCCO

• Evaluation updates with Michelle Cornette

• Mindful Minute

• Updates from the 988 CSR TTAC
SAMHSA 988 & BHCCO Leads

Monica Johnson, 

M.A., LPC

Director of the 988 & 

Behavioral Health 

Crisis Coordinating 

Office

John Palmieri, M.D., 

M.H.A.

Deputy Director of 

the 988 & Behavioral 

Health Crisis 

Coordinating Office

Jill D. Mays, M.S., 

LPC

Division Director of 

Crisis System 

Transformation for 

the 988 & Behavioral 

Health Crisis 

Coordinating Office

Charissa Pallas

Division Director of 

Communications for 

the 988 & Behavioral 

Health Crisis 

Coordinating Office

James Wright

Division Director of 

Crisis Operations, 

Office of the 

Assistant Secretary
Updates

With the 988 BHCCO Leadership Team
Guest Speaker

Michelle Cornette Ph.D.

Data Analytics Officer for the 

988 & BHCCO

• Michelle Cornette, Ph.D. is the Data Analytics Officer in the 988 & Behavioral Health Crisis Coordinating Office at 

SAMHSA. Her professional history in suicide prevention (28 years) includes roles as administrator, researcher, 

clinician, educator, and survivor of suicide loss. She was most recently Subject Matter Expert and Lead Public Health 

Advisor in the Suicide Prevention Branch at SAMHSA and is Past Executive Director of the American Association of 

Suicidology, the nation’s suicide prevention membership organization. During her tenure with AAS, she introduced a 

number of new products and programs, and was responsible for dramatically expanding the organization’s operating 

budget. 

• At SAMHSA, she was most recently Program Lead for the Zero Suicide grant program, and a program officer for the 

National Strategy for Suicide Prevention and Community Crisis Response Partnership grant/ cooperative agreement 

programs. She co-managed the grant for the National Suicide Prevention Lifeline, which has since become 988, and 

was a contributor to the 988 Reports to Congress. She was also previously a program officer for the Native 

Connections and National Child Traumatic Stress Initiative grant programs. Dr. Cornette serves on the BHCC Data 

Subcommittee, and served previously on the Joining Forces IPC, the Suicide Prevention IPC Data and Surveillance 

Subcommittee, and the BHCC SPCC Implementation Strategy Work Group. 

• Dr. Cornette previously supported the Research & Program Evaluation and Data & Surveillance Divisions at the 

Department of Defense Suicide Prevention Office. She was also Military Suicide SME at the Center for Deployment 

Psychology, where she developed a two-day training for military/ veteran mental health providers on evidencebased approaches to suicide risk assessment, crisis intervention, and treatment, and which she subsequently 

delivered at military treatment facilities across the country. 

• She spent 10 years in the VA system, where she was VISN 12 Suicide Prevention Director, responsible for overseeing 

suicide prevention activities at the 7 VA facilities in VISN 12. She further led the suicide prevention clinical team at 

her local facility. Dr. Cornette has received Federal and private funding for her program of research on suicide risk, 

and she has presented and published extensively in addition to providing many television and print interviews on 

the topics of suicide risk assessment, prevention, and treatment. She earned her PhD in Clinical Psychology at the 

University of Wisconsin-Madison.
988 and Behavioral Health 

Crisis Continuum Office 

Evaluation Portfolio

Michelle Cornette, Ph.D.

Data Analytics & Evaluation Lead

988 & BHCCO, SAMHSA

March 26, 2024
988 Lifeline Evaluation to Date

The 988 Suicide & Crisis Lifeline, formerly known as the National 

Suicide Prevention Lifeline, helps thousands of people overcome 

crisis situations every day.

Proven to work – Our research has shown that after 

speaking with a trained 988 Lifeline crisis counselor, 

most callers are significantly more likely to feel:

• less depressed

• less suicidal

• less overwhelmed

• more hopeful
SAMHSA 988 & BHCCO Data & Evaluation Activities

• 988 & BHCCO Evaluation

• Center for Financing Reform Innovations (CFRI) Evaluation

• Clinical Training Evaluation

• Pre and Post Chat & Text Surveys

• Standardized Clinical Contact Form

• Post-Call Survey Evaluation

• Post-Call Survey Implementation Pilot

• Standardizing 988 Grantee Data Collection

• National Survey on Drug Use and Health (pending)
988 Suicide & Crisis Line 

and Crisis Services 

Program Evaluation
Project Purpose

• To gather detailed information about the implementation and utilization 

of the 988 Suicide & Crisis Lifeline, as well as clinical/functional, 

service, and outcomes across the crisis services continuum. 

• The evaluation is centered on the behavioral health crisis services 

continuum:

• Crisis Contact Centers

• Mobile Crisis Services

• Crisis Stabilization Services

Evaluation 

Aims

• Capture changes experienced at the systems- and client- levels, along with 

overarching impact. 

• Provide SAMHSA with data needed to understand the implementation of the 988 

Lifeline and strengthen the full continuum of crisis care services
Evaluation Design Overview

988 Crisis Services 

Program Evaluation

•System-Level 

Evaluation 

•Client-Level 

Evaluation 

•Impact 

Evaluation
System-Level Evaluation Design Overview

Study Name Key Evaluation Questions Participants

System Composition & 

Collaboration Study

• How have states/territories structured their crisis service continuums 

following the implementation of 988 Lifeline?

• How have crisis services providers integrated services since the 

implementation of the 988 Lifeline? 

• How are crisis continuum partners communicating and coordinating 

in their implementation of the 988 Lifeline?

• What is the national profile of crisis workers by demographics, 

credentials, language, and years of service?

All

System-Level Service 

Utilization and Resolution 

Study

• How and to what extent are crisis contacts resolved as they move 

through the crisis care continuum?

• What is the demographic profile and are there disparities in 

individuals who engage with 988?

• What is the impact of 988 on the utilization of other crisis continuum 

services?

• Does use of 988 result in overall cost savings considering impact on 

emergency department utilization and criminal justice system 

involvement?

Reference: 3 icons
System-Level Evaluation – Data Sources

Study Name Data Sources Participants

System Composition & 

Collaboration Study

• System Implementation Survey

• Crisis Continuum Provider Survey

• 988 Implementation Case Studies

• Grantee Reports and Performance Data

o Includes: SPARS IPP data

• 988 State/Territory 

grantees

• 988 Tribal nations 

grantees

• Crisis Contact Centers

• Mobile Crisis Services

• Crisis Stabilization 

Services

System-Level Service 

Utilization and Resolution 

Study

• Grantee Document and Data Review Form

o Includes: SPARS IPP, grantee performance indicators

• System Outcome Summary Inventory 

• Crisis Continuum Provider Survey

• Extant service utilization data

o Vibrant’s Clinical Contact Form

• 988 State/Territory 

grantees

• 988 Tribal nations 

grantees

• Crisis Contact Centers

• Mobile Crisis Services

• Crisis Stabilization 

Services
Client-Level Evaluation Design Overview, Questions

Study Name Key Evaluation Questions Participants

Client-Level Service Utilization 

and Outcome Study

• What are the needs of help-seekers who engage with 988? 

• Through which pathways do individuals access and engage 

with 988?

• How does engagement with 988 impact clinical outcomes 

among help-seekers?

Reference: 4 icons

Client-Level Risk Reduction 

Study

• What are the immediate outcomes of 988 contacts on 

suicide, homicide, and substance use risk?

• How do help-seekers experience their contact with the 

988 Lifeline?

Reference: 2 icons

*Dates may change based on timing of OMB approval
Client-Level Evaluation Design Overview, Sources

Study Name Data Source Participants

Client-Level Service Utilization 

and Outcome Study

• Client Contact Disposition Form 

• Client Experience Survey 

• Extant Lifeline Post-Contact Survey Data

• Grantee Document and Data Review Form

o aggregate suicide attempt and death data (i.e., IPP 

indicators T7 & T8)

Help-seekers - 

recruited through:

• Crisis Contact 

Centers

• Mobile Crisis 

Services

• Crisis Stabilization 

Services

Client-Level Risk Reduction 

Study

• Recorded crisis contacts 

• Client key informant interviews

• Extant Lifeline Post-Contact Survey Data

• Help-seekers

*Dates may change based on timing of OMB approval
Impact Evaluation Design Overview

Study Name Key Evaluation Questions Participants

Impact Study

• What is the impact of 988 on suicidal ideation over time? 

• What is the impact of 988 on suicide attempts over time? 

• What is the impact of 988 on non-fatal overdoses over time?

• What is the impact of 988 on suicide and overdose deaths over 

time?

Reference: icon
Center for Financing Reform and Innovations (CFRI) Evaluation:

Utilizing Medicaid, Medicare, and Private Insurance Data to Understand Behavioral Health Crisis 

Continuum Service Trajectories, Clinical Outcomes, and Risk for Suicide or Other Violent Death

Available Datasets:

• Medicaid

• Medicare

• Marketplace

• National Death Index (NDI)

Analyses:

1. Prospective analysis of the subsequent crisis, emergency department, inpatient, residential and outpatient 

services received by individuals following an index crisis contact. 

*Subanalyses examining differences in outcomes by race/ethnicity, and SOGI (For analyses 1 through 10)

2. Retrospective analysis of behavioral health services received leading up to a crisis contact.

3. Cross-sectional analysis of behavioral health diagnoses at the time of crisis visits.

4. Prospective analysis of diagnostic outcomes following an index crisis contact.

• 4a. Examining differences in diagnostics outcomes among crisis service users covered by 

Medicaid/Medicare, versus among those covered by private insurance.

5. Comparing crisis service utilization pre vs. post-COVID.
Center for Financing Reform and Innovations (CFRI) Project

Analyses Continued:

6. Among those with a billing code for suicide attempt or ED visit with self injury, compare risk of 

death by suicide among those with differing crisis and behavioral health service histories.

7. Among those with a billing code for suicide attempt or ED visit with self-injury, compare risk of 

death by suicide among those covered by Medicaid/ Medicare, versus by private insurance.

8. Among those with crisis service utilization histories, comparing risk of death by suicide to 1) 

risk for other forms of violent death as well as to 2) risk for all other causes of death.

9. Comparing risk of death by suicide among those with histories of behavioral health diagnoses 

plus crisis service utilization to those with behavioral health histories without crisis service 

utilization. 

10. Examining difference in risk of death by suicide among crisis service users covered by 

Medicaid/Medicare, versus among those covered by private insurance.
Vibrant Clinical Training Evaluation, Aims

• Aim 1: To assess the immediate, short-, and long-term effect of training on 

crisis counselors’ self-efficacy, knowledge, and skills. 

• Aim 2: To evaluate the extent to which changes in self-efficacy, knowledge, 

and skills differ based on comfort working with various presenting concerns, 

including those related to minority stress and cultural fluency. 

• Aim 3: To examine the extent to which crisis counselors effectively transfer 

knowledge from training to behavior, including the investigation of 

discrepancies between self-efficacy, knowledge, skills, and behavior. 

• Aim 4: To understand the training experiences and types of support (e.g., 

training gaps, post-training supervisory and peer support) required to best 

meet the training needs of crisis counselors.
Vibrant Clinical Training Evaluation, 4 Areas

REACTION

The degree to which

participants find the 

training favorable, 

engaging, and relevant 

to their jobs. 

Aim 4

LEARNING

The degree to which 

participants acquire 

the intended 

knowledge, skills, 

confidence, and 

commitment to apply 

training principles. 

Aims 1, 2

BEHAVIOR

The degree to which 

participants apply 

what they learned 

during training when 

they are back

on the job. 

Aims 1, 2, 3

RESULTS

The degree to which 

targeted outcomes 

occur as a result 

of the training and 

on-the-job support and 

accountability.

Aims 1, 3
Evaluation Design

Flowchart: All Lifeline Call 

Centers, n = ~200, leads to: 

Organizational Catalyst & 

Capacity & Inventory (newly 

proposed). This leads to 2 

flowcharts for Study 1 and 

Study 2. Each of them have 6 

content boxes.

Study 1: Core Training Outcomes Study (All Crisis Call Centers) 

2 shapes: Trained Staff and Newly Trained Staff lead to 2 boxes, Existing 

Pre/Post Data and Enhanced Pre/Post Data (newly proposed) respectively. 

Those 2 boxes lead to Training Feedback which leads to the end box, 6- or 12-

month Workforce Follow-Up Survey (newly proposed).

Study 2: Training Transfer Sub-Study (Subset of 24 Crisis Call Centers)

The first box: Crisis Center Sample: Ability to Record Contacts, Contact 

Volume, Willingness to Participate, Geographic Distribution, and PostContact Survey Participation leads to 2 boxes. National Backup Centers 

(leads to box: ΟΙ Data) and Matched Centers. OI Data and Matched 

Centers lead to Crisis Contact Recordings (newly proposed) which leads 

to the end box, Focus Groups (newly proposed).
Core Training Outcomes Study

The Core Training Outcomes Study assess the extent to which Vibrant’s training courses are effective in improving the selfefficacy, knowledge, and skill of crisis counselors in implementing the Lifeline Safety Assessment Model (LSAM) across all 

Lifeline crisis contact centers. 

Evaluation Questions:

• To what extent do crisis counselors’ self-efficacy, knowledge, and skill change in the immediate, short-, and long-term as a result of 

training? (Aim 1) 

• How do self-efficacy, knowledge, and skill vary based on comfort working with various presenting concerns, including those related 

to minority stress and cultural fluency? (Aim 2)

• How do these outcomes vary based on crisis counselor characteristics (e.g., prior training exposure, professional experience,

education) and center-level training supports? 

• How do crisis counselors describe their training experiences and needs? (Aim 4)

• What do crisis counselors perceive as the utility and outcomes of training participation? What barriers or facilitators influence the 

implementation of training material? 

Proposed Data Sources:

• Organizational Catalyst and Capacity Inventory (one-time survey, one crisis contact center director/administrator). 

• Enhanced Pre/Post Tests & Workforce Follow-Up Survey (longitudinal follow-up surveys either 6-months or 12-months following 

training; newly trained counselors also complete enhanced pre- and post-training surveys)

• Extant Data (existing pre/post test data, knowledge assessment scores, registration data, training feedback)
Training Transfer Sub-Study

The Training Transfer Sub-Study explores the extent to which crisis counselors effectively apply training 

principles in their work within a subset of 24 recruited crisis centers. This sub-study also gathers feedback 

from crisis counselors implementing training principles in varying contexts (e.g., call, chat, text), along with 

suggestions to strengthen training resources. 

Evaluation Questions:

• To what extent do crisis counselors effectively transfer knowledge from training to behavior? (Aim 3)

• How do crisis counselors describe their training experiences and needs? (Aim 4)

o What do crisis counselors perceive as the utility and outcomes of training participation? What barriers 

or facilitators influence the implementation of training material? 

o What types of support (e.g., training gaps, post-training supervisory and peer support) would enhance 

training experience or application for crisis counselors?

Proposed Data Sources:

• Extant QI Data (3 months prior to and after each center reaches 90% training completion)

• Crisis Contact Recordings (34 per participating center)

• Extant Post-Contact Survey Data

• Focus Groups
Thank you!

You can email questions to 

michelle.cornette@samhsa.hhs.gov

This project is supported by the Substance Abuse and Mental Health Services 

Administration (SAMHSA), the agency within the U.S. Department of Health and 

Human Services that leads public health efforts to advance the behavioral health 

of the nation. The Crisis Systems Response Training & Technical Assistance 

Center works in conjunction with the 988 Suicide & Crisis Lifeline. In 2020, 

Congress designated the new 988 dialing code to be operated through the 

existing National Suicide Prevention Lifeline. SAMHSA sees 988 as a first step 

towards a transformed crisis care system in America. Points of view or opinions 

in this document are those of the author and do not necessarily represent the 

official position or policies of SAMHSA or the 988 Suicide & Crisis Lifeline.
Mindful Minute
Upcoming

Webinar Series: Connecting to Serve: Promising Practices for 988 & 911

Collaboration, Session 4

April 18, 2024, at 1:00 pm EST

“3C” Session: April 16, 2024, at 2:00–3:00 pm EST
This project is supported by the Substance Abuse and Mental Health Services Administration (SAMHSA), the agency within the U.S. Department of Health and Human Services that leads public health efforts to advance the behavioral health of the nation. The Crisis 

Systems Response Training & Technical Assistance Center works in conjunction with the 988 Suicide & Crisis Lifeline. In 2020, Congress designated the new 988 dialing code to be operated through the existing National Suicide Prevention Lifeline. SAMHSA sees 988 as a 

first step towards a transformed crisis care system in America. Points of view or opinions in this document are those of the author and do not necessarily represent the official position or policies of SAMHSA or the 988 Suicide & Crisis Lifeline.

Contact Information 

SAMHSA’s 988 CSR TTAC

• support@988crisisttac.org

• 844-464-8338 (toll free)

Subscribe to the CSR-TTAC contact list to get the 

latest 988 news and invitations to our events, or 

use the QR code to the left

Crisis Systems Response Training and Technical Assistance | SAMHSA
Appendix 
System Composition and Collaboration Study

The System Composition and Collaboration Study seeks to understand the structure of each 

state or territory 988 Lifeline and crisis services continuum, as well as the extent to which crisis 

continuum agencies work together to support 988 implementation. 

• The annual System Implementation Survey will gather information from funded states and territories 

about the structure of their crisis system, partnerships and collaboration efforts in place to support 988, 

funding sources, and barriers/facilitators to implementation.

• All agencies providing crisis continuum services within each state and territory will complete the bi-annual 

Crisis Continuum Provider Survey, which examines workforce composition, partnership engagement and 

collaboration, and policies and procedures that guide crisis work. 

• A series of annual 988 Implementation Case Studies (including Cost Case Sub-Studies) will provide indepth picture of 988 successes, barriers, funding, and collaboration efforts at the national, state/territory, 

and provider levels.

• Grantee Reports and Performance Data will be used to gather additional insight into partnerships, crisis 

services providers within each state/territory, and 988/911 collaboration.
System-Level Service Utilization and Resolution Study

The System-Level Service Utilization Study examines the relationship between 988 implementation and system-wide 

changes in crisis continuum service utilization as well as the impact of 988 on other systems of care to determine success 

in directing behavioral health crisis response to appropriate resources and decreasing costs.

• An Excel-based Grantee Document and Data Review Form will be used to abstract information from grantee reports 

related to service utilization, including contact volume, contact dispositions, follow-up program enrollment, ED or justice 

system diversions, relevant IPP indicators, and (where applicable) KPIs. 

• Extant data from the Grantee Document and Data Review Form will be summarized and used to populate the System 

Outcome Summary Inventory each quarter. Grantees will be asked to review, confirm, and if needed, revise data.

• Data from the System Implementation Survey and the System Outcome Summary Inventory, will be used to 

understand the relationship between contact resolutions, crisis service engagement, crisis system characteristics, and 

system-level functional outcomes. 

• Data from the Crisis Continuum Provider Survey, described previously, will be used to understand characteristics of 

the crisis workforce and other center-level factors that may impact system-level service utilization. 

• Extant emergency services and behavioral health service utilization data will be used to assess the relationship 

between 988 and related systems of care, including considerations of cost savings.
Client-Level Service Utilization and Outcome Study

The Client-Level Service Utilization and Outcome Study explores the effectiveness of the 988 

Lifeline in linking help-seekers to services and supports after the initial crisis contact, as well as the 

relationship between 988 continuum contact and behavioral health outcomes and client perceptions of 

care. 

• The Client Contact Disposition Form will be completed by crisis counselors for each client interested in 

evaluation participation, and will document the presenting concern, contact disposition, and response. 

• The Client Experience Survey (conducted 1 week after initial contact, 3 months, 6 months, and 12 months) 

will be used to assess behavioral health outcomes, service engagement, perceptions of care, and pathways 

through care. 

• Extant Lifeline Post-Contact Survey Data, where available, will be used to understand the relationship 

between immediate, short-term, and long-term behavioral health outcomes and provide additional information 

about initial counselor-help seeker rapport. 

• Data from the Grantee Document and Data Review Form, where available, will be used to triangulate 

client-provided information about risk assessment and follow-up information, as well as aggregate suicide 

attempt and death data (i.e., IPP indicators). 
Client-Level Risk Reduction Study

The Client-Level Risk Reduction Study assesses the extent to which the 988 Lifeline effectively 

reduces immediate suicidal, homicidal, and overdose risk. This study also provides in-depth 

understanding of client experiences with 988.

• A sample of recorded crisis contacts will be coded to assess the efficacy of 988 in reducing 

immediate suicidal, homicidal, and substance use risk throughout the course of the initial crisis 

contact, along with the counselor behaviors and contact characteristics most closely associated with 

risk reduction. 

• A series of client key informant interviews will be conducted to better understand client experience 

navigating the 988 crisis continuum, satisfaction with services and providers, and barriers and 

facilitators to service engagement.

• Extant Lifeline Post-Contact Survey Data, where available, will be used to understand the 

relationship between observed and self-assessed risk status changes. 
Impact Study

The Impact Study examines the impact of the 988 Lifeline and Crisis Services Continuum in 

decreasing suicide and overdose morbidity and mortality.

• Extant behavioral health service utilization data (i.e., HCUP, NSDUH, and Medicaid claims 

data) will be used to understand links between 988 implementation, behavioral health service 

utilization, and morbidity and mortality data.

• Extant suicide and overdose morbidity data (i.e., CDC Wonder and data) will be used to 

examine changes in suicide and overdose morbidity related to 988 implementation. 

• Extant suicide and overdose mortality data (i.e., CDC Wonder and Medicaid claims data) will be 

used to examine changes in suicide and overdose mortality related to 988 implementation.

• Extant Lifeline metrics and grantee KPIs, along with data from the System Implementation 

Survey, will be used to assess system-level characteristics that influence service utilization, 

morbidity, and mortality data.