Quality Measurement as a Tool for 

Continuous Quality Improvement at 

Certified Community Behavioral Health 

Clinics (CCBHCs)

June 1, 2023

Peggy O’Brien, PhD & Shweta Palakkode, MPH 

Center for Behavioral Health Statistics & Quality, Office of 

Evaluation

Substance Abuse and Mental Health Services Administration

U.S. Department of Health and Human Services
This webinar is being recorded and closed captioning is 

provided. Slides and the recording will be available 

about a week after the presentation. They will posted

on the SAMHSA CCBHC website for later reference.

Please mute your lines (Thank you!).

Please turn off your cameras to conserve bandwidth.

We will have time for questions at the end. Please feel 

free to put questions in the Q&A throughout the 

presentation. We will address them at the end. Please 

use the chat for logistics issues only.

2

Logistics
• Our panelists today:

– From the State of New Jersey Department of Human Services: Robert 

Eilers, MD, Medical Director; Vicki Fresolone, Manager of Integrated 

Services; and Charlotte Sadashige, Primary Researcher for the State’s 

CCBHC program

– Emma Kelly-Robinson, Operations Manager for Options of Southern 

Oregon

• The CCBHC SAMHSA team in the Center for Mental Health Services 

(CMHS)

• National Council for Mental Wellbeing 

• A-G Associates

• Selby (Zoom expert)

3

Acknowledgments
Please select the options that accurately describe your role:

1. Potential CCBHC demonstration state

2. Existing CCBHC demonstration state

3. Potential CCBHC demonstration clinic

4. Existing CCBHC demonstration clinic

5. Independent state CCBHC initiative

6. CCBHC-PDI grantees

7. CCBHC-IA grantees

8. Other (feel free to identify your role if not listed above)

4

Poll #1: Who is in the audience today? 

This Photo by Unknown Author is licensed under CC BY-SA
1. Prospective and existing CCBHC Section 223 

Demonstration state staff

2. Prospective and existing CCBHC Section 223 

Demonstration CCBHCs or prospective CCBHCs

3. Independent state CCBHC initiative personnel

4. CCBHC-Expansion (CCBHC-PDI and -IA) grant 

clinics and SAMHSA GPOs

5

Intended audience for this webinar
What is 

continuous 

quality 

improvement 

(CQI)? 

The value of 

quality 

measurement 

for quality 

improvement 

Guidance on 

preparing to 

use quality 

measurement 

for CQI 

Preview of 

future 

webinars 

Questions 

6 

Agenda 
• Quality improvement is the 

framework used to systematically 

improve care. 

7

What is a quality improvement? 

Source: Centers for Medicare & Medicaid Services. Quality Measurement and Quality Improvement | CMS
• CQI is the continuous

and ongoing effort to 

achieve measurable 

improvements in the 

efficiency, effectiveness, 

performance, 

accountability, 

outcomes, and other 

indicators of quality 

improvement. 

8

What is continuous quality improvement (CQI)? 

Source: MN Department of Health Continuous Quality Improvement - MN Dept. of Health (state.mn.us)

Visual based on source cited.
Continuous quality 

improvement 

requirements in the 

updated Certification 

Criteria 

9 SAMHSA 2023 Updated Criteria (pdf)
5.b.1

The CCBHC: 

• Develops, implements, and maintains an effective, CCBHC-wide 

continuous quality improvement (CQI) plan for services provided

• Establishes a critical review process to review CQI outcomes and 

implement changes to staffing, services, and availability that will improve 

the quality and timeliness of services

The CQI plan focuses on:

• Indicators related to improved behavioral and physical health outcomes 

and takes actions to demonstrate improvement in CCBHC performance

• Improved patterns of care delivery, such as reductions in emergency 

department use, rehospitalization, and repeated crisis episodes. 

The Medical Director is involved in the aspects of the CQI plan that apply to 

the quality of the medical components of care, including coordination and 

integration with primary care.

10

Criteria 5.B: Continuous Quality Improvement (CQI) Plan 
5.b.2

The CQI plan is to be developed by the CCBHC and addresses 

how the CCBHC will review known significant events including, 

at a minimum: 

(1) deaths by suicide or suicide attempts of people receiving 

services; 

(2) fatal and non-fatal overdoses; 

(3) all-cause mortality among people receiving CCBHC 

services; 

(4) 30 day hospital readmissions for psychiatric or substance 

use reasons; and 

(5) such other events the state or applicable accreditation 

bodies may deem appropriate for examination and 

remediation as part of a CQI plan.

11

Criteria 5.B: Continuous Quality Improvement (CQI) Plan 
5.b.3

The CQI plan:

• Is data-driven and considers use of quantitative and 

qualitative data in CQI activities

• At a minimum, addresses the data resulting from the 

CCBHC-collected and, as applicable, State-Collected, 

quality measures required as part of the Demonstration. 

• Includes an explicit focus on populations experiencing 

health disparities and addresses how the CCBHC will use 

disaggregated data from the quality measures and, as 

available, other data to track and improve outcomes for 

populations facing health disparities.

12

Criteria 5.B: Continuous Quality Improvement (CQI) Plan 
Value and use of 

quality measures for 

CQI: Insights from 

Evaluation

13

The Photo by Unknown Author is licensed under CC BY-SA
• Changes in quality of care due to clinic-collected measures:

o Clinics adapted to the following routines as a result of quality 

measures:

 Conducting screening and assessments (alcohol, tobacco, depression, 

suicidality) 

 Using structured and standardized instruments for screening

 Conducting and documenting follow-up to those screenings 

 Conducting rapid or same-day client intakes

• Limited evidence on changes in quality of care due to state- collected measures

14

Insights from evaluation on value of quality measures
Measure Description 

Measure 

Name MN MO NJ NY OK OR PA

Time to Initial Evaluation, adult I-EVAL

Child and Adolescent Major Depressive 

Disorder: Suicide Risk Assessment SRA-BH-C

Adult Major Depressive Disorder: 

Suicide Risk Assessment SRA-A

Screening for Clinical Depression and 

Follow-up Plan CDF-BH

Depression Remission at 12 Months DEP-REM-12 n/a

Adult Body Mass Index Screening and 

Follow-up Plan BMI-SF

Weight Assessment for Nutrition and 

Physical Activity for Children/ 

Adolescents

WCC-BH

Tobacco Use - Screening and 

Cessation Intervention TSC

Unhealthy Alcohol Use - Screening 

and Brief Counseling ASC

15

Change in quality of care for CCBHC clients during Demonstration Years One and Two, 

clinic-collected measures

Improved 5% or more improvement in aggregate performance across CCBHCs in state from DY1 to DY2.

Declined = 5% or more decline in aggregate performance across CCBHCs in state from DY1 to DY2.

Stable. Source: Derived from Table 7, ASPE. Report to Congress 2021: Certified Community Behavioral Health Clinics Demonstration Program
Preparing to use quality 

measurement for CQI –

Insights from the Field

16
State of New Jersey: Dr. Robert Eilers, Vicki Fresolone, and Charlotte Sadshige

Options for Southern Oregon (CCBHC): Emma Kelly-Robinson

17

Insights from the field on the use of quality measures

Panel Discussion
What are the greatest benefits you have 

seen from using quality measures to 

improve quality of services in the 

CCBHCs? 

18

Question 1 – For CCBHCs and States
What techniques did your CCBHC use to 

provide feedback to your providers and 

staff in the CCBHC on quality measure 

results and needed action? 

19

Question 2 – For CCBHCs
Have you been successful in getting 

feedback on your state-collected measure 

results to the CCBHCs? 

How did you typically provide feedback? 

Did you use a certain process to expedite the 

provision of timely feedback so that CCBHCs could 

react more quickly? 

20

Question 3 – For States
What would you have wanted to know about 

using QMs for CQI before you started the 

Demonstration? 

What other advice about using data for 

quality improvement might you give to those 

who are gearing up for the Demonstration or 

for newly reporting CCBHC-Es?

21

Question(s) 4 – For CCBHCs and States
Do you have any advice for new 

demonstration states or clinics regarding 

how to optimize use of the Quality Bonus 

Measures to improve quality and to 

improve revenue?

22

Question 5 – For CCBHCs and States
Upcoming Webinars

23
July 20, 2023 

‒ Preparing CCBHCs for quality measure data 

collection and reporting (administrative, 

technical, and clinical aspects)

September-November

‒ Five measure-specific webinars

December

‒ Introduction to the reporting format/template

24

Preview of future webinars in 2023
25

Preview of Webinar 3: How clinics can prepare to collect data

Work with your EHR vendor to build out needed technical 

infrastructure 

Work across departments that include IT staff, clinical teams, 

administrative staff to implement into existing workflow

Understand appropriate codes for billing and codes to record 

other aspects of performance 

Make use of learning collaboratives
26

Questions
• When will the technical specifications for the 

measures be available? Measure specifications will be 

released in late August or very early September. 

Measure-specific webinars begin then and are yet to 

be scheduled.

• Do CCBHC-IAs or PDIs who are either not in a 

demonstration state or not a demonstration statecertified CCBHC have to report the “state-collected” 

measures? No, only the five clinic-collected measures.

27

Answers to a few questions from the last webinar (1)
• What SDOH screening tools are acceptable for the 

SDOH measure? 

– It must be a standardized tool that allows you to screen for: 

food insecurity, housing instability, transportation needs, utility 

difficulties, and interpersonal safety.

– No specific tool is required but ones suggested include: 

• Accountable Health Communities Health-Related Social Needs Screening Tool

(2017) 

• Accountable Health Communities Health-Related Social Needs Screening Tool

(2021)

• The Protocol for Responding to and Assessing Patients’ Risks and Experiences (PRAPARE) Tool

(2016) 

• WellRx Questionnaire (2014)

• American Academy of Family Physicians (AAFP) Screening Tool (2018)

28

Answers to a few questions from the last webinar (2)
• Will all CCBHC-IA or CCBHC–PDI grantees in a demonstration 

state be eligible for the Medicaid Prospective Payment 

System (PPS)? No, a grantee clinic must be in a 

demonstration state and be selected by that state to 

participate in the demonstration and be certified by the state 

as a demonstration clinic.

29

Answers to a few questions from the last webinar (3)
In the last hour, I have learned (please 

select the best option):

A. A lot of useful new information

B. Some useful new information

C. Very little new information

D. Not sure

30

Poll #2
Thank you

31

SAMHSA’s mission is to reduce the impact of substance 

abuse and mental illness on America’s communities.

www.samhsa.gov

1-877-SAMHSA-7 (1-877-726-4727) ● 1-800-487-4889 (TDD)
32

Supplemental Slides
33

Updated clinic-collected quality measures

Measure Steward Changes

Required

Time to Services(I-SERV) SAMHSA Replaces I-EVAL, includes average time to: Initial Evaluation, 

Initial Clinical Services, and Crisis Services

Depression Remission at Six Months (DEP-REM-6) MNCM Changed from the 12- month version, with updates

Preventive Care and Screening: Unhealthy AlcoholUse: Screening 

and Brief Counseling (ASC)

NCQA Updated, based on MIPs measure

Screening for Depression and Follow-Up Plan (CDF-CH and CDF-AD) CMS Updated, based on the Medicaid Core Measure Set (MCMS) 

(2023)

Screening for Social Drivers of Health (SDOH) CMS New, based on MIPS measure 

Optional

Preventive Care & Screening: Tobacco Use: Screening & 

Cessation Intervention (TSC)

NCQA Updated, based on MIPS measure

Child and Adolescent Major Depressive Disorder (MDD): Suicide

Risk Assessment(SRA)

(SRA-C)

Mathematica Updated

Adult Major Depressive Disorder: Suicide Risk Assessment (SRA) 

(SRA-A)

Mathematica Updated

Weight Assessment and Counseling for Nutrition and Physical 

Activity for Children/Adolescents(WCC-CH)

NCQA Updated, based on the MCMS (2023)

ControllingHigh Blood Pressure (CBP-AD) NCQA Updated, based on the MCMS (2023)
34

Updated state-collected quality measures (slide 1)

Measure Steward Changes

Required

Patient Experience of Care Survey SAMHSA Updated

Youth/Family Experience of Care Survey SAMHSA Updated

Adherence to Antipsychotic MedicationsforIndividuals

with Schizophrenia (SAA-AD)

CMS Updated, based on the Medicaid Core Measure Set (MCMS) 

(2023)

Follow-UpAfter Hospitalization for Mental Illness, ages 

18+ (adult) (FUH-AD)

NCQA Updated, based on the MCMS (2023)

Follow-UpAfter Hospitalization for Mental Illness, ages 6 

to 17 (child/adolescent) (FUH-CH)

NCQA Updated, based on the MCMS (2023)

Initiation and Engagement of Substance Use Disorder 

Treatment (IET-AD)

NCQA Updated, based on the MCMS (2023)

Follow-Up After Emergency Department Visit for Mental 

Illness(FUM-CH and FUM-AD)

NCQA Updated, based on the MCMS (2023)

Follow-Up After Emergency Department Visit for 

Substance Use (FUA-CH and FUA-AD)

NCQA Updated, based on the MCMS (2023)
35

Updated state-collected quality measures (slide 2)

Measure Steward Changes

Required (cont’d from prior slide)

Plan All-Cause Readmissions Rate (PCR-AD) NCQA Updated, based on the Medicaid Core Measure Set (MCMS) 

(2023)

Follow-Up Care for Children Prescribed Attention-Deficit 

HyperactivityDisorder(ADHD) Medication (ADD-CH)

NCQA Updated, based on the MCMS (2023)

AntidepressantMedication Management (AMM-AD) NCQA Updated, based on the MCMS (2023)

Use of Pharmacotherapy for OpioidUse Disorder(OUDAD)

CMS Added, based on the MCMS (2023)

Hemoglobin A1c Control for Patientswith Diabetes

(HBD-AD)

NCQA Added, based on the MCMS (2023)

Optional

Use of First-Line Psychosocial Care for Children and

Adolescents on Antipsychotics (APP-CH)

NCQA Updated, based on the MCMS (2023)

Metabolic Monitoring for Children and Adolescents on 

Antipsychotics (APM-CH)

NCQA Added, based on the MCMS (2023)