A Person-Centered, Longitudinal Approach to HCBS 

Outcome Measurement

May 3rd, 2022
What is HCBS Outcome Measurement?

• The goal of HSBS Outcome Measurement is to 

better understand...

• The quality of services and supports received by HCBS 

recipients

» Timeliness

» Based on best-practices

» Coordinated 

» Meet the recipients’ needs 

• The outcomes recipients experience when 

services and supports are received

» Are outcomes person-centered

» Meaningful, and 

» Contribute to a high quality of life 
Levels of Measurement

• HCBS measurement can take place at a variety 

of levels

• National

• State

• Provider

• Individual
Levels of Measurement & Need for Precision
Importance to Providers 

• Documentation of high-quality services & 

outcomes

• Having data to market the quality of services and 

supports you provide to HCBS recipients

• Meeting state and federal service provision 

guidelines

• Quality improvement efforts

• It’s difficult to fix something if you have no data 

that suggests that thing needs fixing

• Data is needed to support that program 

innovations are having their intended outcomes 
Principles Underlying Measure Development Process

Outcome Measures need to be:

• Person-centered

• Sensitive to change over time (can be used 

longitudinally)

• Feasible to administer, minimizing respondent and 

provider burden

• Guided by National Quality Forum Framework for 

HCBS Outcome Measurement (revised); and

• Have utility at service provider and individual levels 

(actionable data) with the capacity to contribute to 

Quality Improvement efforts
Person-Centered Measurement

• Measurement is person-centered when the 

individual’s experiences, needs, goals, and values,

as expressed by persons themselves, inform 

decisions about..

 What we measure 

 How we assess outcomes, and 

 How we evaluate the performance of communitybased supports

• Has many dimensions and is not the same as assessing 

person-centered supports and practices

 Should underlie all measurement in this area

• Balances measurement of what is important for the person 

with what is important to the person.
Revised National Quality Forum Measurement Framework
Measure Development - Selecting Concepts

 Phase 1: Participatory Planning and Decision-Making w/ 

Stakeholder Groups

• People with disabilities, family members, providers, policy makers and 

researchers

• Provided input on NQF framework 

• Importance weightings w/ respect to domains most important to measure

 Phase 2: Gap Analysis

• Development of RTC/OM database of existing measures, assessments

• Coded items on NQF Framework coverage from over 130 HCBS-related 

instruments 

• Results with input from RTC/OM Leadership and National Advisory groups led 

to development of measures in 13 NQF Domains/ Subdomains.

 Phase 3: Technical Expert Panels

• Weightings of Importance, utility, & feasibility of measures
Phase 4: Measure Development - Prioritized Domains/Subdomains

• Choice & Control

• Community Inclusion

• Employment

• Human/Legal Rights

• Transportation 

• Consumer Leadership in System 

Development

• Person-Centered Planning & Coordination

• Personal Choices & Goals

• Choice of Services & Supports

• Self-Direction

• Meaningful Community Activities

• Social Connectedness

• Currently Employed

• Seeking Employment

• Freedom from Abuse/Neglect

• Access to, affordability & quality of 

transportation

• Meaningful Involvement

• System Support & Resources

• Person-Centered Assessment

• Person-Centered Planning

• Person-Centered Service Delivery & Coordination
Phases of Measure Iterative Development Process

• Initial Measure Development

 National Quality Forum HCBS Outcome Measurement 

Framework

 Intensive review of current research literature related to 

each measurement construct

 Development of guiding questions/claim statements

• Designed for 5 Disability Groups

 Intellectual and Developmental Disabilities

 Physical Disabilities, 

 Psychiatric Disabilities, 

 Traumatic/Acquired Brain Injury, & 

 Age-Related Disabilities
Characteristics of the Measurement System

• Modular in Format

 Each measure can be used independently or in conjunction with 

other measures

• Ability to be Used Longitudinally

 Measures need to be able to be sufficiently sensitive to detect 

changes in outcomes in response to policy and programmatic 

changes

• Two - Tiered Measure Structure

 Tier 1: 3-5 general items intended to provide overview of 

outcomes within subdomain

 Tier 2: 10-15 specific items intended to provide more specific, 

actionable data

• Respondents

 Persons with disabilities (whenever possible)

 Proxy nominated by person and knows them well 
Measure Development Trajectory (CMS Measure Development Blueprint)
Measure Development and Testing

• Technical Expert Panels Item Reviews

 Each measure reviewed and revised following feedback 

from 10-20 national content and measurement experts

• Cognitive Testing

 27 PWD from across disability groups

 Comprehension, Judgement, Retrieval, and Response 

Options

• Pilot Testing

 Tested for feasibility with 107 participants from across 

disability groups in 2 states

 Initial psychometric estimates 
Pilot Study - Sample

Sample: N=107 from across five disability 

groups

• MN and PA

• Age: 22 - 101 years

• Race: Representative of U.S. pop. 

• Geographic representation:

» Rural - N=31

» Urban/suburban - N=76
Pilot Study - Data Collection

• Structured Interview

 Qualtrics survey platform (offline app)

• Interview Type

 In-Person (n=85) & Video Conference Interview (n=22)

 Verbal-Only Response Scale or Verbal with Visual 

Representations

• Inter-Rater Observations

• Test-Retest at 10-14 Days
Pilot Summary Table

Measure

Internal Consistency (α) Test-Retest Inter-Rater

Abuse and Neglect .62 .81 .98

Employment: Job Experiences .81 .99 .89

Employment: Barriers .70 .95 .89 

Choice and Control (overall) .85 .76 .94

C&C: Services and Supports .77 .74 .93

C&C: Personal/Daily Choices .75 .72 .98

C&C: Self-Directed Supports Insufficient sample size Insufficient sample size Insufficient sample size

Transportation .86 .76 .92

Social Connectedness .88 .91 .94

Meaningful Activity .94 .79 .92
General Takeaways & Highlights 

• Overall, measures demonstrated good 

psychometric properties

• internal consistency

• test-retest

• interrater reliability)

• Abuse and Neglect and Employment

(Barriers) measures had insufficient 

internal consistency
What We Learned, Challenges, & Changes - Part 1

• Disability groups did not significantly differ with respect 

to their responses to measures

 Initial evidence that we can use similar measures 

across different disability groups

 Indirect evidence for usefulness of NQF Conceptual 

Framework for HCBS Outcome Measurement

• Similar measure completion times for tests and retests 

across disability groups

• Evidence that individuals with significant cognitive 

disabilities could respond to items in a manner that 

appear valid and reliable

• Several response scales changed to ensure sufficient 

variance (some scales initially appeared to have a ceiling 

effect)
What We Learned, Challenges, & Changes - Part 2

Tier 1 and Tier 2 questions across measures

• Tier 1: 2-4 questions intended to broadly capture construct

• Tier 2: Additional items that dig deeper into more specific 

aspects of the construct

• Strong relationships between global and specific items

 Relationship will be further explored in field study

• This study will provide data to analyze these items

 Factor analysis to investigate Global-Specific subdomains

 Examine the relationship between measures
Field Study

• Nationally diverse sample of 1,000 across disability 

groups - target

 Organizations supporting recruitment & data collection 

include UCEDDS, ACL, and data collection organizations

• 8 original measures + 5 additional measures

• Online (Zoom) - majority with in-person data 

collection option

• Three data collection points – to ensure sensitivity 

to change

• Group of up to 400 individuals without disabilities 

who will complete an abbreviated online version
Recruiting for the Field Study!

• We continue to seek MCOs & HCBS provider 

organizations in the U.S. interested in partnering 

with us on recruitment of the people they serve 

to test these measures

 Respondents receive gift cards each time they are 

interviewed

 Minimal effort on part of providers

 Contribute to development of measures you can use 

to determine whether the people you serve are 

experiencing the outcomes they desire

• Contact Matt Roberts (Center Coordinator) at: 

robe0290@umn.edu if you are interested in helping 

us recruit participants from your organization. 
Looking Toward the Future

• Current measurement programs: 

 Goal is “sell” the program either to states or providers

 Organization supports data collection, analysis, & 

interpretation

• RTC/OM goal is not to develop a measurement program but 

rather...

 Generate and validate high quality, person-centered measures that 

provide actionable data for use...

 At the state and provider level

 In quality improvement efforts

 Educate others (States, MCOs, Provider Organization, Advocacy & SelfAdvocacy groups on how to most effectively utilize measures 

developed to enhance...

 Quality of services received by HCBS recipients and

 The outcomes they experience

• Give this information away in a manner that ensures it will be used 

effectively and ethically to improve the lives of HCBS beneficiaries.
Thank you!

Contact Information:

Brian Abery, Ph.D.

University of Minnesota – Twin Cities

Institute on Community Integration

E-mail: abery001@umn.edu

Phone: 612-625-5592 (Office) 612-327-3678 (cell)