The Webinar Will Begin Shortly. 

Please remember to mute your phone lines and 

computers. 

Please use this link for closed captioning: 

http://www.fedrcc.us//Enter.aspx?EventID=3261304&CustomerID=321 

This link is also included in the invite. 

If you have any questions or concerns please let us know VIA chat. 
What is ACL’s Interest in HCBS 

Quality?

Webinar 

April 5, 2017 2pm EST
Webinar Agenda 

i. Introduction: 

1. Welcoming Remarks: Edwin Walker, J.D. 

ii.Overview of ACL Quality Initiatives:

1. Overview of quality measurement: Susan Jenkins, PhD 

2. National Quality Forum work on HCBS quality: Eliza Bangit, J.D. 

3. National Core Indicator Contract: Shawn Terrell, MSW

4. Rehabilitation and Research and Training Center on Outcomes 

Measurement – Amanda Reichard, PhD

iii.Closing:

1. Questions/comments 

2. Next webinar topic and date 

3
Welcoming Remarks

Edwin Walker, J.D., Deputy Assistant Secretary for Aging 

Why it is important and the progress made 

4

ACL’s involvement in Home and Community-Based Services (HCBS) 

quality measures: 
What is ACL’s Interest in HCBS Quality?

The Basics of Quality Measurement 

Susan Jenkins, PhD

Director, Office of Performance and Evaluation 
Why is Quality Measurement Important? 

The original mission of program evaluation and 

performance management was to assist in improving the 

quality of programs. 

6
GPRA-Modernization Act of 2010

The GPRA Modernization Act of 2010 

(GPRAMA) aims to ensure that agencies 

use performance information in decision 

making and holds them accountable for 

achieving results and improving 

government performance. 

7
Office of Management and Budget

●In Memorandum M-13-17 the Office of Management and 

Budget provides guidance to Federal agencies about 

“harnessing evidence and evaluation”

● It discussed delivering “a smarter, more innovative, and 

more accountable government for citizens. 

● An important component of that effort is strengthening 

agencies' abilities to continually improve program 

performance by applying existing evidence about what works, 

generating new knowledge, and using experimentation and 

innovation to test new approaches to program delivery.” 

8
FY 2018 Budget Blueprint

The Administration will take an evidence-based approach to 

improving programs and services—using real, hard data to 

identify poorly performing organizations and programs. We will 

hold program managers accountable for improving performance 

and delivering high-quality and timely services to the American 

people and businesses...”

9
Why Measure? 

For many reasons, including: 

• Measures drive improvement. 

• Measures inform consumers and other stakeholders. 

• Measures influence payment. 

Measurement is a quality improvement tool, not an end in 

and of itself 

• A performance measure is a way to calculate whether and how 

often the system does what it should. 

10
What to Measure?

Not everything that counts can be counted, and not everything 

that can be counted counts

But...

You can’t improve what you don’t measure

11
Areas for Measurement

1. Quality 

– Structures of care 

– Processes of care 

– Outcomes 

– Intermediate clinical outcomes 

– Health outcomes (mortality, complications, etc.) 

– Patient-reported outcomes (experience, functional status, engagement, 

quality of life, etc.) 

2. Resource use/cost 

3. Efficiency (combination of quality and 

resource use) 

12
How to Create Measures?

In order to be most effective, objectives should be 

clear and leave no room for interpretation. S-M-A-R-T 

is a helpful acronym for developing objectives that 

are 

●specific, 

●measurable, 

●achievable, 

●relevant, and 

●time-bound

● Source: http://www.cdc.gov/healthyyouth/evaluation/pdf/brief3b.pdf 

13
Explanation of a basic performance 

formula

14
Cartoon Depiction 

15
What is ACL’s Interest in HCBS Quality?

Quality in Home and Community-Based Services to Support 

Community Living:

Addressing Gaps in Performance Measurement

Eliza Navarro Bangit, J.D.

Director of the Office of Policy and Analysis and Development
National Quality Forum (NQF)

• The organization that endorses national 

consensus standards for performance 

measurement located here: 

http://www.qualityforum.org/story/Abou

t_Us.aspx

•NQF is the designated consensus-based 

entity

17
Project Components

• Develop a standard home- and communitybased services (HCBS) definition

• Design a conceptual framework for HCBS 

measurement, including domains and 

subdomains of development

• Identify characteristics of high-quality HCBS

• Produce an environmental scan of existing HCBS 

measure, measure concepts, and instruments

18
Project Components (cont.)

•Identify promising measures and measure 

concepts

•Identify gaps in measurement 

• Craft recommendations for prioritization 

in HCBS measurement

19
How Was This Project Conducted?

• NQF formed a 22-person multi-stakeholder committee of 

national experts on HCBS quality

• NQF partnered with a five-person Federal Liaison Team from 

HHS to help advise on its work

• NQF held Committee in-person and Web meetings (open to 

the public), workgroup calls, online surveys

• NQF published four reports and taken public comments - final 

report released in September 2016

20
HCBS Definition

• The term “home and community-based services” (HCBS) refers 

to an array of services and supports delivered in the home or 

other integrated community setting that promote the 

independence, health and well-being, self-determination, and 

community inclusion of a person of any age who has 

significant, long-term physical, cognitive, sensory, and/or 

behavioral health needs. 

21
Characteristics of High Quality HCBS (1 of 4) 

• Provides for a person-driven system that optimizes individual 

choice and control in the pursuit of self-identified goals and life 

preferences 

• Promotes social connectedness and inclusion of people who 

use HCBS, in accordance with individual preferences 

• Includes a flexible range of services that are sufficient, 

accessible, appropriate, effective, dependable, and timely to 

respond to individuals’ strengths, needs, and preferences and 

that are provided in a setting of the individual’s choosing 

22
Characteristics of High Quality HCBS (2 of 4) 

• Integrates healthcare and social services to promote well-being 

• Promotes privacy, dignity, respect, and independence; freedom 

from abuse, neglect, exploitation, coercion, and restraint; and 

other human and legal rights 

• Ensures each individual can achieve the balance of personal 

safety and dignity of risk that he or she desires 

• Supplies and supports an appropriately skilled workforce that 

is stable and adequate to meet demand

23
Characteristics of High Quality HCBS (3 of 4) 

• Supports family caregivers

• Engages individuals who use HCBS in the design, 

implementation, and evaluation of the system and its 

performance

• Reduces disparities by offering equitable access to, and 

delivery of, services that are developed, planned, and provided 

in a culturally sensitive and linguistically appropriate manner

• Coordinates and integrates resources to best meet the needs 

of the individual and maximize affordability and long-term 

sustainability

24
Characteristics of High Quality HCBS (4 of 4) 

• Delivers—through adequate funding—accessible, affordable, 

and cost-effective services to those who need them

• Supplies valid, meaningful, integrated, aligned, accessible, 

outcome-oriented data to all stakeholders

• Fosters accountability through measurement and reporting of 

quality of care and recipient outcomes

25
Conceptual Framework 

26
HCBS Domains and Subdomains

• 11 Domains and 40 Subdomains

• Domains

– Service Delivery and Effectiveness

– Person-Centered Planning and Coordination

– Choice and Control

– Community Inclusion

– Caregiver Support

– Workforce

– Human and Legal Rights

– Equity

– Holistic Health and Functioning

– System Performance and Accountability

– Consumer Leadership in System Development

27
Gaps in HCBS Measurement

• Gaps within all of the domains and 

subdomains

• Barriers to measuring HCBS quality include:

– Lack of standardized measures

– Lack of or limited access to timely data

– Variability in reporting across programs

– Administrative burden

28
Global Recommendations to Advance 

HCBS Quality Measurement

• Support quality measurement across all domains and subdomains 

• Build upon existing quality measurement efforts

• Develop and implement a standardized approach to data collection, 

storage, analysis, and reporting 

• Ensure that emerging technology standards, development, and 

implementation are structured to facilitate quality measurement 

• Triangulate assessment of HCBS quality using an appropriate balance of 

measure types and units of analysis

• Develop a core set of standard measures for use across the HCBS system, 

along with a menu of supplemental measures that are tailorable to the 

population, setting, and program 

• Convene a standing panel of HCBS experts to evaluate and approve 

candidate measures 

29
Resources

NQF page on Measuring HCBS Quality 

• http://www.qualityforum.org/Measuring_HCBS_Quality.aspx

Final Report

• http://www.qualityforum.org/Publications/2016/09/Quality_in

_Home_and_CommunityBased_Services_to_Support_Community_Living__Addressing_

Gaps_in_Performance_Measurement.aspx

Project archive

• http://www.qualityforum.org/ProjectMaterials.aspx?projectID

=77692

30
What is ACL’s Interest in HCBS Quality?

National Core Indicator Contract

Shawn Terrell, MSW

Office of Policy Analysis and Development 

31
Developing HCBS Quality Measures from 

National Core Indicators for Intellectual and 

Development Disabilities, and Aging and 

Physical Disabilities (NCI-IDD/AD)

• The NCI-IDD/AD: 

– Two distinct but related sets of survey instruments

– Designed to assess state systems performance 

along a number of key indicators related to 

community living for various populations.

32
NCI

Focus population: 

Adults with I/DD who receive at least one service in 

addition to case management from state IDD systems 

 - NCI began development in 1997

– Survey Suite: Adult Consumer, Family, Staff Stability

– 46 states and DC

– Funded in by state membership fees with ACL support for 

specific expansion activity 

– Most participants are receiving services under an HCBS 

Waiver 

– A small % are either state only, or ICF/ID

33
NCI-AD

Focus population:

Older adults and adults with physical disabilities accessing publicly funded 

services in:

– Skilled Nursing Facilities

– Medicaid Waivers

– Medicaid State Plan

– State Funded Programs

– Older Americans Act Programs

– PACE

– MLTSS

• NCI-AD began development in 2012

• Adult in-person survey only 

• Pilot report on initial 3 states published 2014

• Survey was revised in 2015 based on pilot results

• 20 states are participating in 2017-2018 survey year.

34
NCI/NCI-AD Survey State Participants

35
NCI-AD/IDD Indicators

NCI-IDD

• Community Inclusion

• Choice and Decision Making

• Relationships

• Satisfaction

• Service Coordination

• Work

• Self-Determination

• Access 

• Health

• Medications

• Wellness

• Respect and Rights

• Safety

NCI-AD

• Community Participation

• Choice and Decision Making

• Relationships

• Satisfaction

• Service Coordination

• Care Coordination

• Work

• Self-Direction

• Access

• Health Care

• Medications

• Wellness

• Rights and Respect

• Safety

• Everyday Living

• Affordability

• Future Planning

• Control

36
ACL Contract with Human Services Research 

Institute 

• NASDDDS and NASUAD are collaborators 

• Refine and expand use of NCI and NCI-AD surveys:

– Publish results in peer reviewed journals:

• Psychometric testing (reliability, validity)

• Sampling procedures

• Interview protocols

• Implementation consistency & validity across states

– Revise existing measures for person-centered planning 

questions/survey to adapt to current expectations

37
ACL Contract with Human Services Research 

Institute 

• Submit at least 20 measures for NQF 

endorsement from NCI and NCI-AD surveys

• Technical Assistance to States

– Sampling procedures

– Implementing surveys and analyzing results

– How to use results for system improvement

– Support recruitment of additional NCI and NCI-AD

38
What is ACL’s Interest in HCBS Quality?

Rehabilitation Research Training Center: University of Minnesota 

Home and Community Based Services Outcome Measurement 

Amanda Reichard, PhD

National Institute on Disability, Independent Living, and 

Rehabilitation Research 
Research and Training Center (RRTC) on 

Outcomes Measurement for Home and 

Community Based Services

• ACL’s (NIDILRR) announced a new grant on 

September 30, 2015 – the Research and 

Training Center on Outcomes Measurement for 

Home and Community Based Services

– Grantee: University of Minnesota 

– The grant is for $875,000 per year for 5 years. 

40
UMN RRTC on Outcomes Measurement for 

Home and Community Based Services

• Through this grant measures are being developed that can be 

used across populations of people with disabilities and aging 

populations who receive HCBS services. 

• The grantee team includes partners and collaborators 

organizations that cross disability populations and age groups.

41
The UMN RRTC Responds to the Following 

NIDILRR/ ACL Priorities:

• Identify or develop measures, and then test the reliability, validity, and 

usability of those measures to assess the person-centered outcomes of 

individuals with disabilities who are receiving home and community-based 

services

• Work closely with NIDILRR, ACL, and the National Quality Forum’s project on 

HCBS Quality

• Develop procedures and mechanisms for applying HCBS outcome measures 

in policy and service delivery settings to maximize quality and 

appropriateness of HCBS from the end-user perspective

• Collaborate with stakeholder groups to develop, evaluate, or implement 

strategies to increase utilization of new HCBS outcome measures

• Serve as a national resource center related to person-centered 

measurement of HCBS outcomes

42
Starting Point 

NQF OHCBS Outcome Measurement Framework

• 11 Domains (each with 2 to 7 Subdomains):

– Consumer Leadership in System Development 

– System Performance & Accountability 

– Equity

– Service Delivery & Effectiveness

– Person-Centered Service Planning and Coordination

– Caregiver Support

– Workforce

– Holistic Health and Functioning

– Community Inclusion

– Human and Legal Rights

– Choice and Control

43
RRTC/OM Research Studies

• Study 1: Soliciting broad stakeholder input on NQF HCBS 

Measurement Framework

• Study 2: Gap analysis – NQF HCBS Measurement 

Framework & Current Instruments 

• Study 3: Identification of high quality/fidelity 

implementation practices

• Study 4: Refinement and development of measures

• Study 5: Ascertaining Reliability, Validity & Sensitivity to 

Change of Measures

• Study 6: Identification & testing of risk adjusters

44
Q&A:

“HOW DO YOU MEASURE HCBS 

QUALITY?”

45
Next Webinar: June 28, 2pm EST 

Representatives from the National Quality Forum and the SCAN 

Foundation will discuss the progress made in quality framework 

development in our next webinar. 

• National Quality Forum Final 

Report: 

– Quality in Home and 

Community-Based Services to 

Support Community Living: 

Addressing Gaps in 

Performance Measurement

• The SCAN Foundation 

Publication: 

– Essential Attributes of a 

High-Quality System of Care 

for Adults with Complex 

Care Needs

46