U.S. GOVERNMENT PRINTING OFFICE

WASHINGTON : 

For sale by the Superintendent of Documents, U.S. Government Printing Office

Internet: bookstore.gpo.gov Phone: toll free (866) 512–1800; DC area (202) 512–1800

Fax: (202) 512–2250 Mail: Stop SSOP, Washington, DC 20402–0001

95–101 PDF 2005

S. HRG. 108–794

PERFORMANCE AND OUTCOME MEASUREMENT IN

SUBSTANCE ABUSE AND MENTAL HEALTH

PROGRAMS

HEARING

BEFORE THE

SUBCOMMITTEE ON SUBSTANCE ABUSE AND

MENTAL HEALTH SERVICES

OF THE

COMMITTEE ON HEALTH, EDUCATION,

LABOR, AND PENSIONS

UNITED STATES SENATE

ONE HUNDRED EIGHTH CONGRESS

SECOND SESSION

ON

EXAMINING PERFORMANCE AND OUTCOME MEASUREMENT IN SUBSTANCE ABUSE AND MENTAL HEALTH PROGRAMS, FOCUSING ON

THE MISSION OF THE SUBSTANCE ABUSE AND MENTAL HEALTH

SERVICES ADMINISTRATION TO BUILD RESILIENCE AND FACILITATE

RECOVERY

JULY 20, 2004

Printed for the use of the Committee on Health, Education, Labor, and Pensions

(

(II)

COMMITTEE ON HEALTH, EDUCATION, LABOR, AND PENSIONS

JUDD GREGG, New Hampshire, Chairman

BILL FRIST, Tennessee

MICHAEL B. ENZI, Wyoming

LAMAR ALEXANDER, Tennessee

CHRISTOPHER S. BOND, Missouri

MIKE DEWINE, Ohio

PAT ROBERTS, Kansas

JEFF SESSIONS, Alabama

JOHN ENSIGN, Nevada

LINDSEY O. GRAHAM, South Carolina

JOHN W. WARNER, Virginia

EDWARD M. KENNEDY, Massachusetts

CHRISTOPHER J. DODD, Connecticut

TOM HARKIN, Iowa

BARBARA A. MIKULSKI, Maryland

JAMES M. JEFFORDS (I), Vermont

JEFF BINGAMAN, New Mexico

PATTY MURRAY, Washington

JACK REED, Rhode Island

JOHN EDWARDS, North Carolina

HILLARY RODHAM CLINTON, New York

SHARON R. SODERSTROM, Staff Director

J. MICHAEL MYERS, Minority Staff Director and Chief Counsel

SUBCOMMITTEE ON SUBSTANCE ABUSE AND MENTAL

HEALTH SERVICES

MIKE DEWINE, Ohio Chairman

MICHAEL B. ENZI, Wyoming

JEFF SESSIONS, Alabama

JOHN ENSIGN, Nevada

EDWARD M. KENNEDY, Massachusetts

JEFF BINGAMAN, New Mexico

JACK REED, Rhode Island

KARLA CARPENTER, Staff Director

DAVID NEXON, Minority Staff Director

(III)

CONTENTS

STATEMENTS

TUESDAY, JULY 20, 2004

Page

DeWine, Hon. Mike, Chairman, a U.S. Senator from the State of Ohio, opening statement ........................................................................................................ 1

Kennedy, Hon. Edward M., a U.S. Senator from the State of Massachusetts,

prepared statement .............................................................................................. 2

Curie, Charles G., Administrator, Substance Abuse and Mental Health Services Administration .............................................................................................. 4

Prepared statement .......................................................................................... 6

McLellan, A. Thomas, Director, Treatment Research Institute .......................... 18

Prepared statement .......................................................................................... 20

Goldman, Howard H., Professor of Psychiatry, University of Maryland School

of Medicine ............................................................................................................ 21

Prepared statement .......................................................................................... 22

Tester, Gary, Director, Ohio Department of Alcohol and Drug Addiction Services ......................................................................................................................... 27

Prepared statement .......................................................................................... 28

Medalie, Marsha, Vice President and Chief Operating Officer, Riverside Community Care .......................................................................................................... 41

Prepared statement .......................................................................................... 42

ADDITIONAL MATERIAL

Victor A. Capoccia, Ph.D., Senior Program Officer, the Robert Wood Johnson

Foundation, Princeton, New Jersey .................................................................... 54

Eric Goplerud, Ph.D., Executive Director, Ensuring Solutions to Alcohol Problems, The George Washington University Medical Center, prepared statement ...................................................................................................................... 69

(1)

PERFORMANCE AND OUTCOME MEASUREMENT IN SUBSTANCE ABUSE AND MENTAL

HEALTH PROGRAMS

TUESDAY, JULY 20, 2004

U.S. SENATE,

SUBCOMMITTEE ON SUBSTANCE ABUSE AND MENTAL

HEALTH SERVICES,

COMMITTEE ON HEALTH, EDUCATION, LABOR, AND PENSIONS,

Washington, DC.

The subcommittee met, pursuant to notice, at 10:07 a.m., in

Room 430, Dirksen Senate Office Building, Hon. Mike DeWine

(chairman of the subcommittee) presiding.

Present: Senator DeWine.

OPENING STATEMENT OF SENATOR DEWINE

Senator DEWINE. Good morning. Thank you all for being here

today. We are meeting to discuss performance and outcome measurements in substance abuse and mental health programs. With so

much effort and funding focused on these programs, it is, of course,

crucial to understand the effectiveness of these services, especially

as we work on re-authorizing the Substance Abuse and Mental

Health Services Administration. All SAMHSA programs are moving towards a core set of performance measures.

Our hearing today will help us understand just how this will

work. It will also help us better understand the challenges faced

by government and providers in making such an approach work

and how these measurements can over time improve the effectiveness of all substance and mental health services programs.

To the degree possible at this hearing, we want to look at the big

picture. While we are, of course, interested in specific effective approaches to substance abuse and mental health treatment and prevention, our focus today is really on the overall systems of care and

their effectiveness.

According to the Agency for Health Research and Quality, health

care performance measurement is the process of using a tool based

on research, a performance measure to evaluate a managed care

plan, health plan, or program, hospital or health care practitioner.

Performance measures generally are developed to establish clear

standards of accountability that in turn will lead to efforts to improve the quality of care for people with specific health problems.

Performance also implies that the responsible health care providing

entity can be an identified, held accountable, and has control over

the aspect of care being evaluated.

2

Using these sorts of measures can lead us directly to measuring

and understanding the health outcomes associated with programs.

That information then can help policy makers decide where and at

what levels to make program investments. Unfortunately, as reported by many researchers, development of performance measures

in substance abuse and mental services has lagged behind similar

development for many other chronic medical conditions. Despite

that lag, several organizations and initiatives are now focused one

way or another on performance and outcome measurement in substance abuse and mental health.

We have heard Mr. Curie, for example, talk about SAMHSA’s

seven outcome domains. Organizations involved in recent and or

current efforts in the field include the Institute of Medicine, the

Washington Circle Group, the National Committee for Quality Assurance, the National Association of State Mental Health Program

Directors and its research institute, the National Association of

Psychiatric Health Systems, Ensuring Solutions to Alcohol Problems, Joined Together, and, of course, SAMHSA itself.

So while there is a great deal of recently developed information

out there, there is also still much more work to be done to fill in

the gaps in our knowledge. SAMHSA and the Administration for

several years now have been getting ready to fully transform the

current block grants into performance and accountability-based

programs. We have heard often about the seven domains I mentioned earlier. I hope that we can get a clear picture today of exactly where that effort is, more detail about data strategies, where

the current challenges lie, and what we can expect in addressing

these issues as we move forward on SAMHSA re-authorization. I

look forward to hearing the members of our second panel comment

on these issues from their unique perspective.

For me, all of this boils down to doing what works best for people

with mental health and substance abuse problems. That means

using the best information we have to help guide us in the implementation, management, and funding of Federal programs.

At this time, I would like to submit a statement from Senator

Kennedy to be included in the record.

[The prepared statement of Senator Kennedy follows:]

STATEMENT OF SENATOR EDWARD M. KENNEDY

Discoveries in the medical sciences in recent years are bringing

new hope, new treatments, and new cures within reach of millions

of our citizens. The benefits have been well-documented for physical illnesses such as cancer and diabetes, but too little effort has

been made to document the comparable benefits that treatment

makes for people fighting substance addiction and mental illness.

It’s encouraging therefore that the Substance Abuse and Mental

Health Services Administration has been making a significant effort to close this gap in recent years.

Working in coordination with State officials, SAMHSA has led a

broad effort to reach agreement on a range of outcome measurements to demonstrate that treatment and prevention are working

nationwide, improve methods for collecting data, and make the results widely known.

So far, SAMHSA has taken a number of worthwhile steps.

3

They initially involved States, providers and consumers. They decided on seven key outcome measurements to assess the current

state of care in the States. They are moving forward with Access

to Recovery and other grants to test the ability of providers to

measure new outcomes. And they have begun the process of investing resources in system transformation.

But clearly, more needs to be done to reach consensus and this

hearing is a worthwhile first step. Public debate can help to resolve

lasting questions about how to measure new outcomes in a way

that accommodates existing State efforts and produces the most

useful information about State systems of care as well as other

emerging questions that will surface as this process moves forward—as it must.

At this point, it makes sense to review SAMHSA’s proposed

changes in block grant applications for fiscal year 2005 in light of

State concerns about timing and the cost of newly proposed mandatory and voluntary reporting requirements.

A recent letter to this Subcommittee from Administrator Charles

Curie acknowledged that ‘‘the process had lost sight of the ultimate

goal and that there are major impediments’’ to the previously

agreed-upon plan for transforming the block grants to a performance partnership. We very much appreciate this candor.

First and foremost among the impediments are obvious concerns

about the cost of meeting any new requirements. Currently,

SAMHSA is tackling the issue in a piece-meal fashion by awarding

small discretionary grants to States, but we know from discussions

with State Directors that the Federal contribution will need to be

in the millions of dollars annually.

The SAMHSA reauthorization gives us the opportunity to consider this issue in detail and I look forward to receiving and reviewing the agency’s Reauthorization Proposal and their Performance Partnership Grant Report this year. Major changes to the

block grants should take place in the context of reauthorization and

following receipt of these reports, and not prior.

I look forward to today’s testimony by Mr. Curie, and I commend

him for his leadership in moving this process forward so well since

he took office.

We will also hear from Marsha Medalie of Riverside Community

Care in Massachusetts. Riverside is one of our largest and best providers, and we’re proud of all they do to improve the lives of people

with mental illness and substance abuse addiction. Her testimony

emphasizes the fundamental importance of measuring outcomes as

the key to improving the quality and availability of care.

In addition, Dr. Howard Goldman will discuss the importance of

using this information to aid in the transformation of our mental

health system.

We know that millions of Americans who need treatment for

mental illness never obtain it and those who do are often forced to

navigate a broken system that works for only the most well-off and

knowledgeable.

Our committee has broad jurisdiction and can help to break

down the barriers that keep mental health services out of reach for

millions of Americans of all ages. Some States are making signifiVerDate 11-MAY-2000 14:03 Apr 22, 2005 Jkt 000000 PO 00000 Frm 00007 Fmt 6633 Sfmt 6601 S:\DOCS\95101.TXT SLABOR3 PsN: SLABOR3
4

cant progress, and it should be as broadly available as possible in

all States.

I welcome our witnesses, and I look forward to working with Senator DeWine and other colleagues in Congress to put the best ideas

into action.

Senator DEWINE. Now we will turn to our first panel. We thank

everyone, of course, for joining us today.

For our first panel this morning, I would like to introduce

Charles Curie, Administrator of the Substance Abuse and Mental

Health Services Administration. He has served in this role since

October 2001. He reports, of course, directly to Health and Human

Services Secretary Tommy Thompson and leads $3.2 billion agency

responsible for improving the accountability, the capacity, and effectiveness of our Nation’s substance abuse prevention, addictions

treatment, and mental health services.

Thank you very much for being back with us.

Mr. CURIE. Thank you.

Senator DEWINE. We look forward to your testimony and we look

forward to having the chance to talk with you again.

STATEMENT OF CHARLES G. CURIE, ADMINISTRATOR, SUBSTANCE ABUSE AND MENTAL HEALTH SERVICES ADMINISTRATION

Mr. CURIE. Absolutely. Well, thank you very much for having me

this morning, Mr. Chairman, and I do request that my written testimony be submitted for the record.

Senator DEWINE. It will be made a part of the record.

Mr. CURIE. I am very pleased that you have selected performance

measurement and management as the topic for this morning. I

want to recognize this as a challenging issue, a complex issue. It

is an issue that I have struggled with as SAMHSA Administrator.

In fact, I have struggled because the SAMHSA I entered had hundreds of measures and millions of dollars of activities around data

collection, analysis, and reporting, but there was no strategy, no direction, and the links to agency vision and mission were vague at

best.

I am happy to report that we are changing the way we do business at SAMHSA. We have established a vision at SAMHSA to life

in the community for everyone. Our vision is based on the precept

that all people deserve the opportunity for a life that includes a job,

a home, education, and meaningful relationships with family and

friends, recognizing that these outcomes in the lives of people who

are in recovery also more fully assure that relapse may not occur.

So these are critical outcomes reflecting recovery.

We have established a mission. It is building resilience and facilitating recovery. We have established a matrix of priorities and

management principles, and we are finalizing and implementing a

data strategy that is firmly based on the best of our past activities

and linked directly to our vision and our mission.

Over the years, we have developed through a 16-state pilot program the uniform reporting system for mental health. It contains

over 20 measures of mental health services, each reported by

States in their block grant. I want to commend the National Association of State Mental Health Program directors for their leaderVerDate 11-MAY-2000 14:03 Apr 22, 2005 Jkt 000000 PO 00000 Frm 00008 Fmt 6633 Sfmt 6601 S:\DOCS\95101.TXT SLABOR3 PsN: SLABOR3
5

ship in this process. We have also convened over 30 State substance abuse agency meetings on performance measurement and

funded two treatment outcome and performance pilot studies.

These studies have resulted in the careful identification of performance measures for substance abuse treatment. Many States have

been reporting on these measures voluntarily since 2000, and I personally have seen amazing things done by the State substance

abuse authorities as a result of these efforts.

Mostly recently, I visited North Carolina, last fall Texas and last

summer in Washington State, and I would recommend to the committee to be examining their States in particular in terms of how

they are arriving at outcomes which fit within the seven domains,

as well as you will be hearing from Gary Tester from the Ohio on

the work that they have been doing.

As an illustration of our commitment to performance measurement as well, because we know money is needed, especially in

these tight times, SAMHSA will have invested just over $277 million in data infrastructure-related technical assistance to States

over the past 5 years, up from 49 million in Fiscal Year 2001 to

a requested 66 million in 2005.

Also, for the first time, we are asking for outcomes to be measured in our grant programs that reflect the seven domains and access to recovery, which the applications for that grant are being reviewed as well as the strategic prevention framework which is on

the road right now in terms of people responding and us evaluating

responses to that application. These are all concrete examples of

our steadfast commitment to build State data capacity to measure

and manage performance.

Our intention at SAMHSA is to keep moving forward with our

partners. We will maintain an open and transparent relationship.

Change comes with challenges. Our data strategy is simple. We are

looking at what data we are collecting. We are asking why are we

collecting it, and we are asking how are we using it to manage and

measure performance, and if we do not use it, we need to lose it.

Since all of our programs are aligned with our vision and our mission, it only makes sense that the same outcomes are used across

all of our programs. The tighter our measures become, the more we

can prove our effectiveness. The greater our effectiveness, the

greater the number of people served, the greater the chances for

that life in the community for everyone.

Our emphasis is on a limited number of national outcomes. This

emphasis is built on a history of extensive dialogue with researchers, providers, colleagues in the States, and most importantly the

people we serve. We have learned that a limited number of key

outcomes will minimize the reporting burden on the States and

others and will promote a more effective monitoring of client outcomes and system improvements. All of this leads me to the status

of performance partnership grants, a topic that I know is of both

interest and concern to the subcommittee.

The goal and intent of PPGs were clear, to promote greater flexibility and accountability in the block grant program; however, what

I discussed when I moved from the State of Pennsylvania to the

Federal side of the PPG equation was that the process had gotten

in the way of achieving the purpose. Talk and debate and discusVerDate 11-MAY-2000 14:03 Apr 22, 2005 Jkt 000000 PO 00000 Frm 00009 Fmt 6633 Sfmt 6601 S:\DOCS\95101.TXT SLABOR3 PsN: SLABOR3
6

sion had gone on far longer than necessary, a decade and multitudes of meetings and workshops on block grant performance

measurement alone. SAMHSA had funded data-related grant programs and data collection activities. SAMHSA had analyzed them

and reanalyzed them, and SAMHSA made agreements and then remade the same agreements.

As a result, performance partnerships had not happened when I

arrived at SAMHSA. Process seemed to have supplanted progress.

The report we were to submit to Congress was drafted, but its

focus was on process and not action. I accept full responsibility for

stopping that report, which I discussed with the subcommittee

staff. We still owe you that report. One of the reasons this hearing

is so important is to help ensure that we are moving forward together to meet the needs of people with or at risk for mental and/

or substance use disorders.

Dr. Gary Tisler, who served as study director for the Carter

Commission of Mental Health recently observed when he saw that

the results of the New Freedom Commission and the Carter was

similar. He said: ‘‘It seems as though the advances of science and

technology far exceed our abilities to solve problems related to attitudes, bureaucracies, and the human condition.’’ I think he is on

to something. Attitudes, bureaucracy, and the human condition are

what I fear will get most in the way of our efforts to move forward.

It is time to bring performance measurement and management

to the next level. It is time to begin reporting on what really needs

to be measured. Part of the challenge before us is to change current

attitudes and bureaucracies. Only when we find common ground

can we transcend those old attitudes.

The driving force for our work, as verbalized in our vision and

mission, is the hope of recovering the life in the community.

Through performance measurement and management, we open

ourselves to accountability for the work we do for you, for our many

partners, and most importantly for the people we serve in this Nation.

Thank you, and I would be very open to and look forward to having a dialogue and answering any questions.

[The prepared statement of Mr. Curie follows:]

PREPARED STATEMENT OF CHARLES G. CURIE, M.A., A.C.S.W.

Mr. Chairman and Members of the Subcommittee, good morning. I am Charles

G. Curie, Administrator of the Substance Abuse and Mental Health Services Administration (SAMHSA), part of the U.S. Department of Health and Human Services

(HHS).

I am pleased to appear before you today to focus on performance and outcome

measurement activities being undertaken by SAMHSA. The issue of performance

and outcome measurement is paramount, particularly since our budget for fiscal

year 2004 totals nearly $3.4 billion and since the President’s fiscal year 2005 budget

request for SAMHSA raises that to almost $3.6 billion. Moreover, they are issues

with which we at SAMHSA have been grappling as a priority matter since I came

on board as its Administrator.

I am happy to report that we are changing the way SAMHSA does business. Instead of continuing a history of talking about performance measurement and management, we have taken action to achieve performance measurement and management across all SAMHSA programs. Through decisive action—grounded in years of

deliberation that have preceded it—we are poised to hold our discretionary and

block grant recipients—and ourselves—accountable not only for how we spend, but

also for how we serve people with or at risk for mental and substance use disorders.

7

SAMHSA VISION AND MISSION

We have good reason to believe that, working with our partners at the Federal,

State and community levels, we can achieve SAMHSA’s mission of building resilience and facilitating recovery. We have good reason to believe that we can realize

the SAMHSA vision of a life in the community for people nationwide with or at risk

for substance use or mental disorders. Both our vision and our mission are consistent with the President’s New Freedom Initiative and with the precept that all people deserve the opportunity for a life that includes a job, a home, education, and

meaningful relationships with family and friends.

Both research and clinical experience have shown that people with mental and

addictive disorders can and do recover when they receive timely and effective care

in their communities. According to SAMHSA’s 2002 National Survey on Drug Use

and Health, an estimated 22 million persons, age 12 or older, needed treatment for

an illicit drug problem or an alcohol problem, or both. In the same year, an estimated 17.5 million people, age 18 and older, had serious mental illnesses. An estimated 4 million adults experienced co-occurring serious mental and substance use

disorders during the year. Further, in any given year, about 5 to 9 percent of children and youth have a serious emotional disturbance.

Unfortunately, we also know that for too many people, the need for care is not

matched by the availability of evidence-based substance abuse treatment and mental health services to meet those needs. Some people seek care and cannot get it;

others do not seek it at all. Under either circumstance, their quest for recovery and

a life in the community are frustrated; our mission and vision are not being

achieved.

THE SAMHSA ROLE

As this Subcommittee is well aware, since I became SAMHSA Administrator, the

Agency has been working in partnership with other Federal agencies, with States

and with communities to improve how we approach substance abuse treatment and

prevention and mental health services delivery. By restructuring our work around

the vision and mission, we have eliminated the functions that were not within our

scope as a services agency.

As a result, our work has become more finely honed and our dollars more carefully directed—nurturing a few solid redwoods that can endure over time, instead

of cultivating a garden of annuals pleasing for a season but with little lasting impact.

Further, to refine SAMHSA’s program development and resources, we developed

a Matrix of program priorities and crosscutting principles that pinpoints SAMHSA’s

leadership and management responsibilities. These responsibilities and program directions were developed as a result of discussions with Members of Congress, our

advisory councils, constituency groups, people working in the field, and people working to obtain and sustain recovery. The content is dynamic—and will change over

time. We’ll be able to know when we’ve reached a change point through performance

measurement and management, both at SAMHSA and in communities and States

across the country.

Today’s Matrix priorities are aligned with the priorities of both President Bush

and HHS Secretary Tommy Thompson whose support and confidence we greatly appreciate. They have recognized that it is time that program and policy—and America as a whole—recognize that substance use and mental disorders should be treated

with the same concern and urgency as diabetes, obesity, heart disease, stroke, and

cancer.

To that end, they have supported key elements of SAMHSA’s matrix: transforming the mental health care system; improving services for people with co-occuring

disorders; strengthening prevention efforts; expanding substance abuse treatment

capacity; and, critically, performance measurement and management.

THE ACE PRINCIPLES

From the perspective of today’s hearing, it is also critical that you know that we

are building our priority programs around three key principles. They are principles

that, I am sure resonate with your interests and concerns about SAMHSA’s programs and policy future. I am speaking of the principles of Accountability, Capacity,

and Effectiveness—ACE.

To promote accountability, SAMHSA tracks national trends, establishes measurement and reporting systems, develops standards to monitor service systems, and

works to achieve excellence in management practices in addiction treatment and

substance abuse prevention. We are demanding greater accountability of our grantVerDate 11-MAY-2000 14:03 Apr 22, 2005 Jkt 000000 PO 00000 Frm 00011 Fmt 6633 Sfmt 6621 S:\DOCS\95101.TXT SLABOR3 PsN: SLABOR3
8

ees in the choice of treatment and prevention interventions they set in place and

in the ways in which program outcomes meet the identified needs for services. Increasingly, we are promoting accountability—through performance measurement

and management.

By assessing resources, supporting systems of community-based care, improving

service financing and organization, and promoting a strong, well-educated workforce

that is grounded in today’s best practices and known-effective interventions,

SAMHSA is enhancing the Nation’s capacity to serve people with or at risk for substance use and mental disorders.

Further, SAMHSA also helps assure service effectiveness by assessing delivery

practices, identifying and promoting evidence-based approaches to care, implementing and evaluating innovative services, and providing workforce training. For example, our National Registry of Effective Programs and Practices—with 60 known effective prevention and early intervention programs in mental health and substance

abuse—provides a foundation on which States and communities can build to meet

prevention needs and reduce treatment needs. Our Treatment Improvement Protocols (TIPS) bring the latest knowledge about effective interventions, including treatment for adolescents, co-occurring disorders, and treatment for older adults, to professionals in the field. And our mental health services best practices toolkits, on topics ranging from medication management to assertive community treatment and

from supported employment to illness management and recovery, are being tested

in community-based settings across the country.

To measure our effectiveness and to be accountable, SAMHSA must have the capacity to gather and analyze data about our programs. We are continuing to build

on our long history of national surveys, such as the National Survey of Drug Use

and Health (which now includes measures of mental health and illness), the Drug

Abuse Warning Network and the Drug and Alcohol Services Information System

(which includes the Treatment Episode Data Set (TEDS)). At the same time, we are

working with States to build the infrastructure needed to capture and evaluate their

own measures and to identify and agree upon specific national outcome measures.

These national outcome measures, to the extent possible, have been drawn from

already tested instruments in use by mental health and substance abuse authorities

across the Nation. Many States are already reporting or are substantially ready to

begin reporting on these measures, thanks to this work. Data on specific populations, including women and children, and racial and ethnic minorities, are being

and will continue to be captured by these measures. In this way, the majority of

specific components of each measure already are known to and in use by many

States, and come from existing data sets, discussed next.

Mental Health Data Sets

Since its inception, SAMHSA’s Center for Mental Health Services (CMHS) has

worked with the States to develop a mental health services data system, including

the identification and specification of performance measures and data. This resulted

in the CMHS Uniform Reporting System (URS) that contains over 20 measures of

mental health services, each reported by States in URS ‘‘data tables’’ in their CMHS

Block Grant applications. Today, most States can report on the basic measures contained in the URS. These measures are indicated as change measures, since annual

totals for these measures will be compared year to year. Work is underway to develop more refined methodologies that can demonstrate system change and transformation. Currently, under the CMHS Block Grant, States will be expected to report on all 20 URS measures and to establish performance goals and targets for

mental health. In the future, SAMHSA expects that the number of measures the

States will report will be refined as specific measures are agreed upon for the Mental Health System Transformation effort.

Substance Abuse Treatment Data Sets

During the past several years SAMHSA’s Center for Substance Abuse Treatment

(CSAT) convened over 30 SAMHSA/State substance abuse agency meetings on performance measurement and funded two ‘‘Treatment Outcome and Performance Pilot

Studies’’ (TOPPS) that resulted in careful identification and delineation of performance measures for substance abuse treatment. The outcome measures identified

through TOPPS included changes in client alcohol and drug use; changes in client

illegal activity; changes in employment status; and, changes in homelessness. Many

States have been reporting on these measures voluntarily since 2000. To add yet

another way to help, we have created the Web Infrastructure for Treatment Services

or (WITS) which is an interactive technology system designed to aid States in data

collection. I’ve seen and heard about amazing things done through these efforts—

9

most recently in North Carolina, last fall in Texas, and last summer in Washington

State.

In addition, Federal and State substance abuse treatment data also build upon

the foundation of the TEDS admission data, generally available for most publicly

funded programs throughout the States. Information produced through a survey

conducted by the National Association of State Alcohol and Drug Abuse Directors

(NASADAD) indicates that most States exceed the minimum specifications of TEDS

and are now collecting many of the relevant variables at discharge and beyond. To

this end, the handful of States that have on-going problems submitting their TEDS

reports will be offered an opportunity to participate in a pilot State level operation

to help determine which data collection and management system can best generate

the most accurate data on a real-time basis. SAMHSA believes that this will result

in States being fully prepared to report on the same performance measures regardless of whether they are reporting on the Block Grants or discretionary grant programs.

Substance Abuse Prevention Data Sets

SAMHSA has also worked carefully over the years with State substance abuse

prevention officials to specify and define performance measures for substance abuse

prevention activities. Since 1990, SAMHSA’s Center for Substance Abuse Prevention

(CSAP) and a group of State prevention officials have met regularly to identify and

define the 30+ performance measures currently being addressed by the States as

part of the State Incentive Grant program (SIG), many of which are taken from existing data sources, such as CSAP’s Minimum Data Set (MDS). In the future,

SAMHSA expects to work with the States also to identify and finalize a smaller

group of environmental measures—measures that address the impact of programs

on the community or ‘‘environmental’’ level—that will be used in both discretionary

programs and the prevention portion of the SAPT Block Grant.

These are all concrete examples of our steadfast commitment to build State data

capacity to measure and manage performance. This foundation has been laid to reorient ourselves to a State-friendly and consumer-friendly performance environment.

Our intention at SAMHSA is to keep moving forward with our partners. Change

comes with challenges. One of the reasons this hearing is so important is to help

ensure that we are moving forward together to meet the needs of people with or

at risk for mental and or substance use disorders.

FROM TALK TO ACTION: MEASURING AND MANAGING PERFORMANCE

To help us present consistent and reliable information we have been developing

and implementing a data strategy. The strategy is simple: The tighter our measurements become, the more we can prove our effectiveness. The greater our effectiveness—the greater the number of people served, the greater the chances for a life

in the community for everyone. Developing a data strategy is a task that has been

hanging around for years. Now, we have gotten real about doing it.

Our SAMHSA data strategy is a critical building block to achieve true accountability in a performance environment by transforming the way we do business. We

are looking at what data we are collecting. We are asking why we are collecting it.

And, we are asking how we are using it to manage and measure performance. If

we don’t use it, we need to lose it.

We have learned that a limited number of key outcomes measured in structured

ways can help all of us know how well SAMHSA and its grant programs are building resilience and facilitating recovery. Our emphasis on a limited number of national outcomes and related national outcome measures is built on a history of extensive dialog with our colleagues in State mental health and substance abuse service agencies and the people we serve.

While the discussions with States focused specifically on SAMHSA’s block grant

programs—something I will address in a bit more detail later in this testimony—

the application of national outcomes and national outcome measures extends across

all SAMHSA grant programs. All of our programs are about achieving our vision

of a life in the community for everyone and our mission building resilience and facilitating recovery. So it only makes sense that we use the same outcomes across

our programs. And it only makes sense that we stop talking about national outcomes and start implementing them.

NAMING THE NATIONAL OUTCOMES

So let me tell you more about the National Outcomes we have identified in our

deliberations with the States. Together we have highlighted specific domains of resilience and recovery as National Outcomes. These are:

10

• Abstinence from alcohol abuse or drug use, or decreased symptoms of mental

illness;

• Increased or retained employment and school enrollment;

• Decreased involvement with the criminal justice system;

• Increased stability in family and living conditions;

• Increased access to services;

• Increased retention in services (substance abuse) or decreased utilization of psychiatric inpatient beds (mental health); and

• Increased social connectedness.

These domains are joined by additional outcomes identified by the OMB Program

Assessment Rating Tool (PART) process—for example client perception of care, cost

effectiveness, and use of evidence-based practices. Together they constitute the National Outcomes that SAMHSA is applying to its discretionary and block grant portfolio activities. Already, SAMHSA is implementing these National Outcomes, including them in the grant announcements for its Access To Recovery Program (ATR),

and its Strategic Prevention Framework (SPF). States have voluntarily been collecting and reporting performance information on a variety of measures for SAMHSA’s

Block Grants and we have required reporting on many of these measures in our discretionary programs, as is evident in our fiscal year 2005 budget submission/GPRA

plan and report.

Focusing on this handful of National Outcomes will minimize the reporting burden on the States and other grantees, and will promote more effective monitoring

of client outcomes and system improvements.

SAMHSA has also worked carefully with the States to identify and agree upon

specific performance measures for each of the National Outcomes. These measures,

to the extent possible, have been drawn from already tested instruments in use by

mental health and substance abuse authorities across the Nation. Now, we need to

ensure that we collect the data in the same way across all of our programs, so that

we can present aggregated results wherever possible.

However, some of the measures are developmental and require further work by

SAMHSA and the States to delineate the best measures to assess progress toward

reporting National Outcomes. For mental health, such developmental measures include ones for decreased symptomatology, criminal justice involvement, school attendance, readmission rates, and number of persons receiving evidence-based practices. For substance abuse treatment, developmental measures include those for stable living situation, unduplicated counts, length of stay, and services provided within cost bands. For substance abuse prevention, developmental measures include

those for returning to/staying in school, decreased criminal justice involvement, increased stability in family and living conditions, and cost effectiveness (increase

services provided within cost bands).

Other measures remain to be identified, including those for people with co-occurring disorders, the presence of both mental and substance use disorders. Collecting

data on co-occurring disorders poses unique challenges for States—especially for

those with separate mental health and substance abuse treatment systems. These

systems will need to work together to identify measures and methods of measurement that will be reliable, valid, and non-duplicative, and to share data for reporting. SAMHSA will continue to work with States to further develop and refine these

measures.

IMPLEMENTING NEW DIRECTIONS MEANS SUPPORTING SYSTEM CHANGE

Critically, the implementation of the National Outcomes is being accompanied by

a real-time infusion of SAMHSA support for the improvement of the data infrastructures in place at the Federal, State and local levels to manage this sea change from

counting to accounting for success.

As an illustration of SAMHSA’s commitment to performance measurement, we

will have invested just over $277 million in data infrastructure and related technical

assistance to the States over the past 5 years, up from $49 million in fiscal year

2001 to a requested $66 million in fiscal year 2005, consistent with the President’s

fiscal year 2005 Budget.

The following table provides greater detail regarding SAMHSA’s commitment to

States to build the data infrastructure needed to make performance measurement

and management realities in how States do business with communities and with

SAMHSA, and how SAMHSA does business to achieve its vision and mission for the

American people.

11

SAMHSA RESOURCES FOR PERFORMANCE MEASUREMENT AND

PERFORMANCE MANAGEMENT

[in millions]

SAMHSA Center FY 2001

Actual

FY 2002

Actual

FY 2003

Actual

FY 2004

Estimate

FY 2005

Requested Total

CMHS .................................................................. $12.2 $12.6 $13.7 $14.9 $15.8 $69.2

CSAP ................................................................... 10.1 10.6 8.5 12.3 17.5 59.0

CSAT ................................................................... 22.8 25.4 26.8 25.4 28.4 128.8

OAS ..................................................................... 3.9 3.9 4.0 4.1 4.1 20.1

Total ............................................................... $49.0 $52.5 $53.0 $56.7 $65.8 $277.1

PERFORMANCE PARTNERSHIPS—THE BLOCK GRANT PROGRAMS

All of this leads me to the status of Performance Partnership Grants (PPGs), one

of the topics I know is of both interest and concern to this Subcommittee. After all,

Congress, in its 2000 reauthorization of SAMHSA, called for the transformation of

the existing substance abuse prevention and treatment block grant and the mental

health services block grants into performance partnership grants.

The goal and intent of PPGs were clear—to promote greater flexibility and to infuse greater accountability into the block grant program. I’ve already described the

years of discussion we have had with State mental health and substance abuse authorities. I have described the collaboration over that time with them that led to

the identification of the National Outcomes on which our performance measurement

and management focus. And I have described the broad range of existing data sets

and outcome measures—many of which already are in place.

Yet, what I discovered when I moved from the State of Pennsylvania to the Federal side of the PPG equation, was that clearly, the PPG process had gotten in the

way of achieving the PPG purpose. Talk and debate and discussion had gone on far

longer than necessary: a decade and multitudes of meetings and workshops on block

grant performance measurement alone. SAMHSA had funded data-related grant

programs and data collection activities. SAMHSA had analyzed them and reanalyzed them. And SAMHSA had made agreements and then remade the same agreements.

As a result, Performance Partnerships still had not happened when I reached

SAMHSA. Process had supplanted progress. The Report we were to submit to Congress on our progress on Performance Partnerships was drafted, but its focus was

on the process and not on the action. A recent GAO report reminds us that we owe

Congress that report.

In general, the Report delineates how we are changing the relationship between

the Federal and State governments to create more flexibility for States and accountability based on outcome and other performance measures.

By using the National Outcomes, we are changing the questions from ‘‘How did

you spend the money’’ and ‘‘Did you stay within the spending rules’’ Instead, we are

asking questions relevant to building resilience and facilitating recovery, questions

like ‘‘How did you put the dollars to work?’’ and ‘‘How did your consumers benefit?’’

As the change in questions suggests, our focus is squarely on National Outcomes

and National Outcome Measures. The National Outcomes are true measures of recovery. They assess whether our programs are helping people attain and sustain recovery. They show that people are achieving a life in the community—a home, a job,

and meaningful personal relations.

Clearly, the time for action is long past. Somehow, we lost sight that block grants

are a means to build resilience and facilitate recovery. Instead, the goal became implementing PPGs solely for the sake of implementing them and not the implementation of performance measurement and performance management.

That is why we are moving forward with our National Outcomes and National

Outcome Measures across all of SAMHSA’s funding streams. They will reduce State

and community reporting requirements while simultaneously presenting reliable information to you, to other key stakeholders and to SAMHSA about the effectiveness

of our services and how they are being applied across the country.

CONCLUSION

As this testimony suggests, SAMHSA has invested a decade preparing for action,

debating about action, and thinking about action. The time for preparation is over;

the time for implementation is now. We have the knowledge, we have the capacity,

12

and we most certainly have the obligation to be accountable to the American taxpayer—and to you—to show that what we do, what we fund, and what we propose

in policy are effective. Beyond this obligation, we have a responsibility to the millions of Americans who are battling addiction; struggling with a serious mental illness or emotional disturbance; or are fighting a co-occurring serious mental and

substance use disorder and their families to put into motion this long-overdue due

diligence.

That is why, in our programs, our grant announcements, and our policies, we are

taking that long-overdue action. We have looked to the past and found the delays

unacceptable. And we have looked to the future and found our direction clear.

It is built on the solid ground of customer service—making decisions based on the

needs of the people we serve, not on the needs of bureaucracies. The driving force

for our work—as verbalized in our vision and mission—is what people with or at

risk for substance use or mental disorders desire—the hope of recovery and a life

in the community. We must open ourselves to accountability for the work that we

do for you; for our many partners and for the public health of this Nation.

SUBSTANCE ABUSE AND MENTAL HEALTH SERVICES ADMINISTRATION

NATIONAL OUTCOME MEASURES

Outcome

Treatment Prevention

Mental health Substance abuse Substance abuse prevention

Abstinence from Drug Use/

Alcohol Abuse.

Not applicable ...................... Change in percentage of

clients abstinent at discharge compared to the

number/proportion at admission2.

30-day substance use (nonuse/reduction in use)2

Availability of alcohol and

tobacco. Availability of

other drugs.1

Percentage of program participants and percentage

of population who perceive drug use as harmful.2

Attitude toward use among

program participants and

among population at

large

Decreased Mental Illness

Symptomatology1.

Decreased symptomatology1 Not applicable ...................... Not applicable

Increased/Retained employment or return to /Stay in

school.

Profile of adult clients by

employment status, increased school attendance (children)1.

Change in percentage of clients employed at discharge compared to the

percentage at admission.

Increase in school attendance 1; Decrease in

ATOD-related suspensions/expulsions 1; Decrease in drug-related

workplace injuries 1.

Decreased criminal justice

involvement.

Profile of client involvement

in criminal and juvenile

justice systems 1.

Change in percentage of clients with criminal justice

involvement at discharge

compared to the percentage at admission.

Reduction in drug-related

crime 1.

Increased stability in family

and living conditions.

Profile of clients’ change in

living situation (including

homeless status).

Percentage of clients in stable living situations at

discharge compared to

the number/proportion at

admission (i.e., housing)*1.

Increase in parent participation in prevention activities 1

Increased access to services

(service capacity).

Number of persons served by

age, gender, race and

ethnicity 2.

Unduplicated count of persons served1 2.

Penetration rate—Numbers

served compared to those

in need1.

Number of persons served by

age, gender, race and

ehtnicity.

Increased retention in treatment—substance abuse.Not applicable ...................... Length of stay1 .....................

Unduplicated count of persons served*2.

Not applicable.

13

SUBSTANCE ABUSE AND MENTAL HEALTH SERVICES ADMINISTRATION

NATIONAL OUTCOME MEASURES—CONTINUED

Outcome

Treatment Prevention

Mental health Substance abuse Substance abuse prevention

Reduced utilization of psychiatric inpatient beds—

mental health.

Decreased rate of readmission to State psychiatric

hospitals within 30 days

and 180 days1 2.

Not applicable ...................... Not applicable

Increased social supports/

Social connectedness3.

TO BE DETERMINED (Initial

indicators and measures

have not yet been identified).

TO BE DETERMINED (Initial

indicators and measures

have not yet been identified).

TO BE DETERMINED (Initial

indicators and measures

have not yet been identified).

Client perception of care2 .... Clients reporting positively

about outcomes2.

.........................................

Cost effectiveness2 ............... Number of persons receiving

evidence-based services

1 2.

Number of evidenced-based

practices provided by

State 2.

Percentage of States providing substance abuse

treatment services within

approved cost per person

bands by the type of

treatment1 2.

Increase services provided

within costs bands1 2

Use of evidence-based practices2.

......................................... ......................................... Increase services provided

within cost bands1 2

Total number of evidencebased programs and

strategies funded by

SAPTBG2

1 Developmental.

2 Required by OMB PART Review.

3 For ATR, Social Support of Recovery’’ is measured by client participation in voluntary recovery or self-help groups, as well as interaction.

Senator DEWINE. Thank you very much. I have a question. Senator Kennedy could not be here today, but he has asked me to submit this question to you.

According to your testimony, SAMHSA is, quote, moving forward

with the process of implementing national outcomes and national

outcome measures across all of SAMHSA’s funding streams. In

light of this significant regulatory movement, what role do you now

envision re-authorization of SAMHSA next year will play in the

PPG transition?

Mr. CURIE. It is an excellent question. What I would envision is

that we will be in a position to be discussing with the subcommittee what exactly we would need to be doing that has to be translated in statute which reflect outcomes that we all have come to

consensus reflect those seven domains. We will have some models

to look at. ATR, Access to Recovery, that is the first grant where

we really operationalized outcome measures reflective of the seven

domains, and that will give us, I think, a good foundation along

with strategic prevention framework to consider what type of outcomes would be required and how will that impact the block grant.

Obviously, the block grant is the major bulk of what we fund,

and the block grant historically has been viewed as more of an allocation in practice that goes to the States as opposed to having what

is desired around PPGs, both the flexibility and accountability. So

what we need to look at is how can we assure that the seven domains are appropriately reflected in the block grant which are

those measures that are germane to all States while giving States

the flexibility to address their individualized needs and then consider how accountability will be built in.

14

We have approached this from the perspective that this is not a

way of penalizing States or trying to put States in an awkward or

difficult position if outcomes are not being attained, but it needs to

be a way of assuring that along with any regulatory changes we

feel need to be made in the re-authorization process, that we do it

in such a way that we have technical assistance and supports and

an understanding of how those outcomes are going to be used.

Senator DEWINE. All right. Senator Kennedy has another question. What process is in place to consider input on the developmental outcome measures still under consideration to ensure that

all appropriate stakeholders are involved and working groups between SAMHSA and State directors represented here today be reconvened?

Mr. CURIE. We are committed to continuing to sit down and have

a dialogue with the directors, both on the substance abuse side of

the equation as well as the mental health side of the equation, and

we recognize that clearly some of these measures are developmental. One, for example, is connectiveness, which is the domain

which has emerged over the past year as critical for recovery, but,

in all frankness, there are not a lot of specific measures yet that

have been agreed to that reflect that connectiveness.

So, yes, we will be meeting with on a regular basis State directors and the appropriate associations to determine how to approach

this from a development perspective, recognizing that States are all

at different levels right now. We have certain States, States that

I mentioned, that have a fairly advanced approach to demonstrating outcome measures. Other States are not as far along. Some

States have greater capacity than other States.

So a lot of that discussion also will be along the lines of how we

can use what we have learned from the States and models that

have worked and bring it to scale in other States. So we are committed to a transparent process, a dialogue. It may not be necessarily always in the context of an ongoing work group, but it will

be in the context of having ongoing input and ongoing transparent

communication with the appropriate associations.

Senator DEWINE. You point out in your testimony that the Government Accounting Office just released a report that reviews several SAMHSA operations. In it, they criticize your significant delay

in reporting to Congress your implementation plan for performance

partnership grants. Why is it late? Are there some insurmountable

policy or other issues which maybe we should be aware of?

Mr. CURIE. I appreciate that, and that is an excellent question.

As I said in my opening remarks, I own responsibility for that

deadline not being met. I made a conscious decision after reviewing

the process of PPGs during my first year, trying to determine what

were we accomplishing, and there was, as I would put it, a rush

to get a report pulled together and get it submitted to meet that

particular deadline.

I was concerned about the fact that it was not tied to any particular strategy. There was a discussion about measures. There

was still a discussion about it being all very developmental, and I

thought it was very important for us to embrace those things that

we know were being measured already in the field and bring those

things to scale as well as tie it to an overall data strategy. And the

15

reason that we talk about the seven domains which we have developed over the past year is because, for the first time, it begins to

put the outcomes into a structure which will reflect whether the

dollars we are investing are helping people attain and sustain recovery and helping to build resilience, and it goes to real outcomes

in people’s lives; and the PPG report, I thought was critical to be

reflective of strategy and not have, if you will, the tail wagging the

dog, but the PPG itself being more of an outcome we are looking

for. It was important for a data strategy to be driving the PPG

process.

So we met with the members of your subcommittee to indicate

to them what we were examining and that we were looking to revamp the approach to PPGs in the sense of tying it to that strategy

and that we would not be meeting that deadline, but that we would

continue in dialogue. I am pleased to say that the PPG report is

being vetted at this point through several Federal agencies, and

there is a concrete document from all the work that has been done

which has been taking under consideration input from stakeholders. We have been also looking to examine these seven domains in

the context of experts, are these valid, also is there buy-in that this

makes a lot of sense.

So we have been in that process as we have been developing this

report. So the report is on its way, and I would also, because of the

critical aspects of this to our re-authorization, offer to you that we

schedule a briefing with your committee staff just on this matter

of PPGs and performance measurement on at least an every 60-day

basis, that we sit down and demonstrate the concrete progress we

are making in our discretionary grant process, where we are actually beginning to take those measurements, and how we begin

translating that to the block grant, and then have discussions of

that report in depth as its submitted.

Senator DEWINE. So we should see it initially when?

Mr. CURIE. Well, it is in the process of being examined by appropriate policy and budgetary entities.

Senator DEWINE. I understand.

Mr. CURIE. So it is going through that process, and if it was coming just out of my shop, I probably could give you a more pertinent

deadline, but since there is a wide range of folks looking at it, we

have actually gone through preliminary clearance. It is going

through some final clearance at this point. I would anticipate that

it is going to be out hopefully soon, and it depends how that process is.

Senator DEWINE. That would be in my term of office, would it?

Mr. CURIE. I think in terms of definitely soon and imminent, I

know typically are within at least 1 or 2 years.

Senator DEWINE. That is what I was afraid of.

Mr. CURIE. It is my hope it will be sooner than that, and that

is another reason I would like to convene these meetings with the

staff on a regular basis up until re-authorization, so we can have

discussion about what we anticipate is coming out in the report, as

well as once the report is released, we will have a venue together

to begin to digest it together and determine a direction.

Senator DEWINE. Just for a reference, I have 2 more years on my

term of office.

16

Mr. CURIE. I think it is safe to say soon fits in that category, but

it will be—I think it is on its way.

Senator DEWINE. I am a patient man.

Well, we appreciate your testimony. We look forward to working

with you.

Mr. CURIE. Thank you, Mr. Chairman.

Senator DEWINE. This is very important. We are looking forward

to our testimony of our second panel because they can give us some

of the practical aspects of this. I am interested in some of the

States that you have mentioned, and we will kind of delve into

some of those States.

Mr. CURIE. States have done some very good things with the

money we have put out, and I think we have some good models

upon which to build. We are not starting from scratch at all, and

we can really move this along once we have it.

Senator DEWINE. You found the States that you mentioned are

doing particularly good work?

Mr. CURIE. Yes, absolutely. In fact, the seven domains we talked

about, there are measures related to those, employment, education,

lack of involvement with the criminal justice systems, and also

there are ways States are able to obtain these outcomes from using

the capacities in other State agencies, being connected to criminal

justice, being connected to labor, being connected to housing and

education and cross-referencing people who have substance abuse

issues or who have a mental illness. So there are models out there

that we can utilize.

Senator DEWINE. Let me ask you one final question.

Mr. CURIE. Sure.

Senator DEWINE. I saw this, and I kind of struggled with this

when I was Lieutenant Governor in Ohio. You want your agencies

always to be accountable, and we would put money out to county

agencies that were doing programming, and we would want them

to be able to tell us that whatever programming they were doing

worked. On the other hand, we did not want to burden them with

so much red tape and have them spend so much of their programming money on accountability that burned it all up and spent all

their time doing it. How do you do that? How do you philosophically and practically approach that? How do you go at it? Because

that is the age-old problem. You want to know that it works. You

want to test it so that you know that it works, and yet you do not

want everybody out there spending all their time filling out what

they consider to be very burdensome paperwork and constantly

measuring it so you spend 25 percent of your money on making

sure that the other 75 percent is not wasted. How do you deal with

that?

Mr. CURIE. Well, I think, as you described it, it is an age-old

problem. I do think that has been a major obstacle to this whole

thing over the past decade, and I think what is critical is to have

this strategy in place that we have understanding and consensus

around what measures do we really need to use, number one. In

the past, many times measures have been approached by a grantto-grant basis or even a county-to-county basis, and we have not

necessarily come to agreement historically on what are those few

17

measures we need, so trying to keep it down to a minimum of what

we really need.

Second is examine has some of these stuff already been measured somewhere? Are we already measuring it, and if we are, let

us talk to criminal justice, the other systems that we talked about

that relate to the domains. We may not have to create a new data

infrastructure or we may be able to build on and have linkages,

and I think today with web-based technology and how we could

garner reporting and using what is available and has not been

available in the past, I think there also could be some breakthroughs to help us find cost effective ways of gathering the data.

And again. I think if we approach this from a systems perspective,

and in North Carolina clearly is an example of that, of having all

the pertinent agencies together around this, not just the substance

abuse authority or just the mental health authority, but all those

agencies that represent those domains, you also can gain an economy of scale around that.

But then we have to always keep the provider in mind, and that

is what I think you were describing, the county and provider, and

make sure that what we are requesting is pertinent. I think during

re-authorization, as we look at what is required in statute, also

having discussions with the Administration ongoing on GEPRA and

part scores that we require in grants, trying to get those all aligned

around the measures that reflect recovery can help reduce a burden.

Also, the struggle is when you go through the process of trying

to reduce a burden, many times it becomes more burdensome just

because you are changing the way you are doing things. So that is

something we need to recognize. Also, in terms of cost, we typically

have tried to allow a certain percentage in grants to go toward

that. Also, as I have indicated, we have tried to have some separate

line items, if you will, around data infrastructure itself so it would

not put an undue burden on it.

Substance abuse, I am particularly concerned about because it is

a fragile field in some senses in terms of SAMHSA and the State

match pretty much funds the public substance abuse treatment

system, and you are right. If a lot of those resources are put

around evaluation, it already begins to undercut a system that is

trying to grow capacity, and that has always been its greatest challenge.

Senator DEWINE. Okay. We look forward to working with you.

Thank you very much.

Mr. CURIE. Thank you Mr. Chairman.

Senator DEWINE. Thanks for coming.

Mr. CURIE. Thank you.

Senator DEWINE. Let me invite our second panel to start coming

up now, and I will begin to introduce you.

First we have Dr. A. Thomas McLellan, an internationally recognized researcher in the substance abuse field. He is a psychologist,

professor of psychiatry at the University of Pennsylvania, and Director of the Treatment Research Institute in Philadelphia. He has

published extensively, received many professional awards, and currently serves as the editor in chief of the Journal of Substance

Abuse and Treatment.

18

Next we have Dr. Howard Goldman. Dr. Goldman is an internationally recognized mental health researcher and is a professor

of psychiatry at the University of Maryland School of Medicine.

Next we have Gary Tester, Director of the Ohio Department of

Alcohol and Drug Addiction Services. In this cabinet-level position,

Mr. Tester oversees a staff of 110 employees and a budget of $172

million.

Finally, we have Marsha Medalie, who is Vice President and

Chief Operating Officer of Riverside Community Care. She joined

Riverside in 1995, having been the CEO of one of Riverside’s predecessor organizations. She has 30 years of experience in health care

and human services, much of it in leadership positions and community-based non-profit organizations.

We thank all of you very much for being with us.

Dr. McLellan, we will start with you. And what we are going to

do, we are going to have 5 minutes. We have your written statements from each one of you. They will be made a part of the record.

We thank you very much for that, and we are going to stay rigidly

to five minutes. So when you get a sign of a yellow light up here,

you have a minute to go, and we will stop when you get a red light,

and we will go to next witness, and we will go through all four of

you, and then we will have the opportunity to have some questions

and kind of discussion maybe among all four of you.

So, Dr. McLellan, thank you very much.

STATEMENT OF A. THOMAS MCLELLAN, DIRECTOR,

TREATMENT RESEARCH INSTITUTE

Dr. MCLELLAN. Thank you very much for asking me. I just wanted to say that, prior to my testimony, I am not an advocate. I do

not represent any organization that provides treatment. We do only

evaluation, and the work that I will discuss comes from some of my

own work and many studies that have been reviewed by the scientific community, and it really is just five simple statements, really. We could talk more if you want to talk about the specifics.

First, which is quite important with regard to addiction treatment, it can be evaluated. It is amenable to scientific inquiry in exactly the same way as all other forms of medicine and commerce.

The same kinds of procedures have been used as currently used by

the FDA to evaluate medication and medication procedures.

Okay. Two, effectiveness does not mean cure, but it means more

than abstinence. You just heard Mr. Curie. I think the field would

agree with him. The seven domains have been in existence for 20

years, and they basically revolve around giving a person a kind of

life that they are entitled to and society the kind of reward, results,

that they have paid for.

Effectiveness really means three things. It means significant reductions, ideally abstinence, from substance use, improvement in

personal health and social function, and reduction in public health

and public safety problems. The first two are identical to the same

dimensions that are used in the rest of medicine. If you will, there

is a holy trinity, and the holy trinity is abstinence, employment,

and no crime. That is what the public wants. That is what patients

want.

19

Okay. A treatment program consists of many components, and

these components are therapies, various kinds of social and medical

services, and medications. So the truth is that not all treatment

components are effective and not all treatment programs are competent. Better treatments have the following characteristics: They

are longer, longer duration in an outpatient setting, more social

and medical services, regular monitoring of the patient, and involvement in the family. Frankly, most treatment components have

not been evaluated, and many of the things that have been evaluated are not in practice because of financing and structural issues

that I will get to.

Fourth statement: Addiction treatment has changed over the

years, and it has made significant impact on the way it is evaluated. In the old days, addiction was a bad habit or a sin or a vile

idea. You went away to Shady Acres Treatment Program for ‘‘X’’

number of days and you were expected to emerge rehabilitated.

Evidence of that was lasting sobriety, abstinence. Well, like the

rest of health care—well, first of all, addiction now is more commonly thought of in the same way that other chronic illnesses are

thought of, and like other chronic illnesses, addiction treatment is

now 90 percent in an outpatient setting. That is very important.

People do not go away to treatment anymore. They stay in the community and they are allowed to function in the community, appropriately so.

Meanwhile, the same kinds of evaluation techniques are no

longer appropriate. You do not want to wait a year after the end

of treatment to find out if something is effective. What you want

to do is the same kind of thing that they do in the rest of medicine,

performance monitoring. Monitoring is pertinent to your last questions. The monitoring is the outcome and it is done as the routine

part of standard care, to manage the patient and to develop the

treatment program. The evaluation merely collects those measures

and uses them and reports them to maintain accountability. That

is the way to make it efficient. That is the way to keep it out of

the treatment people’s hair and at the same time get more pertinent responsive accountability.

The last statement, all this said, evaluation can happen. Performance monitoring can occur. It cannot happen in today’s addiction treatment system. The infrastructure of today’s addiction

treatment system is so deteriorated that it cannot sustain. You

have program directors all through this country making less than

prison guards and having fewer benefits. The majority of programs,

the great majority of programs, have no full-time physician, no fulltime psychologist, no full-time social worker, no full-time nurse,

none of the traditional professions that represent health care. It

does not look like health care. It looks like something else.

Okay. Counselor turnover in the United States is comparable to

turnover in the fast food stray industry, and while the fast food industry has accommodated to this by engineering systems to allow

standardization and ensure quality, we do not. We could, but we

do not. The point here is only that you are not going to regulate

this into higher quality at this point. It is going to need some resources. It is going to need to earn some resources. I think it can,

but that is my testimony.

20

That is it.

[The prepared statement of Dr. McLellan follows:]

PREPARED STATEMENT OF A. THOMAS MCLELLAN, PH.D.

I am Thomas McLellan and I am a researcher in the substance abuse treatment

field from the University of Pennsylvania and the Treatment Research Institute.

I am not an advocate and neither I nor my Institute represent any treatment or

government organization.

I can offer evidence on the effects of treatments for alcohol, opiate, cocaine and

amphetamine addiction based on my own work of over 400 reviewed studies published in scientific journals—and based on several reviews of the scientific literature—also reviewed by organizations such as the IOM.

My testimony contains only five points:

1. Addiction treatment can be evaluated in a scientific manner using exactly the

same procedures and standards presently used by the FDA to evaluate new medications and devices.

There are over 700 published studies using these methods to evaluate various

types of addiction treatments and the findings show that—when properly applied— addiction treatments CAN be effective. Treatment response rates and relapse rates

are quite similar to those seen in other chronic illnesses such as diabetes, hypertension and asthma.

2. Effectiveness does NOT mean cure—it does mean more than abstinence. There

is no reliable cure for alcohol or drug addiction. Many people can become abstinent

and resume normal lives but once addicted it is very unlikely that a person can

drink or use drugs socially.

From an evaluation perspective ‘‘Effectiveness’’ means three things:

• Significant reduction in substance use; • Improvement in personal health and social function; • Reduction in public health and public safety problems.

3. Not all treatments are effective—not all treatment programs are competent.

Treatments that do NOT work include: Detoxifications not followed by continuing

care; and acupuncture.

Many contemporary treatment components have not been evaluated.

Many evidence based treatments are not in practice—financing & training issues.

Better treatments have the following characteristics: Longer length and monitoring—in outpatient setting; Tailored social/medical services; and Involvement of family.

4. Addiction treatment has changed in concept and delivery over the past 10 years

and it has significant implications for treatment evaluation. Addiction was considered a bad habit and over 60 percent of treatment was provided in an inpatient setting. Discharged patients were expected to emerge ‘‘rehabilitated’’ and the evidence

was sustained abstinence measured 6–12 months following treatment discharge.

Now addiction is considered like other chronic illnesses (evidence can be briefly

reviewed if necessary) and today over 90 percent of addiction treatments are provided in outpatient settings for unspecified periods of time.

Consequently, the post-treatment measurement of outcomes in the traditional

way, inappropriate, slow and expensive. Traditional post-treatment outcome evaluations cannot provide clinicians with information they need to iteratively improve

care—or the policymaker with evidence of accountability about those issues the public is most interested in—crime, employment, ER utilization.

The clinical monitoring approaches used in the treatment of other chronic illnesses are also appropriate in the treatment of addiction. These approaches stress

patient responsibility for disease and lifestyle management and the early detection

of threats to clinical stability (relapse). These contemporary clinical approaches require modern information management techniques and systems that provide standardized, relevant monitoring information to the clinician and to the payors.

5. The basic infrastructure of the United States addiction treatment system is in

very bad condition. Program closures or takeovers are over 20 percent per year. Program directors make less than prison guards and have fewer benefits. The great majority of programs have no full time physician, no psychologist and no social worker.

Counselor turnover is comparable to that of the fast food industry. There are no

standardized data collection protocols designed for clinical use in monitoring patients.

Although there are now well-tested medications and therapies that could be helpful, the present system cannot adopt most of them.

This system ultimately could meet the accountability demands of the public and

could adopt the evidence-based treatments developed by NIH—but ONLY if it gets

21

investment to improve information infrastructure, basic management training and

to attract professional staff.

Senator DEWINE. Very good.

Dr. GOLDMAN.

STATEMENT OF HOWARD H. GOLDMAN, PROFESSOR OF PSYCHIATRY, UNIVERSITY OF MARYLAND SCHOOL OF MEDICINE

Dr. GOLDMAN. Good morning, Mr. Chairman. As you noted, I am

Howard Goldman. I am a professor of psychiatry at the University

of Maryland. I am pleased to appear before you. I am testifying on

behalf of the Campaign for Mental Health Reform.

The campaign was established to advocate for the recommendations of the President’s New Freedom Commission on mental

health to transform mental health care in America. It was created

to serve as the mental health community’s united voice in promoting Federal policy changes that will transform mental health care

from a fragmented, unresponsive, and inefficiently funded delivery

system to one that meets the needs of service users and their families.

I am pleased to respond to your invitation today to discuss what

we have learned about performance and outcomes in mental health

services and about our capacity to measure effectiveness of programs for multiple perspectives. My written testimony presents an

elaborate argument about the importance of accountability through

performance and outcome measurement, and I will not go into the

details other than to say that we outline the prevalence of mental

illness and its associated burden and point out that fewer than half

of individuals who have a diagnosable mental illness, even the

most serious conditions, seek care. This is particularly unfortunate

because we know that treatment is effective and compounding the

problem is that the care that is delivered is not the best that the

advances in science indicate are effective and are likely to produce

the agreed upon outcomes that we have heard about from Mr.

Curie and from Dr. McLellan, such as reduced symptomology, increased community participation in work in school, for example.

This quality gap, if you will, is the reason that accountability is so

critical.

I want to make two basic points about accountability with my

oral testimony. One focuses on the traditional role of SAMHSA as

the mental health steward for the specialty and particularly public

mental health system, but the other point is about that role in an

unconventional form that will be necessary if we were to transform

mental health care as the President’s commission has suggested.

That first point is that SAMHSA’s re-authorization is critical to

funding an infrastructure for performance and for outcomes measurement. Considerably more than the current level of expenditure

is needed if States are to be able to report to the Federal Government in an effort to assess the performance of the public mental

health system.

$100,000 to $150,000 grants each year is a start, but more is

needed to make performance measurement work. The performance

partnership grants must build a meaningful infrastructure and

they must require data that will be useful to the States and to local

22

governments, the counties in particular, as well as to the Federal

Government, or the whole process will be viewed as too burdensome and will not be effective. This accountability is a critical element of SAMHSA’s Federal stewardship for mental health. That

leadership role for the Nation is critical.

Now, conventional testimony, I would say would end with this

single point, calling for re-authorization of SAMHSA and increased

spending to build the infrastructure for performance and for outcomes monitoring. We could stop now in a conventional sense, but

that conventional approach will not stimulate the transformation

that is needed and that was at the heart of the President’s New

Freedom Commission recommendations and its vision for recovery.

The second point is that stewardship of mental health must extend beyond the traditional mental health system to all of the service systems in which people with a mental illness and a substance

abuse problem are found. The traditional stewards of mental

health have been asked to be responsible for meeting the many

needs of individuals who are affected by mental health illness, yet

they do not control the majority of the resources needed to accomplish this task. If we are to take seriously our responsibility for

these outcomes for individuals with mental illness, then we must

hold all of the systems accountable for their performance.

If SAMHSA is to be the Federal mental health steward, then this

stewardship must empower the agency to oversee this broad accountability process. SAMHSA must be invested with more authority to work collaboratively with all of the other systems and agencies.

In short, focusing on SAMHSA and the State mental health

agencies and requiring reporting performance measures in their

programs alone without at the same time looking at the performance of other programs will merely perpetuate the fragmentation

of the current mental health system and do little to advance the

goals of the President’s commission. If we are serious about recovery and about improving outcomes for adults and children with

mental disorders in all of the systems where people with these disorders are found, we must empower leadership. We must hold all

of these systems accountable.

Now, intentionally, the report of the President’s commission with

its enumerated goals and recommendations left us with its own set

of rudimentary performance measures. We think that this a serious

place to start, and one measure of the campaign’s performance is

the re-authorization of SAMHSA. We appreciate that the committee is approaching the task in the same vein, and thanks for the

opportunity to present before you. I look forward to future questions.

[The prepared statement of Dr. Goldman follows:]

PREPARED STATEMENT OF HOWARD H. GOLDMAN, M.D., PH.D.

Mr. Chairman, Senator Kennedy, and Members of the Subcommittee:

Good morning Mr. Chairman, Senator Kennedy, and Members of the Subcommittee. My name is Howard Goldman. I am a psychiatrist and mental health services

researcher at the University of Maryland School of Medicine and served as the senior scientific editor of the Surgeon General’s 1999 Report on Mental Health and as

a consultant to the President’s New Freedom Commission on Mental Health. I am

honored to participate in today’s hearing and am proud to be doing so on behalf of

the Campaign for Mental Health Reform.

23

Our Campaign, galvanized by the call of the President’s New Freedom Commission on Mental Health to transform mental health care in America, was created to

serve as the mental health community’s united voice in promoting Federal policy

changes that will transform mental health care from a fragmented, unresponsive,

and inefficiently funded delivery system to one that meets the needs of services

users and their families, is integrated across programs, and is adequately and responsibly funded.

I am pleased to respond to your invitation to discuss what we have learned about

performance and outcomes in mental health services and our capacity to measure

effectiveness of programs from multiple perspectives. I will review what we know

about this important topic and its implications for mental health policy generally

and for the Federal role and SAMHSA leadership in particular. My comments will

draw upon current research and numerous publications, as well as two reports of

the Surgeon General and the reports of the President’s New Freedom Commission

on Mental Health.

In the course of a year, about one in five persons has a diagnosable mental disorder, excluding substance use disorders. Almost everyone’s life has been touched

in some way by mental illness—if not due to one’s own impairment, then in caring

for family members, close friends, or colleagues. Unfortunately, notwithstanding the

existence of effective treatments and services and the real prospect for recovery, the

majority of individuals who have a diagnosable disorder do not seek or find the help

they need. This personal tragedy and public health failing is even worse for members of ethnic and racial minorities.

There are many reasons for this crisis: inadequate funding, lack of parity in insurance coverage, stigma, shortage of mental health professionals, and lack of political

will to make mental health a priority. Another relates to the focus of this hearing,

namely, the challenges associated with documenting performance and outcomes of

mental health interventions.

Fortunately, we can do far better. The Surgeon General’s 1999 Report on Mental

Health established that mental health is fundamental to health. Mental disorders

are real health conditions that impose a tremendous burden on the population in

terms of disability, economic loss, and human suffering. Yet, recovery—wherein people with mental disorders are able to live, work, learn, and participate fully in their

communities—is possible, even expected. The literature makes clear that there is a

range of well-researched and efficacious interventions that successfully treat most

mental disorders of adults of all ages, children, and adolescents.

The hopeful findings concerning scientific advances and recovery are tempered by

the wide gap between science and practice. Evidence-based services and other valuable though less thoroughly documented promising and emerging practices are often

not available in many communities, and implementing such practices can be complex and difficult. Barriers impede their use, including resistance to change by entrenched and threatened organizational structures, obsolete reimbursement rules,

and, most importantly, lack of resources necessary to support training and dissemination and to provide incentives for innovation. The hard reality is that millions of

Americans who need mental health services to achieve positive clinical outcomes do

not receive any and, for many, the care that is furnished is inappropriate, inadequate, ineffective or obsolete. There are too many stark manifestations of our system’s failure, including the 30,000 lives lost each year to suicide and the hundreds

of thousands of people with a mental disorder who are homeless, unemployed, or

inappropriately institutionalized or incarcerated.

The promise of recovery combined with the sobering reality of the enormous gaps

in the system of services set the stage for President Bush’s New Freedom Commission on Mental Health. The President, aware of the promise, sought to reveal and

tear down the barriers to appropriate care and community participation. Following

a year of study and consultation, the Commission transmitted to the President its

report calling for the transformation of mental health in America. The report—

Achieving the Promise—is organized around six goals that assert that in a transformed mental health system:

1. Americans understand that mental health is essential to overall health.

2. Mental health care is consumer and family driven.

3. Disparities in mental health services are eliminated.

4. Early mental health screening, assessment, and referral to services are common

practice.

5. Excellent mental health care is delivered and research is accelerated.

6. Technology is used to access mental health care and information.

Within each goal are specific recommendations designed to transform mental

health care and improve systems performance and individual outcomes. The Commission recognized and the Campaign for Mental Health Reform firmly agrees that

24

accountability is fundamental to each of the goals articulated in the report. An accountable system empowers consumers and family members by enabling them to

make informed decisions about treatment. It supports policymakers and administrators who must make informed decisions about planning and resource allocation. It

improves the quality of provider practice and results in improved clinical outcomes.

And it is critical in generating the political support necessary to fund and maintain

the system.

An accountable system is one that can measure both the performance of its programs and the outcomes achieved by the people it serves. With such data, policymakers and mental health providers may monitor and continually refine their programs. They will learn whom they are reaching (and not reaching), what supports

they are providing, what outcomes they are achieving, and what refinements or

modifications are needed to enhance its effectiveness.

Leaders in the field understand the value of performance and outcome measurement, and over the last 10 years we have seen tremendous progress. There is consensus and remarkable consistency across jurisdictions and stakeholders regarding

the outcomes that mental health systems and services are intended to achieve: reduction in symptom distress; building social supports; community participation; improvement in work or, in the case of children and adolescents, age-appropriate functioning; reduced homelessness and inappropriate hospitalization; improved general

health status; and decreased contact with criminal and juvenile justice systems.

Over the past few years, States, with only modest Federal support, have worked to

develop performance measurement systems along these lines. A handful of States—

Ohio, Texas, Colorado, Washington, and Oklahoma among them—have implemented

systems to obtain these data on a statewide basis, but the majority of States are

currently in the process of building such systems.

But implementing these systems is not just a matter of administrative fiat or will.

Identifying and implementing measures for uses such as planning, budgeting, monitoring, and quality improvement is enormously complex, expensive, and labor intensive. Resources are necessary to update or, in some cases, create information technology systems that would enable States and counties to collect, access, link, and

analyze the relevant data. Investing in infrastructure at a time when budgets are

being slashed and public mental health systems are already failing to provide the

services and supports needed by most consumers and family members can be difficult for States.

This suggests the critical role that the Federal Government must play in helping

enhance and expand performance measurement systems: first, in consultation with

stakeholders, developing meaningful measures and definitions; second, ensuring the

dissemination and implementation of these measures; and third, funding States and

counties that are creating performance and outcome measurement systems, particularly to the extent the measures are federally mandated and designed to present a

national picture. To date, the Federal commitment has been minimal, with States

receiving grants of between $100,000 and $150,000 per year to move billion-dollar

systems. To be sure, SAMHSA and the States, through changes to the mental

health block grant program, are making progress by placing greater emphasis on

performance and outcome measures, but SAMHSA must be sure that the data it is

requiring the States to report are of value not only to the Federal Government, but

also to the States and counties in planning, quality improvement, and contracts

management. To the extent those goals are not aligned, the Federal Government

must be prepared to cover more of the financial burden.

Much more needs to be done in the area of mental health performance and outcomes measures, and we must move quickly: the future of mental health services

in this country depends on our ability to improve the quality and accountability of

mental health systems. But without the leadership, investment, and defined expectations that the Federal Government is in a position to provide, the impetus for

change in this area is likely to atrophy.

We cannot, however, end our testimony here. Certainly we must consider accountability in the context of reauthorizing the programs of the Substance Abuse and

Mental Health Services Administration. But we already know that SAMHSA programs have value in communities. For example, SAMHSA programs play a crucial

role in piloting and disseminating information about innovative programming as

well as established best practices. The issue goes far beyond SAMHSA, however,

and we urge that you heed one of the most important observations of the President’s

Commission: that transforming mental health care in America will require fundamental change in all social services settings at Federal, State, and local levels.

Although SAMHSA must be looked to for its leadership at this time, we must not

lose sight of the fact that the resources it controls are dwarfed by those of the myriad programs and supports that serve adults and children with mental disorders in

25

other systems, such as criminal justice, housing, Medicaid, Medicare, child welfare,

vocational rehabilitation, special education, and SSI and SSDI.

We are encouraged by the seriousness with which this Committee is responding

to the call of the President’s Commission. We look forward to working with you to

craft legislation that will translate that call into bold action. A conventional approach to reauthorizing this agency will not result in transformation. Indeed, how

can the stewards of mental health care, namely SAMHSA at the Federal level, and

State mental health agencies, remain accountable and properly assess performance

and outcomes when they each control only a small fraction of the resources needed

to address these needs? The lesson of the Commission is that transforming the mental health system will require change in social services policy broadly. If SAMHSA

is to be tasked with monitoring performance and outcomes of mental health programs, then it must be able to work collaboratively with all of the other systems

and agencies whose policies affect individuals with consumers and their families.

That will require an investment of greater authority in SAMHSA. This, the Campaign believes, would be a sound investment. Only SAMHSA has as its core mission

the delivery of effective services to people with mental disorders, and with so many

competing interests, its leadership now is more important than ever before.

In short, focusing on SAMHSA and the State mental health agencies and requiring reporting of performance measures in their programs, without at the same time

looking to the performance of other programs will merely perpetuate the fragmentation in the public mental health system and do little to advance the goals of the

President’s Commission. If we are serious about recovery and about improving the

outcomes for adults and children with mental disorders in all systems where people

with mental disorders are found, we must hold all of these systems accountable. But

we cannot do this in good conscience without empowered leadership and without investing the resources necessary to achieve our goals.

Intentionally, the report of the President’s Commission with its enumerated goals

and recommendations left us with its own set of rudimentary performance measures. The Campaign for Mental Health Reform, for example, holds itself accountable

for robust policy change that will achieve the outcomes envisioned by the Commission. We view the reauthorization of SAMHSA as one measure of our performance.

We appreciate that this committee is approaching its task in the same vein.

Thank you for the opportunity to appear this morning before you and your subcommittee. I would be more than happy to answer any questions.

REFERENCES

Daniels, A.S. & Adams, N. (2004). From Policy to Service: A Quality Vision for

Behavioral Health: Using the Quality Chasm and New Freedom Commission Reports

as a Framework for Change. Pittsburgh, PA: American College of Mental Health Administration.

Drake, R.E. & Goldman, H.H. (Eds.). (2003). Evidence-Based Practices in Mental

Health Care. Arlington, VA: American Psychiatric Association.

Executive Order # 13263 Establishing the President’s New Freedom Commission

on Mental Health. April 29, 2002. Washington, DC.

Ganju, V., Smith, M.E., Adams, N., & Allen, J., Bible, J., Danforth, M. et al.

(2004). The MHSIP Quality Report: The Next Generation of Mental Health Performance Measures. Rockville, MD: Center for Mental Health Services, Mental Health

Statistical Improvement Program.

Ganju, V. & Lutterman, T. (1998). Five-State Feasibility Study: Implementing

Performance Measures Across State Mental Health Systems. In Mental Health,

United States, 1998 (pp. 45–52). Rockville, MD: Substance Abuse and Mental Health

Services Administration, Center for Mental Health Services.

Ganju, V. & Lutterman, T. (Eds.). (1998). Recommended Operational Definitions

and Measures to Implement the NASMHPD Framework of Mental Health Performance Indicators: Report of the Technical Workgroup on Performance Indicators. Alexandria, VA: National Association of State Mental Health Program Directors.

Lehman, A.F., Goldman, H.H., Dixon, L.B. & Churchill, R. (2004). Evidence-Based

Mental Health Treatments and Services: Examples to Inform Public Policy. New

York, NY: Milbank Memorial Fund.

Lutterman, T., Ganju, V., Schact, L., Shaw, R., Monihan, K. et al. (2003). 16 State

Study on Mental Health Performance Measures. Rockville, MD: Substance Abuse

and Mental Health Services Administration, Center for Mental Health Services.

The President’s New Freedom Commission on Mental Health. (2003). Achieving

the Promise: Transforming Mental Health Care in America: Final Report. Rockville,

MD.

26

The President’s New Freedom Commission on Mental Health. (2002). Interim Report to the President. Rockville, MD.

United States Public Health Service Office of the Surgeon General. (2001). Mental

Health: Culture, Race and Ethnicity: A Supplement to Mental Health: A Report of

the Surgeon General. Rockville, MD: Department of Health and Human Services,

U.S. Public Health Service.

United States Public Health Service Office of the Surgeon General. (1999). Mental

Health: A Report of the Surgeon General. Rockville, MD: Department of Health and

Human Services, U.S. Public Health Service.

STATE PERFORMANCE AND OUTCOME MEASUREMENT SYSTEMS: SAME EXAMPLES

1. OHIO MENTAL HEALTH CONSUMER OUTCOMES SYSTEM

The Ohio Mental Health Consumer Outcomes System is a standardized way of

measuring levels of health and well being experienced by consumers of Ohio’s public

mental health system. The outcomes being measured were selected by the Ohio Outcomes Task Force a Hogan, Ph.D, in September 1996. The measures were pilot tested by a multi-constituency work group in 1998–1999.

The outcomes System is now in operation in a majority of board areas in the State

and Data in the Department’s Outcomes data base have been used to produce a series of statewide reports for local systems. These reports and other information

about the Ohio Mental Health Consumer Outcomes System can be found on the

Outcomes Web site.

The Ohio system includes measures related to quality of life, Symptom distress,

community functioning, safety, employment and involvement with the criminal justice system.

2. OKLAHOMA PERFORMANCE AND OUTCOME MONITORING SYSTEM

The Performance and Outcomes Monitoring Report for Community Mental Health

Centers has been prepared for use by consumers, advocates, planners, treatment

providers, administrators, and other decisionmakers. The report consists of two volumes. Volume One contains performance and outcome and indicators based on a

framework adopted by the National Association of State Mental Health program Directors (NASMHPD). Volume Two contains service utilization data. Also, a Statewide Summary is presented.

Both Volumes contain three sections of charts and corresponding tables that display summarized information for (1) all clients, (2) adults with a serious mental illness (SMI), and (3) children with a serious emotional disturbance (SED). Also included are appendices for definitions, data selection criteria, service categories, and

a State map that depicts community mental health center (CMHC) service areas.

Data for the current fiscal year and the previous fiscal year are presented for yearto-year comparisons.

3. WASHINGTON STATE PERFORMANCE INDICATES SYSTEMS

Performance indicates for the Washington State mental health systems are divided from data from remains data bases and surveys. Regular quarterly reports are

produced which provide data for each administrative region and allow for comparison crossing regions. The performance and outcomes measures include: Penetration

rates inpatients and outpatient utilization, follow up after hospital discharge, employment status, living situation and consumer perceptions of access, quality of care

and outcomes.

4. TEXAS MENTAL HEALTH OUTCOMES SYSTEMS

Performance and outcomes measures for the Texas mental health system developed by representatives of stakeholder groups and staff are used for strategic planning, legislative reports, contracts management and quality improvement. Data are

obtained from all adults and children and youth receiving services. The performance

measurement and outcomes system includes measures related to: functioning symptoms, employment, school functioning, involvement with criminal/juvenile Justice

system and implementation of evidence-based practices.

Senator DEWINE. Doctor, thank you very much.

Mr. Tester, thank you for joining us.

27

STATEMENT OF GARY TESTER, DIRECTOR, OHIO

DEPARTMENT OF ALCOHOL AND DRUG ADDICTION SERVICES

Mr. TESTER. Thank you for inviting me to testify on this issue.

I am presenting both the viewpoints of the National Association of

State Alcohol and Drug Abuse Directors as well as the concerns

from Ohio regarding this issue.

First of all, I think Administrator Curie was exceptionally accurate in talking about the partnership that had evolved around the

discussions for the performance partnership grants. Prior to becoming director of the State Department, I was chief of Prevention

Services for Ohio and served as the State’s National Prevention

Network representative. Beginning in early 2001 I had the opportunity to participate in a number of committee meetings and work

group meetings that focused on the prevention measures in alcohol

and other drug issues associated with the performance partnership

grant discussions. Similar work groups were facilitated through

CSAT and SAMHSA with State directors to discuss the treatmenttype issues.

Personally, I found those meetings quite helpful. They provided

a rich dialogue and an opportunity to hear from a diverse sector

of States, from the very large States on my committee of California,

New York, Texas, and Ohio to the very small States. Rhode Island

and Connecticut were two that were representing the smaller side,

and we were able to talk about the various elements of the infrastructure and the various concerns we had about how we would

meet core measures if we were to move there. And I think that we

should not lose perspective on just how significant those work

groups were and those conversations were to help us get where we

needed to be.

Through that process, we were able to develop a set of what I

will call probable core measures for both prevention and treatment.

Preventionists being as we are, it took us a great many more measures to feel good about what it is that we wanted to accomplish,

but, nonetheless, we reached what we felt was a good conclusion

about many of the measures and anxiously awaited then the opportunity to learn about which ones we would narrow down in order

to make sure our States were moving forward.

As the chief of Prevention Services at this time, I was challenged

by my director at that time, Lucille Flemming, to begin to create

core measures in prevention for Ohio based on the discretionary

grants that the Department of Alcohol and Drug Addiction Services

administers so that we could begin to align our system with what

we anticipated would be the performance partnership grant process.

With that in mind, one of the difficulties that we experienced

was assessing exactly what it would take from a cost standpoint to

implement the appropriate infrastructure to make this happen. As

you know, Senator, Ohio is a state-funded, county-administered

State. We have furious home rule, and as we look at our county

alcohol and drug boards and we look at our local providers, one of

the issues that we face is we can assess what it would take at the

State level to implement as Mr. Curie noted. I think the ideal

would be a web-based platform that would allow us to report both

prevention and treatment outcomes; however, in simply looking at

28

what we think it would take at the State level from the department

in order to get our pieces in place, we are estimating conservatively

about $3.8 million in the first year alone to get us to where we can

accept outcomes from all providers on core measures and then funnel that information both up to the Federal Government so it could

be used wisely by SAMHSA and in reporting to Congress, but just

as importantly to be able to put that information back down to

local boards and then to the local providers, because as Dr.

McLellan indicated, this information is critical to helping them understand what processes they are using. If we know we can keep

a client, for example, on the treatment side of the aisle, if we are

doing well for 30 days post-treatment, but we do not know what

happened 60 days post-treatment, we have to go back and take a

look at that, and right now, our system is not set up to do that. So we are estimating at the State level alone $3.8 million for the

first year, 1.8 Million for the second year.

Under the current domains or categories that Administrator

Curie has noted, the outcome measures that are proposed do make

intuitive sense, and I agree with Dr. McLellan. He is far more intelligent on this issue than I, but we do have a good idea of what

it takes. The critical part from a State perspective comes in what

the exact measures will be, because each time we look at a measure or tweak a measure, we are faced with what it is that we have

to do then at the local provider level to help them gather that appropriate information from each of the consumers that they serve,

and this becomes then more costly because we are taking time from

clinical folks and we are putting it into evaluations.

States very clearly, we want to work with SAMHSA. I do not

think there is any question that we agree that this needs to be an

outcome-based system. We just really need to put some things in

place to put the finishing touches on this dialogue so that we can

move forward in an effective model. I think Ohio is a State that

clearly is prepared to move forward with that.

I conclude my testimony and will look forward to your questions.

[The prepared statement of Mr. Tester follows:]

PREPARED STATEMENT OF GARY Q. TESTER

BACKGROUND

There was a time, when in order to generate more funding for alcohol and other

drug addiction services, I would paint you a picture—a figurative picture—a compelling picture—of a sick, crack-addicted mom and her three young children to tug at

your heartstrings and hopefully loosen the purse strings. But we all know that those

days are over. We still care strongly about that mom and her three children, but

today, we want to know more; we must know more.

Did she reach a sustained recovery? Is she employed? Is she going to school? Has

she found safe, affordable housing? Has she been reunited with her children before

the Adoption and Safe Families clock stopped ticking? Are the children succeeding

in school? Is she a good parent?

How do we know if our services are working to improve lives? We create performance measures covering many of the categories just listed.

THE CHILDREN’S HEALTH ACT—A ROADMAP FOR A PERFORMANCE DATA SYSTEM

The National Association of State Alcohol and Drug Abuse Directors (NASADAD)

and other national organizations joined the Substance Abuse and Mental Health

Services Administration (SAMHSA) to support language in the Children’s Health

Act of 2000 (P.L. 106–310) triggering a transition from the current Substance Abuse

Prevention and Treatment (SAPT) Block Grant to a Performance Partnership Grant

29

(PPG). The goal of the transition is to increase State flexibility in the use of funds

in return for increased accountability based on performance. Both SAMHSA and

NASADAD also agreed that the transition should be based on a ‘‘Continuous Quality Improvement (CQI)’’ mechanism versus a punitive system that could threaten

the flow of much needed resources to our already strained system.

The Act required SAMHSA to work with States to release a report to Congress,

due October 17, 2002, detailing the transition to a PPG, including: (1) a description

of the flexibility that would be given to States; (2) the common set of prevention

and treatment performance measures that would be used for accountability; (3) definitions for the data elements to be used under the plan; (4) the obstacles to implementation of the plan, and the manner in which such obstacles would be resolved;

(5) the resources needed to implement the performance partnership; and, (6) an implementation strategy complete with recommended legislative language.

NASADAD POSITION STATEMENT ON PPG TRANSITION

NASADAD outlined core priorities pertaining to the transition to the PPG in a

Position Statement released this year. The Position Statement summarized

NASADAD’s previous correspondence and testimony regarding the Association’s

views. Some core priorities are as follows:

(1) A True State-Federal Partnership.—States must be an equal partner as the

PPG transition is developed and implemented. State input must be incorporated

into (a) legislation addressing the PPG, (b) any proposed changes to the Block Grant

application seeking performance data, and (c) the timing of the transition and other

aspects of PPG implementation.

(2) Federal Funding For Data Management and Infrastructure.—As SAMHSA

noted in its own December 24, 2002 Federal Register Notice, ‘‘Critical to the collection and reporting on performance measures is the ability to upgrade the data infrastructure of the State . . . without improved data infrastructures in States, many

will not be able to collect and report performance measures.’’ We could not agree

more.

(3) Incentives Yes—Penalties No.—NASADAD agrees with SAMHSA’s statement,

also included in its December 24, 2002 Federal Register Notice, that ‘‘The new partnerships will be built on incentives to improve services rather than penalties for

noncompliance.’’ This is vital.

POSITION PAPER OUTLINES NEXT STEPS

NASADAD outlined recommended next steps needed to be taken in terms of PPG

transition, including:

• The submission by SAMHSA of a report to Congress, as required by P.L. 106–

310, that provides a suggested roadmap for the transition,

• An assessment of State capabilities and readiness to report PPG data as required by P.L. 106–310,

• Allocation of new and additional resources to assist with the transition, particularly in terms of data system conversions, and

• A process whereby legislation that incorporates State input is considered and

passed.

I have submitted the NASADAD Position Statement to the Committee for the

Record.

PPG ACTIVITY

Since the Children’s Health Act was passed, SAMHSA, NASADAD and its members, including State directors and National Prevention Network representatives,

worked to develop and refine performance measures that we all can work toward.

States have been preparing to transition from the current SAPT Block Grant to

PPGs for a number of years. SAMHSA released an excellent overview of the

progress on PPG in a December 24, 2002 Federal Register Notice. NASADAD provided comments along with specific proposed measures and other recommendations.

MORE RECENT ACTION

As you know, SAMHSA Administrator Charles Curie recently announced his

Agency’s policy that seeks to require SAMHSA grant recipients to report information on seven core ‘‘domains’’ or categories. In general, NASADAD agrees that the

seven categories represent important information. NASADAD is concerned, however,

with some specific requirements and measures included in some of the categories.

For example, SAMHSA proposed to measure clients’ connectiveness to society or

participation in recovery support activities at discharge. We agree that information

30

pertaining to a client’s participation in self-help groups and other data is important.

Much more work is needed, however, to develop ways to accurately define and measure elements within this category.

These concerns, along with the principles included in NASADAD’s PPG Position

Statement, led NASADAD to oppose SAMHSA’s recent proposed changes to the fiscal year 2005–2007 SAPT Block Grant application that appeared in the Federal

Register on March 30th of this year. In a May 28th letter to SAMHSA opposing the

changes, NASADAD President Michael Couty (Missouri) wrote,

NASADAD supports the use of performance measurement and other data to

help reach our ultimate goal: improving our substance abuse service delivery

system. We applaud and share the Administration’s dedication and desire to improve the lives of millions across the country who are at risk for or have substance abuse problems. We also appreciate and share the Administration’s desire to avoid unnecessary delay in developing a Federal performance measurement system.

However, a review of the Federal Register Notice found (1) no increase in

flexibility, (2) no substantial increase in resources, (3) no reduction in reporting

burden, (4) a substantial increase in reporting burden and (5) a small set of performance measures that are inappropriate. As a result, we look forward to continuing our work with the Substance Abuse and Mental Health Services Administration (SAMHSA) and others to change our data reporting system in a manner consistent with our core principles outlined above.

Support for any data changes in the SAPT Block Grant application is predicated

on the need to provide States with increased flexibility and resources—along with

reduced reporting burden in other aspects of the application.

TIMING

It is also important to note that States must submit a completed SAPT Block

Grant application for fiscal year 2005 by September 30th. This Application is complex and takes many person hours to complete. It is our understanding that the Office of Management and Budget (OMB) will consider the initial comments sent to

SAMHSA. Subsequently, OMB will release in the Federal Register the Administration’s final proposal to change the Block Grant application with a 30-day comment

period. As a result, even if the OMB proposal came out today, States would still not

be able to begin to complete the final SAPT Block Grant application until late August—giving States only 1 month to complete a large and complex application. This

is problematic given (1) the application could ask for new and expanded data requirements, (2) States are required to seek and consider public input into the application, and (3) the sheer person hours required to complete the application.

As a result, we again recommend that meetings move forward as soon as possible

between NASADAD and SAMHSA in order to achieve consensus on these key

issues. In particular, we believe the existing performance partnership workgroups

from SAMHSA’s Center for Substance Abuse Treatment (CSAT) and Center for Substance Abuse Prevention (CSAP) jointly meet with NASADAD. To date, meetings to

discuss the development of the prevention and treatment measures have moved forward separately—with separate work groups. In order to encourage collaboration

and coordination, a joint meeting is imperative.

NASADAD OUTREACH AND COMMUNICATION

NASADAD has focused on communicating our views regarding the transition to

PPG clearly and consistently. On several occasions, NASADAD highlighted the benefits of working collaboratively with States on many aspects of the SAPT Block

Grant. For example, NASADAD Executive Director Lewis E. Gallant, Ph.D., noted

the following in a response to SAMHSA’s December 24, 2002 PPG Federal Register

Notice:

NASADAD recommends that any changes in the Block Grant Application and

thus reporting related to performance measures, only begin after the following

move forward:

• An assessment by the Secretary of HHS of States’ readiness to report PPG

data,

• The allocation of new and additional resources to assist with data infrastructure and other administrative costs, and

• A process whereby legislation is passed by Congress, and signed by the

President, that truly reflects the principles of the PPG—including CQI and

a true State-Federal partnership.

31

Other examples where NASADAD iterated its position on changing the Application and other issues pertaining to the PPG transition include: (1) July 15th, 2003

testimony presented before the Senate Health, Education, Labor and Pensions’

(HELP) Subcommittee on Substance Abuse and Mental Health Services; (2) discussions held during the June, 2003 SAMHSA-NASADAD PPG workgroup meeting; (3)

a December 9, 2003 letter to Administrator Curie; (4) a January 22, 2004 meeting

with Administrator Curie and staff; (5) a February 4, 2004 letter to Administrator

Curie; (6) a February 17, 2003 meeting with Administrator Curie and staff; and (7)

the NASADAD Position Statement on PPG Transition released February 18, 2004.

OHIO-SPECIFIC EFFORTS

In Ohio, where we’re in year 3 of an across-the-board outcomes framework initiative, we’ve aligned State and local investor targets with anticipated Federal PPGs.

It is vital that these PPG targets remain consistent across grant opportunities and

Federal reporting needs so that the ongoing work of Ohio and other States is not

in vain.

In October of 2001 ODADAS began a 3-year implementation of its Outcome

Framework Initiative. The results to date have been significant: • ODADAS has re-designed its discretionary grant application process which now

fully incorporates the investor approach of the Outcome Framework. • ODADAS staff members have received substantial training and technical assistance to ensure that they can use investor tools and practices within ODADAS’ outcome management framework.

• Over 1000 providers have been trained in Outcome Management with an emphasis on results and the processes that lead to them.

• Every provider who requested it (over 300) received technical assistance on how

to apply Outcome Management to its program(s).

• All grant-funded providers responded to the grant application using an outcome

management framework with a focus on results and outcomes.

• All boards have attended Board-specific training sessions which introduced

them to investor thinking and practices.

• All boards have been invited to participate in technical assistance sessions with

providers.

• All boards responded to ODADAS’ outcomes questions in their Community

Plans and thus have begun to incorporate outcome planning and strategies into

their planning processes.

• Individuals employed by the Department, boards and providers have received

extensive skills training to facilitate ‘‘peer-to-peer’’ training and consultation in

order to sustain the effort.

ODADAS continues to progress to a fully integrated outcome framework in its

policies and operations. To that end, the focus has been on:

• Building sustaining capacity within the entire system.—The Train-the-Trainer

component will ensure that there are people within the system who can provide

training and technical assistance as needed.

• Management structure.—Investor thinking and practices are being integrated

into the management system to ensure the focus on results and outcomes into monitoring activities and contract management.

• Instrumentation.—Reporting structures and content are being designed to ensure that ODADAS, as well as providers and Boards, have the appropriate data

base for results-focused State and local strategic planning processes.

• Gathering and sharing of learning and best practices approaches.—Through the

use of the Outcome Framework: Investor Thinking and Practices, Outcome Management, Strategic Mapping and best practices will be evident to all within the system

and can be shared so that planning and implementation of prevention, intervention,

treatment and recovery services will be effective for Ohioans.

• Preparation for Federal direction.—Ohio has planned for the Federal focus on

results and outcomes that will be operationalized through proposed changes in the

Federal Substance Abuse Prevention and Treatment (SAPT) Block Grant application. The investment ODADAS has and is making in integrating the Outcome

Framework will ensure that the State SSA is well prepared for this Federal direction.

PREVENTION SERVICES AND PERFORMANCE MEASURES

ODADAS and its county Alcohol, Drug Addiction and Mental Health Services/Alcohol and Drug Addiction Services Boards and community providers recognize the

value of an alcohol/drug services system that is data driven, outcome focused,

grounded in evidence-based practices and continually updated.

32

Consistent with the Department’s Outcome Framework Initiative, prevention provider grant applicants must address two or more of the Center for Substance Abuse

Prevention’s strategies which include: • Information Dissemination; • Education; • Community-Based Process; • Environmental; • Problem Identification and Referral; • Alternatives.

All prevention grantees must develop performance targets that contribute to the

ODADAS investor targets that were developed to correspond directly to the proposed core prevention measures within the Performance Partnership Grants.

ODADAS investor targets are what define investor success in a quantitative way.

The challenge for the service provider is to clearly define how many customers

will reach the defined targets and what changes the provider is committed to

achieving for the people they serve. The prevention investor targets are attached at

the end of this testimony (Table 1).

TREATMENT/RECOVERY SERVICES AND PERFORMANCE MEASURES

ODADAS has taken a number of steps to ensure that its Outcome Framework is

aligned with the proposed PPG core treatment measures. These can be divided into

three categories: outcomes for grant-funded programs; outcomes for county boards,

and infrastructure to support the Outcome Framework.

Outcomes for Grant-Funded Programs

Each year, ODADAS provides grants to programs that provide treatment services.

These grants support Ohio’s investment in key areas such as: Women’s services, Adolescent services, Drug Courts, Therapeutic Communities, Juvenile Re-entry services and Treatment Alternatives to Street Crime (TASC). Ohio has established Investor Targets that define success. Programs contribute to the Investor Targets by

addressing one or more of them in their funding applications. ODADAS provides a

significant amount of training and technical assistance to its grant-funded programs

each year to insure understanding of this process.

For State Fiscal Year 2005, investor targets and target area(s) for treatment programs were established and aligned with the PPG core treatment measures. A table

comparing the PPG, Investor Targets and Target Areas is listed below (see Table

2).

Outcomes for County Boards

Alcohol, Drug Addiction Services (ADAS) Boards and Alcohol, Drug Addiction and

Mental Health Services Boards (ADAMHS)—the county agents for the State—are

required by Ohio law to prepare and submit to ODADAS a community plan for the

provision of alcohol and other drug addiction services in their service areas. The

plan, which constitutes the Board’s application for funds, is prepared in accordance

with procedures and guidelines established by ODADAS every 2 years.

Among the legislatively mandated responsibilities of the Board are: (1) assessing

service needs and evaluating the need for programs; (2) setting priorities; (3) reviewing and evaluating substance abuse programs; and (4) assuring effective services

that are of high quality.

The evaluation section of the Community Plan guidelines addresses outcomes (results) of the previous year’s plan. Boards are required to describe what constitutes

success in their systems. In the most recent iteration of the guidelines, ODADAS

incorporated the Outcome Framework as a means for Boards to comply with the

evaluation requirements and to make sure that the data collected was consistent

with the PPG measures.

Changes in Infrastructure to Support the Outcome Framework

ODADAS, through its Governor’s Advisory Council on Alcohol and Drug Addiction

Services, has taken steps to build on the Outcome Framework by establishing a

standing committee on outcomes issues. Other steps include expanding the number

of individuals who are trained Outcome Framework trainers and by providing training to county Boards on outcome-based planning.

The Department’s organizational structure has also been altered to better align

State resources for maximum impact on quality, accessible services for all Ohioans.

ODADAS has added a Division of Planning, Outcomes and Research to spearhead

long range quality improvement and expanded its Division of Treatment and Recovery Services to encompass all of the continuum of care services that comprise holistic wrap-around care. All of these efforts have been undertaken in the context of

33

a connection between enhanced customer service, Ohio’s Investor Targets and the

PPGs.

DATA COLLECTION—ACCESS TO RECOVERY (ATR) VS. PPG

While the data elements collected in ATR are going to provide grantees with good

information on their programs, closer alignment and consistency with PPG measures would be beneficial. The States anxiously await the joint meetings proposed between NASADAD and SAMHSA to establish a definitive listing of those measures

so that all preliminary planning can become finalized.

New opportunities such as Access to Recovery are welcomed by every State. Clearly, ATR performance outcomes, PPGs and State outcome targets must be consistent

and trackable.

IMPLEMENTATION COSTS

Resources are needed to help States build systems that will collect, track, refine,

manage, analyze and disseminate data in accordance with the anticipated new requirements in the PPG. Funding is needed to reengineer the business processes in

substance abuse prevention, intervention, treatment and recovery to effectuate a

performance measurement system.

Based on conservative figures, ODADAS estimates that implementation of the

proposed Federal PPG infrastructure would cost the State $3.8 million in the first

year alone. The second year and annual costs would be $1.8 million per year. Should

SAMHSA require implementation of the PPG structure for next Federal fiscal year,

Ohio would have to pull at least $4 million from prevention and treatment services

funding. This amount does not include the local cost to county Boards and service

providers who have staffing and information technology needs that must be addressed if they are to meet these requirements. A sample of other State cost estimates, provided by NASADAD, is included below:

• California—$6.2 million for treatment data—this does not include prevention

data or out-year estimates;

• Texas—$1.9 million initial costs, $1 million each of the following years to maintain;

• Michigan—$2.3 million in new costs;

• Washington State—$750,000 to initiate the transition, and $350,000 each of the

following years.

States are not simply asking for Federal assistance without substantial investments of their own. In a report written in November 2001 by NASADAD for

SAMHSA, research found that the total State expenditures for the operation and

maintenance of alcohol and other drug data delivery systems in a year was over $35

million. As a result, we know that substantial resources are already being spent by

States on substance abuse data management. It is estimated that millions more will

be required to upgrade State data systems to meet PPG data requirements. The

States fully intend to work with SAMHSA to achieve the desired goals related to

PPG implementation and request Federal funding support to further existing State

efforts.

CONCLUSION

Ohio is ready and willing to partner with the Federal Government in establishing

and working toward well-defined performance measures. We have been laying the

groundwork for the past 3 years. For Ohio and other States, however, a financial

burden comes with a change of this magnitude. We’ve all heard the dreaded phrase

‘‘unfunded Federal mandate.’’ I ask you, on behalf of all Single State Authorities,

to carefully consider and review where we are, where we need to be and precisely

how we should all get there. The SSA’s, through NASADAD, will diligently work

with Congress and SAMHSA to reach a new level of accountability and quality performance.

I’ll be happy to entertain your questions.

34

TABLE 1.—PREVENTION

Proposed Federal Performance Partnership

Grants (PPGs) ODADAS Investor Targets Implementor Target Areas

Youth who have not used ATOD in the

past 12 months.

1. Programs that increase the number

of customers who avoid ATOD use

and perceive non-use as the norm..

a. Increase the number of youth and/or

adults who avoid ATOD for a defined

period of time.

b. Increase the number of youth and/or

adult who perceive an ATOD using

lifestyle unacceptable and do not

use.

c. Increase involvement of youth engaged in ATOD-free alternative activities.

d. Increase the number of youth who

become positive peer prevention

leaders.

e. Increase the number of youth with

enhanced resistance skills.

f. Increase the number of youth who

have more non-using peers than

using peers

Youth who obtain resistance/refusal

skills.

..........................................................

Youth who understand the risks/harm

of use of ATOD.

2. Programs that increase the number

of customers who perceive ATOD use

as harmful..

a. Increase the number of youth and/or

adults who have increased knowledge of the risk and harm of ATOD

use and avoid ATOD use for defined

period of time.

b. Increase the number of women who

have increased knowledge of the

risk and harm from ATOD use and

eliminate use while pregnant.

c. Increase the number of women who

deliver a drug-free baby.

Youth who have favorable attitudes toward non-use...........................................................

Youth who have increased protective

factors.

3. Programs that increase the number

of customers who experience positive family management..

a. Increase the number of families who

provide increased clear consistent

expectations, rules and consequences including non-acceptance

of ATOD use.

b. Increase the number of youth who

gain protective factors at home,

school and/or community.

c. Increase the number of youth who

reside in a safe and violence-free

home environment.

Perceived parental attitude .................... ..........................................................

Reduced availability of ATOD ................. 5. Programs that increase the number

of initiatives that demonstrate an

impact on community laws norms..

a. Increase the impact toward reduction or elimination of ATOD use.

b. Increase the compliance of ATODrelated laws and regulations.

c. Increase productivity, performance

and attendance at the workplace.

d. Decrease accidents and worker’s

compensation costs and/or reduce

health care costs, theft and other

losses.

e. Decrease the availability of ATOD in

the community.

f. Increase the number of medical professionals who identify at-risk behavior concerning the problematic

use of alcohol and other drugs.

35

TABLE 1.—PREVENTION—CONTINUED

Proposed Federal Performance Partnership

Grants (PPGs) ODADAS Investor Targets Implementor Target Areas

6. Programs that reduce the number of

customers who misuse prescription

and/or over-the-counter medications.

a. Increase the number of youth and/or

adults who demonstrate an understanding of the proper use of prescription medications and/or overthe-counter medications.

b. Increase the number of adults who

demonstrate and commit to the

monitoring of prescription medications in the home.

TABLE 2.—TREATMENT

Proposed Federal Performance Partnership

Grants (PPGs) ODADAS Investor Targets Implementor Target Areas

Abstinence at discharge ........................ Customers who are abstinent for at

least 1 year beyond completion of

the program.

Minimum Requirement:

The number of customers who are abstinent at program completion.

Employed at discharge ........................... Customers who are gainfully employed

for at least 1 year beyond completion of the program.

Minimum Requirement:

The number of customers who are employed at discharge.

No criminal justice involvement ............ Customers who incur no new arrests

for at least 1 year beyond completion of the program.

Minimum Requirement:

The number of customers who incur no

new arrests at program completion.

Any target that was reported and approved from the SFY ’04 application

that you wish to report on this year.

Last year’s (SFY ’04) approved target(s)

36

37

38

39

40

41

Senator DEWINE. Mr. Tester, we appreciate your testimony very

much.

Ms. Medalie, thank you very much for joining us. You are our

last witness.

STATEMENT OF MARSHA MEDALIE, VICE PRESIDENT AND

CHIEF OPERATING OFFICER, RIVERSIDE COMMUNITY CARE

Ms. MEDALIE. Chairman DeWine, thank you for the opportunity

to present testimony today on behalf of Riverside Community Care

and Mental Health and Substance Abuse Corporations of Massachusetts. Riverside is a nonprofit behavioral health care organization serving over 50 communities in eastern and central Massachusetts. Through more than 60 programs, Riverside provides a comprehensive system of community-based mental health care, substance abuse treatment, developmental disability services, and

services to individuals with traumatic head injuries as well as community crisis response. We employ 1000 people and provide care to

over 12,000 people annually. Over our $33 million budget, about 68 percent of funds are

through contracts with State agencies, cities and towns, hospital

systems, and private foundations. Third-party payers make up

about 28 percent of our funding, and the remaining 4 percent of

revenue includes donations and other miscellaneous income. Mental Health and Substance Abuse Corporations of Massachusetts is

a State association of over 100 community-based providers.

MHSACM’s mission is to promote community-based mental health

and substance abuse services as the most appropriate clinically effective and cost sensitive method for providing care to those in

need.

Riverside values performance and outcome measures to help inform our quality of care assessment and strategic planning. Because we are a large organization, we cannot hope to truly know

how we are doing without formal data. Our out-patient mental

health clinics measure outcomes in multiple clinical spheres such

as depression, psychosis, suicidality, and mania. Our vocational

programs measure number employed, length of time employed, and

average wages. Our short-term adolescent day treatment program

uses a homegrown outcome measurement to survey participants’

perception of improvement on a number of functional measures

such as ability to manage anger, get along with family, communicate feelings and concerns.

State reporting requirements also dictate what data we collect.

Our adult residential programs report on the number of psychiatric

and substance abuse hospital days utilized, number of consumers

who achieve a majority of their treatment plan goals, and number

moving to lower intensity settings. Our out-patient substance

abuse intervention and outreach program gathers and reports extensive data to the State on a monthly basis, including many of the

seven treatment domains currently under consideration by

SAMHSA, such as arrests, substance use, etc.

However, it is a constant struggle to balance our data collection

efforts with competing pressures of limited funding and the myriad

of record-keeping and reporting requirements already imposed by

payers and accreditors. In Massachusetts, the number one comVerDate 11-MAY-2000 14:03 Apr 22, 2005 Jkt 000000 PO 00000 Frm 00045 Fmt 6633 Sfmt 6601 S:\DOCS\95101.TXT SLABOR3 PsN: SLABOR3
42

plaint from consumers is that staff are kept so busy with paperwork requirements that they do not have enough time for direct

service. From our 40-plus years as a provider and our experience

with outcome measurements, we strongly support the movement

towards performance measurement on a uniform national basis,

but also note that any change in funding or in data collection and

reporting requirements must ensure that it will not come at the expense of services, staff time to serve consumers, or provider viability. This is especially important for Massachusetts where providers

have been largely level-funded in State mental health and substance abuse contracts for 14 years despite the fact that our costs

have increased due to inflation and other factors and where State

agencies and some services have also sustained recent cuts.

Neither providers nor State agencies can afford to divert resources for further performance measurement programs. So I respectfully submit the follow recommendations:

Investment in building performance partnerships must come

from new Federal funds specifically for data management infrastructure, development, and maintenance, rather than eroding base

funding which could dramatically hurt providers like Riverside.

Providers do not have the ability to self-fund hardware, software,

etc., or spend additional staff time that would be required for data

collection and reporting.

Federal funding should require financial support for such new

mandates at the provider level. New mechanisms for developing

Federal block grant funding should not delay payments to the

States. This might delay payments to providers, many of whom

could not survive such a situation.

Until full evaluation of proposed measurements prove their validity and given the fact that many providers are already collecting

valuable data, proposed national measurements must be regarded

as guidance for further queries rather than determinants of program’s value.

Determining State funding of by outcomes risks incorrectly penalizing or rewarding programs for results beyond their full control. Federal funding should not be based on outcomes until experience is allowed for proper weighting of outside variables such as

the state of the local economy, availability of drugs, and unemployment statistics as well as the efficacy of the services being studied.

Performance measurement should support quality improvement

and assist in developing best practices, not create uncertain funding.

Finally, review of performance measurement programs should include ongoing feedback from all stakeholders, including providers

like Riverside and consumers of service.

Thank you for your consideration of my testimony.

[The prepared statement of Ms. Medalie follows:]

PREPARED STATEMENT OF MARSHA MEDALIE, LICSW, ACSW

INTRODUCTION

Chairman DeWine, Senator Kennedy, and Members of the Subcommittee, thank

you for the opportunity to present testimony on behalf of Riverside Community Care

and Mental Health & Substance Abuse Corporations of Massachusetts.

Riverside Community Care is an award winning, non-profit behavioral healthcare

organization serving over 50 communities in Eastern and Central Massachusetts

43

with a service area of one million people. Through more than 60 programs, Riverside provides a comprehensive system of community-based mental health care, substance abuse treatment, developmental disabilities services, services to individuals

with traumatic head injuries, community crisis response and other health and

human services for children, adults and elders.

Mental Health and Substance Abuse Corporations of Massachusetts is a State association of over 100 community-based providers. MHSACM’s mission is to promote

community-based mental health and substance abuse services as the most appropriate, clinically effective, and cost-sensitive method for providing care to those in

need. Accordingly, the organization advocates for appropriate public policy and adequate funding for each service and works with the administration and the legislature at both the State and national levels to support this goal. MHSACM serves as

a forum for the exchange of information and ideas among local mental health and

substance abuse providers and other constituents and encourages and supports education, research and evaluation, technical assistance, professionalism, family/consumer involvement and outcome-oriented service. Riverside Community Care is an

active member of MHSACM and I personally am a former officer of the Board of

Directors.

To provide some context, Riverside has developed through a series of mergers of

small and medium-sized organization and through creative new ventures. For example, we have developed unique relationships with local hospitals to deliver emergency psychiatric services, urgent behavioral healthcare, and collaboration between

medical and behavioral health services. We are committed to providing communitybased alternatives to institutional care and to offering the same single, high standard of care to all consumers, whether their care is publicly or privately funded.

Recent national awards include the Eli Lilly Reintegration Award in recognition

of our employment of people with mental illness, helping more than 300 adults with

mental illness secure and maintain competitive employment, the Negley Award for

Excellence in Risk Management for our multi-faceted program to safely treat highrisk consumers, and the National Council for Community Behavioral Healthcare’s

Award for Excellence for Community Crisis Response for our work in the aftermath

of local and national disasters.

Our organization employs 1000 full and part-time people and provides care to over

12,000 people annually.

OVERVIEW OF SERVICES PROVIDED BY RIVERSIDE COMMUNITY CARE

Riverside offers an integrated network of services designed to help individuals and

families challenged by behavioral health problems—including those with dual diagnoses of mental illness and substance abuse, developmental disabilities, and other

disabling conditions to live and function as independently as possible and to be contributing members of their own communities. The merger of several organizations

enabled us to gain economies of scale, reduce administrative overhead, and build a

system of care to ensure access to quality services for consumers needing comprehensive, coordinated treatment. Riverside’s original predecessor organizations

began in the 1960’s following the passage of the Community Mental Health Center

Act.

Today, Riverside is one of the largest community-based providers in Massachusetts and is highly regarded for our innovative, high quality services, progressive

and successful employment practices, and positive relationships with the State

Agencies and cities and towns that count on us to care for their constituents. Our

services are organized into four divisions:

The Family & Behavioral Health Division includes office-based and community

outreach clinical and support services for children, adolescents, adults and elders.

Programs include: six licensed outpatient mental health and substance abuse clinics; two 24-hour emergency service programs—the State designated emergency service providers for their geographies; two crisis stabilization/respite facilities; one adolescent and four adult psychiatric day treatment programs; an adolescent substance

abuse prevention program; five home and school-based treatment and outreach programs for youth and their families; a consultation and treatment program for adults

and children with both developmental disabilities and behavioral disorders; and two

early intervention programs—serving families with children from birth to age three.

A new addition to this Division is the Urgent Behavioral Care Center created in

conjunction with Milford-Whitinsville Regional Hospital in Central Massachusetts.

This program completes Riverside’s range of services as the behavioral healthcare

provider for this hospital and its large associated physician practice. Riverside provides the behavioral health emergency services for several other community hospitals within our core communities and is the contracted provider for emergency

44

psychiatric and substance abuse assessments for several managed care organizations.

Programs within the Family and Behavioral Health Division led our disaster response following national and local tragedies. Staff provided counseling and support

following events such as the workplace shooting at Edgewater Technologies in

Wakefield, the city of Newton bus accident in which four middle school children

were killed while on a class field trip in Canada, as well as 9/11 which had a devastating affect on many Massachusetts families and communities. Our staff were at

Boston’s Logan Airport immediately after the terrorist attack and we were part of

the MASS Counseling Network, a FEMA funded support network established by the

Massachusetts Department of Mental Health. Riverside also provided two half-day

trainings entitled Caring For Your Staff While They Care for the Community: What

Every Manager Should Know About Disaster Planning. The trainings were geared

to managers of organizations and local services that directly respond to disasters as

well as agencies that may be indirectly involved because of their role in the community. The seminars were offered free to participants from funding provided by the

Substance Abuse & Mental Health Services Administration/Center for Mental

Health Services through the Mass. Department of Mental Health.

The Mental Health Residential Division provides a wide range of residential services to over 232 adults with serious mental illness. Many of these consumers are

dually diagnosed with both mental illness and substance abuse problems. Programs

range from highly supervised group homes of four or five individuals with 24-hour

staffing to apartment programs where staff are located within easy reach of consumers who live in their own apartments to supported living in which staff are mobile

and do outreach to consumers in their own homes or apartments. These residential

options enable us to provide services to adults across the spectrum of needs, from

individuals requiring intensive help with activities of daily living or those needing

structured treatment environments and supervision to allow them to live safely with

others—including people with serious forensic histories of violence or sexual offenses, to those who can live more independently with reliable staff support. Our

residential services include a specialized residence and ‘‘step-down’’ outreach program for adults with mental illness and substance abuse.

Also included within this Division is a Peer Support program run by and for consumers of mental health services. Peer helpers are hired and trained to enhance the

social support networks and provide guidance in recovery for consumers who are

graduating from residential services.

The Clubhouse and Employment Services Division includes three psychosocial

clubhouse programs that utilize the strength of extensive peer support and a rehabilitative environment to provide vocational, social and independent living experiences for individuals who have a history of mental illness. Currently 683 members

are enrolled. Extensive employment placement services and on-the-job support are

offered. Club housing supports members who need intermittent help with activities

such as budgeting, negotiating with landlords, or getting along with roommates.

Two other Supported Education and Employment programs, Riverside Career Services, provide comprehensive career placement services designed to meet the needs

of adults whose education or careers have been interrupted by mental health problems. These programs offer pre-employment and education assessment and counseling along with individualized education and career planning, job placement, access

to colleges and job training programs and flexible ongoing support. They are highly

regarded for their success in helping adults achieve meaningful careers rather than

‘‘dead end’’ jobs and for their employment of staff with their own histories of mental

illness and serve as role models.

Also within this Division is a new Care Management program that helps caregivers concerned about an aging parent or a family member with a developmental

disability, mental illness, or traumatic brain injury by providing a thorough assessment and creating and implementing an appropriate care plan. Plans maximize

independence and promote the family member’s safety, community involvement and

skill building.

The Developmental and Cognitive Disabilities Division offers services designed to

meet the complex needs of individuals with mental retardation or traumatic brain

injury. Over one hundred adults receive residential services, in small group homes,

supported living (where individuals reside in their own homes and are visited by

mobile staff), and specialized homecare (individuals are placed with families who

agree to foster them, often for a lifetime). Family and individual support programs

provide services such as respite, recreational activities, provision of adaptive equipment, skill-training and specialized staff support to adults and children living in the

community with their families or by themselves. Four hundred and fifty people are

served through these support programs.

45

OVERVIEW OF FUNDING SOURCES

Riverside’s fiscal year 2005 annual budget of over $33 million includes a blend

of private and public funding. Approximately 68 percent of funds are through contracts with State agencies, cities and towns, hospital systems, and private foundations. Riverside maintains contracts with the Massachusetts Departments of Mental

Health, Mental Retardation, Public Health—Bureau of Substance Abuse Services

(BSAS) and Early Intervention, and the Massachusetts Rehabilitation Commission

(primarily for head injury services). State contract funding includes State and Federal funds, inclusive of Medicaid Rehabilitation Option funds and Block Grant

funds. Third party payers makes up 28 percent of Riverside’s funding. This includes

Medicaid, Medicare, HMO’s, insurance companies, and self pay from clients. Third

party payers are the largest source of revenue for our clinical services such as outpatient therapy and medication services, emergency services and psychiatric day

treatment. The remaining 4 percent of Riverside’s revenue include donations and

miscellaneous income such as donations, interest on accounts, small grants, and

consumer rents.

RIVERSIDE’S PERFORMANCE MEASUREMENTS AND QUALITY MANAGEMENT

Riverside’s senior management highly values meaningful performance and outcome measurements as well as consumer and payer feedback to help inform our

quality of care assessment and future strategic planning. Because we are a large

and complex organization, we cannot hope to know how we are truly doing without

formal mechanisms to provide data. With our extensive range of services, the instruments we use need to be appropriate for the specific programs, so that the feedback

we receive provides meaningful information that our managers can use for quality

improvement efforts.

Our Quality Management Department oversees the organization’s collection of

data and measurement of outcomes with the goal of assessing our effectiveness, efficiency and consumer satisfaction. Instruments used include standardized, validated

tools where available, performance measurements required by State Agencies, and

internally created measurements tailored to specific service modalities. Our commitment to ongoing assessment and quality improvement begins each year with our annual goals and objectives development at the organization and division levels. Following formal needs assessments in which consumers, payers and staff are surveyed, measurable goals and objectives are established. Progress is reviewed at regular intervals by a senior management committee and ultimately, the Board of Directors.

We have devoted substantial resources to developing and collecting quantitative

data on our performance (and complying with mandatory performance data collection), but are mindful of the need to carefully balance this with competing pressures

of limited funding and sizable staff workloads. The myriad of record keeping and

reporting requirements already imposed by payers, regulators, and accreditors are

highly labor intensive activities. In Massachusetts, we often hear from consumers

that their No. 1 complaint is that staff are kept so busy with paperwork requirements that they are not available to provide direct service.

We are very pleased that in our most recent results of consumer and family satisfaction surveys across Riverside, we yielded a 97 percent overall satisfaction rating

with 98 percent of consumers saying they would recommend our services to others.

Annual Performance Based Contracting Meetings with our State Agency funders

(such as the Department of Mental Health) have consistently yielded high praise for

the quality and effectiveness of our work. Massachusetts has instituted measurement requirements for many contracts with annual contract performance review

meetings. Some specific examples will be presented below.

In addition to these measures of Riverside’s success, all recent accreditation and

licensure surveys have been positive. For example, our organization and our vocational programs are accredited by CARF—the Rehabilitation Accreditation Commission. Our clubhouses all have the highest available certification from the International Center for Clubhouse Development (ICCD). Our residential programs for

adults with mental retardation received 2-year (longest possible) certification from

the Department of Mental Retardation’s QUEST survey, and all mental health and

substance abuse programs are licensed by the Department of Mental Health and/

or the Department of Public Health, where applicable.

EXAMPLES OF PERFORMANCE MEASUREMENTS AT RIVERSIDE

The Treatment Outcome Package (TOP) published by Behavioral Health Laboratories of Ashland, MA. measures outcomes in multiple clinical spheres such as deVerDate 11-MAY-2000 14:03 Apr 22, 2005 Jkt 000000 PO 00000 Frm 00049 Fmt 6633 Sfmt 6621 S:\DOCS\95101.TXT SLABOR3 PsN: SLABOR3
46

pression, psychosis, suicidality, mania, etc. and has nationally recognized, proven reliability. Riverside has been using the TOP in our outpatient mental health clinics

with adults at the initial intake session and at an established follow-up time to

measure improvement in clinical outcomes from treatment. Results are particularly

valuable because it is the most widely used instrument of its kind in Massachusetts,

and Riverside’s results can be compared to other similar programs as well as to our

own performance. Specific demographics of consumers can be tabulated to allow

comparison of similar populations as well as global comparisons. Our outcomes

measurements have consistently shown that consumers improve substantially in all

domains. One of our clinics was found to have the highest rate of improvement in

treatment of depression and was asked to present at a statewide conference on best

practices. We also have the highest rate of follow-up test administration in the State

and have again been asked to share best practices with other organizations. We believe this is a direct result of our commitment to outcomes measurements at all levels of the organization. In fiscal year 2005 Riverside will expand the use of this instrument to our psychiatric day treatment programs and institute the children and

adolescent TOP outcomes measurement in our clinics.

Performance measurements from Riverside’s three clubhouses demonstrate the

impressive success being achieved by them and by clubhouses in Massachusetts in

helping adults with mental illness find employment, despite locally high unemployment rates. For example, our program in Newton had 113 working members and

our program in Norwood had 74 working members in 2003 compared to a State average of 64 per program and a national average of 58. Both clubhouses are average

size programs. Additionally, Riverside club members had a job longevity of about

53 months in independent employment and 37 months in supported employment,

compared to the Statewide averages of 32 and 29 months respectively. They also

earned wages that were slightly higher than the Massachusetts average.

An example of a ‘‘home-grown’’ outcome measurement is the instrument used in

Riverside Lifeskills Program, a short-term adolescent day treatment program primarily serving youth referred by the Massachusetts Department of Mental Health.

The tool surveys participants’ perception of improvement on a number of functional

measures, such as ability to manage anger, get along with family, and communicate

feelings and concerns. Data is available for the previous 3 years and shows that

nearly 100 percent of the adolescents report improvement on all 13 functional domains.

The Massachusetts Department of Mental Health Performance Based Contracting

requirements designate specific measures for different service types. Adult residential programs report on the number of psychiatric and substance abuse hospital

days utilized, number of consumers who achieve a majority of their residential treatment plan goals, and number moving to lower intensity settings. Our results consistently meet or exceed contract requirements. While these results tell part of the

picture, the development of quality indicators is still in relatively early stages and

there is potential for identifying measurements that would further demonstrate the

success of these programs. This is especially important as Massachusetts continues

to move adults with mental illness out of State hospitals and into the community.

For example, a provider that accepts consumers at higher risk can be under-credited

for skill and capability when the measure solely considers the number of hospitalizations.

Our outpatient substance abuse intervention and outreach program, funded by

the Department of Public Health, gathers and reports extensive data to the State

on a monthly basis. These include many of the seven treatment domains currently

under consideration by SAMHSA such as arrests/incarcerations, substance use, and

living situations. Our reports also include such measures as number of participants

who completed treatment and who report abstinence at discharge. Our adolescent

substance abuse prevention program that uses environmental strategies to change

community attitudes to reduce youthful substance abuse also reports extensive information to the Department of Public Health. This program is measured by how

well it achieves agreed upon benchmarks for such outcomes as decrease in middle

school age youth using alcohol and increase in the number of protective factors identified by youth. This program converted to a new model during this past year and

results are not yet available.

COMMENTS ON THE PROPOSED PERFORMANCE AND OUTCOME MEASUREMENT PROGRAMS

From our experience with outcome measurements and our longstanding work as

a provider in Massachusetts, we have come to both respect the need for performance

and outcome studies and the need to proceed cautiously in their use. Applying our

experience to a review of the proposed measurements for mental health and subVerDate 11-MAY-2000 14:03 Apr 22, 2005 Jkt 000000 PO 00000 Frm 00050 Fmt 6633 Sfmt 6621 S:\DOCS\95101.TXT SLABOR3 PsN: SLABOR3
47

stance abuse funding we strongly support the movement toward performance measurement on a uniform, national basis but also offer several concerns for your consideration.

First, let me offer some local context. Providers in Massachusetts have been largely level funded in State mental health and substance abuse contracts for 14 years,

despite the fact that our costs have increased due to inflation and other factors. In

the past few years the economy in this State has been in critical condition, resulting

in cut backs to some State funding and services at the State Agency and provider

levels. At the same time, community-based providers have experienced mounting

regulations with associated mounting costs. We are also managing more challenging/high risk consumers in the community who cost more to serve as State institutions close or downsize, there are more rapid discharges from community hospitals

of under-stabilized patients due to managed care, and we are experiencing a shrinking workforce since we are unable to compete for employees as our salaries fall further behind other industries.

While many organizations in Massachusetts have closed or are in poor financial

condition, Riverside and a number of other providers have been able to grow

through mergers, find economies of scale, reduce administrative overhead, implement creative business practices and clinical strategies that identified new funding

sources. We have also worked to improve collection rates, worked to share resources

across programs, and developed other means to stay ahead of costs. However, even

strong providers such as Riverside are now coming to the end of our ability to continue to deliver high quality services without funding relief and the entire system

of care in Massachusetts is very fragile. Neither providers nor State Agencies can

afford to divert resources to the development of an infrastructure to support further

performance measurement programs. Therefore, any change in funding or in data

collection and reporting requirements must first ensure that it will not come at the

expense of services, staff time to serve consumers, or provider viability.

I respectfully submit the following recommendations:

• Resources directed to Performance Measurements should not be taken from existing funding for State Agencies or services. In Massachusetts, State Agencies have

already had major funding cuts and are already struggling to maintain their commitment to maintain core services in the community. Therefore, we would hope that

the investment in building Performance Partnerships would arise from new Federal

funds specifically for data management infrastructure development and maintenance, rather than eroding the base funding now in place, which could dramatically

hurt providers like Riverside. • No unfunded mandates should be passed onto providers. Providers do not have

the ability to self-fund the hardware, software, retooling or additional staff time that

would be required to implement further management information systems to collect

and report new data to the State. Nor can the consumers who depend on our services afford to give up staff support that is directly or indirectly diverted to data collection. In short, changes to Federal funding should incorporate requirements that

ensure funds are provided to support new mandates at the provider level without

reducing current rates or service levels.

• New mechanisms developed for Federal Block Grant funding should not delay

payments to the States. Such delays would ultimately result in uncertainty and or

delay in payment to providers, many of whom could not survive such a situation.

• While the proposed performance measurements appear to be both reasonable

and informative, the certainty that any measures in behavioral health are true and

meaningful indicators requires careful study over time. Until such full evaluation

can be achieved in the future and the validity of the measurements proven and

given the fact that many providers are already collecting valuable data, we suggest

that proposed national measurements be regarded as useful for informing further

queries rather than determinants of programs’ value and that modifications and refinements be made over time.

• Any move to determine State funding levels by demonstrated outcome improvements risks incorrectly penalizing or rewarding programs for outcomes beyond their

full control. Outcome measurements in mental health and substance abuse are still

in an early stage of development, with many questions yet to be answered about

which results directly correspond to treatment factors and which are influenced or

linked to outside, unrelated factors. For instance the success of any program, or

State, in reducing substance abuse in a population may be greatly influenced by the

local economy, availability of drugs, unemployment statistics, etc. as well as the effectiveness of programs being studied. Similarly, the success of a residential program in graduating consumers to more independent settings may depend on the

availability of affordable housing, the availability of outpatient and support services,

and consumers’ perceptions of opportunities to socialize with peers and avoid isolaVerDate 11-MAY-2000 14:03 Apr 22, 2005 Jkt 000000 PO 00000 Frm 00051 Fmt 6633 Sfmt 6621 S:\DOCS\95101.TXT SLABOR3 PsN: SLABOR3
48

tion after leaving a program. Therefore, basing Federal funding levels on outcomes

should not be implemented at least until sufficient measurement experience has allowed for proper weighting of these outside variables. Even then, it is debatable

whether reducing funding to under-performing States will help them improve programming or set them further behind. Performance measurements should support

quality improvement and assist in developing best practices, rather than create

variable and uncertain funding.

• The ongoing review of performance measurement programs, implementation,

practices, and applications should include ongoing feedback from all stakeholders,

including providers like Riverside and consumers of service.

CONCLUSION

As a community-based provider that works daily with thousands of vulnerable

consumers who depend on our services to avoid unnecessary institutionalization and

to recover from their mental health and substance abuse problems, we support

SAMHSA’s efforts to evaluate programs and promote quality practices across the

country. Our Nation needs to invest more in helping individuals and families struggling with behavioral healthcare challenges. Demonstrating the effectiveness of

services through outcome measurements can be an important step in increasing

public support for funds for behavioral healthcare programs. Defining best practices

and extending them to more people in need is a valuable aim, as is continuing support for the existing service system. Therefore, we would hope that current

SAMSHA funding would remain intact and new investment would be added to develop measurements, infrastructure, and dissemination of what is learned.

Thank you for your consideration of my testimony.

Senator DEWINE. Thank you very much.

Ms. Medalie has brought up an interesting point, and of course

that is the point that I brought up previously, and that is that we

all want facts. We all want to know what works, but no one wants

to pay for it. And I think that the point is well taken that, you

know, if we want this data, we ought to pay for it, but on the other

hand, I think we all have to understand that it all comes out of

the same pot anyway. So if the money was not going to be used

for the data, it could be used for treatment. It goes back to what

Dr. McLellan said, that your description of the treatment situation

in the country today was pretty grim.

So I guess I will start with you, Doctor. How do we get this balance of that data we want, the information we want, versus not

wasting any of that precious money that you describe correctly as

we do not have enough of for treatment?

Dr. MCLELLAN. If you go to your doctor, and you have, let us say,

hypertension, the first thing—actually, the second thing. The first

thing, of course, is the insurance, but the second thing they are

going to do is they are going to put a cuff around your arm and

they are going to measure your blood pressure. Now, is that an outcome measure that you ought to pay for or is that part of clinical

management? It is both, and that is what I am suggesting. For too

long, these systems have been thought to be separate. You need the

same kind of performance measures, patient status measures, to

direct and help a patient achieve self-sustaining care as the Senate

and the finance committees and the insurance companies and everybody else does. They are, in my view, one in the same.

The testimony of Riverside is really illustrative. So many agencies want so many different things for so many different reasons.

These programs are, you know, besieged by measurement and they

are in a desert in terms of actual functional information they can

really use.

49

So I do not think it is a difficult issue. I do not think it is a costly

issue. I think it is an issue of leadership and agreement on what

will be measured and when it will be reported.

Senator DEWINE. Of course in that case, I am not sure I totally

follow you, because in the case of the individual patient you are

tracking, where that patient is, and then how do you take that so

that we know what will work? That is one patient.

Dr. MCLELLAN. Absolutely.

Senator DEWINE. How do I know what is going to work then?

Dr. MCLELLAN. Right. The reason we know what a good blood

pressure is is because across all those individual patients, across

all those individual times they were monitored, you can see trends

in whether they do well or they do not, and they aggregate to a

group level, and you can divide it by age, race, gender, and other

conditions, all those kinds of things. It seems to be, based on research that has been done, that is the only viable system. If you

cannot get real information into the hands of, first, the patient

about his own condition and, second, the clinician that is actually

charged with treating that patient and, third, the evaluators that

need to report that information, it is not going to go. It will not be

self-sustaining.

Senator DEWINE. Does anybody else on the panel want to take

a shot at my question? What is the balance? What should you expect to pay? Maybe another way of putting it is what should you

expect to pay? You have got ‘‘X’’ number of dollars. What should

research cost you? I mean, maybe that is not a legitimate question.

I have seen research could cost you some inordinate amount of

money.

Doctor?

Dr. GOLDMAN. I was attempting to respond to your earlier question. When you asked what the amount ought to be, I became a little more timid. Let me take a shot at it. It is very difficult to assess

precisely what the right amount is, but everyone can agree what

the wrong among and the wrong strategy is, and the wrong strategy that some people on this panel have spoken about is the unnecessary inefficient redundancy of the collection of data that if it

were lined up or properly aligned would not lead to the need for

repetitive monitoring or different measures.

So I think much of what the Federal leadership, the stewardship

we spoke about, what is important and what has been successful

in recent years is overseeing a process of alignment between the individual person-level measures that a clinician would use and the

way Dr. McLellan has spoken about it to think about then aggregating those up to become measures of performance based on outcomes at the local level and have those measures be the same that

the State wishes to know about from its dependant counties and

the same set of relationships between the Federal Government and

its reporting needs be the same as the State to the county. Now,

if we can align those, I do not think that clinicians and directors

of programs will be as resistant to the collecting of these performance measures if they are clinically meaningful and useful for planning.

Now, with respect to your more difficult question, what is the

right amount, I have been working in evaluation for a long time.

50

Senator DEWINE. What is the wrong amount?

Dr. GOLDMAN. I do not even know if I can put the right dollar

amount on the wrong amount, but the point I wanted to make is

I have been working in evaluation for long enough to remember

when the Federal Government set aside 1 or 2 percent of direct resources for use in evaluation programs. That was done at the departmental level in health—well, it was done in Health, Education,

and Welfare—I am revealing my age, but more recently in the Department of Health and Human Services and all the way along the

line, whether it was the community mental health centers program

or other Federal programs, set aside resources at the 1 to 2 percent

level for the performance evaluation, and that could be used as a

benchmark now for the kind of resources we would need to build

this infrastructure.

Senator DEWINE. What about the situation where—and I will go

back again. There is nothing worse than a politician who goes back

and says, Well, why was such and such, you know, When I was a

Mayor, but I will do it. When I was Lieutenant Governor, one of

the things I was involved in is I had some jurisdiction over Mr.

Tester’s department and some other departments, and we were involved in drug treatment in our prisons, and Mr. Tester’s predecessor was an advocate for taking a program of drug treatment

that had already been tested and where she felt had been a model

program in other States. There was a set program, tested, and she

convinced me that it had been used before, model program, had

good test results, and we put it into a few of our prisons. We could

not afford to put it in too many, but we put it into a couple of our

prisons.

Now, assuming she was right, and I think she was, there would

be an example or would that be an example of a place where you

would not have to spend much on testing in the sense that you already had the data? Let us assume you already had the data of 10

years of testing. No? You still would have to do the testing?

Dr. MCLELLAN. With respect, I do not think you are getting it.

You do not want to tell people——

Senator DEWINE. That is not the first time.

Dr. MCLELLAN [CONTINUING]. With respect, you do not want to

tell people what to do. You tell them what you want. Now, as I understand it, what you want is people not going back to jail for drugrelated crimes, and that is what you pay for, and one time-tested

empirically validated procedure toward that goal might be a very

good way of getting that, but do not lose sight of what you want.

The fact that you put the miracle cure program into effect does not

necessarily mean that you are going to get your miracle cures. That

has happened over and over and over and over.

I commend to you the efforts of the State of Delaware.

Senator DEWINE. Well, I understand does not mean you are necessarily going to get it. Maybe you and I are not communicating.

The point is you have got to choose. You are running a prison. You

and I are running a prison. I do not want to belabor because we

will bore everybody else. You and I are running a prison, and we

got a whole bunch of people in there, and 70 percent of them have

got a drug addiction.

Dr. MCLELLAN. Got it. Right.

51

Senator DEWINE. What are you and I going to do? Well, we are

going to spend some money.

Dr. MCLELLAN. Yes. We are going to try something.

Senator DEWINE. We are going to try something. Well, we can

do the A, B, C, or D, and E is something that has worked, and A,

B, C, and D had never worked before because we never tried it before. We have got E. Why shouldn’t we try E that has worked before?

Dr. MCLELLAN. Now, you are right. You better try the thing that

has worked someplace.

Senator DEWINE. The problem with E is that it costs a little

more money than the others.

Dr. MCLELLAN. Well, that is a whole separate problem.

Senator DEWINE. Well, we decided to try E because we thought

it would probably work better.

Dr. MCLELLAN. So what you want to make sure, though, is that

it actually is giving you the outcomes that you want.

Senator DEWINE. You are not going to know that for 10 years because you are not going to know whether these people are recidivists. I am not going to know that for 10 years, because I am not

going to know if they come back. I understand that.

Dr. MCLELLAN. Well, about 50 percent of all recidivism occurs

within the first year.

Senator DEWINE. Well, I am going to know something in the first

year, but I am not going to know——

Dr. MCLELLAN. Okay.

Senator DEWINE [CONTINUING]. I can measure that. The point is

I have got to make a decision initially.

Dr. MCLELLAN. Right, and I would say, my own view is, that you

are using the right criteria to the make your decision. If somebody

else has shown it to be effective and it has been effective by the

standards that you are looking for in your own State and it has

been independently evaluated, that is your best guess.

I used in my testimony the word ‘‘earn’’, because I think with respect to the addiction treatment system, it ought to be given the

opportunity to earn additional revenue by defraying costs of re-arrest and re-incarceration and improved welfare status and things

like that and have some of the money, the savings that are measured, go back into the system that produces them. At this point,

that is not the case.

Mr. TESTER. Senator, I just wanted to add Dr. McLellan earlier

in his testimony talked about hypertension and he talked about

both the clinical management and the outcome piece, and I think

the dialog you are having now with Dr. McLellan around what we

would do in prisons is very much part of the process that Ohio has

looked to implement through what we call our outcome framework

initiative. It is both a quality improvement process and an outcome

process.

The quality improvement or in this case management piece is

clinical. There are certain junctures during a treatment process

and a prevention process where we know where we want our client

or consumer to be, and if we can tell at that juncture that they are

on target, then we can continue to move forward, and quite frankly, when we talk about how do we know what the right balance is

52

in terms of the money that we invest in this, I think the bottom

line—and I will just speak from my Ohio perspective. We are in the

process of using the Federal information that we have worked on

through the PPG process, and we are having dialogue with consumers, providers, and boards to talk exactly about what it is that

we can measure at the provider level with the consumer to make

sure that we are doing the right thing.

From there, if we have designed a system that meets the needs

of the consumer and takes into account what providers are in a position to be able to address comfortably, comfortably financially

without dedicating too many resources to the other side, then that

should give me the information I need from both the county board

perspective and a State perspective to understand what our system

is doing, and in order for me to make that work in Ohio, I have

to have my consumers, providers, and boards sitting with me in the

dialogue, and I think that is part of what you are hearing here. We

have had that dialogue with SAMHSA. We have had periods of

very good dialogue, and we just need to finish that so that we know

where we are headed, and then from there, I think we are in a position where, ideally, I would like to invest more prevention and

treatment and out of that have the provider determine what part

of that they need in order to make these measures work. That is

where we seem to be stubbing our toe, if you will.

Senator DEWINE. Well, how likely is it in all this discussion that

you are going to get that kind of either mandate or guidance from

the Federal Government, or are you better off just doing it yourself?

Mr. TESTER. In Ohio, Senator, we have concluded that right now

we are going to move ahead on our own, and we think the body

of literature is sufficient that we have a good understanding. We

think that the dialog with SAMHSA has given us a foundation, and

through NASADAD with the other States, we have a good feel for

where we ought to head, and then what I have told folks in the

State is when we get to the process where we know what the block

grant or the PPG is going to require, we will do our darnedest to

line up what Ohio has concluded; but quite frankly, because we are

committed to the process of clinical management and outcomes, we

need to put some things in place now.

It is not nearly as sophisticated as what we had envisioned. If

we were going to take that big first step at the right time, that is

where those infrastructure dollars do come into play.

Senator DEWINE. Now, I am saying this almost in jest, but, of

course, to the counties, you are the Federal Government.

Mr. TESTER. That is exactly right.

Senator DEWINE. You are sort of like the Federal Government is.

So they look at you and say, Oh, those guys up in Columbus, they

are making us do this, this, and this.

Mr. TESTER. You are absolutely correctly, and that, Senator, is

why I have those folks sitting at my table through the Governor’s

Advisory Council, and, quite frankly, having dialog with a diverse

group of providers, consumers, and boards is critical to my success,

and I think that is what you have heard us talk about this morning, and I think Administrator Curie talked about that too. It is

53

just that we encourage a more formalized process to make sure

that we have a clear perspective on where we are headed.

Senator DEWINE. Okay.

Ms. MEDALIE. I wonder if I might add something.

Senator DEWINE. Jump right in.

Ms. MEDALIE. I think from the provider perspective, it is important to note that we truly do value performance measurements and

even uniform performance measurements so that we can, first of

all, prove to the general public, if to nobody else, that what we do

really does work and that it really is worthy of being supported.

And we also are quite willing to collect data. The problem is that

much of the data that is even being looked at is already collected.

It is already in the medical record.

In our dreams, we have electronic medical records so once it is

in there, you do not have to spend additional resources to then aggregate it, but, frankly, we are a long way from that happening.

I see it, you know, maybe before I retire and maybe not. But it is

in there, and the things that we collect now are things that are

clinically meaningful to both the provider, to the clinician, and to

the consumer and most meaningful when it is shared, and we are

happy to share that if could have assistance in being able to do the

aggregating and the reporting. It is that intermediate step that is

really very, very difficult without the additional funds.

And, finally, also, for us as providers, the devil is always in the

details, what happens with the information. When we get information, when we provide information for performance measurements

that is then given back in a way that is clinically useful, that really

does help inform treatment, that is useful. We value it. Our clinicians value it. It is shared with the consumers and it has impact

on the programs. But when the information is either used in some

aggregate way that never translates back to something that is

clinically meaningful to help actual program choices or individual

clinical choices with consumers or, even worse, if the detail is let

out so that something—there is some gross measurement being

made, but it leaves out the details that would really say, Yes, it

looks this, but that is because of the special population and special

circumstances, you know, such as measuring hospital usage of people in residential programs coming out of State hospitals, a low

hospital readmission rate would seem to be good except for what

about when you it is a specialized program and you are taking

folks that are very high-risk people and you are really succeeding

in identifying when the risk starts to go up. You would think that

in year 1, you would probably see more hospitalization if your good

clinicians are recognizing this and intervening before there is a

safety breach, and maybe in years 2 and 3, you would see it go

down; but if the measures are not detailed enough, then it might

be measuring something, but it may not be measuring something

that is meaningful for the program that is being looked at.

Senator DEWINE. Okay. Listen, I appreciate your testimony. It

has been very, very helpful. I look forward to working with all of

you.

Thank you for coming in.

[Additional material follows.]

54

ADDITIONAL MATERIAL

PREPARED STATEMENT OF VICTOR A CAPOCCIA, PH.D.

Mr. Chairman and Members of the Committee, thank you for the opportunity to

present a written statement to the Committee on the topic, ‘‘measuring performance

and outcomes in addiction and mental health programs.’’ This statement will primarily discuss improving performance and outcomes in addiction treatment settings.

I believe however that the basic principles described in the statement are also applicable to treatment in mental health as well as prevention settings.

Up until recently, measuring performance and outcomes in addiction treatment

was often a function more informed by belief than by science, with little regard toward empirically validated standards of success. In consideration of the mismatch

between what works for treating addiction disorders and what is practiced, the Robert Wood Johnson Foundation embarked on a strategy to improve the quality of addiction disorder treatment by implementing programs that encourage the use of evidence-based approaches that can be measured by standardized definitions of success. Our plan to accomplish this objective involves several partnerships that include:

the Substance Abuse Mental Health Services Administration (SAMHSA), National

Quality Forum, State Mental Health and Addiction Authorities and Medicaid agencies, purchasers, and providers of addiction treatment services. There are three

basic strategies that we will follow: • We will work with Federal partners, researchers, providers, and purchasers (including States) through a consensus process guided by the National Quality Forum

to develop preliminary and simple measures that indicate the use of proven practices in treatment settings. For example are medications used in this setting? Are

patients admitted quickly after first contact? How long are patients retained in a

treatment or aftercare activity? • We will work to remove and minimize the policy and practical barriers that discourage the more than 14,000 publicly oriented treatment programs in this country

from using scientifically informed treatment approaches. For example, is the admission process organized to encourage same or next day appointments? Are levels of

care sufficiently linked to promote seamless transition by patients from more to less

intense interventions without re-admission delays?

• We will work with States to use the considerable purchasing and licensing authority that they have to encourage the use of treatment based on science not belief.

For example, a State might establish that 80 percent of calls for admission receive

appointments within 3 days. Such a standard would reduce no shows and take advantage of the specific window of opportunity presented by the call for help, and

quickly closed by the next neuro-biologically based need to continue using alcohol

and or drugs.

In partnership with the Center for Substance Abuse Treatment, one of our current initiatives to improve quality is the Network for the Improvement of Addiction

Treatment (NIATx). NIATx is supported by $9.5 million from the Robert Wood

Johnson Foundation’s Paths to Recovery program and $7.7 million from the Center

for Substance Abuse Treatment’s Strengthening Treatment Access and Retention

(STAR) program. NIATx is a vehicle for improving quality in the addiction treatment field that is equivalent to the role the Toyota Production System plays for the

Pittsburgh Regional Health Improvement Initiative or that the Institute for Health

Care Improvement plays for America’s acute health care services.

Research demonstrates that organizational factors are more significant barriers to

admitting and retaining patients into treatment than are personal or policy-related

factors. Therefore, the overall goal of NIATx is to make improvement of organizational functioning an integral part of the work of addiction treatment agencies. The

specific aims of the NIATx are to:

• reduce the time between a client’s first request for service and their first treatment session;

• reduce the percentage of client no-shows;

• increase admissions; and

• increase the treatment continuation rate.

These four aims translate into measures of performance improvement and are

consistent with the measures developed by the SAMHSA-sponsored Washington Circle Group.

How does it actually work? The National Program Office (NPO) at the University

of Wisconsin provides 29 grantee agencies with an expert process improvement

coach and resources for building their organization’s capacity to apply, spread, and

sustain successful changes within their organization. Within each organization,

55

1Gustafson D, and Hundt A (1995). ‘‘Findings of Innovation Research Applied to Quality Management Principles for Health Care.’’ Health Care Management Review 20(2), pp 10–27. 2Gustafson DH (2002). ‘‘Designing Systems to Improve Addiction Treatment: The Foundation.’’ Alcoholism and Drug Abuse Weekly. 14(42).

there must be a committed executive sponsor, a powerful change leader and a dynamic change team using an improvement model that allows for changes to be rapidly tested and implemented. The improvement model is based on five key principles

drawn from extensive empirical research that separate successful from unsuccessful

organizations: (1) thoroughly understand what it is like to be a customer/user of the

process you are trying to improve; (2) select processes to improve that, if successful,

will help senior leaders achieve important overarching goals; (3) have only powerful

and respected change agents; (4) engage external expertise to provide ideas and

pressure to improve; and (5) quickly and repetitively test and (based on those tests)

revise solutions before full-scale implementation.1 2

In 8 months of NIATx participation, these 29 agencies have made impressive improvements in treatment access and retention, developed ideas and tools to share

with the rest of the field, and begun to create the groundwork needed for fundamental change in their agencies. For example, a subset of participating programs,

through using rapid change cycles, have reduced wait times to get into treatment

by 68 percent, reduced the number of no-shows for treatment by 29 percent, increased admissions by 64 percent for inpatient and 142 percent for outpatient treatment settings, and increased treatment continuation by 7 to 17 percent depending

on the level of care. Programs are demonstrating that dramatic change may be a

lot simpler and take less time than is often presumed. They are proving that, when

faced with seemingly insurmountable hurdles, addiction treatment providers find innovative ways of getting more from existing resources.

Behind the numbers are a variety of specific changes that began after members

conducted a walk-through of their own agency where they experienced the barriers

to treatment faced by their clients. The barriers identified by the applicant organizations led to the categorization of nine main areas in need of systemic improvements: (1) outreach; (2) first request for service; (3) intake and assessment; (4)

therapeutic engagement; (5) levels of care; (6) paperwork; (7) scheduling; (8) social

support systems; and (9) maximizing revenue sources. Examples of changes in these

areas include: central admission centers; guaranteed next day appointments; expanded evening, weekend and morning hours; reduced barrier transition between

levels of care; elimination of ‘‘prove you are ready’’ requirements; and targeted reminder and follow up contacts.

Attached to this statement you will find a document that summarize specific accomplishments of the 29 agencies.

We welcome your questions and interest in this work. Thank you for this opportunity.

56

57

58

59

60

61

62

63

64

65

66

67

68

69

PREPARED STATEMENT OF ERIC GOPLERUD, PH.D.

While SAMHSA is just beginning to implement performance measurements and

outcome measurements to ensure accountability, the private sector has been utilizing performance measurements in behavioral health for several years. The model

provides accountability at the plan level and an Ensuring Solutions to Alcohol Problems analysis has shown how attention to a particular measure can ensure quality

improvements. Ensuring Solutions is a research-based initiative that examines barriers to access to alcohol treatment.

PERFORMANCE MEASUREMENT IS A FIRST STEP IN QUALITY IMPROVEMENT

Addiction specialists have made tremendous progress in performance measurement. In just 5 years, they have developed a core set of measures and incorporated

several into tools already familiar to health care purchasers. The inclusion of these

measures alongside those for treating other chronic illnesses—asthma, diabetes and

high blood pressure—gives addiction to alcohol and other drugs a place on the Nation’s health care agenda that is commensurate with its devastating impact on individuals, families and communities.

Improving the quality of alcohol treatment serves everyone’s interests. Alcohol

problems are the third leading cause of preventable death, killing 100,000 Americans annually. They drain $185 billion from the Nation’s economy by reducing productivity and increasing health care costs. Despite these enormous costs, however,

the quality of treatment for alcoholism ranks dead last when compared to treatment

for the Nation’s 25 leading causes of illness, death, hospitalization and doctor’s visits. In fact, RAND researchers have found that only 10 percent of Americans with

alcoholism receive evidence-based care.

PRIVATE SECTOR TAKES PERFORMANCE MEASURE INITIATIVE

The National Committee for Quality Assurance (NCQA), a nonprofit accreditor for

managed care organizations, developed and maintains a leading tool to measure

health care value and improve quality—the Health Plan Employer Data and Information Set (HEDIS). Almost 90 percent of America’s health plans now use HEDIS

to measure performance on important dimensions of care and service for many different health conditions, making it possible to compare the performance of health

care providers in both the private and public sectors on an ‘‘apples-to-apples’’ basis.

70

Public reporting on performance by NCQA and other entities has improved the

delivery of care for a variety of health conditions. Holding health care providers accountable for their treatment of patients with hypertension, for example, has helped

increase blood pressure control efforts substantially over the past 3 years. On average, private health plans in 1999 helped just 39 percent of their patients who had

been diagnosed with hypertension keep their blood pressure within limits specified

by a performance measure; by 2002 that average increased to 58 percent. The best

performing health plans assisted 68 percent of their hypertensive patients in controlling their blood pressure. While there still is room for significant quality improvement, if every health plan performed at least this well, researchers estimate

that 28,000 lives would be saved and 50,000 fewer Americans would suffer from

strokes.

A MILESTONE IN MONITORING CARE

NCQA’s announcement that it will begin to measure performance in treatment for

alcohol problems has heightened expectations for quality improvement in addiction

treatment. These measures, developed with the Washington Circle, a group focused

on performance measurement in addiction treatment, mark a milestone: health

plans will be asked for the first time to account for their success at both initiating

and engaging treatment for alcohol problems once they have been identified.

Public reporting of performance measurement is key. NCQA, for example, publishes an annual report on the State of health care in America and provides tools

for purchasers and consumers to evaluate health care. Public reporting increases the

pressure on health care providers to perform at least as well as their competitors

or risk losing market share. This pressure can lead to quality improvement by encouraging heath care providers to identify problem areas and take the necessary administrative or clinical actions to fix them.

Performance measurement also increases purchasers’ leverage in negotiating

health care contracts. In 1996 the largest business coalition in the Nation, the Pacific Business Group on Health, negotiated a contract with 13 of California’s largest

health plans that put $8 million in premium income at risk if the plans didn’t meet

specific performance measure targets. Poor performance in childhood immunization

resulted in a $2 million refund for the employers on whose behalf the coalition had

been negotiating. A financial penalty of this kind provides the strongest possible incentive for a health plan to improve performance and enables employers to get maximum value from their health care investment. Within a year, all of the health

plans had brought the quality of care up to the business group’s standard, demonstrating the power of objectively measured performance tied to financial incentives.

A NATIONAL BUSINESS INITIATIVE

Performance measurement for addiction treatment also has begun to take root in

other areas of the private sector, including the National Business Coalition on

Health (NBCH). Through its membership of 90 State and regional coalitions, NBCH

represents more than 7,000 employers—including several of the Nation’s largest—

who provide insurance for an estimated 34 million workers and their families. Since

1999, NBCH has offered these and other interested groups a Web-based tool called

eValue8 that enables them to conduct a uniform, annual assessment of the quality

of care for a wide range of health conditions. Independent analysis of the results

permits comparison of health plan performance on a local, regional and national

basis.

In 2003 eValue8 featured, for the first time, an alcohol module that includes several of the Washington Circle performance measures. This development is significant for two important reasons:

1. Health plans are likely to respond to eValue8 because purchasers use the tool

to assess health plan quality directly. For example, when the Pacific Business Group

on Health (which uses eValue8 and is the largest business health coalition in the

country), requests that a plan complete eValue8, the health plan has a strong incentive to comply because of the coalition’s enormous purchasing power.

2. About half of the participating employers offer financial rewards to high performing plans or provide employees with financial incentives to choose these plans

for their health care needs. Employers can use these rewards and incentives to drive

quality improvement in addiction treatment.

A CRITICAL FIRST STEP

Performance measurement can improve the quality of addiction treatment but it

will lead to positive change only if everyone with a stake in health care actively

71

looks for ways to accomplish this. The development of a core set of performance

measures for addiction treatment is a critical first step. Now that stakeholders at

every level of health care delivery have real tools at their disposal, quality improvement in alcohol treatment is moving from theory into practice. It is essential that

the Federal Government through the Substance Abuse and Mental Health Services

Administration (SAMHSA) be actively engaged with both the public and private sectors to support publicly reported common measures of quality care.

Working with policymakers, employers and concerned citizens, Ensuring Solutions

provides research-based information and tools to help curb the avoidable health care

and other costs associated with alcohol use and improve access to treatment for

Americans who need it. The project is supported by a grant from The Pew Charitable

Trusts.

[Whereupon, 11:20 a.m., the subcommittee was adjourned.]

Æ