THE 
UTILIZATION AND 
QUALITY

CONTROL 
PEER REVIEW

ORGANIZATION (PRO) 
PROGRAM

REVIEW ACTIVITIES 
QUALJTY 
'1 
,.1' SuvrC"I"
l. 
OFFICE OF 
JNSPECTOR GENERAL 
OFFCE OF 
ANALYSIS AND INSPECIONS 
AUGUST 198.  THE UTILIZATION AND QUALITY CONTROL

PEER REVIEW ORGANIZATION (PRO) PROGRA: 
QUALITY REVIEW ACTIVITIES

RICHAD P. 
KUSSEROW 
INSPECTOR GENERAL

AUGUST 1988 
CONTROL # OAI-Ol-88-00570
 . .
. .. .. . 
TABLE OF CONTENTS

EXECUTIVE SUMRY 
INTRODUCTION 
BACKGROUND OF THE PRO PROGRA 
Creation of the PRO Proqram

First Scope of Work

Second Scope of Work

COBRA and OBRA Provisions

Visibility and Vulnerabilitv of the PRO Proqram

Administration and Oversiqht of the PRO Proqram

The PROs' Ouali ty Review and Intervention Procedures

The Sanction Process

FINDINGS. . 

Respondents considered quality review to be the most

cri 
tical element of the PROs mission and thouqht that 
it has received increased focus durinq the second

contract period 
(1986-1988) 
Generic quality screens appear to be useful instruments

for quali tv review. but may be in need of refinement

The PROs I quali ty review efforts are limited by a lack 
of consensus reqardinq the definition of quality medical

care. by the amount of resources available for such care. 
and by the current lack of sophisticated technoloqy to

assess quali tv 
The PROs have qreat variations in their approaches to

and outcomes of quality review

I impact on ensurinq hiqh quality care would be

The PROs 
enhanced by qreater coordination with other health care

enti ties. particularlY with the State medical licensure

0 . 

boards. 
Most PROs are findinq it difficult to provide effective

outreach to Medicare beneficiaries. .

The PROs will face serious challenqes and obstacles in

reviewinq care in nonhospi tal settinqs

RECOMMENDATIONS  . . . .
APPENDIX I
ENDNOTES 
APPENDIX II
BACKGROUND OF THE PRO INSPECTION. 
APPENDIX III
METHODOLOGICAL NOTES. 
APPENDIX IV
SUMY 
OF PRO-RELATED STUDIES. 
APPENDIX V
THE PROS WITH MORE THA ONE CONTRACT. 
APPENDIX VI
COMPARISON OF 1984 SCOPE OF WORK TO 1986
SCOPE OF WORK

APPENDIX VII
SANCTION PROCEDURES FOR CASES OF GROSS AND FLAGRANT

AND SUBSTANTIAL NUBER OF SUBSTANTIAL VIOLATIONS 
APPENDIX VIII
SUMY 
OF RECENT PRO-RELATED 
LEGISLATIVE PROVISIONS

APPENDIX IX
MEMORABLE "PRO-ISMS"  8 .

APPENDIX X
TABLE OF GENERIC QUALITY SCREENS: PERCENT OF FAILURES 
AND CONFIRMED PROBLEMS (1986-1987) BY SCREEN AND

CASE STUDY SITE 
APPENDIX XI
HCFA. ASPE. AAP AND AHA COMMNTS ON THE DRAFT REPORT 
AND OIG RESPONSES TO THE COMMENTS. . 
 EXECUTIVE SUMRY 
PURPOSE AND OBJECTIVES: 
The purpose of this inspection was to assess the performance of

the Utilization and Quality Control Peer Review Organization

(PRO) program and to promote a better understanding of the PROs

mission and activities. Because of the major importance of the

PROs' quality assuranc  activities to the integrity of the

Medicare program, the inspection has paid particular attention to

ROs identify and address quality of

tchea prroceess esi sby swhuiche thse 
  activities. 
, including their education and sanction 
BACKGROUND: 
The inspection grew out of the Inspector General' 
s desire to

obtain a broad perspective on the PROs' performance during their

second contract period (1986-1988). To that end, we pursued

inquiry: 
three primary lines of  1) interviews with 211 in
dividuals associated with the PRO program, including all the PRO

chief executive officers (CEOs) and representatives of other

Government, provider, and consumer groups associated with the

PROs, 2) site visits to 12 PROs selected for case study, and 

nd data bases.

review of pertinent literature 

FINDINGS: 
RESPONDENTS CONSIDERED QUALITY REVIEW TO BE THE MOST

CRITICAL ELEMENT OF THE PROS' MISSION AND THOUGHT IT HAS

RECEIVED INCREASED EMPHASIS DURING THE SECOND CONTRACT

PERIOD (1986-1988). 
GENERIC QUALITY SCREENS APPEAR TO BE USEFUL 
INSTRUMNTS FOR 
QUALITY REVIEW , BUT MAY BE IN NEED OF REFINEMENT. Screens 
provide the PRO program with national quality review

procedures, but they have produced excessive referrals to

physician advisers and may be inconsistently applied. 
THE PROS' QUALITY REVIEW EFFORTS AR LIMITED BY A LACK OF 
CONSENSUS REGARDING THE DEFINITION OF QUALITY MEDICAL CAR,
BY THE AMOUNT' OF . RESOURCES AVAILABLE FOR SUCH CAR, AND BY
CUNT 
THE  LACK OF SOPHISTI.CATED TECHNOLOGY TO ASSESS 
QUALITY. 
THE PROS EXHIBIT GREAT VARIATION IN THEIR APPROACHES TO AND

OUTCOMES OF QUADITY REVIEW. Some PROs identify many quality
of care problems' using the generic quality screens while 
others identify !few, if any problems. The quality review

processes among the 12 case study PROs showed variations in

both the identification and the treatment of physicians and

providers with quality of care problems.  I IMPACT ON ENSURING HIGH-QUALITY CAR 
THE PROS  WOULD BE 
ENHACED BY GREATER COORDINATION WITH OTHER HEALTH CAR 
ENTITIES, PARTICULALY WITH THE STATE MEDICAL LICENSUR 
BOARDS. The sharing of important information has been 
limited by the PROs' confusion about their confidentiality

restrictions under the Social Security Act and HCFA '

failure to require PROs to report physician misconduct or

incompetence to State Medical Boards. 
AR 
MOST PROS  FINDING IT DIFFICULT TO PROVIDE EFFECTIVE 
OUTREACH TO MEDICARE BENEFICIARIES. Al though most PROs 
viewed themselves as reasonably effective in this area, 
other national and local representatives were generally

unaware of the PROs' outreach activities.

THE PROS WILL FACE SERIOUS CHALLENGES AND OBSTACLES IN

REVIEWING CAR IN NONHOSPITAL SETTINGS. In addition to the

normal learning curve, respondents anticipated that the PROs

will face problems concerning adequate funding, guidance,

data sources 1 review technology, coordination with skilled

nursing facilitie' 
(SNFs) and home health agencies (HHs),
as well as resistance from the medical community. 
RECOMMNDATIONS 
QUALITY OF CARE

THE HCFA SHOULD:

WORK WITH THE PROS TO DEVELOP MORE CONSISTENT APPROACHES TO

CA 
DEFINING AND ADDRESS ING QUALITY OF 
PROBLEMS 
WORK WITH THE PROS TO ENSUR MORE CONSISTENT REPORTING OF 
QUALITY INTERVENTION ACTIVITIES;

DEVELOP AND WIDELY DISSEMINATE INFORMTION ON THE FULL RAGE 
OF EDUCATIONAL CORRECTIVE ACTIONS BEING TAKN BY THE PROS TO 
ADDRESS QUALITY OF CAR PROBLEMS; 
SHA 
CREATE MORE OPPORTUNITIES FOR THE PROS TO  INFORMTION 
WITH EACH OTHER CONCERNING THEIR QUALITY REVIEW AND

INTERVENTION APPROACHES; and

INTENSIFY ITS SUPPORT OF RESEARCH TO DEFINE AND ASSESS THE

QUALITY OF MEDICAL CAR. IN ADDITION, HCFA SHOULD COLLECT

AND INTEGRATE QUALITY-RELATED RESEARCH FINDINGS FROM ALL

RELEVANT SOURCES AND DISSEMINATE THEM TO THE PROS.  GENERIC OUALITY SCREENS

THE HCFA SHOULD:

ENSUR THAT ANY MAOR MODIFICATIONS TO THE GENERIC QUALITY 
SCREENS ARE PILOT-TESTED BEFORE NATIONAL 
IMPLEMENTATION; 
PROMOTE GREATER CONSISTENCY IN THE APPLICATION OF GENERIC

SCREENS THROUGH ITS TRAINING EFFORTS AND OPERATIONAL

INSTRUCTIONS; AND

SUPPORT THE DEVELOPMENT AND PILOT TESTING OF MORE EFFECTIVE

AND LESS LABOR-INTENSIVE REVIEW METHODOLOGIES.

PROS I COORDINATION WITH OTHER ENTITIES:

THE HCFA SHOULD PROMOTE CLOSER WORKING RELATIONSHIPS 
BETWEN 
THE PROS AND OTHER HEALTH CAR ENTITIES AND ENCOURAGE MORE

SHANG OF INFORMTION ABOUT PHYSICIANS AND PROVIDERS WITH 
QUALTY OF CAR PROBLEMS. THESE EFFORTS SHOULD INCLUDE
CLAFICATION BY HCFA OF THE PROS I CONFIDENTIALITY 
RESTRICTIONS AND A REGULATORY REQUIREMENT THT 
PROS REPORT 
INSTANCES OF PHYSICIAN MISCONDUCT OR INCOMPETENCE TO STATE

MEDICAL LICENSUR BOARDS. THIS REPORTING REQUIREMENT WAS 
AGREED TO BY HCFA IN MACH 1986 IN A BRIEFING ON THE OIG 
REPORT, "MEDICAL LICENSURE AND DISCIPLINE: AN OVERVIEW.

THIS SHOULD BE ACCOMPLISHED THROUGH THE NOTICE OF PROPOSED

RULEKING 
(NPRM) ON CHAGES TO PEER REVIEW ORGANIZATION 
REGULTIONS. 
PROS' OUTREACH , TO MEDICARE BENEFICIARIES 
THE HCFA SHOULD SEEK WAYS TO STRENGTHEN THE BENEFICIARY

OUTRECH EFFORTS OF THE PROS. IN 
THIS CONTEXT / IT SHOULD
COORDINATE MORE CLOSELY WITH THE HSOCUIAL  SECUITY 
ADMINISTRATION AND THE OFFICE OF 
DEVELOPMENT SERVICES 
AND SHOULD PROVIDE MORE OPPORTUITIES FOR THE PROS TO SHAE 
BEST PRACTICE APPROACHES WITH EACH 
OTHER. 
REVIEW OF CARE IN NONHOSPITAL SETTINGS

THE HCFA SHOULD:

WORK WITH THE PROS TO DELINEATE THE MOST SIGNIFICANT

CONSTRAINTS THAT WILL BE ' FACED IN REVIEWING. CAR IN 
NONHOSPITAL SETTINGS AND TO DEVELOP STRATEGIES FOR

ADDRESSING THESE CONSTRAINTS; AND 
SEEK WAYS TO IMPROVE THE AVAILABILITY OF DATA ON OUTPATIENT

AND INPATIENT SERVICES PROVIDED TO THE MEDICAR POPULATION. 
IN THIS CONTEXT, IT SHOULD BUILD UPON DEMONSTRATION PROJECTS

CURRNTLY UNDERWAY IN TEXAS AND MAYLAD TO LINK THE
MEDICAR PART A (INPATIENT) AND PART B (OUTPATIENT) DATA
FILES. 
iii
 COMMNTS ON DRAFT REPORT 
We received written comments from the Health Care Financing

Administration, the Assistant Secretary for Planning and

Evaluation, the American Association of Retired Persons and the

American Hospital Association. All four entities reflected

positive reactions to our recommendations but AH raised some 
important methodological considerations which led us to modify

two of our findings. In addition, HCFA and ASPE highlighted

several steps that HCfA is planning to undertake in the third

scope of work to address several of the concerns raised in our

report. 
See appendix XI for a complete summary of the 
aforementioned comments , and our response to them.  INTRODUCTION

The Office of Inspector General (OIG) has recently completed an

inspection of the utilization and Quality Control Peer Review

Organization 
(PRO) program. The primary purpose of this broad-
based study was to assess PRO performance and to promote a better

understanding of the PROs' mission and activities. To that end,

factors: 
the study focused on the following 
o the implications of the changes in the PROs' scope of work

from the first to second contract period:

o the maj or differences in perception among the PROs and

other entities (e. g. ,  heal th, providers, consumers,
public ,
Government officials,  interest advocates) regarding
1 mission 
the PROs  and  performance: 
o the significant variations that exist among PROs in

carrying out their scope of work responsibilities:

o the PRO practices that appeared to be exemplary: and

o the potential weaknesses or vulnerabilities of the

program 
(See appendix II for a more detailed explanation of the back

ground for this inspection.

In the course of this OIG inspection of PRO performance, we

conducted in-depth interviews with a wide range of individuals

associated with the PRO program, including all PRO chief execu

tive officers (CEOs), and a sample of other PRO staff, as well as

visited 
national and local external entities. We  12 of the 44 
jurisdictions:
PROs who are conducting reviews in the 54 PRO 
California, Colorado, Florida , Georgia, Indiana, Iowa,
Massachusetts 
, New York, Oregon, Rhode Island, Texas, and West
Virginia (for 
the Delaware PRO area). We created a computer file
interviews. 
to store and sort information gathered from our 
addi tion to this primary data, we collected and analyzed PRO-
entities. 
related performance data from HCFA and other  (See
appendix III for a more detailed' description of our methodology. 
Because the PROs' quality assurance activities are of major

integri.ty of the 
importance to the  Medicare program, this 
inspection has paid particular attention to the processes by

which the PROs identify and address quality of care issues. 
This report summarizes the OIG' s inspection findings related to

the PROs' quality review activities. It is the first in a series

t S 
of reports on the OIG  inspecton of the PRO program. The second 
report will focus on the PROs' sanction activities and the third

and final report will offer an explanation of the overall
 program. 
effectiveness of the PRO  That report will focus on an 
assessment of both the PROs themselves and HCFA' s oversight of
them. It is important to note that our assessment of the PRO

program is based primarily on interviews with a wide variety of

well-informed individuals closely associated ' 
with the PROs and 
HCFA. ' Wherever possible , we have supplemented our analyses of 
the qualitative data with available quantitative data. 
BACKGROUND OF THE PRO PROGRA 
Creation of the PRO Proqram

The Utilization and Quality Control Peer Review Organization

(PRO) program was created by the Peer Review Improvement Act 

1982, Title I, Subtitle C of the Tax Equity and Fiscal Respon

sibility Act of 1982, Public Law 97-248.

Peer Review Organizations succeeded Professional Standards Review

Organizations 
(PSROs) in the provision of Medicare peer review. 
Part B of 
The PSRO program had been established by Congress 
 (in 
Ti tIe XI of the Social Security Act) in 1972 to ensure that 
heal th care services provided under the Medicare , the Medicaid

and the Maternal and Child Health and Crippled Children'
programs were "medically necessary, conformed to appropriate
professional 
standards, and were delivered in the most efficient 
and economical manner possible. ,,1 The PSRO program was a 
response to increasing Medicare and Medicaid costs and the

failure of existing utilization and claims review mechanisms to

deal with widespread inappropriate usage of health care services. 
The congressional rational,e ,fo2r rep lacing the PSRO program with

the PRO concept was based on the fact that the PSRO program had

"been faced with certain structural problems: overregulation and

laws 
too detailed specification in  (had) restricted innovation in
new approaches to review. 
The PRO legislation emphasized ,greater accountability by requir
ing PROs to have performance-based contracts with specific

measurable obj ecti ves. 
The PRO legislation shortly preceded the 
Prospective Payment System (PPS) legislation and the PROs were

expected to address concerns about the potential negative

incentives of the PPS for increases in hospitalizations and

reductions in the quality of care provided to Medicare benefi

ciaries. 
Compared with former cost-reimbursement systems , PPS 
gave hospitals much stronger incentives to increase Medicare

numer of 
payments by increasing their  admissions and to reduce 
earlier. 
costs by limiting services or discharging patients 
Hence, the PROs were charged w th monitoring the system to

protect against potential provider abuses. such as unnecessary
admissions, substandard care, and premature discharge.  The Secretary of the Department of Health and Human Services

(HHS) was mandated legislatively to enter into PRO contracts with

physician-sponsored" or "
physician-access" organizations. 
physician-sponsored organization is composed of a substantial

number of physicians in the review area and is representative of
those physicians. A physician-access organization has an

adequate number of available physicians practicing medicine or

surgery in the review area. 
First Scope of Work (1984-86) 
The PRO program was implemented i  54 States and territories

through 2-year, fixed-price contracts with " 
peer review
organizations. " Each of the 50 States , the District of Columia,
Puerto Rico , and the Y:irgin Islands was designated as a separate 
PRO area. Guam, American Samoa, the Northern Mariana Islands, and

the Trust Territory of the Pacific Islands were designated a

area. 
single PRO 
The first contracts, which became effective over a 5-month period

from July to November 1984, emphasized detection of inappropriate

system. 
utilization and paYments under the new PPS  To that end,
contract goals included reducing unnecessary admissions
, ensuring

that paYment rates matched the diagnostic and procedural

information contained in patient records, and reviewing patients

transferred or readmitted within 7 days of discharge to determine

whether readmission was for the same condition as the first

hospital visit. In addition, each PRO contract included at least

ectives: 
reducing unnecessary readmissions because of
five obj 
substandard care during the prior admission, ensuring the
provision of medica,l services critical to avoidance of

unnecessary patient complications, reducing unnecessary surgery

or other invasive procedures, reducing the risk of mortality and

reducing avoidable postoperative or other complications. The

PROs were also expected to develop and analyze Medicare patient

data to identify instances and patterns of poor quality. 
When the PROs identified problems with given physicians or

hospi tals, they were expected to address those problems through

education and consultation, intensified review, or denial of
paYment for care that was not reasonable or was provided in . 
inappropriate setting. The PROs were also authorized to recom

mend the sanction of physicians or  providers in cases of a

substantial violation" in a " substantial numer of 
cases" or a 
case. 
gross and flagrant" violation even :in a single  Such cases 
were referred to the Inspector General' s Office for review and

sanction determination. 
(A further discussion of the sanction 
process appears later in this introduction.
 Second Scope of Work (1986-88) 
During the first contract period, several entities, including the

General Accounting Office, the Inspector General of the Depart

ment of Health and Human Services, and the Rand Corporation,

studied the PROs ' performance and recommended that their quality
review be strengthened.  sumary of 
(See appendix IV for a  PRO-
related studies.  In response to these findings and general

pressure from 
w.i thin and outside of the Department, HCFA 
strengthened the quality review requirements in the second PRO

1986. In 
contracts which began in July  those new contracts, the
44 PROs that were responsible for review in the 54 PRO areas (see
appendix V for a summary of PROs with more than one 
contract) had

the following requirements:

review of readmissions to the same hospi tal within 15 days; 
review of a sample of discharges to assess whether there was

evidence of premature discharge or transfer;

review of hospitals . with unexplained statistical outliers in
the PRO data on high mortality rates or utilization

patterns; 
application of a standard set of quality-related criteria

(called generic quality 
screens) to all cases selected for
PRO review. These six generic quality screens included: 
adequacy of discharge planning, medical stability of patient

at discharge, deaths, nosocomial infection, unscheduled

return to surgery, and trauma suffered in the hospital;

special review of s ort hospital stays; and

development and implementation of community outreach

programs. 
Thus, the second scope of work intensified the PROs I review

requirements. In 
screens, all
addition to generic quality 
records selected for retrospective review for any reason were

also subj 
ected to admiss ion review, DRG validation, coverage
review, ' and discharge review. 
The Senate Finance Committee staff in a background paper,
Quality and Access to Health Care Under Medicare I s Prospective
Payment System, " noted that , "these changes in the PRO review
effort were designed to increase detection of premature

discharges; to improve review of care in the hospital,
particularly the detection of situations where under-service may

impact the quality of patient cases; and to improve the patients'
understanding regarding their rights and appeals under the

system. "  sumary of the 
(See appendix VI for a  differences between the 
first and second scopes of work.

COBRA and OBRA Provisions:

I responsibilities were substantially increased through 
The PROs 
provisions of the Consolidated Omnibus Reconciliation Act of 1985

(COBRA) and the Omnibus Reconciliation Act of 1986 (OBRA). The

COBRA legislation required the implementation of preadmission

review for 10 surgical procedures and preprocedure review of any

cases involving assistants at cataract surgery. It also gave

PROs the authority to deny payment for quality of care concerns. 
The OBRA legislation extended the PROs ' review from only

inpatient hospital settings to ambulatory and posthospital

settings. 
Over the n xt several years, the PROs will be expected 
to review care delivered in hospital outpatient departments, 
ambulatory surgical centers, skilled nursing facilities, home

' offices. 
health settings, and doctors  In addition, in certain 
States, PROs are performing quality review of health maintenance

organizations  (CMPs). 
(HMOs) and competitive medical plans  (See 
appendix VIII for a summary of COBRA and OBRA provisions.

Visibili ty and Vulnerability of the PRO Proqram

As reflected in the legislative history, the scope of the PRO

inception. 
program significantly expanded after its  That 
expansion has been accompanied by extensive scrutiny from many

oversight entities within Government and from provider and

consumer groups outside Government. To date, Congress has held

eight hearings related to the PRO program and numerous research

evaluations. 
and oversight entities have conducted PRO-related 
(See appendix IV. 
The complex identity and inherent vulnerability of the PROs was

sumed 
up by one PRO spokesman: 
"It is clear from my vantage point that PROs are

quickly becoming all things to all people... The

Inspector General of the Department of Health and Human

Services is searching vigorously for a policeman of the

marketplace. The Executive Office of ,Management and
Budget is' looking hard  services, 
for cost containment 
particularly to hold the line on Medicare admissions. 
The Medicare beneficiary community earnestly desires a

protector of quality as the incentives of diagnosis

related group (DRG) payment and capitated arrangements

invi te under-service. 
Heal th care consumers seek ready 
access to the information that review activities can

generate. How else will a competitive market place

work? Local practicing physicians remain wedded to a

responsibility to monitor and evaluate their own

practice behavior. All this and more for one-fifth of
 1 percent of the Medicare hospital trust fund 
(the PRO

budget for a single year) . .. Can there be any doubt

that PROs will surely fail on 
someone' s scorecard?,,4

Administration and Oversiqht of the PRO Proqram

The Health Care Financing Administration 
(HCFA) is responsible

for administering and overseeing the PRO program through its

Office of Medical Review in the Health Standards and Quality

Bureau (HSQB). 
These functions are shared by central and
regional office staff. 
' The former group is responsible for
establishing the operational and evaluation policies and

mechanisms for the program and for negotiating the PRO contracts.
The latter group is responsible for implementing program

requirements and providing regular oversight and technical

assistance to the PROs in their respective regions. 
Since the inception of the PRO 
program, both HHS and HCFA

changed. In 
leadership have  response to substantial concerns
raised about their predecessors' management of the program, the
Secretary of HHS and the Administrator of HCFA have met periodi

cally since 1986 with physician, 
hospital, consumer, and PRO

representatives to hear their concerns about and suggestions for

improving the PRO program. 
In response to those meetings
, a "PRO

action plan" has been developed to improve both 
HCFA' s management
of the program and the PROs' performance and effectiveness. The

action plan has served as a resource for HCFA in 
its ongoing

efforts to strengthen the PRO program. 
The , HCFA has faced numerous challenges in overseeing the PRO
program. 
Like the PROs , HCFA has responded to competing expecta
tions from wi thin and outside the Government. 
For instance, HCFA

has juggled pressures to make the PROs accountable for quantifi

able outputs with those to give the PROs the, proper flexibility
to carry out their mission in an efficient and effective 
manner.
In addition, HCfA has had to balance the expectation that

additional PRO provisions 
(such as COBRA and OBRA) would be

implemented quickly with the pressures to follow formal mechan

isms. All 
the while allowances had to be made for the limi 
ta
tions of available quality review technology. The HCFA has also

had to juggle its mandates to carry out congressional intent for
the PRO program and to operate wi 
thin the apportioned funding
levels prescribed by the Executive Office of Management and

Budget. A more detailed description of 
HCFA' s  the PRO

program will be included in a subsequent OIG reprole ort. in 
The PROs' Ouali ty Review and Intervention Procedures

As part of their ongoing quality assurance 
efforts, the PROs draw

a sample of hospital records for a review of both quality and

utilization elements. 
Those records are reviewed on a case-by-
case basis 
, using six HCFA-generated generic quality screens and  discharge  screens. 
criteria, as well as PRO-specific  The PROs 
employ nurse reviewers, or other health care professionals, to

records. 
perform the initial review of  They refer any potential 
cases to physician reviewers for final determination. 
quali ty 
The PROs also identify potential quality problems through

profiling, 
" in which they use their data system to identify 
care. 
patterns of inappropriate 
Once the PROs have determin d that a quality problem exists,  they
are required to initiate corrective action which may include the
following: 
intensified review, alternate timing of review 
sanctions. 
education, and 
Intensified review involves sampling a larger percentage of a

particular , physician' s or provider' s, records (often 100 percent)
in the subsequent quarter to verify whether or not the identified

problem has continued. Intensified review may also be used after

contact with the physician to ensure that the particular problem

has been corrected. 
Al though most PRO review is done on a retrospective basis, the 
PRO may choose to alter the timing of that review to address

particular problems. 
For instance, if a particular physician has 
a large number of unnecessary admissions , the PRO might initiate

preadmission review of the physician I s patients. 
The PRO may also require that a doctor enroll in continuing

medical education. This could include the physician' s taking

specialized courses or possibly retaining a physician consultant

to review his or her cases. 
If the PRO determines that corrective action has failed to

address the quality problem adequately, the PRO is expected to

recommend the physician or provider for sanction. 5

The Sanction Process

The Secretary of Health and Human Services. is authorized to 
impose sanctions on Medicare-reimbursed physicians or providers

if they have "gross and flagrantly" violated or " substantially"
failed in a "substantial number of cases" to comply with their

statutory obligations to provide (1) services which are . provided
economically and 'only when, and to the extent they arE: medically
necessary,  (2) services that are " 
of a quality which meets
professionally recognized standards of health care, 
" and, (3)
services that are properly documented. For offenses related 
the first two obligations, the Secretary may impose one, of two
sanctions--ei ther a monetary penalty for no more than the  actual 
or estimated cost of the medically improper or unnecessary

services so provided" or exclusion from the Medicare. program for.
time. 
a specified period of  And for improper documentation, the 
Secretary may impose a sanction or exclusion.  The PRO must provide the practitioner or provider with

reasonable notice and opportunity for discussion" before making

its recommendation to the Secretary (42 U.  C. S 1320c-5(b) (1)).
Under the regulations, the provider or practitioner is entitled

to an opportunity to submit additional information and/or meet

with the PRO to discuss an allegation of "gross and flagrant"
violation  With allegations of " 
(s) .  substantial" violations , the 
physicians or providers are entitled to submit additional

information and to receive two notices of potential violation and

PRO. In either 
two opportunities to meet with the  case, if the 
PRO recommends the imposition of a sanction, the physician or

provider must be given 30 days' notice and an additional oppor

tuni ty to submit written comments to the Secretary. 
The Secretary has delegated the authority for sanction determina

tions to the Office of Inspector General (OIG). Upon receipt of

the PRO' s sanction recommendation, the Inspector General must

determine whether he agrees with the ,recommendation and whether 
the physician or proYider has "demonstrated an unwillingness or

lack of ability substantially to comply with statutory obliga

tions. " The Inspector General may accept, rej ect, or modify the

PRO. In 
sanction recommendation forwarded by a  cases where the 
PRO has recommended exclusion, the OIG must act on that recommen

dation within 
120 days or the exclusion automatically goes into 
effect pending final determination by the OIG. 
Wi thin the Office of Inspector General, the Office of Investiga
tions is responsible for processing and reviewing each PRO

sanction referral. If the OIG accepts the PRO' s recommendation

the sanction goes into effect 15 days upon the relevant physi

cian' 
s or provider s receipt of sanction notification. The 
physician or provider may appeal the sanction to an administra

tive law judge (ALJ), who will conductc a ase. If  de novo hearing to
review the facts of the  dissatisfied with that appeal,
the sanctioned party may then appeal to the Secretary' 
cosu rAtp.p eals

Council and may thereafter seek judicial review in  (See
appendix VII for a detailed description of the process for

addressing a case of gross and flagrant violation versus a case

numer of substantial violations. 
of substantial 
The peer review statute and imple enting regulations related to
the sanction process have attempted to balance the competing

priorities to pr tect both the ' rights , of Medicare beneficiaries 
to receive high-
quality care and the rights of physicians and

process. 
providers to receive adequate due  That delicate balance 
has meant that although a physician or provider has had an

opportunity to have at least two administrative entities 
(the PRO

and the OIG) review a case prior to the imposition of a sanction, 
full evidentiary hearing until after

the process has deferred 

the sanction has been imposed. 6
 The PROs' sanction procedures have precipitated ongoing qebate

among all parties associated with the PRO program. Organized 
medicine has argued vociferously that the PROs should provide

physicians and providers with stronger due process protection. 
In an effort to address such concerns, the American Association

of Retired Persons (AAP), the 
American Medical Association
(AM), the 
Health Care Financing Administration , and the Office 
of Inspector General drafted an agreement (hereafter rererred to
as the AAPjAMjHCFAjOIG agreement) in May 1987 to standardize 
ocedures. 
the PROs' due process p  The new PRO sanction 
instructions strengthened PRO notice procedures, clarified the

role of an attorney for the physician or provider at the PRO

discussions, ensured that physicians or providers would be

provided records of the PRO proceedings, and permitted expert

witnesses to provide relevant medical evidence at the PRO discus

sions with the physician or provider.

Over the last year, both organized medicine and Medicare benefi

ciaries from some affected communities have also argued that the

sanction process has adversely affected rural communi ties since

sanctioned physicians and providers have been excluded from the

hearings. 
Medicare program pending their ALJ  Such concerns led 
to a recent legislatively mandated requirement for ALJ review

prior to the imposition of an exclusion for any physician or

provider who is practicing in a "Health Manpower Shortage Area" 
(HMSA) or in a county with a population of 70, 000 or less. 

addi tion, Congress has directed the Secretary of HHS to conduct a

year-
end study of how the PROs' new standardized due process 
procedures have impacted the PRO program. 
Like other aspects of the PRO program, the sanction process has

been dynamic--changing as the PROs, the HCFA, and the OIG have

gained more experience and as particular groups have highlighted

its ambiguities and vulnerabilities. In addition to the afore

mentioned changes, the tOiIeGs h.a s recently proposed changes in the

use of monetary penali 
The current statutory formula for 
assessing monetary penalties is outdated giyen that the PROs' 
sanction provisions were enacted prior to the advent of the

prospective payment system (PPS). For example, under PPS, it is 
almost impossible to determine what Medicare Part A costs are for

improper or unnecessary care. Hence, the monetary penal ties 
impo  ed have generally been based on only Medicare Part B costs 
44. 
and have resulted in amounts as low as $65.  In response to
that problem, the OIG issued a technical memorandum to all the

PROs in July 1987 highlighting the lack of cost efficiency of

monetary penal 
ties and suggesting new guidelines for forwarding 
OIG. In 
such recommendations to the  addition, the OIG submitted 
a legislative proposal to the Department in the Fiscal Year 1989 
legislative process to modify the current monetary penalty

provisions so that a penalty of up to $10, 000 would be set for

each instance in which medically improper or unnecessary health

care services were provided.
 , ' 
As of December 31, 1987 , the OIG had received 151 referrals from

areas. 
follows: 
38 of the 54 PRO  The status of those cases is as 
facility) 
- 61 exclusions ( 60 physicians and 1 
facilities) 
- 26 monetary penalties (24 physicians and 2 
2 physicians expired

- 54 rej 
ections by the OIG 
8 cases still pending

A more thorough discussion of sanction issues and figures will be

process. 
included in our forthcoming report on the sanction 
FINDINGS 
Respondents considered quality review to be the most critical

element of the PROs' mission and thouqht it has received

increased focus durinq the second contract period (1986-1988) 
The PRO program emerged at a time of record budget deficits and

thereby had the inherent mandate to limit Government expenditures

care. 
by eliminating incentives to deliver unnecessary  However, 
I s 
the PROs were also expected to alleviate the public  concern 
that the PPS would precipitate a "deterioration in quality or an

negative patient outcomes. ,,7 Hence, the PROs were
increase  in 
the key instrument used by the Government to "exercise a degree

of surgical precision vs. a meat ax approach when planning to

eliminate waste while avoiding negative patient outcomes. ,,8 
An overwhelming majority of the PRO chief executive officers

(CEOs), along with other groups interviewed, articulated the

primary importance of quality review to the PROs' mission. (See
figure I. 
FI GURE I 
PERCEPT IONS OF THE PROS  PRI HAR  HI SS I ON 
PRO CH I EF EXECUT I UE 
rL OFF I CERS (CEOS ) 
REV I EW . 
QUA LIT  CASE STUD  PRO STAFF

A ND BOARD MEMBERS 
HCFA REGIONAL AND 
COST CO NT A INMENT 
CENTRAL OFFICE STAFF

NAT I ONAL EXTERNAL 
CONTRACT OBLI GAT IONS  UENTITIES 
LOCAL EXTERNAL

ENTITIES 
OTHER RESPONSES'.  TOTAL FOR ALL 
RESPONDENTS 
PERCENT A GE MENT I ON I NG  CONCEPT 
esponden ts

No  te:  N= 1BB 
Sou  ce: 01 G I nspec t i on I n te  v i ews  In addition, although the PROs and others were equally divided

I primary mission 
about whether the PROs  had changed since the
start of the PRO program, the maj ori ty of those who noted such a 
change described it as a shift toward more quality review in the

period. 
second contract  Furthermore, a maj ori ty of respondents
thought that quality review was more important than utilization

' mission. 
review for carrying out the PROs 
ty  (See figure II.
particular, a vast maj ori  (64 percent) of the PRO CEOs thought 
quality was more important than utilization review and most of

others 
the '  (32 percent) thought that quality and utilization were 
equally important. While most other respondents shared similar

high regard for quality review, a maj ori ty of HCFA staff thought

that quality and utilization were equally important. The

national external entities were equally divided between quality

being more important than, and being as important as , utilization

review. As reflected in figure II, only 3 percent of the total

respondents described utilization review as being more important

than quality review. 
FIGURE I I 
PERCEPTIONS OF THE IMPORTANCE OF QUALITY REVIEW

RELATIVE TO UTILIZATION REVIEW FOR PRO PROGRAM

QUALITY REVIEW= 55x

UTIL. REVIEW=3x

NOW= 3 x 
DO NOT 

Utilization 
x Noting Quality o 
Rev i ew as Mo e I Mpo  tan t 
esponden ts . 
No  te:  N= 186  ion I 
S 0 u  0 I G  In s pe c t  ws . 
e: n t  v i  Many of the respondents who viewed quality and utilization as

ission highlighted the inap

equally important for the PRO 
propriate dichotomy that exists in peoples' minds between the

activities. 
PROs' cost containment and quality monitoring 
One
Government official explained that "
quality assurance and cost

savings go hand in hand because any unnecessary admission that is

avoided is also a potential quality problem avoided.

Generic quality screens appear to be useful instruments for

quali ty review. but may be in need of refinement

The HCFA required all PROs in their second contract period

(1986-1988) to review all cases against a set of criteria (called
generic quality screens) designed to identify potential quality

problems. 
The generic quality screen process includes reviewing 
elements: 
the following six 
1) adequacy of discharge planning,
2) medical stability at t me of discharge, 3) deaths

4) nosocomial infections, 
hospS)i tuanlssc.h eduled return to surgery, and
6) trauma suffered in 
Two-thirds of the PRO CEOs 
thought that the screens have been at least moderately effective

identifying quality problems. 
in  A greater proportion (89
percent) of HCFA staff thought they were at least moderately

effective. In 
fact, one HCFA staff person described the screens
as "a stro,ke 0 f genius.  On the other hand, a majority 0 f the
local external entities, particularly State medical licensure

board and American Association of Retired Persons 
(AAP)
representatives 
, thought the generic screens were only marginally
useful or not effective , and a sizable proportion of the national

external entities (47 percent) noted that they 
"did not know"

about the effectiveness of the generic quality screens. 
figure III.  (See
FI GURE I I I 
PERCEPT IONS OF THE EFFECT I VENESS, OF 
GENERI C QUALI TY SCREENS 
PRO CHIEF EXECUTIVE

V ERY EFFECT I V  OFFICERS (CEOS) 
CASE STUDY PRO STAFF

A ND BOA RD MEMBERS
MODERATELY EFFECT I VE ' 
HCFA REGIONAL AND

CENTRAL OFFICE STAFF

HI HI HALLY EFFECT I VE  nNAT' IONAL EXTERNAL 
LLENT I TIES 
LOCAL EXTERNAL 
HOT EFFECT I 
V E 
ENTITIES 
TOTAL FOR ALL

DO NOT HNOW 
RESPONDENTS 
PERCENTAGE OF CATEGORY RESPONDING

esponden ts

No  te:  H=i85  ion I 
S 0 u  0 I G  In s pe c t 
e: n t  v i 
 , ' " ", . .. ..
A recent Ernst and Whinney telephone survey of the PROs and a

sample of hospitals supported the value of the generic quality

screens. 
The study I s findings noted that "the survey par
ticipants strongly agreed the current generic quality of care

screens are effective in flagging cases of suboptimal quality. " 9 
Such sentiments were echoed by many of the ' respondents in 
the OIG 
inspection, who found the screens useful because they provide the

PRO program with national quality review procedures and encourage

reviewers to focus on quality. On the other hand , as reflected

in figure IV, several respondents from each group noted a variety

of problems with the screens. 
F I GU RE 
I U 
CONCERNS REGARDING GENERIC QUALITY SCREENS

V7PRO CHIEF EXECUTIVE 
OFFI CERS ( CEOS )
EXCESS I VE REFERRALS 
CASE STUD  PRO STAFF

AND BOA RD MEMBERS 
HCFA REGIONAL AND

I MPROPERL  FOCUSED  CENTRAL OFFICE STAFF

NATIONAL EXTERNAL

ENTITIES 
NEED REFI NEMENT  LOCAL EXTERNAL 
ENTITIES 
TOTAL FOR ALL

OTHER ISSUES  RESPONDENTS 
PERCENTAGE OF CATEGOR  RESPONDING 
esponden ts

No  te:  85 
Sou  ce: 01 G I nspec t i on I n te  v i ews 
The concerns most frequently mentioned by respondents included

that the screens have produced excessive referrals to physician

advisers  have been improperly 
focused, and need refinement. The 
HCFA introduced the generic quality screens without, pilot-testing
them. 
Instructions for implementing the screens required nurse 
reviewers to refer all cases identified through the screens

(screen failures) to physician advisers 
, regardless of whether

the nurse reviewers thought that such cases merited a higher

level of review. 
Hence, the screens produced many  false
positives" 
(cases that were identified as potential quality 
problems but were subsequently dismissed by the physician

reviewers), which proved expensive for the PROs, professionally
 frustrating for both the nurse reviewers and the physician

advisers, and caused much ill feeling in the medical community. 
In response to loud protests from the PRO and medical com

muni ties, HCFA has allowea the PRO nurse reviewers some discre

tion in using their "professional judgment" with cases identified

by the generic quality screens. 
Other problems indicated by respondents were that the screens

lack flexibility for local use and that their interpretation and

PROs. This inconsistency is
application are inconsistent among 
also suggested by our recent analysis of the generic quality

screen data reports that the PROs submit to HCFA. We analyzed 
30, 1987. 
those reports through July  Given the staggered start
numer of 
up of the PROs and hence the variable  months reflected 
in each PRO' s aggregated totals as of July 30, 1987 , we are

data. 
unable to draw definitive conclusions from the  However, 
the figures do reflect wide variations in both the percentages of

cases that failed particular screens and the percentages that

problems. 
were later confirmed to be quality  The national ranges 
table 1 below:

for each of the six screens are reflected  in 
ITABLE 1 
NATIONAL AGGREGATED PERCENTAGE RAGE OF GENERIC QUALITY 
SCREEN FAILURS AND CONFIRMD PROBLEMS (1986-1987) 
Screen 
Ranqe of Screen Ranqe o f Conf irmed 
Number  Failure  Problems 
1 - 17.  o - 100% 
6 - 43.  o - 69. 
1 - 11.  o - 63. 
3 - 21.  o - 98. 
O. 1 - 12. 7 %  o - 73. 
4 - 17.  5 - 82. 
1 Percentage of all records reviewed that failed the screen

2 Percentage of screen failures that were subsequently confirmed

as quality problems

Source:  HCFA/HSQB/OMR, "Peer Review Organizat-ion Data Swnary,
July 
1987. 
Analysis of quality screen data for the 12 case study sites

further supports the suggestion that a substantial variation

exists among the PROs relative to their interpretation and

application of the generic quality screens. (See' appendix X.
Among the case study sites , generic quality screen # 1 (adequacy
of di charge planning) appears to be the most consistently

applied and useful screen, given the relatively high percentages
 of cases identified by most of the PROs that were subsequently

confirmed as quality problems. Furthermore , analysis of the

quality screen data for the case study sites identified one site

) as a particular outlier because of its consistently low

percentages both of screen failures and of confirmed problems

with five of the six screens. 
Such analysis supports the value of the PROs using some standard

generic quality screens in order for HCFA to compare and contrast

their outcomes. However, analysis of the data also suggests that

the current screens may need refinement. 
The PROs' auality review efforts are limited by a lack of

consensus reqardinq the definition of quality medical 
care. by

the amount of resources available for such care. and bY the

current lack of sophisticated technoloqy to assess quality

Physicians must provide services and the PROs must evaluate them

within an environment where differin  and sometimes competing

views of high-quality care coexist. l  One economic explanation
of the recent reemergence of the debate about the quality of and

resource allocations for medical care is that we are making a

transi tion 
away from a seller' s market (the relative scarcity of
physicians and health facilities of the previous 
2 decades) to a

buyer' s market 
ties).  (an abundance of physicians and health facili
Hence, buyers of health care--businesses, Government, and
individual consumers--are asserting their priorities for quality

contexts. In 
heal th care in both micro and macro  the micro
context, 
the various buyers want their notions of quality 
incorporated into the individual care provided by physicians. In
the macro context, they want " 
a voice in the money transfers made
to providers per unit of service and in the use of resources in

the treatment of illness. 
" 11 
The changing health care delivery environment is reflected in

s recent article, 
Secretary Bowen'  "What Is Quality Care?" in
which he tells physicians that in approaching qual i ty
of care:

"You must constantly weigh two matters. One is the

enduring concern for patients as people that you bring

with you as a physician, and the other, more public, i:
a recognition that your desire to extend the physi

cians' healing touch must be tempered by the limits of

society' 
s resources.... We who practice medicine have
not until recently had to consider the public perspec

ti ve. 
first,
For us, ciot shtass. .a.l.w ays been the patient 
regardless of  But now.. . events 
force a 
different and wider perspective on every practicing

physician. ,,12  p. 

One analysis of the differing views of what constitutes quality

is summarized in table 2 below:

TABLE 2 
DIFFERING VIEWS OF QUALITY

Interested 
Party 
Hiqh Priority Elements of Care 
Consumers  Responsiveness to perceived care
needs 
Level of communication, concern,
and courtesy 
Degree of symptom relief

Level of functional improvement

Practi tioners 
Degree to which care meets the 
current technical state-of-the-art

Freedom to act in the full interest

of the patient

Purchasers 
Efficient use of funds available 
for health care

Appropriate use of health care

resources 
Maximum possible contribution of

heal th care to reduction in lost

producti vi ty 
Source:  J ames Roberts, M. D. and J ames Prevost, M. D. "Using 
Outcome Indicators to Evaluate Quality of Care  The
Internist  Health Policy in Practice 
, September 1987
11. 
Just as different views coexist about how to define qiutal.i ty,
several perspectives also exist about how to measure  The 
quality, developed by
classic model for measuring 
components: structure, 
Avedis Donabedian, focuses on three 
outcome. Structure 
process, and  includes such factors as whether 
a hospital has a regularly operating quality assurance committee

credentialed; 
and whether the physicians on staff are properly 
process includes such factors as whether a physician or provider

delivers care that conforms with generally recognized standards

of quality; and outcome includes such factors as morbidity, 
and patient satisfaction. 
mortali ty 
All these factors interact in the, measurement of quality.
Several private entities are responding to the Public' s growing

demand for quality assurance by developing new quality assessment
 methodologies. For  ta
instance, the Joint Commission on Accredi 
tion of Heal thcare Organizations (JCAHO) is currently identifying

key clinical indicators of quality in order to create a national

data base of normative performance to which an organization can

compare its own performance. Such data will be adjusted for

mix. 
difference in case  Other academic institutions and private 
companies have developed various computer programs to evaluate

clinical outcomes. Al  valuated their effec

ti veness, these  though we have not  include: 
new software systems  the Medical 
Illness Severity Grouping System (MedisGroups), the Acute

Physiology and Chronic Health Evaluation (APACHE II), the

Computerized Severity Index (CSI), the Disease Staging and the

Patient Management Categories (PMC). 14

In the course of our interviews with PRO CEOs and staff, many

voiced concerns about their lack of accessibility to the latest

quali ty assessment technologies. The PROs' current case-by-case

review of patient records is a highly labor-intensive process.
Virtually all the case study sites are experiencing high turnover

among nurse reviewers who constitute the backbone of the review

process. In 
addition, several PROs, as well as national external
anti ties 
, voiced the feeling that the PROs should be more in the 
forefront of quality assurance research--both in field testing

severi ty measures and developing new quality assurance tech

niques. As one PRO medical director put it, "We should be the

national leaders of quality assurance research. "IS 
The PROs exhibit qreat variations in their approaches to 

outcomes of quality review

As previously stated, our analysis of HCFA' s data reports through

July 
30, 1987 , suggested that the PROs' interpretation and
application of the generic quality screens were inconsistent. 
In addition, recent congressional testimony by Sidney 
Wolfe, M. D. ,
Director of Public citizens Health Research Group (PCHRG),
reflected that the PROs vary in their provision of quality-

related information to the public. Most of the PROs surveyed by

PCHRG appeared to receive and process little or no requests for

public information, while a few PROs appeared to be very active

in providing such information.

Our case study site visits enabled us' to! compare and contrast the 
PROs' processes for reviewing and addressing quality' of care
problems. We found great variation among, the PROs ' review

systems regarding the following: 
" three to six); 
the number of severity levels 
(i. e. 
the number of review layers (i. e.  number of different 
groups of individuals who review .a given case) ; 
the number and type of intervention steps and time frames

between the identification of a quality problem and the

potential sanctioning of a physician or provider;  the number of cases required to constitute a " substantial
numer of substantial violations; 
" and 
ties 
the guidelines for determining what penal  (monetary
penal ty or exclusion) to recommend in sanction cases. 
Furthermore 
tion  , our analysis of HCFA' s aggregated Quality Interven
(544J) Report for the 12' case study sites through
September 30, 1987 suggested some interesting patterns. 

general, the PROs repo ted using very little formal education as

an intervention tool and relatively little intensified review. 
One State reported no new physicians with quality problems within

four quarters, whereas another found 5, 694 in, the same time
period. 
Most significantly, the two case study site PROs that 
had the most complicated quality review systems 
(in terms of

layers of review) also were among the three least active PROs

report. 
according to the 544J  The variations in the PROs' 
reporting reflected either inconsistent interpretation of the

report instructions or differences in their approach to quality

review. 
The PROs' impact on ensurinq hiqh-
quality care would be enhanced

bv qreater coordination with other health care ent

particularl y with the State 
medical licensure boards 
PRO Relationships with State Medical Licensure Boards: 
Sixty-seven percent of the  EOs no  d a relatively poor

44 PRO 
working relationship 
(i. e. , less than "moderately effective"

with State medical licensure boards 
(See figure V) and

27 percent of them indicated that they needed clarification of

restrictions. 
their Federal confidentiality 
Several CEOs noted 
that their wish to work more cooperatively with the State medical

licensure boards is inhibited by their uncertainty about what

boards. 
information they are allowed to share with those 
The 
HCFA has yet to release clarifying instructions 
(i. e. , the

confidentiality chapter of the, PRO Manual) to the PROs on this
issue. 
In only one of the 12 case study States did the PRO and the State

medical licensure board representatives agree that they had a

very effective 
relationship. ". In that particular best. practice
State, the close working relationship between the organizations

was fostered by the cross-fertilization of personnel--two current

medical licensure board staff were former PRO 
employees, and one

physician on the State medical board also sat on the PRO board of

directors. 
every other case study site, at least one
However,  in 
of the two organizations perceived a poor working relationship

organizations. 
between the two  Several PRO CEOs mention d that 
they had referred sanction cases to their State medical board but

had received no feedback.  , ,/

FIGURE V

PRO CEOs  PERCEPT IONS OF EFFECT I VENESS OF 
PRO RELATIONSHIPS WITH STATE EXTERNAL ENTITIES

liVERY EFFECTIUE 
MED. LICENSURE BRDS. 
CjMODERATELY EFFECTIUE 
MI NI MALLY EFFECT I UE 
EFFECT I V
EJ NOT 
HOSP. ASSOCI AT IONS 
LJNO RELAT I ONSH I P 
MEDICAL SOCIETIES

PERCENTAGE OF PRO CEOS RESPONDING

esponden ts
No  te:  N=44 
Sou  ce: 01 G I nspec t i on I n te  v i ews 
The continuing pattern of poor working relationships between the

PROs and the state medical licensure boards is particularly

disturbing in view of a previous OIG inspection report on state

medical licensure boards which included recommendations for the

boards to improve their working relationships with other review

enti 
ties. 17 
PRO Relationships with Hospital Associations:

As reflected in figure V, the PRO CEOs perceived having much

better relationships with their local hospital associations. 
Fifty-eight percent of the PROs thought that their relationships

wi th their hospital associations were "very effective" and

33 percent thought that they were "moderately effective.  Our 
pattern. 
case study site visits confirmed this  In two sites, 
both the PRO and the hospital association rated their relation

ship as being "very effective. 
In both those States, hospital
board. In 
association board members sat on the PRO  eight of the
10 other States, the PROs and hospital associations ranked their

relationships as at least "
moderately effective.  Despite their
generally positive working relationships  f the PRO and

hospital association staff mentioned thatr  sevterhal e PROs had had

indi 
problems wi th  idual hospitals regarding th  PROs 
unwillingness to share information about particular physicians

who had been identified as having quality problems.
 ionships with Medical Societies: 
PRO Rela 
The PROs' relationships with their respective State medical

societies, as reflected in figure V, appear to be stronger than

wi th the state medical licensure boards but less effective than

wi th the hospital associations. Seventy-seven percent of the PRO

CEOs reported at least moderately effective relationships with

their medical societies. Again , our case study si te visits 
supported the pattern reflected by the CEO interviews. Two case 
site PROs had "very effective" relationships with their respec

In both cases , the PROs were medical-

tsivoe mecdiciael stocyieti ess 
ponsored. In 
most of the remaining sites , the PROs had
at least moderately effective relationships with the medical

societies. 
Two States reported poor working relationships 
between the PRO and the State medical society; one of the two had

a large number of sanction referrals and the other had none. 
Most PRO and medical society respondents mentioned that the 
maj or

issues between their organizations focused on whether the PROs

quality intervention procedures provided adequate physician and

provider education and whether the PROs' sanction procedures

provided physicians with adequate due process. 
Importance of Communication Between PROs and External Entities: 
In comparing the 12 PRO case study sites , we found that greater

coordination between the PRO and external entities seemed to be

enhanced when the PRO initiated some formal ongoing communication

link with that entity, either through representation on the PRO

board or through regular issue-related meetings. 
Implementation of the Medicare and Medicaid Patient and Program

Protection Act of 1987 is intended, in part, to facilitate closer

working relationships and greater information sharing between the

PROs and other entities, especially State medical licensure

boards. 
That legislation requires States to make available to 
the HHS Secretary information concerning adverse actions taken by

State medical licensure boards against health care practitioners.
It also requires that the HHS Secretary disseminate information

on these actions to State medical licensure boards and to other

officials. 
State and Federal 
Most PROs are findinq it difficult to provide effective outreach

to Medicare beneficiaries.

Eighty-four percent of all PRO CEOs, 62 percent of HCFA staff,
and 72 percent of the AAP representatives  thought that the PROs
had at least moderately effective relationships with beneficiary

organizations. In 
ori ty 
addition, the vast maj  of the PROs 
thought they were at least moderately effective in their outreach

iciaries 
efforts to benef  (See figure VI.  .. '
FIGURE VI 
PERCEPT IONS OF THE PROS  EFFECT I VENESS 
OUTREACH TO HEDI CARE BENEFI CI ARI 
PRO CHIEF EXECUTIVE

rL OFF I CERS (CEOS ) 
VERY EFFECT I VE . 
CASE STUDY PRO STAFF

A ND BOA RD MEMBERS 
MODERATELY EFFECTIVE ,  HCFA REGIONAL AND 
CENTRAL OFFI CE STAFF 
r/NATIONAL EXTERNAL 
MINIMALLY EFFECTIVE  UENTITIES 
LOCAL EXTERNAL

NOT EFFECTIVE  ENTITIES 
TOTAL FOR ALL

RESPONDENTS 
DO NOT KNOW'

PERCENTAGE OF CATEGORY RESPONDING

esponden ts
No  te:  85 
Sou  ce: 01 G I nspec t i on I n te  v i ews 
ori ty 
However, in contrast, a vast maj  of the national and l 
external entities either thought the PROs were only minimally or

less effective or "did not know" how effective the PROs were in

their outreach efforts to beneficiaries. As one national entity

representative said:  it. 
"I know that if they were doing much 

this area that I would have heard about  And I haven  t heard
anything. 
The major outreach activities mentioned by the PROs included

aspretaikcilneg se.n gagements, hot lines, radio spots, and newspaper

The major reasons cited for the PROs' relative lack of 
activities in this area included the PROs I limited resources and

the fact that the elderly are a difficult population to reach

effectively. In .
our case study visits, we observed three. PROs 
wi th especially strong beneficiary outreach efforts. In all of

those organizations, particular staff have been assigned to that

function and have developed extensive video and written

materials. 
They, along with consumer representatives on the PRO 
beneficiaries. Al 
boards, hold ongoing meetings with  though staff 
at one case study site raised questions about the rationality of

the PROs having responsibility for beneficiary, outreach, most PRO 
staff felt they could do more outreach activities if they wer

provided with the necessary resources and opportunities to share

ideas with other PROs. Aside from the AAP, we heard of few 
other private or Government organizations providing information
 to the beneficiaries about their local PROs. For instance, only
6 of the 44 PRO CEOs reported that their local Social Security

offices distribute PRO-related information. 
The PROs will face serious challenqes and obstacles in reviewinq

care in nonhospi tal settinqs

roblems 
Seventy-five percent of all CEOs interviewed anticipate 
reviewing care in nonhospi tal settings and the four other

in  concern. 
groups interviewed voiced similar levels of  Besides the 
normal learning curve associate1 with any new 
activity, the

primary concerns anticipated by various groups are reflected in

figure VI!. 
F I GU HE 
U I I 
EXPECTED PROBLEMS WI TH REV I EW IN 
NON-HOSPI TAL SETT I NGS 
PRO CHIEF EXECUTIVE

FUNDI NG  OFFI CERS (CEOS ) 
CASE STUDY PRO STAFF

AND BOA RD MEMBERS 
REGULAT IONS

HCFA REGIONAL AND

CENTRAL OFFICE STAFF

DATA SOURCES  r7 NA T I ONA L EXT ERNA L 
ENTITIES 
LOCAL EXTERNAL

MED. COMMUN. SUPPORT 
ENTITIES 
TOTAL FOR ALL

RES PONDENT S 
REV I EW TECH NOLOGY 
COORD. w/S NFs 

PERCENT A GE OF CA T EGORY RES POND I NG 
esponden ts/Sou ce: 01 G I nsp.
*N=1S85 NFs--Ski 11 ed  s ing  ac i  I n t  v i 
*HHAs--HoMe heal  it i 
th agenc 
The PROs' maj or concerns focus. on the lack of adequate funding,
guidance (regulations) and data sources , as well as the lack of

support that they anticipate from the medical community. In  addition, they voiced concern about their inability to link data

on medical care provided to Medicare beneficiaries in hospital

settings 
(Medicare Part A) with data on medical care provided to 
settings 
nonhospi tal  (Medicare Part B). 
them  in 
The HCFA staff anticipated that the PROs' maj or problems in
reviewing care in nonhospital settings would include 
(in descend

ing order of frequency) the lack of adequate data sources and

review technology for ambulatory settings as well as the inherent

difficul ties 
of coordinating with multiple skilled nursing
facilities 
(HHs). 
(SNFs) and home health agencies  The national 
and local external entities anticipated that the PROs' major

problems would include (in descending order of frequency) the
lack of adequate review technology, the lack of support from the

the lack of available data sources and the

medical community 
lack of funding.

In both our macro-level and case study interviews
, individuals

from all groups voiced concerns about the feverish pace at which

the PRO program has moved. The PROs have been asked to do ever-
increasingly complicated reviews without the time and technical

job. 
assistance to do a credible  As one PRO CEO staff person
noted: "If 
why don  t you
you really want to help us do better 
leave us alone for a while?" 
RECOMMNDATIONS 
Quality of Care

The HCFA should work with the PROs to develop more

consistent approaches to defining and addressing quality of

care problems.

The HCFA should work with the PROs to ensure more consistent

reporting of quality intervention acti 
vi ties. 
The HCFA should develop and widely disseminate information

on the full range of educational corrective actions being

taken by the PROs to address quality of care problems. 
The HCFA should create more opportunities for the PROs to

share information with each other concerning their quality

review and intervention approaches. 
The HCFA should intensify its support of research to define

care. 
and assess the quality of medical  In addition, HCFA 
should collect and integrate quality-related research

findings from all relevant sources and disseminate them to

the PROs.
 Discussion: 
The HCFA has an opportunity to build upon the work it has already

care. 
undertaken in rasearch related to quality of 
The PROs' 
quality review efforts are difficult and complex, and they have a

limi ted track record. 
Implementation of the above 
recommendations will help the PROs learn from one another'

experience and broaden the pUblic' s understanding of the PROs' 
quali ty review efforts.

Generic Ouali ty Screens 
The HCFA should ensure that any maj or modifications to the

generic quality screens are pilot-tested before national

implementation. 
The HCFA should promote greater consistency in the

application of generic screens through its training efforts

and operational instructions. 
The HCFA should support the development and pilot testing of

more effective and less labor-intensive review

methodologies. 
Discussion: 
Implementation of the generic quality screens was central to the

efforts. 
PROs' quality of care review  The HCFA has already taken
screens. 
steps to refine the  By also implementing the above
recommendations, HCFA will help to avoid unnecessary expenditure

of PRO resources and enhance the credibility of the 
PROs' quality

review efforts.

PROs' Coordination with Other Entities

The HCFA should promote closer working relationships between

the PROs and other health care entities and encourage more

sharing of information about physicians and providers with

of care problems. 
quali ty  These efforts should include 
clarification by HCFA of the PROs' confidentiality

restrictions and a regulatory requirement that PROs report

physician misconduct or incompetence to State Medical

Licensure Boards. 
This should be accomplished through the 
Notice of Proposed Rulemaking  hanges to peer

review organization regulations(N.PR M) on 
Discussion: 
everal different 
types of entities respon
The PROs are one of 

sible for ensuring quality of medical care. ' By encouraging PROs
to work more cooperatively with other entities that share similar

missions 
, the HCFA will help maximize the PROs' impact on quality 
of care.
 PROs' Outreach to Medicare Beneficiaries

The HCFA should seek ways to strengthen the beneficiary

outreach efforts of the PROs. In this context, it should

coordinate- more 
closely with the Social Security 
Administration and the Office of Human Development Services

and should provide more opportunities for the PROs to share

best practice approaches with each other. 
Discussion: 
An extensive network of Federally funded agencies provide

services to the Medicare population. By coordinating with those

agencies, the HCFA can help PROs gain better access to the

serve. 
Medicare population it is charged to  In addition, because 
relatively few PROs have had extensive experience with such

outreach efforts 
, it is important to provide opportunities for 
the PROs to learn from each other' s experiences.

Review of Care in Nonhospi tal settinqs

The HCFA should work with the PROs to delineate the most

significant constraints that will be faced in reviewing care

in nonhospi tal settings and to develop strategies for

these constraints. 
address ing 
The HCFA should seek ways to improve the availability of

data on outpatient and inpatient services provided to the

Medicare population. In this context, it should build upon
demonstration proj ects
currently underway in Texas and
Maryland to link the Medicare Part A (inpatient) and Part B

(outpatient) data files. 
Discussion: 
The PROs' review of care in nonhospital settings will lead them

into unchartered waters. To prepare fo  such broad and compli
cated responsibilities, the PROs will need to use lessons learned

efforts. 
from their inpatient review  Implementation of the above
recommendations will help the PROs address the most critical

concerns in their efforts to carry out effsctively their expanded

responsibilities  APPENDIX I 
ENDNOTES 
49, No. 
Federal Register, Vol.  39, February 27, 1984 
p. 7202.
2 .  Tax Equity and Fiscal Responsibility Act of 1982, Commerce 
37D. 
Clearing House, Inc. p"  Senate Rept. 97-494 at 41. 
3 .  "Examination of Quality of Care Under Medicare' s Propective
PaYment System, " Hearing before the Committee on Finance, 
United States Senate , June 3, 1984 , p. 44.

Andrew Webber, "A PRO Report Card,  The Internist: Heal 

ice 
Policy in Pract, , the American Society of Internal 
Medicine, July 1986, pp. 7-9. 
Material on the PROs ' quality review process was taken from

Conqress: 
Conqressional Research Service Report for  The 
the 
Peer Review Orqanization Proqram, Th  Library of Congress

October 23, 1987, pp. 20-21. 
Material for the description of the sanction process was

taken from the transcript of Inspector General Richard

Kusserow' s testimony at a hearing before the Subcommittee on 
Intergovernmental Relations and Human Resources, Committee

on Government Operations, United States House of

Representatives, October 20, 1987.

Carole Magoffin, " 
7 .  Can PROs Uncover Quality Concerns?" 
Business and Health, the Washington Business Group on

Health, July/August, 1986, p. 14. 
Ibid 
., p. 15. 
Ernst and Whinney National Office , 2000 National City

Center, Cleveland, Ohio 44114, Tel. (216) 861-5000

ext. 5800 , unpublished survey report.

10.  D. and James Prevost, M.  ing 
D., "Us 
James S. Roberts, M

OIunttceormen iIsntd:i cators to Evaluate Quality -of Care:  The
Health Policy in Practice ;, the American Society 
of Internal Medicine, September, 19S6; pp. 10-15. This
thought-
provoking article offers a definition of high
care that bridges diverse perspectives: 
"High 
quali ty 
quality care identifies accurately and fully the health care

needs (educatiQnal, preventative, restorative and main

tenance) of an individual or group and.  pplies  the resources 
(human and other) necessary to meet those needs in a timely

manner as effectively as the practical state of the art

allows. "  , p. 
, " 
.,. .':" .
11.  Uwe Reinhardt, "Quality of Care in Competi ti ve Markets,
Business and Heal  , the Washington Business Group on 
Health, July/August, 1987 , p. 7.

otis Bowen, H.  , "Hhat is Quality Ca Nev! Enqland 
re?" 
Journal of Medicine , Massachusetts Medical Society,

June 
18, 1987 , p. 1578. 
13.  Arnold Milstein, David Nash and Jonathan Sands, , "Auditing 
Qual i ty of Care: An Employer Based Approach,  Bus iness and 
Heal th , the Washington Business Group on Health,

July/August, 1987  10. 
For a detailed summary of the elements and costs of these

computer packages, see Charles M. Jacobs, Alan Brewster, 
D., J. 
D. and James B. Couch, M.  Severity of Illness in 
the Cost/Qual i ty Equation,  The Internist: Health Pol icv in 
Prac  ice , the A erican Society of Internal Medicine
Septe  ber 1987 , pp. 16-31. 
15.  The following research efforts have recently been undertaken 
and should provide some valuable quality-related information

for the PRO community:

The National Academy of Sciences is conducting a study

to develop a strategy for reviewing and assuring the

and 
quali ty of care for Medicare beneficiaries 
The consulting firm of Lewin/IMC, under contract with 
the HHS Assistant Secretary for Planning and Evaluation

is conducting tv!O related studies. The first, "A 
Taxonomy and a Critical Review of Existing Quality of

Care Guidel ines" will critique existing systems for

enhancing qual i ty medical care. The second, " A Forward 
Plan for Expanding the Use of Quality of Care

Guidelines 
" will develop a systematic plan for HHS to 
encourage the dissemination and application of

clinically useful information on medical practices.

16.  Testimony of Sidney M. Wolfe, M. D., Director, Public
ci tiz ens Health Research Group before the Subcommittee on 
Health and the Environment, Committee on Energy and Com

merce , U. S. House of Representatives , October 26, 1987. 
17.  Medical Licensure and Discipline: An Overview, Office of
Analysis and Inspections 
, Office of Inspector General, 
Services, June 
Department of Health and Human  1986.  APPENDIX II

BACKGROUND OF THE PRO INSPECTION

The Office of the Inspector General (OIG) is mandated by statute

to provide leadership and coordination within the Department of

Heal th and Human Services (HHS). The OIG is charged with 
ensuring the economy, efficiency, and effectiveness of HHS

program operations. 
Because of the PRO program' s vi tal role in protecting both the

quality of care of Medicare beneficiaries and the financial

integri ty of the 
Medicare program, the OIG has taken a keen 
interest in and has had a close association wi th the PRO program

vi ties, 
since its inception. Among other acti  the OIG has
conducted pre-
award audits of the PRO and SuperPRO contracts and 
of sanction cost estimates for HCFA and has made sanction

determinations on cases referred by the PROs. 
The impetus for this inspection of PRO performance grew out of

the Inspector General'  gaining a broad

s personal interest  in 
perspective on the PROs' performance during the second scope of

work. 
Al though several other entities had reviewed various elements of 
it. 
the PRO program (see appendix IV), no one had undertaken a broad

evaluation of  Hence, in the fall of 1986 , the Inspector
General asked the Office of Analysis and Inspections to conduct

an inspection of the PRO program in the spring/sumer of 1987. 
In addition, the OIG' s Office of Investigations asked us to

incorporate a review of the sanction process into our overall

inspection. We designed the PRO inspection to integrate some

original PRO data collection and analysis with other existing

PRO-related data collected from primary and secondary sources.
Due to other OIG priori 
ties, completion of the PRO inspection 
field work was delayed until the fall of 1987. 
We designed this PRO inspection to provide the Inspector General

and other departmental officials, policy makers, and the public

wi th a broad perspective on the PRO program and how it has
changed over time. 
tiori to t 
In addi  is inspection, the OIG has done and continues to 
program. 
do other work related to the PRO  The following is a
summary of some key audits and inspections. 
Past Work

Review of FinanC'ial Operations of Peer Review Organiza
tions 
(Audit No. 14-62158), which conclud'ed that most 
PROs made a sizable profit on their initial contracts

and recommended administratiye and fiscal procedures
 for HCFA to incorporate into its management of the PRO

program. 
Inspection of Inappropriate Discharges and Transfers

March 1986, which concluded that many PROs had not

effectively used the authorities or processes available

to address poor quality of care associated with

transfer. 
premature discharges and inappropriate  The 
report incluged recommendations regarding HCFA' s 
reporting and evaluation systems and suggested that the

PROs be given authority to deny payments for substan

dard care. 
Report on Evaluation of California Medicare Review

Inc. 
(CMRI) Price Proposal for Development of
Sanction Cases (Audit No. 09-61658), 
September 1986 
ich concluded that HCFA failed to include

reimbursable sanction activities into its fixed-price
PROs. 
contracts with the  The report recommended that
HCFA: 
(1) provide guidance to all PROs on the 
costs; 
reporting of and accounting for sanction  (2)
require all PROs to establish adequate cost accounting

systems for summarizing the costs of sanc ion
ties; 
acti vi 
(3) ensure all PROs develop and implement 
bid estimating procedures that more reasonably reflect

activity;
the estimated costs of performing sanction 
(4) ensure that all PRO contracts contain the necessary
cost-reimbursement provisions required by the Federal

acquisition regulations 
(FAR) before initiating any 
sanctions; and 
reimbursement of  (5) require CMRI to 
provide an accounting of funds advanced for sanction

acti vi ties 
and return any excess funds to the Federal 
Government. A subsequent report (Audit No. 09-8661662)

was issued in May 1987 on CMRI' s price proposals for

its first 117 sanction cases. That report reinforced

the recommendations of the first report and also

recommended that HCFA issue modifications to the PROs'
fixed-
price contracts to make only the PROs 
incremental sanction costs reimbursable.

Current Wo

Tha Region VII Office of Audit is conducting a national

cost-benefit analysis of five types of PRO review

including retrospective admission , DRG validation, day

outlier, cost outlier, and pre-admission. The audit is

focusing on 
14 PROs and is expected to be completed 
before the spring of 1988. 
The Office of Analysis and Inspections (OAI) is 
coordinating the national DRG Validation Study, an

analysis of medical record data collected from 239 PPS
 hospitals (for 
the period October 1, 1984 to March 31, 
s. 
1985) for DRG validation and identification of quality

of care proble  The OAI central office staff are 
coordinating the review but regional staff are respon

sible for analysis of the PRO-related data on DRG

val ida tion and qual i ty . Those reports are expected to

be completed by the spring of 198?  APPENDIX III

METHODOLOGICAL NOTES

Summary of Interviews

Because we wanted to examine how' both the PROs and HCFA viewed
themselves and were viewed by others 
, we conducted in-depth
(approximately 
2-hour) interviews with a variety of people
associated with the prQgram. Those 211 individuals included "the
following: 
3 areas;
PRO chief executive officers (i. e., all PRO CEOs as

8 of the 44 PROs manage 2 PRO areas and 1 PRO manages
other PRO staff and board members (i. e. , the medical

directors 
, program directors , review directors , board
chairs 
, and consumer representatives from the 12 PROs 
visits) 
selected for case study site 
ti ves (i. e. 
na tional external entity representa  , the 
American Association of Retired Persons, the American

Hospital Association, the American Medical Association,

the American Medical Peer Review Association, and the

Public Citizen Health Research Group as well as the

Department of Health and Human Services , Executive

Office of Management and Budget , and congressional

committee staff);

local external entity representatives (i. 
e., State
medical societies 
, medical licensure boards , hospital 
associations, fiscal intermediaries, and the American

Association of Retired Persons chapters associated with

visits) 
the 12 PROs selected for case study site 
Heal th Care Financing Administration (HCFA) central

office staff (i. 
e., from the Health Standards and 
Quality Review Bureau, the Office of Management and

Budget, and the Bureau of Program Operations); and

(i. e., all 
HCFA regional office staff  10 Associate
Regional Administrators for ' Health Standards and 
Quali ty, al-l 10 branch chiefs and a sample of the

proj ect  the Medical Review Branch) .
officers  in 
Case Study Selection

In an effort to gain a first-hand perspective on the PROs'
operations , we made 3- "to 4-day site visits to at least one PRO 
' As part of that 
from each of the 10 HCFA geographic regions. 
case study effort, we planned to compare those case study
 PRO-specific evaluation documents (i. 
assessments to HCFA' s  e. , 
work. 
PROMPTS and SuperPRO) for the second scope of  Hence, we 
er 
eliminated those PROs with a Nove  1, 1987 contract start date
( i. e. , Group 5 PROs) from the case study sel ect ion, pool since we 
would be unable to obtain their corresponding HCFA evaluation

documents in time for revievl. We also eliminated the Pennsyl

vania PRO frcm the selection pool since its second contract

period only began on July  1987. 
We then drew a judgmental sampl,e of the PROs based on the

size 
(as reflected by funding level), 
following criteria: 
geographic location, a d sa ction activity level. We divided the

PROs into four groups based on their Medicare contract awards

9 million or less 
($2.  , $3-5. 9 million, $6-8. 9 million, and 
million or more) and calculated the appropriate number of PROs to

category. 
select from each funding  That selection of particular 
PROs focused on ensuring a group of PRO sites with a distribution

aphic area (i. e. 
of geog  , at least one PRO from each of the 
HcrA regions) and of sanction activity levels and with at least

some representation of PROs that had both Medicare and Medicaid

contracts. 
The final group of 12 organizations selected for site visits in

the 10 HCFA regions were as follows: 
HCFA Reaio  PRO Area  Orqanization 
Massachusetts 
Massachusetts Peer Review 
Organization, Inc. 
Waltham, MA

Rhode Island  Heal th Care Review, Inc.

Providence, RI

New York  Empire State Medical, Scientific and
Educational Foundation, Inc. 
Lake Success, NY

Delaware  West Virginia Medical
Insti tute, Inc. 
Charleston 
, WV 
Florida 
Professional Foundation for Health 
Care, Inc. 
Tampa, FL

Georgia  Georgia Medical Care Foundation 
At lanta, GA  HC?':. Reo-ion  PRO Area  Orcranization 
Indiana 
PEERVIEW , Inc. 
Cannel, IN

Texas  Texas Medical Foundation 
Austin, TX

Iowa  Iowa Foundation for Medical Care 
West Des Moines, IA

Colorado  Colorado Foundation for 
Medical Care

Denver, CO

California 
California Medical Review, Inc. 
San Francisco, CA

:. 0  Oregon  Oregon Medical Professional Review
Organization
Portland 
, OR 
In the case study selection process, we opted to choose PROs

based on their individual contracts with HCFA, rather than

combining multiple contracts held by one PRO for different PRO

areas. 
We chose Delaware as a PRO site to visit, although the
West Virginia PRO actually holds the contract for Delaware. 
Hence, we refer to the West Virginia PRO in listing the case

study PROs but note parenthetically that discussions focused on

the Delaware contract. On the other hand , our site visit to the

Rhode Island PRO focused on its operation in that state, although

ode Island PRO also holds the PRO contract for Maine. 
the 
Discussion Guides

We designed seven separate but interrelated discussion guides to

capture the perspectives of PRO executive directors
, national

external entities, other PRO board and staff, lsotcaalf fe.x ternal

enti ties and HCFA central and regional office  The 
discussion guide questions were structured so that we could later

compare responses wi thin and a' cross groups.  scussion 
All di
categories: 
guides grouped questions under three or four' . 
PRO
assessment  ty review and , 
sanctions, HCFA oversight, and in 
, qual i 
some cases  : The discussion 
guides included
, descriptive material 
about equal numbers of closed and ' 
open-ended questions , but most
of the closed questions had an open-ended probe following them. 
Interview ADProach

We conducted approximately half the 211 interviews by phone and

person. 
the other half in  For methodological consistency, we 
chose to interview all 44 PRO chief executive officers (CEOs) by
 telephone and held subsequent additional on-site isnitteervsi.e ws with

those CEOs associated with the 12 case study PRO 
the 
addi tioTI to  12 case study PRO sites, we conducted on-site 
rather than telephone interviews with most of the national and

ice staff. 
local external entities and with HCFA central off 
The primary PRO inspection team consisted of four individuals

from Region I who conducted, 95 percent of the telephone inter-

ones. 
v iews and 80 percent of the on-s i te  Four additional field 
team members (two from' Region I and two from OAI' s central 
interviews. At 
office) conducted the other  least two team 
members participated in each of the 12 PRO case study site

visi ts. Inspection inte  'iews ranged up to five hours with an 
hours. 
average length af two  We informed all participants 
interviewed for this study that the confidentiality of their

specific responses to questions would be maintained, unless

otherwise cleared by them. 
plan, the proj 
As part of our quality control  ect leader assigned 
I s 
one person to be the proj ect  administrative coordinator. That 
all 
individual developed and maintained a tracking system for 
discussion 
guides, correspondence, supplementary materials, and a 
interviews. 
master schedule of team 
Codincr and Analysis 
We designed three primary and six relational data files, using

dBASE III PLUS, to store and tabulate interview responses. We

developed codes for all questions and one team member generally

coded all questions in a given file to maximize coding

consistency. In 
addition, a different team member checked at 
accuracy. 
least a 20 percent random sample of the files to assure 
As part of the PRO inspection team I S quality control plan, the

proj ect  proj ect I s data
leader assigned one team member to be the 
coordinator. In  i ty for 
addition to having primary resP9nsibil 
designing the PRO data base, that individual had responsibility

for developing and enforcing data-related quality control

procedures. 
We used dBASE III PLUS to tabulate all interview data by respon

dent group (i. e., 
PRO CEOs, other PRO staff and board , HCFA, 
entities). 
national external entities, and local external 
Other PRO-Related Data

In addition to interview data, we collected and analyzed other

PRO-related data including: HCFA' s monthly and quarterly data

summary reports for all PROs and HCFA' s PROMPTS and SuperPRO

reports for the 12 PRO case study sites. We also collected and

reviewed a wide array of other materials concerning PROs, 
hearings, 
including newspaper and journal articles, congressional 
audits. 
and GAO , Library of Congress, OIG, and other studies and  , "

Methodoloqical Considerations in Interpretinq PRO Interview Data

The reader should keep three caveats in mind when reviewing this

report. First'
, because we wanted to give as comprehensive a view 
of PROs as possible, we have integrated the case study data with

data. 
the universal  The case study data is generally used to
amplify broader-based findings 
, and such data is always clearly
labeled. 
Although we used a judgmental rather than random 
sampling methodology for choosing the case study sites, it is

worth noting that those sites are broadly representative of PROs

by size , geographic location , and sanction activity level. 
A second caveat to keep in mind is that although we interviewed a

total of 211 individuals, a given question may have been directed

to only a subset of tha 
t universe. Therefore , in this report, we
figures. 
have sought to clarify the number of people responding,  tNo =a g i

question by noting the universe of respondents (i. e. 
) in
all relevant summary tables and 
The third but perhaps most important consideration to highlight

is that much of the information gathered in this study came from

questions with both closed and open-ended parts (e. g.  Do you
have any recommendations to the Federal Government regarding

actions it might take that would help PROs be more effective in

addressing quality of care issues "? Explain. 
Because we chose

not to distribute the discussion guides prior to the interviews

the open-ended questions required the respondent to spontaneously

formulate his or her answers. Therefore , the per-centages of

people noting any particular answer vary much more than 
if the

respondents had been presented with limited response options or

had reviewed the discussion cruides prior to the interviews.  ~~~
APPENDIX IV 
RY OF PRO-RELATED STUDIES

is vi tal 
Because the PRO program  to the Medicare program and 
hin a highly vis 
ible pol  tical arena I several entities 
exists wi 
 program. 
have evaluated the  The following is a summary of some 
key studies related to the PROs: 
Past Studies

The Congressional Research Service (CRS): 
"The Peer Review Organization Program, " October 23 I
1967: 
The study presented a summary of the legislative 
history, program features, and relevant issues of the

PRO program. The CRS report was prepared at the

request of the House Committee on Energy and Commerce 

Heal th and the Environment. It revised 
Subcommi ttee  on 
a prior report prepared at the request of the Senate

Committee on Finance. 
The General Accounting Office (GAO): 
"Strategies for Assessing Medicare Health Care

1987: 
Qual i ty I " December 3 0,  The study eva1 ua ted the 
systems for assessing quality of care in the Medicare

carriers 
program (i. e.,  I intermediaries , and PROs) and 
identified short- and long-term strategies for measur

care. 
i ty 
ing and monitor ing qual  0 f  Among other
suggestions  HCFA: review the 
I the GAO recommended that 
PROs' methods for dealing with quality issues; evaluate

the spheres of responsibilities of the PROs, FIs, and

carriers to determine that their responsibilities are

appropriately divided; require the PROs, FIs, and

carriers to maintain data related to 
quality; require

that patient diagnoses be recorded on Medicare out

patient Part B claims and develop HCFA data files of

information; 
that Part B  and develop a mechanism to 
allow SuperPRO to evaluate PRO cases that were selected

through both the PROs' random sample and specific

records. 
samples of hospital 
"Better Controls Needed for Peer Review Organization

" October 8, ' 1987: 
Eval uations,  The study assessed 
HGFA I S evaluation process for the 1986-1988 contract 
awards and concluded that HCFA' s process was fraught 
with inconsistent and inadequate documentation and

instructions. Al 
improper application of  though GAO 
found no evidence of inappropriate contract decisions

that: 
it recommended  HCFA develop sufficient internal 
controls for PRO evaluation, provide better ongoing
 monitoring to the PROs and collect and use adequate

cost and performance data to set each PRO I S contract
funding level. 
Reviews of Quality of Care at Participating

ber 15, 1986: 
Hospi tals, " Septerr. The study was based 
on a survey of California, Florida , and Georgia PROs

and focused on the monitoring of inappropriate

discharges and profiling of hospital and physician

of care problems. 
quali ty  The GAO recommended that 
HCFA require PROs to include quality of care review

data available from the 1984-1986 contract period in

their prof iling of hospitals and physicians and that

the PROs review the appropriateness of the discharge

destinations as part of their discharge reviews to

better, ensure that 
patients needing skilled nursing 
awaiting placement. 

care are allowed to remain in the hospital while

The Prospective Pa  ent Assessment Commission (ProPAC): 
Report and Recommendations to the Secretary, 
U. S.
Department of Heal  " April
th and Human Services 
1986: 
ProPAC, which is an indeperident commission 
established by Congress to' analyze and recommend 
changes in the prospective payment system
, recommended

in its second annual report that better information

about PPS be provided to benef iciari 
, hospitals, and
physicians. 
ProPAC also recommended that PRO review be 
extended to the overall episode of care
, including

skilled nursing facilities 
1 home health care and
outpatient surgery. 
The Rand Corporation: 
Kathleen N. Lohr, " 
Peer Review Organization: Quality
" July 1985: 
Assurance in Medicare,  Study focused on
the first scope of work for PROs' review of quality

PPS. 
during the first two years of  Rand recommended 
that the quality objectives in the 1984-1986 PRO

contracts be broadened to include the use of generic

screens and that quality review be given greater weight

activities. 
in the PRO review  Rand also.' recommended 
that the PROs I quality review be extended beyond 

hospital to include the Medicare beneficiary' s entire

episode of care. 
Current and Future Studies bv the GAO

The Financial Integrity Act Group at GAO is currently

reviewing the internal controls for payments by

Medicare intermediaries. As part of that study, the
 group is assessing the effectiveness of the SuperPRO as

a control mechanism for PRO performance. A draft

soon. 
report is expected  (Herb Danzler--Proj ect
Leader) . 
At the request of the Senate Aging Committee, the 
Program Evaluation and Methodology Division (PEMD) is

exploring how outcome data can be used to monitor

care. 
quality of  That study will include a review of
how PROs use , available data in their profiling of 
providers. A draft report is expected soon. 
(Eric Peterson--Team Leader) . 
At the request of the House Ways and Means Committee, 
the PEMD is also designing a study to evaluate the

issues. 
The study is 
PROs' handling of quality of care 
stage. 
still at the design  ("Jill Bernstein--Team 
Leader) 
At the request of the House Ways and Means Committee, 
the Human Resources Division (HRD), is undertaking a

program:
brief review of two aspects of the PRO 
analysis of the reasons for the lack of information

exchange between the PROs and other quality review

entities and an analysis of the variation among PROs in

their criteria, for 
determining the appropriate sanction 
(monetary penalty or excl 
penal ty  us ion) to recommend. 
The HRD is also contemplating an evaluation of HCFA' s 
program. 
management of the PRO  (Steven Fox--Proj ect
Leader) 
sumary of the 
See appendix II for a  OIG' s past and current work 
related to the PRO program.  APPENDIX V

THE PROS h"ITH MORE THAN on:: CON'!RACT* 
tional 
Addi 
Orcanization Name/Location  PRO Areas Reviewed

Professional Review Organization.  Alaska 
for Washington  Idaho 
Seattle, WA

Institute, 
West Virginia Medical  Inc.  Delaware 
Charleston 
Delmarva Foundation for  Inc  District of 
Medical Care 
Easton, MD  Columbia 
laii Hedical Services Association  Guam/American Sar. 
Honolulu, HI

Inc  Kentucky 
PEERVIEI': 
Carmel, IN

Health Care Review, Inc.  Maine
Providence 
, RI 
Iowa Foundation for Medical Care.  Nebraska 
West Des Moines, IA

New Hampshire Foundation for Medical Care  Vermont 
Dover , NE 
Montana-Wyoming Foundation for.  Wyoming 
Medical Care

Hel ena, 
*Note: Eight PROs hold two contracts; one PRO holds three contracts.  ;'.
APPENDIX VI 
COMPARISON OF 1984 SCOPE OF WORK TO 1986 SCOPE OF WORK

Revie  e2.  1984  1986 
Obj ectives  3 Admission Obj ectives  5 Obj ecti ves 
5 Quality Objectives
 Based on PRO data

All proposed and vali
 from first 90 days

dated by PROs. Very  of generic quality

limited areas for
 screen review. 
focusing obj ectives  HCFA- identif ied
outliers. 
Broader obj ecti ves 
Ra:;dom Samples  5% Admission Sample
 3% random sample

DRG Sample ranging
 (incl udes 1 and 2
day stays) 
from 3% to 100%

based on hospital
discharge size 
?rcad 
ission Review  5 Procedures proposed
 Pacemakers plus

by PRO
 procedures pro
posed by PRO

Pacemakers  100% retrospective
 100% preadmission

(see above)

Transfers 
From P?S to another
 Same but lower

hospi tal, exempt  level of review

uni t, swing bed

Readmiss ions
 All readmissions
 All readmissions

thin 
wi  7 days  wi thin 15 days 
Hedicare Code
 100% of 9 diagnoses
 Same 
tor 
Edi 
Focused  468  468, 462, 088 
(462 added during 
DRGs 
contract period)

Outl iers
 100% (reduced to 50%
 50% 
during contract
period) 
Percutaneous 
Not in contracts
 Review all claims

thotripsy 
Li  for percutaneous

Ii thotripsy in 
hospitals which

have an extra
corporeal shock wave

Ii thotripter  Revie\.1 Area  1984  1986 
Validation of 
Not in contracts
 Sample of one

Objectives  quarter  s dis
charges to validate

obj ective performan 
Notices 
Hospi tal  100% where patient
 Same 
or physician dis

agrees. 
100% where
patient is liable. 
10% of remaining

Specialty Hospital
 Proposed by each PRO
 15% of discharges

Rev i ew 
ission Pattern
 Discontinued during Not in Scope of Work

to ring  CO!1t:-act 
!1oni 
igger:  Trigger: 
ensified Review
 Tr 
5% or 3 cases
 5% or 6 cases

(whichever is 
(whichever is greater)

of cases reviewed
 greater) of cases

Review increased to:  reviewed. Review 
100% or subsets
 increased to: 50% or 
subsets (first quarter)
100% or subsets (two o

more consecutive 
quarters) 
Communi ty Outreach  Not in contracts  All PROs to propose

program 
*All cases reviewed

are subj ect to generic 
screens, 
qual i ty 
discharge review

admission review, DRG

validation, and coveragE
review. 
Source: 
HCFA  APPENDIX VII

SANCTI01; PROCEDURES  ND FLAGRANT 
FOR CASES OF GROSS 
AND S\.BS'!AN7IAL N\:!1BER OF SUBSTAl;TI  L VIOLATIONS 
chart sur..
The, follov,ring  marizes the steps that a PRO and the 
Department of Health and Human Services must follow in reviewing

the heal th care services rendered to Medicare beneficiaries by

individuals or entities and in determining whether statutory

occurred. 
violations have  Because of the extreme seriousness of 
gross and flagrant violations, the two-tier review process treats

those cases more expeditiously than cases of substantial number

of substantial violations.

SUBSTA  '!IAL VIOLATIONS IN A 
SUBSTANTIAL NGMBER OF CASES
 GROSS AND FLAGRJ.NT VIOlJ.TION 
Definition  Definition 
A pattern of care  - a violation of an 
has been provided that 
 obI iga tion has occurred in one 
inappropriate, unnecessary, or
 or more instances which prese

does not meet recognized profes
 an imminent danger to the

sional standards of care 
heal th, safety or well-being 

or is not supported by the
 Medicare beneficiary or places

necessary documentation of care 
the beneficiary in high risk

situations 
as required by the PRO. 
42 C.  R. Section l004. 1(b).  42 C.  R. Section 1004. 1(b). 
Step 1 - Written notice of a  Written notice of a potential
potential violation by the PRO.  violation by the PRO. Subj ect 
Subj ect has 20 days to request a
 has 30 days to ask for a meet i: 
meeting and/or submit additional 
and/ or submit additional

explain, clarify, 
material to explain, clarify, or
 material to 
resolve the potential violations.
 or resolve the potential viola
42 C.  R. Section 1004. 40.  tions. 
42 C.  R. Section 1004. 50. 
Step 
2 - PRO review of additional 
material submitted and/or the
 PRO review of additional

meeting. 
material submitted and/or the

meeting. On the basis of 
addi t,iona l information re
ceived 
, the PRO may affirm, 
modify, or reverse its

determination. 
42 C.  R. Section 1004. 50(c). 
Step 
3 - If the PRO determines,  it  PRO proceeds to Step 

that a violation has occurred 
attempts to develop a corrective

action plan, which is agreed to

by the provider or practitioner, 
to resolve the case.  ep 
4 - In those cases 
step 
unresol ved at  3, the PRO is 
to provide written notice to a

provider or practitioner that

possible violations have been

identified. 
Subject has 
30 days 
to request a meeting and/or

submi t additional material to

explain, clarify, or r sol ve the
potential violations. 
42 C.  R. Section 1004. 50. 
Step 5 - Second meeting between

the PRO and the provider of

practitioner and/or review of

addi tional material submitted. 
Step 
6 - At the conclusion of the aDove process and following 
consideration of all the information presented by provider or

if the PRO comes to the conclusion that a violation

practi tioner 
has occurred which should be the subj ect of a sanction action by

the Department, it is required to send its recommendation for

impos i tion of a sa ction and a supporting report to the OIG for 
independent review. The PRO must notify the individual or entity

that there is an additional opportunity to submit information

regarding the violations, and that this  ormation should be

OIG. 
sent to the  42 C.  R. sections 1004. 60-80. 
I S recommendations 
Step 7 - After reviewing the PRO  and any
practitioner,
addi tional material submitted by the provider or 
the OIG must determine: (1) whether the statutory and regulatory 
with; 
requirements have been complied  (2) whether an adequate 
sanction; 
legal and medical basis exists for imposing a  (3) the 
imposed. 
appropriate sanctioInt  ctoa nbe:   The OIG can exercise one of 
several options.  (1) sustain the PRO I S recommendation 
(3) rej ect the 
in its entirety; (2) alter the recommendation; or 
recommendation. If OIG fails to act within 120 days, an ex

clusion recommended by a PRO is automatically imposed. 
mpose on a provider or

The sanctions that the Secretary may 

practitioner following a PRO' s recommendation are either ex

clusion from participation in the Medicare program or, in lieu of

exclusion, the imposition of a monetary penalty as a condition

for continued eligibility to receive reimbursement under the

Medicare program. 42 C.  R. section 1004. 90. 
Step 
8 - A provider or practitioner who wishes to appeal the 
imposition of a sanction has the right to a formal administrative

hearing in accordance with section 205(b) of the Act. (42 U.

405(B). 
This hearing, conducted in accordance with specified
procedures (42 C. F. R. Part  de novo , and the 
405, Subpart 0), is 
entire factual basis of the case is presented to a Departmental
 (ALJ). 
administrative law judge  Each side may present evidence 
o cross-examination. The

and witnesses, and each has the right 
ALJ is required to issue a decision sustaining, modifying, or

dismissing the sanction , and setting forth findings of fact and

law. 
conclusions of  42 C.  R. Section 498. 74. 
Step 9 - Subsequent to the administrative hearing, review of the

re-
AqLJ udeciesiosn bty ethed Dep.ar tmental Appeals Council may be

42 C.  R. Sections 498. 80-95. 
Step 10 - After the Appeals Council renders a decision, a

provider or practitioner has a right to request judicial review

4 05 (g) . 
of the Department' s decision, in accordance with Section 205(g)
of the Act, 4 2 U. S . C. 
(Source: 
Summary taken from the transcript of Inspector General
Richard P. Kusserow I S testimony before The House Committee on 
Government Operations, Human Resources and Intergovernmental

Relations Committee. 
October 20 , 1987.  APPENDIX VIII

SUMRY OF RECENT PRO-RELATED 
LEGISLATIVE PROVISIONS

STATUS OF IMPLEMENTATION OF COBRA 1985 PROVISIONS

Assistants at Cataract Surqerv

Provides that no Medicare payment may be made be made

for assistant surgeon at cataract surgery unless

carrier or PRO approves use of assistant before

performed. 
procedure is  Also prohibits physician from 
knowingly and willfully billing Medicare beneficiary if

hej she has not 
obtained prior approval and where 
presence of assistant surgeon has been found to be

unnecessary. 
Instructions for PRO review of this activity, effective

with assistants proposed to be used after March 

1987  were issued to PROs on December 30, 1986 and 
review has been implemented. 
PRO Deniais for Substandard Care

Provides for denial of payment when a PRO determines

that the quality of health care services rendered to a

Medicare beneficiary fails to meet professionally

care. 
recognized standards of health  Also specifies
that denials for care of substandard quality shall be

made only on the basis of criteria which are consistent

wi th guidelines established by the Secretary. 
followed. 
Formal rUle-making process is being  Regula
tions are in the final stages of Departmental clearance

and will be published soon. 
PRO 100 Percent Preprocedure Review

Requires peer review on a preadmissionjpreprocedure

basis of non-emergency cases for at least 10 surgical

procedures. 
Second opinion will be required if PRO 
cannot make determination as to medical necessity of

services. 
followed. 
Formal rule-making process will be  Proposed
regulations are still in process of Departmental

clearance.  STATVS OF IHPI-EMENTATION OF OBRA 1986 PROVISIONS 
Review of services provided in hospital outpatient

depa  tments and ambulatory surqical centers  Effective 
for PRO contracts entered into or renewed on or after

January 1, 1987. 
This provision has been implemented 
by the Pennsylvania PRO, which entered a new contract

1, 1987. It 
period on July  will be implemented by 
other PROs as they enter their next contract periods. 
Review of hospital denial notice  Implemented on

statute. 
December 1, 1986 , as required by the 
FIs must provide PROs with "timel y" monthlY informa
tion, or hospitals will be recrired to provide such

Effective April
information directlY to the PROs 
1987. 
sar,ple of readmissions occurrina 
Review of at least a 
within 31 days o: discharqe and any interveninq post

care 
hospi tal  Effective for contracts entered into or 
1, 1987. 
renewed on or after January  This provision
has been impl emented by the Pennsylvania PRO and will 
be implemented by the other PROs as they enter their

next contract periods. 
oportion of' 
A reasonable p  PRO funds must be allocated
to 
revie\v of quality of care p ovided in all settinas 
The HCFA has no plans for separate implementation of

this provision. It will be implemented as part of

visions. 
other OBRA pr 
Review of HMOs/CMPs  The HCFA published a listing of

ti vely bid in  ster 
States to be competi  Federal Reqi 
1987.  the 
on January 5,  RFP for those States was issued on 
March 9 , 1987. Review began July 1, 1987.

The Secretary is to identify and make available to PROs

methods of identifyinq those cases which are more

likel y 
than others to be associated with substandard
cral i ty 
of care, and to provide at least 12 PROs with 
data and data processinq assistance to perform small-

area analvsis 
Both provisions effective upon enact
ment. 
is' an 
The first  ongoing activity. The HCFA is 
planning to contract with the American Medical Review

(AMC) for the 
Research Center  small-area analysis 
which will utilize feedback from 12 pilot PROs.

PRO boards must include at least one consumer

Effective with contracts entered into 
representati ve 
1, 1987. Officially 
or renewed on or after January  implemented with new contracts, but most PROs have

already implemented this provision.

PROs must respond to beneficiary complaints about poor

quality care provided in all settinqs  Implemented
1987. 
October 1,  PROs had already been required to 
respond to complsaoin.t s referred to them and will

continue to do  ClarifyinJ regulations in process. 
PROs will be required to share (when requested)

information ' State 
relatinq to substandard care with. 
licensure or certification bodies and with national

Effective April 1, 1987. Clarify
accreditinq bodies 
process. 
ing regulations in 
Hospitals. home health aqencies. HMOs and skilled

nursinq facilities will be required to have aqreements

with 
PROs. under which costs of PRO review activities 
are to be paid bY the Secretary to the PRO  Effective 
1987. Will 
October 1,  be implemented by the PROs as 
they enter their next contract periods. 
Source: 
HCFA/HSQB/OMR . 
Other PRO-Reievant Leqislation

The Medicare and Medicaid Patient Program Protection

Act of 1987 greatly expanded the sanction and civil

monetary penalty authorities under the Medicare and

Medicaid program. 
The bill also required the reporting 
of all disciplinary actions made by State medical

licensure boards.

The recently enacted 1987 Omnibus Budget Reconciliation

Act has made the following PRO changes: 
three-
year PRO contracts with staggered expiration
dates ; 
a ban on informing Medicare beneficiaries of

substandard quality payment denials before

offering providers the 
9Pportunity for recon

sideration; 
Federal Reqister 
of PRO work 
publication in the 

plans and evaluation techniques;

a prohibition of automatic renewals of PRO

contracts held by out-of 
state groups , provided
in-
state physician groups wish to compete;  a ban on physicians billing Medicare patients for

assigned claims denied for payment on grounds of

substandard quality; 
a requirement that PROs give special consideration

to the needs of "remote rural areas" in setting

review standards; 
mandatory onsi te review at 20 percent of rural

area; 
hospitals in a review 
mandatory meetings several times a year between

PROs and hospitals ' leadership;

assessment of access provided to Medicare

enrollees in risk-sharing HMOs and mandatory

beneficiary outreach;

a provision encouraging PROs to use physician

specialists in initial review of psychiatric and

rehabilitation cases;

demonstra  ion proj ects assessing the educational 
value of video communications between academic

medical centers and physicians treating patients

in rural areas;

administrative law hearings for rural physicians

or hospitals facing PRO-initiated exclusions from

Medicare; and

a report to Congress on the sanctions "due

process" reforms agreed to by HHS, the American

Medical Association, and the American Association

of Retired Persons.

Medical utilization Review, vol. 15 (no. 24),

Source: 
December 24, ' 1987.  -- 
APPENDIX IX 
Memorable "PRO-isms" 
We appreciated the candor and thoughtfulness with which in-

di viduals  program. In
responded to our questions about the PRO 
an effort to share more of those diverse opinions than could be

integrated into the main body of this report, we offer the

following examples of memorable opinions that we heard about the

(i. e. 
PRO program  "PRO  isms" 
In regard to quality review: 
"It' 
everyone agrees it is 
s like feeding the hungry

important, but no one wants to go down to the mission

and do it" (from a PRO CEO) . 
Quali ty Review is carried out too much and too long on

record.. . 
the written  the Government came in with a ball 
and chain and a meat ax" (from a medical community

representative) 
In regard to the generic quality screens: 
so good it' 
"They t re  s scary" (from a PRO medical
director) 
"They' re so frustrating, they make me scream" (from a 
PRO medical director).

In regard to the PROs' upcoming review of care in nonhospital

settings: 
re invading the last bastion ,
"They'  of individuality" 
(from a state medical board representa ti ve) . 
"They will see us like a lynching squad in the

communi ty and we will meet with hostility" (from a PRO

board chair). 
In ,regard to heal  settings: 
th care in rural vs. urban 
"Just because you' re out in the sticks, doesn't mean

you treat people differently" (from a PRO' CEO) . 
e in 
"You simply cannot practice the same medicin
Dallas, Texas. 
Muleshoe, Texas as  You can call that 
two-tier medicine or whatever you want" (from a medical

community representative).
 sanctions: 
In regard to the use of 
It' 
s important -to remember that most doctors are 
competent most of the time and all doctors are

incompetent some of the time" (from a medical community

representative) 
Once you get a sanction in doctors ' hands , their

follow" 
hearts and minds will  (from HCFA
representative) 
"We have a moral obligation to underaccuse more than

overaccuse" 
(from a PRO medical director). 
"The only way to affect physicians is through their

back pockets" (from a PRO board chair).

"They (sanctions) make us the most hated group in the

State. . . but they 
(physicians) read their mail now" 
(from a PRO staff person) .  D 
E
L
AI
1. f
O 
VIII-I:
E
R
Or. 
E
L5 
FAIU 11
x m
O U
N IIEVlfl-r.( U IIl'lS f A I  23. 41.0 33, 40.  :n. JI.9 J5. 32. 45. 27. 14.  J2. 
SCREE IAlllD;PIIOm, EI1 1..  7. 1.2  ?. I  I. 7 1.. 1.0 1..  3. '1  11.. 
q..  AP!'rNO I x x
 1IIIlf or Iii NI II II: 0111\ II V !lCml, NS: uI9Irll T or fAIUJlllo !' AND (IINf IIiHI II PIIIIII fl1!i 11911/, N AND CA!;t DV !iUIU.!.IIJOV 5 lIE  sa:rfN 5 SOIf(N 4 2 SCllffN 3 H\IUOX  m:Vlfl-fO x FAHIIIY. R(VIHJI:O x HrVILI-EO xRllnLl" fA H UII' f Allo IIOIILEI15 EDr M!i fAIL t UPIIOIIL  2... 1.3 17. 10. 21.) JO. '1. 1. 2 1!' I. I 23. 1.2 57. 31.J 21. :11. 1.0 3/.. IO. I'I. 1'1. II. 7 ,t - 5 42. 15. :12. 21.5 1.0 19. j.. 5 . II JI. 7 I. I i'? 1.5 1.4 '1. J 4. () J. '1 15. 1.1. 1111. 17. 2:1. b. 'I '111. 1.6 1.2  42. 22. 1.1 15.  a
 p.en. all re led the sc evip.wed that fai ds subsequently confi screen 'ailu med He.e es that  ns Dala Sum..uy. " July 1907.  " I

N N X!' h    o
PUICl SCREEfAILED AILED  31..  1.1. 17. 15.  11.0 10.  10. 1..  7.   .t disc t.1 ntage ofntaqe of qanilati
X f 9 ni nt piceee O 
  ne s    
seliE: EN I 11 R(VI( REVI(I-(U OfAILED PROnLEM5  91.. 1.. 100. 25. 100. 75. 40. 99. I?. 5 100. 100. 71. 100. '17  7J3, Ib3 4. I  of d h;chuge p I.cal stability of patihs i"fee t ions O" i a I n to su
hedule" retu ed In the hoa suffe wed fai Ie"" is the peoblems d P . is the peoble"5. ty p  Pee ReviewSOH/OHR 
PRO NUI0fR CASE SHIOV SitE  792 575 b4i! 124 . J02 140 1.2' 0'rI 171 . 3110 I 4 , J . S02 1b2 01'1 ,'1 J 1 571  TOTAL  teE V SCREEN I: A"equac SCREEN 2: MediDeatSCllf(N 3: SCIIE(N 4: N050CSCREEN 5: Un5c I.: SCREEN NOIE: . X RevieX faileas quali ee: Sou UCFA/H APPENDIX XI 
HCFA, ASPE, AAP 
AND 
COMMENTS ON THE DRAFT REPORT AND

OIG RESPONSES TO THE COMMENTS

We sent copies of the draft report to a wide array of entities

who participated in our PRO study and received comments from the

Administrator, Health ' 
Care Financing Administration, the
Assistant Secretary of Planning and 
Evaluation, the Acting

Executive Director, American Association of Retired Persons and

the Executive Vice President, American Hospital Association.

the sections that follow 
1 we offer their comments , either
sumary 
verbatim or in  form, and our responses to the comments 
HCFA COMMENTS

The following are HCFA' s comments in full and our response to it: 
General Comments

We have reviewed OIG' s draft report regarding the effectiveness

and utilization of quality care reviews conducted by the peer

review organizations (PROs) and agree with its content. The
report accurately reflects problems already identified by our

Standards and Quality Bureau 
Heal th  (HSQB). Planned and 
completed remedies, discussed by HSQB staff during the course of

their interviews with OIG staff, appear as recommendations in the

draft. 
Our specific actions for each of the findings contained in the

report are attached for your consideration. 
Specific Comments

Quali ty of Care:

A task force made up of PRO physicians and central and regional

office personnel has developed a model Quality Intervention Plan.
Its use will be mandated in the new Scope of Work which was

OIG. 
reviewed by  This Plan prescribes the following quality
review procesa: 
Problem identification/timing of review

Determination of source of problem

3 .  Assignment of severity levels (The HCFA severity levels will
be mandated.

Notification of quality problems
 Quarterly profiling and weighted severity scores

interventions 
Quali ty 
We have pilot studies and research efforts underway which shoulc 
future. 
assist us in assessing quality care in the 
Generic Quality Screens: 
A task force was set up by HSQB and during the period September

through November 1987 , meetings were held to develop consistent

definitions for the hospital inpatient generic quality screens. 
The task force was comprised of representatives from each HCFA

regional office 
, each PRO and centr l office staff. The revised 
generic quality screens are included in the new Scope of Work. 
We have solicited PROs to participate in several different pilot

studies. 
The PROs will be asked to develop review methodologies 
and submit to HCFA a study design which will include evaluation

criteria 
Entities: 
Coordination with Other 
Al though we are restricted, by law as to the amount of information 
we can share, we are attempting to remedy this situation. We

have a regulation in process that will relax restrictions and

provide for a larger exchange of information. 
Beneficiaries: 
PROs' Outreach to Medicare 
The current PRO Scope of Work requires PROs to describe the

methods they use to implement programs such as hotlines

seminars 
, brochures I and consumer representation on PRO governing
bodies. We collected and reviewed each of the PROs' community

outreach plans and developed a model community outreach program.
This model is included in the new Scope of Work. 
Review of Care in Nonhospi tal Settings:

In preparation for starting review of Intervening care between

hospi tal discharges , we worked with the PROs , as well as national

associations 
, in developing the review requirements. 
We have been working to provide the data needed to link

care. 
outpatient and inpatient  HCFA has developed the Medicare 
Automated Data Retrieval System (MADRS). The PRO is able to 
electronically query this system. . The PRO will provide the two 
hcoasprie .t al stay dates and the MADRS will ide tify any intervening

This system is currently being used by the Pennsylvania
PRO. In addition 
, the HCFA demonstration proj ects , known as the 
Common Work File (CWF), are being expanded. In addition to

combining the data from our carriers and intermediaries into
 single files in Maryland and Texas , we will add additional

jurisdictions to those files in FY 1988. We are also taking

steps to assure that all Pa  A and Part B claims for

beneficiaries assigned to CWFs, regardless of which contractor in

the country actually processes the bills , are incorporated into

the CWF. 
This will provide complete Medicare history and permit 
higher quality medical review. We are currently discussing how

the CWF might best support PRO review functions. We also have

plans to conduct several pilot studies to test alternatives for

settings. 
reviewing care in nonhospital 
Other Comments

Fin  lly, we would like to suggest that pages 48 and 49, C., Other
PRO-Relevant Legislation, be revised to incorporate the actual

legislative language. 
Parts of this summary, which was prepared 
Medical Utilization Review, are misleading and should be

bcy  larified. 
orG Res"Oonse 
We support the ini tiati ves that 
HCFA has undertaken to address 
issues raised in our report. However, we reiterate our

suggestion that HCFA "develop and widely disseminate information

on the full range of educational corrective action being taken by

the PROs to address quality of care problems. 
It is important

for all relevant parties to know the breadth of activities other

than sanctioning, in which the PROs engage. 
Furthermore, we would like to highlight the importance of HCFA

expedi tiously developing the confidentiality regulation referred 
to in its comments. In addition, HCFA failed to address' whether 
reauire 
PROs to report instances of physician
mit isi sintcenodinng tdo 
uct or. 
incompetence to State medical licensure boards,
which HCFA first agreed to in March 1986 in a briefing on the OIG

discipline. If 
report on medical licensure and 
the intention of 
the PRO program is to protect Medicare beneficiaries from

substandard medical practice, it is imperative that the PROs be

given clear direction to maximize coordination with other review

enti 
ties, especially State medical licensure boards and hospital 
committees. 
quality assurance 
It is worth noting that, since HCFA responded to our draft

report, it has informed PROs that "the physician-specific quality

concern (i. e., one or more confirmed quality problems) may be

disclosed . by a 
PRO, with or without a specific request, to the
hospi tal where the service in, question was provided.  As with 
the release of information to, medical licensure boards, we 
believe that the PROs should  required  to provide quality-
:be 
related information to hospitals and that HCFA should issue the

necessary regulations immediately. The PROs' confidentiality
 restrictions have been a festering issue that is worthy of

immediate attention.

I S 
Finally, while we coremend HCFA  efforts to develop a model 
commu  ity outreach plan, we agree with the AH I S suggestion that 
efforts. 
HCFA clarify the purpose of the PROs ' community outreach 
ASPE COH1-ENTS 
The Assistant Secretary for Planning and Evaluation echoed HCFA' s 
comments regarding the model community outreach program which

HCFA will be incorporating into the PROs' third scope of work. 
lude 
In addition, ASPE suggested that we revise our report to in 
reference to three quality-related studies that are currently

underway, which should help address concerns ti ted in our report 
I "failure to define and assess quality.

regarding the PROs 
OIG Res'tonse 
I S 
We appreciate ASPE  suggestion and have added an endnote to our
report in order to inc  ude re f erence to these three qual i ty
related studies. 
However, we would like to reiterate the 
importance of HCFA collecting and disseminating the findings of

su=h research to the PROs so that they can integrate it into

their review approach. 

AARP COMHENTS

RP' s comments in full and our response to

The following are 

them: 
Thank you for the opportunity to comment on your draft report

entitled ' 
The Utilization and Quality Control Peer Review
Organization Program: Quality 
Review Activities.  The 
Association appreciates your leadership to safeguard quality in

the Medicare program.  assurance 
We share your view that quality 
is of maj or importance to the integrity of Medicare. 
The Association is in general agreement with the content and

overall thrust of the report '  draft 
s recommendations. The 
report I s 
findings and recommendations, if carried out, will
enhance the nation I s abili 4Y to understand quality medical care 
it. 
and the role of the PROs in protecting 
More specif ically, tne Association agrees with the series of 
recommendations that are aimed at bringing greater uniformity to

the quality review process. In this connection, the report I
f indings 
on page twelve with respect to generic quality screens
are particularly noteworthy. From a national perspective AAP 
continues to believe that mandating the screens was an important

review. At the 
step forward in the evolution of quality of care  same time , we endo se the observation that the screens themselves
are in need of ref  application. 
inement and greater consistency in 
Moreover, a better understanding of what constitutes quality and

how to measure and report it are critical elements of the

unfinished agenda. 
PROs have a central role in the development 
of the health care quality assurance system for this country.
PROs should be in the forefront of quali ty assuranc  research 
because they represent, the Nation' s commitment to quality in 
medical care; a strategy of both strengthening PROs and holding

them increasingly accountable is obviously in the public

interest. 
The Association commends the report I 
S recommendations that HCFA 
promote closer working relationships between the PROs and other

, licensure
heal th  particularly state medical 
boardsca.re  Ientniti es 
an era of tight budgetary constraints the need to 
coo  dinate community services becomes even more important.
Better coordination is essential among bodies charged with

assuring that practitioners and providers of health care are

competent. 
Yet, with few exceptions, in most of the country,
professional societies, medical licensing boards, peer review

organizations, etc. 
, do virtually nothing to coordinate data 
resources, analysis, monitoring, and enforcement responsibilities

public. 
on behalf of the  The Association believes that greater 
coordination and communication between PROs and medical licensure

boards is essential to the development of an efficient quality

assurance system. 
AAP would give greater 
prominence to the recommendation that 
HCFA improve the availability of data on outpatient and inpatient

services. 
Parts A and B data must be linked so that policy 
makers have a better understanding of an entire episode of

illness. 
In addition, Medicare contractors must begin to use 
o that measures are consistent

common data processing systems 
and will yield comparisons of medical outcomes and provider

behavior. 
The Association is optimistic about the development of small area

variation analysis as a tool for identifying both costs and

system. 
qual i ty problems in the health care  Small area vari  ion 
analysis is key to understanding both costs and quality in

medical care. Describing these variations creates new ways to

care. 
assess the delivery of medical  Understanding the
variations in medical practice suggests opportunities for

reducing expenditures under Medicare and Medicaid without

care. 
reducing the benefits of medical  Moreover, the variations 
reveal the need to evaluate the outcomes of diffe ent approaches

to treatment so that patients and physicians can better

understand the significance of their choices in care.  year. 
Small area analysis of Medicare data begins this  The 
research should be a routine function of HCFA with the data

public. 
regularly reported to the  PROs will have an important 
role in organizing the physician community to help explain, the 
data and make judgments about what it means for both the costs

care. 
and quality of medical  The Association, urges the OIG to 
tool. 
help promote this important new 
We note your expression of caution with respect to the extension

of PRO review to post-hospital settings. AARP strongly supported

the Congressional decision in 1986 to mandate such review, and we

continue to believe that the PRO mechanism is the best hope for

creating a system that assures quality across the continuum of

care. At 
the same time we acknowledge the concerns regarding
untested review methodologies, data availabil i ty, and funding 
constraints. The Association seeks to work with all interested

groups and HCFA to maintain the review direction and momentum

established in OBP.A '  , as well as a measured and sensible pace 
of implementation.

I beneficiary outreach activi ties--
wi th respect to PROs  a matter 
of obvious interest and concern to AAP--we welcome your 
highlighting the continued deficiencies in this area. We 

perceive a growing desire within PROs to work with the

benef iciary community to improve communication and understanding. 
We will continue to do as much as possible to contribute ideas 
and resources to this effort , as well as make the case for

increased contract funds earmarked for this important purpose. 
AARP was pleased to have participated in the survey phase of this

proj ect 
both at the national and state levels. We greatly look
forward to reviewing your findings and recommendations in the

study. 
remaining areas of the  Thank you again for allowing the
Associat  report. 
on to comment on your draft 
OIG Res'Donse 
I S 
We appreciate AARP  support of the report, given its unique 
perspective on the PRO program. We would like to clarify that in

our report the order of our recommendations to HCFA parallels the

relate. 
order of the findings to which they  Hence, the fact that 
I S need 
our last recommendation relates to HCFA  to improve the 
availabili ty of data , on; outpatient and inpatient services, is no 
indication that we find  it less important than other

recommendations. We have modified the format of our

recommendations in our  executive summary to help clarify this

fact.  AHA Comments

ents in full: 
The following are the AHA' s co 
On behalf of its nearly 6, 000 institutional members, the American

Hospi tal Association appreciates this opportunity to comment on

report, " 
the draft  The Utilization and Quality Control Peer 
Review Organization Program  Quality Review Activities  , which

was prepared by the Inspector General' s Office of Analysis and

Inspections. 
This report sets for itself a very important task--
assessment of PRO performance--which has not yet been publicly

attempted, It is particularly timely, given the increases in PRO

activity mandated by Congress in 1986, for HES to step back and

analyze the direction the program has taken and to evaluate its

real and potential impact. 
We have reviewed the report carefully, and agree with many of

your recor.  endations. However I while the report does identify 
that there are differences among PROs, it provides little insight

into why those differences occur or whether the existence of

functions. 
differences constitutes a problem in how the program 
Apparent differences in effectiveness and efficiency may be due

I S 
to differences in each PRO  process of review, or they may be 
due to basic inefficiencies in the overall program structure. We

offer some specific questions about the actual conduct of review

in the detailed co  ents attached that might help to sort out the

sources of the differences among PROs. With respect to program 
structure, HCFA may have condemned some PROs to inefficient and

ineffective review by creating basically uniform contracts and

structuring contract deli 
verables around anticipated provider 
behavior rather than on problems in the area identified by the

PROs. 
We continue to feel that a great deal remains to be known 
about what is the most effective and efficient way to structure

the PRO contracts and to conduct the review. 
We also think it is important for HHS to give careful

interventions, 
consideration to the effectiveness of PRO 
o which the

including sanctions, and to consider the extent 
intervention is appropriate to the nature of the problem

identified by the PRO. . Education and consensus building may be

more effective and appropriate than sanctions if the ,problems 
identified reflect a lack of knowledge of Medicare standards or

lack of clearcut standards of care within the medical community. 
Finally, we believe it is important to consider the overall

effect of all PRO review , including utilization review, on the

quality of care provided to Medicare patients , not ;just on the

PROs. For 
actual quality review efforts of the  example, what has 
been the effect of the PRO priority to shift medical treatment

and surgery from an inpatient to an outpatient setting on the
 heal th and safety of Medicare beneficiaries?  This is something

we do not know. 
We look forward to future OIG reports which , we hope, will

here. 
address some of the issued we have raised  We hope these 
perspective. 
comments will provide some useful 
In order to make the following comments easier to follow, they

have been organized in 
o two sections: first, general remarks
about the purpose and methodology of the study and comments

relating to the bac :ground discussion, and second , comments on

specific findings. 
For ease of reference , we have for the most 
report. 
part used the subheadings that appear in the 
The purpose of the study

Our chief concern has to do with the limited extent to which the

research actually addresses the broad purpose announced at the

report. 
beginning of the  The research is intended to " assess the 
performance" of the PRO program and to "promote a better

understanding of the PRO I S miss ion and acti vi ties"  Al though
many of the report' s findings and recommendations seem quite

reasonable 
, the research does not really address the outcomes of 
review and does not analyze differences in PRO review methods or

interventions. 
I quality review and 
The discussion of the PROs 
adequate. 
intervention procedures on pp. 6-7 is not  The analysis
necessary t 
judge what makes some PROs more effective than 
others with respect to both their quality review processes and

their chosen intervention strategies is simply not there. 
As your review of the data reported by PROs on the use of generic

quali ty screening shows, there is wide variation not only in the 
level of referrals to physicians for review but in the number of

confirmed problems. This variation should lead to questions

about what the PROs' do differently to yield such different 
resul ts. For example:

What kind of training dO. they provide to the nurse
reviewers? 
Do they vary in the level of instructions they give to

the nurse reviewers who. perform the initial screening of
cases? 
Do they vary in the extent to which they build a

consensus for the standards of care among their physician

reviewers? 
se of specialists to conduct the

Do they vary in their 

physician review?
 Is a final decision about the quality of services made on

the basis of a single physician s judgment , or is the

I s 
physician reviewer  opinion bolstered by a second 
physician review, or by a quality committee review?

Wha  sort of system does the PRO have--telephone , or 
mail ?--to' allow discussion to take place between the

attending and reviewing physicians? 
Do the PROs vary in 
the way they perceive these discussions?

What sort, 
of interventions have been tried?  Have some 
failed? 
been tried and 
These questions were not . asked. As a consequence , the report
identifies that there are differences between PROs but provides

no insights on why those differences occur or whether the

existence of differecncoes ncondstiututces tae prdobl.em  in how the progra

functions--ei ther in how the contracts are structured or how the

review is actually 
The report mentions in passing (at the bottom of page 1) that

subse 
ROs' sanction process, on 
ent reports will focus on the

their ovetall activities and effectiveness , and on the HCFA

oversight of the program. 
How does the current report fit in? 
Is it intended as background only? Clearly a report on " overall 
activities and effectiveness" would come closer to answering the

ask. 
questions we would have you  But if you would defer these 
questions to future reports, the langua e you use to describe the

current proj 
ect is inaccurate and should be changed. 
If an overall assessment of PRO performance is contemplated 
, we

would make two other general suggestions. First, it should

review. 
address both the results and the processes of  As with 
any kind of quality evaluation, whether of hospital performance

or of PRO performance, outcomes should not be evaluated without

involved. 
looking at the processes  Observed differences in 
outcome must be accompanied by an evaluation of differences in

process if they are going to deepen our understanding of what

results. 
needs to be done differently to achieve better 
Second , an evaluation of the results of PRO review should i clude
not ' only a comparative assessment of the number of PRO findings 
of quality problems, but an evaluation of their appropriateness.
In addition, if HHS really wants to assess the impact of PROs on

the quality of care provided to Medicare beneficiaries, it must

review: is 
consider the effect of PRO utilization  the shift from 
effective? Is 
inpatient to outpatient services always safe and' 
it better for the Medicare beneficiary?
 p. 

The studv ' s methods 
A study' s mietth.o ds 1 imi t the concl us ions that can 1 eg i tima tely be
drawn from , Because this study relies most heavily on 
interviews to evaluate the program, several of its findings

perceptions 
of PRO performance rather than PRO
prefelecrt fboroard m
 ance i tsel f. 
In several places , th  language used to summarize the results of

findings. 
the research somewhat overstates the  For example , the
finding, that " 
study concludes , in its first 
I missioqnu aanlidt y review is
the most critical element of the PROs  has received 
increased emphasis during the second contract period. 
It is

certainly true that quality review has been given more emphasis

in the second contract period , as shown by a comparison of the

work. 
two scopes of  But the research shows that quality is
perce i ved  as the most important function, not that qual i ty review 
the most critical element of the PROs' mission. Our 
perception is that quality has received more emphasis in the

is still 
second contract period, but that the PRO workload 
review. 
predominantly concsrned with utilization  Therefore, it
is questionable y.'hether it can justly be said that "
the most important function of the PRO program.  quality"
Similarly, in its analysis of the generic quality 
screens, the

"finding" is that the generic screens are useful, but are in need

of refinement in that they produce excessive referrals to

applied. 
physicians and may be inconsistently  While this may
well be true 
, the research merely shows that this is the way the 
screens are perceived by' PRO executive directors and others
interviewed.  well. 
This is our perception as  But it would be 
nice to see these perceptions reinforced by some evidence. 
Creation of the PRO Proqram

In the historical discussion on  2, you suggest that the PRO

program was created simultaneously with the PPS. Al 
though it is
true that the PRO program might not have been implemented without

the implementation of PPS, the PRO program was conceived and

legislated as an administrative reform six months before PPS was

passed in March 1983. 
The PRO reforms were introduced with TEFRA in September 1982. 
The PRO reforms streamlined program administration by creating

statewide organizations and enhanced their accountability to HCFA

by moving from a grant structure to perfo  ance-based contracts. 
The PPS legislation, passed in March 1983, gave a particular

focus to PRO review , based on the fear that ' such a sig ificant 
shift in provider payment might result in incentives to

ways. 
compromise the Medicare program in new 
Quality review had
always and continues to be concerned with poor "technical" 
qual i ty. In addition , under cost-based per-diem payment, concern
 p. 

focused on the quality implications of providing too many

services; under PPS, a per case payment system, the quality

concern shifted to the potential underprovision of services. 
The initial scope of work 
I as you point out I was based on HCFA'

anticipation that under PPS 
, Medicare expenditures were

vulnerable to provider attempts to increase revenues by

manipulating admissions. 
The first contracts therefore called
for reductions of nearly 1. 
25 million admissions, nearly half of
which were to be shifted from inpatient to outpatient care.
During the second and third years of 
PPS, as length of stay

declined 
, congressional interest turned to a perceived problem of
premature discharge" 
, and the budget bills passed in 
. 1986
contained several provisions enhancing PRO review of this aspect

care. 
of quality of  In the meantime , HCFA was preparing the
scope of work for the 1986-1988 contracts. Admissions had begun

to decline before PPS was implemented and continued to drop

sharply despite the analysts 
I expectations that the opposite 
would happen in response to PPS. 
Consequently, HCFA apparently 
fel t  it could reduce the level of review of utilization problems

in the 1986-88 contracts and shift some of its focus to "

quality" 
concerns. 
This historical overv'ievl raises an issue important to
understanding the program: to 
what extent are the PRO contracts 
structured around problems identified by the PROs in the course

of their review or through original research? 
To what extent

the structure of the review system dictated by hypotheses about

the effect of the financing system on hospital behavior? 
And how

sensitive has the program been to the political climate? 
As you

point out on  6, HCFA has had to respond to a variety of

competing pressures in implementing this 
program. 
HCFA was forced to put a great deal of emphasis on the review of

premature discharge" 
, because there was a growing public
perception that this was becoming a problem under PPS. 
Yet HCFA-
sponsored research by several PROs found that the actual level of

premature discharge is quite small. We understand that

forthcoming research by the OIG confirms this view.
' It remains

to be seen whether this research will lead to significant change

in future PRO contracts or ' whether the contracts will continue to 
be driven by the potential for problems under prospective 
'payment
even though that potential has not been realized. 
An examination of this issue might demonstrate that the PRO

program could be more effective and efficient if the PRO

contracts were based more on identifying and resolving local

problems than on testing perceptions about provider behavior.
One of the great advantages of the contract structure created by

the statute is the flexibility it gives HCFA to adapt each

state. 
contract to the problems found in each 
What has been 
implemented, by contrast, is a very uniform program. 
 p. 

cognize that such uniformity has 
its advantages--for example

generated. 
in the consistency of information 
HCFA may believe
that it has sacrificed the principle of local determination for

more efficient program administration and greater programmatic

in fact, the choice to create a uniform

consistency  when 
contract may lead to much wasted effort and a less effective

program. 
The Sanction Process

As mentioned in your description of the sanction process on 
p. 7

sanctions are issued for violations of the provider' 
s obligations

to providtee s;e rvices which are (a) medically reasonable and
appropr ia  and (c) of appropriate
(b) provided economically 
quali ty. The appropriateness of a sanction thus hinges on the

appropriateness of the determination that a violation of these

obligations exists. 
Is it appropriate for a sanction to be issued if there was no

willful 
attempt to provide inappropriate care or to circumvent
PPS? 
The sanction process  like all penal ties , serves not only
as retribution for an inappropriate action but as a deterrent to

actions. 
all those who might contemplate such 
But one might
wonder whether it is appropriate to sanction someone whose

inappropriate. 
actions are unknowingly 
Certainly it is true that
sanctions will not have a deterrent effect on anyone who is not

willfully engaged in wrongdoing. 
If the PROs are enforcing standards of care that all physicians

and hospitals know beforehand 
 then penal ties for failure to

conform to those standards might be appropriate. 
But if the PROs
are enforcing standards which are 
controversial, or which are

different than those that prevail in the 
community, then the

appropriate step is education and consensus development
sanction.  , not

HFCA repeatedly assures the provider and practitioner 
communi ties that PROs are not attempting to change standards of

practice. 
Let us assure you , this is not the way the PROs are
perceived by providers. 
In its regulations governing the

program, HCFA requires the PROs to use national 
criteria, norms

and standards rather than local standards, unlike the statute,
whose emphasis is precisely the reverse 
This emphasis of

national over local" criteria and norms would likely be confirmed
by an analysis of the criteria sets used by PROs to '
col1duct
review. 
Finally, as a factual matter, the statute allows for both

monetary and exclusion sanctions
, but does not allow monetary

violations. 
sanctions to be imposed for documentation 
This needs
to be sorted out on 
 ). 

FINDINGS 
Im=  ovement in Oual itv Sc eens and the Process of Review 
The discussion (p. 12-14) of the adequacy of the generic quality

screens is misleading, since the research attempts to identify

perceptions: of their effectiveness rather than to establish just

how sensitive (how well the screens correctlY identify problems)

and specific (to what extent the screens incorrectly identify as

problems cases which are not problems ) the screens are. 
The comparison of screen failure and confirmed problem

identification is particularly confused. The 
questions you need

to address are' (a) whether the screens allow quality problems to

and  they identify are truly
(b) whether the cases 
escape notice 

problematic. It has been well known that one of the biggest

problems with the screens was the large number of unnecessary

referrals to physician review (" false posi ti ves" 
The American

Medical Peer Review Association has made many suggestions for

refinements to the screens that make them more accurate
, and HCFA

has adopted many of these suggestions. 
variation: first 
There a=e two potential sources of 
, the amount
of d scretion granted to the nurse reviewer to identify

second, the judgment of the reviewing

potential" problems 
 cases. 
physician in confirming true problem  One PRO may refer
few cases to physician review, but, the ones referred may be the

right cases (i. e. 
, the ones reflecting real quality problems) .
Another PRO may refer many cases to physician review
, but confirm

portion. 
problems in only a small  The two may identify the same
percentage of confirmed . 
cases in relation to overall cases 
would appear to use the screens more

reviewed, so the first  PRO 
efficiently than the second., You have (a) defined the percentage

of confirmed cases in relation to screen failures rather than

cases reviewed; and 
(b) expressed these as ranges without
attempting to establish the relationship, if any exists, between

screen failures and confirmed problems. 
The comparison of screen
failuras to confirmed cases as presented here is therefore not

meaningful. 
To truly assess the differences among PROs in the use of the

screens , you . have to investigate precisely how they conduct the 
final 
initial screening and the  deci.sions. How much instruction 
do the PROs give the initial reviewer? How many physicians

review a case to determine whether a confirmed prqblem exists?

Do the PROs use specialists in a related field of medicine to

make these determinations? The answers to these questions could

provide useful ,
insights- into just how effective and appropriate
the generic screens would be as' the  quality
review process. You point out in thbeas is dfoir tshec  ussion onPRO   pp. 17-18

that there is wide variation among PROs in their review processes

and approaches tn r  but without any detail comparing the 
vi  differences in process to differences in result, it is difficult

to say just what this shows about PRO performance. 
The Definition of Oualitv and the Technolocry of Ouality Review 
The report concludes that PRO quality review is limited by a

"lack of consens s regarding the definition of quality medical

care" 
, by the lack of available resources to provide such care 
and by lack of sophisticated technologyt rtuo eas.s ess quality. These
conclusions are widely ' 
perceived to be  However, this
discussion, which covers very broad territory in a mere four

paragraphs, contains an alarming confusion of key concepts. 
First, it is certainly true that quality is multifarious, that

, that no single element will capture all the dimensions of

quali ty. While it might be useful for HCFA to consider what

purposes--whose interests--they hope to serve in doing this

review , it is not at all clear that this is a particular handicap

to the PROs. 
uali ty review function has been limited to 
The PRO 
determining whether a particular course of treatment can be

considered to meet  appropriate standards of quality"  The real

is the lack of definition at the "
handicap the PROs face  micro"
level 
-lack of consensus on what consti tutes quality care in the
individual instance--because these "appropriate standards" are

not articulated. 
Second, in discussing qual i ty measurement 
, the report suggests
that PRO access to seve rty of illness measures and

sophisticated quality assurance techniques" would eliminate some

of the current labor- intens i ve, .case-by-
case review process.
This is only partially true. These techniques can prove useful

for more eff  review. 
icient screening to select cases for  The 
generic quality screens require extensive chart review in the

screening process, which is , indeed , time consuming. But there 
is no magic to the use of statistical techniques or a severity of

ure. 
In the end, the basis, 
illness mea  of PRO quality assessment
must remain the detailed review of medical records, because it

only by looking at the medical record that a judgment can be made

about whether the care provided a particular patient was

appropriate. 
Relationships between PROs and: other Health Care Entities 
The study rated the PRO CEO' s perceptions of the effectiveness of

I S 
the PRO  relationship with other state health care
organizations 
, including state medical licensure boards, hospital
associations, and medical societies, and found that PROs

generally believe their relationships with hospital associations

good. 
and medical societies are  While interesting observations
the interviews revealed two issues of importance to hospitals and

physicians which didn 
t appear in the findings summary, but which
are well worth pursuing.
 regard to hospitals 
wi th  I the PROs reported that hospitals had
been troubled by the PROs I unwillingness to share information 
about particular physicians who had been identified as having

i ty probl ems. 
This has become an enormous ly important issue
qual 
for hospitals.

HCFA argues that current regulations prohibit PROs from

identifying physicians with ' 
confirmed quality problems to their 
hospital medical staffs. PRO regulations at Sec. 476. 133 allow

disclosure to an institution of information about its physicians

only "to the extent that the information displays practice or

performance patterns" of the physician within the hospital. 
HCFA 
contends that confirmation of a quality problem does not

constitute " patterns 
" of performance, and therefore cannot be 
The institution is therefore notif 
disclosed to the hospital.  ied 
only if the problem is so severe that a sanction recommendation

is forwarded to the OIG or I indirectly, the PRO undertakes

intensified revie  of the physician. Yet without proper

notification, the hospital medical staff can not implement

internal meas res necessary to address the physician I s practice
proble!:.. 
The AHA has recommended that HCFA change these regulations to

allow PROs to notify the hospital' s medical staff whenever it

practice. 
discovers a quality problem with physician 
Better coordination with State medical review boards is certainly

important when the PRO uncovers egregious quality problems that

safety. 
threaten the public health and  But it is equally 
important that PRO intervention begin early, while it is still

possible  o effect some change in practice I before invoking an

investigation by the medical license boards. 
Beneficiary Outreach

OIG found that PROs have trouble providing effective outreach to

Medicare beneficiaries. This observation is not surprising given

HCFA' s failubree t.o  define what the purpose of this outreach 
supposed to  Is the outreach supposed to improve beneficiary
understanding of Medicare coverage? 
Or increase beneficiary
awareness of. appeals 
mechanism for lodging a protest of a 
hospital notice of noncoverage? Or some other purpose? ' The OIG 
recommendation--that HCFA seek ways to strengthen beneficiary

specific: 
outreach efforts--should be made more  the best way for 
HCFA to strengthen PRO outreach programs is to clarify their

purpose. 
Review of Non-hospi tal Car

The study concludes , after interviewing HCFA staff and others

that the maj or obstacles to review of non-hospital settings will

technology, lack of 
include lack of data, inadequate review  Second , consider whether PRO effectiveness and efficiency might 
be enhanced by making the structure of the review program conform

more to local problems as determined by the PRO than by

assumptions about provider behavior based on an analysis of the

payment system. 
Thirdj in evaluating differen es in PRO effectiveness, consider

effect. 
both the level of effect and the appropriateness of the 
It is important to look at the overall effect of all PRO review, 
including utilization review, on the quality of care provided to

Medicare beneficiaries , not just on the actual quality review

efforts of the PROs. 
Finally, in evaluating PRO effectiveness, consider the range of

al ternati ve interventions at the PROs I disposal, and assess 
whether the interventions--including sanctions--are appropriately

identified. 
geared to the level of the problem 
O!G Response:

We appreciate the time and effort that the AHA put into crafting

report. 
its detailed and thoughtful response to our draft  The 
AHA has raised several important issues about the PRO program

xploration by both HCFA and other

that are worthy of further 
 ourselves. 
entities like 
In regard to the concerns raised about our  methodological
study' 
limitations, it is worth noting that although our assessment of

the PRO program is based primarily on interviews with a wide

variety of well-informed individuals closely associated with the

PROs, wherever possible , we have' supplemented our analysis of

this qualitative date with available quantitative data. 
Unfortunately, as reflected in our third PRO inspection report,
the limitations of available data hampered our ability to draw

def ini ti ve 
concl us ions about PRO effectiveness. 
In response to legitimate methodological and format concerns

raised by AHA, we have reworded our first two findings and

al tered the 
presentation of recommendations in the executive 
summary to correspond with the groupings in the text. We have 
also changed our introduction to better reflect the

methodological limitations of the ' 
study, and to clarify how our 
together. 
three inspection reports fit  In addition, we have 
I s 
included AHA  historical and sanction clarifications in the 
report. 
appropriate parts of the PRO background section of our 
As reflected in our response to HCFA' s comments , we also agreed 
wi th AH I S suggestions: that HCFA clarify the' purpose of community
outreach and facili tat' 
the PRO.5 release of quality-related 
, AH' 
information to hospitals. Furthermore  s concerns about the 
prescriptive nature of the PRO contracts are addressed in our

entitled: 
third PRO inspection report  "The Utilization and  Quality Control Peer Review Organization (PRO) Program:
ss" 
Exploration of Effectivene. (OAI-01-88-00572). 
Finally, while we appreciate AH' s careful criticism of our

report and support of our recommendations, we take exception with

its assessment that our PRO inspection 
fails in any meaningful

way to assess PRO performance.  We believe that the three

reports of this inspection provide more evaluative information

about the PRO program than any other entity has compiled to date.