"\' 
., 
Department of Health and Human Servces

OFFICE OF

INSPECTOR GENERAL

EDUCATING PHYSICIANS RESPONSIBLE

FOR POOR MEDICAL CARE: A REVIEW

OF THE PEER REVIEW ORGANIZATIONS'

EFFORTS

.. SERVI(' 
.. . U: 
Richard P. Kusserow 
INSPECfOR GENERA 
OEI-Ol-8-00 
cf:fa  . . . . . . . . . . . . . . . . . . . . ..  ..  ...  . . . .  ...   ...   ... ...  . 
. ..  ..  ...   ....    .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     .....     ..;..     .....     .....     .....     .....     ....    ....    ...   ...  .  ..  ..  ...  . . .. 
TABL.  OF CONTENTS

EXECUTIVE SUMY 
INODUCTION. . 
...................................................... 3

FININGS 
RECOMMNDATIONS 
COMMNTS ON TH DRA REPORT 
APPENDIX A 
Methodological Notes 
APPENDIX B 
Sample Overvew.  . . . . . . . .. B
APPENDIX C 
Summary of Physician Profie Data. . . . . . . . . . . . . . . . . . i . . . . . . . . . . . .. C
APPENDIX D 
Generic Quality Screens. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .'  ..  ..  ..  .' . D
APPENDIX E 
Summary of Intervention Profie Data  . . . . . . . . . . . . . . . . . . . . . . . . . . . .. E
APPENDIX F 
Detailed Comment . on the Draft Report 
and OIG Response to the Comments. . . . . . . . . . . . . . . . . . . . . . . . . . . 
.. F
APPENDIX G 
Endnotes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . G EXECUTIVE  SUMMAR 
PUROSE 
In this study, we assess me educational interventions that the Medicare-funded Peer 
Review Organizations (PROs) direct ' to physicians responsible for serious quality-of
care problems. 
BACKGROUN 
A major objective of the PROs is to oversee the quality of care rendered to Medicare 
carrg 
beneficiaries. In  out this objective, they have two major functions: (1) 
identif and confirm physicians and/or health care entities responsible for quality-of
care problems, and (2) to direct follow-up actions to the responsible parties. 
In this report, we address the latter function. We examine educational interventions 
because the PROs rely heavily on them as corrective actions, and we focus on the 
most severe problems (those the PROs assign a Level III) because the PROs 
determined that those problems involved adverse effects on patients. For a sample of 
eight PROs
, Vfe reviewed all such problems confirmed between January 1 and June 30
1990. We present our fidings around 
four key questions concerning the 131 
physicians identifed in our sample. 
FIINGS 
Who were the physician identied by the PROs as being responsible for serious
quaty-of-ce problems? 
Twen-eght peen of th physi were in genl practie, 60 peen were betwee 
ma. 
45  yea o  peen were  State in whih the physins
an  64  and  98  geInn thl  
peen of aO physin.fwere in  peen  and 
practied  17  practe,  32  betwee  45 
peen ma 
64 
an  83
idle  
At  th peen of th  131 physins ha more thn one quli-of-care proble
by the PROs. 
What kids of problems were they' responsible for? 
physi  resg 
Eight peen of th  were resnsle for  care  in serus or
potell  deii  gn( 1)figs le unesed
serus compliatins abnorml 
, or  (2) 
(3) both 
What kids of educational interventions did the PROs diec to thes physician? 
edatinal 
PROs diecte no  peent of the physins. 
Th  inerentins to  14  physi  edatina inenn, th 
Amng th  who di receie an  PROs used lett 
call coures, and metigs mo oft 
Fort-two percent received a letter. 
Thirt percent received a telephone call. 
Twenty-two percent had to attend a continuing medical education course. 
Eighteen percent had to attend a meeting with PRO physicians. 
Hosls tyal  wi  the inenns. 
were not involved  or aware of 
Fif-one percent of the hospitals were not inormed that a quality-of-care 
problem had occurred there. 
Eighty-eight percent of the hospitals where the quality-of-care problems 
occurred ha 
no part in the educational intervention. 
What conclusions ca we drw from the PROs' educational interventions? 
th inentins  unerin 
edatinql valu of  is 
Th 
The letters are tyically brief and contain more case summary than educational

material.

The calls are not documented enough to determne their content.

The available courses often do not address the specific quality-of-care problems

the PROs have identifed. 

The meetings with the PRO physicians have the most educational value 
according to both the PROs and the relevant literature, but were used in only 
18 percent of the cases we reviewed. 
PROs confrnt thee bas constrin in  edatinal
seekig to carr out efectie 
iTh nenns. 
They have limited information on which to base their educational interventions. 
They perform oversight roles that inhibit their capacity as educators. 
They find that remedial medical education programs are seldom available.  REMMATIONS 
The Health Care Financing Administration (HCF A) should ensure that the PROs 
initiate timely educational interventions in all confirmed Level III cases. 
The HCF A should issue a regulation or, if necessary, seek legislation mandating that 
for all confirmed Level III cases, the PRO share case information with the hospitals at 
which the problems occurred. That information should include the identity of the 
physician, the nature of the problem, and the tye of educational and other 
interventions imposed by the PRO. 
The HCF A should issue a regulation or, if necessary, seek legislation allowig and 
encouragig the PROs to involve the hospitals in the planning, conduct, and 
assessment of educational interventions in all Level III cases. 
The HCF A should instruct the PROs to give greater attention to personal meetings as 
an educational intervention and to explore ways of conducting one-to-one meetings 
between physicians with confirmed Level III quality-of-care problems and physician 
consultants. 
The HCF A should mandate that before directing an educational intervention to a 
physician in a level III case, the PROs take into account all prior quality-of-care 
problems they have identified concerning the physician during the life of the PRO 
contract. 
The HCF A should obtain and disseminate information on the effectiveness of PRO 
educational interventions. 
The HCF A should obtain and distribute to all the PROs the listing of 
focused/remedial education programs included in the national registry maintained by 
the American Medical Association. 
The Public Health Servce, through the Area Health Education Centers Program 
and/or the Agency for Health Care Policy and Research, should provide 
demonstration funding for the establishment and refinement of medical education 
programs that seek to enhance the clinical competence of physicians through 
individualized assessments and remedial education. 
COMMNT 
Within the Department of Health and Human Servces, we received comments from 
HCF A and PHS. The HCF A agreed with the fist recommendation, indicated that it 
would seek legal guidance concerning the second and third, offered to consider the 
fourth, suggested that the PROs' fourth scope of work addresses the fifth , and 
commented on more generalized education efforts concerning the sixh.  In response to HCF A's comments, we changed the second and third recommendations 
so that they call for HCF A to issue a regulation or, if necessary, introduce legislation 
to further PRO-hospital interaction in Level III cases. We also amended 
the recommendation urgig PROs to take into account al prior quality-of-care 
problems before makig an educational intervention. In that recommendation, we 
specified that we were referrng to Level III cases and to problems identified by the 
PRO durig the life of its contract with HCFA 
The PHS, in relation to the comment we directed to it, suggested that the Area 
Health Education Centers (AHC) Program might be a more appropriate vehicle for 
funding demonstrations than the Agency for Health Care Policy and Research. 
Accordingly, we added the AHC program to our recommendation. 
Outside the Department, we received comments, generally supportive, from the 
American Association of Retired Persons (AAP), the American Medical Peer 
(AM). 
Review Association (AMRA), and the American Medical Association 
response, we made some minor clarifyng changes in recommendations and added a 
new recommendation callng for HCFA to keep the PROs informed of 
focused/remedial education programs included in the national registry of such 
AM. 
programs maintained by the  INTRODUCTION 
PURSE 
The purpose of this study is to assess the educational interventions that the Medicare-
funded Peer Review Organitions (PROs) direct to physicians responsible for serious 
quality-of-care problems. Toward tha  end, we address four basic questions: 
1. Who were the physicians identified by the PROs as being responsible for serious 
quality-of-care problems? 
2. What kids of problems were they responsible for? 
3. What kids of educational interventions did the PROs direct to tnese physicians? 
4. What conclusions can we draw from the PROs' educational interventions? 
MEODOLOY 
Our methodology is based on a stratifed random sample of eight PROs (appendix A), 
for which we reviewed all confirmed Level III quality-of-care problems involving 
physicians identifed between January 1 and June 30, 1990. The Health Care 
Financing Administration (HCFA) defines such problems, which involved 131 
physicians, as involving "medical mismanagement with significant adverse effects on the 
patient.  We supplemented our review by conducting discussions with admistrators 
and physicians associated with the eight PROs. 
BACKGROUN 
When the PROs were established in, 1982, their mission was almost completely focused 
on controllng Medicare costs. Today their mission is much broader and includes 
major responsibilty for overseeing the quality of care rendered to Medicare 
beneficiaries. The PROs carr out this responsibilty by reviewig inpatient medical 
records against standards of established professional practice. These reviews 
constitute about 15 percent of all Medicare admissions to hospitals. 
When the PROs identify what appears to be a quality-of-care problem, they conduct a 
second and sometimes third review. If at this point the PRO physician reviewers 
agree that a likely problem exists, the PROs invite the physicians and/or hospitals 
involved to provide more information. Upon further review, the PROs either confirm 
the problem or overturn their previous findings.  As a part of this process, the PROs determine the degree of harm done the patient 
and, in accord with HCF A instructions, assign a severity level and weight value. 
Incidents involving no potential for harm are assigned a Severity Level I, with a weight 
of 1; those with a potential for harm are Level II, with a weight of 5; and those with 
actual harm are Level III, with a weight of 25. HCF A defies harm, or adverse 
effects, as prolonged treatment, readmission, impairment, disabilty, or death. Durig 
our sample time frame, the 53 PROs completed 1,055,964 reviews and identifed 9 620 
confed 
quality-of-care problems: 4 217 Level I 711 Level II, and 692 Level III. 
, 4,
At the end of each quarter, the PROs, in accord with HCF A's profilig requirements 
havig 
must total the weighted severity score for each physician they have identifed as 
l The 
one or more.quality-of-care problems. interventions the PROs then take are 
based on this score.2 For physicians responsible for a Level III problem, HCFA 
requires that the PROs invoke an educational intervention, conduct an intensifed 
, consider sharing the information with the 
review of subsequent hospital admissions
medical licensure board, and consider recommending a sanction to the Offce of 
Inspector General (OIG).3 In addition, HCFA requires the PROs to notif the 
confired quality-of-care problem in wrting and to take an 
physicians of the 
, such as requiring predischarge or preadmission approval. 
additional intervention
The educational interventions, which are the focus of this report, involve a range of 
possible actions. For example, they may involve coursework, suggested medical 
readings, or discussions with PRO physician advisors. These interventions may be 
taken singly or in any combination. The HCFA allows the PROs to determine the 
most appropriate tye of educational intervention for each physician. 
Overall, these educational interventions are a vital component of the PROs' quality 
assurance efforts. The HCF A mandates them for Le:vel III problems and for Level II 
problems with a score of 10 or more. And they far outnumber the more punitive 
interventions involving referrals to the medical licensure boards and the OIG. Yet 
little is known about the use or effectiveness of the educational interventions the 
PROs undertake.  INDINGS

physici 
Who were the  identied by the PROs as being responsible for serious
quaty-of-ce problems? 
Twen-egh peen of th physi were in gen  peen were betwee 
prctie, 60 
45  peen were male  State in whih the physins
pracn  t 64  yea op~ ean en of aU98  physins were in geIn nth   peen 
17p  een ma  prctie,  32  betwee  45 
64, 
an  83
4 for all but one of the 
We identifed the specialties 131 physicians in our sample. 
These 130 physicians represented 22 specialties which we grouped into four tyes:
general practice, medical specialties, surgical specialties, and osteopathy (appendix C). 
In comparing their distribution among the four tyes with that of the non-Federals 
physicians in their States, we found more physicians responsible for serious quality-of
care problems to be in general practice and less to be in medical specialties than 
would be the case if their distribution mirrored that of the 8 States (figure 1). The 
difference was statistically signficant. 
FIGUR 
PHYSICIAS WIH SERIOUS QUALY-OF-CARE

PROBLEMS AND ALL PHYSICIANS IN 

STATES, BY SPECIALTY 
Percent 
50% 
40% 
40% 
30% 
20% 
10% 
General Practice  Medical Specialtes' Sureical Specialties  Osteopathy 
Specialty 
Physicians, 8 States  Phys. w/QOC Problems 
idfid 
NOTE: N=79 581 non-Fedeal paJ-care physiia in  Sta an N=130 physiia  by  PROs as resnsible for a

confi Lel III qu-of-care proble an repned to HCFA for rAevs sciomlp led 1/1190-6/30190.
 A.an

MSOeUdiRaCl DEi:r OecItGor/vO:E PI haynssiiia o ifn  dtoh f rUm nid SPtRaO.s 3;2 A  emd. (c1a99 0M) eand iaPl hysiia ChDaiavcint eo fa Snu rD aisnu tDna  (1Re9so9u0rc)es;  

Amean Osteopath Asocian,  Yearbook an Directorv of Osovath Physiia (1990).
 We also compared the gender and age distribution of the physicians responsible for 
serious quality-of-care problems with the non-Federal physicians in their States. 
found a smaller proportion of female  percent 
physicians than in the 8 States (2 
compared with 17 percent), and we found fewer physicians either younger than 35 or 
older than 65 and more between 45 and 64 than in those States (figure 2). These 
differences in distribution were statistically signcant, too. 
FIGURE 2 
PHYSICIAS 1rH SERIOUS QUAL-OF-CARE

PROBLEMS AND ALL PHYICIAS

IN 8 STATES. BY AGE

Percent 
40% 
35% 
32% 
30% 
24% 
25% 
20% 
15% 
10% 
.. 35  35-44  45-54  55-64  65+ 
Age Category 
Physicians. 8 States  Phys. "./QOC Problems 
NLOeTeEl :II I Nq=u12f4-1c63a rItr np-IrFoe bdeleal  apnh ryeiar tin  to HSCtaFteA a fnor  Nr=e130  cpohmypliead  1i/1d19f0i-6d/3 0b1y9 0. PROs as resnsle for  confi 
MSmewOd:s UiaRlmC wDE, i:c OtPDoIGirhryc/Oy:to EiPryaI h  oyf sMiiaed oianf l d tSoh fp rUoemni iateids  S2P4t aRteOed s.3  2(A 19e9ntL0 a)(;1 na99n 0M )A aen dmaPl eAh ysOsiitaaeo npCat,  hADsaocaiiavcn,it ne ro fa Snu rDeYye iaasrnbuo Dtkn aa tn( 1R e9Dsoi9ue0rcc)es;t  oMrya Aroqf um an 
Ostopath  (1990). 
Finally, we compared the foreign medical graduate (FMG) distribution among these 
physicians and the non-Federal physicians in their States.s Thirty percent (35) of the 
physicians responsible for serious quality-of-care problems were FMGs, which reflects 
the proportion of FMGs in those States. 
id 
let  physi ha 
At  th peen of  more thn one quli-of-care proble
th  131
by th PROs. 
We asked staff at each PRO if they had identified any quality-of-care problems for 
these physicians prior to our sample time frame. Although the information was not 
always available, the PROs were able to identify 20 physicians (15 percent) who had 
confirmed quality-of-care problems prior to our sample time. Those problems
 included Levels I, II, and III, and in one case, the PRO was recommending sanction. 
The PROs identified another 20 physicians (15 percent) who were responsible for 
additional9 quality-of-care problems during the sample time or identifed in intensifed 
review. 
What kids of problems were they responsible for? 
resg 
physi 
Eight peen of th  wer resnsle for  care  in serus or 
pote  deii  gn( 1)figs le unesed 
abnorm 
serus compliatins or  (2) 
(3) both 
Fort percent (52) of these  in serious or 
physicians provided care that resulted 
For example, one problem involved an attempt 
potentially serious complications. to 
rod implant split the bone 
to repair a patient's broken femur with a rod implant. The 
wie wrap. The wie wrapped the artery and nerve instead of 
therefore requirig a 
just the bone. Despite the patient's symptoms of vascular impairment, corrective 
surgery was delayed and the patient died of sepsis. 
Twenty-three percent (30) of these physicians provided care that left abnormal 
diagnostic findings unaddressed. One such problem involved a patient suffering a low-
discharge and whose white blood cell count was 
grade fever for three days prior to 
high, indicating infection, on the day of discharge. Eight days later the patient was 
readmitted because of the infection. 
Seventeen percent (22) of these physicians provided care that failed both screens. For 
example, one problem involved a surgeon s premature discharge of a patient after a 
'below-the-knee amputation. The stump was open and draing at the time of 
painl 
discharge and the patient was readmitted with an infected and  stump. The 
surgeon performed a debridement and closure, but they were ineffective, and the leg 
was fially amputated above-the-knee. The PRO noted that the patient's complicated 
postoperative period was a result of the inappropriate below-the-knee amputation. 
The remaing 20  were responsible for various other 
percent of the physicians 
problems such as medication errors and nosocomial infections. 
What kids of educational interventions did the PROs diec to these physician? 
edatinal inenns  physi. 
PROs diected no  peen of th 
Th  to  14
The PROs' response to the 131 physicians responsible for serious quality-of-care 
problems was multi-faceted (appendix E). They sent notifcations and conducted 
intensified reviews on the hospital admissions of all the physicians.u They also took 
, ranging from predischarge screening to preoperative 
various other interventions
physicians. And we saw documentation that 
consultation, against 44 percent of the 
they considered referrg 53 percent of the physicians to the OIG for sanctioning and 
39 percent to the State licensure boards. These considerations, however, led to few 
actual referrals, with only two to the OIG and three to the boards.  .. 
;:;:;  ;:;:;  ;:; ;:;:;  ;:;:;:;:;:;:;:;:;  ;:;:;  ;::  ;:  ;:;:;:;:;:;  ;:::::;  ;:;:;:;  ;:;:;  ;:::;  ::::::: 
For the educational interventions, which provide the focus of this study, it is strikig 
that the PROs directed no such interventions to 14 percent (18) of the 131 physicians 
found to be responsible for serious quality-of-care problems (figure 3). Ths omission 
occurred despite HCF A's requirement that an educational intervention be directed to 
each such physician. Like the others, these 18 physicians each received the 
notifcation of a confrmed quality-of-care problem from the PRO, and each was 
subject to intensified review (appendix E). 
FIGURE 3 
NUER  INVEONS 
OF EDUCATIONAL 
PROs IMSED PER PHYSICI 
;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;TWiJ:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:; 
None 
NOTE: N=131 prhyesivias i dfied by  PROs as resnsible for Q confi Lel III Il  b/e an reponed to 
HCFA for  completed 11/90-6130/90. 
SOURCE: OIGIOEI ansi of da from  PROs 
These 18 physicians involved 5 of the 8 PROs in our sample. For eight of the 
physicians, the PROs' records offef( d no documentation on why the educational 
interventions were not made. For seven of them, al involvig one PRO, the PRO 
deferred the interventions for many months so that it could undertake retrospective 
medical reviews to ascertain whether the quality-of-care problems were isolated or
12 The PROs did not pursue an educational intervention for two
part of a pattern.

physicians because they were recommending sanctions 13 and for one such an 
intervention was unnecessary because the physician retired. 
inenn, 
Amng th physins who di receie an edatinal  the PROs used lett 
metigs mo oft 
call coures and 
Fort-two percent received a letter. 
Thirt percent received a telephone call.  Twenty-two percent had to attend a continuing medical education course. 
Eighteen percent had to attend a meeting with PRO physicians. 
The PROs used these four educational interventions alone and in combinations. 
Physicians responsible for serious quality-of-care problems usually received one 
educational intervention (figure 3), and that intervention most often was a letter. The 
most common combination was a letter and a phone call, usually from the medical 
director or, in some cases, a physician adviser or a quality assurance committee 
member. 
The PROs required about one of every five physicians responsible for a serious 
quality-of-care problem to attend a continuing medica education (CME) course. The 
CME requirement ranged from as few as 12 hours to as many as 50. In every case 
the PRO approved the course syllabus and required proof of attendance. 
The meetings imposed as an educational intervention usually involved the physician 
attending a regularly scheduled meeting of the quality assurance commttee at the 
PRO to discuss the quality-of-care problem. The PRO medical director, peer 
physicians, and nurse reviewers usually staff these committees. 
The PROs' inventory of educational interventions contains more than the four noted 
above. For example, the PROs occasionally suggested medical readings and, in one 
case, required the physician to attend a refresher residency. 
Hosit  wi  the inenns. 
or aware of 
typicall were not involved 
Fift-one percent of the hospitals were not informed that a quality-of-care 
problem had occurred there. 
Eighty-eight percent of the hospitals where the quality-of-care problems 
occurred had no part in the educational intervention. 
may release physician-specific inormation about 
According to HCF A policy, the PROs 

confrmed quality-of-care problems to hospitals where the problem occurred without a 
must release it upon the hospital's request. Thee of the eight sample 
request and 
PROs routinely inform hospitals that a physician has been notified of a confrmed 
problem. The others have informed the hospital in only a few cases, citing both 
confdentiality concerns and HCF A's policy as reasons. 
Usually, the PROs that inform the hospitals do so by sending a copy of the physician 
notice to the hospital's quality assurance committee, relevant department chair, and/or 
designated liaison. The PRO would not necessarily know if the hospital took any 
action upon receiving the notice, or if the hospital already knew about the problem.  One PRO not only informed hospitals of problems, but also involved them in the 
educational interventions. In fact, this particular PRO was responsible for 23 of the 25 
cases in which the hospital was involved in the educational interventions. The 
involvement usually included a discussion of the quality problem between a hospital 
physician (for example, the chair of the quality assurance committee or relevant 
department) and the physician responsible for the problem. In some cases, they 
discussed the problem during grand rounds or a morbidity and mortality committee 
meeting. 
For 6 of the 131 quality-of-care problems, the PROs' interventions resulted in changes 
in hospital policies or protocols that could prevent other quality-of-care problems. For 
example, one problem involved the delayed treatment of an elderly patient's severe 
pulmonary edema resulting in the patient s death. The PRO determned that both the 
attending physician and the surgical resident were responsible for the problem. The 
PRO notifed the hospital of the problem, required both physicians to attend the 
PRO' s quality assurance meeting, and issued a first sanction notice for one physician. 
In addition, the PRO prompted the hospital's implementation of new policies for 
geriatric consultations and resident training in geriatric medicine. 
What conclusions ca we drw from the PROs' educational interventions? 
inenns 
unerin 
edatinal valu of the  is 
Th 
The letters are tyically brief and contain more case summary than educational 
material. 
The calls are not documented enough to determine their content. 
The available courses often do not address the specific quality-of-care problems 
the PROs have identifed. 
The meetings with the PRO physicians have the most educational value 
according to both the PROs and the relevant literature, but were used in only 
18 percent of the cases we reviewed. 
None of the eight sample PROs had formally assessed the effectiveness of their 
14 Some representatives of the PROs
educational interventions. , however, indicated 
that these interventions were important not only for the 
physicians involved but for 
other physicians in the State. In this context, they sometimes referred to a sentinel 
effect, whereby physicians change their practice because of the expectations and 
oversight activities of the PROs. 
In its recent report  Medicare: A Strategy for Quality AssurancelS  the Institute of 
Medicine (10M) reviewed the evolving research on educational interventions involving 
physicians. On the basis of this research, it suggested five elements that must be  present if educational interventions are to have much effect in changig physician 
behavior. These elements seldom appear to be associated with PRO interventions. 
First, the physician must accept that he or she needs to change. The PROs reported 
PRO as soon as they 
to us that physicians often engage lawyers to respond to a 
receive a notice of a potential quality-of-care problem.  , one PRO offcial said 
In fact
the medical society in its State identified a particular law firm physicians use when 
contacted by a PRO. In reviewig the case fies for this PRO and others, we saw 
much contentious correspondence between the physicians, their lawyers, and the 
PROs, and few indications that the physicians accepted the PROs' conclusions. 
The second element is that the content of the education must be specifc to the area 
needing change. PRO offcials often indicated that they could not fid courses that 
were suffciently responsive to a physician s particular educational need. Ths problem 
is exacerbated in rural areas where access to medical semiars and conferences can be 
lited. 
The remainig three elements focus on a personal meeting with the physician. 
Physicians, reports the 10M, are likely to respond most positively to an educational 
intervention when it (1) is offered in a personal meeting, (2) is conducted one-to-one 
16 Although most PRO offcials
and (3) is with a trusted and respected colleague.

reported believing that meetings are effective in changing physician behavior, fewer 
than one in five of the physicians in our sample had had an intervention that involved 
such a meeting. Moreover, those who did attend such a meeting often brought their 
lawyers and almost always found themselves confonting a number of PRO physicians 
.' rather than a single, trusted colleague. That is not to suggest that the meetings had no 
, educational value, only that according to the literature the setting did not appear to be 
conducive to providing the maxmum educational benefit. 
edatinal 
PROs confront thee bas constrin in seekig to cany out efectie 
Th 
inenns. 
They have limited information on which to base their educational interventions. 
The PROs usually base their interventions on one incident of care in 
one three-month period. 
The PROs' fies usually don t identify the physician s specialty, 
, certification status, medical school, or other information about 
credentials. 
The PROs rarely know about medical board or hospital actions taken 
against the physician.

The PROs cannot query the National Practitioner Data Bank about 
specific physicians.  The PROs tyically conduct a thorough inquiry in documenting a serious quality-of
determg 
care problem. However, the PROs act in a more restricted manner in 
what kid of educational. intervention to take once the problem is documented. Ths 
is in large part because the PRO physician advisors who decide on the intervention 
usually make their decision on the basis of one incident of care in one three month 
period, in accord with HCF A's quarterly profiing requirement. At some PROs, they 
do not review their own data for previous quality-of-cae problems involvig a 
given
physician. They are also unlkely to know if the State medical board, a hospital, or 
other health care entity has taken or is considerig taking action against the 
physicianP And Federal law stil does not allow the PROs access to the National 
Practitioner Data Bank, which would identif any adverse actions taken by medical 
boards or health care entities such as hospitals and any malpractice payments involving 
the physician.

inbit 
They perform oversight roles that  their capacity as educators. 
, PROs review the medical practice of physicians, identify those 
In their oversight roles
physicians responsible for quality-of-care problems, and then impose certain corrective 
actions. They must do this in accord with due process safeguards and myrad formal 
requirements imposed by the Federal governent. Understandably, the interactions 
between the PRO offcials and these physicians often become adversarial, as the 
physicians express concerns about the accuracy of the PROs' judgments and/or the 
implications for their medical practices. 
Amid such interactions, it is difficult for the PRO offcials and the physicians with 
./ confrmed quality-of-care problems to relate to one another in an essentially 
, educational sense. The PROs impose a certain threat to the physicians, however 
much they might attempt to deemphasize it. This threat limits, but does not 
altogether preclude, their opportunity to serve as medical educators. 
progr 
They fid that remedal medca education  are seldom avaable. 
Most PROs are unable to refer a physician with a serious quality-of-care problem to 
an educational program that offers an intensive and individualized response to the 
physician s needs. The limited availabilty of such programs, a number of PRO 
restricts what they can accomplish through their 
offcials report, signficantly 
educational interventions.  .' 
RECOMMENDATIONS

The physicians we focused on in this report were found responsible for "medical

mismanagement with signficant adverse effects on the patient." The educational

interventions we reviewed were imposed to help these physicians improve their

medical management skills and avoid the need for corrective actions of a punitive

nature. This educational orientation is central to the purpose of the PRO program. 
Yet, as the fidings in this report indicate, the educational efforts of the PROs appear 
to fall well short of their potential. We urge HCFA to give major attention to 
correcting tJ:s deficiency. Toward that end, we offer seven recommendations directed 
to HCFA We also present one recommendation to the Public Health Servce. 
TI HETI 
CAR FIANCIG ADMISTTION 
inenns 
ene tht th 
HCFA shoul  PROs inte tily edatinal  in aU

cTh onfi 
Leel m caes.

The PROs should be held accountable for carrng out the educational interventions 
currently mandated in all Level III cases unless, of course, the physician voluntariy 
surrenders his/her medical license. In our sample, 14 percent of the cases involved no 
such intervention at least six months after the PRO had confirmed a Level III quality-
of-care problem. 
regtin  th 
HCF A shoul is a  or, if necesary, seek lelatin mandtig  for 
/ Tah U confi  cases, th PRO share case informtini wdi  th hoitls at whih 
th proble oLeecl CUed Tht informtin shoul inlu th  of th physi th
natie of th proble an th  edatina and oth inenns impoed by 
type of 
PRO. 
regtin 
Th  HCF A shoul is a  th horo, ifl nse esary, seek lelatin aOowig aansd es
enoeudragitign tah  iPnReOsn tno sin volve  in th planng, condt, and 
of  in aU Leel m cases. 
Through their own quality assurance efforts, hospitals can and in some cases do play 
an important role in helping PROs achieve their educational objectives. The PROs
, it 
would appear, could take much greater advantage of this complementary role. Yet 
for confidentiality and other reasons, some PROs are reluctant to inform hospitals 
about quality-of-care problems and any interventions directed to physicians, and even 
more so to involve them in the conduct of the educational efforts and in the ongoing 
assessment of these efforts. By carrng out the above recommendations, HCF A could 
help correct this situation.  int  attn 
HCFA shoul  th PROs to gie greate  to peona metigs as an
eTh datna inenn an  condg  megs 
to exlore ways of  one-to-one  betwee 
physi wi confi Leel m quli-of-care prble and physin consnt. 
In our sample, only 18 percent of the educational interventions involved a meeting 
with PRO physicians. And even when a meeting was held, the educational value was 
uncertain. A meeting tyically involved a number of PRO offcials, with an attorney 
often accompanying the physician, therefore resemblig a hearg more than an 
educational exchange. We urge that HCFA and the PROs examie ways of making 
more effective use of one-to-one personal meetings as an educational tool and in so 
doing take account of the research findings concerning how physicians learn. 
inenn 
mate tht  edatina 
HCFA shoul  id  before diectig an  to a 
Th  ta in 
physi ina Leel m cas th PROs  accoun al prir qu-of-care 
concerg tht physi dug th  th 
prble th hae  PRO 
lie of 
contct 
PRO physician advisors often lack suffcient information about a physician s practice 
deficiencies to craft a well-suited educational intervention. Ths can be corrected by 
obtainig more inormation from "outside" sources, such as State medical boards, but 
also from the PRO itself. At a minimum, we believe that the PRO, in developing an 
, should review any prior quality problems, be 
educational intervention for a physician
they Level I, II, or III, that the PRO itself has identified for that physician over the 
term of the PRO contract. This would help identify if there is a pattern of medical 
, practice problems that the PRO should address in its educational effort. 
diemte 
HCFA shoul obtain and  informtin on th efectien of PRO 
Th  inenns. 
edatinal 
Do these interventions result in changed physician behavior? What tyes of 
interventions are likely to be most effective, under what conditions and in what 
combinations? Are medical education efforts sometimes relied upon to address 
problems that in essence are unrelated to medical knowledge or practice skis? To 
help answer such questions, HCF A should require that PROs prepare a wrtten 
assessment of the effectiveness of each Level III educational intervention. It should 
also support research efforts on the relative effectiveness of diferent educational 
actions directed to physicians responsible for poor medical care. By doing so, it could 
help the PROs identify and take advantage of the lessons learned from their 
considerable experience in undertaking educational interventions. 
eTh daHtCinF Ap rsohomuls o bitnailnu an din d tihter untaet itnoa la Ure tgh trP RmOs atihn liint ibgy  othfe fAocmeadnlr eMmedl al 
Asoctin. 
The number of such programs is limited, but those that are available can be an 
extremely helpful resource for physicians whose medical knowledge or skis are  deficient. The HCF A should assure that the PROs are fully aware of the programs 
, that do exist and consider them when meeting with physicians to discuss appropriate 
educational activities. 

HETI 
TH PUBUC  SERVICE

Pr 
Ce 
Edatn 
Puli Hea See,r theugh  th Area Hieanlt    fatndg/or
Th  sho prvi denstrn 
th AgenJor Healt Ca Poli and Reseah,  ene 
eslihm 
for tch li  and  of meal edn pramss ethst  see troe  m
th 
comptee of physi thug  and 
edtin. 
Individualized assessment and education programs, such as Ontario s Physician 
Enhancement Program, can serve as an effective approach for addressing medical 
practice deficiencies. Yet, with few such programs available in the United States 
PROs seldom have the option of referrg a physician to one as part of an educational 
intervention. The American Medical Association has encouraged State and local 
20 The Public Health 
medical societies to assist in the development of such programs.
Servce, through demonstration funding, can assist by using some of its demonstration 
funds to help determne the tye of physician enhancement programs likely to be most 
effective.  COMMENTS ON THE DRAFT REPORT

Within the Department of Health and Human Servces, we received comments from 
the Health Cae Financing Administration (HCF A) and the Public Health Servce 
(PHS). In addition, we received comments from a number of private organiations. 
(AA), 
These included the American Association of Retired Persons  the American 
Medical Peer Review Organition (AMRA), and the American Medical Association 
(AM). 
In appendix F, we present the full comments offered and our response to 
them. 
Our second and thid recommendations calling for the PROs to share more 
inormation about Level III cases with hospitals and to involve hospitals in the
plang 
and conduct of the educational interventions generated the most attention. 
, AA
The PHS , and AMRA expressed support for such action. The HCFA, 
which the recommendation was directed, indicated it would seek guidance from its 
general counsel to determne what if any action could be taken. 
In view of the legal uncertainty and the vital importance of these recommendations to 
the future effectiveness of the PROs' educational efforts in Level III cases , we have 
changed them so that they call for HCF A to issue regulations or, if necessary, seek 
legislation to foster closer PRO-hospital interactions in Level III cases. 
In our fifth recommendation, callig for PROs to take into account all prior quality-of
care problems before making an educational intervention, we made two changes to 
clari that we were referrg to Level III cases and to prior problems identified by a 
" PRO durig the life of its contract with HCFA We agree with AMRA that the 
PROs should "maintain a capacity to profie quality-of-care problems by individual 
practitioner(s) over the term of a PRO contract." 
In response to the AM' s comment  about its national registry on focused/remedial 
education programs, we added a recommendation urging HCF A to see that all PROs 
, such programs can 
are kept informed about such programs. For some physicians
serve as a crucial educational resource. 
Finally, with respect to our recommendation callng for PHS to provide demonstration 
funding for such programs, we added the Area Health Education Centers Program as 
a potential source for such funding. This was in response to comments from PHS 
addressing the potential relevance of that program.  APPENDIX A

MEODOLOICAL NOTE

We reviewed all confirmed Level III quality-of-care problems involving physicians 
identified by eight PROs and reported to HCF A for reviews completed between 
January 1 and June 30, 1990. We selected the eight PROs by dividing all PROs into 
confed 
three strata, according to their volume of  Level III problems. We then 
randomly selected 4 PROs from the high (20 or more problems) and 4 from the 
medium (6-19 problems) volume strata. Those 2 strata represent 30 PROs and 86 
confed 
percent of all the  Level III quality-of-care problems reported durig our 
time frame. We excluded the PROs with fewer than six confrmed Level III qualty 
problems due to the cost of including them and their lesser experience with 
interventions. 
We made site visits to six of the eight PROs, collecting our information by phone and 
mail for the others. We reviewed the PROs' records for each of the 131 physicians 
identifed by those 8 PROs (see appendix B for a sample overvew). Those 131 
physicians account for 22 percent of the confirmed Level III quality-of-care problems 
identifed by the 30 PROs in those 2 strata, and 19 percent of the confrmed Level III
problems in all PROs. 
Durig our review we recorded physicians' names , a description of the quality-of-care 
problem, the name of the hospital, and the interventions taken by the PROs. We also 
recorded the generic quality screen failures for each case, as a way to group the 
problems. Using a discussion guide, we went over the intervention process with each 
PRO' s medical director. In some cases, the chief executive offcer, physician 
reviewers, and the quality review director participated. 
We collected the physician profile c4ta from the data bases and publications of the 
American Medical Association and the American Osteopathic Association. 
We used Chi-square comparison of proportions tests to compare the distribution of 
the 131 physicians with non-Federal physicians in the 8 States by specialty, age 
gender, and foreign medical graduate status. 
A 
 * * 
APPENDIX 

SAMLE OVEVIW 
CONFIRMED LEVEL III  N USED 
QUALIT-OF CAR  CASES  FOR 
PROBLEMS REPORTED 
REVIEWED  MOST 
6/301//19/900- 
WIIN 
TO HCFA,  *TOTAL CASES  SCOPE  ANALY
PRO 
REVIEWED  OF INSPECTION  SES
6*** 
10**** 
' 47**** 
/ 8

TOTAL  161  158  135  131 
* Some of these PROs identifed c&ses not reported to HCF A. 
** We omitted cases from our scope for the following reasons: 6 problems occurred 
in a setting other than inpatient, 4 problems involved the provider rather than the 
physician as the responsible party, and 13 problems were either overturned or reduced 
to a lesser severity level. 

*** In this PRO, two physicians were each responsible for two quality-of-care 
problems, or cases, durig the sample time. To avoid double 
counting the physicians 
we omitted one case for each from our analysis. 
**** In each of these two PROs, one physician was responsible for two quality

problems, or cases, during the sample time. To avoid double counting these

physicians, we omitted one case each from our analysis.
 APPENDIX C

SUMY 
OF PHYSICI PROFl DATA 
Followig are summary data of the 131 physicians identifed by 8 PROs as being 
confied 
responsible for  Level III quality-of-care problems that were reported to 
HCFA for reviews completed between January 1 and June 30, 1990. Our sources for 
this data include: 
The 8 PROs. 
American Medical Association  American Medical Directory: Physicians in the 
United States. 32nd ed., Division of Survey and Data Resources, Department of 
Physician Data Servces, 1990. 
American Medical Association, Physician ProfIe Data, Division of Survey and 
Data Resources, Department of Physician and Data Servces, correspondence 
2/13/91 and 2/22/91. 
American Osteopathic Association  Yearbook and Directory of Osteopathic 
Physicians. 1990 ed. 
Marquis Who s Who  Directory of Medical Specialists, 24th ed., Wilmette, IL 
1990. 
C - 1
 -- 
SUMY 
OF PHYSICI PROFl DATA

FRQUECY  PERCE 
GENER 
Male  110  98% 
Female 
112  100% 
Unkown 
AGE (in 199) 
c:35

35

45

55

65+

130  100% 
Unkown 
SPECITY 
General. and Family Practice  28% 
Medical Specialties 
Surgica Specialties 
Osteopathy 
130  100% 
Unkown 
CETICATION 
STATUS 
Certifed  65% 
Not Certifed 
(excludes 11 osteopaths)  120  100% 
FOREIGN MEICAL GRAUATE 
Yes  30% 
(excludes 11 osteopaths)  117  100% 
Unkown 
C - 2
 DETAI ON SPECITY CATEGORITION 
SPECITY CATEGORITION  NUER OF PHYSICIS 
SEROUS QUAL -OF-CAR 
PROBLE 
GEN 
PRACfCE 
General Practice 
Family Practice 
SPECITI 
MEICAL 
Internal Medicine 
Cardiovascular Diseases 
Medical Oncology

Nephrology

Pulmonary Diseases

Anesthesiology

Geriatrics

Hematology

Neurology

Pathology

Rheumatology

SPECITI 
SURGICAL 
General Surgery
 27* 
Orthopedic Surgery

Urology 
Neurosurgery 
Thoracic Surgery

Obstetrics and Gynecology 
Otolarygology 
Vascular Surgery

OSOPATH 
130 
SPECITY UNOWN 
TOTAL  131 
lnc1udes three surgical residents 
C - 3
 APPENDIX D

GENC QUAL SCRS

Below we provide HCFA' s defitions from the third Scope of Work for the inpatient 
hospital generic quality screens. Following the definitions, we provide a summary of 
, 1990. 
the screens failed by the 131 physicians the 8 PROs identifed as responsible for
confed 
Level III quality-of-care problems reported to HCF A for reviews completed 
between January 1 and June 30
DEFONS  TH INATI  GENC QUAL SCRS 
OF  HOSPITAL 
1. Adequacy of Discharge Planning 
No documentation of discharge planning or appropriate follow-up care with 
consideration of physical, emotional, and mental status needs at time of 
discharge. 
Stabilty of the Patient 
2. Medical 
Blood pressure within 24 hours of discharge (systolic less than 85 or 
less than 50 or greater than 110). 
greater than 180; diastolic 
Temperature within 24 hours of discharge greater than 101 degrees 
Fahrenheit (38.3 Centigrade) oral, greater than 102 degrees Fahrenheit 
(38.9 Centigrade) rec 
Pulse less than 50 (or 45 if the patient is on a beta blocker), or greater 
discharge. 
than 120 within 24 hours of 
Abnormal diagnostic findings which are not addressed and resolved or 
where the record does not explain why they are not resolved. 
Intravenous fluids or drugs after 12 midnight on the day of discharge. 
Purulent or bloody drainage of wound or open area within 24 hours 
prior to discharge.

D - 1
 3. Deaths

During or following any surgery performed during the current admission. 
Following return to intensive care unit, coronary care or other special

care unit within 24 hours of being transferred out.

Other unexpected death.

4. Nosocomial Infection 
Hospital-acquired infection. 
5. Unscheduled Return to Surgery 
Within the same admission for same condition as previous surgery or to 
correct operative problem. 
6. 
Trauma Suffered in the Hospital 
Unplanned surgery which includes, but is not limited to, removal or 
repair of a normal organ or body part (i.e., surgery not addressed 
specifcally in the operative consent) 
Fall 
Serious complications of anesthesia 
Any transfusion error or serious transfusion reaction 
Hospital-acquired decubitus ulcer and/or deterioration of existing 
decubitus 
Medication error or adverse drug reaction (1) with 
serious potential for 
harm or (2) resulting in measures to correct 
Care or lack of care resulting in serious or potentially serious 
complications 
7. Optional Screen

Medication or treatment changes (including discontinuation) within 24 
hours of discharge without adequate observation 
D - 2  SUMY OF QUAL-OF CAR PROBLE PROFI DATA: 
NUER  FAIG GENC QUAL SCRS 
OF PHYSICIAS 
FRQUECY  PERCE

DISCHGE
PLAG SCR

Fail

Pass
 120 
129  100% 
Unkown

MEICAL

SCRS

STABIL 
Fail
 43% 
Pass

129  100% 
Unkown

DEATI SCRENS

Fail
 15% 
Pass
 109 
129  100% 
Unknown 
NOSOMI

SCR

INCTON 
Fail

Pass
 123 
129  100% 
Unkown

URNSCEH ULD

SURGERY SCREN 
Fail

Pass
 121 
129  100% 
Unkown

D - 3
 FRQUECY  PERCE 
SUF 
TRUM 
TH 
IN  HOSPITAL 
Fail  59% 
Pass  41% 
129  100% 
Unkown 
OPTONAL OR 
PROS' OWN
SCRS 
Fail  18% 
Pass  106 
129  100% 
Unknown 
D - 4
 APPENDIX E

SUMY  INVEON 
OF  PROFI DATA 
Followig are summary data of the interventions the 8 PROs directed to the 131 
physicians identified as responsible for confirmed Level III quality-of-care problems 
that were reported to HCFA for reviews completed between January 1 and June 30 
1990. 
E - 1
 SUMY OF QUAL INVEON PROFI DATA

INVEON

FRQUECY  PERCE 
NOTCATION 
Imposed  131  100% 
EDUCATIONAL
INVEON 
Imposed  113  86% 
Not Imposed
 14% 
131  100% 
INSIF 
REVIW 
Imposed  131  100% 
OTH
INVEONS 
Imposed  44% 
Not Imposed

131  100% 
SHAG 
UCESUR BODIE 
Considered  39% 
Not Considered

Shared 
131  100% 
SANCTON 
Considered  53% 
Not Considered

1st Notice Issued

Sanctioned

131  100% 
E - 2  SUMY  INVEON 
OF EDUCATIONAL  PROFI DATA

INVEON 
FRQUECY  PERCE 
Imposed  42% 
Not Imposed

113  100% 
CAl 
PHONE 
Imposed  30% 
Not Imposed

113  100% 
COUREWORK 
Imposed  22% 
Not Imposed

MEG WI  113  100% 
PRO PHYSICIS 
Imposed  18% 
Not Imposed

113  100% 
SUGGESTE 
REING 
Imposed 
Not Imposed
 110 
REHE  113  100% 
REIDENCY 
Imposed 
Not Imposed
 112 
113  100% 
METIG
WI 
HOSPITAL 
PHYSICIS 
Imposed  22% 
Not Imposed

113  100% 
E - 3  APPENDIX 
EfAn  REORT
COMM  ON  TH TDRA H COMM 
OIG REPONSE TO 
In this appendix, we present in full the comments on the draft 
report offered by the 
Health Care Financing Admistration (HCFA), the Public Health Servce (PHS), the 
(AA), 
American Association of Retired Persons  the American Medical Peer 
Review Association (AMRA), and the American Medical Association (AM). 
each case, we also include our response to the comments. 
F 
 &. 
Health Care 
DEPARTME!\T or HEALTH  HVMAN SERVICES	 Financing Administration 
Memorandum 
"."..a 
.NOV I 8 1991
Date 
Gai R. 
From	 Wilensky, Ph.D. 

Adminstrator

Subject	 Cae: A

OIG Draft Report - "Educating Physicians Responsible for Poor Medical 
Review of the Peer Review Organiztions' Effort," OEI-Ol-89-0020

Inpector General

Ofce of 
the Secretary

We have reviewed the subject draft report which assesses the educational 
interventions the peer review organitions (PROs) direct to physicians responsible 
problems. The report found that the educational vaue 
for serious quality-of-care 
PROs' interventions is uncertain and that the PROs face basic constraints that liit 
the effectiveness of these interventions. 
The report addresses six recommendations to HCF  Our comments on these 
recommendations, as well as general comments on the report are attached. 
Thank you for the opportunty to review and comment on this report. Please 
advise us whether you agree with our position on the report s recommendations at 
your earliest convenience. 
Attachment 
F 
 Comments of the Health Cae Financig Administration 
Draft Report. "Eucating Physician Responsible 
for Poor Medical Care: A Review of the Peer Revew 
Organizations' Effort." OEI-Ol-89-O20 
Recommendation 
tiely 
should ensure that the PROs intiate  educational interventions in 
The HCF A 
al confed Level III cases, unless the physician surenders bi/her medica license. 
esponse 
organtions 
We concur with ths recommendation. We require peer review 
(PROs) to intiate interventions withi 30 days afer the close of the review quarer, 
uness the error is so egregious that it warants imediate intervention. We 
Organtion Monitorig 
evaluate the performance of PROs using the Peer Review 
Protocol and Trackig System (PROMPTS). When we perform the PROMPTS 
determe whether educational interventions are 
evaluations, we review to 
plan 
requied to submit correctie acton 
implemented as directed. PROs are 
PROMP identies deficiencies in PRO 
which are monitored closely when 
wi pul a 
implementation of educational efforts. In the nex PROMP review, we 
focused sample of Level III caes to ensure that educationa interventions were 
properly intiated. 
Recommendation 2

confed 
The HCFA should mandate that for all  Level il cases, the PRO share 
hospitals at which the problems occured. That 
case information with the 
problem, 
inormation should include the identity of the physician the nature of the 
and the tye of educational and other interventions imposed by the PRO. 
Response 
obtai 
We wi confer with the Offce of the General Counel (OGC) in order to 
educational interventions 
more specific guidance as to what inormation concernng 
hospitals. 
can be shared with 
F 
 Page 2

Recommendation 3

interventions. 
The HCF A should encourage the PROs to involve the hospitals in the planng and 
conduct of the educational 
esponse 
sharg 
As previously stated, we need to confer with OGe on the issue of 
determine 
inormation on educational interventions with hospitals in order to 
whether we can implement this recommendation. In addition, we believe that many 
fuctions. 
, small hospitals) are not equipped to perform such 
spitals (e.g.
Recommenqation 4

The HCF A should instruct the PROs to give greater attention to personal meetings 
as an education intervention and to explore ways of conducting one-to-one meetings 
between physicians with confmned Level III quality-of-cre problems and physician 
consultants. 
Response 
We are currently revewing options for improvig our quality intervention plan and 
wil consider this recommendation in conjunction with other appropriate changes. 
Recommendation 5

The HCF A should mandate that before directing an education intervention to a 
physician, the PROs take into account all prior quality-of-care problems they have 
identifed concernng the physician. 
Response 
The PROs take prior history into account when decidig what intervention to take. 
The scope of work for the fourth contracting cycle claries that the PRO is to take 
into account prior history when deciding upon the appropriate intervention. 
F 
 Page 3

Recommendation 6

The HCFA should obtain and disemiate inormation on the effectiveness of PRO 
education interventions. 
Besponse 
progr 
HCFA is commtted to a strong  of education and inormation 
dissemiation among the PROs and though the PROs to the communty. The 
specifc recommendation to dissemiate inormation on the effeceness of PRO 
educational interventions wi be reaed with the implementation of a program of 
epidemiologic evaluation and oversight though the PROs. Under the scpe 
signcat 
work for the next round of contract with the PROs, there is  emphasis 
on education and feedback to the heath cae provders and practitioners on 
pattern of 
practice and the associated outcomes of cae. 
Genera) Comments 
Page i - OIG's background statement implies that the PROs' two major 
functions are policing activties. We suggest th pargraph be rewrtten 
priar 
to emphasize that the  responsibilty of the PROs is to ensure 
Medicare beneficiaries receive good quaty cae. 
lited 
Page 10 - OIG indicates that the PROs have  inormation and that 
the PROs rarely know about medica board or hospita actons taen 
th 
against physician. We agree that  is tre. Although OIG wants the 
PROs to share inormation freely with the medica boards and hospita, 
there is no companion recommendation that these other entities share 
information with the PROs. In the futue, PRO accss to the National 
th 
wi 
Practitioner Data Ban  help in  regard. 
F 
 COMM 
OIG REPONSE TO HCFA 
We respond as follows to HCFA's comments on each of the recommendations: 
Recmmendation 1: We welcome HCFA' s follow up to ensure that educational 
interventions are properly initiated. 
Recmmendation 2: In view of HCF A' s reservation about the legal basis of the 
recommendation as stated in the draft report, we have modified it to 
cal for the 
necessary regulatory or statutory changes. We urge that HCFA move quickly in 
carrg 
out this recommendation because it has a major bearig on the effectiveness 
of PRO educational interventions in Level III cases. 
In this context, we suggest that HCF A take into account that a close workig 
relationship with hospitals (and their own qualty assurance commttees) supports the 
continuous quality improvement principles which HCF A is embracing for the PRO 
program. By involvig the hospital in the intervention process and by keeping the 
hospital regularly informed of educational interventions taken against its physicians 
the PRO would be contributing to a more collegial process that involves more 
effective use of data concerning the quality of care. 
We also suggest that HCFA take note of the support for this recommendation 
expressed by the Public Health Servce, the American Association of Retired Persons 
and the American Medical Peer Review Association. The American Medical 
Association did not respond directly to the desirabilty of the recommendation, but did 
.I 
, emphasize that there should be strict liits to any educational activities undertaken 
directly by the PROs. Our recommendation is made in recogntion of such limits and 
of the major contributions that hospitals can make in assisting physicians found to 
have serious quality-of-care problems. Indeed, one of the PROs in our sample was 
working very closely with hospitals, just as we call for in the recommendations. 
Recmmendation 3: Our response concerng HCF A's comments on the above 
recommendation also apply to this recommendation. Hospitals can and should be 
major participants in crafting and overseeing educational plans for physicians who 
have been responsible for serious quality-of-care problems. We urge HCFA and the 
PROs to move quickly in this direction. 
Recmmendation 4: We regret that HCFA did not offer a more definitive response 
to this recommendation. One-to-one meetings, as research findings indicate, can lead 
to more effective educational interventions. We urge HCFA to identify ways to 
incorporate such meetings into the educational process in Level III cases. 
Recmmendation 5: HCF A indicates that the scope of work for the fourth contract 
PRO is to take into account prior history when deciding upon
cycle will clarify that the 

an educational intervention. Such language moves in the direction we call for, but is 
F 
 not suffciently precise. We urge that HCFA follow the suggestion offered in 
AMRA' s response to us and require that the PROs "maintain a capacity to profile 
quality of care problems by individual practitioner(s) over the term of a PRO contract 
and not just for consecutive quarters which is the mandate at present." Such an 
approach would help identif any pattern of quality problems and would allow for 
educational interventions more appropriately tailored to a physician s needs. 
(Accordingly, we have specifed in our recommendation that before fashioning 
educational interventions in Level III cases, PROs should review all quality-of-care 
problems they have identifed on those physicians durig the life of their PRO 
contracts. ) 
Recmmendation 6: In response to this recommendation, HCF A references the 
epidemiologic evaluation and oversight that wi be emphasized in future contracts with 
PROs (using HCFA's Uniorm Cliical Data Set). Ths future direction cited by 
HCFA represents an important and potentially constructive redirection in the PRO 
program. It does not, however, respond to the point of our recommendation. 
In our recommendation, as in the report as a whole, we make it clear that we are 
focusing on physicians responsible for poor medical care--that is, those physicians who 
have confrmed Level III quality-of-care problems. Our recommendation urges HCF A 
to determine the effectiveness of the educational interventions undertaken against 
these physicians. What interventions work best for them? Under what circumstances? 
Why? These are important questions to answer and should not be confused with 
other important questions about how practice-related data can best be used to 
improve the performance of most practicing physicians. 
F 
 ~~~
- - . . - ..
i,  , .J l- DEPARTMENT OF HEALTH & HUMA SERVICES	 Puic He 
Memorandum 
NOV '9 1991 
Date 
Assistant Secretary for Health 
From	
S\Jbje	 Office of Inspector General (OIG) Draft Report "Educating
Physicians Responsible for Poor Medical Care: A Review of the 
Peer Review Organizations' Efforts," OEI-Ol-89-00020 
Inspector General, OS

Attached are the Public Health Service comments on 
the subject
draft reporito.n W.e  concur with the intent of the 
re commenda t  Howeve r, we re commend tha t the recommenda t ion 
be directed toward more appropriate organizations 8uch as the

ervices Admini8tration 
Healch Resources and 
 (HRSA), with
technical assistance from the Agency for Health Care Policy

and Research. or perhaps to medical organizations and

themselves. In 
professional societies  our comments we discuss 
some activities undertaken by HRSA in previous years that

could be responsive to the issues identified in this report. 
Attachment 
F 
 . .
PUBLIC HEALTH SERVICE (PHS) COMMENTS ON THE OFFICE OF
INSPECTOR GENERA (OIG) DRAT REPORT . EDUCATING PHYSICIANS 
RESPONSIBLE FOR POOR MEDICAL CAREI A REVIEW OF THE PEER 
REVIEW ORGANIZATIONS' EFFORTS. . OEI-OI-B9-00020 
General Comments

The draft report presents findings regarding the lack of

consistency and perceived value of Peer Review Organization

(PRO) educational interventions, and discusses the significant

barriers undermining these efforts. The recommendations are

designed, overall, to rationalize this system. We are

particularly supportive of efforts to involve hospitals (or

other appropriate facilities, such as nursing homes) in the

planning of educational efforts.

It is not clear, however, that intensified educational 
interventions by PROs will overcome the basic problems 
described in the report. Additional consideration needs to be 
given to ways to change the adversarial relationship between 
PROs and individual physicians, and to encourage more active
invol vement of 
a range of medical peers and professional 
organizations in the education and discipline of colleagues. 
OIG Recommendation 
The Public Health Service, through the Agency for Health Care

Policy and Research (AHCPR), should provide demonstration

funding for the establishment and refinement of medical

education programs that seek to enhance the clinical competence

of physicians through individualized assessment and remedial

! education.

PHS Comment

We agree with the intent of this recommendation, but note that 
there are several problems with it as currently drafted. 
AHCPR 
has the authority to fund -demonstrations, but is not currently 
doing so, nor are specific demonstration funds included in the
AHCPR budget .  As part of its growing program to address issues 
related to quality assurance in health care, AHCPR hopes to
fund research, including demonstrations, designed to determine 
the effectiveness of alternative programs for improving 
physician performance and reducing the incidence of substandard 
medical care. This is consistent with the agency's statutory
authority to focus on research and evaluation questions. 
An evaluation of the effects of personalized assessment and

. remedial education programs on improved physician performance 
would be eligible for consideration in such an effort. 
However, the development and establishment of such programs is 
not wi thin the purview of the AHCPR. The recommendation  should therefore be directed toward more appropriate 
organizations, such as the Health Services and Resources
Administration 
(HRSA), or perhaps to medical organizations and
professional societies themselves. 
AHCPR could provide

technical assistance to HRSA by reviewing methodologies and

evaluating proposals for demonstration funding.

HRSA' s Area Health Education Centers (AHEC) Program is 
authorized under section 781 (a)(1) and 781 (a)(2) of the 
Public Health Service Act. Under section 781 (a)(1), basic 
support in the form of cooperative agreements is provided to 
successful applicants (medical schools or their parent 
organizations) to carry out an AHEC Program in a region of' a 
State or in an entire State, over a' 3- 9 year period. Under 
projects. 
(a)(2), grant support is provided to former 
section  7B1 
reci pien ts  1 (a) ( 1) funds to carry out 2-year AHEC Special 
of 7 
Initiative projects, including innovative demonstration
Quality assurance has been one of the funding priorities

established for the AHEC Program and the AHEC Special

Initiatives Program during competitive cycles held in previous
years. 
In Fiscal Year (FY) 1992, quality assurance was not 
included as a funding priority, since substance abuse was added 
to the other funding priority areas: geriatrics, HIV-AIDS, and

infant mortality prevention. 
We will reconsider whether
quali ty assurance 
should be reestablished as one of the funding
priori ties for 
the AHEC Program and the AHEC Special
! Initiatives 
Program beginning in the FY 1993 competitive grant
cycle. 
This would encourage a range of quality assurance 
applications that would focus on physician effectiveness 
educational programs at a State or local level, and would be 
responsive to the needs and issues cited in the OIG draft
report. 
The AHEC Program appropriation in FY 1991 was $19. 2 million,
with 90 percent of the funds ($17.3M) awarded to support 15
ongoing AHEC programs and six new AHEC programs. Currently,
to 10 percent of the annual AHEC Program appropriation may be 
projects. In 
used to support AHEC Special Initiatives  FY 1991,
a total of approximately $1. 9M was awarded to support
15 continuation and eight new Special Initiatives projects,
some of which address quality assurance issues. The average
award was $75, 000. 
TECHNICAL COMMENT

The name of the Agency for Health Care Policy and Research 
should be corrected in the recommendation to the Public Health 
report. 
Service on pages iii and 13 of the 
F  COMM

OIG REPONSE TO PHS 
In response to PHS' comments, we have amended our recommendation to clari that 
both the Agency for Health Care Policy and Research and the Area Health Education 
Centers Program in the Bureau of Health Professions could serve as resources to 
support and assess demonstration efforts cited. We have also made the techncal 
correction noted.

F 
 .\..
==--
Brinlfin" lifftimts Oft:l:pooimcr and ltlldfrsl1ip to Sfn'/" all "mtratio", 
September 17, 1991

usserow
RIicnharsd Pp.
 ector General 
Services 
Department of Health and Human 
330 IndependencCe .A ve., S. 
Washington, D. 20201 
Dear Mr.  usserow: 
I am responding to your letter to Horace Deets regarding the

draft report entitled "Educating Physicians Responsible for Poor

Medical Care: A Review of the Peer Review Organizations'

Efforts. 
The American Association of Retired Persons is very 
pleased to submi t these comments and commends the authors of the

-report for their contribution to increased understanding of how

to improve a program of great importance to Medicare

beneficiaries. 
The report provides empi rical data indicating that the quali 
intervention plans pursued by the Peer Review Organizations 
(PROs) are falling far short of thei r goals. A major conclusion 
from the report is that, although PROs are effective in 
identifying Level III problems, they do not have an effective and 
consistent process for ameliorating deficiencies and certifying 
the competency of problem physicians. The report is also
instructi ve in 
not only providing an inventory of the problems 
that lead to Level III citations, but in recognizing th 
frequently difficult environment in which remedies for such
ci tations 
are considered. 
AARP continues to believe that a properly functioning peer review
system provides vi 
tal protection for Medicare beneficiaries. 
Accordingly, every effort must be made to strengthen existing 
mechanisms for intervention in cases of poor quality. We have 
reviewed the draft report and find that, for the most part, we 
findings.
are in agreement wi th its recommendations and 
Accordingly, the following comments address those areas where we 
have addi tional suggestions. 
Background  ( pg. 2) 
We suggest that you reference the Office of the Inspector
General' s (OIG) earlier report on PRO sanctions so that readers 
may be informed about the PROs' experience in exercising their
more puni tive options for addressing quali ty of care problems. 
DC 
\rm:nc.1n .\ssexi;1ton of Retired Persom  bOJ E Street.  uhing:tnn.  :!004Q  :!o 
Runen B,  1.l::wel1  PrmdmT  F - HorJ':c B. Dcct  EX((/IT:Y  DII' aror  . '
Recommendations  ( pg 12 - 14) 
In its fi  for: 
rst three recommendations, the OIG calls 
Ensuring timely educational interventions in all

cases; 

confirmed Level III 
Mandating sharing of case information with the hospitals

in which the problems occurred;

Encouraging the PROs to involve the hospitals in the

interventions.

planning and conduct of the educational 
We strongly endorse these recommendations. Wi th respect to the 
second, we note that this sharing, given the PROs existing 
discretionary authority, would not constitute a significant
change in the current confidentiali 
regulation.  ty and disclosutrhei rfdra,m ehwoosrpki tal 
established in  With respect to the 
role nationally in 
quali ty assurance commi ttees play a cri tical 
maintaining quali ty of care, yet the report indicates 88\ had no 
interventions. In light of 
part in PRO educational  the potential
importance of hospi tals' involvement, the OIG in its final report 
that: 
also should recommend 
interaction wi 
o HcrA mandate PRO-hospi tal  th respect to both 
the planning and evaluation of interventions through changes in
and  in hospi tals' Condi tions of Participation. 
PRO regulations 
In addi tion, PROs should be encouraged to reach out and involve 
medical schools, medical centers, and other appropriate
! institutions in their 
areas in the design, implementation, and
. evaluation of 
corrective action plans, including innovative
remedial education programs. In this connection, we note the
report' 
s observation that lack of access ,to medical seminars and 
problem. 
other appropriate remedial courses is particularly acute in many
rural areas. 
Particular attention should be paid to this
In its fourth recommendation, the OIG stresses the value of 
personal meetings as an educational intervention. It is our 
respect 
understanding that considerable research is underway with 
to strategies for changing physician practice patterns which may 
be relevant to this issue. We concur that HcrA and the PROs 
should take into account research findings on how physicians 
learn in evaluating the efficacy of personal meetings and other
interventions. 
The fifth recommendation points up a serious deficiency 
utili ze all 
identi fied in the report, namely the PROs' failure to 
available data affecting the judgment about a physician'
performance. 
Accordingly, we strongly concur wi th the OIG' 
that: 
recommendation and additionally recommend 
Efforts by the PROs and HcrA to achieve PROs' access to

F  state medical licensure board information pertinent to

investigation of Level III cases should be accelerated. This

report again documents the reluctance of PROs and state boards to

problems 
communicate wi th each other in cases of 5e rious quali ty 
the barriers to such information sharing should be evaluated. 
PROs should be given authori ty to query the National

Practitioner Data Bank about specific physicians.

t. 
The OIG' s sixth recommendation, that HcrA obtain and disseminate

information on the effectiveness of PRO educational

In order to implement this
interventions, has great meri
 that: 
recommendation uni formly across PROs, we also recommend 
o PROs should be instructed to prepianrtee ravned ndtoiocunm.e nt reports 
These 
on the effectiveness of each Level III 
reports would serve to communicate Important information to 
physicians and their hospitals, as well contribute to the

development of a HcrA data base on the impact of PRO

interventions. 
We appreciate the opportuni ty to review this important report and 
th you. If 
you should have any 
to share our comments wi 
questions, please contact Mary Jo Gibson of the Division of
Legi slation and Public Policy at (202) 434-3896. 
othe r 
ren -c
 tor 
Dl 
Division of Legislation and Public policy

F 
 AA COMMNT 
OIG REPONSE TO 
In response to AAP' s comments, we have added a footnote referencing the prior 
OIG report on PRO sanctions and have modified our third recommendation to call for 
the PROs and HCF A to work together not only in planning and conducting 
educational interventions, but also in asssing them. 
With respect to AA' s additional recommendations: (1) we do not view it 
appropriate to recommend that PRO-hospital interactions be included in hospitals 
Medicare conditions of participation, given that our focus in this report has been on
the PROs rather than hospitals; (2) we have addressed in a prior report the need for 
closer interactions between PROs and State medical boards (see "State Medical 
Boards and Medical Discipline " OEI-01-89-00560, August 1990); and (3) we agree 
that PROs should be able to query the National Practitioner Data Bank and 
understand that wi be the case once Section 5 of the Medicare and Medicaid 
Protection. Act of 1987 is implemented. 
F 
 "". "" 
,-:-" ...';
'L\mi' 
AMERICAN MEDICAL PEER REVIEW ASSOCIATION
810 First Street. N. E. . Suite 410. . Washington, D.C. 20002  (202) 371-5610 .  M: (202) 371-8954 
OFICE"S  September 30, 1991 
flESIDENT 
WI I'..'" H Moner.' . Jr M 0 
San FrlncllCO. Callto,nl8 
VICE PRESIDENT 
J Ol.,a 811Dv M 0 
Richard Kusserow 
Alma 1."'''"

SECIIETAIIY  Inpector General

Dorl,a K WIII.ce M 0  Deparent of Health and Human Servces 
Sot..'" P,,,. NO"" Cero"""

HHS Cohen Buildig 
TlEASUIIEli 
L- M.

Gery 0 
80""1....1. Uln  Room 525(; 
IMMEDIATE PAST ",ESIOENT  330 Independence Avenue, S. 
TIIO G WDeMt.I Ml, 0n   . 
MII.au.....  Washigton, DC 20201

DIRECTORS 
De.1I lo 8M M D  Dear Mr. Kusserow:

LOI 1.";'''". c."tom. 
AAlMCn AarroO Er..  8M"orenn.'g lM"  D  The American Medica Peer Review Assocadtiron  (AMRA) appreciates the 
Oo'G E Harro M 0
 opportty to respond to the prepublication  report entitled, "Educatig 
Ptnl8vllle. Penn'fl".n'l 
Physician for Poor Medica Cae: A Review of the Peer Review Organtions 
CI- 10_ 1J o M 
 Effort." The draf has not been shared with the AMRA membership nor dicused 
DEae,.",Go" I  .K.' ";JS8''r' tsI. 0   vbiye wtheed A asM pRrAe lBioaarrd a onfd D niorte cretoflresc. tTveh eo ff oAllMowRigA c oBmomaredn tpso, ltihcyer aeft otrhe,  sthieou.l d be 
IIICflro P McLI"g""". Jr.. M 0 
811'''If'g  .m AIIDlf'' 
AMRA is supportve of the report's recommendations and believe adoption would 
eol'G POIeFlon D C

DaIiIS. T...as  s effort to modi 
enhance the PRO' the behavior of individual practtioners. 
'51_.r LoOCku '"0' 1 .P"1e9i"F"l o" M 0
 parcuar, we support tberecommendation requg PROs to share inormation with 
L,wr."ce J 5"'101': M C  the hospitals about Level il caes attrbuted to physician. We would suggest going 
Proen.J.. AriZona  one step fuher and mandate that al confed 
qualty problems be shared with
TOnlodm .B.r OJO lSonwelnl.l. e.. .
. J'" 0'M"O.  hospital. We believe that hospita are legaly responsible for the qualty of cae 
therefore, should be entitled to inormation 
AMRI'C  delivered in their intitutions and, 
II ....ne. Pllme  Y E.. e C" S M  confed 
8otO" MISSIC",,"!!'  relating to  qualty problems identied by the PRO. 
CEO SECTION 
A'!. P AClI" Jr  We are also strongly supportve of the recommendation to encourage PROs to 
De"..' COlor.GO

plang 
and conduct of educational interventions. We 
.1""" C Cen"on
 involve hospital in the 
Sail LI.e C :.. L;:ln 
believe that this would greatly enbance the educational impact by PROs. In the past, 
MEDICAL OIREC ::I:S SEC':ION  HCFA General Counel has taken the position that to share specifics of PRO 
JaM C Pe"Flon ,,' M 0 
Sum. wu,ungt:l 
facity would requie a waiver of the confdentialty proviions 
intervention with the 
PNoU.BmLl ICt IO L.IIIE"Cg '!PC-ReS MS t'  by the afected physician. 
Mllw.  W'ICOr.I,r: 
tae 
tCoeOrlloY,W,,O OLOI OFeIO
 fiCl.  The recommendation that PROs be mandated to  into account al previous 
Vlt. Os '.nCl.'  qualty of cae probJems when diectg an educational intervention for a physician 
Atl."tl Geo' g I  makes good sense. HCFA should consider requig PROs to maitain a capacity 
SPEA"E" OF T"'E "OL:SE  to profie qualty of care problems by individual practtioner over the term of a PRO 
f:rp:," C. C  ,"''' t-' D 
JJ. .",aJ-e- i/IIC:"  contract and not just for consecutive quarers which is tbe mandate at present. 
v'CE S'EAKE" O' T--E "'O.;SE

If.f'  '" Ce::' ,o 

II; ::" t.e ."..0

F 
-='f':'  I! ;'.a.!J :"'..

I":'
 $.. .
(,(

Richard Kusserow

September 30, 1991 
Page two 
Once agai than you for tbe opportty to comment on th draf report AMRA looks 
AMRA membership. 
report for distrbution to the 
forward to receivig the fial 

Sincerely, 
Andrew Webber 
Executive Vice President 
AW/aw 
F 
 COMM 
OIG REPONSE TO AMRA 
We agree with AMRA' s point that all quality problems be shared with hospitals and 
suggest that HCFA consider it. However, given our study s focus on Level III cases 
and educational interventions, we must limit our formal recommendations accordingly. 
We also agree with AMRA' s point about HCFA requirg the PROs to maintain a 
capacity to profie quality-of-care problems by an, individual practitioner over the term 
of a PRO contract. We have reinforced this point in our response to HCFA and have 
amended the recommendation to clarify that this profiling requirement should apply 
for the life of a PRO contract. 
F 
 American Medical Association 
Physicians dedicated to the health of America 
James S. Todd. MD  515 North State Stret  312464-500

312 464-4184 Fax

Executive Vice Preident  Chicago. Illois 60610 
September 9, 1991

Richard P. Kusserow 
General 
Office of Inspector 
Department of Heal th and Human Services 
200 Independence Avenue, SW
Washington, DC 20201 
RE: Draft Report - "Educat ing Phys icians 
Responsible For Poor Medical Care: 
A Review of the Peer Review Organizat ions I 
Efforts  tJ/-f'i- 1J.;6 
Dear Mr. Kusserow:

The American Medical Association (AM) welcomes the opportuni ty to 
comment on the draft report, "Educat ing Phys icians Responsible For Poor 
Medical Care: A Review of the Peer Review Organizat ions I Efforts. 
The AM commends the Office of Inspector General for undertaking the 
We support the 
examination of current educational activities of PROs. 
i ty 
goal of the peer review organizat ion (PRO) program, to ensure qual 
medical care for all Medicare beneficiaries, and we believe it can best 
be achieved by di rect ing current PRO resources toward educat ional and 
interventions. 
quality assurance endeavors rather than punitive 
90) provides that 
The Omnibus Budget Reconci I iat ion Act of 1990 (OBRA '
before a PRO submi 15 its report and recommendations to the Secretary of 
Intervent ion Plan of 
Heal th and Human Services pursuant to the Qual i ty 
the PR 
the Heal th Care Financing Admini strat ion (HCFA),  may provide the 
physician or entity being scrutinized an opportunity to enter into and 
CA 
complete a corrective action plan (CAP). The  may include remedial

appropriate. In 
education, if  our view, however, PROs should not

uni laterally develop or be involved in the actual provision of an

educational CA, nor should they be involved directly in a system for

coordinating the educational activity.

PRO ent i ties do not possess the necessary educational resources, facul 
or expert i se to engage in such an endeavor. Any such act ions by the PROs 
would also constitute a serious conflict of interest, especially if 
undertaken as a revenue generating activity. We believe, moreover, that 
educational 
PROs must not mandate or require the use of any specific 
program or resource by physicians designated through the problem 
ident if icat ion process. Ins tead, the PRO should recognize the resources 
already available within the community and rely on those institutions to 
provide educat ional programs and remediat ion. 
F 
 - 2 

recommendations set forth in the draft

The following analysis of tht es  views on appropriate  ivi ties to 
report wi 11 present the AM  PRO act 
ice that affect the qual i ty 

correct deficiencies in technique or pract 
medical services rendered. 
Finacing 
Recommendation 1 - The Health Care  Administration should 
ensure that the PRs initiate ti.ely educational interventions in all 
confirmed Level III cases. unless the physician surrenders hislher
.edical license. 
The AM believes that the recommendation should be modified by deleting 
the phrase which states, "unless the physician surrenders his/her medical 
license.  Such a recommendation appears contrary to the intent of the 
draft report which attempts "to assess the educ(aPtRiosn)a ld iirnetcetr vteon tions that 
phys icians 
the Medicare-funded Peer Review Organizat ions 
The phrase in
responsible for serious quality of care problems.

quest ion, however, impl ies that PROs may. impose a puni t ive or coercive

measure. rather than an educat ional strategy, to induce a physician to 
icense. .Deletion of the phrase "unless the 
surrender his/her medical 
" would eliminate the, 
physician surrenders his/her medical license
impl ication that a physician may be encouraged to surrender his or her
ini tiate 
an educational
icense. and a PRO would not be required to 

intervention (based on timeliness) in situations where a license 

surrendered. 
Recommendat ion 2 - The Heal th Care Financing AdminisPtrRat ion should
share case
.andate that for all confirmed Level III cases, the  Tht 
infornt ion wi th the hospi tals at wbich the problems occurred. 
information should include the identity of tbe physician. the nature 
of the problem. and the type of educational and other interventions 
PR. 
imposed by the 
PROs should institute an approach to the notification process in sharing 
confirmed Level III information with hospitals that will maximize the 
AM has urged HCFA to modi 
educat ional impact of the program. The  Ii ty problems 
current regulat ions so that:, (1) in regard to confirmed qua 
i ty Assurance
which have been finally adjuaicated by the PRO Qual 

Committee, the PRO is required to notify both the physician and president 
all such cases; and (2) the PRO is 
of the hospital medical staff in 
veri fy receipt 
requi red to implement a mechanism to  of the PRO' s not ice 
physician. We

of both potential and confirmed quality problems by the 
the PRO statute: (1) to require

have further recommended amendments to 
at when the PRO review goes beyond the generic screen for intensified 
review , the phys ician mus t be not i fied wi thin 48 hours of the exact 
reason for said review; and (2) to repeal the existing prohibi tion on the 
release (to a PRO proposed sanctioned physician) of documents or other

its deliberations in

information produced by a PRO in connection with 
making qual i ty determinat ions. The AM al so recommends that HCFA 
regulations be revised to permit notification to residency training 
programs of a Quality Intervention Plan letter of inquiry received by a 
res ident during part icipat ion in an accredi ted res idency program. 
PRO, notification

Prior to confirmation of a quality problem by the 
should occur only at the request of/or with the consent of the affected

F  - 3 

The AM opposes di sclosure of unconfirmed PR qual i ty 
phys ician. 
s right to 
inquiries as this would unfairly impinge on a physician 
privacy and confidentiality and also deleteriously affect the physician
profess ional reputat ion. 
Finacing Adainistration should 
Recommendat ion 3 - The Heal th Care 
encourage the PRs to involve the hospitals in the planing and 
interventions. 
conduct of the educational 
PRs to 
The recommendat ion to encourage  involve hospi tal s in the planning 
ions requires further 
and conduct of the educat ional intervent 
specification. It is essential that a CA apply only to the physician 
ident i fied as the source of the confirmed qual i ty problem. Evidence 
instances, a PR  rge
has required all or a 
suggests that, in certain 
portion of a facility s active medical or other professional staff to be 
CA which should have been more precisely directed 
involved in a focused 
to the ident i fied phys ician. The AM stands opposed to CAs of an 
intrusive and overly expansive nature. Also, in recognizing hospital 
interventions, it 
involvement in the planing and conduct of educational 
should be clarified that such involvement is to include the organized 
medical staff and not jus t the hospi tal administrat ion. 
HeaUhCare Financing Adainistration should 
Recommendat ion 4 - The 
PRs to give 
instruct the  greater attention to personal .eetings as 
an educat ional intervent ion and to explore ways of conducting
III
one-to-one .eetings between physicians with confir.ed Level 

tants.

qual i ty-of-care problems and physician consul 
The AM supports the concept of increased personal meetings between
tool.
physicians and physician consultants as a constructive educational 
i ty to 
Physicians have expressed repea1 d frustrat ions at their inabi 1 
contact and meet wi th a pbys ician representat ive of a PRO. 
these
Implementation of Recommendation 4 would be of value in mitigating 

concerns. 
Consistent with our view that PROs should limit their activities in 
problems and 
developing CAPs to the idendfication of confirmed 
notification  CA, we 
thereof, as well as monitoring compliance with a 
bel ieve that the PRO should not uni laterally be involved in determining 
an educational CAP, providing such education or requiring the use of a
specific educational program. The PRO, therefore, should not participate 
in any form of instructional arrangement between the cited practitioner
problem.
and an independent educational program selected to remediate the 

Recommendat ion 5 - The Heal th Care Financing Administration should 
aadate that 
before directing an educational intervention to a
physician, the PRs take into account all prior qulity-of-care
problems they have ident i fied concerning that physician. 
AM believes 
When the PRO considers prior quality of care problems, the 
that it would be useful to evaluate only those cases that relate directly 
hand. An arbitrary 
review of all 
to the confirmed quality problem at 
F 
 - 4 

previous unre lated qual i ty quest ions would not serve to correct technical 
deficiencies in technique or procedure that have been revealed by the 
scrutiny. 
current Level III confirmed quality problem under 
Recommendation 6 - The Health Care Financing Adainistration should

obtain and disse.inate inforaation on the effectiveness of 

educational interventions. 
PROs generally lack the expertise, resources and faculty to provide the 
necessary assessment, educat ion and evaluat ion of those educat ional 
ident i fied  i ty 
intervent ions undertaken by phys icians  in the Qual 
Intervention Plan. It may be appropriate, however, to monitor the 
referral of physicians to appropriate educational programs or 
institutions, as well as the evaluation by the relevant educational 
program of the phys ician ' s progress. 
In addition, we believe that the PRO program should be redesigned to 
provide rout ine feedback to the medical communi ty about its review 
findings. PROs should be required to distribute aggregate information on 
a regular basis to state medical societies. Provision of such 
educat ional feedback is essent ial to improvi g qual i ty of care for the 
Medicare population and others. Sharing of aggregated information at the

state level would enable physician organizations to identify specific

areas for which educat ional material s or programs could be developed or

provided. 
for
Recommendation 7 - The Public Health Service, through the Agen

Heal th Care Pol i  and Research, should provide demonstration funding 
for the establishaent and refinement of .edical education prograas

enhce 
that seek to  the clinical coapetence of physicians through 
individualized assess.ents and remedial education.

intent 
The AM supports funding for demonstration grants to implement the 
of Recommendation 7. Such grants should be awarded to educational 
institutions with the ability to provide individualized physician 
assessment, necessary education, and evaluation. PROs, however, should 
not  be the recipients of sucID educat ional grants. 
lable 
Wi th respect to the comment in the draft report indicating few avai 
address ing medical pract ice deficiencies, 
educat ional programs for 
wish to point out areas where substantial progress has been made in the

last two years. The AM has compi led and maintains a national registry 
of focused/remedial educat ion programs. A task force comprised of 
cont inuing medical educat ion providers, I icens ing bodies, 
representatives and others has been assembled to meet on a regular basis 
to assess progress in this area. To date, the national registry has not 
fic 
been contacted by a PRO or by HCFA regarding informat ion about a speci 
medical educat ion program. 
In conclusion, the AM applauds your effort to assess educational 
believe 
report. We 
interventions of PROs as articulated in the draft 
which minimize undue disrupt ion of the phys ician' s 
that educat ional CAPs  - s 

practice, 
incorporate principles of fairness and due process, and serve
will 
to truly provide an educational approach to quality intervention 

ients. We would be pleased to work

bes t serve the interests of our pat  I ines to improve
wi th HCFA and others to formulate principles and guide 

the effect i veness of the PRO program in this area and thereby enhance the 
qual i ty of medical care provided to Medicare beneficiaries.

Sincerely, 
d-ml)
,E. 
James S. Todd, MD

JST:hl 
F 
 AM COMMNT 
OIG REPONSE TO 
We agree with the AM that the PROs' own role in conducting educational activities 
should be limited. It is for that reason that we recommend that HCF A involve 
hospitals more in the educational activities and that PHS provide demonstration 
funding for medical education programs that seek to enhance the clinical competence 
of physicians through individualied assessments and remedial education. 
With respect to the AM' s comments on each of our seven recommendations, we 
respond as follows: 
Recmmendation 1: In accord with the AM' s suggestion, we omitted the phrase 
unless the physician surrenders his/her medical license" from the recommendation. 
We moved it to a less promient place in the supporting text, not to imply that the 
PROs "may impose a punitive or coercive measure," but simply to clarify that an 
educational intervention is obviously unnecessary in cases where physicians of their 
own accord have chosen to surrender their license. 
AM 
Recmmendation 2: The  presents its position on a number of issues concerning 
the PRO review process, but does not comment directly on our recommendation 
mandating that information on confrmed Level III cases be shared with the hospitals. 
It does urge that HCF A take an approach in sharig Level III inormation with 
hospitals that "will maxmize the educational impact of the program." Our 
fulfent 
recommendation, if implemented, would faciltate the  of that objective. 
Recmmendation 3: In accord with the AM' s suggestion, we have amended the 
recommendation to specify that it applies to Level III cases. As to which parties 
should be involved in representing the hospital, we would regard a hospital quality 
assurance committee as being the focal point, but look to HCF A to provide the 
appropriate specification. 

Recmmendation 4: We share the AM' s concern that the process of developing an 
lited 
educational intervention be a collaborative one. However, given the  nature of 
the educational actions that we have found PROs take in Level III cases, we would 
not lessen the PRO's authority to determine an appropriate educational intervention 
or to obtain feedback concerning that intervention. 
Recmmendation 5: In callng for the PROs to consider prior quality-of-care 
problems involving a physician with a confirmed Level III problem, we are not at all 
callng for "an arbitrary review of all previous unrelated quality questions." To the 
contrary, we are urgig the PROs to have before them relevant information 
concerning a physician  performance - information that can help them craft an 
s -
s particular needs. Such information 
educational intervention suited to a physician 
could reveal a certain pattern of problems that could be quite helpful to the PROs 
and eventually to the physician. This approach is in accord with the AM' s desire 
F 
 AM 
that the PROs maxmize their educational impact. Accordingly, we urge the 
reconsider its position on this recommendation. 
Recmmendation 6: We recognze that the PROs lack resources or expertise to 
conduct in-depth assessments of their educational interventions in Level III cases. It is 
for that reason that we urge HCF A to address the issue directly by supporting 
research efforts and identifng ways of obtaining and dissemiating information on 
this vital matter. Ths tye of information is important in its own right, and should not 
be confused with information on the effects of more generalized effort to disseminate 
practice-related information to the medical community. 
Recmmendation 7: To clarify, we do not urge that the PROs be the recipients of 
demonstration funding for the establishment and refiement of medical education 
programs. Rather, our intent here is to increase the availabilty of signcant 
educational vehicles available both to PROs and physicians. Finally, in accord with the 
AM 
information provided by the  we have added a recommendation calling for 
HCFA to obtain and distribute to all the PROs information about the 
focused/remedial education programs included in the AM' s national registry. 
F 
 APPENDIX G

ENDNOTE 
The PROs must also total the weighted severity scores for each institutional 
provider identifed as having one or more quality-of-care problems. We have 
excluded such proviqers, which, according to HCF A, account for about 20 
percent of all confrmed quality-of-care problems, from the scope of this study. 
The weighted severity score triggers specific interventions which the PRO must 
impose. At a minium, HCF A requires the PROs to use the followig 
interventions at the noted weighted triggers: 
Intervention  Weighted Trigger 
Notification

Education

Intensified Review

Other Interventions 
Consider Coordinating with

Licensing Bodies 
Consider Recommending 
Sanction 
Because the PROs must apply these interventions in a cumulative manner, a 
confrmed Level III quality-of-care problem, which 
physician responsible for a 
, would be subject to each intervention. 
has a weight of 25
In an October 1988 report entitled "The Utiltion and Quality Control Peer 
Review Organition (PRO) Program: Sanction Activities" (OAI-01-88-00571), 
we focused on the PROs' sanction activities in some detail. One of our findings 
cited in that report was that conflct between the PROs' concurrent education 
and sanction roles appeared to undermine the effectiveness of the sanction 
process. 
Physician specialty does not indicate board certification in this context. Rather 
it is the specialty reported by the physician to the American Medical 
Association which accounts for the greatest number of professional hours. 
Non-Federal physicians are those not employed by the Federal Governent 
i.e., the Ary, Navy, Air Force, Department of Veterans' Afairs, the Public 
Health Servce, and other federally funded agencies. 
G  Chi-square = 14.5, dJ. = 3, and p c: .005. 
Chi-square = 55.4, dJ. = 4, and p c: .005. 
Excluding osteopaths. 
By additional, we mean other than the confirmed Level III quality-of-care 
problem reviewed in our sample. 
10.  To describe the serious quality-of-care problems, we recorded the generic 
quality screen failures for each. Generic quality screens are criteria applied to 
the medical records for identifyg events that could indicate poor quality of 
care. We found that the screen failures clustered in 2 of the 20 screens HCFA 
requires the PROs to apply: (1) care or lack of care resulting in serious or 
potentially serious complications, and (2) abnormal diagnostic fidings which 
are not addressed and resolved or where the record does not explain why they 
are not resolved. See appendix D for more information on generic quality 
screens. 
11.  The intensified reviews ranged from the next 10 cases with a particular 
diagnosis or procedure to 100 percent of all cases for a three-month period. 
many cases, the PRO kept a physician on intensifed review for three months 
following the ,completion of the educational intervention. 
12.  According to this PRO, its practice of retrospective reviews prior to imposing 
an educational intervention has since changed, and interventions will now be 
imposed within 30 days followig notifcation. 
13.  At that time, imposing an educational intervention could undermne the 
sanction recommendation by indicating the physician was willing and able to 
change. An amendment in the 1990 Omnibus Budget Reconcilation Act 
stipulates that a physician s failure to comply with a PRO's educational 
interventions can be considered evidence of his or her unwiingness and 
inabilty to change, thereby not interfering with the sanction recommendation. 
14.  PRO officials tyically consider a quality-of-care problem resolved when the 
intensified review revealed no further problems in the quarter following the 
interventions. 
15.  Institute of Medicine  Medicare: A Strategy for Quality Assurance  Vol. 1 
(Washington, D.  : National Academy Press, 1990). 
16.  All in  Medicare: A Strategy for Quality Assurance:  F. Davidoff, R. Goodspeed 
and J. Clive; "Changing Test Orderig Behavior: A Randomized Controlled 
Trial Comparing Probabilstic Reasoning with Cost-Containment Education 
58; J. Doctor s Decision and the Cost 
Medical Care  27 (1989): 45- M. Eisenberg, 
(An Arbor, Mich.: Health Administration Press, 1986); K. 
of Medical Care 
G 
 , "
Lohr, J.D. Winkler, and R.H. Brook  Peer Review and Technology Assessment in 
: RAD 
2820-0TA (Santa Monica, Calif. Corporation, 1981); S. 
Medicine, 
Schroeder  Strategies for Reducing Medical Costs by Changing Physicians 
Behavior  International Journal of Technology Assessment in Health Care 
(1987): 39-50.16. 
17.  Office of Inspector General Medical Licensure and Discipline: An Overview 
June 1986, and Offce of Inspector General  State Medical Boards and Medical 
August 1990.
Discipline, 
18.	 The National Practitioner Data Bank operates under Title IV of the Health 
Care Quality Improvement Act of 1986 (PL 99-660). 
begig 
19.	 Ths conflct has existed from the  of the PRO program and its 
predecessor, the Professional Standards Review Organization program. See 
Medicare: A Strategy for Quality Assurance,  pp. 145-147, op. cit. 
20.	 American Medical AssoCiation  Report of the Council on Medical Education: 
Focused Continuing Medical Education Programs for Enhanced Clinical 
Competence 89). 
, (1-
G