Department of Health and Human Services 
OFFICE OF 
INSPECTOR GENERAL 
The External Review of Hospital Quality 
The Role of Accreditation 
JUNE GIBBS BROWN 
Inspector General 
JULY 1999 
OEI-01-97-00051  OFFICE OF INSPECTOR GENERAL 
The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, is to 
protect the integrity of the Department of Health and Human Services programs as well as the 
health and welfare of beneficiaries served by them.  This statutory mission is carried out through a 
nationwide program of audits, investigations, inspections, sanctions, and fraud alerts.  The 
Inspector General informs the Secretary of program and management problems and recommends 
legislative, regulatory, and operational approaches to correct them. 
Office of Evaluation and Inspections 
The Office of Evaluation and Inspections (OEI) is one of several components of the Office of 
Inspector General.  It conducts short-term management and program evaluations (called 
inspections) that focus on issues of concern to the Department, the Congress, and the public.  The 
inspection reports provide findings and recommendations on the efficiency, vulnerability, and 
effectiveness of departmental programs. 
OEI's Boston regional office prepared this report under the direction of Mark R. Yessian, Ph.D., 
Regional Inspector General.  Principal OEI staff included: 
BOSTON  HEADQUARTERS 
Joyce M. Greenleaf, Project Leader  Jennifer Antico, Program Specialist 
Lynne G. Dugan, Senior Analyst  Alan Levine, Program Specialist 
Meredith A. Vey, Program Analyst 
To obtain copies of this report, please call the Boston Regional Office at 617/565-1050. 
Reports are also available on the World Wide Web at our home page address: 
http://www.dhhs.gov/progorg/oei  E X E C U T I V E   S U M M A R Y  
PURPOSE 
To assess the role of Joint Commission accreditation in the external review of hospital 
quality. 
BACKGROUND 
External Quality Review of Hospitals in the Medicare Program 
Hospitals routinely offer valuable services, but also are places where poor care can 
lead to unnecessary harm.  The external quality review of hospitals plays an important role 
not only in protecting patients from such harm, but also in complementing the hospitals 
own internal quality efforts.  The Federal Government relies on two types of external 
review to ensure that hospitals meet the minimum requirements for participating in 
Medicare:  accreditation, usually by the Joint Commission on Accreditation of Healthcare 
Organizations, and Medicare certification, by State agencies.  About 80 percent of the 
6,200 hospitals that participate in Medicare are accredited by the Joint Commission. 
This Inquiry 
This report, part of a series of four companion reports that resulted from our 
inquiry, focuses on the Joint Commission because it dominates the hospital accreditation 
market.  Our inquiry draws on aggregate data, file reviews, surveys, and observations 
from a rich variety of sources, including HCFA, the Joint Commission, State agencies, and 
other stakeholders. 
We organize this report around a framework we developed for considering the 
external review of hospital quality.  This framework consists of five components: 
announced, on-site surveys of hospitals; unannounced, on-site surveys of hospitals; 
responses to complaints concerning hospitals; responses to major adverse events in 
hospitals; and collection and dissemination of standardized performance measures. 
FINDINGS 
ANNOUNCED SURVEYS 
1 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  Joint Commission surveys are undertaken in a collegial manner and are tightly structured. 
This approach fosters consistency but leaves little room for probing. 
Surveys look the same hospital to hospital.  Surveyors are well-versed in the Joint 
Commission standards and aim to educate the hospital staff about the significance and 
intent of those standards.  To get an overview of the hospital, the surveyors maintain a 
rapid pace with survey sessions scheduled back-to-back, leaving little opportunity for 
following up leads or developing hunches. 
Joint Commission surveys serve as a means of both reducing risk and fostering attention to 
continuous quality improvement, but are unlikely to either surface substandard care or 
identify individual practitioners whose judgement or skills to practice medicine are 
questionable. 
Hospitals take Joint Commission surveys seriously.  The surveys prompt their 
attention to minimum protections that are important to patients and promote projects 
aimed at improvement.  But surveyors lack much background information on the hospital 
that could help them hone their survey, thus they get a broad rather than in-depth view of 
the hospital.  This, coupled with the tight structure, make it unlikely that the survey will 
identify patterns or instances of poor care. 
While they matter enormously to hospitals, Joint Commission survey results fail to make 
meaningful distinctions among hospitals. 
Hospitals attach great significance to survey results and use them as a way of 
distinguishing themselves from other hospitals.  However, the distinction between 
accreditation with commendation and accreditation with or without recommendations for 
improvement can be difficult to discern.  In fact, little variation exists in accreditation 
levels and scores:  99 percent of the hospitals surveyed between May 1995 and June 1998 
clustered in just 2 of the 5 possible accreditation levels. 
UNANNOUNCED SURVEYS 
The Joint Commissions reliance on unannounced surveys is limited. 
The Joint Commission conducts 1-day, random unannounced surveys to ensure 
continued compliance with accreditation standards between triennial surveys.  From June 
1995 through May 1998, it conducted such surveys, providing 24 to 48 hours notice, on 
about 5 percent of its accredited hospitals. 
RESPONSES TO MAJOR ADVERSE EVENTS 
2 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  The Joint Commission treats major adverse events as opportunities for improvement. 
Accordingly, it emphasizes education, prevention, and confidentiality but limits public 
disclosure on the causes, consequences, and responses to such events. 
The Joint Commissions sentinel event policy stresses self-reporting and analysis 
on the part of the hospitals.  Through this approach, it aims to develop a database of 
events that it can analyze for frequency and causes.  But ensuring confidentiality to self-
reporting hospitals limits public accountability.  This presents particular difficulties if, as it 
often the case, local concern is heightened because of media reports on the events. 
RESPONSES TO COMPLAINTS 
The Joint Commission devotes little emphasis to complaints. 
The Joint Commissions accreditation process is not particularly geared to dealing 
with complaints.  Although it receives complaints during surveys, surveyors must squeeze 
time from other survey activities to respond to them.  The Joint Commission also receives 
and responds to complaints centrally. 
STANDARDIZED PERFORMANCE DATA 
Despite the Joint Commissions early plans, standardized hospital performance data 
remain of little value to external assessments of hospital quality. 
In 1986, the Joint Commission unveiled its plans for a performance-based 
accreditation system that included uniform data from all hospitals.  But as that vision 
unfolded, the Joint Commission faced resistance from hospitals.  Accredited hospitals 
must now participate in a Joint Commission-approved measurement system (of which 
there are many), but collecting uniform data is as yet unrealized. 
CONCLUSION 
Unquestionably, the Joint Commission is the central force in the external review of 
hospital quality.  It accredits about 80 percent of the hospitals in the country and, for 
Medicare purposes, it has a congressionally granted deeming status that is unique among 
accrediting bodies.  Medicare beneficiaries and others who rely upon hospital services 
have much at stake in how and how well the Joint Commission does its job. 
Our review underscores that the core element of the Joint Commissions approach 
to accreditation is the announced, on-site survey of hospitals--a survey that is heavily 
oriented toward educational and performance improvement objectives.  The other 
elements of external review--unannounced surveys, responses to complaints and serious 
3 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  incidents, and standardized performance measures--play relatively minor roles in the Joint 
Commissions accreditation process. 
Given the significance of the Joint Commissions role and its emphasis on one 
approach to external quality review, our inquiry surfaces important policy questions for 
HCFA:  How can it best ensure an appropriate balance in external quality reviews of 
hospitals?  How can it best hold the Joint Commission accountable for the important 
public role it performs while enabling it, at the same time, to have enough flexibility to 
continue to advance the state-of-the-art of hospital accreditation?  We address these 
questions in our summary report, A Call for Greater Accountability.  That report also 
contains our recommendations, which we direct to HCFA. 
COMMENTS 
Within the Department of Health and Human Services, we received comments 
from HCFA.  We also solicited and received comments from the following external 
parties:  Joint Commission on Accreditation of Healthcare Organizations, Association of 
Health Facility Survey Agencies, American Osteopathic Association, American 
Association of Retired Persons, Service Employees International Union, National Health 
Law Program, and Public Citizens Health Research Group.  We include the detailed text 
of all of these comments and our responses to them in our summary report, The External 
Review of Hospital Quality: A Call for Greater Accountability (OEI-01-97-00050). 
4 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  T A B L E  O F  C O N T E N T S  
PAGE

EXECUTIVE SUMMARY

INTRODUCTION  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  6

FRAMEWORK FOR EXTERNAL QUALITY REVIEW OF HOSPITALS  . . . . . . . . . . .  9

PROFILE OF ACCREDITATION  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  10

ANNOUNCED SURVEYS 
Approach:  Collegial Tone and Tight Structure  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Strengths:  Dynamic Standards, Risk Reduction, Quality Improvement . . . . . . . . . . . . .  13

Limitations:  Unlikely to Uncover Substandard Care or Poor Performers  . . . . . . . . . . .  15

Scoring:  Valued by Hospitals, But Scores Make Few Distinctions . . . . . . . . . . . . . . . .  18

UNANNOUNCED SURVEYS 
Limited Reliance  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

RESPONSE TO MAJOR ADVERSE EVENTS 
Opportunities for System Improvement  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Lack of Public Disclosure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  25

RESPONSE TO COMPLAINTS 
Little Emphasis  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

STANDARDIZED PERFORMANCE DATA 
Currently of Little Value to External Review  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

CONCLUSION  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  29

APPENDICES 
A: Medicare Conditions of Participation  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

B: Methodology  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

C: Sample Accreditation Survey Agenda  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

D: Advertising Accreditation Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

E: Endnotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  I N T R O D U C T I O N  
PURPOSE 
To assess the role of the Joint Commission in the external review of hospital quality. 
BACKGROUND 
Hospital Safety 
Hospitals are an integral part of our healthcare system, offering services that 
improve, extend, and even save lives.  But they are also places where inappropriate care 
can lead to unnecessary harm.  This reality was clearly underscored in 1991, when a 
Harvard medical practice study revealed the results of its review of about 30,000 
randomly selected records of patients hospitalized in New York State during 1984.  The 
study found that 1 percent of the hospitalizations involved adverse events caused by 
negligence.1  On the basis of its sample, the study team estimated that during that year, 
negligent care provided in New York State hospitals was responsible for 27,179 injuries, 
including 6,895 deaths and 877 instances of permanent and total disability.  Many other 
more recent studies have reinforced the concerns raised by the Harvard study.  Of 
particular note was one that focused on the care received by 1,047 hospitalized  patients in 
a large teaching hospital affiliated with a medical school.  It found that 17.7 percent of 
these patients received inappropriate care resulting in a serious adverse event--ranging 
from temporary disability to death.2  In the public eye, these scholarly inquiries have been 
overshadowed by media reports that describe, often in graphic detail, the harm done to 
patients because of poor hospital care.3 
Hospitals rely upon a variety of internal mechanisms, from physician credentialing, 
to peer review and benchmarking, in order to try to avoid such incidents and to improve 
the quality of care provided in their facilities.  External quality review serves as a vital 
additional safeguard.  It provides a more detached, independent mechanism for assessing 
the adequacy of hospital practices.  Such oversight is of fundamental importance to 
patients and to the public and private entities that purchase health care services on their 
behalf.  Protecting patient safety and improving the quality of patient care must be a top 
priority of external review. 
Medicares Interest in External Hospital Quality Review 
The Medicare program covers about 38 million elderly and disabled individuals, 
many of whom are high users of hospital care.4  In 1997, Medicare spent $136 billion on 
Part A, the hospital insurance benefit.  This figure is just over half the total amount the 
6 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  Federal Government spent on all Medicare benefits.5  In the same year, Medicare spent 
over $80 billion for inpatient acute hospital care alone.6 
Since Medicares inception, external quality review has been a part of the 
Medicare program.  When Congress enacted the Medicare Act in 1965, it required 
hospitals to meet certain minimum health and safety requirements to participate in the 
program.7  Those minimum requirements, called the Medicare conditions of participation, 
were published in 1966, revised in 1986, and are now being revised again (see appendix 
A).8  Within the Medicare Act itself, however, Congress provided that hospitals accredited 
by the Joint Commission on Accreditation of Healthcare Organizations were deemed to be 
in compliance with the conditions of participation.  Congress also provided that hospitals 
accredited by the American Osteopathic Association could be considered in compliance, 
but only to the extent that the Secretary deemed appropriate.9  Thus, accreditation by the 
Joint Commission or the American Osteopathic Association provides entree into the 
Medicare program.  About 80 percent of the 6,200 hospitals that participate in Medicare 
are accredited by the Joint Commission.  Those hospitals wishing to participate in 
Medicare without accreditation must go through a Medicare certification process.  The 
Health Care Financing Administration (HCFA) relies on State survey and certification 
agencies (hereafter called State agencies) to conduct certification surveys at these 
hospitals to determine compliance with the Medicare conditions.  States currently certify 
1,442 nonaccredited hospitals nationwide.10 
Regardless of the route a hospital takes to Medicare participation, Medicare bears 
a cost for the external review, either directly by funding State surveys or indirectly through 
hospital charges that include the overhead cost of periodic accreditation surveys. 
The Nature of Accreditation 
Accreditation is a form of self-regulation.  It is a voluntary conformity-
assessment process whereby industry experts define what standards organizations must 
meet in order to be accredited and then systematically review the organizations 
performance against those standards.11  It typically aims to improve performance.12 
Organizations wishing to be accredited pay a fee for that service.13 
This Inquiry and Report 
This report examines the role of accreditation in external hospital quality review. 
It focuses on the Joint Commission on the Accreditation of Healthcare Organizations 
because it dominates the hospital accreditation market.14  In this report, we offer a 
framework for considering the external review of hospital quality and apply that 
framework to the Joint Commissions approach to hospital accreditation. 
7 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  This report is part of a series of four companion reports that resulted from our 
inquiry.  A second, The Role of Medicare Certification (OEI-01-97-00052), describes the 
extent and nature of the external review for nonaccredited hospitals.  A third, Holding the 
Reviewers Accountable (OEI-01-97-00053), describes how HCFA oversees both the Joint 
Commission and the State agencies.  The fourth report, A Call for Greater Accountability 
(OEI-01-97-00050), provides a summary of external hospital quality review and presents 
the recommendations emerging from our inquiry. 
Our inquiry draws on a variety of sources.  These include:  data from HCFAs 
Online Survey Certification and Reporting System; aggregate data from the Joint 
Commission concerning hospital survey activity; a mail survey to State agencies in the 50 
States and District of Columbia (hereafter referred to as a State); observations of seven 
routine hospital surveys conducted by the Joint Commission and one conducted by a State 
agency; observations of two separate complaint surveys conducted by a State Agency; 
reviews of accreditation manuals, policies, and hospital survey files from the Joint 
Commission; the systematic gathering of information from representatives of HCFA 
central and regional offices, State agencies, the Joint Commission, American Hospital 
Association, consumer groups, professional associations, and representatives of other 
organizations we considered to be stakeholders in hospital oversight; and reviews of laws, 
regulations, and articles from newspapers, journals, newsletters, and magazines.  We also 
interviewed officials from the American Osteopathic Association and reviewed their 
accreditation materials.  The American Osteopathic Association accredits about 100 to 
150 hospitals, some of which are also accredited by the Joint Commission.  For purposes 
of this inquiry, however, we focused on the Joint Commission.  See appendix A for more 
details on our methodology. 
We conducted this inspection in accordance with the Quality Standards for 
Inspections issued by the Presidents Council on Integrity and Efficiency. 
8 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  HOSPITAL  REVIEW  FRAMEWORK 
The following five components present a framework for considering the external quality review 
of hospitals. They are intended to complement the internal quality assurance and improvement efforts 
that hospitals undertake themselves. They are approaches that health care purchasers, such as Medicare 
and Medicaid, can rely upon to ensure that their beneficiaries receive quality services from hospitals. 
They can also be of use to beneficiaries and other consumers concerned about the quality of their hospital 
care. 
We present the components to facilitate analysis of the extent and type of external review that is 
desirable, whether carried out by accreditation bodies, certification agencies, Medicare Peer Review 
Organizations, HCFA, or others. Each component has strengths and limitations. Moreover, each can be 
used in support of a review philosophy based on continuous quality improvement, more traditional 
compliance enforcement, or some combination thereof. 
We omitted a sixth component: the retrospective review of medical records to determine 
appropriateness of care. Formerly a role of the Medicare Peer Review Organizations, such medical record 
review is no longer carried out on such a large scale. However, some medical record review does occur as 
part of the components described below. 
1. Announced, On-Site Surveys of Hospitals 
These involve some combination of observations of facility and equipment; reviews of medical 
credentials, and other records and documents; and interviews. They result in a pass/fail or some kind of 
score intended to distinguish level of performance. They can also involve follow-up to correct or improve. 
2. Unannounced, On-Site Surveys of Hospitals 
The approach is basically the same as above except that the hospital has not had time to prepare. 
The intent is to gain a clear assessment of the facility as it typically functions and to trigger any necessary 
follow-up. 
3. Response to Complaints Concerning Hospitals 
These involve complaints of a particular instance of care or more encompassing matters 
concerning a hospitals performance. The response to complaints can range from a minimal distant 
review to a thorough on-site review. The process can trigger corrective actions and system improvements. 
4. Response to Major Adverse Events in Hospitals 
These involve cases where substantial patient harm resulted from what may be poor performance 
on the part of the hospital and/or its practitioners. Here, too, the response can range from minimal to 
thorough and can trigger corrective actions and system improvements. 
5. Collection and Dissemination of Standardized Performance Measures 
The aim here is to establish the standardized use of measures in ways that enable purchasers, 
consumers, accrediting bodies, and others to compare hospital performance. The comparisons can focus 
on a hospitals own performance over time and/or on how its performance compares to other hospitals. 
The data can be drawn from surveys of patients or providers, billing claims, and the hospitals own 
records. 
9 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  P R O F I L E   O F   A C C R E D I T A T I O N  
The table below summarizes our assessment of the Joint Commissions approach 
to hospital accreditation, based on the framework we presented on the previous page. 
The remainder of the report follows this framework and elaborates on the five elements of 
the summary. 
Element  Description  Degree of Emphasis 
Routine, Announced, On-Site  Conducted every 3 years  Major. Core of existing accreditation process 
Surveys  to measure compliance 
with over 500 standards 
in 45 performance areas. 
Random, Unannounced, On- Conducted on a sample  Minor. Affects about 5 percent of accredited 
Site Surveys  of hospitals as a measure  hospitals per year. 
of continued compliance 
with standards in five 
performance areas. 
Responses to Complaints  Includes tracking for  Minor. Few complaints result in on-site surveys; 
trends but also some  most are reviewed for trends. Limited attention to 
follow-up on-site,  complaints during triennial surveys. 
depending on the nature 
of the complaint. 
Responses to Major Adverse  Policy calls for hospital  Minor to Moderate. Emphasis has been growing. 
Events  self-reporting and 
determining cause of 
event. Focus is on 
identifying opportunities 
for improvement and 
education. 
Standardized Performance  Hospitals participate in  None. Performance data not linked to 
Data  one of many Joint  accreditation. 
Commission-approved 
indicator system and 
begin submitting data in 
1999. 
10 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  A N N O U N C E D   S U R V E Y S  
APPROACH 
Joint Commission surveys are undertaken in a collegial manner. 
Joint Commission surveyors are experienced nurses, physicians, and 
administrators.  They are keenly aware of their roles as outside reviewers responsible for 
documenting compliance with Joint Commission standards.  But they seek to carry out 
that role more as peer reviewers than as enforcers or regulators.  In interacting with 
hospital staff, they strive to foster a collegial atmosphere characterized by a joint interest 
in improving hospital performance. 
It follows, therefore, that surveyors view education as a central part of their 
mission.  Well-versed in the Joint Commission standards, they tend to be ambassadors of 
those standards, helping hospital staff to understand their intent and significance.  Beyond 
that, they also use the survey as a way of informing hospitals about promising approaches 
they have found at other hospitals, even providing the names of individuals to contact. 
For example, we observed surveyors informing hospital staff of alternate methods for 
monitoring crash carts and sharing other hospitals approaches to defining privilege lists 
and adverse drug reactions.  We also observed a surveyor briefly chat with a patient and 
then scan his chart to identify how the hospital staff recorded the goals set for this patient. 
While the surveyor identified dietary, rehabilitation, and medical goals in the chart, the 
chart lacked documentation reflecting the patients own stated goal:  to ride his 
motorcycle again.  This led to some educational discussion between the surveyor and 
caregivers on reflecting goals not only for individual disciplines, but also in terms of the 
patients and their needs as individuals. 
Given their collegial and educational orientation, surveyors tend to approach their 
information gathering in a nonchallenging manner.  For example, they allow the hospital to 
select the majority of records to be reviewed.15  They tend to be accepting of information 
and answers provided by the hospital staff, with little probing to determine if surface 
appearances describe actual practice.  They seldom reflect any wariness that hospital staff, 
in their eagerness to impress surveyors, may exaggerate points or omit information.  For 
example, in one hospital, a surveyor scanned some performance data prominently 
displayed on a storyboard, questioned the cause of significant deviation in the charted 
data, but when staff failed to respond did not pursue the matter.  In another, a surveyor 
noted with surprise that the hospital staff had failed to pull certain files he requested, but 
did not insist that the requested files be provided. 
11 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  Joint Commission surveys are tightly structured. 
The Joint Commission uses a standard agenda in surveying hospitals.  In 2 to 5 
days, depending on hospital size, surveyors conduct dozens of individually scheduled 
sessions that cover the major aspects of a hospitals operations (see appendix B for a 
sample agenda).  The survey pace is rapid, typically allowing 45 to 60 minutes for each 
meeting or patient care visit before moving on to the next visit.16  A few 15-minute 
intervals might be allotted each day as unscheduled for surveyors personal time or phone 
calls.  Sessions are scheduled back to back. 
The survey provides surveyors with a general overview of the hospital.  It 
facilitates standardized assessments across facilities and protects against inconsistencies or 
the special interests of individual surveyors.  Also, because the schedule, which hospitals 
receive in advance, spells out exactly which surveyor will be where over the course of the 
survey, individual departments can prepare by having everyone ready for the surveyor and 
the area cleaned and organized.17 
The packed agenda, however, affords few opportunities for surveyors to develop 
hunches, follow leads, or even respond to complaints.  Racing from session to session, 
surveyors have little time to probe deeply.  As one surveyor described after surveying a 
700-bed hospital, The team found this to be a harried, rushed survey due to the number 
of areas covered and the timeframe allowed...Patient units were cut short to allow for 
travel time between buildings...The time allotted for the building tour and medical record 
review was grossly inadequate.18 
The surveyors know little going in to a hospital that might help them probe more 
effectively.  Their advance knowledge is limited to some basics on the services offered, 
last triennial survey results, and, in some cases, complaints and media articles.  They know 
little, if anything, about the local health care market in which the hospital operates, 
including whether there are local labor shortages or union disputes.  While such limited 
knowledge deliberately reduces bias and promotes consistency, it also hinders targeted 
probing and a deeper understanding of the hospitals operations.  For example, on the first 
day of a hospital survey, the local newspaper published a lead editorial about a heated 
battle between this hospital and another for dominance in childrens services.  This 
competition was much on the minds of the hospitals leadership, but the surveyors failed 
to ask about it because they were unaware of it.  The surveyors were also unaware that 
the hospital had a random unannounced survey 18 months earlier. 
12 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  STRENGTHS 
Joint Commission standards are dynamic. 
The Joint Commissions ability to quickly adapt the standards that form the 
foundation of the accreditation process enables it to respond to changes in health care 
delivery and evaluation methodology.  The Joint Commission is constantly developing 
new, refining existing, and eliminating out-of-date standards.  Practitioners, experts, and 
stakeholders from around the country weigh in on this process through the Joint 
Commissions committee structure.  That committee includes representatives from 
professional and industry groups, such as the American Medical Association, the National 
Association of Healthcare Quality, and the American Academy of Family Physicians, as 
well as from the Health Care Financing Administration.  The stakeholders we surveyed 
considered ability to improve standards without a long regulatory process to be one of the 
greatest strengths of the accreditation process.  Even a critic of the Joint Commission 
conceded that the standards are wonderful. [One] cant help but admire them. 
Over the past decade, the Joint Commissions standards changed dramatically.19 
The new and revised standards reflect a more patient-centered focus, with greater 
emphasis on patient rights.  The Joint Commission replaced old standards concerning 
committee meeting minutes with standards on strategic planning, resources for continuous 
quality improvement, and managing information.  These changes were so substantial that, 
by 1996, the Joint Commission eliminated 80 percent of standards from its 1986 manual.20 
Joint Commission surveys serve as a means of reducing risk. 
At its core, the survey process is about reducing risk.  The surveys prompt 
hospitals attention to minimum protections that are important to patients.21  Hospitals 
know that once every 3 years the surveyors will be onsite and they take the accreditation 
process seriously.22  They know the survey dates in advance and prepare for them.  The 
survey process gives the hospitals leadership a platform from which to rally attention to 
assessing its condition and correcting problems in time for the survey; hospitals want to 
put their best foot forward.  For example, prior to surveys, hospitals put extra efforts in 
their appearance and make sure manuals are up-to-date.  Many hospitals hire consultants 
to help them with this process by conducting mock surveys or reviewing existing 
conditions to identify areas falling short of accreditation standards.  While some of these 
efforts are cosmetic, others extend beyond the cosmetic and include efforts such as 
training staff on standards. 
Of course, the Joint Commission also has a platform from which it can promote 
particular patient-centered concerns by simply making them survey priorities.  For 
example, over the past several years, the Joint Commission has identified hospital use of 
patient restraints as a such a priority.23  Indeed, surveyors found problems with restraints 
13 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  at 55 percent of hospitals surveyed between June 1995 and May 1997.24  Hospitals were 
well aware of this focus--newsletters and Internet chat rooms devoted to accreditation 
surveys often feature survey trends--and responded through efforts such as improvement 
projects aimed at reducing restraint use.25  Other topics garnering attention from the Joint 
Commission recently include smoking in hospitals, conscious sedation, and emergency 
medicines. 
In addition to focusing on priority areas such as restraints, the triennial surveys 
reinforce attention to other, basic protections that span administrative and clinical realms. 
For example, surveyors review hospitals for their fire safety procedures, including 
reviewing fire drill logs and checking firedoors and firewalls.  They check the security and 
expiration of medicines and supplies.  They look for infection control issues such as 
separation of clean and dirty.  They check how hospitals record drug allergies and monitor 
adverse drug reactions.  They review the timely inclusion of histories and physicals in the 
medical records.  They verify the existence and staff knowledge of hospital policies.  For 
example, at one hospital, we observed the nurse surveyor review policies and minutes of 
the ethics committee.  He then verified that nurses on patient care units used the 
committee appropriately by asking about their most recent referrals to it. 
Joint Commission surveys foster attention to continuous quality improvement. 
Consistent with its mission to improve the quality of care provided to the public 
through...performance improvement in health care organizations, the Joint Commission 
promotes the use of continuous quality improvement through the measurement of 
performance based-standards.26  Continuous quality improvement has been a major trend 
in health care over the past decade, and the Joint Commission adopted its focus on 
measurement and outcomes through its Agenda for Change.27  In fact, 88 percent of the 
stakeholders we interviewed expect 
accreditation to play a great or very 
Fostering An Understanding of Measurement 
great role in fostering continuous 
quality improvement in hospitals. 
While surveying a unit serving the frail elderly, 
a nurse surveyor engaged staff in a discussion 
Attention to continuous 
about identifying and measuring patient 
quality improvement pervades the 
outcomes. Through careful questioning, he led 
entire hospital survey process.  The 
the staff to understand the difference between the 
performance improvement overview 
process of care (in this case, a moist healing 
session, one of the first sessions of  environment for a pressure ulcer) and the 
the survey, provides an opportunity  outcome of care (a smaller and, eventually, 
for hospital staff to present their  healed ulcer).  He also helped the staff 
performance improvement  understand the importance of measuring the 
methodology, to which surveyors  impact of the staffs interventions on patient 
refer throughout the survey.  outcomes. 
14 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  Surveyors reinforce the importance of  quality improvement in sessions with hospital leaders and 
medical staff, where they confirm leadership involvement and ask how projects are identified and 
prioritized.  The survey agenda also reserves specific sessions for performance improvement 
teams to showcase their most successful projects for surveyors.  The performance improvement 
projects presented may involve topics such as reducing patient waiting times, improving the 
timeliness of medical record retrieval, reducing medication errors, and reducing the use of patient 
restraints.  As the surveyors move throughout the hospital, staff are eager to explain the 
performance improvement storyboards on display in their units.  And surveyors often ask staff 
what their units are doing better this year than last and what they will do better next year. 
Depending on the hospitals sophistication in performance improvement, surveyors 
act as either educators, sharing advice and encouragement on measuring outcomes, or as 
cheerleaders, applauding the hospitals performance improvement efforts.  Some hospitals 
have moved beyond Joint Commission requirements, hiring consultants to help them with 
their performance improvement efforts, sending medical staff to training on these 
techniques, or paying medical staff for time spent on performance improvement projects. 
In hospitals with sophisticated performance improvement projects, surveyors offer 
compliments and collect ideas to pass on to other hospitals. 
LIMITATIONS 
Joint Commission surveys are unlikely to surface patterns, systems, or incidents 
of substandard care. 
Looking for questionable care runs counter to the tight structure and collegial 
nature of the Joint Commissions survey process.  That process leaves little opportunity 
for identifying such patterns, systems, or incidents.  It is more oriented toward reducing 
risks, improving performance, and educating. 
The Joint Commissions approach to medical record review illustrates its limits. 
While surveyors review both closed and open medical records during surveys, their focus 
is more on processes of care than appropriateness of care.  In fact, surveyors do not 
judge directly whether the care given to a specific patient is good or bad, right or 
wrong.28  Nor do they choose medical records based on indications of poor quality. 
Rather, the hospitals generally choose the records for surveyors, and, in some cases, 
hospital staff also conduct the review.  While Joint Commission surveyors find problems-
even patterns--through medical record review, they tend to be related to processes, such 
as documenting verbal orders, timely recording of the history and physical, and assessing 
patients educational needs.  While important, these process issues are apparent through 
even a quick review of the records.  The Joint Commissions approach is unlikely to reveal 
problems or patterns that might take a more thorough examination to uncover, such as 
inappropriate surgeries, high complication rates, or poor or unexpected outcomes. 
15 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  The Joint Commission survey process falls short of identifying patterns or systemic 
weaknesses because, at least in part, surveyors lack the time or the up-front knowledge 
that would help them dig deeper.  They get a broad rather than in-depth view of the 
hospitals operations.  Thus, the survey would be more apt to reveal unsecured narcotics 
or outdated drugs than it would a pattern of mismedication.  It would be more apt to 
reveal lapses in fire drills than lapses in continuity of care.  It would be more apt to reveal 
overdue staff evaluations than staff performing tasks for which they are untrained. 
Indeed, the survey process may not reveal any problems when in fact systemic 
problems exist.  For example, in the Spring of 1996, the Joint Commission awarded one 
hospital its highest level of accreditation:  accreditation with commendation.  That Fall, 
the hospital experienced an unexpected death, triggering the State agency to investigate. 
In the spring of 1997, more unexpected deaths occurred, and the agency returned.  After a 
3-week investigation, that agency found systemic problems in both quality assurance and 
medical staffing.29  While no system of oversight is foolproof, the Joint Commission 
survey process failed to uncover what the State agency identified as deep-rooted 
problems. 
Joint Commission surveyors have limited information from which to draw to help 
them identify patterns or incidents.  For example, surveyor preparation packets lack 
practice pattern or performance data that could provide surveyors with insights into 
identifying patterns of questionable care.  In fact, the surveyors have virtually no 
interactions or information from other stakeholders, such as Medicare Peer Review 
Organizations or consumer groups, who might have insights into the hospitals 
performance. 
Joint Commission surveys are unlikely to identify individual practitioners whose 
judgment or skills to practice medicine are questionable. 
The collegial process, tight structure, limited time, approach to medical records, 
and lack of background information all make it unlikely that the survey process will 
identify individual practitioners who pose risks to their patients.  In fact, the Joint 
Commissions own publications note that the accreditation process does not evaluate the 
quality of care provided by individual medical staff members and does not provide a 
warranty that a particular individual will receive quality care in a specific health care 
setting at a particular time.30 
A standard part of an accreditation survey is a review of hospitals own processes 
in ensuring the competence of their practitioners.  But that review of credentialing and 
privileging offers, at best, a preliminary and superficial assessment.  It generally lasts 45 to 
60 minutes, during which the surveyor both interviews the medical staff leadership and 
reviews files (in some cases, the hospital staff also reviews files). 
16 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  While the hospitals generally choose the files for review in the credentials session, 
they do so at the direction of the surveyors, ensuring different privileges (active, courtesy, 
consulting, and temporary, for example) and specialties are represented.  But time is too 
short for any in-depth review of these files.  For example, in one hospital with over 500 
active staff, the surveyor reviewed three practitioners credentials and privileges.  Even 
though he found a problem with one of the three (a podiatrist who was performing surgery 
for which he was not privileged), the surveyor reviewed no additional files. 
The credentials session falls short in other ways, as well.  While the surveyors do 
ask important questions about the hospitals processes for matching privileges to 
competencies and verifying licenses, we observed no surveyor ask how the hospital 
identified or dealt with physicians whose knowledge or practice skills were marginal.31 
Even though Federal law requires hospitals to report any practitioner they have disciplined 
to the National Practitioner Data Bank, it is unlikely that surveyors will make any 
determination about hospitals compliance with that law:  Joint Commission standards do 
not currently reference the law for reporting to the Data Bank, and the likelihood is low 
that surveyors will choose a file of a disciplined physician randomly.32  In fact, one 
physician surveyor chastised the staff for pulling confidential physician files for the 
surveyor to review.  Such files contain peer review assessments and other details on the 
physicians skills. 
The collegiality of the sessions hinders raising sensitive topics, such as physicians 
fitness to practice, in a meaningful way.  For example, during the review of credentials, 
one physician surveyor shared a story about a hospital that was about to reduce privileges 
of an older surgeon whose skills had declined.  The surgeon had practiced at the hospital 
for many years and was a stalwart of the hospital community and respected throughout the 
State.  Before the hospital could act, the State legislature named the surgeon Physician of 
the Year.  The story was met with nervous laughter by the medical staff.  While the story 
touches on the difficulties of dealing with those whose skills are declining, the surveyor 
followed up with neither questions of whether the hospital was faced with anything similar 
nor education about how to deal with such situations. 
Other survey sessions offer some insights into the practitioners skills, but only 
peripherally.  For example, the document review session provides an opportunity for 
surveyors to review any minutes or records that might document some problems with 
individuals, though whether the surveyors would have the time to identify those 
documents amidst the hundreds they scan is uncertain at best.  The surveys also include 
interview sessions with nursing and medical leadership.  But the high attendance and 
collegial atmosphere at these sessions is generally counter to discussing individuals. 
17 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  SCORING 
Joint Commission survey results matter enormously to hospitals. 
Hospitals attach great significance to accreditation scores and status.  They know 
the accreditation scores of competing hospitals, and aim to beat them with higher scores 
of their own.  It is not enough to simply be accredited--nearly every hospital surveyed gets 
accredited--they want to be accredited with commendation, the highest level awarded by 
the Joint Commission.  After all, hospitals operate in an increasingly competitive 
marketplace, where edging out the competition in gaining contracts with managed care 
plans and other purchasers can be the difference between surviving and going out of 
business.  They know that their survey results will be available to the public.33  Thus, 
hospitals that do achieve accreditation with commendation promote their survey results 
(see appendix D).34  They advertise their status in banners hung outside their hospitals. 
They announce it in full-page ads in their local papers.  They post it on their websites and 
throughout the hospital. 
This focus on the survey scores permeates the survey process, introducing some 
tension into the otherwise collegial process.  Surveyors know hospitals aim to do well on 
their surveys--after all, the hospital leadership often follows them around during the entire 
survey process.  Hospitals often hire consultants to help them prepare and increase the 
likelihood they will achieve accreditation with commendation.  Surveyors and hospitals 
alike know that just one deficiency can thwart accreditation with commendation.  This can 
lead to pressure to mitigate the impact of deficiencies, which can play out in a couple of 
ways.  One approach, for example, involves what we have dubbed the denominator game: 
increasing the numbers of documents reviewed (the denominator) in order to decrease the 
significance of problems identified in any one record (the numerator).35  We observed 
hospitals attempting to mitigate findings on restraints, dental histories in the medical 
record, and security of drugs with this approach, with varied success.  Surveyors, of 
course, also have some flexibility in scoring the standards.  For example, we observed one 
surveyor identify a problem with a podiatrist performing surgery that exceeded his 
privileges.  The surveyor scored this as a documentation rather than medical staff problem, 
thereby mitigating its impact on the final score. 
18 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  Joint Commission survey results fail to make meaningful distinctions among 
hospitals. 
Even though the Joint 
Table 1 
Commission has 5 levels of 
Accreditation Levels for Hospitals Surveyed 
accreditation, 99 percent of 
June 1995 - May 1998 
the hospitals surveyed 
Accreditation  6/95  6/96  6/97 
between May 1995 and June 
Level  to  to  to  Total 
1998 clustered in just 2 of 
5/96  5/97  5/98 
those levels (see table 1): 
accreditation with  Accreditation with 
Commendation  13%  18%  15%  16% 
commendation (16 percent) 
and accreditation with type 1  Accreditation without 
Type 1 Recommendations  <1%  1%  0  <1% 
recommendations (83 
percent).36  The difference  Accreditation with Type 1 
between some levels of  Recommendations  86%  80%  83%  83% 
accreditation can be difficult 
Conditional Accreditation  <1%  1%  1%  1% 
to discern.  For example, 
Preliminary Non-
hospitals accredited without 
Accreditation  0  0  <1%  <1% 
recommendations for 
Source: Joint Commission on the Accreditation of Healthcare

improvement may miss the 
Organizations.

commendation status by a  Note: n=17130 for 6/95-5/96 , n=1709 for 6/96-5/97, and n=1719

narrow margin.  On the other  for 6/97-5/98.

hand, accreditation with 
recommendations for improvement (type 1 recommendations) is a broad category that 
encompasses hospitals with just one recommendation to hospitals having many.37 
In addition to the accreditation level, hospitals accredited by the Joint Commission 
receive a score based on a 100-point scale.  Fully 99 percent of hospitals accredited in 
1997 received scores over 80: 73 percent received scores of 90 or higher and 26 percent, 
from 80 to 89.  Just 1 percent received from 70 to 79.38  Given the Joint Commissions 
own interpretation of scores, this spread implies few real differences across accredited 
hospitals: 
The smaller the difference in scores between hospitals, the less likely there is an actual 
difference in the levels of performance between them. There may not be a real difference 
between a hospital that scores 88 and a hospital that scores 81.  However, the greater the 
difference in scores, the more likely there is a difference in patient care.39 
On at least one occasion, the Joint Commission has found its own survey too blunt 
a tool to clearly distinguish a hospital as unacceptable.  In this case, two State agencies, 
HCFA, and the Joint Commission were engaged in surveying and resurveying a hospital. 
Concerns about this hospital ranged, for example, from rat infestations to the hospital 
taking away a patients mattress and bedframe, to improper patient restraints.  Yet it was 
19 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  not until the Joint Commission devised a special survey that the hospital failed to achieve 
accreditation. 
20 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  U N A N N O U N C E D  S U R V E Y S  
The Joint Commissions reliance on unannounced surveys is limited. 
The Joint Commission uses random unannounced surveys as a way to ensure 
continued compliance with accreditation standards between triennial surveys.40  From June 
1995 through May 1998, the Joint Commission conducted 250 random unannounced 
surveys, representing about 5 percent of its accredited hospitals.  Eleven percent of those 
surveys (28) resulted in a change in the hospitals accreditation status.  The Joint 
Commission required some corrective follow-up (a written progress report or a focused 
survey) for about a third (80, or 32 percent). 
The Joint Commission does, in fact, give advance notice of its unannounced 
surveys.  But unlike the triennial surveys, which are scheduled several months in advance, 
the Joint Commission provides considerably less notice to hospitals undergoing these 
random surveys: generally 24 to 48 hours.  By contrast, the New York State Health 
Department recently sent 72 inspectors to 12 hospitals without any warning at all.  The 
inspectors arrived at the hospitals on a Thursday afternoon and spent the weekend 
investigating the hospitals supervision of residents.41  State agencies, in conjunction with 
the Health Care Financing Administration, often give no warning to hospitals prior to 
showing up to investigate a complaint or incident of patient harm.42  In fact, in the case of 
nursing homes, Federal law allows a fine of up to $2,000 for anyone warning the facility of 
an upcoming survey.43  Some hospitals appear to recognize some intrinsic value to the 
surprise of an unannounced evaluation, as they themselves hire people to pose as patients 
and then report on their experience.44  The Joint Commission has not used that approach. 
The focus of random unannounced surveys is narrower than the triennial surveys. 
Rather than encompassing all hospital accreditation standards, they focus on the top five 
performance areas that the Joint Commission finds out of compliance.  Because of the 
narrower focus, one surveyor conducts the random unannounced and the entire survey 
takes 1 day.  The one random unannounced survey we observed began at 8:30 in the 
morning and ended at 3:15 in the afternoon. The Joint Commission notified that hospital 
24 hours before the surveyor, an administrator, arrived at the hospital.  The survey 
resembled the triennial survey in approach.  For example, the surveyor conducted the 
survey in the same collegial manner as in a triennial; he often focused on educating the 
hospital staff on particular standards and their intent.  His advance knowledge of the 
hospital was limited to a copy of the notice to the hospitals executive explaining the focus 
of the survey and a sheet describing some basic information about the hospital and its 
services.  He displayed little skepticism.  In fact, during one 5-minute interview of the 
human resources staff, the surveyor proclaimed their answers to be textbook perfect, 
thereby negating his need to review any of the files they had pulled for him. 
21 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  R e s p o n s e s  t o  M a j o r  A d v e r s e  E v e n t s  
The Joint Commissions approach to major adverse events has evolved to one 
that addresses such events as opportunities for system improvement.  It is in 
accord with recently developed precepts on reducing errors in medicine. 
In recent years, as the media have reported many events involving serious harm to 
hospitalized patients, the Joint Commission has devoted more attention to them under the 
rubric of its sentinel event policy.45 
When it first established this policy in 
April 1996, it assessed the hospitals 
The Joint Commissions Definition of a 
responsibility for them through an on-
Sentinel Event 
site investigation.46  If the Joint 
Commission found that the incident  (1) The event has resulted in an unanticipated death 
or major permanent loss of function, not related to 
was, indeed, serious, and that the 
the natural course of the patients illness or 
hospital could reasonably have been 
underlying condition, or (2) The event is one of the 
expected to prevent it, then the Joint 
following (even if the outcome was not death or 
Commission typically downgraded  major permanent loss of function): Suicide of a 
the hospitals accreditation to  patient in a setting where the patient receives 
conditional--a publicly disclosed  around-the-clock care (e.g., hospital residential 
treatment center, crisis stabilization center); Infant 
status that called for the hospital to 
abduction or discharge to the wrong family; rape; 
determine the cause of the event.47 
Hemolytic transfusion reaction involving 
administration of blood or blood products having 
Since then, the Joint  major blood group incompatibilities; or Surgery on 
Commissions sentinel event policy  the wrong patient or wrong body part. 
has evolved to one that is gentler to 
hospitals.  Its first important move in 
this direction was in October 1996 when it replaced the downgrade to conditional 
accreditation, which hospitals regarded as unnecessarily punitive, with a status called 
Accreditation Watch-- a publicly discloseable status but one that in itself did not affect a 
hospitals existing accreditation status.  More significantly, in April 1998, the Joint 
Commission recast its overall approach toward such events to one that is much more 
analytic than investigative.  This altered approach builds on the premise that medical errors 
occur regularly, even in the best hospitals.  It seeks continuous quality improvement in 
health care by treating  errors as opportunities to improve systems of care rather than to 
find and punish culpable individuals or institutions.  Toward this end, it aims to create  a 
protected venue where it is safe to acknowledge errors and where practitioners and 
administrators analyze them to develop better error-prevention strategies--ones that get at 
the causes for errors.48 
22 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  Thus, the Joint Commissions current policy on major adverse events stresses 
voluntary and timely self-reporting both of the incidents and of the hospitals analyses of 
them.49  Hospitals that proceed in this way avoid Accreditation Watch and are much less 
likely to be subject to an on-site investigation of an incident.  Through this approach, the 
Joint Commission expects to develop a substantial database of events that it can analyze 
for patterns of frequency and causes.50  It intends to communicate this information to 
accredited hospitals so that they can prevent such events. 
It is still early to fully assess the effects of this new approach.  But the initial 
indications suggest that this gentler, educationally oriented approach is beginning to take 
hold.  From April 1, 1998 (when the policy was implemented) until January 7, 1999 (the 
latest date for which data are available), the Joint Commission has been less inclined to 
place hospitals on Accreditation Watch or to change their accreditation status as a result 
of an adverse event, and hospitals have been more inclined to self-report sentinel events to 
the Commission (see table 2).51 
23 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  Table 2 
Sentinel Event Tally 
Sentinel Events in the Database 
Total Number of Events Added to the

Joint Commission Database between

4/1/96 and 1/7/99  ........................... 374

Number Self-reported

4/1/96 - 3/31/98 ......................  36

Number Self-reported since

the New Policy (4/1/98 and 1/7/99)  ....... 115

Hospitals on Accreditation Watch 
Total Number of Hospitals Placed on 

Accreditation Watch 10/1/96 - 1/7/99  ............. 38

Number 10/1/96 - 3/31/98  ............... 35

Number since the New Policy 

(4/1/98 and 1/7/99)  ..................... 3

Hospitals with an Accreditation Status Change 
Total Number of Hospitals with a Change in

Accreditation Status Due to Sentinel Event 

Review between 10/1/96 and 1/7/99.............. 149

Number 10/1/96 - 3/31/98  .............. 146

Number since the New Policy 

(4/1/98 and 1/7/99)  ..................... 3

Source: Joint Commission on the Accreditation of Healthcare 
Organizations. 
24 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  The Joint Commissions approach emphasizes confidentiality, thereby inhibiting 
public disclosure on the causes, consequences, and responses to major adverse 
events. 
For hospitals complying with the Joint Commissions self-reporting format, the 
Joint Commission does not publicly release any information, nor upon inquiry of a 
hospitals accreditation status does it identify that any analysis is underway.  If the inquirer 
specifically mentions the adverse event, the Joint Commission simply indicates that it is 
working with the hospital on it.  But even with this restrictive approach, hospitals have 
been wary of the self-reporting policy as they fear that the confidentiality of the Joint 
Commissions database would not be adequately protected under State and Federal laws 
and that as a result they would be exposed to malpractice liability.52  The Joint 
Commission has been examining legislative remedies, at State and Federal levels,  to 
address this concern and to buttress its limited disclosure policy. 
This emphasis on the confidentiality of adverse event related information may 
facilitate the quality improvement agenda of the Joint Commission, but it is at the expense 
of its public accountability.  It leaves the general public with minimal information about 
how an incident happened and what if anything is being done to prevent more.  This 
presents particular difficulties if media reports of the incident heighten local concern.  In 
such cases, the public is likely to be looking for assurances from an objective outside party 
that any problems at the hospital have been addressed and that patient safety is maintained. 
Under its current  policy, the Joint Commission can offer few specifics to support any such 
assurances.53  The results of investigations conducted by State agencies or HCFA itself are 
likely to be more available to the public. 
25 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  R E S P O N S E S   T O   C O M P L A I N T S  
The Joint Commission devotes little emphasis to complaints. 
The Joint Commissions accreditation process addresses complaints more 
tangentially than centrally.  Although it receives and responds to complaints during 
triennial surveys, the limited time surveyors devote to complaints is squeezed from other 
survey activities in an already-packed agenda.  And while the Joint Commission requires 
hospitals to inform the public about upcoming surveys 30 days in advance with notices 
that explain how to register complaints, it has no standing requirement for hospitals to 
post such notices continuously.54 
The rate of complaints the Joint Commission received through its triennial survey 
process (referred to as public information interviews) remained flat from 1996 to 1997: 
surveyors conducted 111 such interviews in 1996 and 103 in 1997, involving about 6 
percent of surveys in each year.55  Thirteen percent of the public information interviews 
resulted in areas identified for improvement (i.e., type 1 recommendations).  Historically, 
such public information interviews took place with both surveyors and hospital 
representatives present, which likely had a chilling effect on staff wishing to complain. 
But in 1997, the Joint Commission changed its policy to allow confidential opportunities 
for public information interviews.  It remains to be seen how this revised policy will affect 
the complaints received over time.56 
The Joint Commission also receives complaints in its central office, which it tracks 
for trends or sends to surveyors for review during triennial surveys.57  In March of 1999, it 
implemented a new toll-free complaint hotline.58  When a complaint or incident such as an 
unexpected death alerts the Joint Commission of a possibly grave threat to patient care or 
safety in an accredited hospital, it conducts either an unannounced or unscheduled survey 
(these differ from the random unannounced surveys discussed previously).  During the 3-
year period from June 1995 through May 1998, the Joint Commission conducted these 
surveys in about 6 percent of accredited hospitals (172 unannounced and 136 unscheduled 
surveys).  While hospitals do not receive prior notice of an unscheduled survey, the Joint 
Commission gives 24 to 48 hours notice before an unannounced survey.  Both types can 
result in changes in accreditation status and areas requiring follow-up.59 
Sometimes, the Joint Commission, HCFA, and/or the States agencies receive the 
same complaints.  While some sharing of complaint information occurs, officials from 
State agencies, HCFA, and the Joint Commission all expressed some concern about the 
consistency of that sharing.  Some also expressed concerns about the lack of coordination 
in following up on complaints. 
26 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  S t a n d a r d i z e d   P e r f o r m a n c e   D a t a  
Despite the Joint Commissions early plans to incorporate standardized hospital 
performance data into accreditation, it remains of little value to external 
assessments of hospital quality. 
In 1986, the Joint Commission unveiled its Agenda for Change, a vision for a 
performance-based accreditation system that included clinical indicators.  It envisioned a 
system where data based on such indicators would enhance the accreditation process by 
making it more data-driven and providing insights into hospitals performance on a 
continuous basis.60  That vision evolved to include public access to hospital performance 
reports and links between accreditation and performance measurement.  While it has made 
strides in the former, linking accreditation to performance measurement has been more 
troublesome. 
As its vision unfolded, the Joint Commission faced considerable hurdles.  For some 
stakeholders, the Joint Commission was moving too fast on performance measures; for 
others, too slow.  For example, its initial vision required hospitals to participate voluntarily 
in its own data system, called the Indicator Measurement System (IMS), until 1996, when 
participation would become mandatory.  The hospital industry resisted the IMS, finding it 
expensive and burdensome.61  Concerns about the science of performance measurement 
and risk adjustment, considered by many to be still in its infancy, also plagued the Joint 
Commissions implementation.  Many hospitals had already invested in other competing 
performance measurement systems.  The Joint Commissions commitment to making 
hospital performance data public also rankled. 
Thus, while the Joint Commission has maintained its vision of a performance-based 
accreditation system, it has proceeded at a much slower pace and with more flexibility 
than it initially sought.  For example, it no longer requires hospitals to participate in the 
IMS.  Instead, the Joint Commission offers two programs:  Oryx and Oryx+.  Oryx 
reflects the minimum requirement for accredited hospitals, which are required to choose a 
performance measurement system from a list of approved systems, of which IMS is one 
option.62  Hospitals wishing to do more than the minimum participate in Oryx+, which is 
an accelerated version of Oryx that requires participating hospitals to use the same 10 
performance measures.  Oryx supports comparisons only to individual hospitals own 
performance over time, whereas Oryx+ allows comparisons across participating hospitals. 
With the exception of the hospitals participating in Oryx+, this approach left the 
Joint Commission a long way from its earlier vision of uniform data from all hospitals.  In 
the words of one stakeholder, Oryx is akin to saying you have to wear a school uniform 
27 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  but may choose it yourself.  To advance its vision toward uniform data, the Joint 
Commission began collecting and evaluating potential core performance measures in late 
1998.63  Considered the next phase of the Oryx initiative, the Joint Commission aims to 
choose five or six areas (such as diabetes or breast cancer) to focus on by the end of 1999. 
How this new phase of the Oryx initiative will unfold is unknown, but it has 
already caused considerable backlash.  Seventeen State hospital associations wrote to the 
Joint Commission in January 1999 to voice their concerns about the core performance 
measures.64  They raised many of the same concerns raised in 1995 when the Joint 
Commission introduced the IMS, including the costs, burdens to hospitals, and timing of 
implementation.65 
In addition to its Oryx initiatives, the Joint Commission recently joined forces with 
the American Medical Association and the National Committee for Quality Assurance to 
create the Performance Measurement Coordinating Council.  The Council aims to develop 
standards for performance measurement with an eye toward more consistency.66 
28 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  C O N C L U S I O N  
Unquestionably, the Joint Commission is the central force in the external review of 
hospital quality.  It accredits about 80 percent of the hospitals in the country and, for 
Medicare purposes, it has a congressionally granted deeming status that is unique among 
accrediting bodies.  Medicare beneficiaries and others who rely upon hospital services 
have much at stake in how and how well the Joint Commission does its job. 
Our review underscores that the core element of the Joint Commissions approach 
to accreditation is the announced, on-site survey of hospitals--a survey that is heavily 
oriented toward educational and performance improvement objectives.  The other 
elements of external review--unannounced surveys, responses to complaints and serious 
incidents, and standardized performance measures--play relatively minor roles in the Joint 
Commissions accreditation process. 
Given the significance of the Joint Commissions role and its emphasis on one 
approach to external quality review, our inquiry surfaces important policy questions for 
HCFA: How can it best ensure an appropriate balance in external quality reviews of 
hospitals?  How can it best hold the Joint Commission accountable for the important 
public role it performs while enabling it, at the same time, to have enough flexibility to 
continue to advance the state-of-the-art of hospital accreditation?  We address these 
questions in our summary report, A Call for Greater Accountability.  That report also 
contains our recommendations, which we direct to HCFA. 
29 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  APPENDIX A 
Medicare Conditions of Participation 
The Medicare Conditions of Participation (COP) were first published in 1966 and revised in 1986 (42 C.F.R. 
482.1-482.66). On December 19, 1997, HCFA published a proposed COP for hospitals ( 62 Fed. Reg. 66,726). 
On July 2, 1999, it published an interim final rule on patients rights (64 Fed. Reg. 36,070). Below are the 
components of the existing COP for non-specialty hospitals from 1986, followed by the components of the new 
proposed COP. 
Existing COP  Proposed COP 
Subpart A- General Provisions  Subpart A- General Provisions 
Provision of emergency services by  Patient Rights (issued as an interim final rule on July 7, 
nonparticipating hospitals  1999) 
Subpart B- Administration  Subpart B- Patient Care Activities 
Compliance with Federal, State, and local laws  Patient Admissions, assessment, and plan of care

Governing Body  Patient care

Quality assessment and performance improvement

Subpart C- Basic Hospital Functions  Diagnostic and therapeutic services or 

rehabilitation services 
Quality assurance
 Pharmaceutical services 
Medical staff
 Nutritional services 
Nursing services
 Surgical and anesthesia services 
Medical record services
 Emergency services 
Pharmaceutical services
 Discharge Planning 
Radiologic services

Laboratory services
 Subpart C- Organizational Environment 
Food and dietetic services

Utilization review
 Administration of organizational environment

Physical environment
 Infection control

Infection control
 Human resources

Physical environment

Subpart D- Optional Hospital Services  Life safety from fire

Blood and blood products transfusions

Surgical services  Potentially infectious blood and blood products

Anesthesia services  Utilization review

Nuclear medicine services 
Outpatient services 
Emergency services 
Rehabilitation services 
Respiratory care services 
30 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  APPENDIX B 
Methodology

We collected information presented in this report from the following sources: 
The HCFA 
We obtained dates of certification surveys from HCFAs Online Survey 
Certification and Reporting System (OSCAR).  The HCFA authorizes States to update 
and maintain this database with survey information.  We extracted survey data pertaining 
to the frequency of certification surveys.  We subsequently verified the accuracy of our 
extraction by comparing it to on-line OSCAR system information to ensure the dates we 
used corresponded to routine certification surveys, rather than complaint investigations or 
other types of surveys.  We are satisfied that our information is as accurate as HCFAs 
OSCAR system. 
Additionally, we selected 4 States (California, Kansas, Minnesota, and Texas) that 
contain over 50 nonaccredited hospitals and represented different geographic regions of 
the United States.  We then examined the OSCAR data for those States in greater detail. 
We verified the operational status of the nonaccredited hospitals in those States that had 
not been surveyed in over 5 years using the American Hospital Associations 1997 
Hospital Guide. 
We also interviewed staff and managers at each HCFA regional office and the 
central office.  We reviewed a variety of HCFA documents, including budget call letters, 
reinvention materials, and reports to Congress, among others. 
The State Survey and Certification Agencies 
In August 1997, we mailed a pretested survey to the hospital certification agencies 
in the 50 States and the District of Columbia.  The response rate for this survey was 100 
percent.  The State survey addressed four areas of hospital quality oversight:  private 
accreditation of hospitals, Medicare certification of hospitals, HCFA oversight of State 
certification agencies, and State licensure of hospitals.  We interviewed State officials on 
the telephone or in person as well. 
Accrediting Organizations 
We interviewed officials from both the Joint Commission on Accreditation of 
Healthcare Organizations and the American Osteopathic Association.  We also reviewed 
31 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  APPENDIX B 
documents such as mission statements, accreditation manuals, policies, and hospital survey 
files.  We requested and received aggregate data from the Joint Commission reflecting its 
recent hospital survey activity.  All Joint Commission data are presented as reported by 
the Joint Commission, unless otherwise noted.  For purposes of this inquiry, we focused 
our analysis on the Joint Commission. 
Survey Observations 
Based on schedules of upcoming triennial surveys, we identified nine hospitals in 
which to observe triennial Joint Commission surveys.  Of those, we were able to observe 
seven.  In two cases, the hospitals declined the Joint Commissions request that we be 
allowed to observe.  The 7 hospitals varied in size from 80 to 775 beds, represented both 
teaching and community hospitals, and were located in different areas of the country (both 
rural and urban).  We also observed one random unannounced Joint Commission survey. 
Although we observed different teams of surveyors, the survey agenda, lines of inquiry, 
and tone were consistent across the surveys, which were conducted in accordance with 
Joint Commission policy, based on review of survey manuals and interviews with 
representatives of the Joint Commission.  Finally, we observed a certification survey and 
parts of two complaint investigations performed by State surveyors under HCFAs 
auspices. 
Stakeholder Interviews 
We interviewed representatives of organizations we considered to be stakeholders 
in hospital oversight.  These stakeholder organizations included a union, professional 
organizations, hospital associations, and consumer groups. 
Other Documents 
In addition to the documents referenced above, we reviewed statutory and 
regulatory language and a variety of articles from newspapers, journals, magazines, and 
newsletters. 
32 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  APPENDIX C 
Sample Accreditation Survey Agenda

We took the following sample agenda from the Joint Commissions web page 
(http://www.jcaho.org/acr_info ).  This agenda is representative of the agendas surveyors 
followed when we observed accreditation surveys. 
Sample Generic Survey Agenda: Three-Day Survey/Three Surveyors 
In most instances, the survey team is composed of an administrator, nurse, and physician. 
In smaller hospitals, the survey team includes two surveyors, usually a nurse and a 
physician. 
Day One: 
All Surveyors: 
8:30-8:45 am Opening Conference 
8:45-9:15 am Performance Improvement Overview Presentation 
9:15-11:00 am Document Review Session 
11:00-11:45 am Leadership Interview 
11:45-12:15 pm Lunch 
Physician Surveyor: 
12:30-2:00 pm Patient Care Setting Visit (Includes inpatient units and other sites where

care is provided, including ambulatory settings and anesthetizing locations.)

2:00-3:00 pm Anesthetizing Location Visit (Includes inpatient units and other sites where

care is provided, including ambulatory settings and anesthetizing locations.)

3:00-4:00 pm Emergency Services Visit 

Nurse Surveyor: 
12:30-2:00 pm Patient Care Setting Visit (Includes inpatient units and other sites where

care is provided, including ambulatory settings and anesthetizing locations.)

2:00-3:00 pm Anesthetizing Location Visit (Includes inpatient units and other site where

care is provided, including ambulatory settings and anesthetizing locations.)

3:00-4:00 pm Patient Care Setting Visit (Includes inpatient units and other sites where

care is provided, including ambulatory settings and anesthetizing locations.)

Administrator Surveyor: 
12:30-1:15 pm Rehabilitation Services Visit 
33 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  APPENDIX C 
1:15-4:00 pm Building Tour (Includes visits to admitting, kitchen, resource center, 
storage, central supply, and laundry, if applicable.) 
All Surveyors: 
4:00-4:30 pm Survey Team Meeting 
Day Two: 
All Surveyors: 
8:15-8:45 am Daily Briefing 
Physician Surveyor: 
8:45-9:45 am Pathology and Clinical Laboratory Services Visit (Includes survey of blood

and blood components.) 

9:45-10:45 am Medical Staff Credentials Interview 

10:45-11:45 am Medical Staff Leadership Interview 

11:45 am-12:45 pm Lunch: Medical Staff Conference (Optional) 

Nurse Surveyor: 
8:45-10:00 am Patient Care Setting Visit (Includes inpatient units and other sites where

care is provided, including ambulatory settings, and anesthetizing locations.)

10:00-10:45 am Patient Care Setting Visit (Includes inpatient units and other sites where

care is provided, including ambulatory settings, and anesthetizing locations.) 

10:45-11:30 am Infection Control Interview 

11:30 am-12:15 pm Nursing Leadership Interview 

Administrator Surveyor: 
8:45-9:30 am Patient Care Setting Visit (Includes inpatient units and other sites where 

care is provided, including ambulatory settings, and anesthetizing locations.) 

9:30-10:15 am Hospital Department Directors Interview 

10:15 am-12:15 pm Review of Environment of Care Documents 

Nurse and Administrator Surveyors: 
12:15-12:45 pm Lunch 
Administrator Surveyor : 
1:00-1:45 pm Chief Executive Officer/Strategic Planning and Resource Allocation

Interview (Includes Ethics Interview issues.)

1:45-2:15 pm Pharmacy Services Visit 

34 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  APPENDIX C 
Physician and Nurse Surveyors: 
1:00-2:00 pm Medical Record Interview 
Physician Surveyor: 
2:00-2:45 pm Patient Care Setting Visit (Includes inpatient units and other sites where 
care is provided, including ambulatory settings, and anesthetizing locations.) 
2:45-4:00 pm Imaging Services Interview 
Nurse and Administrator Surveyors: 
3:00-4:00 pm Human Resources Interview 
All Surveyors: 
4:00-4:30 pm Survey Team Meeting 
Day Three: 
All Surveyors: 

8:15-8:45 am Daily Briefing 

8:45-9:45 am Patient Care Interview 

Physician Surveyor: 
9:45-10:30 am Performance Improvement Team Interview 
Nurse Surveyor: 
9:45-10:30 am Performance Improvement Team Interview 
Administrator Surveyor: 
10:00-11:00 am Information Management Interview 
All Surveyors: 
10:30-11:30 pm Performance Improvement Coordinating Group Interview 

11:30-12:00 pm Lunch 

12:00-3:00 pm Team Meeting to Integrate Survey Findings 

3:15-4:15 pm Leadership Exit Conference 

35 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  APPENDIX D 
Advertising Hospital Accreditation Status

During the course of our inquiry, we saw a range of advertisements extolling 
hospitals accreditation status.1  We saw advertisements in newspapers, magazines, fund-
raising materials, yellowpages, and on billboards and the Internet.  We also saw a banner 
strung prominently outside a hospital.  Most touted the hospitals achievement of 
accreditation with commendation.  For example, one advertisement from a web page 
included the following: 
Accreditation is the way hospitals are judged.  Commendation is how they are 
judged superior...Were proud to be recognized as one of the best hospitals...in 
the country...Our accreditation with commendation puts [us] in the upper 10% of 
all hospitals surveyed in the past 3 years. 
Another ad, this one from a magazine, pitched a hospital chain and its accreditation as 
follows: 
As for quality, the Joint Commission on the Accreditation of Healthcare 
organizations, an independent group that reviews staffing, safety and patient care 
at hospitals around the country, recognizes [our] hospitals with its highest 
ratings:  Accreditation with Commendation, at a rate three times the national 
average. 
Yet another, from a newspaper noted: 
Accreditation with Commendation.  Its the highest level of achievement awarded 
by the Joint Commission on the Accreditation of Healthcare Organizations.  And 
a distinction earned by only 4 percent of hospitals nationwide. 
1The Joint Commission issues guidelines on how hospitals can characterize their 
accreditation status in such ads and considers false or misleading advertising of accreditation as 
grounds for denying accreditation (Joint Commission on Accreditation of Healthcare 
Organizations, Comprehensive Accreditation Manual for Hospitals, 1997, p AC-18). 
36 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  APPENDIX E 
Endnotes

1.  Troyen A. Brennan et al, Incidence of Adverse Events and Negligence in Hospitalized 
Patients, The New England Journal of Medicine 324 (February 7, 1991) 6: 370-376. 
2.  The 17.7 percent refers to adverse events considered by the authors to be serious.  The authors 
defined adverse events as situations in which an inappropriate decision was made when, at the 
time, an appropriate alternative could have been chosen and serious as ranging from temporary 
disability to death.  See Lori B. Andrews et al, An Alternative Strategy for Studying Adverse 
Events in Medical Care, The Lancet 349 (February 1, 1997)309-313. 
See also Lucian L. Leape, Error in Medicine, Journal of the American Medical 
Association 272 (December 21, 1994) 23: 1851-1857; David C. Classes et al, Adverse Drug 
Events in Hospitalized Patients, Journal of the American Medical Association 277 (January 
22/29, 1997) 4: 301-306; Mark R. Chassin et al, The Urgent Need to Improve Health Care 
Quality:  Institute of Medicine National Roundtable on Health Care Quality, Journal of the 
American Medical Association 280 (September 16, 1998) 11: 1000-1005; David W. Bates et 
al,Effect of Computerized Physician Order Entry and a Team Intervention on Prevention of 
Serious Medication Errors, Journal of the American Medical Association 280 (October 21, 
1998) 1311-1316; Robert A. Raschke et al, A Computer Alert System to prevent Injury from 
Adverse Drug Events:  Development and Evaluation in a Community Teaching Hospital, Journal 
of the American Medical Association 280 (October 21, 1998) 1317-1320; and David C. Classen, 
Clinical Decision Support Systems to Improve Clinical Practice and Quality of Care, Journal of 
the American Medical Association 280 (October 21, 1998) 1360-1361. 
3.  One example is a November 1998 New York Times article under the headline: Death in 
Surgery Reveals Troubled Practice and Lax Hospital.  The article described a botched 
operation on a young woman by a surgeon who was on probation by the State medical board and 
who used unauthorized medical equipment brought in to the operating room by a medical supply 
salesman.  Such incidents can happen even in the best of hospitals, but they underscore the point 
that hospitals can be dangerous places and that oversight systems can be lax.  See also 
Overdoses Still Weigh heavily at Dana Farber, The Boston Globe (26 December 1995); Florida 
Doctor Sanctioned in New Amputation, The Boston Globe (19 July 1995); Two Surgeons 
Surrender Licenses After Mistakenly Removing Kidney, The Boston Globe (6 June 1996); How 
Can We Save the Next Victim? New York Times Magazine (15 June 1997); Another Hospital 
Death Probed, The Boston Globe (26 July 1997); Patient Dies After Drinking Poison Left on 
Nightstand, San Diego Union-Tribune (6 March 1998); Man Arrested for Posing as Doctor for 
4 years, posted at the CNN interactive webpage (16 May 1998); Deadly Restraint:  Patients 
Suffer in a System Without Oversight, The Hartford Courant (13 October 1998). 
37 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  APPENDIX E 
4. http://WWW.hcfa.gov/Medicare/Medicare.htm 
5. Ibid. 
6. Ibid. 
7. P.L. 89-97. 
8. The HCFA published its proposed hospital conditions of participation on December 19, 1997 
(62 Fed. Reg. 66,726).  On July 2, 1999, it published an interim final rule concerning patients 
rights (64 Fed. Reg. 36070). 
9.  Social Security Act, sec. 1865, 42 U.S.C 1395bb. 
10. Health Care Financing Administration, OSCAR 10 Report, as of May 30, 1998. 
11. Michael S. Hamm, The Fundamentals of Accreditation (Washington, D.C.: American Society 
of Association Executives, 1997) 3. 
12. Ibid. 
13. Nonaccredited hospitals can go through the Medicare certification process for free. 
14. Although we conducted some interviews with staff from the American Osteopathic 
Association, another hospital accrediting body, we decided to exclude it from our inquiry.  Our 
decision was based on our own time and resource constraints.  We may examine accreditation 
through the American Osteopathic Association in a future inspection. 
15. In some cases, the surveyors will identify a type of record (e.g., a patient with recent restraint 
orders or a personnel file of a respiratory therapist) or will request that a particular staff persons 
personnel file be pulled. 
16. The building tour and document review sessions take longer.  The unscheduled time built into 
the structure is generally in 15-minute blocks.  We observed surveyors often use that time for 
telephone calls, further document review, or as a buffer in case other sessions ran over. 
17. Indeed, even the likely focus of the surveys or surveyors is available to those hospitals looking 
for it.  For example, hospitals can purchase individual surveyor profiles through an internet site or 
visit chat rooms to exchange details on recent surveys.  One surveyor told us he casually 
mentioned liking chocolate chip cookies at one hospital, and was thereafter inundated with them-
an occurrence he attributed to these web sites and chat rooms.  Hospitals can also subscribe to 
38 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  APPENDIX E 
newsletters that track the latest trends and focus of Joint Commission surveys. 
18.  Surveyor notes to the Joint Commission found in the Joint Commission files. 
19.  The Joint Commission changed its standards as part of their Agenda for Change, an 
initiative calling for a more performance-based accreditation process which was unveiled in 1986. 
20.  Dennis OLeary,  Presidents Column: On Becoming Ten, Joint Commission Perspectives, 
16 (May/June 1996) 3: 2-3. 
21.  The accreditation process itself is fundamentally a risk-reduction exercise.  We dont 
warrant outcomes or that nothing bad will ever happen.  Wed be crazy to do that because you 
can never take the level of risk down to zero  (Dennis OLeary, quoted in Setting Standards for 
Hospitals: An Interview with Dennis OLeary in The Long Term View 3 (4) 23). 
22.  In 1995, the Joint Commission launched an experiment in a more continuous accreditation 
process, called the Orion Project.  Operating in four States (Arizona, Pennsylvania, Georgia, and 
Tennessee), the Orion Project involves partial surveys conducted quarterly by the same surveyors 
for participating hospitals. 
23.  In early 1997, the Executive Committee of the Board of Commissioners approved an interim 
cap on the scores of three standards related to restraints. 
24.  Data as reported by the Joint Commission. 
25.  The most frequently cited deficiencies are often identified in newsletters such as Inside the 
Joint Commission and Joint Commission Perspectives. 
26.  http://jcaho.org. 
27.  Continuous Quality Improvement is a method of performance improvement involving a 
continuous loop of performance measurement, intervention, and remeasurement.  It aims to 
improve the majority rather than marginal performers.  D. E. Berwick, Continuous Quality 
Improvement as an Ideal in Health Care, New England Journal of Medicine 320 (January 5, 
1989) 1: 53-56; The Institute of Medicine, Medicare: A Strategy for Quality Assurance 
(Washington, D.C.: National Academy Press, 1990). 
28.  Joint Commission on Accreditation of Healthcare Organizations, Understanding the 1997 
Hospital Performance Report, 4. 
39 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  APPENDIX E 
29.  In response to the unexpected deaths, the Joint Commission conducted a special announced 
survey lasting one day, during which the hospital presented its analysis of the cause and the 
surveyor identified four deficiencies regarding assessing patients and assessing competence. 
30.  Joint Commission on Accreditation of Healthcare Organizations, Understanding the 1997 
Hospital Performance Report, 10, and 1997 Performance Report, ii. 
31.  According to data reported by the Joint Commission, the most commonly identified 
deficiencies in credentialing between May 1995 and June 1997 were related to the inclusion of 
pending or successful challenges to licensure, relinquishment of licensure, criteria for autopsies, 
and primary verification of license, experience, training, and competencies. 
While we found no examples of surveyor probing to ensure that hospitals deal fully and 
responsibly with marginal performers, we did observe one medical leadership session where the 
physician surveyor gave a stern message about the significance of credentialing and privileging 
decisions.  He said that the standard that physician reviewers should use in making such decisions 
about their colleagues was The Mom Test.  Would you feel comfortable, he asked, in having 
the colleague care for your mother?  Someday, he warned the assembled physicians, you will be 
walking or taken through hospital doors as a patient and you will be dependent on the 
conscientiousness and skill of the medical staff at that hospital.  Think about it. 
32.  In a 1995 Office of Inspector General report, we found that 75 percent of the hospitals in the 
country never reported an adverse action to the Data Bank between September 1, 1990 and 
December 31, 1993 (Office of Inspector General, Hospital Reporting to the National Practitioner 
Data Bank, February 1995). 
33.  In December 1994, the Joint Commission began releasing accreditation information about 
hospitals, on request, in the form of performance reports.  These reports include the hospitals 
overall survey score from the last triennial survey, the hospitals scores in the 45 performance 
areas surveyed, comparative data on the percentage of hospitals receiving each possible score in 
the 45 performance areas, and a list of the performance areas in which the hospital received 
deficiencies.  In November 1996, the Joint Commission waived the $30 report fee. 
As of late 1998, 13,408 performance reports for hospitals, nursing homes, and other 
provider types were available, and that list keeps growing.  Between late 1996 and early 1997, 
requests for performance reports grew from 13 per quarter to over 2,000.  Beginning in early 
1998, the Joint Commission made the reports available on its website, and since then requests for 
reports have tapered off, suggesting some are accessing the information directly through the 
website rather than ordering reports. 
40 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  APPENDIX E 
34.  The Joint Commission allows hospitals to be accredited as single entities or, if they are a part 
of a multi-hospital system, to be accredited through a system survey.  The multi-hospital option 
includes a corporate orientation and/or a corporate summation in addition to a consecutive survey 
of the systems hospitals with the same survey team.  The Joint Commission charges an extra fee 
for this option.  The Joint Commission also offers network accreditation, so hospitals that are part 
of a particular health care network could also be surveyed under the auspices of a network 
survey--in addition to their own hospital survey.  The Joint Commission could award a different 
accreditation status to the hospital than to the network, making it difficult for consumers to 
discern the meaning of each. 
35.  Some Joint Commission scoring guidelines are based on algorithms 
36.  The levels of accreditation are: accreditation with commendation, accreditation without type 
1 recommendations, accreditation with type 1 recommendations, conditional accreditation, and 
preliminary nonaccreditation.  Preliminary nonaccreditation is generally considered to be a 
temporary status. 
From June 1995 through May 1998, the Joint Commission awarded accreditation with 
commendation to hospitals surveyed as a part of a hospital system much more often (22 percent) 
as hospitals surveyed as single entities (13 percent).  Achieving any accreditation level qualifies a 
hospital to participate in Medicare.  By contrast, the National Committee for Quality Assurance, 
which accredits health plans, has used four accreditation levels.  Eleven percent of plans met its 
lowest, or provisional level, and 11 percent were denied accreditation.  Unlike Joint Commission 
hospital accreditation, Congress has not deemed the National Committee for Quality Assurance 
(NCQA) accreditation for purposes of participating in Medicare.  We extracted the data on 
NCQA accreditation from its web page, NCQA Accreditation Status List: A Fact Sheet 
(http://www.ncqa.org). 
37.  When the Joint Commission identifies a Type 1 recommendation in a hospital, that hospital 
must either submit a written progress report and/or be subject to a focused survey.  Written 
progress reports far exceed focused surveys as a form of follow-up:  between June 1995 and May 
1998, the Joint Commission called for 3,264 written progress reports and 417 focused surveys. 
Between June 1995 and May 1997, the Joint Commission rejected about 2 percent of written 
progress reports for triennial surveys and required an additional follow-up, usually an on-site 
survey, for those hospitals.  Of course, the Joint Commission can call for multiple written 
progress reports and focused surveys from a single hospital, depending on the particular situation. 
Hospitals accredited with commendation are subject to no such follow-up, but could be selected 
for a random, unannounced survey. 
41 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  APPENDIX E 
38.  The Joint Commission allows hospitals the opportunity to improve their scores by correcting 
problems identified in the survey (also referred to as type 1 recommendations).  It conducts two 
kinds of follow-up to ensure the hospitals have made the appropriate corrections:  written 
progress reports and focused surveys. 
About a fourth of all hospitals scores improved on average 1 point after they addressed 
their deficiencies; the others remained the same or went down.  From June 1995 to May 1996, 
hospital scores averaged 91 (for multi-hospital option) and 90 (for single hospital option).  From 
June 1996 to May 1997, hospital scores averaged 93 (for multi-hospital option) and 91 (for single 
hospital option). 
39.  Joint Commission on Accreditation of Healthcare Organizations, 1997 Performance Report, 
p. 3. 
40.  In addition, the Joint Commission uses written progress reports and focused surveys both to 
ensure hospitals address deficiencies and as a way to monitor continued compliance.  Written 
progress reports and focused surveys are considered follow-up actions to Joint Commission 
surveys.  It also uses unannounced and unscheduled surveys to respond to concerns about patient 
safety in a hospital, which are discussed in the section on Responding to Complaints. 
41.  Raid of Hospitals Probes Overworked Doctors, The Wall Street Journal (11 March 1998); 
Inspectors Pay Surprise Visits to 12 New York Hospitals, The New York Times (12 May 1998). 
42.  State agencies do, however, give notice to hospitals prior to conducting validation and 
certification surveys. 
43.  42 USC 1819(g)(2). 
44.  David R. Morrow, To Rate Hospitals, She Dons a Wig and Practices Her Cough, New 
York Times, Sunday, 30 March 1997. 
45.  Following are some illustrations of newspaper reports on serious incidents involving patient 
harm: Florida Doctor Errs Anew, Taking Toe Without Asking, The New York Times, July 19, 
1995;  Lawrence K. Altman, State Issues Scathing Report on Error at Sloan-Kettering, The 
New York Times, November 16, 1995; Richard Saltus, Patients Death Leads to Change at the 
Brigham, The Boston Globe, February 16, 1996; Richard Saltus, Two Surgeons Surrender 
Licenses after Mistakenly Removing Kidney, The Boston Globe, June 6, 1996; and Death in 
Surgery Reveals Troubled Practice and Lax Hospital, The New York Times (15 November 1998). 
42 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  APPENDIX E 
46.  The Commission typically conducted these investigations on its own, independent of any 
investigations being conducted by State and/or HCFA officials. 
47.  The Joint Commission refers to this as a root-cause analysis. 
48.  For articles supporting and or explaining this approach to medical errors, see Lucian L. 
Leape, Error in Medicine, The Journal of the American Medical Association 272 (December 
21, 1994) 23:1851-57; Lucian L. Leape, et al, Systems Analysis of Adverse Drug Events, The 
Journal of the American Medical Association 274 (July 5, 1995) 1:35-43; David W. Bates, 
Incidence of Adverse Drug Events and Potential Adverse Drug Events, The Journal of the 
American Medical Association, 274 (July 5, 1995) 1:29-34; Lawrence K. Altman, Medical 
Errors Bring Calls for Change, The New York Times, July 18, 1995; Linda O. Praeger, 
Reducing Medical Errors, American Medical News 39 (November 4, 1996) 41; Denise Grady, 
Doctors Urged to Admit Mistakes, The New York Times, December 9, 1997; and Lucian L. 
Leape et al, Promoting Patient Safety by Preventing Medical Error, The Journal of the 
American Medical Association 280 (October 28, 1998) 16: 1444-1447. 
49.  The Joint Commissions policy calls for a hospital voluntarily to notify it of a reportable 
major adverse event within 5 business days and subsequently to conduct an acceptable analysis of 
the events cause within 30 days. 
50.  In this respect, the Joint Commission indicates that its approach is similar to the aviation 
accident and incident reporting system of the National Transportation Safety Board.  See 
http://nasdac.faa.gov/safety_data/. 
51.  During this period, the Joint Commission has drawn on its sentinel event database to issue 
four alerts with specific risk reduction strategies for hospitals.  The first, issued on February 27, 
1998, focused on preventing medication errors and involved the misadministration of potassium 
chloride.  The second, issued August 28, 1998, was on wrong-site surgery.  The third and fourth 
were both issued in November 1998 and dealt with preventing inpatient suicides and restraint 
deaths.  (See Joint Commission on Accreditation of Healthcare Organizations, Sentinel Event 
Alert, February 27, 1998; August 28, 1998; November 6, 1998; and November 18, 1998.) 
52.  Joint Commission Statement on the Revised Sentinel Event Policy, February 17, 1998. 
53.  Further, if  the public recognizes that the Joint Commission rarely changes a hospitals 
accreditation status as a result of a adverse event, the public is likely to increasingly view the Joint 
Commission as being soft on hospitals at the expense of public safety.  The Joint Commission 
itself is aware of this danger.  In a column in its own newsletter, the Joint Commissions president 
noted that while hospitals tend to regard such events as their own business, those speaking on 
43 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  APPENDIX E 
behalf of the public see sentinel event identification and follow-up as central to the Joint 
Commissions role, and credibility, as an accrediting body.  For hospitals wary even of the 
gentler Joint Commission policy toward such events, he cautioned, [I]n the end, the Joint 
Commission may be the one constructive force standing between the organization that has 
experienced a sentinel event and societys handmaidens of punishment.  He did not identify the 
latter.  See Dennis S. OLeary, Presidents Column: The Sentinel Event Policy: A Work in 
Progress, Joint Commission Perspectives, November/December 1996, pp. 2-3. 
54.  Despite occasional newspaper articles, how aware consumers are of the Joint Commission 
remains unknown (Before You Go Into the Hospital, Parade Magazine, 4 May 1997, 6-8). 
Consumers may be more aware of their State governments or HCFA as places to file complaints. 
When we asked stakeholders the extent to which they expected accreditation to play a role in 
responding to complaints, their responses were mixed: about a third (34 percent) expected no or a 
minor role; 18 percent, a moderate role and; nearly half (48 percent), at least a great role. 
55.  Reflects data as reported by the Joint Commission.  However, based on our file review, the 
public information interviews may be undercounted to some extent.  We reviewed the Joint 
Commissions hospital files for the seven surveys we observed, of which four had public 
information interviews.  We found no record of the four public information interviews we 
observed in the files. 
56.  The most recent data available from the Joint Commission indicates that surveyors conducted 
31 public information interviews during the first 6 months of 1998, involving about 4 percent of 
surveys conducted during that time.  Five of the 31 (16 percent) resulted in type 1 
recommendations. 
57.  In 1996, for example, it reported receiving 2,718 complaints (43 percent by mail and 
57 percent by telephone), a 3.5 percent decrease from those it received in 1995.  Twenty-seven 
percent of the written complaints were either unrelated to an accredited hospital or unintelligible, 
thus prompting no further action from the Joint Commission.  But the Joint Commission tracked 
30 percent of the written complaints for trends and sent 27 percent to surveyors for triennial or 
focused surveys. 
58.  Joint Commission on Accreditation of Healthcare Organizations, March 3, 1999.  The Joint 
Commission also accepts complaints by electronic mail, fax, and mail. 
59.  In the 3-year period from June 1995 through May 1998, 17 percent of the unscheduled and 
unannounced surveys resulted in a change in the hospitals accreditation status and 50 percent 
resulted in some follow-up (involving focused surveys and/or written progress reports). 
44 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051  APPENDIX E 
60. On its webpage (http://www.jcaho.org), the Joint Commission identifies the following ways 
that integrating performance measures will enhance the value of accreditation: 
C	 increase public and private sector reliance on Joint Commission accreditation, 
thereby building support for the consensus development and use of standards and 
performance measures, as well as reducing justification for duplicative evaluation 
activities; 
C	 provide the Joint Commission with performance data that it can use to 
C  monitor organization performance on a continuous basis, 
C  help organizations identify issues that require attention, and 
C  focus triennial surveys on areas within an organization that represent the 
greatest potential opportunities for improvement; 
C  help identify both exemplary performance and best practices to facilitate the 
provision of benchmarking services to health care organizations; and 
C	 facilitate refinement of the Joint Commissions performance-based standards and 
the analysis and understanding of the relationships between standards and 
outcomes. 
61. Christina Kent, Fix it or forget it: The JCAHO in Crisis, Medicine & Health (January 23, 
1995). 
62. The Joint Commission regularly reviews performance measurement systems, so new systems 
found to meet its minimum requirements can be added to the list.  The Joint Commission also can 
remove approved systems. (http://www.jcaho.org/perfmeas/). 
63.  On March 15, 1999, the Joint Commission invited accredited institutions, physicians, 
universities, consumer groups, and researchers, among others, to identify and prioritize potential 
areas for it to develop core measures. 
64.  American Hospital Association to Joint Commission on Accreditation of Healthcare 
Organizations, January 5, 1999. 
65.  Ibid. 
66. J. Duncan Moore, Jr., Standardizing Standards: Accrediting agencies form council to end 
confusion, Modern Healthcare (May 25, 1998); Three for Quality, American Medical News 
(June 15-22, 1998); Three Accreditation Groups to Collaborate on Standard Approach to 
Quality Measures, BNAs Managed Care Reporter (May 27, 1998). 
45 
Hospital Quality: The Role of Accreditation  OEI-01-97-00051