Department of Health and Human Services 

OFFICE OF 

INSPECTOR GENERAL 

JUNE GIBBS BROWN 

Inspector General 

JULY 1999 

OEI-01-97-00051 

The External Review of Hospital Quality 

The Role of Accreditation 
OFFICE OF INSPECTOR GENERAL 

The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, is to 

protect the integrity of the Department of Health and Human Services programs as well as the 

health and welfare of beneficiaries served by them. This statutory mission is carried out through a 

nationwide program of audits, investigations, inspections, sanctions, and fraud alerts. The 

Inspector General informs the Secretary of program and management problems and recommends 

legislative, regulatory, and operational approaches to correct them. 

Office of Evaluation and Inspections 

The Office of Evaluation and Inspections (OEI) is one of several components of the Office of 

Inspector General. It conducts short-term management and program evaluations (called 

inspections) that focus on issues of concern to the Department, the Congress, and the public. The 

inspection reports provide findings and recommendations on the efficiency, vulnerability, and 

effectiveness of departmental programs. 

OEI's Boston regional office prepared this report under the direction of Mark R. Yessian, Ph.D., 

Regional Inspector General. Principal OEI staff included: 

BOSTON HEADQUARTERS 

Joyce M. Greenleaf, Project Leader Jennifer Antico, Program Specialist 

Lynne G. Dugan, Senior Analyst Alan Levine, Program Specialist 

Meredith A. Vey, Program Analyst 

To obtain copies of this report, please call the Boston Regional Office at 617/565-1050. 

Reports are also available on the World Wide Web at our home page address: 

http://www.dhhs.gov/progorg/oei 
EXECUTIVE SUMMARY 

PURPOSE 

To assess the role of Joint Commission accreditation in the external review of hospital 

quality. 

BACKGROUND 

External Quality Review of Hospitals in the Medicare Program 

Hospitals routinely offer valuable services, but also are places where poor care can 

lead to unnecessary harm. The external quality review of hospitals plays an important role 

not only in protecting patients from such harm, but also in complementing the hospitals’ 

own internal quality efforts. The Federal Government relies on two types of external 

review to ensure that hospitals meet the minimum requirements for participating in 

Medicare: accreditation, usually by the Joint Commission on Accreditation of Healthcare 

Organizations, and Medicare certification, by State agencies. About 80 percent of the 

6,200 hospitals that participate in Medicare are accredited by the Joint Commission. 

This Inquiry 

This report, part of a series of four companion reports that resulted from our 

inquiry, focuses on the Joint Commission because it dominates the hospital accreditation 

market. Our inquiry draws on aggregate data, file reviews, surveys, and observations 

from a rich variety of sources, including HCFA, the Joint Commission, State agencies, and 

other stakeholders. 

We organize this report around a framework we developed for considering the 

external review of hospital quality. This framework consists of five components: 

announced, on-site surveys of hospitals; unannounced, on-site surveys of hospitals; 

responses to complaints concerning hospitals; responses to major adverse events in 

hospitals; and collection and dissemination of standardized performance measures. 

FINDINGS 

ANNOUNCED SURVEYS 

Hospital Quality: The Role of Accreditation 1 OEI-01-97-00051 
Joint Commission surveys are undertaken in a collegial manner and are tightly structured. 

This approach fosters consistency but leaves little room for probing. 

Surveys look the same hospital to hospital. Surveyors are well-versed in the Joint 

Commission standards and aim to educate the hospital staff about the significance and 

intent of those standards. To get an overview of the hospital, the surveyors maintain a 

rapid pace with survey sessions scheduled back-to-back, leaving little opportunity for 

following up leads or developing hunches. 

Joint Commission surveys serve as a means of both reducing risk and fostering attention to 

continuous quality improvement, but are unlikely to either surface substandard care or 

identify individual practitioners whose judgement or skills to practice medicine are 

questionable. 

Hospitals take Joint Commission surveys seriously. The surveys prompt their 

attention to minimum protections that are important to patients and promote projects 

aimed at improvement. But surveyors lack much background information on the hospital 

that could help them hone their survey, thus they get a broad rather than in-depth view of 

the hospital. This, coupled with the tight structure, make it unlikely that the survey will 

identify patterns or instances of poor care. 

While they matter enormously to hospitals, Joint Commission survey results fail to make 

meaningful distinctions among hospitals. 

Hospitals attach great significance to survey results and use them as a way of 

distinguishing themselves from other hospitals. However, the distinction between 

accreditation with commendation and accreditation with or without recommendations for 

improvement can be difficult to discern. In fact, little variation exists in accreditation 

levels and scores: 99 percent of the hospitals surveyed between May 1995 and June 1998 

clustered in just 2 of the 5 possible accreditation levels. 

UNANNOUNCED SURVEYS 

The Joint Commission’s reliance on unannounced surveys is limited. 

The Joint Commission conducts 1-day, random unannounced surveys to ensure 

continued compliance with accreditation standards between triennial surveys. From June 

1995 through May 1998, it conducted such surveys, providing 24 to 48 hours notice, on 

about 5 percent of its accredited hospitals. 

RESPONSES TO MAJOR ADVERSE EVENTS 

Hospital Quality: The Role of Accreditation 2 OEI-01-97-00051 
The Joint Commission treats major adverse events as opportunities for improvement. 

Accordingly, it emphasizes education, prevention, and confidentiality but limits public 

disclosure on the causes, consequences, and responses to such events. 

The Joint Commission’s sentinel event policy stresses self-reporting and analysis 

on the part of the hospitals. Through this approach, it aims to develop a database of 

events that it can analyze for frequency and causes. But ensuring confidentiality to selfreporting hospitals limits public accountability. This presents particular difficulties if, as it 

often the case, local concern is heightened because of media reports on the events. 

RESPONSES TO COMPLAINTS 

The Joint Commission devotes little emphasis to complaints. 

The Joint Commission’s accreditation process is not particularly geared to dealing 

with complaints. Although it receives complaints during surveys, surveyors must squeeze 

time from other survey activities to respond to them. The Joint Commission also receives 

and responds to complaints centrally. 

STANDARDIZED PERFORMANCE DATA 

Despite the Joint Commission’s early plans, standardized hospital performance data 

remain of little value to external assessments of hospital quality. 

In 1986, the Joint Commission unveiled its plans for a performance-based 

accreditation system that included uniform data from all hospitals. But as that vision 

unfolded, the Joint Commission faced resistance from hospitals. Accredited hospitals 

must now participate in a Joint Commission-approved measurement system (of which 

there are many), but collecting uniform data is as yet unrealized. 

CONCLUSION 

Unquestionably, the Joint Commission is the central force in the external review of 

hospital quality. It accredits about 80 percent of the hospitals in the country and, for 

Medicare purposes, it has a congressionally granted deeming status that is unique among 

accrediting bodies. Medicare beneficiaries and others who rely upon hospital services 

have much at stake in how and how well the Joint Commission does its job. 

Our review underscores that the core element of the Joint Commission’s approach 

to accreditation is the announced, on-site survey of hospitals--a survey that is heavily 

oriented toward educational and performance improvement objectives. The other 

elements of external review--unannounced surveys, responses to complaints and serious 

Hospital Quality: The Role of Accreditation 3 OEI-01-97-00051 
incidents, and standardized performance measures--play relatively minor roles in the Joint 

Commission’s accreditation process. 

Given the significance of the Joint Commission’s role and its emphasis on one 

approach to external quality review, our inquiry surfaces important policy questions for 

HCFA: How can it best ensure an appropriate balance in external quality reviews of 

hospitals? How can it best hold the Joint Commission accountable for the important 

public role it performs while enabling it, at the same time, to have enough flexibility to 

continue to advance the state-of-the-art of hospital accreditation? We address these 

questions in our summary report, A Call for Greater Accountability. That report also 

contains our recommendations, which we direct to HCFA. 

COMMENTS 

Within the Department of Health and Human Services, we received comments 

from HCFA. We also solicited and received comments from the following external 

parties: Joint Commission on Accreditation of Healthcare Organizations, Association of 

Health Facility Survey Agencies, American Osteopathic Association, American 

Association of Retired Persons, Service Employees International Union, National Health 

Law Program, and Public Citizen’s Health Research Group. We include the detailed text 

of all of these comments and our responses to them in our summary report, The External 

Review of Hospital Quality: A Call for Greater Accountability (OEI-01-97-00050). 

Hospital Quality: The Role of Accreditation 4 OEI-01-97-00051 
TABLE OF CONTENTS 

PAGE

EXECUTIVE SUMMARY

INTRODUCTION ..................................................... 6

FRAMEWORK FOR EXTERNAL QUALITY REVIEW OF HOSPITALS ........... 9

PROFILE OF ACCREDITATION ........................................ 10

ANNOUNCED SURVEYS 

Approach: Collegial Tone and Tight Structure .............................. 11

Strengths: Dynamic Standards, Risk Reduction, Quality Improvement ............. 13

Limitations: Unlikely to Uncover Substandard Care or Poor Performers ........... 15

Scoring: Valued by Hospitals, But Scores Make Few Distinctions ................ 18

UNANNOUNCED SURVEYS 

Limited Reliance ..................................................... 21

RESPONSE TO MAJOR ADVERSE EVENTS 

Opportunities for System Improvement .................................... 22

Lack of Public Disclosure ............................................... 25

RESPONSE TO COMPLAINTS 

Little Emphasis ...................................................... 26

STANDARDIZED PERFORMANCE DATA 

Currently of Little Value to External Review ................................ 27

CONCLUSION ........................................................... 29

APPENDICES 

A: Medicare Conditions of Participation ......................................... 30

B: Methodology ........................................................... 31

C: Sample Accreditation Survey Agenda ......................................... 33

D: Advertising Accreditation Status ............................................. 36

E: Endnotes ............................................................... 37

Hospital Quality: The Role of Accreditation 5 OEI-01-97-00051 
INTRODUCTION 

PURPOSE 

To assess the role of the Joint Commission in the external review of hospital quality. 

BACKGROUND 

Hospital Safety 

Hospitals are an integral part of our healthcare system, offering services that 

improve, extend, and even save lives. But they are also places where inappropriate care 

can lead to unnecessary harm. This reality was clearly underscored in 1991, when a 

Harvard medical practice study revealed the results of its review of about 30,000 

randomly selected records of patients hospitalized in New York State during 1984. The 

study found that 1 percent of the hospitalizations involved adverse events caused by 

negligence.1 On the basis of its sample, the study team estimated that during that year, 

negligent care provided in New York State hospitals was responsible for 27,179 injuries, 

including 6,895 deaths and 877 instances of “permanent and total disability.” Many other 

more recent studies have reinforced the concerns raised by the Harvard study. Of 

particular note was one that focused on the care received by 1,047 hospitalized patients in 

a large teaching hospital affiliated with a medical school. It found that 17.7 percent of 

these patients received inappropriate care resulting in a serious adverse event--ranging 

from temporary disability to death.2 In the public eye, these scholarly inquiries have been 

overshadowed by media reports that describe, often in graphic detail, the harm done to 

patients because of poor hospital care.3 

Hospitals rely upon a variety of internal mechanisms, from physician credentialing, 

to peer review and benchmarking, in order to try to avoid such incidents and to improve 

the quality of care provided in their facilities. External quality review serves as a vital 

additional safeguard. It provides a more detached, independent mechanism for assessing 

the adequacy of hospital practices. Such oversight is of fundamental importance to 

patients and to the public and private entities that purchase health care services on their 

behalf. Protecting patient safety and improving the quality of patient care must be a top 

priority of external review. 

Medicare’s Interest in External Hospital Quality Review 

The Medicare program covers about 38 million elderly and disabled individuals, 

many of whom are high users of hospital care.4 In 1997, Medicare spent $136 billion on 

Part A, the hospital insurance benefit. This figure is just over half the total amount the 

Hospital Quality: The Role of Accreditation 6 OEI-01-97-00051 
Federal Government spent on all Medicare benefits.5 In the same year, Medicare spent 

over $80 billion for inpatient acute hospital care alone.6 

Since Medicare’s inception, external quality review has been a part of the 

Medicare program. When Congress enacted the Medicare Act in 1965, it required 

hospitals to meet certain minimum health and safety requirements to participate in the 

program.7 Those minimum requirements, called the Medicare conditions of participation, 

were published in 1966, revised in 1986, and are now being revised again (see appendix 

A).8 Within the Medicare Act itself, however, Congress provided that hospitals accredited 

by the Joint Commission on Accreditation of Healthcare Organizations were deemed to be 

in compliance with the conditions of participation. Congress also provided that hospitals 

accredited by the American Osteopathic Association could be considered in compliance, 

but only to the extent that the Secretary deemed appropriate.9 Thus, accreditation by the 

Joint Commission or the American Osteopathic Association provides entree into the 

Medicare program. About 80 percent of the 6,200 hospitals that participate in Medicare 

are accredited by the Joint Commission. Those hospitals wishing to participate in 

Medicare without accreditation must go through a Medicare certification process. The 

Health Care Financing Administration (HCFA) relies on State survey and certification 

agencies (hereafter called State agencies) to conduct certification surveys at these 

hospitals to determine compliance with the Medicare conditions. States currently certify 

1,442 nonaccredited hospitals nationwide.10 

Regardless of the route a hospital takes to Medicare participation, Medicare bears 

a cost for the external review, either directly by funding State surveys or indirectly through 

hospital charges that include the overhead cost of periodic accreditation surveys. 

The Nature of Accreditation 

Accreditation is a form of self-regulation. It is a voluntary “conformityassessment process” whereby industry experts define what standards organizations must 

meet in order to be accredited and then systematically review the organization’s 

performance against those standards.11 It typically aims to improve performance.12 

Organizations wishing to be accredited pay a fee for that service.13 

This Inquiry and Report 

This report examines the role of accreditation in external hospital quality review. 

It focuses on the Joint Commission on the Accreditation of Healthcare Organizations 

because it dominates the hospital accreditation market.14 In this report, we offer a 

framework for considering the external review of hospital quality and apply that 

framework to the Joint Commission’s approach to hospital accreditation. 

Hospital Quality: The Role of Accreditation 7 OEI-01-97-00051 
This report is part of a series of four companion reports that resulted from our 

inquiry. A second, The Role of Medicare Certification (OEI-01-97-00052), describes the 

extent and nature of the external review for nonaccredited hospitals. A third, Holding the 

Reviewers Accountable (OEI-01-97-00053), describes how HCFA oversees both the Joint 

Commission and the State agencies. The fourth report, A Call for Greater Accountability 

(OEI-01-97-00050), provides a summary of external hospital quality review and presents 

the recommendations emerging from our inquiry. 

Our inquiry draws on a variety of sources. These include: data from HCFA’s 

Online Survey Certification and Reporting System; aggregate data from the Joint 

Commission concerning hospital survey activity; a mail survey to State agencies in the 50 

States and District of Columbia (hereafter referred to as a State); observations of seven 

routine hospital surveys conducted by the Joint Commission and one conducted by a State 

agency; observations of two separate complaint surveys conducted by a State Agency; 

reviews of accreditation manuals, policies, and hospital survey files from the Joint 

Commission; the systematic gathering of information from representatives of HCFA 

central and regional offices, State agencies, the Joint Commission, American Hospital 

Association, consumer groups, professional associations, and representatives of other 

organizations we considered to be stakeholders in hospital oversight; and reviews of laws, 

regulations, and articles from newspapers, journals, newsletters, and magazines. We also 

interviewed officials from the American Osteopathic Association and reviewed their 

accreditation materials. The American Osteopathic Association accredits about 100 to 

150 hospitals, some of which are also accredited by the Joint Commission. For purposes 

of this inquiry, however, we focused on the Joint Commission. See appendix A for more 

details on our methodology. 

We conducted this inspection in accordance with the Quality Standards for 

Inspections issued by the President’s Council on Integrity and Efficiency. 

Hospital Quality: The Role of Accreditation 8 OEI-01-97-00051 
HOSPITAL REVIEW FRAMEWORK 

The following five components present a framework for considering the external quality review 

of hospitals. They are intended to complement the internal quality assurance and improvement efforts 

that hospitals undertake themselves. They are approaches that health care purchasers, such as Medicare 

and Medicaid, can rely upon to ensure that their beneficiaries receive quality services from hospitals. 

They can also be of use to beneficiaries and other consumers concerned about the quality of their hospital 

care. 

We present the components to facilitate analysis of the extent and type of external review that is 

desirable, whether carried out by accreditation bodies, certification agencies, Medicare Peer Review 

Organizations, HCFA, or others. Each component has strengths and limitations. Moreover, each can be 

used in support of a review philosophy based on continuous quality improvement, more traditional 

compliance enforcement, or some combination thereof. 

We omitted a sixth component: the retrospective review of medical records to determine 

appropriateness of care. Formerly a role of the Medicare Peer Review Organizations, such medical record 

review is no longer carried out on such a large scale. However, some medical record review does occur as 

part of the components described below. 

1. Announced, On-Site Surveys of Hospitals 

These involve some combination of observations of facility and equipment; reviews of medical 

credentials, and other records and documents; and interviews. They result in a pass/fail or some kind of 

score intended to distinguish level of performance. They can also involve follow-up to correct or improve. 

2. Unannounced, On-Site Surveys of Hospitals 

The approach is basically the same as above except that the hospital has not had time to prepare. 

The intent is to gain a clear assessment of the facility as it typically functions and to trigger any necessary 

follow-up. 

3. Response to Complaints Concerning Hospitals 

These involve complaints of a particular instance of care or more encompassing matters 

concerning a hospital’s performance. The response to complaints can range from a minimal distant 

review to a thorough on-site review. The process can trigger corrective actions and system improvements. 

4. Response to Major Adverse Events in Hospitals 

These involve cases where substantial patient harm resulted from what may be poor performance 

on the part of the hospital and/or its practitioners. Here, too, the response can range from minimal to 

thorough and can trigger corrective actions and system improvements. 

5. Collection and Dissemination of Standardized Performance Measures 

The aim here is to establish the standardized use of measures in ways that enable purchasers, 

consumers, accrediting bodies, and others to compare hospital performance. The comparisons can focus 

on a hospital’s own performance over time and/or on how its performance compares to other hospitals. 

The data can be drawn from surveys of patients or providers, billing claims, and the hospitals’ own 

records. 

Hospital Quality: The Role of Accreditation 9 OEI-01-97-00051 
PROFILE OF ACCREDITATION 

The table below summarizes our assessment of the Joint Commission’s approach 

to hospital accreditation, based on the framework we presented on the previous page. 

The remainder of the report follows this framework and elaborates on the five elements of 

the summary. 

Element Description Degree of Emphasis 

Routine, Announced, On-Site 

Surveys 

Conducted every 3 years 

to measure compliance 

with over 500 standards 

in 45 performance areas. 

Major. Core of existing accreditation process 

Random, Unannounced, OnSite Surveys Conducted on a sample 

of hospitals as a measure 

of continued compliance 

with standards in five 

performance areas. 

Minor. Affects about 5 percent of accredited 

hospitals per year. 

Responses to Complaints Includes tracking for 

trends but also some 

follow-up on-site, 

depending on the nature 

of the complaint. 

Minor. Few complaints result in on-site surveys; 

most are reviewed for trends. Limited attention to 

complaints during triennial surveys. 

Responses to Major Adverse 

Events 

Policy calls for hospital 

self-reporting and 

determining cause of 

event. Focus is on 

identifying opportunities 

for improvement and 

education. 

Minor to Moderate. Emphasis has been growing. 

Standardized Performance 

Data 

Hospitals participate in 

one of many Joint 

Commission-approved 

indicator system and 

begin submitting data in 

1999. 

None. Performance data not linked to 

accreditation. 

Hospital Quality: The Role of Accreditation 10 OEI-01-97-00051 
ANNOUNCED SURVEYS 

APPROACH 

Joint Commission surveys are undertaken in a collegial manner. 

Joint Commission surveyors are experienced nurses, physicians, and 

administrators. They are keenly aware of their roles as outside reviewers responsible for 

documenting compliance with Joint Commission standards. But they seek to carry out 

that role more as peer reviewers than as enforcers or regulators. In interacting with 

hospital staff, they strive to foster a collegial atmosphere characterized by a joint interest 

in improving hospital performance. 

It follows, therefore, that surveyors view education as a central part of their 

mission. Well-versed in the Joint Commission standards, they tend to be ambassadors of 

those standards, helping hospital staff to understand their intent and significance. Beyond 

that, they also use the survey as a way of informing hospitals about promising approaches 

they have found at other hospitals, even providing the names of individuals to contact. 

For example, we observed surveyors informing hospital staff of alternate methods for 

monitoring crash carts and sharing other hospitals’ approaches to defining privilege lists 

and adverse drug reactions. We also observed a surveyor briefly chat with a patient and 

then scan his chart to identify how the hospital staff recorded the goals set for this patient. 

While the surveyor identified dietary, rehabilitation, and medical goals in the chart, the 

chart lacked documentation reflecting the patient’s own stated goal: to ride his 

motorcycle again. This led to some educational discussion between the surveyor and 

caregivers on reflecting goals not only for individual disciplines, but also in terms of the 

patients and their needs as individuals. 

Given their collegial and educational orientation, surveyors tend to approach their 

information gathering in a nonchallenging manner. For example, they allow the hospital to 

select the majority of records to be reviewed.15 They tend to be accepting of information 

and answers provided by the hospital staff, with little probing to determine if surface 

appearances describe actual practice. They seldom reflect any wariness that hospital staff, 

in their eagerness to impress surveyors, may exaggerate points or omit information. For 

example, in one hospital, a surveyor scanned some performance data prominently 

displayed on a storyboard, questioned the cause of significant deviation in the charted 

data, but when staff failed to respond did not pursue the matter. In another, a surveyor 

noted with surprise that the hospital staff had failed to pull certain files he requested, but 

did not insist that the requested files be provided. 

Hospital Quality: The Role of Accreditation 11 OEI-01-97-00051 
Joint Commission surveys are tightly structured. 

The Joint Commission uses a standard agenda in surveying hospitals. In 2 to 5 

days, depending on hospital size, surveyors conduct dozens of individually scheduled 

sessions that cover the major aspects of a hospital’s operations (see appendix B for a 

sample agenda). The survey pace is rapid, typically allowing 45 to 60 minutes for each 

meeting or patient care visit before moving on to the next visit.16 A few 15-minute 

intervals might be allotted each day as unscheduled for surveyors’ personal time or phone 

calls. Sessions are scheduled back to back. 

The survey provides surveyors with a general overview of the hospital. It 

facilitates standardized assessments across facilities and protects against inconsistencies or 

the special interests of individual surveyors. Also, because the schedule, which hospitals 

receive in advance, spells out exactly which surveyor will be where over the course of the 

survey, individual departments can prepare by having everyone ready for the surveyor and 

the area cleaned and organized.17 

The packed agenda, however, affords few opportunities for surveyors to develop 

hunches, follow leads, or even respond to complaints. Racing from session to session, 

surveyors have little time to probe deeply. As one surveyor described after surveying a 

700-bed hospital, “The team found this to be a harried, rushed survey due to the number 

of areas covered and the timeframe allowed...Patient units were cut short to allow for 

travel time between buildings...The time allotted for the building tour and medical record 

review was grossly inadequate.”18 

The surveyors know little going in to a hospital that might help them probe more 

effectively. Their advance knowledge is limited to some basics on the services offered, 

last triennial survey results, and, in some cases, complaints and media articles. They know 

little, if anything, about the local health care market in which the hospital operates, 

including whether there are local labor shortages or union disputes. While such limited 

knowledge deliberately reduces bias and promotes consistency, it also hinders targeted 

probing and a deeper understanding of the hospital’s operations. For example, on the first 

day of a hospital survey, the local newspaper published a lead editorial about a heated 

battle between this hospital and another for dominance in children’s services. This 

competition was much on the minds of the hospital’s leadership, but the surveyors failed 

to ask about it because they were unaware of it. The surveyors were also unaware that 

the hospital had a random unannounced survey 18 months earlier. 

Hospital Quality: The Role of Accreditation 12 OEI-01-97-00051 
STRENGTHS 

Joint Commission standards are dynamic. 

The Joint Commission’s ability to quickly adapt the standards that form the 

foundation of the accreditation process enables it to respond to changes in health care 

delivery and evaluation methodology. The Joint Commission is constantly developing 

new, refining existing, and eliminating out-of-date standards. Practitioners, experts, and 

stakeholders from around the country weigh in on this process through the Joint 

Commission’s committee structure. That committee includes representatives from 

professional and industry groups, such as the American Medical Association, the National 

Association of Healthcare Quality, and the American Academy of Family Physicians, as 

well as from the Health Care Financing Administration. The stakeholders we surveyed 

considered ability to improve standards without a long regulatory process to be one of the 

greatest strengths of the accreditation process. Even a critic of the Joint Commission 

conceded that “the standards are wonderful. [One] can’t help but admire them.” 

Over the past decade, the Joint Commission’s standards changed dramatically.19 

The new and revised standards reflect a more patient-centered focus, with greater 

emphasis on patient rights. The Joint Commission replaced old standards concerning 

committee meeting minutes with standards on strategic planning, resources for continuous 

quality improvement, and managing information. These changes were so substantial that, 

by 1996, the Joint Commission eliminated 80 percent of standards from its 1986 manual.20 

Joint Commission surveys serve as a means of reducing risk. 

At its core, the survey process is about reducing risk. The surveys prompt 

hospitals’ attention to minimum protections that are important to patients.21 Hospitals 

know that once every 3 years the surveyors will be onsite and they take the accreditation 

process seriously.22 They know the survey dates in advance and prepare for them. The 

survey process gives the hospital’s leadership a platform from which to rally attention to 

assessing its condition and correcting problems in time for the survey; hospitals want to 

put their best foot forward. For example, prior to surveys, hospitals put extra efforts in 

their appearance and make sure manuals are up-to-date. Many hospitals hire consultants 

to help them with this process by conducting mock surveys or reviewing existing 

conditions to identify areas falling short of accreditation standards. While some of these 

efforts are cosmetic, others extend beyond the cosmetic and include efforts such as 

training staff on standards. 

Of course, the Joint Commission also has a platform from which it can promote 

particular patient-centered concerns by simply making them survey priorities. For 

example, over the past several years, the Joint Commission has identified hospital use of 

patient restraints as a such a priority.23 Indeed, surveyors found problems with restraints 

Hospital Quality: The Role of Accreditation 13 OEI-01-97-00051 
at 55 percent of hospitals surveyed between June 1995 and May 1997.24 Hospitals were 

well aware of this focus--newsletters and Internet chat rooms devoted to accreditation 

surveys often feature survey trends--and responded through efforts such as improvement 

projects aimed at reducing restraint use.25 Other topics garnering attention from the Joint 

Commission recently include smoking in hospitals, conscious sedation, and emergency 

medicines. 

In addition to focusing on priority areas such as restraints, the triennial surveys 

reinforce attention to other, basic protections that span administrative and clinical realms. 

For example, surveyors review hospitals for their fire safety procedures, including 

reviewing fire drill logs and checking firedoors and firewalls. They check the security and 

expiration of medicines and supplies. They look for infection control issues such as 

separation of clean and dirty. They check how hospitals record drug allergies and monitor 

adverse drug reactions. They review the timely inclusion of histories and physicals in the 

medical records. They verify the existence and staff knowledge of hospital policies. For 

example, at one hospital, we observed the nurse surveyor review policies and minutes of 

the ethics committee. He then verified that nurses on patient care units used the 

committee appropriately by asking about their most recent referrals to it. 

Joint Commission surveys foster attention to continuous quality improvement. 

Consistent with its mission to “improve the quality of care provided to the public 

through...performance improvement in health care organizations,” the Joint Commission 

promotes the use of continuous quality improvement through the measurement of 

performance based-standards.26 Continuous quality improvement has been a major trend 

in health care over the past decade, and the Joint Commission adopted its focus on 

measurement and outcomes through its Agenda for Change.27 In fact, 88 percent of the 

stakeholders we interviewed expect 

accreditation to play a great or very 

great role in fostering continuous 

quality improvement in hospitals. 

Attention to continuous 

quality improvement pervades the 

entire hospital survey process. The 

performance improvement overview 

session, one of the first sessions of 

the survey, provides an opportunity 

for hospital staff to present their 

performance improvement 

methodology, to which surveyors 

refer throughout the survey. 

Fostering An Understanding of Measurement 

While surveying a unit serving the frail elderly, 

a nurse surveyor engaged staff in a discussion 

about identifying and measuring patient 

outcomes. Through careful questioning, he led 

the staff to understand the difference between the 

process of care (in this case, a moist healing 

environment for a pressure ulcer) and the 

outcome of care (a smaller and, eventually, 

healed ulcer). He also helped the staff 

understand the importance of measuring the 

impact of the staff’s interventions on patient 

outcomes. 

Hospital Quality: The Role of Accreditation 14 OEI-01-97-00051 
Surveyors reinforce the importance of quality improvement in sessions with hospital leaders and 

medical staff, where they confirm leadership involvement and ask how projects are identified and 

prioritized. The survey agenda also reserves specific sessions for performance improvement 

teams to showcase their most successful projects for surveyors. The performance improvement 

projects presented may involve topics such as reducing patient waiting times, improving the 

timeliness of medical record retrieval, reducing medication errors, and reducing the use of patient 

restraints. As the surveyors move throughout the hospital, staff are eager to explain the 

performance improvement storyboards on display in their units. And surveyors often ask staff 

what their units are doing better this year than last and what they will do better next year. 

Depending on the hospitals’ sophistication in performance improvement, surveyors 

act as either educators, sharing advice and encouragement on measuring outcomes, or as 

cheerleaders, applauding the hospitals’ performance improvement efforts. Some hospitals 

have moved beyond Joint Commission requirements, hiring consultants to help them with 

their performance improvement efforts, sending medical staff to training on these 

techniques, or paying medical staff for time spent on performance improvement projects. 

In hospitals with sophisticated performance improvement projects, surveyors offer 

compliments and collect ideas to pass on to other hospitals. 

LIMITATIONS 

Joint Commission surveys are unlikely to surface patterns, systems, or incidents 

of substandard care. 

Looking for questionable care runs counter to the tight structure and collegial 

nature of the Joint Commission’s survey process. That process leaves little opportunity 

for identifying such patterns, systems, or incidents. It is more oriented toward reducing 

risks, improving performance, and educating. 

The Joint Commission’s approach to medical record review illustrates its limits. 

While surveyors review both closed and open medical records during surveys, their focus 

is more on processes of care than appropriateness of care. In fact, surveyors “do not 

judge directly whether the care given to a specific patient is good or bad, right or 

wrong.”28 Nor do they choose medical records based on indications of poor quality. 

Rather, the hospitals generally choose the records for surveyors, and, in some cases, 

hospital staff also conduct the review. While Joint Commission surveyors find problems-­

even patterns--through medical record review, they tend to be related to processes, such 

as documenting verbal orders, timely recording of the history and physical, and assessing 

patients’ educational needs. While important, these process issues are apparent through 

even a quick review of the records. The Joint Commission’s approach is unlikely to reveal 

problems or patterns that might take a more thorough examination to uncover, such as 

inappropriate surgeries, high complication rates, or poor or unexpected outcomes. 

Hospital Quality: The Role of Accreditation 15 OEI-01-97-00051 
The Joint Commission survey process falls short of identifying patterns or systemic 

weaknesses because, at least in part, surveyors lack the time or the up-front knowledge 

that would help them dig deeper. They get a broad rather than in-depth view of the 

hospitals’ operations. Thus, the survey would be more apt to reveal unsecured narcotics 

or outdated drugs than it would a pattern of mismedication. It would be more apt to 

reveal lapses in fire drills than lapses in continuity of care. It would be more apt to reveal 

overdue staff evaluations than staff performing tasks for which they are untrained. 

Indeed, the survey process may not reveal any problems when in fact systemic 

problems exist. For example, in the Spring of 1996, the Joint Commission awarded one 

hospital its highest level of accreditation: accreditation with commendation. That Fall, 

the hospital experienced an unexpected death, triggering the State agency to investigate. 

In the spring of 1997, more unexpected deaths occurred, and the agency returned. After a 

3-week investigation, that agency found systemic problems in both quality assurance and 

medical staffing.29 While no system of oversight is foolproof, the Joint Commission 

survey process failed to uncover what the State agency identified as deep-rooted 

problems. 

Joint Commission surveyors have limited information from which to draw to help 

them identify patterns or incidents. For example, surveyor preparation packets lack 

practice pattern or performance data that could provide surveyors with insights into 

identifying patterns of questionable care. In fact, the surveyors have virtually no 

interactions or information from other stakeholders, such as Medicare Peer Review 

Organizations or consumer groups, who might have insights into the hospitals’ 

performance. 

Joint Commission surveys are unlikely to identify individual practitioners whose 

judgment or skills to practice medicine are questionable. 

The collegial process, tight structure, limited time, approach to medical records, 

and lack of background information all make it unlikely that the survey process will 

identify individual practitioners who pose risks to their patients. In fact, the Joint 

Commission’s own publications note that the accreditation process does “not evaluate the 

quality of care provided by individual medical staff members” and “does not provide a 

warranty that a particular individual will receive quality care in a specific health care 

setting at a particular time.”30 

A standard part of an accreditation survey is a review of hospitals’ own processes 

in ensuring the competence of their practitioners. But that review of credentialing and 

privileging offers, at best, a preliminary and superficial assessment. It generally lasts 45 to 

60 minutes, during which the surveyor both interviews the medical staff leadership and 

reviews files (in some cases, the hospital staff also reviews files). 

Hospital Quality: The Role of Accreditation 16 OEI-01-97-00051 
While the hospitals generally choose the files for review in the credentials session, 

they do so at the direction of the surveyors, ensuring different privileges (active, courtesy, 

consulting, and temporary, for example) and specialties are represented. But time is too 

short for any in-depth review of these files. For example, in one hospital with over 500 

active staff, the surveyor reviewed three practitioners’ credentials and privileges. Even 

though he found a problem with one of the three (a podiatrist who was performing surgery 

for which he was not privileged), the surveyor reviewed no additional files. 

The credentials session falls short in other ways, as well. While the surveyors do 

ask important questions about the hospitals’ processes for matching privileges to 

competencies and verifying licenses, we observed no surveyor ask how the hospital 

identified or dealt with physicians whose knowledge or practice skills were marginal.31 

Even though Federal law requires hospitals to report any practitioner they have disciplined 

to the National Practitioner Data Bank, it is unlikely that surveyors will make any 

determination about hospitals’ compliance with that law: Joint Commission standards do 

not currently reference the law for reporting to the Data Bank, and the likelihood is low 

that surveyors will choose a file of a disciplined physician randomly.32 In fact, one 

physician surveyor chastised the staff for pulling confidential physician files for the 

surveyor to review. Such files contain peer review assessments and other details on the 

physicians’ skills. 

The collegiality of the sessions hinders raising sensitive topics, such as physicians’ 

fitness to practice, in a meaningful way. For example, during the review of credentials, 

one physician surveyor shared a story about a hospital that was about to reduce privileges 

of an older surgeon whose skills had declined. The surgeon had practiced at the hospital 

for many years and was a stalwart of the hospital community and respected throughout the 

State. Before the hospital could act, the State legislature named the surgeon “Physician of 

the Year.” The story was met with nervous laughter by the medical staff. While the story 

touches on the difficulties of dealing with those whose skills are declining, the surveyor 

followed up with neither questions of whether the hospital was faced with anything similar 

nor education about how to deal with such situations. 

Other survey sessions offer some insights into the practitioners skills, but only 

peripherally. For example, the document review session provides an opportunity for 

surveyors to review any minutes or records that might document some problems with 

individuals, though whether the surveyors would have the time to identify those 

documents amidst the hundreds they scan is uncertain at best. The surveys also include 

interview sessions with nursing and medical leadership. But the high attendance and 

collegial atmosphere at these sessions is generally counter to discussing individuals. 

Hospital Quality: The Role of Accreditation 17 OEI-01-97-00051 
SCORING 

Joint Commission survey results matter enormously to hospitals. 

Hospitals attach great significance to accreditation scores and status. They know 

the accreditation scores of competing hospitals, and aim to beat them with higher scores 

of their own. It is not enough to simply be accredited--nearly every hospital surveyed gets 

accredited--they want to be accredited with commendation, the highest level awarded by 

the Joint Commission. After all, hospitals operate in an increasingly competitive 

marketplace, where edging out the competition in gaining contracts with managed care 

plans and other purchasers can be the difference between surviving and going out of 

business. They know that their survey results will be available to the public.33 Thus, 

hospitals that do achieve accreditation with commendation promote their survey results 

(see appendix D).34 They advertise their status in banners hung outside their hospitals. 

They announce it in full-page ads in their local papers. They post it on their websites and 

throughout the hospital. 

This focus on the survey scores permeates the survey process, introducing some 

tension into the otherwise collegial process. Surveyors know hospitals aim to do well on 

their surveys--after all, the hospital leadership often follows them around during the entire 

survey process. Hospitals often hire consultants to help them prepare and increase the 

likelihood they will achieve accreditation with commendation. Surveyors and hospitals 

alike know that just one deficiency can thwart accreditation with commendation. This can 

lead to pressure to mitigate the impact of deficiencies, which can play out in a couple of 

ways. One approach, for example, involves what we have dubbed the denominator game: 

increasing the numbers of documents reviewed (the denominator) in order to decrease the 

significance of problems identified in any one record (the numerator).35 We observed 

hospitals attempting to mitigate findings on restraints, dental histories in the medical 

record, and security of drugs with this approach, with varied success. Surveyors, of 

course, also have some flexibility in scoring the standards. For example, we observed one 

surveyor identify a problem with a podiatrist performing surgery that exceeded his 

privileges. The surveyor scored this as a documentation rather than medical staff problem, 

thereby mitigating its impact on the final score. 

Hospital Quality: The Role of Accreditation 18 OEI-01-97-00051 
Joint Commission survey results fail to make meaningful distinctions among 

hospitals. 

Even though the Joint 

Commission has 5 levels of 

accreditation, 99 percent of 

the hospitals surveyed 

between May 1995 and June 

1998 clustered in just 2 of 

those levels (see table 1): 

accreditation with 

commendation (16 percent) 

and accreditation with type 1 

recommendations (83 

percent).36 The difference 

between some levels of 

accreditation can be difficult 

to discern. For example, 

hospitals accredited without 

recommendations for 

improvement may miss the 

commendation status by a 

narrow margin. On the other 

hand, accreditation with 

Table 1 

Accreditation Levels for Hospitals Surveyed 

June 1995 - May 1998 

Source: Joint Commission on the Accreditation of Healthcare

Organizations.

Note: n=17130 for 6/95-5/96 , n=1709 for 6/96-5/97, and n=1719

for 6/97-5/98.

Accreditation 

Level 

6/95 

to 

5/96 

6/96 

to 

5/97 

6/97 

to 

5/98 

Total 

Accreditation with 

Commendation 13% 18% 15% 16% 

Accreditation without 

Type 1 Recommendations <1% 1% 0 <1% 

Accreditation with Type 1 

Recommendations 86% 80% 83% 83% 

Conditional Accreditation <1% 1% 1% 1% 

Preliminary NonAccreditation 0 0 <1% <1% 

recommendations for improvement (type 1 recommendations) is a broad category that 

encompasses hospitals with just one recommendation to hospitals having many.37 

In addition to the accreditation level, hospitals accredited by the Joint Commission 

receive a score based on a 100-point scale. Fully 99 percent of hospitals accredited in 

1997 received scores over 80: 73 percent received scores of 90 or higher and 26 percent, 

from 80 to 89. Just 1 percent received from 70 to 79.38 Given the Joint Commission’s 

own interpretation of scores, this spread implies few “real differences” across accredited 

hospitals: 

The smaller the difference in scores between hospitals, the less likely there is an actual 

difference in the levels of performance between them. There may not be a real difference 

between a hospital that scores 88 and a hospital that scores 81. However, the greater the 

difference in scores, the more likely there is a difference in patient care.39 

On at least one occasion, the Joint Commission has found its own survey too blunt 

a tool to clearly distinguish a hospital as unacceptable. In this case, two State agencies, 

HCFA, and the Joint Commission were engaged in surveying and resurveying a hospital. 

Concerns about this hospital ranged, for example, from rat infestations to the hospital 

taking away a patient’s mattress and bedframe, to improper patient restraints. Yet it was 

Hospital Quality: The Role of Accreditation 19 OEI-01-97-00051 
not until the Joint Commission devised a special survey that the hospital failed to achieve 

accreditation. 

Hospital Quality: The Role of Accreditation 20 OEI-01-97-00051 
UNANNOUNCED SURVEYS 

The Joint Commission’s reliance on unannounced surveys is limited. 

The Joint Commission uses random unannounced surveys as a way to ensure 

continued compliance with accreditation standards between triennial surveys.40 From June 

1995 through May 1998, the Joint Commission conducted 250 random unannounced 

surveys, representing about 5 percent of its accredited hospitals. Eleven percent of those 

surveys (28) resulted in a change in the hospitals’ accreditation status. The Joint 

Commission required some corrective follow-up (a written progress report or a focused 

survey) for about a third (80, or 32 percent). 

The Joint Commission does, in fact, give advance notice of its “unannounced” 

surveys. But unlike the triennial surveys, which are scheduled several months in advance, 

the Joint Commission provides considerably less notice to hospitals undergoing these 

random surveys: generally 24 to 48 hours. By contrast, the New York State Health 

Department recently sent 72 inspectors to 12 hospitals without any warning at all. The 

inspectors arrived at the hospitals on a Thursday afternoon and spent the weekend 

investigating the hospitals’ supervision of residents.41 State agencies, in conjunction with 

the Health Care Financing Administration, often give no warning to hospitals prior to 

showing up to investigate a complaint or incident of patient harm.42 In fact, in the case of 

nursing homes, Federal law allows a fine of up to $2,000 for anyone warning the facility of 

an upcoming survey.43 Some hospitals appear to recognize some intrinsic value to the 

surprise of an unannounced evaluation, as they themselves hire people to pose as patients 

and then report on their experience.44 The Joint Commission has not used that approach. 

The focus of random unannounced surveys is narrower than the triennial surveys. 

Rather than encompassing all hospital accreditation standards, they focus on the top five 

performance areas that the Joint Commission finds out of compliance. Because of the 

narrower focus, one surveyor conducts the random unannounced and the entire survey 

takes 1 day. The one random unannounced survey we observed began at 8:30 in the 

morning and ended at 3:15 in the afternoon. The Joint Commission notified that hospital 

24 hours before the surveyor, an administrator, arrived at the hospital. The survey 

resembled the triennial survey in approach. For example, the surveyor conducted the 

survey in the same collegial manner as in a triennial; he often focused on educating the 

hospital staff on particular standards and their intent. His advance knowledge of the 

hospital was limited to a copy of the notice to the hospital’s executive explaining the focus 

of the survey and a sheet describing some basic information about the hospital and its 

services. He displayed little skepticism. In fact, during one 5-minute interview of the 

human resources staff, the surveyor proclaimed their answers to be “textbook perfect,” 

thereby negating his need to review any of the files they had pulled for him. 

Hospital Quality: The Role of Accreditation 21 OEI-01-97-00051 
Responses to Major Adverse Events 

The Joint Commission’s approach to major adverse events has evolved to one 

that addresses such events as opportunities for system improvement. It is in 

accord with recently developed precepts on reducing errors in medicine. 

In recent years, as the media have reported many events involving serious harm to 

hospitalized patients, the Joint Commission has devoted more attention to them under the 

rubric of its “sentinel event” policy.45 

When it first established this policy in 

April 1996, it assessed the hospital’s 

responsibility for them through an onsite investigation.46 If the Joint 

Commission found that the incident 

was, indeed, serious, and that the 

hospital could reasonably have been 

expected to prevent it, then the Joint 

Commission typically downgraded 

the hospital’s accreditation to 

“conditional”--a publicly disclosed 

status that called for the hospital to 

determine the cause of the event.47 

Since then, the Joint 

Commission’s sentinel event policy 

has evolved to one that is gentler to 

hospitals. Its first important move in 

The Joint Commission’s Definition of a 

Sentinel Event 

“(1) The event has resulted in an unanticipated death 

or major permanent loss of function, not related to 

the natural course of the patient’s illness or 

underlying condition, or (2) The event is one of the 

following (even if the outcome was not death or 

major permanent loss of function): Suicide of a 

patient in a setting where the patient receives 

around-the-clock care (e.g., hospital residential 

treatment center, crisis stabilization center); Infant 

abduction or discharge to the wrong family; rape; 

Hemolytic transfusion reaction involving 

administration of blood or blood products having 

major blood group incompatibilities; or Surgery on 

the wrong patient or wrong body part.” 

this direction was in October 1996 when it replaced the downgrade to conditional 

accreditation, which hospitals regarded as unnecessarily punitive, with a status called 

“Accreditation Watch”-- a publicly discloseable status but one that in itself did not affect a 

hospital’s existing accreditation status. More significantly, in April 1998, the Joint 

Commission recast its overall approach toward such events to one that is much more 

analytic than investigative. This altered approach builds on the premise that medical errors 

occur regularly, even in the best hospitals. It seeks continuous quality improvement in 

health care by treating errors as opportunities to improve systems of care rather than to 

find and punish culpable individuals or institutions. Toward this end, it aims to create a 

protected venue where it is safe to acknowledge errors and where practitioners and 

administrators analyze them to develop better error-prevention strategies--ones that get at 

the causes for errors.48 

Hospital Quality: The Role of Accreditation 22 OEI-01-97-00051 
Thus, the Joint Commission’s current policy on major adverse events stresses 

voluntary and timely self-reporting both of the incidents and of the hospitals’ analyses of 

them.49 Hospitals that proceed in this way avoid Accreditation Watch and are much less 

likely to be subject to an on-site investigation of an incident. Through this approach, the 

Joint Commission expects to develop a substantial database of events that it can analyze 

for patterns of frequency and causes.50 It intends to communicate this information to 

accredited hospitals so that they can prevent such events. 

It is still early to fully assess the effects of this new approach. But the initial 

indications suggest that this gentler, educationally oriented approach is beginning to take 

hold. From April 1, 1998 (when the policy was implemented) until January 7, 1999 (the 

latest date for which data are available), the Joint Commission has been less inclined to 

place hospitals on Accreditation Watch or to change their accreditation status as a result 

of an adverse event, and hospitals have been more inclined to self-report sentinel events to 

the Commission (see table 2).51 

Hospital Quality: The Role of Accreditation 23 OEI-01-97-00051 
Table 2 

Sentinel Event Tally 

Sentinel Events in the Database 

Total Number of Events Added to the

Joint Commission Database between

4/1/96 and 1/7/99 ........................... 374

Number Self-reported

4/1/96 - 3/31/98 ...................... 36

Number Self-reported since

the New Policy (4/1/98 and 1/7/99) ....... 115

Hospitals on Accreditation Watch 

Total Number of Hospitals Placed on 

Accreditation Watch 10/1/96 - 1/7/99 ............. 38

Number 10/1/96 - 3/31/98 ............... 35

Number since the New Policy 

(4/1/98 and 1/7/99) ..................... 3

Hospitals with an Accreditation Status Change 

Total Number of Hospitals with a Change in

Accreditation Status Due to Sentinel Event 

Review between 10/1/96 and 1/7/99 .............. 149

Number 10/1/96 - 3/31/98 .............. 146

Number since the New Policy 

(4/1/98 and 1/7/99) ..................... 3

Source: Joint Commission on the Accreditation of Healthcare 

Organizations. 

Hospital Quality: The Role of Accreditation 24 OEI-01-97-00051 
The Joint Commission’s approach emphasizes confidentiality, thereby inhibiting 

public disclosure on the causes, consequences, and responses to major adverse 

events. 

For hospitals complying with the Joint Commission’s self-reporting format, the 

Joint Commission does not publicly release any information, nor upon inquiry of a 

hospital’s accreditation status does it identify that any analysis is underway. If the inquirer 

specifically mentions the adverse event, the Joint Commission simply indicates that it is 

working with the hospital on it. But even with this restrictive approach, hospitals have 

been wary of the self-reporting policy as they fear that the confidentiality of the Joint 

Commission’s database would not be adequately protected under State and Federal laws 

and that as a result they would be exposed to malpractice liability.52 The Joint 

Commission has been examining legislative remedies, at State and Federal levels, to 

address this concern and to buttress its limited disclosure policy. 

This emphasis on the confidentiality of adverse event related information may 

facilitate the quality improvement agenda of the Joint Commission, but it is at the expense 

of its public accountability. It leaves the general public with minimal information about 

how an incident happened and what if anything is being done to prevent more. This 

presents particular difficulties if media reports of the incident heighten local concern. In 

such cases, the public is likely to be looking for assurances from an objective outside party 

that any problems at the hospital have been addressed and that patient safety is maintained. 

Under its current policy, the Joint Commission can offer few specifics to support any such 

assurances.53 The results of investigations conducted by State agencies or HCFA itself are 

likely to be more available to the public. 

Hospital Quality: The Role of Accreditation 25 OEI-01-97-00051 
RESPONSES TO COMPLAINTS 

The Joint Commission devotes little emphasis to complaints. 

The Joint Commission’s accreditation process addresses complaints more 

tangentially than centrally. Although it receives and responds to complaints during 

triennial surveys, the limited time surveyors devote to complaints is squeezed from other 

survey activities in an already-packed agenda. And while the Joint Commission requires 

hospitals to inform the public about upcoming surveys 30 days in advance with notices 

that explain how to register complaints, it has no standing requirement for hospitals to 

post such notices continuously.54 

The rate of complaints the Joint Commission received through its triennial survey 

process (referred to as public information interviews) remained flat from 1996 to 1997: 

surveyors conducted 111 such interviews in 1996 and 103 in 1997, involving about 6 

percent of surveys in each year.55 Thirteen percent of the public information interviews 

resulted in areas identified for improvement (i.e., type 1 recommendations). Historically, 

such public information interviews took place with both surveyors and hospital 

representatives present, which likely had a chilling effect on staff wishing to complain. 

But in 1997, the Joint Commission changed its policy to allow confidential opportunities 

for public information interviews. It remains to be seen how this revised policy will affect 

the complaints received over time.56 

The Joint Commission also receives complaints in its central office, which it tracks 

for trends or sends to surveyors for review during triennial surveys.57 In March of 1999, it 

implemented a new toll-free complaint hotline.58 When a complaint or incident such as an 

unexpected death alerts the Joint Commission of a possibly grave threat to patient care or 

safety in an accredited hospital, it conducts either an unannounced or unscheduled survey 

(these differ from the random unannounced surveys discussed previously). During the 3-

year period from June 1995 through May 1998, the Joint Commission conducted these 

surveys in about 6 percent of accredited hospitals (172 unannounced and 136 unscheduled 

surveys). While hospitals do not receive prior notice of an unscheduled survey, the Joint 

Commission gives 24 to 48 hours notice before an unannounced survey. Both types can 

result in changes in accreditation status and areas requiring follow-up.59 

Sometimes, the Joint Commission, HCFA, and/or the States agencies receive the 

same complaints. While some sharing of complaint information occurs, officials from 

State agencies, HCFA, and the Joint Commission all expressed some concern about the 

consistency of that sharing. Some also expressed concerns about the lack of coordination 

in following up on complaints. 

Hospital Quality: The Role of Accreditation 26 OEI-01-97-00051 
Standardized Performance Data 

Despite the Joint Commission’s early plans to incorporate standardized hospital 

performance data into accreditation, it remains of little value to external 

assessments of hospital quality. 

In 1986, the Joint Commission unveiled its “Agenda for Change,” a vision for a 

performance-based accreditation system that included clinical indicators. It envisioned a 

system where data based on such indicators would enhance the accreditation process by 

making it more data-driven and providing insights into hospitals’ performance on a 

continuous basis.60 That vision evolved to include public access to hospital performance 

reports and links between accreditation and performance measurement. While it has made 

strides in the former, linking accreditation to performance measurement has been more 

troublesome. 

As its vision unfolded, the Joint Commission faced considerable hurdles. For some 

stakeholders, the Joint Commission was moving too fast on performance measures; for 

others, too slow. For example, its initial vision required hospitals to participate voluntarily 

in its own data system, called the Indicator Measurement System (IMS), until 1996, when 

participation would become mandatory. The hospital industry resisted the IMS, finding it 

expensive and burdensome.61 Concerns about the science of performance measurement 

and risk adjustment, considered by many to be still in its infancy, also plagued the Joint 

Commission’s implementation. Many hospitals had already invested in other competing 

performance measurement systems. The Joint Commission’s commitment to making 

hospital performance data public also rankled. 

Thus, while the Joint Commission has maintained its vision of a performance-based 

accreditation system, it has proceeded at a much slower pace and with more flexibility 

than it initially sought. For example, it no longer requires hospitals to participate in the 

IMS. Instead, the Joint Commission offers two programs: Oryx and Oryx+. Oryx 

reflects the minimum requirement for accredited hospitals, which are required to choose a 

performance measurement system from a list of approved systems, of which IMS is one 

option.62 Hospitals wishing to do more than the minimum participate in Oryx+, which is 

an accelerated version of Oryx that requires participating hospitals to use the same 10 

performance measures. Oryx supports comparisons only to individual hospitals’ own 

performance over time, whereas Oryx+ allows comparisons across participating hospitals. 

With the exception of the hospitals participating in Oryx+, this approach left the 

Joint Commission a long way from its earlier vision of uniform data from all hospitals. In 

the words of one stakeholder, Oryx is akin to saying “you have to wear a school uniform 

Hospital Quality: The Role of Accreditation 27 OEI-01-97-00051 
but may choose it yourself.” To advance its vision toward uniform data, the Joint 

Commission began collecting and evaluating potential “core performance measures” in late 

1998.63 Considered the next phase of the Oryx initiative, the Joint Commission aims to 

choose five or six areas (such as diabetes or breast cancer) to focus on by the end of 1999. 

How this new phase of the Oryx initiative will unfold is unknown, but it has 

already caused considerable backlash. Seventeen State hospital associations wrote to the 

Joint Commission in January 1999 to voice their concerns about the core performance 

measures.64 They raised many of the same concerns raised in 1995 when the Joint 

Commission introduced the IMS, including the costs, burdens to hospitals, and timing of 

implementation.65 

In addition to its Oryx initiatives, the Joint Commission recently joined forces with 

the American Medical Association and the National Committee for Quality Assurance to 

create the Performance Measurement Coordinating Council. The Council aims to develop 

standards for performance measurement with an eye toward more consistency.66 

Hospital Quality: The Role of Accreditation 28 OEI-01-97-00051 
CONCLUSION 

Unquestionably, the Joint Commission is the central force in the external review of 

hospital quality. It accredits about 80 percent of the hospitals in the country and, for 

Medicare purposes, it has a congressionally granted deeming status that is unique among 

accrediting bodies. Medicare beneficiaries and others who rely upon hospital services 

have much at stake in how and how well the Joint Commission does its job. 

Our review underscores that the core element of the Joint Commission’s approach 

to accreditation is the announced, on-site survey of hospitals--a survey that is heavily 

oriented toward educational and performance improvement objectives. The other 

elements of external review--unannounced surveys, responses to complaints and serious 

incidents, and standardized performance measures--play relatively minor roles in the Joint 

Commission’s accreditation process. 

Given the significance of the Joint Commission’s role and its emphasis on one 

approach to external quality review, our inquiry surfaces important policy questions for 

HCFA: How can it best ensure an appropriate balance in external quality reviews of 

hospitals? How can it best hold the Joint Commission accountable for the important 

public role it performs while enabling it, at the same time, to have enough flexibility to 

continue to advance the state-of-the-art of hospital accreditation? We address these 

questions in our summary report, A Call for Greater Accountability. That report also 

contains our recommendations, which we direct to HCFA. 

Hospital Quality: The Role of Accreditation 29 OEI-01-97-00051 
APPENDIX A 

Medicare Conditions of Participation 

The Medicare Conditions of Participation (COP) were first published in 1966 and revised in 1986 (42 C.F.R. 

482.1-482.66). On December 19, 1997, HCFA published a proposed COP for hospitals ( 62 Fed. Reg. 66,726). 

On July 2, 1999, it published an interim final rule on patients’ rights (64 Fed. Reg. 36,070). Below are the 

components of the existing COP for non-specialty hospitals from 1986, followed by the components of the new 

proposed COP. 

Existing COP 

Subpart A- General Provisions 

Provision of emergency services by 

nonparticipating hospitals 

Subpart B- Administration 

Compliance with Federal, State, and local laws 

Governing Body 

Subpart C- Basic Hospital Functions 

Quality assurance

Medical staff

Nursing services

Medical record services

Pharmaceutical services

Radiologic services

Laboratory services

Food and dietetic services

Utilization review

Physical environment

Infection control

Subpart D- Optional Hospital Services 

Surgical services 

Anesthesia services 

Nuclear medicine services 

Outpatient services 

Emergency services 

Rehabilitation services 

Respiratory care services 

Proposed COP 

Subpart A- General Provisions 

Patient Rights (issued as an interim final rule on July 7, 

1999) 

Subpart B- Patient Care Activities 

Patient Admissions, assessment, and plan of care

Patient care

Quality assessment and performance improvement

Diagnostic and therapeutic services or 

rehabilitation services 

Pharmaceutical services 

Nutritional services 

Surgical and anesthesia services 

Emergency services 

Discharge Planning 

Subpart C- Organizational Environment 

Administration of organizational environment

Infection control

Human resources

Physical environment

Life safety from fire

Blood and blood products transfusions

Potentially infectious blood and blood products

Utilization review

Hospital Quality: The Role of Accreditation 30 OEI-01-97-00051 
APPENDIX B 

Methodology

We collected information presented in this report from the following sources: 

The HCFA 

We obtained dates of certification surveys from HCFA’s Online Survey 

Certification and Reporting System (OSCAR). The HCFA authorizes States to update 

and maintain this database with survey information. We extracted survey data pertaining 

to the frequency of certification surveys. We subsequently verified the accuracy of our 

extraction by comparing it to on-line OSCAR system information to ensure the dates we 

used corresponded to routine certification surveys, rather than complaint investigations or 

other types of surveys. We are satisfied that our information is as accurate as HCFA’s 

OSCAR system. 

Additionally, we selected 4 States (California, Kansas, Minnesota, and Texas) that 

contain over 50 nonaccredited hospitals and represented different geographic regions of 

the United States. We then examined the OSCAR data for those States in greater detail. 

We verified the operational status of the nonaccredited hospitals in those States that had 

not been surveyed in over 5 years using the American Hospital Association’s 1997 

Hospital Guide. 

We also interviewed staff and managers at each HCFA regional office and the 

central office. We reviewed a variety of HCFA documents, including budget call letters, 

reinvention materials, and reports to Congress, among others. 

The State Survey and Certification Agencies 

In August 1997, we mailed a pretested survey to the hospital certification agencies 

in the 50 States and the District of Columbia. The response rate for this survey was 100 

percent. The State survey addressed four areas of hospital quality oversight: private 

accreditation of hospitals, Medicare certification of hospitals, HCFA oversight of State 

certification agencies, and State licensure of hospitals. We interviewed State officials on 

the telephone or in person as well. 

Accrediting Organizations 

We interviewed officials from both the Joint Commission on Accreditation of 

Healthcare Organizations and the American Osteopathic Association. We also reviewed 

Hospital Quality: The Role of Accreditation 31 OEI-01-97-00051 
APPENDIX B 

documents such as mission statements, accreditation manuals, policies, and hospital survey 

files. We requested and received aggregate data from the Joint Commission reflecting its 

recent hospital survey activity. All Joint Commission data are presented as reported by 

the Joint Commission, unless otherwise noted. For purposes of this inquiry, we focused 

our analysis on the Joint Commission. 

Survey Observations 

Based on schedules of upcoming triennial surveys, we identified nine hospitals in 

which to observe triennial Joint Commission surveys. Of those, we were able to observe 

seven. In two cases, the hospitals declined the Joint Commission’s request that we be 

allowed to observe. The 7 hospitals varied in size from 80 to 775 beds, represented both 

teaching and community hospitals, and were located in different areas of the country (both 

rural and urban). We also observed one random unannounced Joint Commission survey. 

Although we observed different teams of surveyors, the survey agenda, lines of inquiry, 

and tone were consistent across the surveys, which were conducted in accordance with 

Joint Commission policy, based on review of survey manuals and interviews with 

representatives of the Joint Commission. Finally, we observed a certification survey and 

parts of two complaint investigations performed by State surveyors under HCFA’s 

auspices. 

Stakeholder Interviews 

We interviewed representatives of organizations we considered to be stakeholders 

in hospital oversight. These stakeholder organizations included a union, professional 

organizations, hospital associations, and consumer groups. 

Other Documents 

In addition to the documents referenced above, we reviewed statutory and 

regulatory language and a variety of articles from newspapers, journals, magazines, and 

newsletters. 

Hospital Quality: The Role of Accreditation 32 OEI-01-97-00051 
APPENDIX C 

Sample Accreditation Survey Agenda

We took the following sample agenda from the Joint Commission’s web page 

(http://www.jcaho.org/acr_info ). This agenda is representative of the agendas surveyors 

followed when we observed accreditation surveys. 

Sample Generic Survey Agenda: Three-Day Survey/Three Surveyors 

In most instances, the survey team is composed of an administrator, nurse, and physician. 

In smaller hospitals, the survey team includes two surveyors, usually a nurse and a 

physician. 

Day One: 

All Surveyors: 

8:30-8:45 am Opening Conference 

8:45-9:15 am Performance Improvement Overview Presentation 

9:15-11:00 am Document Review Session 

11:00-11:45 am Leadership Interview 

11:45-12:15 pm Lunch 

Physician Surveyor: 

12:30-2:00 pm Patient Care Setting Visit (Includes inpatient units and other sites where

care is provided, including ambulatory settings and anesthetizing locations.)

2:00-3:00 pm Anesthetizing Location Visit (Includes inpatient units and other sites where

care is provided, including ambulatory settings and anesthetizing locations.)

3:00-4:00 pm Emergency Services Visit 

Nurse Surveyor: 

12:30-2:00 pm Patient Care Setting Visit (Includes inpatient units and other sites where

care is provided, including ambulatory settings and anesthetizing locations.)

2:00-3:00 pm Anesthetizing Location Visit (Includes inpatient units and other site where

care is provided, including ambulatory settings and anesthetizing locations.)

3:00-4:00 pm Patient Care Setting Visit (Includes inpatient units and other sites where

care is provided, including ambulatory settings and anesthetizing locations.)

Administrator Surveyor: 

12:30-1:15 pm Rehabilitation Services Visit 

Hospital Quality: The Role of Accreditation 33 OEI-01-97-00051 
APPENDIX C 

1:15-4:00 pm Building Tour (Includes visits to admitting, kitchen, resource center, 

storage, central supply, and laundry, if applicable.) 

All Surveyors: 

4:00-4:30 pm Survey Team Meeting 

Day Two: 

All Surveyors: 

8:15-8:45 am Daily Briefing 

Physician Surveyor: 

8:45-9:45 am Pathology and Clinical Laboratory Services Visit (Includes survey of blood

and blood components.) 

9:45-10:45 am Medical Staff Credentials Interview 

10:45-11:45 am Medical Staff Leadership Interview 

11:45 am-12:45 pm Lunch: Medical Staff Conference (Optional) 

Nurse Surveyor: 

8:45-10:00 am Patient Care Setting Visit (Includes inpatient units and other sites where

care is provided, including ambulatory settings, and anesthetizing locations.)

10:00-10:45 am Patient Care Setting Visit (Includes inpatient units and other sites where

care is provided, including ambulatory settings, and anesthetizing locations.) 

10:45-11:30 am Infection Control Interview 

11:30 am-12:15 pm Nursing Leadership Interview 

Administrator Surveyor: 

8:45-9:30 am Patient Care Setting Visit (Includes inpatient units and other sites where 

care is provided, including ambulatory settings, and anesthetizing locations.) 

9:30-10:15 am Hospital Department Directors Interview 

10:15 am-12:15 pm Review of Environment of Care Documents 

Nurse and Administrator Surveyors: 

12:15-12:45 pm Lunch 

Administrator Surveyor : 

1:00-1:45 pm Chief Executive Officer/Strategic Planning and Resource Allocation

Interview (Includes Ethics Interview issues.)

1:45-2:15 pm Pharmacy Services Visit 

Hospital Quality: The Role of Accreditation 34 OEI-01-97-00051 
APPENDIX C 

Physician and Nurse Surveyors: 

1:00-2:00 pm Medical Record Interview 

Physician Surveyor: 

2:00-2:45 pm Patient Care Setting Visit (Includes inpatient units and other sites where 

care is provided, including ambulatory settings, and anesthetizing locations.) 

2:45-4:00 pm Imaging Services Interview 

Nurse and Administrator Surveyors: 

3:00-4:00 pm Human Resources Interview 

All Surveyors: 

4:00-4:30 pm Survey Team Meeting 

Day Three: 

All Surveyors: 

8:15-8:45 am Daily Briefing 

8:45-9:45 am Patient Care Interview 

Physician Surveyor: 

9:45-10:30 am Performance Improvement Team Interview 

Nurse Surveyor: 

9:45-10:30 am Performance Improvement Team Interview 

Administrator Surveyor: 

10:00-11:00 am Information Management Interview 

All Surveyors: 

10:30-11:30 pm Performance Improvement Coordinating Group Interview 

11:30-12:00 pm Lunch 

12:00-3:00 pm Team Meeting to Integrate Survey Findings 

3:15-4:15 pm Leadership Exit Conference 

Hospital Quality: The Role of Accreditation 35 OEI-01-97-00051 
APPENDIX D 

Advertising Hospital Accreditation Status

During the course of our inquiry, we saw a range of advertisements extolling 

hospital’s accreditation status.1 We saw advertisements in newspapers, magazines, fundraising materials, yellowpages, and on billboards and the Internet. We also saw a banner 

strung prominently outside a hospital. Most touted the hospital’s achievement of 

accreditation with commendation. For example, one advertisement from a web page 

included the following: 

Accreditation is the way hospitals are judged. Commendation is how they are 

judged superior...We’re proud to be recognized as one of the best hospitals...in 

the country...Our accreditation with commendation puts [us] in the upper 10% of 

all hospitals surveyed in the past 3 years. 

Another ad, this one from a magazine, pitched a hospital chain and its accreditation as 

follows: 

As for quality, the Joint Commission on the Accreditation of Healthcare 

organizations, an independent group that reviews staffing, safety and patient care 

at hospitals around the country, recognizes [our] hospitals with its highest 

ratings: Accreditation with Commendation, at a rate three times the national 

average. 

Yet another, from a newspaper noted: 

Accreditation with Commendation. It’s the highest level of achievement awarded 

by the Joint Commission on the Accreditation of Healthcare Organizations. And 

a distinction earned by only 4 percent of hospitals nationwide. 

1

The Joint Commission issues guidelines on how hospitals can characterize their 

accreditation status in such ads and considers false or misleading advertising of accreditation as 

grounds for denying accreditation (Joint Commission on Accreditation of Healthcare 

Organizations, Comprehensive Accreditation Manual for Hospitals, 1997, p AC-18). 

Hospital Quality: The Role of Accreditation 36 OEI-01-97-00051 
APPENDIX E 

Endnotes

1. Troyen A. Brennan et al, “Incidence of Adverse Events and Negligence in Hospitalized 

Patients,” The New England Journal of Medicine 324 (February 7, 1991) 6: 370-376. 

2. The 17.7 percent refers to adverse events considered by the authors to be serious. The authors 

defined adverse events as “situations in which an inappropriate decision was made when, at the 

time, an appropriate alternative could have been chosen” and serious as ranging from “temporary 

disability to death.” See Lori B. Andrews et al, “An Alternative Strategy for Studying Adverse 

Events in Medical Care,” The Lancet 349 (February 1, 1997)309-313. 

See also Lucian L. Leape, “Error in Medicine,” Journal of the American Medical 

Association 272 (December 21, 1994) 23: 1851-1857; David C. Classes et al, “Adverse Drug 

Events in Hospitalized Patients,” Journal of the American Medical Association 277 (January 

22/29, 1997) 4: 301-306; Mark R. Chassin et al, “The Urgent Need to Improve Health Care 

Quality: Institute of Medicine National Roundtable on Health Care Quality,” Journal of the 

American Medical Association 280 (September 16, 1998) 11: 1000-1005; David W. Bates et 

al,“Effect of Computerized Physician Order Entry and a Team Intervention on Prevention of 

Serious Medication Errors,” Journal of the American Medical Association 280 (October 21, 

1998) 1311-1316; Robert A. Raschke et al, “A Computer Alert System to prevent Injury from 

Adverse Drug Events: Development and Evaluation in a Community Teaching Hospital,” Journal 

of the American Medical Association 280 (October 21, 1998) 1317-1320; and David C. Classen, 

“Clinical Decision Support Systems to Improve Clinical Practice and Quality of Care,” Journal of 

the American Medical Association 280 (October 21, 1998) 1360-1361. 

3. One example is a November 1998 New York Times article under the headline: “Death in 

Surgery Reveals Troubled Practice and Lax Hospital.” The article described a “botched” 

operation on a young woman by a surgeon who was on probation by the State medical board and 

who used unauthorized medical equipment brought in to the operating room by a medical supply 

salesman. Such incidents can happen even in the best of hospitals, but they underscore the point 

that hospitals can be dangerous places and that oversight systems can be lax. See also 

“Overdoses Still Weigh heavily at Dana Farber,” The Boston Globe (26 December 1995); “Florida 

Doctor Sanctioned in New Amputation,” The Boston Globe (19 July 1995); “Two Surgeons 

Surrender Licenses After Mistakenly Removing Kidney,” The Boston Globe (6 June 1996); “How 

Can We Save the Next Victim?” New York Times Magazine (15 June 1997); “Another Hospital 

Death Probed,” The Boston Globe (26 July 1997); “Patient Dies After Drinking Poison Left on 

Nightstand,” San Diego Union-Tribune (6 March 1998); ”Man Arrested for Posing as Doctor for 

4 years,” posted at the CNN interactive webpage (16 May 1998); “Deadly Restraint: Patients 

Suffer in a System Without Oversight,” The Hartford Courant (13 October 1998). 

Hospital Quality: The Role of Accreditation 37 OEI-01-97-00051 
APPENDIX E 

4. http://WWW.hcfa.gov/Medicare/Medicare.htm 

5. Ibid. 

6. Ibid. 

7. P.L. 89-97. 

8. The HCFA published its proposed hospital conditions of participation on December 19, 1997 

(62 Fed. Reg. 66,726). On July 2, 1999, it published an interim final rule concerning patients’ 

rights (64 Fed. Reg. 36070). 

9. Social Security Act, sec. 1865, 42 U.S.C 1395bb. 

10. Health Care Financing Administration, OSCAR 10 Report, as of May 30, 1998. 

11. Michael S. Hamm, The Fundamentals of Accreditation (Washington, D.C.: American Society 

of Association Executives, 1997) 3. 

12. Ibid. 

13. Nonaccredited hospitals can go through the Medicare certification process for free. 

14. Although we conducted some interviews with staff from the American Osteopathic 

Association, another hospital accrediting body, we decided to exclude it from our inquiry. Our 

decision was based on our own time and resource constraints. We may examine accreditation 

through the American Osteopathic Association in a future inspection. 

15. In some cases, the surveyors will identify a type of record (e.g., a patient with recent restraint 

orders or a personnel file of a respiratory therapist) or will request that a particular staff person’s 

personnel file be pulled. 

16. The building tour and document review sessions take longer. The unscheduled time built into 

the structure is generally in 15-minute blocks. We observed surveyors often use that time for 

telephone calls, further document review, or as a buffer in case other sessions ran over. 

17. Indeed, even the likely focus of the surveys or surveyors is available to those hospitals looking 

for it. For example, hospitals can purchase individual surveyor profiles through an internet site or 

visit chat rooms to exchange details on recent surveys. One surveyor told us he casually 

mentioned liking chocolate chip cookies at one hospital, and was thereafter inundated with them-­

an occurrence he attributed to these web sites and chat rooms. Hospitals can also subscribe to 

Hospital Quality: The Role of Accreditation 38 OEI-01-97-00051 
APPENDIX E 

newsletters that track the latest trends and focus of Joint Commission surveys. 

18. Surveyor notes to the Joint Commission found in the Joint Commission files. 

19. The Joint Commission changed its standards as part of their “Agenda for Change”, an 

initiative calling for a more performance-based accreditation process which was unveiled in 1986. 

20. Dennis O’Leary, “ President’s Column: On Becoming Ten,” Joint Commission Perspectives, 

16 (May/June 1996) 3: 2-3. 

21. “The accreditation process itself is fundamentally a risk-reduction exercise. We don’t 

warrant outcomes or that nothing bad will ever happen. We’d be crazy to do that because you 

can never take the level of risk down to zero” (Dennis O’Leary, quoted in Setting Standards for 

Hospitals: An Interview with Dennis O’Leary in The Long Term View 3 (4) 23). 

22. In 1995, the Joint Commission launched an experiment in a more continuous accreditation 

process, called the Orion Project. Operating in four States (Arizona, Pennsylvania, Georgia, and 

Tennessee), the Orion Project involves partial surveys conducted quarterly by the same surveyors 

for participating hospitals. 

23. In early 1997, the Executive Committee of the Board of Commissioners approved an interim 

cap on the scores of three standards related to restraints. 

24. Data as reported by the Joint Commission. 

25. The most frequently cited deficiencies are often identified in newsletters such as “Inside the 

Joint Commission” and “Joint Commission Perspectives.” 

26. http://jcaho.org. 

27. Continuous Quality Improvement is a method of performance improvement involving a 

continuous loop of performance measurement, intervention, and remeasurement. It aims to 

improve the majority rather than marginal performers. D. E. Berwick, “Continuous Quality 

Improvement as an Ideal in Health Care,” New England Journal of Medicine 320 (January 5, 

1989) 1: 53-56; The Institute of Medicine, Medicare: A Strategy for Quality Assurance 

(Washington, D.C.: National Academy Press, 1990). 

28. Joint Commission on Accreditation of Healthcare Organizations, “Understanding the 1997 

Hospital Performance Report,” 4. 

Hospital Quality: The Role of Accreditation 39 OEI-01-97-00051 
APPENDIX E 

29. In response to the unexpected deaths, the Joint Commission conducted a special announced 

survey lasting one day, during which the hospital presented its analysis of the cause and the 

surveyor identified four deficiencies regarding assessing patients and assessing competence. 

30. Joint Commission on Accreditation of Healthcare Organizations, “Understanding the 1997 

Hospital Performance Report,” 10, and “1997 Performance Report,” ii. 

31. According to data reported by the Joint Commission, the most commonly identified 

deficiencies in credentialing between May 1995 and June 1997 were related to the inclusion of 

pending or successful challenges to licensure, relinquishment of licensure, criteria for autopsies, 

and primary verification of license, experience, training, and competencies. 

While we found no examples of surveyor probing to ensure that hospitals deal fully and 

responsibly with marginal performers, we did observe one medical leadership session where the 

physician surveyor gave a stern message about the significance of credentialing and privileging 

decisions. He said that the standard that physician reviewers should use in making such decisions 

about their colleagues was “The Mom Test.” “Would you feel comfortable,” he asked, “in having 

the colleague care for your mother?” Someday, he warned the assembled physicians, “you will be 

walking or taken through hospital doors as a patient and you will be dependent on the 

conscientiousness and skill of the medical staff at that hospital. Think about it.” 

32. In a 1995 Office of Inspector General report, we found that 75 percent of the hospitals in the 

country never reported an adverse action to the Data Bank between September 1, 1990 and 

December 31, 1993 (Office of Inspector General, Hospital Reporting to the National Practitioner 

Data Bank, February 1995). 

33. In December 1994, the Joint Commission began releasing accreditation information about 

hospitals, on request, in the form of performance reports. These reports include the hospital’s 

overall survey score from the last triennial survey, the hospital’s scores in the 45 performance 

areas surveyed, comparative data on the percentage of hospitals receiving each possible score in 

the 45 performance areas, and a list of the performance areas in which the hospital received 

deficiencies. In November 1996, the Joint Commission waived the $30 report fee. 

As of late 1998, 13,408 performance reports for hospitals, nursing homes, and other 

provider types were available, and that list keeps growing. Between late 1996 and early 1997, 

requests for performance reports grew from 13 per quarter to over 2,000. Beginning in early 

1998, the Joint Commission made the reports available on its website, and since then requests for 

reports have tapered off, suggesting some are accessing the information directly through the 

website rather than ordering reports. 

Hospital Quality: The Role of Accreditation 40 OEI-01-97-00051 
APPENDIX E 

34. The Joint Commission allows hospitals to be accredited as single entities or, if they are a part 

of a multi-hospital system, to be accredited through a system survey. The multi-hospital option 

includes a corporate orientation and/or a corporate summation in addition to a consecutive survey 

of the system’s hospitals with the same survey team. The Joint Commission charges an extra fee 

for this option. The Joint Commission also offers network accreditation, so hospitals that are part 

of a particular health care network could also be surveyed under the auspices of a network 

survey--in addition to their own hospital survey. The Joint Commission could award a different 

accreditation status to the hospital than to the network, making it difficult for consumers to 

discern the meaning of each. 

35. Some Joint Commission scoring guidelines are based on algorithms 

36. The levels of accreditation are: accreditation with commendation, accreditation without type 

1 recommendations, accreditation with type 1 recommendations, conditional accreditation, and 

preliminary nonaccreditation. Preliminary nonaccreditation is generally considered to be a 

temporary status. 

From June 1995 through May 1998, the Joint Commission awarded accreditation with 

commendation to hospitals surveyed as a part of a hospital system much more often (22 percent) 

as hospitals surveyed as single entities (13 percent). Achieving any accreditation level qualifies a 

hospital to participate in Medicare. By contrast, the National Committee for Quality Assurance, 

which accredits health plans, has used four accreditation levels. Eleven percent of plans met its 

lowest, or provisional level, and 11 percent were denied accreditation. Unlike Joint Commission 

hospital accreditation, Congress has not deemed the National Committee for Quality Assurance 

(NCQA) accreditation for purposes of participating in Medicare. We extracted the data on 

NCQA accreditation from its web page, “NCQA Accreditation Status List: A Fact Sheet” 

(http://www.ncqa.org). 

37. When the Joint Commission identifies a Type 1 recommendation in a hospital, that hospital 

must either submit a written progress report and/or be subject to a focused survey. Written 

progress reports far exceed focused surveys as a form of follow-up: between June 1995 and May 

1998, the Joint Commission called for 3,264 written progress reports and 417 focused surveys. 

Between June 1995 and May 1997, the Joint Commission rejected about 2 percent of written 

progress reports for triennial surveys and required an additional follow-up, usually an on-site 

survey, for those hospitals. Of course, the Joint Commission can call for multiple written 

progress reports and focused surveys from a single hospital, depending on the particular situation. 

Hospitals accredited with commendation are subject to no such follow-up, but could be selected 

for a random, unannounced survey. 

Hospital Quality: The Role of Accreditation 41 OEI-01-97-00051 
APPENDIX E 

38. The Joint Commission allows hospitals the opportunity to improve their scores by correcting 

problems identified in the survey (also referred to as type 1 recommendations). It conducts two 

kinds of follow-up to ensure the hospitals have made the appropriate corrections: written 

progress reports and focused surveys. 

About a fourth of all hospitals’ scores improved on average 1 point after they addressed 

their deficiencies; the others remained the same or went down. From June 1995 to May 1996, 

hospital scores averaged 91 (for multi-hospital option) and 90 (for single hospital option). From 

June 1996 to May 1997, hospital scores averaged 93 (for multi-hospital option) and 91 (for single 

hospital option). 

39. Joint Commission on Accreditation of Healthcare Organizations, 1997 Performance Report, 

p. 3. 

40. In addition, the Joint Commission uses written progress reports and focused surveys both to 

ensure hospitals address deficiencies and as a way to monitor continued compliance. Written 

progress reports and focused surveys are considered follow-up actions to Joint Commission 

surveys. It also uses unannounced and unscheduled surveys to respond to concerns about patient 

safety in a hospital, which are discussed in the section on “Responding to Complaints.” 

41. “Raid of Hospitals Probes Overworked Doctors,” The Wall Street Journal (11 March 1998); 

“Inspectors Pay Surprise Visits to 12 New York Hospitals,” The New York Times (12 May 1998). 

42. State agencies do, however, give notice to hospitals prior to conducting validation and 

certification surveys. 

43. 42 USC 1819(g)(2). 

44. David R. Morrow, “To Rate Hospitals, She Dons a Wig and Practices Her Cough,” New 

York Times, Sunday, 30 March 1997. 

45. Following are some illustrations of newspaper reports on serious incidents involving patient 

harm: “Florida Doctor Errs Anew, Taking Toe Without Asking,” The New York Times, July 19, 

1995; Lawrence K. Altman, “State Issues Scathing Report on Error at Sloan-Kettering,” The 

New York Times, November 16, 1995; Richard Saltus, “Patient’s Death Leads to Change at the 

Brigham,” The Boston Globe, February 16, 1996; Richard Saltus, “Two Surgeons Surrender 

Licenses after Mistakenly Removing Kidney,” The Boston Globe, June 6, 1996; and “Death in 

Surgery Reveals Troubled Practice and Lax Hospital,” The New York Times (15 November 1998). 

Hospital Quality: The Role of Accreditation 42 OEI-01-97-00051 
APPENDIX E 

46. The Commission typically conducted these investigations on its own, independent of any 

investigations being conducted by State and/or HCFA officials. 

47. The Joint Commission refers to this as a “root-cause analysis.” 

48. For articles supporting and or explaining this approach to medical errors, see Lucian L. 

Leape, “Error in Medicine,” The Journal of the American Medical Association 272 (December 

21, 1994) 23:1851-57; Lucian L. Leape, et al, “Systems Analysis of Adverse Drug Events,” The 

Journal of the American Medical Association 274 (July 5, 1995) 1:35-43; David W. Bates, 

“Incidence of Adverse Drug Events and Potential Adverse Drug Events,” The Journal of the 

American Medical Association, 274 (July 5, 1995) 1:29-34; Lawrence K. Altman, “Medical 

Errors Bring Calls for Change,” The New York Times, July 18, 1995; Linda O. Praeger, 

“Reducing Medical Errors,” American Medical News 39 (November 4, 1996) 41; Denise Grady, 

“Doctors Urged to Admit Mistakes,” The New York Times, December 9, 1997; and Lucian L. 

Leape et al, “Promoting Patient Safety by Preventing Medical Error,” The Journal of the 

American Medical Association 280 (October 28, 1998) 16: 1444-1447. 

49. The Joint Commission’s policy calls for a hospital voluntarily to notify it of a reportable 

major adverse event within 5 business days and subsequently to conduct an acceptable analysis of 

the event’s cause within 30 days. 

50. In this respect, the Joint Commission indicates that its approach is similar to the aviation 

accident and incident reporting system of the National Transportation Safety Board. See 

http://nasdac.faa.gov/safety_data/. 

51. During this period, the Joint Commission has drawn on its sentinel event database to issue 

four alerts with specific risk reduction strategies for hospitals. The first, issued on February 27, 

1998, focused on preventing medication errors and involved the misadministration of potassium 

chloride. The second, issued August 28, 1998, was on wrong-site surgery. The third and fourth 

were both issued in November 1998 and dealt with preventing inpatient suicides and restraint 

deaths. (See Joint Commission on Accreditation of Healthcare Organizations, Sentinel Event 

Alert, February 27, 1998; August 28, 1998; November 6, 1998; and November 18, 1998.) 

52. “Joint Commission Statement on the Revised Sentinel Event Policy,” February 17, 1998. 

53. Further, if the public recognizes that the Joint Commission rarely changes a hospital’s 

accreditation status as a result of a adverse event, the public is likely to increasingly view the Joint 

Commission as being soft on hospitals at the expense of public safety. The Joint Commission 

itself is aware of this danger. In a column in its own newsletter, the Joint Commission’s president 

noted that while hospitals tend to regard such events as their own business, “those speaking on 

Hospital Quality: The Role of Accreditation 43 OEI-01-97-00051 
APPENDIX E 

behalf of the public see sentinel event identification and follow-up as central to the Joint 

Commission’s role, and credibility, as an accrediting body.” For hospitals wary even of the 

gentler Joint Commission policy toward such events, he cautioned, “[I]n the end, the Joint 

Commission may be the one constructive force standing between the organization that has 

experienced a sentinel event and society’s handmaidens of punishment.” He did not identify the 

latter. See Dennis S. O’Leary, “President’s Column: The Sentinel Event Policy: A Work in 

Progress,” Joint Commission Perspectives, November/December 1996, pp. 2-3. 

54. Despite occasional newspaper articles, how aware consumers are of the Joint Commission 

remains unknown (“Before You Go Into the Hospital,” Parade Magazine, 4 May 1997, 6-8). 

Consumers may be more aware of their State governments or HCFA as places to file complaints. 

When we asked stakeholders the extent to which they expected accreditation to play a role in 

responding to complaints, their responses were mixed: about a third (34 percent) expected no or a 

minor role; 18 percent, a moderate role and; nearly half (48 percent), at least a great role. 

55. Reflects data as reported by the Joint Commission. However, based on our file review, the 

public information interviews may be undercounted to some extent. We reviewed the Joint 

Commission’s hospital files for the seven surveys we observed, of which four had public 

information interviews. We found no record of the four public information interviews we 

observed in the files. 

56. The most recent data available from the Joint Commission indicates that surveyors conducted 

31 public information interviews during the first 6 months of 1998, involving about 4 percent of 

surveys conducted during that time. Five of the 31 (16 percent) resulted in type 1 

recommendations. 

57. In 1996, for example, it reported receiving 2,718 complaints (43 percent by mail and 

57 percent by telephone), a 3.5 percent decrease from those it received in 1995. Twenty-seven 

percent of the written complaints were either unrelated to an accredited hospital or unintelligible, 

thus prompting no further action from the Joint Commission. But the Joint Commission tracked 

30 percent of the written complaints for trends and sent 27 percent to surveyors for triennial or 

focused surveys. 

58. Joint Commission on Accreditation of Healthcare Organizations, March 3, 1999. The Joint 

Commission also accepts complaints by electronic mail, fax, and mail. 

59. In the 3-year period from June 1995 through May 1998, 17 percent of the unscheduled and 

unannounced surveys resulted in a change in the hospital’s accreditation status and 50 percent 

resulted in some follow-up (involving focused surveys and/or written progress reports). 

Hospital Quality: The Role of Accreditation 44 OEI-01-97-00051 
APPENDIX E 

60. On its webpage (http://www.jcaho.org), the Joint Commission identifies the following ways 

that integrating performance measures will “enhance the value of accreditation: 

C increase public and private sector reliance on Joint Commission accreditation, 

thereby building support for the consensus development and use of standards and 

performance measures, as well as reducing justification for duplicative evaluation 

activities; 

C provide the Joint Commission with performance data that it can use to 

C monitor organization performance on a continuous basis, 

C help organizations identify issues that require attention, and 

C focus triennial surveys on areas within an organization that represent the 

greatest potential opportunities for improvement; 

C help identify both exemplary performance and best practices to facilitate the 

provision of benchmarking services to health care organizations; and 

C facilitate refinement of the Joint Commission’s performance-based standards and 

the analysis and understanding of the relationships between standards and 

outcomes.” 

61. Christina Kent, “‘Fix it or forget it’: The JCAHO in Crisis,” Medicine & Health (January 23, 

1995). 

62. The Joint Commission regularly reviews performance measurement systems, so new systems 

found to meet its minimum requirements can be added to the list. The Joint Commission also can 

remove approved systems. (http://www.jcaho.org/perfmeas/). 

63. On March 15, 1999, the Joint Commission invited accredited institutions, physicians, 

universities, consumer groups, and researchers, among others, to identify and prioritize potential 

areas for it to develop core measures. 

64. American Hospital Association to Joint Commission on Accreditation of Healthcare 

Organizations, January 5, 1999. 

65. Ibid. 

66. J. Duncan Moore, Jr., “Standardizing Standards: Accrediting agencies form council to end 

confusion,” Modern Healthcare (May 25, 1998); “Three for Quality,” American Medical News 

(June 15-22, 1998); “Three Accreditation Groups to Collaborate on Standard Approach to 

Quality Measures,” BNA’s Managed Care Reporter (May 27, 1998). 

Hospital Quality: The Role of Accreditation 45 OEI-01-97-00051