Department of Health and Human Services 

OFFICE OF 

INSPECTOR GENERAL 

JUNE GIBBS BROWN 

Inspector General 

JANUARY 2000 

OEI-01-97-00054 

The External Review of Hospital Quality 

State Initiatives 
OFFICE OF INSPECTOR GENERAL 

The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, is to 

protect the integrity of the Department of Health and Human Services programs as well as the health 

and welfare of beneficiaries served by them. This statutory mission is carried out through a nationwide 

program of audits, investigations, inspections, sanctions, and fraud alerts. The Inspector General 

informs the Secretary of program and management problems and recommends legislative, regulatory, 

and operational approaches to correct them. 

Office of Evaluation and Inspections 

The Office of Evaluation and Inspections (OEI) is one of several components of the Office of Inspector 

General. It conducts short-term management and program evaluations (called inspections) that focus 

on issues of concern to the Department, the Congress, and the public. The inspection reports provide 

findings and recommendations on the efficiency, vulnerability, and effectiveness of departmental 

programs. 

OEI's Boston Regional Office prepared this report under the direction of Mark R. Yessian, Ph.D., 

Regional Inspector General. Principal OEI staff included: 

REGION HEADQUARTERS 

Joyce M. Greenleaf, Project Leader Jennifer Antico, Program Specialist

Maria E. Maddaloni

Laura C. McBride

To obtain copies of this report, please call the Boston Regional Office at (617)565-1050. 

Reports are also available on the World Wide Web at our home page address: 

http://www.dhhs.gov/progorg/oei 
EXECUTIVE SUMM ARY 

Purpose 

To describe State government initiatives concerning the external review of hospital quality. 

Background 

State Initiatives Aimed at Hospital Oversight 

State agencies play important roles in overseeing hospitals, either as agents of the Health Care 

Financing Administration (HCFA), or under their own authorities to license such facilities and 

protect the public. Indeed, many States have developed and implemented initiatives aimed at 

addressing the quality of care in hospitals. This report presents a snapshot of six such initiatives 

that appear promising and could be instructive not only to other States but also to the Federal 

government. We present the initiatives in three categories: standardized performance 

measures, on-site surveys, and public disclosure of information on hospital performance. 

This report is a follow-up to our recent series of reports that assessed the roles of HCFA and 

the Joint Commission in overseeing hospitals that participate in Medicare. In that series, we 

directed recommendations for improvement to HCFA. 

Our information comes from discussions with the pertinent State officials and reviews of 

relevant documents. 

Standardized Performance Measures 

These are quantitative indicators that enable regulators, purchasers, and consumers to compare 

hospital performance to itself over time or to other hospitals. They can provide insights into a 

hospital’s performance, foster improvements, and identify outliers. 

New York: Using Mortality Data to Measure Hospital Performance 

New York collects and publishes mortality data on coronary artery bypass graft surgery 

(CABG) and other procedures, fulfilling both quality improvement and regulatory functions. 

Hospital Quality: State Initiatives 

1 OEI-01-97-00054 
Pennsylvania: Creating Performance Report Cards 

Pennsylvania analyzes inpatient data from every hospital in the State to create reports that 

evaluate hospitals on quality-of-care measures such as length of stay, charges, and admission 

rates for CABG, breast cancer, and diabetes, among others. In 1998, it documented a 22 

percent drop in in-hospital mortality for CABG from 1991 to 1995. 

On-site Surveys 

On-site surveys are a traditional way to assess directly a hospital’s compliance with Federal, 

State, and local requirements. Many States, however, largely rely on the Joint Commission on 

Accreditation of Healthcare Organizations or surveys funded by HCFA for an on-site presence 

in their hospitals. 

Utah: Observing Accreditation Surveys 

Utah relies on surveys by the Joint Commission to determine compliance with its hospital 

licensure requirements. However, Health Department officials participate in the summary 

session at the end of the Joint Commission’s on-site survey. The Department looks to the 

findings of the Joint Commission in determining whether to pursue its own enforcement actions. 

New York: Surprise Inspections of Residency Programs 

In 1998, New York launched 12 simultaneous surprise inspections of residency programs at 

teaching hospitals across the State. New York regulates resident working conditions, and these 

surprise inspections marked the State’s first effort to ensure compliance. 

Public Disclosure through the Internet 

The Internet provides enormous opportunity for sharing performance information quickly and 

broadly. It can spur hospital improvements and reassure the public that an external review 

process is protecting its safety. 

New Jersey: Listing Hospital Enforcement Actions 

New Jersey’s website details resolved hospital enforcement actions, such as fines or other 

penalties imposed for violating licensure or certification regulations. The website also includes 

information about the State’s inspection, licensure, and complaint processes. It updates the site 

quarterly. 

Hospital Quality: State Initiatives 

2 OEI-01-97-00054 
Colorado: Posting Compliance Summaries 

Colorado posts hospitals’ compliance histories on its website. In addition to basic hospital 

information, the website includes a summary of all complaints and serious events reported to the 

State since January 1999. The summaries describe the reported incident, what actions the 

facility took in response, and the Department’s follow-up actions. 

Conclusion 

The State initiatives presented in this report show that States can draw on their own authorities 

and resources to add a measure of public protection not provided by either HCFA or the Joint 

Commission. The States have advantages, such as simply being closer to the action, that 

national reviewers would be hardpressed to match. And these advantages help States 

contribute a valuable complement to the existing, national approaches to external hospital 

review. 

State initiatives can also serve as important catalysts for continued national efforts aimed at 

improved hospital oversight. Indeed, the States’ experiences can be instructive to HCFA, the 

Joint Commission, and other States. The initiatives highlighted herein reinforce themes of 

balance and accountability that we promoted in our prior inquiry, which assessed the roles of 

HCFA and the Joint Commission. The States’ experiences with performance measures, 

surprise inspections, and public disclosure demonstrate that such efforts can be both feasible 

and constructive. 

Hospital Quality: State Initiatives 

3 OEI-01-97-00054 
TABLE OF CONTENTS 

EXECUTIVE SUMMARY ............................................... 1

INTRODUCTION ..................................................... 5

Standardized Performance Measures .................................... 7

New York: Using Mortality Data ..................................... 8

Pennsylvania: Creating Report Cards ................................ 9

Conducting On-site Surveys .......................................... 12

Utah: Observing Accreditation Surveys .............................. 13

New York: Surprise Hospital Inspections ............................. 13

Public Disclosure through the Internet .................................. 16

New Jersey: Listing Hospital Enforcement actions ..................... 17

Colorado: Posting Compliance Summaries ........................... 18

CONCLUSION ...................................................... 21

APPENDICES

A: Methodology ................................................ 23

B: Endnotes ................................................... 24

Hospital Quality: State Initiatives 

4 OEI-01-97-00054 
INTRODUCTION 

PURPOSE 

To describe State government initiatives concerning the external review of hospital quality. 

BACKGROUND 

States as HCFA Agents 

State agencies play important roles in helping the Health Care Financing Administration 

(HCFA) ensure that hospitals meet the minimum requirements for participating in the Medicare 

program. Funded by HCFA, they conduct validation surveys of those hospitals accredited by 

the Joint Commission on Accreditation of Healthcare Organizations, determine Medicare 

certification for those hospitals that choose not to be accredited, and respond to complaints and 

adverse events involving hospital care. In a number of recent reports, we addressed how the 

States performed these roles and how HCFA held them (and the Joint Commission) 

accountable for their performance.1 We directed our recommendations for improvement to 

HCFA, which responded with an action plan. 

States Under their Own Authorities 

During the course of our prior work, we learned of initiatives that State governments were 

taking under their own authorities to address the quality of care provided in hospitals. Many of 

these were significant initiatives that could be instructive to the hospital quality review efforts of 

the Federal government and of other States. In this report we offer a snapshot of six such 

initiatives. 

We found each of the initiatives promising enough to warrant wider attention, although we did 

not evaluate them. Our information comes from discussions with pertinent State officials and 

from reviews of relevant documents (see appendix A for more details on our methodology). 

We do not suggest that the initiatives presented here represent a comprehensive listing of quality 

review efforts being undertaken by the States; nor do we suggest that they have necessarily 

been successful in achieving their objectives. 

We present the initiatives in three categories: (1) standardized performance measures, 

(2) on-site surveys, and (3) public disclosure on hospital performance. We begin each by 

discussing the relevance of these categories to the external quality review of hospitals. 

Hospital Quality: State Initiatives 

5 OEI-01-97-00054 
We conducted this inspection in accordance with the Quality Standards for Inspections 

issued by the President’s Council on Integrity and Efficiency. 

Hospital Quality: State Initiatives 

6 OEI-01-97-00054 
Standardized Performance Measures 

Tools for Improving Performance and Enforcing Minimums 

Standardized performance measures are quantitative indicators that enable regulators, 

purchasers, and consumers to compare hospital performance and ensure that patients are 

receiving quality services. The comparisons can focus on a hospital’s own performance over 

time and/or how its performance compares to other hospitals. Examples of performance 

measures include rates of complication from specific procedures and the preventive 

administration of antibiotics prior to surgery. The data can be drawn from sources such as a 

hospital’s records, billing claims, or surveys of patients or providers. 

External reviewers can use performance data in two fundamentally different ways, both of 

which have value. One way is at root collegial: to foster continuous quality improvement. 

External reviewers collect performance data and distribute them with the intent of providing 

hospitals with comparative information they can use, voluntarily, to improve their own 

performance. If particular hospitals find that their performance is significantly poorer than that 

of others, they can search for factors that explain the difference and for changes that will 

improve their performance. 

The other way is more regulatory: to identify hospitals that are performing poorly in relation to 

any of the designated measures and to hold those hospitals accountable for raising their 

performance to acceptable levels. External reviewers encourage the hospitals to improve 

voluntarily, but could also mandate corrective actions and even penalties. 

Realizing the Potential 

The Joint Commission on Accreditation of Healthcare Organizations has been one of the 

leaders in recognizing the potential of standardized performance measures. More than a 

decade ago it set forth a vision for hospital accreditation that would be based largely on datadriven clinical performance indicators. But its progress toward this end has been much slower 

than envisioned because of a number of significant obstacles. Three such obstacles have been 

particularly prominent: (1) the technical difficulties associated with the science of measurement 

and risk management, (2) the costs involved in collecting and distributing data, and (3) the 

political concerns raised by sharing performance data with others. 

Although these obstacles remain imposing, the quest to institutionalize performance 

measurement remains strong. The Joint Commission requires, at a minimum, that accredited 

hospitals use a performance measurement system from its list of approved systems. 

Furthermore, in an effort to foster consistent national standards for performance measurement, 

the Joint Commission recently joined with the American Medical Association and the National 

Hospital Quality: State Initiatives 

7 OEI-01-97-00054 
Committee for Quality Assurance to create the Performance Measurement Council. And at the 

Federal level, HCFA, in the proposed revisions to the Medicare conditions of participation, 

stresses the potential of performance measurement and calls for Medicare-certified hospitals to 

conduct a minimum number of performance improvement projects 

In the presentations below, we address the considerable experience that Pennsylvania and New 

York have had in using discrete hospital performance measures. 

NEW YORK: Using CABG Mortality Data to Measure Hospital 

Performance 

New York’s Department of Health publishes reports on risk-adjusted coronary artery bypass 

graft (CABG) mortality rates. The annual reports present the number of CABG surgeries 

performed, the number of deaths, and 

three types of mortality rates for each 

surgeon and hospital in the State (see Mortality Rates 

box). The Department distributes 

about 2,000 reports per year, which Reports published by the New York

also are available on the Internet. Department of Health present three mortality

rates for each surgeon and hospital 

included: Risk Adjusting and Public Scrutiny 

Observed mortality rate - the number of Development of these measures observed deaths divided by the total number 

began in 1988 at the direction of the of patients who underwent isolated CABG 

Department’s Commissioner as a surgery. 

way to understand the disparities

among the mortality rates from Expected mortality rate - the sum of the

CABGs in the State’s hospitals. A predicted probabilities of death for all

Department statistician developed the patients divided by the total number of

risk-adjustment methodology based patients. 

on risk factors compiled by the Risk-adjusted mortality rate - the best State’s Cardiac Advisory Committee, estimate, based on the statistical model, of a longstanding group of cardiologists what the provider’s mortality rate would have 

and cardiac surgeons that advises the been if the provider had a mix of patients 

Department on cardiac-related identical to the statewide mix. The report also 

matters. These risk factors include presents a confidence interval for the riskage, gender, comorbidities, and adjusted mortality rate. 

previous open heart operations.2 

Source: Coronary Artery Bypass Surgery in New York 

State 1994-1996, New York State Department of Health, 

October 1998, p. 8. 

Hospital Quality: State Initiatives 

8 OEI-01-97-00054 
The Department released the first risk-adjusted CABG mortality rates in 1990 for each 

hospital. An article on the data in the Journal of the American Medical Association noted 

that surgeons performing fewer CABGs had mortality rates that were higher than the surgeons 

who performed a higher volume of CABGs.3 Following this, a Freedom of Information Act 

request and subsequent lawsuit by a New York newspaper led to the release of surgeonspecific data. The Department now analyzes and reports data on individual surgeons who have 

performed 200 procedures over a 3-year period or performed at least 1 procedure in each of 

the 3 years evaluated.4 

Since 1994, the Department has published a similar report presenting angioplasty mortality 

data. Within the next year, it will begin publishing a report with data on heart valve surgery 

mortality rates. 

Quality Improvement and Regulatory Effects 

The publication of the data has spurred both regulatory actions on the part of the State as well 

as improvements in the quality of care provided in the hospitals. Risk-adjusted data enable the 

Department to compare every facility’s outcomes on a level playing field. The Department has 

used the risk-adjusted mortality rates to identify outlier hospitals, and required one such outlier 

to affiliate with an academic institution. It required two other hospitals to close or suspend their 

cardiac services as a result of their performance data. 

Since the Department began collecting and reporting CABG data, the mortality rate from this 

surgery in New York fell from 3.52 per 100 patients in 1989 to 2.44 per 100 patients in 1998.5 

Some hospitals in the State have initiated their own quality improvement programs based on the 

data from the CABG reports.6 For example, one hospital in the State had higher mortality rates 

and concluded that emergency cases were rushed into surgery without being stabilized. That 

hospital then made changes to ensure patients were stabilized prior to surgery, leading to an 

improvement in its mortality rate. In another hospital, the mortality data led to personnel 

changes. That hospital determined it was unable to handle the severe cardiac cases, so it 

referred such cases to other hospitals until it could expand its own ability to handle them. 

PENNSYLVANIA: Creating Performance Report Cards 

Pennsylvania, too, collects and disseminates data on CABG surgery and has done so annually 

since 1993. In 1998, the Pennsylvania Health Care Cost Containment Council (hereafter 

referred to as the Council) released its fifth CABG report, which includes risk-adjusted 

measures of mortality, length of stay, and average charges. The report lists each measure 

individually by hospital, physician, and most recently, by health plan in the State. In addition to 

its annual CABG reports, the Council produces other reports addressing a range of health 

issues such as caesarean-section rates, breast cancer, heart attacks, osteoporosis, drug use, 

Hospital Quality: State initiatives 

9 OEI-01-97-00054 
and diabetes. These issue briefs provide State-wide information on, for example, charges, 

average length of stay, and admission rates. The Council recently reissued for the first time 

since 1994 the “Hospital Effectiveness Report,” which includes data on 15 DRGs.7 For each 

DRG and each hospital, this report highlights risk-adjusted mortality, average length of stay, 

and average charges (see box). 

Source: Pennsylvania Health Care Cost Containment Council, 

A Hospital Performance Report-Region 1, 1999 (99-07/04-11/1), p. 7. 

Pennsylvania began this data initiative in 1986, when, at the urging of State business and labor 

leaders, its legislature created the Health Care Cost Containment Council.8 One of the 

Council’s charges is to contain health care costs by stimulating competition among hospitals. 

By compiling and releasing hospital data on charges and outcomes, the Council aims to provide 

purchasers and consumers with information—thereby promoting more informed decisions and 

choices among health care providers—and thus drive down health care costs. The 21-member 

Council includes representatives from business, organized labor, hospitals, physician groups, 

and insurance companies, as well as consumers. The Secretary of the Department of Health is 

also a Council member, however there is little interaction between the Department and the 

Council.9 

Documented Improvements in Data Indicators 

The 1998 CABG report identified a 22 percent drop in in-hospital mortality for CABG surgery 

from 1991 to 1995 while the number of cases in the State increased. Also, the average charge 

for CABG surgery decreased 3.9 percent.10 Similarly, caesarean-section rates have dropped 

across the State while the rate of vaginal births after a caesarean-section has increased.11 

Many attribute the measured improvements to hospitals’ own use of the data for internal 

improvement projects and to guide staff recruitment efforts.12 There has been some debate on 

the actual impact on consumers of the Council’s data releases. Some argue that the data is 

Hospital Quality: State initiatives 

10 OEI-01-97-00054 
most helpful for hospitals and purchasers, rather than consumers. However, a 1996 survey 

found that 20 percent of the heart-surgery patients surveyed were aware of the CABG report.13 

Data Collection Is Resource Intensive 

These improvements come at a significant cost to hospitals. The Hospital Association estimates

that the software mandated by the State costs Pennsylvania hospitals $10 million and an

additional $40 million for the separate data abstraction. Hospitals can use the software system

for their own efforts but must buy back any data generated by the State. Furthermore,

hospitals are concerned about the validity of the data. They note that certain factors, such as

“Do Not Resuscitate” orders, are not taken into account by the software. As a result, the data

may show a hospital with a significantly higher mortality than is actually true. Despite the

burden of data collection and submission, it appears that hospitals find the available data helpful. 

Hospital Quality: State initiatives 

11 OEI-01-97-00054 
On-site Surveys 

The Core Element of the External Review Process 

Traditionally, the on-site survey of hospitals has been the key element of the external review 

process. Through on-site surveys, the surveyors can assess first-hand how well hospitals are 

meeting established requirements and can offer information and advice for hospitals seeking to 

improve their performance. The surveyors can observe conditions, review records, and 

interview administrators, clinical staff, and patients. 

As part of its accreditation process, the Joint Commission conducts on-site surveys every 3 

years to review hospital compliance with over 500 standards in 45 performance areas. The 

surveys range from 2 to 5 days, depending on the size of a hospital. The Joint Commission 

supplements these surveys with a small number of random, unannounced, and briefer surveys 

intended to assess continued compliance with standards. Occasionally, it will also visit hospitals 

to investigate complaints or adverse events. 

The State agencies also conduct surveys. For accredited hospitals, they perform validation 

surveys and respond to complaints and adverse events under agreement with HCFA. For 

nonaccredited hospitals, they conduct routine on-site surveys, but because of resource 

constraints they use on-site surveys primarily for responding to complaints or adverse events. 

Using Survey Resources Strategically 

On-site surveys serve as a central, resource-intensive part of the external review process. 

Given that, it is especially important that the resources devoted to on-site surveys be used 

strategically. Among the key questions that the Joint Commission, State agencies, and HCFA 

must consider are these: How can surveys best be targeted to problem areas? How often 

should surveys be conducted? What should be the balance between announced and 

unannounced surveys? Between routine surveys and surveys conducted in response to 

complaints and/or adverse events? 

These questions have particular relevance for the Joint Commission because it accredits about 

80 percent of the 6,200 hospitals that participate in Medicare and because 30 States deem 

hospitals accredited by the Joint Commission to meet their licensure requirements. But a 

number of States have taken initiatives that aim to add an on-site presence and a measure of 

patient safety beyond that afforded by the accreditation process. Below, we present Utah’s 

participation in Joint Commission accreditation surveys and New York’s unannounced visits to 

teaching hospitals. 

Hospital Quality: State initiatives 

12 OEI-01-97-00054 
UTAH: Observing Accreditation Surveys 

Utah retains the unique authority to attend a deemed hospital’s accreditation survey and to take

action based on that survey. This authority allows the State to balance scarce resources and

hospital burden while maintaining a prominent presence in hospitals. 

Each year, when hospitals must apply for a license to operate in Utah, they have the option of

initiating, continuing, or relinquishing deemed status. Thirty-two of the State’s 51 hospitals are

accredited and have chosen to be deemed. By opting for deemed status, hospitals forego the

annual licensure surveys conducted by the State, and the State relies instead on the survey

conducted by the accrediting organization (usually the Joint Commission). However, as

outlined in regulation, request for deemed status automatically authorizes officials from the

Department of Health to attend part of the hospital’s accreditation survey. Deemed hospitals

must notify the State of their upcoming accreditation surveys. On the last day of the survey, a

Department official attends the closing conference. Generally, the Department official takes

notes and acts as an observer, but he or she can and will ask questions if necessary. In

addition, Joint Commission surveyors have, on occasion, made themselves available after the

conference for further discussion. After the conference, the official reports back to the

Department on all deficiencies. The Department can then take action based on this report and

its assessment of the accrediting body’s findings. If warranted, the Department can cite a

deficiency under licensure authority while on-site and often will interview hospital officials to find

out more information about the deficiencies. 

Observing Accreditation Surveys Allows for Timely Action 

Utah requires hospitals to submit, as many other States do, the final Joint Commission 

accreditation report to the State. However, the Joint Commission does not issue its final report 

until up to 120 days after the survey, and hospitals then have another 60 days to respond. If 

there were significant or immediate problems at the hospital, the Department would be unaware 

of them until well after the fact. By attending the conference, officials can hear the results firsthand and take immediate action if they feel it is warranted. In 1998, the Department cited three 

hospitals on-the-spot based on attendance at such conferences. The Department’s presence at 

the conferences also allows it to learn about any supplemental deficiencies, which are generally 

excluded from the Joint Commission’s final report. 

NEW YORK: Surprise Inspections of Residency Programs 

In March 1998, New York’s Department of Health conducted surprise inspections of 

residency programs simultaneously in 12 of the State’s teaching hospitals. This was the 

beginning of an effort to oversee hospitals’ compliance with the State’s residency law, known 

Hospital Quality: State Initiatives 

13 OEI-01-97-00054 
as the Bell Regulations. New York is the only State that regulates resident working conditions

in its State hospital code. 

The State conducted the inspections by sending survey teams into the 12 teaching hospitals

simultaneously on a Thursday afternoon. Surveyors then stayed on-site through the weekend. 

Survey teams consisted of Department

field workers, including registered

nurses who conducted medical record

reviews to assess physician supervision 

of residents’ cases. 

The inspections focused on residents’ 

working hours and supervision by 

attending physicians. Surveyors 

assessed compliance with the State’s 

working hour limitations by examining 

posted schedules, then observing 

residents as they went on- and off-shift 

to assess their true working hours. The 

Department used the data collected on 

residents’ working hours to calculate a 

sample work week, which estimated 

residents’ work schedules. To 

determine adequacy of supervision, 

surveyors reviewed medical records for 

evidence that physicians supervised the 

residents’ cases, and tested the on-call 

system by having residents contact their 

respective attending physicians. In 

Bell Regulations 

In 1989, New York established the Bell 

Regulations to cover supervision of residents 

by attending physicians and to limit the hours 

worked by residents in the State’s teaching 

hospitals. The impetus for these regulations 

was the death of a woman in a New York City 

emergency room; a grand jury probe 

subsequently found problems with residency 

training. 

New York City’s public advocate published 

reports in 1994 and 1997 that revealed 

violations of the Bell regulations in New York 

City’s teaching hospitals. The cited violations 

included both insufficient supervision and 

excessive hours worked by residents. In 

1998, State legislators questioned the 

Department’s Commissioner about Bell 

Regulation compliance, which led to the 

surprise inspections. 

addition, surveyors interviewed residents and physicians to learn more about supervision and 

working hours. 

Violations of working hour limits 

In May 1998, the Department released a report with its inspection findings. The State found 

supervision of residents by attending physicians that was both timely and in-person. The main 

problem uncovered was with residents’ working hours: 37 percent of residents worked more 

hours than the regulations allow. Surgical residents violated regulations the most: 79 percent 

exceeded the working hour limits. Residents exceeded the working hour limits the most in New 

York City area hospitals; there, 40 percent of all residents and 94 percent of surgical residents 

exceeded the working hour limits.14 

Hospital Quality: State Initiatives 

14 OEI-01-97-00054 
The Department intends to conduct these surprise inspections in each teaching hospital in the 

State targeted for review. To date, the Department has surveyed about 30 percent of the 

teaching hospitals. Once it conducts an initial surprise inspection in a teaching hospital, it plans 

to enforce the Bell Regulations with follow-up surveys and by assessing corrective action plans 

produced by the hospitals. Follow-up surveys have been conducted in 50 percent of the 

hospitals surveyed in the original study. Since these inspections began, the Department and the 

State’s hospital associations have conducted training sessions on compliance with the Bell 

Regulations with teaching hospitals. 

Hospital Quality: State Initiatives 

15 OEI-01-97-00054 
Public Disclosure through the Internet 

A Mechanism for Fostering Accountability 

With the rapid development of information technology, new opportunities have opened up for 

sharing more information, more quickly, to wider audiences. In regard to the external quality 

review of hospitals, this development means that the Internet now exists as a significant forum 

for informing consumers, health care purchasers, the media, and other interested parties about 

the performance of hospitals. The hospitals’ traditional accountability to private accrediting 

entities and public bodies can now be supplemented with direct accountability to the public. 

In particular, posting on Internet websites can be an important way for external reviewers to 

present information to the public on how hospitals fared during the review process. It can serve 

as a way of revealing hospital shortcomings as well as strengths, of indicating how performance 

has changed over time, and even of indicating how they compare with one another on certain 

measures. Furthermore, it can serve as a key motivator for hospitals to improve their 

performance and can reassure the public that an external review process is protecting its safety. 

In the same context, the posting of information on the Internet can also be a tool for holding the 

reviewers--the Joint Commission, the State agencies, and HCFA--more fully accountable to 

the public. Such information can address the extent and type of review efforts conducted as 

well as any evaluative information on the reviewers themselves. 

Finding the Way There 

The public disclosure of information along the lines indicated above is in the earliest of stages. 

Little is now available. To a significant degree, the technical, financial, and political constraints 

that inhibit the use of standardized performance measures also restrict the use of the Internet as 

an information-sharing forum. 

HCFA now lacks a website offering information on the performance of hospitals or 

reviewers.15 The Joint Commission has been more proactive in this regard as it makes hospital 

performance reports available on its website. But the extent and type of information it offers is 

minimal. A few States, acting on their own authority, are becoming even more proactive in this 

regard. Below we discuss the early experiences of New Jersey and Colorado. 

Hospital Quality: State Initiatives 

16 OEI-01-97-00054 
NEW JERSEY: Listing Hospital Enforcement Actions on the 

Internet 

In May 1998, the New Jersey Department of Health and Senior Services unveiled a website

detailing hospital enforcement actions, such as fines or other penalties imposed for violating

licensure or certification regulations.16 Visitors to the website can access information on

resolved hospital enforcement actions in New Jersey hospitals that the Department

investigated.17

The website contains documentation of the enforcement actions, as well as the actual penalty

letter the Department sent to the hospital, which outlines the findings of the investigation,

describes the licensing violations in

greater detail, and gives legal citations

for each violation (see box). It also

includes information about how the 

Department inspects hospitals and the 

State’s hospital licensure and complaint 

process. It explains how consumers 

can file a complaint against a hospital or 

review an inspection file for further 

information on an enforcement action. 

The Department updates the site 

quarterly with resolved enforcement 

actions and leaves the information on 

the website for 15 months. 

Making information available to 

consumers 

Information on New Jersey’s Website: 

Name and address of hospital

Enforcement date

Enforcement action

The issue

General area of Licensing Standards

How the violation was found

Hospital’s plan of correction

Hospital’s appeal status

Penalty letter sent to the hospital

Source: 

http://www.state.nj.us/health/hcsa/hospfines/hfines.htm 

Over a 6-month period in 1996, the Asbury Park Press, a New Jersey newspaper, published 

a series of articles called “Vital Signs” that examined the quality of hospital care in the State. 

The series focused, in part, on the lack of information available to consumers on the quality of 

hospitals. It found that “[hospital] fines are effectively a private matter. Seldom are fines 

announced or made readily available to the general public, even though such information is a 

public record. Fines are not printed in any official state publication and no press releases are 

sent to the media.”18 At the time, a consumer would have to contact the Department and 

request information about a hospital’s violation of licensing standards. 

During the publication of “Vital Signs,” the newspaper’s editorial board met with the 

Commissioner of the Department to discuss issues raised in the series. The series included 

recommendations that the Department make information about hospital violations more 

Hospital Quality: State Initiatives 

17 OEI-01-97-00054 
available to consumers. In early 1997, the Commissioner’s Office began a workgroup to 

develop a website detailing hospital fines and enforcement actions. 

Consideration and concerns about the website’s information 

When creating the site, the workgroup carefully considered what information to include and 

how to present it. Because New Jersey keeps information about hospital enforcement actions 

confidential until the hospital has a chance to respond, the Department makes it clear to the 

website’s readers that the hospital violations listed in the site have been corrected and the 

hospitals listed are now in compliance. In addition, the website cautions the reader: 

Please note that this report offers only a one-time “snap-shot” of hospital 

performance. To more fully assess the quality of care provided by a hospital, it is 

important that you review current and past survey reports. You may also wish to 

discuss hospital services and performance levels with your doctor and with family 

members or friends who have used the facility. 

Given that the information provided is publicly available, the New Jersey Hospital Association 

does not object to the website. However, it has concerns with how the Department presents 

the information. It is concerned that visitors to the website will be unable to easily discern the 

gravity of the deficiencies listed due to the regulatory language used, which may make the 

complaints seem worse than they actually are. In addition, the Association felt that keeping the 

information on the website for 15 months was too long, since a hospital would still be listed on it 

after having rectified the deficiency. 

COLORADO: Posting Compliance Summaries on the Internet 

On the Colorado Department of Public Health and Environment website, the public can find a 

compliance history covering the past couple of years for each hospital in the State.19 The 

website includes basic information on the hospital, such as its accreditation status, ownership, 

address, and associated facilities, in addition to summaries of all complaints and serious events, 

referred to as “occurrences,” reported to the State since January 1999.20 The summaries 

describe the reported incident, what actions the hospital took in response, and the 

Department’s follow-up actions (see the box on the following page for an example). The 

hospitals are also able to provide responses to the summaries. In the future, the State plans to 

include a list of deficiencies found during State licensure surveys and Medicare certification 

surveys in addition to the occurrence and complaint summaries. 

Hospital Quality: State Initiatives 

18 OEI-01-97-00054 
Emerging Activity and Refinement 

Since 1987, Colorado has mandated that hospitals file occurrence reports with the State. All 

hospitals, and more recently all licensed health facilities in the State, must submit reports of 

specific types of incidents within one business day of when the incident took place. Examples 

of such reportable 

occurrences include 

deaths from 

unexplained causes, 

missing persons, 

physical and sexual 

abuse, and lifethreatening reactions 

to anesthesia or 

transfusions. In 1995, 

the Colorado media 

criticized the 

confidential nature of 

these reports. Soon 

after, the State 

decided to make 

summaries of the 

reports available to 

the public from their 

offices in Denver. To 

make the summaries 

more widely available 

to the rest of 

Colorado, the 

Department began 

posting occurrence 

summaries on the 

Internet in early 1998 

as they were released 

to the public. More 

recently, the State has 

reorganized its 

database capabilities 

and its website, and 

Occurrence Summary Report 

Facility: XYZ

Date of Occurrence: 1/30/99

Report Timely: Yes

Type of Occurrence: Brain Injury 

Description of Occurrence: On 01/30/99, a female patient was given 

10mgm of morphine for pain intravenously. It was ordered 

intramuscularly. The patient suffered a respiratory arrest. The patient 

expired in intensive care on 02/03/99. 

Facility Action: The facility was unaware of the medication error at 

the time of the respiratory arrest. The patient was resuscitated and 

moved to intensive care. The physician and family were notified. The 

facility became aware of the medication error about three hours after 

the medication was given. The physician and family were made aware 

of the error. The nurse administering the medication was suspended 

and terminated. 

Department Findings: The Department conducted an on-site 

investigation and a HCFA authorized hospital survey as a result of 

this occurrence. The facility was cited for deficient practice for not 

assuring the competency of the nursing staff, for lack of accountability 

of the governing body and for deficient practice in the quality 

management functions of the hospital. The Department will conduct 

ongoing monitoring of facility compliance under the supervision of 

HCFA. The Department will review all facility occurrences during this 

monitoring process. 

Sent to Facility: 7/27/99 

Facility Comment: No facility comment received at this time 

Released to Public: 8/6/99 

An example of an occurrence summary from the Colorado website. 

now posts the reports by facility and includes information on complaints. 

Hospital Quality: State Initiatives 

19 OEI-01-97-00054 
Recognizing the Sensitivities 

With such widely available information, hospitals in Colorado are concerned that what might be 

an isolated event could be generalized across an entire facility. The media, also, are able to run 

attention-grabbing stories about incidents from the information made public. In an effort to 

further educate the public and alleviate any potential concerns from hospitals, the website 

includes language helping put occurrence reports in context and excludes the names of staff or 

patients. 

Hospital Quality: State Initiatives 

20 OEI-01-97-00054 
CONCLUSION 

States rely on both the Health Care Financing Administration (HCFA) and the Joint 

Commission for the external review of the hospitals operating within their boundaries. They 

receive funds from HCFA to determine nonaccredited hospitals’ compliance with the Medicare 

conditions of participation and to respond to adverse events and complaints in all Medicareparticipating hospitals. Over half the States deem Joint Commission accreditation to meet their 

State licensure requirements.21 

State Initiatives as Complements to HCFA and the Joint Commission 

The State initiatives presented in this report show that States can draw on their own authorities 

and resources to add a measure of public protection not provided by either HCFA or the Joint 

Commission. States have advantages that the other external reviewers would be hardpressed 

to match. In particular, States are closer to the action in their hospitals than a national reviewer 

like the Joint Commission could ever be. They are more likely to know the hospitals’ histories 

and the local market and be up-to-date on events such as union disputes and mergers, and 

trends such as local nursing shortages. As a result, States can act swiftly when needs arise. 

These advantages can help States contribute a valuable complement to the existing, national 

approaches to external review. They can craft oversight initiatives to meet a need unique to 

their State, like New York did with its surprise surveys that assessed compliance with its 

residency regulations. Alternatively, they can build on that existing national system of oversight, 

like Utah, thereby extending that system’s value to the State. Or they can develop initiatives 

that fill in gaps left by the approaches of HCFA or the Joint Commission. Both New York and 

Pennsylvania did that with their data collection efforts. 

State Initiatives as Instructive to HCFA, the Joint Commission, and Other States 

States can serve as important catalysts for continued national efforts aimed at improved hospital 

oversight. In fact, these States’ initiatives and others that are underway can be instructive to 

HCFA and the Joint Commission, as well as to other States. 

In our previous series, which examined the roles of HCFA and the Joint Commission, we 

stressed two themes in our recommendations to HCFA that are reflected in the initiatives 

highlighted herein: balance and accountability. We called for HCFA to promote balance 

between approaches to oversight that are collegial (oriented toward education and 

improvement) and regulatory (oriented toward ensuring minimum protections). This has 

particular relevance for the use of performance measures, which we called for to be used in 

ways that not only foster improvements but also in ways that help identify and deal with poor 

Hospital Quality: State Initiatives 

21 OEI-01-97-00054 
performers. New York and Pennsylvania’s experiences with performance measures 

demonstrate that such balanced uses are both feasible and constructive. 

Our previous series also called for HCFA to better use unannounced hospital surveys as a 

another way to introduce more balance between collegial and regulatory approaches to hospital 

oversight. New York’s recent surprise hospital inspections provide insights into how one State 

managed to coordinate a State-wide unannounced inspection process. 

The other theme from our prior series--increased accountability--is reflected in the States’ 

initiatives as well. New Jersey, Colorado, New York, and Pennsylvania all now have a body 

of experience with releasing hospital- or physician-specific performance information in various 

forms. Releasing such information promotes accountability on the part of the hospital as well as 

the State. Each of these States overcame concerns that have inhibited national efforts at similar 

disclosures, such as those about risk-adjustment and public misunderstanding the information. 

Their efforts illustrate the potential of public disclosure as a means to increasing accountability. 

Hospital Quality: State Initiatives 

22 OEI-01-97-00054 
APPENDIX A 

Methodology

We identified the initiatives highlighted in this report through a mail survey of State survey and 

certification agencies, follow-up telephone calls, and a literature review. 

As a part of our larger inquiry on the external review of hospital quality, we mailed a pretested 

survey to the State survey and certification agencies in the 50 States and the District of 

Columbia in August 1997. The response rate was 100 percent. The State survey addressed 

four areas of hospital quality oversight: State licensure of hospitals, private accreditation, 

Medicare certification, and Health Care Financing Administration oversight of State certification 

agencies. We also interviewed some State officials on the telephone and in person. 

We followed up the mail survey with telephone calls in July and August, 1998 to collect more 

information about hospital licensure and any special initiatives underway in each State. The 

response rate for this survey was 100 percent. 

We identified several State initiatives through the mail and telephone surveys, of which we 

selected nine for follow-up: Colorado, Massachusetts, New Jersey, New York (two initiatives), 

Ohio, Pennsylvania, South Carolina, Utah. Our criteria for defining a State initiative in hospital 

oversight were rather broad: that the initiative be led by the State, that it be implemented as 

opposed to planned, and that some minimum time with it had elapsed. 

With the exception of New York, which we visited on-site, we conducted the follow-up on the 

initiatives through telephone interviews, reviewing documents, and searching the literature and 

popular media. Our telephone interviews included not only State officials but also 

representatives of the State hospital associations for some States. We conducted this follow-up 

in the fall of 1998 and then updated our information in the summer and fall of 1999. During this 

time, we eliminated three States from our sample of initiatives: Massachusetts, because its 

initiative was not led by the State; Ohio, because its experience with its initiative was too 

limited; and South Carolina, because it abandoned its initiative over the course of our inquiry. 

For each of the remaining initiatives, we confirmed the accuracy of our information with officials 

directly involved in the initiatives. 

Hospital Quality: State Initiatives 

23 OEI-01-97-00054 
APPENDIX B 

Endnotes

1. Office of Inspector General, The External Review of Hospital Quality: A Call for Greater 

Accountability (OEI-01-97-00050), The External Review of Hospital Quality: The Role of 

Accreditation (OEI-01-97-00051), The External Review of Hospital Quality: The Role of 

Medicare Certification (OEI-01-97-00052), and The External Review of Hospital Quality: 

Holding the Reviewers Accountable (OEI-01-97-00053), July 1999. 

2. New York State Department of Health, Coronary Artery Bypass Surgery in New York State 

1994-1996. See also Edward L. Hannan et al, “Improving the Outcomes of Coronary Artery Bypass 

Surgery in New York State” Journal of the American Medical Association 271 (March 1994) 10: 

761-766. 

3. Edward L. Hannan et al, “Investigations of the Relationship between Volume and Mortality for 

Surgical Procedures Performed in New York State Hospitals,” Journal of the American Medical 

Association 262 (28 July 1989) 4: 503-510. 

4. Previously, surgeons included in the report had to have performed 200 CABG operations during the 

3-year reporting period. ( New York State Department of Health, Coronary Artery Bypass Surgery 

in New York State 1994-1996, October 1998, p. 11) 

5. Current results are based on data from 1994-1996; the latest mortality rates were released in 

October 1998. 

6. The Department recently reprogrammed its data collection software, enabling hospitals to more 

readily examine their own data. 

7. The DRGs were selected based on their high volume, cost, and wide variation in mortality. 

8. “Because of the continuing escalation of costs, an increasingly large number of Pennsylvania citizens 

have severely limited access to appropriate and timely health care. Increasing costs are also 

undermining the quality of health care services currently being provided.” Health Care Cost 

Containment Act, P.L. 783, No. 123, Sec. 2 (1992). 

9. The Council and the Department did collaborate and combine data for a report on caesareansection rates. 

10. Pennsylvania Health Care Cost Containment Council, Pennsylvania’s Guide to Coronary Artery 

Bypass Graft Surgery 1994-1995, May 1998 (98-05/01-07), p.1. 

Hospital Quality: State Initiatives 

24 OEI-01-97-00054 
APPENDIX B 

11. Pennsylvania Health Care Cost Containment Council, Cesarean Section Deliveries in 

Pennsylvania 1995, October 1997 (97-10/01-02), p. 1. 

12. J.M. Bentley and D.B. Nash, “How Pennsylvania hospitals have responded to publicly released 

reports on coronary artery bypass graft surgery,” Joint Commission Journal of Quality 

Improvement 24 (January 1998)1: 40-49; Ron Winslow, “Making the Grade: Improvements in quality 

of care suggests hospitals are taking reports cards to heart” Wall Street Journal, 19 October 1998, 

16. 

13. E.C. Schneider and A.M. Epstein “Use of Public Performance Reports: A Survey of Patients 

Undergoing Cardiac Surgery,” Journal of the American Medical Association 279 (May 1998) 20: 

1638-1642. 

14. For the purpose of the inspections, a sample work week for residents was calculated. Bell 

Regulations limit resident working hours to 80 hours per week, averaged over a 4-week period. 

15. It does, however, offer such a website with regard to nursing homes. 

16. The website’s address is http://www.state.nj.us/health/hcsa/hospfines/hfines.htm (accessed 

September 1999). 

17. New Jersey also has websites that have similar information on nursing homes and emergency 

medical services. 

18. Paul D’Ambrosio, “Some Say Low Fines Discourage Hospitals from Following Rules,” Asbury 

Park Press, 3 November 1996. 

19. The site includes facility profiles for all healthcare facilities in Colorado, including physical therapy 

clinics, psychiatric hospitals, dialysis facilities, and nursing homes. 

20. See http://www.hfd.cdphe.state.co.us/info.asp (accessed September 1999). 

21. Thirty-four of 51 States (including the District of Columbia) deem Joint Commission accreditation 

to meet their State licensure requirements for hospitals. Of those 34, 5 deem only for the year 

coinciding with the Joint Commission’s on-site survey. 

Hospital Quality: State Initiatives 

25 OEI-01-97-00054