Department of Health and Human Services 
OFFICE OF 
INSPECTOR GENERAL 
The External Review of Hospital Quality 
State Initiatives 
JUNE GIBBS BROWN 
Inspector General 
JANUARY 2000 
OEI-01-97-00054  OFFICE OF INSPECTOR GENERAL 
The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, is to 
protect the integrity of the Department of Health and Human Services programs as well as the health 
and welfare of beneficiaries served by them. This statutory mission is carried out through a nationwide 
program of audits, investigations, inspections, sanctions, and fraud alerts. The Inspector General 
informs the Secretary of program and management problems and recommends legislative, regulatory, 
and operational approaches to correct them. 
Office of Evaluation and Inspections 
The Office of Evaluation and Inspections (OEI) is one of several components of the Office of Inspector 
General. It conducts short-term management and program evaluations (called inspections) that focus 
on issues of concern to the Department, the Congress, and the public.  The inspection reports provide 
findings and recommendations on the efficiency, vulnerability, and effectiveness of departmental 
programs. 
OEI's Boston Regional Office prepared this report under the direction of Mark R. Yessian, Ph.D., 
Regional Inspector General. Principal OEI staff included: 
REGION  HEADQUARTERS 
Joyce M. Greenleaf, Project Leader  Jennifer Antico, Program Specialist

Maria E. Maddaloni

Laura C. McBride

To obtain copies of this report, please call the Boston Regional Office at (617)565-1050. 
Reports are also available on the World Wide Web at our home page address: 
http://www.dhhs.gov/progorg/oei  E X E C U T I V E   S U M M A  R Y  
Purpose 
To describe State government initiatives concerning the external review of hospital quality. 
Background 
State Initiatives Aimed at Hospital Oversight 
State agencies play important roles in overseeing hospitals, either as agents of the Health Care 
Financing Administration (HCFA), or under their own authorities to license such facilities and 
protect the public. Indeed, many States have developed and implemented initiatives aimed at 
addressing the quality of care in hospitals. This report presents a snapshot of six such initiatives 
that appear promising and could be instructive not only to other States but also to the Federal 
government. We present the initiatives in three categories: standardized performance 
measures, on-site surveys, and public disclosure of information on hospital performance. 
This report is a follow-up to our recent series of reports that assessed the roles of HCFA and 
the Joint Commission in overseeing hospitals that participate in Medicare. In that series, we 
directed recommendations for improvement to HCFA. 
Our information comes from discussions with the pertinent State officials and reviews of 
relevant documents. 
Standardized Performance Measures 
These are quantitative indicators that enable regulators, purchasers, and consumers to compare 
hospital performance to itself over time or to other hospitals. They can provide insights into a 
hospitals performance, foster improvements, and identify outliers. 
New York: Using Mortality Data to Measure Hospital Performance 
New York collects and publishes mortality data on coronary artery bypass graft surgery 
(CABG) and other procedures, fulfilling both quality improvement and regulatory functions. 
Hospital Quality: State Initiatives 
1 
OEI-01-97-00054  Pennsylvania: Creating Performance Report Cards 
Pennsylvania analyzes inpatient data from every hospital in the State to create reports that 
evaluate hospitals on quality-of-care measures such as length of stay, charges, and admission 
rates for CABG, breast cancer, and diabetes, among others.  In 1998, it documented a 22 
percent drop in in-hospital mortality for CABG from 1991 to 1995. 
On-site Surveys 
On-site surveys are a traditional way to assess directly a hospitals compliance with Federal, 
State, and local requirements. Many States, however, largely rely on the Joint Commission on 
Accreditation of Healthcare Organizations or surveys funded by HCFA for an on-site presence 
in their hospitals. 
Utah:  Observing Accreditation Surveys 
Utah relies on surveys by the Joint Commission to determine compliance with its  hospital 
licensure requirements. However, Health Department officials participate in the summary 
session at the end of the Joint Commissions on-site survey. The Department looks to the 
findings of the Joint Commission in determining whether to pursue its own enforcement actions. 
New York: Surprise Inspections of Residency Programs 
In 1998, New York launched 12 simultaneous surprise inspections of residency programs at 
teaching hospitals across the State.  New York regulates resident working conditions, and these 
surprise inspections marked the States first effort to ensure compliance. 
Public Disclosure through the Internet 
The Internet provides enormous opportunity for sharing performance information quickly and 
broadly. It can spur hospital improvements and reassure the public that an external review 
process is protecting its safety. 
New Jersey: Listing Hospital Enforcement Actions 
New Jerseys website details resolved hospital enforcement actions, such as fines or other 
penalties imposed for violating licensure or certification regulations.  The website also includes 
information about the States inspection, licensure, and complaint processes. It updates the site 
quarterly. 
Hospital Quality: State Initiatives 
2 
OEI-01-97-00054  Colorado: Posting Compliance Summaries 
Colorado posts hospitals compliance histories on its website.  In addition to basic hospital 
information, the website includes a summary of all complaints and serious events reported to the 
State since January 1999. The summaries describe the reported incident, what actions the 
facility took in response, and the Departments follow-up actions. 
Conclusion 
The State initiatives presented in this report show that States can draw on their own authorities 
and resources to add a measure of public protection not provided by either HCFA or the Joint 
Commission. The States have advantages, such as simply being closer to the action, that 
national reviewers would be hardpressed to match. And these advantages help States 
contribute a valuable complement to the existing, national approaches to external hospital 
review. 
State initiatives can also serve as important catalysts for continued national efforts aimed at 
improved hospital oversight.  Indeed, the States experiences can be instructive to HCFA, the 
Joint Commission, and other States.  The initiatives highlighted herein reinforce themes of 
balance and accountability that we promoted in our prior inquiry, which assessed the roles of 
HCFA and the Joint Commission.  The States experiences with performance measures, 
surprise inspections, and public disclosure demonstrate that such efforts can be both feasible 
and constructive. 
Hospital Quality: State Initiatives 
3 
OEI-01-97-00054  T A B L E   O F   C O N T E N T S  
EXECUTIVE SUMMARY  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  1

INTRODUCTION  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  5

Standardized Performance Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  7

New York: Using Mortality Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  8

Pennsylvania: Creating Report Cards  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  9

Conducting On-site Surveys  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  12

Utah: Observing Accreditation Surveys  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  13

New York: Surprise Hospital Inspections . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  13

Public Disclosure through the Internet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  16

New Jersey: Listing Hospital Enforcement actions  . . . . . . . . . . . . . . . . . . . . .  17

Colorado: Posting Compliance Summaries . . . . . . . . . . . . . . . . . . . . . . . . . . .  18

CONCLUSION  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  21

APPENDICES

A: Methodology  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  23

B: Endnotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  24

Hospital Quality: State Initiatives 
4 
OEI-01-97-00054  I N T R O D U C T I O N  
PURPOSE 
To describe State government initiatives concerning the external review of hospital quality. 
BACKGROUND 
States as HCFA Agents 
State agencies play important roles in helping the Health Care Financing Administration 
(HCFA) ensure that hospitals meet the minimum requirements for participating in the Medicare 
program. Funded by HCFA, they conduct validation surveys of those hospitals accredited by 
the Joint Commission on Accreditation of Healthcare Organizations, determine Medicare 
certification for those hospitals that choose not to be accredited, and respond to complaints and 
adverse events involving hospital care.  In a number of recent reports, we addressed how the 
States performed these roles and how HCFA held them (and the Joint Commission) 
accountable for their performance.1  We directed our recommendations for improvement to 
HCFA, which responded with an action plan. 
States Under their Own Authorities 
During the course of our prior work, we learned of initiatives that State governments were 
taking under their own authorities to address the quality of care provided in hospitals. Many of 
these were significant initiatives that could be instructive to the hospital quality review efforts of 
the Federal government and of other States. In this report we offer a snapshot of six such 
initiatives. 
We found each of the initiatives promising enough to warrant wider attention, although we did 
not evaluate them. Our information comes from discussions with pertinent State officials and 
from reviews of relevant documents (see appendix A for more details on our methodology). 
We do not suggest that the initiatives presented here represent a comprehensive listing of quality 
review efforts being undertaken by the States; nor do we suggest that they have necessarily 
been successful in achieving their objectives. 
We present the initiatives in three categories: (1) standardized performance measures, 
(2) on-site surveys, and (3) public disclosure on hospital performance. We begin each by 
discussing the relevance of these categories to the external quality review of hospitals. 
Hospital Quality: State Initiatives 
5 
OEI-01-97-00054  We conducted this inspection in accordance with the Quality Standards for Inspections 
issued by the Presidents Council on Integrity and Efficiency. 
Hospital Quality: State Initiatives 
6 
OEI-01-97-00054  Standardized  Performance  Measures 
Tools for Improving Performance and Enforcing Minimums 
Standardized performance measures are quantitative indicators that enable regulators, 
purchasers, and consumers to compare hospital performance and ensure that patients are 
receiving quality services. The comparisons can focus on a hospitals own performance over 
time and/or how its performance compares to other hospitals. Examples of performance 
measures include rates of complication from specific procedures and the preventive 
administration of antibiotics prior to surgery. The data can be drawn from sources such as a 
hospitals records, billing claims, or surveys of patients or providers. 
External reviewers can use performance data in two fundamentally different ways, both of 
which have value. One way is at root collegial: to foster continuous quality improvement. 
External reviewers collect performance data and distribute them with the intent of providing 
hospitals with comparative information they can use, voluntarily, to improve their own 
performance.  If particular hospitals find that their performance is significantly poorer than that 
of others, they can search for factors that explain the difference and for changes that will 
improve their performance. 
The other way is more regulatory: to identify hospitals that are performing poorly in relation to 
any of the designated measures and to hold those hospitals accountable for raising their 
performance to acceptable levels. External reviewers encourage the hospitals to improve 
voluntarily, but could also mandate corrective actions and even penalties. 
Realizing the Potential 
The Joint Commission on Accreditation of Healthcare Organizations has been one of the 
leaders in recognizing the potential of standardized performance measures. More than a 
decade ago it set forth a vision for hospital accreditation that would be based largely on data-
driven clinical performance indicators.  But its progress toward this end has been much slower 
than envisioned because of a number of significant obstacles. Three such obstacles have been 
particularly prominent: (1) the technical difficulties associated with the science of measurement 
and risk management, (2) the costs involved in collecting and distributing data, and (3) the 
political concerns raised by sharing performance data with others. 
Although these obstacles remain imposing, the quest to institutionalize performance 
measurement remains strong.  The Joint Commission requires, at a minimum, that accredited 
hospitals use a performance measurement system from its list of approved systems. 
Furthermore, in an effort to foster consistent national standards for performance measurement, 
the Joint Commission recently joined with the American Medical Association and the National 
Hospital Quality: State Initiatives 
7 
OEI-01-97-00054  Committee for Quality Assurance to create the Performance Measurement Council.  And at the 
Federal level, HCFA, in the proposed revisions to the Medicare conditions of participation, 
stresses the potential of performance measurement and calls for Medicare-certified hospitals to 
conduct a minimum number of performance improvement projects 
In the presentations below, we address the considerable experience that Pennsylvania and New 
York have had in using discrete hospital performance measures. 
NEW YORK:  Using CABG Mortality Data to Measure Hospital 
Performance 
New Yorks Department of Health publishes reports on risk-adjusted coronary artery bypass 
graft (CABG) mortality rates.  The annual reports present the number of CABG surgeries 
performed, the number of deaths, and 
three types of mortality rates for each 
surgeon and hospital in the State (see  Mortality Rates 
box). The Department distributes 
about 2,000 reports per year, which  Reports published by the New York

Department of Health present three mortality

also are available on the Internet. 
rates for each surgeon and hospital 
included:
Risk Adjusting and Public Scrutiny 
Observed mortality rate - the number of 
Development of these measures 
observed deaths divided by the total number 
began in 1988 at the direction of the  of patients who underwent isolated CABG 
Departments Commissioner as a  surgery. 
way to understand the disparities

among the mortality rates from  Expected mortality rate - the sum of the

CABGs in the States hospitals. A  predicted probabilities of death for all

patients divided by the total number of

Department statistician developed the 
patients. 
risk-adjustment methodology based 
on risk factors compiled by the 
Risk-adjusted mortality rate - the best
States Cardiac Advisory Committee, 
estimate, based on the statistical model, of
a longstanding group of cardiologists 
what the providers mortality rate would have 
and cardiac surgeons that advises the  been if the provider had a mix of patients 
Department on cardiac-related  identical to the statewide mix. The report also 
matters.  These risk factors include  presents a confidence interval for the risk-
age, gender, comorbidities, and  adjusted mortality rate. 
previous open heart operations.2 
Source: Coronary Artery Bypass Surgery in New York 
State 1994-1996, New York State Department of Health, 
October 1998, p. 8. 
Hospital Quality: State Initiatives 
8 
OEI-01-97-00054  The Department released the first risk-adjusted CABG mortality rates in 1990 for each 
hospital. An article on the data in the Journal of the American Medical Association noted 
that surgeons performing fewer CABGs had mortality rates that were higher than the surgeons 
who performed a higher volume of CABGs.3  Following this, a Freedom of Information Act 
request and subsequent lawsuit by a New York newspaper led to the release of surgeon-
specific data. The Department now analyzes and reports data on individual surgeons who have 
performed 200 procedures over a 3-year period or performed at least 1 procedure in each of 
the 3 years evaluated.4 
Since 1994, the Department has published a similar report presenting angioplasty mortality 
data. Within the next year, it will begin publishing a report with data on heart valve surgery 
mortality rates. 
Quality Improvement and Regulatory Effects 
The publication of the data has spurred both regulatory actions on the part of the State as well 
as improvements in the quality of care provided in the hospitals.  Risk-adjusted data enable the 
Department to compare every facilitys outcomes on a level playing field.  The Department has 
used the risk-adjusted mortality rates to identify outlier hospitals, and required one such outlier 
to affiliate with an academic institution.  It required two other hospitals to close or suspend their 
cardiac services as a result of their performance data. 
Since the Department began collecting and reporting CABG data, the mortality rate from this 
surgery in New York fell from 3.52 per 100 patients in 1989 to 2.44 per 100 patients in 1998.5 
Some hospitals in the State have initiated their own quality improvement programs based on the 
data from the CABG reports.6  For example, one hospital in the State had higher mortality rates 
and concluded that emergency cases were rushed into surgery without being stabilized. That 
hospital then made changes to ensure patients were stabilized prior to surgery, leading to an 
improvement in its mortality rate. In another hospital, the mortality data led to personnel 
changes. That hospital determined it was unable to handle the severe cardiac cases, so it 
referred such cases to other hospitals until it could expand its own ability to handle them. 
PENNSYLVANIA:  Creating Performance Report Cards 
Pennsylvania, too, collects and disseminates data on CABG surgery and has done so annually 
since 1993. In 1998, the Pennsylvania Health Care Cost Containment Council (hereafter 
referred to as the Council) released its fifth CABG report, which includes risk-adjusted 
measures of mortality, length of stay, and average charges. The report lists each measure 
individually by hospital, physician, and most recently, by health plan in the State. In addition to 
its annual CABG reports, the Council produces other reports addressing a range of health 
issues such as caesarean-section rates, breast cancer, heart attacks, osteoporosis, drug use, 
Hospital Quality: State initiatives 
9 
OEI-01-97-00054  and diabetes.  These issue briefs provide State-wide information on, for example, charges, 
average length of stay, and admission rates. The Council recently reissued for the first time 
since 1994 the Hospital Effectiveness Report, which includes data on 15 DRGs.7  For each 
DRG and each hospital, this report highlights risk-adjusted mortality, average length of stay, 
and average charges (see box). 
Source: Pennsylvania Health Care Cost Containment Council, 
A Hospital Performance Report-Region 1, 1999 (99-07/04-11/1), p. 7. 
Pennsylvania began this data initiative in 1986, when, at the urging of State business and labor 
leaders, its legislature created the Health Care Cost Containment Council.8  One of the 
Councils charges is to contain health care costs by stimulating competition among hospitals. 
By compiling and releasing hospital data on charges and outcomes, the Council aims to provide 
purchasers and consumers with informationthereby promoting more informed decisions and 
choices among health care providersand thus drive down health care costs.  The 21-member 
Council includes representatives from business, organized labor, hospitals, physician groups, 
and insurance companies, as well as consumers. The Secretary of the Department of Health is 
also a Council member, however there is little interaction between the Department and the 
Council.9 
Documented Improvements in Data Indicators 
The 1998 CABG report identified a 22 percent drop in in-hospital mortality for CABG surgery 
from 1991 to 1995 while the number of cases in the State increased.  Also, the average charge 
for CABG surgery decreased 3.9 percent.10  Similarly, caesarean-section rates have dropped 
across the State while the rate of vaginal births after a caesarean-section has increased.11 
Many attribute the measured improvements to hospitals own use of the data for internal 
improvement projects and to guide staff recruitment efforts.12  There has been some debate on 
the actual impact on consumers of the Councils data releases. Some argue that the data is 
Hospital Quality: State initiatives 
10 
OEI-01-97-00054  most helpful for hospitals and purchasers, rather than consumers.  However, a 1996 survey 
found that 20 percent of the heart-surgery patients surveyed were aware of the CABG report.13 
Data Collection Is Resource Intensive 
These improvements come at a significant cost to hospitals. The Hospital Association estimates

that the software mandated by the State costs Pennsylvania hospitals $10 million and an

additional $40 million for the separate data abstraction.  Hospitals can use the software system

for their own efforts but must buy back any data generated by the State.  Furthermore,

hospitals are concerned about the validity of the data. They note that certain factors, such as

Do Not Resuscitate orders, are not taken into account by the software. As a result, the data

may show a hospital with a significantly higher mortality than is actually true. Despite the

burden of data collection and submission, it appears that hospitals find the available data helpful. 

Hospital Quality: State initiatives 
11 
OEI-01-97-00054  On-site  Surveys 
The Core Element of the External Review Process 
Traditionally, the on-site survey of hospitals has been the key element of the external review 
process. Through on-site surveys, the surveyors can assess first-hand how well hospitals are 
meeting established requirements and can offer information and advice for hospitals seeking to 
improve their performance. The surveyors can observe conditions, review records, and 
interview administrators, clinical staff, and patients. 
As part of its accreditation process, the Joint Commission conducts on-site surveys every 3 
years to review hospital compliance with over 500 standards in 45 performance areas. The 
surveys range from 2 to 5 days, depending on the size of a hospital.  The Joint  Commission 
supplements these surveys with a small number of random, unannounced, and briefer surveys 
intended to assess continued compliance with standards. Occasionally, it will also visit hospitals 
to investigate complaints or adverse events. 
The State agencies also conduct surveys. For accredited hospitals, they perform validation 
surveys and respond to complaints and adverse events under agreement with HCFA.  For 
nonaccredited hospitals, they conduct routine on-site surveys, but because of resource 
constraints they use on-site surveys primarily for responding to complaints or adverse events. 
Using Survey Resources Strategically 
On-site surveys serve as a central, resource-intensive part of the external review process. 
Given that, it is especially important that the resources devoted to on-site surveys be used 
strategically.  Among the key questions that the Joint Commission, State agencies, and HCFA 
must consider are these: How can surveys best be targeted to problem areas?  How often 
should surveys be conducted?  What should be the balance between announced and 
unannounced surveys? Between routine surveys and surveys conducted in response to 
complaints and/or adverse events? 
These questions have particular relevance for the Joint Commission because it accredits about 
80 percent of the 6,200 hospitals that participate in Medicare and because 30 States deem 
hospitals accredited by the Joint Commission to meet their licensure requirements. But a 
number of States have taken initiatives that aim to add an on-site presence and a measure of 
patient safety beyond that afforded by the accreditation process.  Below, we present Utahs 
participation in Joint Commission accreditation surveys and New Yorks unannounced visits to 
teaching hospitals. 
Hospital Quality: State initiatives 
12 
OEI-01-97-00054  UTAH:  Observing Accreditation Surveys 
Utah retains the unique authority to attend a deemed hospitals accreditation survey and to take

action based on that survey. This authority allows the State to balance scarce resources and

hospital burden while maintaining a prominent presence in hospitals. 

Each year, when hospitals must apply for a license to operate in Utah, they have the option of

initiating, continuing, or relinquishing deemed status.  Thirty-two of the States  51 hospitals are

accredited and have chosen to be deemed. By opting for deemed status, hospitals forego the

annual licensure surveys conducted by the State, and the State relies instead on the survey

conducted by the accrediting organization (usually the Joint Commission). However, as

outlined in regulation, request for deemed status automatically authorizes officials from the

Department of Health to attend part of the hospitals accreditation survey. Deemed hospitals

must notify the State of their upcoming accreditation surveys. On the last day of the survey, a

Department official attends the closing conference.  Generally, the Department official takes

notes and acts as an observer, but he or she can and will ask questions if necessary. In

addition, Joint Commission surveyors have, on occasion, made themselves available after the

conference for further discussion. After the conference, the official reports back to the

Department on all deficiencies.  The Department can then take action based on this report and

its assessment of the accrediting bodys findings. If warranted, the Department can cite a

deficiency under licensure authority while on-site and often will interview hospital officials to find

out more information about the deficiencies. 

Observing Accreditation Surveys Allows for Timely Action 
Utah requires hospitals to submit, as many other States do, the final Joint Commission 
accreditation report to the State. However, the Joint Commission does not issue its final report 
until up to 120 days after the survey, and hospitals then have another 60 days to respond. If 
there were significant or immediate problems at the hospital, the Department would be unaware 
of them until well after the fact. By attending the conference, officials can hear the results first-
hand and take immediate action if they feel it is warranted. In 1998, the Department cited three 
hospitals on-the-spot based on attendance at such conferences.  The Departments presence at 
the conferences also allows it to learn about any supplemental deficiencies, which are generally 
excluded from the Joint Commissions final report. 
NEW YORK:  Surprise Inspections of Residency Programs 
In March 1998, New Yorks Department of Health conducted surprise inspections of 
residency programs simultaneously in 12 of the States teaching hospitals. This was the 
beginning of an effort to oversee hospitals compliance with the States residency law, known 
Hospital Quality: State Initiatives 
13 
OEI-01-97-00054  as the Bell Regulations.  New York is the only State that regulates resident working conditions

in its State hospital code. 

The State conducted the inspections by sending survey teams into the 12 teaching hospitals

simultaneously on a Thursday afternoon.  Surveyors then stayed on-site through the weekend. 

Survey teams consisted of Department

field workers, including registered

Bell Regulations 
nurses who conducted medical record

reviews to assess physician supervision 
In 1989, New York established the Bell 
of residents cases. 
Regulations to cover supervision of residents 
by attending physicians and to limit the hours 
The inspections focused on residents  worked by residents in the States teaching 
working hours and supervision by  hospitals. The impetus for these regulations 
attending physicians. Surveyors  was the death of a woman in a New York City 
emergency room; a grand jury probe 
assessed compliance with the States 
subsequently found problems with residency 
working hour limitations by examining 
training. 
posted schedules, then observing 
residents as they went on- and off-shift 
New York Citys public advocate published 
to assess their true working hours. The 
reports in 1994 and 1997 that revealed 
Department used the data collected on  violations of the Bell regulations in New York 
residents working hours to calculate a  Citys teaching hospitals. The cited violations 
sample work week, which estimated  included both insufficient supervision and 
residents work schedules. To  excessive hours worked by residents. In 
1998, State legislators questioned the 
determine adequacy of supervision, 
Departments Commissioner about Bell 
surveyors reviewed medical records for 
Regulation compliance, which led to the 
evidence that physicians supervised the 
surprise inspections. 
residents cases, and tested the on-call 
system by having residents contact their 
respective attending physicians.  In 
addition, surveyors interviewed residents and physicians to learn more about supervision and 
working hours. 
Violations of working hour limits 
In May 1998, the Department released a report with its inspection findings.  The State found 
supervision of residents by attending physicians that was both timely and in-person.  The main 
problem uncovered was with residents working hours: 37 percent of residents worked more 
hours than the regulations allow.  Surgical residents violated regulations the most:  79 percent 
exceeded the working hour limits.  Residents exceeded the working hour limits the most in New 
York City area hospitals; there, 40 percent of all residents and 94 percent of surgical residents 
exceeded the working hour limits.14 
Hospital Quality: State Initiatives 
14 
OEI-01-97-00054  The Department intends to conduct these surprise inspections in each teaching hospital in the 
State targeted for review. To date, the Department has surveyed about 30 percent of the 
teaching hospitals. Once it conducts an initial surprise inspection in a teaching hospital, it plans 
to enforce the Bell Regulations with follow-up surveys and by assessing corrective action plans 
produced by the hospitals. Follow-up surveys have been conducted in 50 percent of the 
hospitals surveyed in the original study. Since these inspections began, the Department and the 
States hospital associations have conducted training sessions on compliance with the Bell 
Regulations with teaching hospitals. 
Hospital Quality: State Initiatives 
15 
OEI-01-97-00054  Public  Disclosure  through  the  Internet 
A Mechanism for Fostering Accountability 
With the rapid development of information technology, new opportunities have opened up for 
sharing more information, more quickly, to wider audiences.  In regard to the external quality 
review of hospitals, this development means that the Internet now exists as a significant forum 
for informing consumers, health care purchasers, the media, and other interested parties about 
the performance of hospitals.  The hospitals traditional accountability to private accrediting 
entities and public bodies can now be supplemented with direct accountability to the public. 
In particular, posting on Internet websites can be an important way for external reviewers to 
present information to the public on how hospitals fared during the review process. It can serve 
as a way of revealing hospital shortcomings as well as strengths, of indicating how performance 
has changed over time, and even of indicating how they compare with one another on certain 
measures. Furthermore, it can serve as a key motivator for hospitals to improve their 
performance and can reassure the public that an external review process is protecting its safety. 
In the same context, the posting of information on the Internet can also be a tool for holding the 
reviewers--the Joint Commission, the State agencies, and HCFA--more fully accountable to 
the public.  Such information can address the extent and type of review efforts conducted as 
well as any evaluative information on the reviewers themselves. 
Finding the Way There 
The public disclosure of information along the lines indicated above is in the earliest of stages. 
Little is now available.  To a significant degree, the technical, financial, and political constraints 
that inhibit the use of standardized performance measures also restrict the use of the Internet as 
an information-sharing forum. 
HCFA now lacks a website offering information on the performance of hospitals or 
reviewers.15  The Joint Commission has been more proactive in this regard as it makes hospital 
performance reports available on its website.  But the extent and type of information it offers is 
minimal. A few States, acting on their own authority, are becoming even more proactive in this 
regard.  Below we discuss the early experiences of New Jersey and Colorado. 
Hospital Quality: State Initiatives 
16 
OEI-01-97-00054  NEW JERSEY:  Listing Hospital Enforcement Actions on the 
Internet 
In May 1998, the New Jersey Department of Health and Senior Services unveiled a website

detailing hospital enforcement actions, such as fines or other penalties imposed for violating

licensure or certification regulations.16  Visitors to the website can access information on

resolved hospital enforcement actions in New Jersey hospitals that the Department

investigated.17

The website contains documentation of the enforcement actions, as well as the actual penalty

letter the Department sent to the hospital, which outlines the findings of the investigation,

describes the licensing violations in

greater detail, and gives legal citations

Information on New Jerseys Website: 
for each violation (see box).  It also

includes information about how the 
Name and address of hospital

Department inspects hospitals and the 
Enforcement date

States hospital licensure and complaint 
Enforcement action

process.  It explains how consumers  The issue

can file a complaint against a hospital or  General area of Licensing Standards

review an inspection file for further  How the violation was found

information on an enforcement action.  Hospitals plan of correction

Hospitals appeal status

The Department updates the site 
Penalty letter sent to the hospital

quarterly with resolved enforcement 
actions and leaves the information on 
Source: 
the website for 15 months. 
http://www.state.nj.us/health/hcsa/hospfines/hfines.htm 
Making information available to 
consumers 
Over a 6-month period in 1996, the Asbury Park Press, a New Jersey newspaper, published 
a series of articles called Vital Signs that examined the quality of hospital care in the State. 
The series focused, in part, on the lack of information available to consumers on the quality of 
hospitals.  It found that [hospital] fines are effectively a private matter.  Seldom are fines 
announced or made readily available to the general public, even though such information is a 
public record.  Fines are not printed in any official state publication and no press releases are 
sent to the media.18  At the time, a consumer would have to contact the Department and 
request information about a hospitals violation of licensing standards. 
During the publication of Vital Signs, the newspapers editorial board met with the 
Commissioner of the Department to discuss issues raised in the series.  The series included 
recommendations that the Department make information about hospital violations more 
Hospital Quality: State Initiatives 
17 
OEI-01-97-00054  available to consumers. In early 1997, the Commissioners Office began a workgroup to 
develop a website detailing hospital fines and enforcement actions. 
Consideration and concerns about the websites information 
When creating the site, the workgroup carefully considered what information to include and 
how to present it.  Because New Jersey keeps information about hospital enforcement actions 
confidential until the hospital has a chance to respond, the Department makes it clear to the 
websites readers that the hospital violations listed in the site have been corrected and the 
hospitals listed are now in compliance.  In addition, the website cautions the reader: 
Please note that this report offers only a one-time snap-shot of hospital 
performance. To more fully assess the quality of care provided by a hospital, it is 
important that you review current and past survey reports.  You may also wish to 
discuss hospital services and performance levels with your doctor and with family 
members or friends who have used the facility. 
Given that the information provided is publicly available, the New Jersey Hospital Association 
does not object to the website.  However, it has concerns with how the Department presents 
the information.  It is concerned that visitors to the website will be unable to easily discern the 
gravity of the deficiencies listed due to the regulatory language used, which may make the 
complaints seem worse than they actually are.  In addition, the Association felt that keeping the 
information on the website for 15 months was too long, since a hospital would still be listed on it 
after having rectified the deficiency. 
COLORADO:  Posting Compliance Summaries on the Internet 
On the Colorado Department of Public Health and Environment website, the public can find a 
compliance history covering the past couple of years for each hospital in the State.19  The 
website includes basic information on the hospital, such as its accreditation status, ownership, 
address, and associated facilities, in addition to summaries of all complaints and serious events, 
referred to as occurrences, reported to the State since January 1999.20  The summaries 
describe the reported incident, what actions the hospital took in response, and the 
Departments follow-up actions (see the box on the following page for an example). The 
hospitals are also able to provide responses to the summaries.  In the future, the State plans to 
include a list of deficiencies found during State licensure surveys and Medicare certification 
surveys in addition to the occurrence and complaint summaries. 
Hospital Quality: State Initiatives 
18 
OEI-01-97-00054  Emerging Activity and Refinement 
Since 1987, Colorado has mandated that hospitals file occurrence reports with the State.  All 
hospitals, and more recently all licensed health facilities in the State, must submit reports of 
specific types of incidents within one business day of when the incident took place.  Examples 
of such reportable 
occurrences include 
Occurrence Summary Report 
deaths from 
unexplained causes,  Facility: XYZ

missing persons,  Date of Occurrence: 1/30/99

physical and sexual  Report Timely: Yes

Type of Occurrence: Brain Injury 

abuse, and life-
threatening reactions 
Description of Occurrence: On 01/30/99, a female patient was given 
to anesthesia or  10mgm of morphine for pain intravenously. It was ordered 
transfusions. In 1995,  intramuscularly. The patient suffered a respiratory arrest. The patient 
expired in intensive care on 02/03/99. 
the Colorado media 
criticized the 
Facility Action: The facility was unaware of the medication error at 
confidential nature of 
the time of the respiratory arrest. The patient was resuscitated and 
these reports.  Soon  moved to intensive care. The physician and family were notified. The 
after, the State  facility became aware of the medication error about three hours after 
the medication was given. The physician and family were made aware 
decided to make 
of the error. The nurse administering the medication was suspended 
summaries of the 
and terminated. 
reports available to 
the public from their  Department Findings: The Department conducted an on-site 
investigation and a HCFA authorized hospital survey as a result of 
offices in Denver.  To 
this occurrence. The facility was cited for deficient practice for not 
make the summaries 
assuring the competency of the nursing staff, for lack of accountability 
more widely available  of the governing body and for deficient practice in the quality 
to the rest of  management functions of the hospital. The Department will conduct 
ongoing monitoring of facility compliance under the supervision of 
Colorado, the 
HCFA. The Department will review all facility occurrences during this 
Department began 
monitoring process. 
posting occurrence 
summaries on the  Sent to Facility: 7/27/99 
Internet in early 1998 
Facility Comment: No facility comment received at this time 
as they were released 
to the public.  More 
Released to Public: 8/6/99 
recently, the State has 
reorganized its  An example of an occurrence summary from the Colorado website. 
database capabilities 
and its website, and 
now posts the reports by facility and includes information on complaints. 
Hospital Quality: State Initiatives 
19 
OEI-01-97-00054  Recognizing the Sensitivities 
With such widely available information, hospitals in Colorado are concerned that what might be 
an isolated event could be generalized across an entire facility. The media, also, are able to run 
attention-grabbing stories about incidents from the information made public. In an effort to 
further educate the public and alleviate any potential concerns from hospitals, the website 
includes language helping put occurrence reports in context and excludes the names of staff or 
patients. 
Hospital Quality: State Initiatives 
20 
OEI-01-97-00054  C O N C L U S I O N  
States rely on both the Health Care Financing Administration (HCFA) and the Joint 
Commission for the external review of the hospitals operating within their boundaries.  They 
receive funds from HCFA to determine nonaccredited hospitals compliance with the Medicare 
conditions of participation and to respond to adverse events and complaints in all Medicare-
participating hospitals.  Over half the States deem Joint Commission accreditation to meet their 
State licensure requirements.21 
State Initiatives as Complements to HCFA and the Joint Commission 
The State initiatives presented in this report show that States can draw on their own authorities 
and resources to add a measure of public protection not provided by either HCFA or the Joint 
Commission.  States have advantages that the other external reviewers would be hardpressed 
to match.  In particular, States are closer to the action in their hospitals than a national reviewer 
like the Joint Commission could ever be.  They are more likely to know the hospitals histories 
and the local market and be up-to-date on events such as union disputes and mergers, and 
trends such as local nursing shortages. As a result, States can act swiftly when needs arise. 
These advantages can help States contribute a valuable complement to the existing, national 
approaches to external review. They can craft oversight initiatives to meet a need unique to 
their State, like New York did with its surprise surveys that assessed compliance with its 
residency regulations.  Alternatively, they can build on that existing national system of oversight, 
like Utah, thereby extending that systems value to the State. Or they can develop initiatives 
that fill in gaps left by the approaches of HCFA or the Joint Commission. Both New York and 
Pennsylvania did that with their data collection efforts. 
State Initiatives as Instructive to HCFA, the Joint Commission, and Other States 
States can serve as important catalysts for continued national efforts aimed at improved hospital 
oversight.  In fact, these States initiatives and others that are underway can be instructive to 
HCFA and the Joint Commission, as well as to other States. 
In our previous series, which examined the roles of HCFA and the Joint Commission, we 
stressed two themes in our recommendations to HCFA that are reflected in the initiatives 
highlighted herein: balance and accountability. We called for HCFA to promote balance 
between approaches to oversight that are collegial (oriented toward education and 
improvement) and regulatory (oriented toward ensuring minimum protections).  This has 
particular relevance for the use of performance measures, which we called for to be used in 
ways that not only foster improvements but also in ways that help identify and deal with poor 
Hospital Quality: State Initiatives 
21 
OEI-01-97-00054  performers. New York and Pennsylvanias experiences with performance measures 
demonstrate that such balanced uses are both feasible and constructive. 
Our previous series also called for HCFA to better use unannounced hospital surveys as a 
another way to introduce more balance between collegial and regulatory approaches to hospital 
oversight. New Yorks recent surprise hospital inspections provide insights into how one State 
managed to coordinate a State-wide unannounced inspection process. 
The other theme from our prior series--increased accountability--is reflected in the States 
initiatives as well.  New Jersey, Colorado, New York, and Pennsylvania all now have a body 
of experience with releasing hospital- or physician-specific performance information in various 
forms. Releasing such information promotes accountability on the part of the hospital as well as 
the State.  Each of these States overcame concerns that have inhibited national efforts at similar 
disclosures, such as those about risk-adjustment and public misunderstanding the information. 
Their efforts illustrate the potential of public disclosure as a means to increasing accountability. 
Hospital Quality: State Initiatives 
22 
OEI-01-97-00054  APPENDIX A 
Methodology

We identified the initiatives highlighted in this report through a mail survey of State survey and 
certification agencies, follow-up telephone calls, and a literature review. 
As a part of our larger inquiry on the external review of hospital quality, we mailed a pretested 
survey to the State survey and certification agencies in the 50 States and the District of 
Columbia in August 1997. The response rate was 100 percent. The State survey addressed 
four areas of hospital quality oversight:  State licensure of hospitals, private accreditation, 
Medicare certification, and Health Care Financing Administration oversight of State certification 
agencies. We also interviewed some State officials on the telephone and in person. 
We followed up the mail survey with telephone calls in July and August, 1998 to collect more 
information about hospital licensure and any special initiatives underway in each State.  The 
response rate for this survey was 100 percent. 
We identified several State initiatives through the mail and telephone surveys, of which we 
selected nine for follow-up: Colorado, Massachusetts, New Jersey, New York (two initiatives), 
Ohio, Pennsylvania, South Carolina, Utah.  Our criteria for defining a State initiative in hospital 
oversight were rather broad: that the initiative be led by the State, that it be implemented as 
opposed to planned, and that some minimum time with it had elapsed. 
With the exception of New York, which we visited on-site, we conducted the follow-up on the 
initiatives through telephone interviews, reviewing documents, and searching the literature and 
popular media. Our telephone interviews included not only State officials but also 
representatives of the State hospital associations for some States. We conducted this follow-up 
in the fall of 1998 and then updated our information in the summer and fall of 1999. During this 
time, we eliminated three States from our sample of initiatives: Massachusetts, because its 
initiative was not led by the State; Ohio, because its experience with its initiative was too 
limited; and South Carolina, because it abandoned its initiative over the course of our inquiry. 
For each of the remaining initiatives, we confirmed the accuracy of our information with officials 
directly involved in the initiatives. 
Hospital Quality: State Initiatives 
23 
OEI-01-97-00054  APPENDIX B 
Endnotes

1. Office of Inspector General, The External Review of Hospital Quality: A Call for Greater 
Accountability (OEI-01-97-00050), The External Review of Hospital Quality: The Role of 
Accreditation (OEI-01-97-00051), The External Review of Hospital Quality: The Role of 
Medicare Certification (OEI-01-97-00052), and The External Review of Hospital Quality: 
Holding the Reviewers Accountable (OEI-01-97-00053), July 1999. 
2. New York State Department of Health, Coronary Artery Bypass Surgery in New York State 
1994-1996. See also Edward L. Hannan et al, Improving the Outcomes of Coronary Artery Bypass 
Surgery in New York State Journal of the American Medical Association 271 (March 1994) 10: 
761-766. 
3. Edward L. Hannan et al, Investigations of the Relationship between Volume and Mortality for 
Surgical Procedures Performed in New York State Hospitals, Journal of the American Medical 
Association 262 (28 July 1989) 4: 503-510. 
4.  Previously, surgeons included in the report had to have performed 200 CABG operations during the 
3-year reporting period.  ( New York State Department of Health, Coronary Artery Bypass Surgery 
in New York State 1994-1996, October 1998, p. 11) 
5. Current results are based on data from 1994-1996; the latest mortality rates were released in 
October 1998. 
6.  The Department recently reprogrammed its data collection software, enabling hospitals to more 
readily examine their own data. 
7. The DRGs were selected based on their high volume, cost, and wide variation in mortality. 
8.  Because of the continuing escalation of costs, an increasingly large number of Pennsylvania citizens 
have severely limited access to appropriate and timely health care. Increasing costs are also 
undermining the quality of health care services currently being provided. Health Care Cost 
Containment Act, P.L. 783, No. 123, Sec. 2 (1992). 
9.  The Council and the Department did collaborate and combine data for a report on caesarean-
section rates. 
10. Pennsylvania Health Care Cost Containment Council, Pennsylvanias Guide to Coronary Artery 
Bypass Graft Surgery 1994-1995, May 1998 (98-05/01-07), p.1. 
Hospital Quality: State Initiatives 
24 
OEI-01-97-00054  APPENDIX B 
11. Pennsylvania Health Care Cost Containment Council, Cesarean Section Deliveries in 
Pennsylvania 1995, October 1997 (97-10/01-02), p. 1. 
12.  J.M. Bentley and D.B. Nash, How Pennsylvania hospitals have responded to publicly released 
reports on coronary artery bypass graft surgery, Joint Commission Journal of Quality 
Improvement 24 (January 1998)1: 40-49; Ron Winslow, Making the Grade: Improvements in quality 
of care suggests hospitals are taking reports cards to heart Wall Street Journal, 19 October 1998, 
16. 
13. E.C. Schneider and A.M. Epstein Use of Public Performance Reports: A Survey of Patients 
Undergoing Cardiac Surgery, Journal of the American Medical Association 279 (May 1998) 20: 
1638-1642. 
14. For the purpose of the inspections, a sample work week for residents was calculated. Bell 
Regulations limit resident working hours to 80 hours per week, averaged over a 4-week period. 
15. It does, however, offer such a website with regard to nursing homes. 
16.  The websites address is http://www.state.nj.us/health/hcsa/hospfines/hfines.htm (accessed 
September 1999). 
17. New Jersey also has websites that have similar information on nursing homes and emergency 
medical services. 
18. Paul DAmbrosio, Some Say Low Fines Discourage Hospitals from Following Rules, Asbury 
Park Press, 3 November 1996. 
19. The site includes facility profiles for all healthcare facilities in Colorado, including physical therapy 
clinics, psychiatric hospitals, dialysis facilities, and nursing homes. 
20. See http://www.hfd.cdphe.state.co.us/info.asp (accessed September 1999). 
21. Thirty-four of 51 States (including the District of Columbia) deem Joint Commission accreditation 
to meet their State licensure requirements for hospitals.  Of those 34, 5 deem only for the year 
coinciding with the Joint Commissions on-site survey. 
Hospital Quality: State Initiatives 
25 
OEI-01-97-00054