Department of Health and Human Services

OFFICE OF 

INSPECTOR GENERAL 

 

 

ADVERSE EVENTS IN HOSPITALS: 

METHODS FOR 

IDENTIFYING EVENTS 

 

Daniel R. Levinson 

Inspector General 

March 2010 

OEI-06-08-00221 
 

 EXECUTIVE SUMMARY 

OBJECTIVE

To evaluate the usefulness of selected methods for identifying events 

that harm hospitalized Medicare beneficiaries. 

BACKGROUND 

The term “adverse event” describes harm to a patient as a result of 

medical care or harm that occurs in a health care setting. The term 

“never events” refers to a specific list of serious events, such as surgery 

on the wrong patient, that the National Quality Forum deemed “should 

never occur in a healthcare setting.” The Tax Relief and Health Care 

Act of 2006 (the Act) mandated that the Office of Inspector General 

(OIG) report to Congress about such events, including making 

recommendations about processes for identifying events. To meet the 

requirements of the Act, OIG published a series of reports in 2008 and 

will publish additional reports based on ongoing work. 

In 2008, we conducted a case study to determine the incidence of 

adverse events (hereinafter referred to as events) by reviewing a 

random sample of 278 Medicare beneficiary hospitalizations selected 

from all Medicare discharges from acute care hospitals in two selected 

counties during a 1-week period in August 2008. Using a two-stage 

review process, the case study identified 120 events. The first stage 

consisted of using five selected methods to screen for events, including 

nurse reviews of medical records, interviews of Medicare beneficiaries, 

two types of billing data analysis, and reviews of internal hospital 

incident reports. Each time a screening method indicated the possibility 

that an event occurred during the hospitalization, we designated the 

possible event as a “flag.” The second stage consisted of physician 

reviews of medical records for 183 of the 278 beneficiary 

hospitalizations—those with at least 1 flag. This report provides an 

indepth examination of the usefulness of the five screening methods 

used for identifying events. We considered the most useful methods to 

be those that identified the greatest number of events. 

FINDINGS 

The five screening methods were useful in identifying events that 

harmed patients; however, most flags were not associated with 

events. Physician reviews determined that 256 of the 662 flags 

(39 percent) generated by the screening methods were associated with 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS i 
 

EXECUTIVE SUMMARY 

1 or more of the 120 events found by the case study. Nurse reviews and 

one type of billing data analysis identified the greatest number of 

events. Although the five screening methods helped to identify events, 

they also generated many flags (61 percent) that were not associated 

with events. 

Shortcomings in two screening methods have implications for 

Medicare payments and Federal initiatives to identify, track, and 

monitor events. Our analysis revealed vulnerabilities regarding both 

accuracy and completeness of two critical sources of information about 

events. Through analysis of the billing data, we found that diagnosis 

codes were inaccurate or absent for 7 of the 11 Medicare hospitalacquired conditions identified by the case study. These problems would 

prevent Medicare’s automated payment software from identifying the 

hospital-acquired conditions, which could result in Medicare 

overpayments and inhibit use of billing data to monitor quality of care 

in hospitals. We also found that hospitals participating in the case 

study apparently did not have any internal incident reports for 112 of 

the 120 events (93 percent), including some of the most serious events 

involving death or permanent disability to the patients. The lack of 

such reports could prevent hospitals from tracking events as required by 

regulation or reporting events to outside entities. It also suggests that 

hospital incident-reporting systems may be an unreliable source of 

information for Patient Safety Organizations (PSO), which are entities 

that aggregate and analyze information about events voluntarily 

reported by hospitals. 

RECOMMENDATIONS 

Overall, the case study findings suggest that an effective way to identify 

events is through review of medical records by nurses and/or physicians, 

whereas other screening methods identified far fewer events. 

Additionally, it demonstrated opportunities to address shortcomings that 

limited the usefulness of some screening methods. 

Therefore, we recommend to the Centers for Medicare & Medicaid 

Services (CMS) and the Agency for Healthcare Research and Quality 

(AHRQ) that: 

CMS and AHRQ should explore opportunities to identify events 

when conducting medical record reviews for other purposes. 

Examples of such efforts include, but are not limited to: CMS’s Medicare 

Comprehensive Error Rate Testing and the work of Quality Improvement 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS ii 
 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS iii 

EXECUTIVE SUMMARY 

Organizations; AHRQ grant awards for patient safety research; and State 

survey agency work in certifying hospital compliance with the Medicare 

conditions of participation. 

CMS should ensure that hospitals code claims accurately and 

completely to allow for identification of hospital-acquired conditions 

affected by Medicare’s payment policy. To identify Medicare hospitalacquired conditions, CMS relies on hospitals to code diagnoses and 

present on admission (POA) indicators (which indicate whether 

diagnoses were present upon admission). CMS should determine 

whether additional guidance is needed to ensure that hospitals code 

diagnoses and POA indicators accurately and completely. 

CMS should provide interpretive guidelines for State survey 

agencies to assess hospital compliance with requirements to track 

and monitor adverse events. The CMS State Operations Manual

contains no guidance to State survey agencies regarding assessing 

hospital compliance with Federal requirements to track and monitor 

events; therefore, it is unclear how surveyors are to assess hospitals’ 

compliance. 

AHRQ should inform PSOs that internal hospital incident reporting 

may be insufficient to provide needed information about events to 

PSOs. As the key Federal agency involved with PSOs, AHRQ indicated 

that PSOs will rely primarily upon hospitals for identifying, tracking, 

and reporting information about events. However, we found that 

hospital incident reports existed for only 8 of the 120 events identified 

in the case study. In providing technical assistance to support PSOs, 

AHRQ should convey the importance of hospitals’ having strong 

internal incident-reporting procedures. 

AGENCY COMMENTS 

We received comments on our draft of this report from AHRQ and CMS. 

AHRQ concurred with the report as written. 

CMS stated that it agrees with recommendations relevant to its 

programs. 
 TABLE OF CONTENTS 

EXECUTIVE SUMMARY ................................. i

INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 

FINDINGS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 

The five screening methods were useful in identifying events 

that caused harm; however, most flags were not associated with 

events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 

Shortcomings in two screening methods have implications for 

Medicare payments and Federal initiatives to identify, track, 

and monitor events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 

RECOMMENDATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 

Agency Comments and Office of Inspector General Response . . . 19 

SCREENING METHODS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 

APPENDIXES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 

 A: Tax Relief and Health Care Act of 2006 . . . . . . . . . . . . . . . . . . 29 

 B: Medicare List of Hospital-Acquired Conditions . . . . . . . . . . . . 31 

 C: Glossary of Selected Clinical Terms . . . . . . . . . . . . . . . . . . . . . 32 

 D: Case Study Protocol for Nurse Reviews . . . . . . . . . . . . . . . . . . 34 

 E: Medicare Beneficiary Interview Questions . . . . . . . . . . . . . . . . 36 

 F: Patient Safety Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 

 G: Case Study Protocol for Physician Reviews . . . . . . . . . . . . . . . 41 

 H: Results of Screening Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 48 

 I: Agency Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 

ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
 

 INTRODUCTION 

OBJECTIVE 

To evaluate the usefulness of selected methods for identifying events 

that harm hospitalized Medicare beneficiaries. 

BACKGROUND 

Statutory Mandate and Office of Inspector General Response 

The Tax Relief and Health Care Act of 2006 (the Act) requires that the 

Office of Inspector General (OIG) report to Congress regarding the 

incidence of “never events” among Medicare beneficiaries, payment by 

Medicare or beneficiaries for services furnished in connection with such 

events, and the processes that the Centers for Medicare & Medicaid 

Services (CMS) uses to identify such events and deny or recoup 

payment.1, 2 (For relevant text of the Act, see Appendix A.) The Act 

also requires that OIG make recommendations, as appropriate, 

regarding processes for identifying such events. To meet the 

requirements of the Act, OIG published a series of reports and will 

publish additional reports based on ongoing work.3 

Expanding beyond the term “never events,” OIG studies use the term 

“adverse event” to allow for a more comprehensive examination of the 

topic. As used in these studies, the term “adverse event” describes harm 

to a patient as a result of medical care or harm that occurs in a health 

care setting. Although an adverse event often indicates that the care 

resulted in an undesirable clinical outcome and may involve medical 

errors, adverse events do not always involve errors, negligence, or poor 

quality of care and may not always be preventable.4 

1 The Act, P.L. 109-432 § 203. 

2 For purposes of the Act, the term “never events” means “an event that is listed and 

endorsed as a serious reportable event by the National Quality Forum (NQF) as of 

November 16, 2006.” The Act, § 203(d). The NQF used the term “Serious Reportable 

Events” to describe a specific list of events associated primarily with patient death or 

serious disability that “should never occur in a healthcare setting.” These colloquially 

became known as “never events.” Available online at 

http://www.qualityforum.org/Topics/Safety.aspx. Accessed on August 12, 2009. 

3 OIG, Adverse Events in Hospitals: Overview of Key Issues, OEI-06-07-00470, 

December 2008; OIG, Adverse Events in Hospitals: State Reporting Systems; 

OEI-06-07-00471, December 2008; and OIG, Adverse Events in Hospitals: Case Study of 

Incidence Among Medicare Beneficiaries in Two Counties, OEI-06-08-00220, 

December 2008; OIG, Adverse Events in Hospitals: Public Disclosure of Information About 

Events, OEI-06-09-00360. 

4 R.M. Wachter, Understanding Patient Safety, McGraw-Hill, 2008. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 1 
 

INTRODUCTION 

Following a review of Medicare policies and expenditures, as well as 

consultation with officials from CMS and the Agency for Healthcare 

Research and Quality (AHRQ), we chose to focus much of our work on 

inpatient acute care hospitals. In 2006, 12.5 million Medicare 

beneficiaries were hospitalized,5 with inpatient hospital costs 

constituting the largest portion of Medicare expenditures (32 percent in 

2006).6 Federal regulations require that hospitals, as a condition of 

participation in the Medicare and Medicaid programs, develop and 

maintain a quality assessment and performance improvement (QAPI) 

program.7 As a part of the QAPI program, hospitals must “measure, 

analyze, and track quality indicators, including adverse patient 

events.”8 To accomplish this, hospitals must “track medical errors and 

adverse patient events, analyze their causes, and implement preventive 

actions and mechanisms that include feedback and learning throughout 

the hospital.”9 State health agencies, otherwise referred to as State 

survey agencies, perform survey and review functions for Medicare and 

certify that hospitals comply with these Federal requirements.10

Hospitals may also report information about adverse events to various 

entities, such as Patient Safety Organizations (PSO), which seek to 

improve quality of patient care by identifying and reducing the risks 

and hazards associated with care. 11, 12 PSOs must certify that they 

5 CMS, Statistics Book, p. 43, Table IV.1: Medicare/short-stay hospital utilization, 

2008. 

6 Based on data from the Congressional Budget Office (CBO), “Fact Sheet for CBO’s 

March 2007 Baseline: Medicare,” March 7, 2007. Available online at 

http://www.cbo.gov/budget/factsheets/2007b/medicare.pdf. Accessed on September 8, 2008. 

7 42 CFR § 482.21. In the preamble to the final rule adding QAPI to the conditions of 

participation, CMS said QAPI focuses providers on the care delivered to patients, the 

performance of the hospital as an organization, and the effect of treatment. 

68 Fed. Reg. 3435 (Jan. 24, 2003). 

8 42 CFR § 482.21(a)(2). 

9 42 CFR § 482.21(c)(2). 

10 42 CFR § 488.10. Providers accredited by an accreditation organization are deemed to 

be in compliance with Medicare conditions of participation. 42 CFR § 488.5. 

11 Sections 923 and 924 of the Public Health Service Act, which were added by the 

Patient Safety and Quality Improvement Act of 2005, required the Department of Health 

and Human Services (HHS) to determine that PSOs meet certain criteria to perform 

“patient safety activities” and establish a Network of Patient Safety Databases to receive, 

analyze, and report on patient safety information submitted by the PSOs. Patient Safety 

and Quality Improvement Act of 2005, P.L. 109-41 § 2, Public Health Service Act, §§ 923 

and 924, 42 U.S.C. §§ 299b-23 and 24. 

12 Other entities, such as States operating adverse event reporting systems and the Joint 

Commission, also accept reports from hospitals regarding adverse events. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 2 
 

INTRODUCTION 

have policies and procedures in place to perform “patient safety 

activities,” such as aggregation and analysis of reported events received 

from hospitals that voluntarily report patient safety information.13 The 

Secretary of Health & Human Services has delegated to AHRQ the 

responsibility for determining whether certifications submitted by 

entities seeking to be PSOs meet Federal requirements.14 The 

Secretary may also provide technical assistance to PSOs on matters 

such as methodology, communication, data collection, or privacy 

concerns.15 An AHRQ-funded study found that 98 percent of hospitals 

reported having adverse event reporting systems and nearly all 

hospitals have safety and quality functions that would facilitate 

participating in a PSO.16 HHS indicated that PSOs would rely 

primarily on “existing hospital activities,” such as internal procedures 

for identifying and reporting adverse events.17 

Present on Admission Indicators and Medicare’s Hospital-Acquired 

Conditions Policy 

Since October 1, 2007, hospitals have been required to assign a present 

on admission (POA) indicator to each principal and secondary diagnosis 

for acute Inpatient Prospective Payment System (IPPS) claims for all 

discharges.18 This was an initial step in complying with the Deficit 

Reduction Act of 2005 (DRA), which required CMS to select at least two 

hospital-acquired conditions for which hospitals would not be paid 

higher Medicare reimbursement than if the conditions had not occurred 

for discharges occurring on or after October 1, 2008.19 In the fiscal year 

(FY) 2009 IPPS Final Rule, CMS established the Medicare policy to 

deny hospitals higher payment for hospital admissions complicated by 

any of 10 categories of hospital-acquired conditions, that is, conditions 

that were not present upon admission.20 Appendix B contains a list of 

these 10 hospital-acquired conditions. 

3 

13 Public Health Service Act, § 924(a), 42 U.S.C. § 299b-24(a). 

14 73 Fed. Reg. 70732 (Nov. 21, 2008). 

15Public Health Service Act, § 925, 42 U.S.C. § 922b-25. 

16 73 Fed. Reg. 70732, 70793 (Nov. 21, 2008). 

17 Ibid. 

18 CMS, CMS Manual System, Change Request 5679 (July 20, 2007). 

19 DRA, P.L. 109-171 § 5001(c)(1), Social Security Act, § 1886(d)(4)(D), 

42 U.S.C. § 1395ww(d)(4)(D). 

20 FY 2009 Final IPPS Rule, 73 Fed. Reg. 48434, 48471-48491 (Aug. 19, 2008). 

Effective October 1, 2008, CMS began denying hospitals higher payment for Medicare 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 
 

INTRODUCTION 

Efforts by CMS to identify and deny payment for certain adverse events 

are part of a broader quality improvement initiative. In 2005, CMS 

released its Quality Improvement Roadmap, which expressed a vision 

for health care quality—“The right care for every person every time.”21 

To achieve this vision, CMS stated its commitment to safe, effective, 

timely, patient-centered, efficient, and equitable medical care. A central 

part of quality improvement is the value-based purchasing (VBP) 

initiative, which ties payment to quality medical care. Medicare’s 

hospital-acquired conditions policy is part of the VBP initiative.22 

In the preamble to the FY 2009 IPPS Proposed Rule, CMS indicated 

that evaluating POA indicators in Medicare billing data could be used to 

better understand and prevent the occurrences of hospital-acquired 

conditions. CMS posed that this information could be used to measure 

hospital performance as part of the VBP program. Such information 

could also be publicly reported, enabling consumers to make more 

informed choices about their health care. Additionally, CMS indicated 

that researchers could use POA data in a variety of ways to increase 

understanding and identify best practices for prevention of hospitalacquired conditions.23

Case Study of Events That Harmed Hospitalized Medicare Beneficiaries 

To best respond to the Act, we conducted a pilot study in 2008 to 

determine the incidence of adverse events among the Medicare 

population in two selected counties so that we could learn about various 

methods for identifying adverse events. The results of that study, 

Adverse Events in Hospitals: Case Study of Incidence Among Medicare 

Beneficiaries in Two Counties (hereinafter referred to as “case study”), 

are the basis of this report.24 

In the case study, we found that an estimated 15 percent of hospitalized 

Medicare beneficiaries in two selected counties experienced adverse 

admissions complicated by these conditions. CMS, CMS Manual System, Change 

Request 6189 (October 3, 2008). 

21 CMS Quality Improvement Roadmap. Available online at 

http://www.cms.hhs.gov/CouncilonTechInnov/downloads/qualityroadmap.pdf. Accessed on 

June 30, 2009. 

22 FY 2009 Final IPPS Rule, 73 Fed. Reg. 48434, 48471 (Aug. 19, 2008); CMS, “Roadmap 

for Implementing Value Driven Healthcare in the Traditional Fee-for-Service Program,” 

pp. 10-12. 

23 FY 2009 Proposed IPPS Rule, 73 Fed. Reg. 23528, 23561 (Apr. 30, 2008). 

24 OIG, Adverse Events in Hospitals: Case Study of Incidence Among Medicare 

Beneficiaries in Two Counties, OEI-06-08-00220, December 2008. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 4 
 

INTRODUCTION INTRODUC T ION 

events that resulted in harm during their hospital stays.25 The case 

study was based on a random sample of 278 Medicare beneficiary 

hospitalizations selected from all Medicare discharges from acute care 

hospitals in two selected counties during a 1-week period in August 

2008. We calculated the estimated adverse event incidence rate as the 

percentage of Medicare beneficiaries with at least one qualifying 

adverse event. 

Qualifying adverse events met one or more of the following criteria: 

1. included in NQF’s list of Serious Reportable Events; 

2. included in Medicare’s list of hospital-acquired conditions; and 

3. resulted in a level of patient harm determined by physician 

reviewers to be associated with a prolonged hospital stay, 

permanent harm, life-sustaining intervention, or death. 

Further, we found that another 15 percent of beneficiaries experienced 

less serious occurrences that resulted in harm that was temporary but 

required intervention, referred to as “temporary harm events” in the 

case study.26 For the purposes of this report, we refer to all occurrences 

that harmed patients (both adverse events and temporary harm events) 

as “events.” The case study identified a total of 120 events that affected 

sample beneficiaries. 

Methods for Identifying Events in the Case Study 

We used a two-stage review process to identify events in the case study. 

The first stage consisted of screening for possible events using the 

following five screening methods: nurse reviews of medical records, 

interviews of Medicare beneficiaries, two types of billing data analysis, 

and reviews of internal hospital incident reports. If any screening 

method identified a possible event, it was labeled a “flag” and the 

medical record proceeded to the second stage of review. The second 

stage consisted of physician reviews of those medical records for which 

at least one of the screening methods indicated that an event had 

possibly occurred. 

First-Stage Review: Screening Methods

 Method 1: Nurse Reviews. Contracted registered nurses reviewed 

medical records for each sampled Medicare beneficiary’s 

25 Case study, p. 10. 

26 Case study, p. 16. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 5 
 

INTRODUCTION 

hospitalization. Nurses used a standardized review process developed 

by the Institute for Healthcare Improvement (IHI) as part of its 

Global Trigger Tool (GTT) protocol, which we adapted for use in the 

case study.27 The nurse review used IHI’s GTT worksheet that listed 

54 “triggers” that could be found within a medical record to indicate 

the possibility of an event. Examples of triggers include transfusions 

or a return to surgery.28 (For a glossary of clinical terms, see 

Appendix C.) When a trigger was found, the nurse reviewer explored 

the medical record further to identify possible events and associated 

level of harm. (See Appendix D for a copy of the nurse review protocol 

and the IHI GTT worksheet.) 

 Method 2: Analysis of POA Indicators. For the case study, we 

obtained administrative billing data directly from hospitals for each of 

the 278 sample Medicare beneficiary hospitalizations.29 We used 

POA indicators in the billing data to identify hospitalizations that 

may have had events. When the POA indicator showed that a 

diagnosis was not present upon admission, we concluded that the 

condition developed during the hospital stay and might have been the 

result of an event.30 For example, a diagnosis code for acute renal 

failure that was not present upon admission may be a sign that an 

event occurred during the hospital stay. 

 Method 3: Beneficiary Interviews. We conducted telephone 

interviews with 220 of the 278 Medicare beneficiaries or their family 

members to learn about the medical care experienced during sampled 

hospitalizations.31 The interview protocol was designed to determine 

27 For more information about the IHI GTT protocol, see F.A. Griffin and R.K. Resar, IHI 

Global Trigger Tool for Measuring Adverse Events, IHI Innovation Series white paper. 

Cambridge, MA: Institute for Healthcare Improvement Innovation Series 2007. 

28 The case study used the 2007 version of the IHI GTT worksheet. A revised version 

with 53 triggers was published in 2009. Available online at 

http://www.ihi.org/IHI/Results/WhitePapers/IHIGlobalTriggerToolWhitePaper.htm. 

Accessed June 15, 2009. 

29 Hospital administrative billing data may include up to 30 diagnosis codes with POA 

indicators. In contrast, Medicare claims data submitted by hospitals to CMS for payment 

are limited to nine diagnosis codes with POA indicators. 

30 The ICD-9-CM system assigns diagnoses and procedure codes associated with 

hospitalizations and is maintained jointly by the National Center for Health Statistics 

(NCHS) and CMS. NCHS, The International Classification of Diseases, 9th Rev., Clinical 

Modification (ICD-9-CM), 6th Edition, was issued for use beginning October 1, 2007. 

31 The remaining 58 beneficiaries or family members either could not be reached or 

declined to be interviewed. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 6 
 

INTRODUCTION 

whether beneficiaries experienced any episodes while in the hospital 

that might have involved events. It also included questions about 

such topics as medications, procedures, infections, and falls. 

Interviews typically lasted 5–7 minutes, but often took longer when 

interviewees reported occurrences and the interviewer probed for 

details. (See Appendix E for a list of interview questions.) 

 Method 4: Hospital Incident Reports. We requested that hospitals 

provide any internal incident reports, such as submissions to any 

hospital incident-reporting systems, adverse drug reaction reports, 

complaints, peer reviews, and mortality and morbidity reviews 

associated with the 278 sample Medicare beneficiary hospitalizations. 

Reports provided by hospitals included issues related to risk 

management, hospital infections, surgical management, and others. 

 Method 5: Analysis of Patient Safety Indicators. We applied AHRQ’s 

Patient Safety Indicator (PSI) software program to hospital 

administrative billing data for the 278 sample Medicare beneficiary 

hospitalizations. AHRQ developed the PSI software to monitor health 

care quality using administrative data, such as patient demographics 

(e.g., age, gender), and diagnoses and procedure codes.32 The PSI 

software is based upon a series of algorithms that detect 20 providerlevel complications that indicate possible events (e.g., death of a lowrisk patient).33 (See Appendix F for a list of provider-level PSIs.) 

Flags. As mentioned, each time a screening method indicated the 

possibility that an event occurred during the hospitalization, we 

designated the possible event as a “flag.” For example, if a nurse review 

indicated that a patient contracted an infection during the sampled 

hospitalization, we considered that a flag. If the nurse review also 

indicated that the same patient fell during the hospital stay, the fall 

was considered another flag for the same patient. Similarly, possible 

events identified through any of the other screening methods were all 

considered flags. Thus, each sampled hospitalization could have 

multiple flags from one or more screening methods. Medical records for 

32 AHRQ, Guide to Patient Safety Indicators, AHRQ Quality Indicators. March 2003. 

Available online at http://www.qualityindicators.ahrq.gov/archives/psi/psi_guide_v30.pdf. 

Accessed on June, 4, 2008. 

33 AHRQ also has seven “area-level” PSIs designed to indicate possible events on a 

regional level, which were not used for the case study. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 7 
 

INTRODUCTION 

hospitalizations with one or more flags were forwarded to the second 

stage of review (i.e., physician review of medical records). 

Second-Stage Review: Physician Review of Medical Records

During the second stage of the case study review, 3 contracted 

physicians reviewed medical records for 183 of the 278 beneficiary 

hospitalizations—those with at least 1 flag. The physicians reviewed 

both the full medical records and the information concerning each flag. 

Physicians completed a structured medical review protocol through 

which they described each incident that had been flagged in the firststage review and noted what documentation led to flagging the incident. 

The physicians then determined whether the incidents flagged in the 

first-stage review qualified as events for purposes of our study. (See 

Appendix G for a copy of the Protocol for Physician Reviews of Medical 

Records.) 

To ensure consistency among the three physician reviewers, we 

conducted weekly conference calls to discuss issues and reach consensus 

about complex cases. Additionally, after physicians initially identified 

122 events, we gave the associated hospitals an opportunity to provide 

additional information or documentation. We received 18 hospital 

submissions; the additional information led the physicians to change 

their determinations for 2 cases and reduce the number of events to 120. 

Only events that were identified and confirmed by physicians were 

included in the case study results. 

METHODOLOGY 

Scope 

This study provides an in-depth examination of the usefulness of the 

five screening methods for identifying events in the case study. The 

case study was based on a random sample of 278 Medicare beneficiary 

hospitalizations selected from all Medicare discharges from acute care 

hospitals in 2 selected counties during a 1-week period in August 2008. 

Data Collection and Analysis 

To evaluate the usefulness of the methods for identifying events, 

we compared all flags generated by each method to the 120 events 

identified and/or confirmed through physician reviews. We classified 

each flag as either “associated with an event” (i.e., it provided 

information that led the physician to identify an event) or “not 

associated with an event” (i.e., the possible event did not meet our 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 8 
 

INTRODUCTION 

criteria to be classified as an event). We then calculated the number 

and percentage of flags associated and not associated with events. We 

considered the most useful methods to be those that identified the 

greatest number of events. We also examined the types of events 

identified through each method and determined which events had no 

associated flags (i.e., events identified only through physician reviews of 

the medical records). 

Limitations 

Several aspects of the case study warrant mention because of possible 

implications and limitations for this report. First, results of the case 

study can be projected only to the population of Medicare beneficiaries 

hospitalized in the two selected counties during a 1-week period in 

August 2008. Second, it is possible that the case study underestimated 

incidence of adverse events in the sample. Specifically, the two-stage 

methodology meant that physicians conducted medical record reviews 

for only 183 of the 278 Medicare beneficiary hospitalizations. Therefore, 

it is unknown whether the 95 hospitalizations with no flags had any 

events. Third, the case study included beneficiary interviews for only 

220 of the 278 sampled hospitalizations. It is unknown whether 

interviews with the other 58 beneficiaries would have identified 

additional flags or events. Fourth, it is possible that hospitals did not 

provide all incident reports associated with the Medicare beneficiaries’ 

hospitalizations. Finally, because data collection occurred soon after the 

sample beneficiary hospitalizations, staff from a few hospitals indicated 

that the hospital billing data had not undergone complete data cleaning 

and validation. We do not know whether this additional processing by 

hospitals would have affected any screening results. 

Standards 

This study was conducted in accordance with the Quality Standards for 

Inspections approved by the Council of the Inspectors General on 

Integrity and Efficiency.

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 9 
 

The five screening methods were useful in 

identifying adverse events; however, most flags 

were not associated with events 

 FINDINGS 

The 5 screening methods used in 

the case study generated a total of 

662 flags (i.e., possible events). 

(See Figure 1.) Physician reviews 

determined that 256 of the 662 flags (39 percent) were associated with 

events and 406 of the 662 flags (61 percent) were not associated with 

events. Because more than 1 flag identified many events, the 256 flags 

revealed a total of 114 events. Physicians identified an additional 

6 (of the 120) events in their medical record reviews, finding events 

within cases that were flagged for other reasons. (See Appendix H for 

an expanded graphic of the results of the screening methods.) 

All Flags Identified 

in Case Study

(n = 662)

Flags Not Associated 

With Events 

(n = 406)

Flags Associated 

With Events (n = 256)

Unique Events Identified 

Through Screening 

Methods

(n = 114)

Events Identified in 

Case Study

(n = 120)

Figure 1: Flags and Events Identified Through Case Study 

Screening Methods

Source: OIG analysis of 278 Medicare beneficiaries’ hospitalizations in 2 selected counties, 2008.

Events Identified 

Only Through Physician 

Reviews

(n = 6)

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 10 
 

FINDINGS 

Nurse reviews and POA analysis identified the greatest number of events 

Nurse reviews identified 93 of the 120 events in the case study and 

POA analysis identified 61 events. (See Table 1.) The third most 

useful screening method, beneficiary interviews, identified 22 events, 

and the remaining 2 screening methods identified 8 events each. 

Table 1: Number and Percentage of Events Identified 

Through Each Screening Method 

Method Number of Events* Percentage of 

120 Events 

Nurse Reviews 93 78% 

POA Analysis 61 51% 

Beneficiary Interviews 22 18% 

Hospital Incident Reports 8 7% 

PSI Analysis 8 7% 

Source: OIG analysis of 278 Medicare beneficiary hospitalizations in 2 selected counties, 2008. 

*Column does not sum to 120 because many events were identified through more than 1 method. 

The usefulness of nurse reviews and POA analysis appears attributable 

to both the breadth and precision of each of these methods. Both were 

broad, in that nurse reviews examined all documents in the medical 

records and POA analysis considered up to 30 diagnosis codes 

representing patient conditions. This allowed for a review of a range of 

possible events unlike, for example, a hospital incident report related 

specifically to a medication error. Both methods were also precise. The 

nurse review protocol provided a clinical description of the health care 

condition; the ICD-9-CM codes used for POA analysis specified each 

patient diagnosis. This specific clinical information later enabled the 

physicians to more easily pinpoint related information in the medical 

records. 

The usefulness of nurse reviews and POA analysis is further 

demonstrated by their performance in identifying events not flagged by 

any other screening methods. Of the 120 events, 55 (46 percent) were 

identified by only 1 screening method. Nurse reviews identified 

35 events not flagged by any other screening method, (29 percent of the 

120 events) and POA analysis alone flagged 14 events (12 percent). 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 11 
 

FINDINGS 

Although the five screening methods were useful in identifying events, 

most flags were not associated with events 

As reported on page 11, 406 of the 662 flags generated by the screening 

methods were not associated with any of the 120 events identified in the 

case study. The POA analysis generated the most flags that were not 

associated with events (183 flags) and PSI analysis generated the fewest 

(4 flags). (See Figure 2.) 

183

123

50 46

4

113

104

21

9 9

0

50

100

150

200

Flags Not Associated With Events

Flags Associated With Events

POA 

Analysis

(n = 296)

Nurse 

Reviews

(n = 227)

Beneficiary 

Interviews

(n = 71)

Hospital 

Incident 

Reports

(n = 55)

PSI 

Analysis

(n = 13)

Source: OIG analysis of 278 Medicare beneficiary hospitalizations in 2 selected counties, 2008.

Note: Some flags were associated with more than one event, and some events were identified through more than one flag.

Figure 2: Number of Flags Not Associated With Events and Associated With Events by 

Screening Method

We noted several possible explanations for the large number of flags 

that were not associated with events: 

 First, physician reviews determined that many flags represented 

conditions or occurrences that did not cause harm to the patients. For 

example, six hospital incident reports described beneficiary falls that, 

according to the medical record, did not result in injury or require 

additional treatment. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 12 
 

FINDINGS 

 Second, physician reviews determined that some flags represented 

occurrences that were part of the natural course of the patients’ 

disease or medical condition. In one such case, a flag from the nurse 

review indicated that a beneficiary experienced postoperative atrial 

fibrillation. However, the physician review determined that this was 

not an event because the beneficiary’s underlying medical condition 

likely led to the atrial fibrillation, rather than care provided during 

the hospitalization. 

 Third, in some cases, physician reviews did not find supporting 

documentation in the medical records to determine that flags were 

associated with events. For example, a beneficiary interview included 

a report of pressure ulcers, yet the physician reviewer did not find 

pressure ulcers documented in the medical record and therefore could 

not determine that an event occurred. 

 Finally, to ensure that physicians reviewed medical records with any 

chance of an event, we defaulted to including all flags, even when the 

screening method suggested only a slight possibility of harm. For 

example, physicians reviewed all cases for which the hospitals 

provided associated incident reports, even if the reports did not 

initially appear to be related to patient harm. 

Although some events were flagged by one screening method, other 

events were flagged by multiple methods, especially in cases in which 

the patients had several complicating health care conditions. For 

example, one of the most complex hospitalizations involved a patient 

who died during the hospitalization. Even though the family declined to 

be interviewed, the 4 remaining screening methods generated a total of 

29 flags. Physician reviews determined that the patient experienced 

4 events. One of the four events was a “cascade” with 19 associated 

flags.34 This cascade included several events, such as deep vein 

thrombosis, respiratory failure, renal failure, and sepsis. Two other 

events were hospital-acquired infections, which were both flagged by 

nurse reviews and POA analysis. The fourth event involved kidney 

damage, which was flagged by nurse review and POA analysis. Overall, 

13 

34 IHI defines cascade event as one in which an initial event causes a series of additional, 

related events for the same patient, and advocates collapsing these series into single events 

in determining event counts. F.A. Griffin and R.K. Resar, IHI Global Trigger Tool for 

Measuring Adverse Events, IHI Innovation Series white paper. Cambridge, MA: Institute 

for Healthcare Improvement Innovation Series 2007, p. 11. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 
 

FINDINGS 

physician reviews determined that 26 of the 29 flags were associated 

with events, and the remaining 3 flags were not associated with events. 

For more details on each screening method, see page 21. 

Shortcomings in two screening methods have 

implications for Medicare payments and Federal 

initiatives to identify, track, and monitor events 

For the Medicare program and 

other entities to learn of events 

that harm patients, they must 

receive accurate and complete 

information from hospitals. Two critical sources of such information are 

hospital billing data (diagnosis codes and associated POA indicators) 

and hospital incident reports. Our analysis of data from these sources 

revealed vulnerabilities regarding both accuracy and completeness, 

which could result in Medicare overpayments and inhibit initiatives to 

improve patient safety. 

POA analysis revealed problems that could inhibit CMS’s ability to identify 

hospital-acquired conditions and appropriately deny Medicare payments 

In the case study, we found that only 4 of the 11 events involving 

Medicare hospital-acquired conditions could be identified through the 

POA indicators contained in billing data.35 As previously stated, 

Federal regulations require that hospitals submit with each diagnosis 

code on a Medicare claim a POA indicator designating whether the 

condition was present upon admission. The ICD-9-CM codes and POA 

indicators submitted by hospitals enable Medicare’s automated payment 

processing software to identify claims with hospital-acquired conditions, 

enabling denial of payment for associated care and also providing 

valuable information for use in monitoring quality of care in hospitals. 

Among the seven hospital-acquired conditions found in the case study 

that POA analysis did not flag, five had no related ICD-9-CM code in 

the hospital billing data.36 Thus, Medicare automated payment 

processing software could not determine that a hospital-acquired 

condition had occurred. Although the hospital would not receive 

additional payment for care associated with the conditions (the desired 

14 

35 Medicare’s payment policy for hospital-acquired conditions was not yet in effect at the 

time of the case study, and therefore these findings do not indicate that Medicare made any 

improper or incorrect reimbursements for claims associated with these hospital-acquired 

conditions. 

36 It was beyond the scope of the case study to determine the underlying causes for these 

absent or inaccurate codes. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 
 

FINDINGS 

outcome of the Medicare payment policy), the absence of these codes 

inhibits the usefulness of billing data for gauging how often such events 

occur and for monitoring quality of care in hospitals. 

In the two remaining cases, hospital billing data contained diagnosis 

codes related to the hospital-acquired condition, but the codes were not 

specific enough to invoke the Medicare payment policy. In each of these 

two cases, physician reviews determined that the beneficiary 

experienced a “catheter-associated urinary tract infection,” yet the 

billing data included a more general diagnosis code for “urinary tract 

infections, not otherwise specified.” Although these less descriptive 

diagnosis codes led physicians to identify the events in these cases, they 

would not have allowed Medicare’s automated payment-processing 

software to identify the hospital-acquired conditions and, therefore, 

could have resulted in Medicare overpayments to the hospitals if the 

payment policy had been in effect at the time of the case study. Such 

discrepancies would also affect use of Medicare claims and POA 

indicator data to monitor events and quality of care in hospitals. 

The lack of hospital incident reports could prevent hospitals from tracking 

events as required or reporting events to outside entities 

Identifying events within hospitals and capturing event information in 

hospital incident reports are critical first steps toward addressing 

problems that lead to patient harm. We requested that hospitals 

provide any internal incident reports involved with the hospitalization, 

including submissions to the hospital incidence-reporting systems, 

adverse drug reaction reports, complaints, peer reviews, and morbidity 

and mortality reviews, for the 278 sample Medicare beneficiary 

hospitalizations. However, hospitals did not provide, and apparently 

did not have, any reports for 112 of the 120 events (93 percent) found in 

the case study. Further, hospitals had no incident reports for two of the 

three events that resulted in death to the patients or two of the four 

events that resulted in serious disability. 

The implications of hospitals’ failure to identify and capture event 

information can be significant. First, although we did not assess 

hospital compliance with Federal requirements to “track medical errors 

and adverse patient events,” it raises concerns that only four of the 

seven most serious events had no associated hospital incident reports. 

Further, the lack of incident reports for 93 percent of events suggests 

that hospital incident-reporting systems may be an unreliable source of 

information for PSOs, States operating adverse event reporting 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 15 
 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 16 

FINDINGS 

systems, and other entities. These entities often seek to learn from the 

combined experiences of many hospitals to generate lessons to improve 

patient safety. Unless events are reported within the hospital first, the 

event information is unlikely to be available to outside entities for 

learning.
 

FINDINGS RECOMMENDATIONS 

 

Identifying events that harm hospital patients is a topic of interest to 

Congress, HHS, and the health care community and is important to 

reduce their occurrence. This report found that, combined, the selected 

screening methods used for the case study were useful in identifying 

events. However, we also identified weaknesses in their use; physicians 

determined that most flags generated by the five screening methods 

were not associated with events. Further, we found that shortcomings 

revealed within two screening methods have implications for Medicare 

payments and Federal initiatives to identify, track, and monitor events. 

Overall, the case study findings suggest that an effective way to identify 

events that result in harm to patients is through review of medical 

records by nurses and/or physicians, methods that are both costly and 

time consuming. Other less costly and time-consuming methods, such 

as analyzing billing data or examining hospital incident reports, would 

identify far fewer events. Nonetheless, there is no current mandate for 

HHS to conduct medical reviews to identify events. 

Therefore, we recommend that: 

CMS and AHRQ should explore opportunities to identify events when 

conducting medical record reviews for other purposes 

CMS and AHRQ could reduce the cost of efforts to identify adverse 

events by adding such efforts to ongoing medical record reviews. 

Examples of such efforts include, but are not limited to: CMS’s 

Medicare Comprehensive Error Rate Testing (CERT) and the work of 

Quality Improvement Organizations (QIO); AHRQ grant awards for 

patient safety research; and State survey agency work in certifying 

hospital compliance with the Medicare conditions of participation. 

CMS should ensure that hospitals code claims accurately and completely to 

allow for identification of hospital-acquired conditions affected by 

Medicare’s payment policy 

CMS relies on hospitals to code diagnoses and POA indicators to identify 

Medicare hospital-acquired conditions subject to the payment policy. 

However, we found that ICD-9-CM codes were inaccurate or absent for 

7 of the 11 events that involved Medicare hospital-acquired conditions 

found in the case study. Because the payment component of the Medicare 

hospital-acquired conditions policy was not in effect at the time of the case 

study, this problem did not result in improper payments to any providers. 

However, given that the payment policy is now in effect, similar problems 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 17 
 

RECOMMENDATIONS 

could result in improper payments. Further, incorrect or absence of 

diagnosis codes would prevent CMS from determining that the events 

occurred, which would inhibit its ability to measure rates of hospitalacquired conditions and limit public reporting of the measured rates. 

CMS should determine whether additional guidance or other action is 

needed to ensure that hospitals code diagnoses and POA indicators 

accurately and completely. We are conducting further research examining 

the accuracy of claims related to hospital-acquired conditions for a 

nationally representative sample of Medicare beneficiaries’ 

hospitalizations. 

CMS should provide interpretive guidelines for State survey agencies to 

assess hospital compliance with requirements to track and monitor adverse 

events 

Although hospitals must “track medical errors and adverse patient 

events” under Medicare’s conditions of participation, few events found 

in the case study had associated hospital incident reports. The State 

Operations Manual, through which CMS provides guidance to State 

survey agencies that assess hospital compliance with Federal 

regulations, contains no interpretive guidelines regarding these 

requirements.37 Therefore, when State agency staff perform surveys of 

hospitals (e.g., standard compliance surveys and surveys based on 

complaints), it is unclear how surveyors are to assess hospital 

operations for tracking medical errors and other events that result in 

harm to patients. CMS should provide guidance to surveyors for 

assessing hospitals’ compliance with these requirements. Such 

guidance could include what types of medical errors and adverse patient 

events hospitals should track and monitor, as well as what information 

should be captured in hospital incident reports. 

AHRQ should inform PSOs that internal hospital incident reporting may be 

insufficient to provide needed information about events to PSOs 

PSOs were established to aggregate and analyze events that pose risks 

to patient safety. AHRQ indicated that it expects PSOs will rely 

primarily upon existing hospital infrastructures for identifying, 

tracking, and reporting information about events to PSOs. However, we 

found that hospital incident reports existed for only 8 of the 120 events 

37 CMS, State Operations Manual, 2009, Appendix A, A-0267; A-0286 through A-0288. 

Available online at http://www.cms.hhs.gov/manuals/downloads/som107ap_a_hospitals.pdf. 

Accessed on September 1, 2009. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 18 
 

RECOMMENDATIONS 

identified in the case study. In providing technical assistance to 

support PSOs, AHRQ should convey the importance of strong hospital 

internal incident-reporting procedures that capture relevant 

information about events that cause harm to patients. 

AGENCY COMMENTS AND OFFICE OF INSPECTOR GENERAL 

RESPONSE 

We received comments on our draft of this report from AHRQ and CMS. 

AHRQ concurred with the report as written. CMS stated that it 

appreciates the original contribution that the report makes in 

advancing the patient safety agenda in hospitals and agrees with the 

recommendations made to CMS. We made no changes to the report 

based on agency comments. 

In its comments, CMS outlined several opportunities to identify events 

when conducting medical record reviews for other purposes. First, CMS 

noted that it expects State Survey Agencies to investigate patient 

adverse events while conducting a survey of a hospital’s compliance 

with the Medicare conditions of participation. Second, CMS stated that 

it is increasingly investigating the extent to which adverse events are 

tracked and analyzed by hospital internal quality assurance and 

performance systems. Third, CMS noted that the QIO program 

addresses many quality-of-care concerns through the beneficiary 

complaint and medical review processes or through referral by other 

entities. Regarding QIOs, CMS indicated that it has proposed policy 

revisions to the QIO beneficiary complaint and quality-of-care review 

process to include adverse events as indicators of quality and will 

explore a possible quality-referral mechanism between the CERT 

contractor and the QIO program. Lastly, CMS stated that it is 

collaborating with AHRQ on the Medicare Patient Safety Monitoring 

System to estimate prevalence of certain conditions and adverse events. 

CMS indicated that to ensure that hospitals accurately code claims to 

enable identification of hospital-acquired conditions, it has established a 

process for providing coding advice to hospitals. Further, CMS 

indicated that it will study the accuracy of coding hospital-acquired 

conditions and reporting of POA indictors to use in developing 

refinements in coding instructions or to focus review on areas of 

inaccurate reporting. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 19 
 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 20 

RECOMMENDATIONS 

CMS indicated that it will ensure that the State Operations Manual

includes full guidance for surveyors to assess hospital QAPI systems 

and strengthen CMS surveyor-training programs to enhance surveyor 

abilities to evaluate compliance with the QAPI requirements.

For the full text of AHRQ and CMS comments, see Appendix I.
 

 SCREENING METHODS 

The following section provides specific results for each of the screening 

methods used for identifying events in the case study, as well as factors 

that appeared to affect their usefulness. 

Method 1: Nurse Reviews of Medical Records

Nurse reviews identified the most events among the screening methods and 

also generated many flags that were not associated with events 

Nurse reviews generated 227 flags across 127 Medicare beneficiaries: 

1 flag each for 70 beneficiaries and more than 1 flag for the remaining 

57 beneficiaries. In an unusually complex case, the nurse reviewer 

identified 11 flags. Because the nurse review protocol required a broadbased examination of the medical record, nurses identified flags 

associated with a variety of conditions and occurrences. The most 

prominent types of flags related to surgery and other procedures, 

medication, and infections. (See Table 2.) 

Table 2: Number of Flags and Events Identified by Nurse 

Reviews of Medical Records 

Number of Flags 

Type of Conditions or Occurrences Identified by 

Method 

Associated 

With Events 

Surgery and other procedures 58 34 

Medication 51 29 

Infections 45 20 

Skin care 19 11 

Patient care 9 5 

Glycemic control 8 3 

Patient fall 7 1 

Readmission 7 0 

Intravenous fluid 4 2 

Use of restraints 4 0 

Other 15 0 

 Total 227 104* 

Source: OIG analysis of 278 Medicare beneficiary hospitalizations in 2 selected counties, 2008. 

*Column will not sum to 104 because some flags were associated with more than 1 event and some events 

were identified through more than 1 flag. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 21 
 

SCREENING METHODS 

Physician reviews determined that 104 of the 227 flags (46 percent) 

identified by nurse reviews were associated with events. Because more 

than 1 flag was associated with some of the events, this method 

identified a total of 93 events (78 percent of the 120 events in the case 

study). Physician reviews determined that 123 of the 227 flags 

(54 percent) generated by nurse reviews were not associated with 

events. These flags were typically not associated with events because 

the condition or occurrence flagged did not involve harm to the patient 

or was related to the patient’s underlying disease or medical condition 

or because the medical record did not contain sufficient documentation 

to determine that an event occurred. 

Method 2: Analysis of Present on Admission Indicators

POA analysis generated the most flags among the five methods and 

identified about half of the events 

To identify diagnoses that were not coded as present on admission and 

therefore presumed to be hospital acquired, POA analysis of hospital 

billing data considered up to 30 ICD-9-CM codes and POA indicators for 

each case. This analysis generated 296 flags across 98 Medicare 

beneficiaries: 1 flag for 40 beneficiaries and more than 1 flag for the 

remaining 58 beneficiaries. The method generated 10 or more flags for 

each of 4 beneficiaries who had very complex hospitalizations. 

The 296 flags included 155 different ICD-9-CM codes across the 

98 beneficiaries. Of these 155 codes, 95 were flagged once (i.e., for only 

1 beneficiary). The most frequently represented code, anemia because 

of blood loss, was flagged for 16 different beneficiaries. (See Table 3.) 

Physician reviews determined that 113 of the 296 flags (38 percent) 

identified through POA analysis were associated with events. Because 

more than 1 flag was associated with some of the events, the method 

identified a total of 61 events (51 percent of the 120 events in the case 

study).38 However, POA analysis generated the largest number flags 

38 POA analysis would have identified 34 percent fewer events (21 of 61 events) if we 

had used Medicare claims data rather than billing data obtained directly from hospitals. 

For these 21 events, either there were no Medicare claims for the hospitalizations or the 

relevant diagnosis codes were not in the Medicare claims. (Medicare claims data include 

only the first nine diagnosis codes and POA indicators for each hospitalization.) This 

suggests that researchers who cannot obtain billing data directly from hospitals may 

identify a smaller percentage of adverse events than suggested by the case study results. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 22 
 

SCREENING METHODS 

that were not associated with events (183) among the screening 

methods (62 percent of the 296 POA analysis flags). For POA flags that 

were not associated with events, physician reviews determined that 

they typically did not involve harm to the patient or were related to the 

patients’ diseases or medical conditions or that the medical records did 

not contain documentation to determine that events occurred. For 

example, physician reviews often did not find accompanying patient 

signs or symptoms associated with laboratory tests that caused actual 

harm or required an intervention or prolonging of the hospital stay. 

Physician reviewers reported that the three most frequently occurring 

POA flags—anemia because of acute blood loss, potassium deficiency, 

and acute renal failure—each represented temporary laboratory 

findings common during care for complex patients. 

Table 3: Seven Most Frequently Occurring Flags With Diagnoses 

Not Coded as Present on Admission 

Number of Flags ICD-9-CM 

Code Description Identified by 

Method 

Associated 

With Events 

285.1 Anemia because of acute blood loss 16 2 

276.8 Potassium (K) deficiency 12 0 

584.9 Acute renal failure 10 4 

48.6 Other enterovirus diseases of central 

nervous system 6 4 

511.9 Unspecified pleural effusion 6 1 

707.03 Decubitus ulcer 5 5 

997.1 Cardiac complications 5 1 

Source: OIG analysis of 278 Medicare beneficiary hospitalizations in 2 selected counties, 2008. 

Method 3: Beneficiary Interviews

Beneficiary interview was the only method to rely on information from 

outside the hospitals and identified 21 percent of events 

Beneficiary interviews identified 70 flags across 42 Medicare 

beneficiaries: 1 flag each for 25 beneficiaries and more than 1 flag for 

each of the remaining 17 beneficiaries. Two beneficiaries each reported 

five different flags, the most reported during interviews. Flags from 

interviews covered a smaller range of conditions and occurrences than 

nurse reviews or POA analysis. (See Table 4.) 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 23 
 

SCREENING METHODS 

Table 4: Number of Types of Flags and Events Identified 

Through Beneficiary Interviews 

 Number of Flags 

Type of Condition or Care Identified by 

Method 

Associated 

With Events 

Medication 18 8 

Skin care 14 3 

Hospital-acquired infections 13 3 

Surgery and other procedures 12 6 

Patient falls 4 1 

Intravenous volume overload 1 1 

Other general care complaints 8 0 

 Total 70 21* 

Source: OIG analysis of 220 Medicare beneficiaries’ interviews. 

*Column does not sum to 21 because 1 flag was associated with 2 events. 

During interviews, beneficiaries and their representatives revealed both 

specific and general concerns. Among the 70 flags, 62 involved specific 

health care circumstances (e.g., medication problems, infections). For 

the other eight flags, interviews gleaned general complaints about the 

quality or outcome of care. For example, one beneficiary indicated that 

his mobility changed shortly after the hospital stay. To ensure that we 

did not miss events, we considered each of these 8 as flags and 

forwarded them to physicians for medical record reviews, along with the 

62 specific flags. 

Several reasons may explain why beneficiary interviews identified 

relatively few flags as compared to some of the other methods. 

Although we conducted interviews within about 1 month of their 

hospitalizations, a number of beneficiaries had difficulty recalling their 

experiences, although some remembered more as the interview 

proceeded. For example, during the initial series of general questions, 

several respondents indicated that they had no problems, yet later 

identified problems when asked about specific topic areas such as 

medication or falls. Respondents who had more than one recent 

hospital stay often had difficulty distinguishing during which stay 

problems occurred. Although assured that information would be kept 

confidential, a few beneficiaries expressed concern that hospitals would 

disapprove of their participating in the interviews. Finally, this method 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 24 
 

SCREENING METHODS 

generated no flags for the 58 sample beneficiaries that we could not 

contact or declined to be interviewed. 

Despite these barriers, physician reviews determined that 21 of the 

70 flags (30 percent) generated from beneficiary interviews were 

associated with events. One event that was flagged through an 

interview but missed by the other screening methods demonstrates the 

usefulness of beneficiary interviews in learning about events. In this 

case, the beneficiary experienced an allergic reaction determined by the 

physicians to be an event. The beneficiary indicated in the interview 

that hospital staff had been informed in advance of the allergy. 

Method 4: Hospital Incident Reports

Although few hospital incident reports were associated with events, these 

were useful in understanding what transpired 

Counting each incident report as a flag, this method identified 55 flags 

across 40 Medicare beneficiaries. Hospitals submitted a single incident 

report each for 29 of these beneficiaries and multiple reports for each of 

the other 11 beneficiaries. In a few of the cases with multiple reports, 

the reports were related to the same condition or occurrence, with initial 

reports and then subsequent reports detailing further analysis and 

outcomes. In the other cases, the multiple reports indicate that the 

beneficiaries experienced multiple unrelated occurrences. 

We grouped the 55 incident reports into 3 distinct categories: patient 

incidents (29 reports), systems incidents (21 reports), and nonmedical 

incidents (5 reports). (See Table 5.) Each patient incident report 

involved a medical condition or occurrence specific to one of the sample 

Medicare beneficiaries. The most common of these reports involved 

adverse medication reactions. The systems incident reports also related 

to particular patients, but the reports described incidents related to 

hospital systems rather than the patients’ conditions. The most 

common hospital systems incident report addressed medication 

administration errors that, despite involving errors, did not cause harm 

to the patients. Finally, nonmedical incident reports were unrelated to 

the health care provided to the patients, such as two reports about 

stolen property. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 25 
 

SCREENING METHODS 

Table 5: Hospital Incident Reports by Category 

Number of Flags 

Category of Hospital Incident Reports 

Identified 

by Method 

Associated 

With Events 

PATIENT INCIDENTS 29 9 

Adverse medication reactions 11 3 

Skin care-related incidents 7 3 

Falls 6 0 

Other patient incidents 5 3 

SYSTEMS INCIDENTS 21 0 

Medication administration errors 7 0 

Hospital policy-driven reports* 6 0 

Other systems incidents 5 0 

Staff care incidents 3 0 

NONMEDICAL INCIDENTS 5 0 

 Total 55 9 

Source: OIG analysis of 278 Medicare beneficiary hospitalizations in 2 selected counties, 2008. 

*These reports were generated because of hospital policies that called for such reports whenever specific 

circumstances occurred, such as an unplanned readmission of a patient. 

Physician reviews determined that 9 of the 55 flags (16 percent) were 

associated with events. (The method identified eight unique events 

because one event had two associated incident reports.) The events 

involved three adverse medication reactions, two skin care events, two 

surgical complications, and one blood-clotting event. The eight events 

involved only reports that we categorized as patient incident reports, 

indicating that reports related to hospital systems and nonmedical 

incidents were not useful for identifying events. 

The detail included in the hospital incident reports was useful in 

understanding what transpired. These reports typically described the 

nature and types of incidents and their effect on patients. Some reports 

also indicated whether the events were reportable to other entities, such 

as States operating adverse event reporting systems. More serious 

events tended to require reporting to other entities and sometimes 

resulted in highly detailed and/or multiple reports. For example, for 

one beneficiary, who experienced multiple complications following 

surgery, the hospital generated three separate incident reports: a 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 26 
 

SCREENING METHODS 

report describing a hospital-acquired infection that led to sepsis, a 

report describing the patient’s return to surgery for a bowel obstruction, 

and a more general report about hypotension and confusion. 

The low number of events identified through hospital incident reports 

(8), and the relatively high number of incident reports not related to 

events (46), suggest a disconnect between the hospitals’ purpose for 

internal incident-reporting systems and the goal of identifying events 

that result in patient harm. Reports often referred to incidents with 

hospital systems and nonmedical issues that could be useful for hospital 

administration purposes, but the incidents did not involve harm to 

patients. 

Method 5: Analysis of Patient Safety Indicators

PSI analysis identified the fewest flags among the screening methods and 

did not identify some events related to conditions covered by PSI software 

PSI analysis identified 13 flags across 11 Medicare beneficiaries, the 

lowest number of flags among the screening methods used in the case 

study.39 (See Table 6.) This low number of flags is, in part, because 

PSI software is limited to only 20 health care conditions.40 The most 

frequently occurring condition with a PSI was pressure ulcer (five flags). 

Physician reviews determined that 9 of the 13 flags (69 percent) 

generated by PSI analysis were associated with events. (The method 

identified eight events because one of the events had two associated 

PSIs.) Although this method was one of the methods that identified the 

fewest events among the screening methods, it also had the fewest flags 

that were not associated with events (four). 

39 The PSI analysis identified 1 flag for 10 of these 11 beneficiaries and 3 flags for the 

other beneficiary. 

40 Additionally, 4 of these 20 PSIs involved obstetrics and were therefore unlikely to 

affect the Medicare beneficiary population included in the case study. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 27 
 

SCREENING METHODS 

Table 6: Number of Flags and Events Identified Through 

PSI Analysis 

Number of Flags 

Patient Safety Indicator Identified by 

Method 

Associated 

With Events 

Pressure ulcer 5 5 

Postoperative pulmonary embolism or deep vein 

thrombosis 3 2 

Accidental puncture or laceration 2 1 

Selected infections because of medical care 1 1 

Iatrogenic pneumothorax 1 0 

Complications of anesthesia 1 0 

 Total 13 9 

Source: OIG analysis of 278 Medicare beneficiary hospitalizations in 2 selected counties, 2008. 

Additionally, PSI analysis did not identify as flags some events 

involving health care conditions targeted by the PSI software typically 

because related diagnosis codes were not included in the billing data 

provided to OIG. For example, although PSI software is designed to 

identify pressure ulcers, our PSI analysis identified only 5 of the 

13 pressure ulcers found in the case study. Regarding the other eight 

hospitalizations with pressure ulcers, billing data for seven had no 

ICD-9-CM diagnosis codes indicating that the Medicare beneficiaries 

had pressure ulcers. The remaining case had an ICD-9-CM diagnosis 

code for the pressure ulcer, but the PSI software intentionally invoked 

an exemption because the hospitalization had another complication, 

specifically anoxic brain damage. In the absence of the brain damage, 

PSI analysis would likely have flagged the pressure ulcer. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 28 
 

 APPENDIX ~ A 

Tax Relief and Health Care Act of 2006

P.L. 109-432 

DIVISION B – MEDICARE AND OTHER HEALTH PROVISIONS 

TITLE II—MEDICARE BENEFICIARY PROTECTIONS

SEC 203 OIG STUDY OF NEVER EVENTS 

(a) Study.— 

(1) In general.—The Inspector General in the Department of Health and 

Human Services shall conduct a study on— 

(A) incidences of never events for Medicare beneficiaries, including 

types of such events and payments by any party for such events; 

(B) the extent to which the Medicare program paid, denied payment, or 

recouped payment for services furnished in connection with such events 

and the extent to which beneficiaries paid for such services; and 

(C) the administrative processes of the Centers for Medicare & Medicaid 

Services to detect such events and to deny or recoup payments for 

services furnished in connection with such an event. 

(2) Conduct of study.—In conducting the study under paragraph (1), the 

Inspector General— 

(A) shall audit a representative sample of claims and medical records of 

Medicare beneficiaries to identify never events and any payment (or 

recouping of payment) for services furnished in connection with such 

events; 

(B) may request access to such claims and records from any Medicare 

contractor; and 

(C) shall not release individually identifiable information or facilityspecific information. 

(b) Report.—Not later than 2 years after the date of the enactment of 

this Act, the Inspector General shall submit a report to Congress on the 

study conducted under this section. Such report shall include 

recommendations for such legislation and administrative action, such as 

a noncoverage policy or denial of payments, as the Inspector General 

determines appropriate, including— 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 29 
 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 30 

A PPENDIX ~ A 

(1) recommendations on processes to identify never events and to deny 

or recoup payments for services furnished in connection with such 

events; and 

(2) a recommendation on a potential process (or processes) for public 

disclosure of never events which— 

(A) will ensure protection of patient privacy; and 

(B) will permit the use of the disclosed information for a root cause 

analysis to inform the public and the medical community about safety 

issues involved. 

(c) Funding.— Out of any funds in the Treasury not otherwise 

appropriated, there are appropriated to the Inspector General of the 

Department of Health and Human Services $3,000,000 to carry out this 

section, to be available until January 1, 2010. 

(d) Never Events Defined.— For purposes of this section, the term 

“never event” means an event that is listed and endorsed as a serious 

reportable event by the National Quality Forum as of 

November 16, 2006. 
 

Medicare List of Hospital-Acquired Conditions 

The Centers for Medicare & Medicaid Services (CMS) list of 

hospital-acquired conditions is divided into 10 categories. Effective 

October 1, 2008, CMS no longer pays a higher reimbursement for 

hospitalizations complicated by these categories of conditions that were 

not present upon admission. 

Table B-1: Medicare List of Hospital-Acquired Conditions 

01. Foreign object retained after surgery 

02. Air embolism 

03. Blood incompatibility 

04. Pressure ulcers (stages III and IV) 

05. Falls 

A. Fracture 

B. Dislocation 

C. Intracranial injury 

D. Crushing injury 

E. Burn 

F. Electric shock 

06. Manifestations of poor glycemic control 

A. Hypoglycemic coma 

B. Diabetic ketoacidosis 

C. Nonkeototic hyperosmolar coma 

D. Secondary diabetes with ketoacidosis 

E. Secondary diabetes with hyperosmolarity 

07. Catheter-associated urinary tract infection 

08. Vascular catheter-associated infection 

09. Deep vein thrombosis/pulmonary embolism associated with 

A. Total knee replacement 

B. Hip replacement 

10. Surgical site infection 

A. Mediastinitis after coronary artery bypass graft 

B. Associated with certain orthopedic procedures involving the 

a. Spine 

b. Neck 

c. Shoulder 

d. Elbow 

C. Associated with certain bariatric surgical procedures for obesity 

a. Laprascopic gastric bypass 

b. Gastroenterostomy 

c. Laparoscopic gastric restrictive surgery 

Source: Fiscal Year 2009 Final Inpatient Prospective Payment System Rule, 73 Fed. Reg. 48434, 

48490–48491 (Aug. 19, 2008). 

 APPENDIX ~ B 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 31 
 

A PPENDIX B APPENDIX ~ C 

Glossary of Selected Clinical Terms41 

Acidosis—An abnormal condition of reduced alkalinity of the blood and 

tissues that is marked by sickly sweet breath, headache, nausea and 

vomiting, and visual disturbances and is usually a result of excessive 

acid production. 

Acute renal failure—A sudden loss of the ability of the kidneys to remove 

waste and concentrate urine without losing electrolytes.

Anemia—A condition in which the blood is deficient in red blood cells, in 

hemoglobin, or in total volume.

Anoxic brain damage—Brain damage that occurs when the brain does 

not receive sufficient oxygen.

Atrial fibrillation—Very rapid uncoordinated contractions of the atria of 

the heart resulting in a lack of synchronism between the heartbeat and 

pulse beat. 

Blood clot—A coagulated mass produced by clotting of blood.

Coronary artery bypass graft—Heart bypass surgery performed to route 

blood flow around clogged arteries supplying the heart. 

Deep vein thrombosis—A condition marked by the formation of a 

thrombus within a deep vein (as of the leg or pelvis) that may be 

asymptomatic or be accompanied by symptoms (such as swelling and 

pain) and that is potentially life threatening if dislodgment of the 

thrombus results in pulmonary embolism. 

Enterovirus Disease—A kind of ribonucleic virus that multiplies 

especially in the gastrointestinal tract but may infect other tissues, such 

as nerve and muscle. 

Hypotension—Abnormally low pressure of the blood; also called low 

blood pressure. 

Hypoglycemia—Abnormal decrease of sugar in the blood. 

41 National Institutes of Health, U.S. National Library of Medicine, Medline Plus 

Medical Dictionary, updated February 4, 2003. Available online at 

http://www.nlm.nih.gov/medlineplus/mplusdictionary.html. Accessed on June 23, 2009. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 32 
 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 33 

A PPENDIX ~ C 

Pleural effusion—An oozing of fluid from the blood or lymph into a 

pleural cavity. 

Potassium deficiency—Also known as hypokalemia, a lower than 

normal amount of potassium in the blood. It may result from a number 

of conditions.

Pressure ulcer—An ulceration of tissue deprived of adequate blood 

supply by prolonged pressure; called also decubitus, decubitus ulcer, 

and pressure sore. 

Pulmonary embolism—A sudden blockage of a lung artery or one of its 

branches that is produced by foreign matter; most often a blood clot 

originating in a vein of the leg or pelvis. 

Sepsis—A systemic response typically to a serious, usually localized 

infection (as of the abdomen or lungs) especially of bacterial origin that 

is usually marked by abnormal body temperature and white blood cell 

count, tachycardia, and tachypnea; specifically, systemic inflammatory 

response syndrome induced by a documented infection. 

Urinary tract infection—An infection of the tract through which urine 

passes. It consists of the renal tubules and renal pelvis of the kidney, 

the ureters, the bladder, and the urethra. 
 

 APPENDIX ~ D 

Case Study Protocol for Nurse Reviews 

Figure D-1 presents the form used by nurses to record the results of 

medical record reviews for the first stage for the case study. Figure D-2 

contains the Institute for Healthcare Improvement (IHI) Global Trigger 

Tool (GTT) worksheet.. 

Figure D-1: Nurse Review Results Form 

IHI GTT – TRIGGER TOOL RESULTS – OEI-06-08-00220 Start Time:__________AM / PM

DATE: __________ RN ID: __________ OIG CASE ID: __________ HOSPITAL: ______________ 

Beneficiary Information

NAME: ________________________________ MR#: ___________________________ 

PHYSICIAN REVIEW INFORMATION 

Should this case be reviewed by a physician? 

 YES, Adverse Event Identified 

 YES, Other (please explain reason in notes below) 

 NO 

Notes: 

ADVERSE EVENT INFORMATION 

Harm Category Key:

E – Temporary harm, intervention required 

F – Temporary harm, initial or prolonged hospitalization 

G – Permanent patient harm 

H – Life-sustaining intervention required 

I – Contributing to death 

Assign AE Number:_______(#) of _______(#) 

(Ex: 2 of 3 – this is 2nd of 3 adverse events 

identified for this patient) 

Harm Category: _________ 

Adverse Event Description: 

Notes to Physician about Adverse Event (e.g., date, location in chart, questions) 

 End Time:__________AM / PM

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 34 
 

A PPENDIX ~ D 

Figure D-2: IHI GTT Worksheet (2007)42

42 F.A. Griffin and R.K. Resar, IHI Global Trigger Tool for Measuring Adverse Events, 

IHI Innovation Series white paper. Cambridge, MA: Institute for Healthcare Improvement 

Innovation Series 2007, Appendix B. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 35 
 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 36 

 APPENDIX ~ D APPENDIX ~ E 

Medicare Beneficiary Interview Questions 

Table E-1 provides the list of the questions asked of Medicare 

beneficiaries to screen for possible events that may have occurred during 

the target hospitalization in the case study. 

Table E-1. Beneficiary Interview Questions

Part A: General Questions 

1. Why did you go to the hospital? 

2. In general, were there any problems that happened during this hospital stay? (If NO, go to Question 4.) 

If YES or UNSURE/ CANNOT DECIDE: 

A. Please tell me more about that. 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

D. Describe how you became aware of this problem. For example, did you notice it yourself or did a 

physician, nurse, or someone else from this hospital tell you about it? 

3. Were there any other problems? (If NO, go to Question 4.) 

If YES or UNSURE/ CANNOT DECIDE: 

A. Please tell me more about that. 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

D. Describe how you became aware of this problem. For example, did you notice it yourself or did a 

physician, nurse, or someone else from this hospital tell you about it? 

4. Was there anything [else] that seemed unusual about your medical care or anything else during your stay in this 

hospital? ( If NO, go to Question 5) 

If YES or UNSURE/ CANNOT DECIDE: 

A. Please tell me more about that. 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

D. Describe how you became aware of this problem. For example, did you notice it yourself or did a 

physician, nurse, or someone else from this hospital tell you about it? 

continued on next page 
 

A PPENDIX ~ E 

Table E-1. Beneficiary Interview Questions 

Part B: Medication Questions 

5. Did you receive any medications or anesthesia during this hospital stay? (If NO, go to Question 8.) 

6. Were there any problems with the medication you were given? (If NO, go to Question 8.) 

If YES or UNSURE/ CANNOT DECIDE: 

A. Were you given the wrong type of medication? Yes or No. 

B. Were you given the wrong dosage or amount of medication? Yes or No. 

C. Did you have an allergic reaction to the medication? Yes or No. 

D. Or did you have some other problem with the medication? Yes or No. 

E. Was the medicine administered correctly? Yes or No. 

Please tell me more about that [these issues]: 

A. What was the name of the medication? If you can’t remember the name, please tell me what the 

medication was for: [Name/ Purpose] 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

D. Describe how you became aware of this problem. For example, did you notice it yourself or did a 

physician, nurse, or someone else from this hospital tell you about it? 

7. Were there any other problems related to the medication you received during your stay at this hospital? (If NO, go to 

Question 8.) 

If YES or UNSURE/ CANNOT DECIDE: 

A. Please tell me more about that. 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

D. Describe how you became aware of this problem. For example, did you notice it yourself or did a 

physician, nurse, or someone else from this hospital tell you about it? 

Part C: Procedure Questions 

8. Did you have any surgeries or procedures in this hospital stay? (If NO, go to Question 11.) 

9. Were there any problems related to the surgery? (If NO, go to Question 11.) 

 If YES: 

A. Please tell me more about that. 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

D. Describe how you became aware of this problem. For example, did you notice it yourself or did a 

physician, nurse, or someone else from this hospital tell you about it? 

10. Were there any other problems related to the surgery? (If NO, go to Question 11.) 

 If YES: 

A. Please tell me more about that. 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

D. Describe how you became aware of this problem. For example, did you notice it yourself or did a 

physician, nurse, or someone else from this hospital tell you about it? 

continued on next page

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 37 
 

A PPENDIX ~ E 

Table E-1. Beneficiary Interview Questions

Part C: Procedure Questions (Continued) 

11. Did you have any blood transfusions while you were in this hospital stay? (If NO, go to Question 14.) 

12. Were there any problems related to the transfusion? (If NO, go to Question 14.) 

If YES or UNSURE/ CANNOT DECIDE: 

A. Please tell me more about that. 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

D. Describe how you became aware of this problem. For example, did you notice it yourself or did a 

physician, nurse, or someone else from this hospital tell you about it? 

13. Were there any other problems related to the blood transfusion? (If NO, go to Question 14.) 

If YES or UNSURE/ CANNOT DECIDE: 

A. Please tell me more about that. 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

D. Describe how you became aware of this problem. For example, did you notice it yourself or did a 

physician, nurse, or someone else from this hospital tell you about it? 

14. Were there any problems with the equipment? (If NO, go to Question 15.) 

If YES or UNSURE/ CANNOT DECIDE: 

A. Please tell me more about that. 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

D. Describe how you became aware of this problem. For example, did you notice it yourself or did a 

physician, nurse, or someone else from this hospital tell you about it? 

Part D: Infections and Other Issues

15. Did you develop any infections during this hospital stay? (If NO, go to Question 17.) 

If YES: 

A. Please tell me more about that. 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

D. Describe how you became aware of this problem. For example, did you notice it yourself or did a 

physician, nurse, or someone else from this hospital tell you about it? 

16. Did you develop any additional infections during this hospital stay that we have not already discussed? (If NO, go 

to Question 17.) 

If YES: 

A. Please tell me more about that. 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

D. Describe how you became aware of this problem. For example, did you notice it yourself or did a 

physician, nurse, or someone else from this hospital tell you about it? 

continued on next page

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 38 
 

A PPENDIX ~ E 

Table E-1. Beneficiary Interview Questions 

Part D: Infections and Other Issues (Continued)

17. Did you develop any bedsores during the hospital stay? (If NO, go to Question 18.) 

If YES: 

A. Please tell me more about that. 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

D. Describe how you became aware of this problem. For example, did you notice it yourself or did a 

physician, nurse, or someone else from this hospital tell you about it? 

18. Did you ever fall during this hospital stay? (If NO, go to Question 19.) 

If YES: 

A. Please tell me more about that. 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

19. Did you ever feel unsafe during this hospital stay? (If NO, go to question 20.) 

If YES: 

A. Please tell me more about that. 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

20. Did a physician or other health care official tell you that you received the wrong diagnosis while you were in this 

hospital? (If NO, go to Question 21.) 

If YES: 

A. Please tell me more about that. 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

21. Did you discover any problems with the care you received at this hospital, once you were discharged? (If NO, go to 

Question 22.) 

If YES: 

A. Please tell me more about that. 

B. Please describe how this affected you. 

C. How, if at all, did this affect the rest of your medical care at this hospital? 

Part E: Hospital Response

22. Did you file a formal complaint about this hospital stay for any reason? (If NO, go to Question 23.) 

If YES: 

A. Please describe the nature of the complaint. 

B. How did this hospital stay respond to your complaint? 

C. Were you fully satisfied by this hospital stay’s response to your complaint? 

23. Did anyone from this hospital talk to you about the issues you have described today? (Note: Asked only if a 

possible event was discussed in prior questions. (If NO, go to Question 24.) 

If YES: 

A. If you can recall, who spoke to you about these issues? 

B. What did they say? 

C. How, if at all, did they address your concerns? 

D. Were you fully satisfied by this hospital stay’s response? 

Part F: Close

24. Is there anything else you would like to tell us about the care you received at this hospital? 

Source: Office of Inspector General interview protocol used to collect information about Medicare beneficiaries. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 39 
 

 APPENDIX ~ F 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 40 

Patient Safety Indicators 

Table F-1 lists the provider-level Patient Safety Indicators developed by 

the Agency for Healthcare Research and Quality (AHRQ), which, when 

flagged, may indicate possible adverse events. 

Table F-1: AHRQ Patient Safety Indicators (Provider-Level) 

 Patient Safety Indicators PSI 

Number

Complications of anesthesia 1 

Death in low mortality 2 

Decubitus ulcer 3 

Failure to rescue 4 

Foreign body left in during procedure 5 

Iatrogenic pneumothorax 6 

Selected infections due to medical care 7 

Postoperative hip fracture 8 

Postoperative hemorrhage or hematoma 9 

Postoperative physiologic and metabolic derangements 10 

Postoperative respiratory failure 11 

Postoperative pulmonary embolism or deep vein thrombosis 12 

Postoperative sepsis 13 

Postoperative wound dehiscence in abdominopelvic surgical patients 14 

Accidental puncture and laceration 15 

Transfusion reaction 16 

Birth trauma - injury to neonate 17 

Obstetric trauma - vaginal delivery with instrument 18 

Obstetric trauma - vaginal delivery without instrument 19 

Obstetric trauma - cesarean delivery 20 

Source: AHRQ, Guide to Patient Safety Indicators, AHRQ Quality Indicators. March 2003. Available online at 

http://www.qualityindicators.ahrq.gov/archives/psi/psi_guide_v30.pdf. Accessed on June, 4, 2008 
 

 APPENDIX ~ G 

Case Study Protocol for Physician Reviews 

Figure G-1 presents the form used by physicians to record the results of 

medical record reviews for the second stage of review in the case study. 

Physicians generated one form for each adverse event. Figure G-2 is the 

case summary completed by physicians at the end of their reviews. Table 

G-1 describes the categories used to classify the level of harm to the 

patient caused by the event. 

 Figure G-1: Adverse Event Review Form 

CASE ID: _______ 

Medical Review Protocol - OIG Case Study of Adverse Events 

Adverse Event Review 

1. Adverse Event #: ____ (Case # + Alpha) 

2. Please briefly identify the adverse event: _______________________________ 

____________________________________________________________________ 

3. Please provide a full description of the adverse event, including the: 

 circumstances of the adverse event (note as AE), 

 possible contributing factors (note as CF), and 

 impact of the adverse event on the patient (note as PI). 

AE: 

CF: 

PI: 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 41 
 

A PPENDIX ~ G 

Figure G-1: Adverse Event Review Form (Continued) 

CASE ID: ______ 

4. Using the NCC MERP categories of harm, what level of harm resulted from the event? 

 E – Temporary harm, intervention required 

 F – Temporary harm, initial or prolonged hospitalization 

 G – Permanent patient harm 

 H – Life-sustaining intervention required 

 I – Contributing to death 

5. Does the harm incurred appear to be the result of a “cascade”? 

 

 Yes (If yes, please insure that Q3 includes the sequence of occurrences.) 

 No 

If yes, please lay out the sequence of occurrences below (with arrows): 

6. On what date or range of dates did the adverse event occur? 

7. During which time period did the adverse event occur? 

 During time in emergency department 

 During time in observation 

 During the targeted inpatient hospitalization 

 Other, please describe: 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 42 
 

A PPENDIX ~ G 

Figure G-1: Adverse Event Review Form (Continued) 

CASE ID: ______ 

8. Please review the provided billing summary of diagnosis and procedure codes. Which (if any) 

appear to result from this adverse event? 

ICD-9 Code Brief Description 

 

 

 

 

 

 

 

 

 

 

 

9. Which case flags reflected information about the specific adverse event? (Check all that apply.) 

 Patient interview 

 Global Trigger Tool summary 

 POA analysis 

 PSI analysis 

 Hospital incident report or other hospital report 

 No case flags reflected this adverse event 

10. Is there anything additional you would like to note about the case flags for this adverse event? 

(case flag or element of a case flag that was particularly valuable in leading you to the event, thoughts 

about selecting case flags for our future work, etc.) 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 43 
 

A PPENDIX ~ G 

Figure G-1: Adverse Event Review Form (Continued)

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 44 
 

A PPENDIX ~ G 

Figure G-2: Case Summary 

CASE ID: _______

Medical Review Protocol - OIG Case Study of Adverse Events 

Case Cover Page 

Hospital: __________________

Patient name: _______________

OIG ID#: ______ MR#: ________ 

Review Date: _______________, 2008 

Physician Initials: _________________ 

Directions: Review the provided case documents and medical record. 

Complete Adverse Event Review for any adverse events. 

Complete Summary Questions and Exit Check List. 

Beneficiary Information 

Summary Questions

Name: _______________________________ D.O.B.:_________________ 

Dates of Stay: __________________________ L.O.S. (days):____________ 

Principal Diagnosis: _______________________ 

Flags: ___ Beneficiary Interview ___ IHI Global Trigger Tool Review 

 ___ POA Analysis ___ Hospital Incident Report 

 ___ PSI Analysis ___ Other: ________________ 

1. Is this review complete? 

 Yes 

 Partially complete, estimate _______ additional time needed 

2. Did you identify any adverse event(s)?

 Yes, how many? _______________ 

 No 

AE #s and brief descriptions: 

________ _____________________________________ 

________ _____________________________________ 

________ _____________________________________ 

________ _____________________________________ 

3. Should this case be targeted for further discussion? 

 Yes 

 No 

If yes, please briefly state reason: 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 45 
 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 46 

A PPENDIX ~ G 

Figure G-2: Case Summary (Continued) 

CASE ID: _______

Case Exit Check List 

1. Which of the following information sources did you review? (Check all that apply.) 

 Medical record 

 Results of administrative data reviews 

 Patient interview 

 Hospital incident report or other hospital report 

 Other, please describe:_____________________________________________________ 

2. Please circle on the scale below to assess the documentation in the medical record. A scale of 

1=worst (unclear, difficult to determine event) and 5=best (well-described and definitive). 

1 2 3 4 5 

3. Was the medical record recorded as (check both if applies): 

 Paper records? 

 Easy to navigate 

 Some difficulty navigating 

 Difficult to navigate 

 Electronic records? 

 Easy to navigate 

 Some difficulty navigating 

 Difficult to navigate 

4. Did you consult with another physician reviewer? 

 Yes 

 No 

If yes, please describe: _________________________________________________________ 

5. How much time did you spend on this review (including any consultation with others)? 

 Less than 30 minutes 

 Between 30 minutes – one hour 

 Between one hour – two hours 

 More than two hours 

 Difficult to determine because of stop and start and/or consultation 

6. Did this case suggest modifications of this form to improve subsequent studies? 

 Yes

 No

If yes, please describe: ________________________________________________________ 

___________________________________________________________________________ 

7. Is there anything else about this case that you would like to note? _________________________ 

______________________________________________________________________________ 
 

A PPENDIX ~ G 

Patient Harm Index 

Table G-1 lists the categories of patient harm adapted by the Institute 

for Healthcare Improvement (IHI) and used by the Office of Inspector 

General in the case study. The National Coordinating Council for 

Medication Errors Reporting and Prevention (NCC MERP) developed an 

index to categorize the level of patient harm resulting from medication 

errors. Researchers have modified this index for use in measuring and 

distinguishing adverse events of all types, rather than only medication 

errors. IHI adapted the index to reflect only events that cause harm to 

patients. 

Table G-1: IHI Global Trigger Tool Adaptation of the NCC 

MERP Index to Categorize Harm 

Category Description 

E Temporary harm to the patient and required intervention 

F Temporary harm to the patient and required an initial or prolonged hospital stay 

G Permanent patient harm 

H Intervention required to sustain life 

I Patient death 

Source: F.A. Griffin and R.K. Resar, IHI Global Trigger Tool for Measuring Adverse Events, IHI Innovation 

Series white paper. Cambridge, MA: Institute for Healthcare Improvement Innovation Series 2007, pp. 4–5. 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 47 
 

 APPENDIX ~ H 

Results of Screening Methods 

Present on 

Admission (POA) 

Analysis

(n = 296)

Nurse Review

(n = 227)

Beneficiary 

Interviews

(n = 71)

Hospital Incident 

Reports

(n = 55)

Patient Safety 

Indicators (PSI) 

Analysis

(n = 13)

POA Analysis

(n = 111)

Nurse Review

(n = 104)

Beneficiary 

Interviews

(n = 21)

Hospital Incident 

Reports

(n = 9)

PSI Analysis

(n = 9)

Unique Events 

Identified Through 

Screening Methods

(n = 114)

Events

(n = 120)

Flags by 

Screening Method

(n = 662)

Flags Associated 

With Events by 

Screening Method

(n = 254)

Events

(n = 120)

Events* 

(Unduplicated)

Figure H-1: Results of Screening Methods Used for Identifying Events in Case Study

Source: Office of Inspector General analysis of 278 Medicare beneficiary hospitalizations in 2 selected counties, 2008.

*Some flags were associated with more than one event, and some events were identified through more than one flag.

Events Identified 

Only Through 

Physician Reviews

(n = 6)

Physician Review of 

Medical Records 

With Flags

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 48 
A P PEN D x 

Agency Comments: Agency for Healthcare Research and Quality 

DEPARTMEN1'OF HEALTH & HUMAN SERVICES Agency· for H~althcare. 

Research and Quality 

540 Gaither Road 

Rockville MD 20850 

www,ahrq.gQv 

DEC 162009 

TO: Daniel Levinson, Inspector GeneraJ 

FROM: Cal'olynM, Clancy,M,D•• Dii'ectol' 

SUBJECT: Comments on draft Office of Inspec(orGenetal Report entitled, "Adverse 

BVeiltsin Hospitals: Metl1odsJ'or Identffying Events" 

Thmlk you [wr the oPpOI:lunity to review the draft Qffj.ce·oflirspe.ctor General Report 

entftled: "Ad.\lerse EVents in Hospitals: Methbds fDr Identifying Events," The Agency 

for Hefllthcare Re:seal'ch.and Quality (AHRQ)cqllctJfswith thereport.as written. 

If you have any questions; please feel freeto call Wendy Perry of my staff at 301-427­

l216. . , . 

oE 1-0 6-08-00 221 ADVERSE EVENTS IN HOSPITALS: METHODS FOR IDENTIFYING EVENTS 49 
-----------------

APPEND X 

Agency Comments: Centers for Medicare &Medicaid Services 

DEPARTMENT OF HEALTH & HUMAN SERVICES Centers lor Medicare & Medicaid Services 

Administrator 

Washington. DC 20201 DEC 1 7 2009 

TO: 	 Daniel R. Levinson 


Deputy Inspector General 


FROM: 	 Charlene Frizzera 


Acting Administrator 


SUBJECT: 	 Office OfInspector General (OlG) Draft Report: "Adverse Events in Hospitals: 


Methods for Identifying Events," (OEI-06-08-00221) 


Thank you for the opportunity 10 review and comment on this draft report. "Adverse Events in 

Hospitals: Methods for Identifying Events." This is a particularly useful report that offers timely 

recommendations. The OlG undertook original data collection to assess five different methods 

of identifying the occurrence of serious adverse events in short-term acute care hospitals. The 

OlG then identified strengths and weaknesses of each method. Finally, the OlG explored 

implications of its findings with regard to the ability of the Centers for Medicare & Medicaid 

Services (CMS) to monitor adverse events, promote value-based purchasing, and enforce quality 

of care and patient safety requirements. 

We appreciate the original contribution that this study makes in advancing the patient safety 

agenda in hospitals and bolstering CMS efforts to prevent adverse events. As the OlG noted, 

CMS enforces quality of care expectations through its survey and certification onsite reviews. 

eMS has actively advanced value-based purchasing across a variety of provider types. CMS has 

also instituted payment policies that prevent coverage of certain serious but clearly avoidable 

medical events, often referred to as "Never Events." Information about the most recent policies 

(relating to performance of the wrong surgery or surgery on the wrong body part or on the wrong 

patient) can be found at: 

http://W'oNw.cms.hhs.gov/MLNMattersArticlcs/dowl1load~nY!M6405.pdf. 

We endorse each recommendation addressed to eMS, and provide more detail below. 

OIG Recommendation 1: 

eMS and the Agency for Healthcare Research and Quality (AHRQ) should explore opportunities 

to identify events when conducting' medical record reviews for other purposes. 

CMS Response 

We agree with the recommendation and will consider creating or taking advantage of existing 

opportunities to identi fy adverse events, including consideration ofthe specific options identified 

by the OlG. 

oEI-06-08-002 21 ADVERSE EVENTS IN HOSPITALS: METHODS FOR IDENTIFYING EVENTS 50 
 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 51 

A PPENDIX ~ I 
 

A PPENDIX ~ I 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 52 
 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 53 

A PPENDIX ~ I 
 

 O E I - 0 6 - 0 8 - 0 0 2 21 A DVERSE E VENTS IN H OSPITALS : METHODS FOR I DENTIFYING E VENTS 54 

 ACKNOWLEDGMENTS

This report was prepared under the direction of Kevin K. Golladay, 

Regional Inspector General for Evaluation and Inspections in the Dallas 

regional office, and A. Blaine Collins, Deputy Regional Inspector 

General. 

Amy Ashcraft and Ruth Ann Dorrill served as the team leaders for this 

study. Deborah Cosimo served as the lead analyst. Other principal 

Office of Evaluation and Inspections staff from the Dallas regional office 

who contributed to the report include Anthony Guerrero-Soto, Deborah 

McGurk, and Lyndsay Patty; other central office staff who contributed 

include Rob Gibbons and Rita Wurm. 
Office of Inspector General 

http://oig.hhs.gov 

The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452,

as amended, is to protect the integrity of the Department of Health and Human Services 

(HHS) programs, as well as the health and welfare of beneficiaries served by those 

programs. This statutory mission is carried out through a nationwide network of audits, 

investigations, and inspections conducted by the following operating components: 

Office of Audit Services 

The Office of Audit Services (OAS) provides all auditing services for HHS, either by

conducting audits with its own audit resources or by overseeing audit work done by others. 

Audits examine the performance of HHS programs and/or its grantees and contractors 

in carrying out their respective responsibilities and are intended to provide independent 

assessments of HHS programs and operations. These assessments help reduce waste, 

abuse, and mismanagement and promote economy and efficiency throughout HHS. 

Office of Evaluation and Inspections 

The Office of Evaluation and Inspections (OEI) conducts national evaluations to provide 

HHS, Congress, and the public with timely, useful, and reliable information on 

significant issues. Specifically, these evaluations focus on preventing fraud, waste, or 

abuse and promoting economy, efficiency, and effectiveness in departmental programs. 

To promote impact, the reports also present practical recommendations for improving 

program operations. 

Office of Investigations 

The Office of Investigations (OI) conducts criminal, civil, and administrative 

investigations of allegations of wrongdoing in HHS programs or to HHS beneficiaries 

and of unjust enrichment by providers. The investigative efforts of OI lead to criminal 

convictions, administrative sanctions, or civil monetary penalties. 

Office of Counsel to the Inspector General 

The Office of Counsel to the Inspector General (OCIG) provides general legal services to 

OIG, rendering advice and opinions on HHS programs and operations and providing all 

legal support in OIG's internal operations. OCIG imposes program exclusions and civil 

monetary penalties on health care providers and litigates those actions within HHS. 

OCIG also represents OIG in the global settlement of cases arising under the Civil False 

Claims Act, develops and monitors corporate integrity agreements, develops compliance 

program guidances, renders advisory opinions on OIG sanctions to the health care 

community, and issues fraud alerts and other industry guidance.