U.S. Department of Health and Human Services 
Office of Inspector General 
Adverse Events Toolkit: 
Medical Record Review 
Methodology 
Ann Maxwell  
Deputy Inspector General 
   for Evaluation and Inspections 
July 2023, OEI-06-21-00030   
TABLE OF CONTENTS 
 
 
INTRODUCTION .......................................................................................................................................................... 1 
PROCESS FOR IDENTIFYING PATIENT HARM EVENTS ...................................................................................... 2 
DEFINITION OF PATIENT HARM ............................................................................................................................. 4 
SEVERITY OF HARM ................................................................................................................................................... 7 
PREVENTABILITY OF HARM ..................................................................................................................................... 9 
COMPLEX CASES ........................................................................................................................................................ 11 
MEDICAL RECORD REVIEW PROTOCOL .............................................................................................................. 13 
QUALITY ASSURANCE ............................................................................................................................................. 16 
APPENDICES ............................................................................................................................................................... 18 
Appendix A: IHI Methodology for Identifying Harm ............................................................................................. 18 
Appendix B: Contributing Factors for Preventability ............................................................................................. 20 
ACKNOWLEDGMENTS AND CONTACT ............................................................................................................... 21 
ABOUT THE OFFICE OF INSPECTOR GENERAL ................................................................................................. 22 
REFERENCES .............................................................................................................................................................. 23 
 
 
 
  INTRODUCTION 
This toolkit provides information about how we conducted 
Companion Resource 
medical record reviews to identify patient harm and our 
For clinical guidance used during 
decision criteria for adverse events, which may be useful to 
OIGs medical record reviews, see our 
health care providers and researchers dedicated to patient 
companion resource: Adverse Events 
safety.  The toolkit describes the methods used in a recent 
Toolkit: Clinical Guidance for 
report, Adverse Events in Hospitals: A Quarter of Medicare  Identifying Harm. 
Patients Experienced Harm in October 2018, OEI-06-18-00400, 
and builds upon a broader series of reports about adverse 
events in hospitals and other health care settings.  Of the 18 reports currently in this series, 7 studies 
used nurses and physicians to review medical records to identify adverse events (each focused on a 
different health care setting).  
The goal of our medical record reviews was to establish a national, point-in-time rate of patient harm.  
The methodology builds upon the Global Trigger Tool (GTT) methodology developed by the Institute 
for Healthcare Improvement (IHI), which we adapted and used for screening and flagging medical 
records for possible patient harm.  
During the course of our reviews, we recorded internal decisions, researched clinical literature and 
guidelines, interviewed experts in various fields, and conducted routine calls to gain consensus among 
reviewers on decisions regarding what constitutes harm and how to categorize harm events.  A 
companion resource, Adverse Events Toolkit: Clinical Guidance for Identifying Harm, describes the clinical 
decision rules we developed for 29 conditions as well as suggestions for how to identify and document 
evidence of patient harm.   
Standards 
We conducted this work in accordance with the Quality Standards for Inspection and Evaluation issued 
by the Council of the Inspectors General on Integrity and Efficiency. 
Legal Notice 
This toolkit is a technical resource and is not intended to be used to determine compliance with any laws, 
regulations, or other guidance.  It is not intended to, and does not, create any rights, privileges, or benefits, 
substantive or procedural, enforceable by a party against the United States; its agencies or instrumentalities; its 
officers or employees; or any other person.  OIG does not endorse external content or material linked in this 
toolkit. 
   
Adverse Events Toolkit: Medical Record Review Methodology  Introduction | 1 
OEI-06-21-00030  PROCESS FOR IDENTIFYING PATIENT HARM 
EVENTS 
Patient harm events are commonplace in inpatient care settings, affecting approximately one in four 
hospital patients and similar numbers of patients in post-acute care settings, such as nursing homes.  
However, harm is not always acknowledged as such by providers and clinical staff, often because it is so 
routine.  We believe it is important to identify all causes of harmknown risks, failure to provide care, 
and substandard care, as well as errors and other causes of harm.  Tracking all causes of harm allows for 
better comparisons across time and provides a fuller picture of the patient experience.  The challenge 
associated with tracking all causes of harm is systematically identifying events in an efficient way. 
We conducted retrospective reviews of medical records to identify harm events using a two-stage 
process.  Sample sizes ranged between 300 and 800, depending on the setting.  The stages are (1) a 
manual screening process using a Global Trigger Tool (GTT) first devised by the Institute for Healthcare 
Improvement (IHI) and (2) a comprehensive physician review.  Combined, this process was effective at 
identifying patient harm events.1, 2 
Stage 1  Screening: In the first stage of review, 
Suggested order of review 
screeners (typically nurses experienced in GTT 
When using a GTT methodology, reviewing medical 
reviews) systematically reviewed records using a 
record components in the order listed below may 
modified version of IHIs GTT methodology.  The 
shorten the timeframe needed to conduct reviews. 
screeners searched the record for triggers, which 
are clinical clues that may indicate that harm  1)  Diagnosis and procedure codes 
occurred.  (In some cases, the trigger is itself the  2)  Discharge summary 
harm, such as a pressure injury.)  When a trigger 
3)  History and physical 
was identified, the screeners investigated further 
and recorded their results.    4)  Medication administration record 
5)  Laboratory results and radiology reports 
Screeners also did a targeted review to investigate 
diagnoses listed in claims data that were not  6)  Prescriber orders 
present on admission to assess whether the  7)  Operative record 
diagnoses were potentially related to a harm event.  
8)  Daily progress notes 
Medical conditions that develop during a stay are 
9)  Skin/wound assessment 
sometimes the result of harm events.  
10) Other pertinent records (e.g., contiguous 
Medical records flagged by the screeners as having 
emergency department records) 
potential harm events were referred to the second 
stage of review.  Detailed information about the 
triggers we used is included in the companion document: Adverse Events Toolkit: Clinical Guidance for 
Identifying Harm, and a comparison of our methodology to the original IHI-developed methodology is 
described in Appendix A. 
In addition to the manual screening process, we automatically referred to the second stage of review 
the medical records for any patients who were readmitted within 30 days of the stay, as these 
Adverse Events Toolkit: Medical Record Review Methodology  Process for Identifying Patient Harm Events | 2 
OEI-06-21-00030  readmissions may have indicated that the patient was harmed during the original admission.  Screeners 
also had the discretion to refer medical records in which they did not identify potential harm but found 
the case to be complex or to involve circumstances often associated with harm. 
Stage 2  Physician review: In the second stage, we used an interdisciplinary panel of physicians to 
conduct a comprehensive review of the records referred by the screeners.  A physician reviewer 
independently reviewed each patients medical records to confirm (or refute) harm events identified in 
the first stage of review and to identify any additional harm events.  They assessed whether events were 
preventable and classified the events according to severity, the harm event type, and other 
characteristics.  We routinely held calls, during which the physician reviewers discussed complex cases 
and questionable events to promote consensus around harm determinations.  We included a wide 
range of specialties (e.g., infectious disease, cardiology, surgery, neurology, physiatry, orthopedics, 
critical care, and hospital medicine) on the physician panel to promote discussion, and this provided 
opportunities for reviewers to consult other members for nuanced or complex cases.  See Quality 
Assurance section on page 16 for more information.  For each referred record, the physician reviewer 
made the final determination of patient harm. 
 
Adverse Events Toolkit: Medical Record Review Methodology  Process for Identifying Patient Harm Events | 3 
OEI-06-21-00030  DEFINITION OF PATIENT HARM 
We define patient harm as any undesirable clinical outcomenot caused by underlying diseasethat 
was the result of medical care or that occurred in a health care setting, including the failure to provide 
needed care.  Below, we further describe the criteria we used to identify patient harm and we explore 
additional boundaries pertaining to patient harm in subsequent sections.  Patient harm refers 
collectively to adverse events and temporary harm events.  An adverse event indicates that the harm 
prolonged the hospital stay; resulted in an elevation of care or transfer to another facility; caused 
permanent harm; required life-saving intervention; or contributed to death.  A temporary harm event 
is an event that resulted in patient harm and required medical intervention but did not prolong the 
patient stay, cause permanent harm, or require life-sustaining intervention.  (See the Severity of Harm 
section on page 7 for details about the distinction between adverse events and temporary harm events.)   
Patient harm includes all causes of harm that occur as a result of medical care, or lack thereof, and that 
necessitate treatment or intervention.  It includes both preventable and nonpreventable harm events.  
Nonpreventable harm can include expected side effects where harm was foreseeable but considered 
acceptable given alternatives or known complications of treatment that required intervention.  Our 
definition of patient harm does not include near misses that had the potential to cause harm.  For each 
harm event, we identify (1) a clinical cause (commission or omission) demonstrating that the event is 
not the result of underlying disease, (2) signs and symptoms of harm to the patient, and (3) a medical 
intervention to treat the harm.  Other researchers and government agencies have tracked harm events 
by creating lists of specific harm events, see box below. 
Differences in patient harm definitions, terms, and lists 
Researchers and organizations use a variety of terms and lists to define patient harm, so it is important to 
articulate the specific definition used in each research effort.  The literature on patient safety sometimes uses the 
term adverse events to indicate a specific list of harm events but OIG focuses on all causes of harm and does 
not limit to a specific list.   
Some researchers use alternative terms such as never events, serious reportable events (SREs), sentinel events, 
hospital-acquired conditions (HACs), healthcare-associated infections (HAIs), and adverse drug events (ADEs) to 
focus on a specific subset of harms or that fit specific criteria.  Below are some examples of terms currently used: 
  The Centers for Medicare & Medicaid Services (CMS) uses a specific list of HACs, that includes HAIs, to 
meet statutory requirements for its payment programs and policies.  See Deficit Reduction Act HAC list, 
the HAC Reduction Program list, and provider-preventable conditions list.  
  The Joint Commission (TJC) focuses on sentinel events when instructing onsite surveyors to assess hospital 
compliance with accreditation standards.  See Sentinel Event Policy for Hospitals. 
  The Centers for Disease Control and Prevention (CDC) focuses on HAIs in its surveillance programs to track 
infections acquired in health care facilities and tracks ADEs as part of the Healthy People initiative.  See 
Types of Healthcare-associated Infections and see National Electronic Injury Surveillance System. 
  The Agency for Healthcare Research and Quality (AHRQ), with support from CMS and CDC, tracks 35 types 
of adverse events in the Quality Safety and Review System.  See Quality Safety and Review System. 
  The National Quality Forum (NQF) developed a list of serious reportable events (also known as never 
events) to facilitate uniform and comparable public reporting.  See List of Serious Reportable Events. 
Adverse Events Toolkit: Medical Record Review Methodology  Definition of Patient Harm | 4 
OEI-06-21-00030  Patient Harm Event Criteria  
The sections below provide details regarding what we, in OIG, include and exclude as patient harm.  
Exclude underlying disease.  We excluded events that resulted from the natural progression of 
underlying disease.  For example, symptomatic anemia would not be considered a harm event if it 
resulted from an underlying condition, such as bone marrow cancer.  In contrast, we included disease 
exacerbations that resulted directly from hospital care or omissions of care during the hospital stay.  For 
example, we would consider symptomatic anemia that is due to anticoagulants requiring a blood 
transfusion a harm event. 
Include both commission and omission of care.  We included harm events that resulted from acts of 
commission (i.e., events related to the active delivery of care) and omission of care (i.e., events related 
to the failure to provide necessary care).  Omissions of care may represent errors or substandard care, 
such as failing to implement preventative measures.  Some researchers exclude omission events 
because the linkage between the care and patient harm is more ambiguous.  
Examples of commission events:   
  administration of penicillin to a patient who had an allergic reaction 
  administration of beta blocker to a patient that resulted in complete heart block 
Examples of omission events: 
  delayed recognition of Type 2 myocardial infarction leading to pulmonary edema   
  failure to diagnose and treat a patient with serious hypertension (i.e., not providing 
antihypertension medication) resulting in a stroke   
Exclude present on admission (POA).  We excluded harm events caused by care provided in another 
setting prior to the hospital stay.3  These are categorized as POA events.  Some researchers include POA 
events in order to track harm across health care providers rather than limiting to a single hospital stay.   
POA events are those that occurred: 
  during a previous stay in that same facility,  
  in another facility (e.g., a skilled nursing facility, a different hospital), 
  in an ED observation or outpatient encounter from which the patient was discharged prior to 
the admission,4   
  in an ambulance on the way to the hospital, or  
  at the patients home.   
Adverse Events Toolkit: Medical Record Review Methodology  Definition of Patient Harm | 5 
OEI-06-21-00030  Examples of POA events:  
  a complication, such as an infection, from a procedure performed during a preceding 
hospitalization that was recognized upon admission and treated appropriately during the 
hospital stay 
  a pressure injury that developed prior to admission and did not progress to a higher stage 
during the hospital stay 
Include expected side effects.  We included harm events resulting from expected side effects when 
they required intervention.  This was true even when the provider explained the potential complication 
during informed consent.  For example, we would consider vomiting and dehydration due to 
chemotherapy treatment a harm event if the patient required repeated use of antinausea medication 
that was more than expected or needed intravenous fluids.   
Exclude solely abnormal laboratory test results.  We did not typically consider abnormal laboratory 
test results without symptoms to be harm events even when providers intervened to correct the 
abnormal finding.  For example, we would not consider a patient with no signs or symptoms of 
infection but who had an indwelling urinary catheter and a positive urine culture of 100,000 colony-
forming units (CFUs) per milliliter to have experienced a harm event.  If the same patient had 
accompanying signs and symptoms, such as pain or fever, we considered the bacteriuria a harm event. 
Exclude pain.  We excluded events composed solely of pain without evidence of harm.  For example, 
we would not count routine post-surgical site pain as a harm event unless it was associated with a 
hospital-acquired infection or other surgical complication.  In addition, we may consider significant pain 
that impacted patient care, such as the patient being unable to ambulate, part of a harm event if this 
was the result of a fall or other trauma. 
 
Adverse Events Toolkit: Medical Record Review Methodology  Definition of Patient Harm | 6 
OEI-06-21-00030  SEVERITY OF HARM 
We assigned each patient harm event a severity level based on a modified version of the National 
Coordinating Council for Medication Error Reporting and Preventions (NCC MERPs) index for 
categorizing events.  NCC MERP devised this index to categorize medication errors by the degree of 
harm and the resources expended to treat the patient.5  The NCC MERP Index ranks errors from levels A 
to I on the index.  Levels A through D constitute near misses in which the error did not result in 
patient harm but had the potential to cause harm.  These errors typically consist of quality-of-care 
problems such as staff not following national clinical guidelines or best practices.  Levels E through I 
progress from temporary harm (E-level harm) to harm that contributes to or results in death (I-level 
harm).  These harm levels are not necessarily sequential, since an H-level harm could have a short-
lasting effect if resolved quickly whereas F-level and G-level harms may result in longer-lasting effects. 
Although the NCC MERP Index was initially developed to categorize the severity of medication errors, 
researchers, such as those at IHI, have modified the Index to measure and distinguish other types of 
patient harm events.  Our physician reviewers assigned each harm event to a severity level between E 
and I.  We also used the index to distinguish between temporary harm events and adverse events.  See 
Exhibit 1 below for a description of each severity level.   
Exhibit 1: OIG-Modified Version of the NCC MERP Index for Categorizing Events 
Event Type  Level  Description 
Harm occurred that may have contributed to or resulted in the 
  I 
patients death. 
  H  Harm occurred that required intervention to sustain the patients life. 
Harm occurred that contributed to or resulted in permanent patient 
Adverse Event  G 
harm. 
Harm occurred that contributed to or resulted in prolonged facility 
  F  stay, elevation in level of care, transfer to another facility, or 
subsequent admission. 
Temporary Harm Event  E  Harm occurred that caused temporary harm that required intervention. 
Source: Adapted from the NCC MERP Index for Categorizing Errors.  Revised February 20, 2001. 
In determining the level of harm, reviewers considered the impact to the patient and specific 
circumstances of the harm event.  Below we provide more detail about each level of the index.   
E-level.  We categorized harm events that caused temporary harm and required an intervention, 
including the need for additional monitoring, as E-level.  Although adverse events are often more 
serious, temporary harm events can also be serious and lead to severe consequences if left untreated.  
For example, if treated promptly, clinically significant hypoglycemia (low blood glucose) is usually a 
temporary harm event, but it can become a life-threatening adverse event if left untreated.  Other 
examples of temporary harm events could include a surgical bladder perforation with immediate 
Adverse Events Toolkit: Medical Record Review Methodology  Severity of Harm | 7 
OEI-06-21-00030  surgical repair, an asymptomatic procedural bleed corrected with blood transfusions, a skin rash caused 
by allergies, and a Stage 1 pressure injury. 
F-level.  We categorized harm events as F-level if they prolonged the hospital stay, required an 
elevation in level of care (e.g., patient moved to the intensive care unit (ICU)), required a transfer to 
another facility (for observation, emergency care, or inpatient admission), or required a subsequent 
admission.  If the patient was transferred, we considered the reason for the transfer when determining 
the level of harm and only assessed harm events as F-level if the patient was transferred to receive a 
higher level of care for the event.  Examples of F-level harm could include surgical site infections and 
injuries from falls which prolonged the patient stays.   
We expanded the original NCC-MERP definition of an F-level event to include harm events that 
contributed to or resulted in an elevation in level of care, transfer to another facility, or subsequent 
admission.  We expanded the definition to include these cases after identifying events in long-term care 
facilities that would be considered an F in an acute care setting but did not meet the F-level criteria due 
to a lengthy anticipated stay.  Similar to other F-level events, these scenarios indicate the need for 
additional care and resources to treat the harm event.  
G-level.  We categorized harm events that contributed to or resulted in permanent patient harm as 
G-level.  This included either cognitive or physical impairment and often represents very significant 
implications for the patient.  Examples of G-level harm could include complications (e.g., strokes) from 
major surgeries resulting in lifelong impaired coordination and mobility. 
H-level.  We categorized harm events that required an intervention to sustain the patients life as 
H-level.  The intervention typically occurs within 1 hour of the act of commission or omission that led 
to the patient harm to save the patients life.  These events could include permanent harm and always 
required a life-saving intervention to prevent death.  In distinguishing life-saving intervention (H-level) 
from permanent harm (G-level harm), reviewers considered which had the more profound impact on 
the patient.  Examples of H-level harm could include respiratory failure due to excessive use of opioids 
and hypoglycemic coma due to insulin, which can be life-threatening if not corrected within the hour. 
I-level.  We categorized harm events that may have contributed to or resulted in a patients death as 
I-level.  If a patient experienced multiple events that contributed to or resulted in their death during 
their hospital stay, only one event that most directly contributed to or resulted in their death could be 
categorized as I-level.  Therefore, each patient could only have one I-level event.  Other events may be 
cited at another level of severity.  A patient with a terminal illness or receiving end-of-life care can also 
experience an I-level event that may likely not be preventable.  In such cases, reviewers considered 
whether the patients death was hastened as a result of the event.   
 
Adverse Events Toolkit: Medical Record Review Methodology  Severity of Harm | 8 
OEI-06-21-00030  PREVENTABILITY OF HARM 
Measuring all causes of harm results in capturing 
Measuring Preventability  Pros and Cons 
both preventable and nonpreventable patient 
Pros: 
harm events.  This comprehensive measurement of 
  Improves our understanding of harm events 
harm improves our ability to compare rates across 
  Assists with designing and prioritizing 
time and across research.  However, we believe it 
interventions to prevent the recurrence of 
is important to allow providers, researchers, and 
preventable harm events 
policymakers to focus on preventable events, in    Assists with designing policies to avert 
which they have greater opportunity to reduce  payment for preventable harm events 
harm.  Therefore, we determined the preventability  Cons:  
of harm events.  Preventability assessments,    Changes in clinical guidance over time may 
however, are inherently subjective and  affect how reviewers assess preventability 
  Inherent subjectivity can make it difficult to 
assessments can be affected by the quality of 
achieve high inter-rater reliability in 
documentation and advances in medical practice.  
determinations of preventability 
Physician reviewers assessed the preventability of 
  Reviewers may be affected by hindsight bias in 
each harm event individually and considered the 
determining preventability 
clinical cause of the event, whether the event was    Limited information in medical records may 
anticipated, and other questions (see list below).   
hinder preventability assessments 
Questions To Consider When Determining 
Preventability 
  Was an identifiable error or system failure documented in the medical record?  If yes, the event can 
be considered preventable.   
  Could the care provider have anticipated this event with the information available at the time?  If no, 
the event is likely not preventable. 
  Were appropriate precautions taken to prevent this event?  If yes, the event is likely not preventable.  
If no, the event is likely preventable. 
  Did the providers follow national clinical guidelines or best practices when providing care?  If not, the 
event may be preventable. 
  Was there an omission of care (e.g., delay in providing care)?  If yes, this would be considered an 
error and would be preventable. 
  Was the harm event the result of an expected side effect of medication that was more severe or 
prolonged than normal and required intervention?  If yes, this could be considered likely preventable 
if preventative measures were considered the standard of care and providers did not take those 
measures. 
We then assigned each harm event to one of five preventability determinations.  We created a 
preventability scale to provide our physician reviewers with flexibility in calling an event likely to clearly 
preventable or likely to clearly not preventable.  If physicians were unable to determine preventability, 
Adverse Events Toolkit: Medical Record Review Methodology  Preventability of Harm | 9 
OEI-06-21-00030  they could label an event unable to determine.  Physicians also explained their rationale for each 
preventability determination based on a list of contributing factors gleaned from prior research and 
experience in OIG studies of adverse events.  See Appendix B for a list of contributing factors.   
Assessing an event as clearly preventable or clearly not preventable required a greater degree of 
certainty on the part of the reviewer.  Although we collapsed the designations of clearly and likely into 
the larger categories of preventable or not preventable for presentation in the report, the five-point 
scale enabled physicians to make more precise determinations, which was useful during quality 
assurance activities.  Reviewers also selected from a pre-determined list of factors that could have 
contributed to harm events.  For example, preventable events may be related to substandard treatment, 
medical error, or inadequate monitoring depending on the factors involved.  Events that were not 
preventable may be related to a patients diagnosis or treatment being unusual or complex and thereby 
making care difficult or a patient being highly susceptible to a particular type of harm event because of 
poor health.  These factors are not necessarily exclusive of each other and we asked clinicians to use 
their clinical experience and judgment in selecting the factors that applied to each harm event.  
Clinicians based their decisions on the circumstances of the specific case and also considered accepted 
standards of care, guidance developed during the review process, and group discussion of the patients 
and events.  Additionally, clinicians referenced condition-specific guidance developed by OIG, see the 
Adverse Events Toolkit: Clinical Guidance for Identifying Harm.   
 
 
Adverse Events Toolkit: Medical Record Review Methodology  Preventability of Harm | 10 
OEI-06-21-00030  COMPLEX CASES 
During our reviews, some patients had complex medical conditions or circumstances that made it 
difficult for our reviewers to determine whether a patient harm event occurred and how to assess its 
preventability.  Below, we describe some of these circumstances and how we addressed them.   
Cascade harm events.  When a patient experienced an initial harm event that causes a series of related 
and dependent events, physician reviewers combined the events into a cascade and counted it as a 
single event.  The purpose of combining the events was to avoid overcounting harm events.  This was in 
contrast to cases where a patient experienced multiple, unrelated harm events during a hospital stay, 
which we counted as separate events (see Examples of two independent harm events below).  To 
ensure accuracy and consistency, we instructed clinicians to clearly distinguish between each harm 
event that occurred in the cascade and to identify the initial event that cascaded to other harms.   
Examples of cascade events: 
  After placement of a contaminated central venous catheter, a patient developed an infection 
that resulted in sepsis, shock, acute kidney injury, respiratory failure, and ultimately the patients 
death.  Our reviewers assigned this series of related events to the most serious applicable 
severity levelI-level harm (an event that contributed to or resulted in death).  Each harm event 
shares an initial act of commission, which was the central line-associated bloodstream infection. 
  A patient undergoing a transcatheter aortic valve replacement experienced prolonged 
hypotension during the procedure which led to pulseless electrical activity requiring 
cardiopulmonary resuscitation, vasopressor medication, a heart pump insertion, and fluid 
resuscitation.  The hypotension contributed to permanent brain injury from a stroke.  Our 
reviewers assigned this as a cascade event with a severity level of G-level harm (an event that 
contributed to permanent harm). 
Examples of two independent harm events (i.e., not cascade events): 
  A patient developed a catheter-associated urinary tract infection (CAUTI) and oversedation due 
to medication during the hospital stay.  The harm events are not related. 
  A patient developed a CAUTI and was treated with antibiotics and then developed a rash as a 
reaction to the antibiotics.  The use of antibiotics leading to the rash was considered an 
independent harm event because it had a clinical cause that was separate and distinct from the 
CAUTI.  
Latent harm events.  We reviewed medical records for subsequent admissions within 30 days of the 
patients discharge.  This review allowed us to identify some adverse events that were not known at the 
time of discharge, such as the development of a deep surgical infection.  In cases where harm from care 
in an initial stay became evident in a subsequent stay, we attributed the harm to the initial stay and not 
to the readmission. 
   
Adverse Events Toolkit: Medical Record Review Methodology  Complex Cases | 11 
OEI-06-21-00030  Repeat similar events.  When a patient experienced repeat similar events during a hospital stay, we 
generally counted each event separately.  For example, if a patient fell twice during the hospital stay 
and was injured both times, we counted both falls as events. 
Exceptions: The following are exceptions to separately counting multiple, similar events. 
  We collapsed multiple hypoglycemic events within 24 hours into a single event because these 
repeated events are often related to a single cause (e.g., insulin or other diabetic agents) that 
can affect a patient for an extended time (e.g., 24 hours).  We counted recurrences after 
24 hours as separate events. 
  We collapsed multiple pressure injuries into a single event when the pressure injury (1) was at 
the same anatomical site, including bilateral pressure injury of the same anatomical site (e.g., 
bilateral buttocks), and (2) occurred within 24 hours of the initial pressure injury.  We counted 
pressure injuries at different anatomic sites as separate events.  This is because they have 
different causes or means of prevention.  For example, pressure injuries of the foot are often 
caused by not using a foam boot, whereas pressure injuries of the buttock are often caused by 
failure to turn the patient.   
Drug abuse.  Although we included any adverse drug event that occurs in a health care setting, our 
reviewers considered the patients history of drug abuse or dependence when assessing whether an 
event was preventable.  A harm event may not be preventable if a drug-dependent patient failed to 
disclose recent drug use prior to admission, posing increased risk for interaction effects or oversedation.   
End of life and hospice.  Acute care patients at the end of life may be converted to hospice during 
their stay.  For these patients, we included certain harm events if the harm was not the intended result 
of palliative care.  For example, our reviewers would consider central-line infections to be harm events 
for hospice patients.  If patients were particularly susceptible to the harm, given their frailty status or 
other risk factors, reviewers designated the harm as likely not preventable.  In contrast, a fall from bed 
with a resultant fracture or other injury would be considered preventable harm if staff did not take the 
appropriate preventative measures.  Oversedation due to medication was not considered a harm event 
if the intention was to improve comfort and care for these patients.   
In some cases, providers had to weigh the potential impact of appropriate intervention with known side 
effects or physical toll on an already fragile patient.  For example, providers may decide to avoid routine 
turning to lessen a hospice patients pain.  In such situations, reviewers may determine that a related 
pressure injury is a nonpreventable harm event.  In addition, our physician reviewers considered 
documented patient or family wishes to determine if end-of-life events were preventable. 
 
Adverse Events Toolkit: Medical Record Review Methodology  Complex Cases | 12 
OEI-06-21-00030  MEDICAL RECORD REVIEW PROTOCOL 
In this section, we provide a simplified and condensed version of OIGs physician protocol for collecting 
information about adverse events from the medical record.  Our reviewers captured data elements 
listed below for each harm event identified during their medical record reviews and entered their 
responses into an electronic instrument. 
Core Data Elements  
1.  Harm: Did you find patient harm?   
 Yes 
 No 
2.  Harm Event Date: On what date did the harm event begin? __________________ 
3.  Description of Event: Briefly describe the harm event in one-two sentences; include the cause of 
the event. 
 
 
 
 
4.  Clinical Cause: Identify the commission or omission that precipitated this harm event.   
 
 
 
 
5.  Background: Provide background information about the patients condition, including medical 
history, comorbidities, history of present illness, and reason for hospitalization.  
 
 
 
 
6.  Evidence: Describe the evidence in the medical record that supports your findings, including signs 
and symptoms, exam results, laboratory results, imaging results, and other clinical findings.   
 
 
 
 
   
Adverse Events Toolkit: Medical Record Review Methodology  Medical Record Review Protocol | 13 
OEI-06-21-00030   
7.  Intervention: Describe the medical intervention administered to treat the event. 
 
 
 
Severity and Preventability Data Elements 
8.  Severity Level: Which severity level best describes this harm event?  
 E  Temporary harm, intervention   G  Permanent patient harm 
required and/or performed  
 H  Life-sustaining intervention 
 F  Prolonged inpatient stay, elevation  required 
in level of care, or transfer to another 
 I  Contributed to death 
facility 
9.  Preventability Rating: Provide your clinical assessment regarding whether this event was 
preventable and select the factors that contributed to the harm event.  
 Clearly preventable   Likely not preventable 
 Likely preventable   Unable to determine 
 Clearly not preventable 
10. Contributing Factors: Select the factors that contributed to the event.  (See Appendix B for 
Contributing Factors used in OIG reports.) 
11. Preventability Rationale: In your own words, describe the reason for the preventability rating and 
rationale(s) selected. 
 
 
 
Harm Event Category and Type Data Elements 
12. Clinical Category: Which clinical category is most related to this harm event?  
 Medication    Procedure/Surgery 
 Patient Care   Infection 
13. Harm Event Type: Which event type best describes this harm event?  (See the Adverse Events 
Toolkit: Clinical Guidance for Identifying Harm for examples of common harm events we identified 
in our reviews.) 
Adverse Events Toolkit: Medical Record Review Methodology  Medical Record Review Protocol | 14 
OEI-06-21-00030  14. NQF/HAC Event:  Was the event on the CMS HAC and/or NHQ lists?  (This inquiry was composed 
of a series of skip pattern questions to facilitate navigation and overlap of these lists.  See CMSs 
HACRP and DRA-HAC lists, and the NQF SRE list for more information.) 
Quality Assurance-related Data Elements 
15. Cascade: Is this harm a cascade event? 
 Yes 
 No 
16. Group Discussion Call: Would you like to discuss this admission during a consensus call?  (See 
Group discussions for more information about these calls.) 
 Yes 
 No 
17. Present on Admission: Was the harm event present on admission? 
 Yes 
 No 
18. Portion of Stay: During which portion of the stay did the harm event begin?  
 During the inpatient hospitalization   In a contiguous observation stay 
 In a contiguous emergency room visit   Other (describe) 
 In a contiguous outpatient department   Unable to determine (describe) 
visit 
19. Triggers: Which triggers were most helpful in identifying this potential harm event?  (See the 
Adverse Events Toolkit: Clinical Guidance for Identifying Harm for the list of triggers used.) 
20. Record Quality: Did the quality of the medical record documentation negatively affect your ability 
to review this admission?  
 Yes  
 No  
21. Anything Else: Is there anything else about this admission that you would like to convey, such as 
care problems that did not lead to harm? 
 Yes 
 No 
   
Adverse Events Toolkit: Medical Record Review Methodology  Medical Record Review Protocol | 15 
OEI-06-21-00030  QUALITY ASSURANCE 
In this section, we provide suggestions for ensuring the quality and integrity of medical record reviews 
to identify harm events.  Quality assurance reviews ensure accuracy and consistency by helping 
promote standardization across reviews and by minimizing false-positive (as well as false-negative) 
results.  Below is a description of the quality assurance practices used in our reviews. 
Pre-review trainings.  Prior to beginning medical record reviews, we conducted training sessions for all 
OIG reviewers to ensure that they understood our screening method, definition of harm, data collection 
instrument, and study protocols and practices found in this toolkit.  Each reviewer performed pre-test 
reviews and received feedback on the results of those reviews.  These training sessions and pre-test 
reviews helped improve reviewer accuracy and consistency. 
Insufficient records.  We also assessed each record for completeness prior to beginning the reviews.  If 
critical components of a record were missing, we considered excluding the record from the reviews.  In 
cases where the records appeared to be complete, but some portion of documentation was insufficient 
to determine the circumstances surrounding an event, we instructed our reviewers to rely on their 
clinical judgment.  Our reviewers based their harm event and preventability determinations on clear and 
logical conclusions supported by existing documentation.  In some cases, it was clear that a patient 
harm event occurred, but it was not clear why it occurred.  In such cases, our reviewers chose to identify 
preventability as unable to determine. 
Documentation of events.  During the process of the reviews, we instructed our reviewers to capture 
sufficient details from the medical records to support their findings.  Without sufficient details, 
reviewers may need to re-review the medical record to answer questions that might arise during group 
discussions and quality assurance reviews, which can be time-consuming and costly.  We provide 
documentation recommendations for individual conditions in the Adverse Events Toolkit: Clinical 
Guidance for Identifying Harm.   
Group discussions.  Throughout the duration of the study, we facilitated regular conference calls to 
promote consistency across reviewers and to reach a final consensus determination about harm events.  
During these calls, reviewers discussed categories of events as follows: 
  Complex or unusual,     Possible implications for other events,  
  Difficult to assess,     Contributed to or resulted in death,  
  Involving clinical matters outside their    Cascade event, and/or 
area of expertise,  
  Unable to determine preventability. 
We documented the discussions and conclusions made during these calls and compared these notes to 
the determinations to further promote consistency.  
Once physician reviewers completed initial reviews and discussions for all cases, we compared 
groupings of cases for each type of event to identify outliers in severity levels and preventability 
Adverse Events Toolkit: Medical Record Review Methodology  Quality Assurance | 16 
OEI-06-21-00030  determinations.  The physician reviewers discussed case-specific differences in the cases to ensure 
consistency across reviewers. 
Systematic logic checks.  During the reviews, we conducted systematic logic checks of each reviewers 
findings by reading through case summaries to identify inconsistencies and through data analysis of 
key variables.  These checks involved identification of missing values, aberrancies, and logical 
inconsistencies.  They were employed for all records in our sample.  After conducting the systematic 
logic checks, we followed up with individual reviewers and reassessed information in the records as 
needed.  The following were important checks during our review: 
  The harm date recorded by the reviewer was not before admission or after discharge, 
  Infections were not counted as harm if diagnosed before the third calendar day after admission 
(with exceptions for Clostridioides difficile and infections attributable to devices), 
  The description of the harm or evidence recorded by the reviewer was consistent with the harm 
event types criteria (see the Adverse Events Toolkit: Clinical Guidance for Identifying Harm), 
  Evidence recorded by the reviewer supported the harm events severity level,  
  The rationale provided by the physician in the second stage of review explained the reason for 
not confirming a potential harm event identified in the first stage of review, 
  Preventability determinations were supported by narrative explanations and similar cases were 
assessed consistently across reviewers unless case-specific differences were noted, and 
  Cascade events only included a series of logically connected harm events and did not include 
unrelated harm events. 
Quality assurance re-reviews.  We also conducted quality assurance re-reviews, in which a second 
reviewer reviewed the medical record and compared their responses to those of the initial reviewer.  We 
conducted these re-reviews to ensure the accuracy of reviewers findings and their adherence to the 
guidelines in this toolkit.  We conducted two main types of quality assurance re-reviews listed below. 
  In the first stage, we re-reviewed all records that were not forwarded to the second stage of 
review to identify any potential harm events missed by screeners using the GTT method.  This 
review allowed us to identify false negative results. 
  In the second stage, we re-reviewed a small sample of records that were forwarded to the 
second stage of review to validate the physician review results.  These records were selected 
after the initial physician review but were cases that were not included in group discussions.  
These re-reviews supplemented the quality assurance efforts of the group discussions and 
systematic logic checks. 
We selected the medical records for re-review based on potential areas of concern (e.g., complicated 
cases not included in group discussions) and to ensure representation from all reviewers.   
 
Adverse Events Toolkit: Medical Record Review Methodology  Quality Assurance | 17 
OEI-06-21-00030  APPENDICES 
Appendix A: IHI Methodology for Identifying Harm 
For our medical record screening process, we adopted and modified the Institute for Healthcare 
Improvements (IHIs) Global Trigger Tool (GTT) process.  Understanding the modifications and reasons 
for them may be helpful to hospitals and researchers as they develop and/or refine their own medical 
record review procedures.  Toolkit users may want to consult additional resources for adapting the GTT 
for their facilities.6  
IHI developed the IHI GTT as a tool for hospitals to assess patient safety by providing on-going, 
statistically valid monitoring of patient harm, including trends of events over time.  The methodology 
relies on a retrospective medical record review to identify triggers that then prompt reviewers to 
search for adverse events.7, 8  This method was designed to be used primarily by acute care hospitals 
while conserving limited resources.  To increase efficiency of reviews, IHI recommends using two 
independent medical record reviewers (e.g., nurses) who are limited to 20-minute reviews of the 
records.  The reviewer determines if harm was present and its severity.  Then, a physician, who is not 
tasked with conducting reviews, authenticates the results from the primary reviewers and arbitrates any 
differences of opinion.  IHI also recommended criteria regarding which events to include or not include 
in results.  More information about IHIs GTT method can be found at IHIs website. 
In contrast, our primary objective was to provide statistically valid point-in-time estimates of patient 
harm across the Nation for use in policy decisions and planning.  We conducted two stages of review.  
Nurses screened the records for triggers in the first stage, and physicians conducted a full medical 
record review in the second stage.  We allowed longer timeframes for reviews to reduce the likelihood 
of false negative results (i.e., missing events in the record).  We gave physicians a more significant 
responsibility for identifying events and encouraged discussion across reviewers to improve the 
accuracy of our decisions, particularly with respect to complex cases.  This also allowed us to gather rich 
clinical information about each harm event with nuanced detail about the event and its circumstances, 
such as the patients condition prior to and following the event.  A hospital with the goal of monitoring 
basic trends over time may not require the same level of rigor in its estimates of harm. 
In addition to these differences in the review process, we captured and classified harm events 
differently.  We expanded our definition of harm to include events that were a result of omission of care 
(e.g., delayed care) because we believe these events can be correctly identified during our in-depth 
review process and are of interest to clinicians and policymakers.  On the other hand, we did not include 
present on admission (POA) events because we attributed the patient harm events to the hospital or 
other health care setting that provided the care and did not want to include events that occurred in the 
extended community.  We were also interested in distinguishing events by preventability because we 
believe that this information can help policymakers and clinicians identify where to focus resources.  
IHIs alternative approach is designed to identify a broad range of common harm events in a way that 
can be replicated over time.  Standards of care and expectations about preventability can evolve over 
time and affect results.  See Exhibit A-1 for a summary of key differences between the OIG and IHI 
methods. 
Adverse Events Toolkit: Medical Record Review Methodology  Appendix A | 18 
OEI-06-21-00030  Exhibit A-1: Key Differences between the OIG and IHI Methods in Identifying Harm  
Component  OIG Method  IHI Method 
   
ELEMENTS OF REVIEW 
Primary Objective  OIGs primary objective is to provide  IHIs primary objective for the GTT is 
statistically valid estimates of patient  to provide a tool for hospitals to 
harm for use in policy decisions and  assess patient safety and provide 
planning.    on-going monitoring of all patient 
harms through tracking and 
trending over time. 
Number of Reviewers per  One screener (typically a nurse) and  Two reviewers (typically a nurse, 
Patient Record  1 physician reviewer if the record is  pharmacist, or respiratory therapist) 
referred by the screener  and 1 physician who authenticates 
the results from the 2 reviewers and 
arbitrates differences in findings of 
harm 
Process for Arbitrating  Consensus by a panel of physician  One physician who serves as a 
Complex Cases  reviewers from multiple specialties,  tiebreaker if reviewers disagree 
(typically 5-7 members) 
Time Limit for Medical Record  No   Yes  
Review per Patient Record  (typically, 30 to 45 minutes)  (20 minutes) 
   
HARM EVENT CRITERIA 
Include POA Events  No  Yes 
Include Commission Events  Yes  Yes 
Include Omission Events  Yes  No 
Include Events That Are Not  Yes  Yes 
Preventable 
Include Events Related to  No  No 
Underlying Disease 
   
HARM CLASSIFICATION 
Use a Modified NCC MERP  Yes*  Yes 
Index for Severity Level 
Distinguish Between Adverse  Yes  No 
Events and Temporary Harm 
Events 
Distinguish by Harm   Yes  No 
Event Preventability 
 Source: OIG comparison of OIG-modified GTT with IHI GTT method at Griffin, F.A., and Resar, R.K., IHI Global Trigger Tool for Measuring 
Adverse Events (Second Edition), Institute for Healthcare Improvement Innovation Series, 2009. 
*OIG expanded severity level F on the NCC MERP index to include harm that contributed to or resulted in an elevation in level of care, transfer 
to another facility, or subsequent admission.   
Adverse Events Toolkit: Medical Record Review Methodology  Appendix A | 19 
OEI-06-21-00030  Appendix B: Contributing Factors for Preventability 
Preventable Factors That Contributed to the Event 
Inadequate admission assessment  Equipment failure or other system breakdown 
Inadequate care plan  Poor communication between caregivers 
Inadequate monitoring   Possible abuse, neglect, or other trauma 
associated with care 
Substandard or inadequate preventative care  Lack of access to physician/specialist 
Substandard treatment or therapeutic care   Event that rarely happens when proper 
precautions and procedures are followed 
Necessary treatment not provided  Other, describe 
Error related to medical judgment, skill, or   
patient management 
 
Nonpreventable Factors That Contributed to the Event 
Event occurred even though providers followed  Patient was highly susceptible to this type of 
proper preparation and procedures  event due to health status 
Provider could not have anticipated this event  Harm was foreseeable but was considered 
with the information available at the time  acceptable given alternatives 
Patients diagnosis was unusual or complex,  Other, describe 
making care difficult 
Patients treatment was unusual or complex,   
making care difficult  
 
Rationales for Unable to Determine 
Poor and/or absent documentation  Patients health status was particularly complex 
or unusual, making determination difficult 
Care was particularly complex or unusual,  Other, describe 
making determination difficult 
   
Adverse Events Toolkit: Medical Record Review Methodology  Appendix B | 20 
OEI-06-21-00030  ACKNOWLEDGMENTS AND CONTACT 
Acknowledgments 
Demetrius Martinez served as the team leader for the development of this toolkit.  
Office of Evaluation and Inspections headquarters staff who provided support include 
Joe Chiarenzelli, Robert Gibbons, Jennifer Gist, and Sara Swisher. 
We would also like to acknowledge the contributions of the Office of Inspector 
Generals Chief Medical Officer, Dr. Julie Taitsman.   
This toolkit was prepared under the direction of Ruth Ann Dorrill, Regional Inspector 
General for Evaluation and Inspections in the Dallas Regional Office, and 
Amy Ashcraft, Deputy Regional Inspector General. 
Contact 
To obtain additional information concerning this report, contact the Office of Public 
Affairs at Public.Affairs@oig.hhs.gov.  OIG reports and other information can be found 
on the OIG website at oig.hhs.gov. 
Office of Inspector General 
U.S. Department of Health and Human Services 
330 Independence Avenue, SW 
Washington, DC 20201 
 
Adverse Events Toolkit: Medical Record Review Methodology   
OEI-06-21-00030   Acknowledgments and Contact | 21   ABOUT THE OFFICE OF INSPECTOR GENERAL 
The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, as amended, is 
to protect the integrity of the Department of Health and Human Services (HHS) programs, as well as the 
health and welfare of beneficiaries served by those programs.  This statutory mission is carried out 
through a nationwide network of audits, investigations, and inspections conducted by the following 
operating components: 
The Office of Audit Services (OAS) provides auditing services for HHS, either by conducting 
audits with its own audit resources or by overseeing audit work done by others.  Audits examine the 
performance of HHS programs and/or its grantees and contractors in carrying out their respective 
responsibilities and are intended to provide independent assessments of HHS programs and 
operations.  These audits help reduce waste, abuse, and mismanagement and promote economy and 
efficiency throughout HHS. 
The Office of Evaluation and Inspections (OEI) conducts national evaluations to provide 
HHS, Congress, and the public with timely, useful, and reliable information on significant issues.  These 
evaluations focus on preventing fraud, waste, or abuse and promoting economy, efficiency, and 
effectiveness of departmental programs.  To promote impact, OEI reports also present practical 
recommendations for improving program operations. 
The Office of Investigations (OI) conducts criminal, civil, and administrative investigations of 
fraud and misconduct related to HHS programs, operations, and beneficiaries.  With investigators 
working in all 50 States and the District of Columbia, OI utilizes its resources by actively coordinating 
with the Department of Justice and other Federal, State, and local law enforcement authorities.  The 
investigative efforts of OI often lead to criminal convictions, administrative sanctions, and/or civil 
monetary penalties. 
The Office of Counsel to the Inspector General (OCIG) provides general legal services 
to OIG, rendering advice and opinions on HHS programs and operations and providing all legal support 
for OIGs internal operations.  OCIG represents OIG in all civil and administrative fraud and abuse cases 
involving HHS programs, including False Claims Act, program exclusion, and civil monetary penalty 
cases.  In connection with these cases, OCIG also negotiates and monitors corporate integrity 
agreements.  OCIG renders advisory opinions, issues compliance program guidance, publishes fraud 
alerts, and provides other guidance to the health care industry concerning the anti-kickback statute and 
other OIG enforcement authorities.
Adverse Events Toolkit: Medical Record Review Methodology   
OEI-06-21-00030   About OIG | 22   REFERENCES 
 
1 OIG, Adverse Events in Hospitals: A Quarter of Medicare Patients Experienced Harm in October 2018, OEI-06-18-00400, 
May 2022. 
2 OIG, Adverse Events in Hospitals: Methods for Identifying Events, OEI-06-08-00221, March 2010. 
3 In some cases, patients had multiple admissions in OIGs sample period.  If an event was attributable to care provided during 
a sample admission, we included it in our harm rate, regardless of whether it was a POA event for a subsequent admission.   
4 Exception: We included harm events that occurred in the emergency department (ED), observation unit, or other outpatient 
department contiguous to the inpatient admission as long as the patient was directly admitted to the hospital.  For example, 
we would include a patient who experienced a harm event during outpatient surgery and was then directly admitted to the 
hospital as an inpatient.  We also counted harm events that occurred during a prior hospital stay if it was within our study 
period but that resulted in a readmission 30 days after discharge. 
5 NCC MERP, Index for Categorizing Medication Errors, 2001.  Accessed at 
http://www.nccmerp.org/sites/default/files/indexColor2001-06-12.pdf on April 27, 2021. 
6 For more information on the development of the triggers and their applications to different health care settings, Classen, et 
al., Development and Evaluation of the Institute for Healthcare Improvement Global Trigger Tool, Journal of Patient Safety, 
Volume 4, Issue 3, September 2008.  Accessed at https://journals.lww.com/journalpatientsafety/Abstract/2008/09000/ 
Development_and_Evaluation_of_the_Institute_for.6.aspx on May 25, 2023.  Also see Hibbert, et al., The application of the 
Global Trigger Tool: a systematic review, International Journal for Quality in Health Care, Volume 28, Issue 6, December 2016.  
Accessed at https://doi.org/10.1093/intqhc/mzw115 on May 25, 2023. 
7 Griffin F.A., and Resar R.K., IHI Global Trigger Tool for Measuring Adverse Events (Second Edition), Institute for Healthcare 
Improvement Innovation Series, 2009.  Accessed at https://www.ihi.org/resources/Pages/Tools/IHIGlobalTriggerToolfor 
MeasuringAEs.aspx on May 25, 2023. 
8 IHIs first GTT was aimed at detecting adverse medication events.  IHIs GTT methods have since expanded to a wider range of 
triggers for different types of adverse events and across different health care settings. 
Adverse Events Toolkit: Medical Record Review Methodology   
OEI-06-21-00030   References | 23