A Cluster Randomized Controlled Trial Comparing the Safety Action Feedback and Engagement (SAFE) Loop With an Established Incident Reporting System
Phase: NA
Conditions: Medication Errors, Patient Safety
Sponsor: Cedars-Sinai Medical Center

 
 
 
SAFE Loop Grant Proposal v.3 (20211015) 
 
IRB HRP-651 - 9.8.2020 
A Cluster Randomized Controlled Trial Comparing the 
Safety Action Feedback and Engagement (SAFE) Loop 
with an Established Incident Reporting System 
 
 
Protocol Number: 1 
National Clinical Trial (NCT) Identified Number:  N/A 
Principal Investigator: Teryl K. Nuckols, MD, MSHS 
Funded by: Agency for Healthcare Research and Quality 
Version Number:  22 March 2022 
 
Summary of Changes from Previous Version: 
Affected 
Section(s) 
Summary of Revisions Made 
Rationale 
5/6/2021 
Clarified that all surveys and interviews of 
nurses will occur during nursing hours. (p.10) 
Changed incentive for interviews from gift 
cards to a lottery for an iPad (p. 10, 20). 
Added details on dissemination of intervention 
and recruitment procedures (p. 10, 11).  
The intervention itself is considered a part of 
usual quality improvement activities. 
Change req by HR because gift cards are 
wages but a lottery for an iPad is not. 
Ensure recruitment of nurses working 
evenings and weekends. 
10/15/2021 
Revised to add information about the Advisory 
Board in section 10.1.5. 
 
3/22/2022 
Implemented changes to hopefully increase 
response rate of nursing surveys: 1) study team 
will attend huddles and other meetings as 
feasible, 2) recruitment emails will be sent 
from the nursing director, 3) snacks will be 
provided regardless of participation, (section 
5.5). 
Response rates have been very low so far. 
These changes are to hopefully increase the 
response rate by increasing awareness of 
the survey and its importance. 

  
 
Table of Contents 
STATEMENT OF COMPLIANCE ............................................................................................................................. 1 
1 
PROTOCOL SUMMARY ............................................................................................................................... 1 
1.1 
Synopsis ................................................................................................................................................ 1 
1.2 
Schema ................................................................................................................................................. 3 
1.3 
Schedule of Activities (SoA) ............................................................................................................... 3 
2 
INTRODUCTION ............................................................................................................................................ 1 
2.1 
Study Rationale .................................................................................................................................... 1 
2.2 
Background........................................................................................................................................... 1 
2.3 
Risk/Benefit Assessment .................................................................................................................... 3 
2.3.1 
Known Potential Risks ..................................................................................................... 3 
2.3.2 
Known Potential Benefits ................................................................................................ 5 
2.3.3 
Assessment of Potential Risks and Benefits ................................................................ 5 
3 
OBJECTIVES AND ENDPOINTS ................................................................................................................ 5 
4 
STUDY DESIGN ............................................................................................................................................. 7 
4.1 
Overall Design ...................................................................................................................................... 7 
4.2 
Scientific Rationale for Study Design................................................................................................ 7 
4.3 
Justification for Dose ........................................................................................................................... 7 
4.4 
End of Study Definition ....................................................................................................................... 8 
5 
STUDY POPULATION .................................................................................................................................. 8 
5.1 
Inclusion Criteria .................................................................................................................................. 8 
5.2 
Exclusion Criteria ................................................................................................................................. 9 
5.3 
Lifestyle Considerations ...................................................................................................................... 9 
5.4 
Screen Failures .................................................................................................................................... 9 
5.5 
Strategies for Recruitment and Retention ........................................................................................ 9 
6 
STUDY INTERVENTION ............................................................................................................................ 10 
6.1 
Study Intervention(s) Administration ............................................................................................... 10 
6.1.1 
Study Intervention Description ..................................................................................... 10 
6.1.2 
Dosing and Administration ............................................................................................ 13 
6.2 
Preparation/Handling/Storage/Accountability ................................................................................ 14 
6.3 
Measures to Minimize Bias: Randomization and Blinding ........................................................... 14 
6.4 
Intervention Compliance ................................................................................................................... 14 
6.5 
Concomitant Therapy ........................................................................................................................ 16 
6.5.1 
Rescue Medicine ............................................................................................................ 16 
7 
STUDY INTERVENTION DISCONTINUATION AND PARTICIPANT 
DISCONTINUATION/WITHDRAWAL .................................................................................................................... 16 
7.1 
Discontinuation of Study Intervention ............................................................................................. 16 
7.2 
Participant Discontinuation/Withdrawal from the Study ............................................................... 16 
7.3 
Lost to Follow-Up ............................................................................................................................... 16 
8 
STUDY ASSESSMENTS AND PROCEDURES ..................................................................................... 16 
8.1 
Efficacy Assessments ....................................................................................................................... 16 
8.1.1 
Unanticipated Problem Reporting ................................................................................ 20 
8.2 
Safety and Other Assessments ....................................................................................................... 21 
8.3 
Adverse Events and Serious Adverse Events ............................................................................... 22 
8.3.1 
Definition of Serious Adverse Events (SAE) .............................................................. 22 
8.3.2 
Classification of an Adverse Event .............................................................................. 23 
8.3.3 
Time Period and Frequency for Event Assessment and Follow-Up ....................... 24 

  
 
8.3.4 
Adverse Event Reporting .............................................................................................. 24 
8.3.5 
Serious Adverse Event Reporting ............................................................................... 25 
8.3.6 
Reporting Events to Participants ................................................................................. 25 
8.3.7 
Events of Special Interest ............................................................................................. 25 
8.3.8 
Reporting of Pregnancy ................................................................................................ 25 
8.4 
Unanticipated Problems .................................................................................................................... 25 
8.4.1 
Definition of Unanticipated Problems (UP) ................................................................. 25 
8.4.2 
Reporting Unanticipated Problems to Participants ................................................... 26 
9 
STATISTICAL CONSIDERATIONS .......................................................................................................... 26 
9.1 
Statistical Hypotheses ....................................................................................................................... 26 
9.2 
Sample Size Determination .............................................................................................................. 26 
9.3 
Populations for Analyses .................................................................................................................. 28 
9.4 
Statistical Analyses............................................................................................................................ 28 
9.4.1 
General Approach .......................................................................................................... 28 
9.4.2 
Analysis of the Primary Efficacy Endpoint(s) ............................................................. 28 
9.4.3 
Analysis of the Secondary Endpoint(s) ....................................................................... 30 
9.4.4 
Safety Analyses .............................................................................................................. 30 
9.4.5 
Baseline Descriptive Statistics ..................................................................................... 30 
9.4.6 
Planned Interim Analyses ............................................................................................. 30 
9.4.7 
Sub-Group Analyses ..................................................................................................... 31 
9.4.8 
Tabulation of Individual participant Data .................................................................... 31 
9.4.9 
Exploratory Analyses ..................................................................................................... 31 
10 
SUPPORTING DOCUMENTATION AND OPERATIONAL CONSIDERATIONS .............................. 31 
10.1 
Regulatory, Ethical, and Study Oversight Considerations ........................................................... 31 
10.1.1 
Informed Consent Process ........................................................................................... 31 
10.1.2 
Study Discontinuation and Closure ............................................................................. 32 
10.1.3 
Confidentiality and Privacy ........................................................................................... 32 
10.1.4 
Future Use of Stored Specimens and Data ............................................................... 33 
10.1.5 
Key Roles and Study Governance .............................................................................. 33 
10.1.6 
Safety Oversight ............................................................................................................. 35 
10.1.7 
Clinical Monitoring .......................................................................................................... 36 
10.1.8 
Quality Assurance and Quality Control ....................................................................... 36 
10.1.9 
Data Handling and Record Keeping ............................................................................ 36 
10.1.10 
Protocol Deviations ........................................................................................................ 39 
10.1.11 
Publication and Data Sharing Policy ........................................................................... 39 
10.1.12 
Conflict of Interest Policy .............................................................................................. 40 
10.2 
Additional Considerations ................................................................................................................. 40 
10.3 
Abbreviations ...................................................................................................................................... 41 
10.4 
Protocol Amendment History ........................................................................................................... 43 
11 
REFERENCES ............................................................................................................................................. 45 
 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
1 
STATEMENT OF COMPLIANCE 
 
The trial will be conducted in accordance with International Conference on Harmonisation Good Clinical 
Practice (ICH GCP), applicable United States (US) Code of Federal Regulations (CFR), and the Agency for 
Healthcare Research and Quality Terms and Conditions of Award. The Principal Investigator will assure 
that no deviation from, or changes to the protocol will take place without prior agreement from the 
funding agency and documented approval from the Institutional Review Board (IRB), except where 
necessary to eliminate an immediate hazard(s) to the trial participants. All personnel involved in the 
conduct of this study have completed Human Subjects Protection and ICH GCP Training. 
 
The protocol, informed consent form(s), recruitment materials, and all participant materials will be 
submitted to the Institutional Review Board (IRB) for review and approval.  Approval of both the 
protocol and the consent form must be obtained before any participant is enrolled.  Any amendment to 
the protocol will require review and approval by the IRB before the changes are implemented to the 
study.  In addition, all changes to the consent form will be IRB-approved; a determination will be made 
regarding whether a new consent needs to be obtained from participants who provided consent, using a 
previously approved consent form. 
 
1 
 
PROTOCOL SUMMARY 
1.1 
SYNOPSIS  
Title: 
A Cluster Randomized Controlled Trial Comparing the Safety Action 
Feedback and Engagement (SAFE) Loop with an Established Incident 
Reporting System 
Study Description: 
This cluster RCT will test whether a novel  intervention, the Safety Action 
Feedback and Engagement (SAFE) Loop, enhances incident reporting 
practices, improves nurses’ perceptions of incident reporting, and lowers 
rates of high-priority medication events, as compared with using an 
existing incident reporting system.   
Objectives: 
 
This project seeks to transform hospitals’ existing voluntary incident 
reporting systems into effective tools for improving patient safety. In this 
cluster RCT, we propose to test whether, compared with using an existing 
incident reporting system, implementing the SAFE Loop in acute care 
nursing units: 
• 
Aim 1: improves incident reporting practices by increasing the rate at 
which nurses report high-priority medication incidents (H1.1) and the 
number of contributing factors described per report (H1.2). 
• 
Aim 2: improves nurses’ attitudes toward incident reporting, including 
perceptions of feedback and communication about error (H2.1) and of 
the frequency with which events are reported (H2.2).  
• 
Aim 3: reduces rates of high-priority medication events (H3.1) and 
high-priority medication events involving harm (H3.2). 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
2 
Endpoints: 
1) Incident reporting practices (rates at which nurses report high-priority 
medication incidents and numbers of contributing factors described per 
report). 
2) Nurses’ attitudes toward incident reporting using the AHRQ Hospital 
Survey on Patient Safety Culture™ (SOPS), particularly perceptions of 
feedback and communication about error and of the frequency with which 
events are reported. 
3) Rates of high-priority medication events (determined via Institute for 
Healthcare Improvement Trigger Tool method of medical record review).  
 
Qualitative evaluation of implementation as per opinions of nurses. 
Study Population: 
A) Nurses: All 1,980 nurses on the nursing units will be eligible to 
participate in the SAFE Loop as well as Aims 1 and 3. Nurses will be eligible 
for participating in the survey (Aim 2) if they worked >50% time on one 
study nursing unit during a 6-month study period. Nursing unit managers 
and nurses in the study units will also be eligible to participate in semi-
structured interviews to help characterize the implementation of SAFE 
Loop (32 participants). 
  
B) Patients: Cedars-Sinai has 49,000 inpatient, 7,700 observation, and 
58,000 emergency department encounters in a typical year, resulting in a 
total of 294,470 patient-days of care in study nursing units per year. Under 
Aim 1, incident reports for about 1,663 patients will be analyzed. Under 
Aim 3, medical records for 1,520 eligible hospitalizations during the SAFE 
Loop study period will be randomly sampled and analyzed.  
Description of 
Sites/Facilities Enrolling 
Participants: 
20 Cedars-Sinai acute care nursing units in the main Medical Center 
Description of Study 
Intervention: 
We will conduct a cluster RCT to compare nursing units that implement 
the SAFE Loop, a system-level change in incident reporting, with nursing 
units that continue to use the existing reporting system. The Safety Action 
Feedback and Engagement (SAFE) Loop has three phases: Engagement, 
Fact-finding, and Action & Feedback.  
 
Across the three phases, the SAFE Loop has five Key Attributes: (1) 
obtaining input from nurses on which problems to address; (2) focusing on 
selected high-priority events, (3) prompting nurses to report the high-
priority events for a designated period and write more informative 
reports; (4) following standardized investigative procedures to integrate 
information from sources internal and external to CSMC, and (5) providing 
feedback to nurses about safety problems and mitigation plans. During all 
phases, the SAFE Loop Team collaborates with frontline nurses and Unit 
Managers. (See schemata in Section 1.2) 
Study Duration: 
60 months 
Participant Duration: 
24 weeks 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
3 
1.2 
SCHEMA 
 
Figure: SAFE Loop Reporting System (intervention arm, which is added to conditions in control arm) 
 
 
 
1.3 
SCHEDULE OF ACTIVITIES (SOA) 
Timeline depicting individual nursing unit involvement in 3 phases and 5 attributes of SAFE Loop 
Week→ 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
… 
24 
1. Engagement 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Obtain input 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Select Target Event 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2. Fact-finding 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Train nurses  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Stimulate reporting 
 
 
 
 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
 
 
 
… 
 
Analyze & investigate  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
… 
 
Integrate information 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
… 
 
3. Action, Feedback 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Develop plan 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Distribute feedback  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Implement, sustain 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
… 
… 
 
 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
4 
Timeline depicting study activities over the entire 5-year study period, including milestones 
♦ 
Milestone 
Year 
→ 
1 
 
 
 
2 
 
 
 
3 
 
 
 
4 
 
 
 
5 
 
 
 
Quarter Year → 
1 
2 
3 
4 
1 
2 
3 
4 
1 
2 
3 
4 
1 
2 
3 
4 
1 
2 
3 
4 
Clinical Events 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Randomize nursing units 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Study Periods 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--Baseline periods 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--Implementation/control periods 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--Follow-on periods (Aim 3) 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Trial Preparation & Oversight 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Prepare DSM Plan, Protocol 
 
 
 
♦ 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Prepare & submit IRB application 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Advisory board meetings 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Develop SAFE Loop training materials 
 
 
♦ 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Monitoring trial & potential effects 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
DSM oversight assessments 
 
 
 
 
 
 
♦ 
 
 
 
 
 
 
 
 
♦ 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Data Collection 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Incident reports (Aim 1, retrospective) 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--Refine & test scoring tools 
 
♦ 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--Train abstractors 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--Evaluate incident reports 
 
 
 
 
 
 
 
 
 
 
 
 
♦ 
 
 
 
 
 
 
 
Surveys (Aim 2, prospective) 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--Develop & test survey interface 
 
 
♦ 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--Survey clinicians & staff 
 
 
 
 
 
 
 
 
 
♦ 
 
 
 
 
 
 
 
 
 
 
Med events (Aim 3, retrospective) 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--Develop chart review screening tools 
 
 
 
♦ 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--Develop event scoring tools 
 
 
 
♦ 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--Train nurse abstractors 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--Screen for possible med events 
 
 
 
 
 
 
 
 
 
♦ 
 
 
 
 
 
 
 
 
 
 
--Train physician reviewers 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--Confirm, classify medication events 
 
 
 
 
 
 
 
 
 
 
 
 
 
♦ 
 
 
 
 
 
 
Qualitative Analysis of Implementation 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--Develop interview guides 
 
 
 
 
 
♦ 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--Conduct interviews 
 
 
 
 
 
 
 
 
 
 
♦ 
 
 
 
 
 
 
 
 
 
--Code and analyze interviews 
 
 
 
 
 
 
 
 
 
 
 
 
♦ 
 
 
 
 
 
 
 
Analysis 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Data cleaning 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Descriptive analyses 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Analyses of primary outcomes 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
♦ 
 
 
 
 
Secondary analyses 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Dissemination  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Manuscripts, conferences 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Media campaign 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Dissemination together with ISMP 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
1 
2 
 
INTRODUCTION 
2.1 
STUDY RATIONALE  
This project seeks to transform hospitals’ existing voluntary incident reporting systems into effective 
tools for improving patient safety. 
Substantial progress has been made in some areas of patient safety, but medication errors and other 
problems continue to harm many thousands of patients each year. In other high-risk industries, 
voluntary incident reporting is widely used to improve safety. Incident reporting is also widely used in 
hospitals, but the reporting systems do not function optimally. Nurses file most reports, but they 
experience multiple barriers to reporting, including uncertainty about what to report, lack of feedback, 
and doubt about how hospitals will use the reports. Nurses submit thousands of reports each year, but 
the reported incidents represent a lower-risk subset of medical errors, the reports impart few actionable 
insights, and no standardized procedures exist for conducting follow-up investigations.  
The Safety Action Feedback and Engagement (SAFE) Loop has five key attributes designed to transform 
hospital incident reporting systems into effective tools for improving patient safety: obtaining nurses’ 
input about which medication safety problems to address; focusing on selected high-priority events; 
prompting nurses to report high-priority events during a designated period and training them to write 
more informative reports; integrating information from reports, investigations, and other internal and 
external sources; and providing feedback to nurses on the problems identified and mitigation plans.  
If effective, the SAFE Loop will have several benefits: increasing nurses’ engagement with reporting, 
producing more informative reports, enabling safety leaders to understand problems and design 
system-based solutions more effectively and more efficiently, and lowering rates of medication errors. 
In turn, receiving feedback about problems and system-based solutions will further improve nurses’ 
perceptions of reporting. In addition to the local benefits to hospitals that implement the SAFE Loop, 
these changes will create secondary benefits nationally by enhancing the functioning of AHRQ Patient 
Safety Organizations, which rely on incident reports as critical sources of insights into safety concerns in 
hospitals and potential solutions. 
 
2.2 
BACKGROUND  
 
Despite decades of effort by policymakers, hospitals, and clinicians, medical errors still contribute to 
numerous patient deaths and injuries in U.S. hospitals each year. In other high-risk industries, such as 
aviation, voluntary incident reporting is a widely used and effective technique through which frontline 
personnel describe events—particularly near misses—that serve as early warnings of safety problems. 
Personnel who witnessed the incidents write narrative descriptions that reveal critical details including 
contributing system and human factors, then organizations analyze the narratives, conduct follow-up 
investigations, and ultimately modify systems to reduce the likelihood of future harm.1,2  
 
Incident reporting systems have existed in U.S. hospitals for over 50 years, but they have not been as 
effective at improving patient safety, for three basic reasons. First, although nurses file most reports and 
believe reporting is a professional responsibility, they experience barriers to reporting including 
uncertainty about which events to report and lack of feedback about how reports are used. Nurses 
describe hospital reporting systems as a “black hole.” Second, hospitals receive thousands of reports 
each year, but most reports address low-risk problems and provide little information on contributing 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
2 
factors. Third, how hospitals follow-up on incident reports varies and no optimal procedures for 
following up have been described. 
Despite Years of Effort, Medical Errors Still Harm Thousands of Hospitalized Patients Each Year. 
Progress has occurred in important areas such as healthcare-associated infection, but improvements in 
other areas have been modest.10,11 A 2010 study revealed that the rate of harms due to medical care 
remained constant at 25% of hospitalizations over a 5-year period.12 The same year, 13.5% of Medicare 
beneficiaries were found to experience adverse events during hospitalization.13 A 2019 study of death 
certificates found that adverse effects of medical treatment increased from 1990 to 2016 due to the 
aging population.14  
Voluntary Incident Reporting Has Been an Effective Strategy for Improving Safety in Other High-risk 
Industries. Voluntary incident reporting is a technique through which frontline personnel describe 
events that posed risks to safety, particularly incidents that were intercepted or happened not to cause 
harm. A fundamental tenet of effective reporting is that first-hand witnesses write narrative descriptions 
that reveal contributing factors (the specific conditions leading up and potentially causing each event), 
sharing insights that might otherwise be rapidly forgotten. By analyzing the narratives and conducting 
follow-up investigations, organizations learn how system factors (facilities, equipment, tasks, personnel 
availability and qualifications, and organizational culture and goals) contributed to incidents, either 
directly or by influencing human performance (human factors, including errors, mistakes, and 
violations). These insights enable organizations to modify systems to reduce the likelihood of future 
harm.1,2  
Voluntary incident reporting has a long history and is widely used in the aviation, maritime, chemical, 
nuclear, and railway industries. Research in these settings has demonstrated that a major accident is 
usually preceded by many weak signals, called near misses (errors that might have led to an adverse 
event but that were intercepted or happened not to cause harm).15 Studies in the oil, food, construction, 
health and transport sectors in the U.K. revealed that, for each major event, there are 7 minor events 
and 189 near misses.16  Furthermore, studies in the aviation and chemical industries showed that near 
misses and harmful incidents share the same causes.17 Thus, reporting near misses expands 
opportunities to avoid future harmful incidents.  
Incident Reporting Systems Already Exist in Most U.S. Hospitals. Whereas safety has been the sole 
purpose in other industries, reporting has served dual objectives in U.S. hospitals since 1965: (1) alerting 
risk managers when a specific event could lead to litigation and (2) preventing problems that are 
common sources of malpractice claims.18 Hospitals rely on nurses to file most reports.3 How hospitals 
handle the reports varies: reports may be transmitted to the manager of the nursing unit, safety leaders, 
and/or risk managers. Reports can trigger investigations locally or systemwide. Sometimes reports are 
used to discipline the individuals involved. Nurses describe what happens after reporting as a “black 
hole” because they seldom learn how reports were used (Figure 1).7,19-21  
Hospital Reporting Systems Are Not Employed Effectively. To share insights about patient safety issues 
and opportunities for improvement, AHRQ created Patient Safety Organizations (PSOs) with the unique 
ability to aggregate incident reports from multiple health systems.22 However, aggregating data requires 
effective reporting systems within hospitals and current systems within hospitals function poorly.19,20  
Three types of problems limit the effectiveness of hospital reporting systems. 1. Nurses and other staff 
experience barriers to reporting, particularly lack of engagement and feedback. While nurses believe 
that incident reporting is a professional obligation, they are often uncertain about what to report and 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
3 
what information to include. Nurses fear triggering punitive responses and doubt whether any 
responsive actions will be taken to improve systems of care.19,23,24 Indeed, lack of feedback is the 
primary deterrent to reporting.24 In over 2000 hospitals, only 13 reporting systems provided any 
feedback to personnel who filed reports.25 The dearth of feedback not only contributes to a sense that 
reports go into a “black hole,” it undervalues the importance of nurses in designing and implementing 
solutions. When feedback is provided, it is a powerful facilitator of reporting.24 2. The reports currently 
received are often of limited usefulness to improving safety. Although underreporting is pervasive, 
hospitals typically receive thousands of reports per year.26,27 Research by Nuckols found that many 
reports describe similar events and address lower-risk problems like infiltrated peripheral intravenous 
(IV) catheters or missed doses of medication.3 Higher-risk incidents are underreported.3 Simply 
increasing the quantity of reports, as opposed to the priority of the reported events, may degrade the 
signal-to-noise ratio, obscuring the ability to identify critical events. Additionally, studies by Nuckols and 
Cohen (Co-I) showed that only 32-65% of reports contain any information on contributing system 
factors.4,8,9 If incident reports do not capture high-priority events or reveal helpful insights, safety 
leaders may learn too little to act on the information in reports. 3. Optimal procedures for following up 
on incident reports in hospitals have not been described. Hospitals have over-emphasized the collection 
and rudimentary analysis of large volumes of reports, unlike in aviation, where systematic investigation 
and prompt action are the goals.28,29  
Specific Changes Would Transform Hospital Reporting Systems into Effective Tools for Improving 
Safety. Engagement is needed between nurses and safety leaders throughout the process of reporting, 
investigation, and improvement. Events that are common, preventable, and potentially severe should be 
the highest priority. The systems should emphasize near misses, to reduce the chance that reporters 
might be blamed. Reports should provide rich detail on contributing factors, facilitating follow-up 
investigations. Lastly, reporting systems should provide feedback about the problems identified and 
mitigation plans.1  
 
2.3 
RISK/BENEFIT ASSESSMENT   
 
2.3.1 KNOWN POTENTIAL RISKS  
There are two categories of study subjects who will be affected by SAFE Loop:  
1. Nurses and nursing unit managers who work in the nursing units, and  
2. Patients who receive care there.  
Table of Study Procedures and Associated Risks for Each Group.  
Study Procedure 
Risks for Nurses 
Risks for Patients 
1. Organizational 
Quality Improvement 
Intervention: 
Implementation of SAFE 
Loop in nursing units  
Psychological distress or loss of 
professional reputation may occur 
if a nurse is blamed for a 
medication error after an incident 
report is submitted. Risks are 
similar to those in usual nursing 
practice because nurses already 
submit incident reports.  
Medication error. Risks are unlikely 
but may occur if the nursing units 
inadvertently implement changes 
that worsen rather than improve 
medication safety, as intended. 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
4 
To further mitigate the risks, we 
will train nurses to emphasize 
reporting of near misses and no-
harm events (which are less likely 
to result in punitive responses by 
unit managers) and train unit 
managers in the “just culture” 
approach, which emphasizes 
changing systems of care rather 
than blaming individual nurses 
when medication errors occur.  
2a. Research data 
collection: nursing work 
schedules, to assess 
nurse eligibility for 
survey 
Inadvertent disclosure of 
individually identifiable 
information on nursing work 
schedules. This data source does 
not include sensitive information.  
Not applicable. 
2b. Research data 
collection: surveys of 
nurses on attitudes 
toward reporting and 
patient safety culture 
Inadvertent disclosure of 
individually identifiable 
information on nursing survey. 
Survey does not involve sensitive 
information and it is already 
routinely deployed in this hospital.  
Not applicable. 
3. Research data 
collection: analysis of 
incident reports to 
assess rate and quality 
of reports 
Inadvertent disclosure of 
individually identifiable 
information. This data source 
includes sensitive information such 
as errors that nurses may have 
committed. Data to be extracted 
will not include individually 
identifiable information about 
nurses. 
Inadvertent disclosure of protected 
health information (PHI). Incident 
reports contain patient identifiers 
and PHI. Data to be extracted will not 
contain patient identifiers aside from 
randomly assigned study 
identification numbers to facilitate 
linkage to other datasets. 
4. Research data 
collection: analysis of 
medical records to 
identify medication 
events  
Not applicable. Data to be 
extracted will not include 
individually identifiable 
information. 
Inadvertent disclosure of PHI. 
Medical records contain patient 
identifiers and PHI. Data to be 
extracted will not contain patient 
identifiers aside from randomly 
assigned study identification 
numbers to facilitate linkage to other 
datasets. 
5. Research data 
collection: interviews 
with nurses who 
Inadvertent disclosure of 
individually identifiable 
information. The interviews will 
not include sensitive information. 
Not applicable. 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
5 
implemented the 
intervention 
6. Research data 
collection: Extraction of 
electronic health record 
variables 
Not applicable.  
Inadvertent disclosure of PHI. Data to 
be extracted will not contain patient 
identifiers aside from randomly 
assigned study ID numbers to 
facilitate linkage to other datasets. 
 
 
2.3.2 KNOWN POTENTIAL BENEFITS  
Nurse participants: Nurses may benefit from the SAFE Loop intervention through improvements in their 
skill in reporting medication errors; such reporting is considered a basic responsibility in nursing 
practice. Additionally, participation may enhance professional satisfaction, because the nurses on each 
study unit come together to choose a medication safety problem to solve and then help to collect useful 
information on that problem by filing incident reports. The intervention is also designed to improve the 
reporting culture at the institution, shifting from a culture that may blame reporters for events to a 
culture that uses reports as tools for improving systems of care.  
Patient participants: Patients may benefit from the SAFE Loop intervention through reductions in rates 
of medication errors. Additionally, improved incident reporting systems may help to lower the rates of 
harmful medication errors at the study hospital.  
Society: If effective, the research will demonstrate a strategy for improving patient safety in hospitals 
through the use of incident reporting. In the future, hospitals that implement the SAFE Loop may 
experience several benefits: increasing nurses’ engagement with reporting, collecting more informative 
reports, enabling safety leaders to understand problems and design system-based solutions more 
effectively and more efficiently, and lowering rates of medication errors. In turn, receiving feedback 
about problems and system-based solutions will further improve nurses’ perceptions of reporting. In 
addition to the local benefits to hospitals, these changes will create secondary benefits nationally by 
enhancing the functioning of AHRQ Patient Safety Organizations, which rely on incident reports as 
critical sources of insights into safety concerns in hospitals and potential solutions.  
2.3.3 ASSESSMENT OF POTENTIAL RISKS AND BENEFITS  
The risks of nurse and patient participation are no greater than they normally encounter during the 
course of clinical care in the hospital setting, while the potential benefits to participants and society are 
substantial due to possible reduction in harmful adverse events. Thus, on balance, the risks of 
participation are counterbalanced by the potential for substantial benefits to participants and society. 
3 
OBJECTIVES AND ENDPOINTS 
 
In this cluster RCT, we propose to test whether, compared with using an existing incident reporting 
system, implementing the SAFE Loop in acute care nursing units:  
Aim 1: improves incident reporting practices by increasing the rate at which nurses report high-priority 
medication incidents (H1.1) and the number of contributing factors described per report (H1.2). 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
6 
Aim 2: improves nurses’ attitudes toward incident reporting, including perceptions of feedback and 
communication about error (H2.1) and of the frequency with which events are reported (H2.2).  
Aim 3: reduces rates of high-priority medication events (H3.1) and high-priority medication events 
involving harm (H3.2). 
 
OBJECTIVES 
ENDPOINTS 
JUSTIFICATION FOR ENDPOINTS 
Primary 
 
 
Aim 1: To determine whether 
SAFE Loop implementation 
increases the number of 
contributing factors described 
per report (H1.2) 
(H1.2) The number of system 
plus human contributing factors 
per incident report 
Reports including more 
contributing factors will allow 
quality improvement personnel 
to develop solutions that are 
better-informed 
Aim 2: To determine whether 
SAFE Loop implementation 
improves nurses’ attitudes 
toward incident reporting, 
including perceptions of 
feedback and communication 
about error (H2.1)  
(H2.1) Hospital Survey on Safety 
(SOPS) Culture Survey 
performance on composite 
measure on “Feedback & 
Communication About Error” 
Nurses involved in SAFE Loop will 
confirm that an effective safety 
action loop has been established 
by scoring higher on measures of 
feedback/communication about 
error  
Aim 3: To determine whether 
SAFE Loop implementation 
reduces rates of high-priority 
medication events (H3.1)  
(H3.1) Rate per 1,000 patient-
days of Target Medication 
Events selected by nursing units 
in intervention groups 
Nursing units involved in the 
intervention arms will reduce the 
number of target medication 
events compared to control 
group units 
Secondary 
 
 
Aim 1: To determine whether 
SAFE Loop implementation 
increases the rate at which 
nurses report high-priority 
medication incidents (H1.1) 
and the number of 
contributing factors described 
per report 
(H1.1) The rate per 1000 patient-
days of incident reports 
addressing one of the Target 
Medication Events  
The number and type of patient 
factors (e.g. language barrier) 
associated with incident reports 
 
If nurses report more important 
events per patient-day, quality 
improvement personnel will have 
more opportunities to develop 
targeted improvement solutions 
Patient factors may lead to an 
increased risk of adverse drug 
events 
Aim 2: To determine whether 
SAFE Loop implementation 
improves nurses’ attitudes 
toward incident reporting, 
including perceptions of the 
frequency with which events 
are reported. 
(H2.2) SOPS Culture Survey 
performance on composite 
measure on “Frequency of 
Events Reported” 
Nurses will demonstrate greater 
engagement in SAFE Loop by 
scoring higher on measures of 
frequency of events reported  
Aim 3: To determine whether 
SAFE Loop implementation 
reduces rates of high-priority 
medication events involving 
harm 
 
 
Rate of events involving harm 
 
The most severe events are the 
highest priority for prevention.  

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
7 
OBJECTIVES 
ENDPOINTS 
JUSTIFICATION FOR ENDPOINTS 
Tertiary/Exploratory  
 
 
Qualitative analysis of 
implementation:  
Qualitative reports from nurses 
about the process of 
implementation 
 
If the SAFE Loop improves 
outcomes, understanding how it 
worked will enable the 
intervention to be adapted for 
use at other hospitals.  
 
4 
STUDY DESIGN  
4.1 
OVERALL DESIGN 
We plan to conduct a single-site cluster RCT of 20 acute care nursing units in the main Medical Center to 
compare use of the SAFE Loop (intervention nursing units) and an existing incident reporting system 
(control nursing units). Implementation of the intervention will be sequential in two phases (5 
intervention and 5 control units will be randomized at a time). Randomization will occur in blocks to 
assure the similarities of the nursing units, with 4 nursing units per block, matched on patient 
population, size of the nursing unit, and other factors.   
 
Corresponding to three study Aims above, outcome measures include:  
• 
(Aim 1) incident reporting practices (rates at which nurses report high-priority medication incidents 
and numbers of contributing factors described per report),  
• 
(Aim 2) nurses’ attitudes toward incident reporting (AHRQ Hospital Survey on Patient Safety 
Culture™, particularly perceptions of feedback and communication about error and of the frequency 
with which events are reported), and  
• 
(Aim 3) rates of high-priority medication events (determined via IHI Trigger Tool method of medical 
record review).  
Analyses will compare changes in outcomes between from before to after implementation in 
intervention and control arms. Qualitative interviews of nurses the SAFE Loop intervention arm will 
provide insight into implementation. 
 
To minimize bias while also facilitating study feasibility, randomization will be done at the nursing unit 
level, and research staff involved in data collection will not have access to information on which nurses 
and patients were assigned to which arm. Also to minimize bias, nurses participating in the survey in Aim 
2 will be informed that the results will be analyzed as part of a study on patient safety and incident 
reporting, without specifically linking the survey to the SAFE Loop study. 
 
4.2 
SCIENTIFIC RATIONALE FOR STUDY DESIGN 
 
A cluster RCT is an appropriate balance the rigors of a randomized control trial against the practical 
considerations of implementing a study across multiple nursing units of a large medical center. The 
design will enable the research team to adequately engage and train participants in nursing units while 
also using control nursing units to measure the effectiveness of the SAFE Loop intervention.  
 
4.3 
JUSTIFICATION FOR DOSE 
 
Not Applicable 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
8 
 
4.4 
END OF STUDY DEFINITION 
 
A nurse or nursing unit will be considered to have completed the study upon finishing all phases of the 
study including the last procedure shown in Section 1.3. The end of the study is defined as completion of 
last procedure shown in the trial globally. 
 
5 
STUDY POPULATION 
5.1 
INCLUSION CRITERIA 
Study Nursing Units:  
The study will include 34 acute care areas within Cedars-Sinai medical Center, which are grouped 
together into 20 nursing units by virtue of sharing the same nursing unit manager (Table). To facilitate 
blocked randomization, we grouped similar nursing units together into 5 blocks with four nursing units 
per block. The four nursing units within each block are as similar as possible (e.g., medical units are 
grouped together, ICUs are grouped together, etc.). 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Nurses:  
• Aims 1 & 3 (SAFE Loop Incident Reports and Medical Record Review): All 1,980 nurses on the nursing 
units will be eligible to participate in the SAFE Loop and therefore participate in Aims 1 and 3.  
Table: Study Nursing Units, Patient Populations, Type, Size, & Allocation to Blocks 
 
Unit Areas 
Typical Patient Population 
Type 
Nurses 
Pt-Days* 
Block 1: Larger medical floor units  
A 4SE, 4SW 
Adult oncology 
Floor 
118 
   22,322  
B 5NE, 5NW 
Cardiology 
Floor 
116 
21,534  
C 5SE, 5SW 
Adult medicine, gastroenterology 
Floor 
103 
22,025  
D 7SE, 7SW 
Adult medicine 
Floor 
92 
22,025  
Block 2: Smaller medical floor units and emergency department 
A 6SE, 6SW 
Congestive heart failure 
Floor 
104 
18,552  
B ED 
Emergency department (ED) 
ED 
167 
19,108  
C 3-SCCT 
Medicine, congestive heart failure 
Floor 
50 
10,188  
D 3NW 
Short stay medical 
Floor 
41 
2,437  
Block 3: Surgical floor units 
A 6NE, 6NW 
6ICU 
Kidney & liver transplant, cardiac surgery,  
Post cardiac-catheterization recovery 
Floor, 
ICU 
122 
21,493 
B 7NE, 7NW 
Med/surg orthopedics  
Floor 
106 
20,901 
C 8SE, 8SW 
Med/Surg bariatrics, GI surgery 
Floor 
100 
15,230  
D 8NE, 8NW 
Neurosurgery, Med/surg spine 
Floor 
78 
15,230  
Block 4: Adult intensive care units 
A 4N-SCCT, 
6-SCCT 
Cardiac ICU,  
Cardiothoracic surgery ICU  
ICU 
130 
10,711 
B 5-SCCT 
Surgical and trauma ICU 
ICU 
91 
6,381 
C 7-SCCT 
Respiratory/medical ICU 
ICU 
69 
    7,085  
D 8-SCCT, 
4NW 
Neurosurgical ICU,  
Adult stroke/medicine 
ICU, 
Floor 
124 
17,424 
Block 5.1: Obstetric and postpartum units 
A 3NE 
Maternal-fetal care (obstetrics) 
Floor 
123 
14,486  
B 3SE, 3SW 
Post-partum 
Floor 
115 
13,476  
Block 5.2: Pediatric units 
C 4S-SCCT, 
4NE 
Pediatric and congenital cardiac ICU,  
Pediatrics 
ICU, 
Floor 
44 
3,903  
D 4NICU 
Neonatal ICU 
ICU 
87 
9,959 
*Patient-days of care on nursing unit per year.                            Total 
1980 
294,470 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
9 
• Aim 2 (Survey): Nurses will be eligible for survey administration if they worked >50% time on one study 
nursing unit during a 6-month study period; this will ensure adequate exposure to the study conditions 
and the opportunity to complete baseline and follow-up surveys. About 1,980 nurses will be eligible. 
• Qualitative analysis of implementation: Nurse unit managers and nurses will be eligible for the 
qualitative interview if they worked at least 50% time on the study nursing units for the entire duration 
of a SAFE Loop implementation period. (We will enroll 32 participants out of a pool of approximately 
800 eligible nurses.) 
Patients:  
Cedars-Sinai has a total of 294,470 patient-days of care in eligible nursing units per year. Patients 
located in experimental and control group nursing units will be eligible for participation. If patients 
transfer between nursing units during the study, analyses will allocate each day of the hospitalization to 
the unit where the patient was that day. For the review of medical records under Aim 3, we will 
randomly select 1,520 patients hospitalized during the baseline and follow-on periods. 
 
5.2 
EXCLUSION CRITERIA 
We will exclude outpatient clinics, operating/ procedure rooms, post-anesthesia care, and diagnostic 
and therapeutic services. 
5.3 
LIFESTYLE CONSIDERATIONS 
 
Not applicable 
 
5.4 
SCREEN FAILURES 
 
Not applicable 
 
5.5 
STRATEGIES FOR RECRUITMENT AND RETENTION 
 
Recruitment is not applicable to participation in the SAFE Loop because the intervention is implemented 
at the level of the nursing unit. This is true for both nurses and patients on the study nursing units. 
Outcome Assessment 
Nurses 
• Aim 2 (AHRQ Hospital Survey on Patient Safety Culture): To identify nurses eligible for the survey, 
hospital administrators will access internal staffing databases listing the nurses who work on each of 
the nursing units, including names, email addresses, title/position, percent effort overall, dates of work 
on the study units (to identify nurses who joined or left during a study wave), and work schedule (to 
determine which nurses worked >50% on each unit). For the survey procedures, the response rate is 
relevant rather than the recruitment rate. During routine deployments at Cedars-Sinai of the AHRQ 
Hospital Survey on Patient Safety Culture, response rates on study nursing units have been 60% or 
higher on average. To optimize response rates, leaders on the study team will visit nursing units during 
the huddles and during other meetings on the nursing units shortly before and during each survey 
deployment. To reinforce completion of the survey and we will ask the nursing unit directors to send 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
10 
eligible nurses an email notice expressing support for the research project and survey, explaining that 
completing the survey is voluntary, and letting the nurses know to expect the survey. The survey itself 
will be delivered electronically, via REDCAP using a weblink embedded in an email. During each 
deployment, we will deliver the survey up to four times to optimize response rates. We will also place 
fliers approved by the IRB on tack boards in common areas like break rooms and conference rooms 
away from public areas. We will incentivize completion of the survey at both the individual level and 
the nursing unit level. Nurses will be asked to complete the survey during working hours. Individual 
nurses will be entered into a lottery to win an iPad. When visiting the nursing huddles/meetings to 
prompt completion of the survey, study team members will bring gift baskets of healthy beverages and 
snacks. Study subjects will include women and minority groups, and will be diverse in terms of race 
and ethnicity. Because study subjects are nurses and nurses are more likely to be female, a majority of 
subjects will be female. We will neither conduct outreach for nor exclude any individuals on the basis 
of sex/gender, race, or ethnicity.  
• Qualitative analysis of implementation: We will recruit 22 frontline nurses and 10 nurse managers to 
participate in a 30-minute qualitative interview post SAFE Loop implementation by visiting nursing 
huddles that are convened daily as part of routine organizational activities on the unit. We will also 
post fliers approved by the Marketing Dept in nursing break rooms and conference rooms away from 
public areas. Nurses will be entered into a lottery for an iPad for their participation. The recruitment 
rate is not applicable to participation in the SAFE Loop because we will continue to recruit nurses to 
participate in the qualitative interviews until we have completed 32. Study subjects will include 
women and minority groups, and will be diverse in terms of race and ethnicity. Because study subjects 
are nurses and nurses are more likely to be female, a majority of subjects will be female. We will 
neither conduct outreach for nor exclude any individuals on the basis of sex/gender, race, or ethnicity. 
Nurses will be asked to complete the interviews during working hours. 
Patients 
• Aim 1 (SAFE Loop Incident Reports): Not applicable; data are collected retrospectively. 
• Aim 3 (Medical record and other record review to identify medication errors): Not applicable; data are 
collected retrospectively.  
 
6 
STUDY INTERVENTION 
6.1 
STUDY INTERVENTION(S) ADMINISTRATION 
6.1.1 STUDY INTERVENTION DESCRIPTION 
The SAFE Loop has three phases: Engagement, Fact-finding, and Action & Feedback. Across the three 
phases, the SAFE Loop has five Key Attributes. During all phases, the SAFE Loop Team collaborates with 
frontline nurses and Unit Managers. During the Fact-finding Phase, the team obtains and integrates 
information from an AHRQ Patient Safety Organization.  
Although the SAFE Loop could apply to any problem, we will focus on medication errors because they 
are common,34-37 occur on diverse nursing units, and can be evaluated using well-established methods. 
Also, nurses file most incident reports and are uniquely able to detect problems not only during drug 
administration but also during the ordering, dispensing, and monitoring stages in drug therapy.38  
Five Key Attributes Designed to Maximize the Effectiveness of Reporting: 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
11 
(1) Input from Nurses 
(Engagement Phase): Nurses are 
more likely to report events and 
adopt innovations addressing 
safety problems that they find to 
be particularly common, severe, 
or preventable.39-41 The SAFE 
Loop Team, therefore, solicits 
input from nurses and Unit 
Managers on which problems to 
select and solve. 
(2) Focus on Selected Target 
Event (Engagement Phase): To 
make the best use of 
investigative resources and 
obtain enough reports to understand each safety problem, the SAFE Loop focuses on Target Medication 
Events (errors and adverse events related to a selected, defined problem). These events will involve a 
particular clinical situation, drug class (e.g., opioids), population (e.g., kidney disease), type of error (e.g., 
duplicate therapy), and/or system factor (e.g., different protocols in ICU and floor units). Examples 
include: (1) nurses having challenges reconciling medications after patients go to the operating room, (2) 
pharmacists dispensing mislabeled medication, and (3) physicians ordering multiple sedating 
medications concurrently.  
(3) Stimulated Time-limited Reporting (Fact-finding Phase): Lack of guidance about what and how to 
report deters reporting and limits reports’ usefulness. The SAFE Loop Team prompts nurses to report 
many Target Medication Events during a certain period and teaches nurses to write more informative 
reports.  
(4) Integration of Information (Fact-finding Phase): Because most safety problems are not unique to a 
particular nursing unit or hospital, the SAFE Loop Team integrates information from incident reports, 
follow-up investigations, and internal and external sources, including an AHRQ PSO.   
(5) Feedback to Nurses (Mitigation Phase): To demonstrate that reports are used and can drive system 
changes, the SAFE Loop Team provides feedback about medication safety problems and mitigation 
plans. 
Implementation of Intervention on an Individual Nursing Unit: 
Engagement Phase (4 weeks):  
Obtain Input: Nursing units convene daily huddles with their staff to discuss various issues. For three 
weeks, the SAFE Loop Team will attend one huddle per week to provide information on local safety 
events and obtain input on problems that frontline nurses believe to be common, severe, and 
preventable.  
To include nurses who work at night and on weekends, the intervention will be disseminated through 
nursing huddles, which occur at shift change. We will also place fliers approved by the Marketing Dept 
on tack boards in common areas like break rooms and conference rooms away from public areas.  
Select Target Medication Events: Each study nursing unit will select its own unique event. Through 
discussions with nurses and the SAFE Loop Team, the Unit Manager will choose a medication safety 
problem with a narrow enough scope to facilitate mitigation through targeted actions (e.g., problems 
1. Input  
3. Stimulated Time-limited  
Reporting  
Action & 
Feedback Phase 
Fact-finding  
Phase 
5. Feedback  
to Nurses 
SAFE Loop Team 
Nurses on a specific 
nursing unit 
Engagement 
Phase 
Unit Manager 
4. Integration  
of Information 
AHRQ Patient Safety  
Organization 
2. Focus on  
Target Events 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
12 
with opioids). The SAFE Loop Team will define inclusion/exclusion criteria for the unit’s Target 
Medication Event.  
Fact-Finding Phase (12 weeks):  
Train Nurses: The SAFE Loop Team will train nurses to: (1) submit reports on Target Medication Events 
(in addition to usual reports); (2) focus on unsafe conditions and near misses, and (3) write detailed 
narrative descriptions of circumstances they observed leading up to each event. The SAFE Loop Team 
will train Nursing Unit Managers in a just-culture approach (i.e., addressing system issues rather than 
blaming individuals). 
Stimulate Reporting: The SAFE Loop Team will attend huddles weekly to ask nurses about Target 
Medication Events, remind them to submit reports, and reinforce how to write high-quality reports. A 
data analyst will provide quantitative data reports for review at the weekly huddles. Based on the pilot 
study, 12 weeks is long enough to collect enough reports but short enough to retain nurses’ attention 
and facilitate rapid action.  
To train nurses in how to write high-quality incident reports, the SAFE Loop Team will work with a 
nursing educator at Cedars-Sinai and develop content for a HealthStream module approved by the Chief 
Patient Safety Officer. This module will include a short set of slides with voiceover narration explaining 
the importance of focusing on a Target Event and the value of providing a detailed narrative description 
of events including contributing factors when filing reports. Completing the module will take no more 
than 10 minutes. All nurses on the study units will be delivered the module in their HealthStream 
accounts and asked to complete it; they will receive this request via usual channels of communication 
(e.g., email reminders). The information in the HealthStream module will be reinforced via visits by SAFE 
Loop staff to daily nursing huddles, reminders from nursing unit managers, and via a small laminated 
reminder card that can be affixed to the nurses ID badge. 
Analyze and Investigate: This iterative process will involve the SAFE Loop Team and Nursing Unit 
Manager. First, the Incident Report Clinical Reviewer will examine incident reports to identify Target 
Medication Events. The SAFE Loop Team and Nursing Unit Manager will provide input on key terms used 
to search for Target Medication Events, confirm which events meet the definitions, and discuss insights 
from the reports. Next, the SAFE Loop Team and Nursing Unit Manager will work together on follow-up 
investigations, including “mini-root-cause-analyses” whereby the Safety Officer and Nursing Unit 
Manager discuss events and contributing factors with frontline nurses for 10 to 15 minutes during 
morning huddles. Depending on the event, additional steps may include reviewing medical records, 
staffing records, pharmacy reporting systems, and patient complaints, and working with representatives 
of diverse physician specialties and hospital departments.  
Integrate Information: Concurrently, the Safety Champion will seek external information on the Target 
Medication Event, with assistance from the Institute for Safe Medication Practices. The Safety Champion 
will access resources from ISMP, the National Alert Network by the National Coordinating Council for 
Medication Error Reporting and Prevention (NCC MERP), other AHRQ PSOs relevant to medication safety 
(Alliance for Patient Medication Safety, ECRI, CHPSO), the American Society of Health-System Pharmacy, 
the Institute for Healthcare Improvement, the Collaborative Alliance for Nursing Outcomes, AHRQ’s 
PSNET, the Joint Commission, and the FDA. Next, the Safety Champion will integrate information from 
incident reports, internal investigations, and external resources to draft a synopsis of the Target 
Medication Events, contributing factors, and any mitigation strategies used elsewhere. The SAFE Loop 
Team and Nursing Unit Manager will edit the synopsis for relevance and clarity. The Nursing Unit 
Managers will share it with frontline nurses through email and the huddles. 
Action and Feedback Phase (16 Weeks):  

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
13 
Develop Plan: The SAFE Loop Team and Nursing Unit Manager will employ the information summarized 
in the synopsis to develop a mitigation plan that involves system changes such as protocols, practices, 
equipment, staffing, and oversight (not solely educating providers). Ideally, stronger actions, such as 
process changes and hard stops, will be implemented.  Where possible, they will draw from prior 
interventions on other nursing units or in other hospitals. To assure that the mitigation plan is feasible 
and aligned with clinical practice, the Safety Champion will elicit ideas from nurses on the unit. The SAFE 
Loop Team will discuss advantages and disadvantages of promising strategies, then the Nursing Unit 
Manager and Safety Officer will establish the final plan. Where necessary, the Safety Officer will 
coordinate with other departments and senior hospital leaders.  
Distribute Feedback: The Safety Champion will draft a summary of the mitigation plan and obtain 
feedback. The Nursing Unit Managers will share the mitigation plan with frontline nurses through email 
and the huddles. 
Implement and Sustain Plan: The Nursing Unit Manager will take local actions to improve systems of 
care, assure that the planned changes are implemented, and make any subsequent refinements through 
ongoing Plan-Do-Study-Act cycles. The Safety Officer will help to overcome any barriers, assure 
alignment with other departments, and involve senior hospital leaders when necessary. The SAFE Loop 
Team will visit the nursing unit weekly during a 6-month follow-on period to reinforce and facilitate 
refinements to the plan. 
Table: Roles in the Existing Reporting System and the SAFE Loop Intervention 
 
Position 
Roles in Existing Reporting System 
ADDITIONAL Roles in SAFE Loop 
Intervention 
 
Nurses on 
Study 
Nursing Units 
Spontaneously submit incident reports. 
Use the SBAR (situation, background, 
assessment, and recommendation) 
structure when writing reports.9 
Provide input on the Target Events, 
undergo training in reporting, submit 
reports, receive feedback, and implement 
mitigation strategies. 
 
Nursing Unit 
Managers 
Administratively responsible for overseeing 
efforts to improve the safety of care on 
each nursing unit. Participate in analyzing 
reports, conduct any investigations, design 
and implement any solutions. 
Select the Target Medication Events and 
work with the SAFE Loop Team throughout 
implementation. (Nursing units will be the 
unit of randomization and are defined by 
having a unique Manager).  
SAFE Loop Team 
Incident Report 
Clinical 
Reviewer 
Classifies events, assigns harm severity, 
identifies source nursing unit, and triages 
high-risk events to relevant departmental 
and unit leaders. 
Identifies and extracts information from 
reports on Target Medication Events. 
(Different reviewers will assist intervention 
and control nursing units).   
Safety Officer 
Assists nursing units with investigations, 
obtains resources, engages leaders of 
relevant departments, and overcomes 
barriers. Oversees root-cause analyses of 
harmful events. 
Directs the SAFE Loop Team. Facilitates 
investigation of near misses. (Different 
Safety Officers will assist intervention and 
control nursing units).  
Safety 
Champion 
Not applicable. 
Obtains input from and trains nurses, 
assists with investigations, integrates 
information from reports and other sources, 
drafts feedback to nurses. 
 
AHRQ Patient 
Safety 
Organization 
Not applicable. 
Institute for Safe Medication Practices 
(ISMP) serves as a resource to the SAFE 
Loop Team, providing information on 
events at other hospitals. 
6.1.2 DOSING AND ADMINISTRATION 
 
Not applicable 
 
 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
14 
 
6.2 
PREPARATION/HANDLING/STORAGE/ACCOUNTABILITY 
 
Not applicable 
 
6.3 
MEASURES TO MINIMIZE BIAS: RANDOMIZATION AND BLINDING 
 
Random Sequence Generation, Allocation Concealment, and Blinding: A researcher (not on the SAFE 
Loop or Data Collection teams) will use computer-generated random numbers to randomly assign the 4 
nursing units within each of the 5 randomization blocks to one of 4 study groups, reflecting 2 arms 
(intervention and control) and 2 periods of time (sequential implementation; see Table below). Next, the 
researchers will place lists of the nursing units assigned to each group in opaque, sealed envelopes 
labeled with group numbers. At the start of each implementation period, the SAFE Loop Team will open 
the corresponding envelopes to determine the 5 nursing units that are assigned to each group. Research 
staff involved in data collection will be blinded to which nurses and patients are assigned to which arm. 
 
 
 
 
6.4 
INTERVENTION COMPLIANCE 
 
Implementation Processes: Importantly, this is a quality improvement intervention that will be 
implemented flexibly by hospital staff and not a rigid protocol to which they must comply. The Chief 
Patient Safety Officer (Dr. Seferian), will oversee implementation of the SAFE Loop, given his extensive 
prior experience implementing other performance improvement interventions. Planning: During year 1, 
the research team will refine the implementation plan, which considers stakeholders' needs, tailors the 
SAFE Loop for subgroups (e.g., nurses vs. unit managers), employs appropriate communication channels 
(e.g., daily nursing huddles), and tracks progress toward milestones. Engaging: Engagement is 
fundamental to the SAFE Loop, particularly nurses’ input on medication safety problems, participation in 
“mini-root-cause analyses,” and receipt of feedback. Executing: SAFE Loop Team members will reinforce 
the SAFE Loop via stimulated reporting and feedback. Nursing Unit Managers will serve as local 
implementation leaders, and each nursing unit has one or more additional nurses designated as Safety 
Representatives who will assist with this process. Reflecting and Evaluating: We will perform a detailed 
qualitative analysis of implementation. 
 
The SAFE Loop intervention is designed to shape but not constrain or determine or control systems of 
medication management on each nursing unit. These systems of medication management, in turn, 
influence but do not constrain or determine how individual clinicians provide care to individual patients. 
The care provided to individual patients can, unfortunately, involve medication errors. Medication 
Table: Study Design, Including Study Arms and Measurement Periods 
Group 
½ Year 1 
Year 2 
Year 3 
1 
Baseline 
b SAFE Loop 
f i Follow-on 
 
 
2 
Baseline 
b Control 
f  Follow-on 
 
 
3 
 
 
Baseline 
b SAFE Loop 
f i Follow-on 
4 
 
 
Baseline 
b Control 
f  Follow-on 
Light green = baseline measurement periods for Aims 1 & 3 
Orange = baseline (b) and follow-up (f) survey deployments for Aim 2 
Dark blue = SAFE Loop arm 
Medium blue = control arm 
Yellow = qualitative interviews (i) on implementation and follow-on period 
Dark green = follow-on period for Aim 3 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
15 
errors are problems with the ordering, dispensing, administration, or monitoring phases of medication 
therapy. Patient outcomes resulting from medication errors are influenced by the errors themselves as 
well as patient characteristics including the reason for hospitalization, the set of medications, age, 
gender, comorbidities, and diverse other factors.  
 
Under the SAFE Loop, the goal is to have nurses use incident reports to identify problems with systems 
of care on the nursing units and then develop and implement changes to those systems of care that 
reduce the likelihood of medication errors. As indicated by the diagram below, the SAFE Loop 
intervention has the potential to influence systems of medication therapy, care for individual patients, 
and patient outcomes, but these are not under direct investigator control. 
 
 
 
 
Relationship between Intervention, Systems of Care, Care for Individual Patients, and Outcomes  
 
SAFE Loop Intervention 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
16 
 
6.5 
CONCOMITANT THERAPY 
 
Not Applicable 
 
6.5.1 RESCUE MEDICINE 
 
Not Applicable 
 
7 
STUDY INTERVENTION DISCONTINUATION AND PARTICIPANT 
DISCONTINUATION/WITHDRAWAL 
7.1 
DISCONTINUATION OF STUDY INTERVENTION 
 
The DSM will be able to interrupt the trial if there are serious concerns about data safeguarding or 
safety based on potential adverse events or other information, and if there have been any adverse 
events (particularly serious events). The study team will work with the DSM to develop rules for 
stopping the trial, based on the occurrence of severe adverse events. 
 
Discontinuation of the SAFE Loop intervention does not necessarily mean discontinuation of the study, 
and remaining study procedures should be completed as indicated by the study protocol.  If a clinically 
significant finding is identified in a participant or one arm (including, but not limited to changes from 
baseline) after enrollment, the investigator or qualified designee will determine if any change in 
participant management is needed. 
 
7.2 
PARTICIPANT DISCONTINUATION/WITHDRAWAL FROM THE STUDY 
 
The SAFE Loop intervention is implemented at the level of the nursing unit. Additionally, the nursing 
units choose how to modify systems of medication therapy to improve safety, and those local changes 
can be considered hospital quality improvement interventions. As such, individual nurses and patients 
do not have the ability to withdraw from the study. However, nurses can choose not to engage in SAFE 
Loop activities such as filing incident reports on Target Medication Events. Nurses can also decline to 
respond to the survey under Aim 2. No data are collected directly from patients. 
 
7.3 
LOST TO FOLLOW-UP 
 
Not applicable. We will have complete data on all patients who receive care on the study nursing units. 
If nurse choose not to complete the survey, this will not constitute “loss to follow-up.” 
 
8 
STUDY ASSESSMENTS AND PROCEDURES 
8.1 
EFFICACY ASSESSMENTS  
Methods for Aim 1: Incident Reporting Practices 
A. Measures: There are two key outcome measures and several descriptive measures. One key outcome 
measure is the rate per 1000 patient-days of incident reports addressing one of the Target Medication 
Events. Ten nursing units will implement the SAFE Loop, leading to 10 Target Medication Events total.  

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
17 
The primary outcome measure will be the number of system plus human factors per incident report. In 
industries where incident reporting has been effective, obtaining insights into contributing system and 
human factors has been the purpose of reporting. To characterize system and human factors, we will 
use the Human Factors Analysis and Classification System for Healthcare (HFACS-Healthcare).44 
Introduced in 2001, the HFACS has widely been widely used to analyze incidents in diverse industries 
internationally.44-47 More recently, HFACS developers 
adapted it for use in healthcare settings.48 HFACS-
Healthcare lists 21 categories of contributing factors 
within 4 tiers (Table 6).9 We will also examine the number 
and diversity of factors reported.  
Descriptive Measures: Patient factors also represent 
opportunities to improve care. To classify patient factors, 
we will use categories from prior work.4 Experts consider 
severity and preventability when prioritizing incidents for 
follow-up and action.49 To classify severity, we will use the 
NCC MERP Medication Error Index.50-52 To classify 
preventability, we will adapt an existing scale.53 
Data Collection: Analysts with clinical experience (e.g., 
nursing, pharmacy) and training in HFACS-Healthcare will 
manually review incident reports related to medication 
issues. Pairs of analysts will independently judge whether 
each incident matches one of the definitions of Target 
Medication Events, extract information related to the measures above, and then meet to discuss and 
reach consensus. A human factors expert (Dr. Cohen) will adjudicate any ties. Analysts will be blinded to 
nursing unit, time period, and study arm. Data collection instruments will be adapted from our prior 
work.4,9,54 As part of training, analysts will practice on at least 30 sample reports, consulting with each 
other and the human factors expert until scores are consistent. To evaluate reliability, we will calculate 
Cohen’s Kappa for 10% of records. 
Methods for Aim 2: Nurses’ Attitudes toward Incident Reporting 
Measures: The planned outcome measures include 
two composite measures from the AHRQ Hospital 
Survey on Patient Safety (SOPS) Culture™, Version 
1.0 (English). AHRQ developed the survey in 2004 to 
assess the perspectives of frontline providers and 
staff on patient safety, error, and event reporting. 
Since then, hundreds of hospitals across the U.S. and 
internationally have used it. The survey has 
moderate-to-strong validity and reliability,55 and the 
items and dimensions are psychometrically sound at 
the individual, unit, and hospital levels.56 A 2015 
systematic review found associations between 
Hospital SOPS scores and patient outcomes at 
hospital and nursing-unit levels of analysis.57 Scores 
represent percentages (range 0-100%, higher is better). Changes in overall Hospital SOPS scores of 8.6 
percentage points (range, 6.5-10.6) have been associated with changes in practices by hospital leaders.58  
Table 6: Incident Report Characteristics 
Contributing Factors (HFACS, 4 tiers) 
Organizational influences (system factors) 
 
Unsafe supervision (system factors) 
 
 
Preconditions for unsafe acts (system) 
 
 
 
Unsafe acts (human factors) 
Severity (NCC MERP, 9 Categories) 
A: Safety hazard (did not involve patient) 
B: Event did not reach patient 
C: Event reached patient, no harm 
D: Monitoring/intervention to prevent harm 
E: Temporary harm requiring intervention 
F: Temporary harm, prolonged hospitalization 
G: Permanent harm 
H: Intervention to sustain life 
I: Death 
Preventability (Brook et al.) 
Definitely or probably preventable 
Definitely or probably not preventable 
Uncertain/unable to determine (added) 
Table 8:  Hospital Survey on Patient Safety (SOPS) Culture, Including 
Data from Study Site (CSMC) and Nationally  
12 Composite Measures  
CSMC 
National 
Feedback & Communication About Error 
75% 
68% 
Frequency of Events Reported 
72% 
67% 
Communication Openness 
63% 
64% 
Handoffs & Transitions 
52% 
46% 
Management Support for Patient Safety 
72% 
73% 
Nonpunitive Response to Error 
35% 
44% 
Organizational Learning 
83% 
73% 
Overall Perceptions of Patient Safety 
70% 
66% 
Staffing 
49% 
53% 
Supervisor Expectations & Actions  
74% 
79% 
Teamwork Across Units 
66% 
61% 
Teamwork Within Units 
84% 
82% 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
18 
The survey takes 10-15 minutes to complete and includes 42 items used to derive 12 composite 
measures (Table 8), 2 single-item measures, and respondents’ backgrounds (unit, staff position, 
interaction with patients, years of experience in current profession, hospital, and current nursing unit). 
Most items are scored on a scale from 1 (strongly agree) to 5 (strongly disagree). Creating composite 
scores involves: (1) inverting the scales of any negatively worded items, (2) identifying positive 
responses (1 or 2); (3) calculating the percent of responses that were positive for each item (excluding 
missing values), and (4) averaging the percent positive across all items within a composite measure.  
 The two composite measures we plan to use are directly relevant to incident reporting and likely to be 
responsive to system-level changes. Feedback and Communication About Error includes three items: 
staff are informed about errors, discuss ways to prevent errors, and are given feedback about changes 
implemented. The Frequency of Events Reported also includes three items: staff report mistakes that are 
caught before affecting the patient, with no potential to harm, and that could harm but did not. In a 
prior analysis from three nursing units at three different hospitals, scores on these two composite 
measures were correlated (ρ=0.56) but factor analyses indicated that they captured distinct concepts 
(loaded separately). Respectively, Cronbach α values for these measures were 0.77 and 0.78 among 
nurses, demonstrating good internal consistency.55  
B. Data Collection: Cedars-Sinai routinely surveys staff every two years using the Hospital SOPS. For the 
study, we will survey ~1980 eligible nurses on study nursing units twice via 4 total deployments of the 
full Hospital SOPS during two-week intermissions between study periods (Section 3A). To identify 
eligible nurses, we will obtain staffing databases including names, email addresses, title/position, 
percent effort, dates of work on the unit, and work schedule. At least 85% of nurses on study units will 
be eligible. Two weeks before deployment, we will send email notices to expect the survey and a 
researcher will visit nursing units to reinforce survey completion. The survey will be delivered via 
REDCap using a weblink embedded in an email. During each deployment, we will deliver the survey 
three times. We will incentivize completion at the individual (lotteries for iPads) and unit levels (gift 
baskets of snacks for response rates >65%). To minimize detection bias, the survey will not refer to the 
SAFE Loop intervention. To avoid having incomplete outcome data, we will monitor survey response 
rates and make adjustments to procedures as needed. 
Methods for Aim 3: Rates of Target Medication Events 
A. Measures: The primary outcome will be the rate per 1,000 patient-days of Target Medication Events 
(preventable plus potential adverse drug events) selected by the 10 nursing units in Groups 1 and 3. We 
will also examine the rate of events involving harm. We will define events as in prior research (Table 
10).38,61,62  
Table 10: Medication Event Definitions 
Examples 
ADE (NCC MERP 
severity E-I) 
A patient injury* resulting from a medical 
intervention related to drug(s). 
A rash due to an antibiotic. 
 
Preventable 
An adverse drug event due to a medication error.† 
A rash due to an antibiotic given despite 
known allergy. 
 
Non-preventable 
An adverse drug event not due to an error. 
A rash occurring after first exposure to an 
antibiotic. 
Potential ADE 
A medication error that had the potential to cause 
injury but did not. 
 
 
Intercepted (NCC 
MERP B) 
A medication error that was detected and stopped 
before the patient received the medication. 
A pharmacist filling orders discovers a 
known allergy and prompts the prescriber 
to switch antibiotics.  
 
Non-intercepted 
(NCC MERP C-D) 
A medication error in which the patient received 
the medication but did not to experience harm. 
A patient with a known allergy to an 
antibiotic receives the drug anyway but 
does not develop a rash. 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
19 
*An injury is an unintended consequence of care that negatively affects a patient’s health and/or quality of life. † A 
medication error is an error anywhere in the process of drug ordering, dispensing, administration, and monitoring. 
Descriptive Measures: We will characterize event severity/harm (NCC MERP categories), medication 
class (e.g., cardiovascular, diabetes, etc.), and route of administration (intravenous, oral, etc.). We will 
classify any errors by stage in drug therapy (ordering, dispensing, administering, monitoring), 
profession(s) involved (physician, pharmacist, nurse, other), and error types (allergy, drug-disease 
interaction, drug-drug interaction, drug-lab interaction, duplicate therapy, therapeutic omission, 
concentration, duration, route/dosage form, dose/frequency/rate, wrong medication, wrong patient, 
wrong timing, incomplete order).5,6,38,61 We will asses which events are Target Medication Events used 
by Group 1 and 3 nursing units (yes, no; and which unit).   
B. Data Collection: The nursing units will implement mitigation plans designed to prevent Target 
Medication Events toward the end of the SAFE Loop implementation (Section 2B), which means that 
medication events will not start to fall until after that. Accordingly, we will monitor medication events 
during a baseline period, the implementation period, and 6-month follow-on period after 
implementation (Section 3A). To identify medication events, we will randomly sample 1,520 
hospitalizations divided equally between the four study groups and the baseline and follow-on periods. 
(We will exclude the emergency department from Aim 3).  
To detect medication events, we will employ the widely accepted Trigger Tool Method by Institute for 
Healthcare Improvement. It involves two stage: (1) nurses systematically screen medical records for 
“triggers” (clues that an event occurred) and write synopses of possible events, and (2) physicians 
review synopses to confirm and classify them. Many studies have used these methods, including studies 
by Dr. Nuckols.5,6,38,61-64  
Screening: Research nurses will search for several types of triggers including the use of medications that 
can counteract other medications (e.g., naloxone), abnormal lab results (e.g., serum glucose <50 mg/dL), 
clinical events (e.g., rash), abrupt cessations of medication, and transfers to a higher level of care.52 To 
increase our ability to detect potential ADEs, we will add an additional “trigger”: events in the Cedars-
Sinai “Ivents” database, where pharmacists routinely record changes to medication therapy made to 
reduce the risk of harm (overseen by Dr. Leang). Ivents reports are distinct from voluntary incident 
reports (we will not include regular incident reports). Two research nurses will perform these 
screenings, blinded to study arm. To train the nurses, we will adapt training manuals and procedures 
from the Institute for Healthcare Improvement and our prior research.5,6,52 Manuals will include 
descriptions of the Trigger Tool method, Ivents reports, and variables employed in classifying the 
medication events. Through a 3-day in-person training session, nurses will become familiar with the 
manuals, ask questions, review 3 sample charts with supervision (by Dr. Nuckols), review an additional 5 
to 7 sample charts independently, and then meet with the trainer assess proficiency. During data 
collection, quality monitoring and training reinforcement will occur in two ways (1) the research nurses 
will submit questions and maintain a log of trainers’ responses, and (2) physician reviewers will ask for 
clarifications when the event synopses lack essential information. To obtain potential covariates for use 
in regression models, nurses will also extract patient age, gender, race, ethnicity, discharge diagnosis 
codes needed to score the Elixhauser comorbidity index, and primary and secondary payer. To ascertain 
reliability (Cohen’s kappa statistics), a randomly chosen 10% of medical records will undergo duplicate 
review.  
Classification: Two physicians will independently review each synopsis, employing a standardized rating 
form that includes the variables above. Next, they will meet to discuss responses. When there are 
discrepancies, physicians may ask the research nurses for additional information or involve a third 
physician to break ties. To ensure blinding, synopses will not include identities of patients, clinicians, or 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
20 
nursing units. For training, physicians will receive similar manuals as the nurses and score 20 sample 
events with an experienced reviewer (Dr. Nuckols). We will select 10% of synopses to calculate 
interrater reliability (Cohen’s kappa). 
Qualitative Analysis of Implementation: If the SAFE Loop improves outcomes, understanding how and 
why it worked will enable the intervention to be improved and adapted for use at other hospitals as well 
as at Cedars-Sinai. Similarly, if the SAFE Loop is ineffective, insights into its limitations will reveal how 
incident reporting systems could be improved through other types of changes.  
A. Data Collection: After SAFE Loop implementation, we will conduct one-on-one in-person interviews 
with 10 Nursing Unit Managers and 22 frontline nurses. This sample size will enable us to perform 
purposive sampling to acquire broad representation by randomization block, study group, and nursing 
unit. Interviews will occur in locations convenient to interviewees, last approximately 30 minutes, and 
be audio recorded (with permission). We will recruit interviewees by visiting nursing huddles. 
Participants will be entered into a lottery for an iPad. We will use a semi-structured interview guide with 
open-ended questions and follow-up probes to examine several topics, including: fidelity to SAFE Loop 
as planned, adaptations to SAFE Loop, SAFE Loop “dose” (e.g., did nurses learn about it once or several 
times?) and “reach” (e.g., were all nurses on the study units aware?), facilitators/barriers to 
implementation, mechanisms of action (e.g., did nurses respond to providing input, receiving guidance 
on how to report, or emphasizing near misses?), exposure to the SAFE Loop in the control arm, and 
contextual factors that may have moderated its effectiveness. Finally, we will discuss how to adapt the 
SAFE Loop for ongoing use over the long term and how to enhance its effectiveness.  
B. Analysis: Interviews will be transcribed from audio recordings, with personal identifiers removed. 
Pairs of researchers trained in qualitative analysis will analyze transcripts in Dedoose using a 
combination of content-analysis and qualitative inquiry, allowing them to discover and quantify the 
nurses’ experiences and perceptions. We will use an iterative process to identify a priori themes based 
on the domains above, and to create in vivo themes as they emerge during coding (e.g., specific barriers 
to implementation).65 The two coders will code each interview independently and then discuss 
variations until consensus is reached. After coding all interviews, we will use the constant comparative 
method to combine similar themes with limited data under more general themes.66 In the final step of 
analysis, we will review Dedoose code reports (participant N and density) of salient themes and coding 
matrices related to significant interview findings.  
 
8.1.1 UNANTICIPATED PROBLEM REPORTING  
 
The principal investigator (PI) will report unanticipated problems (UPs) to the reviewing Institutional 
Review Board (IRB). The UP report will include the following information: 
 
• 
Protocol identifying information: protocol title and number, PI’s name, and the IRB project number; 
• 
A detailed description of the event, incident, experience, or outcome;  
• 
An explanation of the basis for determining that the event, incident, experience, or outcome 
represents an UP;  
• 
A description of any changes to the protocol or other corrective actions that have been taken or are 
proposed in response to the UP. 
 
To satisfy the requirement for prompt reporting, UPs will be reported using the following timeline:   
 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
21 
• 
UPs that are serious adverse events (SAEs) will be reported to the IRB within 10 days of the 
investigator becoming aware of the event, to allow the investigator to conduct an investigation.  
• 
Any other UP will be reported to the IRB and to the DCC/study sponsor within 10 days of the 
investigator becoming aware of the problem.  
• 
All UPs should be reported to appropriate institutional officials (as required by an institution’s 
written reporting procedures), the supporting agency head (or designee), and the Office for Human 
Research Protections (OHRP) within 10 days of the IRB’s receipt of the report of the problem from 
the investigator. 
 
8.2 
SAFETY AND OTHER ASSESSMENTS 
The level and frequency of monitoring are designed to be commensurate with the risks, nature, and 
complexity of this clinical trial. Additionally, the data safeguarding procedures will adhere to all current 
standards established under the Common Rule, the Health Insurance Portability and Accountability Act 
(HIPAA), policies by AHRQ and NIH, and policies established by the Cedars-Sinai Institutional Review 
Board. All investigators have undergone required trainings in human subjects’ protections and have 
extensive experience in handling sensitivity information. As PI, Dr. Nuckols has ultimate responsibility for 
monitoring data and safety. 
Monitoring procedures will serve the following functions: (1) reviewing the research protocol and plans 
for data and safety monitoring issues; (2) monitoring data quality, timeliness, survey response rates, and 
factors that may affect the risks and benefits of the study such as emerging literature; (3) monitoring for 
potential and actual adverse events, and (4) making recommendations regarding continuation of the 
trial.  
Internal Monitoring Procedures: The study team will learn of potential issues with data and safety for 
nurses and patients through the following procedures.  
Nurses: Dr. Coleman will meet periodically with nursing unit managers to qualitatively monitor nurses’ 
perceptions of the study, including whether they have any concerns or any potential or actual adverse 
events have occurred. As Chief Patient Safety Officer, Dr. Seferian will monitor for increases in the 
number of referrals of nurses to the Human Resources Department on the intervention nursing units 
during the study period. Additionally, the Analytical Team will monitor responses to a relevant domain 
on the AHRQ Hospital Survey on Patient Safety Culture, Nonpunitive Response to Error, which reflects 
whether nurses feel that they have been blamed for reported incidents. This survey will be deployed 
before and after the implementation of the SAFE Loop, enabling the research team to detect problems 
with the first implementation phase before the subsequent phase begins. 
Patients: Dr. Coleman will work with the Cedars-Sinai Patient Safety Department, which handles incident 
reports, to qualitatively monitor whether there may be any reports addressing risks to patients 
associated with changes in practice on study nursing units that were implemented in response to the 
intervention.  
Inadvertent Disclosures of Identifiable Information: The team will self-monitor whether data are 
collected, stored, transmitted, and analyzed in accordance with the Data and Safety Monitoring Plan and 
for any inadvertent disclosures of sensitive information. 
The research team will include data and safety monitoring on the agenda of each routine team meeting, 
enabling any concerns to be identified and addressed quickly. The study will submit annual progress 
reports to the Cedars-Sinai Institutional Review Board and meet annually with the external DSM during 
Years 1 to 4. 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
22 
External Monitoring Procedures: The DSM will monitor throughout the portions of the study that involve 
human subjects, including implementation of the SAFE Loop, surveys of nurses, and extraction of data 
from incident reports and medical records. The DSM will meet with the Executive Oversight Team four 
times via teleconference to review the Data and Safety Monitoring Plan, proposed study procedures, 
data safeguarding practices, and data on potential and actual adverse events.  
• 
Year 1: The DSM will meet to review study protocols and the Data and Safety Monitoring Plan prior 
to the start of data collection. 
• 
Year 2: The DSM will meet to monitor progress in the study and consider any new information 
related to risks to nurse and patients. This meeting will represent a project milestone (see timeline) 
because it will provide a more complete picture of the risks associated with the study. 
• 
Year 3: The DSM will meet to monitor progress and any potential or actual adverse events. 
Additionally, the study Analytical Team will send the external monitoring body the survey responses 
on Nonpunitive Responses to Error after each deployment of the survey. 
• 
Year 4: The DSM will meet after the intervention and data collection are complete to review 
information on potential or actual adverse events. This meeting will represent a project milestone 
(see timeline) because it will be the concluding meeting of the DSM.   
• 
Additional Reviews: The DSM will reserve the right to request additional interim reviews, in the 
unlikely event that concerns are raised about data safeguarding or safety.  
Investigation of Suspected Adverse Events: Any potential or actual adverse events will be immediately 
reported to Dr. Nuckols (PI) and Dr. Coleman (internal DSM). Dr. Coleman will contact the parties 
involved to investigate: (1) nature of the event, (2) the type and severity of harm, (3) potential 
attribution to the research study, (4) circumstances that led up to the event including whether any study 
procedures deviated from the study protocol, and (5) potential for recurrence. Dr. Coleman will report 
this information to the research team, which will discuss how to address the event and to reduce the 
risk of recurrence.    
8.3 
ADVERSE EVENTS AND SERIOUS ADVERSE EVENTS 
8.3.1 DEFINITION OF SERIOUS ADVERSE EVENTS (SAE)  
 
Nurses:  
• 
Loss of professional reputation: Adverse job consequences for a nurse due to study participation, 
such as having documentation of providing suboptimal care placed in employment/human 
resources files, will be considered a serious adverse event. 
• 
Economic harm: Adverse job consequences for a nurse due to study participation, such as 1) having 
work hours reduced, 2) being passed up for a professional opportunity, or 3) being dismissed from 
Cedars-Sinai, will be considered a serious adverse event. 
• 
Disclosure of individually identifiable information: Evidence of significant psychological or other 
harm to nurses due to disclosure of individually identifiable information outside of the study team 
will be considered a serious adverse event. 
• 
Psychological distress: If a nurse reports psychological distress due to study participation (e.g., via 
loss of professional reputation or economic harm) that interferes with functioning on a prolonged 
basis or warrants referral to a mental health professional, this will be an serious adverse event. 
Patients: 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
23 
• 
Medication error: A medication error resulting from the SAFE Loop that required caused a patient 
temporary or permanent physical harm (a preventable adverse drug event) will be considered a 
serious adverse event. 
• 
Disclosure of individually identifiable protected health information: Evidence of significant 
psychological or other harm to patients due to disclosure of individually identifiable information 
outside of the study team will be considered a serious adverse event. 
 
8.3.2 CLASSIFICATION OF AN ADVERSE EVENT 
8.3.2.1 SEVERITY OF EVENT 
For adverse events (AEs) not included in the protocol defined grading system, the following guidelines 
will be used to describe severity.  
 
• 
Mild –  
o Events that do not interfere with the participant’s daily activities, cause significant distress, 
or interfere with major life functions.  
o Medication errors that require minimal or no treatment (NCC MERP categories B and C). 
• 
Moderate – 
o Events that cause inconvenience, interference with functioning, disruption of usual daily 
activity, or greater worry or concern.  
o Medication errors that require monitoring and/or intervention to prevent harm (NCC MERP 
categories D and E).  
• 
Severe –  
o Events that significantly interrupt a participant’s usual daily activity, cause distress, or 
interfere with major life functions.  
o Medication errors that prolong hospitalization, are potentially life-threatening, are 
incapacitating, or are fatal (NCC MERP categories F to I).   
 
8.3.2.2 RELATIONSHIP TO STUDY INTERVENTION 
 
All adverse events (AEs) must have their relationship to study intervention assessed by the clinician who 
examines and evaluates the participant based on temporal relationship and his/her clinical judgment. 
The degree of certainty about causality will be graded using the categories below. In a clinical trial, the 
intervention and/or study procedures must always be suspect.  
 
• 
Related – The AE is known to occur with the study intervention, there is a reasonable possibility that 
the study intervention caused the AE, or there is a temporal relationship between the study 
intervention and event. Reasonable possibility means that there is evidence to suggest a causal 
relationship between the study intervention and the AE. 
• 
Not Related – There is not a reasonable possibility that the administration of the study intervention 
caused the event, there is no temporal relationship between the study intervention and event 
onset, or an alternate etiology has been established. 
8.3.2.3 EXPECTEDNESS  

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
24 
The Principal Investigator and Chief Safety Officer will be responsible for determining whether an 
adverse event (AE) is expected or unexpected.  An AE will be considered unexpected if the nature, 
severity, or frequency of the event is not consistent with the risk information previously described for 
the study intervention. 
8.3.3 TIME PERIOD AND FREQUENCY FOR EVENT ASSESSMENT AND FOLLOW-UP 
 
• 
Events that are judged to be “related” to the SAFE Loop should be, at a minimum, temporally linked 
to the filing of an incident report for a Target Medication Event during the intervention period in an 
intervention nursing unit.  
• 
Alternatively, medication errors that are judged to be “related” to the SAFE Loop should generally 
occur during the latter part of the “SAFE Loop” period (when the intervention designed to prevent 
medication errors is designed and implemented) or the “follow-on” intervention period. It should 
occur in an intervention nursing unit and match the definition of a Target Medication Event, as well 
as have features suggesting that they were caused by the changes on the nursing unit made in 
response to the SAFE Loop.  
 
All AEs not meeting the criteria for SAEs will be captured on the appropriate case report form (CRF). 
Information to be collected includes event description, time of occurrence, an assessment of severity, 
relationship to study (assessed only by those with the training and authority to make this assessment), 
and time of resolution/stabilization. All AEs occurring while on study must be documented appropriately 
regardless of relationship. All AEs will be followed to adequate resolution. 
 
8.3.4 ADVERSE EVENT REPORTING  
The study team will report any potential or actual adverse events to the Cedars-Sinai Institutional 
Review Board and the external DSM within 7 days, to allow time to begin to investigate the event. A 
final report adjudicating the nature and severity of the event and relationship to the intervention will be 
provided to the IRB and external DSM within 30 days. 
The team will respond to any adverse events with a written mitigation and prevention plan to avoid 
recurrence, and consider whether doing so may be beneficial following any potential adverse 
events. This plan will be developed within 60 days, to allow for a thorough understanding of 
circumstances leading up to the event. 
 
 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
25 
 
8.3.5 SERIOUS ADVERSE EVENT REPORTING  
If an event may meet criteria for being serious, the study team will report it to the Cedars-Sinai 
Institutional Review Board and the external DSM within 24 hours, while the investigation proceeds. 
Following the final adjudication of the event, the report will also be submitted to the AHRQ Project 
Officer.  
8.3.6 REPORTING EVENTS TO PARTICIPANTS  
In the event of disclosures of protected health information outside the research team, we will adhere 
with HIPAA and other regulations regarding notifying affected patients. Events will be reported to 
participants if deemed appropriate by Dr. Coleman (internal DSM), Dr. Nuckols (PI), the external DSM, or 
the Cedars-Sinai OHRP. 
We will also report to nurses if individually identifying information of a sensitive nature is inadvertently 
disclosed outside the research team. 
8.3.7 EVENTS OF SPECIAL INTEREST  
Not Applicable 
 
8.3.8 REPORTING OF PREGNANCY  
Not Applicable 
 
8.4 
UNANTICIPATED PROBLEMS 
8.4.1 DEFINITION OF UNANTICIPATED PROBLEMS (UP) 
The Office for Human Research Protections (OHRP) considers unanticipated problems involving risks to 
participants or others to include, in general, any incident, experience, or outcome that meets all of the 
following criteria: 
• 
Unexpected in terms of nature, severity, or frequency given (a) the research procedures that are 
described in the protocol-related documents, such as the Institutional Review Board (IRB)-approved 
research protocol and informed consent document; and (b) the characteristics of the participant 
population being studied; 
• 
Related or possibly related to participation in the research (“possibly related” means there is a 
reasonable possibility that the incident, experience, or outcome may have been caused by the 
procedures involved in the research); and 
• 
Suggests that the research places participants or others at a greater risk of harm (including physical, 
psychological, economic, or social harm) than was previously known or recognized. 
 
This definition could include an unanticipated adverse device effect, any serious adverse effect on 
health or safety or any life-threatening problem or death caused by, or associated with, a device, if that 
effect, problem, or death was not previously identified in nature, severity, or degree of incidence in the 
investigational plan or application (including a supplementary plan or application), or any other 
unanticipated serious problem associated with a device that relates to the rights, safety, or welfare of 
subjects (21 CFR 812.3(s)). 
 
 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
26 
 
8.4.2 REPORTING UNANTICIPATED PROBLEMS TO PARTICIPANTS  
 
Unanticipated problems will be reported to participants if deemed appropriate by Dr. Coleman (internal 
DSM), Dr. Nuckols (PI), the external DSM, or the OHRP. 
 
9 
STATISTICAL CONSIDERATIONS  
9.1 
STATISTICAL HYPOTHESES 
Aim 1 Hypotheses: 
1.1 The rate per 1000 patient-days at which nurses report incidents will increase more in nursing units 
engaged in the SAFE Loop than in units using the existing reporting system. 
1.2 The number of system plus human factors per incident report will increase more in nursing units 
engaged in the SAFE Loop than in units using the existing reporting system. (Primary outcome for Aim 1) 
Aim 2 Hypotheses: 
2.1 Nurses’ perceptions of feedback and communication about error on the nursing unit will improve 
more in nursing units engaged in the SAFE Loop than in units using the existing reporting system. 
(Primary outcome for Aim 2) 
2.2 Nurses’ perceptions of the frequency with which events are reported on the nursing unit will 
increase more in nursing units engaged in the SAFE Loop than in units using the existing reporting 
system. 
Aim 3 Hypotheses: 
3.1 The rate of medication events designated as high priorities will decline more in nursing units 
engaged in the SAFE Loop than in units using the existing reporting system. (Primary outcome for Aim 3) 
3.2 The rate of medication events designated as high priorities and involving harm will decline more in 
nursing units engaged in the SAFE Loop than in units using the existing reporting system. 
9.2 
SAMPLE SIZE DETERMINATION 
Aim 1: 
On average per 6-month study period, each unit 
will provide 7,362 patient-days of care (294,470 
pt-days/20/2) and submit 16.6 reports (665 
reports/20/2). H1.1. The baseline rate is 2.26 
incident reports per 1000 patient-days (665 
reports/294,470). H1.2. Based on prior studies, 
the expected baseline number of system plus 
Table 1: Minimum Detectable Difference in Outcome 
Implementation trend 
Betw-Cluster Coeff. of Variation 
0.01 
0.05 
0.10 
0.20 
1.1. Incident Reports per 1000 Patient-days 
Yes 
1.241 
1.277 
1.392 
1.851 
No 
0.794 
0.815 
0.877 
1.115 
1.2. System plus Human Factors per Incident Report 
Yes 
0.471 
0.481 
0.515 
0.651 
No 
0.299 
0.305 
0.322 
0.391 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
27 
human factors is 0.70 per report.1 Table 1 presents the minimum detectable differences in outcomes as 
functions of the between-cluster coefficient of variation (CV) and the presence of implementation trend, 
using two-sided t-tests for Poisson rates with 80% power at 5% significance level. These differences 
would be meaningful to safety leaders, reflecting >35% increases in reporting rates and enhanced 
detection of the multiple contributing multiple factors that are typically involved in every error.  
Aim 2: 
Power calculations are the same for both 
composite outcome measures. The baseline 
percentages of positive responses are 75% (H2.1) 
and 72% (H2.2). We assume 85% of 1980 nurses 
will be eligible and a 60% response rate (50.5 
responses per nursing unit per deployment). We also assume a discrete uniform distribution for each 
composite measure with only four possible values (0%, 33%, 66%, 100%) per nurse because each 
measure contains 3 items. Therefore, we assume standard deviation (SD) equal to 37%, and a between-
cluster coefficient of variation of 0.65 as recommended by Campbell and Walters.2 The Table presents 
the minimum detectable difference in the percentage of positive responses as a function of the intra-
cluster correlation (ICC) and presence of implementation trend, using a two-sided t-test for Normal 
means with 80% power at 5% significance level. Based on prior studies, a meaningful difference in the 
percentage of positive responses is 8.6 percentage points (pp).3 Missing data patterns will be examined 
using the method proposed by Little4 if the data is found to be not missing at completely random, 
missing values will be imputed with values drawn from fully conditional specifications using the 
multivariate imputation by chained equations (MICE) algorithm,5,6 with the number of imputed datasets 
chosen such that the maximum loss of efficiency of 5%.7 The plausibility of imputed values will be 
checked by corresponding diagnostics. The outlined statistical analysis will be performed for each 
imputed dataset and results will be pooled. All hypotheses will be two-sided at 5% significance level. 
Calculations will be performed using R-package, version 3.5.3. 
 
 
 
 
1 Nuckols TK, Bell DS, Paddock SM, Hilborne LH. Contributing factors identified by hospital incident report narratives. Qual Saf Health Care. Oct 
2008;17(5):368-372. 
2 Campbell MJ, Walters SJ. How to design, analyse and report cluster randomised trials in medicine and health related research. John Wiley & 
Sons; 2014. 
3 Campione J, Famolaro T. Promising Practices for Improving Hospital Patient Safety Culture. Jt Comm J Qual Patient Saf. Jan 2018;44(1):23-32. 
4 Little RJ. A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association. 
1988;83(404):1198‐1202. 
5 Audigier, V., White, I., Jolani, S. Debray, T., Quartagno, M., Carpenter, J., van Buuren, S. and Resche-Rigon, M. Multiple imputation for 
multilevel data with continuous and binary variables. Statistical Science 2018;33(2):160-183 
6 Resche-Rigon, M. and White, I. R. Multiple imputation by chained equations for systematically and sporadically missing multilevel data. 
Statistical Methods in Medical Research 2018; 27(6): 1634-1649 
7 White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice. Stat Med. 2011 Feb 
20;30(4):377-99. 
Table 2: Minimum Detectable Difference in Outcome 
Implementation 
trend 
Intra-cluster Correlation 
0.01 
0.05 
0.10 
0.20 
Yes 
10pp* 
11pp 
12pp 
13pp 
No 
7pp 
8pp 
10pp 
11pp 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
28 
Aim 3:  
Table 3 presents the minimum detectable 
difference in event rates as functions of 
between-cluster coefficient of variation and 
implementation trend using a two-sided t-test 
for Poisson rates with 80% power at 5% 
significance level. A difference of 0.50 events per 
1000 patient days is clinically meaningful, 
representing the avoidance of 147 events and 
116 harmful events per year hospital-wide.  
Qualitative Interviews: 
Qualitative methods do not involve statistical tests or rely on calculations of statistical power. Instead, 
we provide a rationale for the proposed sample size and describe analytical methods. 
After SAFE Loop implementation, we will conduct one-on-one in-person interviews with 10 Nursing Unit 
Managers and 22 frontline nurses. This sample size will enable us to perform purposive sampling to 
acquire broad representation by randomization block (there are 5 blocks), study group (there are four 
groups, with five nursing units per group – one from each block; half of the groups receive the 
intervention), and nursing unit (there are 10 nursing units that will be randomized to the SAFE Loop 
arm). Thus, we will interview one nurse manger and two nurses from each study nursing unit, about 6 
per block, and 16 per group.  
 
9.3 
POPULATIONS FOR ANALYSES 
The trial will randomize 20 nursing units to two arms: (1) the SAFE Loop, and (2) the existing reporting 
system (control). All analyses will compare data between these two arms. 
9.4 
STATISTICAL ANALYSES 
 
9.4.1 GENERAL APPROACH 
 We will employ a cluster RCT to compare nursing units that implement the SAFE Loop, a system-level 
change in incident reporting, with nursing units that continue to use the existing reporting system. 
Outcomes will include changes in incident reporting practices, nurses’ attitudes toward reporting, and 
rates of high-priority medication events. Qualitative interviews with nurses will provide insight into 
implementation. See Section 9.4.2 for detailed information describing the analytic plan. 
9.4.2 ANALYSIS OF THE PRIMARY EFFICACY ENDPOINT(S) 
Aim 1 Analyses: 
Descriptive Analysis: We will perform descriptive analyses of reporting practices during the baseline 
periods and intervention periods, stratified by study arm. We will calculate rates of reported Target 
Medication Events and total reported medication events per 1000 patient-days (# of reports*1000/# of 
patient-days). Next, we will calculate the percentages of reported incidents in each NCC MERP category 
and the percentages of reports for which preventability could be determined. Finally, we will calculate 
Table 3: Minimum Detectable Difference in Outcome 
Implementation trend 
Betw-Cluster Coeff of Variance 
0.01 
0.05 
0.10 
0.20 
3.1 Target Event Rate 
Yes 
0.766 
0.929 
1.292 
2.108 
No 
0.516 
0.631 
0.886 
1.474 
3.2 Harmful Event Rate 
Yes 
0.322 
0.335 
0.373 
0.486 
No 
0.220 
0.230 
0.257 
0.340 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
29 
the number of contributing system, human, and patient factors per report and describe the types of 
factors reported. 
Statistical Analyses: We will compare changes over time between study arms for: (a) rate of incident 
reports per 1000 patient-days, and (b) number of system plus human factors per report. Multivariable 
Poisson regression models will be fitted with response variables as (a) number of reports per patient, 
with offset given by the length of stay divided by 1000; and (b) number of system plus human factors 
per report. For both models, the main hypothesis is whether there is an interaction between study arm 
and time period (baseline, post-intervention). Nursing units will be divided between two 
implementation steps, creating the possibility of an implementation trend. Therefore, a three-way 
interaction among study arm, time period, and implementation step (Step 1: Group 1 and 2, Step 2: 
Group 3 and 4) will be tested. If this is statistically significant, results will be presented separately for 
each implementation step. Otherwise, implementation step will be an additive effect. Random effects 
will describe nursing units and models will adjust for demographics, Elixhauser comorbidity index, and 
primary payer (collected under Aim 3). All hypotheses will be two-sided at 5% significance level. 
Calculations will be performed in R-package, version 3.5.3. 
Aim 2 Analyses: 
Descriptive Analysis: We will calculate survey response rates, examine respondent characteristics (years 
worked in current profession, hospital, and current nursing unit and hours worked per week), and 
describe outcome measures (overall scores and composite measures) during each study period, 
stratified by study arm.  
Assessment of Survey Measurement Properties: To evaluate the two selected composite measures’ 
internal consistency, we will calculate Cronbach’s α using data from baseline survey. To assess construct 
validity and convergent validity, respectively, we will evaluate associations (Pearson correlation 
coefficients) between scores on the composite measures and: (1) the full Hospital SOPS, and (2) incident 
reporting rates at the nursing unit level. To assess responsiveness, we will compare scores from baseline 
and follow-up surveys among nurses in the SAFE Loop arm (paired t-tests). If the composite measures do 
not demonstrate good internal consistency, construct, and convergent validity, we will use the full 
Hospital SOPS. 
Statistical Analyses: We will compare changes over time between study arms for Feedback and 
Communication About Error and Frequency of Events Reported. Multivariable Normal regression models 
will be fitted for both endpoints and we will test hypotheses similarly to Aim 1. We will adjust for nurse 
characteristics from the AHRQ survey: years working in this hospital, years worked in current nursing 
unit, and hours worked per week. We will perform subgroup analyses by years of experience in current 
profession, because experience has been associated with medication error rates.59 
Aim 3 Analyses: 
Descriptive Analysis: We will calculate rates of preventable ADEs and potential ADEs per 1,000 patient-
days, both overall and for the Target Medication Events for the Group 1 and 3 nursing units. We will 
describe the severity of the events, medication classes, errors, clinicians involved, and stages in drug 
therapy.  
Statistical Analysis: We will compare changes over time between study arms for rates of (a) Target 
Medication Events per 1000 patient-days and (b) harmful Target Medication Events per 1000 patient-
days. Multivariable Poisson models will be fitted with response variable as the number of (a) target 
medication events and (b) harmful medication events, with offset the length of stay divided by 1000 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
30 
days. We will test hypotheses similarly to under Aim 1. Model covariates will include patient 
demographics, biological sex, Elixhauser comorbidity index, and insurance payer. 
 
9.4.3 ANALYSIS OF THE SECONDARY ENDPOINT(S) 
 
Qualitative Analysis of Implementation:  
Qualitative interviews will be transcribed from audio recordings, with personal identifiers removed. 
removed. Pairs of researchers trained in qualitative analysis will analyze transcripts in Dedoose using a 
combination of content-analysis and qualitative inquiry, allowing them to discover and quantify the 
nurses’ experiences and perceptions. We will use an iterative process to identify a priori themes based 
on the domains above, and to create in vivo themes as they emerge during coding (e.g., specific barriers 
to implementation).65 The two coders will code each interview independently and then discuss 
variations until consensus is reached. After coding all interviews, we will use the constant comparative 
method to combine similar themes with limited data under more general themes.66 In the final step of 
analysis, we will review Dedoose code reports (participant N and density) of salient themes and coding 
matrices related to significant interview findings.  
 
9.4.4 SAFETY ANALYSES 
 
Nurses:  
• 
The Analytical Team will monitor responses to a relevant domain on the AHRQ Hospital Survey on 
Patient Safety Culture, Nonpunitive Response to Error, which reflects whether nurses feel that they 
have been blamed for reported incidents. This survey will be deployed before and after the 
implementation of the SAFE Loop, enabling the research team to detect problems with the first 
implementation phase before the subsequent phase begins. 
• 
Dr. Coleman will meet periodically with nursing unit managers to qualitatively monitor nurses’ 
perceptions of the study, including whether they have any concerns or any potential or actual 
adverse events have occurred.  
• 
Dr. Seferian will monitor for qualitative increases in the number of referrals of nurses to the Human 
Resources Department on the intervention nursing units during the study period.  
Patients:  
• 
Dr. Coleman will work with the Cedars-Sinai Patient Safety Department, which handles incident 
reports, to qualitatively monitor whether there may be any reports addressing risks to patients 
associated with changes in practice on study nursing units that were implemented in response to 
the intervention.  
 
9.4.5 BASELINE DESCRIPTIVE STATISTICS 
 
Not Applicable 
 
9.4.6 PLANNED INTERIM ANALYSES  
 
Not Applicable 
 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
31 
9.4.7 SUB-GROUP ANALYSES 
 
We will perform two prespecified secondary analyses, stratifying by biological sex and age group (18-44, 
45-65, over 65). Effectiveness may differ in these populations because Hug et al. found that 60.3% of 
preventable plus potential ADEs occur among women while 77.6% occur among adults age 65+.61  
 
9.4.8 TABULATION OF INDIVIDUAL PARTICIPANT DATA 
 
Not Applicable 
 
9.4.9 EXPLORATORY ANALYSES 
 
Not Applicable 
 
10 SUPPORTING DOCUMENTATION AND OPERATIONAL CONSIDERATIONS 
10.1 REGULATORY, ETHICAL, AND STUDY OVERSIGHT CONSIDERATIONS 
10.1.1 INFORMED CONSENT PROCESS 
10.1.1.1 CONSENT/ASSENT AND OTHER INFORMATIONAL DOCUMENTS PROVIDED TO 
PARTICIPANTS 
 
We are requesting a waiver of informed consent for implementation of the SAFE Loop intervention 
(both nurse and patients) and data collection procedures for Aim 1 (incident reports, both nurses and 
patients), and Aim 3 (adverse medication events, patients). 
 
For nurses participating in Aim 2 (surveys) we are requesting a waiver of consent documentation. As the 
survey is via REDCap and is not collecting any identifiable information, requiring a signature would be 
the only link to a subject’s identify. An information sheet will be provided with full consent information. 
Agreeing to complete and submit the survey will indicate their consent to participate. 
 
For nurses participating in the qualitative interviews of implementation, we will provide consent forms 
describing in detail the study procedures and risks to the participant. Written documentation of 
informed consent is required prior to starting intervention/administering study intervention.  The 
following consent materials are submitted with this protocol. 
 
10.1.1.2 CONSENT PROCEDURES AND DOCUMENTATION 
 
Informed consent is a process that is initiated prior to the individual’s agreeing to participate in the 
study and continues throughout the individual’s study participation. For the survey, electronic consent 
forms will be Institutional Review Board (IRB)-approved and the participant will be asked to read and 
review the document. The form will explain the research study to the participant and an FAQ will answer 
any questions that may arise. A verbal explanation will be provided in terms suited to the participant’s 
comprehension of the purposes, procedures, and potential risks of the study and of their rights as 
research participants. Participants must be informed that participation is voluntary and that they may 
withdraw from the study at any time, without prejudice. A copy of the informed consent document will 
be given to the participants for their records. The informed consent process will be conducted and 
documented in the source document (including the date), and the form signed, before the participant 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
32 
undergoes any study-specific procedures. The rights and welfare of the participants will be protected by 
emphasizing to them that the quality of their medical care will not be adversely affected if they decline 
to participate in this study. 
 
10.1.2 STUDY DISCONTINUATION AND CLOSURE 
 
This study may be temporarily suspended or prematurely terminated if there is sufficient reasonable 
cause.  Written notification, documenting the reason for study suspension or termination, will be 
provided by the suspending or terminating party to study participants, investigator, funding agency, and 
regulatory authorities.  If the study is prematurely terminated or suspended, the Principal Investigator 
(PI) will promptly inform study participants, the Institutional Review Board (IRB), and sponsor and will 
provide the reason(s) for the termination or suspension.  Study participants will be contacted, as 
applicable, and be informed of changes to study visit schedule. 
  
Circumstances that may warrant termination or suspension include, but are not limited to: 
• 
Determination of unexpected, significant, or unacceptable risk to participants 
• 
Demonstration of efficacy that would warrant stopping    
• 
Insufficient compliance to protocol requirements 
• 
Data that are not sufficiently complete and/or evaluable 
• 
Determination that the primary endpoint has been met 
• 
Determination of futility 
 
Study may resume once concerns about safety, protocol compliance, and data quality are addressed, 
and satisfy the sponsor and IRB. 
 
10.1.3 CONFIDENTIALITY AND PRIVACY  
 
Participant confidentiality and privacy is strictly held in trust by the participating investigators, their 
staff, and the sponsor(s) and their interventions. This confidentiality is extended to cover testing of 
biological samples and genetic tests in addition to the clinical information relating to participants. 
Therefore, the study protocol, documentation, data, and all other information generated will be held in 
strict confidence. No information concerning the study or the data will be released to any unauthorized 
third party without prior written approval of the sponsor.  
 
All research activities (e.g., nursing interviews) will be conducted in as private a setting as possible. 
 
The study monitor, other authorized representatives of the sponsor, representatives of the Institutional 
Review Board (IRB), regulatory agencies or pharmaceutical company supplying study product may 
inspect all documents and records required to be maintained by the investigator, including but not 
limited to, medical records (office, clinic, or hospital) and pharmacy records for the participants in this 
study. The clinical study site will permit access to such records. 
 
The study participant’s contact information will be securely stored at each clinical site for internal use 
during the study. At the end of the study, all records will continue to be kept in a secure location for as 
long a period as dictated by the reviewing IRB, Institutional policies, or sponsor requirements. 
 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
33 
Study participant research data, which is for purposes of statistical analysis and scientific reporting, will 
be transmitted to and stored at in Redcap. This will not include the participant’s contact or identifying 
information. Rather, individual participants and their research data will be identified by a unique study 
identification number. The study data entry and study management systems used by clinical sites and by 
Cedars-Sinai research staff will be secured and password protected. At the end of the study, all study 
databases will be de-identified and archived at Cedars-Sinai Medical Center. 
 
Confidentiality Statute: Studies funded by the Agency for Healthcare Research and Quality are protected 
under the AHRQ Confidentiality Statute. This statute protects identifiable research information from 
forced disclosure. It allows the investigator and others who have access to research records to refuse to 
disclose identifying information on research participation in any civil, criminal, administrative, legislative, 
or other proceeding, whether at the federal, state, or local level. By protecting researchers and 
institutions from being compelled to disclose information that would identify research participants, this 
statute helps achieve the research objectives and promote participation in studies by helping assure 
confidentiality and privacy to participants. 
 
10.1.4 FUTURE USE OF STORED SPECIMENS AND DATA  
 
Data collected for this study will be analyzed and stored at Cedars-Sinai. After the study is completed, 
the de-identified, archived data will be destroyed.  
 
10.1.5 KEY ROLES AND STUDY GOVERNANCE 
 
Principal Investigator 
Internal DSM 
Teryl Nuckols, MD, MSHS, Vice Chair for Clinical 
Research, Director of General Internal Medicine           
Bernice Coleman, PhD, ACNP-BC, Associate Director of 
Nursing Research and the Associate Director of the 
Nursing Performance Improvement Department 
Cedars-Sinai Medical Center 
Cedars-Sinai Medical Center 
8700 Beverly Boulevard, Becker 113 
Los Angeles CA 90048 
8700 Beverly Boulevard, Room 1132  TSB Building 
Los Angeles CA 90048 
310‐423‐2760 
310-423-6178 
teryl.nuckols@cshs.org 
bernice.coleman@cshs.org 
Executive Oversight Team: The organizational chart below shows the various roles of members of the 
study team. This team will be comprised of Dr. Nuckols (PI), Dr. Seferian (Chief Patient Safety Officer at 
Cedars-Sinai), Dr. Coleman (Associate Director of Nursing Research and of Nursing Performance 
Improvement), Dr. Cohen (Human factors expert), Dr. Leang (Pharmacist involved in medication safety 
at Cedars-Sinai) and Ms. Kim (Biostatistics and Bioinformatics at Cedars-Sinai). Dr. Nuckols will assume 
oversight and responsibility of all aspects of the project. Dr. Seferian will provide senior leadership from 
the perspective of implementing system changes designed to improve patient safety. Ms. Kim will 
provide senior expertise in the design of RCTs and statistical analysis. The other members of the 
advisory board will provide specific content expertise. All team members, especially the Executive 
Oversight Team and the DSMs, will safeguard data and monitor the safety of participants. The Data and 
Safety Monitors include an internal team member (Dr. Coleman) and an external Data and Safety 
Monitor (DSM, to be named).  

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
34 
Organizational Chart
 
SAFE Loop Team: The SAFE Loop team plays a key role in all aspects of the intervention, as described in 
the Approach section. This team will solicit input from nurses on study nursing units, train nurses to 
report Target Events, emphasize near misses, and describe circumstances leading up to events in detail. 
The team will partner with study units on the analysis and interpretation of events, as well as the 
development of a plan to mitigate events. The team will include Dr. Seferian as the Safety Officer, Dr. 
Van Groningen as the Champion, and an experienced analyst who routinely evaluates incident reports in 
the hospital. 
 
Institute for Safe Medication Practices (ISMP): ISMP works with healthcare systems and practitioners to 
advocate for patient safety and promote safe medication practices. ISMP will serve as a resource to the 
SAFE Loop team during the analysis of reports and integration of events, and an ISMP staff member will 
also serve as a member of the Advisory Board. Finally, ISMP will assist with the dissemination of study 
findings via its usual channels for sharing the results of investigations of incident reports and for 
advancing medication safety nationally. 
Advisory Board: We will recruit 5-7 national experts in incident reporting, human factors, medication 
safety, nursing care, and implementation science as well as a professional patient advocate to advise the 
project leaders at key junctures. The board will participate in one meeting before implementation to 
advise on compatibility with nursing practice, acceptability to nurses, and likelihood of improved safety, 
and overcoming barriers. After data collection is complete, the Advisory Board will discuss interpretation 
and dissemination of results, particularly adaptation of the intervention for use in other hospitals. Each 
meeting will last about 60 minutes, and there will be materials sent ahead to participants. Advisory 
Board members will only review aggregate information or completely de-identified qualitative 
information (e.g., a description of a medication safety event or a quote from an interviewee). They will 
not receive any sensitive or confidential data of any kind. Pursuant to discussions with the IRB, Advisory 
Board members will be considered external advisors, similar to a DSMB (so neither consent of the board 
members nor CITI training is required).  
Outcome Assessment Team: This team will include both leaders who will direct each of the data 
collection efforts and support staff who will conduct the data collection. All data will be collected in a 
manner that blinds the assessors to the study arm. 
Dr. Cohen will direct the extraction of data from hospital incident reports; she has a PhD in human 
factors and has applied the HFACS-Healthcare to incident reports at Cedars-Sinai previously. She will 
oversee two analysts with experience in patient safety and assure reliability in scoring. 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
35 
Dr. Kaiser and Ms. McCleskey will oversee the deployment of the survey of nurses including assuring 
adequate response rates are obtained, and they will perform the qualitative interviews of nurses to 
examine intervention implementation. In addition, these two individuals will conduct qualitative 
analyses of implementation. Dr. Kaiser will provide senior expertise in qualitative methods, while Ms. 
McCleskey will oversee and perform day to day data collection activities.  
Dr. Nuckols will direct the extraction of data on medication events from electronic medical records. She 
will train nurses and physicians in methods for screening for possible medication events and 
adjudication of such events, respectively. She will oversee three nurses and three physicians who will be 
blinded to the study arms. 
Analytical Team: Sungjin Kim, MS and André Rogatko are statisticians with experience in experimental 
design including RCTs, psychometric analyses of surveys, biostatistics, and bioinformatics. They will: (1) 
coordinate statistical activities to ensure that investigators have ready access to statistical consultation 
and support, (2) provide statistical expertise in the design of experiments and studies, including research 
proposal development, sample size determination, and plans for interim reviews and final analysis, (3) 
assist with the writing of statistical components of manuscripts, (4) review the integrity and statistical 
soundness of all studies, (5) provide statistical analysis for all projects using appropriate statistical and 
computing methodologies, and (6) assist in the interpretation and presentation of results. Dr. Rogatko 
will advise Ms. Kim and the project team on advanced issues related to trial design and biostatistics. 
 
10.1.6 SAFETY OVERSIGHT 
 
Internal Data and Safety Monitor: Dr. Coleman has been designated as an internal Data and Safety 
Monitor because Dr. Coleman is a nurse with extensive experience in human subjects research. Dr. 
Coleman will be responsible for recruiting and interfacing with the DSM and for refining and maintaining 
the Data and Safety Monitoring Plan. In addition, she will oversee internal monitoring procedures, 
described below.  
External Data and Safety Monitor (DSM): The DSM will be an expert in nursing and research ethics who 
is a single individual unaffiliated with Cedars-Sinai. The DSM will be tasked with independent oversight 
to monitor for adverse events, defined in Section 8.3.1. The DSM will review evidence of adverse events 
to recommend whether trials should be continued, altered, or terminated. We will recruit the DSM prior 
to randomization or data collection and convene meetings between the Executive Oversight Committee 
and DSM at designated points in the trial, in accordance with standard AHRQ and NIH procedures. The 
DSM will have no associations with the study team members or affiliations with Cedars-Sinai, and no 
conflicts of interest with study outcomes. 
The Executive Oversight Team concluded that this pragmatic trial does not require a full Data and Safety 
Monitoring Board (DSMB) after conferring with the Cedars-Sinai Institutional Review Board and the 
UCLA Clinical and Translational Science Institute (CTSI) as well as reviewing a relevant publication.8 
Factors that argue against convening a full DSMB are: (1) recruitment and randomization are at the level 
of the nursing unit, obviating the need to monitor recruitment rates; (2) the intervention is a quality 
improvement intervention implemented at the nursing unit level with the support of the Cedars-Sinai 
 
 
8 Gregory E Simon, Susan M Shortreed, Rebecca C Rossom, Robert B Penfold, Jo Ann M Sperl-Hillen, Patrick 
O'Connor. Principles and procedures for data and safety monitoring in pragmatic clinical trials. Trials. 2019 Dec 
9;20(1):690. doi: 10.1186/s13063-019-3869-3.  

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
36 
nursing and patient safety leadership; (3) the intervention has an indirect effect on patient care; treating 
clinicians retain full freedom and responsibility for making decisions in the care of individual patients 
(see Section 8.3.2.2); (4) the intervention poses a low risk of causing physical harm to study subjects; (5) 
study measures reflecting physical harm—rates of Target Medication Events—are collected 
retrospectively from existing data sources long after the care is provided, and it would be infeasible to 
detect a statistically significant increase in these events and shut down the trial in a timely manner; (6) 
other study outcome measures are not measures of physical harm but rather the behaviors and 
attitudes of clinicians. See Section 8.3 for additional information on Adverse Events. 
 
10.1.7 CLINICAL MONITORING 
 
Clinical site monitoring is conducted to ensure that the rights and well-being of trial participants are 
protected, that the reported trial data are accurate, complete, and verifiable, and that the conduct of 
the trial is in compliance with the currently approved protocol/amendment(s), with International 
Conference on Harmonisation Good Clinical Practice (ICH GCP), and with applicable regulatory 
requirement(s). 
 
10.1.8 QUALITY ASSURANCE AND QUALITY CONTROL 
 
Our research team at Cedars-Sinai will perform internal quality management of study conduct, data and 
biological specimen collection, documentation and completion.  An individualized quality management 
plan will be developed to describe a site’s quality management. 
 
Quality control (QC) procedures will be implemented beginning with the data entry system and data QC 
checks that will be run on the database will be generated. Any missing data or data anomalies will be 
communicated to the site(s) for clarification/resolution. 
 
Following written Standard Operating Procedures (SOPs), the monitors will verify that the clinical trial is 
conducted and data are generated and biological specimens are collected, documented (recorded), and 
reported in compliance with the protocol, International Conference on Harmonisation Good Clinical 
Practice (ICH GCP), and applicable regulatory requirements (e.g., Good Laboratory Practices (GLP), Good 
Manufacturing Practices (GMP)).  
 
Our site will provide direct access to source data/documents and reports for the purpose of monitoring 
and auditing by the sponsor, and inspection by local and regulatory authorities. 
 
10.1.9 DATA HANDLING AND RECORD KEEPING  
10.1.9.1 DATA COLLECTION AND MANAGEMENT RESPONSIBILITIES  
 
Data collection is the responsibility of the research team under the supervision of Dr. Nuckols (PI). The PI 
is responsible for ensuring the accuracy, completeness, legibility, and timeliness of the data reported. 
 
All source documents will be completed in a neat, legible manner to ensure accurate interpretation of 
data.   
 
Hardcopies of the study visit worksheets will be provided for use as source document worksheets for 
recording data for each participant enrolled in the study.  Data recorded in the electronic case report 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
37 
form (eCRF) derived from source documents should be consistent with the data recorded on the source 
documents.  
 
Clinical data (including adverse events (AEs), concomitant medications, and expected adverse reactions 
data) and clinical laboratory data will be entered into Redcap, a 21 CFR Part 11-compliant data capture 
system. The data system includes password protection and internal quality checks, such as automatic 
range checks, to identify data that appear inconsistent, incomplete, or inaccurate. Clinical data will be 
entered directly from the source documents. 
 
Procedures for Assuring Data Integrity and Confidentiality 
Data safeguarding is the responsibility of the study PI as well as all individuals working with human 
subjects data on the research team. The PI will ensure the accuracy, completeness, and timeliness of the 
data reported as well as protect the data from inadvertent disclosure.  
Data Management: Research data will be extracted into the Research Electronic Data Capture (REDCap) 
system, a secure, web-based application that supports data capture and management. REDCap 
(Research Electronic Data Capture) is a web application platform for creating and managing online 
surveys and databases. REDCap provides interfaces for data entry, audit trails for tracking data 
management and data cleaning and recoding activities, automated export procedures to common 
statistical software packages, procedures for important data from external sources, and standard 
methodsUsers can either create and design projects using 1) the online method from a generic web 
browser using the Online Designer; and/or 2) the offline method by constructing a 'data dictionary' 
template file in Microsoft Excel, which can be later uploaded into REDCap. Both surveys and databases 
(or a mixture of the two) can be built using these methods. REDCap provides audit trails for tracking 
data manipulation and user activity, as well as automated export procedures for seamless data 
downloads to Excel, PDF, and common statistical packages (SPSS, SAS, Stata, R). Also included are a 
built-in project calendar, a scheduling module, ad hoc reporting tools, and advanced features, such as 
branching logic, file uploading, and calculated fields.9  
REDCap is widely used for clinical research projects and has been adopted by the CTSI (Clinical and 
Translational Sciences Institute) consisting of Cedars-Sinai Medical Center (CSMC), University of 
California Los Angeles (UCLA), King Drew, and Harbor General as the tool for CTSI databases. 
REDCap servers are housed at the CSMC Enterprise Information Systems (EIS) secure Data Center. The 
EIS Data Center is a modern facility with 24/7/365 power, cooling, connectivity, security and monitoring. 
Only authorized EIS Data Center staff have access to manage the physical servers. Security measures for 
the database include password management, inactivity logouts and maximum login attempts. 
Database/server TSM tape backups occur on a regular basis (daily and weekly) as per EIS NAS Backup 
Retention Schedule Policy and Tape Recall Procedure Policy (offsite secured storage). 
Network transmissions (data entry, survey submission, web browsing, etc.) in REDCap are protected via 
Secure Sockets Layer (SSL) encryption to guard against unauthorized access to electronic protected 
health information (EPHI). Network security includes IP/ firewall restrictions, and SSL enabled web 
 
 
9 Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap) - A 
metadata-driven methodology and workflow process for providing translational research informatics support, J 
Biomed Inform. 2009 Apr;42,377-81. 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
38 
servers for https browser access. REDCap database meets the Health Insurance Portability and 
Accountability Act (HIPAA) Security Rule requirements. Appropriate administrative, physical, and 
technical safeguards are in place to ensure the confidentiality, integrity, and security of electronic 
protected health information (EPHI). 
Assuring Data Integrity: All research staff who use REDCap will undergo training, and all data collection 
procedures will be monitored throughout the study period to ensure data quality and integrity. The 
Analytical Team will be responsible for receiving, reviewing, and cleaning all study data as well as 
generating regularly scheduled data quality reports and study status reports. Standardized reports will 
be developed during Year 1, during creation of the REDCap databases, and will be provided to the full 
study team and the Data and Safety Monitor at pre-specified times. These reports will include survey 
response rates for each study wave, progress on the analyses of incident reports and medical records, 
inter-rater reliability for those data extraction efforts, data quality concerns, and adverse events, and 
other elements requested by the study team. The Analytical Team will ensure secure data storage of 
verified and documented data, audit trails for data, and ensure back-up systems are working. Any 
missing or anomalous data will be referred to the respective Outcomes Assessment Teams for 
investigation.   
Protecting Data from Disclosure: The study team will develop a detailed data safeguarding plan and 
submit it to the Cedars-Sinai Institutional Review Board and the Data and Safety Monitor and DSMB, if 
applicable. In brief, all data will be stored on secure, encrypted, password protected servers in locked 
facilities at Cedars-Sinai. Data transmission will be via secure, encrypted, password protected electronic 
interfaces. REDCap is a 21 CFR Part 11-compliant web-based database system. Only HIPAA-compliant 
Limited Datasets will be shared with the Analytical Team (containing dates and study identification 
numbers). 
Details on Data Management by Study Procedure: 
Data sources will include nurses’ work schedules, surveys of nurses, incident reports, medical records, 
and qualitative interviews (audiorecordings and transcripts).  
Work Schedules, Surveys, and Interviews with Nurses: The work schedules, surveys, and interviews with 
nurses will be individually identifiable (names, email addresses, possibly dates of birth) but will not 
contain sensitive information. To determine which nurses are eligible for the survey and interview 
procedures, we will obtain nurses’ work schedules to ascertain how often the nurses work on a 
particular study nursing unit during the study waves of implementation and whether the nurses worked 
on the units for an entire study wave. If multiple nurses have the same name, we will use dates of birth 
and email addresses to distinguish them. We will distribute the survey via Research Electronic Data 
Capture system (REDCap).  
Analysis of Incident Reports: The incident reports themselves constitute sensitive information for nurses 
and protected health information for patients. The data elements extracted from incident reports will 
not include individually identifiable information about nurses, but will include protected health 
information on patients and meet the definition of Limited Datasets under HIPAA. Data elements will 
pertain to the quality of incident reports and include incident dates and patient study identification 
numbers.  
The analysts will obtain standard certifications in human subjects’ protections and be trained to apply 
study instruments, use the electronic data collection interface, and adhere to study data protection 
procedures. Analysts will record the extracted data elements in a specially developed data collection 
interface in REDCap. Data extraction will be conducted by duplicate reviewers and then reconciled to 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
39 
ensure data completeness and reliability. Analysts will conduct extensive pilot tests of study instruments 
using sample data sets until interrater reliability (agreement) as to scoring is very high. 
To create patients study identification numbers to use the incident reporting dataset, the research team 
will create a master list of patient names, dates of birth, and unique identification numbers created 
using a random number generator. This linkage dataset will be maintained apart from other datasets 
and will be destroyed following the completion of the final paper from the study.   
Analysis of Medical Records: Medical records include protected health information for patients. The data 
elements extracted will not include individually identifiable information about nurses, but will include 
protected health information on patients and meet the definition of Limited Datasets under HIPAA. Data 
elements will pertain to medication events (including medications, outcomes, errors, and dates of 
events, etc.), patient age, gender, race, ethnicity, medical comorbidities, and nursing units where the 
patient received care on each date in the study. In addition to patients electronic medical records, this 
data collection procedure also includes a pharmacy dataset called iVents, which captures issues related 
to medication orders. 
Data extraction will occur in two stages. First, trained research nurses who work at Cedars-Sinai will 
manually extract data elements and draft summaries of any medication events; these summaries will 
include event dates and patient study identification numbers but no other identifiers, meeting the 
definition of a limited dataset under HIPAA. Second, trained physicians will review these summaries and 
classify the medication events. Both nurses and physicians will have standard certifications in human 
subjects’ protections and be trained in data collection procedures. Both nurses and physicians will 
record the extracted data elements in a specially developed data collection interface in REDCap. The 
study team will conduct extensive supervised trainings (separately) for nurses and physicians, and 
require duplicate reviews of selected events to assess reliability. We will conduct extensive pilot tests of 
study instruments using sample data sets until nurse and physician interrater reliability (agreement) as 
to scoring is very high. 
Qualitative Interviews: Researchers trained in qualitative methods will meet one-on-one with nurses 
who consent to interviews. With respondent permission, the interviews will be audiorecorded. 
Interviews will be transcribed from audio recordings, with personal identifiers removed from the 
transcriptions. Audiorecordings will be destroyed at the conclusion of the research. 
 
10.1.9.2 STUDY RECORDS RETENTION  
 
Study documents should be retained for a minimum of 2 years after the end of the study. These 
documents will be retained for a longer period, however, if required by local regulations. No records will 
be destroyed without the written consent of the sponsor, if applicable. It is the responsibility of the 
sponsor to inform the investigator when these documents no longer need to be retained. 
 
10.1.10 
PROTOCOL DEVIATIONS  
 
Note that implementation of the intervention is designed to be flexible and is performed by hospital 
leaders, clinicians, and staff who bear the ultimate responsibility for patient care and patient safety, as 
noted in Section 6.4. Protocol deviations will NOT be considered noncompliance or reported, since they 
have only indirect effects on the care of individual patients. 
 
10.1.11 
PUBLICATION AND DATA SHARING POLICY 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
40 
 
This study will be conducted in accordance with the following publication and data sharing policies and 
regulations: 
 
National Institutes of Health (NIH) Public Access Policy, which ensures that the public has access to the 
published results of NIH funded research. It requires scientists to submit final peer-reviewed journal 
manuscripts that arise from NIH funds to the digital archive PubMed Central upon acceptance for 
publication. 
 
This study will comply with the NIH Data Sharing Policy and Policy on the Dissemination of NIH-Funded 
Clinical Trial Information and the Clinical Trials Registration and Results Information Submission rule. As 
such, this trial will be registered at ClinicalTrials.gov, and results information from this trial will be 
submitted to ClinicalTrials.gov. In addition, every attempt will be made to publish results in peer-
reviewed journals. Data from this study may be requested from other researchers for 5 years after the 
completion of the primary endpoint by contacting Dr. Teryl Nuckols (PI).  
 
In addition, this study will comply with the NIH Genomic Data Sharing Policy, which applies to all NIH-
funded research that generates large-scale human or non-human genomic data, as well as the use of these 
data for subsequent research. Large-scale data include genome-wide association studies (GWAS), single 
nucleotide polymorphisms (SNP) arrays, and genome sequence, transcriptomic, epigenomic, and gene 
expression data. 
 
10.1.12 
CONFLICT OF INTEREST POLICY 
 
The independence of this study from any actual or perceived influence, such as by the pharmaceutical 
industry, is critical.  Therefore, any actual conflict of interest of persons who have a role in the design, 
conduct, analysis, publication, or any aspect of this trial will be disclosed and managed. Furthermore, 
persons who have a perceived conflict of interest will be required to have such conflicts managed in a 
way that is appropriate to their participation in the design and conduct of this trial.  The study 
leadership in conjunction with the Agency for Healthcare Research and Quality have established policies 
and procedures for all study group members to disclose all conflicts of interest and will establish a 
mechanism for the management of all reported dualities of interest. 
 
10.2 ADDITIONAL CONSIDERATIONS 
Not applicable 
 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
41 
10.3 ABBREVIATIONS 
 
The list below includes abbreviations utilized in this template.  However, this list should be customized for 
each protocol (i.e., abbreviations not used should be removed and new abbreviations used should be 
added to this list). 
 
AE 
Adverse Event 
ANCOVA 
Analysis of Covariance 
CFR 
Code of Federal Regulations 
CLIA 
Clinical Laboratory Improvement Amendments 
CMP 
Clinical Monitoring Plan 
COC 
Certificate of Confidentiality 
CONSORT 
Consolidated Standards of Reporting Trials 
CRF 
Case Report Form 
DCC 
Data Coordinating Center 
DHHS 
Department of Health and Human Services 
DSMB 
Data Safety Monitoring Board 
DRE 
Disease-Related Event 
EC 
Ethics Committee 
eCRF 
Electronic Case Report Forms 
FDA 
Food and Drug Administration 
FDAAA 
Food and Drug Administration Amendments Act of 2007 
FFR 
Federal Financial Report 
GCP 
Good Clinical Practice 
GLP 
Good Laboratory Practices 
GMP 
Good Manufacturing Practices 
GWAS 
Genome-Wide Association Studies 
HIPAA 
Health Insurance Portability and Accountability Act  
IB 
Investigator’s Brochure 
ICH 
International Conference on Harmonisation  
ICMJE 
International Committee of Medical Journal Editors 
IDE 
Investigational Device Exemption 
IND 
Investigational New Drug Application 
IRB 
Institutional Review Board 
ISM 
Independent Safety Monitor 
ISO 
International Organization for Standardization 
ITT 
Intention-To-Treat 
LSMEANS 
Least-squares Means 
MedDRA 
Medical Dictionary for Regulatory Activities 
MOP 
Manual of Procedures 
MSDS 
Material Safety Data Sheet 
NCT 
National Clinical Trial 
NIH  
National Institutes of Health 
NIH IC 
NIH Institute or Center 
OHRP 
Office for Human Research Protections 
PI 
Principal Investigator 
QA 
Quality Assurance 
QC 
Quality Control 
SAE 
Serious Adverse Event 
SAP 
Statistical Analysis Plan 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
42 
SMC 
Safety Monitoring Committee 
SOA 
Schedule of Activities 
SOC 
System Organ Class 
SOP 
Standard Operating Procedure 
UP 
Unanticipated Problem 
US 
United States 
 
 
 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
43 
10.4 PROTOCOL AMENDMENT HISTORY 
 
The table below is intended to capture changes of IRB-approved versions of the protocol, including a 
description of the change and rationale. A Summary of Changes table for the current amendment is 
located in the Protocol Title Page.  
 
Version 
Date 
Description of Change  
Brief Rationale 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
44 
 
 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
45 
11 REFERENCES  
 
1. 
Agency for Healthcare Research and Quality. Reporting Patient Safety Events. 2019; 
https://psnet.ahrq.gov/primers/primer/13/reporting-patient-safety-events. Accessed 04/04, 
2019. 
2. 
Woods D. Improving the reporting and analysis of incidents. Enhancing Patient Safety and 
Reducing Errors in Health Care, Chicago. 1999. 
3. 
Nuckols TK, Bell DS, Liu H, Paddock SM, Hilborne LH. Rates and types of events reported to 
established incident reporting systems in two US hospitals. Qual Saf Health Care. Jun 
2007;16(3):164-168. 
4. 
Nuckols TK, Bell DS, Paddock SM, Hilborne LH. Contributing factors identified by hospital 
incident report narratives. Qual Saf Health Care. Oct 2008;17(5):368-372. 
5. 
Nuckols TK, Paddock SM, Bower AG, et al. Costs of intravenous adverse drug events in academic 
and nonacademic intensive care units. Med Care. Jan 2008;46(1):17-24. 
6. 
Nuckols TK, Bower AG, Paddock SM, et al. Programmable infusion pumps in ICUs: an analysis of 
corresponding adverse drug events. J Gen Intern Med. Jan 2008;23 Suppl 1:41-45. 
7. 
Nuckols T. Incident reporting: More attention to the safety action feedback loop, please. Online 
verfügbar unter http://www. webmm. ahrq. gov/perspective. aspx. 2011. 
8. 
Nuckols TK, Bell DS, Paddock SM, Hilborne LH. Comparing process- and outcome-oriented 
approaches to voluntary incident reporting in two hospitals. Jt Comm J Qual Patient Saf. Mar 
2009;35(3):139-145. 
9. 
Cohen TN, Francis SE, Wiegmann DA, Shappell SA, Gewertz BL. Using HFACS-Healthcare to 
Identify Systemic Vulnerabilities During Surgery. Am J Med Qual. Nov/Dec 2018;33(6):614-622. 
10. 
Baines R, Langelaan M, De Bruijne M, Spreeuwenberg P, Wagner C. How effective are patient 
safety initiatives? A retrospective patient record review study of changes to patient safety over 
time. BMJ Quality and Safety. 2015. 
11. 
Centers for Disease Control and Prevention. 2017 National and State Healthcare-Associated 
Infections Progress Report. 2017; https://www.cdc.gov/hai/data/portal/progress-report.html. 
12. 
Landrigan CP, Parry GJ, Bones CB, et al. Temporal Trends in Rates of Patient Harm Resulting from 
Medical Care. N Engl J Med. 2010. 
13. 
D L. Adverse events in hospital: National incidence Medicare beneficiaries. Department of 
Health and Human Services-USA, Office of Inspector General. 2010. 
14. 
Sunshine JE, Meo N, Kassebaum NJ, Collison ML, Mokdad AH, Naghavi M. Association of Adverse 
Effects of Medical Treatment With Mortality in the United States: A Secondary Analysis of the 
Global Burden of Diseases, Injuries, and Risk Factors Study. JAMA network open. 
2019;2(1):e187041-e187041. 
15. 
Jehring J, Heinrich HW. Industrial Accident Prevention: A Scientific Approach. Industrial and 
Labor Relations Review. 2006. 
16. 
Gordon F RD. The cost to Britain of workplace accidents and work-related ill health in 1995/96. 
1999; http://www.hse.gov.uk/pUbns/priced/hsg101.pdf. Accessed 05/31, 2019. 
17. 
Johnson CW. A Handbook of Incident and Accident Reporting. Glasgow: Glasgow University 
Press; 2003. 
18. 
Vincent C. Clinical risk management: Enhancing patient safety. BMJ Publishing group; 2001. 
19. 
Mitchell I, Schuster A, Smith K, Pronovost P, Wu A. Patient safety incident reporting: A 
qualitative study of thoughts and perceptions of experts 15 years after 'To Err is Human'. BMJ 
Quality and Safety. 2016. 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
46 
20. 
Battles JB, Stevens DP. Adverse event reporting systems and safer healthcare. Quality and Safety 
in Health Care2009. 
21. 
Macrae C. The problem with incident reporting. BMJ Qual Saf. 2016;(25):71-75. 
22. 
Clancy CM. New patient safety organizations can help health providers learn from and reduce 
patient safety events. 2009. 
23. 
Evans SM, Berry JG, Smith BJ, et al. Attitudes and barriers to incident reporting: A collaborative 
hospital study. Quality and Safety in Health Care. 2006. 
24. 
Archer S, Hull L, Soukup T, et al. Development of a theoretical framework of factors affecting 
patient safety incident reporting: A theoretical review of the literature. 2017. 
25. 
Benn J, Koutantji M, Wallace L, et al. Feedback from incident reporting: information and action 
to improve patient safety. Qual Saf Health Care. Feb 2009;18(1):11-21. 
26. 
Cullen DJ, Bates DW, Small SD, Cooper JB, Nemeskal AR, Leape LL. The incident reporting system 
does not detect adverse drug events: a problem for quality improvement. The Joint Commission 
journal on quality improvement. 1995. 
27. 
Sari ABA, Sheldon TA, Cracknell A, Turnbull A. Sensitivity of routine system for reporting patient 
safety incidents in an NHS hospital: Retrospective patient case note review. 2007. 
28. 
Kapur N, Parand A, Soukup T, Reader T, Sevdalis N. Aviation and healthcare: a comparative 
review with implications for patient safety. JRSM Open. 2016. 
29. 
Erickson SM, Wolcott J, Corrigan JM, Aspden P. Patient safety: achieving a new standard for 
care. National Academies Press; 2003. 
30. 
Pronovost P, Needham D, Berenholtz S, et al. An intervention to decrease catheter-related 
bloodstream infections in the ICU. N Engl J Med. Dec 28 2006;355(26):2725-2732. 
31. 
The Joint Commission. CLABSI Toolkit - Chapter 3, CLABSI Prevention Strategies, Techniques, and 
Technologies. 2019; https://www.jointcommission.org/topics/clabsi_toolkit__chapter_3.aspx. 
Accessed 04/25, 2019. 
32. 
Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and 
mortality in a global population. N Engl J Med. Jan 29 2009;360(5):491-499. 
33. 
The Joint Commission. Universal Protocol. 2019; 
https://www.jointcommission.org/standards_information/up.aspx. Accessed 04/25, 2019. 
34. 
Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients. 
Results of the Harvard Medical Practice Study II. N Engl J Med. Feb 7 1991;324(6):377-384. 
35. 
Thomas EJ, Studdert DM, Burstin HR, et al. Incidence and types of adverse events and negligent 
care in Utah and Colorado. Med Care. Mar 2000;38(3):261-271. 
36. 
Agency for Healthcare Research and Quality. Medication Errors and Adverse Drug Events. 
Patient Safety Network 2019; https://psnet.ahrq.gov/primers/primer/23/Medication-Errors-
and-Adverse-Drug-Events. Accessed 5/18, 2019. 
37. 
Hauck K, Zhao X. How dangerous is a day in hospital? A model of adverse events and length of 
stay for medical inpatients. Med Care. Dec 2011;49(12):1068-1075. 
38. 
Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse drug events and potential adverse drug 
events. Implications for prevention. ADE Prevention Study Group. JAMA. Jul 5 1995;274(1):29-
34. 
39. 
Rogers E. Diffusion of Innovations, 5th Edition. New York: Free Press; 2003. 
40. 
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service 
organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581-629. 
41. 
Moore GC BI. Development of an Instrument to Measure the Perceptions of Adopting an 
Information Technology Innovation. Institute for Operations Research and the Management 
Sciences. 1991;2(3):173-239. 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
47 
42. 
CFIR Research Team-Center for Clinical Management Research. Consolidated Framework for 
Implementation Research (CFIR). 2019; https://cfirguide.org/. Accessed 11/5, 2018. 
43. 
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation 
of health services research findings into practice: a consolidated framework for advancing 
implementation science. Implement Sci. Aug 7 2009;4:50. 
44. 
Shappell SA WD. A Human Error Approach to Aviation Accident Analysis: The Human Factors 
Analysis and Classification System. Burlington, VT: Ashgate Press; 2003: https://dvikan.no/ntnu-
studentserver/reports/A%20Human%20Error%20Approach%20to%20Aviation%20Accident%20
Analysis.pdf. Accessed 1/24/2019. 
45. 
Reinach S, Viale A. Application of a human error framework to conduct train accident/incident 
investigations. Accid Anal Prev. Mar 2006;38(2):396-406. 
46. 
Li WC, Harris D. Pilot error and its relationship with higher organizational levels: HFACS analysis 
of 523 accidents. Aviat Space Environ Med. Oct 2006;77(10):1056-1061. 
47. 
Patterson JM, Shappell SA. Operator error and system deficiencies: analysis of 508 mining 
incidents and accidents from Queensland, Australia using HFACS. Accid Anal Prev. Jul 
2010;42(4):1379-1385. 
48. 
Diller T, Helmrich G, Dunning S, Cox S, Buchanan A, Shappell S. The Human Factors Analysis 
Classification System (HFACS) applied to health care. Am J Med Qual. May-Jun 2014;29(3):181-
190. 
49. 
Committee on Identifying and Preventing Medication Errors. Identifying and Preventing 
Medication Errors. In: Aspden P BJ, Cronenwett L, ed. Preventing Medication Errors: Quality 
Chasm Series. Washington, DC: The National Academies Press; 2006. 
50. 
Hartwig SC, Denger SD, Schneider PJ. Severity-indexed, incident report-based medication error-
reporting program. Am J Hosp Pharm. Dec 1991;48(12):2611-2616. 
51. 
National Coordinating Council for Medication Error Reporting and Prevention. Types of 
Medication Errors. 2001; https://www.nccmerp.org/types-medication-errors. Accessed 5/7, 
2019. 
52. 
Institute for Healthcare Improvement. Trigger tool for measuring adverse drug events. 2004; 
http://www.ihi.org/resources/Pages/Tools/TriggerToolforMeasuringAdverseDrugEvents.aspx. 
Accessed 05/06, 2019. 
53. 
Dubois RW, Brook RH. Preventable deaths: who, how often, and why? Ann Intern Med. Oct 1 
1988;109(7):582-589. 
54. 
Cohen TN, Wiegmann DA, Reeves ST, Boquet AJ, Shappell SA. Coding Human Factors 
Observations in Surgery. Am J Med Qual. Sep/Oct 2017;32(5):556-562. 
55. 
Blegen MA, Gearhart S, O'Brien R, Sehgal NL, Alldredge BK. AHRQ's hospital survey on patient 
safety culture: psychometric analyses. J Patient Saf. Sep 2009;5(3):139-144. 
56. 
Sorra JS, Dyer N. Multilevel psychometric properties of the AHRQ hospital survey on patient 
safety culture. BMC Health Serv Res. Jul 8 2010;10:199. 
57. 
DiCuccio MH. The Relationship Between Patient Safety Culture and Patient Outcomes: A 
Systematic Review. J Patient Saf. Sep 2015;11(3):135-142. 
58. 
Campione J, Famolaro T. Promising Practices for Improving Hospital Patient Safety Culture. Jt 
Comm J Qual Patient Saf. Jan 2018;44(1):23-32. 
59. 
Blegen MA, Vaughn TE, Goode CJ. Nurse experience and education: effect on quality of care. J 
Nurs Adm. Jan 2001;31(1):33-39. 
60. 
Campbell MJ, Walters SJ. How to design, analyse and report cluster randomised trials in 
medicine and health related research. John Wiley & Sons; 2014. 

Comparing SAFE Loop with an Established Incident Reporting System 
Version 2 (20190604) 
CS IRB Protocol 1 | 12.8.2020 
01 September 2020 
Comparing SAFE Loop with an Established Incident Reporting System v.3 
48 
61. 
Hug BL, Witkowski DJ, Sox CM, et al. Adverse drug event rates in six community hospitals and 
the potential impact of computerized physician order entry for prevention. J Gen Intern Med. 
Jan 2010;25(1):31-38. 
62. 
Parry G, Cline A, Goldmann D. Deciphering harm measurement. Jama. May 23 
2012;307(20):2155-2156. 
63. 
Classen DC LR, Provost L, Griffin FA, Resar R. Development and evaluation of the Institute for 
Healthcare Improvement Global Trigger Tool. Journal of Patient Safety. 2008;4(3):169-177. 
64. 
Adler L DC, McKeever M, Purunton R, Guilloteau F, Moorhead D, Resar R. . Global Trigger Tool: 
Implementation basics. Journal of Patient Safety. 2008;4(4):245-249. 
65. 
Miles MB, Huberman AM, Saldaña J. Qualitative data analysis: A methods sourcebook. 3rd. 
Thousand Oaks, CA: Sage; 2014. 
66. 
Glaser BG. The constant comparative method of qualitative analysis. Social problems. 
1965;12(4):436-445.