Novel Application of Simulation for Providers to Overcome Decisional Gaps in High-risk Prescribing
Phase: NA
Conditions: Medication Administered in Error, Antipsychotics and Neuroleptics Toxicity
Sponsor: Brigham and Women's Hospital

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
1 
 
 
 
CLINICAL TRIAL PROTOCOL 
 
Novel application of simulation for providers to overcome decisional gaps in high-risk 
prescribing 
 
National Clinical Trial (NCT) Identified Number: NCT04668248 
Principal Investigators: 
Julie C. Lauffenburger, PharmD, PhD 
Niteesh K. Choudhry, MD, PhD 
Center for Healthcare Delivery Sciences and Division of Pharmacoepidemiology and 
Pharmacoeconomics, Brigham and Women’s Hospital and Harvard Medical School 
 
Funded by the National Institute on Aging 
Grant Number: P30AG064199 
Version Number: 2 
February 9, 2021 
 
 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
2 
 
TABLE OF CONTENTS 
SUMMARY OF CHANGES FROM PREVIOUS VERSION ....................... ERROR! BOOKMARK NOT DEFINED. 
1. BACKGROUND AND RATIONALE ........................................................................................................4 
2. STUDY AIMS ...........................................................................................................................................5 
3. STUDY DESIGN ......................................................................................................................................6 
3.1 STUDY SITE ................................................................................................................................................... 6 
3.2 OVERALL DESIGN ........................................................................................................................................... 6 
3.3 STUDY SCHEMA ............................................................................................................................................. 9 
3.4 SCIENTIFIC RATIONALE FOR STUDY DESIGN ................................................................................................. 10 
3.5 JUSTIFICATION FOR INTERVENTION ............................................................................................................. 10 
3.6 END-OF-STUDY DEFINITION ......................................................................................................................... 10 
3.7 DATA SOURCES ........................................................................................................................................... 10 
3.8 SCHEDULE OF ACTIVITIES ............................................................................................................................ 11 
4. STUDY POPULATION .......................................................................................................................... 12 
4.1 INCLUSION CRITERIA ................................................................................................................................... 12 
4.2 EXCLUSION CRITERIA .................................................................................................................................. 13 
4.3 RECRUITMENT AND RETENTION ................................................................................................................... 13 
4.3.1 Inclusivity of study subjects ............................................................................................................... 14 
5. STUDY INTERVENTIONS ..................................................................................................................... 14 
5.1 THERAPEUTIC AREAS ................................................................................................................................... 14 
5.2 STUDY INTERVENTIONS ............................................................................................................................... 15 
5.3 MEASURES TO MINIMIZE BIAS: RANDOMIZATION AND BLINDING .................................................................... 16 
6. STUDY ASSESSMENTS AND PROCEDURES .................................................................................... 16 
6.1 
BASELINE DATA ..................................................................................................................................... 16 
6.2  
OUTCOMES ........................................................................................................................................... 16 
6.3 
ADVERSE EVENTS AND UNANTICIPATED PROBLEMS .............................................................................. 17 
7. STATISTICAL CONSIDERATIONS ...................................................................................................... 19 
7.1  
STATISTICAL HYPOTHESES ................................................................................................................... 19 
7.2  
SAMPLE SIZE DETERMINATION ............................................................................................................... 20 
7.3  
STATISTICAL ANALYSES ......................................................................................................................... 20 
7.3.1 
ANALYSIS OF THE PRIMARY ENDPOINT ............................................................................................... 20 
7.3.2 
ANALYSIS OF SECONDARY ENDPOINTS ............................................................................................... 20 
7.3.3 
BASELINE DESCRIPTIVE ANALYSES .................................................................................................... 21 
7.3.4 
SUBGROUP ANALYSES ....................................................................................................................... 21 
8. ETHICAL AND REGULATORY REQUIREMENTS ............................................................................... 21 
8.1 
ETHICAL CONDUCT ................................................................................................................................ 21 
8.2 
INFORMED CONSENT ............................................................................................................................. 22 
8.3  
CONFIDENTIALITY AND PRIVACY ............................................................................................................ 24 
8.4 
SAFETY OVERSIGHT .............................................................................................................................. 25 
8.5 BENEFIT RISK ASSESSMENT ......................................................................................................................... 26 
8.5.1 KNOWN POTENTIAL RISKS ........................................................................................................................ 26 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
3 
 
8.5.2 KNOWN POTENTIAL BENEFITS ................................................................................................................... 26 
8.5.3 ASSESSMENT OF POTENTIAL RISKS AND BENEFITS ................................................................................... 26 
9. LIST OF REFERENCES ........................................................................................................................ 27 
 
 
 
 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
4 
 
1. Background and Rationale 
The overuse of medications with psychoactive properties such as antipsychotics, 
benzodiazepines, or sedative hypnotic “Z-drugs”, is extremely common in inpatient and acute 
care settings to manage delirium and agitation, despite the considerable risks associated with 
their use and guidelines that recommend their avoidance in older adults.1-4 In addition to their 
short-term risks, many patients are discharged on these potentially inappropriate medications 
which leads to longer-term adverse consequences such as falls and cognitive decline.5,6 While 
there are many contributors to the prescribing of these high-risk medications in the inpatient 
setting, provider-level barriers are thought to be central. These include clinical inertia (i.e., 
habits/culture of prescribing), perceived pressure from front-line nursing staff or patients and/or 
their caregivers, a lack of familiarity with a particular patient and fatigue.7-9  
 
Unfortunately, efforts to address these barriers have been largely unsuccessful.10-13 We 
hypothesize that this reflects inadequate addressing of the specific needs of complex and 
acutely-ill inpatients in stressful clinical situations by providers.  
 
In stressful situations, people rely on automatic, quick, or even emotional thinking, sometimes 
referred to as “System 1” thinking and may make different decisions than they would if they 
were instead using controlled, slow or rational or “System 2” thought.14-17 One example of a 
behavioral principle describing these differences is the “hot-cold” empathy gap which is based 
on the observation that when people are in rational or “cold” states, they incorrectly predict 
what their behavior will be during “hot states”.18 Other related behavioral principles include 
those that describe deliberative versus impulsive thinking and have found similar gaps in 
behaviors when people are in different states.19-21 
 
In the case of health care, the “empathy gap” has been thought to lead physicians to 
underestimate their actual prescribing of antibiotics or opioids when experiencing pressure to 
do so.17 For the prescribing of potentially-inappropriate medications to acutely-ill inpatients, 
most interventions have focused on educating and training providers when they can access 
System 2 thinking mechanisms, and may not have adequately prepared them for making 
complex and urgent therapeutic decisions under System 1 conditions.14,22 
 
Efforts to address decisional gaps between System 1 and System 2 thinking in other fields have 
involved role-playing or games to simulate System 2 thinking states, such as having participants 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
5 
 
evaluate their cravings for tobacco during hot state and cold state sessions.14,15,23,24 In clinical 
medicine, simulation has increasingly been used to help health care professionals, alone and in 
teams, practice how they would handle stressful situations such as cardiac arrest or emergent 
trauma situations in emergency rooms.25-31 By extension, these approaches could help address 
decisional gaps for prescribing high-risk medications for older adults, thereby improving patient 
safety and welfare.  
 
2. Study Aims 
We propose a 2-phase project consisting of provider qualitative interviews and a 2-arm pilot 
randomized controlled pragmatic trial in which physicians in training (interns) caring for elderly 
inpatients are assigned to receive simulation training or an educational control. The main goal 
of this project is to improve patient safety.  
 
The objectives and endpoints for the trial are summarized below. 
OBJECTIVES 
ENDPOINTS 
JUSTIFICATION FOR 
ENDPOINTS 
Primary 
To determine whether 
simulation-based training 
reduces the prescribing 
of high-risk medications 
to hospitalized older 
adults. 
Number of high-risk medications doses 
prescribed per day to eligible patients 
assigned to intervention providers 
compared with control providers. 
This outcome is measurable using 
electronic health record (EHR) 
data and will provide evidence of 
provider behavior change during 
the follow-up period. 
Secondary 
To evaluate whether 
simulation-based training 
reduces the continued 
prescription for high-risk 
medication to hospitalized 
older adults at discharge. 
Discharge medication order written for one 
of the eligible high-risk medications to 
eligible patients assigned to intervention 
providers versus control providers 
This outcome is measurable using 
EHR data and will provide 
evidence of whether the 
intervention results in fewer high-
risk medications for patients at 
discharge. 
To determine whether 
there are impacts of 
simulation-based training 
on prescribing of other 
related medications. 
Number of medication doses prescribed 
per day to eligible patients assigned to 
intervention providers versus control 
providers for opioids, trazodone, and 
melatonin 
This outcome is measurable using 
EHR data and will provide 
evidence of whether there are 
spillover effects on other related 
medications.  

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
6 
 
OBJECTIVES 
ENDPOINTS 
JUSTIFICATION FOR 
ENDPOINTS 
To evaluate adoption and 
implementation outcomes 
for the intervention 
 
Implementation outcomes informed by the 
Reach Effectiveness Adoption 
Implementation Maintenance (RE-AIM) 
framework: 
• 
Characteristics of consenting and non-
consenting providers 
• 
Completion of the simulation training 
(intervention) or online educational 
training (control) 
• 
Feedback from and issues reported by 
providers about the intervention 
• 
Self-reported satisfaction with the 
intervention 
These outcomes capture the extent 
to which the intervention could be 
replicated on a larger scale. This 
approach will provide valuable 
information regardless of the 
results of the trial, as evaluating 
implementation will enable us to 
explain why the intervention did not 
work, should that turn out to be the 
case. These outcomes are also 
easily measurable using baseline 
and endline (follow-up) 
questionnaires.  
 
 
3. Study Design 
3.1 Study site 
Study participants will be providers recruited from the general medicine inpatient service at 
Brigham and Women’s Hospital (BWH) and Brigham and Women’s Faulkner Hospital (BWFH). 
BWH is part of Mass General Brigham (MGB), formerly known as Partners Healthcare, a large 
hospital system in Boston, MA, and are both staffed by the same general medicine service at 
the main BWH campus.   
 
3.2 Overall design 
We propose a 2-phase project consisting of provider qualitative interviews and a 2-arm pilot 
randomized controlled pragmatic trial in which physicians in training (interns) caring for elderly 
inpatients are assigned to receive simulation training or an educational control.  
 
The first phase, qualitative interviews, are a separate aim from the pragmatic trial and 
have already been reviewed and approved by the MGB IRB and is governed under a 
separate protocol (MGB IRB: 2020P002467). This aim consists of brief virtual qualitative 
interviews with approximately 25 inpatient providers and allied team members (e.g., nurses or 
pharmacists) to elicit perspectives on prescribing of high-risk medications in the in-patients 
setting as well as simulation-based training. These interviews are being conducted virtually 
with providers at other MGB sites and former interns who are no longer part of the general 
medicine service to avoid contamination with the trial using an MGB-approved video platform. 
This aim is being used to inform the design of the simulation-based training intervention used 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
7 
 
in Aim 2 (pragmatic trial). Providers are identified through referrals by clinicians and recent 
prescribing history and contacted through email invitation. They are providing verbal consent 
for their participation and receiving renumeration for their participation. The remainder of this 
protocol is about the 2-arm pilot randomized trial.  
 
The 2-arm pilot randomized controlled trial will determine whether a newly-designed 
simulation-based training program for providers based on underlying principles of System 1 
and System 2 thinking reduces prescribing of high-risk medications for hospitalized older adults 
versus control, with the ultimate goal of improving patient safety. We will also measure 
subsequent prescribing for patients cared for by other providers and other adoption and 
implementation outcomes to explore the extent to which the intervention could be used at 
scale. 
 
BWH and BWFH uses 2-week service blocks for its inpatient general medicine evening coverage, 
and the service overall consists of 6 day and 8 evening (twilight) teams covering geographically 
distinct wards at 2 hospital sites with daily censuses of 15-20 patients. These teams typically 
consist of an attending, a resident, an intern (i.e., 1st year of residency), and medical students. 
There is no cross-coverage across teams. We will focus on interns from the “twilight” (i.e., evening) 
teams as they cover the shifts during which the high-risk medications of interest (specifically anti-
psychotics, benzodiazepines, and sedative hypnotic “Z-drugs”) are typically first initiated and are 
also the first point of contact for nurses, pharmacists, and other physicians. Given their stage of 
training and the nature of their workflow, they are also most prone to make decisions using System 
1 thinking. 
 
At the beginning of each rotation, we will identify the interns assigned to the twilight teams (i.e., the 
primary prescriber and responding clinician on the general medicine service team). We will invite 
these interns to participate in the study via email invitation. Interns who consent will be randomized 
in a 1:1 ratio with a random number generator to one of 2 arms: (a) Arm 1: simulation intervention 
and (b) Arm 2: control (online education) intervention. The unit of randomization will therefore be 
the intern. Randomizing at the intern level will reduce the possibility of contamination across 
patients. Providers will receive $25 for completing the initial provider survey, $75 for completing the 
intervention or control assignment (e.g., simulation or online education training), and $50 for 
completing the endline (follow-up) questionnaire. Other ward staff will be blinded to arm 
assignment to the extent possible. Data analysts will be blinded to arm assignment. This is a highly 
feasible approach used by investigators conducting provider-facing intervention trials at BWH.  

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
8 
 
 
Providers assigned to the intervention arm will participate in a short simulation training at the 
beginning of their 2-week block (planned for their second day). The training will take place at the 
Neil and Elise Wallace STRATUS Center for Medical Simulation at Brigham and Women’s Hospital  
and will follow all of their recommended and hospital-recommended practices on social distancing, 
including the learning limits. The training will also be led by Matthew DiFrancesco, hospitalist, 
Instructor in Medicine, and Assistant Director for Medicine at STRATUS. Based on discussion with 
Dr. DiFrancesco, the study team, and intern and resident interviews completed in the first phase, it 
is clear that in-person simulation will have be more effective at creating the stressful situations that 
could lead to inappropriate prescribing. Moreover, this approach has been used in numerous prior, 
unrelated trainings conducted by STRATUS and by the general medicine service.  
 
Providers assigned to the control arm will receive online educational training about other poorly-
prescribed medications, including albumin, transfusion, and blood product repletion guidelines. 
This information will be in the form of electronically-delivered links to information already housed 
and available at BWH, including reviewing BWH guidelines and literature about transfusion 
reactions. The interns will be asked to answer several clinical questions about optimal prescribing. 
Albumin and related products are also high-risk medications often overprescribed on twilight shifts. 
Receiving education about another high-risk medication will reduce contamination and allow the 
ability to measure outcomes to other high-risk medications.  
 
Follow-up time for trial evaluation will begin on the day in which the interventions are delivered 
(planned for Day 2 of their service) and will continue until the end of their coverage block on the 
twilight team. The interventions in both arms will be delivered on the same day, so there will be 
equal follow-up time. Upon completion of the trial, we will evaluate outcomes, including prescribing 
and implementation outcomes, using structured EHR data on the patient-level and a follow-up 
survey of the interns at 2 weeks after randomization.  
 
We believe that this is a minimal-risk study for several reasons. First, the goal of the study is to 
improve patient safety, by reducing the prescribing of high-risk medications in older adults in a way 
that is concordant with well-established clinical guidelines. Second, the intervention itself is a 
quality-improvement intervention, leveraging resources already available at BWH. Third, all 
prescribing decisions will be made by the patients’ covering providers. Fourth, providers will 
provide consent for their participation and will not be required to participate. Finally, all patient-level 
outcomes will be evaluated only through structured EHR data, collected during routine care.  

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
9 
 
 
3.3 Study Schema 
 
Screening 
 
 
 
 
 
 
Enrollment 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Follow-up 
 
 
 
Screen potential participants by inclusion and exclusion criteria 
Total N: 96 interns (1440 patients for analysis) 
Conduct consent process.  
Perform baseline assessments. 
Final Assessments 
36 providers 
 
(432 patients for analysis) 
Arm 1 
N = 18  
Randomize 
Arm 2 
N = 18  

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
10 
 
3.4 Scientific rationale for study design 
The use of a randomized trial to evaluate the primary outcome and secondary clinical outcome is 
scientifically justified, as this design will be able to provide evidence of causality in the 
effectiveness of the tools on reducing prescribing. An observational study design, by contrast, 
would not provide the same degree of scientific rigor. 
 
3.5 Justification for intervention 
Simulation has increasingly been used to help health care professionals, alone and in teams, 
practice how they would handle stressful situations such as cardiac arrest or emergent trauma 
situations in emergency rooms. By extension, leveraging behavioral science principles within 
simulation to address decisional gaps in System 1 and System 2 thinking holds great promise 
for reducing for high-risk medications for older adults. Of note, the intervention in this study will 
be integrated in the intern schedule with minimal impact on workflow. We believe this is a minimal 
risk intervention overall, as providers could seek to ignore the simulation training and are 
ultimately responsible for making any prescribing decisions. Patients will not be specifically 
engaged as part of the trial, as any non-specific attention from the trial protocol in the control 
arm will undermine the ability to observe the effect of the intervention. 
 
3.6 End-of-study definition 
The trial will be completed 1 month after completing randomization. Providers and their eligible 
patients will be followed until the end of this follow-up date, or until censoring. 
 
3.7 Data sources 
Sources of research material, data that will be recorded, when data will be collected 
We will use electronic health record (EHR) data to retrospectively identify eligible patients. 
Data regarding patients' medical history, disease control, medication use, and health care 
utilization will be obtained from EHR data (Epic). In specific we will extract clinical and financial 
information from the electronic medical record system (EpicCare), the patient accounting 
system (Resolute), the patient web portal and the master patient index (Identity). The various 
tables within these databases are refreshed on a daily, weekly or monthly basis. Data 
regarding intern scheduling and team assignments will be gathered from Amion®, the residency 
scheduling system. We will also gather information from provider surveys, one at baseline and 
one at the end of follow-up. 
 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
11 
 
Linkages to subjects, access to subject identities 
Individually-identifiable data are maintained for patient care purposes within the MGB network 
and are needed to identify patients for evaluation and to monitor care recommendations by 
providers. Without this linkage, we could not fulfill the study’s objectives. To protect the 
confidentiality of these data, only the minimal necessary research staff will have access to 
personal identifiers. This will be necessary for linking data and contacting providers. After 
linkage is completed and study variables are created, all identifiable information will be deleted 
from the study database. All research staff are properly trained in research management and 
will be approved by the IRB. All personally identifiable health information will be kept under 
lock and key.  
 
3.8 Schedule of activities 
 
Screening (7 days 
prior to rotation) 
Randomization (2 
days prior to rotation) 
Day 2 of rotation 
2-week follow-up 
Review of general medical service schedule and outreach to eligible interns  
X 
 
 
 
Completion of consent 
 
X 
 
 
Completion of baseline questionnaire* 
 
X 
 
 
Randomization 
 
X 
 
 
Provision of interventions (simulation training and online education) 
 
 
X 
Adverse event reporting measurement 
 
 
X 
X 
Provision of endline (follow-up) questionnaire* 
 
 
 
X 
EHR and scheduling system data  evaluation* 
 
 
 
X 
 
*Baseline Questionnaire (Providers): Demographics (age, sex, race/ethnicity), 6-item State-Trait Anxiety Inventory32, 8-item 
Physicians’ Reaction to Uncertainty Stress from Uncertainty subscale 
Endline Questionnaire (Providers): 6-item State-Trait Anxiety Inventory32, 8-item Physicians’ Reaction to Uncertainty Stress 
from Uncertainty subscale, Satisfaction with the interventions 
EHR fields:  
Baseline: Provider: Prior prescribing of high-risk medications; Eligible patients: Demographics (age, sex, race/ethnicity, 
insurance status), prior in-hospital use of one of the high-risk medications, admitted on one of the high-risk medications, 
hospitalization within last 30 days, total no. of home medications; 
Follow-up: Prescribing of one of the high-risk medications, prescribing of albumin and other high-risk medications (e.g., opioids), 
discharge medication orders, administration of the one of the high-risk medications 
Scheduling system (Amion®) fields: 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
12 
 
Provider: Baseline; Prior inpatient service blocks (i.e., ICU service, ambulatory service), no. of months of internship, 
characteristics of the rest of the team (i.e., number of residents, seniority of attending, no. of other team members);  
Follow-up: Completion of simulation training, no. of days on twilight service 
 
4. Study Population 
The study will intervene upon physicians and their older adult patients admitted to the adult 
inpatient general internal medicine services at BWH and BWFH. We will focus on interns from the 
“twilight” (i.e., evening) teams as they cover the shifts during which the key high-risk medications of 
interest (specifically anti-psychotics, benzodiazepines, and sedative hypnotic “Z-drugs”) are 
typically first initiated and are also the first point of contact for nurses, pharmacists, and other 
physicians. Given their stage of training and nature of their workflow, they are also most prone to 
make decisions using System 1 thinking. We plan to exclude patients for the purposes of analysis 
who were on the relevant high-risk medication of interest prior to admission. For example, patients 
who were admitted on a benzodiazepine will not be included in the benzodiazepine/sedative 
hypnotic analysis but will be included in the antipsychotic analysis. Benzodiazepines and sedative 
hypnotics will be considered interchangeable for this analysis, due to the fact that they have 
overlapping indications. We will conduct stratified analyses by whether patients by whether they 
had exposed to one of the high-risk medications during their present admission prior to being 
under the care of the consenting intern. 
 
4.1 Inclusion Criteria 
The study will include provider subjects for intervention and analysis and patient-subjects for 
analysis. The study criteria are defined below.  
 
In order to be eligible to participate in this study, providers must meet all of the following criteria: 
1. Intern practicing on the general internal medicine service at Brigham and Women’s Hospital 
(including Faulkner) 
2. Assigned to the “twilight” team 
3. Consent to participate 
4. Not previously identified for this study 
 
Patients who meet the following criteria will be included in the analysis:  
1. Adults ≥65 years 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
13 
 
2. Admitted to the BWH/BWFH general medicine service under the care of one of the 
consenting interns (specifically, with the intern listed as the “Responding Clinician”) 
 
4.2 Exclusion Criteria 
 
Our study population will be restricted to adult subjects (≥65 years) who were not on the high-risk 
medication of interest before admission. The specific age range was selected because clinical 
guidelines for high-risk prescribing for the medications of interest are clearer for this age range, 
and because overprescribing is a well-documented issue in older patients. Patients who were 
previously on the medication of interest before their present admission will be excluded because of 
potential withdrawal concerns. To enhance generalizability, we will not apply any additional 
exclusion criteria. 
 
4.3 Recruitment and retention 
 
As described in Section 3.2, BWH uses 2-week service blocks for its inpatient general medicine 
evening (twilight) coverage, and at the beginning of each block, we will identify the interns 
assigned to twilight teams (i.e., the primary prescribers on the general medicine service team) one 
week prior to their service. We will focus on interns from the “twilight” (i.e., evening) teams as they 
cover the shifts during which the high-risk medications of interest (specifically anti-psychotics, 
benzodiazepines, and sedative hypnotic “Z-drugs”) are typically first initiated. In addition to being 
the least experienced physicians on the medical team, they are also the first and primary point of 
contact for nurses, pharmacists, and other physicians for these patients (i.e., the responding 
clinician). Their participation in twilight teams does not repeat once they have previously been 
assigned, so they cannot be re-randomized.  
 
Over the 6-month window, at least 96 interns are expected to be eligible during the proposed 6-
month recruitment window. Based on preliminary estimates and consultation with the general 
medicine service and chief residents, we expect that ≥40% of the interns will agree to participate 
and that therefore at least 36 interns will be included in the study. This rate is highly feasible based 
on feedback from the clinical leadership engaged in our study team.  
 
Considering the number of potential eligible patients, these interns typically have daily census of 
10-20 patients. Each patient is typically admitted for 3-5 days on each general medicine service, so 
interns typically care for approximately 40 unique patients during their time on the twilight team. 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
14 
 
Based on preliminary estimates, ≥60% of their patients are older adults, and ≥50% will be naïve to 
either benzodiazepines/sedative hypnotics or antipsychotics. With 36 participating interns, we 
therefore expect that at least 430 unique patients will be included in the trial analyses (mean: 12 
per intern).  
 
We will provide compensation for participation in this trial, which will help ensure retention.  
Providers will receive $25 for completing the initial provider survey, $75 for completion of the 
simulation intervention or online educational training, and $50 for completing the follow-up 
questionnaire. Outside of the brief intervention itself and short baseline and endline (follow-up) 
questionnaires delivered electronically, there will otherwise be no action required by the 
participating interns. We have also engaged the chief medical residents in this project, which along 
with providing access to the scheduling system, will help us track their schedules and help ensure 
retention. 
4.3.1 Inclusivity of study subjects  
This study will be conducted on the adult inpatient general internal medicine services at 
Brigham and Women’s Hospital (BWH), which serve a diverse range of patients. As such, we 
plan to include patients of all race/ethnicities in the analysis, regardless of their primary 
language. The specific age range (≥65 years) was selected because clinical guidelines for the 
prescribing of high-risk medications of interest are clearer for this age range, and because 
inappropriate prescribing of these medications is a well-documented issue in the older adult 
population. Based on the inclusion criteria, we expect that 59.2% of the trial patient participants 
will be female. In addition, we expect 84.6% will be white, 6.4% will be black or African 
American, 1.4% will be Hispanic/Latino, 4.2% will be biracial/other, and 3.1% will be 
Asian/Pacific Islander.  
 
 
5. Study Interventions 
5.1 Therapeutic areas 
 
The focus areas for simulation training will be high-risk medications for older adults, which will 
be primarily drawn from the outpatient Choosing Wisely recommendations in geriatric medicine 
but are also informed by the Beers Criteria and other major clinical guidelines.33,34 In specific, 
we plan to focus on the following therapeutic classes: (1) benzodiazepines; (2) sedative 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
15 
 
hypnotics (sleep medicines) and (3) antipsychotics. We are also using albumin and blood 
products as a control, given that they are also high-risk and overprescribed, typically on twilight 
or evening shifts as patients are admitted to the hospital or are actively decompensating.  
 
 
5.2 Study interventions 
 
BWH uses 2-week service blocks for its inpatient general medicine evening (twilight) coverage. Prior 
to the beginning of each block, we will identify the interns assigned to twilight teams (i.e., the primary 
prescribers on the general medicine service team). We will invite these interns to participate in the 
study.  
 
• 
Intervention: Providers in this arm will undergo simulation training at the onset of their 2-
week block. The simulation will consist of a one-time, short immersive simulation session at 
the STRATUS Center for Medical Education at BWH (which is located within the hospital 
building). We plan to conduct this session with up to 4 interns at the same time, in 
accordance with social distancing practices at BWH. This simulation session will consist of 
1 scenario with two parts of simulated patient experiences with expert facilitators in the 
simulated hospital rooms to help providers identify when they are in the hot state and their 
reactions, and work on improving communication skills, differential diagnoses, and 
alternative therapeutic options. These scenarios are intended to simulate both in-person 
and virtual interactions that are common in the inpatient setting. During these trainings, we 
will use behavioral principles like time pressure and increasing cognitive load to simulate a 
“hot state” environment. After the scenarios, the facilitator will perform a debriefing session 
for the interns, which will be used to complete the training. 
• 
Control: Providers assigned to the control arm will receive online educational training about 
other poorly-prescribed medications, including albumin, transfusion, and blood product 
repletion guidelines. These treatments are often overprescribed on twilight shifts. Receiving 
education about another high-risk medication will reduce contamination and allow the ability 
to measure outcomes to other high-risk medications.  
 
These interventions are described in further detail in the “Simulation Scenarios and Online 
Educational Training overview” document. 
 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
16 
 
5.3 Measures to minimize bias: randomization and blinding 
 
Interns who consent will be randomized in a 1:1 ratio to receive either intervention or control 
using a random number generator. We plan to conduct stratified randomization based on the 
service to which the intern is assigned to (GMS, ITU, or Faulkner).  
 
The providers will not be blinded to which arm to which they were assigned, as blinding in the 
context of an intervention that is intended to motivate action will be infeasible. Physicians 
randomized to control will not have any way of accessing the intervention. Of note, once on a 
twilight team, interns are unlikely to rotate back onto a service, so contamination across arms 
should be minimal. Either way, if this does happen, they will not be re-randomized in the trial. 
Other ward staff and analysts will be blinded to arm assignment to the extent possible.  
 
6. Study Assessments and Procedures 
 
6.1 
Baseline data 
 
We will collect baseline data on providers using a brief baseline questionnaire at the time of 
consent as well as extracted EHR data and data from the Amion scheduling system. This 
baseline data will be used to assess any potential imbalances in the characteristics of 
providers despite randomization. The baseline data will include relevant demographic 
characteristics, such as age, sex, race/ethnicity, and number of months in training. Given the 
strong potential for characteristics of the study team to influence outcomes, despite 
randomization, we will also measure characteristics about the interns’ team using the Amion 
scheduling system. Finally, we will also collect patient data from the EHR for eligible patients in 
follow-up (please see the footnote Section 3.8 for more detail).  
 
 
6.2  Outcomes 
The primary outcome will be measured on the patient-level as the number of high-risk medication 
doses prescribed per day to eligible patients beginning on the day the intervention arms are 
delivered to both arms until the end of follow-up. We will only measure doses ordered by the 
interns in the study. As described above, we will censor follow-up time on when patients transition 
from the intern’s service, when the intern transitions off the twilight service, discharge, or in-hospital 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
17 
 
death. The interventions in both arms will be delivered on the same day, so there will be no 
immortal time bias. Medications that are written pro re nata (PRN) will be treated the same as 
standing orders for the primary analysis. If patients are eligible to be measured for multiple 
medication classes (i.e., antipsychotics and benzodiazepines/sedative hypnotics), we will sum the 
medication doses across the classes. 
 
In secondary analyses, we will only measure doses that are standing orders (i.e., not PRN doses) 
and doses that were actually administered to patients. We chose to measure this outcome on the 
patient-day level, because this would enable the fairest comparison between the arms, given that 
patients have variable lengths of stay and, as a result, the lengths of time under the interns’ care. 
These data will be collected through structured EHR data from MGB paired with data collected 
from the scheduling system. 
As secondary outcomes, we will measure whether patients are ultimately discharged on one of 
these medications and prescribing by the interns of other related medications such as opioids, 
trazodone, or melatonin, to measure spillover effects. We will also measure prescribing and 
monitoring of albumin and blood products. As above, if patients are not discharged within the 
timeframe for evaluation (i.e., are still admitted or experience in-hospital death), we will not include 
them in this analysis. We will also evaluate implementation outcomes informed by the RE-AIM 
framework. These implementation outcomes are: 1) Characteristics of consenting and non-
consenting providers (measured by the Amion scheduling system), 2) Adoption: whether the 
simulation training was completed and whether the email with the online educational training was 
accessed/opened (measured by study staff), 3) Implementation: feedback and issues reported by 
the interns or study staff during the study (collected by study staff), and 4) Maintenance: reported 
satisfaction with the intervention and likelihood of incorporating insights into future practice 
(measured in the endline questionnaire).  
 
6.3 
Adverse events and unanticipated problems 
 
Oversight: 
Oversight of the pilot will be the responsibility of Drs. Lauffenburger and Choudhry, the 
Principal Investigators. 
 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
18 
 
The co-leads and study investigators will meet on a regular basis throughout the study period 
and will be in direct contact with clinical leadership involved in the project to obtain ongoing 
feedback. In addition, the protocol will be overseen by the Institutional Review Board (IRB). 
 
De-identified study data will be accessible at all times for the BWH PI and coinvestigators to 
review during the conduct of the trial and during the study analysis, if applicable. We will also 
ensure that all protocol deviations for the pilot study are reported to the NIH and the IRB 
according to the applicable regulatory requirements. Compliance of regulatory documents and 
study data accuracy and completeness will be maintained through an internal study team 
quality assurance process. 
 
Definition: 
Adverse Event (AE): Any untoward or unfavorable medical occurrence in a human study 
participant, including any abnormal sign (e.g. abnormal physical exam or laboratory finding), 
symptom, or disease, temporally associated with the participants’ involvement in the research, 
whether or not considered related to participation in the research. 
 
- 
Adverse Events will be classified using the following rating scales: 
o Severity: Mild, Moderate or Severe 
 
Mild: Awareness of signs or symptoms but are easily tolerated 
 
Moderate: Events introduce a low level of inconvenience or concern but may 
interfere with daily activities but are usually improved by simple therapeutic 
measures. 
 
Severe: Events interrupt the participants’ normal daily activities and 
generally require systemic drug therapy 
o Expectedness: Unexpected or Expected 
 
Unexpected: nature or severity of the event is not consistent with the 
condition under study 
 
Expected: event is known to be associated with the intervention or condition 
under study. 
 
Serious Adverse Event (SAE): Any adverse event that: 
• Results in death 
• Is life threatening, or places the participant at immediate risk of death from the 
event as it occurred 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
19 
 
• Requires or prolongs hospitalization 
• Causes persistent or significant disability or incapacity 
• Results in congenital anomalies or birth defects 
• Is another condition which investigators judge to represent significant hazards 
 
Determination: 
Given the minimal risk nature of the study where the overall goal is to improve patient safety, 
the intervention is a quality improvement intervention, all prescribing decisions are made by 
providers who will provide consent, we do not anticipate any SAEs or AEs. 
 
Reporting: 
No SAEs or AEs are expected as part of this study, which aims to increase patient safety through 
the avoidance of high-risk medications.  The study team will not be providing any direct care to 
patients and all treatment decisions will ultimately be made by the patients’ medical teams at 
Brigham and Women’s Hospital. While we will not be actively monitoring the occurrence of adverse 
events, which if done would require direct patient contact, detailed patient-level chart-reviews, and 
clinical assessments conducted by the study team, we anticipate that the study team will be 
informed of any AEs or SAEs that do occur through hospital safety reports.  
 
If we become aware of any AEs or SAEs throughout the course of the study, we will collect this 
information. Any reports of deaths will be submitted to the NIA Program Officer and to the Safety 
Officer (SO) within 24 hours. Any unexpected SAEs will be reported to the NIA PO, SO and the 
IRB within 48 hours of the study’s knowledge of the SAE. All other reported SAEs and AEs 
received by the study team will be reported to the NIA Program Officer and to the SO quarterly, 
unless otherwise requested by the Safety Officer or Roybal Center Program Data Safety 
Monitoring Board (DSMB). 
 
7. Statistical Considerations 
 
7.1  Statistical Hypotheses 
Our null hypothesis will be that number of high-risk medication doses prescribed per day to 
eligible patients assigned to intervention providers will be no different than those in the control 
arm.  
 
 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
20 
 
7.2  Sample size determination 
As described in Section 4.3, we expect at least 36 interns to participate in the trial (18 per arm), 
and that at least 430 patients will be eligible for inclusion in the analysis, with approximately 3 
eligible days per patient (i.e., 1290 patient-days of observations). With these estimates, we 
would have >80% power to detect a mean difference of 0.5 high-risk medication doses per 
patient-day in the intervention arm compared with the control arm, assuming a standard 
deviation of 1.1, power=0.8, alpha of 0.05 (two-sided), and patient correlation of 0.2, a 
conservative estimate.13,35 We also assumed an average cluster size of 12 patients (and 36 
patient-days) per provider based on preliminary estimates. As these data are intended to 
provide pilot data for larger evaluations, any positive signal from this pilot trial will help inform 
next steps. 
 
 
7.3  Statistical analyses 
 
7.3.1 Analysis of the primary endpoint 
 
The unit of analysis is at the patient-level. We will evaluate the primary outcome using 
generalized estimating equations to adjust for physician-level clustering and multiple patient-
day observations per physician using a log-link function, Poisson-distributed errors, and fixed 
effects for the treatment group and month of the year to account for seasonality. This approach 
will account for correlations between repeated measurements. Because this is a randomized 
trial, our primary analyses are planned as unadjusted; however, if there are strong predictors of 
the outcomes not balanced by stratified randomization, we will adjust for these in the primary 
analyses. We will conduct analyses using intention-to-treat principles. Given the nature of the 
data and how the outcomes are measured in EHR data, there will not be missing data for the 
primary outcome. 
 
 
7.3.2 Analysis of secondary endpoints 
 
We will use a similar approach for the secondary outcomes. For discharge medication orders 
(secondary outcome), we will also use generalized estimating equations that adjust for physician-
level and patient-level clustering using a logit-link function, binary-distributed errors, and fixed 
effects for the treatment group and month of the year. For the outcomes measuring spillover 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
21 
 
effects, the approach will be the same, except using a log-link function and Poisson-distributed 
errors. Given the nature of the data and how these outcomes are being measured primarily in EHR 
data, there should not be missing values. Because this is a randomized trial, our primary analyses 
are planned as unadjusted; however, if there are strong predictors of the outcomes not balanced 
by randomization, we will adjust for these in the primary analyses.  
 
The implementation outcomes will be measured at the provider-level. The characteristics of 
consenting and non-consenting physicians will be measured descriptively, comparing the values 
using t-tests and chi-square tests or non-parametric analogs, as appropriate. Other implementation 
outcomes will be measured descriptively.   
 
 
7.3.3 Baseline descriptive analyses 
 
We will report the means and frequencies of pre-randomization variables separately for 
intervention and control subjects. Comparisons of these values will be performed using t-tests 
and chi square tests and their non-parametric analogs, as appropriate. The outcomes will be 
evaluated using intention-to-treat principles among all randomized patients.  
 
 
7.3.4 Subgroup analyses 
 
In subgroup analyses, we will explore whether there were any modifiers of the effects of the 
simulation-based intervention. For example, we will explore if the intervention effectiveness 
differed by month of the year, whether there were team-level differences, or if there were 
observable differences in patients who were less likely to receive high-risk medications, such 
as gender or race/ethnicity. 
 
 
8. Ethical and regulatory requirements 
8.1 
Ethical conduct 
General oversight of the project by the principal investigators (Drs. Lauffenburger and 
Choudhry) will occur throughout the study period, including regular contact with clinical 
leadership to obtain ongoing feedback. In addition, this protocol will be overseen by the MGB 
Institutional Review Board (IRB). Study data will be accessible at all times for the principal 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
22 
 
investigators and co-investigators to review, if applicable. The principal investigators will review 
study conduct (e.g., protocol deviations) on a monthly basis. The principal investigators will 
also ensure that all protocol deviations for the trials are reported to the NIH and the IRB 
according to the applicable regulatory requirements. 
 
We believe that the risks to participation for both sets of subjects (i.e. providers and patients) 
are no more than minimal for several reasons. We believe there is no more than minimal risk 
involved to the provider subjects, as the providers will simply undergo training to alter their 
behaviors towards guideline recommended care, as opposed to being forced to do so. The 
intervention is delivered as a simulation training that incorporates information already available 
to providers 
 
In terms of patient-subjects, the intervention aims to emphasize guideline-recommended 
information for providers to assist in their decision-making when caring for older patients who 
have been admitted to the general medicine service. All treatment decisions will ultimately be 
made by licensed health care providers.  
 
Patient data to inform the study and conduct evaluations will be drawn from EHR data 
maintained by the health systems. We will also safeguard any identifiable information in 
accordance with IRB practices, limit access to any information in accordance with IRB 
practices, limit access to the information to study investigators actively involved in the research 
who have all undergone human subjects research training, and destroy any data upon 
completion of the research. Finally, as is our routine practice, throughout this work, great care 
will be taken to ensure the confidentiality of all data and to protect the privacy of participants 
through translation of all potentially traceable identifiers into untraceable coded subject 
numbers. 
 
 
8.2 
Informed consent 
Interns will be eligible if they are scheduled to work for one of the general medicine service 
wards during an evening block on one of the “twilight” teams.  
 
We request a waiver of documentation of informed consent and will obtain e-consent without a 
signature requirement. We believe this waiver is justified for this study for several reasons. 
First, the subjects being intervened upon are physicians at BWH/BWFH and the study team will 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
23 
 
not have any contact with patients. The proposed intervention being tested is very similar to 
existing quality improvement initiatives already offered at BWH and intends to evaluate 
interventions to encourage physicians to follow care processes based on widely-accepted 
clinical practice guidelines. The interventions involve providing them with tools to improve 
prescribing in accordance with these guidelines and they can ignore this guidance. 
Additionally, a formal written informed consent process would be infeasible for this study. 
Interns are busy and the components of this intervention are very familiar to them; therefore, 
the standard consent process would be burdensome and likely be a barrier to study enrollment. 
If we have difficulty recruiting sufficient interns for the study, and therefore unable to reach our 
sample size targets, our study will be insufficiently powered.  Further, if we have highly self-
selected sample, this can bias our results and make them less generalizable.  
 
Instead, after reviewing the information sheet that we will electronically provide to potential 
participants and prior to their agreeing to participate, subjects will be invited to talk with the 
study team by phone or Zoom to address questions or concerns, and our contact information 
will be easily accessible for potential participants. For these reasons, we feel that a waiver of 
documentation of informed consent with the option for video or phone conversation for any 
questions or concerns would be the best consent process for this study. We also have approval 
from clinical leadership and chief residents for this approach. Finally, the surveys as part of this 
study take only 10-15 minutes, are all validated and widely used, and PCPs will be 
compensated for their time. 
 
We will obtain e-consent with a waiver of written consent (signature) and waiver of 
documentation of informed consent for participation in the trial through e-consent 
documentation in REDCap, a secure HIPAA-compliant web-based platform hosted on our Mass 
General Brigham servers. Intern physician subjects will be invited to participate via email.  The 
information sheet below will be attached to the email, and contains information about the study, 
their participation, and the names and contact information for study staff who can be reached for a 
phone or video conversation if they have any questions or concerns.  We have received approval 
from clinical leadership for this approach. Interested intern physicians will be asked to click on a 
link in the email to open a secure RedCap survey.  There, interns will again view an electronic copy 
of this information sheet.  If the physician agrees to participate, he or she will click a button 
indicating “I agree”.  E-consent documentation (without signature or documentation of written 
consent) will be captured in REDCap, which is a secure, HIPAA-compliant web-based platform 
hosted on the MGB servers. We will provide sufficient information to the prospective provider-

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
24 
 
subjects about the nature of the study and their rights within the fact sheet, as well as provide 
contact information for study staff and IRB for any additional questions.  
 
BWH study staff and data analysts will track any physician turnover in the study using the Amion 
electronic residency scheduling system and the paging directory. While minimal turnover before 
the end of the block is expected, we expect any turnover to bias the trial results towards the null. If 
providers wish to revoke their consent and no longer participate in the study, they will also be 
removed from the study. 
 
We also request a waiver of patient informed consent and authorization for the collection of 
routinely-collected patient data for outcome evaluation, given the minimal risk nature of the study, 
as we have done in prior work. We believe this waiver is justified as investigators will have no 
direct patient contact and all prescribing decisions are made under the care of licensed physicians, 
including supervising residents and attending who are directing their care. Finally, the intervention 
is exclusively provider-facing with the underlying goal of promoting patient safety in accordance 
with clinical practice guidelines.  
 
 
8.3  Confidentiality and privacy 
We will enroll provider-subjects based on their being employed by MGB as interns. For the 
trial, we will seek informed consent from providers. We will also seek a HIPAA waiver of patient 
authorization to access the EHR data necessary for outcome evaluation. 
 
To ensure the confidentiality and security of all data, the research team operates a secure, 
state-of-the-art computing facility housed at MGB’s data center. The MGB data center is a 
secure facility that houses both computing environments as well as clinical systems and 
electronic medical records for several large hospitals in Eastern Massachusetts. Entry into the 
computer room requires staffed computer room security. The Division’s computers are 
connected to the MGB networking backbone with 10 gigabit-per-second fiber links. Network 
security is overseen by electronic medical records systems to the research team’s data. All 
data are transmitted to programmers’ workstations in an encrypted state. Backups are created 
using 256-bit AES encryption, the current Department of Defense standard for data security, 
and are stored in a locked facility. The redundancy, extensive data power, and security of our 
computer facility confirm our capacity to collect and manage data and ensure confidentiality for 
all project participants.  

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
25 
 
 
We will also safeguard any identifiable information from the providers in accordance with IRB 
practices, limit access to any information in accordance with IRB practices, limit access to the 
information to study investigators actively involved in the research who have all undergone 
human subjects research training, and destroy any recordings from the qualitative interviews 
upon completion of the research.  
 
All members of the research team have completed appropriate human subjects research training 
and patient privacy training related to the Health Insurance Portability and Accountability Act 
(HIPAA). The setup for analysis of these HIPAA-limited data will be exactly the same as all of the 
other IRB applications that our MGB research division submits for secondary use of data. In fact, 
we have an umbrella-approval place in place with the MGB IRB for using these types of HIPAA-
limited data. All of the datasets, including limited PHI, will be stored only on secure servers at the 
MGB data center and will only be accessed by a limited number of individuals in the study team 
from this division who are all trained in data security and patient privacy.  
 
8.4 
Safety oversight 
General oversight of this project by the BWH co-leads (Drs. Lauffenburger and Choudhry) will 
occur throughout the study period, including regular contact with clinical leadership of the general 
medicine service wards involved in the project to obtain ongoing feedback. In addition, this protocol 
will undergo Institutional Review Board (IRB) evaluation. Drs. Choudhry and Lauffenburger have 
previously overseen numerous large, multi-site pragmatic trials.  
 
This study will include safety monitoring from both the Roybal Centers Program DSMB and an 
independent safety officer (SO) to perform data and safety monitoring activities. This SO will advise 
NIA Program staff and the PI regarding participant safety, study risks and benefits, scientific 
integrity, participant recruitment, and ethical conduct of the study. The SO will also provide the 
Program DSMB with periodic safety reports in a manner determined by the Program DSMB.  
 
As directed by the NIA, the co-leads will appoint a licensed physician with relevant study and 
disease-specific expertise to serve as SO, submitted to the NIA PO for approval. The Program 
DSMB consists of members approved by the Director of NIA. Following approval, the SO and 
Program DSMB will receive a manual of operating procedures containing the study protocol and 
DSMP prior to study enrollment. The co-leads (Drs. Choudhry and Lauffenburger) have nominated 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
26 
 
Gina Kruse, MD, MS, MPH, as the independent Safety Officer (SO), pending approval by the NIA 
PO, to act in an advisory capacity to the NIA PO and to evaluate the progress of the study. 
 
8.5 Benefit risk assessment 
8.5.1 Known potential risks 
We believe there is no more than minimal risk involved to the provider subjects, as the 
providers will simply undergo training to alter their behaviors towards guideline recommended 
care, as opposed to being forced to do so. In terms of patient-subjects, all medical decisions 
are ultimately made by the provider. This trial will otherwise not interfere with the ordinary 
workings of the inpatient service.  
 
8.5.2 Known potential benefits 
Simulation training could help reduce high-risk prescribing in stressful situations as a way to 
improve patient safety. However, very few studies have evaluated the use of simulation training 
in the context of inpatient prescribing, and not in the context of bridging System 1 and System 
2 prescribing behaviors. Thus, the research will have both immediate benefits for subjects 
enrolled in the study as well as for the larger population of older adults. We will also produce 
several deliverables for this work for the public, researchers, and policymakers, which will be 
shared as generalized knowledge. These deliverables include the results from the study and 
implementation toolkits. If effective, this intervention could be expanded to other high-risk 
prescribing situations. 
 
8.5.3 Assessment of potential risks and benefits 
We will enroll provider-subjects based on their being employed by MGB as interns. For the 
trial, we will seek informed consent. We will also receive a HIPAA waiver of patient 
authorization to access the EHR data necessary for outcome evaluation.  
 
To protect against the risk of inappropriate disclosure of personal health information, the 
investigators at BWH will only access study data with encrypted identifiers. As described, all 
members of the research team have completed or will complete appropriate human subjects 
research training and patient privacy training related to the Health Insurance Portability and 
Accountability Act (HIPAA). We have a history of collaborative evaluations with delivery 
organizations that involves transfer of the minimum data necessary to complete rigorous 
evaluations, involving the use of encrypted identifiers to ensure patient confidentiality.  

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
27 
 
To ensure the confidentiality and security of all data, the research team operates a secure, 
state-of-the-art computing facility housed at MGB’s data center.  
 
We will also safeguard any identifiable information from the providers in accordance with IRB 
practices, limit access to any information in accordance with IRB practices, and limit access to 
the information to study investigators actively involved in the research who have all undergone 
human subjects research training.  
 
For all of these reasons, we believe that this trial is a minimal risk trial and that we have the 
appropriate protections in place for all study subjects.  
 
 
9. List of references 
1. 
Hill KD, Wee R. Psychotropic drug-induced falls in older people: a review of interventions aimed at reducing the 
problem. Drugs Aging. 2012;29(1):15-30. 
2. 
Jordan S, Gabe-Walters ME, Watkins A, et al. Nurse-Led Medicines' Monitoring for Patients with Dementia in Care 
Homes: A Pragmatic Cohort Stepped Wedge Cluster Randomised Trial. PloS one. 2015;10(10):e0140203. 
3. 
Craftman AG, Johnell K, Fastbom J, Westerbotn M, von Strauss E. Time trends in 20 years of medication use in older 
adults: Findings from three elderly cohorts in Stockholm, Sweden. Archives of gerontology and geriatrics. 2016;63:28-
35. 
4. 
By the American Geriatrics Society Beers Criteria Update Expert P. American Geriatrics Society 2019 Updated AGS 
Beers Criteria(R) for Potentially Inappropriate Medication Use in Older Adults. Journal of the American Geriatrics 
Society. 2019;67(4):674-694. 
5. 
Marcum ZA, Kisak A, Visoiu A, Resnick N. Medication Reconciliation: Will the Real Medication List Please Stand Up? 
Journal of the American Geriatrics Society. 2015;63(12):2639-2641. 
6. 
Michaelsen MH, McCague P, Bradley CP, Sahm LJ. Medication Reconciliation at Discharge from Hospital: A Systematic 
Review of the Quantitative Literature. Pharmacy (Basel). 2015;3(2):53-71. 
7. 
Embi PJ, Leonard AC. Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized 
controlled study. Journal of the American Medical Informatics Association : JAMIA. 2012;19(e1):e145-148. 
8. 
Payne TH, Hines LE, Chan RC, et al. Recommendations to improve the usability of drug-drug interaction clinical 
decision support alerts. Journal of the American Medical Informatics Association : JAMIA. 2015;22(6):1243-1250. 
9. 
Lewis PJ, Tully MP. Uncomfortable prescribing decisions in hospitals: the impact of teamwork. Journal of the Royal 
Society of Medicine. 2009;102(11):481-488. 
10. 
Roumie CL, Elasy TA, Wallston KA, et al. Clinical inertia: a common barrier to changing provider prescribing behavior. 
Joint Commission journal on quality and patient safety. 2007;33(5):277-285. 
11. 
Litvin CB, Ornstein SM, Wessell AM, Nemeth LS, Nietert PJ. Adoption of a clinical decision support system to promote 
judicious use of antibiotics for acute respiratory infections in primary care. International journal of medical informatics. 
2012;81(8):521-526. 
12. 
Markota M, Rummans TA, Bostwick JM, Lapid MI. Benzodiazepine Use in Older Adults: Dangers, Management, and 
Alternative Therapies. Mayo Clinic proceedings. 2016;91(11):1632-1639. 
13. 
Davey P, Marwick CA, Scott CL, et al. Interventions to improve antibiotic prescribing practices for hospital inpatients. 
Cochrane Database Syst Rev. 2017;2:CD003543. 
14. 
Loewenstein G. Hot-cold empathy gaps and medical decision making. Health psychology : official journal of the Division 
of Health Psychology, American Psychological Association. 2005;24(4S):S49-56. 
15. 
Sayette MA, Loewenstein G, Griffin KM, Black JJ. Exploring the cold-to-hot empathy gap in smokers. Psychol Sci. 
2008;19(9):926-932. 
16. 
Webster C. More on "fast" and "slow" thinking in diagnostic reasoning. Academic medicine : journal of the Association 
of American Medical Colleges. 2015;90(1):3. 
17. 
Alsoufi B. Thinking, fast and slow. The Journal of thoracic and cardiovascular surgery. 2017;153(3):646-647. 
18. 
Kang MJ, Camerer CF. fMRI evidence of a hot-cold empathy gap in hypothetical and real aversive choices. Frontiers in 
neuroscience. 2013;7:104. 
19. 
Tay SW, Ryan P, Ryan CA. Systems 1 and 2 thinking processes and cognitive reflection testing in medical students. Can 
Med Educ J. 2016;7(2):e97-e103. 

Provider simulation for high-risk prescribing 
 
      Version 2 
PIs: Lauffenburger/Choudhry 
 
February 9, 2021 
28 
 
20. 
Ward T, Garety PA. Fast and slow thinking in distressing delusions: A review of the literature and implications for 
targeted therapy. Schizophrenia research. 2019;203:80-87. 
21. 
Siebolds M. [Evidence-based medicine as a model of decision making in clinical practice]. Z Arztl Fortbild Qualitatssich. 
2003;97(4-5):257-262. 
22. 
Sequist TD, Zaslavsky AM, Colditz GA, Ayanian JZ. Electronic patient messages to promote colorectal cancer screening: 
a randomized controlled trial. Archives of internal medicine. 2011;171(7):636-641. 
23. 
Angner ELG. Behavioral Economics. Handbook of the Philosophy of Science: Philosophy of Economic. 2007:641-690. 
24. 
Datta S, Burns J, Maughan-Brown B, Darling M, Eyal K. Risking it all for live? Resetting beliefs about HIV risk among 
low-income South African teens. Journal of Economic Behavior & Organization. 2015;118:184-198. 
25. 
Schoenthaler A, Albright G, Hibbard J, Goldman R. Simulated Conversations With Virtual Humans to Improve Patient-
Provider Communication and Reduce Unnecessary Prescriptions for Antibiotics: A Repeated Measure Pilot Study. JMIR 
Med Educ. 2017;3(1):e7. 
26. 
Armenia S, Thangamathesvaran L, Caine AD, King N, Kunac A, Merchant AM. The Role of High-Fidelity Team-Based 
Simulation in Acute Care Settings: A Systematic Review. Surg J (N Y). 2018;4(3):e136-e151. 
27. 
Dale-Tam J, Posner GD. Alice's Delirium: A Theatre-based Simulation Scenario for Nursing. Cureus. 2018;10(4):e2411. 
28. 
Molloy MA, Cary MP, Jr., Brennan-Cook J, et al. Preparing Clinicians for Transitioning Patients Across Care Settings and 
Into the Home Through Simulation. Home healthcare now. 2018;36(4):225-231. 
29. 
Kannampallil TG, McNutt R, Falck S, et al. Learning optimal opioid prescribing and monitoring: a simulation study of 
medical residents. JAMIA Open. 2018;1(2):246-254. 
30. 
Muller E, Diesing A, Rosahl A, Scholl I, Harter M, Buchholz A. Evaluation of a shared decision-making communication 
skills training for physicians treating patients with asthma: a mixed methods study using simulated patients. BMC 
health services research. 2019;19(1):612. 
31. 
Peltan ID, Shiga T, Gordon JA, Currier PF. Simulation Improves Procedural Protocol Adherence During Central Venous 
Catheter Placement: A Randomized Controlled Trial. Simul Healthc. 2015;10(5):270-276. 
32. 
Tluczek A, Henriques JB, Brown RL. Support for the reliability and validity of a six-item state anxiety scale derived from 
the State-Trait Anxiety Inventory. Journal of nursing measurement. 2009;17(1):19-28. 
33. 
By the American Geriatrics Society Beers Criteria Update Expert P. American Geriatrics Society 2015 Updated Beers 
Criteria for Potentially Inappropriate Medication Use in Older Adults. Journal of the American Geriatrics Society. 
2015;63(11):2227-2246. 
34. 
McCormick WC. Revised AGS Choosing Wisely((R)) list: changes to help guide older adult care conversations. Journal of 
gerontological nursing. 2015;41(5):49-50. 
35. 
Lauffenburger JC CN. Optimizing Electronic Health Record Prompts With Behavioral Economics to Improve Prescribing 
for Older Adults (NUDGE-EHR). 2020; https://clinicaltrials.gov/ct2/show/NCT04284553.