Pragmatic Trial of User-centered Clinical Decision Support to Implement EMergency Department-initiated BuprenorphinE for Opioid Use Disorder
Phase: NA
Conditions: Clinical Decision Support, Buprenorphine, Opioid-use Disorder
Sponsor: Yale University

Page 1 of 22 
UH3 Version 1 (8/22/18) 
 
 
Title: Clinical Decision Support to Implement ED-initiated Buprenorphine for OUD 
ClinicalTrials.gov ID: NCT03658642 
Date: 8/22/2018 
 
 

Page 2 of 22 
UH3 Version 1 (8/22/18) 
                    
                                    
HRP-503B – BIOMEDICAL RESEARCH PROTOCOL  
(2017-1) 
 
 
Protocol Title: EMBED: Pragmatic trial of user-centered clinical decision support to implement EMergency 
department-initiated BuprenorphinE for opioid use Disorder 
 
Principal Investigator: Edward R. Melnick, MD, MHS 
 
Version Date: 8/22/18 
 
(If applicable) Clinicaltrials.gov Registration #: NCT03658642 
 
 
INSTRUCTIONS 
 
This template is intended to help investigators prepare a protocol that includes all of the necessary information 
needed by the IRB to determine whether a study meets approval criteria. Read the following instructions 
before proceeding: 
 
1. Use this protocol template for a PI initiated study that includes direct interactions with research 
subjects. Additional templates for other types of research protocols are available in the system Library. 
 
2. If a section or question does not apply to your research study, type “Not Applicable” underneath. 
 
3. Once completed, upload your protocol in the “Basic Information” screen in IRES IRB system.  
 
 
 
SECTION I: RESEARCH PLAN 
 
1. Statement of Purpose: State the scientific aim(s) of the study, or the hypotheses to be tested.  
The goal of this multicenter, pragmatic, parallel cluster randomized trial is to compare the 
effectiveness of user-centered CDS for ED-initiated BUP and referral for ongoing MOUD treatment 
to usual care on the rates of ED initiation of BUP and referral in ED patients with OUD. We 
hypothesize that rates of ED-initiation of BUP and referral will be higher in the user-centered CDS 
arm of the trial. 
 
2. Probable Duration of Project: State the expected duration of the project, including all follow-up and data 
analysis activities.   
18 months 
 
3. Background: Describe the background information that led to the plan for this project. Provide references to 
support the expectation of obtaining useful scientific data. 

Page 3 of 22 
UH3 Version 1 (8/22/18) 
Dependence on opioids is a major public health problem in the United States, taking a devastating 
toll on Americans, their families, and communities.[1,2] An estimated 2.1 million people in the U.S. 
have opioid use disorder (OUD)[3] and more than 33,000 opioid-related deaths occur annually.[4] In 
2011, there were 605,000 ED visits related to opioids in the United States.[5] From 2016-2017, 
emergency departments (EDs) experienced a 30% increase in visits for opioid overdose.[6] The ED 
offers a unique treatment opportunity for patients receiving care for acute and comorbid conditions 
related to opioid use.  
 
One of the most promising treatments for OUD is buprenorphine/naloxone (BUP), a partial opioid 
agonist combined with an antagonist, that can be prescribed by an appropriately trained clinician in 
an office setting for use at home. BUP decreases mortality as well as symptoms of withdrawal, 
craving, and opioid use.[7,8] In a placebo-controlled randomized trial of 40 OUD patients who all 
received cognitive-behavioral group therapy, weekly individual counseling, and weekly urine drug 
screening, cumulative retention in treatment at one year was 75% for individuals in the BUP group 
compared to 0% in the placebo group (p = 0.0001).[9] A recent Cochrane review including 31 trials 
with 5430 participants found high quality evidence that BUP is superior to placebo in retention of 
participants in treatment and can reduce illicit opioid use effectively compared to placebo.[10]  
 
Currently, ED clinicians often provide OUD patients referral to opioid treatment programs rather 
than initiating MOUD treatment in the ED. In a randomized clinical trial involving 329 individuals with 
OUD, we found that ED-initiation of BUP with referral for ongoing MOUD treatment was superior to 
referral alone, resulting in nearly twice the percentage of patients who were engaged in formal 
addiction treatment at 30 days (78% with BUP vs 37% with referral alone vs 45% with brief 
intervention, p < 0.001) and less illicit opioid use.[11] Despite the efficacy of ED-initiated BUP with 
referral for ongoing MOUD treatment, it is currently not offered in routine ED practice due to multiple 
complex medical, regulatory, and logistical barriers.[11–13] Adopting this evidence-based practice 
into routine care would shift the clinical practice paradigm for early OUD identification and treatment 
by initiating treatment at a time when the patient may be motivated and particularly vulnerable to 
morbidity and mortality.[14,15] 
 
Clinical decision support (CDS), computerized tools that offer patient-specific assessments or 
recommendations to clinicians, represents one approach to embed this complex intervention into 
routine emergency care.[16,17] However, CDS faces its own challenges, including unintended 
consequences such as alert fatigue and increased cognitive load.[18–22] CDS design 
recommendations suggest careful consideration of the socio-technical environment and delivery of 
the right information, to the right person, in the right format, at the right time in clinical workflow to 
optimize medical decision-making.[23–26] 
 
For these reasons, we employed a user-centered design process to design and formatively 
evaluate the EMBED (Emergency Department-Initiated Buprenorphine for Opioid Use Disorder) 
CDS intervention. The user-centered design and formative evaluation of the EMBED intervention is 
reported elsewhere. Given the current opioid epidemic in the US, there is great urgency for 
prospective trials to identify the best approaches to BUP implementation and integration into routine 
practice. The goal of this multicenter, pragmatic, parallel cluster randomized trial is to compare the 
effectiveness of user-centered CDS for ED-initiated BUP and referral for ongoing MOUD treatment 
to usual care on the rates of ED initiation of BUP and referral in ED patients with OUD. We 
hypothesize that rates of ED-initiation of BUP and referral will be higher in the user-centered CDS 
arm of the trial. 
 
 

Page 4 of 22 
UH3 Version 1 (8/22/18) 
4. Research Plan: Summarize the study design and research procedures using non-technical language that can 
be readily understood by someone outside the discipline. Be sure to distinguish between standard of care vs. 
research procedures when applicable, and include any flowcharts of visits specifying their individual times 
and lengths. Describe the setting in which the research will take place. 
 
Design 
 
The study design is an 18-month pragmatic, parallel, cluster randomized, superiority trial using 
constrained randomization of clusters to arms (schematic diagram, Figure 1).[27–29] The unit of 
randomization (i.e. cluster) is the ED. EDs will be randomly allocated with an allocation ratio of 1:1. 
Adequate lead time will be allotted to install the intervention in the EHR at all intervention sites--
including a three month implementation and washout phase. The intervention will then begin at the 
same time across all sites with the CDS intervention fully implemented in the intervention sites’ 
EHRs at the start of the trial. Clinicians at control sites will retain all control of their practice and 
practice as usual without the CDS intervention installed in their EHR. 
  
 
 
Pragmatic trials study an intervention under the usual conditions in which it will be applied; as 
opposed to an explanatory trial which would test an intervention under ideal conditions. [27,30] In 
cluster randomized trials, treatment intervention is allocated to clusters (i.e. groups of individuals) 
rather than individuals. This is done to manipulate the physical or social environment of the 
intervention when an individual intervention would likely result in contamination between 
intervention and control participants at the group level.[28] The parallel cluster randomized design 
was chosen over a stepped wedge design due to the high likelihood of confounding by temporal 
trends from ongoing efforts to mitigate the opioid epidemic.[31][32] A major challenge of the cluster 
randomized design is from potential confounding due to a limited number of heterogeneous 
groups.[28] Constrained randomization offers a solution to this source of confounding by balancing 
key cluster-level prognostic factors across the study to avoid distorting estimates of treatment effect 
due to the confounding factors.[29] This allocation technique more evenly distributes potential 
confounders between intervention arms by specifying the confounding factors, characterizing each 
cluster in terms of these factors, identifying a subset of randomization combinations of clusters that 
adequately balance confounding factors between intervention arms and randomly selecting one of 
these combinations as the allocation scheme.[29] Potential confounders that will be used for this 
trial are: EHR vendor, ED annual volume, ED type (e.g., academic, community, urban, rural, etc), 
ratio of ED clinicians who have a waiver to prescribe BUP, current rate of ED BUP prescribing, 
resources in ED to facilitate management of patients with OUD, and willingness of staff to adopt the 
practice of ED-initiation of BUP. 

Page 5 of 22 
UH3 Version 1 (8/22/18) 
 
Intervention 
The intervention for this study includes the user-centered CDS as well as education of ED clinicians 
practicing at all study sites.  
 
The need for flexibility in the graphical user interface of the intervention resulted in the decision to 
develop the CDS as a web application. This provides the ability to access the tool both embedded 
within the EHR or directly over the Internet. The web application was developed as a single-page 
application (SPA) based on React JavaScript library. The CDS is a user-initiated, SMART on FHIR 
(Substitutable Medical Applications and Reusable Technologies on Fast Health Interoperability 
Resources)[37] application that streamlines a flow diagram of our clinical protocol for ED-initiated 
BUP (Figure 2).  
 
The intervention’s graphical user interface (Figure 3) is an intuitive, simple layout presenting four 
care pathways in columns based on the patient’s diagnosis of OUD, the severity of withdrawal, and 
readiness to start treatment. There is additional, optional decision support available for guidance to: 
1) evaluate OUD severity based on DSM-5 criteria, 2) assess withdrawal severity using the clinical 
opiate withdrawal scale (COWS) score, and 3) motivate patient willingness and readiness to initiate 
MOUD treatment with a brief motivational interview.[38,39] These materials are also available to 
share with other members of the care team via a web address, text messaging, or QR code. The 

Page 6 of 22 
UH3 Version 1 (8/22/18) 
interface also includes a toggle switch for the user based on whether or not they have a waiver to 
prescribe BUP. Non-waivered clinicians cannot prescribe BUP but can administer a one-time dose 
of BUP in the ED for up to 72-hours.[40] When integrated into the local EHR system, launching a 
care pathway enables the user to: place orders, refer for ongoing MOUD treatment, and update 
clinical notes. 
 
 
 
The educational plan will be site-specific and tailored to the usual care at that institution. It will be 
administered within three months of the study start date. The details of the plan will be developed in 
partnership with local champions who self-identify an interest in helping to implement an ED-
initiated BUP protocol at their site. Specifically, the education plan will be required to include:  
1. A didactic on opioid use disorder, its diagnosis, assessment of withdrawal severity, and local 
resources for referral for ongoing MOUD treatment 
2. Circulation and posting in each study site ED of the flow diagram of the study’s clinical protocol 
for ED-initiated BUP (Figure 2). Since this protocol is considered best practice, clinicians at 
control sites will retain all control of their practice and be encouraged to follow this protocol even 
though the CDS will not be available to them. 
3. Intervention sites will include strategies to increase use of the intervention by training clinicians 
on how to launch and use the CDS. Use of the intervention will be tracked with site-specific 
audit and feedback that is consistent with typical quality improvement initiatives at that site.  
 
Given the ongoing and escalating opioid epidemic and wide scope of this trial, we anticipate that 
there may be concomitant interventions to stem OUD at study sites during the trial. We plan to 
permit these interventions as long as they are: (1) implemented before randomization so that they 
can be tracked and accounted for in the constrained randomization process, and (2) they are not a 
health IT intervention targeted at clinicians to initiate BUP in the ED.  
 
Outcomes 
The primary study hypothesis is that there will be higher rates of provision of ED-initiated BUP with 
referral for ongoing MOUD with user-centered CDS compared with usual care. Therefore, the primary 
outcome will be BUP initiation in the ED, defined as whether or not an eligible patient is administered 

Page 7 of 22 
UH3 Version 1 (8/22/18) 
BUP in the ED and/or prescribed BUP upon discharge from the ED. Although this is not a patient-
centered outcome, it is a pragmatic and meaningful surrogate that will serve as a lead indicator of the 
CDS intervention’s effect on engaging more OUD patients in treatment.  
 
We will also evaluate the effect of user-centered CDS on the following secondary implementation 
outcomes as compared to usual care, including several following the RE-AIM framework:[41,42]  
1. Referral to follow-up for ongoing MOUD treatment (patient-level; Y/N)treatment 
2. Prescription for naloxone at ED discharge (patient-level; Y/N) 
3. Receipt of discharge instructions on opioid use, overdose education, naloxone education, and 
buprenorphine education (patient-level; Y/N) 
4. Clinician adoption rates (clinician level): 
a. Provision of any ED-initiated BUP during the trial (Y/N) 
b. Provision of any referral for ongoing MOUD treatment during the trial (Y/N) 
5. Receipt of Drug Addiction Treatment Act of 2000 training during trial (clinician level; Y/N)  
Additional secondary implementation outcomes to be obtained from the web application include: 
clinician fidelity with the intervention assessed via a critical action checklist[43] and error rate of the 
intervention (using surrogates based on tool usage, e.g., application launched but not used, launching a 
page in the web application and spending less than two seconds on that page). The intervention will 
continue to be made available for use after the trial concludes; three months after trial completion, 
medical record review of eligible patients will be conducted at a subset of intervention sites to 
determine the maintenance rate of the intervention. 
 
Data Collection 
Outcome data will be collected via SQL query of the local EHR at regular intervals from data routinely 
collected in each hospital’s EHR. This will facilitate large-scale data collection that would not otherwise 
be practical in an explanatory trial.  
 
To enable consistent EHR data collection across sites, a master data dictionary of all data elements will 
be created. At each study site, the variables in the data dictionary will be validated against the 
institutional EHR to ensure that the variables are correctly mapped to the EHR field that corresponds to 
the clinical intent of the variable after accounting for documentation practices and workflow at each 
site.[47] In particular, the outcome variables of BUP initiated in the ED and referral made for ongoing 
MOUD treatment will be validated against the EHR to ensure accuracy. For data quality assurance, the 
mapped variables will be validated against the EHR to ensure that the data are clinically relevant to the 
goals of the project and correctly represents the clinical data that clinicians use to make decisions. 
Additionally, data to determine compliance, use, and fidelity with the CDS intervention that could not be 
reliably abstracted from the EHR (e.g., DSM-5 OUD score, COWS score) will be abstracted from the 
web application’s use logs. Information on whether the patient attended the referred follow-up visit and 
whether the patient  was prescribed BUP as an outpatient will be abstracted from the EHR if available 
(e.g., if the patient is seen for follow-up within the same system). 
 
Data will be sent from study sites to the study DCC at predetermined, regular intervals. The DCC will 
conduct ongoing data monitoring activities on study data from all participating sites to ensure data 
received is what it is intended to be. Baseline data for the study participants will include demographic 
and clinical data such as age, gender, race, ethnicity, insurance status, past medical and psychiatric 
history, recent medical or psychiatric hospital admissions, recent enrollment in formal addiction 
treatment, active prescriptions for other opioids, and urine drug screen results as ascertained by 
regularly collected data in the EHR. 
 
5. Genetic Testing    
N/A ☒ 

Page 8 of 22 
UH3 Version 1 (8/22/18) 
 
6. Subject Population: Provide a detailed description of the types of human subjects who will be recruited into 
this study. 
 
Subjects for this trial will be the clinicians in both the control sites and those sites implementing the 
user-centered CDS. A waiver of informed consent will be obtained for data collection of clinicians. 
 
OUD patients are not considered human subjects since: (1) no identifiable private information will be 
collected, (2) the intervention does not target the patient, and (3) EHR data will be collected 
retrospectively without interaction with the patient. 
 
 
7. Subject classification: Check off all classifications of subjects that will be specifically recruited for enrollment in 
the research project. Will subjects who may require additional safeguards or other considerations be enrolled 
in the study? If so, identify the population of subjects requiring special safeguards and provide a justification 
for their involvement. 
 
☐Children 
 
 
☒ Healthy  
 
 
☐Fetal material, placenta, or dead fetus 
☐Non-English Speaking  
☐ Prisoners 
 
 
☐Economically disadvantaged persons 
☐Decisionally Impaired  
☒ Employees  
 
☐Pregnant women and/or fetuses 
☐Yale Students  
 
☐ Females of childbearing potential 
 
NOTE: Is this research proposal designed to enroll children who are wards of the state as potential subjects? 
Yes ☐  No ☒  
 
8. Inclusion/Exclusion Criteria: What are the criteria used to determine subject inclusion or exclusion? 
 
There will be 20 participating EDs from hospitals within approximately six health care systems 
(HCS) of which clinicians will be recruited.  
 
Adult ED patients (age 18 years or older) meeting an EHR-derived phenotype suggesting possible 
OUD will be included in the analysis: those who are discharged from the ED, not pregnant, and not 
currently taking a MOUD. The CDS will also be available for physicians to use when patients do not 
meet the EHR phenotype. The initial phenotype has been developed by the study team and is 
currently undergoing validation via emergency physician chart review to determine the phenotype’s 
validity in identifying the target patient population.[33] The phenotype will use routinely collected 
structured data elements from the EHR (e.g., diagnosis code of opioid overdose). All ED patients 
meeting the EHR phenotype criteria will be eligible for the trial. For patients with more than one ED 
visit during the study period, only the initial ED visit will be eligible for inclusion in the primary 
analysis. The CDS will also be available for clinicians to use for patients who are not identified by 
the phenotype. These patients will be excluded from the primary analyses.  
 
9. How will eligibility be determined, and by whom?  
 
Patient eligibility will be determined by the EHR phenotype described above. All ED clinicians 
practicing in the study sites will be eligible. 
 
10. Risks: Describe the reasonably foreseeable risks, including risks to subject privacy, discomforts, or 
inconveniences associated with subjects participating in the research.  

Page 9 of 22 
UH3 Version 1 (8/22/18) 
 
OUD Patients: Information and identifiers from patients will not be collected, as this is a trial for 
clinician use of the Clinical Decision Support (CDS) for opioid use disorder (OUD). Clinicians will 
retain complete control over treatment decisions and at intervention sites have the option whether 
or not to use the intervention. The patient retains the right to refuse treatment or request treatment 
at any time. All tools included in the CDS are validated clinical tools that are part of recommended 
best practices. The OUD population has a high underlying risk of morbidity/mortality (approximately 
5% risk of death in 12 months per LaRochelle Annals of IM 2018). The risk to a patient with OUD 
who is not receiving medication for opioid use disorder in their ordinary daily lives greatly exceeds 
the risk of the EMBED intervention.  
 
Clinicians: Clinicians in the control group will have access to all standard OUD medications and 
services to which they would otherwise have access to treat OUD patients. Clinicians retain all 
control of their practice. The intervention group will receive interventions which are already 
accepted as best practices; none of the proposed interventions are experimental, and they do not 
carry any risks beyond what is expected in standard medical care. Clinician identifiers will be 
collected in order to follow practice patterns. However, the investigators will be blinded to both site 
and clinician identifiers. Each system will use an Honest Broker to protect the welfare and identity of 
each site and clinician and allow adjudication for analyses. 
 
11. Minimizing Risks: Describe the manner in which the above-mentioned risks will be minimized. 
 
Study data will only be available to members of the Yale Data Coordinating Center who are 
authorized for this study.  To ensure the privacy and confidentiality of data for this project, we will 
store and use identifiable data in a Yale University ITS hosted environment that is approved by Yale 
ITS Information Security Office. The physical address to the facility is limited to ITS, and server 
access is limited only to those who are authorized. All personnel who have access to the data will 
pass appropriate HIPAA training coursework. The main levels of security are:  
 
• 
Physical media that are received from the distributer or any physical copies of the data will be 
encrypted while at rest and will be held in a locked, fireproof cabinet within the office of Dr. 
Melnick. 
• 
Project computers are all password protected, are protected by the Yale University firewall, are 
encrypted using Microsoft BitLocker, and are in locked offices within a building having limited, 
electronic passkey access.  
• 
All servers and workstations have been certified by Yale’s Information Security Officer as 
compliant with Yale’s HIPAA policy (http://hipaa.yale.edu/). 
• 
All computing devices follow Yale’s password policy, which requires strong passwords with 
periodic mandatory changes. 
• 
All computers are on the Yale internal network which is maintained and monitored by Yale ITS. 
The servers on the Yale internal network have no direct connection to the external network 
without special setup by Yale ITS after serious security screening performed by the Yale 
Information Security Office. 
• 
The PHI database will reside on the local network and will be accessible only by selected data 
project staff. 
• 
All servers employ redundant drives to protect against data loss in the event of hardware 
failure. All databases are backed up by Yale ITS and can be recovered in the event of 
database corruption. 

Page 10 of 22 
UH3 Version 1 (8/22/18) 
• 
All servers, including the PHI server, are located in a secure, environmentally-controlled facility. 
Electrical power to this facility is protected by a standby power system maintained by Yale 
Facilities. This system includes generators to protect against complete building shutdowns. 
• 
Files used for analysis are required to reside on servers and are never stored on desktop 
computers.  
• 
All staff requiring access to PHI must complete HIPAA training provided by Yale University, and 
must additionally follow procedures for the protection of electronic and printed data. 
 
The electronic data files for this study will be processed on this dedicated, layered-security system, 
which can be accessed only by the Yale Data Coordinating Center and designated project staff that 
are under the direct supervision of the PI. Since the system is behind multiple firewalls, is monitored 
regularly, and is accessible only to key personnel, the risk of unlawful penetration is not a significant 
data safeguard concern.  
 
Individually identifiable or deducible data will not be transmitted by unsecured telecommunications, 
which include the Internet, email, and electronic File Transfer Protocol (FTP).   
 
At the conclusion of this study, we will follow the NIH’s data retention guideline. 
  
Lastly, all output containing individual identifiable information is treated as confidential data.  This 
information is never transferred electronically via email or other protocols.  Shredders are used on 
any printed material containing individual identifiers.   
 
12. Data and Safety Monitoring Plan: Include an appropriate Data and Safety Monitoring Plan (DSMP) based on 
the investigator’s risk assessment stated below. (Note: the HIC will make the final determination of the risk to 
subjects.) 
 
a.  
What is the investigator’s assessment of the overall risk level for subjects participating in this 
study? Minimal 
b. 
If children are involved, what is the investigator’s assessment of the overall risk level for the 
children participating in this study? Not Applicable 
c. 
Include an appropriate Data and Safety Monitoring Plan. Examples of DSMPs are   
available here http://your.yale.edu/policies-procedures/forms/420-fr-01-data-and-safety-
monitoring-plans-templates for 
i. Minimal risk 
ii. Greater than minimal 
 
As a minimal risk implementation study of established best practices, an Independent Study Monitor will 
be utilized in place of a formal Data Safety Monitoring Board (DSMB). Interim monitoring will focus on 
adherence to the protocol, completeness of data retrieval from each ED’s EHR, and uptake of the CDS 
intervention. A set of monitoring tables will be generated for this purpose. The Independent Study 
Monitor will report directly to the study DCC. No interim analyses for effectiveness are planned.  
 
d. For multi-site studies for which the Yale PI serves as the lead investigator: 
i. How will adverse events and unanticipated problems involving risks to subjects or others be 
reported, reviewed and managed? Not Applicable 
ii. What provisions are in place for management of interim results? Not Applicable 
iii. What will the multi-site process be for protocol modifications? Not Applicable 
 
 

Page 11 of 22 
UH3 Version 1 (8/22/18) 
13. Statistical Considerations: Describe the statistical analyses that support the study design.  
 
Analysis Plan 
General Considerations: This is a cluster randomized trial to test the hypothesis that there will be higher 
rates of provision of ED-initiated BUP and referral for ongoing MOUD with user-centered CDS 
compared with usual care. Analyses will be conducted as intention to treat including all individuals 
regardless of intervention receipt. While the unit of randomization is at the level of the ED, the unit of 
analysis will be the patient or clinician. Analyses of primary and secondary outcomes will be conducted 
using generalized linear mixed models (GLMM) to account for clustering from the EDs and clinicians in 
patient outcome models. [31] Analyses will be performed in SAS v9.4 (Cary, NC) with a two-sided type I 
error of 0.05 (unless otherwise specified). For the primary and secondary analyses described below, 
only the first ED encounter for an individual patient will be used.  Supportive analyses will include 
patients with repeated ED visits. 
 
Comparability of Baseline and Intervention Patients: Distributions of baseline demographic and clinical 
characteristics will be described during baseline and intervention periods. Comparability for continuous 
variables will be examined graphically and by summary statistics (means, medians, quartiles, 
etc.).  Categorical variables will be examined by calculating frequency distributions. 
 
Analysis of Primary Outcome: The primary outcome, initiation of BUP in the ED, will be assessed for all 
patients that meet the criteria for the EHR phenotype. Intervention differences (CDS vs usual care) for 
this dichotomous outcome will be examined using  mixed effect logistic regression (GLMM). This model 
will contain a fixed effect for intervention (CDS vs usual care). Random effects will be included for ED 
and the primary clinician to account for clustering of responses. The model will also include cluster-level 
covariates included in the constrained randomization and patient-level covariates that may be 
associated with the delivery of BUP (age, gender, race, ethnicity, insurance status, past medical and 
psychiatric history, recent medical or psychiatric hospital admissions, recent enrollment in formal 
addiction treatment, active prescriptions for other opioids, and urine drug screen results). Linear 
contrasts will be used to estimate treatment differences along with 95% confidence intervals in the 
proportions of ED patients that received BUP in intervention vs. usual care. Given the relative 
advantages of GLMM and Generalized Estimating Equations, sensitivity analyses will compare 
treatments using a logistic regression with Generalized Estimating Equations, clustering on ED. 
 
Analysis of Secondary Outcomes: Secondary outcomes such as referral for MOUD appointment, 
attendance at an MOUD appointment (if available in the EHR), prescription for naloxone at ED 
discharge and receipt of discharge instructions. will be evaluated using random effects logistic 
regression as described above. Assessments of the clinician including provision of any ED-initiated 
BUP during the trial, provision of any referral for ongoing MOUD treatment during the trial and receipt of 
Drug Addiction Treatment Act of 2000 training during the trial will be compared between intervention 
and usual care using a GLMM. These models will be stratified by the number of eligible patients the 
clinician encountered during the trial and will include a fixed effect for intervention, cluster-level 
covariates included in the constrained randomization, and a random effect for ED. Discrete numeric 
outcomes such as clinical fidelity will be compared using the GLMM with an log link and a negative 
binomial distribution. 
 
Plan for Missing Data: Several strategies will be imposed to accommodate the likelihood that missing 
data will occur during this study. Prevention is the most obvious and effective manner to control bias 
and loss of power from missing data.[48] As noted in the Data Collection section above, prior to the trial 
we will pilot data collection procedures. Variables with large proportions of missing will be excluded 
from collection. We will follow the intent to treat principle, requiring follow-up of all EDs randomized 
regardless of the treatment received.[49] Regular data retrieval from EHR combined with monitoring 

Page 12 of 22 
UH3 Version 1 (8/22/18) 
and missing data reports will trigger protocols for tracking and obtaining missing data. Despite these 
prevention efforts it is reasonable to assume missing data will occur. Our primary analysis is valid under 
the assumption that missing data is missing at random (MAR).[50] We will evaluate the plausibility of 
this assumption by determining the extent of missing data and use logistic regression to identify factors 
associated with missing data. As appropriate, we will conduct sensitivity analysis using pattern-mixture 
and selection models under missing not at random (MNAR) assumptions to examine the robustness of 
conclusions of the primary analysis to missing data.[48,50] 
 
SECTION II: RESEARCH INVOLVING DRUGS, BIOLOGICS, RADIOTRACERS, PLACEBOS AND DEVICES 
 
 
If this section (or one of its parts, A or B) is not applicable, check off N/A and delete the rest of the section. 
 
A.  RADIOTRACERS 
☒N/A 
 
B.  DRUGS/BIOLOGICS    ☒N/A 
 
 B.  DEVICES 
 ☒N/A 
 
1. 
Are there any investigational devices used or investigational procedures performed at Yale-New Haven 
Hospital (YNHH) (e.g., in the YNHH Operating Room or YNHH Heart and Vascular Center)?  ☐Yes   ☒No    
 
SECTION III: RECRUITMENT/CONSENT AND ASSENT PROCEDURES  
1. Targeted Enrollment: Give the number of subjects:  
a. Targeted for enrollment at Yale for this protocol: 800 patient charts, 200 clinicians  
b. If this is a multi-site study, give the total number of subjects targeted across all sites: Not Applicable 
 
2. Indicate recruitment methods below.  Attach copies of any recruitment materials that will be used. 
☐ Flyers 
☐ Internet/web postings 
☐ Radio 
☒ Posters 
☒ Mass email solicitation [email to 
Department listserv] 
☐ Telephone 
☐ Letter 
☐ Departmental/Center website 
☐ Television 
☒ Medical record review*  
☐ Departmental/Center research boards 
☐ Newspaper 
☐ Departmental/Center newsletters 
☐ Web-based clinical trial registries 
☐ Clinicaltrails.gov  
☐ YCCI Recruitment database 
☐ Social Media (Twitter/Facebook):  
 
☐ Other:  
 
 
 
* Requests for medical records should be made through JDAT as described at 
http://medicine.yale.edu/ycci/oncore/availableservices/datarequests/datarequests.aspx 
 
3.  Recruitment Procedures:  
a. Describe how potential subjects will be identified. Clinician subjects will be identified if they work in the EDs of 
any of the health systems identified for participation. 
b. Describe how potential subjects are contacted. Clinicians will be contacted by email and at staff meetings.  
c. Who is recruiting potential subjects? Investigative team 
 
 
4. Assessment of Current Health Provider Relationship for HIPAA Consideration: 

Page 13 of 22 
UH3 Version 1 (8/22/18) 
Does the Investigator or any member of the research team have a direct existing clinical relationship with any 
potential subject?  
☐Yes, all subjects 
☐Yes, some of the subjects 
☒No 
 
If yes, describe the nature of this relationship. Write here 
5. Request for waiver of HIPAA authorization: (When requesting a waiver of HIPAA Authorization for either the 
entire study, or for recruitment purposes only.  Note: if you are collecting PHI as part of a phone or email 
screen, you must request a HIPAA waiver for recruitment purposes.) 
Choose one:  
☒ For entire study (for clinicians only) 
☐ For recruitment/screening purposes only  
☐ For inclusion of non-English speaking subject if short form is being used and there is no translated HIPAA 
research authorization form available on the University’s HIPAA website at hipaa.yale.edu. 
 
i. Describe why it would be impracticable to obtain the subject’s authorization for use/disclosure of this 
data:  
 
We anticipate a waiver of informed consent under the Common Rule (45 code of federal regulations 
(CFR) 46.116 given that:[51,52] (1) the research involves no more than minimal risk to the subjects;[53] 
(2) the waiver or alteration will not adversely affect the rights and welfare of the subjects; (3) the 
research could not practicably be carried out without the waiver or alteration; and (4) subjects will be 
provided with additional pertinent information after participation.  
 
Patients are not considered human subjects by HHS regulation 45 CFR 46.102(f)[52] since: (1) no 
identifiable private information will be collected, (2) the intervention does not target the patient, and (3) 
EHR data will be collected retrospectively without interaction with the patient. Therefore, consent is not 
applicable to this population. Furthermore, all recommendations included in the CDS intervention are 
considered best practices in treatment of OUD. The OUD population has a high underlying risk of 
morbidity/mortality (approximately 5% risk of death in 12 months).[8] Patient rights and welfare will be 
protected per standard practice. Therefore, the risk to a patient with OUD who is not receiving MOUD 
treament in their ordinary daily lives greatly exceeds the risk of the EMBED intervention. All study sites 
will post details about the study in a location visible to patients to make them aware of the option to 
receive BUP and referral to treatment so as best to offer an informed decision for requesting care. 
Patients will retain the right to request MOUD treatment at any study site.  
 
Clinicians at all study sites will have access to all standard OUD medications and services to which 
they would otherwise have access to treat OUD patients. Clinicians will retain all control of their practice 
and at intervention sites have the option whether or not to use the intervention (i.e., can opt out). 
Clinician identifiers will be collected in order to follow practice patterns. However, the investigators will 
be blinded to both site and clinician identifiers. Each system will use an Honest Broker to protect the 
welfare and identity of each site and clinician and allow adjudication for analyses. Clinicians will be 
made aware of the study, its outcomes, the data to be collected and, at intervention sites, how to use 
and opt out of using the CDS via broadcast e-mail and direct communication by site champions. A flow 
diagram of the study’s clinical protocol (Figure 2) will shared with clinicians and posted in the clinical 
work area of all study sites. Since this protocol is considered best practice, clinicians at control sites will 
retain all control of their practice and be encouraged to follow this protocol even though the CDS will 
not be available to them. As this is a pragmatic trial focused on implementing this intervention in a way 

Page 14 of 22 
UH3 Version 1 (8/22/18) 
that is as close to routine care as possible, consenting clinicians would not be consistent with routine 
CDS implementation and could jeopardize the scientific validity of the CDS intervention to overcome 
barriers to adoption of this practice[51]. Given the stigma[11] associated with treating individuals with 
OUD, the additional burden of the consent process could be a deterrent for clinicians to provide MOUD 
treatment to appropriate patients and bias the sample to clinicians with less stigma toward treating 
these patients. For this reason and since clinician data will be de-identified and unavailable to the 
investigators, we propose a waiver of consent of the clinicians to ensure the scientific validity of our 
findings. There is precedent for such a waiver in a similar situation.[54] Results will be published in 
open-access, peer-reviewed journals, presented at national meetings, and shared with the clinicians at 
participating sites via a broadcast e-mail notification of publications. 
 
 
ii. If requesting a waiver of signed authorization, describe why it would be impracticable to obtain the 
subject’s signed authorization for use/disclosure of this data: We are preparing for a trial that meets a 
framework for consent in cluster randomized trials that recommends a waiver of informed consent of 
patients if the following conditions are met: (1) the intervention is at the level of the hospital and 
informed consent of large numbers of individual patients in the hospital is not possible; (2) the study 
meets the Common Rule criteria for waiver of informed consent (see Section 5.ii above) and (3) 
participants are approached after randomization with information on the intervention. Also this study 
uses no investigational agents or devices, and simply attempts to implement giving a single dose of an 
evidence-based, FDA-approved medication in a hospital setting with referral to treatment to 
appropriate adult OUD subjects. During software development stage there is minimal risk to the 
clinician in giving input on the software’s format and content. 
 
 
 
The investigator assures that the protected health information for which a Waiver of Authorization has been 
requested will not be reused or disclosed to any person or entity other than those listed in this application, except 
as required by law, for authorized oversight of this research study, or as specifically approved for use in another 
study by an IRB. 
 
Researchers are reminded that unauthorized disclosures of PHI to individuals outside of the Yale HIPAA-Covered 
entity must be accounted for in the “accounting for disclosures log”, by subject name, purpose, date, recipients, 
and a description of information provided.  Logs are to be forwarded to the Deputy HIPAA Privacy Officer. 
 
6. Process of Consent/Assent: Describe the setting and conditions under which consent/assent will be obtained, 
including parental permission or surrogate permission and the steps taken to ensure subjects’ independent 
decision-making.  
 
We will be requesting a waiver of consent for all clinicians for the duration of this trial.  Clinicians 
participating in both the control sites and the intervention sites will be given an information sheet 
prior to participation. All clinicians in the practice will be forwarded the information sheet.  We will 
also utilize an e-mail notification system. OUD patients would not be considered human subjects in 
the case of this trial. 
 
7. Evaluation of Subject(s) Capacity to Provide Informed Consent/Assent: Indicate how the personnel obtaining 
consent will assess the potential subject’s ability and capacity to consent to the research being proposed.  
 

Page 15 of 22 
UH3 Version 1 (8/22/18) 
As practicing clinicians, these subjects have capacity to make medical decisions as a part of their 
daily lives. Therefore, they also have capacity to consent.  
 
8. Non-English Speaking Subjects: Explain provisions in place to ensure comprehension for research involving 
non-English speaking subjects. If enrollment of these subjects is anticipated, translated copies of all consent 
materials must be submitted for approval prior to use.  
Research will only be conducted with English speaking subjects 
 
As a limited alternative to the above requirement, will you use the short form* for consenting process if you 
unexpectedly encounter a non-English speaking individual interested in study participation and the translation of 
the long form is not possible prior to intended enrollment?  YES ☐  NO ☐ Not Applicable 
 
9. Consent Waiver: In certain circumstances, the HIC may grant a waiver of signed consent, or a full waiver of 
consent, depending on the study. If you will request either a waiver of consent, or a waiver of signed consent 
for this study, complete the appropriate section below.   
 
☐Not Requesting any consent waivers  
 
☒Requesting a waiver of signed consent: 
☐ Recruitment/Screening only (if for recruitment, the questions in the box below will apply to 
recruitment activities only) 
☒ Entire Study (Clinicians participating in the study will be given an information sheet in 
place of written consent)  
 
 
  
  ☒ Requesting a waiver of consent: 
☐ Recruitment/Screening only (if for recruitment, the questions in the box below will apply to 
recruitment activities only) 
☒ Entire Study  (Clinicians participating in the study will be given an information sheet in 
place of written consent) 
 For a waiver of signed consent, address the following: 
• Would the signed consent form be the only record linking the subject and the research?  YES ☐  NO ☐  
• Does a breach of confidentiality constitute the principal risk to subjects? YES ☐  NO ☐ 
 OR 
• Does the research pose greater than minimal risk? YES ☐    NO☒  
• Does the research include any activities that would require signed consent in a non-research context? YES ☐   
NO ☒ 

Page 16 of 22 
UH3 Version 1 (8/22/18) 
 
 
 
 
 
 
 
 
SECTION IV: PROTECTION OF RESEARCH SUBJECTS 
 
    Confidentiality & Security of Data: 
1. What protected health information (medical information along with the HIPAA identifiers) about subjects will 
be collected and used for the research?    
 
No patient private information will be collected. 
 
2. How will the research data be collected, recorded and stored?  
 
OUD patients: OUD private information will not be collected. Patients will instead be identified with a 
unique study identifier that The Honest Broker in each system could in theory, link to patient data. 
Identifiers may be system-dependent, but it will be mandatory that the identifier which is used is not 
an identifiable piece of protected health information (PHI) and that special administrative access is 
required to the local EHR to use this identifier to link back to patient data. Further, we will not be 
collecting or considering the actual date of OUD patient visit, but rather, we will collect the day on 
For a full waiver of consent, please address all of the following: 
• 
Does the research pose greater than minimal risk to subjects?   
☐ Yes If you answered yes, stop. A waiver cannot be granted.   
☒ No 
• 
Will the waiver adversely affect subjects’ rights and welfare? YES ☐    NO☒ 
• 
Why would the research be impracticable to conduct without the waiver? Yes 
As this is a Pragmatic Trial focused on implementing this intervention in a way that is as close to 
routine care as possible, it would be impractical to request consent from each clinician. We will 
be applying for a waiver of consent for this population. It would be a deterrent for clinicians to 
participate in this intervention with the added complications of consent. Additionally, clinician 
stigma to treating individuals with OUD could bias the sample if clinicians that have a 
stronger stigma toward these patients can refuse to participate. For this reason and since 
clinician data will be de-identified and unavailable to the investigators, we propose a waiver of 
consent of the clinicians to ensure the scientific validity of the CDS intervention to overcome 
barriers to adoption of this practice. There is precedent for such a waiver in a similar situation, 
Suffoletto et al. The Effect of a Statewide Mandatory Prescription Drug Monitoring Program on 
Opioid Prescribing by Emergency Medicine Providers Across 15 Hospitals in a Single Health 
System. J Pain. 2018;19(4):430-8 
• 
Where appropriate, how will pertinent information be returned to, or shared with subjects at a later date?  
OUD patients: As OUD patient data will not be used, there will be no findings that would 
be reportable to this population.  In an effort of good faith, we will incorporate a 
broadcast system in the ED with approved IRB text to be used locally at each site as 
they see fit (i.e., posters, screen savers, information sheets). 
Clinicians:  Clinicians will be made aware of study findings by use of a broadcast e-mail 
to all participating sites referencing the ClinicalTrials.gov record as well as notification of 
publications to open-access journals and articles attributable to the study, in which 
results of the study will be disseminated. 

Page 17 of 22 
UH3 Version 1 (8/22/18) 
which the OUD patient visited (i.e. Day 42 of the trial) without knowing the exact start date for each 
site.  As such, we would not be able to link a visit day with a OUD patient. 
 
Clinicians: We intend to integrate the use of an Honest Broker for each health system to de-identify 
the specific sites in which this trial is taking place and data is being collected.  The Honest Broker for 
each external health system will remove all identifiers and be responsible for the key to the identifiers. 
 
3. How will the digital data be stored? ☐CD  ☐DVD  ☐Flash Drive  ☐Portable Hard Drive   ☒Secured Server  
☐Laptop Computer  ☐Desktop Computer  ☐Other 
 
4. What methods and procedures will be used to safeguard the confidentiality and security of the identifiable 
study data and the storage media indicated above during and after the subject’s participation in the study?  
 
We will use study codes on data documents (e.g., completed questionnaire) instead of recording 
identifying information and keep a separate document that links the study code to subjects' identifying 
information locked in a separate location and restrict access to this document (e.g., only allowing 
primary investigators access). WE will encrypt identifiable data. Identifiable information will only be 
available to the PI and the Data Coordinating Center Co-I’s. We will securely store data documents 
within locked locations and assign security codes to computerized records. 
 
All portable devices must contain encryption software, per University Policy 5100.  If there is a technical reason a 
device cannot be encrypted please submit an exception request to the Information Security, Policy and Compliance 
Office by clicking on url http://its.yale.edu/egrc or email it.compliance@yale.edu 
 
5. What will be done with the data when the research is completed? Are there plans to destroy the identifiable 
data? If yes, describe how, by whom and when identifiers will be destroyed. If no, describe how the data and/or 
identifiers will be secured.  
 
Identifiable data will be destroyed within 6 months of study completion by the Principal Investigator. 
 
6. If appropriate, has a Certificate of Confidentiality been obtained?  
 
Yes, a COC will be provided as the study is funded by the NIH. 
 
SECTION V: POTENTIAL BENEFITS 
 
 
Potential Benefits: Identify any benefits that may be reasonably expected to result from the research, either to 
the subject(s) or to society at large. (Payment of subjects is not considered a benefit in this context of the risk 
benefit assessment.)  
 
As there is minimal risk to the clinician subjects, the benefits substantially outweigh the risks. Potential 
benefits for the OUD patients are the possibility of initiating BUP for OUD as well as referral to ongoing 
medical treatment. This treatment has been shown to be effective for staying in treatment and 
decreasing mortality and efficacious to be initiated in the ED. Therefore, ED patients are likely to benefit 
greatly from treating their addiction and having the opportunity to rebuild their lives. Additionally, the 
CDS would benefit the emergency clinicians in terms of organization and identification of those within 
this subject population and streamlining initiation of treatment—which would otherwise likely be too 
medically and bureaucratically complex to be adopted into routine ED care.  This system also provides 

Page 18 of 22 
UH3 Version 1 (8/22/18) 
a mechanism to remove clinician stigma to the OUD population. This generalizable knowledge could be 
applied in healthcare system EDs nationally and internationally. Knowledge gained from this study will 
be invaluable towards better identifying and treating opioid use disorder and ED management of the 
population. Given the anticipated benefits to OUD patients and to society, the low risks are reasonable.  
 
         SECTION VI: RESEARCH ALTERNATIVES AND ECONOMIC CONSIDERATIONS 
 
1. Alternatives: What other alternatives are available to the study subjects outside of the research?  
OUD patients: consent will not be applicable for OUD patients as identifiers will not be collected.  
They will receive usual care. 
Clinicians:  Opt out and broadcast notification will be used. Clinicians will be informed of the 
research by broadcast. Posters targeting the clinicians with information about the study will be 
posted in all participating emergency departments. Additionally, the participating health systems will 
receive broadcast e-mails detailing the trial and the outcomes being studied as well as an 
explanation of the option and process to Opt out.  That is, even though randomization is by site, 
clinicians at each site can opt out of using the intervention. Further, Clinical Champions identified at 
each site will discuss the intervention in the same way they would discuss any CDS being 
implemented in their EHR locally.  
 
2. Payments for Participation (Economic Considerations): Describe any payments that will be made to subjects, 
the amount and schedule of payments, and the conditions for receiving this compensation. 
 
There will be no payments for participation 
 
3. Costs for Participation (Economic Considerations): Clearly describe the subject’s costs associated with 
participation in the research, and the interventions or procedures of the study that will be provided at no cost 
to subjects.  
 
Interventions will be provided at no cost. The only cost of participation is the time of the participant. 
 
4. In Case of Injury: This section is required for any research involving more than minimal risk, and for minimal 
risk research that presents the potential for physical harm (e.g., research involving blood draws). 
 
a. Will medical treatment be available if research-related injury occurs? Not Applicable 
b. Where and from whom may treatment be obtained? Not Applicable 
c. Are there any limits to the treatment being provided? Not Applicable 
d. Who will pay for this treatment? Not Applicable 
e. How will the medical treatment be accessed by subjects? Not Applicable 
 
IMPORTANT REMINDERS 
 
Will this study have a billable service?  Yes ☐  
 No☒ 
 
A billable service is defined as any service rendered to a study subject that, if he/she was not on a study, would 
normally generate a bill from either Yale-New Haven Hospital or Yale Medical Group to the patient or the patient’s 
insurer. The service may or may not be performed by the research staff on your study, but may be provided by 
professionals within either Yale-New Haven Hospital or Yale Medical Group (examples include x-rays, MRIs, CT 
scans, specimens sent to central labs, or specimens sent to pathology). Notes: 1. There is no distinction made 

Page 19 of 22 
UH3 Version 1 (8/22/18) 
whether the service is paid for by the subject or their insurance (Standard of Care) or by the study’s funding 
mechanism (Research Sponsored). 2. This generally includes new services or orders placed in EPIC for research 
subjects.  
 
If answered, “yes”, this study will need to be set up in OnCore, Yale’s clinical research management system, for 
Epic to appropriately route research related charges. Please contact oncore.support@yale.edu 
 
Are there any procedures involved in this protocol that will be performed at YNHH or one of its affiliated entities?  
Yes ☐  No ☒  
 
If Yes, please answer questions a through c and note instructions below.   Not Applicable 
  
IMPORTANT REMINDER ABOUT RESEARCH AT YNHH  
Please note that if this protocol includes Yale-New Haven Hospital patients, including patients at the HRU, the 
Principal Investigator and any co-investigators who are physicians or mid-level practitioners (includes PAs, APRNs, 
psychologists and speech pathologists) who may have direct patient contact with patients on YNHH premises 
must have medical staff appointment and appropriate clinical privileges at YNHH. If you are uncertain whether 
the study personnel meet the criteria, please telephone the Physician Services Department at 203-688-2615. By 
submitting this protocol as a PI, you attest that you and any co-investigator who may have patient contact has a 
medical staff appointment and appropriate clinical privileges at YNHH. 
 
 
REFERENCES  
1 
Rudd RA, Aleshire N, Zibbell JE, et al. Increases in Drug and Opioid Overdose Deaths--United States, 
2000-2014. MMWR Morb Mortal Wkly Rep 2016;64:1378–82. 
2 
Buchanich JM, Balmert LC, Burke DS. Exponential Growth Of The USA Overdose Epidemic. bioRxiv 
Published Online First: 2017.https://www.biorxiv.org/content/early/2017/05/09/134403.abstract 
3 
Abuse S. … Services Administration (2017). Key substance use and mental health indicators in the 
United States: Results from the 2015 National Survey on Drug Use …. Abuse and Mental Health 
Services Administration … 
4 
Seth P, Scholl L, Rudd RA, et al. Overdose Deaths Involving Opioids, Cocaine, and Psychostimulants - 
United States, 2015-2016. MMWR Morb Mortal Wkly Rep 2018;67:349–58. 
5 
Substance Abuse and Mental Health Services Administration. Drug Abuse Warning Network, 2011: 
National Estimates of Drug-Related Emergency Department Visits. Published Online First: 
5/2013.https://www.samhsa.gov/data/sites/default/files/DAWN2k11ED/DAWN2k11ED/DAWN2k11ED
.pdf 
6 
Vivolo-Kantor AM, Seth P, Gladden RM, et al. Vital Signs: Trends in Emergency Department Visits for 
Suspected Opioid Overdoses - United States, July 2016-September 2017. MMWR Morb Mortal Wkly 
Rep 2018;67:279–85. 
7 
Sullivan LE, Fiellin DA. Narrative review: buprenorphine for opioid-dependent patients in office 
practice. Ann Intern Med 2008;148:662–70. 
8 
Larochelle MR, Bernson D, Land T, et al. Medication for Opioid Use Disorder After Nonfatal Opioid 
Overdose and Association With Mortality: A Cohort Study. Ann Intern Med Published Online First: 19 
June 2018. doi:10.7326/M17-3107 
9 
Kakko J, Svanborg KD, Kreek MJ, et al. 1-year retention and social function after buprenorphine-
assisted relapse prevention treatment for heroin dependence in Sweden: a randomised, placebo-
controlled trial. Lancet 2003;361:662–8. 

Page 20 of 22 
UH3 Version 1 (8/22/18) 
10 Mattick RP, Breen C, Kimber J, et al. Buprenorphine maintenance versus placebo or methadone 
maintenance for opioid dependence. Cochrane Database Syst Rev Published Online First: 2014. 
doi:10.1002/14651858.cd002207.pub4 
11 Sharfstein JM. The Opioid Crisis From Research to Practice. Milbank Q 2017;95:24–7. 
12 We have the tools needed to cut overdose death rates by half — but we aren’t using them. Paperpile. 
https://paperpile.com/app/p/964b2025-4e4d-0756-9b7a-d12f213481d7 (accessed 6 Aug 2018). 
13 Walsh L. Buprenorphine Waiver Management | SAMHSA - Substance Abuse and Mental Health 
Services Administration. https://www.samhsa.gov/programs-campaigns/medication-assisted-
treatment/training-materials-resources/buprenorphine-waiver (accessed 17 Jul 2018). 
14 Duber HC, Barata IA, Cioè-Peña E, et al. Identification, Management, and Transition of Care for 
Patients With Opioid Use Disorder in the Emergency Department. Ann Emerg Med Published Online 
First: 4 June 2018. doi:10.1016/j.annemergmed.2018.04.007 
15 Houry DE, Haegerich TM, Vivolo-Kantor A. Opportunities for Prevention and Intervention of Opioid 
Overdose in the Emergency Department. Ann Emerg Med 2018;71:688–90. 
16 Kawamoto K, Houlihan CA, Balas EA, et al. Improving clinical practice using clinical decision support 
systems: a systematic review of trials to identify features critical to success. BMJ 2005;330:765. 
17 Garg AX, Adhikari NKJ, McDonald H, et al. Effects of computerized clinical decision support systems on 
practitioner performance and patient outcomes: a systematic review. JAMA 2005;293:1223–38. 
18 Sittig DF, Wright A, Osheroff JA, et al. Grand challenges in clinical decision support. J Biomed Inform 
2008;41:387–92. 
19 Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: 
the nature of patient care information system-related errors. J Am Med Inform Assoc 2004;11:104–12. 
20 Ash JS, Sittig DF, Campbell EM, et al. Some unintended consequences of clinical decision support 
systems. AMIA Annu Symp Proc 2007;:26–30. 
21 Levin S, France DJ, Hemphill R, et al. Tracking workload in the emergency department. Hum Factors 
2006;48:526–39. 
22 Melnick ER, Nielson JA, Finnell JT, et al. Delphi consensus on the feasibility of translating the ACEP 
clinical policies into computerized clinical decision support. Ann Emerg Med 2010;56:317–20. 
23 Sirajuddin AM, Osheroff JA, Sittig DF, et al. Implementation pearls from a new guidebook on improving 
medication use and outcomes with clinical decision support. Effective CDS is essential for addressing 
healthcare performance improvement imperatives. J Healthc Inf Manag 2009;23:38–45. 
24 Phansalkar S, Edworthy J, Hellier E, et al. A review of human factors principles for the design and 
implementation of medication safety alerts in clinical information systems. J Am Med Inform Assoc 
2010;17:493–501. 
25 Horsky J, Phansalkar S, Desai A, et al. Design of decision support interventions for medication 
prescribing. Int J Med Inform 2013;82:492–503. 
26 Gellert G, Webster S, Gillean J, et al. Should US doctors embrace electronic health records? BMJ 
2017;356:j242. 
27 Roland M, Torgerson DJ. What are pragmatic trials? BMJ 1998;316:285. 
28 Murray DM, Varnell SP, Blitstein JL. Design and analysis of group-randomized trials: a review of recent 
methodological developments. Am J Public Health 2004;94:423–32. 
29 Li F, Lokhnygina Y, Murray DM, et al. An evaluation of constrained randomization for the design and 
analysis of group-randomized trials. Stat Med 2016;35:1565–79. 
30 Loudon K, Treweek S, Sullivan F, et al. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ 
2015;350:h2147. 
31 Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin 
Trials 2007;28:182–91. 

Page 21 of 22 
UH3 Version 1 (8/22/18) 
32 Hemming K, Haines TP, Chilton PJ, et al. The stepped wedge cluster randomised trial: rationale, design, 
analysis, and reporting. BMJ 2015;350:h391. 
33 Richesson RL, Hammond WE, Nahm M, et al. Electronic health records based phenotyping in next-
generation clinical trials: a perspective from the NIH Health Care Systems Collaboratory. J Am Med 
Inform Assoc 2013;20:e226–31. 
34 Hylan TR, Von Korff M, Saunders K, et al. Automated prediction of risk for problem opioid use in a 
primary care setting. J Pain 2015;16:380–7. 
35 Carrell DS, Cronkite D, Palmer RE, et al. Using natural language processing to identify problem usage of 
prescription opioids. Int J Med Inform 2015;84:1057–64. 
36 Reardon JM, Harmon KJ, Schult GC, et al. Use of diagnosis codes for detection of clinically significant 
opioid poisoning in the emergency department: A retrospective analysis of a surveillance case 
definition. BMC Emerg Med 2016;16:11. 
37 Mandel JC, Kreda DA, Mandl KD, et al. SMART on FHIR: a standards-based, interoperable apps platform 
for electronic health records. J Am Med Inform Assoc 2016;23:899–908. 
38 American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®). 
American Psychiatric Pub 2013. 
39 Wesson DR, Ling W. The Clinical Opiate Withdrawal Scale (COWS). J Psychoactive Drugs 2003;35:253–
9. 
40 Walsh L. Special Circumstances for Providing Buprenorphine | SAMHSA - Substance Abuse and Mental 
Health Services Administration. https://www.samhsa.gov/programs-campaigns/medication-assisted-
treatment/legislation-regulations-guidelines/special (accessed 13 Aug 2018). 
41 Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion 
interventions: the RE-AIM framework. Am J Public Health 1999;89:1322–7. 
42 Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion 
research to practice? Rethinking the efficacy-to-effectiveness transition. Am J Public Health 
2003;93:1261–7. 
43 LeBlanc A, Ruud KL, Branda ME, et al. The impact of decision aids to enhance shared decision making 
for diabetes (the DAD study): protocol of a cluster randomized trial. BMC Health Serv Res 2012;12:130. 
44 National Institutes of Health (NIH). National Institutes of Health (NIH). 
https://researchmethodsresources.nih.gov/SampleSizeCalculator.aspx (accessed 13 Aug 2018). 
45 Eldridge SM, Ashby D, Kerry S. Sample size for cluster randomized trials: effect of coefficient of 
variation of cluster size and analysis method. Int J Epidemiol 2006;35:1292–300. 
46 Raab GM, Butcher I. Balance in cluster randomized trials. Stat Med 2001;20:351–65. 
47 Gong MN, Schenk L, Gajic O, et al. Early intervention of patients at risk for acute respiratory failure and 
prolonged mechanical ventilation with a checklist aimed at the prevention of organ failure: protocol 
for a pragmatic stepped-wedged cluster trial of PROOFCheck. BMJ Open 2016;6:e011347. 
48 National Research Council, Division of Behavioral and Social Sciences and Education, Committee on 
National Statistics, et al. The Prevention and Treatment of Missing Data in Clinical Trials. National 
Academies Press 2010. 
49 Lachin JM. Statistical considerations in the intent-to-treat principle. Control Clin Trials 2000;21:167–89. 
50 Molenberghs G, Thijs H, Jansen I, et al. Analyzing incomplete longitudinal clinical trial data. Biostatistics 
2004;5:445–64. 
51 McKinney RE Jr, Beskow LM, Ford DE, et al. Use of altered informed consent in pragmatic clinical 
research. Clin Trials 2015;12:494–502. 
52 Office for Human Research Protections (OHRP). 45 CFR 46. HHS.gov. 
2016.https://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/index.html (accessed 16 
Aug 2018). 

Page 22 of 22 
UH3 Version 1 (8/22/18) 
53 Lantos JD, Wendler D, Septimus E, et al. Considerations in the evaluation and determination of 
minimal risk in pragmatic clinical trials. Clin Trials 2015;12:485–93. 
54 Suffoletto B, Lynch M, Pacella CB, et al. The Effect of a Statewide Mandatory Prescription Drug 
Monitoring Program on Opioid Prescribing by Emergency Medicine Providers Across 15 Hospitals in a 
Single Health System. J Pain 2018;19:430–8.