Application of Economics & Social Psychology to Improve Opioid Prescribing Safety (AESOPS): R21 Pilot Phase
Phase: NA
Conditions: Opioid Use, Unspecified
Sponsor: University of Southern California

1 
 
 
 
STUDY PROTOCOL 
Application of Economics & Social psychology to improve Opioid Prescribing Safety (AESOPS): 
R21 pilot 
 
Co-Principal Investigators: 
 
Jason N. Doctor, Ph.D., Associate Professor, University of Southern California 
Mark Sullivan, Professor, University of Washington 
 
 
Supported by: 
 
The National Institute on Aging 
 
An R21 protocol preceding 4R33AG057395 
Clinicaltrials.gov identifier:  NCT03773484 
 
Version 3   (September 30, 2021)

2 
 
 
TABLE OF CONTENTS 
STUDY TEAM ROSTER 
5 
PARTICIPATING STUDY SITES 
6 
PRÉCIS 
7 
1. Study objectives 
9 
1.1 Primary Objective 
9 
1.2 Secondary Objectives 
9 
2. BACKGROUND AND RATIONALE 
9 
2.1 Background on Condition, Disease, or Other Primary Study Focus 
9 
2.2 Study Rationale 
10 
3. STUDY DESIGN 
11 
4. SELECTION AND ENROLLMENT OF PARTICIPANTS 
11 
4.1 Inclusion Criteria 
11 
4.2 Exclusion Criteria 
12 
4.3 Study Enrollment Procedures 
12 
5. STUDY INTERVENTIONS 
12 
5.1 Interventions, Administration, and Duration 
12 
5.2 Handling of Study Interventions 
12 
5.3 Adherence Assessment 
13 
6. STUDY PROCEDURES 
13 
6.1 Schedule of Evaluations 
13 

3 
 
6.2 Description of Evaluations 
14 
6.2.1 
14 
6.2.2 
14 
7. SAFETY ASSESSMENTS 
14 
7.1 Specification of Safety Parameters 
15 
7.2 Methods and Timing for Assessing, Recording, and Analyzing Safety Parameters 
15 
7.3 Adverse Events 
15 
7.4 Reporting Procedures 
15 
7.5 Safety Monitoring 
15 
8. INTERVENTION DISCONTINUATION 
15 
9. STATISTICAL CONSIDERATIONS 
16 
9.1 General Design Issues 
16 
9.2 Sample Size and Randomization 
17 
9.2.1 
                                                                                                                                                                                                18 
9.3 Interim analyses and Stopping Rules 
18 
9.4 Outcomes 
19 
9.4.1 
19 
9.4.2 
19 
 
9.5 Data Analyses 
20 
10. DATA COLLECTION AND QUALITY ASSURANCE 
20 
10.1 Data Collection Forms 
20 
10.2 Data Management 
20 
10.3 Quality Assurance 
10.3.1 
20 
20 

4 
 
 
10.3.2 
20 
10.3.3 
21 
10.3.4 
21 
10.3.5 
21 
 
11. PARTICIPANT RIGHTS AND CONFIDENTIALITY 
21 
11.1 Institutional Review Board (IRB) Review 
21 
11.2 Informed Consent Forms 
21 
11.3 Participant Confidentiality 
21 
11.4 Study Discontinuation 
21 
12. COMMITTEES 
24 
13. PUBLICATION OF RESEARCH FINDINGS 
24 
14. REFERENCES 
24 
15. SUPPLEMENTS/APPENDICES 
24 
Appendix A. Opioid Prescribing Diagnoses Exclusions and Medications for Outcome Assessments 
24 

5 
 
STUDY TEAM ROSTER 
Jason N. Doctor, Ph.D. (Principal Investigator), Leonard D. Schaeffer Center for Health Policy and Economics, University of 
Southern California, 3335 S. Figueroa Street, Unit A, Los Angeles, CA 90089-7273, Office: 213.821.7943, Fax: 213.740.3460, 
jdoctor@usc.edu 
Mark Sullivan, M.D., Ph.D. (Co-Principal Investigator) University of Washington, Department of Psychiatry & Behavioral Sciences, 
1959 NE Pacific Street, Box 356560, Room BB1644, Seattle, WA 98195-6560 
sullimar@uw.edu 
Daniella Meeker, Ph.D (Co-Investigator), University of Southern California, 2250 Alcazar Street Health Sciences Campus 
Los Angeles 90033, 
dmeeker@usc.edu 
Stephen D. Persell, MD, MPH, .(Co-Investigator), Division of General Internal Medicine and Geriatrics, Institute for Healthcare 
Studies, Feinberg School of Medicine, Northwestern University, 750 N. Lake Shore Drive, 10th Floor, Chicago, IL 60611, 
spersell@nmff.org 
Jeffrey A. Linder, MD, MPH, FACP (Co-Investigator), Division of General Internal Medicine and Geriatrics, Institute for Healthcare 
Studies, Feinberg School of Medicine, Northwestern University, 750 N. Lake Shore Drive, 10th Floor, Chicago, IL 60611, 
jlinder@partners.org 
Mark Friedberg, MD, MPP (Co-Investigator), BlueCross BlueShield of Massachusetts, Boston, MA  02116, 
Mark.Friedberg@bcbsma.com 
Craig R. Fox, PhD (Co-Investigator), UCLA Anderson School of Management, 110 Westwood Plaza D-511, Los Angeles, CA 90095, 
craig.fox@anderson.ucla.edu 
Noah J. Goldstein, Ph.D. (Co-Investigator), UCLA Anderson School of Management, 110 Westwood Plaza, A-412, Los Angeles, CA 
90095, noah.goldstein@anderson.ucla.edu 
Tara K. Knight, Ph.D. (Project Manager) Leonard D. Schaeffer Center for Health Policy and Economics, University of Southern 
California, 3335 S. Figueroa Street, Unit A, Los Angeles, CA 90089-7273, Office: 213.821.7943, Fax: 213.740.3460, knight@usc.edu 

6 
 
PARTICIPATING STUDY SITES 
Northwestern University. Jeffrey Linder, M.D., M.P.H., FACP and Stephen D. Persell, M.D., M.P.H., Division of General Internal 
Medicine and Geriatrics, Institute for Healthcare Studies, Feinberg School of Medicine, Northwestern University, 750 N. Lake Shore 
Drive, 10th Floor, Chicago, IL 60611, jlinder@nmff.org spersell@nmff.org 
 
 

7 
 
 
PRÉCIS 
 
Study Title 
 
Application of Economics & Social psychology to improve Opioid Prescribing Safety (AESOPS): R21 pilot 
 
Objectives 
There is a lack of evidence that long-term opioid use offers benefit for noncancer pain and an abundance of evidence of harm. 
The objective of the Application of Economics & Social psychology to improve Opioid Prescribing Safety (AESOPS) study is 
to develop and test novel behavioral nudges to encourage adherence to Oregon Pain Guidance and CDC guidelines for opioid 
prescribing for persons with noncancer pain. Once we have successfully developed different nudges in the R21 phase, we will 
then compare these nudge interventions in a future cluster randomized trial. 
 
Design and Outcomes 
The purpose of this pilot project is to test EHR behavioral interventions to encourage safer opioid prescribing during ambulatory 
visits when pain is a symptom. Changes in opioid prescribing among primary care clinicians will be measured for three types of patients: 
1): Opioid naïve: Visit where the order is for an included opioid and there is no prior opioid prescription with a start date of 
greater than 1 day and less than 91 days, 2) At-risk for long term use: Visit where the order is for an included opioid there is a 
prior opioid prescription with a start date greater than 1 day and less than 91 days, and there is no prior opioid prescription with a 
start date greater than 90 days, and 3) Long-term opioid recipient: Total opioid doses are at least 50 MME per day, there are 
two or more prior opioid prescriptions with two different start dates both greater than 1 day and less than 91 days, and there is a 
prior opioid prescription with a start date greater than 90 days and less than 181 days. 
Outcome Measure: The primary outcome is average weekly morphine milligram equivalents (MME) prescribed per-clinician. 

8 
 
 
1. 
STUDY OBJECTIVES 
 
1.1 
Primary Objective 
The objective of the R21 pilot phase of the Application of Economics & Social psychology to improve Opioid Prescribing Safety 
(AESOPS) study is to develop and test novel behavioral nudges to encourage adherence to pain and CDC guidelines for opioid 
prescribing for persons with noncancer pain. At the time of opioid prescribing, clinicians will be prompted with an EHR nudge when the 
prescribing history for the patient falls into one of the following three mutually exclusive categories: Opioid naïve, At-risk for long term 
use, or Long-term opioid recipient. The primary outcome is average weekly morphine milligram equivalents (MME) prescribed per-
clinician.  
 
2. 
BACKGROUND AND RATIONALE 
 
2.1 
Background on Condition, Disease, or Other Primary Study Focus 
Over the last two decades, prescription opioids have grown to become a public health crisis. Today, on average 1 in 5 persons with 
chronic noncancer pain receives an opioid prescription in the U.S.1 Yet, despite this record level of prescribing, reports of pain in America 
have not gone down.10,11 The greater availability of prescription opioids has been accompanied by alarming rise in the negative 
consequences related to their use. In 2015, there were over 365,000 emergency department visits for misuse and 20,101 prescription 
overdose deaths, more than have ever been recorded in U.S. history.5 The costs of prescription opioids are staggering. Aggregate costs 
for prescription opioid harms are estimated at over $78.5 billion (in 2013 dollars). One-fourth of the aggregate economic burden is 
publicly funded (i.e., Medicaid, Medicare, and veterans' programs).12,13 
In 2016, the CDC issued the “CDC Guideline for Prescribing Opioids for Chronic Pain” which gives 12 patient care recommendations: 
(1) use alternatives to opioids, (2) set realistic goals for pain and function, (3) discuss opioid risks upon therapy initiation, (4) use 
immediate release agents instead of long-acting ones, (5) use lowest effective starting dose, avoid escalation above 50 milligram (mg) 
morphine equivalent (ME) daily dose and abide by a 90 mg ME/day dose prescribing limit, (6) prescribe lowest dose at lowest quantity 
needed for acute pain—usually 3 days with a maximum of 7 days in rare cases, (7) evaluate for and discuss tapering within 1 - 4 weeks 
and after 90 days, (8) prescribe naloxone and evaluate substance use history with opioid prescription, (9) review the state prescription 
drug monitoring program data (10)  conduct urine drug tests to provide information about drug use that is not reported by the  patient, 
(11) avoid co-prescribing with benzodiazepines, and (12) refer patients with opioid use disorder to medication-assisted treatment in 
combination with behavioral therapies. Our objective is to increase adherence to the CDC guideline and Oregon Pain Guidance pain 
management guideline. To evaluate the effects of doing so, we will measure as a primary outcomes clinician aggregate monthly mg ME 

9 
 
 
for patient visits with >50 mg ME daily dose. Secondary outcome is rate of dose escalation to >50 mg ME/day. 
 
2.2 
Study Rationale 
Rationale for Accountable Justifications. In the Accountable Justifications intervention, clinicians will be prompted to record an explicit 
justification for why they are prescribing an opioid that appears in the patient’s EHR. Accountable justifications incorporate several 
behavioral principles. First, they signal an injunctive norm (a norm, often provided by an authoritative source, that strongly indicates how 
people should behave) indicating that prescribing an opioid is not recommended. This may make the clinician more likely to believe both that 
not prescribing an opioid is the best medical decision and that prescribing when it is not indicated violates professional standards. Second, it 
incorporates social accountability. A clinician justification become an explicit, separate part of the medical record, so a clinician’s decision to 
prescribe is subject to the review and judgment of the provider’s peers. Third, the justification alert implicitly designates guideline-
concordant prescribing as the default action. Defaults are options that are exercised if the decision maker takes no special action to opt in or 
out of a given choice. Prior to our intervention, choosing to deviate from guidelines did not carry a special requirement to document a clinical 
rationale in the EHR. Accountable justifications, therefore, reset the default action. Guideline-concordant treatment choices will not require 
special justification, but a clinician must now “opt-in” to prescribing an opioid by providing a justification for which they are accountable. 
 
3. 
STUDY DESIGN 
The Application of Economics & Social psychology to improve Opioid Prescribing Safety (AESOPS) pilot study involves 41 Northwestern 
Medicine clinicians for which the primary aim is to test the feasibility of three interventions based on behavioral economic principles to reduce 
the reliance on opioids and encourage safe and effective pain management. The study period includes a 34 week baseline period and 34 week 
study intervention period. The primary outcome is average weekly morphine milligram equivalents (MME) prescribed per-clinician. 
 
4. 
SELECTION AND ENROLLMENT OF PARTICIPANTS 
 
4.1 
Inclusion Criteria 
The subjects involved in this trial are clinicians from one clinical site in Illinois. Clinicians must meet the following inclusion criteria 
to participate in this study: 1) treat adult patients and 2) practice at one of the study clinics. 
 
An office visit is eligible for inclusion in the outcome denominator if: 1) the patient was 18 years old or older, 2) the provider and 
practice site were enrolled in the study, and 3) the visit occurred during the 6-month intervention period. 
 
4.2 
Exclusion Criteria 
Visits will be excluded from the primary analysis when they have active cancer. Cancer exclusions (ICD-10 codes) are listed here. 

10 
 
 
4.3 
Study Enrollment Procedures 
We will not be enrolling clinicians. We are approved for a waiver of consent. 
 
5. 
STUDY INTERVENTIONS 
 
5.1 
Interventions, Administration, and Duration 
The intervention period will be 34 weeks in length for all participants. 
 
5.2 
Handling of Study Interventions 
The following interventions will be tested: Accountable Justification (AJ) is an EHR-based intervention that will prompt the 
clinician to justify, in a free text response, the decision to prescribe an opioid or to not reduce high risk/high dose prescribing. The 
prompt is designed to inform the clinician that the justification will be seen by others in the patient’s medical record as an “High risk 
prescribing justification” note, and that if no justification is entered, the phrase “no justification given” will appear in the note. 
 
Delivery of AJ nudges in Epic’s electronic health record (EHR) based on patient type is described below. 
Opioid-naïve: An automated search of EHR data determines whether the patient received an opioid prescription within 90 days. If no 
opioid was prescribed, the clinician receives an alert cautioning against initiating a prescription based on current guidelines. The 
clinician can either remove or continue with the current opioid prescription. The former results in return to normal workflow after 
receiving a prompt recommending physical therapy and non-opioid alternatives. The latter, however, induces a prompt requiring the 
clinician to justify their decision in free text. A Justification Note with the clinician’s response is recorded in the Visit Summary portion 
of the patient’s record. A generic “No justification given” is generated for blank responses.  
At-risk for long term use: If an automated search of EHR data reveals an opioid prescription(s) within 90 days of the current order, but 
no previous opioid prescription prior to 90 days before, a prompt explaining the danger of continuing refills based on current guidelines 
appears. The process is otherwise the same as above; clinicians who cancel a prescription receive a prompt recommending non-opioid 
alternatives and physical therapy, whereas those who continue with the order are required to justify their decision.   
Long term opioid recipient: If an automated search of EHR data reveals total opioid doses are at least 50 MME per day, there are two or 
more prior opioid prescriptions with two different start dates both greater than 1 day and less than 91 days, and there is a prior opioid 
prescription with a start date greater than 90 days and less than 181 days,  a prompt suggesting an opioid taper appears. 
 
 
5.3 
Adherence Assessment 
In order to ensure that the study interventions are being reliably delivered we will create testing scripts that cover logical and coding 

11 
 
variation in EHR-based interventions. Study staff will conduct best practice alerts reports during the intervention to ensure that tests 
do not fail. 
 
6. 
STUDY PROCEDURES 
 
6.1 
Schedule of Evaluations 
 
Assessment 
Screening: 
Baseline 
prescribing 
Baseline, 
Enrollment, 
Intervention 
start (Month 
1) 
Continuously 
Measured or 
monitored 
Intervention 
end: (Month 
6) 
 
 
Clinician-level 
Assessments 
Demographics 
X 
 
Inclusion/Exclusion 
Criteria 
X 
X 
Visit-level assessments 
ICD-10 codes 
X 
X 
X 
X 
X 
Ordering Data 
X 
X 
X 
X 
X 
Adverse Events 
X 
X 
X 
 
6.2 
Description of Evaluations 
6.2.1 
Screening Evaluation 
Consenting Procedure 
The study investigators will seek a waiver of consent. 
 
6.2.2 
Enrollment, Baseline, and/or Randomization 
Enrollment 
Eligible clinicians from one pilot clinic will receive study interventions. 

12 
 
 
Baseline Assessments 
 
None 
 
Randomization 
None 
 
7. 
SAFETY ASSESSMENTS 
Data for patients who were noted to have been abruptly stopped of opioid prescriptions, emergency room eligible study visit with a 
diagnosis that could represent a serious complication of untreated pain will be extracted from study site EHRs and reported to the Data 
Safety and Monitoring Board. Relative rate of ED visits between study conditions will also be evaluated. 
 
7.1 
Specification of Safety Parameters 
Data elements from qualifying visits for providers enrolled in the study will be collected from the electronic health record.  Clinics 
will incorporate exclusions (ICD 10 cancer suppressor codes found here) used in the decision to trigger the clinical decision support. 
Aggregate counts of total visits across sites for which the intervention was triggered, for high dose opioid patients, if abrupt changes to 
dose (> 20% morphine equivalent daily dose) were made. Such cases will be examined closely to determine if unsafe drops in opioids 
occurred. 
 
7.2 
Methods and Timing for Assessing, Recording, and Analyzing Safety Parameters 
Reports of our safety measures will be delivered to our Data Safety Monitoring Board. 
 
7.3 
Adverse Events 
Per CDC guideline clarification, adverse events are defined as an abrupt discontinuation of opioids for persons whose most recent 
prescription exceeds > 49 morphine equivalent daily dose; or as reported to study staff.(Dowell et al. 2019) Emergency department 
visits will also be evaluated as well as increases in prescribing > 20% presumably in response to reports of worsening pain. 
 
7.4 
Reporting Procedures 
The Principal Investigator will report any unanticipated events to the IRB as well as the Data Safety and Monitoring Board (DSMB) 
assembled for this study. When notified of an unanticipated event, the DSMB will convene and make a decision as to whether the 
study should continue. The IRB will also be notified of the DSMB's decision. 
 

13 
 
7.5 
Safety Monitoring 
A Data Safety and Monitoring Board (DSMB) has been established by NIH. The following individuals were recommended to NIH 
with expertise in either opioids, overprescribing or biostatistics/research methods: Joe Frank, Jeanmarie Perrone, and Carl Pieper. 
 
8. 
INTERVENTION DISCONTINUATION 
Following each DSMB meeting, the board will make recommendations to the local IRBs as to whether the study should continue or if 
changes to the protocol are necessary for continuation. 
 
9. 
STATISTICAL CONSIDERATIONS 
 
9.1 
General Design Issues 
Hypotheses 
Our primary hypothesis is that clinicians receiving behavioral economic interventions will have lower opioid prescribing rates post-
intervention as compared to baseline. We will estimate weekly milligram morphine equivalent for each clinician and observe the change 
each month. This will capture CDC guideline prescribing endpoints. 
Design 
The Application of Economics & Social psychology to improve Opioid Prescribing Safety (AESOPS) R21 is a pilot study to test the feasibility 
of clinical decision support nudges within the electronic health record to discourage unnecessary opioid prescribing through the application of 
"behavioral insights"— empirically-tested social and psychological interventions that affect choice. Over a 34 week study period, participating 
clinicians will receive any of three nudges when eligibility criteria are met within a patient's chart: 1): Opioid naïve: Visit where the order is for 
an included opioid and there is no prior opioid prescription with a start date of greater than 1 day and less than 91 days, 2) At-risk for long 
term use: Visit where the order is for an included opioid there is a prior opioid prescription with a start date greater than 1 day and less than 91 
days, and there is no prior opioid prescription with a start date greater than 90 days, and 3) Long-term opioid recipient: Total opioid doses are 
at least 50 MME per day, there are two or more prior opioid prescriptions with two different start dates both greater than 1 day and less than 91 
days, and there is a prior opioid prescription with a start date greater than 90 days and less than 181 days. 
 
Outcome Measures 
The primary outcome is change in weekly milligram morphine equivalent from baseline to post intervention period. We will estimate weekly 
milligram morphine equivalent for each clinician and observe the change each month. This will capture CDC guideline prescribing endpoints. 
 
9.2 
Sample Size and Randomization 
Not applicable as this is a pilot feasibility study within one clinic with no randomization. 
 

14 
 
 
 
9.2.1 
Treatment Assignment Procedures 
Randomization of study sites 
Not applicable as this is a pilot feasibility study within one clinic with no randomization. 
 
9.3 
Interim analyses and Stopping Rules 
No interim analysis will be conducted on primary or secondary outcomes. The Data Safety and Monitoring Board is granted the power to 
recommend discontinuation of the study to each study IRB, if safety concerns are found.  
 
9.4 
Outcomes 
9.4.1 
Primary outcome 
The primary outcome is change in weekly milligram morphine equivalent over 6 months. We will estimate weekly milligram morphine 
equivalent for each clinician and observe the change each month. This will capture CDC guideline prescribing endpoints. 
9.4.2 
Secondary outcomes 
None 
 
9.5 
Data Analyses 
For sample descriptive statistics, we will use means and medians for continuous measures, frequencies for count data, and standard deviations 
and interquartile ranges for variance. For inferential analysis, our primary hypothesis is that milligram morphine equivalent dose will decrease for 
persons on high doses of opioid therapy.  
 
10. 
Statistical Analysis  
Measures   
 
The primary outcome is average weekly morphine milligram equivalents (MME) prescribed per-clinician. We assessed change in average per-
clinician weekly MME from the 34-week baseline period before, to the 34-week period post-intervention using a mixed-effects, knotted spline, 
left censored regression to account for baseline differences in prescribing, a non-linear change in the rate of opioid prescribing from baseline to 
post-intervention, and an inflated number of weeks with an average MME of zero, respectively. Predictors include the fixed effects time in 
weeks, time with a knot at the intervention start, and a random intercept for clinician. The density function to achieve maximum likelihood 
estimates for a left censored regression is the observed probability of yi assuming normal distribution (xi𝛽𝛽, 𝜎𝜎2)  given that yi > 0, weighted by 
the probability that yi is above a censoring threshold, in this case zero.2 Because uncensored observations are often skewed, we used log average 
weekly MME as our dependent variable. The percentage change in log MME per one unit increase in the uncensored, or latent variable was 

15 
 
measured using: 
 
 
 
where log(MME)* is the latent, uncensored variable, time is the number of weeks, t* is the spline knot for the intervention start, and 𝜁𝜁 is the 
clinician random effect.1 We adjusted the coefficients by calculating the average partial effect to interpret not just the linear change in 
log(MME*), but the non-linear likelihood that log(MME*) is uncensored, resulting in comparable estimates to ordinary least squares.2  
 
 
11. 
DATA COLLECTION AND QUALITY ASSURANCE 
11.1 
Data Collection Forms 
Data from electronic medical and billing records will be collected at baseline and post intervention. 
 
11.2 
Data Management 
Northwestern Medicine will create an extract from their Electronic Medical or Billing Records of the Data Elements. These 
records will be transferred to USC coordinating center at the end of the study. The CC has created programs and quality control 
queries for transforming all of the data into a standard model (Observational Medical Outcomes Partnership Common Data 
Model, version 5.1). The electronic data system, Epic, will have native data capture formats. 
 
11.3 
Quality Assurance 
11.3.1 Training 
Staff will be trained on the permissible values present in Electronic Records, frequency of update, and expected volumes of 
data. 
 
11.3.2 Quality Control Committee 
The quality control committee will consist of practicing clinicians from Northwestern Medicine. They will review 
automatically refreshing dashboards for potential deviations in coding systems and appropriate values for codes for 
inclusion in the outcome measures.  
 
11.3.3 Metrics 
Quality control metrics will be based on reports verifying visits were not for cancer exclusions. All drugs prescribed at these 
visits will be categorized as “opioid” or “non-opioid”.  
 
11.3.4 Protocol Deviations 

16 
 
Our task tracking system, Monday.com will be used to track and document issues. Each issue will include both an 
assignee and a reviewer. 
 
11.3.5 Monitoring 
In addition to data quality reviews, we will also review the integrity of the interventions. On an approximately quarterly basis, 
staff will verify functionality of decision support tools. Additionally, practicing clinicians on our study team will have the 
ability to monitor electronic medical record interventions in their own health systems. 
 
12. 
PARTICIPANT RIGHTS AND CONFIDENTIALITY 
 
12.1 
Institutional Review Board (IRB) Review 
The study protocol has been reviewed and approved by the University of Southern California’s Institutional Review Board (IRB). 
Northwestern Medicine will rely on the USC IRB through the SMART IRB online reliance system. 
 
12.2 
Informed Consent Forms 
We will seek a waiver of consent for this study. 
 
12.3 
Participant Confidentiality 
Data will be recorded with SSL protected web sites to a data warehouse, and transferred over secure network protocol. Data will be 
kept in encrypted files on computers in locked offices at USC Schaeffer Center facilities. Only Northwestern Medicine study 
investigators will have access to a list of study ID codes that will be traceable back to actual subject contact identifiers for clinicians. 
These codes will be kept in locked offices at USC Schaeffer Center facilities. 
12.4 
Study Discontinuation 
Following each DSMB meeting, the board will make recommendations to the local IRBs as to whether the study should continue or if 
changes to the protocol are necessary for continuation. 
 
13. 
COMMITTEES 
Data Safety Monitoring Board: Joseph Frank, Jeanmarie Perrone, Carl Pieper 
 
14. 
PUBLICATION OF RESEARCH FINDINGS 

17 
 
Publication of results from our research will follow the NIH Public Access Policy, which requires that we submit to the National 
Library of Medicine’s PubMed Central an electronic version of final, peer-reviewed manuscripts upon acceptance for publication, to 
be made publicly available no later than 12 months after the official date of publication. 
 
15. 
REFERENCES 
1. 
Grijalva C, Nuorti J, Griffin M. Antibiotic prescription rates for acute respiratory tract infections in US ambulatory settings. 
JAMA. 2009;302(7):758-766. 
2. 
Gonzales R, Malone D, Maselli J, Sande M. Excessive antibiotic use for acute respiratory infections in the United States. Clin 
Infect Dis. 2001;33:757-762. 
3. 
Shehab N, Patel P, Srinivasan A, Budnitz D. Emergency department visits for antibiotic-associated adverse events. Clin Infect 
Dis. 2008;47(735-743). 
4. 
Arias C, Murray B. Antibiotic-resistant bugs in the 21st century—a clinical super-challenge. N Engl J Med. 2009;360:439-443. 
5. 
Klevens R, Morrison M, Nadle J, al e. Invasive methicillin-resistant Staphylococcus aureus infections in the United States. 
JAMA. 2007;298:1763-1771. 
6. 
Lerner J, Tetlock P. Accounting for the effects of accountability. Psych Bull. 1999;125(2):255. 
7. 
Johnson EJ, & Goldstein, D. G. (2003). Do defaults save lives? Science, 302, 1338-1339. 
8. 
Madrian B, & Shea, D. (2001). The power of suggestion. Quarterly Journal of Economics, 18-116. 
9. 
Thaler R, Sunstein C. Nudge: Improving decisions about health, wealth, and happiness. Yale University Press.; 2008. 
10. 
Macfarlane J, Holmes, W., Macfarlane, R., & Britten, N. (1997). Influence of patients’ expectations on antibiotic management 
of acute lower respiratory tract illness in general practice: Questionnaire study. British Medical Journal, 315, 1211-1214. 
11. 
Cialdini R, Reno R, Kallgren C. A focus theory of normative conduct: recycling the concept of norms to reduce littering in 
public places. J Pers Soc Psych. 1990;58(6):1015. 
12. 
Cialdini RB. Influence: Science and Practice.  5th Edition. Boston: Allyn & Bacon; 2010. 
13. 
Goldstein N, Cialdini R, Griskevicius V. A room with a viewpoint: Using social norms to motivate environmental conservation 
in hotels. J Consum Res. 2008;35:472-482. 
14. 
Kiefe C, Allison J, Williams O, Person S, Weaver M, Weissman N. Improving quality improvement using achievable 
benchmarks for physician feedback: a randomized controlled trial. JAMA. 2001;285(22):2871-2879. 
15. 
Ihaka R GR. A language for data analysis and graphics. J Comput Graph Stat 1996;5:299-314. 
16. 
Chan KS, Fowles, J. B., & Weiner, J. P. (2010). Electronic health records and reliability and validity of quality measures: a 
review of the literature.Medical Care Research and Review. 
17. 
Kish L. Survey Sampling. New York, NY: John Wiley and Sons; 1965. 

18 
 
15. 
SUPPLEMENTS/APPENDICES 
 
APPENDIX A. OPIOID PRESCRIBING DIAGNOSES EXCLUSIONS AND MEDICATIONS FOR OUTCOMES ASSESSMENTS 
 
Opioids Trigger List 
Opioids Outcome List 
Diagnosis of Exclusion 
Non-Opioid Dosing for Alternatives 
Link 
Link 
Link 
Link 
("Outcome" Tab) 
("Grouper" Tab)