Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in Los Angeles
Phase: NA
Conditions: Substance-Related Disorders
Sponsor: University of Southern California

 
 
Roybal Pilot 1: Comparative Effectiveness of Two Letters to Encourage Judicious 
Prescribing of Opioids: A County-wide Project in Los Angeles 
National Clinical Trial (NCT) Identified Number:  NCT03856593 
Principal Investigator: Jason Doctor, PhD 
Sponsor: National Institutes of Health 
Title of Grant: Roybal Center for Behavioral Interventions in Aging 
Grant #: P30AG024968 
Funded by: NIA 
Version Number:  v.1.0 
April 2019 
 
 

NIH Protocol Template for Behavioral and Social Sciences Research Involving Humans 
CONFIDENTIALITY STATEMENT 
This document is confidential communication. Acceptance of this document constitutes agreement by the 
recipient that no unpublished information contained herein will be published or disclosed without prior 
approval of the Principal Investigator or other participating study leadership and as consistent with the 
NIH terms of award.  
 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
NIH Protocol Template for Behavioral and Social Sciences Research Involving Humans 
Table of Contents 
STATEMENT OF COMPLIANCE ............................................................................................................................. 1 
INVESTIGATOR’S SIGNATURE .............................................................................................................................. 1 
1 
PROTOCOL SUMMARY ............................................................................................................................... 2 
1.1 
Synopsis ................................................................................................................................................ 2 
1.2 
Schema ................................................................................................................................................. 3 
1.3 
Schedule of Activities .......................................................................................................................... 3 
2 
INTRODUCTION ............................................................................................................................................ 4 
2.1 
Study Rationale .................................................................................................................................... 4 
2.2 
Background........................................................................................................................................... 5 
2.3 
Risk/Benefit Assessment .................................................................................................................... 5 
2.3.1 
Known Potential Risks ..................................................................................................... 6 
2.3.2 
Known Potential Benefits ................................................................................................ 6 
2.3.3 
Assessment of Potential Risks and Benefits ................................................................ 6 
3 
OBJECTIVES AND ENDPOINTS ................................................................................................................ 6 
4 
STUDY DESIGN ............................................................................................................................................. 6 
4.1 
Overall Design ...................................................................................................................................... 7 
4.2 
Scientific Rationale for Study Design................................................................................................ 7 
4.3 
Justification for Intervention ............................................................................................................... 7 
4.4 
End-of-Study Definition ....................................................................................................................... 8 
5 
STUDY POPULATION .................................................................................................................................. 8 
5.1 
Inclusion Criteria .................................................................................................................................. 9 
5.2 
Exclusion Criteria ................................................................................................................................. 9 
5.3 
Lifestyle Considerations ...................................................................................................................... 9 
5.4 
Screen Failures .................................................................................................................................... 9 
5.5 
Strategies for Recruitment and Retention ........................................................................................ 9 
6 
STUDY INTERVENTION(S) OR EXPERIMENTAL MANIPULATION(S) .............................................. 9 
6.1 
Study Intervention(s) or Experimental Manipulation(s) Administration ........................................ 9 
6.1.1 
Study Intervention or Experimental Manipulation Description ................................... 9 
6.1.2 
Administration and/or Dosing ....................................................................................... 10 
6.2 
Fidelity ................................................................................................................................................. 10 
6.2.1 
Interventionist Training and Tracking .......................................................................... 10 
6.3 
Measures to Minimize Bias: Randomization and Blinding ........................................................... 10 
6.4 
Study Intervention/Experimental Manipulation Adherence.......................................................... 10 
6.5 
Concomitant Therapy ........................................................................................................................ 10 
6.5.1 
Rescue Therapy ............................................................................................................. 10 
7 
STUDY INTERVENTION/EXPERIMENTAL MANIPULATION DISCONTINUATION AND 
PARTICIPANT DISCONTINUATION/WITHDRAWAL ......................................................................................... 11 
7.1 
Discontinuation of Study Intervention/Experimental Manipulation ............................................. 11 
7.2 
Participant Discontinuation/Withdrawal from the Study ............................................................... 11 
7.3 
Lost to Follow-Up ............................................................................................................................... 11 
8 
STUDY ASSESSMENTS AND PROCEDURES ..................................................................................... 11 
8.1 
Endpoint and Other Non-Safety Assessments .............................................................................. 11 
8.2 
Safety Assessments .......................................................................................................................... 11 
8.3 
Adverse Events and Serious Adverse Events ............................................................................... 12 
8.3.1 
Definition of Adverse Events ........................................................................................ 12 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
NIH Protocol Template for Behavioral and Social Sciences Research Involving Humans 
8.3.2 
Definition of Serious Adverse Events .......................................................................... 12 
8.3.3 
Classification of an Adverse Event .............................................................................. 12 
8.3.4 
Time Period and Frequency for Event Assessment and Follow-Up ....................... 13 
8.3.5 
Adverse Event Reporting .............................................................................................. 14 
8.3.6 
Serious Adverse Event Reporting ............................................................................... 15 
8.3.7 
Reporting Events to Participants ................................................................................. 15 
8.3.8 
Events of Special Interest ............................................................................................. 15 
8.3.9 
Reporting of Pregnancy ................................................................................................ 15 
8.4 
Unanticipated Problems .................................................................................................................... 15 
8.4.1 
Definition of Unanticipated Problems .......................................................................... 15 
8.4.2 
Unanticipated Problems Reporting .............................................................................. 15 
8.4.3 
Reporting Unanticipated Problems to Participants ................................................... 15 
9 
STATISTICAL CONSIDERATIONS .......................................................................................................... 16 
9.1 
Statistical Hypotheses ....................................................................................................................... 16 
9.2 
Sample Size Determination .............................................................................................................. 16 
9.3 
Populations for Analyses .................................................................................................................. 16 
9.4 
Statistical Analyses............................................................................................................................ 16 
9.4.1 
General Approach .......................................................................................................... 16 
9.4.2 
Analysis of the Primary Endpoint(s) ............................................................................ 17 
9.4.3 
Analysis of the Secondary Endpoint(s) ....................................................................... 17 
9.4.4 
Safety Analyses .............................................................................................................. 17 
9.4.5 
Baseline Descriptive Statistics ..................................................................................... 17 
9.4.6 
Planned Interim Analyses ............................................................................................. 17 
9.4.7 
Sub-Group Analyses ..................................................................................................... 17 
9.4.8 
Tabulation of Individual Participant Data .................................................................... 18 
9.4.9 
Exploratory Analyses ..................................................................................................... 18 
10 
SUPPORTING DOCUMENTATION AND OPERATIONAL CONSIDERATIONS .............................. 18 
10.1 
Regulatory, Ethical, and Study Oversight Considerations ........................................................... 18 
10.1.1 
Informed Consent Process ........................................................................................... 18 
10.1.2 
Study Discontinuation and Closure ............................................................................. 18 
10.1.3 
Confidentiality and Privacy ........................................................................................... 19 
10.1.4 
Future Use of Stored Specimens and Data ............................................................... 19 
10.1.5 
Key Roles and Study Governance .............................................................................. 20 
10.1.6 
Safety Oversight ............................................................................................................. 20 
10.1.7 
Clinical Monitoring .......................................................................................................... 20 
10.1.8 
Quality Assurance and Quality Control ....................................................................... 20 
10.1.9 
Data Handling and Record Keeping ............................................................................ 21 
10.1.10 
Protocol Deviations ........................................................................................................ 22 
10.1.11 
Publication and Data Sharing Policy ........................................................................... 22 
10.1.12 
Conflict of Interest Policy .............................................................................................. 23 
10.2 
Additional Considerations ................................................................................................................. 23 
10.3 
Abbreviations and Special Terms ................................................................................................... 23 
10.4 
Protocol Amendment History ........................................................................................................... 24 
11 
REFERENCES ............................................................................................................................................. 25 
 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
1 
STATEMENT OF COMPLIANCE 
 
(1) [The trial will be carried out in accordance with International Council on Harmonisation Good 
Clinical Practice (ICH GCP) and the following:  
 
• 
United States (US) Code of Federal Regulations (CFR) applicable to clinical studies (45 CFR Part 
46, 21 CFR Part 50, 21 CFR Part 56, 21 CFR Part 312, and/or 21 CFR Part 812).  
 
National Institutes of Health (NIH)-funded investigators and clinical trial site staff who are 
responsible for the conduct, management, or oversight of NIH-funded clinical trials have 
completed Human Subjects Protection and ICH GCP Training. 
 
The protocol and all participant materials will be submitted to the IRB for review and approval. 
Approval of both the protocol and the waiver of consent must be obtained before any participant 
is enrolled. Any amendment to the protocol will require review and approval by the IRB before 
the changes are implemented to the study.  
INVESTIGATOR’S SIGNATURE 
The signature below constitutes the approval of this protocol and provides the necessary assurances 
that this study will be conducted according to all stipulations of the protocol, including all statements 
regarding confidentiality, and according to local legal and regulatory requirements and applicable US 
federal regulations and ICH guidelines, as described in the Statement of Compliance above. 
 
Principal Investigator or Clinical Site Investigator: 
Signed: 
 
Date: 
April 1, 2019 
 
Name*:  Jason Doctor 
 
Title*: Professor 
 
Investigator Contact Information 
Affiliation*: University of Southern California 
Address: 635 Downey Way, Los Angeles, Ca 90089 
Telephone: 213.821.8142 
Email: jdoctor@usc.edu 
 
 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
2 
1 
PROTOCOL SUMMARY 
1.1 
SYNOPSIS  
 
Title: 
Comparative Effectiveness of Two Letters to Encourage Judicious 
Prescribing of Opioids: A County-wide Project in Los Angeles 
Grant Number: 
P30AG024968 
Study Description: 
In collaboration with the Los Angeles County Medical Examiner's Office 
and the State of California's controlled Substance Utilization Review 
and Evaluation System (CURES), the investigators propose to review 
opioid poisonings over 12 months and send letters to prescribers in 
California when at least one of the provider's prescription(s) was filled 
by a patient who died of an opioid poisoning in Los Angeles County. 
The letters will be non-judgmental and factual, explaining that a patient 
of the provider who was being treated with prescription narcotics died 
of an opioid poisoning. The letters will also encourage judicious 
prescribing including use of the CURES system before prescribing. The 
investigators will evaluate physician prescribing practices over 24 
months (12 months pre- and 12 months post-letter) using data from 
the CURES database. The hypothesis is that letters will make the risk of 
opioids more cognitively available and that physicians will respond by 
prescribing opioids more carefully, resulting in fewer deaths due to 
misuse and more frequent use of the CURES system. 
Objectives*: 
 
1) To conduct a randomized trial comparing two versions of a 
letter sent to prescribers in Los Angeles County notifying them 
that one of their prescriptions was filled by a patient who died 
of an opioid poisoning, and  
2) Evaluate the effects of the randomized trial and identify factors 
that predict intervention effectiveness.    
Endpoints*: 
1) Average change over time in Milligram Morphine Equivalents 
(MME) dispensed  
2) Average change over time in Benzodiazepine Equivalents (BE) 
dispensed  
Study Population: 
Los Angeles County prescribers 
 
 
Phase* or Stage: 
IV 
Description of 
Sites/Facilities Enrolling 
Participants: 
Los Angeles County Medical Examiner’s Office   

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
3 
Description of Study 
Intervention/Experimental 
Manipulation: 
We will conduct a randomized trial to test the two versions of a letter 
signed by the Chief Medical Examiner-Coroner and County Health Officer 
of Los Angeles County to notify prescribers of the death in their practice. 
Each letter version includes the decedent’s name, date of birth and date 
of death, and outlines the annual number and types of prescription drug 
deaths seen by the medical examiner, discusses the value of and way to 
access the State’s prescription drug monitoring program and includes 
five CDC guideline-recommended safe prescribing strategies: 1) Avoid 
co-prescribing of opioids with benzodiazepines, 2) prescribe minimal 
dose necessary for acute pain, 3) consider slow tapers with pauses to 
below 50 MME per day, 4) avoid prescriptions lasting greater than 3-
months for pain, and 5) prescribe naloxone in conjunction with opioids 
for patients taking > 50 MME per day. Each letter also states that CURES 
review is required by law as of October 2, 2018.    
 
Study Duration*: 
24 months 
Participant Duration: 
24 months 
1.2 
SCHEMA  
 
1.3 
SCHEDULE OF ACTIVITIES  
 
 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
4 
     Assessment 
 Enrollment 
Randomization 
    Baseline 
Intervention 
Period 
    Observation  
 Period 
     Follow-up  
     Period 
 
January 
2019 to 
July 2020 
 
Monthly: 
April 2019 to 
July 2020 
12 months prior 
to  decedent  
date of death  
(DOD) 
   1-6 months  
 after DOD 
2-13 months 
after Letter 
is sent 
     > 13 months 
after letter is sent 
Data attainment 
on demographics 
X 
X 
X 
 
 
 
Inclusion/ 
Exclusion 
Criteria 
X 
 
 
 
 
 
Data attainment 
on controlled 
substances 
 
 
X 
X 
X 
 
Data analysis 
 
 
 
 
 
X 
 
 
2 
 
INTRODUCTION 
2.1 
STUDY RATIONALE  
One promising strategy for changing physician behavior is through use of “nudges”. The term “nudge” 
identifies a set of social and cognitive devices that persuade decisions in subtle ways while preserving 
choice. When prescribing narcotics, physicians may not have recent experiences that come to mind as to 
why these drugs are dangerous, making risks seem remote. They may also not consult data on a patient’s 
other narcotic prescriptions in their State’s pharmaceutical drug monitoring program. One useful nudge 
capitalizes on a finding called “Availability”.  Availability is the notion that people rely upon knowledge 
that is salient, recent and readily available to them to evaluate risks and make decisions. If a physician 
issued a prescription for a narcotic that resulted in a recent opioid poisoning, providing that physician 
with this feedback may make the physician more likely to consult a pharmaceutical drug monitoring 
program database and also prescribe more judiciously. 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
5 
We published a randomized trial in the journal Science demonstrating that carefully crafted letters, 
supportive in tone and designed to increase the availability of patient harms, are effective in reducing 
opioid prescribing among high risk 
prescribers (i.e., those with a death in 
their practice).9 Milligram morphine 
equivalents in prescriptions filled daily by 
patients of letter recipients compared to 
controls 
decreased 
by 
9.7% 
(95% 
confidence interval: 6.2 to 13.2%; p < 
0.001) over a 3-month period following 
intervention (Figure 2). We also observed 
both fewer high dose prescriptions and 
new patients started on opioids among 
letter recipients. Learning of a patient’s 
fatal overdose may instill safe opioid 
prescribing habits in physicians.  
 
2.2 
BACKGROUND  
Much of the increase in opioid prescribing rates from 1999 until the past decade has been driven by an 
increase in the use of prescription opioids to treat non-cancer pain (Guy et al. 2017; Boudreau et al. 2009). 
Although opioids carry significant risks of overdose and addiction, they are no more effective for treating 
chronic non-cancerous pain over a one year period than non-opioid alternatives (Krebs et al. 2018). The 
greater availability of prescription opioids has been accompanied by an alarming rise in the negative 
consequences related to opioid use. In 2017, there were 17,029 prescription opioid overdose deaths in 
the US (Scholl 2019). The costs of prescription opioid adverse outcomes are staggering. Aggregate costs 
for prescription opioid harms are estimated at over $78.5 billion (in 2013 USD) and 25% of the aggregate 
economic burden is publicly funded (i.e., Medicaid, Medicare, and veterans' programs) (Florence et al. 
2016). 
In 2016, the Centers for Disease Control and Prevention (CDC) issued the “CDC Guideline for Prescribing 
Opioids for Chronic Pain” which encourages the use of alternatives to opioids and other practices that 
minimize harm to patients (Dowell et al. 2016). Despite the introduction of this guideline, primary care 
clinicians, who prescribe 45% of all opioid prescriptions in the US, report challenges in following these 
recommendations (Kroenke et al. 2019). The dynamics of opioid use make following guidelines difficult. 
Since opioid analgesia from a given dose declines with chronic use due to opioid tolerance, doses increase 
and the chance of harm grows. Over time, the primary benefit of opioids for many patients becomes the 
avoidance of withdrawal. As patients become dependent on opioids, they may misconstrue the treatment 
of interdose withdrawal hyperalgesia as ongoing effectiveness and they may become reluctant to stop 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
6 
opioids (Juurlink 2017). More cautious opioid prescribing (including fewer new starts, avoidance of high 
doses, and slow, collaborative tapers for those already on high dose long-term therapy) can improve the 
balance of benefits and harms for patients with chronic pain. To embrace more cautious prescribing, a 
clinician may need to be informed that opioid risks are present and relevant to his/her own patients. 
2.3 
RISK/BENEFIT ASSESSMENT  
2.3.1 KNOWN POTENTIAL RISKS  
There is a small risk that people who are not connected with this study will learn a participant’s identity 
or their personal information.  Physicians may also experience some discomfort in learning that a patient 
that they prescribed to has died. 
2.3.2 KNOWN POTENTIAL BENEFITS  
Clinicians may benefit if the quality of care they provide their patients is improved by the intervention. 
They also may learn more about guidelines for appropriate prescribing. 
2.3.3 ASSESSMENT OF POTENTIAL RISKS AND BENEFITS  
In sum, the risks of this study are small relative to the potential benefits in increased adoption of evidence-
based practices, and improved patient outcomes through improved pain management and reduction in 
opioid use.  Unfortunately, most people who develop opioid use disorder, use illicit opiates or synthetic 
opioids or who die of an opioid poisoning are introduced to these drugs medically.  We hope that these 
pilot studies will have a number of benefits that will address the unmet needs of this important group of 
individuals. We anticipate that health care systems, government agencies and other policy makers will 
benefit from implementation of the interventions we develop. Even if the intervention(s) are not found 
to be beneficial, then the knowledge gained will hopefully lead to new insights that will result in successful 
redesign for future studies. Most importantly, we anticipate the approaches we are studying will lead to 
fewer opioid harms, fewer persons suffering from opioid use disorder and poisoning death and fewer 
tragedies for families of persons who find themselves misusing opioid medications. 
3 
OBJECTIVES AND ENDPOINTS  
 
OBJECTIVES 
ENDPOINTS 
Primary 
 
Evaluate the effects of the randomized trial 
and identify factors that predict intervention 
effectiveness.    
 
Average change over time in Milligram Morphine 
Equivalents (MME) dispensed  
Average change over time in Benzodiazepine Equivalents 
(BE) dispensed    
 
 
4 
STUDY DESIGN 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
7 
4.1 
OVERALL DESIGN 
This is a decedent-cluster randomized field experiment; clusters of prescribers within each decedent will 
be randomly allocated to one of two letters interventions. This design avoids treatment contamination—
clinicians prescribing to the same patient will be able to share information about the intervention without 
diluting its effect. If two or more decedents have the same prescriber, that prescriber will be assigned to 
one and only one of those decedent clusters by random draw. Two factors will serve as random strata: 1) 
whether the decedent received a prescription from a clinician with a single or multiple deaths, and 2) 
whether opioids only, opioids in combination with benzodiazepines, or benzodiazepines only were the 
cause of death. The first strata are designed to form equivalent groups on prescriber risk posture. The 
second strata equate preference for type of scheduled drug prescribed in practice. Strata will be crossed 
to form 2 x 3 = 6 randomization groups.  
4.2 
SCIENTIFIC RATIONALE FOR STUDY DESIGN 
Given the recent results collected in San Diego County that were reported by our group in Science,9 on 
August 14th, 2018 the Los Angeles County Board of Supervisors voted to explore the feasibility of sending 
letters to prescribers whose patients suffered a fatal overdose to encourage judicious prescribing. Dr. 
Doctor participated in these discussions and helped formulate along with the County Health Officer Muntu 
Davis, MD, MPH and the County Medical Examiner-Coroner, Jonathan Lucas, MD, two versions of the 
letter (described below). A report was submitted to the County in September on feasibility and a “make 
it so” Board motion on October 2, 2018 approved sending of the letters.  
4.3 
JUSTIFICATION FOR INTERVENTION 
In 2016, the Centers for Disease Control and Prevention (CDC) issued the “CDC Guideline for Prescribing 
Opioids for Chronic Pain” which encourages the use of alternatives to opioids and other practices that 
minimize harm to patients (Dowell et al 2016). Despite the introduction of this guideline, primary care 
clinicians, who prescribe 45% of all opioid prescriptions in the US, report practical challenges in 
implementing these recommendations (Kroenke et al 2019]. The dynamics of opioid use make following 
guidelines difficult. Since opioid analgesia from a given dose declines with chronic use due to opioid 
tolerance, doses increase and the chance of harm grows. Over time, the primary benefit of opioids for 
many patients becomes the avoidance of withdrawal. As patients become dependent on opioids, they 
may misconstrue the treatment of withdrawal hyperalgesia as ongoing effectiveness, and they may 
become reluctant to stop opioids (Juurlink, 2017]. More cautious opioid prescribing (including fewer new 
starts, avoidance of high doses, and slow, collaborative tapers for those already on high dose long-term 
therapy) can improve the balance of benefits and harms for patients with chronic pain. To embrace more 
cautious prescribing, a clinician may need to be informed that opioid risks are present and relevant to 
his/her own patients. 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
8 
Notifying physicians via a letter of the death of their patient has been shown to significantly reduce opioid 
prescribing (Doctor et al. 2018). Two versions of a letter will be signed by the Chief Medical Examiner-
Coroner and County Health Officer of Los Angeles County to notify prescribers of the death in their 
practice. Each letter version includes the decedent’s name, date of birth and date of death, and outlines 
the annual number and types of prescription drug deaths seen by the medical examiner, discusses the 
value of and way to access the State’s prescription drug monitoring program and includes five CDC 
guideline-recommended safe prescribing strategies: 1) Avoid co-prescribing of opioids with 
benzodiazepines, 2) prescribe minimal dose necessary for acute pain, 3) consider slow tapers with pauses 
to below 50 MME per day, 4) avoid prescriptions lasting greater than 3-months for pain, and 5) prescribe 
naloxone in conjunction with opioids for patients taking > 50 MME per day. Each letter also states that 
CURES review is required by law as of October 2, 2018.    
Version B of the letter includes additional text involving an “if/when/then statement” along with an 
injunction to providers to share safety information with patients so that they identify as a "safe 
prescriber."  The text reads as follows: When your next patient presents with pain, keep the above 5 
recommendations close at hand to assist with their safe care. Also, be comfortable voicing your concern 
about prescribing safety with them so that they are also aware of the dangers scheduled drugs may carry. 
“If/when/then” is a form of “pre-suasion” that provides simple rules that tie goals to specific actions and 
has been used successfully to encourage behavior in many areas including medication adherence and drug 
abuse rehabilitation.21 Letters will be signed and posted in the U.S. mail. 
4.4 
END-OF-STUDY DEFINITION 
A participant is considered to have completed the study if he or she has completed the baseline 
assessment, received a letter, and been observed for >13 months following letter receipt.  
 
The end of the study is defined as completion of the >13-month data observation period following receipt 
of letter shown in the Schedule of Activities (SoA), Section 1.3. 
5 
STUDY POPULATION 
We will intervene on all clinicians and allied health professionals with prescribing privileges and at least 
one prescription drug death over the course of a year as part of a public health program to encourage 
safe prescribing in Los Angeles County. Informed consent of patients and clinicians is waived under HHS 
regulations at 45 CFR 46.116(c), as the study was evaluating a County public service safe prescribing 
program. As part of the County safe prescribing program, prescribers will be identified in the Controlled 
Substance Utilization Review and Evaluation System (CURES) system. All California pharmacies and clinics 
that dispense controlled substances must submit reports to CURES on a weekly basis. In consultation with 
the program manager of CURES, a letter notifying CURES administrators that prescriptions from these 
clinicians would be evaluated prospectively after the safe prescribing letters have been sent will be 
submitted to the CURES system.  

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
9 
5.1 
INCLUSION CRITERIA 
The inclusion criteria for clinicians are as follows: a) must be located in Los Angeles County, b) have 
scheduled drug prescribing privileges and, c) have prescribed a schedule II, III, or IV drug to a person in 
the 12 months prior to their opioid-related death.  
5.2 
EXCLUSION CRITERIA 
Clinicians not meeting the inclusion criteria above will be excluded from the study. 
5.3 
LIFESTYLE CONSIDERATIONS 
Not applicable 
5.4 
SCREEN FAILURES 
Not applicable 
5.5 
STRATEGIES FOR RECRUITMENT AND RETENTION 
We will intervene on all clinicians and allied health professionals with prescribing privileges and at least 
one prescription drug death over the course of a year as part of a public health program to encourage 
safe prescribing in Los Angeles County. Informed consent of patients and clinicians is waived under HHS 
regulations at 45 CFR 46.116(c), as the study is evaluating a County public service safe prescribing 
program. We will utilize data from the County Medical Examiner’s Office to identify opioid-related 
deaths and the state’s prescription drug monitoring program (CURES) to identify prescribers to those 
decedents.  
6 
STUDY INTERVENTION(S) OR EXPERIMENTAL MANIPULATION(S) 
6.1 
STUDY INTERVENTION(S) OR EXPERIMENTAL MANIPULATION(S) ADMINISTRATION 
6.1.1 STUDY INTERVENTION OR EXPERIMENTAL MANIPULATION DESCRIPTION 
Two versions of a letter will be signed by the Chief Medical Examiner-Coroner and County Health Officer 
of Los Angeles County to notify prescribers of the death in their practice. Each letter version includes the 
decedent’s name, date of birth and date of death, and outlines the annual number and types of 
prescription drug deaths seen by the medical examiner, discusses the value of and way to access the 
State’s prescription drug monitoring program and includes five CDC guideline-recommended safe 
prescribing strategies: 1) Avoid co-prescribing of opioids with benzodiazepines, 2) prescribe minimal dose 
necessary for acute pain, 3) consider slow tapers with pauses to below 50 MME per day, 4) avoid 
prescriptions lasting greater than 3-months for pain, and 5) prescribe naloxone in conjunction with opioids 
for patients taking > 50 MME per day. Each letter also states that CURES review is required by law as of 
October 2, 2018.    

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
10 
Version B of the letter includes additional text involving an “if/when/then statement” along with an 
injunction to providers to share safety information with patients so that they identify as a "safe 
prescriber."  The text reads as follows: When your next patient presents with pain, keep the above 5 
recommendations close at hand to assist with their safe care. Also, be comfortable voicing your concern 
about prescribing safety with them so that they are also aware of the dangers scheduled drugs may carry. 
“If/when/then” is a form of “pre-suasion” that provides simple rules that tie goals to specific actions and 
has been used successfully to encourage behavior in many areas including medication adherence and drug 
abuse rehabilitation.  Letters will be signed and posted in the U.S. mail. 
6.1.2 ADMINISTRATION AND/OR DOSING 
Not applicable 
6.2 
FIDELITY 
No text is to be entered in this section; rather it should be included under the relevant subheadings below.  
This section refers to efforts made to confirm that the intervention is appropriately conducted by the 
interventionist(s).  It is distinct from the content of Section 6.4, Study Intervention Adherence, which is 
intended to capture a study participant’s adherence to an intervention. 
 
6.2.1 INTERVENTIONIST TRAINING AND TRACKING 
It is the primary role of the data analysts (Marcella Kelley and Emily Stewart) to: 1) determine the 
appropriate number of eligible providers to reach sufficient statistical power, 2) record provider 
information in a secure, password protected database located at LA County Medical Examiner-Coroner’s 
office and, 3) track enrolled providers in regards to their intervention status and progress.  
6.3 
MEASURES TO MINIMIZE BIAS: RANDOMIZATION AND BLINDING 
For randomization, two decedent lists will be generated from the crossed strata levels. Using 
random.org’s sequence generator, two true random integer sequences derived from atmospheric noise 
will determine decedent order in each list. For each ordered list, prescribers to decedents in the first half 
will receive Letter A, the second half of the list will receive Letter B. 
6.4 
STUDY INTERVENTION/EXPERIMENTAL MANIPULATION ADHERENCE 
Not applicable 
 
6.5 
CONCOMITANT THERAPY 
Not applicable 
 
6.5.1 RESCUE THERAPY 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
11 
Not applicable 
 
7 
STUDY INTERVENTION/EXPERIMENTAL MANIPULATION DISCONTINUATION AND 
PARTICIPANT DISCONTINUATION/WITHDRAWAL 
7.1 
DISCONTINUATION OF STUDY INTERVENTION/EXPERIMENTAL MANIPULATION 
Not applicable 
7.2 
PARTICIPANT DISCONTINUATION/WITHDRAWAL FROM THE STUDY 
Not applicable 
7.3 
LOST TO FOLLOW-UP 
Not applicable 
 
8 
STUDY ASSESSMENTS AND PROCEDURES 
8.1 
ENDPOINT AND OTHER NON-SAFETY ASSESSMENTS 
Data will be obtained by providing CURES with a list of prescriber names, alongside randomized digit IDs 
to anonymize human subjects. The list will include the date range for retrieving all schedule II, III, and IV 
drugs written by the prescriber in that time. These dates will be based on the decedent date of death 
and letter sent date for each prescriber - 12 months before the decedent date of death and 13 months 
after the letter was sent, which includes a one-month wash out period. Based on the earliest decedent 
date of death and latest letter sent date, we will obtain data from October 1, 2017 to August 31, 2021. 
CURES will provide the data to USC researchers with the randomized digit IDs (i.e., omitting prescriber 
and patient names) to ensure the confidentiality and privacy of human subjects. All data will be 
transferred and reintegrated into LA Medical Examiner-Coroner’s database using Secure File Transfer 
Protocol. 
 
8.2 
SAFETY ASSESSMENTS 
Although we will be unable to determine if there are hospitalizations or ED visits for opioid withdrawal 
in patients of clinicians in the intervention group, we will monitor this potential safety issue by 
comparing clinicians who received a letter to controls to determine if there are any dramatic reductions 
(>20% from baseline) in opioid prescribing in the post-intervention period as a result of the letter.  The 
U.S. does not have an integrated healthcare system with complete data on hospitalizations for all 
patients to evaluate induced withdrawal. However, it is possible for us to evaluate if providers induce 
withdrawal on patients by exhibiting high rates of MME reductions (see p. 589 Doctor et al., Science 
361, 588–590 (2018).  In our previous work, we demonstrated that for letters, there was no difference in 
the proportion of prescribers in the intervention or control group who made substantial (>20%) 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
12 
reductions in opioid prescribing in the post-intervention period (z =1.279, P > 0.05). We will use this 
same outcome to evaluate potential harm to patients in this trial. 
8.3 
ADVERSE EVENTS AND SERIOUS ADVERSE EVENTS 
8.3.1 DEFINITION OF ADVERSE EVENTS 
This protocol uses the definition of adverse event from 21 CFR 312.32 (a): any untoward medical 
occurrence associated with the use of an intervention in humans, whether or not considered intervention-
related. 
8.3.2 DEFINITION OF SERIOUS ADVERSE EVENTS 
Not applicable 
 
8.3.3 CLASSIFICATION OF AN ADVERSE EVENT 
8.3.3.1 SEVERITY OF EVENT 
For adverse events (AEs) not included in the protocol defined grading system, the following guidelines will 
be used to describe severity.  
 
• 
Mild – Events require minimal or no treatment and do not interfere with the participant’s daily 
activities.  
• 
Moderate – Events result in a low level of inconvenience or concern with the therapeutic 
measures. Moderate events may cause some interference with functioning. 
• 
Severe – Events interrupt a participant’s usual daily activity and may require systemic drug 
therapy or other treatment. Severe events are usually potentially life-threatening or 
incapacitating.  Of note, the term “severe” does not necessarily equate to “serious”.] 
8.3.3.2 RELATIONSHIP TO STUDY INTERVENTION/EXPERIMENTAL MANIPULATION 
All adverse events (AEs) will have their relationship to study procedures, including the intervention, 
assessed by an appropriately-trained clinician based on temporal relationship and his/her clinical 
judgment. The degree of certainty about causality will be graded using the categories below.  
 
• 
Related – The AE is known to occur with the study procedures, there is a reasonable possibility 
that the study procedures caused the AE, or there is a temporal relationship between the study 
procedures and the event. Reasonable possibility means that there is evidence to suggest a causal 
relationship between the study procedures and the AE. 
• 
Not Related – There is not a reasonable possibility that the study procedures caused the event, 
there is no temporal relationship between the study procedures and event onset, or an alternate 
etiology has been established. 
 
OR 
 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
13 
• 
Definitely Related – There is clear evidence to suggest a causal relationship, and other possible 
contributing factors can be ruled out. The clinical event, including an abnormal laboratory test 
result, occurs in a plausible time relationship to study procedures administration and cannot be 
explained by concurrent disease or other drugs or chemicals. The response to withdrawal of the 
study procedures should be clinically plausible. The event must be pharmacologically or 
phenomenologically definitive. 
• 
Probably Related – There is evidence to suggest a causal relationship, and the influence of other 
factors is unlikely. The clinical event, including an abnormal laboratory test result, occurs within a 
reasonable time after administration of the study procedures, is unlikely to be attributed to 
concurrent disease or other drugs or chemicals, and follows a clinically reasonable response on 
withdrawal.  
• 
Potentially Related – There is some evidence to suggest a causal relationship (e.g., the event 
occurred within a reasonable time after administration of study procedures). However, other 
factors may have contributed to the event (e.g., the participant’s clinical condition, other 
concomitant events). Although an AE may rate only as “possibly related” soon after discovery, it 
can be flagged as requiring more information and later be upgraded to “probably related” or 
“definitely related”, as appropriate. 
• 
Unlikely to be related – A clinical event, including an abnormal laboratory test result, whose 
temporal relationship to study procedures administration makes a causal relationship improbable 
(e.g., the event did not occur within a reasonable time after administration of the study 
procedures) and in which other drugs or chemicals or underlying disease provides plausible 
explanations (e.g., the participant’s clinical condition, other concomitant treatments). 
• 
Not Related – The AE is completely independent of study procedures administration, and/or 
evidence exists that the event is definitely related to another etiology. There must be an 
alternative, definitive etiology documented by the clinician.] 
 
8.3.3.3 EXPECTEDNESS  
An AE will be considered unexpected if the nature, severity, or frequency of the event is not consistent 
with the risk information previously described for the study procedures. 
Although we will be unable to determine if there are hospitalizations or ED visits for opioid withdrawal 
in patients of clinicians in the intervention group, we will monitor this potential safety issue by 
comparing clinicians who received a letter to controls to determine if there are any dramatic reductions 
(>20% from baseline) in opioid prescribing in the post-intervention period as a result of the letter.  The 
U.S. does not have an integrated healthcare system with complete data on hospitalizations for all 
patients to evaluate induced withdrawal. However, it is possible for us to evaluate if providers induce 
withdrawal in patients by exhibiting high rates of MME reductions (see p. 589 Doctor et al., Science 361, 
588–590 (2018).  In our previous work, we demonstrated that for letters, there was no difference in the 
proportion of prescribers in the intervention or control group who made substantial (>20%) reductions 
in opioid prescribing in the post-intervention period (z =1.279, P > 0.05). We will use this same outcome 
to evaluate potential harm to patients in this trial. 
8.3.4 TIME PERIOD AND FREQUENCY FOR EVENT ASSESSMENT AND FOLLOW-UP 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
14 
At any time, clinicians can report an adverse event or unanticipated problem potentially related to the 
letters to the medical examiner’s office or the County or City public health officer who co-signs the letter 
and provides contact information. 
We do not expect there to be adverse events directly influenced by the clinical guidance being 
delivered in this study. All study interventions encourage clinicians to follow well-established 
national guideline recommendations and known best practices. While the expectedness of 
adverse events is very low, we will investigate each and every numerator case identified in all 
safety measures described above. For cases identified by the safety monitoring measures, we 
will perform manual physician chart review to examine the clinical circumstances and to make a 
judgment (1) the expectedness of the event [unexpected, expected], (2) the likelihood that the 
safety event was study related [not related/possibly related/definitely related] and (3) judge the 
event’s severity [abnormal clinical finding without symptoms/symptoms requiring clinical 
intervention/short term disability or hospitalization/death AND separately define the severity as 
mild, moderate, or severe].  
These will be conducted only by authorized study personnel. Study personnel will interview 
clinicians treating patients when needed to obtain additional information. Each case identified 
will have a case report form with these variables and will be signed and dated by study staff 
completing the form. These forms will be stored in a locked office. Each adverse event will be 
given an identification number. If study personnel believe that a patient that experienced an 
adverse event would benefit from seeing or communicating with their clinician who previously 
received a letter, the PI will within 2 business days reach out to this clinician advising them to 
contact the patient as soon as possible.  
We will within a business day report any clinician reported adverse events, safety analysis, or 
unanticipated problems to the USC IRB. Our report will include appropriate identifying information for 
the study, a detailed description of the adverse event, and a description of any changes to the protocol 
or other corrective actions that have been taken or are proposed. If an adverse event occurs, we will 
review relevant clinical decision support and ensure others are not at a greater risk of harm than was 
previously known or recognized. We will notify our NIA Program Officer of any serious adverse events. 
Clinician participants are only receiving a letter informing them of a recent patient death due to opioid 
overdose. Additional patient deaths related to this study are not expected. However, should we identify 
a patient death in safety measures described above we will report the death to the NIA Program Officer 
and the USC IRB within 24 hours of our knowledge of the death. 
The seriousness of the adverse event will be documented on the case report form. We will categorize all 
of the following as serious adverse events: patient death, life-threatening event, hospitalization-initial or 
prolonged, disability/incapacity, and events that required intervention to prevent permanent 
impairment. The clinician reviewing the event will determine the seriousness of the event. If it is an 
event other than those listed above that the reviewing clinician feels is an ‘other’ serious event, it will be 
discussed with another clinical study team member to reach consensus. All details will be documented 
on a case report form.   
8.3.5 ADVERSE EVENT REPORTING  
See 8.3.4. 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
15 
8.3.6 SERIOUS ADVERSE EVENT REPORTING  
See 8.3.4. 
 
8.3.7 REPORTING EVENTS TO PARTICIPANTS  
Not applicable 
 
8.3.8 EVENTS OF SPECIAL INTEREST  
Not applicable 
 
8.3.9 REPORTING OF PREGNANCY  
Not applicable 
 
8.4 
UNANTICIPATED PROBLEMS 
8.4.1 DEFINITION OF UNANTICIPATED PROBLEMS 
This protocol uses the definition of Unanticipated Problems as defined by the Office for Human Research 
Protections (OHRP).  OHRP considers unanticipated problems involving risks to participants or others to 
include, in general, any incident, experience, or outcome that meets all of the following criteria: 
 
• 
Unexpected in terms of nature, severity, or frequency given (a) the research procedures that are 
described in the protocol-related documents, such as the Institutional Review Board (IRB)-
approved research protocol and informed consent document; and (b) the characteristics of the 
participant population being studied; 
• 
Related or possibly related to participation in the research (“possibly related” means there is a 
reasonable possibility that the incident, experience, or outcome may have been caused by the 
procedures involved in the research); and 
• 
Suggests that the research places participants or others at a greater risk of harm (including 
physical, psychological, economic, or social harm) than was previously known or recognized. 
 
8.4.2  UNANTICIPATED PROBLEMS REPORTING  
Unanticipated Problems that are unexpected, related or possibly related to this research (including 
anywhere there is a reasonable possibility that the incident or outcome may have been caused or 
associated with the study) and that suggests that the research places clinicians or patients at greater risk 
of harm than was previously known or recognized will be thoroughly and promptly investigated. The 
Unanticipated Problem will be investigated, formally written down with a corrective plan and measures 
to prevent reoccurrence. This report will be shared with the NIA Program Officer and Safety Officer within 
48 hours of study’s knowledge of the problem. 
8.4.3 REPORTING UNANTICIPATED PROBLEMS TO PARTICIPANTS  

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
16 
Not applicable 
 
9 
STATISTICAL CONSIDERATIONS  
9.1 
STATISTICAL HYPOTHESES 
Milligram morphine equivalents in prescriptions filled daily by patients of letter recipients will decrease 
following receipt of letter as compared to baseline. 
9.2 
SAMPLE SIZE DETERMINATION 
We calculated the sample size needed to detect an effect of the letter intervention using standard 
formulas for power analysis in cluster randomized trials and the ‘clusterPower’ package in R statistical 
computing language. For this calculation we used data on decedents and prescribers at the San Diego 
County Medical Examiner between January 1 and December 31, 2013 published in Lev et al. 2016 and 
Doctor et al. 2018.  We assume a two tailed test with a 5% Type I error rate and an 80% chance to detect 
an effect. Doctor et al. (2018) found mean of 5.5 prescribers for each decedent. Coefficient of variation 
we assume to be 1.22, which implies that 99% of decedents had 20 or fewer prescribers in the year before 
their death. We assume a 50% reduction in mean difference as compared to the effect reported in Doctor 
et al. 2018 and a standard deviation of +140 daily MME within cluster. Most clinician clusters for process 
measures have intracluster correlations that fall between 0.05 and 0.15. We assume a more conservative 
intracluster correlation of 0.2; in our design we randomized by decedent to reduce intracluster 
correlation. We utilized the Taylor method for calculating variance inflation due to unequal cluster sizes. 
Under these assumptions, we would need 103 decedents per study arm; given annual opioid overdose 
frequency in Los Angeles, we expect to have 192 per arm. We have ample power to detect an effect. 
9.3 
POPULATIONS FOR ANALYSES 
Intention-to-Treat (ITT) Analysis Population (i.e., all randomized participants) 
 
9.4 
STATISTICAL ANALYSES 
9.4.1 GENERAL APPROACH 
We will convert opioids to milligram morphine equivalents (MME) using formulas published by the CDC.29 
Descriptive and inferential statistics will be carried out in STATA (Version 14.0; Stata Corporation, College 
Station, TX). The cmp command in STATA will be used to compute a difference in differences estimator 
within a mixed-model censored linear regression.30 The difference in differences estimator compares the 
average change over time in MME dispensed for prescribers in the group receiving letter version A, 
compared to the average change over time for those prescribers in receiving version B. To ensure normally 
distributed data, we evaluate the natural log of MME. Censored regression has a continuous component 
and a discrete one. The natural log MME doses estimated over days where opioids were dispensed in the 
name of a prescriber represent the continuous part, and days with no opioids filled in that prescriber’s 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
17 
name represent the discrete part of the model. We denote estimation of the dependent variable as 
𝑙𝑙𝑙𝑙𝑙𝑙(𝑀𝑀𝑀𝑀𝑀𝑀) ∗
෣
 to distinguish it from uncensored estimation. The analysis is represented by: 
𝑙𝑙𝑙𝑙𝑙𝑙(𝑀𝑀𝑀𝑀𝑀𝑀) ∗𝚤𝚤𝚤𝚤𝚤𝚤
෣
 = 𝛽𝛽1𝑥𝑥1𝑖𝑖𝑖𝑖 +  𝛽𝛽2 𝑥𝑥2𝑖𝑖𝑖𝑖 +  𝛽𝛽3 𝑥𝑥3𝑖𝑖𝑖𝑖 + 𝛿𝛿𝑖𝑖(𝑘𝑘)  [1] 
where 𝛽𝛽1, 𝛽𝛽2and 𝛽𝛽3 are fixed effects coefficients on time, 𝑥𝑥1, intervention, 𝑥𝑥2,and time by intervention 
interaction, 𝑥𝑥3, respectively. Alphabetic subscripts describe the  ith prescriber, jth prescription filled and 
kth decedent, the nested random intercept 𝛿𝛿𝑖𝑖(𝑘𝑘)is normally distributed with mean zero and variance, 
𝜎𝜎𝑖𝑖(𝑘𝑘)
2
, for each i prescriber nested in (i.e., having prescribed to) decedent k, i(k). With natural log 
transformed data, the value 100·[exp{𝛽𝛽3 } - 1] measures precisely the percentage change in MME 
attributable to the intervention when data are uncensored (see Woolridge (2008), pp. 212 and 410).  
 
9.4.2 ANALYSIS OF THE PRIMARY ENDPOINT(S) 
 
9.4.3 ANALYSIS OF THE SECONDARY ENDPOINT(S) 
Not applicable 
9.4.4 SAFETY ANALYSES 
Although we will be unable to determine if there are hospitalizations or ED visits for opioid withdrawal 
in patients of clinicians in the intervention group, we will monitor this potential safety issue by 
comparing clinicians who received a letter to controls to determine if there are any dramatic reductions 
(>20% from baseline) in opioid prescribing in the post-intervention period as a result of the letter.  The 
U.S. does not have an integrated healthcare system with complete data on hospitalizations for all 
patients to evaluate induced withdrawal. However, it is possible for us to evaluate if providers induce 
withdrawal on patients by exhibiting high rates of MME reductions (see p. 589 Doctor et al., Science 
361, 588–590 (2018).  In our previous work, we demonstrated that for letters, there was no difference in 
the proportion of prescribers in the intervention or control group who made substantial (>20%) 
reductions in opioid prescribing in the post-intervention period (z =1.279, P > 0.05). We will use this 
same outcome to evaluate potential harm to patients in this trial. 
 
9.4.5 BASELINE DESCRIPTIVE STATISTICS 
For sample (clinician and decedent) descriptive statistics, we will use means and medians for continuous 
measures, frequencies for count data, and standard deviations and interquartile ranges for variance. 
 
9.4.6 PLANNED INTERIM ANALYSES  
Not applicable 
9.4.7 SUB-GROUP ANALYSES 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
18 
All providers meeting the study inclusion criteria will participate in the pilot study. We will not 
undersample or oversample clinicians who are women and/or members of minority racial and ethnic 
groups, so we expect to enroll them in proportion to their population prevalence.  Although we can 
report on sex/gender and race/ethnicity of the decedents (if the information is available in the Medical 
Examiner’s report), we will likely not have racial/ethnic information available for providers through 
CURES data. However, we will have access to the sex/gender of providers and will conduct analyses to 
investigate any differences between groups.  Per the Frequently Asked Questions on NIH policy 
regarding Inclusion on the basis of Sex/Gender and Race/Ethnicity, if we are using previously collected 
data sets that do not conform to the current (1997) OMB standards and have no plans of collecting any 
new/additional data from the subjects, we will note this in the comments section of our Planned 
Enrollment Report.   
9.4.8 TABULATION OF INDIVIDUAL PARTICIPANT DATA 
Not applicable 
 
9.4.9 EXPLORATORY ANALYSES 
Not applicable 
 
10 SUPPORTING DOCUMENTATION AND OPERATIONAL CONSIDERATIONS 
10.1 REGULATORY, ETHICAL, AND STUDY OVERSIGHT CONSIDERATIONS 
10.1.1 INFORMED CONSENT PROCESS 
Informed consent of patients and clinicians is waived under HHS regulations at 45 CFR 46.116(c), as the 
study is evaluating a County public service safe prescribing program. 
 
10.1.1.1 CONSENT/ASSENT 
AND 
OTHER 
INFORMATIONAL 
DOCUMENTS 
PROVIDED 
TO 
PARTICIPANTS 
Not applicable 
 
10.1.1.2 CONSENT PROCEDURES AND DOCUMENTATION 
Not applicable 
10.1.2 STUDY DISCONTINUATION AND CLOSURE 
This study may be temporarily suspended or prematurely terminated if there is sufficient reasonable 
cause. Written notification, documenting the reason for study suspension or termination, will be provided 
by the suspending or terminating party to investigator, funding agency, and regulatory authorities. If the 
study is prematurely terminated or suspended, the Principal Investigator (PI) will promptly inform study 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
19 
participants, the Institutional Review Board (IRB), and sponsor/funding agency and will provide the 
reason(s) for the termination or suspension.  
 
Circumstances that may warrant termination or suspension include, but are not limited to: 
• 
Determination of unexpected, significant, or unacceptable risk to participants 
• 
Insufficient compliance of study staff to the protocol  (ie, significant protocol violations) 
 
The study may resume once concerns about safety, protocol compliance, and data quality are addressed, 
and satisfy the funding agency, sponsor, or IRB. 
 
10.1.3 CONFIDENTIALITY AND PRIVACY  
Participant confidentiality and privacy is strictly held in trust by the participating investigators, their staff, 
the safety and oversight monitor(s), and the sponsor(s) and funding agency. This confidentiality is 
extended to the data being collected as part of this study. All research activities will be conducted in as 
private a setting as possible. 
Data for this project is extracted from Controlled Substance Utilization Review and Evaluation System 
CURES; https://oag.ca.gov/cures). Only designated members of the research team will have access to 
this data. Only data from eligible subjects will be analyzed.  Marcella Kelley and Emily Stewart have been 
cleared and embedded as LA County volunteers and will collect all data on site at LA County Medical 
Examiner's office or from CURES Research Center. A data usage agreement has been put in place 
between LAC and USC to safely transfer de-identified data via a secure hard drive for analyses related to 
the evaluation. All data will be stored on password-protected computers and servers that are accessible 
only to study personnel with appropriate password authorization. These measures should be effective in 
minimizing breaches of confidentiality. Prior to study initiation, approval will be obtained from the 
Institutional Review Board at University of Southern California. This approval will be reevaluated each 
year as part of the Human Subjects Committee annual review process, paying particular attention to 
patient confidentiality and the Health Insurance Portability and Accountability Act of 1996 (HIPAA). 
Human subjects training: All project personnel handling study data will be certified by the Collaborative 
IRB Training Initiative (CITI) program, which consists of courses in the Protection of Human Research 
Subjects for Biomedical Research 
Measures Taken to Ensure Confidentiality of Data Shared per the NIH Data Sharing Policies  
It is NIH policy that the results and accomplishments of the activities that it funds should be made available 
to the public (see https://grants.nih.gov/policy/sharing.htm). The PI will ensure all mechanisms used to 
share data will include proper plans and safeguards for the protection of privacy, confidentiality, and 
security for data dissemination and reuse (e.g., all data will be thoroughly de-identified and will not be 
traceable to a specific study participant). Plans for archiving and long-term preservation of the data will 
be implemented, as appropriate.  
10.1.4 FUTURE USE OF STORED SPECIMENS AND DATA  

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
20 
Data collected for this study will be analyzed and stored at the LA County Medical Examiners Office. After 
the study is completed, the de-identified, archived data will be transmitted to and stored at University of 
Southern California. Data may be made available through data use agreements among researchers 
interested in using the data, the Los Angeles County Medical Examiner's Office and the California State 
Department of Justice. 
10.1.5 KEY ROLES AND STUDY GOVERNANCE 
 
Principal Investigator 
Independent Safety Monitor 
Jason Doctor, PhD 
Daniel Larach, MD 
University of Southern California  
University of Southern California  
635 Downey Way, Los Angeles, Ca 
1450 San Pablo St., Suite 3600, 
Los Angeles, Ca 
213.821.8142 
(410) 274-3034 
jdoctor@usc.edu 
daniel.larach@med.usc.edu 
This project is also governed by the Roybal Steering Committee consisting of the Roybal Center PIs and 
Co-Investigators. 
10.1.6 SAFETY OVERSIGHT 
For this single-site, single study arm, minimal risk intervention pilot study, the study staff and principal 
investigator will be responsible for ensuring participants’ safety on a daily basis and for monitoring and 
responding to any adverse events or unanticipated problems and for reporting them to the IRB and the 
NIA Program Officer. 
The presence of a DSMB is not requested or required by the NIA for this study. However, we have 
nominated a Safety Officer, Daniel Larach, MD.   
In addition to the reporting to the IRB and NIA PO, we will report all adverse events that are both serious 
(SAE) and unexpected (i.e., have not been previously reported for the study's intervention) to the Safety 
Officer within 48 hours of the study's knowledge of SAE. The summary of all other SAEs will be reported 
to NIA PO and to the Safety Officer quarterly, unless otherwise requested by the Safety Officer.  
10.1.7 CLINICAL MONITORING 
Not applicable 
 
10.1.8 QUALITY ASSURANCE AND QUALITY CONTROL 
Quality control (QC) procedures will be implemented as follows: 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
21 
Informed consent --- Not applicable. The USC IRB has issued a waiver of consent for participation in this 
study, but clinician participants who receive a letter will have the opportunity to ask questions and opt 
out of future mailings. 
Source documents and the electronic data --- Data will be entered into the study database.  To ensure 
accuracy site staff will compare a representative sample of source data against the database, targeting 
key data points in that review. 
 
Intervention Fidelity — Consistent delivery of the study interventions will be monitored throughout the 
intervention phase of the study. Procedures for ensuring fidelity of intervention delivery are described in 
Section 6.2.1, Interventionist Training and Tracking.  
 
Protocol Deviations – The study team will review protocol deviations on an ongoing basis and will 
implement corrective actions when the quantity or nature of deviations are deemed to be at a level of 
concern. 
 
Should independent monitoring become necessary, the PI will provide direct access to all trial related 
sites, source data/documents, and reports for the purpose of monitoring and auditing by the 
sponsor/funding agency, and inspection by local and regulatory authorities. 
 
10.1.9 DATA HANDLING AND RECORD KEEPING  
10.1.9.1 DATA COLLECTION AND MANAGEMENT RESPONSIBILITIES  
Decedent data is collected from hard copies of death reports at the LA County office. Prescriber data is 
collected electronically from the CURES online portal, CURES Research Center, and the CA Department 
of Consumer Affairs (DCA) website.  Patient and physician data are recorded in a secure, electronic 
database, and maintained by study analysts. 
CURES provides records of opioids dispensed at California pharmacies attributable to each provider in 
our sample treating all civilian, non-Veteran’s Administration and non-institutionalized patients. The 
Medical Examiner has authority to use CURES for the purpose of educating practitioners and others in 
lieu of disciplinary, civil, or criminal actions, in accordance with the California State’s Health and Safety 
Code § 11165(c)(2). First name, last name, date of birth and address will identify each decedent in the 
medical examiner reports and CURES data. Drug Enforcement Agency (DEA) number will identify the 
prescribers in CURES data. Data from eligible clinicians will be extracted from CURES and kept onsite 
(Windows 10 OS) at the medical examiner’s office for de-identified analytic file preparation. A file 
stripped of patient and clinician identifiers will be prepared and released for analysis on secure servers 
at the University of Southern California. 
Data will be entered, tracked, edited, updated and reported by pre-approved analysts with the 
appropriate clearance.  

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
22 
 
10.1.9.2 STUDY RECORDS RETENTION  
 
Study documents will be retained for a minimum of 3 years after of study completion. No records will be 
destroyed without the written consent of the sponsor/funding agency, if applicable. It is the responsibility 
of the sponsor/funding agency to inform the investigator when these documents no longer need to be 
retained. 
 
10.1.10 
PROTOCOL DEVIATIONS   
This protocol defines a protocol deviation as any noncompliance with the clinical trial protocol, 
International Council on Harmonisation Good Clinical Practice (ICH GCP), or Manual of Procedures (MOP) 
requirements. The noncompliance may be either on the part of the participant, the investigator, or the 
study site staff. As a result of deviations, corrective actions will be developed by the site and implemented 
promptly.  
 
It will be the responsibility of the site investigator to use continuous vigilance to identify and report 
deviations within 2 working days of identification of the protocol deviation. All deviations will be 
addressed in study source documents, reported to NIA Program Official. Protocol deviations will be sent 
to the USC Institutional Review Board (IRB) per their policies. The site investigator will be responsible for 
knowing and adhering to the reviewing IRB requirements. Further details about the handling of protocol 
deviations will be included in the MOP. 
10.1.11 
PUBLICATION AND DATA SHARING POLICY  
This study will be conducted in accordance with the following publication and data sharing policies and 
regulations: 
 
National Institutes of Health (NIH) Public Access Policy, which ensures that the public has access to the 
published results of NIH funded research. It requires scientists to submit final peer-reviewed journal 
manuscripts that arise from NIH funds to the digital archive PubMed Central upon acceptance for 
publication. 
 
This study will comply with the NIH Data Sharing Policy and Policy on the Dissemination of NIH-Funded 
Clinical Trial Information and the Clinical Trials Registration and Results Information Submission rule. As 
such, this trial will be registered at ClinicalTrials.gov, and results information from this trial will be 
submitted to ClinicalTrials.gov. In addition, every attempt will be made to publish results in peer-reviewed 
journals.  
There are restrictions with sharing this data. The data will be extracted from CURES 2.0 (Controlled 
Substance Utilization Review and Evaluation System), a database of Schedule II, III and IV controlled 
substance prescriptions dispensed in California serving the public health, regulatory oversight agencies, 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
23 
and law enforcement. Data may be made available through data use agreements among researchers 
interested in using the data, the Los Angeles County Medical Examiner's Office and the California State 
Department of Justice. 
10.1.12 
CONFLICT OF INTEREST POLICY 
The independence of this study from any actual or perceived influence is critical. Therefore, any actual 
conflict of interest of persons who have a role in the design, conduct, analysis, publication, or any aspect 
of this trial will be disclosed and managed. Furthermore, persons who have a perceived conflict of interest 
will be required to have such conflicts managed in a way that is appropriate to their participation in the 
design and conduct of this trial. The study leadership in conjunction with the NIA has established policies 
and procedures for all study group members to disclose all conflicts of interest and will establish a 
mechanism for the management of all reported dualities of interest. 
10.2 ADDITIONAL CONSIDERATIONS 
 
10.3 ABBREVIATIONS AND SPECIAL TERMS 
 
AE 
Adverse Event 
CURES 
Controlled Substance Utilization Review and Evaluation System 
DSMB 
Data Safety Monitoring Board 
HIPAA 
Health Insurance Portability and Accountability Act  
IRB 
Institutional Review Board 
MME 
Milligram Morphine Equivalent 
NCT 
National Clinical Trial 
NIH  
National Institutes of Health 
PI 
Principal Investigator 
SAE 
Serious Adverse Event 
SAP 
Statistical Analysis Plan 
SMC 
Safety Monitoring Committee 
SOA 
Schedule of Activities 
UP 
Unanticipated Problem 
US 
United States 
 
 
 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
24 
10.4 PROTOCOL AMENDMENT HISTORY 
The table below is intended to capture changes of IRB-approved versions of the protocol, including a 
description of the change and rationale. A Summary of Changes table for the current amendment is 
located in the Protocol Title Page.  
 
Version 
Date 
Description of Change  
Brief Rationale 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Comparative Effectiveness of Two Letters to Encourage Judicious Prescribing of Opioids: A County-wide Project in 
Los Angeles 
 
 
 
 
 
 
 
 
Version 1.0  
Grant #: P30AG024968 
 
 
 
 
 
 
 
04 April 2019 
 
NIH Protocol Template for Behavioral and Social Sciences 
Research 
 
25 
11 REFERENCES  
D. Boudreau et al., “Trends in long-term opioid therapy for chronic non-cancer pain,” 
Pharmacoepidemiol. Drug Saf., vol. 18, no. 12, pp. 1166–1175, 2009. 
 
J. N. Doctor et al., “Opioid prescribing decreases after learning of a patient’s fatal overdose,” 
Science, vol. 361, no. 6402, pp. 588–590, Aug. 2018. 
 
D. Dowell, T. M. Haegerich, and R. Chou, “CDC Guideline for Prescribing Opioids for Chronic Pain - 
United States, 2016,” MMWR Recomm. Rep., vol. 65, no. 1, pp. 1–49, Mar. 2016. 
 
C. S. Florence, C. Zhou, F. Luo, and L. Xu, “The Economic Burden of Prescription Opioid Overdose, 
Abuse, and Dependence in the United States, 2013,” Med. Care, vol. 54, no. 10, pp. 901–906, 
Oct. 2016. 
G. P. Guy Jr et al., “Vital Signs: Changes in Opioid Prescribing in the United States, 2006-2015,” 
MMWR Morb. Mortal. Wkly. Rep., vol. 66, no. 26, pp. 697–704, Jul. 2017. 
 
D. N. Juurlink, “Rethinking ‘doing well’ on chronic opioid therapy,” CMAJ, vol. 189, no. 39, pp. 
E1222–E1223, Oct. 2017. 
 
E. E. Krebs et al., “Effect of Opioid vs Nonopioid Medications on Pain-Related Function in Patients 
With Chronic Back Pain or Hip or Knee Osteoarthritis Pain: The SPACE Randomized Clinical 
Trial,” JAMA, vol. 319, no. 9, pp. 872–882, Mar. 2018. 
 
K. Kroenke et al., “Challenges with Implementing the Centers for Disease Control and Prevention 
Opioid Guideline: A Consensus Panel Report,” Pain Med., vol. 20, no. 4, pp. 724–735, Apr. 
2019. 
 
L. Scholl, “Drug and Opioid-Involved Overdose Deaths — United States, 2013–2017,” MMWR Morb. 
Mortal. Wkly. Rep., vol. 67, 2019, doi: 10.15585/mmwr.mm6751521e1.