PRescribing INterventions for Chronic Pain Via the Electronic Health Record Study - Opioid-Naive Population
Phase: NA
Conditions: Opioid-use Disorder, Opioid Use, Opioid Abuse
Sponsor: University of Minnesota

 
 
Page 1 of 37 
 
Cover Page 
 
Study Title: PRescribing Interventions for Chronic pain using the Electronic Health record (PRINCE) 
 
ClinicalTrials.gov #: NCT04601493 
 
Study Protocol with Statistical Analysis Plan 
 
Document date: May 9, 2022 
 
 
 

 
 
Page 2 of 37 
 
 
 
 
Protocol Title 
PRescribing INterventions for Chronic pain via the Electronic 
health record (PRINCE) study 
Principal 
Investigator/Faculty 
Advisor 
Name: Ezra Golberstein 
Department: Division of Health Policy and Management 
Telephone Number: 612-626-2572 
Email Address: egolber@umn.edu 
Student 
Investigator 
Name: 
Current Academic Status (Student, Fellow, Resident): 
  
Department: 
Telephone Number: 
Institutional Email Address: 
Scientific 
Assessment 
Nationally-based, federal funding organizations 
Version 
Number/Date: 
Version 1.5, May 9, 2022 
 

 
 
Page 3 of 37 
 
REVISION HISTORY 
 
Revision # 
Version Date 
Summary of Changes 
Consent Change? 
1 
July 29, 2020 
Minor edits to web survey text, 
change in data storage plan for 
web survey data, per HIPCO 
No 
2 
November 
30, 2020 
Edits to the plan for the PDMP 
data acquisition, storage, and 
analysis. 
 
3 
December 8, 
2020 
Revised interim outcomes analysis 
plan approved by Data Monitoring 
Committee, and uploading the 
approved Data Monitoring 
Committee charter. 
No 
4 
February 24, 
2021 
Including report from Data 
Monitoring Committee and 
changes to protocol to reflect 
those recommendations. 
No 
5 
May 9, 2022 
Minor changes to the analysis plan 
that were made before starting 
the actual data analysis 
No 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
Page 4 of 37 
 
Table of Contents 
 
1.0 
Objectives 
6 
2.0 
Background 
6 
3.0 
Study Endpoints/Events/Outcomes 
7 
4.0 
Study Intervention(s)/Interaction(s) 
8 
5.0 
Procedures Involved 
13 
6.0 
Data Banking 
17 
7.0 
Sharing of Results with Participants 
18 
8.0 
Study Duration 
18 
9.0 
Study Population 
18 
11.0 Number of Participants 
21 
12.0 Recruitment Methods 
21 
13.0 Withdrawal of Participants 
22 
14.0 Risks to Participants 
23 
15.0 Incomplete Disclosure or Deception 
23 
16.0 Potential Benefits to Participants 
23 
17.0 Statistical Considerations 
24 
18.0 Health Information and Privacy Compliance 
29 
19.0 Confidentiality 
33 
20.0 Provisions to Monitor the Data to Ensure the Safety of Participants 
33 
21.0 Compensation for Research-Related Injury 
34 
22.0 Consent Process 
34 
23.0 Setting 
35 
24.0 Multi-Site Research 
36 
25.0 Coordinating Center Research 
36 
26.0 Resources Available 
36 
27.0 References 
36 
 

 
 
Page 5 of 37 
 
ABBREVIATIONS/DEFINITIONS 
● CDC: Centers for Disease Control and Prevention 
● QI: Quality Improvement 
● EHR: Electronic Health Record 
● (1)ICS: Informatics Consulting Services 
● MMB: Minnesota Management and Budget 
● MME: Morphine Milligram Equivalent 
● PCP: Primary Care Provider 
● PDMP: prescription drug monitoring program 
● PMP: Prescription Drug Monitoring Program 
● MMB: Minnesota Management and Budget 
● UMP: University of Minnesota Physicians 
● DEA: Drug Enforcement Agency 
 
 

 
 
Page 6 of 37 
 
1.0 
Objectives 
1.1 Purpose:  
The objective of this research is to assess the effects of EHR-based decision support 
tools on primary care clinician decisionmaking around pain treatment and opioid 
prescribing. EHR-based clinical decision support tools are a common feature of QI 
The decision support tools are informed by principles of “behavioral economics,” 
whereby clinicians are “nudged,” though never forced, towards guideline-
concordant care. 
To test the effects of these decision support tools for improving the quality of care 
for pain treatment, we will implement a clinic-randomized QI study across the 
primary care clinics of Fairview Medical Group and University of Minnesota 
Physicians. 
The study has two parallel components. The decision support tools to be tested will 
differ somewhat depending on whether a given patient is opioid-naïve, or whether a 
given patient is a current opioid-user. Four sets of analyses will be conducted 
separately: one for the opioid-naïve group using EHR data, one for the current 
opioid-user group using EHR data, one at the PCP-level using web survey data, and 
one at the PCP-level using MN Prescription Drug Monitoring Program data. 
2.0 
Background 
2.1 Significance of Research Question/Purpose:  
Three of the key dimensions to addressing the current opioid crisis are a) how to 
minimize the number of patients who are starting opioids and therefore at risk of 
developing dependence, b) ensuring that opioids that are prescribed are prescribed 
appropriately and safely, and c) how to effectively and safely reduce prescription 
opioid use among current users of prescription opioids. 
Prior research establishes that behavioral economics-informed interventions that 
are embedded within the EHR can successfully improve clinicians’ decisionmaking. 
For example, an influential paper by Meeker and colleagues (1), showed that EHR-
based “nudges,” similar to what we propose to test, successfully reduced rates of 
inappropriate antibiotic prescribing. A few studies use quasi-experimental methods 
to examine how changes in the EHR infrastructure affect opioid prescribing 
decisions, with mixed results.  
As the opioid crisis continues, there is an urgent need to develop effective methods 
for improving pain treatment and opioid prescribing in primary care. Behavioral 
economics-informed interventions may be especially useful in the context of pain 
treatment and opioid prescribing. Specifically, decisions about opioid versus non-
opioid treatments, monitoring opioid use, initial prescribing, and decisions around 
tapering and/or discontinuing opioids are idiosyncratic to the patient. Therefore, 
interventions which nudge towards treatment guidelines, without compelling any 
clinical action, may be especially promising. 

 
 
Page 7 of 37 
 
We are proposing to implement a primary care clinic-randomized study of two sets 
of behavioral economics-informed interventions embedded in the EHR, to improve 
pain treatment and opioid prescribing decisions. 
2.2 Preliminary Data:  
We recruited five clinicians to pilot-test the interventions, and one additional PCP 
from the PRINCE study team (Dr. Elert) also pilot-tested the interventions. The pilot 
testers included a mix of three PCPs and three pain clinicians. The pilot-testing took 
place from 11/27/18 to 12/25/18. We received extensive feedback in the semi-
structured exit interviews about how to improve the usability of the interventions, 
which informed the modifications we subsequently made to the interventions 
2.3 Existing Literature: There is a recent and growing literature that studies how EHR-
based “nudges” can affect clinician behavior around opioid prescribing and pain 
treatment. For example, two recent studies found that reducing the default number 
of pills per opioid prescription order within the EHR led to reduced amount of 
opioids prescribed after surgical procedures and at emergency department 
discharge (2, 3). However, other research on nudges to improve opioid prescribing 
found that introducing safety alerts in the EHR for when a patient was being co-
prescribed an opioid and a benzodiazepine was not associated with changes in 
prescribing patterns (4). Overall, there are many open questions about the types of 
EHR-based alerts and reminders can effectively improve opioid prescribing patterns 
and pain treatment decisions. 
Another important component of safe and appropriate opioid prescribing is the CDC 
guideline recommendation to check the PDMP to identify potentially problematic 
patterns of opioid use. A number of studies examine the effects of laws that 
mandate providers to check their state’s PDMP on prescribing patterns and on 
opioid use patterns (5). Other research notes that many providers do not check the 
PDMP when prescribing opioids, or do so relatively infrequently (6-8). Reasons for 
the underuse of PDMPs by prescribers include that the data in the PDMP are difficult 
and time-consuming to access (7, 8). One approach to “nudge” providers to use the 
PDMP more frequently and effectively is to simply make it much easier to use, by 
integrating it directly into a patient’s EHR. Very limited evidence to date exists on 
the effects of PDMP integration into the EHR (9). We are not aware of any 
randomized trials of PDMP integration into the EHR. 
3.0 
Study Endpoints/Events/Outcomes 
3.1 Primary Endpoint/Event/Outcome:  
Opioid-Naïve Population: Whether or not an opioid is prescribed in a primary care 
visit, without currently receiving a non-opioid alternative pain treatment (including a 
new order for a non-opioid pain treatment). 
Current Opioid-User Population: A variable with three mutually-exclusive 
categories: 

 
 
Page 8 of 37 
 
Category 1 (appropriate taper): Whether a primary care appointment with someone 
currently receiving a “high risk” opioid had an order that would reduce MME by no 
greater than 20%, relative to the current prescription, and there is documented 
evidence that the reduction was consistent with CDC guidelines. “High risk,” is 
defined as a patient is currently receiving opioids with >50 MME and/or currently 
receiving both an opioid and a benzodiazepine. 
Category 2 (“inappropriate” taper): Whether a primary care appointment with 
someone currently receiving a “high risk” opioid had an order that would reduce 
MME without documented evidence that the reduction was consistent with CDC 
guidelines, or, decreased MME by greater amounts than recommended (>20% 
relative reduction in MME). 
Category 3 (no taper): Whether a primary care appointment with someone currently 
receiving a “high risk” opioid had no reduction in MME. 
3.2 Secondary Endpoint(s)/Event(s)/Outcome(s):  
Opioid-Naïve Population: 
1) Whether a CDC-recommended non-opioid treatment is ordered in a primary care 
visit (e.g., referral to physical therapy, prescription of an NSAID, referral to pain 
clinic) 
2) The length of opioid prescriptions 
3) The MME of opioid prescriptions 
 
Current opioid user Population: 
1) Whether there was a partial reduction in the MME or prescription length of refill 
orders, versus a total opioid discontinuation 
2) Whether there was an increase in the MME/day for current opioid users with at 
least 50 MME/day 
 
All PCPs, including visits with opioid-naïve and current opioid-using patients: 
1) The frequency with which PCPs check the PMP. 
2) PCP satisfaction with decision support 
 
Exploratory outcome: 
For primary care appointments with someone currently receiving a “high risk” 
opioid, who did not get tapered, we will assess how frequently there is a 
documented clinical rationale for the decision to not taper, that is consistent with 
the CDC’s guidelines. This outcome is not linked to the experimental study arms. 
4.0 
Study Intervention(s)/Interaction(s) 
4.1 Description:  
The subjects of the study are primary care clinicians working in Fairview and in 
University of Minnesota Physicians. The interventions to be tested in this study are 

 
 
Page 9 of 37 
 
sets of EHR clinical decision support tools (some of which are referred to as, “Best 
Practice Alerts”). These tools are very similar to many other EHR tools and alerts that 
clinicians encounter and use in their practice, and which are frequently adopted by 
health systems.  
There are two sets of interventions to be tested. Both of these intervention sets use 
behavioral economics principles around “choice architecture.” One set includes a 
pair of EHR alerts – one that triggers for patients who are opioid-naïve (i.e., have not 
had an active opioid prescription in the past six months), and one that triggers for 
patients who are current opioid users. The other intervention is essentially the same 
for patients who are opioid-naïve or current opioid users. 
These interventions will be delivered to all of the PCPs working in Fairview and UMP 
primary care clinics that are assigned to one of the intervention arms of the study 
(randomization arms are described in Section 5).  
Detailed description of the Intervention Set A (choice architecture): 
 
For the opioid-naïve group, the interventions will “fire” when a PCP initiates an 
opioid order within the EHR for a patient who has not had an opioid prescription 
within the past six months. The alert, shown in Exhibit 1a, provides guidance 
language about opioid prescribing and prompts the PCP to open the “SmartSet” to 
order non-opioid treatment alternatives. PCPs can choose to ignore this, but 
opening the SmartSet is the default option. When the SmartSet is opened, PCPs can 
choose to click on a variety of treatment order options, including both non-opioid 
pharmacological options and non-pharmacological options (e.g., referral to physical 
therapy or pain clinic). Clicking on a category expands the selections, and Exhibit 1b 
shows the SmartSet with the non-pharmacologic options expanded. The non-opioid 
treatment options were selected for the intervention based on the CDC’s chronic 
pain recommendations and the Fairview Opioid Network Team’s recommendations.  
 
Exhibit 1a. 

 
 
Page 10 of 37 
 
 
 
Exhibit 1b. 
 
 
For the current-opioid user group, the interventions will “fire” when a refill order 
for an opioid is initiated or a new opioid order is initiated for a patient currently 
receiving an opioid. The alert, shown in Exhibit 2, prompts PCPs to consider 
tapering the patient’s opioid. The alert also displays the MME of the patient’s 
current opioid prescription and automatically calculates what a 10% reduction in 
MME relative to the current prescription would be. The alert contains options to 
either cancel the refill order, or to continue with the order. 
 
Exhibit 2. View of the “opioid taper” alert. 

 
 
Page 11 of 37 
 
 
 
Detailed description of the Intervention Set B (PMP Integration): 
 
For all PCPs assigned to this intervention, they will have integrated access to the 
PMP embedded within the EHR. All clinicians can already access the PMP to look up a 
patient’s prior opioid prescriptions and prescription fills. However, this process involves 
signing in to the separate PMP website and can be complicated and time-consuming 
within typical clinical workflow. The integrated PMP tool makes it much easier and faster 
for a PCP to access the PMP information for a given patient. This is a proprietary tool 
called “PMP Gateway,” developed by Appriss, and is being paid for by the state of 
Minnesota. In this tool, there is a link that is embedded directly within the EHR, and does 
not require the PCP to re-enter credentials and password, and automatically shows the 
relevant information for the patient whose EHR is already open. We show screenshots for 
the integrated PMP tool below in Exhibits 4-5.  
 
Exhibit 4. Accessing the integrated PMP in Epic 
  

 
 
Page 12 of 37 
 
 
Exhibit 5. View of the integrated PMP 
 
 
The integrated PMP tool also has a “reminder” function that we will use. This 
reminder is triggered if the PCP attempts to sign an opioid prescription in the EHR without 
having checked the PMP within the past 24 hours. A screenshot example of the PMP 
reminder is shown in Exhibit 6. 
 
 
Exhibit 6. View of the PMP reminder 

 
 
Page 13 of 37 
 
 
 
 
 
The study team will collect four types of data. 
1) Passively-collected data from the Fairview and UMP EHR systems. These data 
involve no interactions with the participants, and is information that is already 
collected by Fairview IT as part of the EHR system. 
2) Retrospective chart reviews of primary care visits with current opioid users. 
These chart reviews will involve no direct interaction with the study participants. 
3) Passively-collected data from the Minnesota PMP. These data involve no 
interactions with the participants, and is information that is already collected 
automatically by the state of Minnesota. 
4) Web survey data from participants. We will use the University of Minnesota’s 
REDCap system to collect information from consenting primary care providers 
via web surveys. This will include basic demographic and background 
information, information about views on pain treatment and opioid prescribing, 
information about how providers interact with EHR systems, and information 
about satisfaction with clinical decision support for pain and opioid treatment. 
Web surveys will be fielded at the start and again at the end of the 12 month 
study period. All PCPs in the participating primary care clinics will be exposed to 
the study interventions their clinic is assigned to, regardless of whether PCPs 
complete the web survey or not. 
5.0 
Procedures Involved 
5.1 Study Design:  

 
 
Page 14 of 37 
 
The study is a clinic-randomized trial of EHR clinical decision support tools for pain 
treatment and opioid prescribing. We propose clinic-level randomization in order to 
minimize the potential for spillovers or “contamination” of the intervention across 
the study arms. 
The randomization plan is described in the figure below. There are two dimensions 
to the randomization. One dimension is treatment arm within a factorial study 
design: some clinics will be randomized to receive “care as usual,” (Arm 1), some 
clinics will be randomized to receive Intervention set A (Arm 2), some clinics will be 
randomized to receive Intervention B (Arm 3), and some clinics will receive both 
Interventions A and B (Arm 4). The other dimension to randomization occurs among 
the clinics that are assigned to Intervention set A or B, and involves those clinics 
being randomized to three different conditions. 1) The clinic receives the 
intervention set for the entire 12 month study period (Subarm A), 2) the clinic 
receives the intervention set for only the first 6 months of the study period, and 
then reverts to “care as usual,” for Intervention set A and PMP integration without 
the reminder function for Intervention B (Subarm B), or 3) The clinic receives the 
intervention set for the first 6 months, but in the second 6 months, Intervention set 
A will only fire in a random half of clinical encounters and the reminder function will 
only fire in a random half of clinical encounters for Intervention B (Subarm C). 
Exhibit 7. Randomization plan 
 

 
 
Page 15 of 37 
 
5.2 Study Procedures:  
● 
All PCPs who are in clinics that are assigned to Intervention set A or B will 
receive communications from the Medical Directors of their respective clinics 
about the implementation of the new EHR alerts and/or PMP integration, in 
the same way that new best-practice alerts and EHR features are normally 
implemented in Fairview and UMP.  
● 
All PCPs who are in clinics that are assigned to Intervention set A or B will 
receive either Intervention set A or B over the course of the study period. As 
described above, both Intervention Sets A and B are extremely similar to the 
types of best-practice alerts and/or new EHR capabilities that clinicians are 
currently exposed to in their usual clinical practice. 
● 
The study team will receive bi-weekly reports from Fairview IT which will 
include data by study arm on how often the PRINCE alerts and PMP reminders 
are firing, and how often PCPs are using versus dismissing the alerts. 
● 
All PCPs in the Fairview and UMP primary care clinics will be recruited to 
complete brief web surveys at the start and end of the 12-month study period. 
PCPs will be recruited to participate in these surveys via email communications 
from the study team, and written communications from the study team and 
from Fairview and UMP clinical leadership. These web surveys will not collect 
any personal health information. They will collect information on PCPs’ 
background characteristics, knowledge and attitudes about pain treatment, 
opioid therapy, and practice guidelines, and satisfaction with clinical decision 
support for pain treatment and opioid prescribing. The web surveys will be 
implemented using the University of Minnesota’s REDCap system. 
● 
Data from the shared EHR of Fairview and UMP will be used for analysis. These 
data are already collected as part of clinical care, and the research team will 
access these data via the University of Minnesota’s Informatics Consulting 
Service. The data that the Informatics Consulting Service already has on hand 
will be augmented with additional EHR data on the activation of the study’s 
EHR alerts and the engagement with these alerts, along with data from the 
legacy HealthEast (which is now part of Fairview) Epic system which are not 
currently received by the Informatics Consulting Service. Fairview IT will share 
those data with the Informatics Consulting Service, who will merge it with the 
other EHR data that the Informatics Consulting Service already has. All of these 
data will be stored and accessed only on the ICS’s secure data shelter. 
● 
We will conduct retrospective chart reviews of the visit records for all visits 
where someone currently receiving a “high risk” opioid had an order that 
would reduce MME by no greater than 20% relative to the current 
prescription. The study team has already created the chart review tool that will 
be used. These chart reviews will ascertain whether there is any 
documentation of the clinical rationale for tapering that is consistent with the 

 
 
Page 16 of 37 
 
CDC’s guidelines. Each month during the 12-month study period, the study 
team will receive a new data extract from the AHC-IE, from which we will 
identify the relevant visits. We will record the Medical Record Number and 
visit date and store them in a database in the AHC-IE’s server. At the 
conclusion of the 12 month study period, all of these files will be destroyed. 
The PI will share that identifying information with the co-Investigator who will 
lead the chart reviews. For each visit that is reviewed, the study team will 
enter the information that is collected into a secure REDCap database. These 
data will then be shared with the ICS staff, who will link it to the EHR data in 
the ICS’ secure data shelter.  
● 
Data on the number of times per month each PCP checks the PMP, or has an 
assigned delegate check the PMP on their behalf, will be taken directly from 
the Minnesota PMP. The PMP data are held by the state of Minnesota Board of 
Pharmacy. The study team will email the PCPs in the study asking for electronic 
authorization to access their data on how frequently they and their delegates 
check the Minnesota PMP between August 2019 and September 2021. 
For the PCPs who provide this electronic authorization, Fairview IT will provide 
the Drug Enforcement Agency (DEA) prescriber identifier number to the PI. The 
list of DEA numbers will be transferred and stored via the PI’s secure “BOX” 
account. The PI will then transfer the list of DEA numbers to the MN Board of 
Pharmacy, also using BOX for file transfer. The Board of Pharmacy will pull the 
data on monthly number of times the PMP was checked between August 2019 
and September 2021, and then transfer back to the PI using BOX. The study 
team will analyze the PMP data on the PI’s HIPAA-compliant secure server. The 
study team will link information on characteristics of the PCPs (i.e., provider 
type, clinic where they practice, treatment arm assignment, and baseline 
information from the web survey) with the PMP data, and then remove all 
identifiers from the analytic dataset. The file linking study identifiers to the 
PMP data will be stored separately in the PI’s BOX. 
● 
In addition, for primary care appointments with someone currently receiving a 
“high risk” opioid, who did not get tapered, we will assess how frequently 
there is a documented clinical rationale for the decision to not taper, that is 
consistent with the CDC’s guidelines. We will do retrospective chart reviews of 
a random 10% sample of primary care appointments with someone currently 
receiving a “high risk” opioid, who did not get tapered (up to 2,000 visits). This 
will shed light on the appropriateness of decisions to not taper opioids among 
current users. For each visit that is reviewed, the study team will enter the 
information that is collected into a REDCap database. 
 
5.3 Follow-Up:  

 
 
Page 17 of 37 
 
At the end of the 12-month study period, participants will be invited to participate in 
a follow-up web survey that is similar in structure and length to the baseline survey.  
The research team will also examine longer-term opioid prescribing patterns using 
the EHR data alone, up to 24 months after the completion of the study period. 
5.4 
Individually Identifiable Health Information: 
This research will require exact dates of service for the EHR data from the AHC-IE. 
In addition, we will require Medical Record Numbers for the subset of the data 
collection that will involve chart reviews. Names and license numbers of the PCPs 
will be collected. 
5.5 
Modifications to Procedures in Response to Data Monitoring Committee 
recommendations: 
The external Data Monitoring Committee reviewed the results of the interim 
outcomes analysis (see Section 17.1 for details). Their letter to the PI is now added 
to the study IRB files. We will follow their recommendations and implement two 
changes to the study procedures. 
Change #1: Open up the PMP Integration arm to the entire study population at the 
6-month mark. 
Change #2: We will not implement the secondary randomization of the intervention 
arm clinics into three subgroups. Instead, the clinics assigned to the “choice 
architecture” intervention set in Arms 3 and 4 will stay the same for months 6-12 of 
the study period. 
 
 
6.0 
Data Banking 
6.1 Storage and Access: 
After the study is completed, the data will be transmitted to and stored by ICS, for use by other 
researchers including those outside of the study.  
6.2 
Data: 
The following identifiable data elements will be stored after the study completion:  
- 
Provider Names  
- 
Dates, except year  
- 
Geographic data  
- 
Email addresses  
- 
Medical record numbers  
- 
Certificate/license numbers  
- 
Any unique identifying number or code (ex: license numbers of the PCPs) 
6.3 
Release/Sharing: 

 
 
Page 18 of 37 
 
Researchers submit a research proposal to the PI to assess scientific merit of proposed work and 
competing projects or initiatives. The PI or his designee communicates approval, disapproval, or 
request revisions. 
Data will be released according to policies established and maintained by the University of 
Minnesota’s Human Research Protection Program and ICS (here for more details) in compliance 
with all state, federal and institutional policies and guidelines. Data will be released after IRB 
determination and/or approvals are obtained. 
7.0 
Sharing of Results with Participants 
7.1 
Aggregate-level study results will be shared with clinical leadership from Fairview 
and UMP before any results are published in peer-reviewed journals. No identifying 
information will be shared. 
8.0 
Study Duration 
8.1 We anticipate that: 
● 
The duration of each participant’s participation in the study will be 12 months. 
● 
The process of enrolling participants to complete the web surveys will be three 
weeks for the baseline survey and three weeks for the follow-up survey. 
● 
Once the 12-month study period has ended, the data analysis will take an 
additional 9-12 months. 
● 
EHR data alone will be examined up to 24 months after the completion of the 
study period. 
9.0 
Study Population 
9.1 Inclusion Criteria:  
Primary Care Providers: 
The final study sample for outcomes derived from EHR and PMP data will include all 
PCPs from all of the Fairview and UMP study clinics. 
Retrospective chart reviews of primary care visits with current opioid users: 
1. Male or female ≥18 years of age at the primary care visit 
2. Current “high-risk” opioid user as defined by having a current opioid 
prescription with at least 50 MME/day and/or having a current opioid 
prescription along with a current benzodiazepine prescription.   
9.2 Exclusion Criteria:  
Primary Care Providers: 
PCPs will be excluded from the final analytic sample if they work less than 20% FTE. 
Retrospective chart reviews of primary care visits with current opioid users: 

 
 
Page 19 of 37 
 
Patient records corresponding to each PCP will only be included in the analysis if the 
patient has Minnesota Research Authorization. 
9.3 Screening:  
Information on which PCPs work in the study clinics will be provided by staff from 
Fairview and UMP. Information on which patient records have Minnesota Research 
Authorization will be provided by the AHC IE staff. 
10.0 Vulnerable Populations 
10.1 
Vulnerable Populations:  
 
Population / Group 
Identify whether any of the 
following populations will be 
targeted, included (not necessarily 
targeted) or excluded from 
participation in the study.  
Children 
Excluded from Participation 
Pregnant women/fetuses/neonates 
Included/Allowed to Participate 
Prisoners 
Excluded from Participation 
Adults lacking capacity to consent 
and/or adults with diminished 
capacity to consent, including, but 
not limited to, those with acute 
medical conditions, psychiatric 
disorders, neurologic disorders, 
developmental disorders, and 
behavioral disorders 
Excluded from Participation 
Non-English speakers 
Excluded from Participation 
Those unable to read (illiterate) 
Excluded from Participation 
Employees of the researcher 
Excluded from Participation 
Students of the researcher 
Excluded from Participation 

 
 
Page 20 of 37 
 
Undervalued or disenfranchised 
social group 
Excluded from Participation 
Active members of the military 
(service members), DoD personnel 
(including civilian employees) 
Excluded from Participation 
Individual or group that is 
approached for participation in 
research during a stressful situation 
such as emergency room setting, 
childbirth (labor), etc. 
Excluded from Participation 
Individual or group that is 
disadvantaged in the distribution of 
social goods and services such as 
income, housing, or healthcare. 
Excluded from Participation 
Individual or group with a serious 
health condition for which there are 
no satisfactory standard treatments. 
Excluded from Participation 
Individual or group with a fear of 
negative consequences for not 
participating in the research (e.g. 
institutionalization, deportation, 
disclosure of stigmatizing behavior). 
Excluded from Participation 
Any other circumstance/dynamic 
that could increase vulnerability to 
coercion or exploitation that might 
influence consent to research or 
decision to continue in research. 
Excluded from Participation 
 
10.2 Additional Safeguards:  
This study focuses on PCPs. Though we are requesting a waiver of informed consent 
for participation in the overall study which involves exposure to the study 
intervention arms, and analysis of passively-collected data (per Section 22.2). We 
will be obtaining modified web-based consent for the portion of the study where 
we will collect information from PCPs via web surveys. The information form for 

 
 
Page 21 of 37 
 
the web surveys will make clear that supervisors will not know whether or not 
PCPs have participated in the study and that there are no negative consequences 
for not participating in the web surveys. 
This study will likely include some PCPs who happen to be pregnant during the 
study. We think that the risk for this population is minimal, as the interventions 
are just EHR tools to improve opioid prescribing practices, and should have no 
bearing on the health of the PCP participants in any way. 
11.0 Number of Participants 
11.1 Number of Participants to be Consented:  
We expect to enroll approximately 300-350 PCPs in both the baseline and follow-up 
web surveys. 
We expect that 400-450 PCPs will be included in the final analytic samples for the 
analysis of EHR data and PDMP data.  
We expect that 20,000 charts will be reviewed in the retrospective chart reviews of 
primary care visits with current opioid users. 
12.0 Recruitment Methods 
12.1 Recruitment Process:  
All PCPs in Fairview and UMP primary care clinics will be included in the study. Those 
who are in the three intervention arms will receive information about the 
interventions that are going to be delivered in their clinics. 
Active recruitment of participants will only be relevant for the web survey 
components of the study. We will recruit all active PCPs who are working in the 
Fairview and UMP primary care clinics. 
We will include two notices about the upcoming web surveys in Fairview’s weekly 
“Neighborhood News” newsletter. The Medical Director of each primary care clinic 
will email their respective PCPs 1-4 days prior to the first survey invitation, 
explaining that all of the PCPs in the clinic will be receiving emails from the study 
team about participating in the web survey. 
The research team will email all of the eligible PCPs directly to recruit them into the 
web survey. We will use the RedCap system to manage email invitations so that 
reminders will only be sent to people who have not yet responded to the survey. 
The first invitation will be sent two weeks prior to the start of the PRINCE 
interventions being activated. The second invitation will be sent one week prior to 
the start of the PRINCE interventions being activated. The third and final email 
invitation will be sent the day after the PRINCE interventions are activated. 
The research team will email all of the eligible PCPs directly to recruit them to 
provide authorization for access to data on their PMP usage. We will use the RedCap 
system to manage email invitations so that reminders will only be sent to people 

 
 
Page 22 of 37 
 
who have not yet responded to the survey. We will send an introductory email, 
followed by up to three follow-up emails in the three weeks after the introductory 
email. 
12.2 Source of Participants:  
We will obtain the list of all PCPs working in the Fairview and UMP primary care 
clinics from both Fairview and UMP clinical leadership. 
12.3 Identification of Potential Participants:  
● 
The PI has obtained the current list of all PCPs in Fairview and UMP primary 
care clinics, along with email addresses. This information was shared by 
Fairview and UMP clinical leadership. 
● 
Each primary care clinic’s Medical Director will first alert all of their respective 
PCPs about the PRINCE study and the web surveys. The study team will then 
contact the PCPs directly via email to recruit the PCPs for the web surveys. 
● 
For the analysis of EHR data, along with the subset of visits that are identified 
for medical chart reviews, we will be restricted by the AHC-IE to only obtain 
data from patient records where there is Minnesota Research Authorization 
status. 
 
12.4 
Recruitment Materials: The following recruitment materials will be used in this 
study:  
- 
communications from the Medical Directors 
- 
email communications from the study team to recruit PCPs for the brief web surveys 
- 
written communications from the study team to Fairview and UMP clinical leadership 
- 
notices about the upcoming web surveys to be published in Fairview’s “Neighborhood 
News” newsletter. 
- 
Email communication from the Medical Directors to PCPs about the authorization for 
accessing PMP data. 
- 
Email communications from the study team to the PCPs to ask for authorization for 
accessing PMP data. 
Recruitment materials are attached. 
12.5 Payment:  
● 
Each PCP that completes the web survey will be compensated $250. PCPs will 
be eligible to receive this payment for both the baseline and 12-month follow-
up surveys, up to a maximum of $500. 
● 
The research team will mail checks directly to the participating PCPs 2-4 weeks 
after the completion of the survey. 
13.0 Withdrawal of Participants 

 
 
Page 23 of 37 
 
13.1 
Withdrawal Circumstances: There are no anticipated circumstances under which 
participants will be withdrawn from the research without their consent. PCPs will not have 
the ability to withdraw from the study intervention arm their clinic is assigned to. 
However, the nature of the interventions is that they are EHR-based nudges that can 
easily be ignored by the PCP, with no negative consequences to either the PCP or to the 
study. 
13.2 
Withdrawal Procedures: Participants will be free to withdraw from completing the 
web surveys at any time, although their EHR data would still be used for analysis.  
13.3 
Termination Procedures: N/A 
14.0 Risks to Participants 
14.1 Foreseeable Risks:  
The only potential foreseeable risks are  
a) a data security breach, which we consider a minimal risk due to our data security 
plan. The records of this study will be kept private. In any sort of report we might 
publish, we will not include any information that will make it possible to identify a 
subject. Research records will be stored securely and only researchers will have 
access to the records. 
b) discomfort answering web survey questions, which we consider a minimal risk 
because the web survey is designed to avoid questions that might make 
respondents uncomfortable and because respondents can choose to not respond to 
any survey question, and  
c) that the interventions themselves may be a minor nuisance in their day-to-day 
clinical practice. We think that latter risk is minimal because the EHR interventions 
can be ignored with no negative implications for the PCPs or for the study data and 
because these interventions are very similar to EHR alerts that are routinely used 
and introduced in clinical practice. 
14.2 Reproduction Risks:  
Not Applicable. 
14.3 Risks to Others: Not Applicable 
15.0 Incomplete Disclosure or Deception 
15.1 Incomplete Disclosure or Deception:  
● This research will not involve incomplete disclosure or deception. 
16.0 Potential Benefits to Participants 
16.1 Potential Benefits: We do not foresee any immediate direct benefits to participants. 
Potential benefits include helping to improve the design of and future investment in 
EHR-based tools to improve pain treatment in primary care which may improve the 

 
 
Page 24 of 37 
 
experience using the EHR for PCPs and may improve outcomes for patients with pain 
symptoms, as well. 
17.0 Statistical Considerations 
17.1 Data Analysis Plan:  
The first formal statistical interim analysis is planned on the primary endpoint. This interim 
analysis is planned to take place when the study is at least 4 months into the study period. 
Formal interim analyses of the data will be reviewed by the Data Monitoring Committee. The 
Committee will be asked to give advice on whether the accumulated data from the study, 
together with results from other relevant studies, justifies continuing exposure to the study 
interventions. A decision to modify or discontinue all or some portions of the study will be made 
only if the result is likely to convince a broad range of experts. In mid-January, 2020, the Data 
Monitoring Committee will meet to review the interim outcomes analysis. 
The final analysis for the primary outcome is planned to take place 12-months after study. The 
first main report/publication of the study will be prepared when PCPs have completed the 12-
month follow-up survey and data for the primary endpoint has been received and cleaned. 
Longer-term endpoints will be analyzed when all data has been received and cleaned. 
An independent Data Monitoring Committee (DMC) will ensure the integrity throughout the 
study by reviewing outcome data for the study interventions. The DMC will be comprised of a 
statistician, an expert in behavioral economics interventions in a clinical setting, and a physician 
with expertise in trials around opioid prescribing interventions. The DMC will review interim and 
final analysis, data quality and completeness, external factors such as scientific advances which 
may impact the study and make recommendations. Stopping rules for efficacy or futility will be 
specified in the charter at the beginning of the study.    
Interim Outcome Analysis Plan 
The DMC is responsible for assessment of the efficacy of the interventions during the course of 
the study i.e. interim analyses of efficacy endpoints. The study protocol version 1.2 pre-specifies 
the following interim analyses: The DMC will be asked to recommend graduation to the second 
stage of randomization when there is clear and substantial evidence of a treatment difference. As 
a guideline, the Lan-DeMets spending function analog of the Pocock boundaries will be used to 
monitor the primary endpoint comparison for the primary aims. The statistical analysis plan is 
given in the protocol. In brief, we will fit mixed effects logistic regression models of the primary 
outcome (described below) for opioid-naïve patients using data from the 12 months of the pre-
intervention period, and through December 31, 2020 (approximately 4 months of post-
intervention data). These models will include fixed effects for whether or not the encounter was 
at a clinic in the intervention a given intervention arm or received both interventions, indicators 
for study month to account for secular trends, and an indicator for whether an observation takes 
place in an intervention arm clinic and in the post-intervention period. These models will include 
random effects for clinic and PCP (nested in clinic) to account for within-clinic and within-PCP 
correlation. 
 

 
 
Page 25 of 37 
 
We propose a single interim analysis occurring sometime shortly after January 1, 2021, using data 
through December 31, 2020. The planned boundaries are given below. The advantage of using 
the Lan-DeMets spending function is that this permits the DMC during the course of the study to 
require more frequent interim analysis and preserve type I error. If more interim analyses are 
desired by the DMC, the Lan-DeMets spending function will be recalculated and provided with 
each DMC report. 
 
 
Interim 
Analysis 
Critical Z- 
score 
Nominal 
One-sided  
p-vale 
Alpha Spent 
1 
2.16 
0.0155 
0.0155 
Final 
2.20 
0.0139 
0.0095 
 
The analysis of the effect of the intervention on key secondary outcomes will also be provided. 
Should the stopping boundary be crossed at the proposed interim analysis and there is clear 
evidence of an intervention’s effect, we would recommend the DMC to move into phase 2 of the 
study in which clinics in the experimental arms for that intervention would be re-randomized to 
continue with intervention for all opioid prescription or for only 50% of opioid prescriptions. The 
decision to move to phase 2 will be made separate for each intervention component (choice 
architecture nudge and PDMP integration/reminders).  
 
We emphasize that data will only be available for the primary outcome in a timely manner for the 
opioid naïve subpopulation and, therefore, decisions on whether or not to move into stage two 
of randomization will be made based on that subpopulation only.  
 
The primary outcome for the opioid-naïve subpopulation is: Whether or not an opioid is 
prescribed in a primary care encounter, without currently receiving a non-opioid alternative pain 
treatment (including a new order for a non-opioid pain treatment). 
 
Key secondary outcomes to be presented to the DMC are: 
• The frequency with which each PCP checks the Minnesota Prescription Drug Monitoring 
Program (derived from the Minnesota PDMP).  
• For the opioid-naïve subpopulation, whether a CDC-recommended non-opioid treatment 
is ordered in a primary care visit (e.g., referral to physical therapy, prescription of an 
NSAID, referral to pain clinic) 
• For the opioid-naïve subpopulation, the length of opioid prescriptions 
• For the opioid-naïve subpopulation, the MME of opioid prescriptions 
 
Additionally, there is a possibility that data on two secondary outcomes for the current opioid-
using subpopulation will be presented to the DMC: 
• Whether there was a partial reduction in the MME or prescription length of refill orders, 
versus a total opioid discontinuation 

 
 
Page 26 of 37 
 
• Whether there was an increase in the MME/day for current opioid users with at least 50 
MME/day 
 
If at the single interim efficacy analysis, the PDMP integration/reminder intervention show 
substantial efficacy, we would recommend that the PDMP integration be made available to all 
clinics. A decision on what constitutes substantial efficacy will be made in consultation with the 
DMC.    
 
Futility Analysis 
There will be no formal futility analyses presented to the DMC. Given that these intervention 
have small potential adverse effects, the DMC will NOT be asked to recommend early termination 
for futility. 
 
 
17.2 Power Analysis: The proposed design has sufficient power to detect clinically 
meaningful changes in opioid prescription practices from the main effects of the two 
interventions and the control group (10). All power calculations assume a two-sided 
test at the 0.05 significance level. Power for the Primary Aim (comparison of 
intervention arms to the control during months 1-6) is shown in Exhibits 2a-b 
assuming a total of 48 clinical are randomized (equal allocation between Arms 1-4) 
with a conservative estimate of 10 providers per clinic (~12 providers per clinic in the 
FHS/UMP system). The range of the number of visits eligible and the percentage of 
visits with an opioid prescription in the control arm (for both the opioid-naïve and 
opioid-using populations) were based on preliminary data from the EHR from a 
subset of clinics that will be randomized. Meeker et al. (1) assumed an intra-clinic 
correlation of 0.05 but that encounters within the same provider were not further 
correlated. We assumed the correlation for visits within the same clinic (but with 
different providers) was 0.025 and the within-provider correlation would be 0.075 as 
well as considering a more conservative scenario (correlation given on the logistic 
scale (11)). We assumed a 5% and 17% prescription rate in the control arm for the 
opioid-naïve and opioid-using populations, respectively, based on preliminary 
analysis of opioid prescribing in Fairview primary care clinics, and based on 
published research (12). We considered a difference in the percentage of eligible 
encounters with an opioid prescription between the intervention and control groups 
of 3.5% and 5% to be of clinical interest in the two populations. Exhibits 8 and 9 
demonstrate that we have sufficient power across a wide range plausible scenarios 
and maintain adequate power even under smaller effect sizes. These calculations 
are also conservative as they assume 48 clinics, whereas we expect to enroll over 50 
clinics. 
We also have sufficient power to detect differences between 2.5 to 4 percentage 
points across the two intervention sets during the first 6 months of the intervention.  

 
 
Page 27 of 37 
 
 
 
 
 
 
 
 
 
 
17.3 Statistical Analysis:  
We will first empirically verify that randomization was successful by testing whether trends in the 
outcome are equal across the study arms in the 12 months prior to the start of the intervention 
period. Second, we will present descriptive statistics by intervention arm of the outcomes, basic 
characteristics of the PCPs, and clinical and demographic characteristics of the patients seeking 
care whose visits meet the inclusion criteria. We will also produce graphs of the unadjusted 
trends in the outcomes by month over the study period, and broken down by study arm.  
Exhibit 8. Power to detect a difference in proportion of eligible visits with an opioid prescription 
between intervention and control arms. Power calculations assume an opioid prescription rate in 
the no intervention arms of 7%. 
30 eligible visits/month/provider 
15 eligible visits/month/provider 
 
% reduction in opioid 
prescribing   
 
 
% reduction in opioid 
prescribing   
Intra-clinic/ 
Intra-provider 
correlation 
15% 
20% 
25% 
 
30% 
Intra-clinic/ 
Intra-provider 
correlation 
15% 
20% 
25% 
 
30% 
0.025/0.075 
96% 
100% 
100% 
100% 
0.025/0.075 
76% 
95% 
100% 
100% 
Exhibit 9. Power to test a difference in proportion of eligible visits resulting in an opioid 
prescription between intervention and controls arms in opioid-using population (Primary Aim). 
Power calculations assume 40 clinics randomized equally to Arms 1-4 with 10 providers/clinic, 
the opioid prescription rate in the control arm was 17%. 
10 eligible visits/month/provider 
5 eligible visits/month/provider 
 
Difference in opioid 
prescription rate between 
control and intervention  
 
Difference in opioid 
prescription rate between 
control and intervention 
Intra-clinic/ 
Intra-
provider 
correlation 
4.5% 
5% 
5.5% 
Intra-clinic/ 
Intra-provider 
correlation 
4.5% 
5% 
5.5% 
0.025/0.075 
81% 
89% 
95% 
0.025/0.075 
77% 
86% 
93% 

 
 
Page 28 of 37 
 
We will run separate analyses for the opioid-naïve population and the current opioid user 
population. The analysis sample for the opioid-naïve population will be primary care visits for 
patients who have not had an active opioid prescription in the six months prior to the visit, and 
no cancer diagnosis in the 12 months prior to the visit. In order to identify the visits that plausibly 
might lead to an opioid prescription, we will restrict to primary care visits that have a recorded 
diagnosis that occurs regularly in opioid-prescribing visits. We are defining those diagnoses as 
diagnoses that were associated with at least a 3% opioid prescribing rate among opioid-naive 
primary care visits across all FMG and UMP primary care clinics over a 12 month period and there 
were at least 25 opioid orders in visits with the diagnosis for all visits in the study clinics during 
the 12-month period before the study, excluding diagnoses for current opioid use, which is 
intended to have very high sensitivity for capturing chronic pain-related diagnoses, but low 
specificity. We have already created this list, which includes 294 specific diagnoses with pain-
related diagnoses the most common.  
The analysis sample for the current opioid user sample will be all primary care visits for patients 
with an active opioid prescription, excluding patients with a cancer diagnosis in the 12 months 
prior to the visit. We will include both in-person visits along with virtual (telephone or video) 
visits for all analyses. 
Primary Outcome Analysis  
To test our primary hypotheses, we will fit mixed effects logistic regression models using data 
from the 12-month pre-intervention period, and the post-intervention period prior to any 
second-stage randomization (the model for the current “high risk” opioid group will be 
multinomial logistic regression due to the 3-category outcome). All models will include fixed 
effects for whether the encounter was at an intervention arm clinic, an indicator for whether the 
measurement was after the interventions began, and their interaction. The interaction is the 
primary measure of the intervention effect. These models include random effects for clinic and 
PCP (nested in clinic) to account for within-clinic and within-provider correlation. Because of the 
cluster-randomized design with a moderate number of clinics, residual imbalances in patient and 
PCP-characteristics among the randomized groups may exist. The primary analysis model will 
adjust for characteristics at all levels of the hierarchy including clinic-level (indicators for the four 
systems), PCP-level (clinician type [MD/DO, NP, or PA], sex, and length of tenure in medicine), 
and patient-level (age, sex, race/ethnicity, insurance status, and in-person or virtual visit). 
Separate models will be fit to assess the PDMP integration and choice architecture intervention 
and for the opioid-naïve and current, high-risk opioid users. In each model, the intervention 
group pools across two arms of the factorial design (e.g., the choice architecture intervention 
pools across arms 2 and 4) and the control condition pools across the remaining two groups (e.g., 
arms 1 and 3 to continue the example). A Bonferroni correction will adjust inference for multiple 
comparison across two different populations (opioid naïve and current, high-risk opioid users) but 
we will not adjust for multiple tests across the two different interventions.  
 
As an exploratory analysis, we will assess whether PCP characteristics (clinician type and tenure in 
medicine) and patient characteristics (age, sex, and race/ethnicity) moderate the interventions’ 
effect. Tests for treatment effect heterogeneity will be implemented by adding the interaction 
between the potential moderator and the intervention indicator to the models for the primary 

 
 
Page 29 of 37 
 
outcome (adjusting for the same factors described above). The effect of the intervention within 
subgroups formed from the potential moderators will be estimated by fitting separate models 
within each subgroup.  
 
Subgroup analyses will not be adjusted for multiple comparisons; they are supportive to the 
primary outcome analysis. Subgroup analyses will be interpreted with caution due to limited 
power and uncontrolled type I error. 
 
If the study proceeds to the second stage of randomization, we will fit mixed effects logistic 
regression models using data from 12 months pre- and post-intervention. All models will include 
fixed effects for intervention arm (control, continued intervention, 50% intervention, or 
intervention turned-off), indicators for whether the measurement was after the initial 
interventions began or after the second stage randomization, and their interaction. The 
interaction is the primary measure of the intervention effect. The models will include the same 
random effects and covariate adjustment as above.  
 
Secondary Outcomes Analysis: PDMP and Web Survey data 
For other EHR-derived secondary outcomes, we will use the same general modeling framework as 
for the primary outcome but will fit mixed effects logistic or linear models depending on whether 
the outcome was categorical or continuous. 
 
To assess intervention effects on the frequency of checking the PDMP we will fit mixed effects 
Poisson regression models using data from the 12 months pre- and post-intervention. All models 
will include fixed effects for whether the PCP was at an intervention arm clinic, indicator for 
whether the month was during the intervention period, and their interaction. The models will 
adjust for clinician type, length of tenure in medicine, and health system indicators, and include 
random effects for clinic and PCP (nested in clinic) to account for within-clinic and within-provider 
correlation. As an exploratory analysis, we will assess whether PCP characteristics (clinician type 
and length of tenure) moderate the effect of the interventions using a similar process as the 
primary outcome.  
 
A similar approach will be used for the web survey data, except we will fit mixed effect linear 
regression models using data from the pre- and post-intervention surveys. All models will include 
fixed effects for whether the PCP was at an intervention arm clinic, an indicator for whether the 
survey was the 12-month follow-up, and their interaction.  
17.4 Data Integrity: See section 20.2 Data Safety Monitoring 
18.0 Health Information and Privacy Compliance 
18.1 
Select which of the following is applicable to your research: 
☐ My research does not require access to individual health information and 
therefore assert HIPAA does not apply.   
☐ I am requesting that all research participants sign a HIPCO approved HIPAA 

 
 
Page 30 of 37 
 
Disclosure Authorization to participate in the research (either the standalone 
form or the combined consent and HIPAA Authorization). 
☒ I am requesting the IRB to approve a Waiver or an alteration of research 
participant authorization to participate in the research. 
Appropriate Use for Research:  
The analysis of patient records will be restricted to patients with Minnesota 
Research Authorization. The University of Minnesota AHC IE staff will only share 
records with us where there is Minnesota Research Authorization. 
☐ An external IRB (e.g. Advarra) is reviewing and we are requesting use of the 
authorization language embedded in the template consent form in lieu of the U 
of M stand-alone HIPAA Authorization.  Note: External IRB must be serving as 
the privacy board for this option. 
 
18.2 
Identify the source of Private Health Information you will be using for your 
research (Check all that apply)   
☒ I will use the Informatics Consulting Services (ICS) available through CTSI (also 
referred to as the University's Information Exchange (IE) or data shelter) to pull 
records for me 
☒ I will collect information directly from research participants. 
☐ I will use University services to access and retrieve records from the Bone 
Marrow Transplant (BMPT) database, also known as the HSCT (Hematopoietic 
Stem Cell Transplant) database. 
☒ I will pull records directly from EPIC. 
☐ I will retrieve record directly from axiUm / MiPACS 
☐ I will receive data from the Center for Medicare/Medicaid Services  
☐ I will receive a limited data set from another institution  
☒ Other.  Describe: Data on the number of times per month each PCP checks the 
PMP, or has an assigned delegate check the PMP on their behalf, will be taken 
directly from the Minnesota PMP. The PMP data are held by the Minnesota 
Board of Pharmacy. The study team will securely share the names and DEA 
numbers of the PCPs in the PRINCE study with the Board of Pharmacy, and the 
Board of Pharmacy will securely share the data back with the PI for analysis. 
18.3 
Explain how you will ensure that only records of patients who have agreed to have 
their information used for research will be reviewed. 
The Informatics Consulting Service will only provide the research team with 
records of patients who have Minnesota Research Authorization status. 

 
 
Page 31 of 37 
 
Fairview IT will only provide the research team with DEA numbers for PCPs that 
have authorized access to their PMP data, and the Minnesota Board of Pharmacy 
will only share PMP usage data for PCPs who have provided electronic 
authorization for access to their data. 
18.4 
Approximate number of records required for review: 
For the chart review component of the study, we anticipate reviewing up to a total of 
20,000 records using our chart audit tool. These records will need to be retrieved directly 
from Epic because the information we will extract is most likely in the providers’ notes 
and assessments. 
 
18.5 
Please describe how you will communicate with research participants during the 
course of this research.  Check all applicable boxes 
☐ This research involves record review only. There will be no communication with 
research participants. 
☐ Communication with research participants will take place in the course of 
treatment, through MyChart, or other similar forms of communication used 
with patients receiving treatment.  
☒ Communication with research participants will take place outside of treatment 
settings. If this box is selected, please describe the type of communication and 
how it will be received by participants. 
 
We will recruit PCPs into the web survey component of the study by email that will 
be sent from the study team via RedCap. We will recruit PCPs to provide 
authorization for access to their PMP data by email that will be sent from the 
study team via RedCap. 
18.6 
Access to participants 
The research team will only be accessing EHR data and medical records from 
patients who have Minnesota Research Authorization status. 
 
18.7 Location(s) of storage, sharing and analysis of research data, including any links to 
research data (check all that apply).   
☒ In the data shelter of the Information Exchange (IE)  
 ☒ Store 
☒ Analyze 
☐ Share 
☐ In the Bone Marrow Transplant (BMT) database, also known as the HSCT 
(Hematopoietic Stem Cell Transplant) Database  
 ☐ Store 
☐ Analyze 
☐ Share 
☒ In REDCap (recap.ahc.umn.edu)  
 ☒ Store 
☒ Analyze 
☐ Share 

 
 
Page 32 of 37 
 
☐ In Qualtrics (qualtrics.umn.edu) 
 ☐ Store 
☐ Analyze 
☐ Share 
☐ In OnCore (oncore.umn.edu)  
 ☐ Store 
☐ Analyze 
☐ Share 
☐ In the University’s Box Secure Storage (box.umn.edu) 
 ☒ Store 
☐ Analyze 
☒ Share 
☐ In an AHC-IS supported server. Provide folder path, location of server and IT 
Support Contact:  
 ☐ Store 
☐ Analyze 
☐ Share 
 
☐ In an AHC-IS supported desktop or laptop.  
Provide UMN device numbers of all devices: 
 ☐ Store 
☐ Analyze 
☐ Share 
☒ Other.  
Indicate if  data will be collected, downloaded, accessed, shared or stored using a 
server, desktop, laptop, external drive or mobile device (including a tablet computer 
such as an iPad or a smartform (iPhone or Android devices) that you have not 
already identified in the preceding questions 
The PDMP data analysis will take place on a Microsoft virtual server that resides in 
the Office of Information Technology (OIT) Data Center. The server location is “HPM-
RHRC2.ad.umn.edu”. Server operating system is maintained by OIT staff. Access to 
the data center is controlled and monitored.   
☐I will use a server not previously listed to collect/download research data 
☐I will use a desktop or laptop not previously listed 
☐I will use an external hard drive or USB drive (“flash” or “thumb” drives) not 
previously listed  
☐I will use a mobile device such as an tablet or smartphone not previously listed 
 
18.8 Consultants. Vendors. Third Parties.  
We will share some information on PCPs with the Minnesota Board of Pharmacy, for 
them to extract PDMP usage data. The information will include the names and DEA 
numbers of the PCPs.  

 
 
Page 33 of 37 
 
18.9 
Links to identifiable data: The PI has the full list of Fairview and UMP PCPs, 
including names. Once the web survey data is collected, baseline web survey information 
will be transferred to the AHC-IE and the study team will link some baseline survey data to 
the EHR data for analyses, using PCP name as a linkage. After the data are linked, PCP 
names will be removed from the data. 
 
Medical record numbers are needed to identify the records for the chart audits, and to 
then link that information back to AHC-IE data. The list of Medical record numbers will be 
stored on the AHC-IE’s secure server. The PI will only share those Medical record numbers 
with the co-Investigator David Satin who will lead the chart audits. Once the chart audits 
are completed, the chart audit data will be linked back to the rest of the AHC-IE data and 
the Medical record numbers will be deleted from our files. 
 
18.10 Sharing of Data with Research Team Members. EHR data will only be accessed via 
the AHC-IE server. Only team members who are directly involved with data 
analysis will have access to these data. 
 
18.11 Storage and Disposal of Paper Documents: There will be no paper documents with 
research data as part of this study. 
19.0 Confidentiality 
19.1 Data Security: All study team members who will access any of the study data will 
have completed the appropriate HIPAA training and CITI Human Subjects 
Protections trainings. All study data will be stored on password-protected servers 
with two-factor authorization.  
20.0 Provisions to Monitor the Data to Ensure the Safety of Participants 
20.1 Data Integrity Monitoring.  
● 
The study team will conduct monthly reviews of data reports from Fairview IT, 
in order to assess whether the PRINCE alerts are firing as intended, and to 
assess the degree of PCP engagement with the alerts (e.g., how often using 
versus dismissing alerts). 
● 
The study team will receive monthly EHR data extracts from the AHC-IE, and 
the study partners at MMB will receive weekly PDMP data extracts. This will 
allow us to do an interim outcome check using data collected through 
December 31, 2020.  
 
20.2 Data Safety Monitoring.  
It is the responsibility of the Principal Investigator to oversee the safety of the study.  This safety 
monitoring will include careful assessment and appropriate reporting of adverse events. 
 

 
 
Page 34 of 37 
 
21.0 Compensation for Research-Related Injury 
21.1 Compensation for Research-Related Injury: N/A 
Contract Language: N/A 
 
 
22.0 Consent Process 
22.1 Consent Process (when consent will be obtained): N/A 
 
22.2 Waiver or Alteration of Consent Process (when consent will not be obtained, 
required information will not be disclosed, or the research involves deception):  
● 
For participation in the overall study by PCPs (exposed to the study 
interventions), we are requesting a waiver of consent. Per the Common Rule 
(§46.116), informed consent can be waived when, “1) The research involves no 
more than minimal risk to the subjects; (2) The waiver or alteration will not 
adversely affect the rights and welfare of the subjects; [and] (3) The research 
could not practicably be carried out without the waiver or alteration.” We 
believe that we meet all three conditions, as we elaborate below. 
● 
Common Rule Criterion 1 (Research involves no more than minimal risk). The 
proposed research is implementing a clinical quality improvement 
intervention. Clinical QI interventions that remind clinicians of practice 
guidelines and encourage clinical decisions that are concordant with well-
established practice guidelines are common, and already carry minimal risk. In 
order to increase the value of the knowledge from the QI intervention, we will 
be randomly assigning clinics to receive different types of QI interventions. In 
addition, the specific nature of the proposed interventions further ensures 
that there is no more than minimal risk. By definition, the QI interventions to 
be tested do not compel any activity or decision by clinicians. Rather, they are 
designed to merely “nudge,” clinicians towards evidence-based care that is 
supported by consensus guidelines from the CDC. 
● 
Common Rule Criterion 2 (Waiving consent will not adversely affect the rights 
and welfare of the subjects). This study is merely applying randomization to a 
QI intervention that would already be deemed acceptable for clinical 
operations, without requiring consent from clinicians. Because this is a QI 
intervention that would normally be implemented without clinician consent, 
implementing the intervention to a random subset of clinicians will not 
adversely affect the rights and welfare of the subjects. 
● 
Common Rule Criterion 3 (The research could not practicably be carried out 
without the waiver or alteration). As this study is a rigorous evaluation of a QI 
intervention, it is imperative to have information from all clinicians working in 

 
 
Page 35 of 37 
 
the study clinics. Seeking out consent from all of the clinicians would be 
disruptive to the operations of the clinics. And, clinicians who would be willing 
to take the extra time to consent for the study would be different from those 
who do not. As such, the study sample would be biased towards only including 
the most motivated clinicians, which would undermine the value of the study, 
making the research not practicable. 
22.3 Waiver of Written/Signed Documentation of Consent (when written/signed consent 
will not be obtained):  
● 
For the web survey component of the study only, we will obtain informed 
consent from PCPs but waive the signed documentation of consent. Because 
this study component is web-only, the consent process will be online-only. The 
first webpage of the web surveys will be the “information sheet” about the 
study, and at the end of the form participants will type their name to 
electronically “consent” to participation and move on to the actual web survey. 
● 
Participants will have the opportunity to carefully review the information sheet 
and ask questions prior to participating. The PCPs will have the opportunity to 
discuss the study with others or think about it prior to agreeing to participate if 
needed. The PCPs may withdraw consent at any time throughout the course of 
the study and/or survey. 
● 
We request a waiver of written/signed documentation of consent because we 
will be using an online consent process, where the typed signature of the 
participant on the front page of the web survey will be used for consent. 
22.4 Non-English Speaking Participants:  
● 
N/A 
22.5 Participants Who Are Not Yet Adults (infants, children, teenagers under 18 years of 
age): 
● 
Not applicable. 
22.6 Cognitively Impaired Adults, or adults with fluctuating or diminished capacity to 
consent: 
● 
Not applicable. 
22.7 Adults Unable to Consent: 
● 
Not applicable. 
23.0 Setting 
23.1 Research Sites:  
● 
All of the potential participants in the study will be PCPs working in Fairview or 
UMP primary care clinics. 

 
 
Page 36 of 37 
 
● 
The web surveys will be conducted on the work or personal computers of the 
PCPs. The trial interventions will take place within the EHR of each PCP in 
Fairview and UMP primary care clinics. 
● 
Data analysis of the EHR data and the web survey data will be conducted at the 
University of Minnesota Division of Health Policy and Management, and on the 
AHC-IE’s servers. 
● 
Data analysis of PMP data will be conducted at the University of Minnesota 
Division of Health Policy and Management. 
23.2 International Research:  
Not applicable. 
23.3 Community Based Participatory Research:  
● 
Not applicable. 
24.0 Multi-Site Research 
Not applicable.  
25.0 Coordinating Center Research 
Not applicable. 
26.0 Resources Available 
26.1 Resources Available:  
● 
This project is funded by an R33 grant from NIDA, over a three year period. The 
PI’s project time is funded at 30% FTE for all three years. All co-Investigators 
are funded at levels from 5%-15% FTE over the three year period. There is also 
a 50% FTE Project Manager and a 50% FTE doctoral student research assistant. 
● 
For the primary outcomes, we will have EHR data on all of the PCPs who are 
working in the study clinics. We expect this will include approximately 400 
PCPs. For the web survey component, we expect a 75% response rate. 
● 
The research team has all of the required statistical software and computing 
resources needed to complete the research, including departmental resources 
and the AHC-IE’s resources. 
● 
All people assisting with the research will be required to undergo training in 
the research background, objectives, and procedures. The PI will be 
responsible for clearly delineating all duties and functions of the entire 
research team. 
27.0 References 
 

 
 
Page 37 of 37 
 
1. 
Meeker D, Linder JA, Fox CR, Friedberg MW, Persell SD, Goldstein NJ, et al. Effect of 
Behavioral Interventions on Inappropriate Antibiotic Prescribing Among Primary Care Practices: 
A Randomized Clinical Trial. JAMA. 2016;315(6):562-70. 
2. 
Chiu AS, Jean RA, Hoag JR, Freedman-Weiss M, Healy JM, Pei KY. Association of 
Lowering Default Pill Counts in Electronic Medical Record Systems With Postoperative Opioid 
Prescribing. JAMA Surg. 2018;153(11):1012-9. 
3. 
Montoy JCC, Coralic Z, Herring AA, Clattenburg EJ, Raven MC. Association of Default 
Electronic Medical Record Settings With Health Care Professional Patterns of Opioid Prescribing 
in Emergency Departments: A Randomized Quality Improvement Study. JAMA Intern Med. 2020. 
4. 
Smith LB, Golberstein E, Anderson K, Christiaansen T, Paterson N, Short S, et al. The 
Association of EHR Drug Safety Alerts and Co-prescribing of Opioids and Benzodiazepines. J 
Gen Intern Med. 2019;34(8):1403-5. 
5. 
Schuler MS, Heins SE, Smart R, Griffin BA, Powell D, Stuart EA, et al. The state of the 
science in opioid policy research. Drug Alcohol Depend. 2020;214:108137. 
6. 
Irvine JM, Hallvik SE, Hildebran C, Marino M, Beran T, Deyo RA. Who uses a 
prescription drug monitoring program and how? Insights from a statewide survey of Oregon 
clinicians. J Pain. 2014;15(7):747-55. 
7. 
Rutkow L, Turner L, Lucas E, Hwang C, Alexander GC. Most primary care physicians are 
aware of prescription drug monitoring programs, but many find the data difficult to access. Health 
Aff (Millwood). 2015;34(3):484-92. 
8. 
Blum CJ, Nelson LS, Hoffman RS. A survey of Physicians' Perspectives on the New York 
State Mandatory Prescription Monitoring Program (ISTOP). J Subst Abuse Treat. 2016;70:35-43. 
9. 
Holmgren AJ, Apathy NC. Evaluation of Prescription Drug Monitoring Program 
Integration With Hospital Electronic Health Records by US County-Level Opioid Prescribing 
Rates. JAMA Netw Open. 2020;3(6):e209085. 
10. 
Stroup WW. Living with Generalized Linear Mixed Models. SAS Global Forum. 
2011;Paper 349-2011. 
11. 
Wu S, Crespi CM, Wong WK. Comparison of methods for estimating the intraclass 
correlation coefficient for binary responses in cancer prevention cluster randomized trials. 
Contemp Clin Trials. 2012;33(5):869-80. 
12. 
Burgess DJ, Nelson DB, Gravely AA, Bair MJ, Kerns RD, Higgins DM, et al. Racial 
differences in prescription of opioid analgesics for chronic noncancer pain in a national sample of 
veterans. The journal of pain : official journal of the American Pain Society. 2014;15(4):447-55.