Digital Therapeutics for Opioids and Other Substance Use Disorders in Primary Care Pilot (DIGITS Trial Pilot)
Phase: NA
Conditions: Drug Use Disorders, Illicit Drug Use, Alcohol-Related Disorders
Sponsor: Kaiser Permanente

 
1 
 
DIGITS Pilot Protocol and Statistical Analysis Plan 
Version 2.0 
 
NCT #04907045 
 
5/15/2024 
 
Joseph E. Glass 
 

 
2 
 
DIGITS Pilot Protocol and Statistical Analysis Plan 
 
The DIGITS pilot study was conducted to evaluate the feasibility of integrating digital 
therapeutics into primary care and to prepare for the DIGITS Trial. In this study, we 
quantitatively examine implementation during the pilot study by describing the 
proportion of eligible patients reached by the digital therapeutics and fidelity related to 
ideal use. The pilot study included the enactment of several implementation strategies 
to be used in the future DIGITS Trial: practice facilitation (PF), health coaching (HC), 
and standard implementation (SI). 
 
The DIGITS pilot study took place at Kaiser Permanente Washington (KPWA), a large 
integrated health system in Washington State. We selected two primary care clinics 
(referred to as Clinics A and B) in Seattle, WA for the pilot based on the presence of 
highly skilled licensed independent clinical social workers (LICSWs) willing to participate 
in a pilot study of digital therapeutics. We also intentionally selected clinics of disparate 
sizes. Clinic A was a large outpatient medical center with approximately 29,000 primary 
care empaneled patients. Clinic B was a newer and smaller medical center with 
approximately 2,500 empaneled patients. 
 
Implementation Strategy Interventions 
 
The SI strategy involved a 2-hour training, quality improvement meetings, an 
implementation toolkit consisting of clinician workflow aids, patient pamphlets, and 
scripts to help clinicians introduce the digital therapeutic to patients, and electronic 
health record (EHR) tools including documentation/charting templates and after-visit-
summaries, an order set, and a population management workbench (i.e., EHR registry). 
The PF strategy is an evidence-based clinician-facing implementation strategy designed 
to support clinicians in overcoming implementation and workflow challenges. HC is a 
patient-facing strategy designed to support patient self-management and engagement 
in treatment.  
 
Digital Therapeutic Intervention 
 
Digital therapeutic prescriptions were entered into the EHR by LICSWs and routed to a 
medical doctor for approval. The digital therapeutics implemented were reSET® and 
reSET-O®, prescription smartphone apps made by Pear Therapeutics which had been 
authorized by the United States Food and Drug Administration (FDA) for the treatment 
of SUDs (not including alcohol use disorders, unless they are accompanied by drug 
use) and opioid use disorder (OUD; only when accompanied by prescription 
buprenorphine use), respectively. reSET is a 90-day prescription and reSET-O is an 84-
day prescription. Both apps leveraged cognitive-behavioral approaches and are shown 
to improve patient outcomes in randomized controlled trials conducted in specialty 
addiction treatment settings. 

 
3 
 
Timeline and population 
 
Pilot activities involved two distinct three-month periods. During the first period (2/10/21 
to 5/5/21), one LICSW from each of the two pilot clinics was trained to offer reSET and 
reSET-O. During this time, LICSWs offered the apps to patients in the absence of PF or 
HC. This supported our goal of testing the pilot standard implementation materials. 
During the second period (5/6/21 to 8/6/21), Clinic A received both PF and HC, and 
Clinic B received HC only.  
 
The study’s quantitative outcome evaluation relied on secondary data sources. We 
report on two analytic cohorts: 1) a “main analytic cohort” consisted of patients who met 
pre-defined eligibility criteria, allowing us to calculate reach of the digital therapeutic in a 
defined patient population (which was consistent with the main trial eligibility criteria), 
and 2) a secondary “prescribed cohort”, which consisted of all patients prescribed the 
digital therapeutic, regardless of whether they met criteria for the analytic cohort.  
 
The main analytic cohort consisted of patients at least 18 years of age who had a 
primary care visit at Clinic A or B during the accrual period (defined below) and had a 
high-scoring screen for drug use the day of the visit or at any time in the prior year. A 
high-scoring screen was defined as a score of four on the Single Item Screen for 
Cannabis (SIS-C) (the highest score, indicating daily or almost daily cannabis use) 
and/or a score greater than zero for other drugs on the Single Item Screen for Drugs 
(SIS-D) (indicating any past year use of other drugs). Patients who have requested 
through the health system not to be contacted for research or not to have their chart 
reviewed for research were excluded. We included patients who had high-scoring 
substance use screens rather than patients with documentation of an active SUD 
diagnosis in analyses to avoid the potential for identification bias. 
 
The second cohort, a prescribed cohort, consisted of patients who were prescribed the 
digital therapeutics. Patients in this cohort did not necessarily meet criteria for the 
analytic cohort (e.g., they may not have had a high-scoring drug screen or a visit at 
Clinic A or B during the accrual period or before their prescription) but were determined 
by clinicians to meet eligibility criteria for one of the digital therapeutics. For example, a 
patient who indicated on the SIS-C that they used cannabis weekly (as opposed to 
daily) would not have met criteria for the analytic cohort, but a clinical discussion may 
have revealed a need for SUD treatment. 
 
Data collection and analysis 
 

 
4 
 
Data was collected from the EHR for an accrual period and an outcomes assessment 
period. The intended launch date was 2/10/21 (see above), but technical problems 
delayed reSET and reSET-O account creation and prescriptions until 3/2/21. 
Specifically, clinicians could not access the vendor’s website from web browser versions 
that were installed in clinical offices; to resolve this, the app vendor ultimately modified 
their website. In addition, the EHR order set did not initially route prescriptions to 
physicians, which required EHR programmer intervention. We set a patient accrual 
period of 2/23/21 to 8/6/21, which captures everyone who was prescribed the digital 
therapeutics, including patients who may have visited clinics slightly before the technical 
problems were resolved. The outcomes assessment period (2/23/21 to 11/12/21) 
included the accrual period (since the outcome of reach of the digital therapeutic, 
defined below, could happen as early as the day of eligibility), a one-week grace period 
(to allow patients who became eligible on the last day of the accrual period to have at 
least one week to be reached), and three follow-up months to capture app use for 
patients who had a 12-week digital therapeutic prescription at any point during the pilot.  
 
Baseline characteristics from the EHR included demographics, address, insurance 
status, healthcare utilization, and mental health and SUD screening information and 
diagnoses. For patients having multiple eligible visits (those where they were 18 or older 
and had a high-scoring screen), we used the first eligible visit as the “qualifying visit” on 
which to anchor time-varying covariates and outcomes. For patients who were 
prescribed the digital therapeutics but did not have an eligible visit, we used the 
prescription date as the qualifying visit. The time-period for constructing baseline clinical 
characteristics was from two years before until the day before the qualifying visit. Age 
was measured on the date of the qualifying visit. Sex (binary male/female), race, and 
ethnicity were from the time of the data pull to get the most current data. Insurance and 
address (used to classify rurality/urbanity was defined using the 2010 rural-urban 
commuting area codes and 2010 Census Tract geocoding) were measured on the date 
of or in the month before the qualifying visit, depending on data availability.  
 
Our primary outcomes were clinic-level measures of reach and fidelity. Reach was 
defined as the number and proportion of patients who were prescribed reSET or reSET-
O, downloaded the app and activated their prescription, and completed at least one 
module within the app. Fidelity was defined as the average (across patients) of the 
number of weeks (out of 12 possible weeks) in which patients completed at least four 
modules per week and had a past-month SUD-related visit. This definition of fidelity is 
based on Pear Therapeutic’s recommendation that patients complete four modules per 
week (the maximum number of modules per week for which a patient can receive a 
contingency management reward) and the FDA label requirement and clinical 
recommendation that patients use the digital therapeutic while under the care of a 

 
5 
 
clinician (our health system leaders determined monthly SUD-related visits satisfied this 
requirement). We used a database provided by the vendor to obtain data related to app 
use such as activations and module completion during the outcomes assessment 
period. 
 
Additional outcomes were treatment engagement and additional reach and fidelity 
measures. Treatment engagement was defined as the average (across patients) of the 
number of months in which patients had at least one visit for SUD treatment. Additional 
reach measures (which were subcomponents of the primary reach outcome measure) 
were the number and proportion of patients who were prescribed reSET or reSET-O 
(regardless of whether they downloaded the app and activated their prescription) and 
the number and proportion of patients who were prescribed the app and subsequently 
downloaded it and activated their prescription. Additional fidelity measures were the 
average number of weeks (out of 12 possible weeks) in which patients completed at 
least one module per week or completed at least four modules per week. Treatment 
engagement and fidelity-3 were subcomponents of the primary fidelity outcome 
measure. Adoption was measured to record if clinicians other than LICSWs completed 
training to prescribe the digital therapeutics.  
 
Table 1. DIGITS pilot study outcome measures 
Measure 
Definition 
Primary outcomes 
Reach 
The number and proportion of patients who were prescribed 
reSET or reSET-O, downloaded the app and activated their 
prescription, and completed at least one module within the 
app. 
Fidelity 
The average of the sum of number of weeks (out of 12 
possible weeks) in which patients completed at least four 
modules per week and had a past-month SUD-related visit. 
Additional pre-specified outcomes 
Treatment 
engagement 
The average of the sum of the number of months in which 
patients had at least one visit for SUD treatment. 
Reach-2 
The number and proportion of patients who were prescribed 
reSET or reSET-O. 
Reach-3 
The number and proportion of patients who were prescribed 
reSET or reSET-O, downloaded the app and activated their 
prescription. 
Fidelity-2 
The average of the sum of number of weeks (out of 12 
possible weeks) in which patients completed at least one 
module per week. 

 
6 
 
Fidelity-3 
The average of the sum of number of weeks (out of 12 
possible weeks) in which patients completed at least four 
modules per week.  
Adoption 
The proportion of other healthcare providers initiating a patient 
on reSET or reSET-O, overall and by provider type. 
 
While both of our primary outcomes required clinician and patient involvement, reach 
relied more heavily on actions from clinicians and fidelity relied more heavily on actions 
from patients. For example, PCPs and LICSWs were responsible for identifying patients 
who were eligible for the digital therapeutics; PCPs would connect potentially eligible 
patients to the integrated LICSW who would offer the digital therapeutics to patients. If 
the patient agreed to try the app and the designated medical doctor approved the 
prescription, that patient would be included in the numerator of reach-2. Downloading 
the digital therapeutic and activating the prescription (reach-3) was the responsibility of 
patients, although a motivated clinician with time to do so could assist with this step. 
Module completion (which was factored into reach, fidelity, fidelity-2 and fidelity-3 
[module completion is critical because it is the primary way patients receive treatment 
from reSET or reSET-O]) required activation from patients, although clinicians were 
instructed to encourage patients to complete modules. Treatment engagement (a 
subcomponent of fidelity) was the joint responsibility of clinicians and patients. Many 
external factors could influence treatment engagement (e.g., appointment/scheduling 
access, patient ability to pay for SUD-related visits, etc.).  
 
Data analyses were descriptive. Patient characteristics were described among eligible 
patients in Clinics A and B and in the sample overall. Outcomes were calculated among 
the eligible sample, overall and separately within each of the two clinics. We additionally 
calculated outcomes among the secondary sample of patients who were prescribed 
reSET (including some patients who did not meet eligibility criteria [e.g., were 
prescribed reSET but did not have a high scoring screen within a year prior to a primary 
care visit]) within each of the two clinics and in the sample overall.