Technology Enabled Services to Enhance Depression Care
Phase: NA
Conditions: Depression, Anxiety
Sponsor: Northwestern University

Technology Enabled Services to Enhance Depression Care
NCT05406791
Date: 3ͬϮϵͬϮϰ

STU#:00211887
Version Date:  8.11.23 1 of 30
HRP-593 / v03152019
fPROTOCOL TITLE:  Technology Enabled Services to Enhance Depression Care
PRINCIPAL INVESTIGATORS:
David C. Mohr, PhD
Director, Center for Behavioral Intervention Technologies (CBITs)
Northwestern University
Feinberg School of Medicine
Department of Preventive Medicine
312-503-1403
d-mohr@northwestern.edu
Sameer Panjwani, MD
Physician, Department of Family & Preventive Medicine, Rush Medical College
Rush University Medical Center
sameer_panjwani@rush.edu
Colleen Stiles-Shields, PhD
Assistant Professor, Institute for Juvenile Research, Department of Psychiatry
University of Illinois at Chicago
ecss@uic.edu
VERSION DATE:  8.11.23
STUDY SUMMARY:
Investigational Agent(s) 
(Drugs or Devices)
The Vira Mental Health Measurement Platform (Vira) is a 
mobile application (app), secure clinician portal/website 
and administrator portal/dashboard.
IND / IDE / HDE # Not Applicable.
Indicate
Special Population(s)
Children 
 Children who are wards of the state 
 Adults Unable to Consent  
 Cognitively Impaired Adults 
 Neonates of Uncertain Viability 
 Pregnant Women 
 Prisoners (or other detained/paroled individuals) 
 Students/Rush Employees 
Sample Size
Clinical Trial: 130 Patients
Funding Source
National Institutes of Health
Indicate the type of consent 
to be obtained
Written- eConsent 
Verbal/Waiver of Documentation of Informed Consent
Waiver of HIPAA Authorization 
Waiver/Alteration of Consent Process  
Site
 Lead Site (For A Multiple Site Research Study)
 Data Coordinating Center (DCC)
Research Related 
Radiation Exposure
Yes 
 No 
DSMB / DMC / IDMC
Yes 
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 2 of 30
HRP-593 / v03152019
No
OBJECTIVES
Depression and anxiety are common1 and impose a tremendous societal burden in terms of 
cost, morbidity, quality of life, and mortality.2-5 Yet, few people are able to obtain adequate or 
appropriate treatment.6,7 While 100s of trials have demonstrated the efficacy of technology-
enabled services (TESs) in reducing depression and anxiety,8-13 the many attempts to 
implement these validated programs into large value-based care systems have failed.14-18 
Patients do not engage with the technologies, they stop answering reminder calls from staff, and 
TESs do not fit into clinician workflows.
The overarching goal of this research project (RP) is to design, develop, and evaluate a TES 
that can be successfully implemented to support the treatment and management of depression 
and/or anxiety in primary care/family medicine clinics. The design innovation focus of this RP 
will be the use of mobile phone sensor data to identify behaviors, psychological states, and 
environmental conditions in real time19 to support patients’ behavior change.  An intelligent 
patient-facing app will be designed and evaluated, which passively acquires behavioral markers 
and automates prompts to the patient, thereby eliminating much of the effort required by users 
to engage with mental health apps. Our software collaborators at Ksana Health have developed 
algorithms that can identify mobility patterns, physical activity, and other behavioral targets that 
can be sensed. While technological capability to track behaviors is increasing, the field does not 
yet understand how to use these technologies to promote behavior change for the treatment of 
depression.14 We will design interfaces that make it easy for patients to understand and act on 
sensed information for managing depression. By reducing the need for patients to manually 
enter information into an app, the value/burden ratio is significantly increased. These tools will 
be designed to be used in short bursts of 30-60 seconds, which is how most people interact with 
apps,27,28 and which we have found promotes sustained engagement.29 The intervention 
protocol will be based on principles of behavioral activation. 
This patient app, called Vira (which is a product developed and managed by Ksana Health), will 
be evaluated with patients receiving care from Rush University Medical Center’s primary care 
and family medicine clinics, which serve racially, ethnically, and economically diverse 
communities. The TES will include a service protocol that defines how a coach will interact with 
the patient around these sensed behavior targets, a coach dashboard that provides visibility into 
app use and patient status, and tools that enable coach communication with the patient to 
provide support. Thus, we will evaluate a TES that can support depression and/or anxiety care 
in general medicine, with a unique innovation focus on patient-facing features that harness 
emerging personal sensing methods to support behavior change in patients with depression. To 
achieve these goals, we have the following 3 aims:
Aim 1: Design Study (completed in years 1-2 of the project period): Design and develop a TES 
for patients receiving care in a primary care/family medicine setting that harnesses personal 
sensing technologies. A TES for depression has been designed, developed, and refined with 
input from key stakeholders including patients, staff, physicians, and others identified during the 
design process. User-centered design (UCD) principles were used to design the technology, 
service protocol (e.g., intervention and coordination protocols), and an implementation plan. 
Design efforts focused on the integration of personal sensing technologies.  
Aim 2: Randomized Clinical Trial. Pilot the feasibility of an Optimization, Effectiveness, 
Implementation (OEI) Hybrid trial of the TES. The OEI Hybrid trial extends the hybrid (type 1) 
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 3 of 30
HRP-593 / v03152019
effectiveness-implementation trial30 design by allowing optimization of the TES technology, 
service protocol, and implementation plan to the delivery context.31,32 The TES will be compared 
to a control treatment (CT) app that provides basic psychoeducation.
Sub-aim 2.1 Effectiveness. The primary outcomes are patients’ depression severity (PHQ-
9)33, and anxiety severity (GAD-7)34. The secondary outcome is quality of life (PedsQl).  Patient 
characteristics (race, ethnicity, income, age, health status, substance and alcohol use, other 
mental health treatment [including pharmacological]), stigma, discrimination, and mobile phone 
use will be examined as moderators.
Sub-aim 2.2 Implementation. Implementation targets will include reach (number eligible, 
treatment uptake, treatment completion) and cost. 
Sub-aim 2.3 Optimization. We will use quality improvement methods32 to continuously 
improve the technology, service protocol (e.g., intervention and coordination protocols), and 
implementation plan. 
Aim 3. Experimental Therapeutics Aims. We hypothesize that the impact of core UCD targets 
(usability, usefulness, and satisfaction) in the TES will result in increased objective and 
subjective markers of engagement relative to the CT, with improved downstream effects on 
depression. We also hypothesize the moderating effects of patient characteristics on 
engagement and depression will be lower levels in the TES arm, relative to CT, as a result of 
inclusion of users’ voices through the design and optimization processes.  
This project will result in the first TES for depression that can be successfully implemented 
in primary care and family medicine clinics. It will be achieved by creating tools that fit into the 
fabric of patients’ lives by harnessing personal sensing.
BACKGROUND
Primary care is the de facto setting for depression and anxiety treatment for the majority of 
people.7,37-39 However, patient outcomes in primary care are extremely poor due to failure to 
identify symptoms, suboptimal treatment, and treatment access barriers.40-42  Digital mental 
health strategies have been used to address depression and anxiety in primary care.
Technology-enabled services (TESs; which we define as a remotely delivered clinical service 
supported by digital tools and/or applications along with a human coach) offer the potential for 
adaptable psychological services that can be delivered more efficiently at scale that is not 
feasible for in-person services. Today, 81-85% of Americans own a smartphone, among whom 
95% almost always have their phones with them.47,48 Rates are similar across racial and ethnic 
groups.49 RCTs consistently show that TESs, when coupled with low intensity coaching, are 
highly effective for depression.50,51 
However, evidence is emerging that implementation efforts of TESs in real world healthcare 
systems have failed.14,15,16-18 Patients do not use the technologies, and TESs do not fit into the 
workflow of staff and providers. Simply put, current generation TESs do not engage patients or 
providers. Patients become frustrated and stop answering reminder calls.16,17 
We used our Accelerated Create-To-Sustainment (ACTS) research model to design and 
evaluate the technology, service protocols, and identify an implementation plan for a TES that is 
usable and useful, aimed at supporting patients receiving primary care from a provider at Rush 
University Medical Center.  The distinct focus of this RP will be to design intelligence, an 
emerging property of digital technologies, into the TES, harnessing personal sensing.52 Sensing 
is changing behavioral health. For example, automated step counting improves physical activity 
interventions by making measurement effortless.53 There are many more behaviors relevant to 
depression and anxiety that until recently have required self-report. Methods of passive sensing 
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 4 of 30
HRP-593 / v03152019
have emerged that can track behaviors related to depression continuously and effortlessly.19
Personal sensing uses data from sensors in common devices to estimate behaviors, 
psychological states, and environmental conditions.19 Using the increasingly large complement 
of sensors embedded in mobile phones, researchers have been able to accurately estimate 
mobility, physical activity, the location types a person visits, sleep duration and circadian 
rhythm, online social activity, and medication adherence.19-26 While our technological capability 
to track behaviors is increasing, we have yet to understand how to use this capacity to intervene 
with patients.  Roughly half of Americans have low graphic literacy, creating challenges in the 
design of patient-centered communication tools.54 In addition, the ability to continuously track 
behaviors opens new intervention opportunities, requiring service design.14  Thus, a primary and 
unique focus of this RP within the Chicago ALACRITY Center will be to design a TES that 
harnesses emerging personal sensing methods to support behavior change in patients with 
depression. 
RANDOMIZED CONTROLLED TRIAL (RCT) STUDY INTERVENTION(S) / INVESTIGATIONAL 
AGENT(S)
Participants enrolled in the Randomized Controlled Trial (RCT) will be assigned to 
receive one of two study interventions: (1) Technology Enabled Service (TES), or (2) Control 
Treatment (CT). The design of the TES and CT interventions were determined through the 
design study and negotiations with our software development partners (Ksana Health and 
Audacious Software). 
Technology Enabled Service (TES)
The TES consists of digital technologies that connect patients to a coach remotely, providing 
tools for monitoring patient symptoms, communication, and delivery of tools and 
psychoeducation that support self-management strategies. Patients randomized to the 
technology enabled service (TES) condition will use Ksana Health’s “Vira” Mental Health 
Measurement Platform with support from a study coach. Vira by Ksana Health delivers passive 
and continuous remote patient monitoring + digital therapeutics for behavioral health. Vira uses 
smartphones as a passive sensing device, requiring no special equipment or additional costs for 
users who already use a smartphone. The app uses passive sensing technology to identify 
activities and behaviors that appear predictive of a users’ enjoyment and accomplishment, 
displays those activities to the user, and prompts participants to reflect on how to increase their 
sense of enjoyment and their sense of accomplishment.  Additionally, the app will contain 
psychoeducational resources to help participants self-manage symptoms of depression and 
anxiety. Coaches will have the benefit of receiving objective patient data and insights, allowing 
them to monitor app usage and provide remote support.  Northwestern University study team 
member(s) will be trained to deliver the digital mental health coaching intervention to support 
study participants in using Vira.  The coach will help participants with app engagement, 
knowledge, fit, and implementation of self-management strategies.  The coach will also provide 
behavioral activation-informed content to participants via messaging. Content will be in the form 
of previously established peer-led delivery models. To supplement telephone communication 
between study coaches and participants, we have contracted with Audacious Software to 
develop a secure text messaging platform to facilitate delivery of the coaching service. All text 
messaging data will be stored on Northwestern University Feinberg School of Medicine’s on-
premises servers. The TES will be delivered over a period of 8 weeks.
  
Control Treatment (CT)
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 5 of 30
HRP-593 / v03152019
Participants randomized to the CT condition will use an app designed by researchers at the 
University of Virginia to deliver psychoeducational content to help people self-manage 
symptoms of stress, depression and anxiety.  The mood education app will be cloned by 
Audacious Software for use in this study and data will be hosted on Northwestern University 
Feinberg School of Medicine’s AWS environment. 
RCT PROCEDURES INVOLVED
We will utilize an RCT based on our OEI hybrid trial design. A total of 130 Rush Patients will be 
randomized to either TES or Control Treatment (CT). Depressive and anxiety symptoms and 
other outcomes will be measured at baseline, 4, 8, and 12 weeks.  Beyond 12 weeks, 
participants will be allowed to continue use of the apps for the study duration, though the TES 
coaching service will be discontinued at the end of 8 weeks. App use will be tracked for the 
entire study duration. Should a patient begin treatment but stop engaging, a single question 
survey will be administered to gauge the reason why and potentially optimize treatment as a 
result of the feedback.
Patient populations will be all Rush patients who are 13+ years old who are experiencing at 
least moderate symptoms of anxiety and/or depression and are open to using a smartphone 
app to get mental health support.  Out of the total number of participants, 10%, or approximately 
13 patients are expected to be adolescents.
All participant management will be performed by personnel at Northwestern University, and 
Rush. 
Assessment Strategy for Sub-aim 2.1 Effectiveness
The primary outcomes are depressive symptom severity, measured using the PHQ-933, and 
anxiety symptom severity, measured by the GAD-7. Administration will occur as part of the 
research protocol via REDCap online, or if not feasible via phone.101 Secondary outcomes 
include quality of life.  Primary and secondary outcome data will be gathered from patients as 
outlined in Table 1  below. 
Table. 1: Primary and Secondary Outcome Measures
Outcome/Construct
Measure
Method
Timing
Type
Depressive 
symptoms 
PHQ-9 
Self-report 
Survey
0w, 4w, 8w, 
12w
Primary
Anxiety symptoms
GAD-7
Self-report 
Survey
0w, 4w, 8w, 
12w
Primary
Quality of Life
PedsQL
Self-report 
Survey
0w, 4w, 8w, 
12w
Secondary
Additional data will be collected to evaluate intervention use, usability, satisfaction, and 
engagement.  Measures of discrimination and stigma will be administered. Patient self-report 
measures have been uploaded to the eirb+ system for review. See data collection packets 
containing the measures to be administered via REDCap at screening, baseline, week 4, week 
8, and week 12. 
Assessment Strategy for Sub-aim 2.2 Implementation
Reach will be measured as follows: 1) Number of eligible patients will be the number of patients 
with PHQ-9≥10 at a clinic visit during the study recruitment period; Treatment uptake will be the 
number of patients consented and randomized; 3) Treatment completion will be the number of 
patients completing at least 6 weeks of the 8-week treatment.
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 6 of 30
HRP-593 / v03152019
Optional Usability/User-Feedback Interviews (4w, 6w, 8w)
Participants will be informed during the informed consent process that they may be invited to 
complete optional interviews.  All interviews will be conducted by telephone.  At the end of the 
week 4 self-report survey, participants will be asked if they would like to provide additional 
feedback about their experience using the Vira app and working with the study coach via paid 
interviews.  Participants who express interest in providing feedback will be contacted by study 
staff to schedule the optional interview session(s).  More information about the nature and 
timing of the optional interviews is provided below.
Vira app usability interviews (4w, 6w, 8w)
As we begin enrolling the clinical trial, we will invite the first participants randomized to the TES 
condition to provide feedback on the usefulness and usability of the Vira application.  These 
optional interviews will be administered to approximately 10 participants.  
The app usability interview guide has been uploaded to the eirb+ portal for review.
Coaching user feedback interviews (4w)
All participants randomized to the TES will be invited to share additional user feedback about 
their experience working with the study coach at the week 4 (mid-treatment) timepoint.  
The coaching user feedback interview guide has been uploaded to the eirb+ portal for review.
Failure to Initiate Treatment
If a participant fails to initiate treatment or fails to continue with treatment but is still engaged 
with the study, a single question survey will be administered to gauge the reason why they 
decided to not engage with the treatment.
Optional Focus Group (Post-Trial)
Approximately 12 participants will be invited to participate in a one-time, virtual focus group after 
completion of the trial to share their feedback on their participation (reasons for enrolling, for 
remaining/not remaining engaged, etc.) as well as their thoughts on how to best disseminate 
study findings to study participants.
Participants will be invited via email to participate in this add-on study activity. Participants will 
be instructed to express interest by replying to the email. Participants will also be informed that 
space is limited. Once the participant expresses interest, the study team will assess number of 
spaces available and other demographic information (to ensure a range of representation) and 
reply with either:

A link to the addendum document which outlines the focus group details (objectives, 
benefits/risks, and confidentiality), followed by a brief form to provide their 
availability. The addendum and the availability form are hosted on Northwestern’s 
instance of REDCap, or

A message to thank participant for their interest and to let them know spaces have 
been filled.
Adolescents 13-17 years old will require parental consent to participate in the focus group. 
Participants who sign the addendum document will be scheduled for a focus group according to 
their availability.
Due to the low-risk nature of the focus group and the fact that participants are being invited after 
having completed all other study activities, we will use an addendum document (versus a 
revised study consent form). An addendum will decrease the participant burden and potential 
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 7 of 30
HRP-593 / v03152019
confusion of re-reading the entire consent form when they have already completed all other 
study activities.
These virtual focus groups will be audio-recorded and transcribed. Transcripts will not contain 
identifiers. This is detailed in the addendum document.
Risks are described in the “Risks” and “Protection Against Risks” sections of this document. 
Risks are explained in the addendum document,
The addendum document will not be translated into Spanish as the study sample contains no 
adolescents with Spanish-speaking parents.
DATA AND SPECIMEN BANKING
Deidentified data will be stored indefinitely for secondary analyses. 
STUDY TIMELINES
RCT Enrollment Timeline
This trial will utilize a rolling recruitment strategy with patient-level randomization. Recruitment 
will begin in the third year of the grant-funded project and continue into year four.
RCT Timeline of Participant Activities
Active participation will last 12 weeks for each study participant. Participants will use the study 
apps and receive coaching (group 1 only) for the first 8 weeks.  We will unobtrusively track all 
communication with the study coach. Assessment will continue through week 12. Study 
assessment milestones are laid out in tables 3 and 4 below.
Table 2. Online surveys to be administered to all participants
Method
Week 0
Week 4
Mid-
Treatment
Week 8
End of 
Treatment
Week 12
Post-
Treatment
Self-report Survey
x
x
x
x
App Usage
Continuous, unobtrusive
 
Table 3. Optional interview data to be collected from patients randomized to TES
Method
Week 4
Mid-
Treatment
Week 6
Week 8
End of 
Treatment
Vira app usability interview
X
X
x
Coaching user-feedback interview 
X
Table 4. Optional focus group be completed with patients randomized to control and TES
Method
Post-Trial Completion
Focus Group (semi-structured interview guide)
X
The anticipated completion date for primary analyses is March 31st, 2024.
RCT INCLUSION AND EXCLUSION CRITERIA:
Trial participation will be offered to patients at Rush University Medical Center aged 13 or older. 
Children under the age of 13 will be excluded given likely differences in development and 
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 8 of 30
HRP-593 / v03152019
smartphone ownership that would make such young children an inappropriate population for 
evaluating the technology enabled service and implementation plan. 
Inclusion Criteria: US Citizen/Resident (for payment purposes) living in the US during the 12-
week study period (for tech functionality, data security, and safety management purposes), 13 
years or older (if age 13-17, parent/caregiver must provide consent, a consent process is in 
place for English-speaking minors with Spanish-speaking parents), English speaking, eligible to 
receive care through Rush University Medical Center or its clinical affiliates, owns an Android or 
iPhone smartphone with an up-to-date operating system, has used a smartphone in the last 7 
days, at least moderate symptoms of depression (PHQ-9 greater than or equal to 10) and/or 
anxiety (GAD-7 greater than or equal to 8) 
Exclusion Criteria: Severe suicidality (as defined by presence of a plan + intent to act on that 
plan)
VULNERABLE POPULATIONS
Adolescents (ages 13-17) meeting the previously described inclusion criteria will be eligible to 
participate. The proposed study poses minimal risks for adolescent participants. All potential 
risks associated with participation in this study will be disclosed in the combined assent and 
parental consent document. Potential risks are identical to those of potential adult participants 
(ages 18+) and fall into the four categories addressed later in this protocol: (a) risks associated 
with the intervention; (b) risks associated with research assessments, consisting of questions 
about depression, anxiety, and personal functioning, and other mental and emotional problems; 
(c) risks associated with potential loss of confidentiality; and (d) risks of worsening mental or 
emotional state. 
RCT RECRUITMENT METHODS
Patients will be recruited from Rush University Medical Center clinic-based sites, social media 
and other platforms, and through subject recruitment registries.
Clinic-based recruitment will occur at Rush University Medical Center primary care, family 
medicine, internal medicine, collaborative care, psychiatry clinics, community health clinics (e.g., 
school-based health centers), pediatrics, the Adolescent Family Center, and may be extended 
to other clinics as needed.
All patients judged as potentially meeting criteria for treatment by the care team and/or a 
member of the research team will be offered recruitment materials. Specifically, they will offer 
participation in the Trial to all patients aged 13 or older whom they deem appropriate via a flyer 
or an email/MyChart message with a description of the study and a link to complete the online 
eligibility screener. Information commonly used by providers to determine need for mental 
health care are routine screening with the PHQ-9, GAD-7, physician referral, and patient 
complaints and requests.
Recruitment will also take place using Rush’s MyChart Recruitment feature on Epic. The 
research team will invite patients who have had a recent visit or have a visit scheduled to their 
Rush primary care doctor. After receiving IRB-approved recruitment materials, they will have the 
option to opt in or out of additional follow-up, including a link to complete the online eligibility 
screener. Patients who do not respond to the initial invitation may receive one additional 
invitation. Patients who express initial interest but do not read the team’s follow-up message will 
receive a reminder. The team will know who has not read the MyChart message via the “un-
read notification” feature.
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 9 of 30
HRP-593 / v03152019
Parents of minors who themselves receive care at Rush may also receive a flyer or an email 
inviting their child to take part in the study. The email scripts have been uploaded into the eirb+ 
system. 
Patients may also receive a teaser advertisement the size of a business card with a QR code 
directing them to an online version of the full recruitment flyer. Patients will confirm their interest 
by initiating the online screening and consent procedure. 
Specifically:

Patients will be offered a flyer or sent an email/MyChart message with a description of 
the study and a link to complete the online eligibility screener.  

A flyer will be included with the PHQ-9 and GAD-7 clinic questionnaires. Patients will 
confirm their interest by initiating the online screening and consent procedure.

The PHQ-9 and GAD-7 screeners will contain an add-on form inviting participants to 
provide their contact information if they’re interested in being contacted about free 
resources, programs, or studies that may help manage their mental health.  Rush 
personnel would follow up with interested patients to review main points from IRB 
approved recruitment flyer and direct them to the screener link. Rush personnel would 
not share contact information for patients who may not be interested.

A member of the research team may approach patients in the waiting room to offer a 
flyer and those who express interest will be invited to visit/initiate the online screening 
either on the patient’s personal device or a Rush-issued tablet.
Social Media and Other Platforms. Rush and Northwestern teams will post recruitment 
messages to their social media pages that direct participants to a link that includes an online 
version of the full recruitment flyer and to the online screening. These have been uploaded to 
the eirb+ system. Rush will also post the study to their Clinical Trials website which will include a 
description of the study and the study team’s contact information.
Community-Based Recruitment
The Rush research team will disseminate study flyers at community events hosted by Rush and 
Rush’s community partners (back-to-school fairs, resources fairs, wellness events, etc.). Event 
attendees will be informed that inclusion criteria include being a Rush patient. If attendees 
report that they are not currently a Rush patient, they will be encouraged to take the flyer in 
case they seek care at Rush at any point. Those who report being a Rush patient will be 
encouraged to scan the QR code to learn more about the study and complete a screener 
survey. Patients will confirm their interest by initiating the online screening and consent 
procedure.
The Rush Education and Career Hub (REACH) supports education through academic 
enrichment, mentoring, and internships by connecting with underrepresented students, their 
parents, educators, and staff to: spark and catalyze STEM learning in and outside the 
classroom, design solutions and address challenges in their communities and beyond, improve 
overall community health. Interested REACH-affiliated adolescents may receive a flyer or an 
email with a description of the study and a link to complete the online eligibility screener. We will 
use the currently approved study flyer and email invitation, only slightly tailored for this younger 
demographic. Patients will confirm their interest by initiating the online screening and consent 
procedure. The REACH team may also include recruitment materials in their social media or 
newsletter platforms.
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 10 of 30
HRP-593 / v03152019
Clinical trial recruitment advertising materials have been uploaded to the eirb+ system for 
review. 
Subjects who do not provide the necessary information to determine eligibility via the online 
screener will be contacted by telephone to obtain the information needed to make an eligibility 
determination. 
COMPENSATION FOR PARTICIPATION IN RESEARCH ACTIVITIES
We will reimburse all RCT participants for completing research assessments.  
All subjects can earn up to $100 for completing online study assessments. The reimbursement 
schedule for each completed assessment is outlined in table 6 below. 
Table 5. Compensation for Completion of Online Surveys
Method
Week 0
Week 4
Week 8
Week 12
Self-report Survey 
$25
$25
$25
$25
Subjects randomized to the TES arm can earn additional compensation if they are interested in 
completing additional optional user feedback interviews.  Compensation for each additional 
interview completed is outlined in table 6 below.
Table 6. Compensation for Completion of Optional Interviews
Method
Week 4
Week 6
Week 8
App Usability 
Interview
$25
$25
$25
Coaching Feedback 
Interview
$25
Table 7. Compensation for Completion of Optional Focus Group
Method
Post-Trial
Focus Group (semi-structured interview guide)
$25
Depending on Northwestern University payment guidelines and availability at the time of 
participation, payments to participants will be issued in one of the following ways:

Gift card: The following gift card payment methods are available to participants earning 
no more than $225 per calendar year: 

Amazon.com gift card: Participants will be sent a code via email that will allow them 
to redeem an Amazon.com gift card. Gift Cards may only be redeemed toward the 
purchase of eligible goods and services provided by Amazon.com. Participants must 
create an account with Amazon to use the card. No fees apply to Amazon Gift Cards 
and the balance will not expire. 

PNC Stored Value (Visa) Card: If participants do not have an Amazon account or do 
not wish to be compensated through Amazon, they may request a Stored Value 
(Visa) Card instead. The cards may be virtual or physical as per participant 
preference.

Check: Participants earning more than $225 per calendar year must be paid by check. 
Checks require a signed W9 form. The check may be mailed to the participant’s 
address.
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 11 of 30
HRP-593 / v03152019

Through Northwestern University (NU) Payroll if the participant is an NU employee
Participants will be informed that it will take approximately 2-4 weeks for payment to be 
processed.  
WITHDRAWAL OF PARTICIPANTS
Patient participants can discontinue the TES and/or study at any time at their own request, or 
they may be withdrawn at the discretion of the investigator for safety, behavioral or 
administrative reasons. 
The reason(s) for discontinuation will be documented and may include:
Patient voluntarily withdraws from treatment (follow-up permitted); 
Patient withdraws consent (termination of treatment and follow-up);
Patient is unable to comply with protocol requirements; (termination of treatment and follow-up);
Patient demonstrates disease progression (termination of treatment and follow-up unless 
continued treatment with study intervention is deemed appropriate at the discretion of the 
investigator);
RISKS TO PARTICIPANTS
Risks
During clinical trial described above, patient participants will be assessed using self-report 
measures over 3 months. Participants randomized to TES may be offered user feedback 
interviews to provide feedback on their experience using the Vira app and working with the 
study coach.  The proposed study poses minimal risks. All potential risks associated with 
participation in this study will be disclosed in consent documents. Any potential risks that might 
exist fall into four categories: (a) risks associated with the intervention; (b) risks associated with 
research assessments, consisting of questions about depression, anxiety, and personal 
functioning, and other mental and emotional problems; (c) risks associated with potential loss of 
confidentiality; and (d) risks of worsening mental or emotional state.  We address each in turn 
below.
Risks of the intervention: Digital mental health intervention programs generally have not been 
shown to cause any harm. The primary risk of the intervention is participants being distracted by 
their mobile phones while engaged in activities that demand their complete attention. 
Participants occasionally try to use mobile apps while driving motor vehicles, and therefore will 
be instructed never to use the mobile phone while driving. Participants will be made aware of 
the physical, financial, and legal risks associated with using the phone while driving. If the 
research team becomes aware of a participant engaging in this behavior, the PI and mentors 
will consult to determine the most appropriate way to eliminate this risk. 
Risks associated with research assessments: Research assessments include questions about 
depression and other mental and emotional problems. Participants will give voluntary responses 
to interview questions; they are told that they can decline to answer any questions that they 
choose. The instruments and methodologies are well tested in adolescents (13-17 years) and 
adults (18+ years old) and are not known to cause problems or distress on the part of the 
participants. All research interview-based assessments are audio-recorded, or video recorded, 
for the purpose of review to ensure quality assurance ratings of assessment performance, 
including ensuring that patients are comfortable with the interview procedures. Audiotapes and 
videos will be maintained on a secure server with no identifying information in the labels for the 
duration of the funded study, unless other arrangements are made. On occasion patients may 
request that audio files or videos be deleted before the end of the study, in which case we will 
comply.
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 12 of 30
HRP-593 / v03152019
Risks associated with potential loss of confidentiality. There is a slight risk of loss of 
confidentiality.  While transmissions from mobile phones or other devices are protected, and 
communication occurs within a secure messaging platform, there is some possibility that others 
may see the participant’s smartphone or device.  There is also a small possibility that databases 
may be hacked, even though they are behind secure firewalls.  Measures to protect security in 
these instances are described below. Confidentiality may be broken by research staff to ensure 
the patient’s safety if there is an imminent threat to self or others. There is also the remote 
possibility that research records will be subpoenaed by a court of law. Given the nature of focus 
groups, there is risk of loss of confidentiality. All of these potential losses of confidentiality will be 
disclosed in the consent documents.
Risk of suicide: Some participants may show suicidality or problems during the study period, 
which is the most serious risk. This risk is inherent in the population and would occur whether or 
not they were enrolled in the study. It is not believed that the risk of these depressive, anxious, 
suicidal, or other adverse outcomes are increased as a function of being enrolled in this study.
All potential risks associated with participation in this study will be disclosed in assent, parental 
consent, and adult consent documents. 
Protections Against Risk
All patient participants will continue to receive all care through their providers within the Rush 
system.  Thus, the full range of available treatment options are available to participants, 
including antidepressant medications, psychotherapy, electroconvulsive therapy, or inpatient 
treatment. The benefits of these alternatives are that they are evidence-based, more intensive 
and specialized. 
Protection for risks associated with potential loss of confidentiality. Data for all participants will 
be kept strictly confidential, except as mandated by law.  All electronic data will be stored on 
secure servers behind firewalls meeting all security requirements of the medical school. Any 
paper documentation (which we do not anticipate) is kept in locked file cabinets or a locked file 
room. Participants will be assigned a numerical code for identification in the files. Names and 
other identifiers will be kept in separate password protected files. Audio and video data will be 
stored on secure servers and will only be available for coding by study staff. All intervention 
technologies will be developed using up-to-date security measures. 
To reduce the risk of loss of confidentiality through the participant’s device, we will instruct 
patient participants on how to add a PIN to their phone to prevent unwanted access. To protect 
against unauthorized access of patient participant intervention data on a mobile phone, 
intervention data will be encrypted on the phone itself and configured to self-destruct at the 
command of study staff (for example in the case a phone is reported missing). We will clearly 
inform the patient participants of the risk of data insecurity. 
To reduce the risk of loss of confidentiality associated with the focus group, participants will be 
asked to log in with first names only; and the research staff will rename anyone who uses both 
first and last names. Participants will also be asked to refrain from sharing the content of the 
discussion with anyone outside of the group.
Protection for risk of suicidality: The development of suicidal ideation during the study remains 
the most serious risk. Patient care will always take precedence over study protocol.
 
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 13 of 30
HRP-593 / v03152019
There are a number of pathways through which suicidal risk can be detected.  During the trial 
phases, suicidality may be detected through automated PHQ-9 self-report assessments, 
through direct communications with the study coach via phone or messaging (email, text, etc.) 
or through direct communication with research staff during user feedback interviews. In each of 
these instances, the goal will be to rapidly acquire additional information that can trigger safety 
procedures if necessary.
 
Suicidality will be monitored through weekly assessments.  Participants who rate Question 9 of 
the Patient Health Questionnaire-9 (i.e., “Over the last 2 weeks, how often have you been 
bothered by thoughts that you would be better off dead or of hurting yourself in some way”) a 1 
or higher will receive a second item from the Beck Depression Inventory (2nd ed.) that is more 
specific in identifying suicidal intent, asking participants who indicate that they would like to kill 
themselves, or would kill themselves if they had the chance will be considered to have “elevated 
suicidality.”  For participants with “elevated suicidality,” a crisis message is displayed to the 
participant (see message verbiage below) and an alert is sent to the research team and a 
trained member of the research team will respond within two business days by calling the 
patient to enact the assessment procedures based on the Columbia Suicide Severity Rating 
Scale and management procedures that follow clinical guidelines described below.
Crisis Message Displayed to Participants in REDCap:
This research study is NOT an emergency or crisis service.
If you are experiencing a life-threatening emergency such as serious thoughts or plans to end 
your own life or the life of anyone else, please call or text the numbers 9-8-8 to reach the 
Suicide and Crisis Lifeline.
If you're looking for crisis supports who specialize in serving members of the LGBTQIA+ 
youth communities, please call the Trevor Project at 1-866-488-7386 or text "START" to 678-
678
For non-crisis support, warmlines are also available, find one here: 
http://www.mhanational.org/warmlines
 
When suicidality is expressed by the participant directly to the study coach in a phone call or 
text, the coach similarly follows up to conduct a risk assessment based on the Columbia Suicide 
Severity Rating Scale  and engage the participant in risk management procedures based on 
their risk status.
 
Finally, participants may express suicidality during user feedback interviews with research study 
staff. While planned questions do not probe suicidality,  participants may, in the course of the 
interviews, express suicidality, in which case they will be considered have “elevated suicidality” 
and trained research staff will conduct risk assessment and management procedures
 
Description of assessment procedures using the Columbia Suicide Severity Rating Scale
All participants with “elevated suicidality” will receive additional follow-up assessment using the 
Columbia Suicide Severity Rating Scale1 to obtain further information on level of risk by the 
research staff member, who will be trained on its administration.  Participants with elevated 
suicidality that is not emergent will be provided with a personalized coping plan that includes 
crisis resources (e.g. the 9-8-8 Suicide Hotline, Crisis Text Line), referred back to their 
physician, and their physician will be informed.  In the case of a psychiatric emergency during 
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 14 of 30
HRP-593 / v03152019
the trial phases (including imminent suicide risk), all necessary efforts will be made to ensure 
the safety of the participant, including a warm handoff to the National Suicide Prevention 
Lifeline, possible outreach to emergency services and informing the participant’s physician of 
risk status and all risk management actions taken. All risk management activities will be 
supervised by the Center’s clinical monitor, a PhD level clinical psychologist. A masters level 
social worker may supervise the suicidality assessments.
 
In the event that a suicidality assessment is completed with an adolescent participant (age 13-
17), a parent or guardian will be notified that a risk assessment was completed with their child. 
Limits to confidentiality have been specified in the adolescent consent/parental consent 
document.
 
We note that these are risks inherent in the population and would occur whether or not they 
were enrolled in the study. We do not believe that the risk of these depressive, suicidal, or other 
adverse outcomes are increased as a function of being enrolled in this study or receiving TES 
during the trial phases. All potential risks associated with participation in this study will be 
disclosed in consent documents.
Protection for Spanish-speaking parents of minors
The study team will use Spanish-language materials that have been translated and certified by 
the Rush Interpreter Services Office. A qualified bilingual study team member who has been 
certified by the Rush Interpreter Services Office will be conducting outreach to Spanish-
speaking parents.
POTENTIAL BENEFITS TO PARTICIPANTS
There is no direct or immediate benefit to participants from whom data will be collected, 
although we anticipate some participants may receive support for their depression through the 
TES intervention platform. The potential to future patients is that the study may provide 
fundamentally new and more effective low-intensity treatment approaches that would be more 
widely available. 
DATA MANAGEMENT AND CONFIDENTIALITY
Research data will be electronically recorded and stored in the following secure locations:

Online survey data: NU’s instance of REDCap

App use data:
o
Vira App: Ksana Health’s secure servers  
o
Mood Education App: Feinberg School of Medicine’s AWS Environment

Text messaging/other communications with participants: Feinberg School of Medicine 
Servers

Interview data: Feinberg School of Medicine Servers
Research Electronic Data Capture (REDCap) tools hosted at Northwestern University.  REDCap 
is a secure, Web-based application designed to support data capture for research studies, 
providing 1) an intuitive interface for validated data entry; 2) audit trails for tracking data 
manipulation and export procedures; 3) automated export procedures for seamless data 
downloads to common statistical packages; and 4) procedures for importing data from external 
sources. Data communication between data entry computers and database servers will be 
encrypted via a VPN client. Access to the VPN connection will require user authentication via 
username and password. Firewall software will restrict communication to the database server 
that is absolutely critical for conducting research. The databases and all data entry procedures 
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 15 of 30
HRP-593 / v03152019
will be designed to maintain the confidentiality of patients enrolled into the study. Numbers will 
be used to identify all written study materials, and no personal identifiers will be on such 
materials. Hard copy files will be stored in a secure location accessible only to the study team. 
Audio and video data will be stored on secure servers and will only be available for coding by 
study staff. All intervention technologies will be developed by our software development 
partners, Ksana Health and Audacious Software, using up-to-date security measures.  The 
study team has worked with Northwestern University to establish contracts with these vendors 
to develop and maintain the digital tools that will be used to deliver the TES and CT 
interventions and manage the resulting data. We’ve laid out a description of the data handling 
practices for each of these vendors in table 2 below:
Table 6.  Software Vendor Data Handling
Vendor
Data Collection 
Platform
Data Elements Handled by 
Vendor
Data Storage 
Location 
During Project
Data Retention 
at End of Project
Ksana 
Health
Vira Mental 
Health 
Measurement 
Platform
Email address to generate user 
login, app use event data (e.g., 
number of logins, time spent 
reviewing written information in 
the app), survey responses, 
smartphone sensor data 
(deidentified location data, 
physical activity, motion), and 
in-app communication with 
study coach 
Ksana's 
Secure 
Servers
Ksana will delete 
all study data at 
the end of trial 
after data is 
transferred to the 
study team. 
Mood 
Education 
Smartphone 
Application
Phone number to generate user 
login, and app use event data 
(e.g., number of logins, time 
spent reviewing written 
information in the app)
NU's AWS 
Environment/ 
FSMIT on-
premises 
servers
Audacious 
Software
Simple SMS 
Platform
Phone number used to facilitate 
sending/receiving text 
messages between study 
coach and participant, log of 
messages exchanged between 
study coach and participant
NU/FSMIT on-
premises 
servers
Data will not 
leave the NU 
environment. 
Data will be 
retained 
indefinitely at 
NU.
To reduce the risk of loss of confidentiality through the participant’s device, we will instruct 
patient participants on how to add a PIN to their phone to prevent unwanted access. We will 
clearly inform the patient participants of the risk of data insecurity.
Optional usability and user feedback interviews and focus group interviews:
Audio data will be stored on secure servers and will only be available for coding by study staff. 
Interview audio files may be transcribed by one or more of the following NU transcription 
vendors:
GMR Transcription
https://www.gmrtranscription.com/
Wordsworth
https://wordsworthcoop.com 
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 16 of 30
HRP-593 / v03152019
Files may be shared via secure access to FSM servers or through vendors’ proprietary web-
based applications for secure file management. 
Data Analysis
Effectiveness (Sub-Aim 2.1). 
All effectiveness analyses for the primary (PHQ-9) and secondary clinical (GAD-7, suicidal 
ideation) outcomes will use intention to treat methods. A comparison of population 
characteristics (race and ethnicity, age, gender, health status, baseline depression and use of 
other mental health treatments) must be made between patients receiving TES and the control 
treatment. Should the populations be different, propensity scores will be estimated for the 
likelihood of receiving TES, and models will adjust for those scores (trimming the data as 
necessary). After the population comparison and appropriate adjustment, we will compare 
overall effectiveness, using generalized linear mixed models (GLMM) to determine if there are 
differences in effectiveness between individuals receiving TES and CT over time.106  Separate 
models will be run for PHQ-9 and GAD-7, using participants who meet entry criteria for each 
(PHQ-9≥10 or GAD-7≥8, respectively). Missing data will be managed through multiple 
imputation. Additional covariates will also be included to identify any substantive changes in the 
service, technologies, and/or implementation plan as a result of optimization during the trial.
Moderation of TES on effectiveness. Moderation of patient characteristics (race, ethnicity, 
income, age, sex, health status, substance/alcohol use, or use of other mental health 
treatments, including pharmacologic) will be analyzed in individual GLM models by estimating a 
treatment by moderator interaction in predicting end of treatment depression scores adjusting 
for baseline levels of depression.    
Power. This is a pilot study aimed at evaluating feasibility, and as such, sample size was 
not determined by power. Nevertheless, we provide an estimate of an effect size that we may 
be able to detect with a conservative estimate of roughly 2 enrolled participants per clinic month, 
or 50 participants in CT and 80 participants in TES. As power calculations for delayed roll-out 
trials would involve speculation of both between period and within period autocorrelations, which 
would be speculative at this point, we estimated an effect size for a cluster randomized trial, with 
estimated ICC of 0.01 within clinic. Based on this sample size, we will have 80% power to detect 
an effect size of 0.62 in an intent-to-treat analysis. This equates to a difference in PHQ-9 scores 
between CT and TES after treatment of 3.1 assuming a SD of 5.  
Implementation (Sub-Aim 2.2). 
We will examine the implementation outcomes: reach and cost.  Reach is defined as the 
implementation cascade including 1) the number of eligible patients (PHQ-9≥10 at clinic visit 
during recruitment period); 2) number of patients consented and randomized; and 3) number of 
patients completed (6 or more weeks in treatment).   As is common in implementation research, 
we will likely be underpowered for standard statistics;102,107 we will therefore examine 
implementation metrics (adoption , and reach) using control charts and run charts over time, as 
well as charts centered on the time when clinics are transitioned from CT to TES.108,109 These 
charts plot observed rates or means together with ±1 & 2 SDs and then plot the measures 
monthly to determine if there is significant trend or change in implementation metrics over time.  
To evaluate time to stable implementation in the TES arm, we will use run charts examining 
runs, shifts, and trends to look for nonrandom variation that indicates actual change or stability 
of the implementation metrics over time.110  Cost for a depression free day (DFD), with DFD 
estimated using linear interpolation of PHQ-9 scores using a criterion of PHQ-9<10 and 
compared across conditions using both a t-test (on cost) and incremental cost effectiveness 
ratio (ICER) analysis.111-113 This metric allows comparison with other studies.114,115
D6.3 Aim 3. Experimental Therapeutics. We will evaluate if the core UCD targets (usability, 
usefulness, and satisfaction) in the TES will result in increased objective and subjective markers 
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 17 of 30
HRP-593 / v03152019
of engagement relative to the CT, with by testing the mediational effects of subjective and 
objective engagement with the apps on the primary outcomes, depression (PHQ-9) and anxiety 
GAD-7).  Subjective engagement will be measured using the Twente Engagement with eHealth 
Technologies Scale (TWEETS), while objective engagement will be measure by app use 
(number of app launches and time from first to last use).  We hypothesize that the difference 
between the TES and CT will be mediated by subjective and objective engagement. We also 
hypothesize the moderating effects of patient characteristics on subjective and objective 
engagement, depression, and anxiety will be lower in the TES arm, relative to CT, as a result of 
inclusion of users’ voices through the design processes used in the creation of the TES app.  
Mediation will be tested using methods by Iacobucci.121 Using analogous adaptations of 
methods to Preacher’s methods,122,123 we will be able to test for moderation and mediation 
together. Between patient characteristics and actionable targets (subjective and objective use) 
we could test many moderated mediation (conditional indirect effects) which may indicate that a 
particular subset of participants (defined by subjective or objective use, and patient 
characteristics) may be more likely to benefit from TES. Multiple models will need to be fit, 
increasing the possibility of an inflated type I error. To prevent this, we will implement a false 
discover rate (FDR) control using methods by Benjamini-Hochberg124 although as these are pilot 
studies, we will test these models at a FDR of 10%.  Depending on the distribution of the 
variables under consideration, we will fit linear or logistic regression models to estimate 
regression parameters a (TES and the relationship to potential mediators) and b (mediator 
relationship to the outcome). Iacobucci shows that a normally distributed test statistic can be 
constructed from standardized regression parameters Za=a/SEa such that Z(mediation)=Za*Zb/ 
(sqrt(Za2+Zb2 + 1)) can be used to test for mediation.  Further, we can examine if there is 
further moderation of these models such that any patient covariate could moderate the 
relationship between TES use and depression outcomes. Power analyses for the individual RPs 
can be found in the RPs.
Power. While sample size and power calculations for mediational analyses are subject to 
speculation, Fritz and Mackinnon116 show that our effective sample size of 112 (130 divided by 
the design effect determined by the number of patients per clinic and ICC within clinic) achieves 
80% power to detect between a medium effect size, and small-medium effect size for the a and 
b paths (medium ES = 0.39, small-medium ES=0.26) using the joint tests we propose. 
SMS Data and Focus Group Analysis Plan
Data from SMS records and focus groups will be analyzed qualitatively via thematic 
analysis as described by Braun and Clarke or via semi-open iterative coding. Semi-open 
iterative coding would allow us to begin with a set of codes relevant for the trial and stay open to 
new codes (as in thematic analysis).
Organizational Level Data
Rush University Medical Center personnel will provide aggregated data on TES Reach/Adoption 
on a routine basis. The Rush team will ensure a de-identified DUA is in place and will work with 
the Research Core to have the data extracted and de-identified prior to the transfer. 
Additionally, the request will be reviewed and approved by the Rush Data Governance 
Committee.  These de identified data exports may include the following information which will 
enable the study team to understand what proportion of patients who screened positive for 
depression and/or anxiety were offered participation in the trial and engaged in TES. For 
patients in each participating clinic:

Number of patients screening positive on PHQ-9, GAD

Number of patients being referred to study
PROVISIONS TO MONITOR THE DATA TO ENSURE THE SAFETY OF PARTICIPANTS
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 18 of 30
HRP-593 / v03152019
CLINICAL TRIAL
Oversight of the trial is provided by the Principal Investigators, Drs. Mohr, Meyerhoff, and Stiles-
Shields. The NU research coordinator, Ms. Barnas will ensure that informed consent is obtained 
prior to engaging in research activities, and that the study is conducted according to the IRB-
approved research plan.  
Study data that do not unblind investigators are accessible at all times for the PIs, the 
biostatistician, and all relevant Co-Is to review. The biostatistical team and study manager will 
conduct analyses of accrual, drop-outs, protocol deviations on a monthly basis initially, which 
may drop to quarterly once we are confident that procedures are functioning adequately.  Drs. 
Mohr, Meyerhoff, and Stiles-Shields will review adverse events (AEs) and serious adverse 
events (SAEs) individually in real-time and in aggregate at each DSMB meeting (not less the 
annually). The PI ensures all protocol deviations, AEs, and SAEs are reported to the IRB and 
the NIMH program officer according to the applicable regulatory requirements.
For this study, the following standard AE definitions are used:
Adverse event:  Any unfavorable and unintended sign or symptom temporally associated with 
the use of digital mental health interventions, regardless of whether it is considered related to 
intervention.
Serious Adverse Event:  Any AE that results in any of the following outcomes:
o
Death
o
Life-threatening situation 
o
Event requiring inpatient hospitalization or prolongation of existing hospitalization
o
Persistent or significant disability/incapacity
AEs are graded according to the following scale:
o
Mild:  An experience that is transient and requires no special treatment or intervention.  
The experience does not generally interfere with usual daily activities.  
o
Moderate:  An experience that is alleviated with simple therapeutic treatments.  The 
experience impacts usual daily activities.
o
Severe:  An experience that requires therapeutic intervention.  The experience interrupts 
usual daily activities. If hospitalization (or prolongation of hospitalization) is required for 
treatment it becomes an SAE.
The study uses the following AE attribution scale:
o
Not related:  The AE is clearly not related to the study procedures (i.e., another cause of 
the event is most plausible and/or a clinically plausible temporal sequence is 
inconsistent with the onset of the event).  
o
Possibly related:  An event that follows a reasonable temporal sequence from the 
initiation of study procedures, but that could readily have been produced by a number of 
other factors.
o
Related:  The AE is clearly related to the study procedures.  
SAEs and specific treatment-associated AEs are reported to the IRB and NIMH program officer 
within 24 hours. 
Data Safety Monitoring Board (DSMB)
The DSMB will be chaired by Greg Simon, MD, from Kaiser Permanente, Seattle.  Dr. Simon is 
a well-respected mental health researcher. He is well versed to chair a study that will include 
participants with potentially multiple minoritized identities (i.e., adolescents ages 13-17, 
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 19 of 30
HRP-593 / v03152019
racial/ethnic minoritized identity, low socioeconomic status). As chair of the DSMB, Dr. Simon 
will appoint additional board members as needed and will determine the meeting schedule. That 
said, we expect that the DSMB will meet twice in the first year of the trial (year 2 of the grant), 
and once in the final year. The PIs and Dr. Kwasny (biostatistician) will be present to facilitate 
the flow of information, however they will not be voting members.
PROVISIONS TO PROTECT THE PRIVACY INTERESTS OF PARTICIPANTS
CLINICAL TRIAL
All data are stored on secure servers. Collaborators working on this project have agreed to 
protect the data via agreements executed with Northwestern University. To protect privacy and 
confidentiality, all identifiable information collected via smartphone apps or messaging is 
encrypted at rest.  Identifiers, such as the contact phone numbers from the participants contact 
lists are hashed, and therefore not identifiable in the research database. To help protect 
participant privacy, this study was awarded a Certificate of Confidentiality from the National 
Institutes of Health. This certificate would help us legally to refuse to disclose information that 
may identify participants in federal, state, or local civil, criminal, administrative, legislative, or 
other proceedings, if there is a court subpoena.  Confidentiality may be broken by research staff 
to ensure the patient’s safety if there is an imminent threat to self. There is also the remote 
possibility that research records will be subpoenaed by a court of law. All of these potential 
losses of confidentiality will be disclosed in the consent documents. All data presentation will be 
of aggregate-level data; participants are never individually named and identifiable data are 
never presented. Researchers do not have plans to evaluate the legality of participants’ 
behaviors, nor do we expect to gather information that would directly reveal engagement in 
illegal behavior.  In the unexpected event that researchers do become aware of participants 
engaging in illegal behavior, we would protect participant confidentiality per the terms set forth in 
the NIH certificate of confidentiality. 
Limits to Confidentiality
Participants will be notified during the informed consent process that a break in confidentiality 
may be necessary if we learn of child abuse, neglect, and/or harm to self or others. Adolescent 
participants (age 13-17) will be notified that their parents/guardians may have to be notified to 
keep them safe. As mandated by Illinois state law, all disclosures of abuse or neglect will be 
reported to the Department of Children and Family Services (DCFS) and all clinical staff will 
complete the Illinois Mandated Reporter Training 
(https://mr.dcfstraining.org/UserAuth/Login!loginPage.action;jsessionid=A12E50700101F9604F
D9FFDBCFA136ED). A pediatric psychologist (Dr. Stiles-Shields) will be involved in managing 
reports of abuse and neglect. 
ECONOMIC BURDEN TO PARTICIPANTS
There will be no cost to participants to receive the TES.  It is possible that some participants 
might have very limited data plans, or text messaging plans, and the use of the study apps and 
coaching via text message may cause them to exceed their data limit resulting in fees. The 
study apps do not use unusually large amounts of data, and text messages are generally limited 
to a few a week, so this is an unlikely occurrence, and the study team asks about participant 
data plans and will inform participants of the expected data usage prior to downloading the 
apps. There are no procedures that will be billed to insurance. 
CONSENT PROCESS
The following description of consent procedures for the clinical trial is broken into two sections.  
Section 1 describes general consent procedures that will apply to all subjects regardless of 
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 20 of 30
HRP-593 / v03152019
age.  Section 2 describes specialized procedures that apply only to the combined adolescent 
assent/parental consent process for participants aged 13-17.
 
Section 1: General Consent Procedures
The informed consent process will be electronic and will be executed through Northwestern 
University’s instance of REDCap. Individuals engaged in the informed consent process will read 
through the consent form, answer a series of questions to assess comprehension, and then will 
be asked “do you wish to participate?”. 
 
People who answer “no” to “do you wish to participate” when completing the consent process 
will be prompted to let the study team know why they do not wish to participate.  The response 
choices will include:

I'd like to speak with a study representative before signing the consent form

I have concerns about the study (please specify below)

Time commitment

Other (please specify below)
A member of the study team will follow up with all individuals who wish to speak to a study 
representative.  Participants who answer “no” to “do you wish to participate” will not be able to 
execute the consent form or complete additional study procedures.
 
People who answer “yes” to “do you wish to participate” will be asked to type their name, 
electronically sign, date, and press submit.
 
Once the electronic consent form has been fully executed and submitted, two things will 
happen:
1.  Participants will receive one of the following messages:

For subjects ages 18+: “Thank you for consenting to participate in the iCan 4 Wellness 
study. As a next step, you may continue on to complete the initial paid survey now, or 
you may press save and return to complete it at a later date.  You will receive daily 
reminders to complete this survey within the next week.”

For subjects ages 13-17: “Thank you for consenting to participate in the iCan 4 Wellness 
study. A member of the study team will review the information you provided and contact 
your parent or guardian to obtain their permission for you to join the study.  Once we 
have your parent or guardian’s permission for you to join the study, we will contact you 
with instructions for completing the initial paid survey.” 
2.  The research coordinator will receive an electronic notification which will prompt them to 
confirm (1) that the form was signed by the participant and (2) that the comprehension 
questions were answered correctly. The research coordinator will then contact the appropriate 
party by telephone to follow-up on any of the comprehension questions that were left blank or 
answered incorrectly.
 
REDCap reports will be generated daily to alert the study coordinator to follow-up with subjects 
who have not submitted the digital consent form within 48 hours of receiving it.  The coordinator 
will follow-up with individuals who have not submitted consent up to 3 times before discontinuing 
efforts to enroll the subject in the study. Participants may contact study staff at any time for a 
copy of the consent form for their records.
 
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 21 of 30
HRP-593 / v03152019
Once the informed consent process has been fully executed, the participant will proceed with 
study assessments and randomization to Treatment as Usual or the TES Intervention.
 
Section 2: Specialized Procedures for obtaining participant assent (ages 13-17)/parental 
consent
The adolescent assent/parent or guardian consent form is one document (administered 
electronically).  A consent protocol that includes bilingual staff and a Spanish-language version 
of the assent/parent or guardian consent is in place for English-speaking minors with Spanish-
speaking parents (see below).
First, participants will be presented with the IRB approved assent language.  Once the 
adolescent participant has finished reading the assent verbiage, they will be asked to sign the 
assent form if they are interested in participating.  Once submitted, a member of the research 
team will contact the parent/guardian whose child has completed the online screening and 
assent (parent/guardian contact information is gathered during the online screening process, as 
well as parent’s preferred language (English or Spanish)) to inform them that their child is 
interested in joining the research study and ask if they are willing to review the online consent 
form and provide permission for their child to participate.  If the parent is willing to review and 
sign the online consent, they will review a copy of the assent form that their child signed and 
they will be asked to add their own signature to the consent portion of the form (or to a Spanish-
language version, as applicable). The parent/guardian will be asked to verify that they have 
legal authority to consent to the child’s participation in the research. A research coordinator will 
verify that the parent/guardian has executed the consent form before engaging the adolescent 
participant in post-consent procedures.
Spanish-speaking Parents
Although we are recruiting only English speakers to participate in the study activities, we will be 
obtaining parental/guardian consent from Spanish-speaking parents as needed using a Spanish 
version of the informed assent/consent form. The Spanish ICF and assent have been translated 
and certified by the Rush Interpreter Services Office. A Rush-qualified bilingual staff (QBS) 
member on the study team will be obtaining permission from Spanish-speaking parents. QBS 
certification is granted by the Rush Interpreter Services Office after confirming the staff 
member’s Spanish-language abilities.
Note regarding the add-on focus group: The focus group addendum will not be translated into 
Spanish as the study sample contains no adolescents with Spanish-speaking parents.
NON-ENGLISH-SPEAKING PARTICIPANTS
Non-English-speaking participants will not be enrolled. 
PROTECTED HEALTH INFORMATION (PHI AND HIPAA)
We will collect the following PHI:

Zip Code
Zip code is currently collected via the demographics survey on REDCap; however, due to the 
later addition of this self-reported variable, these data will be missing for approximately 114 
participants. The missing data will be collected by IRB-approved Rush study staff via manual 
review of Rush’s instance of Epic and securely transferred to Northwestern with a DUA in place. 
In most cases, the zip code data can be obtained from the “patient lookup” screen, which is a 
snapshot of the patient’s demographics that does not require opening the medical chart.
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 22 of 30
HRP-593 / v03152019
We are requesting a HIPPA Waiver of Authorization for this activity because all 114 of these 
participants have completed the study (i.e., none are in follow-up). While it is feasible to reach 
out to the participants post-study completion to obtain permission/zip codes (we have their email 
addresses and phone numbers), we anticipate that the majority will not respond, resulting in a 
dataset that could not adequately characterize the study sample in terms of geographic location. 
This will make it impossible to ascertain the percentage of participants enrolled who reside in 
the underserved communities the study intended to reach (some of which have the highest 
levels of poverty, lowest life expectancy, and worst health outcomes in Chicago). Furthermore, 
the team will be unable to evaluate whether the intervention under investigation can reduce 
health disparities which are prominent across Chicagoland’s communities. The activity is 
minimal risk in nature, as the zip code can be obtained from the demographics snapshot, which 
does not entail opening the record.
We are requesting a waiver of consent as this activity poses minimal risk to subjects (data are 
obtainable from a snapshot of the medical record that does not entail staff to open the chart, 
they will be entered directly into a secure platform (REDCap) and securely stored). Additionally, 
efforts to reach out to 114 participants for their consent may yield a low response rate, resulting 
in an incomplete dataset which will make it impossible for us to ascertain if we reached the 
communities we intended to reach (see above). The waiver of consent does not adversely affect 
the rights and welfare of subjects – these are data participants may already provide for payment 
purposes, data will be protected and shared only with approved research staff, and findings will 
be reported only in aggregate (without personal identifiers).
Zip codes will be manually entered directly into NU’s instance of REDCap, which is accessible 
only to approved research staff. When downloaded from REDCap, data will be stored in secure 
FSM servers. The data will be stored indefinitely for future related analysis, as approved by the 
IRB. To protect the data from improper use or disclosure, participants’ direct personal identifiers 
will not be released without prior consent, except as specifically required by law or for other 
research where the use or disclosure of the PHI is not required by 45 CFR 164.512. Efforts will 
be made to limit the use and disclosure of the data to people who have a need to review this 
information. The data manager will share stored data with collaborators who have executed 
data Data Use Agreements with Northwestern University.
QUALIFICATIONS TO CONDUCT RESEARCH AND RESOURCES AVAILABLE
The research staff at CBITs have extensive experience developing and evaluating remote-
delivered mental health interventions. 
Dr. Mohr is the Director of Northwestern University’s Center for Behavioral Intervention 
Technologies (CBITs) and a tenured professor in the Department of Preventive Medicine. CBITs, 
which was founded in 2011, is a unique center in the United States, focused on the development, 
evaluation and implementation of technologies, including computer, mobile, and sensors, aimed 
at changing behavior in support of health, mental health, function, quality of life, and wellness. Dr. 
Mohr’s work lies at the intersection of behavioral science, technology, and clinical intervention 
research, and is focused on developing and evaluating interventions that harness Internet and 
wireless technologies to promote health. This work in development includes the design, 
development and implementation of mobile and computer-based treatments for depression and 
anxiety in healthcare settings as well as the development of a context sensing mobile application 
that harnesses embedded mobile phone sensor data to identify behavioral phenotypes related to 
depression and mental health. Much of the work has involved clinical trials methods.  The P50 
ALACRITY center (which funds this research project) will provide a significant advancement in 
design and clinical trials methodology, adapting them to fit digital mental health interventions as 
well as implementation and sustainment.  This project also is in many ways the culmination of 
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 23 of 30
HRP-593 / v03152019
more than a quarter of a century of research, continuously funded by the NIMH, as we move from 
efficacy and effectiveness trials to solution-focused trials with the end goal of leaving a sustainably 
implemented digital mental health intervention in a real-world care setting. 
 
Rush and CBITs staff have been working closely on the development of this project for several 
years and are collaborating to put the resources in place to conduct this study. The services 
being provided in this study are a supplement to standard care, and providers will continue to 
provide all standard care services to patients who enroll in this study.
The UIC site PI (Dr. Stiles-Shields) is a licensed clinical psychologist who has worked in clinical 
and research capacities with children under the age of 18 for more than a decade. She was an 
Assistant Professor in the Department of Psychiatry and Behavioral Sciences at Rush University 
Medical Center for over three years and maintains an adjunct faculty appointment there. She is 
now an Assistant Professor at the Institute for Juvenile Research at the University of Illinois 
Chicago, whose campus abuts the research site at Rush University Medical Center.  The Rush 
site PI (Dr. Sanchez-Johnsen) is a clinical psychologist and Associate Professor and Vice-Chair 
for Research in the Department of Family Medicine at Rush University, with extensive 
experience in research with underserved and minoritized populations. 
COLLABORATIVE RESEARCH PARTNERS
The collaborative research team is comprised of personnel from Northwestern University, Rush 
University Medical Center, The University of Illinois at Chicago, Ksana Health, and Audacious 
Software. The proposed roles of personnel from each site are delineated in the table below. 
Table 7. Research Sites and Vendors
Site
Role
Approvals
Northwestern 
University (NU)
NU will serve as the lead site for this study.  NU 
personnel will be responsible for maintaining 
current versions of the protocol, consent 
documents, and other IRB materials. NU staff 
will be responsible for screening and consenting 
study participants, collecting outcome data, 
delivering and supervising the TES, disbursing 
subject payments, managing suicidality 
assessments, and analyzing study data. NU 
staff will facilitate regular project meetings with 
participating sites and will be responsible for 
communicating procedural changes, emergent 
issues, interim results, and study status 
updates.   NU will be responsible for ensuring 
that all sites involved in data handling have 
executed the appropriate agreements. 
IRB protocol number 
STU00211887
Rush University 
Medical Center 
Rush will serve as the main recruitment and 
implementation site for this study.  Rush 
personnel will be responsible for referring study 
participants, monitoring delivery of the TES, 
managing suicidality, and facilitating access to 
organizational-level data .  Investigators from 
Rush will be involved in data analysis.
A reliance agreement, executed 
via Smart IRB, has been 
uploaded to the eirb+ system.  
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 24 of 30
HRP-593 / v03152019
The University of 
Illinois at 
Chicago (UIC)
Effective 9/1/22, Co-PI, Dr. Colleen Stiles-
Shields, will provide study oversight from a 
faculty position at UIC.  Dr. Stiles-Shields will be 
engaged in non-exempt study activities (which 
may include direct contact with participants and 
access to identifying study data).
A reliance agreement will be 
executed via Smart IRB.
A single IRB consultation intake 
form was submitted on 10.7.22
Ksana Health
Software Development/Maintenance: Ksana 
Health established several agreements with 
Northwestern University to make their Vira 
platform available for use in delivery of the TES. 
Data Storage/Retention:  Ksana Health will store 
data on their secure servers during the trial and 
transfer it to NU at the end of the trial. Upon 
termination of this Agreement Ksana Health 
shall erase, destroy, and render unrecoverable 
all Northwestern data.
The following agreements 
executed between NU and 
Ksana Health have been 
uploaded to the eirb+ portal.
Northwestern University - FSM 
and Ksana Subscription 
Services Agreement Signed by 
NU 06302021.pdf
Northwestern University - FSM 
Information Security Addendum 
for Ksana Health Signed by NU 
06302021.pdf
Northwestern University Service 
Level and Support Agreement 
for Ksana Health Signed by NU 
06302021.pdf
Ksana Health has passed all 
security vetting by FSMIT.
Audacious 
Software
Software Development/Maintenance: Chris Karr, 
Founder and CEO of Audacious Software, is 
working as a Northwestern University contractor 
and will be responsible for the development and 
maintenance of two pieces of software that will 
be deployed in the clinical trial:
1.
The Mood Education app that will be 
used by participants randomized to CT
2.
The secure text messaging dashboard 
that will be used to deliver the TES 
coaching service. 
 
Data Storage/Retention:  Mr. Karr is working 
with FSM IT to store all data on FSMIT 
managed servers/cloud storage environments.  
Data resulting from the applications built and 
managed by Audacious Software will not leave 
NU servers.
 
The following agreement has 
been executed between 
Audacious Software and 
Northwestern University and 
have been uploaded to the eirb+ 
portal.
Audacious Software - Services 
Agreement 2016FEB01 
executed.pdf
Audacious Software is not 
subject to FSMIT security vetting 
since all data will be stored in 
NU/FSMIT on-premises servers.
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 25 of 30
HRP-593 / v03152019
REFERENCES
1.
Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and 
comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey 
Replication. Arch Gen Psychiatry. 2005;62(6):617-627.
2.
Ferrari AJ, Charlson FJ, Norman RE, et al. Burden of depressive disorders by country, 
sex, age, and year: findings from the global burden of disease study 2010. PLoS Med. 
2013;10(11):e1001547.
3.
Wells K, Miranda J, Bauer M, et al. Overcoming barriers to reducing the burden of 
affective disorders. Biol Psychiatry. 2002;52(6):655.
4.
Greenberg PE, Fournier AA, Sisitsky T, Pike CT, Kessler RC. The economic burden of 
adults with major depressive disorder in the United States (2005 and 2010). J Clin 
Psychiatry. 2015;76(2):155-162.
5.
Greenberg PE, Sisitsky T, Kessler RC, et al. The economic burden of anxiety disorders 
in the 1990s. J Clin Psychiatry. 1999;60(7):427-435.
6.
Marcus SC, Olfson M. National trends in the treatment for depression from 1998 to 
2007. Arch Gen Psychiatry. 2010;67(12):1265-1273.
7.
Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, Kessler RC. Twelve-month use of 
mental health services in the United States: results from the National Comorbidity 
Survey Replication. Arch Gen Psychiatry. 2005;62(6):629-640.
8.
Wagner B, Horn AB, Maercker A. Internet-based versus face-to-face cognitive-
behavioral intervention for depression: a randomized controlled non-inferiority trial. J 
Affect Disord. 2014;152-154:113-121.
9.
Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided Internet-based vs. 
face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a 
systematic review and meta-analysis. World Psychiatry. 2014;13(3):288-295.
10.
Cuijpers P, Marks IM, van Straten A, Cavanagh K, Gega L, Andersson G. Computer-
aided psychotherapy for anxiety disorders: a meta-analytic review. Cogn Behav Ther. 
2009;38(2):66-82.
11.
Richards D, Richardson T. Computer-based psychological treatments for depression: a 
systematic review and meta-analysis. Clin Psychol Rev. 2012;32(4):329-342.
12.
Donker T, Petrie K, Proudfoot J, Clarke J, Birch MR, Christensen H. Smartphones for 
Smarter Delivery of Mental Health Programs: A Systematic Review. J Med Internet Res. 
2013;15(11):e247.
13.
Firth J, Torous J, Nicholas J, Carney R, Rosenbaum S, Sarris J. Can smartphone mental 
health interventions reduce symptoms of anxiety? A meta-analysis of randomized 
controlled trials. J Affect Disord. 2017;218:15-22.
14.
Mohr DC, Weingardt KR, Reddy M, Schueller SM. Three Problems With Current Digital 
Mental Health Research . . . and Three Things We Can Do About Them. Psychiatr Serv. 
2017:appips201600541.
15.
Bertagnoli A, Trangle M, Marx L, Lacoutture E, Rukanonchai D, Schuster J. Behavioral 
Health. Paper presented at: Annual Meeting of the Alliance of Community Health Plans; 
Sept 28-30, 2015, 2015; Washington, DC.
16.
Gilbody S, Littlewood E, Hewitt C, et al. Computerised cognitive behaviour therapy 
(cCBT) as treatment for depression in primary care (REEACT trial): large scale 
pragmatic randomised controlled trial. BMJ. 2015;351:h5627.
17.
Gilbody S, Brabyn S, Lovell K, et al. Telephone-supported computerised cognitive-
behavioural therapy: REEACT-2 large-scale pragmatic randomised controlled trial. Br J 
Psychiatry. 2017;210(5):362-367.
18.
Duarte A, Walker S, Littlewood E, et al. Cost-effectiveness of computerized cognitive-
behavioural therapy for the treatment of depression in primary care: findings from the 
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 26 of 30
HRP-593 / v03152019
Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy 
(REEACT) trial. Psychol Med. 2017;47(10):1825-1835.
19.
Mohr DC, Zhang M, Schueller SM. Personal Sensing: Understanding Mental Health 
Using Ubiquitous Sensors and Machine Learning. Annu Rev Clin Psychol. 2017.
20.
Saeb S, Cybulski TR, Schueller SM, Kording KP, Mohr DC. Scalable Passive Sleep 
Monitoring Using Mobile Phones: Opportunities and Obstacles. J Med Internet Res. 
2017;19(4):e118.
21.
Saeb S, Lattie E, Schueller SM, Kording K, Mohr DC. The relationship between mobile 
phone location sensor data and depressive symptom severity. PeerJ. 2016;4(e2537).
22.
Saeb S, Zhang M, Kwasny MM, Karr CJ, Kording KP, Mohr DC. The Relationship 
between Clinical, Momentary, and Sensor-based Assessment of Depression. 
International Conference on Pervasive Computing Technologies for Healthcare; 2015; 
Istanbul, Turkey.
23.
Saeb S, Zhang M, Karr CJ, et al. Mobile Phone Sensor Correlates of Depressive 
Symptom Severity in Daily-Life Behavior: An Exploratory Study. J Med Internet Res. 
2015;17(7):e175.
24.
Saeb S, Kording K, Mohr DC. Making Activity Recognition Robust against Deceptive 
Behavior. PLoS One. 2015;10(12):e0144795.
25.
Corden ME, Koucky EM, Brenner C, et al. MedLink: A mobile intervention to improve 
medication adherence and processes of care for treatment of depression in general 
medicine. Digital Health. 2016;2:1-10.
26.
Mohr DC, Montague E, Stiles-Shields C, et al. MedLink: A mobile intervention to address 
failure points in the treatment of depression in general medicine. Paper presented at: 
The Proceedings of Pervasive Health ’15. IEEE2015; Istanbul, Turkey.
27.
Oulasvirta A, Tamminen S, Roto V, Kuorelahti J. Interaction in 4-Second Bursts: The 
Fragmented Nature of Attentional Resources in Mobile HCI. Paper presented at: SIGCHI 
Conference on Human Factors in Computing Systems2005; Portland, Oregon, USA.
28.
Vaish R, Wyngarden K, Chen J, Cheung B, Bernstein MS. Twitch Crowdsourcing: Crowd 
Contributions in Short Bursts of Time. SIGCHI Conference on Human Factors in 
Computing Systems; 2014; Toronto, ON, Canada.
29.
Mohr DC, Tomasino KN, Lattie EG, et al. IntelliCare: An Eclectic, Skills-Based App Suite 
for the Treatment of Depression and Anxiety. J Med Internet Res. 2017;19(1):e10.
30.
Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation 
hybrid designs: combining elements of clinical effectiveness and implementation 
research to enhance public health impact. Med Care. 2012;50(3):217-226.
31.
Mohr D, Lyon AR, Lattie E, Reddy M, Schueller S. Accelerating Digital Mental Health 
Research From Early Design and Creation to Successful Implementation and 
Sustainment. J Med Internet Res 2017;19(5):e153.
32.
Mohr DC, Schueller SM, Riley WT, et al. Trials of Intervention Principles: Evaluation 
Methods for Evolving Behavioral Intervention Technologies. J Med Internet Res. 
2015;17(7):e166.
33.
Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity 
measure. J Gen Intern Med. 2001;16(9):606-613.
34.
Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for assessing generalized 
anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092-1097.
35.
Brooke J. SUS-A quick and dirty usability scale. Usability evaluation in industry. 
1996;189(194):4-7.
36.
Lyon AR. Acceptability, Feasibility, and Appropriateness Scale (AFAS) 2012, 
Unpublished Manuscript.
37.
Regier DA, Narrow WE, Rae DS, Manderscheid RW, Locke BZ, Goodwin FK. The de 
facto US mental and addictive disorders service system. Epidemiologic catchment area 
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 27 of 30
HRP-593 / v03152019
prospective 1-year prevalence rates of disorders and services. Arch Gen Psychiatry. 
1993;50(2):85-94.
38.
Burns BJ, Ryan Wagner H, Gaynes BN, Wells KB, Schulberg HC. General medical and 
specialty mental health service use for major depression. Int J Psychiatry Med. 
2000;30(2):127-143.
39.
Unutzer J, Schoenbaum M, Druss BG, Katon WJ. Transforming mental health care at 
the interface with general medicine: report for the presidents commission. Psychiatr 
Serv. 2006;57(1):37-47.
40.
Simon GE. Evidence review: efficacy and effectiveness of antidepressant treatment in 
primary care. Gen Hosp Psychiatry. 2002;24(4):213-224.
41.
Mohr DC, Hart SL, Howard I, et al. Barriers to psychotherapy among depressed and 
nondepressed primary care patients. Ann Behav Med. 2006;32(3):254-258.
42.
Mohr DC, Ho J, Duffecy J, et al. Perceived barriers to psychological treatments and their 
relationship to depression. J Clin Psychol. 2010;66(4):394-409.
43.
Bower P, Gilbody S. Managing common mental health disorders in primary care: 
conceptual models and evidence base. BMJ. 2005;330(7495):839-842.
44.
Archer J, Bower P, Gilbody S, et al. Collaborative care for depression and anxiety 
problems. Cochrane Database Syst Rev. 2012;10:CD006525.
45.
Blasinsky M, Goldman HH, Unutzer J. Project IMPACT: a report on barriers and 
facilitators to sustainability. Adm Policy Ment Health. 2006;33(6):718-729.
46.
Solberg LI, Crain AL, Jaeckels N, et al. The DIAMOND initiative: implementing 
collaborative care for depression in 75 primary care clinics. Implementation science : IS. 
2013;8:135.
47.
Rainie L, Zickuhr K. Chapter 1: Always on Connectivity. [URL removed per NIH 
guidelines].
48.
Deloitte. 2018 global mobile consumer survey: US edition. A new era in mobile 
continues. 2018.
49.
Pew Research Center: Internet ST. Mobile Fact Sheet. 2018. [URL removed per NIH 
guidelines].
50.
Andrews G, Basu A, Cuijpers P, et al. Computer therapy for the anxiety and depression 
disorders is effective, acceptable and practical health care: An updated meta-analysis. J 
Anxiety Disord. 2018;55:70-78.
51.
Firth J, Torous J, Nicholas J, et al. The efficacy of smartphone-based mental health 
interventions for depressive symptoms: a meta-analysis of randomized controlled trials. 
World Psychiatry. 2017;16(3):287-298.
52.
Kelly K. The inevitable. New York: Viking Press; 2016.
53.
Fanning J, Mullen SP, McAuley E. Increasing physical activity with mobile devices: a 
meta-analysis. J Med Internet Res. 2012;14(6):e161.
54.
Nayak JG, Hartzler AL, Macleod LC, Izard JP, Dalkin BM, Gore JL. Relevance of graph 
literacy in the development of patient-centered communication tools. Patient Educ 
Couns. 2016;99(3):448-454.
55.
Burns MN, Begale M, Duffecy J, et al. Harnessing context sensing to develop a mobile 
intervention for depression. J Med Internet Res. 2011;13(3):e55.
56.
Canzian L, Musolesi M. Trajectories of Depression: Unobtrusive Monitoring of 
Depressive States by means of Smartphone Mobility Traces Analysis. Paper presented 
at: UniComp '152015; Osaka, Japan.
57.
Palmius N, Tsanas A, Saunders K, et al. Detecting bipolar depression from geographic 
location data. Under Review.
58.
Saunders K, Palmius N, de Vos M, Bilderbeck A, Geddes J, Goodwin G. Depression 
detection in bipolar disorder using geolocation data. Bipolar Disorders. 2016;18:153-154.
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 28 of 30
HRP-593 / v03152019
59.
Saeb S, Lattie EG, Kording KP, Mohr DC. Mobile Phone Detection of Semantic Location 
and Its Relationship to Depression and Anxiety. JMIR Mhealth Uhealth. 2017;5(8):e112.
60.
Min J-M, Doryab A, Wiese J, Amini S, Zimmerman J, Hong JI. Toss ‘N’ Turn: 
Smartphone as Sleep and Sleep Quality Detector. Paper presented at: CHI 20142014; 
Toronto, Ontario, Canada.
61.
Poh MZ, McDuff DJ, Picard RW. Non-contact, automated cardiac pulse measurements 
using video imaging and blind source separation. Opt Express. 2010;18(10):10762-
10774.
62.
Poh MZ, McDuff DJ, Picard RW. Advancements in noncontact, multiparameter 
physiological measurements using a webcam. IEEE Trans Biomed Eng. 2011;58(1):7-
11.
63.
McDuff D, Gontarek S, Picard RW. Improvements in remote cardiopulmonary 
measurement using a five band digital camera. IEEE Trans Biomed Eng. 
2014;61(10):2593-2601.
64.
Adams P, Rabbi M, Rahman T, et al. Towards Personal Stress Informatics: Comparing 
Minimally Invasive Techniques for Measuring Daily Stress in the Wild. Pervasive Health; 
2014; Oldenburg, Germany.
65.
Lu H, Rabbi M, Chittaranjan GT, et al. StressSense: Detecting stress in unconstrained 
acoustic environments using smartphones. Paper presented at: ACM International Joint 
Conference on Pervasive and Ubiquitous Computing2012; Pittsburgh, USA.
66.
Park G, Schwartz HA, Eichstaedt JC, et al. Automatic personality assessment through 
social media language. J Pers Soc Psychol. 2015;108(6):934-952.
67.
Schwartz HA, Eichstaedt J, Kern M, et al. Towards assessing changes in degree of 
depression through Facebook. Paper presented at: Conference of the Association for 
Computational Linguistics2014; Baltimore, Maryland.
68.
Kaiser EC, Cambo S, Gergle D, Mohr DC, Schueller SM. Exploring User Needs in the 
Development of Context-Aware Mental Health Interventions. Anxiety and Depression 
Conference; 2017; San Francisco, CA.
69.
Meng J, Hussain SA, Mohr DC, Czerwinski M, Zhang M. Enhancing university 
counseling services: Exploring user needs of a mobile behavioral sensing technology for 
depression management. J Med Internet Res. In Press.
70.
Martell CR, Dimidjian S, Herman-Dunn R. Behavioral activation for depression: A 
clinician's guide. New York: Guilford; 2010.
71.
Emrich A. MicroProse’s strategic space opera is rated XXXX. Computer Gaming World. 
1993;110:92-93.
72.
Reeves B, Nass C. The Media Equation: How people treat computers, television, and 
new media like real people and places. New York, NY: CSLI Publications and 
Cambridge University Press; 1996.
73.
Fogg BJ, Nass C. Silicon sycophants: The effects of computers that flatter. International 
Journal of Human-Computer Studies. 1997;46:551-561.
74.
Lee EJ. I like you, but I won't listen to you: Effects of rationality on affective and 
behavioral responses to computers that flatter. International Journal of Human Computer 
Studies. 2009;67:628-638.
75.
Mishra P. Affective feedback from computers and its effect on perceived ability and 
affect: A test of the computers as social actor hypothesis. Journal of Educational 
Multimedia and Hypermedia. 2006;15:107-131.
76.
Beutler LE, Engle D, Mohr D, et al. Predictors of differential response to cognitive, 
experiential, and self-directed psychotherapeutic procedures. J Consult Clin Psychol. 
1991;59(2):333-340.
77.
Beutler LE, Harwood TM, Michelson A, Song X, Holman J. Resistance/reactance level. J 
Clin Psychol. 2011;67(2):133-142.
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 29 of 30
HRP-593 / v03152019
78.
Bentley F, Tollmar K, Stephenson P, et al. Health Mashups: Presenting Statistical 
Patterns between Wellbeing Data and Context in Natural Language to Promote Behavior 
Change. Acm T Comput-Hum Int. 2013;20(5).
79.
Noah B, Keller MS, Mosadeghi S, et al. Impact of remote patient monitoring on clinical 
outcomes: an updated meta-analysis of randomized controlled trials. npj Digital 
Medicine. 2018;1(2):1-12.
80.
Karkar R, Zia J, Vilardaga R, et al. A framework for self-experimentation in personalized 
health. J Am Med Inform Assoc. 2016;23(3):440-448.
81.
Wolever RQ, Simmons LA, Sforzo GA, et al. A Systematic Review of the Literature on 
Health and Wellness Coaching: Defining a Key Behavioral intervention in Healthcare. 
Global Advances in Health and Medicine. 2013;2(4):38-57.
82.
Schueller SM, Tomasino KN, Mohr DC. Integrating Human Support into Behavioral 
Intervention Technologies: The Efficiency Model of Support. Clinical Psychology: 
Science and Practice. 2016.
83.
Shilton K. Four Billion Little Brothers? Privacy, mobile phones, and ubiquitous data 
collection. Commun Acm. 2009;52(11):48-53.
84.
Wallerstein N, Duran B. Community-based participatory research contributions to 
intervention research: the intersection of science and practice to improve health equity. 
Am J Public Health. 2010;100(S1):S40-S46.
85.
Koo C, Chung N, Kim HW. Examining explorative and exploitative uses of smartphones: 
a user competence perspective. Inform Technol Peopl. 2015;28(1):133-162.
86.
Faulkner L. Beyond the five-user assumption: Benefits of increased sample sizes in 
usability testing. Behavior Research Methods, Instruments, & Computers. 
2003;35(3):379-383.
87.
Nielsen J. Severity ratings for usability problems. Papers and Essays. 1995;54.
88.
Noth KN, Bardsley L, Lattie EG, Mohr DC. IntelliCare Study Coaching Manual. Chicago, 
IL: Northwestern University; 2018: [URL removed per NIH guidelines].
89.
Kroenke K, Spitzer RL, Williams JB. The Patient Health Questionnaire-2: validity of a 
two-item depression screener. Med Care. 2003;41(11):1284-1292.
90.
Brown CH, Ten Have TR, Jo B, et al. Adaptive designs for randomized trials in public 
health. Annu Rev Public Health. 2009;30:1-25.
91.
Brown CA, Lilford RJ. The stepped wedge trial design: a systematic review. BMC Med 
Res Methodol. 2006;6:54.
92.
Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized 
trials. Contemp Clin Trials. 2007;28(2):182-191.
93.
Joiner TE, Jr., Pfaff JJ, Acres JG. A brief screening tool for suicidal symptoms in 
adolescents and young adults in general health settings: reliability and validity data from 
the Australian National General Practice Youth Suicide Prevention Project. Behav Res 
Ther. 2002;40(4):471-481.
94.
Janssen MF, Birnie E, Haagsma JA, Bonsel GJ. Comparing the standard EQ-5D three-
level system with a five-level version. Value Health. 2008;11(2):275-284.
95.
Lund AM. Measuring usability with the USE Questionnaire. STC Usability Newsletter. 
2001;8(2).
96.
Weiner BJ, Lewis, C. C., Stanick, C., Powell, B. J., Dorsey, C. N., Clary, A. C., Boynton, 
M. H., Halko, H. . Psychometric assessment of three newly developed implementation 
outcome measures. Implementation science : IS. 2017;12:108.
97.
Attkisson CC, Zwick R. The client satisfaction questionnaire. Psychometric properties 
and correlations with service utilization and psychotherapy outcome. Eval Program 
Plann. 1982;5(3):233-237.
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.

STU#:00211887
Version Date:  8.11.23 30 of 30
HRP-593 / v03152019
98.
Jenkinson C, Layte R, Jenkinson D, et al. A shorter form health survey: can the SF-12 
replicate results from the SF-36 in longitudinal studies? J Public Health Med. 
1997;19(2):179-186.
99.
Saunders JB, Aasland OG, Babor TF, de la Fuente JR, Grant M. Development of the 
Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early 
Detection of Persons with Harmful Alcohol Consumption--II. Addiction. 1993;88(6):791-
804.
100.
Berman AH, Bergman H, Palmstierna T, Schlyter F. Evaluation of the Drug Use 
Disorders Identification Test (DUDIT) in criminal justice and detoxification settings and in 
a Swedish population sample. Eur Addict Res. 2005;11(1):22-31.
101.
Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic 
data capture (REDCap)--a metadata-driven methodology and workflow process for 
providing translational research informatics support. Journal of biomedical informatics. 
2009;42(2):377-381.
102.
Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: 
conceptual distinctions, measurement challenges, and research agenda. Adm Policy 
Ment Health. 2011;38(2):65-76.
103.
Shortell SM, Bennett CL, Byck GR. Assessing the impact of continuous quality 
improvement on clinical practice: what it will take to accelerate progress. Milbank Q. 
1998;76(4):593-624, 510.
104.
Bhuiyan N, Bahghel A. An overview of continuous improvement: from the past to the 
present. Management Decision 2005;43(5):761-771.
105.
Johnson CM, Johnson TR, Zhang J. A user-centered framework for redesigning health 
care interfaces. Journal of biomedical informatics. 2005;38(1):75-87.
106.
Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster 
randomised trial: rationale, design, analysis, and reporting. BMJ. 2015;350:h391.
107.
Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation 
research in mental health services: an emerging science with conceptual, 
methodological, and training challenges. Adm Policy Ment Health. 2009;36(1):24-34.
108.
Poots AJ, Woodcock T. Statistical process control for data without inherent order. BMC 
medical informatics and decision making. 2012;12:86.
109.
Benneyan JC, Lloyd RC, Plsek PE. Statistical process control as a tool for research and 
healthcare improvement. Qual Saf Health Care. 2003;12(6):458-464.
110.
Wheeler DJ. Understanding statistical process control, 3rd edition. Knoxville, 
Tennessee: SPC Press; 2010.
111.
Vannoy SD, Arean P, Unutzer J. Advantages of using estimated depression-free days 
for evaluating treatment efficacy. Psychiatr Serv. 2010;61(2):160-163.
112.
Willan AR, O'Brien BJ. Confidence intervals for cost-effectiveness ratios: an application 
of Fieller's theorem. Health Econ. 1996;5(4):297-305.
113.
Cohen DJ, Reynolds MR. Interpreting the results of cost-effectiveness studies. J Am Coll 
Cardiol. 2008;52(25):2119-2126.
114.
Simon GE, Katon WJ, VonKorff M, et al. Cost-effectiveness of a collaborative care 
program for primary care patients with persistent depression. Am J Psychiatry. 
2001;158(10):1638-1644.
115.
Simon GE, Manning WG, Katzelnick DJ, Pearson SD, Henk HJ, Helstad CS. Cost-
effectiveness of systematic depression treatment for high utilizers of general medical 
care. Arch Gen Psychiatry. 2001;58(2):181-187.
116.
Fritz MS, Mackinnon DP. Required sample size to detect the mediated effect. Psychol 
Sci. 2007;18(3):233-239.
IRB #: STU00211887 Approved by NU IRB for use on or after 3/29/2024 through 3/28/2025.