Development and Preliminary Testing of a Non-regulated Digital Service That Supports Cognitive Behavioral Therapy for Depressive Symptoms Under Clinical Supervision
Phase: NA
Conditions: Major Depressive Disorder
Sponsor: Massachusetts General Hospital

 
 
1 
 
Study Protocol and 
Statistical Analysis Plan (SAP) 
DEVELOPMENT AND PRELIMINARY TESTING OF A NON-REGULATED DIGITAL 
SERVICE THAT SUPPORTS COGNITIVE BEHAVIORAL THERAPY FOR DEPRESSIVE 
SYMPTOMS UNDER CLINICAL SUPERVISION 
Principal Investigator:  
 
Sabine Wilhelm, PhD 
Chief of Psychology 
Massachusetts General Hospital & Harvard Medical School 
Boston, MA, United States of America 
 
 
Ethical approval:  
 
 
Mass General Brigham Institutional Review Boards  
(protocol # 2020P001958, approved on 04/08/2021) 
 
 
ClinicalTrials.gov identifier:   
NCT05386329 
 
 
Funding:  
 
 
 
This study was funded by Koa Health (formerly Telefonica  
Alpha). Investigators from MGH developed the Mindset app 
in collaboration with technologists and designers from Koa 
Health via a user-centered, iterative design approach. The 
investigators from MGH were responsible for the study 
design and for the execution of the study; authors from Koa 
Health supported the technical activities related to the 
development and deployment of the app, and provided 
funding for the project. Koa Health had no role in the 
recruitment of participants and had no access to data during 
the trial. 
 
 
Document Date:  
 
 
November 14, 2023 (shortened version) 
 
 
 
 
 
Last IRB-approved update: 09/19/23 
 
 
Author  
 
 
 
Susanne Hoeppner, PhD, MApStat 
Assistant Professor of Psychology 
Massachusetts General Hospital & Harvard Medical School 
Boston, MA, United States of America 
 
 
Version 
 
 
 
Draft 1.0 
 
 

 
 
2 
 
PRELIMINARY TESTING OF COMBINED DIGITAL COGNITIVE BEHAVIORAL 
THERAPY PLUS VIRTUAL TREATMENT FOR DEPRESSIVE SYMPTOMS  
 
I. 
SPECIFIC AIMS 
 
The overarching purpose of this project is to conduct an open trial to test the efficacy of digital 
CBT for when combined with virtual visits with a CBT therapist. Specifically, we seek to 
evaluate how digital CBT plus virtual treatment initially performs when deployed in a clinical 
setting to support treatment. This treatment will include the mobile CBT for depression app in 
combination with eight virtual visits delivered by doctoral-level psychologists or psychology 
interns with their master’s degree over a period of 8 weeks. The therapist will be assigned once 
eligibility is determined and will offer 8 virtual treatment sessions in total in addition to 
unlimited access to the app. All virtual sessions will be no longer than 16-25 min. The combined 
CBT app plus virtual treatment will be tested in 28 patients. 
 
Aim 1: Test feasibility and acceptability of the combined digital CBT plus virtual treatment. 
Examine uptake, patient satisfaction, clinician usability and satisfaction, sustained engagement, 
drop-out rates. 
 
Hypothesis 1a: We hypothesize that digital CBT plus virtual treatment will be feasible 
(based on uptake, engagement and drop-out rates).  
 
Hypothesis 1b: We hypothesize that digital CBT plus virtual treatment will be deemed 
acceptable both to patients and clinicians (based on descriptive statistics of patient 
satisfaction, app use, app feedback, clinician ratings (CGI Ratings) as well as qualitative 
feedback). 
 
Aim 2: Examine preliminary efficacy of digital CBT plus virtual treatment for primary outcomes 
(i.e., reduction in severity of depressive symptoms) and secondary outcomes (e.g., quality of life 
at post-treatment, social and occupational functioning at post-treatment).  
 
Hypothesis 2a: We hypothesize that participants receiving digital CBT plus virtual 
treatment will evidence statistically significant reductions in HAM-D scores from 
baseline to post-treatment (week 8). 
 
Hypothesis 2b: We hypothesize that participants receiving digital CBT plus virtual 
treatment will evidence statistically significant improvement on secondary clinical 
outcome measures (i.e., functional impairment (WSAS) and quality of life (Q-LES-Q)) 
from baseline to post-treatment (week 8). 
 
II. 
SUBJECT SELECTION 
 
Overview: 
 
To meet our target of 28 eligible participants, we expect to screen up to about 84 individuals. 
Detailed eligibility criteria are below.  

 
 
3 
 
 
A. Inclusion/Exclusion Criteria: 
 
1. Inclusion criteria 
a. Adults (at least 18 years of age) 
b. Living in Massachusetts 
c. Current diagnosis of primary DSM-5 MDD, based on MINI 
d. PHQ-9 score ≥ 10, indicating the presence of at least moderately severe 
depressive symptoms 
2. Exclusion criteria 
a. Psychotropic medication changes within 2 months prior to enrollment. 
Participants taking psychotropic medication must have been on a stable dose for 
at least 2 months prior to enrollment and not change medication during study 
period 
b. Past participation in > 4 sessions of CBT for depression 
c. Current severe substance use disorder 
d. Lifetime bipolar disorder or psychosis 
e. Acute, active suicidal ideation as indicated by clinical judgment and/or a score ≥2 
on the suicidal ideation subscale of the C-SSRS.  
f. Concurrent psychological treatment 
g. Does not own a supported mobile Smartphone with a data plan  
h. Lack of technology literacy that would interfere with ability to engage with 
smartphone treatment 
 
  
B. Source of Subjects and Recruitment Methods: 
 
We plan to enroll 28 participants. Potential participants may be informed about the study through 
MBTA advertisements, radio advertisements, MGB / Partners clinicians and research colleagues 
nationally, fliers posted in specialty clinics and hospitals, coffee shops, restaurants, laundromats, 
barber shops, churches, daycares, libraries, newspapers, universities, other public locations, 
through our program’s website and a recruitment website we will create for this study, Partners 
Clinical Trials (Rally), listservs (e.g., MGH Broadcast), via social media (e.g., 
Facebook/Instagram ads, our program’s Twitter account) on the Internet, and in online support 
groups. Individuals will also be recruited as part of our clinic’s general recruitment protocol.  
Interested individuals will be referred to the study RA, who will provide more information about 
the research study and assess preliminary eligibility over the telephone. We will also use a 
screening questionnaire hosted on REDCap. The link to access this screener will be embedded 
within our recruitment website. This will help identify individuals who are likely to be eligible 
for the study. 
 
III. 
SUBJECT ENROLLMENT 
 
A.  Method of enrollment, including procedures for patient registration and/or 
randomization 
 
Overview 

 
 
4 
 
Potential participants will be preliminarily screened by a trained study staff member (e.g., RA) 
over the phone to establish their likely eligibility. We only ask for the potential participant’s 
name and contact information at the end of the phone screen if the individual is eligible and 
interested in participating. The participants’ contact information may also be gathered through 
the screening questionnaire on our recruitment website. Additionally, during the phone screen 
the RA may ask permission to send eligible and interested participants a Partners/MGH 
Authorization for Release of Protected or Privileged Health Information form, which participants 
may complete with their psychiatrist’s or other healthcare provider’s contact information and 
then send directly to the study RA. If permission is obtained, the study clinician may then contact 
the participant’s healthcare provider to verify issues surrounding potential eligibility (e.g., if 
there is any question/concern about the anticipated stability of the participant’s medications), 
based on information collected by the RA during the phone screen. Screening information of 
individuals who do not meet study criteria will be destroyed. Subjects will be enrolled by MGH. 
Potential participants will be given as much time as they need to consider participation, prior to 
providing informed consent. If suicidality is expressed to an RA during the phone screen, the RA 
will notify a licensed clinician member of the study staff immediately. Per our internal 
procedures, RAs will keep the suicidal individual on the phone until a licensed clinician member 
of the study staff arrives to conduct a telephone evaluation. RAs may contact a licensed clinician 
member of the study staff directly via e-mail during the phone call, and/or email other research 
assistants who can then alert a licensed clinician member of the study staff to promptly conduct 
an evaluation over the telephone. 
 
Interested and eligible participants will be invited to complete the baseline assessment with the 
independent evaluator (IE) by secure, HIPAA-compliant video calling program (Zoom or phone). 
At that time, patients will be informed about the study’s purpose and procedures and advised 
about alternative treatment options.  Before eligibility is assessed, the IE, Dr. Greenberg, Dr. 
Bernstein, or Dr. Wilhelm will obtain electronic informed consent. The informed consent 
document will be provided to the potential participant electronically through REDCap, a secure 
data capture system, and they will be asked to select an “I agree” button and provide their 
signature in REDCap to indicate their consent (see more REDCap information below: 
“Monitoring and Quality Assurance”). The participant will be informed that they may download 
and save a copy of the consent form for their records. Self-report measures for the baseline 
assessment will be completed through a secure REDCap link emailed to participants if 
participants consent and are deemed eligible.  
 
C. Procedures for obtaining informed consent (including timing of consent process) 
 
Before participating in the screening and baseline assessment, patients will be given information 
about the study via phone and initial eligibility will be assessed. After this initial phone call, the 
RA will email interested potential participants a pdf copy of the informed consent document for 
review, prior to the screening/baseline assessment. Online consent will be used as recruitment is 
state-wide, prohibiting in-person written consent. The potential participants will be asked to 
abstain from signing the consent form until study procedures are discussed with the IE, or Drs. 
Wilhelm, Bernstein, or Greenberg, during the baseline screening video call. In cases where 
Zoom is used, before the baseline assessment appointments, the RA will send participants an 
appointment confirmation e-mail with detailed instructions for logging onto the HIPAA-

 
 
5 
 
compliant Zoom software on their devices, which is used for video call assessments. In advance 
of the appointments, the RA may conduct a brief test call with the subject to ensure that the 
subject installed the software and can access it from their devices. For safety, the RA will collect 
an emergency contact from the participant, so that the IE will have this information on hand 
during the virtual baseline assessment. The emergency contact information will also be collected 
in the demographics form after the baseline interview with the IE for confirmation. At the start of 
the screening/baseline assessment, before beginning study procedures, the IE, or Drs. Wilhelm, 
Bernstein, or Greenberg will inform potential participants about the study’s purpose and 
procedures and advise about alternative treatment options.  If the individual wishes to participate 
in the study, the IE or Drs. Wilhelm, Greenberg, or Bernstein will obtain informed consent 
electronically by asking participants to click an “I agree” button and electronically sign the 
electronic informed consent document sent through REDCap, a secure data capture system. The 
person obtaining informed consent will also verify the participants identity by requesting to view 
a form of identification (e.g., government-issued driver’s license). Participants will have the 
ability to download and print the electronic informed consent document, or save the pdf copy to 
their computer for their records. The research assistant will send participants a signed pdf copy 
of the consent form for their records. 
 
D. Treatment assignment and randomization (if applicable) 
 
N/A for the current study. All participants will receive the same treatment. 
 
STUDY PROCEDURES 
 
Email Correspondence 
All email communications with participants will be sent in accordance with Partners’ Send 
Secure email encryption policy.  
 
A.  Study visits and parameters to be measured 
 
Phone screen: See section IV, “Subject Enrollment,” above.  
 
Screening/Assessment Calls and Procedures: The screening visit will take place over the 
phone (or HIPAA-compliant video conference call) and will last approximately 2-3 hours (with 
up to an additional hour on the computer for self-report assessments). Subjects will provide 
electronic informed consent with study staff (e.g., the IE; See Section C, “Procedures for 
obtaining informed consent” for further details). They will then be assessed by the IE for 
eligibility, using the below assessment instruments (see Table 1). Participants will complete self-
report questionnaires online via REDCap. After the Baseline visit, participants will be notified 
by study staff (see appendix) regarding their eligibility status, and they will be given the 
opportunity to discuss this further with our study staff by phone. Reasons for ineligibility will be 
documented, and ineligible participants will also be provided with treatment referrals and 
resources. Eligible subjects will be taught to download the app and will receive a brief tutorial in 
using the app-based treatment. All participants will receive instruction in downloading the Koa 
Health Habits app as well. Please see the schedule of assessments and measures administered at 
each time point in Table 1. These assessments will require approximately 1 hour. These 

 
 
6 
 
assessment lengths are similar to those used in other studies in our program and have been well-
tolerated by participants. To avoid dropout, participants will be paid $25 for mid-treatment, end-
of-treatment, and 3-month follow up assessments.  
 
Downloading the app(s): The IE or RA will assist participants in downloading the app(s) on their 
smartphone. The IE or RA will generate an enrollment code(s) for each study participant linked 
to their unique study identifier on the study server. Participants will download the study app(s) 
from the Apple iOS or Google Play stores. When subjects launch the study app(s) for the first 
time they will be prompted for the enrollment code. Study content will only be available to 
participants after entering their enrollment code. In this way, the app(s) cannot be accessed by 
individuals who have not been enrolled in our trial and given an enrollment code.  
 
Passively Collected Data to be Measured: Data from sensors in participants’ mobile phones 
may also be collected through the Mindset and/or Habits apps, to optimize the program through 
personalization and improvement of the app. Participants will not be required to use the Habits 
app and may opt out of this component. In order for Habits to continue to run in the background 
of participants’ iPhones across the study period, participants must occasionally interact with the 
app once every 7 days, unless location is activated. For Android users, the participants do not 
need to access Habits apart from the first time for data collection to occur.  
 
Treatment Format: Participants will complete app-delivered treatment on their personal 
smartphones, over the course of approximately 8 weeks (see “The Treatment” section below for 
an outline of the modules that the app includes). Forde et al. (2005) report that 6-8 therapy 
sessions is the optimal number of sessions for treating mild to moderate depression. Each 
treatment component will be presented through modules on the mobile app, and exercises will be 
logged and practiced through the app on one’s smartphone. An advantage of app-based CBT for 
MDD is that participants can self-direct the frequency and duration of modules, and they can re-
visit modules as many times as is useful to them.  
 
Additionally, upon initiating the digital CBT treatment, each participant will be assigned a 
therapist with training in CBT and familiarity with MDD; all therapists will be supervised 
weekly by a licensed psychologist (see “Therapist Training and Qualifications,” below). 
Participants will meet virtually with this therapist for a total of 8 treatment sessions to 
supplement and help guide them through the CBT app for MDD. All visits will be no longer than 
16-25 minutes.  During these sessions, the therapist will help the participants set treatment goals, 
evaluate and enhance motivation, answer questions about skills introduced via digital CBT, 
brainstorm ideas for between-session homework, and problem-solve should treatment barriers 
arise. In addition to sessions, participants can communicate with their therapist through a secure 
messaging system incorporated into the app throughout the 8-week treatment. The therapist will 
have a separate portal through the Mindset server to receive and respond to these in-app 
communications from patients. Patients will be notified that the therapist will respond to all in-
app communications within 36 hours on weekdays. During the follow-up phase of the study (i.e., 
after week 8), participants will continue to have access to the app, but without the therapist 
support or messaging system.   
 

 
 
7 
 
Therapist Training and Qualifications: Doctoral-level clinicians (or pre-doctoral interns with 
master’s degrees) will serve as therapists. All study therapists will be trained in the study 
protocol and participate in weekly supervision with a licensed psychologist. Previous findings 
from our BDD pilot trial (#2017P000293) in which only a BA-level “coach” was available to 
participants in addition to the CBT treatment app, suggest that even a light-touch focused on 
motivation and problem-solving is helpful in supporting treatment, with the app itself serving as 
the primary mode of treatment delivery. As such, though the therapists in this trial will have 
more training in CBT and will spend more time speaking one-on-one with participants compared 
to our previous trials, the current trial is still structured with the intention of having the CBT app 
serve as the primary treatment modality. Therapists will receive training prior to starting the trial 
(e.g., completing relevant MGH Psychiatry Academy CBT training course(s), and will be 
required to pass (>90% correct) the corresponding MGH Psychiatry Academy knowledge test to 
demonstrate proficiency). To ensure ongoing high-quality treatment, Dr. Wilhelm, or another 
study staff member who is a licensed clinician with expertise in CBT for MDD and related 
conditions will provide weekly supervision to therapists, with additional supervision as needed if 
questions arise. Further, sessions will be audio recorded for review, as 15% of randomly selected 
audiotaped sessions will be rated for competency and treatment adherence, similar to other NIH 
studies we have conducted.  
 
The Treatment: 
The CBT app includes the following components of CBT for MDD, which the therapist will also 
follow in virtual sessions: 1) education about a CBT model of MDD; 2) cognitive techniques to 
identify and challenge distorted thoughts related to one’s MDD; 3) enhancing values (e.g., 
behavioral activation); 4) mindfulness (to help patients to learn to balance distressing emotional 
states with rational thinking and to control their attentional processes); 5) deeper level (core) 
beliefs; and 6) relapse prevention (to teach patients to expect and react effectively to setbacks 
that may occur during times of stress). Additionally, passively collected sensor data (described 
above) will provide information on changes in participants’ mobility and sleep patterns. Our 
hope is that, in a future stage, the treatment could be adapted to address changes in sleep and 
mobility patterns (e.g., prompting the participant via a message through the app encouraging him 
or her to engage in the treatment when mobility is notably low). 
 
Measures Descriptions 
 
Diagnostic Measures 
Mini International Neuropsychiatric Interview (M.I.N.I.) (Appendix): The M.I.N.I.  
(Sheehan et al., 2006) is a semi-structured diagnostic assessment of DSM-5 psychiatric illnesses. 
The M.I.N.I. is efficient, reliable, and well-validated.  
 
Assessment of Depression and Related Symptoms 
Hamilton Depression Rating Scale (HAM-D) (Appendix): The HAM-D (Hamilton, 1960) 
is a is a clinician-administered scale considered to be the “gold-standard” means of assessing 
symptom severity in depressed patients. There are 21 items in this measure, but the total score is 
the sum of the first 17 items. Items assess common symptoms of MDD including mood, guilt, 
suicide, insomnia, work and activities, agitation, anxiety, and sexual interest.  
 

 
 
8 
 
Center for Epidemiological Studies Depression Scale – Revised (CESD-R) (Appendix): 
The CESD-R (Eaton et al., 2004) is a revised version of the Center for Epidemiological Studies 
Depression Scale (CES-D; Radloff, 1977). It is a 20-item self-report symptom severity measure 
of depression. Patients endorse one of the answer choices (Not at all or less than 1 day, 1-2 days, 
3-4 days, 5-7 days, nearly every day for 2 weeks) for each of the statements that list certain 
behaviors or feelings common to depression (in the past week). For example, items address 
appetite, sleep, weight, suicide, self-harm, attention, happiness, and interest. Participants will 
complete this form at baseline, midpoint, endpoint, and follow-up timepoints. 
 
 
Clinical Global Impression – Improvement Scale (CGI-I) and severity scale (CGI-S) 
(Appendix): The single-item CGI-I, which ranges from 1 (very much improved) to 7 (very much 
worse), is commonly used in clinical trials (Guy, 1976). Participants will complete a CGI-I for 
MDD symptoms (CGI-P MDD) and the clinician will complete a CGI-I for MDD symptoms 
(CGI-MDD) and overall symptoms (CGI-global). The CGI-I ratings will be secondary outcome 
measures and will also be used to determine clinical deterioration of MDD via weekly 
administration through REDCap (link to be sent via email or in-app message) (see Minimizing 
Risks, below). The CGI also has a severity scale (CGI-S) which is rated by the clinician at 
baseline as well as the following assessments. The CGI-S determines the patient’s overall level 
of symptom severity, in comparison to other patients the clinician has treated or assessed with 
the same diagnosis.   
 
Patient Health Questionnaire-9 (PHQ-9) (Appendix): The PHQ-9 (Spitzer, Kroenke, & 
Williams, 1999) is a self-report measure of depression severity includes 9 Likert scale items 
ranging from 0 (not at all) to 3 (every day). The PHQ-9 will be used as an initial screening 
measure to determine symptom severity for inclusionary purposes, as well as at the midpoint, 
post-treatment, and 3-month follow-up assessments. 
 
Patient Health Questionnaire-8 (PHQ-8) (Appendix): The PHQ-8 is a revised version of 
the PHQ-9 less item #9 (thoughts of death and self-harm). Research shows that the PHQ-8 is 
highly correlated with the PHQ-9 ( > 99%) and shows highly similar sensitivity and specificity 
(Wu et al., 2020). It will be re-administered weekly prior to a patient’s appointment with their 
study clinician. REDCap link to be sent via email or in-app message. 
  
Columbia-Suicide Severity Rating Scale (C-SSRS) (Appendix): The C-SSRS (Posner et 
al., 2008) is a gold-standard, clinician-administered assessment of suicidal ideation and suicide 
behaviors. The baseline version, which assesses both lifetime and recent time frames, will be 
used to establish eligibility at the baseline/screening visit. The “Update” version, which assesses 
suicide risk since the last assessment, will be used at subsequent study administrations.  
 
Depression, Anxiety, and Stress Scale (DASS-21): (Appendix). The DASS-21 is a 21-
iitem self-report questionnaire created from a set of three self-report scales with 7 items each. 
The subscale groups include depression, anxiety, and stress scale (Lovibond & Lovibond, 1995). 
 
Assessment of Functioning and Quality of Life 
 

 
 
9 
 
Work and Social Adjustment Scale (WSAS) (Appendix): The WSAS is a 5-item, self-
report measure of impairment in occupational, social, and family domains (Mundt et al., 2002). 
Items are measured on 9-point Likert scales ranging from 0 (no impairment at all) to 8 (very 
severe impairment). 
 
Quality of Life Enjoyment and Satisfaction Questionnaire – Short Form (Q-LES-Q-SF) 
(Appendix). The Q-LES-Q-SF (Endicott, Nee, Harrison, & Blumenthal, 1993) is a 16-item self-
report measure of subjective quality of life. Higher scores correspond with greater ratings of 
quality of life. The Q-LES-Q-SF has strong psychometric properties (Endicott et al., 1993).   
 
Assessment of Expectancy, Motivation and Satisfaction 
 
 
Credibility/expectancy questionnaire (CEQ) (Appendix): This 6-item, self-report, Likert-
type questionnaire assesses patients’ judgments about the credibility of the treatment rationale 
and treatment expectancy (Borkovec & Nau, 1972). It has good reliability (=.81-.86), and 
validity is evident in its ability to differentiate between treatment rationales (Devilly & 
Borkovec, 2000).   
 
 
The Client Satisfaction Questionnaire (CSQ) (Appendix) is an 8 item self-report 
questionnaire which assesses the satisfaction with clinical services received. It has excellent 
internal consistency and good discriminant validity (Attkisson, 2015). 
 
University of Rhode Island Change Assessment Scale - Psychotherapy Version (URICA) 
(Appendix) is a 32-item Likert-type questionnaire that assesses how participants feel about 
starting and engaging in therapy (e.g., “I am doing something about the problems that have been 
bothering me.”; McConnaughy, Prochaska, & Velicer,1983). 
 
Other Assessments 
 
Treatment History Medication, Treatment History Psychosocial, Treatment History 
Checklist forms (Appendix): These forms will be used to assess any medications taken within the 
last 2 months as well as any current or lifetime psychosocial treatment.  
 
Lifetime Psychiatric Medication Form (Appendix): This self-report form collects 
information about all lifetime psychiatric medications taken and will be administered at the 
baseline screening visit. 
 
Concomitant medication and therapy form: (Appendix): This form probes for any 
changes in psychotropic medication and mental health treatments that the participant has made 
since the prior assessment. Any reported changes in therapy or medication are then logged in the 
CTHER or CMED logs. 
 
 
Demographics Form (Appendix): This self-report form collects basic demographic data.  
 

 
 
10 
 
App Feedback Questionnaire (Baseline) (Appendix): This 3-item self-report 
questionnaire asks about participant’s expectations of using the app (e.g., “How frequently do 
you intend to use Mindset?”). 
 
App Feedback Questionnaire (Midpoint) (Appendix): These multiple choice and open-
ended questions ask participants to describe their ongoing perceptions of the app at midpoint 
(e.g., “How easy are the exercises to understand, overall?”).  
 
App Feedback Questionnaire (Endpoint) (Appendix): This self-report form collects 
participant feedback via multiple choice and open-ended questions pertaining to the content, 
usability, perceived usefulness, and aesthetics of the app (e.g., “How clear was the layout of the 
app?”).  
 
Mobile Application Rating Scale user version (uMARS) (Appendix): This self-report form 
collects participant evaluations of mobile health applications. Dimensions include engagement, 
functionality, aesthetics, and information quality.  
  
Treatment Utilization Questionnaire (TUQ) (Appendix): This 3-item self-report form 
measures how much time participants are practicing treatment skills both on and off the app.  
 
Technologist Interview Notes (Appendix): This interview guide asks about 
credibility/expectancy, client satisfaction, and improvements at post-treatment.  
 
Life Events Questionnaire (Appendix): This question will be used to assess whether any 
major life events occurred since the previous assessment that might have had a psychological 
impact on the subject. 
 
Therapist Satisfaction Questionnaire: (Appendix). This self-report form for clinicians 
will assess their satisfaction with using this app treatment approach. 
 
Everyday Discrimination Scale Short Version: (Appendix). This 5-item self-report form 
asks about participants’ experiences with discrimination in their daily life (Sternthal, Slopen, & 
Williams, 2011).  
 
Adverse Event Form and Log (AE Form) (Appendix). This form probs for any major 
events or concerns (i.e., adverse events) since the last time the form was filled out. 
 
Self Help Questionnaire: (Appendix) This self-report form collects information about any 
additional self-help behaviors that could have impacted their well-being or depression since 
beginning treatment.  
 
IE Training and Qualifications, and Procedures to Ensure Assessment Integrity and 
Interrater Reliability: Assessments will be conducted by an IE who has a Master’s or Doctoral 
degree in clinical psychology or related mental health field. The IE will be otherwise uninvolved 
in study procedures. Training and reliability checks will be done to ensure that IEs conduct 
ratings in a uniform way. IEs will first receive instruction in the MINI and HAM-D from a gold-

 
 
11 
 
standard expert rater, prior to beginning as an IE. The IE will be supervised at minimum twice 
monthly. The IE will be required to demonstrate reliability on the HAM-D at a criterion of .80 
ICC, compared to measures rated by a gold-standard rater. All assessments will be audiotaped 
for reliability ratings. To reduce rater drift, a trained reliability rater meeting the same 
qualifications as the IE will review 15% of randomly selected audiotaped interviews. If 
reliability falls below .75, we will institute retraining procedures. 
 
Table 1 Assessment Measures by Occasions for the Open Trial. 
 
Measure 
Baseline 
Baseline 
Re-assess 
Weekly  
 
Mid-
Treatment 
Assess 
(wk 4) 
Post-
Treatment 
Assess 
(wk 8)  
 
3-month 
Follow-up 
Treatment 
History Checklist 
✔️ 
 
 
 
 
 
Treatment 
History 
Medication  
✔️ 
 
 
 
 
 
Treatment 
History 
Psychosocial 
✔️ 
 
 
 
 
 
Life Events 
Question 
 
✔️ 
 
✔️ 
✔️ 
✔️ 
AE Form 
 
✔️ 
 
✔️ 
✔️ 
✔️ 
CONCOM Form 
 
✔️ 
 
✔️ 
✔️ 
✔️ 
MINI 
✔️ 
 
 
 
 
 
HAM-D 
✔️ 
✔️ 
 
✔️ 
✔️ 
✔️ 
C-SSRS Lifetime 
✔️ 
 
 
 
 
 
C-SSRS Update 
 
✔️ 
 
✔️ 
✔️ 
✔️ 
CGI–S 
✔️ 
✔️ 
 
✔️ 
✔️ 
✔️ 
CGI–MDD, CGI-
Global 
 
✔️ 
 
✔️ 
✔️ 
✔️ 
Technologist 
Interview Notes 
 
 
 
 
 
✔️ 
Demographics  
✔️ 
 
 
 
 
 
Lifetime 
Psychiatric 
Medication Form 
✔️ 
 
 
 
 
 
PHQ-9 
✔️ 
✔️ 
 
✔️ 
✔️ 
✔️ 
CGI-P (MDD) 
 
✔️ 
✔️ 
✔️ 
✔️ 
✔️ 
PHQ-8 
 
 
✔️ 
 
 
 
CEQ 
✔️ 
 
 
✔️ 
 
 
URICA 
✔️ 
 
 
 
 
 

 
 
12 
 
CESD-R 
✔️ 
 
 
✔️ 
✔️ 
✔️ 
Q-LES-Q-SF 
✔️ 
 
 
✔️ 
✔️ 
✔️ 
WSAS 
✔️ 
 
 
✔️ 
✔️ 
✔️ 
DASS-21 
✔️ 
 
 
✔️ 
✔️ 
✔️ 
Everyday 
Discrimination 
Scale 
✔️ 
 
 
✔️ 
✔️ 
✔️ 
Self Help 
Questionnaire 
✔️ 
 
 
 
✔️ 
✔️ 
CSQ 
 
 
 
✔️ 
✔️ 
 
Treatment 
Utilization 
Questionnaire  
 
 
 
✔️ 
✔️ 
✔️ 
App Feedback 
Questionnaire 
(Baseline)  
✔️ 
 
 
 
 
 
App Feedback 
Questionnaire 
(Midpoint)  
 
 
 
✔️ 
 
 
App Feedback 
Questionnaire 
(Endpoint)  
 
 
 
 
✔️ 
 
uMARS 
 
 
 
 
✔️ 
 
Therapist 
Satisfaction 
Questionnaire 
 
 
 
 
✔️ 
 
 
If more than ten days elapse between a patient’s initial baseline screening assessment and the 
start of treatment, the study IE would re-administer select forms, including an additional HAM-D 
form. 
Despite our best efforts to retain participants, we do expect dropouts. Except for subjects who 
withdraw consent to participate, all who are withdrawn or drop out of the study will be asked to 
complete all scheduled assessments and we will provide remuneration for participation in 
assessments. 
 
In addition to the assessment measures in Table 2, de-identified data from sensors in 
participants’ mobile phones will also be collected via the CBT and Habits apps, to inform design 
and improvement of the app. 
 
B. Drugs to be used 
Not applicable 
 

 
 
13 
 
C. Devices to be used 
Participants will download the CBT for MDD and Habits apps onto their personal 
smartphone devices.  
 
D.  Procedures, surgical interventions, etc. 
Not applicable 
 
E. Data to be collected and when the data will be collected 
See above for the assessment schedules and assessment batteries for each phase.  
 
VI. BIOSTATISTICAL ANALYSES 
The IE and the RAs will play the primary role in data entry. Data will be entered in REDCap, a 
HIPAA-compliant, Partners-approved platform for electronic data capture that streamlines data 
collection and management, and ensures data integrity, resulting in improved data quality and 
reduced costs. For more information on REDCap, please see section IX, “Monitoring and 
Quality Assurance.”  
Dr. Susanne Hoeppner will be responsible for data management and analysis. All project staff 
will receive training in data management and data confidentiality procedures. Data checks will 
be done regularly to assure that all forms are entered and available for analysis. Questions or 
problems will be resolved promptly by communication between study staff. Data and analysis 
files will be backed up on the lab server and may also be stored in separate locked cabinets.  
 
To characterize our sample, data will be displayed graphically, and summary statistics (e.g., 
means and frequencies) will be calculated for all variables, including demographic and clinical 
descriptors (e.g., from the MINI).  
 
Prior to data analysis, all major variables will be screened for inconsistent or abnormal values.  
Continuous measures will be assessed for skewness and outliers (based on model residuals), and, 
if needed, will be transformed to better meet modeling assumptions of normality and 
homogeneity of variance. A two-tailed p-value <.05 will be considered evidence of statistical 
significance for the primary and secondary outcomes. 
 
A. Specific data variables being collected for the study (e.g., data collection sheets) 
Specific variables being collected, and their timeline are presented in Table 1, and described in 
the Study Procedures section.  
 
B. Study endpoints 
 
Primary endpoints: Feasibility and acceptability of the combined digital CBT plus virtual 
treatment.  
Hypothesis 1a: We hypothesize that digital CBT plus virtual treatment will be feasible 
(based on uptake, engagement and drop-out rates).  
Hypothesis 1b: We hypothesize that digital CBT plus virtual treatment will be deemed 
acceptable both to patients and clinicians (based on descriptive statistics of patient 

 
 
14 
 
expectancy and satisfaction, app use, app feedback, clinician ratings, as well as 
qualitative feedback). 
 
Secondary endpoints: Preliminary efficacy of digital CBT plus virtual treatment for primary 
outcomes (i.e., reduction in severity of depressive symptoms) and secondary outcomes (e.g., 
quality of life at post-treatment, social and occupational functioning at post-treatment).  
Hypothesis 2a: We hypothesize that participants receiving digital CBT plus virtual 
treatment will evidence statistically significant reductions in HAM-D scores from 
baseline to post-treatment (week 8). 
Hypothesis 2b: We hypothesize that participants receiving digital CBT plus virtual 
treatment will evidence statistically significant improvement on secondary clinical 
outcome measures (i.e., functional impairment (WSAS) and quality of life (Q-LES-Q)) 
from baseline to post-treatment (week 8). 
 
C. Statistical methods 
 
Primary endpoints: feasibility and acceptability of the combined digital CBT plus virtual 
treatment. 
Analysis: 
We will examine feasibility (H1a) and acceptability (H1b) by reporting: 
1. Refusal and dropout rates and reasons 
2. Patient satisfaction, as measured by the Client Satisfaction Questionnaire 
3. Patient feedback, as measured by Mobile Application Rating Scale user version  
4. Patients’ credibility and expectancy ratings for the open-label treatment, as measured 
by the Credibility/Expectancy Questionnaire 
5. Treatment utilization, as measured by the Treatment Utilization Questionnaire 
 
Secondary endpoints: preliminary efficacy of digital CBT plus virtual treatment for reducing 
severity of depression symptoms, and improving quality of life and functional impairment 
Analysis: 
Intention-to-treat mixed model analyses with repeated measures (baseline, mid-treatment, 
post-treatment) will be used to examine differences between pre- and post-treatment 
depression (HAM-D scores), using a two-tailed -level of .05. The pre- to post-treatment 
effect size will be calculated. Similar analyses will also be conducted for change in 
functional impairment (WSAS) and quality of life (Q-LES-Q) over the course of 
treatment. We will not adjust for multiple testing among secondary outcomes based on 
the recommendation of Cook and Farewell (1996), who argue that multiplicity 
adjustments are not necessary if separate test results are interpreted marginally and 
address different aspects of the patient experience and decision-making process rather 
than alternative assessments of efficacy. 
 
D. Power analysis 
 
With n=28, we will have > 80% power to detect pre- to post-treatment effect sizes of d ≥ 1.37 
(very large effect sizes).  The estimate of the detectable effect size is based on a single degree of 
freedom contrast in a paired means test, implemented in SAS for Windows version 9.4.  The 

 
 
15 
 
power model used a two-sided alpha=0.05, a power of 0.80, assumed a pre- to-post-treatment 
correlation of 0.18, and a doubling of the standard deviation from pre- to post-treatment.  The 
pre- to post-treatment correlation estimate was based on the mean pooled correlation between 
pre-test and post-test HAM-D scores in 14 CBT trials for adult depression (Cuijpers et al., 2016); 
with higher pre-post correlations, smaller effect sizes would be detectable in other secondary 
outcomes.  The anticipated increase in outcome standard deviation was based on observations 
reported in treatment studies with participants with MDD, which reported pre- to post-treatment 
standard deviation increases in observed HAM-D scores in the range of 161-191% (Jarrett et al., 
1999; Elkin et al., 1989; Goldberger et al., 2011). We further assumed no drop-out by the end of 
treatment, similar to what we observed in pilot studies for similar app-treatment for body 
dysmorphic disorder (Wilhelm et al., 2020); if drop-out was 20%, we would only be able to 
detect effect sizes of d ≥ 1.55.  For comparison, in a meta-analysis of randomized controlled 
treatment trials that included internet CBT (iCBT) for persons with MDD (Andrews et al., 2018), 
the observed median percentage of participants randomized who finished the treatment course 
was 66% (50/64 trials) and the interquartile range was 29% (Q1 52%, Q3 80%).  Large pre-to 
post-treatment effect sizes for CBT treatments among patients with MDD are to be expected, 
based on a meta-analysis by Cuijpers and colleagues (2016), which found that the mean pre-post 
standardized mean difference (i.e., effect size) in 14 CBT trials was 1.79, 95% CI: [1.37, 2.21].   
 
VII. RISKS AND DISCOMFORTS 
 
A. Complications of surgical and non-surgical procedures 
Not applicable 
 
B. Drug side effects and toxicities 
Not applicable 
 
C. Device complications/malfunctions 
If the app or supporting therapist dashboard malfunctions or does not work for a period of 
time, the patient may be unable to use the app to receive treatment or communicate with their 
therapist.  Similarly, if the therapist dashboard were inaccessible, the therapist would not be able 
to communicate via the app with the patient.  These risks could result in minor harms to users 
such as inconvenience or a delay in treatment. To mitigate risks of a temporary device 
malfunction, participants are given the study staff and investigators’ contact information in the 
consent document. Thus, the clinical and study staff can communicate with the participant by 
phone call instead of through the app. As with in-person therapy, there are times when a therapy 
session gets cancelled and treatment, as well as weekly symptom assessments, are thus 
temporarily delayed. Likewise, in face-to-face therapy, a therapist may be temporarily out of 
reach at times (e.g., on vacation) and backup lines of communication (e.g., pager coverage by a 
colleague) are put in place in case of emergencies.  Thus, device malfunctions that lead to 
temporary delays in accessing the app functionality are similar to risks associated with temporary 
delays in traditional face-to-face therapy. See also, “Minimizing of Risks and Safety Reporting” 
under “D. Psychosocial (non-medical) risks” for further description of protections in place.  

 
 
16 
 
 
D. Psychosocial (non-medical) risks 
Participants may feel uncomfortable due to the sensitive nature of the questions they may be 
asked. Likewise, some participants may feel uncomfortable about having assessment sessions 
digitally recorded and reviewed by project staff (which is necessary for rater supervision as well 
as assessment of the reliability of ratings adherence and competence). Participants could 
experience an increase in symptoms related to the natural waxing and waning of MDD 
symptoms. MDD can be associated with other psychiatric symptoms (e.g., anxiety), as well as 
suicidal thoughts and behaviors, which may also change over time. Breach of confidentiality, 
which great care will be taken to prevent, represents a potential risk. As discussed below, we will 
take precautions to ensure that these potential risks are minimized (see Adequacy of Protection 
Against Risks below). 
 
Privacy and Confidentiality 
 
All information gathered will be kept strictly confidential. We will adhere to the following 
procedures to protect privacy and confidentiality: 
1) Participants will be assigned a code number. A link between ID number and participant’s 
names will be kept in a separate secure password-protected file, saved on our secure MGH lab 
server and/or Partners Dropbox Business. Participants’ names or other identifying information 
will not appear on any questionnaires, study documents, digital recordings, computerized data 
files, or published reports. Case records will be reviewed only by study personnel or, if 
necessary, by institutional or sponsor-assigned regulatory personnel. Research assistants and 
others working on this study (e.g., technologists) will be educated about the importance of 
strictly protecting participants' rights to confidentiality. 
2) All personnel will be trained in research confidentiality procedures and HIPAA, including 
completion of CITI training and Healthstream training. Only the study personnel (or, if 
necessary, institutional or sponsor-assigned regulatory personnel) will have access to the 
identifiable data.  
 
3) Clinical data collected during MGH assessments: Computerized data and digital audio files 
will be stored de-identified, in password protected files saved on the protected lab server. Data 
within Partners is stored automatically and securely on an MS SQL Server, accessed over 
industry standard SSL 128 bit RSA encryption during data transfers.  Data is routinely backed up 
locally onto a redundancy server and stored in a separate database that is locked with 256 AES 
encryption. Long term storage on Partners servers occurs nightly and allows for incremental 
backup over multiple systems. Therefore, should one drive be physically damaged, there will be 
multiples within the chain to replace it. Both data servers are stored within PHS IS corporate 
firewall, in a secure, key access facility with password protected computers. Only trained PHS 
security officials will have access to physical machines storing study data.  Since data are stored 
on a protected server, a compromise of any individual computer at a research facility will not 
lead to a breach of the secure database. Hard copy data (paper forms) will be stored securely in 
locked file cabinets, within locked offices in the MGH Center for OCD and Related Disorders.  
 

 
 
17 
 
4) Technologist data: Computerized data and digital audio files collected from feedback 
interviews with technologists will be stored de-identified, in password protected files in the 
Partners Healthcare Dropbox Business folder. The PHS Dropbox Business folder will be set up 
by a listed owner with a Partners email address. Per Partners Research Information Services and 
Computing, “The enterprise rollout of Dropbox Business at Partners HealthCare is an approved 
storage and collaboration solution. This version of Dropbox Business provides unlimited storage, 
fully encrypted data (AES-256 encrypted) and is compliant with Partners’ policies and 
procedures. Dropbox Business allows you to sync, share, and manage your files online” 
(https://rc.partners.org/kb/article/2285).  
 
5) Mobile data:  All mobile data will be transmitted from the participant’s mobile device to the 
secure Mindset server (managed by Koa Health via public cloud service) via an encrypted 
internet connection. The collected data will be linked to the study identifier only. The 
administrative interface to the clinical study server (channels 1 and 7 below) will be password 
protected with access limited to study staff. 
 
6) Habits: The Habits app and platform will be used to collect passive smartphone data. Habits is 
a secure, HIPPA-compliant research platform developed by Koa Health. Habits includes (a) a 
database inside the app container to store de-identified data, with nothing in shared storage, (b) a 
participant-facing app, and (c) a data analysis pipeline that pre-processes raw mobile data into 
daily aggregations. The current version of Habits has passed an internal security audit, and the 
data uploaded to the Koa data lake is stored in the same way as events collected from the 
Mindset app. The data is restricted to be accessed only by the researchers involved in the study 
and all the processing is done within the restricted access, HIPAA-compliant Amazon Web 
Services (AWS) server. In addition to the assessment measures in Table 2, de-identified data 
from sensors in participants’ mobile phones will also be collected via the Habits app, to inform 
design and improvement of the app. Key features are summarized below: 
• The participant does not need to register as a user. The participant will be provided a 
generated code with a unique study prefix and a non-identifiable string of numbers and 
letters after. 
• Participants can only download and use the study’s Habits app if given proper instruction 
from the study investigator (e.g., once fully consented and determined to be eligible). 
• All data are stored with the de-identified user ID. No identifying data such as name or 
contact information are tied to participants’ data. The only identifier that will be inputted 
into habits are dates.  
• The app caches information locally (temporarily or if there is no connectivity). Any data 
stored locally (cache or otherwise) is stored encrypted in a database inside the app 
container. 
• The app will not integrate with other apps on the phone. The app will only integrate with 
Google Fit, which is not an app. 
• The app updates automatically as long as the default option of auto update is enabled. 
• All data stored on the phone are sent to the servicer over an encrypted TLS connection to 
ensure the security of the data transmitted. The data is received by the server and stored 
in an AWS S3, an industry-standard secure storage platform housed in data centers that 
are protected by armed guards. 

 
 
18 
 
• All data connections to the web service hosting the study are negotiated on industry-
standard SSL/TLS connections, removing the vulnerability of man-in-the-middle attacks 
or packet-sniffing data leaks. 
 
6) All virtual visits (i.e., assessments with the IE and telehealth sessions with the therapist) will 
be conducted via a Partners IS-approved, secure, and HIPAA compliant system for video calls 
(e.g., Zoom). 
 
7) Only de-identified data will be shared with the sponsor (Koa Health), under the terms of the 
Data Use Agreement (i.e., Statement of Work).  
 
See Figure 1 and 2 in “Independent Monitoring of Source Data” for a detailed diagram and 
description of the secure data flow and storage. 
 
Minimizing of Risks and Safety Reporting. 
 
The following procedures will be implemented to protect participants against risks. 
1. Participants with active suicidal ideation at the screening assessment will be excluded 
from participating (see Inclusion/Exclusion criteria). Suicidal ideation at screening is 
captured with the PHQ-9 #9 and the C-SSRS (see Inclusion/Exclusion criteria); if a 
potential participant scores ≥2 on the C-SSRS, a study clinician, PI, or independent 
evaluator will conduct a risk evaluation with the participant and refer the participant to a 
higher level of care if clinically indicated.  
 
2. The therapist will assess risk weekly during virtual appointments and refer the 
participant to a higher level of care if clinically indicated, consistent with routine clinical 
care. If a patient unexpectedly does not attend their weekly session, the clinician, PI, or 
the independent evaluator will follow up with a phone evaluation within 24 hours and 
refer the participant to a higher level of care if clinically indicated.  
 
3. A disclaimer that is accessible from the home page of the digital CBT program will be 
presented. The disclaimer will alert participants that “If you are having thoughts of 
suicide or death, please note that this program is not the right treatment for you. You 
should seek professional help without delay. If you feel unsafe, call 911 or go to your 
nearest emergency room.” Links to 911 and suicide hotline numbers will be provided 
along with this disclaimer.  
 
4. A general resources page will be available on the app at all times to participants, which 
will include a suicide hotline number.  
 
5. Participants’ clinical improvement or deterioration will be assessed weekly via a 
participant-rated CGI-MDD collected before each therapy session. Participants may be 
withdrawn from the study for a significantly deteriorating clinical course, as indicated by 
a score of 6 (much worse) or 7 (very much worse) on the weekly, participant-rated CGI-
MDD, across 3 subsequent weeks and/or PI judgement that remaining in the study is not 
in the participant’s best interest. Of note, study clinicians will review weekly CGI scores 

 
 
19 
 
and follow-up in session as appropriate. The clinician and/or PI will refer the participant 
to a higher level of care if clinically indicated.  
 
6. Participants may also be withdrawn if, in the judgment of the PI, remaining in the study 
poses a substantial risk to the participant or a higher level of care is needed.  
 
7. Treatment through app-based CBT will be supplemented with virtual visits with a 
therapist (8 sessions over the course of 8 weeks), who can answer questions and guide 
participants through the treatment as needed. These therapists will be doctoral level 
clinicians, or pre-doctoral interns supervised by doctoral-level clinicians, who are experts 
in CBT for MDD and related disorders.  
 
8. The independent evaluator(s) will be highly experienced, highly trained, and closely 
supervised. 
 
9. Dr. Wilhelm will be available, if necessary, to discuss the study, alternative treatments, 
or any concerns about the study with participants if requested by the participant, 
therapist, or rater. 
 
10. Drs. Wilhelm, Greenberg, Bernstein, and the participants’ study clinician will be 
available to answer study questions via the app or phone. This will be clearly 
communicated orally and in writing to study participants.  
 
11. All participants who fail to respond to treatment or withdraw prematurely will be 
provided with referral resources. 
 
12. The study therapists and raters will make every attempt to help participants feel 
comfortable when discussing sensitive material. Participants may skip questions on 
assessments that they are uncomfortable answering. 
 
13. Technologists from Koa Health who conduct the technology feedback interviews are 
highly trained professional staff with experience conducting patient interviews and user-
centered design. They have received additional training from our team (MGH Center for 
OCD and Related Disorders) on CBT, MDD, and issues of confidentiality.  They have 
completed CITI training.   
 
14. The subject will designate a relative or friend who could be contacted should the subject 
be unavailable, and the investigator has concerns about the subject’s well-being. 
 
 
We anticipate that the above procedures will be effective in protecting study participants against 
potential risks. 
 
Adverse event reporting:  
See below: “Adverse event reporting guidelines”  
 

 
 
20 
 
E. Radiation risks 
Not applicable 
 
VIII. POTENTIAL BENEFITS 
 
A. Potential benefits to participating individuals 
Participants may benefit from the comprehensive diagnostic assessment with a clinician assessor.  
Participants may benefit from careful clinical monitoring, sessions with their therapist, and 
experiencing some relief from their MDD symptoms through the CBT app.  
 
B. Potential benefits to society 
If the digital service combining app-based CBT for MDD with virtual visits with a therapist is 
effective, it may offer increased, cost-effective access to CBT for MDD, a treatment that is 
empirically supported and otherwise difficult to access.  
 
IX. MONITORING AND QUALITY ASSURANCE 
 
Independent monitoring of source data 
The PI will have overall responsibility for study data and participant safety. Please see “Privacy 
and Confidentiality” above for more information about the data collected for the present study 
including (1) Clinical data (formal assessments with the MGH IE, which includes self-report and 
audio recorded clinician-administered assessments completed on paper and via REDCap) (2) 
Mobile data (self-report questions collected electronically within the mobile app, and passively 
collected data through sensors in participants’ mobile phones). All aspects of the study will be 
conducted in accordance with the hospital’s policy on confidentiality.  
 
Paper research records will be kept de-identified, in a locked file in a locked office at MGH. 
Self-report measures and some clinician administered measures will be collected using REDCap. 
REDCap (Research Electronic Data Capture) is a free, secure, HIPAA compliant web-based 
application hosted by the Partners HealthCare Research Computing, Enterprise Research 
Infrastructure & Services (ERIS) group. Data collection projects rely on a study-specific data 
dictionary built by members of the research team. Once built, participants and study staff can 
enter data directly into REDCap surveys via any computer or tablet with standard web access 
and browsers. To ensure confidentiality, data will be identified in the database solely by subject 
number, mapped patient initials (whereby each initial is replaced with the next letter in the 
alphabet), week number, and visit date (i.e., subjects' names will not be entered into the 
database). By identifying study records in this manner, the information can be considered ‘de-
identified’ and therefore compliant with the Standards for Privacy of Individually Identifiable 
Health Information (“Privacy Rule”) of the Health Insurance Portability Act of 1996 (HIPAA). 
Any data that is transmitted electronically will be encrypted and password protected. IEs will 
enter their clinical assessment data directly into REDCap, except for one form (i.e., MINI) that 
will first be collected as paper forms or as electronic records in PDF format. The electronic form 
of the MINI assessments will be completed by IEs using the Notability software on iPads, where 
the assessment data is entered using a stylus and the PDF record plus data is saved as a new PDF 
file. The completed PDF-copy of the MINI forms are then uploaded into the study-specific 
Partners Healthcare Dropbox Business folder designated for this purpose. One study staff 

 
 
21 
 
member (either the IE or a designated, trained RA) will then enter the data from the paper or 
PDF forms into the designated REDCap forms within 3-5 business days of the completion of the 
assessment visit, and another staff member (trained RA or undergraduate intern) will enter the 
same forms into double data entry records for data entry comparisons. If discrepancies are noted, 
the trained staff responsible for data entry will check the source documentation and correct the 
REDCap form as necessary.  Data entry by participants for self-report measures in this electronic 
data capture system precludes the need for subsequent data entry by staff, thus minimizing 
human error, and resulting in improved data integrity and quality. Patients will enter survey 
responses into an electronic assessment form on subject-facing REDCap, and the responses will 
then be transmitted and stored in a secured and confidential database. Each participant will only 
have access to his or her own survey, but not the survey data. All users will have defined roles 
and privileges pre-determined by the system administrator.  Thus, the PI can determine the level 
of access for each study staff such that only a limited number or people have access to sensitive 
study data. To protect patient privacy, when completing REDCap questionnaires, participants 
will be asked to enter their random codes in place of any identifying information (e.g., name, 
birth date). All identifying information will be stored separately from data in a password-
protected file. Together with the data collected on paper forms or PDF records (i.e., MINI), 
Mindset app, and the Habits app (see sections “Privacy and Confidentiality” above for details), 
the data entered by IEs and RAs into REDCap will be considered the source documentation. 
 
Figure 1. Data flow and storage chart. “Mindset” refers to the CBT for MDD mobile application. 
 
1. Researchers have accounts on the Mindset Server (i.e., the CBT app) that allow them to 
create participant accounts (random IDs) that they then give to study participants. 
2. Participants install mobile app on their mobile devices, then enter the activation code 
provided by MGH staff (e.g., clinician, therapist, RA) and grant access to receive push 
notifications. 
3. The application collects and sends data such as: 
a. progress across the exercises, 
b. information about feature usage (analytics), 
c. answers to questionnaires. 

 
 
22 
 
4. N/A (function that will not be utilized in trial) 
5. N/A (function that will not be utilized in trial) 
6. Researchers collect additional information on subjects using REDCap 
7. Researchers access the data collected via the mobile app, check user progress, 
communicates with the patient through integrated messaging. 
8. Researchers pull data from REDCap 
9. The server sends Push Notifications to notify the user when:  
a. The therapist has sent a message 
b. Data sent to an External Notification Service includes: 
i. The registration token identifying which device should receive 
notification 
ii. Textual content of the notification (title and body) 
10. Application reaches out to the External Notification Service to:  
a. Register for Push Notifications. When registering the app sends a uniquely 
generated identifier that identifies the app installation. 
11. All the info from potential app crashes are stored in mindset servers. Retrieved 
information includes:  
a. Device vendor 
b. Device model 
c. OS version 
d. Error details (stack trace) 
 
Figure 2. data flow and storage chart. “Habits” refers to the passive data collection mobile 
application. 
 
1. Researchers have accounts on the Habits Server that allow them to create participant 
identifier (random IDs) that they then give to study participants. 
2. Participants install Habits mobile app on their mobile devices, then enter the participant 
identifier provided by research staff and grant access to receive notifications. 
3. The application collects and sends data such as: 
a. sensing data coming from smartphone sensors (such as number of steps) 

 
 
23 
 
b. information about feature usage (analytics), 
c. answers to questionnaires. 
4. Researchers access the data collected via the mobile app on the Koa Servers and check 
user data. 
5. (Optional) The user allows the app to send Notifications to notify the user when it is time 
to fill a weekly questionnaire. These are local notifications set up within the app which 
only include and send to the user:  textual content of the notification (title and body) 
6. (Optional) Habits sets Local notifications logic on when to send the notifications to users 
(e.g., every 2 weeks to fill a specific questionnaire) 
 
We anticipate that the above procedures will ensure the confidentiality and integrity of study 
data.  
 
C. Safety monitoring 
The PI will have overall responsibility for monitoring the integrity of study data and participant 
safety. Procedures for managing participant safety, including the monitoring of participants 
throughout the trial and response to clinical deterioration (as defined above) should it occur, are 
detailed above in “Minimizing of Risks and Safety Reporting.”  
 
D. Outcomes monitoring 
Adverse events and data completeness will be monitored as described above.  
 
E. Adverse event reporting guidelines 
 
Adverse event reporting: Adverse events will be reported per PHRC guidelines.  
 
X. REFERENCES 
Andersson, G., Cuijpers, P., Carlbring, P., Riper, H., & Hedman, E. (2014). Guided Internet‐
based vs. face‐to‐face cognitive behavior therapy for psychiatric and somatic disorders: a 
systematic review and meta‐analysis. World Psychiatry, 13, 288-295. 
Borkovec, T. D,. & Nau, S. D. (1972). Credibility for analogue therapy rationales. J Behav Ther 
Exp Psychiatry, 3, 257-260. 
Cook, R., & Farewell, V. (1996). Multiplicity Considerations in the Design and Analysis of 
Clinical Trials. Journal of the Royal Statistical Society. Series A (Statistics in Society), 
159(1), 93-110. doi:10.2307/2983471 
Cuijpers, P., Berking, M., Andersson, G., Quigley, L., Kleiboer, A., & Dobson, K. S. (2013). A 
meta-analysis of cognitive-behavioural therapy for adult depression, alone and in 
comparison with other treatments. The Canadian Journal of Psychiatry, 58, 376-385. 
DeRubeis, R.J., Webb, C.A., Tang, T.Z., & Beck, A.T. Cognitive therapy. In: Dobson KS, ed. 
Handbook of cognitive-behavioral therapy. 3rd ed. New York: Guilford, 2010. 
Devilly, G. J., & Borkovec, T. D. (2000). Psychometric properties of the credibility/expectancy 
questionnaire. J Behav Ther Exp Psychiatry, 31, 47-55. 
Endicott, J., Nee, J., Harrison, W., & Blumenthal, R. (1993). Quality of Life Enjoyment and 
Satisfaction Questionnaire: A new measure. Psychopharmacology Bulletin, 29, 321-326.  
Ferrari, A.J., Charlson, F.J., Norman, R.E., Patten, S.B., Freedman, G., Murray, C.J., et al. 
(2013). Burden of Depressive Disorders by Country, Sex, Age, and Year: Findings from 

 
 
24 
 
the Global Burden of Disease Study 2010. PLoS Med 10: e1001547. 
https://doi.org/10.1371/journal.pmed.1001547 
Goldberger, C., Guelfi, J. D., & Sheehan, D. V. (2011). Assessment of Anxiety in Clinical Trials 
with Depressed Patients Using the Hamilton Depression Rating Scale. 
Psychopharmacology bulletin, 44(3), 34–50. 
Guy, W. (1976). Clinical Global Impressions. In W. Guy (Ed.), ECDEU Assessment Manual for 
Psychopharmacology (rev. ed.; DHEW publication [ADM] 76-338; pp. 217-222).  
Rockville, MD: U.S. Department of Health, Education, and Welfare, National Institute of 
Mental Health. 
Hamilton, M. (1960). A rating scale for depression. Journal of Neurology, Neurosurgery, and 
Psychiatry, 23, 56-62  
Hedman, E., Ljótsson, B., & Lindefors, N. (2012). Cognitive behavior therapy via the Internet: a 
systematic review of applications, clinical efficacy and cost–effectiveness. Expert review 
of pharmacoeconomics & outcomes research, 12, 745-764. 
Kroenke, K. & Spitzer, R.L. (2002). The PHQ-9: A new depression and diagnostic severity 
measure. Psychiatric Annals, 32, 509-521. 
Lovibond, P. F., & Lovibond, S. H. (1995). The structure of negative emotional states: 
Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression 
and Anxiety Inventories. Behaviour research and therapy, 33(3), 335-343. 
McConnaughy, E.N., Prochaska, J.O., & Velicer, W.F. (1983). Stages of change in 
psychotherapy: Measurement and sample profiles. Psychotherapy: Theory, Research and 
Practice, 20, 368-375.  
McMurtry, S. L. & Hudson, W. W. (2000). The Client Satisfaction Inventory: Results of an  
initial validation study. Research on Social Work Practice, 10, 644-663. 
Mojtabai, R. (2005). Trends in contacts with mental health professionals and cost barriers to 
mental health care among adults with significant psychological distress in the United 
States : 1997–2002. American Journal of Public Health, 95, 2009–2014. 
Mundt, J., Marks, I., Shear, M., & Greist, J. (2002). The Work and Social Adjustment Scale: A 
simple measure of impairment in functioning. British Journal Of Psychiatry, 180, 461-
464. 
Posner, K., Brent, D., Lucas, C., Gould, M., Stanley, B., Brown, G., ... & Mann, J. (2008). 
Columbia-suicide severity rating scale (C-SSRS). New York, NY: Columbia University 
Medical Center. 
Rush, A. J., Trivedi, M. H., Ibrahim, H. I., Carmody, T. J., Arnow, B., Klein, D. N., Markowitz, 
J. C., Ninan, P. T., Kornstein, S. G., Manber, R., Thase, M. E., Kocsis, J. H. & Keller, M. 
B. (2003). The 16-item Quick Inventory of Depressive Symptomatology (QIDS) 
Clinician Rating (QIDS-C) and Self-Report (QIDS-SR): Psychometric evaluation in 
patients with chronic major depression. 
Biological Psychiatry, 54, 573–583. 
Sareen J, Jagdeo A, Cox BJ, Clara I, ten Have M, Belik SL, de Graaf R, Stein MB (2007). 
Perceived barriers to mental health service utilization in the United States, Ontario, and 
the Netherlands. Psychiatric Services 58, 357–364. 
Sheehan, D. V., Lecrubier, Y., Sheehan, K. H., Amorim, P., Janavs, J., Weiller, E. ... Dunbar, G.  
C. (1998). The Mini-International Neuropsychiatric Interview (M.I.N.I): The  
development and validation of a structured diagnostic psychiatric interview for DSM-IV  
and ICD-10. Journal of Clinical Psychiatry, 59, 22-33. 

 
 
25 
 
Smith, A. (2017). Record shares of americans now own smartphones, have home broadband. 
Pew Research Center. url: http://www.pewresearch.org/fact-tank/2017/01/12/evolution-
of-technology/ 
Sternthal, M. J., Slopen, N., & Williams, D. R. (2011). RACIAL DISPARITIES IN HEALTH: 
How Much Does Stress Really Matter? Du Bois review: social science research on race, 
8(1), 95. 
van Voorhees BW, Fogel J, Houston TK, Cooper LA, Wang NY, Ford DE (2006). Attitudes and 
illness factors associated with low perceived need for depression treatment among young 
adults. Social Psychiatry and Psychiatric Epidemiology 41, 746–754. 
Webb, C. A., Rosso, I. M., & Rauch, S. L. (2017). Internet-based cognitive-behavioral therapy 
for depression: current progress and future directions. Harvard review of psychiatry, 25, 
114-122. 
Wilhelm S, Weingarden H, Greenberg JL, McCoy TH, Ladis I, Summers BJ, Matic A, Harrison 
O.  Development and pilot testing of a cognitive-behavioral therapy digital service for 
body dysmorphic disorder.  Behavior Therapy, Jan.  2020, 51(1):15-26.  Doi: 
10.1016/j.beth.2019.03.007 
Wu, Y., Levis, B., Riehm, K. E., Saadat, N., Levis, A. W., Azar, M., ... & Thombs, B. D. (2020). 
Equivalency of the diagnostic accuracy of the PHQ-8 and PHQ-9: a systematic review 
and individual participant data meta-analysis. Psychological medicine, 50(8), 1368-1380.