A Pilot Study Aimed at Developing Automated Motivational Messaging for Patient Engagement With Digital Mental Health Interventions
Phase: NA
Conditions: Depression, Anxiety
Sponsor: Brigham and Women's Hospital

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Toward Understanding Drivers of Patient Engagement With Digital  
Mental Health Interventions – Part I 
NCT05360901 
Date: 9/28/2023 

 
Version 2021.06.10 
Page 1 of 8 
 
 
 
Institutional Review Board 
Intervention/Interaction Detailed Protocol 
 
 
Principal Investigator: 
Jessica M Lipschitz, PhD  
 
Project Title: 
Toward Optimizing Digital Mental Health Interventions: A Clinical Trial 
Aimed at Understanding What Drives Patient Engagement – Part 1  
 
Version Date: 
National Institute of Mental Health   
 
Version Name/Number: 
9/28/23 
 
 
 
1. 
 Background and Significance 
 
Digital mental health interventions (DMHIs), delivered via the internet and/or mobile apps, have 
the potential to remove many of the barriers that plague in-person behavioral health interventions. 
They can be widely disseminated at minimal marginal cost, do not require transportation or 
daytime appointments, and promote patient autonomy. Reviews and meta-analyses suggest that 
DMHIs are effective in controlled research settings.1-3 But, in routine care implementations, 
effectiveness is limited by insufficient engagement.4,5 Only 20-50% of patients engage at the 
recommended frequency and/or complete the full course of treatment.4,6,7 The strongest 
hypothesis to date is that achieving reliable engagement with DMHIs requires adding human 
support.8,9 But strategies for adding human support are widely varied—from less scalable options 
like weekly phone calls with a clinician to more scalable options such as asynchronous 
communication with a health coach—and little is known about the impact of the more scalable 
strategies. A less explored, even more scalable strategy for improving engagement is automated 
motivational messaging (AMM). While AMM has been effective in some trials,10,11 little is known 
about what content or delivery characteristics drive its impact. Understanding whether fully 
automatized intervention components can improve engagement without human support, and what 
drives patient engagement in either condition will be essential. This study applies a patient-
centered approach to developing the look, feel and written content for a set of AMMs that could 
be used to enhance patient engagement with DMHIs.  
 
2. 
Specific Aims and Objectives  
 
This study aims to systematically employ patient feedback to develop a set of automated 
motivational messages aimed at enhancing engagement with digital mental health interventions. 
 
This study is a precursor to a subsequent full-scale clinical trial that will test the impact of 
digitally delivering these messages on patient engagement with a digital mental health 
intervention.  
 
 
 

Mass General Brigham Institutional Review Board 
Intervention/Interaction Detailed Protocol 
 
Version 2021.06.10 
 
Page 2 of 8 
3. 
General Description of Study Design 
 
This is an in situ user-centered design study for the automated motivational push messaging we 
are developing. Users will download an example DMHI (IntelliCare) use it for three weeks. In 
tandem, they will receive automated motivational push messages from the set that we have in 
development on a randomized schedule. That is, each day they will be randomized to either 
receive an automated motivational push message (60% likelihood) or not receive an automated 
motivational message (40% likelihood). On days when they receive a push message, they will 
then be promoted to complete a brief survey on their reaction to the push message. They will 
also be asked to complete weekly symptom ratings. At the end of three weeks, they will be 
asked to complete a qualitative interview to provide further feedback on messages under 
development.  
 
The primary outcome in this study is user ratings from the brief surveys provided after push 
notification delivery. We will also, however, look at user feedback on messages under 
development in the qualitative interview as well as whether receiving a message makes users 
more likely to engage with the DMHI (IntelliCare) during the three hours following message 
delivery. 
 
4. 
Subject Selection 
 
We will enroll up to 40 patients with clinically significant anxiety and/or depression. Inclusion 
criteria will be: (i) Age 18+; (ii) Patient Health Questionare-8 (PHQ-8) score ≥10 and/or 
Generalized Anxiety Disorder-7 (GAD-7) score ≥8; (iii) owns a smartphone; (iv) fluent in English; 
and (v) have a MGB primary care provider. Exclusion criteria will include: (i) diagnosis of bipolar 
disorder, any psychotic disorder, or a current substance use disorder per patient’s report or 
medical record; (ii) acute and/or unstable medical problem that may interfere with participation 
(e.g., scheduled for surgery in the next two months). 
 
Thus, enrollment will be open to all English-speaking, adult, MGB primary care patients with 
clinically significant depression and/or anxiety and no current or past serious mental illness or 
acute medical illness that may interfere with participation. Potential subjects who do not speak 
English will be excluded from the study due to the fact that not all of the validated assessments 
used are available in languages aside from English. Additional eligibility criteria will be added as 
needed after recruitment of N=10 participants to ensure adequate representation of racial 
minorities and men (who are historically underrepresented in studies on depression and anxiety 
disorders). 
 
Subjects will be recruited from the Mass General Brigham Clinical Trials website/Rally and 
RPDR patients who have not opted out of Research Invitations. We will send IRB-approved 
recruitment letters by mail and Patient Gateway messages. We will also recruit from ongoing 
studies across the MGB system being conducted by colleagues with IRB approval to offer 
current participants opportunities to participate in other ongoing research. An IRB-approved 
telephone script will be utilized by the study research assistant to pre-screen subjects prior to 
scheduling. 
 
 
 

Mass General Brigham Institutional Review Board 
Intervention/Interaction Detailed Protocol 
 
Version 2021.06.10 
 
Page 3 of 8 
5. 
Subject Enrollment 
 
An IRB-approved telephone script will be utilized to pre-screen subjects prior to scheduling. 
 
Once a potential subject has been pre-screened, the study research assistant will review the 
consent form verbally with eligible subjects. Eligible subjects will be given an opportunity to ask 
questions by phone. Potential subjects are not required to sign the consent on-the-spot, they 
are able to carefully read the document and discuss the research with their family, friends 
and/or physician and develop questions to ask the research staff. Once they have read the 
consent document and their questions are answered by a qualified member of the study staff, if 
they agree to participate in the research, they will be provided a copy of the consent form via 
REDCap to sign.  
 
Consented subjects will be scheduled for a study enrollment call via phone or Zoom based on 
the participants’ preference. During this call, participants will be guided through the download 
process for the study intervention app (Intellicare) and the push notification delivery/survey app 
(MyDataHelps and/or Catalyst by MetricWire) on their personal smartphone. Participants will be 
randomized each day over the next three weeks to either receive a push notification from the 
pool of notifications being tested (60% probability) or not (40% probability). Daily randomization 
will be done using the R package randomizeR. 
 
6. 
Study Procedures 
 
User-centered, participatory design consists of iterative qualitative feedback meetings with 
patients (i.e., participants). Up to 40 participants will be recruited for this study. After screening 
(by phone), eligible participants will be invited to participate in the study. After consent, but prior 
to enrollment, study staff will check diagnoses in participants’ EPIC chart to ensure that 
participants do not meet exclusion criteria. 
 
The first step in study participation will be for participants to complete an enrollment call and an 
enrollment survey in REDCap. The enrollment survey will include a demographic questionnaire, 
the Technology Adoption Propensity Scale, symptom surveys and a brief enrollment survey on 
messaging preferences. The enrollment call will be conducted via phone or Zoom (per the 
participant’s preference) with study staff. The call will entail walking users through downloading 
the study intervention app (IntelliCare) and the push notification delivery/survey app 
(MyDataHelps and/or Catalyst by MetricWire) on their personal smartphone. Participants will be 
told that IntelliCare is most effective when used on a daily basis and be asked to respond to 
surveys attached to push notifications by tapping on the push notification. Participants will then 
be randomized each day to either receive a push notification from the pool of notifications being 
tested (60% probability) or no notification (40% probability). On days when a push notification is 
received, they will have the opportunity to respond to a brief survey in the MyDataHelps and/or 
Catalyst by MetricWire app on their reactions to the notification. Participants will earn $2 for each 
push notification survey they complete. At the end of week 1 and week 2, participants will be 
prompted to complete a depression severity self-report measure (PHQ-8) and an anxiety severity 
self-report measure (GAD-7) and will earn $5 each week these are completed. 
 
Three weeks later, participants will be invited to engage in a study visit, either in lab or remotely 
via Zoom, based on current COVID-19 prevention guidelines and the participant’s preference. 
Study visits will last up to 90 minutes and will involve meeting with one or two members of the 
study team.  

Mass General Brigham Institutional Review Board 
Intervention/Interaction Detailed Protocol 
 
Version 2021.06.10 
 
Page 4 of 8 
 
In the study visit, participants will be given the opportunity to ask any questions they have about 
digital mental health interventions and/or the goal of the automated motivational messaging being 
developed. They will then be asked to complete a brief self-report questionnaire including the 
PHQ-8, GAD-7 and a repeat administration of the enrollment survey (to evaluate whether answers 
have changed). The remainder of the meeting will consist of a qualitative interview focused on 
generating subject feedback on design and content considerations in development of automated 
motivational messages. Subjects will be asked to offer additional feedback related to messages 
received over the preceding 3 weeks and be presented with low-fidelity prototypes of additional 
messages not delivered in the preceding 3 weeks with open-ended questioning used to generate 
feedback.  
 
Participants will be paid $50 for participation in the study visit. All meetings with participants will 
be audio recorded and recordings will be immediately transferred onto Partners secure servers 
and deleted from the recording device. All recording devices will be stored in the PI’s lab in a 
locked file cabinet. 
 
The primary outcome in this study is user ratings on message likeability and message 
motivation from the brief surveys provided after push notification delivery. We will also, however, 
look at user feedback on messages under development in the qualitative interview as well as 
whether receiving a message makes users more likely to engage with the DMHI (IntelliCare) 
during the three hours following message delivery. 
 
The intervention offered as part of this study is only intended to facilitate user-centered design 
and will be presented as such. It will not be considered part of the participant’s treatment plan 
nor will it be presented as an alternative to other treatment. Neither participation in this study nor 
engagement with the intervention app prohibit participants from engaging with other treatments 
for depression and/or anxiety. This study does not involve ongoing assessment of clinical 
symptoms because this is not an outcome consistent with study aims. If at any point, a 
participant expresses that participation in the study may be adversely impacting his or her 
mental health, study staff will make it clear to the participant that he or she can choose to 
withdraw at any time.  Participants can be removed from the study at any time at their own 
request.  
7. 
Risks and Discomforts 
 
Foreseeable risks that might exist are described in the following four categories: (a) risks 
associated with the intervention; (b) risks associated with study visits; (c) risks associated with 
potential loss of confidentiality; and (d) risks of worsening mental or emotional state.  
(a)Risks associated with the intervention: Technology-enabled mental health intervention 
programs have not been shown to cause any harm. The clinical risks of the intervention used in 
this study are very low. The digital mental health intervention being used in this study 
(IntelliCare) has been studied previously with no serious adverse events and potential for 
clinical benefit. The IntelliCare suite is already in public use. Participants may continue with or 
begin any other treatments they choose for anxiety and/or depression while they are engaging 
in the app-based intervention.  
 
(b)Risks associated with study visits: Subjects may experience fatigue from the study visits, and 
potential discomfort from the nature of questions. Some of the questions asked in the qualitative 

Mass General Brigham Institutional Review Board 
Intervention/Interaction Detailed Protocol 
 
Version 2021.06.10 
 
Page 5 of 8 
interview may be unsettling, but most will not be.  Subjects may refuse to answer any questions 
that they do not want to answer. Additionally, subjects are allowed to take breaks throughout the 
study visit at their request.  Risks involving the compromise of privacy and confidentiality during 
study visits will be minimized through the utilization of the PI’s laboratory space, where there are 
private rooms for study visits and infrastructure in place to ensure the proper adherence to IRB-
approved protocol such as the use of password protected computers and locked storage space. 
 
(c) Risks associated with potential loss of confidentiality. As with all studies, potential loss of 
confidentiality is also a risk. Risks in this study are minimized by using secure online data 
collection tools –REDCap, the MyDataHelps and Catalyst by MetricWire are HIPAA compliant. 
The IntelliCare app can be used anonymously with participants labeled by a study ID number 
only. Additionally, questionnaire data collected has been limited to only those measures 
essential for addressing primary aims of this project. Still, there is some possibility that others 
may see the app open on the participant’s smartphone. There is also the possibility that 
databases may be hacked. Confidentiality may be broken by research staff to ensure the 
patient’s safety if there is an imminent threat to self or others. All of these potential losses of 
confidentiality will be disclosed in the consent documents.  
 
(d)Risks of worsening mental or emotional state: It is not believed that the risk of these 
depressive, anxious, suicidal, or other adverse outcomes are increased as a function of being 
enrolled in this study. However, due to the common course of depression and anxiety, it is 
expected that there will be exacerbations in symptoms of some participants over the course of 
the study. Subjects may pursue all treatment services they otherwise would alongside 
participating in this study.  
While there is no point in which we are assessing current suicidality, several procedures are in 
place to protect participants who convey suicidality in the course of their participation. Language 
in participants’ free-form responses to questions in the intervention app (IntelliCare) are 
continuously monitored by the IntelliCare system. If any words related to death, suicidality or 
means of suicidality (e.g., gun, pills) are mentioned, the participant will receive an automated 
message indicating the following “IntelliCare is NOT an emergency service. If you do not feel 
safe and need immediate police or medical assistance, call 9-1-1 or go directly to your nearest 
emergency room.” Additionally, the study PI (or covering clinician) would receive an alert and 
call the participant. Additionally, if at any time, a subject communicates any imminent safety risk, 
they will be assessed by the PI (a licensed psychologist) or a covering clinician and referred for 
immediate care if it is clinically determined. 
8. 
Benefits 
 
There may be no immediate benefits to you from participation in this research study; however, 
results may allow researchers to design effective ways to better engage patients with digital 
mental health interventions. Subjects engaged in this type of participatory design research, 
sometimes report a sense of accomplishment from participation.  
 
9. 
Statistical Analysis 
 
The primary aims of this study are qualitative and descriptive consistent with user-centered 
design standards. Descriptive data will be collected on the following: average likability ratings for 
messages sent, average motivation ratings for messages sent, percentage of the time users 

Mass General Brigham Institutional Review Board 
Intervention/Interaction Detailed Protocol 
 
Version 2021.06.10 
 
Page 6 of 8 
engaged with the intervention app (IntelliCare) within three hours of receiving a message. Daily 
randomization of message delivery is being applied to mirror message frequency delivery to 
what we intend to use in the clinical trial not to infer causality.  
 
Data from qualitative interviews will be analyzed using a rapid qualitative analysis approach.12 
Final decisions on messaging will be made in consultation with my mentorship team with the 
goal of producing 10-15 messages in each “bucket” (i.e., targeting attitude, self-efficacy and 
habit strength respectively) to avoid delivering redundant messages in the clinical trial.  
 
10.   Monitoring and Quality Assurance 
 
As the proposed study involves no more than minimal risk to subjects, there is no formal Data 
Safety Monitoring Board established for this study. The PI will review safety data in real time to 
address any concerning adverse events and discuss these with her mentor, Dr. Burdick. Dr. 
Burdick has served as an active member of three IRBs over the past 15 years and will ensure 
that all AEs are appropriately reported and followed up. 
 
The principal investigator will oversee the data collection and will ensure that any adverse 
events are recorded and properly reported. We do not anticipate frequent problems given that 
this is a minimal risk protocol; however, we will closely monitor data collected in the event of any 
unanticipated problems. In the event of any unanticipated problems and adverse events we will 
report to the PHRC: 1) during the conduct of the study, 2) after study completion, or 3) after 
subject withdrawal or completion. Reports are to be submitted within 5 working days/7 calendar 
days of the date the investigator first becomes aware of the problem, as outlined in Partners 
System-wide policy. 
 
Self-report data will be collected in REDCap and the HIPAA-compliant MyDataHelps and/or 
Catalyst by MetricWire app. Additionally, de-identified participant engagement data will be 
collected automatically via the IntelliCare app. Assurance of data validity and integrity will be 
monitored closely by the PI. Similarly, audio recordings of subject interviews will be uploaded 
onto the research drive (and deleted from the recording device) the day the interview is 
conducted. The principal investigator will work closely with the rest of the study team to ensure 
the study is in adherence to the IRB-approved protocol. 
 
11.   Privacy and Confidentiality 
 
☒ Study procedures will be conducted in a private setting 
☒ Only data and/or specimens necessary for the conduct of the study will be collected 
☒ Data collected (paper and/or electronic) will be maintained in a secure location with appropriate 
protections such as password protection, encryption, physical security measures (locked 
files/areas) 
☐ Specimens collected will be maintained in a secure location with appropriate protections (e.g. 
locked storage spaces, laboratory areas) 
☒ Data and specimens will only be shared with individuals who are members of the IRB-approved 
research team or approved for sharing as described in this IRB protocol 
☒  Data and/or specimens requiring transportation from one location or electronic space to 
another will be transported only in a secure manner (e.g. encrypted files, password protection, 
using chain-of-custody procedures, etc.) 

Mass General Brigham Institutional Review Board 
Intervention/Interaction Detailed Protocol 
 
Version 2021.06.10 
 
Page 7 of 8 
☒   All electronic communication with participants will comply with Mass General Brigham secure 
communication policies 
☒ Identifiers will be coded or removed as soon as feasible and access to files linking identifiers 
with coded data or specimens will be limited to the minimal necessary members of the research 
team required to conduct the research 
☒ All staff are trained on and will follow the Mass General Brigham policies and procedures for 
maintaining appropriate confidentiality of research data and specimens 
☒ The PI will ensure that all staff implement and follow any Research Information Service Office 
(RISO) requirements for this research 
☒ Additional privacy and/or confidentiality protections 
 
 
Risks in this study are minimized by using secure online data collection tools – REDCap, the 
MyDataHelps and Catalyst by MetricWire apps are HIPAA compliant. The IntelliCare app can be 
used anonymously with participants labeled by a study ID number only. Additionally, 
questionnaire data collected has been limited to only those measures essential for addressing 
primary aims of this project. Finally, risks involving the compromise of privacy and confidentiality 
during study visits will be minimized through the utilization of the PI’s laboratory space, where 
there are private rooms for study visits and infrastructure in place to ensure the proper 
adherence to IRB-approved protocol such as the use of password protected computers and 
locked storage space. 
 
12.   References 
 
1. 
Firth J, Torous J, Nicholas J, et al. The efficacy of smartphone‐based mental health interventions 
for depressive symptoms: a meta‐analysis of randomized controlled trials. World Psychiatry. 
2017;16(3):287-298. doi:10.1002/wps.20472 
2. 
Richards D, Richardson T. Computer-based psychological treatments for depression: A 
systematic review and meta-analysis. Clinical Psychology Review. June 2012 2012;32(4):329-342. 
doi:10.1016/j.cpr.2012.02.004 
3. 
Firth J, Torous J, Nicholas J, Carney R, Rosenbaum S, Sarris J. Can smartphone mental health 
interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. Journal of 
Affective Disorders. 2017/08/15/ 2017;218:15-22. doi:https://doi.org/10.1016/j.jad.2017.04.046 
4. 
Hensel JM, Shaw J, Ivers NM, et al. A Web-Based Mental Health Platform for Individuals Seeking 
Specialized Mental Health Care Services: Multicenter Pragmatic Randomized Controlled Trial. J Med 
Internet Res. Jun 2019;21(6):e10838. doi:10.2196/10838 
5. 
Karyotaki E, Riper H, Twisk J, et al. Efficacy of Self-guided Internet-Based Cognitive Behavioral 
Therapy in the Treatment of Depressive Symptoms: A Meta-analysis of Individual Participant Data. JAMA 
Psychiatry. Apr 2017;74(4):351-359. doi:10.1001/jamapsychiatry.2017.0044 
6. 
Gilbody S, Brabyn S, Lovell K, et al. Telephone-supported computerised cognitive-behavioural 
therapy: REEACT-2 large-scale pragmatic randomised controlled trial. The British Journal of Psychiatry. 
May 2017;210(5):362-367. doi:10.1192/bjp.bp.116.192435 
7. 
Cavanagh K, Seccombe N, Lidbetter N. The implementation of computerized cognitive 
behavioural therapies in a service user-led, third sector self help clinic. Behavioral and Cognitive 
Psychotherapy. 2011;39:427-442. doi:10.1017/S1352465810000858 
8. 
Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JE. Persuasive system design does 
matter: a systematic review of adherence to web-based interventions. J Med Internet Res. Nov 
2012;14(6):e152. doi:10.2196/jmir.2104 

Mass General Brigham Institutional Review Board 
Intervention/Interaction Detailed Protocol 
 
Version 2021.06.10 
 
Page 8 of 8 
9. 
Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for 
adult depression: a meta-analysis. Cogn Behav Ther. 2009;38(4):196-205. 
doi:10.1080/16506070903318960 
10. 
Bidargaddi N, Almirall D, Murphy S, et al. To Prompt or Not to Prompt? A Microrandomized Trial 
of Time-Varying Push Notifications to Increase Proximal Engagement With a Mobile Health App. JMIR 
Mhealth Uhealth. Nov 2018;6(11):e10123. doi:10.2196/10123 
11. 
Kelders SM, Bohlmeijer ET, Pots WT, van Gemert-Pijnen JE. Comparing human and automated 
support for depression: Fractional factorial randomized controlled trial. Behav Res Ther. Sep 2015;72:72-
80. doi:10.1016/j.brat.2015.06.014 
12. 
Hamilton AB. Qualitative methods in rapid turn-around health services research. Health services 
research & development cyberseminar. 2013 Dec 11:2023-03.