Does Adding a Tailored Cognitive Behavioral Therapy (CBT) Mobile Skills App Mediate Higher Rates of Depression Recovery, Adjustment, and Quality of Life in OEF/OIF Veterans Compared to Standard CBT?
Phase: NA
Conditions: Depression
Sponsor: VA Office of Research and Development

Does Adding a Tailored Cognitive Behavioral Therapy (CBT) Mobile Skills App mediate 
Higher Rates of Depression Recovery, Adjustment, and Quality of Life in OEF/OIF 
Veterans Compared to Standard CBT? 
NCT 04002063 
PI: Judith A Callan PhD, RN 
'HFHPEHU
Study Protocol 

Recruitment: In addition to the referrals and self-referrals made from the above locations, 
eligible patients will be identified by reviewing CPRS records. These chart reviews include 
patients receiving active clinical care and meet our inclusion criteria i.e., has depressive 
symptoms but are not engaged in therapy.  
For patients that we have reviewed through CPRS via a waiver and who received an 
encrypted email from the Veteran’s identified therapist of notification and Veteran permission 
to call, we will make our first contact by phone. For all other recruitment methods, we will send 
a letter requesting that they contact us by phone if they so desire. Patients who notify the 
research staff by phone of their interest, will then be contacted by the research staff about 
coming in for an evaluation and considering signing the informed consent and the HIPAA 
authorization. During the initial phone call, following a thorough explanation of the study to the 
Veteran who has agreed to be contacted, verbal informed consent for a short eligibility survey 
will be obtained. The eligibility survey will contain the PHQ-9 and Scale of Suicidal Ideation 
(SOSI) on the phone to establish preliminary eligibility. IRB approval will be attained to perform 
eligibility ratings prior to written consent. 
If preliminarily eligible, Veterans will be invited to come for an in-person visit to sign the 
informed consent and the HIPAA authorization and complete the full baseline assessment in 
person. A Suicide Risk Assessment Protocol will be used for telephone interviews.  See Human 
Subjects.  Specific elements of the protocol included further evaluation if Question # 9 of the 
PHQ-9 is “several days, more than half the days or nearly every day” or if the Scale of Suicidal 
Ideation items two or four (wish to die or desire to make an active suicide attempt are moderate 
to strong).  Specifically, the evaluator must determine how serious the threat is.  If the risk is 
moderate or high, i.e., there is a moderate to high risk of potential harm to the participant, the 
participant will be placed on hold briefly, and the Veterans Crisis Line will be called and the 
participant will be transferred to the Veterans Crisis Line Responder. The phone evaluator will 
be  prepared to tell the emergency responder the participant’s name, location (i.e., physical 
address), phone number, last 4 of SSN, and relevant clinical information (e.g., suicide risk level, 
who the participant is talking with about these thoughts).  The evaluator will document this in 
CPRS. 
In addition to the specific suicide protocol we have added, we will: 1) exclude all Veterans 
who are listed as high suicide risk in CPRS; 2) will train study personnel about methods for 
handling endorsement of suicide risk during phone screens; 3) clinicians will be available at all 
times to manage such situations (this will also include the PI, Dr. Haas, and Dr. Thase); and 4)  
the VA has on-site suicide prevention coordinators who will manage a transfer from study staff 
for endorsement of suicide risk.  The standard of practice at the VA includes a phone evaluation 
for outpatient services.  Our initial research evaluation offers no additional risk than this 
standard evaluation.  Finally, the literature supports the feasibility and safety of telephone 
screening of depressed individuals with potential suicidal ideation5,6, 78.  
Informed Consent Procedure: Participants will sign an informed consent prior to initiation of 
the study baseline study procedure. Initial assessments will cover key demographic, diagnostic, 
clinical, psychopathologic assessments (See Table 8).  Informed consent will be obtained during 
an in-person interview with the subject and it will be documented on a VAPHS or VA 
Philadelphia consent form.  
Assessments will be completed via paper and pencil. Once participants complete all T1 
assessments, they will be randomly assigned to either 1) CBT-D as usual or 2) CBT-D+. The CBT 
MobileWork-V app documentation will be provided and downloaded on their phone. In addition 
to an in-person demonstration, we are developing a YouTube video detailing the use of and clear 
demonstrations about how the app works, its contents, how information is entered, etc.  
We have identified a cadre of five CBT certified therapists at each of the Pittsburgh and 
Philadelphia VA’s who are willing to participate with us and have training in VA CBT-D. All 
therapists will receive comprehensive training on the protocol procedures and a protocol 

manual.  All of the therapists will receive an in-person training on the app as well as a manual 
with every screen shot present in the app. They will identify the most appropriate individualized 
skills practice at the end of each therapy session. 
 
We will facilitate the patient’s appointment with the study therapist at the time of 
randomization. All patient participants will receive reimbursement for their participation 
($30 for T1, $35 for T2, and $50 for T3). 
Description of Usual Care and Experimental Conditions 
Usual Care Condition: After completion of the baseline assessment, study participants will have 
an appointment set up to meet with one of the VA-trained/certified CBT therapists who will 
explain how the CBT program will be carried out. CBT-D includes twelve, once weekly, 50-
minute sessions (per the VA-specified manual) over a period of up to 20 weeks. There will be a 
leeway of 8 weeks to complete all sessions (to allow for vacations, illness, etc.). All subjects will 
have the same therapist throughout. All therapists will complete an Assignment Compliance 
Rating Scale65 following each session, referencing the patient’s adherence to last session’s HW 
and additional skills practice.   
Experimental Condition: Participants in the experimental condition will have identical therapy 
procedures as the CBT-D as usual condition (see 3.6.1).  They will have access to CBT 
MobileWork-V throughout the entire study and be instructed to use their study ID as their 
username and will be given a numeric password.  Thus, there will be no identification of the 
participant through PHI.  We will also instruct them to avoid personal identification of 
themselves or their family members in the smartphone app.   
For those that are randomized to the app, we will guide them to a YouTube video detailing 
clear demonstrations about how the app works, its contents, how information is entered, etc.  A 
user manual will be generated and individualized training on all aspects of CBT will occur for 
patient and therapists who access or need to assign HW based upon contents of the app 
CBTMobileWork-V. They will be instructed to use CBT MobileWork-V as the primary method of 
CBT skills practice.  
When the patient opens CBT MobileWork-V, he/she will be prompted to complete a VAS 
rating from “I feel the worst I have ever felt” to “I feel the best I have ever felt.” 74  This 
procedure will be repeated when they close the app.  The patient will identify the location of the 
HW and locate it in the pull-down menu. The smartphone application will have consecutive 
screens that ask the patient to comment on specific areas, i.e. if the patient is doing a 
dysfunctional thought record it will begin by asking them to describe the situation and why it 
was upsetting.  Thus, the HW assignments are not merely forms, but series of questions that 
ascertain the crucial HW responses.  Patients can then save it in the CBT Toolbox for future 
reference. 
If, during the course of the study therapy, a patient is found to have symptoms that “cannot 
wait to be treated,” therapists will intervene, within the scope of safe therapeutic treatment, e.g. 
an emergency contact will be set up by phone or in person (and per the intervention manual).  
Patients with high risk or severe symptoms will be instructed to report symptoms to their 
therapist in a timely manner. 
All assignments cannot be directly completed on the smartphone application.  These 
assignments will be documented on the Assignment Compliance Rating Scale (ACRS) for both 
the Experimental and Treatment as Usual conditions. Especially in the beginning of therapy, 
when behavioral activation is required, HW may include graded physical activity.  The 
smartphone application may record the activity, evaluate how the activity affected their mood, 
or to get ideas for physical activity from My CBT Library.   
 
 
 
Statistical Analysis  

Analysis Strategy: This study is a two-group, two-site randomized controlled trial.  Primary 
effects of interest are greater CBT understanding and skills acquisition (Primary Aim 1) and 
lower levels of depression symptoms (Primary Aims 2a and 2b) in the CBT-D+ group compared 
to CBT-D alone. Secondary effects are lower comorbid diagnosis symptoms (i.e., lower 
psychological adjustment and reintegration problems, greater functional capacity, and quality of 
life, and lower levels of anger, PTSD, and anxiety (see table 8). 
 
Independent variables: Randomized treatment group assignment (CBT-D+ versus CBT-D 
alone). 
 
Dependent Variables: The primary outcomes on which the analysis is powered are CBT 
undertanding and skills acquisition assessed by patient and therapist SoCT and depressive 
symptoms assessed by PHQ-9. 
 
Sample Size Justification: The proposed sample size of 268 randomized (134 per group) was 
estimated based on the expected effect sizes to test hypotheses for the primary aims 
(Hypotheses 1 and 2[a,b]); adjustment of the testwise significance level to .01 for CBT 
understanding and skill acquisition (as measured by patient and therapist SoCT total scores) 
(Primary Aim 1) and depressive symptoms (measured by PHQ-9) (Primary Aims 2a and 2b) to 
limit inflation of type 1 error due multiple testing of hypotheses at 12-weeks post-therapy 
initiation and 6-months post-therapy; and anticipated attrition at 6-months post-therapy7 
recommend clinically meaningful effect sizes of 0.50 (in terms of a standardized mean 
difference, d) when comparing alternative treatments in comparative psychotherapy research. 
When using the patient and therapist SoCT total scores to measure patient’s CBT understanding 
and skill acquisition, standard deviations (SD) of ~0.72 and ~0.80 have been reported post-
therapy by patients and therapists, respectively93.  Given these results, clinically meaningful 
effects of at least d=0.50 (or at least 0.36-point and 0.40-point differences in treatment group 
means assuming SDs of ~0.72 and ~0.80 for SoCT total scores from patients and therapists, 
respectively) may be detected with 0.80 power with 192 total (96 per group) when comparing 
mean SoCT total scores from patients and therapists between treatment groups based on linear 
contrasts from linear mixed model at testwise significance level of .01 (Hypothesis 1). 
 
Regarding Primary Aims 2a and 2b, standard deviations on the order of ~4.0 have been 
reported for PHQ-9 total scores post-therapy8. With minimum group sizes of 96 (192 total), we 
would have .80 power to detect clinically meaningful differences as small as d=0.50 (or at least a 
2.0-point difference in treatment group means assuming a SD of 4.00) when comparing mean 
depressive symptoms (as measured the PHQ-9 total scores) between treatment groups at 12-
weeks post-therapy initiation (Hypothesis 2a) and at 6-months post-therapy (Hypothesis 2b) at 
testwise level of significance of .01. 
 
Although efforts will be made to retain participants through 6-months post-therapy, some 
attrition may occur. Research in CBT for depressive symptoms and the use of technology/apps 
to enhance treatment effects support at most 20% attrition at 12-weeks post-therapy initiation9 
and an additional 8% attrition at 6-months post-therapy10 for an overall attrition rate of 28% at 
6-months post-therapy. To ensure an adequate number of participants with full follow-up, we 
plan to enroll 134 per group (268 total), to conservatively adjust for overall attrition of 28% at 6-
months post-therapy. 
Data Analysis: Data will be analyzed using SAS94 to conduct exploratory data analyses, 
missing data estimation, and repeated measures modeling for Primary Aims 1, 2a and 2b and 
Secondary Aim 2. MPlus95 will be used for mediational analyses of Secondary Aim 1. While 
directional hypotheses have been posited, the level of statistical significance will be set 
conservatively at .01 to limit type 1 error inflation due to hypothesis testing at multiple outcomes 
and at multiple time points. Confidence intervals for point estimates will be computed at 99%. 
Preliminary Analyses: Exploratory analyses will first be performed involving data 
description and screening for anomalies (e.g., outliers, nonnormality, missing data) with these 
results being used to 1) describe univariate/bivariate data distributions; 2) identify treatment 

group imbalances and associations between dependent variables and suspected 
covariates/confounders; 3) evaluate the amount and patterns of missing data; and 4) check for 
violations of assumptions for the planned analysis strategies. If assumptions are violated, data 
transformations or more statistically robust procedures will be considered as appropriate. 
Covariates/confounders will be included in analyses secondarily and their impact on the results 
will be evaluated. The randomness of missing data will be investigated using available 
information on participant characteristics to help discern patterns in the missing data, identify 
possible missing data mechanisms, and inform the strategies used to handle missing data. If 
data are missing completely at random or missing at random, the likelihood based estimation 
procedures to be used will produce unbiased estimates while allowing us to retain observations 
with missing values on the dependent variables. If needed, multiple imputation (e.g., Markov 
Chain Monte Carlo with the number of imputations based on the extent of missing data) will be 
used to impute missing values on covariates. If nonrandom missingness is suspected, we will use 
selection or pattern mixture modeling to explore the sensitivity of results to assumed missing 
data patterns. 
Analysis Plan for Primary Aim 1: An "intent-to-treat" (ITT) approach will first be used for 
efficacy analyses of CBT-D+ compared to CBT-D alone. All participants will be analyzed “as 
randomized”, regardless of protocol adherence, treatment received, withdrawal or protocol 
deviations. Although recommended for efficacy analyses in RCTs, the sensitivity of the results 
assuming ITT will also be explored using data collected on the participant’s adherence to their 
assigned treatment group using an instrumental variables approach 96as well as ad hoc 
approaches (e.g., per protocol, amount of treatment received). 
The primary outcome variable for this aim is CBT understanding and skill acquisition (as 
measured by the patient and therapist SoCT total scores) assessed at baseline prior to 
randomization, completion of the acute phase of treatment (12-weeks post-therapy initiation), 
and 6-months post-treatment. Linear mixed modeling will be used to examine the overall 
efficacy of CBT-D+ on patient’s CBT understanding and skill acquisition from both the patient’s 
and therapist’s perspectives over time. When fitting models to each of these dependent 
variables, time will be treated as fixed, categorical, repeated within-subject factor, while 
randomized treatment assignment (CBT-D+ vs. CBT-D alone) will be a fixed, categorical, 
between-subject factor, with interactions between time and randomized treatment assignment. 
In addition to the baseline value of the SoCT total scores, time-invariant  and/or time-
dependent covariates may be included to adjust for treatment group imbalances or variables 
related to the dependent variables of interest. In keeping with linear mixed modeling, random 
effects for participants will be included as well as possible random site effects. Standard fit 
indices will be used to identify the best-fitting variance-covariance structure for the repeated 
measures. We will also consider time as a continuous variable through the use of mixed effect 
regression models to accommodate individual deviations from the average trends. F-tests will be 
used to test the main effects and interactions included in the model. For each model, residual 
analyses will be used to identify possible model misspecification, outliers and influential cases. 
Sensitivity analyses will be performed to discern the impact of influential observations on 
results. To test hypotheses for Primary Aim 1 (Hypothesis 1) by formulating linear combinations 
of the model parameters from the fitted model, group comparisons between CBT-D+ and CBT-D 
alone will be performed at a) 12-weeks post-therapy initiation and b) 6-months post-therapy. 
Effect size indices with confidence intervals will be reported for the overall effect of the addition 
of CBT MobileWork V to CBT-D and at each time point of interest. Lastly, we will also explore 
possible modification of the treatment effect by participant characteristics (e.g., sex, age, etc.). 
Analysis Plan for Aims 2a and 2b: For the most part, we will use a similar repeated measures 
approach as outlined for Aim 1 to fit linear mixed models for the primary distal outcome of 
depressive symptoms at baseline prior to randomization, 12-weeks post-therapy initiation, and 
6-months post-treatment. Linear mixed modeling will be used to examine the overall efficacy of 

CBT-D+ relative to CBT-D alone on patient’s depressive symptoms (as measured by total scores 
on the PHQ-9) over time. To test aim-specific hypotheses and estimate the corresponding effect 
sizes, group comparisons between treatment groups will be performed at a) 12-weeks post-
therapy initiation (Hypothesis 2a) and b) 6-months post-therapy (Hypothesis 2b). To examine 
the durability of the effect of CBT-D+, we will also examine changes in depressive symptoms 
from a) 12-weeks post-therapy initiation to 6-months post-therapy and b) baseline to 6-months 
post-therapy. 
Analysis Plan for Secondary Aim 1: We will explore CBT skills practice (as measured by the 
Assignment Compliance Rating Scale and the number of homework assignments completed) 
and acquisition of additional CBT skills (as measured by the SoCT P & T) as intermediate 
outcomes that may mediate the effect of the addition of CBT MobileWork-V to CBT-D on the 
distal outcome of depressive symptoms. Initially, we will fit simple mediational applying the 
change score method and estimate whether the effect of the addition of CBT MobileWork-V 
(predictor) on changes in depressive symptoms (outcome) is mediated through changes in the 
suspected mediators of CBT skills practice and acquisition of additional CBT skills. In addition 
to the chi-square goodness of fit statistic, standard summary fit indices (e.g., root mean square 
error of approximation, comparative fit index) will be used to assess the fit of the mediational 
model to the data, followed by model assessment via residual analyses. Total, indirect and direct 
effects will be estimated and reported with confidence intervals. If nonnormality is encountered, 
robust estimation methods (e.g., robust maximum likelihood estimation) will be be used, or 
resampling approaches with the percentile bootstrap will be applied. Depending on the results 
from simple mediational modeling, we may also combine mediators and incorporate additional 
potential mediating/moderating variables (e.g., the number of CBT therapy sessions completed) 
into a multiple-mediator model to develop a more comprehensive picture of the possible 
pathways. Additionally, we will explore CBT skills practice and acquisition of additional CBT 
skills as mediational processes on depressive symptoms as an outcome process using parallel 
process latent growth curve modeling, where the mediational process is modeled by relating the 
randomized treatment group assignment (CBT-D+ vs. CBT-D) and the latent growth factors of 
the two parallel processes. 
Analysis Plan for Secondary Aim 2: Using a similar analysis approach as described for Aim 1, 
we will use linear mixed modeling (or generalized linear mixed modeling, if dependent variables 
are binary with binomial errors) to explore the effect of CBT-D+ compared to CBT-D alone on 
the more secondary outcomes of OEF/OIF comorbid disorders/issues, including work and home 
functioning (assessed by the Military to Civilian Questionnaire and Disability Assessment 
Schedule), quality of life (assessed by the WHO-QOL), anxiety (assessed by the GAD-7), anger 
(assessed by the Dimensions of Anger Reactions), and lastly PTSD (based on the PTSD 
Checklist) following 12 weeks of weekly CBT-D sessions and at 6-months post-therapy. For these 
explorations, the estimation of effect sizes (with confidence intervals) will be emphasized to 
summarize short- and long-term effects of CBT-D+, compared to CBT-D, on work and home 
functioning, quality of life, anxiety, anger and PTSD at 12-weeks post-therapy initiation and 6-
months post-therapy, respectively.