Facilitating Assessment of At-Risk Sailors Using Technology
Phase: NA
Conditions: Psychological Distress
Sponsor: VA Eastern Colorado Health Care System

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 1 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
NCT 04159480 Protocol 
 
Facilitating Assessment of At-Risk Sailors Using Technology (FAAST) 
 
VA Eastern Colorado Health Care System 
 
6 April 2021 
 

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 2 
 
 
w 
COMIRB Protocol 
COLORADO MULTIPLE INSTITUTIONAL REVIEW BOARD 
CAMPUS BOX F-490    TELEPHONE:  303-724-1055 Fax:  303-724-0990 
 
 
 
Protocol #: 18-2844 
Project Title: Facilitating Assessment of At-Risk Sailors using Technology (FAAST) 
Principal Investigator: Lisa A. Brenner, PhD, ABPP 
Version Date: 4/6/21 (v9) 
 
 
 
 
  
 
I. Hypotheses and Specific Aims   
 
Aim 1. Evaluate the efficacy of the Cogito Companion as an upstream suicide prevention 
intervention.  
Hypothesis 1.1. Participants allocated to the Cogito Companion will report significantly fewer 
episodes of distress and less distress over the course of the study (Outcome Questionnaire-45; 
OQ-45)1 compared to those allocated to the Active Control group.  
Hypothesis 1.2. Among those with episodes of clinically meaningful distress, those allocated to 
the Cogito Companion will more rapidly engage in treatment (medical record review and Client 
Services Receipt Inventory-EU-Revised; CSRI-EU-R)2 to address such symptoms versus those 
allocated to the Active Control group.  
Hypothesis 1.3. Participants allocated to the Cogito Companion will report significantly fewer 
episodes of and less severe depressive (Patient Health Questionnaire-9; PHQ-9)3 and post-
traumatic symptoms (Post Traumatic Stress Disorder Checklist-5; PCL-5),4 and suicide-related 
thoughts (Beck Scale for Suicide Ideation; BSS, Columbia-Suicide Severity Rating Scale 
Screener; C-SSRS Screener), as well as overall increased perceived physical/psychological 
health functioning (Rand 36-Item Health Survey; SF-36)5,6 over the course of the study compared 
to those allocated to the Active Control condition.  
 
Aim 2. Evaluate the acceptability and feasibility of the Cogito Companion among military 
personnel (Sailors).7  
Aim 2.1. Acceptability of Cogito Companion will evaluate based on participant satisfaction (CSQ), 
participant engagement in application-related activities (e.g., leaving voice recordings), and time 
spent on the application.8 Acceptability refers to the suitability of an intervention from the 
perspectives of the participants.  
Aim 2.2. Feasibility of Cogito Companion will be evaluated based on participant retention and 
accrual, and number and type (minor, moderate, severe) of technical challenges noted during 
implementation. Feasibility refers to the ease of implementation.  
 
Exploratory Aim. Identify patterns of distress, depressive symptoms, post-traumatic symptoms, 
suicide-related thoughts, and perceived physical and psychological health functioning over the 
course of the study among the entire sample, as well as within the Cogito Companion and Active 
Control groups. 
 
II. Background and Significance  
 
Research suggests that Naval personnel experience concerning levels of anxiety, stress, 
and suicidal ideation while deployed.9 Moreover, the post-deployment period has been shown to 
be particularly risky in terms of suicidal thoughts and behaviors.10 Given this, novel, non-
stigmatizing, and portable upstream prevention strategies are needed to evaluate behaviors, 
moods, and symptoms over time, particularly during periods of transition. Whereas regular and 
frequent monitoring by health diaries can be unreliable, the development of smartphone 
technology has created accessible means to record, store, display, and monitor (self and other) 
health information.11 In specific, technology developed by the Cogito Corporation (Cogito 

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 3 
 
Companion), and evaluated by members of this research team, has the capability to enable 
military personnel self-care and facilitate early identification of risk for suicide. However, research 
is needed to establish evidence for this type of intervention.  
 
Suicide Risk and Periods of Transition. Research suggests that periods of transition 
are associated with changes in mental health,12-14 including increased risk for suicidal ideation 
and behavior.12,15-19 Additionally, studies have shown that increased risk for death by suicide 
following a transition is often closely tied to psychosocial stressors, including loss of 
employment,19,20 romantic relationships,20,21 onset of illness, or declines in physical health.17,22 
Timing of transitional periods may also be of import.23 For example, Warner and colleagues24 
found three time periods of significantly increased risk for suicidality among United States (US) 
Soldiers over a 15-month deployment cycle. The first period occurred in the second month and 
was hypothesized to be a result of separation from important social relationships. The second 
was after six months in theater, which was hypothesized to be related to feelings of isolation after 
returning from leave. Perhaps most pertinent to this proposed study, the third period was near the 
end of the Soldier’s deployment and thought to be related to exposure to increased stressors on 
the home front. Highlighting the need for increased focus on the period of transition from military 
service to Veteran status, the President recently signed an Executive Order entitled, “Supporting 
Our Veterans During Their Transition from Uniformed Service to Civilian Life”.25 Within the order, 
the Departments of Defense (DoD), and Veterans Affairs (VA), as well as Homeland Security 
were directed to develop a plan to ensure that those separating from service receive necessary 
mental health care. However, current strategies (e.g., Post Deployment Health Assessment) have 
proven to have limited utility in identifying those whose symptoms develop in the weeks and 
months following periods of transition (e.g., redeployment to the continental United States 
[CONUS]).26 As such, exploration of novel strategies (e.g., smartphone based mobile applications 
[app]), is indicated. 
Historically, suicide prevention efforts have largely focused on individuals known to be at 
high risk.27 Although this continues to be an important strategy, efforts are also required “to keep 
[military personnel] from ever reaching the point of contemplating or attempting suicide”, by early 
identification of increased risk and rapid intervention.28 To date, limited research has been 
conducted to evaluate such upstream approaches, particularly among military personnel. 
Identification of evidence-based upstream interventions is necessary to decrease risk among 
military personnel during periods of transition.  
 
Smartphone Technology and the Cogito Companion. Smartphone technology can be 
used to record, store, and monitor (self and other) vital information regarding suicide risk.19 
Specifically, the Cogito Corporation has developed a secure, privacy-compliant mobile app 
(Cogito Companion). The Cogito Companion was designed to collect and interpret social signals 
underlying smartphone behaviors via usage, to non-invasively collect, transfer, integrate, analyze, 
and report objective behavioral measures of psychological distress. Measurements of location, 
call, and text patterns are fed into machine learning algorithms to compute behavioral measures. 
These measurements provide proxies for behavioral and cognitive components of depression, 
mood, and anxiety disorders. Over time, these patterns may provide markers for changes in 
occupational and social functioning.29 For example, when a Service member is functioning well, 
he or she is likely to have a routine level of communication and spend certain hours of the day at 
home versus at work, or out and about. However, with the onset of mental health symptoms, 
he/she is more likely to spend increasing amounts of time at home, and limit communication. 
Adaptive algorithms combine these elements over time to derive behaviorally driven changes in 
these patterns. 
 
In addition, individual users are prompted by the Cogito Companion to complete 
standardized psychological measures (see below) and to record short (a few seconds to 10 
minute) audio diaries. Using vocal analytics, recordings can be transformed into predictive 
metrics of depressed mood. Specifically, the recording is analyzed regarding the manner in which 
people speak, rather than what they say. An analysis of vocal qualities, rather than content, 
allows the technology to predict depressed mood from short field-based audio recordings, while 
remaining objective to topic, demographic, or even language spoken. Participants will be asked to 
enter audio diaries every two weeks, which will be used to produce an additional real-time, 
longitudinal, and clinically validated measure of depressed mood. Data regarding social signals 

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 4 
 
underlying smartphone behaviors, as well as from the measures and audio diaries, are displayed 
in the Clinician Dashboard (see below). 
 
III. Preliminary Studies/Progress Report:   
 
Development of Cogito Algorithms. Serving as a contractor to the Defense Advanced 
Research Program Agency (DARPA), Cogito gathered an extensive dataset of individual trauma 
survivors’ mobile data.29 With the goal of developing mobile phone behavioral health monitoring 
for military personnel, the Cogito team ran a 3-month study of 100 participants. The participants, 
both Veteran and civilian, were enrolled for 12 weeks during which time they each carried a 
mobile phone equipped with a research app. The research app passively monitored, on an 
intermittent fixed schedule, probe readings from sensors hardwired into the current generation 
mobile phone. These sensors included call logs, short message service (SMS) logs, locational 
pattern changes, acceleration, and battery levels. During the study, participants also filled out 
weekly self-report metrics around mood disorder symptomatology, left weekly audio diaries, and 
met with a clinician for a Structured Clinical Interview DSM-IV (SCID-IV)30 at both intake and exit 
dates of the study. Cogito used structured machine learning to create models, which took in, as 
inputs, these raw data streams and predicted the clinical level assessment of symptomatology. 
Each model outputs a score from 0-100, indicating the probability of presence of the behavioral 
component of mental health. The model uses a week of data to create a score, and updates on a 
daily basis. Cogito researchers also met with clinical stakeholders to assess the most useful and 
actionable means of presenting feedback to patients.  
 
Civilian RCT. Among civilians, a 171 person two-arm randomized clinical trial (RCT) was 
conducted to compared use of the Cogito Companion to standard care in an integrated 
behavioral health patient-centered medical home setting.31 In this National Institute of Mental 
Health funded clinical trial, the PHQ-9 was the primary outcome. Results showed a statistically 
significant decrease in this score for patients who received the Companion relative to those who 
did not. While these findings were not focused on military personnel or upstream suicide 
prevention, they bolster the hypothesis that Cogito Companion could have a positive effect on 
improving PHQ-9 outcomes in our study population. 
VA Pilot Study. Members of this research team (Brenner, Betthauser, Stearns-Yoder) 
conducted a pilot study to assess the acceptability and feasibility of Cogito Companion among 
military Veterans. Participation in the study included a baseline visit, using the mobile app for 
three months, and one follow-up visit. Participants were sent surveys every two weeks while 
using the app, and were outreached as needed for clinical concerns or administrative and/or 
technical issues. One hundred and six Veterans were enrolled in the pilot study, and 85 have 
completed the study protocol. Initial results support both the feasibility and acceptability of the 
intervention among Veterans from all cohorts. With an eye towards the population of interest for 
this proposal, the following data is from Veterans who served in Operation Enduring 
Freedom/Operation Iraqi Freedom/Operation New Dawn (OEF/OIF/OND; N=53). The vast 
majority of these individuals reported enjoying using the app with half rating their level of 
enjoyment as “Yes, a lot”, another 43% reporting “Yes, a little”, and only 7% reporting "No, not at 
all". Per the study protocol, acceptability was established apriori as 70% of participants scoring 24 
or greater on the Client Satisfaction Questionnaire (CSQ).8 In this sample, 81% scored above this 
cut off; thereby suggesting overall acceptability. Qualitatively, OEF/OIF/OND Veterans reported 
that they liked the ability to self-monitor their symptoms. For example, one participant noted “it 
allows me freedom to sign in and say what I want when I want or I mean [audio diary], give 
information I want to or need to when I want to”. Another Veteran described, “it just reminded me 
to take a step back, to breathe. To understand there’s always people there that are willing to help 
you and talk to you”. In addition, some participants discussed how specific app functions provided 
motivation to be more social and active, such as, “it [app] reminded me…of the need to get out of 
the house on a daily basis. Not just sit and take the pill-pain pills”. These data suggest that the 
intervention will be acceptable to Active Duty Sailors. 
 
 
 

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 5 
 
IV. Research Methods 
 
A.  Outcome Measure(s) 
Aim 1. Outcome Questionnaire (OQ-45). The OQ-45 is a psychometrically sounds 45-item 
measure designed to evaluate key areas of mental health functioning, including symptom 
distress.32 It is a widely-accepted tool for identifying, tracking, and measuring behavioral health 
treatment outcomes.33  
Client Services Receipt Inventory-EU-Revised (CSRI-EU-R). This is a measure which queries 
regarding use of inpatient hospital, outpatient hospital, community based-day services, as well as 
primary and community care contacts. In addition to validating care documented in electronic 
medical records, it will be will be used to identify care received outside of the MHS.2 In a recent 
systematic review of self-report measures, use of this tool was noted as being a valid method for 
data collection on healthcare utilization.2 
Patient Health Questionnaire-9 (PHQ-9)*. This is a frequently used and psychometrically 
sound self-report measure of depression.3  
 
Posttraumatic Stress Disorder Checklist 5 (PCL-5)*. The PCL-5 is a self-report measure 
used to evaluate post traumatic symptom severity.4 Criteria is based on Diagnostic and Statistical 
Manual of Mental Disorders-5 criteria.5 Items are rated on a 5-point Likert scale.  
 
Beck Scale for Suicidal Ideation (BSS)* is the self-report version of the Beck Scale for 
Suicide Ideation.34 Concurrent validity (comparing patient and psychiatrist ratings) was reported 
as .90 (p < .001) and internal reliability was similarly high (α = .93)34 
 
Rand 36-Item Health Survey is a 36-item psychometrically sound quality of life outcome-
measure that yields perceived physical and mental health functioning summary scores.35 
The Columbia-Suicide Severity Rating Scale (C-SSRS) Screener is a screening 
measure, administered by a clinician or electronically, that assesses for suicidal ideation and 
attempt behavior based on established definitions.36 The C-SSRS screener assesses intensity of 
suicidal ideation, specifically asking about frequency, duration, intrusiveness, controllability, and 
deterrents, as well as suicide-related behavior, such as actual attempts and aborted attempts. 
 
Aim 2. Client Satisfaction Questionnaire (CSQ). The CSQ is an eight-item questionnaire 
with adequate reliability and validity, which has frequently been used to evaluate standard 
community mental health care.8,37 Items are scored on a Likert-type scale where ‘1’ indicates the 
lowest degree of satisfaction and ‘4’ the highest. 
 
General. MSRC Standard Demographic Form. Information will be gathered on topics 
such as participant age, gender, race/ethnicity, education, military service, and combat exposure. 
 
University of Washington Risk Assessment Protocol-Revised (UWRAP). The UWRAP will 
be used during clinical outreach phone calls to facilitate risk identification. The UWRAP has been 
recommended by National Institute of Mental Health and has been used in over 20 years of 
research.38,39  
 
Self-Help Mobile Application Survey. The study team will develop a brief survey to 
identify applications that military personnel may use for mental health and wellness, and may 
include brief items such as the purpose of the app, and estimated amount of time (i.e., minutes) 
they interacted with these mobile apps. Participants will be specifically asked about mHealth 
Tools developed by the National Center for Telehealth & Technology), as well as any additional 
apps (free text field).40 Some examples include: the Virtual Hope Box, Breathe2Relax, and the T2 
Mood Tracker. Those in the experimental arm will complete this survey at the beginning and end 
of the study. Participants allocated to the Active Control arm will respond to these survey 
questions every two weeks.  
       MSRC Common Data Elements (CDEs). The Military Suicide Research Consortium’s 
(MSRCs) questionnaire includes shortened measures of the following: current and past suicide 
risk, lethality and intent of past suicide attempts, hopelessness, thwarted belongingness, anxiety 
sensitivity, posttraumatic stress disorder symptoms, traumatic brain injury, insomnia, and alcohol 
abuse. All studies funded by MSRC are required to include the CDEs in their assessment 
protocol. 
       COVID-19 Questionnaire. The brief self-report asks participants about the impact of the 
COVID-19 pandemic and stressors related to the pandemic.48 Questions are also taken from the 
Fear of Illness and Virus Evaluation (FIVE) form.49 

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 6 
 
Additional Data Sources. Mobile Apps. In addition to data obtained from the measures 
outlined above, data from those allocated to the Cogito Companion group will be captured via the 
mobile app including: the number of phone calls made and received, and the length of those 
calls; the number of text messages sent and received; how frequently the mobile app is used; and 
which features of the app the participant chooses to view.  
For those in the Active Control arm, MyCAP data will be captured regarding app usage 
and information accessed (e.g., links to wellness apps) using the Self-Help Mobile Application 
Survey developed by the study team (see description above, as well as measures pushed 
regarding their mental health. Measures demarcated with an * above will be sent every two 
weeks (see Figure 1 below).  
Technical Challenges (Implementation Data). For those in both groups, data will also be 
collected regarding technical challenges faced by participants such as: forgotten passwords; no 
individual model scores; and, not completing biweekly surveys. Moreover, for individuals in both 
conditions, we will capture information regarding research staff outreach (Administrative, Clinical).  
Military Health System (MHS) Data. Study participant medical usage (including medical 
and mental health) data (e.g., dates of visits, hospitalizations), Current Procedural Terminology 
[CPT Codes], International Classification of Diseases codes, Ninth Revision and Tenth Revision 
[ICD-9/10]) will also be obtained from the MHS.42 This will include information regarding all visits 
to a MHS facility during and 1 month post the study period (a total a four months). Likely data 
systems of interest include the MHS Data Repository (MDR)/Military Health System Mart (M2), as 
well as the Mental Health Research File.42 
 
B. Description of Population to be Enrolled   
 
Inclusion Criteria. 1) Member of the Navel Surface or Aviation Forces; 2) Age: 18-55 years at the 
time of enrollment; 3) History of at least one deployment with return to permanent station within 
the past 36 months, or stationed outside the Continental United States with permanent change of 
station (PCS) to the Continental United States within the past 36 months, or any PCS move within 
the past 36 months or on initial command duty assignment; 4) Ability to provide verbal and 
electronic informed consents; 5) Ownership of smartphone; and 6) Willingness to use smartphone 
and personal data plan to participate. 
 
Exclusion Criteria. 1) Having an iPhone (until the Cogito Corporation releases the iOS version of 
the Cogito Companion app). 
 
C. Study Design and Research Methods   
 
Recruitment. Dr. Werbel (Naval Captain, Co-Investigator on grant) will work with US 
Fleet Forces Command to identify units viable for participation. Members of such units may be 
serving on larger (e.g., Aircraft Carrier, ~5000 Sailors) or smaller (e.g., Destroyers, ~300 Sailors) 
vessels. Post-unit identification, Dr. Werbel will work with commanding officers to facilitate 
recruitment efforts, in terms of identifying vessels/Sailors. Members of this research team may  
present information to potential participants. This could occur on the participating vessels and/or 
identified military locations. Potential in-person options to inform potential participants regarding 
the study may range from recruitment on-board vessels during redeployment (recruitment and 
enrollment tables on ships) to pier-side post-leave during redeployment procedures/all-hands 
meetings to approximately 36 months post-redeployment. 
To facilitate remote recruitment, information regarding the study may be provided to 
Sailors via email or text, or flyers posted around ships/bases. Although distribution of such 
information may occur via military personnel, responses to such information (e.g., interest in 
enrollment) will only be directed to members of the study team. In terms of remote recruitment, 
Sailors can express interest in the study by calling the study team or completing a short 
recruitment survey in REDCap. The survey will collect name, phone number, and best day/time to 
contact the Sailor regarding next steps. 
 
Consent. Once interest has been noted, potential participants will be asked to provide 
verbal consent either in person, over the phone, or via a secure and approved video connection 

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 7 
 
platform (e.g., VA Video Connect, Zoom) to: 1) confirm eligibility and complete informed consent 
and baseline measures on the MyCAP/REDCap (Research Electronic Data Capture) smartphone 
app and/or web-based service 2) be randomized; 3) download either the Cogito Companion 
smartphone app (Experimental) or the MyCAP/REDCap smartphone app and/or web-based 
service (Active Control); and 4) complete baseline measures. Further information regarding 
MyCAP/REDCap (https://projectmycap.org/ or https://redcapinfo.ucdenver.edu/) is provided 
below.  
Sailors will be provided with written and/or electronic consent materials. That is, Sailors 
will be provided with a written copy and/or an electronic copy(ies) of the consent materials. 
Consent will be obtained via MyCAP/REDCap. This will be facilitated and observed by the team 
members. During this process, Sailors will be asked to confirm that they have read consent 
materials and received a written and/or electronic version of consent materials. During the study 
visit, members of the research team will also explain monitoring processes, and outreach 
procedures. Such procedures will be initiated in the event that the study team receives technical 
or clinical data suggesting that such a contact is warranted (more information to be provided 
below). Participants will be asked to provide the study team with multiple phone numbers via 
MyCAP/REDCap or in written format for personal (e.g., spouse, friend) and/or provider contact in 
case the participant cannot be reached during a clinical monitoring outreach call (see below for 
clinical monitoring procedures). 
Randomization will be stratified by gender, force (Surface versus Aviation), and 
smartphone type (i.e., Android, iPhone [once the iOS version is released]). Initially, only those 
with Android platform smartphones will be recruited. Once either application is assigned and/or 
downloaded, a member of the study team will provide the Sailor with the information to login (e.g., 
tokens). Participants will be provided with study team contact information, for any technical 
assistance while using the assigned modality, as well as information regarding monthly 
reimbursement for cell phone data usage ($30 per month). They will all be provided with 
Military/Veterans Crisis Line Contact information, and may be provided with additional resources 
(e.g., local resources). All payments may be made via a MasterCard debit type card called a 
“ClinCard” that is provided by Greenphire. Funds are usually available via this card within 24 
hours following the scheduled payment date (possible exceptions are weekends or holidays). 
Authorization. All required elements of informed consent will be provided on 
MyCAP/REDCap, including information regarding the purpose of the study, what to expect if 
participating, possible risks and discomforts, who is paying for the study, the voluntary nature of 
participating, whom to contact with questions, information regarding the privacy of participants’ 
information, and exceptions to confidentiality. We will request a Waiver of Documentation of 
Consent and a HIPAA Waiver to facilitate conducting this study.  
Study Design. Randomized Controlled Trial (RCT), with up to 1,418 individuals being 
allocated to Experimental (Cogito Companion) or Active Control (MyCAP) based on stratified 
randomization tables. Randomization will be stratified by gender, force (Surface, Aviation), and 
smartphone model (Android, I-Phone [once the iOS version is released]), as Cogito algorithms 
differ based on smartphone platforms. Data is also being collected (feasibility/acceptability) to 
inform future implementation. 
Study Procedures. Upon confirming interest in participating in the study, a member of 
the research team will obtain verbal informed consent and confirm eligibility. Electronic consent 
will be provided via MyCAP/REDCap. Baseline measures will be administered via 
MyCAP/REDCap. Military personnel will then be randomized to the Cogito Companion group 
(Experimental) or the Active Control (MyCAP) group. 
Figure 1. Study Procedures 
 

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 8 
 
Biweekly (every other week) post-consent, study measures will be pushed to participants via 
the mobile apps (Cogito Companion, MyCAP). Post-completion of measures at the end of the 
study period (Follow-up Survey), participants will be provided with a link (MyCAP/REDCap). See 
Figure 1 for an outline of study procedures, and Table 1 for proposed measures and 
administration schedule. 
 
Table 1. Measures and Administration 
Measure 
Domain 
Baseline 
During 
Intervention 
(Biweekly) 
Follow-
up  
Survey 
Beck Scale for Suicidal Ideation 
(BSS) 
Suicidal ideation 
X 
X 
X 
Client Satisfaction Questionnaire 
(CSQ) 
Satisfaction with 
intervention 
 
 
X 
Client Services Receipt Inventory-
EU-Revised (CSRI-EU-R) 
Health care utilization 
X 
X 
X 
Columbia-Suicide Severity Rating 
Scale (C-SSRS) Screener 
Suicidal ideation and 
behavior 
X 
X 
X 
MSRC Common Data Elements 
 
 
 
X 
MSRC Standard Demographic 
Form 
Demographics 
X 
 
X 
Outcome Questionnaire (OQ-45) 
Psychological 
Distress 
X 
X 
X 
Patient Health Questionnaire-9 
(PHQ-9) 
Depression 
X 
X 
X 
Posttraumatic Stress Disorder 
Checklist 5 (PCL-5) 
PTSD 
X 
X 
X 
Rand 36-Item Health Survey 
Quality of life 
X 
X 
X 
Self-Help Mobile Application 
Utilization Survey 
Use of readily 
available Apps 
X* 
X** 
X* 
COVID-19 Questionnaire 
 
X 
X 
X 
*All study participants; **Those allocated to the Active Control arm only 
 
Random Allocation and Concealment. Randomization to the 
Experimental or Active Control arms will be stratified by gender, force, 
and smartphone model. The block randomization schemes, using 
random block sizes of 2, 4, and 6, will be produced by Dr. Forster using 
Statistical Analysis System (SAS) PROC PLAN with a specified seed, 
prior to any recruitment. To ensure concealed allocation, participants will 
be designated by an identification number.  
Treatment Arms. Experimental. Those allocated to Experimental arm 
will have access the Cogito Companion for a three-month period post-
consent. Passive 
data collection will 
also occur during 
this period. As a secure, privacy-compliant 
mobile app, the Cogito Companion 
facilitates the non-invasive collection, 
transfer, integration, analysis, and reporting 
of objective behavioral indicators (Figure 2). 
The Cogito mobile sensing platform 
passively gathers behavioral information 
through an individual’s normal smartphone 
usage. Measurements of location, call, and 
text patterns are recorded. These raw 
Figure 2.  
Participant View 
Figure 3. Clinician Dashboard 

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 9 
 
measurements, which number in the tens of thousands per week, are fed into Cogito’s algorithms 
to compute validated behavioral indicators. These indicators are presented as model scores in 
the Clinician Dashboard, which will be used to facilitate monitoring of behavioral changes (Figure 
3). Further information regarding this process will be provided below and in Human Subjects 
Recruitment and Safety Procedures. Participants may also monitor their own behavioral changes 
overtime (Figure 3). Individual users will also be prompted to record an “audio diary” using the 
mobile app via their smartphone. An analysis of vocal qualities, rather than content, will produce a 
model score, which is provided to users and the research team. Participants will also be able to 
subjectively rate their mood at the time of the recording. Reminders to complete these “audio 
diaries” will occur biweekly, on an alternating schedule with the biweekly surveys. Nonetheless, 
participants will be able to leave as many “audio diaries” as they would like during the three-
month period. 
 
Clinical Research Staff Outreach – Cogito Group. Clinically-trained research team members 
will monitor the Clinician Dashboard each business day one time per day (excluding federal 
holidays and weekends). Clinical monitoring may include the review of responses for the 
following: the PHQ-9, BSS, C-SSRS Screener, and individual model scores derived by Cogito 
app algorithms. Lack of data may also prompt a clinical outreach. Further information regarding 
risk identification and clinical monitoring will also be provided below.  
 
PHQ-9 Monitoring. Clinical team members will review PHQ-9, item 9 “Thoughts that you 
would be better off dead or of hurting yourself in some way.”3 A response of 2 (more than half the 
days) or 3 (nearly every day) will prompt a clinical outreach call to the participant. A response of 1 
(several days) will be considered for a clinical outreach call in conjunction with other Dashboard 
data and/or surveys.  
BSS Monitoring. Clinical team members will review the BSS screening items (items 1-5). Any 
response of “2” on items 1- 4, will prompt a clinical outreach call. 
C-SSRS Screener Monitoring. Clinical team members will 
review responses to the C-SSRS and conduct outreach based 
upon recent and/or current suicidal ideation and/or suicidal or self-
harm behaviors. Specifically, we will outreach if the participant 
positively endorses thinking about how they may kill themselves, 
have had thoughts of killing themselves with intention of acting on 
these thoughts, and/or if they’ve started to work out details of how 
to kill themselves. 
Using the same clinical thresholds described above, the Cogito 
app will send an automatic notification to the participant regarding 
their scores, as well as resources (e.g., Military and Veterans 
Crisis Line, Military OneSource, Military Health System provider 
contact information [individualized data entered by participant or 
study team]). 
Cogito Model Score Monitoring. Model scores are presented both in number and color format 
(Figure 4). Red indicates scores 39 or below, yellow indicates scores between 40 and 79, and 
green indicates scores between 80 and 100. The higher the score, the “healthier” the individual’s 
behaviors. Any drastic daily change in score (+/- change of 20) and/or two scores less than 20 
may prompt a clinical outreach call.  
Active Control. Participants randomized to the Active Control group will download MyCAP 
and have access to resources housed within this app for a three-month period post-consent. 
Participants will be provided information regarding the mHealth Tool apps, and provided basic 
information on how to download the apps. As noted above, participants will be provided biweekly 
surveys as listed in Table 1. Participants will get one notification when surveys are available. If 
surveys are not completed, participants may be sent a reminder to complete the surveys via 
MyCAP automatic reminder or the in-app messaging feature. 
Clinical Research Staff Outreach – Active Control. Clinically-trained research team members 
will monitor the MyCAP dashboard each business day one time per day (excluding federal 
holidays and weekends); however, outreach will solely be based on survey data or lack thereof. 
Figure 4. Health-Related 
Feedback  

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 10 
 
Similar to the Cogito group, the Active Control group will receive the PHQ-9, BSS, and C-SSRS 
Screener surveys biweekly. If participants’ responses on the PHQ-9, BSS and/or the C-SSRS 
Screener meet the clinical threshold described above, they will receive an outreach call, as well 
as automatic notification from MyCAP with resources (e.g., Military and Veterans Crisis Line, 
Military OneSource, and Military Health System provider contact information [individualized data 
entered by participant or study team]). Lack of responses to surveys may also prompt clinical 
outreach. Further information regarding risk identification and clinical monitoring will be provided 
below. 
 
D.   Description, Risks and Justification of Procedures and Data Collection Tools 
 
Risk and Benefits. The primary benefit of this trial is facilitating knowledge regarding the 
early identification of risk and intervention among Naval personnel. During the study period, 
participants may become upset or frustrated with technology associated with the mobile 
application itself, or during administration of the interviews or questionnaires. Possible breaches 
of confidentiality may occur, though risks will be minimized as outlined below. The research may 
also involve unforeseeable risks. 
Safety Monitoring. Procedures for Minimizing Risk. Training. Members of the research 
team will receive training on risk assessment, crisis management, and regulatory reporting 
requirements. Clinical team members will receive additional training about clinical monitoring via 
the Cogito and MyCAP apps, using developed tools (e.g., clinical decision-tree).  
Risk Identification and Clinical Monitoring. Informed consent materials (which will be 
stored within the apps and available for participant download) will contain statements explaining 
mandatory reporting requirements for information regarding intention to harm self or others. 
During the informed consent process, all participants will provide multiple phone numbers for 
personal and/or provider contact in case we are unable to reach the participant during a clinical 
monitoring outreach call. All participants will be monitored during the study period. Specifically, 
clinical monitoring will occur via responses to the PHQ-9, BSS, C-SSRS, and for the Cogito 
group, the Cogito Model Scores. Participants may be outreached in two ways. First, if participants 
meet the clinical threshold on the PHQ-9, BSS and/or the C-SSRS, they will receive an automatic 
notification via Cogito Companion or MyCAP smartphone applications with crisis resources. 
Second, clinical team members will contact participant via phone using the following steps. 
Initial Data Assessment. Clinical team members will have clinical decision-tree handouts 
for both groups to help guide decision making. Once a clinical team member has determined the 
need for clinical outreach, the team member will call the participant. The study team will outreach 
the participant first, and if the participant does not answer, the study team may proceed to contact 
the personal and/or provider contacts the participant listed during the consent process. 
Clinical Outreach Calls. At the beginning of the call, a locator form will be completed to 
document the physical location and contact information of the participant in case of imminent risk. 
The participant may be asked for permission to collect contact information for the participants’ 
mental health or health provider, and/or a secondary contact person for additional safety 
outreach. Next, the team member will let the participant know that information gathered from the 
app indicated that the participant may be at increased risk and prompted a check on the 
participant. During the call, the team member will assess the participant’s urge to harm 
themselves or others, and suicidal intent using the UWRAP.38 Additional information may be 
gathered including coping strategies, reasons for living, and assessment of physical/mental 
health symptoms that may be eliciting suicidal ideation, intent, and/or a plan. Referral assistance 
to treatment providers (DoD/VA) may also be provided. All information gathered during the clinical 
call, as well as app monitoring data, will be used to determine level of risk and appropriate action 
steps to be taken. Three risk levels will be used: 1) imminent acute risk; 2) increased acute risk; 
or 3) no perceived increased acute risk. 
Imminent Acute Risk. Clinical team members, Dr. Betthauser, the PI, or their designee 
will stay on the telephone with the participant while another team member immediately calls 911, 
or the appropriate emergency service to initiate an on-site rescue if such action is clinically 
indicated.  

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 11 
 
Increased Acute Risk. If a participant appears to be at increased but not imminent risk of 
suicide, the clinical team member will provide the participant with crisis resources and encourage 
use, as well as referrals for treatment. Increased clinical monitoring and continued clinical 
outreach will occur following this contact until the participant is determined to be at No Perceived 
Increased Acute Risk.  
No Perceived Increased Acute Risk. If a participant appears to be at No Perceived 
Increased Acute Risk with respect to the platform data and suicide risk assessment, the clinical 
team member may provide crisis resources and referral information, as indicated.  
All participants will be informed that they will continue to be monitored via these platforms 
over the course of their participation in the study. Each clinical outreach, including steps taken 
during the outreach call, will be documented per study-specific standard operating procedures 
(SOPs) and reviewed with Drs. Betthauser and Brenner. 
Documentation of Risk. Notes from the call and responses to the UWRAP will be 
documented in electronic study “note to file” for the respective participant for each clinical 
outreach call. Drs. Betthauser and Brenner will be informed of the clinical call and will sign off on 
each note to file. Each note to file will be saved electronically in the participant’s study folder for 
reference and documentation during active participation in the study. Note to file information may 
be shared with participant’s mental health and/or health providers, with the permission of the 
participant, or with emergency responders in the case of clinical action for those at imminent risk. 
Data regarding clinical outreach calls will be compiled. After Action Reviews (AARs) will be 
conducted following the clinical outreach and these events and forms will be reviewed during 
clinical and administrative team meetings and with the PI.  
Administrative Monitoring and Outreach. Clinical team members also will assess the 
need for administrative outreach. Administrative outreach most generally will be initiated based 
on a lack of data being collected. Common issues include: participants’ limited internet 
connectivity; low frequency of text messaging behaviors; changes in app permission (e.g., 
location permission); and, lack of responses to surveys or incomplete surveys. In most cases, 
such issues will be noted by Clinical Team Members who will provided this information to the 
Research Coordinator and/or Assistant, who will make the administrative outreach calls. These 
administrative calls will be documented in an administrative log maintained by the study team for 
tracking purposes. Study team contact information will also be provided in both apps, should the 
participant encounter any administrative issues. The study team will be the primary contact for 
resolving any administrative or technical issues related to use of the application. 
Clinical & Administrative Consultation Reviews. Dr. Betthauser (clinical) and Ms. Stearns-
Yoder (administrative) will provide ongoing consultation to members of the research team. They 
will maintain notes regarding these meetings and will review such notes with the PI.  
Reporting. Any adverse event will be reported to the local Institutional Review Board by 
the PI within 5 days and will be reported to the Safety Monitor and MSRC as indicated. 
Data Management Plan. Minimizing risks pertaining to data security will be achieved by 
employing safeguards). Initially, all data collected via Cogito or REDCap will be stored on the 
secure server and behind the firewall at CU-AMC. Files will be user-restricted and/or password-
protected so that only members of the research team can access the data. Data in electronic form 
created by the study team will be limited data sets and stored in password protected Word 
documents, Excel spreadsheets and/or ACCESS databases within a restricted access folder 
behind the firewall at CU-AMC. A master list of participant names and phone number will be kept 
in a password-protected electronic file, password known only to an authorized member of the 
research team as designated by the PI. A separate research identification number spreadsheet 
will be kept and will include the participant’s names and an assigned unique research 
identification number (UID) in a password-protected electronic file, known only to the authorized 
research team members. All data in paper format or original documents containing sensitive 
information will be stored in locked filing cabinets in the Department of Physical Medicine & 
Rehabilitation, in administrative offices at CU-AMC. 
To facilitate data analysis aggregate de-identified datasets may be stored at the Rocky 
Mountain Regional VA Medical Center. Files will be user-restricted and/or password-protected so 
that only members of the research team can access the data. All VA data in electronic form will 
be limited data sets and stored in password protected Word documents, Excel spreadsheets 

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 12 
 
and/or ACCESS databases within a restricted access folder on the VA shared drive and on the 
secure server and behind the VA firewall. The secure VA servers are located in a secure room 
and are backed up nightly. 
Cogito. Cogito servers and centralized hardware are locked down both physically and 
electronically. Networking infrastructure is done through a series of access-controlled devices, 
which restrict access of outside presences and protect inside resources. Datasets are stored on 
encrypted volumes or under systems with full disk encryption, as appropriate. Data are encrypted 
and backed-up externally. Data access is granted under least privilege. Cogito is fully equipped 
with an internal system for project management and documentation of non-PII (personally 
identifiable information) research materials. This system is fully backed-up and accessible by all 
employees, both on and off-site. The computing infrastructure has automated monitoring 
capacities to alert Cogito engineering for downtime, data anomalies, security risks, and directed 
attacks. End-point data will be validated and confirmed by members of the research team. Cogito 
has secured Federal Wide Assurance (FWA) for the Protection of Human Subjects. All 
participants assigned to the Cogito group will be provided a unique research identification “token.” 
Cogito Corporation will provide the research team with a list of unique tokens prior to study 
recruitment. Authorized members of the research team will distribute the Cogito token at the time 
of enrollment to participants assigned to this group. The research team will keep a password-
protected list of the assigned Cogito token and the participant’s unique research identification 
number (UID). Cogito Corporation will transfer de-identified data via encrypted Word documents 
and/or Excel spreadsheets to authorized members of the research team via email at CU-AMC.  
REDCap. The University of Colorado Anschutz Medical Campus (AMC) REDCap 
platform is a highly secure, nationally-utilized data management system, and it is housed within 
the highly-secure environment on the AMC. MyCAP is an app developed for use within REDCap. 
MyCAP allows research study participants to access and respond to surveys on their mobile 
devices. The app is protected by a 6-digit passcode chosen by the participant, and data are 
stored in an encrypted database. Participants’ survey responses are synchronized with the 
REDCap database. All participants assigned to the Active Control group will be provided a unique 
MyCAP research identification number (ID). A member of the study team will obtain a list of 
unique MyCAP ID prior to study recruitment. Authorized members of the research team will 
distribute the MyCAP ID at the time of enrollment to participants assigned to this group. The 
research team will keep a password-protected list of the assigned MyCAP ID and the participant’s 
unique research identification number (UID). Authorized members of the research team will have 
secure access to MyCAP/REDCap data, and all data will be transferred and stored securely in a 
user-restricted study folder, saved on the VA server behind the VA firewall (see below). 
Clinical & Administrative Outreach Data. All clinical and administrative outreach data will 
be kept using the participant’s assigned UID and protected as described above.  
Military Health System Data. Data management and security for military health 
information will be stored securely in accordance with the Guide for DoD Researchers on Using 
MHS Data, as well as protections in accordance with 32 CFR219, DoDI 3216.02 and other 
applicable Federal laws, regulations, and DoD policies.43,44  
    
E.   Potential Scientific Problems   
 
 
Participant recruitment and retention may be a potential scientific concern; however, we 
have initial support from Naval command to recruit onboard an aircraft carrier (~5000 Sailors) and 
attached Striker units. The PI and Dr. Werbel will continue to work with Naval command to ensure 
ongoing recruitment support and planning. 
Technological issues with Android and/or iPhone platforms with the assigned study arms 
(Active or Control) may occur. The study team has pilot data that assists in problem-solving these 
concerns quickly. Cogito Corporation will have dedicated staff to assist with any Cogito 
Companion technological issues as they arise. The study team will monitor MyCAP issues and 
work closely with REDCap staff at the University in a timely manner. Issues will be documented 
and reviewed as part of Aim 2 and appropriate amendments may occur with consistently 
identified issues. 

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 13 
 
While 21% attrition is expected and the statistical methods proposed below are equipped to 
handle data that are missing at random, characteristics of participants who drop out of the study 
will be compared to those who do not and if there is differential dropout by group or it is plausible, 
based on characteristics of those who did drop out, that the missingness is related to the primary 
outcome of interest, additional methods aimed at accounting for nonignorable dropout will be 
employed where possible. 
 
F.   Data Analysis Plan   
 
Power Analysis. Power is calculated for the two primary hypotheses assessing episodes 
of distress and distress over the course of the study. We will recruit 1418 participants and expect 
approximately 33% to decline participation resulting in 950 participants randomized to either the 
Cogito Companion or Active Control groups, stratified by gender, force and phone type. 
Assuming 21% attrition, it is expected that 750 participants will complete the entire protocol. 
While the analysis will include all participants who are randomized, power is based on 750 
completed (375 per group). Power to address episodes of distress was estimated using the 
Poisson regression procedure in Power and Sample Size (PASS) v13. An episode of distress is 
defined as an OQ-45 total score of ≥ 64 with each participant having a maximum of 6 possible 
episodes (every 2 weeks for weeks 2-12 of the study, inclusive). We wish to detect a 15% 
difference in the number of episodes of distress over the 3-month study period, and we assume a 
two-sided test and a significance level of 0.025 to control for the two primary tests (number of 
episodes and overall distress). If the Active Control mean response rate is 3.5 (i.e., number of 
episodes in the 3-month study period) we will have 97% power to detect a 15% difference. For 
Active Control response rates of 3, 2.5, 2 and 1.5, power is then 94%, 89%, 81% and 68%, 
respectively. For the second primary hypothesis, we will calculate the Area Under the Curve 
(AUC) using an estimate statement within a linear mixed model described below. We wish to 
demonstrate that the Cogito Companion is superior to Active Control by a margin of at least 14 
points (minimally clinically significant difference, translates to an AUC difference of 84). Using the 
Superiority by a Margin procedure in PASS v13, 375 participants per group provides 90% power 
to demonstrate superiority, assuming a true difference of ≥18.7 points (AUC difference of 112.2) 
with a significance level of 0.025.  
Analysis Plan. All analyses will assume a two-sided test of hypothesis, a significance 
level of 0.05 (unless otherwise indicated) and will be run in SAS v9.4 or higher, or R v3.3 or 
higher. To check randomization, demographic and clinical characteristics will be compared 
between groups using t-tests, chi-square tests or nonparametric tests, as appropriate. If any 
variable is found to be significantly different between the groups and is also plausibly associated 
with an outcome, it will be considered as potential confounders in the models described below.  
Aim 1. To address the hypothesis regarding episodes of distress first using Poisson 
regression will be used to model the number (response rate) of episodes of distress as a function 
of group (0=Active Control, 1=Cogito), any potential confounders, and an offset of log (time) to 
account for varying lengths of follow-up. If the deviance is estimated to be >1.5, Negative 
Binomial regression will be used instead, incorporating the same variables. The parameter 
estimate associated with the group variable will be exponentiated, providing an estimate of the 
ratio of response rates between the groups and will be reported with a 97.5% confidence interval 
(CI). To address the hypothesis regarding distress over the course of the study, a linear mixed-
effects model will be employed with longitudinal OQ-45 scores as the dependent variable and 
group, potential confounders, and an interaction between group and time (using either natural 
cubic B-spline transformations on time or polynomial transformations on time) as independent 
variables. These transformations will also be applied to the random time effects so that individual 
trajectories can be calculated. Akaike’s Information Criterion will be used to determine which 
model provides the best fit (both between the B-spline/polynomial models and within them). Once 
a final model is determined, the difference in AUC will be calculated and between groups and 
compared to the clinically meaningful AUC difference of 84 using a one-sided test. Here the null 
hypothesis is that AUCAC - AUCCogito ≥ 84, and the alternative hypothesis is that AUCAC - AUCCogito 
< 84. The estimated difference will be reported with the one-sided 97.5% CI. For hypothesis 1.2, 
a Kaplan-Meier estimate will be used to compare the median time to treatment engagement 

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 14 
 
between the groups and each median will be reported with 95% CIs. Treatment engagement is 
operationalized as a visit with any medical or mental health provider in which a mental health 
related ICD code is identified. and will be determined by medical record review. Additionally, the 
total number of mental health visits calculated from the CSRI-EU-R will be modeled using a 
Poisson or Negative Binomial regression model to estimate the difference in number of visits 
between the groups over the course of the study. The model will be set up similarly to the OQ-45 
model. Hypothesis 1.3 is similar to hypothesis 1.1 such that we wish to look at episodes of 
depressive symptoms (PHQ-9), post-traumatic symptoms (PCL-5) and suicide-related thoughts 
(BSS) as well as severity of each over the course of the study. We also wish to look at perceived 
physical and psychological health functioning (RAND SF-36) over the course of the study. The 
methods employed to address hypothesis 1.1 will be utilized to estimate the difference in number 
of episodes of each outcome (excluding the SF-36) as well as the difference in AUC for each 
outcome. Zero-inflated negative binomial regression will be considered for the BSS if a large 
proportion of the participants do not exhibit suicidal thoughts and transformations will be 
considered for the AUC BSS analysis, given that it is likely to be highly skewed. Cutoffs for 
episodes of depressive/post traumatic symptoms, and suicidal thoughts are ≥10 (PHQ-9),45 ≥33 
(PCL-5),46 and ≥2 (BSS),47 respectively. 
Aim 2. We will assess the proportion of participants who find the Cogito Companion 
acceptable, defined as ≥ 70% of participants scoring ≥ 24 on the CSQ. The proportion of Cogito 
Companion participants with a CSQ score of ≥24 will be calculated with a 95% CI. Additionally, 
the proportion of participants engaging in activities will be reported with a 95% CI as well as the 
median (range) amount of time spent on the application. Feasibility will be measured by the 
proportion of Cogito Companion participants who complete treatment and reasons for termination 
will be examined.  
Exploratory Aim. To explore patterns of distress experienced by participants over the 
course of the study, individual OQ-45 curves modeled in Aim 1 will be qualitatively clustered into 
groups based on visual inspection of the trajectory patterns. Once distinct patterns are identified 
and labeled, baseline demographic and clinical characteristics of participants in each pattern will 
be summarized and reported. This will be performed for the entire sample, as well as within each 
group. This is considered a hypothesis generating aim. This analysis will be repeated for the 
PHQ-9, PCL-5, BSS and SF-36. 
 
G.  Summarize Knowledge to be Gained   
 
Establishing the efficacy of the Cogito Companion has direct military relevance aligned with 
Leaderships' desire to promote wellness and help-seeking behaviors, as well as prevent suicide-
related behaviors among Naval personnel during the redeployment home following a stressful 
forward operating mission. Importantly, this upstream intervention (the Cogito Companion) has 
the potential to facilitate identification of those transitioning (e.g., redeployment, permanent 
change of station, separation) whose risk may: 1) not be easily identified immediately post-
deployment; 2) and/or may increase secondary to intra- and inter-personal stressors during the 
reintegration period. Identifying and engaging such individuals in treatment using traditional 
methods has proven to be challenging. Based on consumer preferences, technology is finding its 
way into the health arena. Identifying means by which technology can be used to evaluate risk 
identification is imperative. Specifically, evidence-based, portable, and non-pathologizing 
interventions, which capitalize on Active Duty Service Member comfort with technology, are 
necessary to prevent suicide among members of this high-risk cohort. Importantly, project aims 
also align with the MSRC objective of “fund[ing] … clinical trial … research with high value to the 
MSRC,” aimed at promoting help-seeking behavior and preventing suicide. 
 
 H. References 
 
1. Ellsworth JR, Lambert MJ, Johnson J. A comparison of the Outcome Questionnaire‐45 and 
Outcome Questionnaire‐30 in classification and prediction of treatment outcome. Clinical 
Psychology & Psychotherapy: An International Journal of Theory & Practice. 2006;13(6):380-391. 

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 15 
 
2. Heinrich S, Deister A, Birker T, et al. Accuracy of self-reports of mental health care utilization 
and calculated costs compared to hospital records. Psychiatry research. 2011;185(1-2):261-268. 
3. Kroenke K, Spitzer RL. The PHQ-9: a new depression diagnostic and severity measure. 
Psychiatric annals. 2002;32(9):509-515. 
4. Blevins CA, Weathers FW, Davis MT, Witte TK, Domino JL. The posttraumatic stress disorder 
checklist for DSM‐5 (PCL‐5): Development and initial psychometric evaluation. Journal of 
Traumatic Stress. 2015;28(6):489-498. 
5. Ware Jr JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36): I. Conceptual 
framework and item selection. Medical care. 1992:473-483. 
6. McHorney CA, Ware Jr JE, Raczek AE. The MOS 36-Item Short-Form Health Survey (SF-36): 
II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. 
Medical care. 1993:247-263. 
7. Feeley N, Cossette S, Cote J, et al. The importance of piloting an RCT intervention. The 
Canadian journal of nursing research = Revue canadienne de recherche en sciences infirmieres. 
2009;41(2):85-99. 
8. Attkisson CC, Zwick R. The client satisfaction questionnaire. Psychometric properties and 
correlations with service utilization and psychotherapy outcome. Evaluation and program 
planning. 1982;5(3):233-237. 
9. McNulty PA. Reported stressors and health care needs of active duty Navy personnel during 
three phases of deployment in support of the war in Iraq. Military medicine. 2005;170(6):530-535. 
10. Mansfield AJ, Bender RH, Hourani LL, Larson GE. Suicidal or self‐harming ideation in military 
personnel transitioning to civilian life. Suicide and Life‐Threatening Behavior. 2011;41(4):392-405. 
11. Bush NE, Ouellette G, Kinn J. Utility of the T2 Mood Tracker mobile application among army 
warrior transition unit service members. Military medicine. 2014;179(12):1453-1457. 
12. Wang Y, Sareen J, Afifi TO, Bolton SL, Johnson EA, Bolton JM. A population-based 
longitudinal study of recent stressful life events as risk factors for suicidal behavior in major 
depressive disorder. Archives of suicide research : official journal of the International Academy for 
Suicide Research. 2015;19(2):202-217. 
13. Kendler KS, Gardner CO. Dependent stressful life events and prior depressive episodes in 
the prediction of major depression: the problem of causal inference in psychiatric epidemiology. 
Archives of general psychiatry. 2010;67(11):1120-1127. 
14. Bonanno GA. Uses and abuses of the resilience construct: loss, trauma, and health-related 
adversities. Social science & medicine (1982). 2012;74(5):753-756. 
15. Liu RT, Miller I. Life events and suicidal ideation and behavior: a systematic review. Clinical 
psychology review. 2014;34(3):181-192. 
16. LeardMann CA, Powell TM, Smith TC, et al. Risk factors associated with suicide in current 
and former US military personnel. Jama. 2013;310(5):496-506. 
17. Hyman J, Ireland R, Frost L, Cottrell L. Suicide incidence and risk factors in an active duty US 
military population. American journal of public health. 2012;102 Suppl 1:S138-146. 
18. Nock MK, Hwang I, Sampson N, et al. Cross-national analysis of the associations among 
mental disorders and suicidal behavior: findings from the WHO World Mental Health Surveys. 
PLoS medicine. 2009;6(8):e1000123. 
19. Duberstein PR, Conwell Y, Conner KR, Eberly S, Caine ED. Suicide at 50 years of age and 
older: perceived physical illness, family discord and financial strain. Psychological Medicine. 
2004;34(1):137-146. 
20. Thoresen S, Mehlum L, Moller B. Suicide in peacekeepers. Social psychiatry and psychiatric 
epidemiology. 2003;38(11):605-610. 
21. Kposowa AJ. Marital status and suicide in the National Longitudinal Mortality Study. Journal 
of Epidemiology & Community Health. 2000;54(4):254-261. 
22. Brown SL, Vinokur AD. The interplay among risk factors for suicidal ideation and suicide: The 
role of depression, poor health, and loved ones' messages of support and criticism. American 
Journal of Community Psychology. 2003;32(1-2):131-141. 
23. Logan J, Skopp NA, Karch D, Reger MA, Gahm GA. Characteristics of suicides among US 
army active duty personnel in 17 US states from 2005 to 2007. American journal of public health. 
2012;102(S1):S40-S44. 

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 16 
 
24. Warner CH, Appenzeller GN, Parker JR, Warner C, Diebold CJ, Grieger T. Suicide prevention 
in a deployed military unit. Psychiatry: Interpersonal & Biological Processes. 2011;74(2):127-141. 
25. President Donald J. Trump signs Executive Order to Improve Mental Health Resources for 
Veterans Transitioning from Active Duty to Civilian Life [press release]. 
https://www.va.gov/opa/pressrel/pressrelease.cfm?id=39952018. 
26. Instruction DoD. Deployment Health Assessments. 2018; http://www.pdhealth.mil/treatment-
guidance/deployment-health-assessments. Accessed July 4, 2018. 
27. Brenner LH, C; Mohatt, N; Forster, JE. Preventing Suicide among Veterans Will Require 
Clinicians and Researchers to Adopt a Public Health Approach. FORUM. 2018;Spring. 
28. Caldwell D. The Suicide Prevention Continuum. Pimatisiwin. 2008;6(2):145-153. 
29. Place S, Blanch-Hartigan D, Rubin C, et al. Behavioral Indicators on a Mobile Sensing 
Platform Predict Clinically Validated Psychiatric Symptoms of Mood and Anxiety Disorders. 
Journal of medical Internet research. 2017;19(3):e75. 
30. APA. Diagnostic and statistical manual of mental disorders 5th Ed. American Psychiatric 
Publishing; 2013. 
31. Place S. Dashboards for Clinician Monitoring of Patients Through a Mobile Sensing Platform.  
https://clinicaltrials.gov/ct2/show/NCT02167373. 
32. Lambert MJ, Burlingame GM, Umphress V, et al. The reliability and validity of the Outcome 
Questionnaire. Clinical Psychology & Psychotherapy: An International Journal of Theory and 
Practice. 1996;3(4):249-258. 
33. Maruish ME, Nelson EA. Psychological testing in the age of managed behavioral health care. 
Routledge; 2001. 
34. Beck AT, Steer RA, Ranieri WF. Scale for suicide ideation: Psychometric properties of a 
self‐report version. Journal of clinical psychology. 1988;44(4):499-505. 
35. Hays RD, Sherbourne CD, Mazel RM. The rand 36‐item health survey 1.0. Health economics. 
1993;2(3):217-227. 
36. Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, Currier GW, Melvin 
GA, Greenhill L, Shen S, Mann JJ. The Columbia-Suicide Severity Rating Scale: Initial validity 
and internal consistency findings from three multisite studies with adolescents and adults. Am J 
Psychiatry. 2011;168(12):1266-1277. 
37. Gaston L, Sabourin S. Client satisfaction and social desirability in psychotherapy. Evaluation 
and program planning. 1992;15(3):227-231. 
38. Linehan M. University of Washington risk assessment protocol. Unpublished instrument 
University of Washington, Seattle. 1998. 
39. Reynolds SK, Lindenboim N, Comtois KA, Murray A, Linehan MM. Risky assessments: 
participant suicidality and distress associated with research assessments in a treatment study of 
suicidal behavior. Suicide and Life-threatening behavior. 2006;36(1):19-34. 
40. DHA. mHealth Tools.  http://t2health.dcoe.mil/sites/default/files/T2-TSWF-AIM-Client-
Handout-Aug2016-web.pdf. Accessed July 4, 2018. 
41. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data 
capture (REDCap)—a metadata-driven methodology and workflow process for providing 
translational research informatics support. Journal of biomedical informatics. 2009;42(2):377-381. 
42. Office of the Assistant Secretary of Defense for Health Affairs (OSAD(HA)). Guide for DoD 
Researchers on Using MHS Data. In: TRICARE Management Activity HRPP, ed2012. 
43. CMS. Code List for Certain Designated Health Services.  
https://www.cms.gov/Medicare/Fraud-and-Abuse/PhysicianSelfReferral/List_of_Codes.html. 
Accessed July 4, 2018. 
44. Office HRP. Regulations, Policies and Procedures.  
http://mrmc.amedd.army.mil/index.cfm?pageid=research_protections.hrpo_policies. 
45. Spitzer R, Williams J, Kroenke K. Instruction manual: Instructions for patient health 
questionnaire (PHQ) and GAD-7 measures. In:1999. 
46. Weathers F, Litz B, Keane T, Palmieri P, Marx B, Schnurr P. The PTSD Checklist for DSM-5 
(PCL-5). Scale available from the National Center for PTSD. Boston (MA): National Center for 
PTSD. 2013. 
47. Brown GK. A review of suicide assessment measures for intervention research with adults 
and older adults. GK Brown; 2001. 

VA Eastern Colorado Health Care System 
NCT04159480 Protocol 
Page 17 
 
48. McLean CP, Cloitre M. Coronavirus Stressor Survey. 2020. 
49. Ehrenreich-May J. Fear of Illness and Virus Evaluation Adult Report Form. 2020.