Increasing Help-Seeking Behavior Among Transitioning Veterans at Risk for Suicide With Online Gatekeeper Training: A Pilot Study of PsychArmor S.A.V.E.
Phase: NA
Conditions: Suicide Prevention
Sponsor: VA Office of Research and Development

1 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
 
 
Protocol and Statistical Analysis Plan 
 
Study Title 
Increasing Help-Seeking Behavior Among Transitioning Veterans at Risk for Suicide With Online 
Gatekeeper Training 
 
Clinical Trials.gov ID 
NCT04565951 
 
Date approved by VA Portland Institutional Review Board 
06/29/2021 
 

2 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
    
VAPORHCS Research Protocol/Local Protocol Addendum  
 
Title      
VAPORHCS/OHSU J: Increasing Help-Seeking Behavior Among Transitioning Veterans At Risk For Suicide 
With Online Gatekeeper Training: A Pilot Study Of PsychArmor S.A.V.E.  
 
Specific Aims/Purpose 
 
Transitioning Veterans (those who have recently separated from the military) are a high-risk group for 
suicide. A key component to advancing suicide prevention efforts among at-risk Veterans is to address low 
levels of help-seeking and engagement in clinical care. 
Gatekeeper training is a type of intervention with the potential to substantially increase help-seeking, and 
thereby contribute to “upstream” suicide prevention efforts. Gatekeeper training teaches individuals the 
skills needed to identify signs of suicide risk and connect others to professional treatment. PsychArmor 
S.A.V.E. is a recently-developed form of brief online gatekeeper training and has been tailored to Veterans 
and their close supports. 
We propose to conduct a pilot randomized controlled trial (RCT) of PsychArmor S.A.V.E. Using a targeted 
advertisement campaign on social media, we will reach and recruit transitioning Veterans and their close 
supports (family and friends) into this online study. Participants will be randomized 1:1 to PsychArmor 
S.A.V.E. or to a “sham” online training. We will use a mixed methods evaluation of website administrative 
data, surveys, and interviews, including follow-up over six months. This pilot will inform a subsequent, larger 
RCT to determine if PsychArmor S.A.V.E. improves trainee behaviors and reduces rates of suicide 
behaviors.  
Our Specific Aims are: 
1. Determine the feasibility of recruiting participants via social media, engaging them to participate 
in an online gatekeeper training program, and retaining them in an online intervention study.  
2. Determine the acceptability of PsychArmor S.A.V.E. in a pilot RCT, and evaluate measures for 
use in a future larger-scale RCT.  
Three key deliverables from this project will be: 1) empirical data on study feasibility (recruitment flow, 
training participation, and loss to follow-up); 2) a mixed methods evaluation of the acceptability of 
PsychArmor S.A.V.E.; and 3) measurement characteristics of key outcome variables for use in design of 
larger-scale RCT. If successful, this project will seamlessly link outreach via social media outreach to an 
online training in suicide prevention skills. This line of research has major potential for implementation and 
impact due to its broad reach and scalability. 
 
 
Hypothesis:   
 
This study has three hypotheses related to Aim 1: 
● Hypothesis 1.1: Twenty participants per week, on average, will enroll in the study 
● Hypothesis 1.2: At least 50% of eligible individuals will enroll in the study 
● Hypothesis 1.3: Loss to follow-up at the study endpoint will be less than 50%  
 
And one hypothesis related to Aim 2: 
● 
Hypothesis 2: Participants will complete on average at least 70% of the PsychArmor S.A.V.E. 
training.  
 
Scientific Rationale and Significance  
     

3 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
 
 
 
Suicide prevention among Veterans not engaged in Veterans Health Administration (VHA) care and 
Veterans transitioning from military service is an urgent priority. An estimated 70% of Veterans who 
die by suicide have not recently used any VHA services.1 The latest data further show that suicide rate 
among Veterans who do not use VHA care increased by 12% between 2016 and 2017.1 As a consequence, 
addressing lack of help-seeking has been called one of the top three short-term research objectives in 
suicidology.2 Addressing this barrier is especially critical among young Veterans transitioning from military 
service to civilian life who may face disruption in their social networks and challenges establishing care with 
VHA. 
Veterans’ close supports—which includes include family members and friends, as well as Veteran 
peers—have a vital role to play in increasing help-seeking and treatment engagement among 
Veterans at risk of suicide. Support and encouragement from Veterans’ close supports is a key facilitator 
of help-seeking.3 In addition, suicidal individuals are most likely disclose their thoughts to a close social 
network member.4 Individuals having frequent social contact with at-risk individuals have the greatest 
potential to impact suicide prevention.5 
Veterans’ close supports are likely to have the opportunity to intervene on behalf of a Veteran at risk 
of suicide and seek training in how to help. A study of 971 National Guard personnel found that 65% 
reported knowing someone who had died by suicide, and on average, participants knew three suicide 
decedents.6 Another study of 931 Veterans found that younger Veterans had an even higher chance of 
interacting with peers at risk of suicide, suggesting that transitioning Veterans may be an especially good 
target for suicide prevention efforts.7 National data suggest that Americans “overwhelmingly agree they 
have an important role to play in preventing suicide – and most (78%) are interested in learning how they 
might be able to play a role in helping someone who may be suicidal.”8 However, few have ever received 
training in suicide prevention. Even among groups most often targeted for training (e.g., college students), 
just 8-10% have had any exposure to training.9 
Gatekeeper training is a community-based, public health suicide prevention strategy. Gatekeeper 
training is explicitly designed to provide close supports with training in how to help a person at risk of 
suicide. It can be delivered outside of the healthcare system, and thus reach Veterans not engaged in VHA 
care who are most at risk of suicide. Further, it can be useful for both selective and universal prevention.10 
In this proposal we focus on transitioning Veterans—due to their elevated risk for suicide—and their close 
supports (selective prevention). However, gatekeeper training may also be relevant at a broader population 
level to support “upstream” suicide prevention (universal prevention).  
Gatekeeper training is promising but understudied. Positive outcomes from gatekeeper training include 
improvements in knowledge, self-efficacy, and behavioral intentions related to suicide prevention.11 
Intervention studies also have shown improvements in attitudes such as stigma and social norms towards 
mental health treatment, which thereby lead to increased help-seeking behaviors.12,13 Studies also support 
these improvements in diverse community-based populations,14 over long-term follow-up (up to two 
years),15 and using brief (  2 hours) gatekeeper training.14,16 Some large studies—including a multi-site 
randomized controlled trial (RCT) and large-scale observational studies—have found preliminary evidence 
of reduction in suicide attempts17 and suicide mortality rates.18 Nonetheless, multiple systematic reviews 
have called for additional research on gatekeeper training.19–22 A primary reason is the lack of studies 
demonstrating that gatekeeper training directly decreases suicide rate. This is likely due to the constraints of 
adequately powering a study to show change in suicide mortality which has a low base rate (a limitation 
common to many suicide prevention interventions), as well as historically suboptimal interactions between 

4 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
healthcare systems and suicide prevention programs (a limitation unique to community-based interventions 
such as gatekeeper training).23 
Two additional areas of research needed on gatekeeper training are: 
1) Feasibility and potential efficacy of online-based gatekeeper training. While online gatekeeper 
training is appealing due to its practicality and accessibility, research into it has been lacking,24 and no 
studies have been conducted in the U.S. or targeted Veteran populations.25,26 One non-comparative 
study examined the feasibility of using an online gatekeeper to direct individuals searching for suicide-
related keywords to a website encouraging use of an e-mail consultation service. While the strength of 
evidence was low, modest levels of treatment engagement and improvement in mood were seen.27 
2) Outcomes for the recipient of the training. While the primary goal of training is to equip a close 
support to help another individual, there is preliminary evidence of an ancillary benefit to the trainee. In 
post-hoc analyses from a recent RCT, researchers found that recipients of gatekeeper training 
increased their own mental health care utilization at three months post-intervention.12 Because close 
supports for Veterans are often Veterans themselves, exploring the potential benefit of gatekeeper 
training for the trainee is especially important in a VHA context. 
PsychArmor S.A.V.E. is an accessible, engaging, brief online gatekeeper training tailored to the 
Veteran community. As with all gatekeeper trainings, it teaches practical skills in how to assist a person at 
risk of suicide and increase formal help-seeking.5,12,18 Unlike other gatekeeper trainings (e.g., QPR, 
Mental Health First Aid, ASIST), though, it is both free and online, which increases its accessibility to 
Veterans and their close supports.28 PsychArmor S.A.V.E. is also intentionally brief (24 minutes), has a 
video format, and includes three scripted role plays; these features may help engage individuals who might 
otherwise be reluctant.25 Finally, it was developed in partnership between VA and the non-for-profit 
PsychArmor Institute; it is culturally tailored to needs and circumstances of the Veteran community. Cultural 
tailoring an intervention may improve treatment outcomes, particularly when the population has unique 
vulnerabilities, and/or when there is a reduced responsiveness to other interventions.29 
Social media is an innovative way to reach transitioning Veterans and their close supports, and 
promote participation in PsychArmor S.A.V.E. Recruitment of participants via social media offers the 
potential for immediate delivery of an online intervention such as PsychArmor S.A.V.E. Among social media 
platforms, Facebook is used by most Veterans and has the largest and most active base of users: 76% log 
on daily and spend on average 50 minutes on the site.30–32 Ads on social media platforms such as 
Facebook have the added benefit of being able to presented to narrowly targeted populations (i.e., “micro-
targeting”) based on demographics, location, interests, and behaviors.33 
 
VA-SPECIFIC REQUIREMENT (relevance to VA mission):  
 
This project is highly responsive to multiple top VHA priorities. This project focuses on VA’s topic 
clinical priority, suicide prevention. The 2019 VA and Department of Defense Clinical Practice Guidelines 
also identified gatekeeper training as an important research gap and priority for future research.23 Finally, 
our research design addresses three separate goals and objectives in VA’s National Strategy for Preventing 
Veteran Suicide: 1) use a “comprehensive public health approach to reduce Veteran suicide rates, one that 
looks beyond the individual to involve peers, family members, and the community”; 2) “Develop and 
promote educational materials about the warning signs for Veteran suicide and how to connect individuals 
in crisis with assistance and care”; and 3) “Provide training on suicide prevention to community groups.”1 
Preliminary Studies 
     
 

5 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
1. Feasibility of a social media ad campaign to recruit a high-risk Veteran population. In our previous 
work, we used a similar recruitment strategy to target younger Veterans (Iraq/Afghanistan service era, 
average age 40 years old) to participate in an online, cross-sectional study. Ads from this campaign were 
seen over 800,000 times (a metric called “impressions”), 85% of eligible individuals enrolled, and we 
recruited 587 participants eligible for data analysis in six weeks (~100 participants/week recruited).34 In the 
same study, we found high rates of screening positive for current suicidal ideation (23%), posttraumatic 
stress disorder (45%), problematic drinking (42%), and major depression (28%). In addition, while 67% of 
respondents reported being enrolled in VHA care, 55% denied using VHA care in the prior year. Together, 
these data indicate our ability to successfully reach Veterans at risk of suicide and support our hypothesis 
that gatekeeper training may be of benefit to trainees themselves. 
2. Acceptability of online suicide prevention training. As part of the same project above,34 we analyzed 
survey responses from the 587 participants and found over 40% were very to extremely interested in 
learning how to help a peer who was having suicidal thoughts, 30% very to extremely interested in learning 
how to spot warning signs of suicide, and 30% very to extremely interested in learning where to find mental 
health services or treatment options (unpublished data). 
3. Efficacy of in-person gatekeeper training. Dr. Cross (Co-I) is a distinguished expert in the area of 
gatekeeper training, including the original development of S.A.V.E. (the precursor to PsychArmor S.A.V.E.). 
She has also led numerous clinical trials of gatekeeper training, which demonstrate improvements in 
knowledge, self-efficacy, and intentions to use and disseminate training information among participants.35–
39 Dr. Teo (PI) has also led a trial of a culturally-tailored, brief format in-person gatekeeper training for lay 
community members (n = 106). Results showed significant increases in behavioral intentions and attitudes 
related to gatekeeping, compared to an attention control.14 
4. Collaboration and partnership among the investigative team. Members of the research team have 
worked together since 2014 on multiple projects related to suicide prevention in Veterans, including suicide 
risk factors40,41 (Teo/Dobscha/ Cross/Gamble), an ongoing training program on discussing lethal means 
safety (Dobscha/Karras-Pilato), and depression (Teo/Dobscha).42 The PI has also built and maintained 
robust partnerships with PsychArmor Institute and OMHSP. 
5. Research experience and technical expertise in the proposed data collection and analytic 
methods. In the area of social media, Dr. Teo (PI) has significant experience with recruitment via social 
media and analyzing data obtained from social media platforms including Facebook and Twitter.34,43 He 
has also collaborated with the Oregon Health & Science (OHSU) Strategic Communications social media 
team who will partner on this project and possess broad expertise in marketing across an array of social 
media platforms. In terms of experience with web-based suicide prevention training, the team includes Dr. 
Rodgers (Consultant; oversight in the development and management of PsychArmor S.A.V.E.)., Dr. Gamble 
(Consultant; collaborates with education leaders across VHA to develop and disseminate suicide prevention 
trainings through web-based modalities), and Dr. Dobscha (Consultant; led a team that systematically 
developed and evaluated web-based training programs for clinicians and patients related to OpenNotes in 
mental health).44 Finally, the research team is experienced with longitudinal data collection and analysis, 
mixed methods research, and intervention studies and human subjects research with individuals at risk of 
suicide.42,45–48 
 
Research Design and Methods  
 
Recruitment via social media. We will recruit participants through a targeted social media advertisement 
campaign. Recruitment through social media offers the advantage of being able to reach veterans outside 
the VA system and immediately deliver an online intervention.  

6 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
We will use Facebook as our primary platform for recruitment; however, in consultation with our partners 
including the Oregon Health and Science University (OHSU) Strategic Communications social media team, 
we may include advertisements on other platforms for which they have extensive experience running 
campaigns (e.g., Instagram). Following a procedure developed in our previous research,36 we will set ads to 
be presented to Facebook users likely to fit our eligibility criteria. Examples of characteristics we will use 
include: age, military-related interests or behaviors (e.g., making a charitable donation to a Veteran service 
organization, listing the military as their industry or employer), and “liking” Facebook pages related to 
Veterans, suicide prevention, or related topics. Ads will be placed in users’ “News Feeds” and optimized for 
computer and mobile devices. To assure that sensitive information is not sent in an unencrypted email, 
these ads will not invite communication from participants. From these feeds, participants will click on the ad 
and be taken to screening and consent in OHSU REDCap to enter the survey. Ad performance will be 
monitored in real-time, allowing the potential to retain only the most cost-effective ads in the campaign. The 
research team will collaborate closely with the OHSU Strategic Communications Social Media team to host 
ads, and monitor, audit and report on interval progress of the communication campaign. OHSU REDCap 
captures and stores the number of screened and consented participants and stores this information; we will 
be able to view and monitor recruitment progress by logging into REDCap and checking these numbers.  
Based on our preliminary work recruiting Iraq/Afghanistan-era veterans, budget for recruitment, and 
empirical data on the slower pace of recruitment for web-based intervention studies that include close 
supports as participants,34,49 we estimate a 10-week recruitment period to meet our target enrollment of 
200 participants (20 participants/week).  
A subset of approximately 15 participants from the intervention group will be invited to participate in an 
interview. These participants may be veterans or their close supports. For this, a research staff member will 
contact potential participants to determine interest and conduct the interview in a format (telephone, video 
call, or in-person) that is best suited to each research participant. 
 
Intervention Description. We will use PsychArmor S.A.V.E. (see description above in Scientific Rationale 
and Significance section). In brief, the online training is tailored to Veterans and their close supports, is free, 
brief (24 minutes), and includes three scripted role plays. It closely follows the S.A.V.E. model employed by 
the VA by teaching four skills: learning Signs of suicide, Asking about suicidal thoughts, Validating feelings, 
Encouraging help and Expediting treatment.50 
Control Group Description. Consistent with best practices for behavioral interventions, our control group 
accounts for the benefits seen with the passage of time and attention received in the intervention group.51 
The control group will consist of a “sham” training – a different online, video-based training course 
developed by PsychArmor Institute designed for transitioning military service members and their family. 
Topics in this training include information on preparing finances, networking, searching for a job, and using 
educational benefits. The length of the training will be matched to the intervention (24 minutes), and the 
“sham” training also contains other design features similar to the intervention (e.g., motion graphics). By 
using another PsychArmor training, we are able to isolate the difference seen in the control group to be the 
educational content related to S.A.V.E. 
 
Sources of Data. We will use three data sources (website administrative data, survey responses, and 
qualitative interviews) to obtain a mixed methods evaluation of PsychArmor S.A.V.E.  
1. Website administrative data. These include measures of engagement with the web-based training 
in the current study: number of log-ins, enrollments, and completed trainings, as well as time spent 
on the website. These data will only be obtained from participants who are enrolled in the study.  
2. Surveys. Participants will complete surveys online at baseline (before and after the training) and at 
monthly follow-up (1, 2, 3, 4, 5, and 6 months). Surveys contain standard items and scales to collect 
self-reported information and attitudes. This includes sociodemographic characteristics, knowledge, 

7 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
self-efficacy, social norms, stigma, behavioral intentions, use of gatekeeper behaviors, suicidal 
ideation, and help-seeking. See Appendix B for baseline survey. Follow-up surveys will contain a 
subset of baseline survey measures—see Table 2 below for a timeline of measures. 
3. Interviews. A small subgroup of study participants (n = 15) who were assigned to the intervention 
group will be invited to participate in an interview to take place approximately one-month post-
intervention. Interviews will include questions on feasibility and acceptability of the social media ad 
campaign and on PsychArmor S.A.V.E. See Qualitative Data Collection section below for a more 
detailed description and Appendix C for the interview guide. Interviews will be audio-recorded and 
written notes will be recorded by the interviewer. 
 
Potential Difficulties, Limitations, and Alternatives. A potential difficulty is struggling to recruit enough 
interested and eligible participants online. We will monitor recruitment progress on a weekly basis using a 
module in our data management software (OHSU instance of REDCap), which allows real-time monitoring 
of progress against targets. If we do not meet a minimum recruitment milestone of 50 participants in three 
weeks, the research team may consult with a recruitment specialist at the Oregon Clinical and Translation 
Research Institute and OHSU’s Strategic Communications social media team about adding recruitment 
strategies in accordance with NIH guidelines,52 and/or placing advertising on additional social media 
platforms. 
 
Given the 6-month length of follow-up, another potential difficulty is sample retention. To promote sample 
retention we will employ methods with empirical validation in increasing online study response and retention 
rates: 1) minimizing survey burden, 2) specifying survey time estimates, and 4) sending reminder emails, 
texts, and small messages of appreciation to study participants.53 Of note, prior research among military 
personnel suggests that survey retention is higher in web-based surveys than paper-based surveys.54 We 
will also employ modest financial incentives. In the case of incomplete follow-up, reminders will be sent with 
use of email and additional modalities (e.g., text message) as consented to by the participant. 
 
Timetable for study. The project timeline is 
presented in the Table 1. Special 
consideration was made for the time required 
to: develop the social media ad campaign (3 
months), setup study infrastructure including 
REDCap data collection and management 
system and a study website containing 
integration of the two study arms (4 months), 
and conduct recruitment (2.5 months) and 
participant follow-up (6 months). During the 
dissemination and next steps phase, we will 
prepare a grant application for the larger-
scale RCT and prepare the study products 
noted above. 
 
Measures. A timeline of measures is presented in Table 2 and described further below.  
Table 2: Summary of Study Measures 
Timepoint(s) 
Measure 
Main Function  
Aim(s) 
0  
Numbers screened, eligible, and enrolled per week; 
proportion eligible who enroll 
Feasibility 
1 
Table 1 
Year 1 
Year 2 
Quarter 
Activity 
Q
1 
Q
2 
Q
3 
Q
4 
Q
1 
Q
2 
Start-up, IRB, and hiring 
X 
 
 
 
 
 
Social media ad development 
 
X 
 
 
 
 
Study infrastructure setup 
 
X 
X 
 
 
 
Online recruitment 
 
 
X 
 
 
 
Baseline & follow-up 
assessments 
 
 
 
X 
X 
 
Interview training 
 
 
X 
 
 
 
Interview conduct 
 
 
 
X 
 
 
Qualitative data analyses 
 
 
 
 
X 
X 
Quantitative data analyses 
 
 
 
 
X 
X 
Dissemination/Next steps 
 
 
 
 
 
X 

8 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
PT, 1, 2, 3, 4, 
5, 6 
Retention rate; survey completion rate; exposure 
Feasibility 
1 
PT 
Completion of training; satisfaction; usability 
Acceptability 
2 
0 
Background characteristics 
Descriptive data 
1, 2  
0, PT, 2, 4, 6 
Intention to use of gatekeeper behaviors 
Mechanisms of action  
2 
0, PT, 2, 4, 6 
Knowledge; stigma; self-efficacy; social norms 
Mechanisms of action  
2 
0, PT, 1, 2, 3, 
4, 5, 6 
Gatekeeper behaviors; suicidal ideation; help-
seeking, social support 
Measurement characteristics 
2 
1  
Semi-structured interview about social media ads and 
PsychArmor S.A.V.E. 
Feasibility and acceptability 
1, 2 
0=pre-training (baseline); PT=post-training (immediately after); 1=1-month follow-up, 2=2-month follow-up, 
etc. 
Feasibility: We have selected feasibility measures most appropriate for a pilot intervention and previously 
used to assess online interventions.55,56 These measures address each stage of the project including 
screening (number screened/eligible per week), recruitment (number enrolled per week), randomization 
(proportion of eligible who enroll), cross-contamination (number of control group participants who report 
taking PsychArmor S.A.V.E. training), retention (i.e., loss to follow-up), and assessment process (survey 
completion rates).  
Acceptability: For acceptability, we will use a combination of three measures: completion of the training, 
satisfaction (measured using the validated eight-item Client Satisfaction Questionnaire (CSQ-8),57 and the 
System Usability Scale (SUS) along with a single item usability rating that is highly correlated (r=0.82) with 
the SUS.58,59 
Background Characteristics: This will include standard items on age, gender, race/ethnicity, education, 
marital status, zip code (rurality), occupation, military experiences, social media platform use, frequency of 
social contact with Veterans, social network size, and VA and other health service use. 
Intentions to Use Gatekeeper Behaviors: Suicide prevention training research has shown that behavioral 
intention has a direct effect on actual behavior and also mediates the relationship between attitudes and 
actual behaviors.60 Thus, we have chosen a three-item scale to assess behavioral intentions to use core 
skills learned in gatekeeper training (recognizing gatekeeper opportunities, inquiring about suicidal ideation, 
and making a professional referral). In research conducted by our research team and others, scales of 
intentions to use gatekeeper behaviors have shown adequate internal reliability.14,60 
Knowledge: The 12-item short form of the Literacy of Suicide Scale (LOSS) measures knowledge about 
suicide. A total literacy score can be calculated as the percent of all correct items.61 
Stigma: The 16-item short form of the Stigma of Suicide Scale (SOSS) measures public stigma towards 
individuals who die by suicide. It shows reliability and validity in community-based samples.62 
Self-Efficacy: A four-item scale we have previously used to assess confidence to use gatekeeper skills. This 
scale has shown adequate internal reliability in our prior trial of gatekeeper training.15 
Social Norms: A two-item scale we have previously used that assesses the degree to which one believes 
that most people do seek (descriptive norm) or should seek (injunctive norm) professional care if having 
suicidal thoughts. Items show adequate internal reliability.14,48 
Gatekeeper Behaviors: Three items that assess actual self-reported frequency of: 1) recognizing 
gatekeeper opportunities (“In the last month, how many people have you encountered that you suspected 
were in an emotional crisis?”); 2) inquiring about suicidal ideation (“…how many times did you directly ask 
such a person if they were having suicidal thoughts?”); and 3) making a professional referral (“…how many 
times did you directly refer such a person to mental health or other professional treatment?”). Items are 
derived from outcome measures in prior gatekeeper training trials.63 
Suicidal Ideation: The Depression Symptom Index Suicidality Subscale (DSI-SS) is a four-item measure of 
suicidal ideation over the past two weeks with strong psychometric properties.64,65 

9 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
Help-Seeking: The Actual Help-Seeking Questionnaire (AHSQ) consists of four dichotomous items asking 
whether in the last month the respondent has sought help for a mental health problem.66 
Social Support: The Patient-Reported Outcomes Measurement Information System (PROMIS) short form – 
Emotional Support 4a is a four-item measure of emotional support.67 
Sample Size Considerations. We first consider whether our proposed sample size (n=200) is sufficient to 
meet our proposed benchmarks for the proportion of individuals who enroll in the study and loss to follow-
up. For the primary analyses in Aim 1, we calculate CI’s for a single proportion to determine whether the 
target benchmarks are within or beyond the CI’s. Correspondingly, we used a confidence approach to 
determine whether our proposed sample size is sufficient to meet our proposed benchmarks.68 Hypothesis 
1.2 is that at least 50% of eligible individuals will enroll in the study. Assuming that 357 people are eligible 
for the study, and 56% enroll (resulting in a sample size of 200 participants), this will give a 95% binomial 
exact confidence interval of 50.7% to 61.2%. The lower bound of this CI does not overlap with our threshold 
for an enrollment rate > 50%. For numbers as low as 300 people eligible for the study, a study enrollment 
rate of at least 56% would be adequate, and with 400 people eligible we would only need 55% to enroll. The 
Hypothesis 1.3 benchmark for loss to follow-up at end of study is < 50%. Assuming that the study loss to 
follow-up rate is 42%, a sample size of 200 gives a 95% binomial exact confidence interval of 35.1% to 
49.2%. The upper bound of this CI does not overlap with our 50% threshold for loss to follow-up.  For a 
sample size as low as 150, we would also need a loss to follow-up rate of at most 40%, and with 250 we 
would need the rate to be at most 42%. 
Aim 2’s hypothesis is that participants will complete on average at least 70% of the PsychArmor S.A.V.E. 
training. Since this hypothesis is restricted to the treatment arm, our proposed sample size is 100. 
Assuming that the mean percentage of time spent on the training is 73% with a standard deviation (SD) of 
15%, a sample size of 100 gives a 95% t-distribution confidence interval of 70.02% to 75.98%. The lower 
bound of this CI does not overlap with our minimum threshold of 70% completion of the training. For a 
sample size as low as 75, we would need a mean of at least 74% with SD = 15% and at least 76% with SD 
= 25%. 
Randomization and Blinding. Block-randomization schemes using random block sizes to assign the two 
study arms will be created in R using the blockrand package (Version 1.3) before study start and pre-loaded 
into REDCap. Following recommendations for blinding in behavioral interventions, participants will be 
blinded by limiting details about the study hypotheses and differences between the study arms.69 Blinding 
will also be achieved because no study personnel are involved in intervention delivery, and the 
biostatistician will be blinded for analyses involving study group comparisons.  
Statistical Analyses. We will examine descriptive statistics and distributions for variables at each of their 
timepoints. Variables will be summarized both with the arms combined and stratified. Differences in 
baseline values between study arms will be tested using t-tests or chi-squared tests. For variables 
measured with multiple items, Cronbach's alpha will be conducted to examine internal reliability, and, when 
necessary, measurement modifications will be made to improve reliability. If missing data meet minimal 
missing data assumptions (i.e., at least missing at random), then multiple imputation or full maximum 
likelihood estimation can be used to improve standard errors compared with analyses based on listwise 
deletion.77 All statistical analyses will be done in R. 
Qualitative Interview Sample Considerations. A small subgroup of study participants (n = 15) who were 
assigned to the intervention group will be invited to participate in an interview to take place approximately 
one-month post-intervention. We will use maximum variation sampling, a purposive sampling strategy which 
increases depth of understanding by selecting a diverse group of individuals who are expected to 
communicate different perspectives.70 Characteristics considered in sampling will be use of gatekeeper 
behaviors, transitioning Veteran vs. close support, rurality, and VHA service use. 

10 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
Qualitative Data Collection. Research staff will set up a time to speak with the participant over an agreed 
upon and VA-approved remote video modality. This includes VA Video Connect, although in specified 
circumstances other video conferencing applications such as Apple Facetime, Facebook Messenger, 
Skype, Zoom, Cisco WebEx, or Google Hangouts may be used. Public facing applications will not be used. 
Participants will be informed that certain third-party video applications may introduce additional privacy 
risks. Research staff will use all available encryption and privacy modes. 
 
Semi-structured qualitative interviews will be used with the goal of providing the qualitative component to 
mixed methods evaluations of the social media ad campaign and PsychArmor S.A.V.E. In the first section of 
the interview, questions will focus on feasibility and acceptability of the social media ad campaign. We will 
probe ad reactions/responses (e.g., “What went through your mind when you saw this Facebook ad?”), ad 
features (e.g., “Who do you think these messages should come from?”), and facilitators and barriers to 
recruitment via social media (e.g., “What can be done with these ads to motivate a person to learn how to 
help Veterans having a crisis?”). In the second section of the interview, questions will focus on PsychArmor 
S.A.V.E. This includes inquiry about its acceptability, usefulness, suggestions for improvement, potential 
modifications to the content (e.g., addition of booster sessions),71 and facilitators and barriers to further 
implementation. A list of interview questions is included in Appendix C. Interviews will last approximately 
one hour and be audio recorded using a handheld VA audio recorder. Audio recordings may also be sent to 
the VHA Salt Lake City (VHASLC) Centralized Transcription Services Program (CTSP) for transcription 
services. 
 
Qualitative Data Analysis: We will employ qualitative content analysis with a directed grounded theory 
approach.71 We have chosen this approach because it will allow us to identify tacit and implicit content in 
transcripts and use our conceptual framework and theory to guide our initial codebook.72 The investigative 
team (Drs. Teo, Karras-Pilato, and Cross) will review the first transcript, discuss general impressions and 
insights, and develop a preliminary codebook. Full coding of the transcripts will then begin with each 
transcript being double coded by team members with at least two years’ experience analyzing qualitative 
data. We will use open coding to add relevant, unexpected, and emergent themes. Conceptual memos to 
track developing themes and parent/child nodes showing the interrelationship among codes will be created 
as necessary. Supervised by the investigative team, the coders will review areas of disagreement in coding, 
develop a consensus, and iteratively refine the codebook as needed. For data management we will use the 
software program Atlas.ti and use an audit trail to document creation and modification of codes.  
Study Endpoints. The study will conclude with an assessment of feasibility and acceptability based on a 
mixed methods analysis of the quantitative (200 surveys, measures in Table 2) and qualitative data (15 
interviews) collected. This will contribute towards the design of a larger RCT and may contribute to 
submission of journal articles for dissemination of findings.  
 
Study Population 
 
Eligibility Criteria. All potential participants will be identified through the targeted social media advertising 
campaign. After a potential participant clicks on an ad, they will complete a screening that will take place 
prior to consent. Half of the participants will be randomized to the intervention and half to the control 
condition. The eligibility criteria are presented in Table 3 below. Through the targeted social media 
advertisement campaign, we will recruit recently transitioned veterans (active duty within past 12 months) 
who have at least weekly social contact with another veteran, and we will recruit friends or family of 
veterans who have at least weekly social contact with a veteran. Potential participants will be excluded for 
lacking necessary means of contact for follow-up, lacking fluency in English needed because the 

11 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
intervention is delivered in English, having already been exposed to the intervention, or misrepresenting 
their Veteran status.  
Table 3: Eligibility Criteria 
Inclusion Criteria 
Exclusion Criteria 
1a) Veteran who has served on active duty 
in the U.S. Armed Forces within the prior 12 
months, or 1b) family member or friend of a 
Veteran 
1) lack a U.S. phone number, email, or computer access 
2) not fluent in English 
3) have previously taken or intend to take PsychArmor 
S.A.V.E. training (to minimize contamination of the control 
group) 
4) duplicate study entry or misrepresentation of Veteran 
status.73 
2) social contact with a Veteran at least 
weekly 
 
Number of Participants. In order to obtain a final study sample of 200 participants (half randomized to the 
intervention and half to the control condition), we anticipate consenting and enrolling a higher number of 
participants as shown below: 
 
Number of intervention group participants consented: 400 
Number of participants enrolled (assumes 50% enroll): 200 
Number of participants analyzed (assumes 50% lost to follow-up): 100 
 
Number of control group participants consented: 400 
Number of participants enrolled (assumes 50% enroll): 200 
Number of participants analyzed (assumes 50% lost to follow-up): 100 
 
Number of total participants consented: 800 
Number of total participants enrolled (assumes 50% enroll): 400 
Number of total participants analyzed (assumes 50% lost to follow-up): 200 
 
Exclusion of Children. In addition to the above exclusion criteria, we will also exclude children under age 
18 years. Children would not qualify as Veterans and would typically not be considered a close support to 
Veterans due to their young age. The primary focus of this study is looking at the outreach and uptake of 
online training in an adult population that can be difficult to reach though other means (for example, 
Veterans that don’t use VHA services for their health care) and are at risk of depression and suicide. 
Although children can suffer from depression and suicide, the proposed methods of this study (an online 
platform) may not adequately address the sensitive nature of working with children under 18 that are at risk 
for suicide. Additionally, the military-focused gatekeeper training contained in this study has not been 
studied or tailored to a younger audience.  
 
Inclusion of Women and Minorities. People of any gender, race, or ethnicity are invited to participate in 
our study. Based on the demographic characteristics of participants in a preliminary study in which we used 
similar recruitment strategies as will be employed in the current project, we anticipate a higher proportion of 
minority and women participants, compared to the national Veteran population. Our communication 
campaign messages and recruitment materials will be developed with readability and low literacy principles 
in mind. However, study materials will only be available in English, which may lead to an under-
representation of some ethnic groups. The resources needed to adequately translate and validate study 
materials into other languages are outside of the scope of this developmental grant. We anticipate that we 
will have approximately the same proportion of minority participants in the study as there are in the 
populations they are drawn from. If we find that we are falling below recruitment targets for minority 
participants, we will make special efforts to increase minority recruitment, which is relatively easy to 

12 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
accomplish through advertisement targeting features available through our recruitment modality (social 
media advertisements).  
 
For our qualitative interviews, we anticipate that the proportion of minority participants in the study will be 
similar to the local environment surrounding the primary study site, Portland, Oregon. In the state of 
Oregon, 9% of Veterans are ethnic minorities and 8% are women. Demographic characteristics of Veterans’ 
close supports is not well known. We anticipate that the proportion of minorities among close supports will 
match that among Veterans, while the proportion of women is expected to be significantly higher given that 
Veterans’ spouses are usually of the opposite gender. 
 
VA-SPECIFIC REQUIREMENT (inclusion of non-Veterans):  
 
Inclusion of non-Veterans. PsychArmor S.A.V.E. is a gatekeeper training geared towards suicide 
prevention in a veteran population. The intended audience for this intervention includes not only Veterans 
but also their close supports, who may be non-Veteran friends or family members. To understand the 
effectiveness of this intervention, when delivered to non-Veteran close supports of Veterans, it will be 
necessary to include non-Veterans in the study.  
 
Subject Identification/Recruitment 
 
Recruitment via Social Media. We will recruit participants through a targeted social media advertisement 
campaign. We will use Facebook as our primary platform for recruitment; however, in consultation with our 
partners including the OHSU Strategic Communications social media team, we may include advertisements 
on other platforms for which they have extensive experience running campaigns (e.g., Instagram).  
Following a procedure developed in our previous research,36 we will set ads to be presented to Facebook 
users likely to fit our eligibility criteria. Examples of characteristics we will use include: age, military-related 
interests or behaviors (e.g., making a charitable donation to a Veteran service organization, listing the 
military as their industry or employer), and “liking” Facebook pages related to Veterans, suicide prevention, 
or related topics. Ads will be placed in users’ “News Feeds” and optimized for computer and mobile devices. 
Ad performance will be monitored in real-time, allowing the potential to retain only the most cost-effective 
ads in the campaign. The research team will collaborate closely with the OHSU Strategic Communications 
Social Media team to host ads, and monitor, audit and report on interval progress. We will monitor 
recruitment progress on a weekly basis using a module in our data management software (OHSU instance 
of REDCap), which allows real-time monitoring of progress against targets. 
 
Based on our preliminary work recruiting Iraq/Afghanistan-era veterans, budget for recruitment, and 
empirical data on the slower pace of recruitment for web-based intervention studies that include close 
supports as participants,34,49 we estimate a 10-week recruitment period to meet our target enrollment of 
200 participants (20 participants/week). We will monitor recruitment progress on a weekly basis using a 
module in our data management software (OHSU instance of REDCap), which allows real-time monitoring 
of progress against targets. If we do meet a minimum recruitment milestone of 50 participants in three 
weeks, the research team may consult with the Oregon Clinical and Translation Research Institute and 
OHSU’s Strategic Communications social media team about adding recruitment strategies in accordance 
with NIH guidelines52 and/or placing advertising on additional social media platforms.  
After interested individuals click on an ad, they will be brought to a study website landing page. This website 
will be hosted and managed through OHSU REDCap, which allows study participants to access materials 
from any computer and stores data on a server at OHSU, separate, secure and independent of Facebook. 
Prior to consent, they will complete a brief screener to determine study eligibility (see Appendix B, submitted 

13 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
for review). No identifying information will be collected as a part of the screener. If they are eligible, they will 
then complete an online informed consent process before being randomized into one of the two conditions. 
Materials for Recruitment via Social Media. As Aim 1 of our study includes determining the feasibility of 
recruiting participants via social media advertisements, a part of the study will involve the development and 
evaluation of the social media advertisements we will use for recruitment. We are planning three months for 
the development of our social media ads as seen in the timeline presented in Table 1 in the Research 
Design and Methods section. In Appendix E we have included a pool of potential text and images that we 
will use in creating our social media ads for recruitment.   
 
Informed Consent & HIPAA Authorization 
 
 VAPORHCS-SPECIFIC REQUIREMENT (inclusion of those with impaired decision-
making capacity): 
 N/A. The study does not include those with impaired decision-making capacity 
  
We will conduct an informed consent process electronically using OHSU’s instance of REDCap. Potential 
participants who click on a social media-based study advertisement will be redirected to the study website 
where they will complete a brief screener. No identifying information until after participants complete study 
screening.  If they are eligible to participate, they will be provided with a study information sheet explaining 
the purpose of the study, procedures, risks, benefits, and alternatives (Appendix A). If they wish to 
participate, the participant will then click on a box to indicate consent to the study. Documentation of 
electronic consent will be captured and stored in the secure REDCap database. No study procedures will 
occur prior to obtaining consent from the participants.  
 
As both study recruitment and delivery of the intervention occurs online, conducting written informed 
consent would not be practical to achieve the study aims. In addition, sending and receiving a written 
informed consent document containing participants’ names would add risk to study participation. Given the 
online nature of the study and that it involves no more than minimal risk to participants, we are submitting a 
request for waiver of informed consent documentation and waiver of authorization. This waiver or alteration 
will not adversely affect the rights and welfare of the participants as the study will yield only generalizable 
results and will not impact individual welfare.  
For the qualitative interviews, before the interview begins, we will obtain and record a verbal consent for 
recording.  
 
Risks and Side Effects: 
     
 
Potential Risk. As the intervention in this study does not include direct contact with participants, this study 
is highly unlikely to involve risks to participants. Nonetheless, it will be the responsibility of the project 
manager to monitor the progress of individual participants and report any concerns to the PI during regular 
study meetings. Interviewers and anyone else who has contact with study participants during study activities 
have the responsibility to monitor for any potential safety issues and report any concerns to the PI. The 
balance of risk and benefit will be continuously monitored by the PI, and the study may be modified or 
terminated if risks begin to outweigh benefits. 
 
Participation includes completion of an online training followed by intermittent longitudinal observations 
(surveys and, for a subgroup of participants, an interview). The risks associated with participation in this 
study are minimal. The primary risk is breach of confidentiality via inadvertent disclosure of personal health 

14 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
information (PHI). For study activities involving direct contact with participants (qualitative interviews), the 
risk is also minimal due to the nature of the interaction. The primary risk is discomfort during the interview. 
 
Participants may experience discomfort during interviews or surveys while discussing sensitive topics such 
as emotional problems, suicidal ideation, and training in suicide prevention skills. Numerous studies have 
established that inquiring about suicide does not induce suicidal ideation, whether in adults or adolescents, 
or in general or at-risk populations. On the contrary, research suggests that asking study participants about 
suicidal thoughts may in fact reduce suicidal ideation, and lead to improvements in mental health 
in treatment-seeking. 
 
Protection Against Risk. Participant confidentiality will be maintained in accordance with HIPAA 
regulations. Numerous efforts will be made to minimize the likelihood of the risk of a breach of 
confidentiality. The project manager will conduct a review of study files every six months to assure 
compliance with approved procedures. The research team will work closely with the VAPORHCS 
Information Security Officer and Privacy  Officer to ensure that any data transfer, storage, and handling by 
non-VA entities adhere to VA security policies. 
 
The confidentiality of participants will be protected in the following ways:  
● Unique identifiers will be assigned to all participants at the time of electronic consent and kept 
consistent for the duration of the study.  
● Surveys, administrative data, and all other measures related to individual participants will contain the 
ID number instead of names or other PHI.  
● A master list linking names and ID numbers will be stored in a password-protected electronic file, 
located behind the VA firewall on a secure server or in locked file cabinets, separate from other 
study data and accessible only to IRB-approved research personnel.  
● With the exception of zip codes needed to assess rurality, all identifiable information will be removed 
from datasets prior to statistical analyses. We will analyze and report participant data in aggregate 
form only, and no PHI will be reported for any participant.  
● Access to study data will be limited to IRB-approved study personnel and those with regulatory 
oversight responsibilities, and all study staff will complete the required research ethics, data security, 
and HIPAA training within the prior 12 months.  
● Any hard copies of data will be stored in locked file cabinets in locked rooms.  
● No study data will be stored on laptops unless they meet data security policies.  
● The study data will be managed and stored in OHSU’s instance of REDCap, a highly secure and 
robust web-based research and data collection and management system, and in a password and 
firewall protected network drive on the secure (Veterans Health Administration) VHA network 
accessible only to study staff.  
 
Access to the REDCap database will be limited to IRB-approved study staff. REDCap is a secure, 
customizable, web-based application for building and managing HIPAA compliant databases. REDCap 
resides on a server housed in ITG’s Advanced Computing Center (ACC), providing locked physical security. 
The servers are maintained by ITG Database Administrators and Systems Administrators in accordance 
with all relevant OHSU policies and guidelines. REDCap is hosted on ACC servers and maintained by 
developers in the Oregon Clinical and Translational Research Institute (OCTRI) in accordance with all 
relevant OHSU policies and guidelines. Additionally, ACC employs a second firewall within the OHSU 
network to attain a high level of security and access control. All web-based data transmissions are 
encrypted with industry-standard SSL methods. The ACC’s architecture has been reviewed by the OHSU 
Office of Integrity and undergoes periodic internal security audits.  
 

15 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
REDCap employs a robust multi-level security system that enables researchers to easily implement 
"minimum necessary" data access for their research staff, including specification of data fields that are 
identifiers. This feature includes “single click” ability to provide completely deidentified (removing all 
identified data fields and shifting dates) for analysis or other purposes.  User activities are logged to enable 
auditing of all data access.   Access is integrated with OHSU's network such that users who are also OHSU 
employees are authenticated against their OHSU network credentials.   
 
REDCap is jointly managed in accordance with OHSU Information Security Directives by ACC staff and 
members of OCTRI's Biomedical Informatics Program, ensuring fidelity of database configuration and back-
ups. User activities are logged to enable auditing of all data changes. 
 
With the exception of ZIP Codes needed to assess rurality, all other identifiers will be removed from the 
dataset prior to export from REDCap. Data containing the ZIP Code identifiers may be stored on OHSU Box 
(OHSU.box.com), an institution supported, cloud-based file sharing and storage option with added 
protections in place for OHSU confidential and restricted data or protected health information. Box.com 
adheres to the highest industry standards for security and is compliant with HIPAA requirements as well as 
federal government agency requirements such as the Department of Defense. All files uploaded to Box are 
encrypted at rest using 256-bit AES encryption, and further protected by an encryption key-wrapping 
strategy. The company also offers customer-managed encryption keys. Box provides you with in-depth, 
unchangeable audit logs, user permission options and restrictions, and organization-wide controls and 
permissions. OHSU and Box have entered into a Business Associate Agreement, including a data use 
agreement, that meets all OHSU privacy, confidentiality, and information technology policy compliance 
requirements. This includes requirements for software applications to use specific login IDs, passwords, 
OHSU-approved encryption, audit software, and reporting requirements to privacy officers when becoming 
aware of any users not in compliance. 
 
Communications with participants will include invitations and reminders for follow-up surveys, reminders 
and notifications about incentive payments, and for a subset of participants, invitations to participate in an 
interview and the interview itself (Appendix D). Study invitations and reminders will be sent by email or 
SMS text message to study participants. For these communications, we will take special precautions to 
preserve privacy, confidentiality, and data security. These communications will not contain PII/PHI. Any 
email communication that does contain PII/PHI will be encrypted and sent via Azure Rights Management 
Services (RMS).  
 
For the qualitative interviews, individual interviews will be digitally audio-recorded using a handheld VA 
audio recorder. After the interviews, digital recordings will be uploaded on a password secured server 
behind the VA firewall. Recordings will then be deleted from the audio recorder. Audio recordings will not be 
destroyed since they will contain the verbal consent for the recorded interviews. Filenames of digital 
recordings will not include participant names; only a unique study ID number will be listed in filenames. 
Transcriptions will be de-identified during the transcription process by personnel who have training and 
experience in transcription. Audio recordings may be sent to the VHA Salt Lake City (VHASLC) Centralized 
Transcription Services Program (CTSP) for transcription services. Only study personnel will have access to 
de-identified transcripts. Any hardcopies made of transcripts will be kept in locked files in investigators’ 
offices. We will ensure that any quotations used in manuscripts or reports do not contain identifying 
information.  Upon completion of the study, all audio files and documents containing participants’ names 
and other identifying information will be destroyed. 
 
Additional Participant Safeguards: 

16 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
 N/A. The study does not include vulnerable populations and/or those with impaired decision-
making capacity. 
 
VAPORHCS-SPECIFIC REQUIREMENT (inclusion of those with impaired decision-
making capacity): 
 N/A. The study does not include those with impaired decision-making capacity  
 
Suicidality: 
 
We expect some participants to indicate symptoms of active suicidal ideation on our self-report 
survey instrument that assesses for suicidal ideation, the Depressive Symptom Inventory – Suicidal 
Subscale (DSI-SS). Due to the online format of this study, it is not feasible for the PI or research staff to 
locate and respond in “real-time” to positive responses to survey items related to suicidality, or provide 
individualized follow-up if concerns are raised by survey responses. Suicidal ideation and suicide risk are by 
their very nature, fluid.  
 
Because of this, we have opted to provide resources at multiple time points in the study, including the time 
of consent, and any time in which we would become aware of a subject’s positive suicide screen, i.e., when 
the person is completing the survey instrument (Appendix B). Providing resources at that time, rather than 
conducting risk assessment hours if not days later, is preferable in the context of the current study 
design. Resources will include information on online, phone, and text messaging-based mental health 
treatment resources. They will include options for crisis situations, non-urgent treatment referral, and—as 
we will not be fully aware of the location of respondents—ways to locate local support and treatment 
resources. Our study team will not provide therapy to directly treat depression or other mental health 
symptoms for participants who experience distress over the course of the study. We are not limiting 
participants’ engagement with mental health providers in any way.  
 
If a participant indicates they are suicidal during an interview, a warm transfer will be initiated. The 
interviewer will ascertain that the participant will be available by phone while calling the warm transfer 
number for the VA National Suicide Hotline (503-393-7938). They will provide the participant’s name and 
telephone number to the VA National Suicide Prevention Hotline. 
 
Benefits:  
 
Potential benefits to the individual include the possible acquisition of gatekeeper training skills in suicide 
prevention for those in the intervention condition. All individuals will also receive information on mental 
health resources.  
 
This study includes two key potentially important impacts to knowledge of suicide prevention in Veterans. 
The first impact is the feasibility of the use of social media to provide outreach to Veterans, particularly 
those who are not VHA connected, and their family and friends. The second impact is the knowledge on the 
feasibility and acceptability of online training that provides the trainee with the skills to help individuals at 
risk of suicide. We believe that the knowledge to be gained and the potential benefits from the proposed 
study outweigh the associated risks.  
 
Protected Health Information:  
     

17 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
 
In terms of the identifiers included, these are the minimum necessary in order to maintain contact with 
consented research participants during the course of the study; identifiers will be exported and removed 
from REDCap once study activities are complete Contact information including names, phone numbers, and 
email addresses is necessary to send them follow-up survey invitations, reminders, and information about 
the study. We will also collect zip codes as a variable of interest reflecting rurality. Because participation is 
remote, we may need additional identifying information to be able to provide payment for participation. 
Depending on the mode of payment preferred by the participant, we may additionally need to collect mailing 
addresses, social security numbers, and/or banking account information in order to provide payment. This 
information will not be recorded as part of the study record or stored with any health information. 
Participants will be informed of the risk of providing this information and that it would be optional but that we 
may not be able to pay them without it. 
 
Health information collected directly corresponds to our research aims. As this study is related to suicide 
risk, suicide prevention, and seeking help for mental health issues, data collected from participants may 
relate to these topics (see Appendix B for survey measures and Appendix C for the interview guide). 
 
Collaborative Research 
 
All research work will be conducted at VAPORHCS by VA staff on VA time using VA equipment, with the 
following exceptions related to data collection software and collaborators located outside VAPORHCS.  
● OHSU’s instance of REDCap, housed on secure servers at OHSU, will be used to collect data, 
store data, and communicate with participants for follow-up. While we will consult with the Oregon 
Clinical and Translational Research Institute (OCTRI) for assistance in building the survey in 
REDCap, all recruitment, data collection, and communication done through REDCap will be 
accessed by VA staff on VA time using VA equipment. OHSU will have ownership of research data 
for this research study.   
● Wendi Cross, PhD, Co-Investigator. Dr. Cross is the Director of Clinical Psychology Training and a 
Researcher at the VA’s VISN 2 Center of Excellence for Suicide Prevention. She will attend 
research team meetings, provide input and oversight on evaluation of PsychArmor S.A.V.E. in our 
randomized pilot study, and participate in manuscript-writing and other dissemination activities. She 
will be paid as an IPA. 
● Elizabeth Karras-Pilato, PhD, Co-Investigator. Dr. Karras-Pilato is the Co-Director for Research 
with the Center of Excellence for Suicide Prevention at the Canandaigua VA Medical Center. She 
will travel to Portland, OR for a workshop to develop messages for the social media ad campaign (or 
do so remotely if travel is not feasible). She will also attend research team meetings, provide input 
and oversight on development of messages for use in the social media advertisement campaign, 
and participate in manuscript-writing and other dissemination activities. Effort is limited to Year 1 
when the social media ad campaign will occur. 
● Carie Rodgers, PhD, Consultant. Dr. Rodgers is the Education Director at the PsychArmor 
Institute and is a Clinical Professor of Psychiatry at the University of California San Diego School of 
Medicine. She will act as liaison between PsychArmor Institute and the primary VA site. She will be 
involved with and provide consultation for the development of a reusable web-based interface and 
clone of PsychArmor S.A.V.E. for use in this project. She will provide consultations for the project 
without compensation.  
● Stephanie Gamble, PhD, Consultant. Dr. Gamble is the Deputy Director for the Center of 
Excellence for Suicide Prevention at the Canandaigua VA Medical Center and an Associate 
Professor in the Department of Psychiatry at the University of Rochester Medical Center. She will 

18 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
provide input regarding the development, preparation, and evaluations of the trainings used in the 
intervention and control arms.  
VA Canandaigua will be notified of the research through the JIT process since they will receive funding. 
The PI will notify the local directors of other sites if engagement of their site is no longer required. 
 
Resources Available 
VA-SPECIFIC REQUIREMENT:    
 
The study will take place at the VA Portland Health Care System (VAPORHCS) in Portland, Oregon. OHSU 
will be the coordinating center for the study. The VAPORHCS HSR&D Center of Innovation is the Center to 
Improve Veteran Involvement in Care (CIVIC) is where the science of this research study will take place, in 
collaboration with other CIVIC investigators, staff, and the CIVIC Veteran Engagement Group.  
  
Most project personnel will be located at the Center to Improve Veteran Involvement in Care (CIVIC). CIVIC 
will provide office space, equipment, and administrative research support to project staff. Dr. Teo and his 
research staff retain private office space in CIVIC. CIVIC has approximately 4,500 square feet of office 
space, which includes the second and third floors of VAPORHCS Building 6. This contiguous space allows 
extraordinary opportunities for collaboration and synergy. All necessary equipment including computers, 
printers, copiers, and scanners are available to the research team. Dr. Teo will devote 2/8 time to this study, 
and his study coordinator will be devoting 8/8 time to this study. 
 
Subject Compensation/Payment: 
 
Participants will receive compensation because they are taking time out of their day to complete the 
surveys, intervention training, and interviews. All participants who complete their follow-up surveys will 
receive compensation valued at $20 and an additional $20 at the completion of the study. The subset of 15 
participants who complete an interview will also receive compensation valued at $25 as a thank you for their 
time. Participants may be paid according to their preferences via cash (in the case of any local participants 
of interviews who may choose to come in), Electronic Funds Transfer, mailed check, ClinCard, gift card, or 
other allowable VA payment method suitable for paying participants remotely. These compensation terms 
are included in the information sheet provided during consent. Additional identifiers may be collected for 
payment purposes; depending on payment method, identifiers may be stored in REDCap, VA server only 
accessible to the study team, or on a secure payment site (such as ClinCard). 
 
The compensation offered is a fair amount for the time and effort involved for each section of study 
participation. It is enough to encourage completion and is not so large as to constitute undue pressure or 
influence. Compensation will not be prorated for participants who do not complete the study because the 
cost is low and our sample large, thus making prorated payments difficult to manage for a pilot project. 
 
Privacy and Confidentiality: 
 
Privacy and confidentiality will be maintained in accordance with HIPAA regulations. Numerous efforts will 
be made to minimize the likelihood of the risk of a breach of confidentiality. The project manager will 
conduct a review of study files every six months to assure compliance with approved procedures. The 
research team will work closely with the VAPORHCS Information Security Officer and Privacy  Officer to 
ensure that any data transfer, storage, and handling by non-VA entities adhere to VA security policies. 
 
The confidentiality of participants will be protected in the following ways:  

19 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
● Unique identifiers will be assigned to all participants at the time of electronic consent and kept 
consistent for the duration of the study.  
● Surveys, administrative data, and all other measures related to individual participants will contain the 
ID number instead of names or other PHI.  
● A master list linking names and ID numbers will be stored in a password-protected electronic file, 
located behind the VA firewall on a secure server or in locked file cabinets, separate from other 
study data and accessible only to IRB-approved research personnel.  
● With the exception of zip codes needed to assess rurality, all identifiable information will be removed 
from datasets prior to statistical analyses. We will analyze and report participant data in aggregate 
form only, and no PHI will be reported for any participant.  
● Access to study data will be limited to IRB-approved study personnel and those with regulatory 
oversight responsibilities, and all study staff will complete the required research ethics, data security, 
and HIPAA training within the prior 12 months.  
● Any hard copies of data will be stored in locked file cabinets in locked rooms.  
● No study data will be stored on laptops unless they meet data security policies.  
● The study data will be managed and stored in OHSU’s instance of REDCap, a highly secure and 
robust web-based research and data collection and management system, and in a password and 
firewall protected network drive on the secure (Veterans Health Administration) VHA network 
accessible only to study staff.  
● Access to the REDCap database will be limited to IRB-approved study staff. See Protection of Risk 
subsection of Risk and Side Effects section for a description of REDCap’s security features.   
 
With the exception of ZIP Codes needed to assess rurality, all other identifiers will be removed from the 
dataset prior to export from REDCap. Data containing the ZIP Code identifiers may be stored on OHSU 
Box, an institution supported, cloud-based file sharing and storage option with added protections in place for 
OHSU confidential and restricted data or protected health information. Box.com adheres to the highest 
industry standards for security and is compliant with HIPAA requirements as well as federal government 
agency requirements such as the Department of Defense. All files uploaded to Box are encrypted at rest 
using 256-bit AES encryption, and further protected by an encryption key-wrapping strategy. The company 
also offers customer-managed encryption keys. Box provides you with in-depth, unchangeable audit logs, 
user permission options and restrictions, and organization-wide controls and permissions. OHSU and Box 
have entered into a Business Associate Agreement, including a data use agreement, that meets all OHSU 
privacy, confidentiality, and information technology policy compliance requirements. This includes 
requirements for software applications to use specific login IDs, passwords, OHSU-approved encryption, 
audit software, and reporting requirements to privacy officers when becoming aware of any users not in 
compliance. Only study personnel with VAPORHCS appointments will work with identifiable data. 
  
Any transfer of data containing identifiable information between the VA network and Box.com will utilize 
encrypted email or encrypted flash drives.  
 
For communication with participants, we will take special precautions to preserve privacy, confidentiality, 
and data security. Personal email accounts will not be used for research communications with participants. 
These communications will not contain PII/PHI. Any email communication that does contain PII/PHI will be 
encrypted and sent via Azure Rights Management Services (RMS).  
 
For the qualitative interviews, individual interviews will be digitally audio-recorded using a handheld VA 
audio recorder. After the interviews, digital recordings will be uploaded on a password secured server 
behind the VA firewall. Recordings will then be deleted from the audio recorder. Audio recordings will not be 
destroyed since they will contain the verbal consent for the recorded interviews. Filenames of digital 
recordings will not include participant names; only a unique study ID number will be listed in filenames. 

20 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
Transcriptions will be de-identified during the transcription process by personnel who have training and 
experience in transcription. Audio recordings may be sent to the VHA Salt Lake City (VHASLC) Centralized 
Transcription Services Program (CTSP) for transcription services. Only study personnel will have access to 
de-identified transcripts. Any hardcopies made of transcripts will be kept in locked files in investigators’ 
offices. We will ensure that any quotations used in manuscripts or reports do not contain identifying 
information. Upon completion of the study, all audio files and documents containing participants’ names and 
other identifying information will be destroyed. 
 
Certificate of Confidentiality 
N/A. The study does not include a Certificate of Confidentiality. 
 
Information and/or Specimen Management 
 
Unique identifiers will be assigned to all participants at the time of electronic consent and kept consistent for 
the duration of the study. Surveys, administrative data, and all other measures related to individual 
participants will contain the ID number instead of names or other PHI. A master list linking names and ID 
numbers will be stored in a password-protected electronic file, located behind the VA firewall on a secure 
server or in locked file cabinets, separate from other study data and accessible only to IRB-approved 
research personnel.  
 
Survey data will be captured and stored in REDCap, which employs a robust multi-level security system that 
enables researchers to easily implement "minimum necessary" data access for their research staff, 
including specification of data fields that are identifiers. This feature includes a “single click” ability to 
provide completely deidentified (removing all identified data fields and shifting dates) for analysis or other 
purposes. All HIPAA identifiers contained in our dataset except for ZIP Code, used to assess rurality, will be 
removed prior to data analysis. Other identifiable information that was collected, such as names and email 
addresses will only be accessed, separate from any health data, to send out invitations and reminders for 
follow-up surveys and interviews.  
 
For the qualitative interviews, individual interviews will be digitally audio-recorded using a handheld VA 
audio recorder. After the interviews, digital recordings will be uploaded on a password secured server 
behind the VA firewall. Recordings will then be deleted from the audio recorder. Audio recordings will not be 
destroyed since they will contain the verbal consent for the recorded interviews. Filenames of digital 
recordings will not include participant names; only a unique study ID number will be listed in filenames. 
Transcriptions will be de-identified during the transcription process by personnel who have training and 
experience in transcription. Audio recordings may be sent to the VHA Salt Lake City (VHASLC) Centralized 
Transcription Services Program (CTSP) for transcription services. Only study personnel will have access to 
de-identified transcripts. Any hardcopies made of transcripts will be kept in locked files in investigators’ 
offices. We will ensure that any quotations used in manuscripts or reports do not contain identifying 
information. Upon completion of the study, all audio files and documents containing participants’ names and 
other identifying information will be destroyed. 
 
All information linking study data to PHI will be kept within VAPORHCS electronically in secure computer 
files stored behind firewalls requiring password access, or in hardcopy form in locked file cabinets in locked 
offices. Except for ZIP Codes, all patient identifiers will be removed prior to analysis. All investigators and 
team members who will have access to the data will have received appropriate background checks as part 
of hiring and/or credentialing, and will have completed Data Security Training within the prior 12 months. 
 

21 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
Disclosure/Sharing: 
N/A. The study does not include disclosure/sharing outside the IRB-approved VAPORHCS 
study personnel. 
 
Transfer of Data Ownership 
 OHSU will have ownership of research data for this research study.  
 
Web Application(s), Mobile Device(s) and/or Mobile Application(s): 
 
Facebook ads will be used for study recruitment. On Facebook advertising, all consumer data is 
automatically “hashed,” meaning their unique identifiers are stripped before we (the advertisers) gain 
access to the information. The only info we see is basic demographics, geographic info and performance 
data.  
 
After interested individuals click on an ad, they will be brought to a study website landing page. This website 
will be hosted and managed through OHSU’s instance of REDCap,74 a highly secure, web-based 
application that allows study participants to access materials from any computer and stores data on a server 
at OHSU, separate, secure and independent of Facebook. Eligibility questions, consent, questionnaires, 
and presentation of intervention will all be done through REDCap.  
 
REDCap employs a robust multi-level security system that enables researchers to easily implement 
"minimum necessary" data access for their research staff, including specification of data fields that are 
identifiers. User activities are logged to enable auditing of all data access. Access is integrated with OHSU's 
network such that users who are also OHSU employees are authenticated against their OHSU network 
credentials.   
 
For communication with participants, we will take special precautions to preserve privacy, confidentiality, 
and data security. Personal email accounts will not be used for research communications with participants. 
VA email, email generated through OHSU REDCap, and SMS text messages sent through REDCap may 
be used to provide reminders about survey completion or related general communication about the study. 
REDCap uses a third party web service, Twilio Inc., to send automated SMS text messages. No PII/PHI 
besides the participant’s phone number will be shared with Twilio and participants will be informed in the 
information sheet that their phone number may be shared with Twilio for this purpose. These 
communications will not contain PII/PHI. Any email communication that does contain PII/PHI will be 
encrypted and sent via Azure Rights Management Services (RMS).  
 
Some data containing PHI may be stored on OHSU Box, an institution supported, cloud-based file sharing 
and storage option with added protections in place for OHSU confidential and restricted data or protected 
health information. Box.com adheres to the highest industry standards for security and is compliant with 
HIPAA requirements as well as federal government agency requirements such as the Department of 
Defense. All files uploaded to Box are encrypted at rest using 256-bit AES encryption, and further protected 
by an encryption key-wrapping strategy. The company also offers customer-managed encryption keys. Box 
provides you with in-depth, unchangeable audit logs, user permission options and restrictions, and 
organization-wide controls and permissions. OHSU and Box have entered into a Business Associate 
Agreement, including a data use agreement, that meets all OHSU privacy, confidentiality, and information 
technology policy compliance requirements. This includes requirements for software applications to use 

22 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
specific login IDs, passwords, OHSU-approved encryption, audit software, and reporting requirements to 
privacy officers when becoming aware of any users not in compliance. 
 
For the qualitative interviews with the subset of 15 participants, research staff will set up a time to speak 
with the participant over an agreed upon and VA-approved remote video modality (i.e. VA Video Connect, 
although in specified circumstances other video conferencing applications such as Apple Facetime, 
Facebook Messenger, Skype, Zoom, Cisco WebEx, or Google Hangouts may be used. Public facing 
applications will not be used). Participants will be informed that certain third-party video applications may 
introduce additional privacy risks. Research staff will use all available encryption and privacy modes. 
 
Data and Safety Monitoring Plan (DSMP)  
     
 
This study does not involve pharmaceuticals or medical devices. Data collection relies on self-report online 
surveys and interviews; no invasive testing or procedures are involved, and the intervention involves no 
direct contact with participants. The risks related to participation are likely much lower than those of a trial 
involving pharmaceuticals, medical devices, or somatic interventions or procedures. Nonetheless, we will 
very carefully monitor the progress of the study and any potential risks or adverse events.  
 
Because the primary risk is a breach in confidentiality, no statistical analysis of safety data is planned. The 
study coordinator will monitor for breaches in confidentiality and problems will be addressed and reported 
according to regulations. Other safety information collected will include self-report information participants 
may tell us while filling out online surveys or, for a subset of 15 participants, through interviews. This data 
will be collected at the time of participation when participants enroll in the study and once a month for six 
months of follow-up surveys. The interviews will occur approximately one month post-intervention. PI Dr. 
Teo holds weekly meetings with his study personnel where they will discuss adverse events and protocol 
deviations associated with this project and ways to reduce repeat occurrences. Research staff will examine 
all cumulative adverse events quarterly to determine if there are any systematic problems and to implement 
protocol corrections as needed after receiving IRB approval. 
 
Monitoring for any potential adverse events and protocol deviations will be performed by research staff that 
oversees data collection activities, including the interviewers or anyone else who has contact with study 
participants during study activities. Dr. Teo will be responsible for ensuring the absence of conflicts of 
interest within the research staff and training them in the responsible conduct of research and reporting 
adverse events. Any adverse event will be reported immediately to PI Dr. Teo, who will contact the 
participant and determine if additional intervention is needed to ensure participant safety. Protocol 
deviations will also be immediately reported to Dr. Teo who will ensure that adverse events deemed to be 
unanticipated problems and protocol deviations are properly reported to the IRB in a timely manner. 
Detailed written documentation will be kept for all adverse events that occur over the course of the study. 
The balance of risk to benefit will be continuously monitored by the PI, and the study may be modified or 
terminated if risks begin to outweigh benefits. 
 
Our data and safety monitoring plan includes adverse event (AE) and serious adverse event (SAE) 
monitoring by all staff working on the project. AEs will be identified by spontaneous reports to any study 
staff member, and all staff will be trained to be receptive to participant complaints and concerns about the 
study. Any breach of confidentiality will be considered an AE. 
 
We will classify an AE as: 
0 - No AE 

23 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
1 - Mild - no treatment needed 
2 - Moderate - resolved with treatment 
3 - Severe - inability to carry on normal activities, required professional medical attention 
4 - Life-threatening or disabling 
5 - Fatal 
 
The PI will review all adverse events to determine if they meet the definition of an Unanticipated Problem 
(UP). All Unanticipated Problems, including AEs that meet the definition of a UP as determined by Dr. Teo, 
including breaches of confidentiality and participant complaints, will be reported to the IRB as soon as 
possible and within the following time frames: 
 
● Deaths and potentially life-threatening events within 7 days of the PI learning of the event days. 
● All other unanticipated problems will be reported within 15 days of the PI learning of the event. 
 
If any AEs or UPs require a change to the protocol or consent form (as determined by the PI or IRB) the PI 
will submit the revised protocol for review and approval by the Joint IRB promptly. A brief summary of UPs 
and all adverse events will be submitted to the IRB at the time of annual continuing review. Adverse events, 
enrollment, and data collection problems will be reviewed quarterly by the PI. In addition, dropouts and AEs 
will be reviewed for patterns indicating problems with the study procedures. Any protocol deviation identified 
during the quarterly reviews will be reported per IRB policy. 
 
Step-by-Step Guidance on Conducting the Study 
 
1. Project Setup 
1.1. Social media campaign (Appendix E) development 
1.1.1. Dr. Karras-Pilato to deliver workshop to develop messages for ad campaign 
1.1.2. Consultation with OHSU Strategic Communications social media team and a Veteran 
engagement representative 
1.1.3. Submit finalized social media ads to IRB  for approval 
1.2. REDCap setup 
1.2.1. Build survey (Appendix B) in REDCap with consultation from Oregon Clinical and 
Translational Research Institute (OCTRI)  
1.2.2. Submit any substantive changes to survey to IRB for approval (if applicable) 
2. Recruitment 
2.1. Implement social media ad campaign on Facebook  
2.1.1. Monitor recruitment pacing through REDCap 
2.2. Interested participants click on a link to connect to the study website on OHSU’s REDCap  
2.3. Interested participants complete electronic eligibility screening 
2.4. Eligible participants will be directed to an information sheet (Appendix A) for electronic informed 
consent 
2.5. Consented participants provide contact information (name, phone number, email address) needed 
for follow-up (if participants choose to withdraw, they will be given the option to stop receiving 
communications of any format from the study team). 
2.6. Participants will be randomly assigned to either the intervention or control group trainings 
3. Intervention 
3.1. Participants will complete the baseline survey (see table 2, reproduced below, for included 
measures) 
3.2. Participants will view either intervention gatekeeper training or control  “sham” training 
3.2.1. Data collected on training completion rates 

24 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
3.3. Participants will immediately complete a follow-up survey (see table 2 for included measures) 
4. Longitudinal survey follow-up 
4.1. Participants will receive a communication (Appendix D) via email and or SMS text messages 
automated through REDCap from the study team 1-2 weeks prior to the time points indicated in 
table 2 to complete the survey responses noted  
4.2. Participants will have the option for compensation to be sent to them via an approved VA modality 
at the conclusion of all surveys  
5. Qualitative interviews 
5.1. Study staff will use maximum variation sampling to select a subset of participants to invite for 
qualitative interview  
5.2. Research staff will set up a time to speak with the participant either in person at VAPORHCS, or 
over an agreed upon and VA-approved remote video modality (i.e. VA Video Connect, although in 
specified circumstances other video conferencing applications such as Apple Facetime, Facebook 
Messenger, Skype, Zoom, Cisco WebEx, or Google Hangouts may be used. Public facing 
applications will not be used).  
5.2.1. Participants will be informed that certain third-party video applications may introduce 
additional privacy risks.  
5.2.2. Research staff will use all available encryption and privacy modes. 
5.3. Informed consent from interview participants will be collected verbally and will be audio recorded  
5.4. Interviews will be audio recorded and will last between 45-60 minutes using the interview guide 
(Appendix C) 
5.5. Participants interviewed in-person will receive compensation at the conclusion of the interview at 
the hospital agent cashier.  
5.6. Participants interviewed remotely will receive compensation at the conclusion of the interview via an 
approved VA modality. 
6. Data analysis 
6.1. Conduct quantitative data analysis of survey data to examine descriptive statistics and distributions 
for variables at each of their timepoints. 
6.2. Conduct qualitative content analysis with a directed grounded theory approach with interview data. 
7. Dissemination Activities 
7.1. Review quantitative and qualitative analysis and develop publications 
7.2. Review data and use to develop future larger RCT 
 
References & Literature Cited 
 
1.  2019 National Veteran Suicide Prevention Annual Report. Office of Mental Health and Suicide 
Prevention; U.S. Department of Veteran Affairs https://www.mentalhealth.va.gov/docs/data-
sheets/2019/2019_National_Veteran_Suicide_Prevention_Annual_Report_508.pdf 
2.  The Action Alliance for Suicide Prevention. The Nation’s Public-Private Partnership for Suicide 
Prevention. Accessed November 19, 2018. https://theactionalliance.org/about-us 
3.  Fischer EP, McSweeney JC, Wright P, et al. Overcoming barriers to sustained engagement in 
mental health care: Perspectives of rural veterans and providers. J Rural Health. 
2016;32(4):429-438. doi:10.1111/jrh.12203 
4.  Drum D, Brownson C, Denmark A, Smith S. New Data on the Nature of Suicidal Crises in 
College Students: Shifting the Paradigm. Professional Psychology: Research and Practice. 
2009;40(3):213-222. doi:10.1037/a0014465 

25 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
5.  Wyman PA, Brown CH, Inman J, et al. Randomized Trial of a Gatekeeper Program for Suicide 
Prevention: 1-year Impact on Secondary School Staff. J Consult Clin Psychol. 
2008;76(1):104-115. doi:10.1037/0022-006X.76.1.104 
6.  Bryan CJ, Cerel J, Bryan AO. Exposure to suicide is associated with increased risk for suicidal 
thoughts and behaviors among National Guard military personnel. Compr Psychiatry. 
2017;77:12-19. doi:10.1016/j.comppsych.2017.05.006 
7.  Cerel J, van de Venne JG, Moore MM, Maple MJ, Flaherty C, Brown MM. Veteran exposure 
to suicide: Prevalence and correlates. J Affect Disord. 2015;179:82-87. 
doi:10.1016/j.jad.2015.03.017 
8.  National Survey Shows Majority of Americans Would Take Action to Prevent Suicide. 
American Foundation for Suicide Prevention. Published September 12, 2018. 
https://afsp.org/harrispoll/ 
9.  Cerel J, Chandler Bolin M, Moore M. Suicide Exposure, Awareness and Attitudes in College 
Students. Advances in Mental Health. 2014;12:46-53. doi:10.5172/jamh.2013.12.1.46 
10.  US. Department of Health and Human Services National Institutes of Health. Drug Abuse 
Prevention: What Works. National Institute on Drug Abuse; 1997. 
11.  Yonemoto N, Kawashima Y, Endo K, Yamada M. Gatekeeper training for suicidal behaviors: A 
systematic review. J Affect Disord. 2018;246:506-514. doi:10.1016/j.jad.2018.12.052 
12.  Lipson SK, Speer N, Brunwasser S, Hahn E, Eisenberg D. Gatekeeper Training and Access 
to Mental Health Care at Universities and Colleges. J Adolesc Health. 2014;55(5):612-619. 
doi:10.1016/j.jadohealth.2014.05.009 
13.  Burnette C, Ramchand R, Ayer L. Gatekeeper training for suicide prevention: A theoretical 
model and review of the empirical literature. Rand Health Q. 2015;5(1):16. doi:PMCID: 
PMC5158249 
14.  Teo AR, Andrea SB, Sakakibara R, Motohara S, Matthieu MM, Fetters MD. Brief gatekeeper 
training for suicide prevention in an ethnic minority population: a controlled intervention. BMC 
Psychiatry. 2016;16:211. doi:10.1186/s12888-016-0924-4 
15.  Litteken C, Sale E. Long-Term Effectiveness of the Question, Persuade, Refer (QPR) Suicide 
Prevention Gatekeeper Training Program: Lessons from Missouri. Community Ment Health J. 
2018;54(3):282-292. doi:10.1007/s10597-017-0158-z 
16.  Matthieu MM, Chen Y, Schohn M, Lantinga LJ, Knox KL. Educational preferences and 
outcomes from suicide prevention training in the Veterans Health Administration: one-year 
follow-up with healthcare employees in Upstate New York. Mil Med. 2009;174(11):1123-1131. 

26 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
17.  Godoy Garraza L, Walrath C, Goldston DB, Reid H, McKeon R. Effect of the Garrett Lee 
Smith Memorial Suicide Prevention Program on Suicide Attempts Among Youths. JAMA 
Psychiatry. 2015;72(11):1143-1149. doi:10.1001/jamapsychiatry.2015.1933 
18.  Walrath C, Garraza LG, Reid H, Goldston DB, McKeon R. Impact of the Garrett Lee Smith 
Youth Suicide Prevention Program on Suicide Mortality. Am J Public Health. 2015;105(5):986-
993. doi:10.2105/AJPH.2014.302496 
19.  Yonemoto N, Kawashima Y, Endo K, Yamada M. Implementation of gatekeeper training 
programs for suicide prevention in Japan: a systematic review. Int J Ment Health Syst. 
2019;13:2. doi:10.1186/s13033-018-0258-3 
20.  Peterson K, Anderson J, Bourne D. Evidence Brief: Suicide Prevention in Veterans. 
Department of Veterans Affairs (US); 2018. Accessed January 29, 2019. 
http://www.ncbi.nlm.nih.gov/books/NBK535971/ 
21.  Mann JJ, Apter A, Bertolote J, et al. Suicide prevention strategies: a systematic review. JAMA. 
2005;294(16):2064-2074. doi:10.1001/jama.294.16.2064 
22.  Zalsman G, Hawton K, Wasserman D, et al. Suicide prevention strategies revisited: 10-year 
systematic review. The Lancet Psychiatry. 2016;3(7):646-659. doi:10.1016/S2215-
0366(16)30030-X 
23.  VA/DoD Clinical Practice Guideline for the Assessment and Management of Patients at Risk 
for Suicide. Department of Veterans Affairs; Department of Defense; 2019. 
https://www.healthquality.va.gov/guidelines/MH/srb/VADoDSuicideRiskFullCPGFinal5088212
019.pdf 
24.  Ghoncheh R, Koot HM, Kerkhof AJFM. Suicide Prevention E-Learning Modules Designed for 
Gatekeepers: A Descriptive Review. Crisis. 2014;35(3):176-185. doi:10.1027/0227-
5910/a000249 
25.  Lancaster PG, Moore JT, Putter SE, et al. Feasibility of a web-based gatekeeper training: 
Implications for suicide prevention. Suicide Life Threat Behav. 2014;44(5):510-523. 
doi:10.1111/sltb.12086 
26.  Ghoncheh R, Gould MS, Twisk JW, Kerkhof AJ, Koot HM. Efficacy of Adolescent Suicide 
Prevention E-Learning Modules for Gatekeepers: A Randomized Controlled Trial. JMIR Ment 
Health. 2016;3(1):e8. doi:10.2196/mental.4614 
27.  Sueki H, Ito J. Suicide Prevention Through Online Gatekeeping Using Search Advertising 
Techniques: A Feasibility Study. Crisis. 2015;36(4):267-273. doi:10.1027/0227-5910/a000322 
28.  Stone DM, Barber CW, Potter L. Public Health Training Online: The National Center for 
Suicide Prevention Training. Am J Prev Med. 2005;29(5 Suppl 2):247-251. 
doi:10.1016/j.amepre.2005.08.019 

27 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
 
 
29.  Wingood GM, DiClemente RJ. The ADAPT-ITT model: a novel method of adapting evidence-
based HIV Interventions. J Acquir Immune Defic Syndr. 2008;47 Suppl 1:S40-46. 
doi:10.1097/QAI.0b013e3181605df1 
30.  Social Media Fact Sheet. Pew Research Center: Internet & Technology. Published February 
5, 2018. http://www.pewinternet.org/fact-sheet/social-media/ 
31.  Stewart JB. Facebook Has 50 Minutes of Your Time Each Day. It Wants More. New York 
Times. May 5, 2016:B1. 
32.  Bush NE, Wheeler WM. Personal Technology Use by U.S. Military Service Members and 
Veterans: An Update. Telemedicine and e-Health. 2015;21(4):245-258. 
doi:10.1089/tmj.2014.0100 
33.  Akers L, Gordon JS. Using Facebook for Large-Scale Online Randomized Clinical Trial 
Recruitment: Effective Advertising Strategies. J Med Internet Res. 2018;20(11):e290. 
doi:10.2196/jmir.9372 
34.  Teo AR, Liebow SB, Chan B, Dobscha SK, Graham AL. Reaching those at risk for psychiatric 
disorders and suicidal ideation: Facebook advertisements to recruit military veterans. JMIR 
Mental Health. 2018;5(3):e10078. doi:10.2196/10078 
35.  Cross WF, West JC, Pisani AR, et al. A randomized controlled trial of suicide prevention 
training for primary care providers: a study protocol. BMC Med Educ. 2019;19(1):58. 
doi:10.1186/s12909-019-1482-5 
36.  Cross WF, Seaburn D, Gibbs D, Schmeelk-Cone K, White AM, Caine ED. Does practice make 
perfect? A randomized control trial of behavioral rehearsal on suicide prevention gatekeeper 
skills. J Prim Prev. 2011;32(3-4):195-211. doi:10.1007/s10935-011-0250-z 
37.  Cross W, Matthieu MM, Lezine D, Knox KL. Does a brief suicide prevention gatekeeper 
training program enhance observed skills? Crisis. 2010;31(3):149-159. doi:10.1027/0227-
5910/a000014 
38.  Cross W, Matthieu MM, Cerel J, Knox KL. Proximate outcomes of gatekeeper training for 
suicide prevention in the workplace. Suicide Life Threat Behav. 2007;37(6):659-670. 
doi:10.1521/suli.2007.37.6.659 
39.  Matthieu MM, Cross W, Batres AR, Flora CM, Knox KL. Evaluation of gatekeeper training for 
suicide prevention in veterans. Arch Suicide Res. 2008;12(2):148-154. 
doi:10.1080/13811110701857491 
40.  Dobscha SK, Denneson LM, Kovas AE, et al. Correlates of suicide among veterans treated in 
primary care: case-control study of a nationally representative sample. J Gen Intern Med. 
2014;29 Suppl 4:853-860. doi:10.1007/s11606-014-3028-1 

28 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
41.  Iovine-Wong PE, Nichols-Hadeed C, Thompson Stone J, et al. Intimate Partner Violence, 
Suicide, and Their Overlapping Risk in Women Veterans: A Review of the Literature. Military 
Medicine. 2019;184(5-6):e201-e210. doi:10.1093/milmed/usy355 
42.  Teo AR, Marsh HE, Forsberg CW, et al. Loneliness is closely associated with depression 
outcomes and suicidal ideation among military veterans in primary care. Journal of Affective 
Disorders. 2018;230:42-49. doi:10.1016/j.jad.2018.01.003 
43.  Pereira-Sanchez V, Alvarez-Mon MA, Barco AA del, Alvarez-Mon M, Teo A. Exploring the 
Extent of the Hikikomori Phenomenon on Twitter: Mixed Methods Study of Western Language 
Tweets. Journal of Medical Internet Research. 2019;21(5):e14167. doi:10.2196/14167 
44.  Denneson LM, Chen JI, Pisciotta M, Tuepker A, Dobscha SK. Patients’ Positive and Negative 
Responses to Reading Mental Health Clinical Notes Online. Psychiatr Serv. 2018;69(5):593-
596. doi:10.1176/appi.ps.201700353 
45.  Bush NE, Smolenski DJ, Denneson LM, Williams HB, Thomas EK, Dobscha SK. A Virtual 
Hope Box: Randomized Controlled Trial of a Smartphone App for Emotional Regulation and 
Coping With Distress. Psychiatr Serv. 2017;68(4):330-336. doi:10.1176/appi.ps.201600283 
46.  Ganzini L, Denneson LM, Press N, et al. Trust is the basis for effective suicide risk screening 
and assessment in veterans. J Gen Intern Med. 2013;28(9):1215-1221. doi:10.1007/s11606-
013-2412-6 
47.  Bossarte RM, Karras E, Lu N, et al. Associations Between the Department of Veterans Affairs’ 
Suicide Prevention Campaign and Calls to Related Crisis Lines. Public Health Rep. 
2014;129(6):516-525. doi:10.1177/003335491412900610 
48.  Chen JI, Hooker ER, Niederhausen M, et al. Social connectedness, depression symptoms, 
and health service utilization: a longitudinal study of Veterans Health Administration patients. 
Soc Psychiatry Psychiatr Epidemiol. Published online November 6, 2019. 
doi:10.1007/s00127-019-01785-9 
49.  Pedersen ER, Osilla KC, Helmuth ED, Tolpadi A, Gore K. Reaching Concerned Partners of 
Heavy Drinking Service Members and Veterans through Facebook. Military Behavioral Health. 
2017;5(3):265-273. doi:10.1080/21635781.2017.1316804 
50.  S.A.V.E. PsychArmor Institute. Accessed November 6, 2018. 
https://psycharmor.org/courses/s-a-v-e/ 
51.  Czajkowski SM, Powell LH, Adler N, et al. From ideas to efficacy: The ORBIT model for 
developing behavioral treatments for chronic diseases. Health Psychol. 2015;34(10):971-982. 
doi:10.1037/hea0000161 
52.  National Institute of Mental Health. Points to Consider about Recruitment and Retention While 
Preparing a Clinical Research Study. Published online June 2005. Accessed January 30, 

29 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
2019. https://www.nimh.nih.gov/funding/grant-writing-and-application-process/recruitment-
points-to-consider-6-1-05_34848.pdf 
53.  Trouteaud AR. How You Ask Counts: A Test of Internet-Related Components of Response 
Rates to a Web-Based Survey. Social Science Computer Review. 2004;22(3):385-392. 
doi:10.1177/0894439304265650 
54.  Smith B, Smith TC, Gray GC, Ryan MAK, Millennium Cohort Study Team. When epidemiology 
meets the Internet: Web-based surveys in the Millennium Cohort Study. Am J Epidemiol. 
2007;166(11):1345-1354. doi:10.1093/aje/kwm212 
55.  Als LC, Nadel S, Cooper M, Vickers B, Garralda ME. A supported psychoeducational 
intervention to improve family mental health following discharge from paediatric intensive care: 
feasibility and pilot randomised controlled trial. BMJ Open. 2015;5(12):e009581. 
doi:10.1136/bmjopen-2015-009581 
56.  Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical 
research. J Psychiatr Res. 2011;45(5):626-629. doi:10.1016/j.jpsychires.2010.10.008 
57.  Larsen DL, Attkisson CC, Hargreaves WA, Nguyen TD. Assessment of client/patient 
satisfaction: development of a general scale. Eval Program Plann. 1979;2(3):197-207. 
58.  Bangor A, Kortum P, Miller J. Determining What Individual SUS Scores Mean: Adding an 
Adjective Rating Scale. JUS. 4(3). 
59.  Ben-Zeev D, Brenner CJ, Begale M, Duffecy J, Mohr DC, Mueser KT. Feasibility, 
acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. 
Schizophr Bull. 2014;40(6):1244-1253. doi:10.1093/schbul/sbu033 
60.  Osteen P, Frey JM, Woods MN, Ko J, Shipe S. Modeling the Longitudinal Direct and Indirect 
Effects of Attitudes, Self-Efficacy, and Behavioral Intentions on Practice Behavior Outcomes 
of Suicide Intervention Training. Suicide Life Threat Behav. 2017;47(4):410-420. 
doi:10.1111/sltb.12288 
61.  Calear AL, Batterham PJ, Christensen H. Predictors of help-seeking for suicidal ideation in the 
community: risks and opportunities for public suicide prevention campaigns. Psychiatry Res. 
2014;219(3):525-530. doi:10.1016/j.psychres.2014.06.027 
62.  Batterham PJ, Calear AL, Christensen H. The Stigma of Suicide Scale. Psychometric 
properties and correlates of the stigma of suicide. Crisis. 2013;34(1):13-21. doi:10.1027/0227-
5910/a000156 
63.  Wyman PA, Brown CH, LoMurray M, et al. An Outcome Evaluation of the Sources of Strength 
Suicide Prevention Program Delivered by Adolescent Peer Leaders in High Schools. 
American Journal of Public Health. 2010;100(9):1653-1661. doi:10.2105/AJPH.2009.190025 

30 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
64.  Joiner Jr. TE, Pfaff JJ, Acres JG. A brief screening tool for suicidal symptoms in adolescents 
and young adults in general health settings: reliability and validity data from the Australian 
National General Practice Youth Suicide Prevention Project. Behaviour Research and 
Therapy. 2002;40(4):471-481. doi:10.1016/S0005-7967(01)00017-1 
65.  Batterham PJ, Ftanou M, Pirkis J, et al. A systematic review and evaluation of measures for 
suicidal ideation and behaviors in population-based research. Psychol Assess. 
2015;27(2):501-512. doi:10.1037/pas0000053 
66.  Calear AL, Brewer JL, Batterham PJ, et al. The Sources of Strength Australia Project: study 
protocol for a cluster randomised controlled trial. Trials. 2016;17(1):349. doi:10.1186/s13063-
016-1475-1 
67.  Hahn EA, DeWalt DA, Bode RK, et al. PROMIS Cooperative Group. New English and Spanish 
social health measures will facilitate evaluating health determinants. Health Psychol. 
2014;33(5):490-499. doi:10.1037/hea0000055 
68.  Hertzog MA. Considerations in determining sample size for pilot studies. Res Nurs Health. 
2008;31(2):180-191. doi:10.1002/nur.20247 
69.  Friedberg JP, Lipsitz SR, Natarajan S. Challenges and recommendations for blinding in 
behavioral interventions illustrated using a case study of a behavioral intervention to lower 
blood pressure. Patient Educ Couns. 2010;78(1):5-11. doi:10.1016/j.pec.2009.04.009 
70.  Creswell J, Plano Clark V. Collecting data in mixed methods research. In: Designing and 
Conducting Mixed Methods Research. SAGE Publications, Inc.; 2011:171-202. 
71.  Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in 
Psychology. 2006;3(2):77-101. doi:10.1191/1478088706qp063oa 
72.  Morgan D. Focus Groups as Qualitative Research. SAGE Publications, Inc.; 1997. 
doi:10.4135/9781412984287 
73.  Kramer J, Rubin A, Coster W, et al. Strategies to address participant misrepresentation for 
eligibility in Web-based research. International journal of methods in psychiatric research. 
2014;23(1):120-129. doi:10.1002/mpr.1415 
74.  Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data 
capture (REDCap)—A metadata-driven methodology and workflow process for providing 
translational research informatics support. Journal of Biomedical Informatics. 2009;42(2):377-
381. doi:10.1016/j.jbi.2008.08.010 
Appendix – Supporting Documents List 
 
 
 
Appendix A: Information Sheet 
Appendix B: Survey 
Appendix C: Interview Guide 

31 
<Protocol version date: 06/24/21  
Research Service Template: 12/20/19 
Appendix D: Communications with participants 
Appendix E: Recruitment materials