Randomized Controlled Trial of AboutFace: A Novel Video Storytelling Resource to Improve Access, Engagement, and Utilization of Mental Health Treatment Among Veterans With PTSD
Phase: NA
Conditions: PTSD, Posttraumatic Stress Disorder
Sponsor: VA Office of Research and Development

1 
 
 
 
 
 
Randomized Controlled Trial of AboutFace: A Novel Video Storytelling Resource 
to Improve Access, Engagement, and Utilization of Mental Health Treatment 
among Veterans with PTSD 
NCT03521999 
10/31/2018 

2 
 
 
PROTOCOL TITLE: Randomized Controlled Trial of AboutFace: A Novel Video Storytelling Resource to Improve 
Access, Engagement, and Utilization of Mental Health Treatment among Veterans with PTSD  
PRINCIPAL INVESTIGATOR(S): Anouk L. Grubaugh, Ph.D. of the Charleston VAMC and the Medical University 
of SC and Jessica L. Hamblen from the National Center for PTSD and the Geisel School of Medicine at 
Dartmouth 
1.0 OBJECTIVES/SPECIFIC AIMS: At least one in ten Veterans meet criteria for posttraumatic stress disorder 
(PTSD) related to their military experience.3, 4 Treatment for PTSD is widely available within the Veterans Health 
Administration (VHA), and national dissemination initiatives have significantly increased Veterans’ access to 
best-practice interventions.15-18 However, mental health treatment seeking among Veterans remains strikingly 
low; most Veterans with mental health problems do not seek mental health services .17, 18 This is in large part 
due to perceived stigma and other barriers. 28, 29 The National Center for PTSD (NCPTSD) developed and 
launched AboutFace, a public awareness campaign to help Veterans recognize PTSD and motivate them to 
seek evidence-based treatment. AboutFace has tremendous potential to increase the likelihood that Veterans 
with PTSD and related mental health conditions will access mental health services. Due to this potential, the 
study team recently partnered with NCPTSD to evaluate AboutFace through a funded HSR&D pilot study. Study 
data from this project were promising with regard to the potential benefit of AboutFace for promoting treatment 
initiation, and the current proposal is an extension of this initial feasibility trial.  
 
The current proposal is significant because it seeks to 1) compare the efficacy of AboutFace [AboutFace] 
to enhanced Usual Care [eUC] for decreasing stigma related to mental healthcare, increasing treatment 
engagement, and improving clinical and quality of life outcomes using a randomized controlled study design with 
n=376 Veterans referred for PTSD specialty care; and 2) obtain valuable data from relevant VA stakeholders 
regarding optimal implementation strategies to ensure the wide-spread dissemination of AboutFace within VHA. 
To date, we know that a large number of Veterans referred for a PTSD assessment by their VA provider do not 
follow through with that recommendation or do not follow through with treatment once they are referred. As such, 
the proposed study represents an invaluable opportunity for HSR&D to fill a critical service delivery gap for a 
significant number of Veterans who are at risk for PTSD and the concomitant adverse impact of this disorder on 
functioning and quality of life. The current proposal is also responsive to the Veterans Affairs 2014-2020 Strategic 
Plan including the priority goals to 1) Improve Veteran Access to VA Benefits and Services; and 2) Enhance and 
Develop Trusted Partnerships both within and between VA and its external partners. Altogether, it is anticipated 
that project data will have broad and significant implications for overcoming barriers to care for Veterans with 
PTSD as well as other stigmatized mental health conditions.  
Specific Aims are detailed below: 
 
Specific Aim 1:  To evaluate the impact of AboutFace compared to eUC on treatment initiation and engagement 
(i.e., total number of sessions completed).  
Hypothesis 1 (primary): Veterans in the AboutFace condition will be more likely than those assigned to 
the eUC condition to initiate PTSD treatment and to attend more therapy sessions. 
 
Hypothesis 2 (secondary): Veterans in the AboutFace condition will have improved clinical and quality 
of life outcomes (e.g., PTSD, depression) at three- and six-month follow-up relative to Veterans in the 
eUC condition.   
 
Specific Aim 2:  To compare stigma and attitudes toward mental health treatment for Veterans receiving  
AboutFace and Veterans receiving eUC. 
Hypothesis 1 (secondary): Veterans in the AboutFace condition will have lower stigma and more 
positive attitudes toward seeking help than those assigned to the eUC condition at one-, three- and six-
month assessments. 
 
Specific Aim 3: Conduct thematic (qualitative) interviews with key stakeholders [i.e., PTSD Clinical Team (PCT) 
directors] at multiple and diverse VAMCs to learn about potential barriers to integrating AboutFace into their 
intake processes and improve future dissemination efforts.   
 
2.0 BACKGROUND:   

3 
 
 
Posttraumatic Stress Disorder is Prevalent among Veterans. Military operations frequently involve 
life-threatening duties, such as patrols and direct fire, witnessed violence and human suffering, and hostile 
responses from civilians. 1, 2 Although the majority of Veterans are resilient and/or recover quickly, 
posttraumatic stress disorder (PTSD) occurs in around 17% of Vietnam Veterans 3 and 13% of Veterans from 
Iraq and Afghanistan.4 PTSD is associated with other mental health problems such as depression and alcohol 
abuse,1, 5-10 and all of these disorders are in turn associated with high levels of distress, functional impairment, 
and morbidity and mortality.11-13 For example, a survey of 103,788 Veterans found that over 1 in 4 Veterans 
met criteria for a psychiatric diagnosis.14 Disparities also are prevalent, with rural and minority Veterans having 
a greater disease burden than their counterparts.15-16 
 
PTSD Treatment Seeking is Low. Despite high levels of distress and impairment, rates of mental health 
treatment seeking are surprisingly low among those with a psychiatric condition.  Fewer than half (45%) of 
individuals with a mental health problem seek treatment.17 Rates of treatment seeking for PTSD are similarly 
low. In a survey of over 47,000 soldiers, only 48% of those who screened positive for PTSD had received any 
treatment in the past 6 months and 24% of those dropped out prematurely.18 Even in VA where evidence-based 
treatments for PTSD are widely available through national dissemination efforts, 19-24 rates of treatment seeking 
are low and premature dropout is common.25-27 Stigma, transportation, time constraints, scheduling difficulties, 
and concerns about confidentiality are noted key barriers to accessing treatment.28, 29 As such, there is a critical 
need for innovative strategies to overcome barriers to accessing and benefitting from mental health care 
for PTSD.30 Widely accessible technology-based resources can play a significant role in addressing this 
need. The potential reach of such resources has increased significantly in recent years due to rapid growth in 
Internet access (88% of the population has access to the internet; 77% use a smartphone).31  
 
Stigma is a Barrier to Mental Health Treatment Seeking. Stigma is frequently reported as a barrier to 
seeking treatment.32, 33 Stigma may be even more problematic among service members who perceive that 
disclosing a mental illness could negatively impact their military career. In one survey of four US combat infantry 
units, among those who met screening criteria for a psychiatric disorder 65% were concerned that others would 
see them as weak, 63% were concerned that leadership would treat them differently, and 59% were concerned 
that members of their unit might have less confidence in them.  Those most in need of care were also the ones 
most concerned about stigma.34 
 
Peers Provide Opportunity to Observe and Learn from Others. Corrigan and colleagues propose 
education and contact with people who have a mental illness as two recommended strategies for reducing 
stigma.35 Peers, especially those with the same psychiatric disorder, can be used to simultaneously fulfill both of 
these strategies. They can provide patients with accurate information (i.e., education) as well as personal contact 
with a similar peer (i.e., contact), which in turn can challenge patients’ misperceptions about mental illness. A 
growing body of evidence supports the effectiveness of peer educators in reducing stigma and 
improving treatment seeking. Peer educators have been shown to improve knowledge, self-efficacy, and 
behavioral outcomes in people with HIV,36, 37 reduce drinking in college students,38 and reduce stigma in 
depressed older adults.39 
 
Conceptual Model. Naslund and colleagues 40 propose a conceptual model explaining how online peer 
networks can improve access to treatment (see Figure 1).  They view online peer support networks as a potent 
mechanism to challenge stigma, increase consumer activation, and ultimately increase access to mental 
healthcare.  The model suggests that people with PTSD may be motivated to access online peer interventions 
because they are struggling to deal with their PTSD symptoms, feel socially isolated and tend to avoid in-
person interactions, and/or are afraid to reach out for traditional face-to-face help due to concerns about how 
others will perceive them.  These factors can lead a person with PTSD to seek out online support, which is 
anonymous and less anxiety provoking than in-person interactions.  The decision to visit an online peer 
network provides opportunities to challenge stigma or other misperceptions that are preventing an individual 
from accessing needed mental healthcare. Additionally, peers help people feel less isolated and alone in their 
illness, more empowered, and more hopeful. These interactions can motivate interest in learning more about 
PTSD and demystify the treatment process resulting in an increased likelihood that evidence-based care for 
PTSD will be accessed. 
Figure 1. Conceptual Model 

4 
 
 
 
AboutFace: A Veteran-to-Veteran Digital Storytelling Resource. Developed by Dr. Hamblen’s (MPI) 
team at the National Center for PTSD, AboutFace is a web-based video gallery that introduces viewers to a 
community of over 77 Veterans—diverse with regard to military experience, age, gender, race/ethnicity—who 
have experienced PTSD and received treatment in VA facilities. The site also contains testimonials from 23 
family members and 22 clinicians. As such, it serves as an online peer network where Veterans can meet similar 
others and learn from them (see Figure 2). 
Figure 2. AboutFace Home Screen 
 
AboutFace aims to use the shared bonds of military service to educate Veterans and their families about 
PTSD and help normalize common reactions that Veterans may experience due to their military service or 
deployment experiences. Consistent with this, visitors to the site can ‘meet’ Veterans and hear how PTSD has 
affected them through unscripted, authentic personal stories. Veterans are filmed in natural settings looking 
directly at the camera. For the viewer, the eye contact is intimate, as if the Veterans have invited the viewer into 
their homes and are sharing personal details about their lives. AboutFace has an open format that allows visitors 
to navigate the site freely based on user preferences. Veterans also can receive advice from expert clinicians 
and hear how PTSD can affect family members, via a separate component of the AboutFace site. Topics 
addressed in the brief (~1 min) video clips are included in Table 1 below.  
Table 1. AboutFace video clips 

5 
 
 
CLIPS OF VETERANS 
CLIPS OF CLINICIANS 
CLIPS OF FAMILY MEMBERS 
WHO I AM 
Example: A.J., US Army (1978-1998),  
Germany/Korea/US 
WHO I AM  
Example: Dr. Peter Tuerk, Clinical 
Psychologist, Director of PCT Clinic, 
Ralph H. Johnson VA Medical Center, 
Charleston, SC 
WHO I AM 
Example: O.J., daughter of A.J. 
HOW I KNEW I HAD PTSD 
I was waking up, sweating, [and I] 
couldn’t go back to sleep… 
 WHAT PTSD IS 
A very, very lonely experience… People 
have thoughts and nightmares they 
experience alone. They want to isolate… 
LIVING WITH SOMEONE WITH 
PTSD 
I didn’t experience a childhood… I 
would give up going to a friend’s house 
in case Dad needed me. 
HOW PTSD AFFECTS THE PEOPLE 
YOU LOVE 
My daughter would ask, ‘Mommy, why is 
Daddy crying?’ 
HOW TO KNOW YOU’RE READY 
FOR HELP 
Have the worst time sleeping, they isolate 
the most, have no relationships, extremely 
on edge all the time… 
THE SIGNS THAT I SAW 
He wouldn’t converse with me as much, 
he was a little distant, his temper… 
WHY I DIDN’T ASK FOR HELP 
RIGHT AWAY 
I didn’t think she [my therapist] could 
relate to what I had been exposed to… 
WHAT TREATMENT IS LIKE 
People are asked to sort of get used to the 
things that are bothering them the most… 
HOW PTSD AFFECTS A FAMILY 
When I got older, he would start 
isolating himself… I would talk “at him” 
without him saying anything. 
WHEN I KNEW I NEEDED HELP 
I heard about Gulf War Vets not being 
able to sleep, etc. I thought “Wow, that’s 
some of the symptoms I have.” 
WHAT TREATMENT CAN DO FOR 
YOU 
Set goals and target treatment to those 
goals. If Veteran wants to get rid of 
nightmares, use exposure… 
THE HARDEST PART 
PTSD made him shelter me a lot…I 
couldn’t go to the movies on the weekend 
or house parties 
WHAT TREATMENT WAS LIKE 
FOR ME 
My homework was to go into the 
Walmart or crowded mall for 30-45 
minutes. 
QUESTIONS WE’VE BEEN ASKED 
Does my family have to be involved? 
HOW 
TREATMENT 
CHANGED 
THINGS 
We don’t argue as much. He’s a 
different person, and I like it. He’s 
happier and taking care of himself. 
HOW TREATMENT HELPS ME 
I still have PTSD, but I’m in control of it 
now… I’m at peace with it, and I can 
talk about it [the trauma]. 
 
 
MY ADVICE TO YOU  
They [the therapists] are waiting for 
Veterans like you and I. Try it. You won’t 
regret it. 
MY ADVICE TO YOU 
You really want a treatment that involves 
some type of exposure… 
MY ADVICE TO YOU 
Support them [the family member] and 
let them know you’re there for them. 
Most importantly - listen. 
 
 
 
 
Following from Naslund’s conceptual model,40 AboutFace visitors to the site can challenge their 
stigmatizing beliefs by choosing to engage with the Veterans on the site with whom they most closely identify. 
They can view the clips that are most relevant to them and learn from others how they have handled similar 
situations.  Real life examples provide corrective information to challenge misperceptions about PTSD or the 
treatment process for PTSD.  Visitors may be motivated to learn about PTSD (i.e., increased consumer 
activation) either through accessing videos of clinicians on the site describing PTSD or by going to the National 
Center for PTSD website which houses AboutFace and reading any of the hundreds of articles or other resources 
available.  The ultimate goal of AboutFace is to motivate Veterans to seek evidence-based care for  PTSD. To 
date, VHA has invested significant time and money on the development and refinement of AboutFace. Despite 
this investment, Web analytic data show that less than 1% of U.S. Veterans have visited AboutFace to date. This 
is in large part likely due to a lack of public awareness of AboutFace as well as limited efforts to integrate 
AboutFace into clinical practice settings. Additionally, there are no systematic data on the efficacy of AboutFace 
for reducing stigma and increasing PTSD treatment initiation and engagement.  
 
Summary. AboutFace is an ideal peer-to-peer digital story-telling resource with which to initiate this line 
of research for several reasons. First, PTSD is a prevalent, stigmatized condition, and we believe that About 
Face’s peer-to-peer approach may be particularly valuable for decreasing stigma related to PTSD and increasing 
treatment initiation and engagement. Survey research has found that U.S. adults with stigmatized illnesses (e.g., 
anxiety, depression, herpes, urinary incontinence) are more likely than those with at least one other chronic 

6 
 
 
illness (e.g., cancer, heart problems, diabetes, back pain) to have used the internet to obtain health-related 
information or to have communicated with providers via the web.41 Thus, what we learn in this research may 
have relevance to a wide range of stigmatized conditions. Second, AboutFace is a widely accessible resource 
that has already been developed and launched by the National Center for PTSD, as well as revised based 
on Veterans’ feedback from our recently completed HSR&D pilot project in partnership with the National 
Center. It is anticipated that study data will provide insight into the value of AboutFace over standard care in 
improving treatment initiation and engagement of evidence-based services for PTSD. Third, the qualitative 
component of the study will allow us to prepare AboutFace for dissemination and implementation into PTSD 
specialized programs. Fourth, preliminary work led by our team in collaboration with the National Center for 
PTSD underscores our ability to successfully lead this project, including significant experience with technology-
based interventions, a strong partnership with the National Center including MPI Dr. Hamblen who led the 
development of AboutFace, and successful completion of pilot project benchmarks.  
 
 
Significance and Impact. How can we overcome the resistance of Veterans to get the help they need 
when they need it most? VHA has devoted tremendous effort and resources to developing and increasing access 
to evidence-based mental health programs, but misconceptions about PTSD treatment nevertheless remain 
pervasive.  Stigma remains a major barrier to care. Developing effective ways to improve Veterans’ engagement 
in evidence-based care for PTSD and reduce stigma associated with help-seeking continues to be a major 
priority for VHA.42 As noted, research has demonstrated that people are most responsive to advice or education 
when it comes from someone they see as similar to themselves (i.e., peer education), and the potential reach of 
web-based resources has increased significantly in recent years due to rapid growth in Internet access (i.e., 88% 
of the population has access to the interne; 77% has access to a smartphone).31 
 
AboutFace has tremendous potential to reduce stigma and improve attitudes toward mental health 
treatment seeking, as well as to increase treatment initiation and engagement among Veterans. By hearing 
Veterans on AboutFace describe what treatment was actually like, what their fears were, and how their lives 
have improved with treatment, it is anticipated that Veterans considering PTSD treatment will be more willing to 
initiate treatment and be better prepared to navigate the challenges that may surface during the treatment 
process. Challenging misperceptions and negative attitudes about treatment should increase engagement and 
retention in treatment, which in turn will increase the likelihood that a Veteran will receive enough sessions to 
experience symptom improvement. However, the efficacy of AboutFace for increasing PTSD treatment initiation 
and engagement or for decreasing stigma related to PTSD mental healthcare has not yet been evaluated. 
Drawing on findings from our recently completed HSR&D pilot study (#PPO 14-360-1), the study team is again 
partnering with the National Center for PTSD in the proposed Merit application to examine the efficacy of 
AboutFace relative to enhanced usual care for increasing treatment initiation and engagement using a 
randomized controlled trial (RCT) design. If AboutFace is found to increase PTSD treatment initiation and 
engagement, this finding would have broad and significant implications for improving Veteran access and 
utilization of evidence-based treatment, and for overcoming barriers to care for Veterans with other stigmatized 
conditions. Additional data from key stakeholders will inform efforts to widely disseminate AboutFace within VHA 
clinical practice settings. 
 
Innovation. Few studies have examined the impact of peer education approaches in the health care 
field, and the majority of existing evaluations have relied on in-person, live interactions. Costs associated with 
training peers and supporting their interactions with the target population inherently limit the scalability of these 
approaches. Additionally, peer education approaches have not been examined extensively in Veteran 
populations, and to our knowledge, there has been no rigorous evaluation of Veterans’ receptivity to digital 
storytelling approaches. This low-cost, highly sustainable and scalable approach to peer education is likely to 
have particular value for VHA and Veterans with stigmatized conditions. Thus, although the proposed study 
focuses on Veterans with PTSD, the study design can be easily adapted for Veterans with a range of other 
stigmatized conditions.  
 
Preliminary Studies and Study Team Background. Our research team has worked collaboratively on 
many successful large-scale projects in technology-based intervention development, evaluation, and 
dissemination. Each team member offers unique, complementary areas of expertise. Of direct relevance, the 
study team recently collaborated on a 1-year HSR&D funded pilot (#PPO 14-360-1) evaluating the usability and 
feasibility of AboutFace as well as the feasibility of our study methodology in preparation for the proposed RCT. 
Drs. Hamblen, Grubaugh, Ruggiero, and Davidson all were key investigators who co-led this pilot study. Drs. 

7 
 
 
Grubaugh, Hamblen, and Ruggiero also recently completed an HSR&D-funded study to develop and evaluate 
an online VA provider training website in Prolonged Exposure for PTSD (IIR-08-323-2), as well as a self-help 
intervention for OEF/OIF Veterans (MHI-08-105-2). Drs. Ruggiero, Hamblen, Grubaugh, and Davidson all have 
significant experience in the conduct of thematic interviews and in qualitative data analysis and interpretation 
(HSR&D CD 207015; NIH R01 MH074151; NIH R21MH065248; HSR&D IIR-08-323). This strong history of 
collaboration demonstrates our ability to lead a project of this scope. Drs. Grubaugh and Hamblen also have 
extensive experience leading clinical trials with Veterans including large scale RCTs. Dr. Hamblen is currently 
Acting Deputy Executive Director of the National Center for PTSD and the Deputy for Education. She led the 
development of the AboutFace digital storytelling site for Veterans and co-led the HSR&D Pilot study that led to 
this Merit proposal. She oversees the national PTSD Mentoring Program which will provide a vehicle for both 
disseminating study findings to all VA specialized PTSD Programs and for working with sites to overcome 
barriers to implementation. Dr. Ruggiero adds significant expertise relating to the use of technology to improve 
access and quality of services. He has been PI on three (3) HSR&D grants focused on the development and 
evaluation of technology-based solutions in mental health among Veterans and VA providers. He also serves as 
Co-Director of the Technology Applications Center for Healthful Lifestyles at our institutional affiliate (Medical 
University of South Carolina), where he has led as PI 9 federally funded grants.  
 
Development of AboutFace (led by the National Center for PTSD). Dr. Hamblen and Vicky Bippart, 
an award winning documentary filmmaker, set out to develop AboutFace as a unique public awareness campaign 
featuring video stories and peer education. It was designed specifically to help Veterans recognize PTSD 
symptoms and increase their readiness to engage in evidence-based mental health treatment. As noted, 
Veterans visit the site, find someone who looks like or appeals to them, and listen to that person’s story. Veterans 
were filmed in comfortable home environments to convey the intimacy of someone’s living room, and Veterans 
speak candidly in the videos, directly to the viewer. Their message is empathic but clear: get into treatment—
treatment can turn your life around. Dr. Hamblen’s team launched AboutFace in May of 2012. Since the launch 
of AboutFace it has had tens of thousands of visits to the site and earned three major awards: the 
Summer/Fall Web Health Award, Bronze, Medical Education, the Interactive Media Award, Best in Class, 
Military, and a Bronze Telly Award. In the first year post-launch, Web Trends data reveal over 35,000 visits to 
the site. In FY16, 153,772 individuals came to the AboutFace website and an additional 400,000 viewed 
AboutFace videos on Facebook.  
 
AboutFace Pilot Results. The 1-year HSR&D pilot project (#PPO14-360-1) conducted by the study 
team included a usability testing phase and a randomized controlled feasibility trial.  
 
(a)Usability Testing. The usability testing component of the study included 20 Veterans who presented for an 
evaluation at the Charleston VAMC PTSD Clinical Team (PCT) and were recommended for treatment. Veterans 
were approached for study participation immediately after their PCT evaluation session. Participants then 
completed informed consent and were given access to AboutFace while the moderator observed their use of the 
site. During the initial introductory observation period, Veterans were encouraged to freely navigate the site using 
a cognitive interviewing approach.43 Specifically, the moderator stated “Here is the website we would like you to 
evaluate. Take some time to use the site. Please walk me through what you are thinking step by step out loud 
while you check it out and use it.” The moderator then took notes during the session to record any behavioral 
observations, impressions, and relevant quotes from Veterans. Following this introductory phase, the moderator 
used a general interview guide that was developed by study staff using questions based on the actions and 
responses of participants as they navigated the site. Example questions included:  
 
• 
What do you think about the layout of the site? 
• 
Why did you choose to click on that Veteran? What characteristics are you looking for as you look at the 
images of Veterans on the screen? How important is branch of service? age? race? gender? 
• 
I noticed you skipped over that video clip—why did you decide not to watch it?  
All interviews were audio recorded for later and analysis.  
 
Veterans were also asked questions about perceived changes in knowledge about the nature of PTSD 
treatment and perceived changes in attitudes toward seeking mental health treatment.  Overall, Veterans felt 
that the site was appealing and the topic questions were readily apparent. AboutFace was designed to allow 
users flexibility to select the videos most relevant to them.  Usability testing supported this concept. Veterans 
reported that they liked the ability to find Veterans “like me.” Videos describing Veterans’ experiences with PTSD, 

8 
 
 
especially the ones on “How I knew I had PTSD” were particularly well received.  Participants also liked hearing 
about the effects of PTSD on family members. Many commented that they also liked the opportunity to 
 Figure 3. AboutFace Sample Content                                    
educate themselves about PTSD. Several 
problems with the website were also noted.  Some 
participants were confused by the navigation 
features and did not recognize that there were 
features in addition to the topic questions that 
were accessible from the top navigation. Some 
also did not recognize initially that the images of 
Veterans on the main screen were connected to 
videos, and only noticed the rollover quotes. 
  
 
 
Veterans offered several recommendations for 
improving the site including adding a greater 
diversity of profiles, more search options, and 
more content describing what PTSD is and how it 
can be treated. Some of these suggestions 
stemmed from a lack of recognition that there was 
a “Clinicians” tab on the site that provides more of 
this education. With regard to dissemination, 
Veterans believed the site would be most 
beneficial to Veterans who were ambivalent about 
seeking help. 
Finally, more than half of 
participants (60%) reported an increase in 
knowledge regarding PTSD symptoms and 
diagnosis, the impact of their symptoms on family 
members, and a greater understanding of PTSD 
and what PTSD treatment involves after visiting 
the site. 
 
 
Feedback-based changes to AboutFace: 
Based on these data, changes to AboutFace were 
implemented in two phases. The first phase 
focused on immediate changes that were made 
without changing the overall architecture of the 
site. These included producing and adding more 
Veterans’ videos to the site to increase diversity 
(especially more women who experienced military 
sexual trauma) and including a “Therapies” page 
to provide more basic information on PTSD and 
PTSD treatment. In the next phase, more 
challenging issues were addressed, including the 
development of a new homepage that more 
clearly communicates the purpose of the site and 
promotes viewing of videos, top navigation that is 
more obvious, a searchable video directory that includes Veteran, clinician, and family videos, and play buttons 
on videos. The new site has been designed and the programming is nearly completed (see Figure 3). The 
updated site will be live several months before the proposed study can be launched. 
 
(b) Randomized Controlled Feasibility Trial. The feasibility pilot included 60 Veterans randomized to one of 
two conditions: AboutFace (Veteran is provided link and instructions for accessing AboutFace or enhanced Usual 
Care (Veteran is provided a link to obtain some basic education material derived from a PTSD brochure). The 
purpose of the pilot was to demonstrate the methodology and set a foundation for the proposed large-scale RCT 
to examine the impact of AboutFace on treatment initiation and engagement, as well as stigma and attitudes 
toward mental health service seeking. Our choice of study design was consistent with expert recommendations 

9 
 
 
regarding the use of pilot mechanisms to 1) test the feasibility of conducting a full-scale RCT; 2) use data yielded 
by the pilot study to “de-bug” the existing methodology; and 3) assess optimal strategies for executing the RCT.44 
 
Findings from our small scale RCT supported the feasibility of our methodology and recruitment plan. We 
recruited an average of 12 Veterans a month (60 Veterans were recruited in 5 months) and retention from 
baseline to follow-up was 81.7%. Our sample was representative with regard to gender (85% male) and ethnically 
diverse (50% of the sample was African American). The mean age of our sample was 42.2 (S.D. 12.6). 
Importantly, 92% of Veterans visited AboutFace as directed indicating that it is feasible to assign AboutFace as 
an intervention strategy. Additional study data indicated that 68% of Veterans in the control condition and 79% 
of Veterans in the AboutFace condition initiated treatment. As would be expected, mean stigma scores in our 
treatment seeking sample were slightly lower than those in the Veteran non-treatment seeking validation study 
on the Endorsed and Anticipated Stigma Inventory 45 but we demonstrated that scores are sensitive to change. 
Veterans in both conditions reported improved attitudes towards mental illness from baseline to the two week 
follow-up suggesting that stigma towards mental health can be reduced through education.  There were no 
statistically significant differences in attitudinal change across the two conditions at post, but the study was not 
powered to detect these differences.  Altogether, our pilot data provides strong support for the methodology 
proposed in the current study and our ability to meet project milestones. 
    
I. Randomized Controlled Trial  (VETERAN) COMPONENT OF STUDY  
3.0a INTERVENTION: We propose to (a) conduct a randomized controlled trial of 376 Veterans to rigorously 
evaluate the impact of AboutFace compared to enhanced usual care for increasing treatment initiation and 
engagement; (b) test the efficacy of AboutFace for decreasing stigma and improving attitudes toward mental 
health treatment seeking; and (c) interview key VHA stakeholders to learn how to broadly implement AboutFace 
into practice.  This is significant because, if AboutFace is found to be effective for reducing stigma regarding 
PTSD and/or for increasing PTSD treatment engagement it would in turn have broad and considerable 
implications for overcoming barriers to care for Veterans with PTSD as well as other stigmatized mental health 
conditions. 
The RCT component of the study (i.e., Veteran component) will randomize 376 Veterans who are referred 
for a PTSD evaluation through the Charleston VAMC PTSD Clinic Team (PCT) and Telehealth Programs to 
either AboutFace or enhanced Usual Care [eUC]. We are focusing on this group of Veterans for several reasons. 
First, AboutFace is highly relevant to Veterans who have recently been referred for a PTSD evaluation session 
and the proposed study design will ensure that we target a group of Veterans for which this resource is most 
relevant and is likely to have the greatest immediate value (i.e., Veterans who are considering PTSD treatment). 
Second, our recruitment approach ensures very high feasibility of meeting our sample recruitment goals and 
study aims. In the pilot we were able to recruit 60 Veterans into the feasibility study over a 5-month period, 
averaging 12 recruited Veterans per month with the limited staff effort and resources inherent in the pilot award 
mechanism. We propose to recruit a total of 376 Veterans during the 41 months of data collection period; only 9 
Veterans per month and only a 20% recruitment rate. We considered recruiting Veterans from general mental 
health as a way of including Veterans who may be more ambivalent about their decision to seek PTSD treatment.  
However, given that the pilot data were obtained from Veterans who came to the PCT, we decided to use the 
same methodology.  Additionally, only 30% of Veterans who come to the evaluation initiate treatment.  Thus, we 
will receive feedback in this study from Veterans at different levels of treatment readiness. 
 
4.0a STUDY TIMEPOINTS: At baseline, a 1-month follow-up assessment (in person or by telephone) will be 
scheduled with participants in both conditions (see follow-up measures section below). Additional assessments 
across both conditions will take place at 3 and 6 months post randomization. The 3- and 6-month assessment 
time-points were selected in order to ensure Veterans will have completed the recommended course of treatment 
within the PCT. A research assistant blind to study condition will administer all telephone interviews. Veterans 
will be reimbursed $25 for completing the baseline interview, $30 for completing the 1-month assessment, $35 
for completing the 3-month assessment, and $40 for completing the 6-month assessment.  
  
5.0a INCLUSION/EXCLUSION CRITERIA/STUDY POPULATION: Veterans are asked, as routine clinical 
practice during their intake through the Charleston PTSD Clinic Team (PCT) and Telehealth Programs, if they 
would be willing to be contacted about possible studies they may be eligible for. If a Veteran consents to be 
contacted during their intake, and are recommended for treatment through the Charleston PTSD Clinic Team 

10 
 
 
(PCT) and Telehealth Programs because they have a diagnosis of PTSD, they will be contacted by study 
personnel to hear more about the study. Since Veterans will be contacted for possible study participation after 
they are diagnosed with PTSD and recommended for PTSD treatment through the Charleston PCT and its 
affiliated clinics, it is anticipated that almost all Veterans who consent will be eligible to participate in the study.  
 
6.0a NUMBER OF SUBJECTS: A total of 376 Veterans will be recruited to participate in the study. 
 
7.0a-8.0a SETTINGS & RECRUITMENT METHODS: Veterans will be recruited through the Charleston VAMC 
PTSD Clinic Team (PCT) and Telehealth Programs, directed by Dr. Tuerk (Co-I). The Charleston PCT provides 
Prolonged Exposure (PE) and Cognitive Processing Therapy (CPT), two best practice interventions for PTSD.  
Program evaluation data are available for over 2,000 Veterans referred to the Charleston PCT. Program 
evaluation data from a recent 4- period found that of the 268 Veterans referred to the clinic for PTSD evaluation 
(67/month), nearly 80% attended the evaluation (53/month), and most (85%) were referred for PTSD treatment 
(45/month). However, nearly 30% referred for PTSD treatment did not attend treatment. Additionally, another 
33% dropped out of treatment prematurely.  Thus, there are 636 eligible Veterans seen for intake each year for 
a total of over 1,900 eligible participants over the three years of study recruitment. 
 
9.0a CONSENT PROCESS: The baseline assessment for Veterans will be conducted face-to-face or via 
telephone [for those who prefer and for telehealth patients who cannot travel to the Charleston PCT due to 
distance and time constraints]. Due to the minimal risk posed to participants and the logistics of the study 
procedures involving telehealth participants, we will obtain a waiver of written informed consent. Rather than 
obtain written informed consent, the study team will obtain verbal consent using a standardized script. The script 
will include an overview of the study and detail aspects of the study procedures and will be administered/reviewed 
immediately prior to the baseline assessment. A copy of this document will be given (face-to-face) and/or mailed 
(telehealth) to participants for their records, and study personnel will document the date of 
administering/reviewing the consent script in the study enrollment log which is housed on a protected VA server. 
Elements included in the consent script will indicate the voluntary nature of the study, detail the information that 
will be gathered as part of study participation, discuss any risks and benefits associated with participation, and 
note how study related information will be safeguarded.   
 
10.0a STUDY DESIGN/METHODS: Veterans are asked, as routine clinical practice during their PCT intake 
through the Charleston PTSD Clinic Team (PCT) and Telehealth Programs, if they would be willing to be 
contacted about possible studies they may be eligible for. Veterans who consent to this and are recommended 
for treatment through the Charleston PCT and affiliated programs, will be contacted by study personnel to learn 
about the study. If the Veteran is interested/willing, he or she will be scheduled for the baseline assessment, 
which will include an overview of the study and details about the study procedures using a preapproved script. 
Verbal consent will be obtained using a document that contains the same information detailed in the phone script, 
and a copy of this document will be given (face-to-face) and/or mailed (telehealth) to participants for their records.  
 
After completion of the baseline assessment, Veterans will be randomly assigned to one of two conditions 
using a Microsoft Excel randomization formula: enhanced usual care (eUC) or AboutFace. All Veterans will be 
given printed psychoeducation materials for PTSD [see Appendix for a copy of this education material] and will 
be given instructions for accessing the study portal. Veterans randomized to eUC will be told that they can use 
the study portal to access the psychoeducation print materials they received during the baseline assessment [in 
the case they lose their materials or prefer to look at them online].  Veterans assigned to AboutFace will be told 
that they can use the study portal to access AboutFace and will be given brief instructions on how to navigate 
the AboutFace website. They will also be told they can download a handout from the portal that includes basic 
recommendations for using the site (e.g., “We suggest you spend a minimum of 15 minutes”) and an explanation 
of the different ways that visitors can search the site (e.g., “You will notice you can search by question and then 
use your mouse to roll over and see responses you want to hear”). 
 
 Between two to three days after randomization, the site coordinator will call all Veterans assigned to 
AboutFace to check in and see if they have visited the site.  If not, the coordinator will encourage the Veteran to 
use the site and problem solve any issues that may have arisen (e.g., cannot remember url, have not found the 
time, tried but could not access, etc.). 
 

11 
 
 
Because we are evaluating the use of a web-based system in this study, our study procedures 
necessitate that eligible Veterans have the ability to access the internet. This can be done in their household, in 
the VA, in a public library, or at the home of a friend or relative. This criterion will have minimal impact on eligibility. 
Evidence shows that 88% of adults in the US use the internet, and roughly two-thirds of US adults access the 
internet through mobile phone devices (AboutFace uses responsive design and can be used on a mobile 
phone)31 and 44% of those who do not have their own accessible devices have gained access through family 
members.47 These percentages are growing rapidly.48 Therefore, we are proposing an approach that stands to 
reach the vast majority of Veterans. If the Veteran does not have access to the Internet in his or her home, study 
staff will assist in problem solving to identify alternative access locations. 
  
The following patient measures will be used at baseline: 1) Demographic Form: age, race, gender, 
education, and employment information will be collected via a study specific demographic form; 2) The Endorsed 
and Anticipated Stigma Inventory (EASI) will be used to assess attitudes and barriers toward seeking mental 
health services. The EASI was developed to assess different dimensions of stigma-related beliefs about mental 
health among military personnel and Veterans.45 It measures 5 dimensions: beliefs about mental illness, beliefs 
about mental health treatment, beliefs about seeking treatment, concerns about stigma from loved ones, and 
concerns about stigma in the workplace.  Items are scored on a 5-point Likert scale with higher scores indicative 
of greater stigma.  Psychometric properties of the EASI suggest that it demonstrates good internal consistency 
reliability, content validity, and convergent and discriminate validity 45; 3) The PTSD Checklist-5 (PCL-5)49 is a 
20-item instrument that parallels DSM-5 criteria for PTSD. Each item has five response options. Studies 
supported the psychometrics of the DSM-IV PCL, including internal consistency (coefficient alphas  .85 for all 
subscales50), test-retest reliability, convergent validity with the gold standard PTSD interview, discriminant 
validity, and sensitivity and specificity.51-54 Psychometrics on the newly developed PCL-5 indicate coefficient 
alphas above .9 in two different samples, test-retest of .8, and correlations with the gold standard PTSD interview 
at .78 55; 4) The PHQ-9 is a brief questionnaire that scores each of the 9 DSM-IV criteria for depression as "0" 
(not at all) to "3" (nearly every day). PHQ-9 score > or =10 have a sensitivity of 88% and a specificity of 88% for 
major depression. 56; 5) The Quality of Life Index (QLI; 57, 58) is a 2-part, 33-item self-report measure of life 
satisfaction and importance regarding various aspects of life. The QLI demonstrates strong internal consistency, 
test-retest reliability, and convergent validity.57, 58 The same battery of instruments will be administered at 1-
month, 3-month, and 6-month follow-up.  
 
Fidelity Measures: Because marketing of the program has been limited to PTSD awareness campaigns 
and Facebook advertisements, it is anticipated that almost all Veterans will be unfamiliar with the site at the time 
of recruitment.  At the 1-month time point we will ask Veterans in the control condition if they have ever accessed 
AboutFace [1-month only] or whether they have accessed it since the baseline assessment or prior assessment. 
Additionally, we will be able to document from user logs the number of times Veterans in the AboutFace condition 
access the website, what they viewed, and for how long.  
 
Treatment Initiation & Number of Treatment Sessions Data: Treatment initiation and number of treatment 
sessions will be extracted from Veterans electronic medical records (Computerized Patient Record System; 
CPRS) by the project coordinator using relevant search parameters. More specifically, the project coordinator 
will document whether or not a Veteran presented for his or her first treatment session as well as the number of 
treatment sessions the Veteran completed. All sessions that take place in the PCT will be counted toward the 
total number of sessions. It is anticipated that the majority of these sessions will consist of PE or CPT, which are 
the two evidence-based therapies offered in the Charleston PCT clinic.   
 
11.0a SPECIMEN COLLECTION AND BANKING: N/A 
 
12.0a DATA MANAGEMENT:  
Sample size/Power (Quantitative Aims 1-2). There is a two-part approach to sample/size power 
assessment for the quantitative aims. First, for longitudinal outcomes (which can have missing data and for 
which longitudinal data modeling methods are used), we determine a sample size of 150/group (N=300) and 
then apply an inflation factor of 20% to account for dilution effect of intent-to-treat (ITT) analysis and number 
needed to impute to achieve a final ITT sample of 188 participants randomized to each group (total ITT 
N=376).  For primary outcomes for Aim 1 (therapy initiation and engagement), there will be no missing data 
and hence the full ITT sample of 188/group is used to assess power for these outcomes. For secondary 

12 
 
 
longitudinal outcomes, calculations are based on the reduced sample size (prior to inflation for missing data) of 
150 participants/group.  The details of these calculations are presented below. 
 
The primary outcomes for Specific Aim 1 (Hypothesis 1) are therapy initiation and therapy engagement. 
The primary analysis uses a two-part model approach (described below in Analysis section) for overall therapy 
engagement, which combines an "occurrence" or initiation component (zeros for number of sessions attended 
for those who do not initiate therapy) with an "intensity" component (number of therapy sessions attended >0 for 
those who did initiate therapy). For this calculation, we hypothesize the proportions of participants who initiate 
treatment in the eUC and AboutFace groups to be 0.70 and 0.83 (proportion of zero responses for two-part 
model: 0.30, 0.17), respectively. The hypothesized population value for initiation in eUC is based on a 0.70 
therapy initiation proportion observed in the PTSD Clinic in Charleston.  
 
To balance the need for determining a clinically meaningful improvement with that of feasibility of sample 
size, we hypothesize the initiation proportion in AboutFace as 0.83 (i.e., we assume an improvement from 0.7 to 
0.83 represents a clinically meaningful change).  The estimate of the proportion for AboutFace in our pilot study 
was 0.79. Based on Veteran feedback from the usability pilot, the study team made changes to AboutFace to 
improve navigation and searchability and added new features and content. In addition to the downloadable 
handout used in the pilot, which describes the purpose of and how to use AboutFace, the project coordinator 
now will contact Veterans to ensure they understand the procedure for accessing and using the site and are 
aware of relevant design features. These changes to the site, should improve the effectiveness of AboutFace. 
As such, we believe that the hypothesized proportion for treatment initiation for AboutFace of 0.83 is reasonable 
and supported by estimates from the pilot study.  
 
For overall therapy engagement, there will be 91% power to detect a standardized effect size of 0.25sd 
(Cohen's d, small effect size, assuming proportions of zeros of 0.3 and 0.17 for eUC and AboutFace groups, 
respectively).46   If the true proportion initiating therapy for AboutFace is as low as 80%, there will be 
approximately 80% power to detect a small standardized effect size (d=0.25) for overall therapy engagement.  
For the secondary comparison of simple proportions initiating therapy, we will have 84% power for the statistical 
test (assuming true population initiation proportions of 0.7 and 0.83 for eUC and AboutFace groups, 
respectively).  For secondary continuous longitudinal clinical (PCL, PHQ-9), quality of life (QLI), and stigma 
(EASI) outcomes (Specific Aim 1, Hypothesis 2; Specific Aim 2), there will be 85% power to detect a standardized 
effect size of 0.27sd [assuming nAF= neUC=150; 3 post baseline time points; intraclass correlation of 0.5; level of 
significance 0.05; two tailed tests]. To account for the impact of  missing data for longitudinal outcomes and the 
dilution effect of ITT analyses, the number of participants randomized to each treatment group to comprise the 
ITT sample is increased by 20% to achieve a final ITT sample size of 188 participants randomized to each 
treatment group (i.e., ITT N = 376). 
 
Descriptive analyses. Demographic and baseline characteristics will be described for the total sample 
and by intervention groups (AboutFace, eUC) using frequency distributions and summaries of central tendency 
(mean, median, variation). The intervention groups will be compared for imbalance using pooled t-test (or 
nonparametric alternative) for continuous variables and chi square (or Fisher's Exact Test) for categorical 
variables. These analyses will identify potential confounding variables to be used as covariates in subsequent 
analyses.   
 
Premature exits (drop-outs) and missing data.  For primary/secondary outcomes [therapy 
engagement/ therapy initiation] there will be no missing data.  For secondary longitudinal outcomes involving 
clinical, quality of life, and stigma/attitudes toward mental health, we will employ longitudinal data methods 
(mixed effects models) which allow for missing data under the assumption of data missing at random (MAR). 
While the MAR missing data mechanism can be justified in many situations, it is possible that the missing data 
will be missing not at random (MNAR) for some of the outcome measures (i.e., the probability of missingness is 
dependent on the missing data even after conditioning on the observed data).  While there are several general 
methods for dealing with NMAR data currently in use (including selection models, pattern mixture models, and 
shared parameter models), there is no current standard recommended method.  We will conduct  sensitivity 
analyses to explore the impact of different strategies and different distributional assumptions for MNAR on study 
conclusions. In addition, we will model the dichotomous outcome, missing/not missing end-of-study scores, using 
logistic regression to describe and compare the characteristics of subjects missing and not missing outcome 
endpoints. Analyses will be carried out separately for ITT and completer/per protocol samples to test sensitivity 

13 
 
 
of conclusions to dropouts/nonadherence. The intent-to-treat (ITT) sample, consisting of all randomized 
participants, will be used for the primary analyses. 
 
Multiple outcome variables. We acknowledge concern about inflation of Type I error given multiple 
secondary outcome variables. This concern must be balanced with the need to avoid overly conservative Type 
I (false positive) error rates that may result in excessively high Type II (false negative) error rates. For analyses 
for which there are a priori hypotheses relating to outcomes within specific domains or for which there are single 
outcomes (e.g. quality of life), we will consider the unadjusted results as primary. In secondary analyses to 
address the impact of multiple outcomes, we will conduct a series of analyses using different approaches. (1) 
We will use a generalized linear mixed model (GLMM) approach to model each dependent outcome separately 
and adjust for the multiple outcomes using a Bonferroni-type adjustment. Both the non-adjusted and adjusted p-
values will be reported as a measure of sensitivity of conclusions to the analysis of multiple outcomes.  (2) We 
will use a multivariate Generalized Linear Mixed Models (mGLMM) approach 59-61  to evaluate the relationship 
between the multiple outcomes and treatment by jointly modeling potentially correlated outcomes (see statistical 
analysis section). The mGLMM approach uses a random coefficient model with shared random intercept and 
random slope for each outcome. These models are capable of accommodating the correlation within each 
subject’s measurements over time, the correlation among the multiple outcomes from each subject, and allow 
for better description of the individual differences.62 Because the resulting overall covariance matrix (D) from this 
model provides standard error estimates that account for correlation among the multiple outcomes, no additional 
adjustment for multiple outcomes will be used.62-63 
 
Analysis plan: Specific Aim 1-Hypothesis 1. The primary/secondary outcomes for Aim 1 (Hypothesis 
1) are therapy engagement and therapy initiation (attend at least 1 therapy session/do not attend at least 1 
session). The outcome, therapy engagement, combines an "occurrence" (initiation) component (zeros for 
number of sessions attended for those who do not initiate therapy) with an "intensity" component (number of 
therapy sessions attended >0 for those who did initiate therapy). This outcome variable will have a distribution 
that is "zero-inflated", i.e., semi-continuous distribution characterized by two parts: (1) a point mass at zero (=0 
for those who do not initiate therapy) and (2) a positive continuous component (=number sessions completed for 
treatment initiators).  Methods for handling semi-continuous data of this type include standard two-part mixture 
models (TP) and, more recently, marginalized two-part models for semi-continuous data (MTP) .64,65 Both 
approaches involve models relating to the binary (occurrence) component for part (1) and models relating to the 
continuous (intensity) component for part (2).  The MTP approach allows a marginal (overall) interpretation of 
intervention and other covariate effects, i.e., allows for a single interpretation of intervention effects for the full 
population of those with both zero and non-zero responses for therapy engagement. The two-part marginal 
model method described by Smith and Gebregziabher 64,65 will be followed for these analyses.  The basic 
approach will include intervention (AboutFace, eUC) as the primary independent variable in the MTP model. The 
binary (occurrence) component will be modeled using the logit function; common choices for distributions of the 
non-zero (intensity) component of the MTP include a wide array of distributions [e.g., log-normal, log-skew-
normal, and the generalized gamma family of distributions]. The choice of distribution for the intensity component 
will be made after examining the observed distributions of the outcome variable and model fit parameters prior 
to breaking the partial (A,B) treatment blinding.  Additional covariates can be added to the basic model to adjust 
for putative confounding variables, if appropriate. Putative confounders include race/ethnicity, rural/urban 
dwelling status, baseline severity of clinical symptoms. In addition, in exploratory analyses, interaction terms can 
be added to the model to examine possible differential effect of intervention on the outcomes by specified 
covariables (e.g. demographic characteristics such as age, initial PTSD severity).  For example, an age-by-
intervention interaction term may indicate that AboutFace is more effective for encouraging younger rather than 
older Veterans to seek treatment. For the secondary comparison of proportion initiating treatment, we will use 
logistic regression analyses to compare treatment groups. In further descriptive analyses, we will estimate 
number (%), using 95% confidence intervals, of participants in both groups who do/do not access the web 
interface and will describe characteristics (e.g., age) of initiators vs non-initiators.   
 
Analysis plan: Specific Aim 1 (Hypothesis 2) and Specific Aim 2 (Hypothesis 1).  The outcomes for 
Aim 1 (Hypothesis 2) are PTSD symptom severity (PCL), depression (PHQ-9), and quality of life (QLI) measured 
at baseline, 3 months, and 6 months; the outcomes for Aim 2 (Hypothesis 1) are stigma and attitudes toward 
mental health care (EASI).   A generalized linear models (GLMM) approach will be used as the general analytic 
framework for inferential analyses for longitudinal outcomes. GLMM analyses allow for measurement of subjects 

14 
 
 
at different time points, missing data (MAR), and time varying or invariant covariates, and can account for the 
correlated measurements within subjects over time. In addition, GLMM can accommodate a wide range of 
distributional assumptions [e.g., continuous (e.g. normal), dichotomous/categorical/proportions (e.g., binomial), 
ordinal, count (e.g., Poisson, negative binomial)] through selection of an appropriate link function.  Each of 
outcomes will first be used separately as the dependent variable in the GLMM model with intervention, time, and 
time by intervention as the primary independent variables, baseline value of outcome as covariable, and within 
subjects correlation accounted for as a random effect in the model. In further multivariable analyses, additional 
covariates, as describe for Hypothesis 1 (above),  will be added to the model to adjust for putative confounding 
variables (if appropriate) and possible moderators of intervention effect will be explored through addition of 
interaction terms (see Hypothesis 1). Unadjusted and covariate-adjusted least squares means will be compared 
at the primary 3-month time point and at the secondary 6-month follow-up time point using appropriate model 
contrasts. Distributions of outcome variables will be evaluated and appropriate link functions will be selected 
prior to breaking the blinding of treatment assignment codes.   
 
Multiple correlated outcomes. As discussed above (see Section on Multiple Outcome Variables), 
mGLMM will be used as one method to control for multiple outcomes by jointly modeling the collection of 
variables within/between the separate health services domains, accounting for the correlation between the 
outcomes through the assumption of shared random intercept and random slope for each outcome. Advantages 
of mGLMM are: (1) flexibility in allowing for a common or global effect of a covariate (e.g., treatment) while 
accommodating dependence in the longitudinal outcomes through correlation in the random effects; (2) all 
univariate (single outcome) models implied by the mGLMM belong to the family of GLMMs 66; and (3) the same 
distribution does not necessarily have to be assumed for each of the responses given the different types of 
outcomes considered in this study.  For mGLMM, we will follow the analysis procedure outlined by Gebregziabher 
and colleagues.63   
 
Sensitivity analysis for MNAR missing data. While there are several general methods (multiple 
imputation, likelihood based modeling) for dealing with NMAR data (nonignorable dropout) currently in use 
(including selection models, pattern mixture models, and shared parameter models), there is no single current 
standard recommended method; the area of research is very active and evolving. For primary analyses of 
individual secondary clinical, quality of life, and stigma variables, we will use the standard longitudinal GLMM 
methods first assuming data are missing at random (MAR). In additional sensitivity analyses, we will use several 
approaches assuming NMAR data, including the shared parameter mixed effects models as described by 
Hedeker and Gibbons.67 For analyses of multiple correlated outcomes when data are NMAR, we will use a 
shared parameter joint modelling approach where the model for missingness is jointly modeled with the multiple 
longitudinal outcomes, which will provide valid inference.68 
 
Effect size estimation. Effect sizes for primary (difference in proportion initiating treatment and 
difference in mean number of therapy sessions completed for AboutFace vs eUC) and secondary (difference in 
EASI subscores and PCL, PHQ-9, QLI scores at each time point) outcomes will be described using 95% 
confidence intervals (CI). Both unadjusted and adjusted CI (obtained using appropriate model contrasts) will be 
estimated. 
 
13.0a PROVISIONS TO MONITOR THE DATA AND ENSURE SAFETY OF SUBJECTS:  The data 
and safety monitoring plan will include an internal Data Safety Monitoring Committee (DSMC), a Data and Safety 
Monitoring Board (DSMB), and the institutional IRB.  The purpose of the DSMC, DSMB, and IRB are to ensure 
the safety of participants and the validity and integrity of the data. Summaries of adverse event reports or patient 
safety concerns raised by the DSMB or IRB will be made to VA HSR&D on a yearly basis unless the nature of a 
particular event is such that it bears reporting immediately. 
  
DSMC: The internal DSMC will consist of the PI and co-investigators/consultants on the proposal. The 
functions of the DSMC will include: 1) providing scientific oversight; 2) reviewing all adverse effects or 
complications related to the study; 3) monitoring enrollment; 4) reviewing summary reports relating to compliance 
with protocol requirements; and 5) providing advice on resource allocation. The DSMC will meet quarterly and 
as necessary by telephone. The recommendations of the DSMC will be reviewed and the PI will take appropriate 
corrective actions as needed.  
 
DSMB: In addition to the internal DSMC, a DSMB will be established. The DSMB will be made up of  

15 
 
 
professionals with appropriate expertise, who are willing to participate, and who do not have any conflicts of 
interest. The DSMB will include: 1) an expert in the area of PTSD, 2) a biostatistician with expertise in the conduct 
of clinical trials, and 3) two members with expertise in the treatment of Veterans. The DSMB will meet annually. 
The DSMB will have the authority to temporarily or permanently discontinue the trial if it perceives that harm is 
occurring due to the intervention. The DSMB will meet with the internal DSMC yearly to review adverse event 
reports, patient complaints if any, and enrollment rates.  Data will be provided at these meetings by the 
investigators on key variables that may indicate harm, including changes in PTSD scores.  The DSMB 
biostatistician will evaluate the confidentiality and integrity of the database and the procedures for recording and 
storing confidential files. The DSMB will also review the elements of the plan to manage emergencies.  
Institutional IRB: MUSC IRB and VA R&D will review and approve the funded protocol, review patient and 
provider consent procedures, ensure protection of patient privacy and safety, and monitor the study on an on-
going basis. Adverse events will be reported to MUSC IRB and VA R&D as they occur. Annual reports to MUSC 
IRB and VA R&D will indicate enrollment rates, adverse events, new findings that may influence continuation of 
the study, and reports of the DSMB.  
14.0a WITHDRAWAL OF SUBJECTS: Veterans will be told that they do not have to answer any questions they 
do not want to and that they can stop using the educational materials at any point in time. They will also be told 
that they can withdraw their consent at any point in time. Veterans will only be withdrawn from the study by study 
personnel if it is deemed counter therapeutic.  
 
15.0a RISKS TO SUBJECT: The risks from being in the study are very small. There is a chance that answering 
questions may be upsetting.  Veterans will be told that they do not have to answer any questions they do not 
want to.  Veterans may also feel upset when reading about PTSD or watching the videos, and they will be told 
they can stop reading the booklet or watching the videos at any time.  For most people stopping the activity 
should make them feel better.  If a Veteran continues to feel upset by the questions or educational materials they 
will be instructed to contact Dr. Anouk Grubaugh at 843-532-6672 who is the onsite PI for the study. She is a 
licensed clinical psychologist and can help you find support for any distress caused by being in the study.  In the 
rare case where it appears that study participation is harmful to a participant, he or she will be terminated from 
the study, and referred for any services that may be indicated. 
There is also the risk of loss of confidentiality of personal information collected.  To minimize these risks, 
Paper documents pertaining to this study will be stored in locked file cabinets in the Charleston VAMC Mental 
Health Research Suite, and data will be entered into secure, password-protected databases developed for this 
study. A database of names, contact addresses, telephone numbers, and other research identification numbers 
will be stored separately from the study database, for purposes of audit by VA R&D or MUSC IRB, if necessary. 
Access to study data will be limited to research personnel. 
 
16.0 POTENTIAL BENEFITS TO SUBJECTS OR OTHERS: Veterans may not benefit from being in the 
research study. However, it is possible that being in the study might help them learn more about PTSD and 
PTSD treatment. Additionally, the information gathered during the course of the study may help other Veterans 
in the future. 
 
17.0a SHARING OF RESULTS WITH SUBJECTS: We expect that it will take about 6 months from the end of 
the study to analyze the information we get in this study.  We will write reports and we will make these reports 
available to subjects who were in this study if they ask for them.  No Veteran will be named in any of these 
reports.  All information gathered will be kept confidential as far as possible by law.  Veterans will be given 
contact information for both PIs, so that they can request a copy of the report(s).  
 
18.0a DRUGS OR DEVICES: N/A 
 
 
II. STAKEHOLDER INTERVIEWS (PROVIDER COMPONENT)  

16 
 
 
 
[See 1.0-2.0 for Aims and Background of Study] 
 
3.0b  INTERVENTION TO BE STUDIED: We will conduct thematic interviews with VA PCT directors. We are 
focusing solely on PCT directors because most have both decision-making control about clinic procedures and 
function as frontline clinicians. We considered interviewing providers from other clinics (e.g., general mental 
health, primary care, etc.) but felt it would be premature given that we have not studied the effectiveness of using 
AboutFace in those settings. Our thematic interview will follow a structured format but will use open-ended 
questions. We have chosen this approach because it offers the advantage of making interviews more systematic 
than informal, conversational interview approaches; at the same time, however, it allows for more flexibility in 
eliciting individual perspectives and experiences relative to a standardized interview approach.69 The study team 
has successfully used this approach in their previous work (CD207015; R01MH074151; R21 MH065248; IIR-
08-323-2). 
 
4.0b STUDY ENDPOINTS: Providers will participate in one (1) thematic interview unless the initial interview is 
cut short for some unforeseen reason and must to divided into 2 sessions in order to complete.  
 
5.0b-6.0b INCLUSION/EXCLUSION CRITERIA/NUMBER OF SUBJECTS: Any PCT Director from a complete 
list of 120 Directors nationwide will be eligible to participate in the thematic interview portion of the study. No 
other restrictions will be applied. We anticipate interviewing between 20-30 providers until a point of saturation 
is achieved (described below) in the sample size section below. 
 
7.0b-8.0b SETTING/RECRUITMENT METHODS: A complete list of the 120 PCTs across the US will be created. 
Each program will be coded on urbanicity (metro/non-metro), region (Northeast, Southeast, etc.), and program 
size (small, medium, and large, defined according to thirds of the distribution of the number of patients served). 
Programs will be organized into strata on the basis of these three characteristics, and then directors of these 
programs will be selected randomly for inclusion in the study. Dr. Davidson, a study Co-Investigator is trained in 
the conduct of qualitative interviews and will consent providers and conduct the thematic interviews. We 
anticipate interviewing between 20-30 providers until a point of saturation is achieved (described below) in the 
sample size section.  
 
9.0b CONSENT PROCESS: Dr. Davidson, a co-investigator on the study, will complete the thematic interviews. 
Using the list described above, Dr. Davidson will call providers to assess their interest in participating in the 
study. If a provider expresses a willingness to participate, Dr. Davidson will set-up an appointment time to obtain 
verbal informed consent and complete the thematic interview. At the time of the telephone call/thematic interview, 
Dr. Davidson will obtain verbal informed consent using a standardized script, and she will answer any questions 
the provider may have. The script will contain details about the study procedures. Due to the nature of the 
questions posed a written informed consent will be waived (i.e., a signed informed consent document will not be 
required). However, in addition to details about the study procedures, providers will be told that their study 
participation is voluntary and that they can refuse to participate in the study, decline to answer any specific 
questions, or discontinue the study at any point in time. The consent script will also briefly detail the information 
that will be gathered as part of their study participation, any risks and benefits to participation, and how this 
information will be safeguarded (see Consent Script in Appendix). Providers will not start the thematic interview 
until verbal consent is given.  
  
10.0b STUDY DESIGN/METHODS: The Director of the National Center for PTSD Mentoring Program will send 
an Email to identified program directors inviting them to participate.  In a previous qualitative study of this 
population using this same methodology we had 92.8% response rate.42 Dr. Davidson will conduct the thematic 
interviews (see budget justification for more detailed role designations).  Based on our previous work, we 
anticipate that the thematic interviews will take about 45 minutes to complete. After hour interviews will be 
offered.  
Thematic interviews will involve asking open-ended questions, listening to and recording the responses, 
taking careful notes, and then following up with additional relevant questions. For this purpose, we have prepared 
a draft interview guide (included in the Appendices) with a list of open-ended questions that are to be explored 
in the course of the interview. The interview guide is flexible, in that it provides topics and subject areas of inquiry 

17 
 
 
but allows the interviewer the freedom to explore, probe, and ask relevant follow-up questions. Example 
questions include:  
 
• 
What do you do in your clinic to try and improve treatment initiation and retention? 
• 
What elements of AboutFace do you think are potentially the most helpful for engaging Veterans in 
treatment? What about for retaining them in treatment? 
• 
If the data supporting AboutFace were compelling with regard to promoting treatment initiation and 
retention, how likely would it be that your team would consider integrating it into your intake process?  
• 
What recommendations do you have for best incorporating AboutFace into clinical practice?   
• 
What barriers do you see to integrating AboutFace into your intake [or clinic process if noted above]?  
• 
What types of training or support would be needed to integrate AboutFace into your intake process [or 
clinic process if noted above]?  
 
11.0b SPECIMEN COLLECTION & BANKING: N/A 
  
12.0b DATA MANAGEMENT:  
Sample Size. There are no clear accepted guidelines or statistical formulas by which to provide a 
rationale for targeted sample sizes in qualitative research. Rather than pre-specify the number of participants, it 
typically makes more sense with qualitative research methods to continue the data collection until there is team 
consensus that a “saturation” point has been reached (i.e., the point at which relatively little new information is 
being revealed in the course of interviews). Consistent with this approach, we will continue to conduct interviews 
until we believe relatively little new information is being collected. Based on our previous work (CD 207015; R01 
MH074151; R21MH065248; IIR-08-323-2), we believe a point of saturation will be reached with approximately 
20-30 PTSD clinic directors. One variable we will be sensitive to is clinic design.  We will continue to conduct 
interviews until we have selected directors from programs with no preparatory or orientation groups, as well as 
from those with brief orientation and longer skills-based groups. This is important because how AboutFace would 
be integrated in the intake process will depend on the clinic design. If after 20 interviews we do not have at least 
5 programs with minimal preparatory/orientation requirements (i.e., 0-1 session), 5 with brief requirements (i.e., 
2-3 preparatory sessions), and 5 with at least moderate requirements (i.e., 4 or more preparatory sessions), we 
will randomly select another 5 programs and continue sampling until we fill these clinic design conditions and 
reach “saturation”. 
Analysis Plan. Qualitative data from stakeholder interviews will be transcribed and imported into NVivo 
10 70 qualitative analysis software. The qualitative approach chosen for this study is derived from constructivist 
grounded theory; this approach acknowledges the researcher’s prior knowledge and influence in the process 
and provides guidelines for building a conceptual framework to understand the interrelations (e.g., the what and 
how) between constructs. 71 NVivo 10 will be used to identify common themes (coded as nodes) as they relate 
to specific aspects of implementation and dissemination of AboutFace. Initial and secondary coding passes will 
be conducted by Drs. Grubaugh and Hamblen to refine theme classifications, to minimize redundancies, and to 
impose a data-derived hierarchy to the nodes identified.  As a form of verification, 72 Drs. Ruggiero and Davidson 
will independently code a random selection of 20% of these cases and interclass correlation coefficients will be 
computed to assess reliability.  
 
Implementation Plan. Our implementation plan will be guided by data obtained from key 
stakeholders [Aim 3]. Using what we learn from these qualitative sources of information, we will work with two 
National Center for PTSD programs to assure we can incorporate AboutFace into clinics as appropriate.  
Through the PTSD Mentoring Program we will be able to reach all of the PCTs. We will formally present 
findings on the monthly program call and ask for a few sites to serve as pilots to incorporate AboutFace into 
their clinic process. Consistent with study methodology we will look for variability across sites to learn how 
AboutFace works in clinics with few preparatory sessions that offer primarily time limited evidenced based 
treatment to clinics with longer term therapy options that tend to emphasize the need for patient readiness. We 
will problem solve barriers to implementation with the pilot sites and then report back to the larger group on the 
monthly PTSD Mentor calls and share feedback.  Once we feel we have a workable implementation strategy 
we will ask the other sites to incorporate AboutFace into their programs.  Although the National Center for 
PTSD does not set policy for the programs, it is our experience that the majority of the specialized PTSD 
programs are excited and willing to try Center initiatives that improve care.  
 

18 
 
 
We will also work with the Practice Based Implementation Network (PBI Network). The PBI Network is a 
collaboration between VA and DoD designed to engage providers and leadership in the implementation of best 
practices.  The Network currently has more than 18 sites enrolled and over 130 providers. They are currently 
focusing on training and support for integration of mobile apps and web-based self-help tools as part of routine 
treatment for Veterans with PTSD. The PBI Network has agreed to add AboutFace as one of the programs they 
will support (see letter of support from Dr. Ruzek, Co-I).  Sites will attend a virtual workshop and will select a 
program(s) to use in their clinic for a 12 week period.  They will also create a formal implementation plan and will 
have access to additional resources including online material and conference calls with peers and subject matter 
experts. Ongoing program evaluation through surveys and interviews will be collected and shared within the 
Network.  
 
The National Center for PTSD will produce relevant training materials guided by what is learned in the 
qualitative interviews.  For example, we expect that a facilitator’s guide would be developed to give programs 
strategies for how to integrate AboutFace in clinical care.  A slide set could be created for use in orientation 
groups.  The patient handout describing AboutFace used in the study may also need to be revised. 
 
13.0b PROVISIONS TO MONITOR THE DATA TO ENSURE THE SAFETY OF SUBJECTS: N/A  
 
14.0b WITHDRAWAL OF SUBJECTS: Providers will be told as part of the consent process that they are free 
to refuse to participate in the study, to not answer specific questions during the thematic interview, or to withdraw 
from the study at any point in time. They will also be told that their employment will not be affected by their 
participation or refusal to participate in the study.  
15.0b RISKS TO SUBJECTS: We do not foresee any risks to providers from the interview. If a provider does 
feel significant distress at any time, or if she/he changes her/his mind about participating, they may stop 
participating at any point in time or choose to not answer any questions during the interview that they do not 
want to answer. Audio recordings will be heard only by study investigators and the person who transcribes the 
sessions. The  name of the provider and  the recording will not be shared with those outside the study and the 
providers name will not appear on the transcripts or in any reports. 
 
16.0b POTENTIAL BENEFITS TO SUBJECTS OR OTHERS: There are no significant benefits to providers 
personally for participating in the study. However, one possible benefit of participating is that Veterans might be 
helped by the findings of this study. 
 
17.0b SHARING OF RESULTS WITH SUBJECTS: We expect that it will take about 6 months from the end of 
the study to analyze the information we get in this study.  We will write reports and we will make these reports 
available to subjects who were in this study if they ask for them.  No PCT Director will be named in any of these 
reports.  All information gathered will be kept confidential as far as possible by law.  PCT Directors will be given 
contact information for both PIs, so that they can request a copy of the report(s). 
 
18.0b DRUGS OR DEVISES: N/A 
 
 
 
Project Management Plan 
The anticipated duration of this study is four years (48 months). Over the course of the project, Drs. Grubaugh 
and Hamblen will hold monthly conference calls with all members of the investigative team to establish goals, 
monitor study progress, and problem solve any difficulties that may have emerged. Dr. Hamblen will also travel 
to Charleston in Year 1 and 4 of the project. During the first six months of the project, Drs. Grubaugh and 
Hamblen will train project staff who are employed through the COIN on other HSR&D funded projects,establish 
the infrastructure for protocol procedures and begin development of the study database. Months 7-42 will be 
used to recruit subjects, conduct assessments for the RCT component of the study, and data entry/cleaning. 
Months 13-42 will be used to conduct thematic interviews, transcription, and thematic coding for the 
stakeholder component of the study. As the research/clinic liaison, the RA will attend relevant PCT staffing 
meetings. Further, he/she will regularly interface with VAMC clinicians about opportunities for their patients to 
participate in the study. Months 30-48 will be used to prepare relevant manuscripts/presentations including 

19 
 
 
preparation of final reports and the primary outcome paper in the last 6 months of the study.  Dissemination of 
project goals/findings will occur as early as year 2 through relevant presentations at professional conferences 
and VA venues and publications (e.g. in journals that publish ongoing clinical trials) and continue through the 
duration of the study and beyond. See Table 2 below for project timeline.  
 
  Table 2: Project Timeline 
1 
2 
3 
4 
                                                                     Quarters → 
1 
2 
3 
4 
1 
2 
3 
4 
1 
2 
3 
4 
1 
2 
3 
4 
Staff Training; Establish Patient Safety Procedures; Purchase Study 
Equipment  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
RCT Enrollment, Intervention; F/U assessments; Data 
Entry/Cleaning/Analysis 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Stakeholder Interviews; Transcription; Data Coding/Analysis 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Dissemination Activities 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Manuscript/Presentation Preparation/Final Reports/Main Outcome 
Paper 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Drs. Grubaugh and Hamblen will jointly oversee all aspects of the proposed project. Dr. Grubaugh will 
provide on-site supervision to the project staff in data collection activities at the Charleston VAMC facility. The 
team in its entirety will have ongoing discussion of recruitment and progress; drafting and editing content for the 
assessments; data collection and management; interviews and interviewer training; and addressing ongoing 
challenges. Regular (weekly and as-needed) supervisory meetings will be held with all investigators and staff 
directly in contact with participants to address recruitment and procedural issues. Meetings with the study team 
also will include discussion of data collection and analyses, manuscript submissions, and dissemination 
activities. Ongoing implementation challenges will be addressed in the monthly calls.  
 
The National Center for PTSD is also well positioned to disseminate the findings to other mental health 
service lines as well, such as general mental health and primary care and to work directly with the evidence 
based practice coordinates within each VA. Other dissemination activities will include (1) authorship of 
descriptive materials in the form of white papers and brochures; (2) publication of study results in professional 
and non-professional outlets; and (3) local and national presentations and seminars. Drs. Grubaugh and 
Hamblen will record and track all dissemination activities, and they will direct the scope and frequency of these 
activities as indicated. Using the above mechanisms, we believe dissemination of our findings will be greatly 
enhanced over standard procedures. 
 
REFERENCES 
1. Hoge CW, Auchterlonie J, & Milliken C. (2006). Mental health problems, use of mental health services, 
attrition from military service after returning from deployment to Iraq or Afghanistan. JAMA, 295, 1023-1032. 
2. Wright KM, Huffman AH, Adler AB, et al. (2002). Psychological screening program overview.  Military 
Medicine, 167, 853-861.   
3. Dohrenwend BP, Turner JB, Turse NA, et al. (2006). The psychological risks of Vietnam for U.S. veterans: 
A revisit with new data and methods. Science, 313, 979-82. 
4. Kok BC, Herrell RK, Thomas JL, et al. (2012). Posttraumatic stress disorder associated with combat 
service in Iraq or Afghanistan: Reconciling prevalence differences between studies. J Nerv Ment Dis., 200, 
444-450. 
5. Blow AJ, Gorman L, Ganoczy D, et al. (2013). Hazardous drinking and family functioning in National Guard 
veterans and spouses post-deployment. J Fam Psyc, 27, 303-313. 
6. Cohen BE, Gima K, Bertenthal D, et al. (2010). Mental health diagnoses and utilization of VA non-mental 
health medical services among returning Iraq and Afghanistan Veterans. J Gen Intern Med, 25, 18-24. 
7. Corson K, Denneson LM, Bair MJ, et al. (2013). Prevalence/correlates of suicidal ideation in OEF/OIF 
veterans. J Affective Disorders, 149, 291-298. 
8. Hankin CS, Spiro A, Miller DR, et al. (1999). Mental disorders and mental health treatment among US 
Department of Veterans Affairs outpatients: The veterans health study. Am J Psychiatry, 156, 1924-1930. 
9. Hoge CW, Grossman SH, Auchterlonie JL, et al. (2014). PTSD treatment for soldiers after combat 
deployment: Low utilization of mental health care and reasons for dropout. Psychiatr Serv. 65, 997-1004. 

20 
 
 
10. Kim PY, Thomas JL, Wilk JE, et al. (2010). Stigma, barriers to care, and use of mental health services among 
active duty and National Guard soldiers after combat. Psychiatr Serv., 61, 582-588. 
11. Bonn-Miller MO, Boden MT, Vujanovic AA, et al. (2013). Prospective investigation of the impact of cannabis 
use disorders on PTSD symptoms among veterans in residential treatment. Psychol Trauma: Theory, Res, 
Pract, Policy, 5, 193-200. 
12. Flynn HA, Walton MA, Curran GM, et al. (2004). Psychological distress and return to substance use two 
years following treatment. Substance Use & Misuse, 39, 885-910. 
13. Kessler RC. (2000). Gender differences in major depression: Epidemiologic findings. In: Frank E, editor. 
Gender and Its Effect on Psychopathology. Washington DC: American Psychiatric Press, 61–84. 
14. Seal KH, Bertenthal D, Miner CR, et al. (2007). Bringing the war back home: Mental health disorders among 
103 788 US veterans returning from Iraq and Afghanistan seen at DVA Facilities. Arch Internal Med, 167, 
476-482. 
15. Grubaugh AL, Cain GD, Elhai JD, et al. (2008). Attitudes toward medical and mental health care delivered 
via telehealth applications among rural and urban primary care patients. J of Nervous & Mental Disease, 196, 
166-170. 
16. Weeks WB, Wallace AE, Wang S, et al. (2006). Rural-urban disparities in health-related quality of life within 
disease categories of Veterans. J Rural Health, 22, 204-211. 
17. Han BL, Hedden SL, Lipari R, et al. (2015). Receipt of services for behavioral health problems: Results 
from the 2014 National Survey on Drug Use and 
Health.https://www.samhsa.gov/data/sites/default/files/NSDUH-DR-FRR3-2014/NSDUH-DR-FRR3-
2014/NSDUH-DR-FRR3-2014.pdf). 
18. Hoge CW, Grossman SH, Auchterlonie JL, et al. (2014). PTSD treatment for soldiers after combat 
deployment: Low utilization of mental health care and reasons for dropout. Psychiatr Serv., 65, 997-1004. 
19. Chard KM, Schumm JA, Owens GP, et al. (2010). A comparison of OEF and OIF veterans and Vietnam 
veterans receiving cognitive processing therapy. J Traumatic Stress, 23, 25-32. 
20. Gros D, Yoder M, Tuerk PW, et al. (2011). Exposure therapy for PTSD delivered to veterans via telehealth. 
Beh Ther, 42, 276-283. 
21. Yoder M, Tuerk PW, Price M, et al. (2012). Prolonged Exposure for combat-related PTSD. Psychol Serv, 9, 
16-25. 
22. Wolf GK, Strom TQ, Kehle SM, et al. (2012). A preliminary examination of prolonged exposure therapy with 
Iraq and Afghanistan veterans with a diagnosis of posttraumatic stress disorder and mild to moderate 
traumatic brain injury. The J of Head Trauma Rehabilitation, 27, 26-32. 
23. Karlin BE, Ruzek JI, Chard KM, et al. (2010). Dissemination of evidence-based psychological treatments for 
posttraumatic stress disorder in the Veterans Health Administration. J Traumatic Stress, 23, 663-673. 
24. Karlin BE, Brown GK, Trockel M, et al. (2012). National dissemination of CBT for depression in the DVA 
Health Care System: Therapist and patient-level outcomes. J Consult Clin Psychol, 80, 707-718. 
25. Hoerster KD, Malte CA, Ahmad Z, et al. (2012). Association between perceived barriers and prospective VA 
mental healthcare use among Iraq and Afghanistan War Veterans. Psychiatr Serv., 36, 380-382. 
26. Iversen A, Dyson C, Smith N, et al. (2005). Goodbye & good luck: The mental health needs and treatment 
experiences of British ex-service personnel. Br J Psychiatry, 186, 480-486. 
27. Seal KH, Maguen S, Cohen B, et al. (2010). VA mental health services utilization in OIF OEF veterans. J 
Traumatic Stress, 23, 5-16. 
28. Blais R & Renshaw K (2013). Stigma and demographic correlates of help‐seeking intentions in returning 
service members. J Traumatic Stress, 26, 77-85. 
29. Kim PY, Britt TW, Klocko RP, et al. (2011). Stigma, negative attitudes about treatment, and utilization of 
mental health care among soldiers. Military Psychology, 23, 65-81. 
30. Zinzow HM, Britt TW, McFadden AC, et al. (2012). Connecting active duty and returning Veterans to mental 
health treatment. Clin Psychol Rev., 32, 741-753. 
31. http://www.pewresearch.org/fact-tank/2017/01/12/evolution-of-technology/ 
32. Clement S, Schauman O, Graham T, et al. (2015). What is the impact of mental health-related stigma on 
help-seeking? A systematic review of quantitative and qualitative studies. Psychol Med., 45, 11-27. 
33. Gulliver A, Griffiths KM, & Christensen H. (2010). Perceived barriers and facilitators to mental health help-
seeking in young people: a systematic review. BMC Psychiatry, 10: 113. 

21 
 
 
34. Hoge CW, Castro CA, Messer SC, et al. (2004). Combat duty in Iraq and Afghanistan, mental health, barriers 
to care. NEJM, 351, 13-22. 
35. Corrigan PW, River LP, Lundin RK, et al. (2001). Three strategies for changing attributions about severe 
mental illness. Schizophr Bull., 27, 187-95. 
36. Mahat G, Scoloveno MA, & Ayres C. (2011). HIV/AIDS knowledge and self-efficacy among Nepalese 
adolescents: A peer education program. Res Theory Nurs Pract., 25, 271-83. 
37. Medley A, Kennedy C, O’Reilly K, et al. (2009). Effectiveness of peer education interventions for HIV 
prevention in developing countries: A systematic review and meta-analysis. AIDS Education and 
Prevention: Official Publication of the International Society for AIDS Education, 21, 181. 
38. White S, Park YS, Israel T, et al. (2009). Longitudinal evaluation of peer health education on a college 
campus: Impact on health behaviors. Journal of American College Health, 57, 497-506. 
39. Conner KO, McKinnon SA, Ward CJ, et al. (2015). Peer education as a strategy for reducing 
internalized stigma among depressed older adults. Psychiatr Rehabil J., 38,186-93. 
40. Naslund JA, Aschbrenner KA, Marsch LA, et al. (2016). The future of mental health care: Peer-to-peer 
support and social media. Epidemiol Psychiatr Sci., 25,113-22.  
41. Berger M, Wagner TH, Baker LC. (2005). Internet use and stigmatized illness. Social Science & Medicine, 
61, 1821-1827.  
42. Hamblen JL, Bernardy NC, Sherrieb K, et al. (2015). VA PTSD clinic director perspectives: How 
perceptions of readiness influence delivery of evidence-based PTSD treatment. Professional Psychology: 
Research and Practice, 46, 90-96. 
43. Knadfl K. (2009). Cognitive interviewing.  In L. Givens (Ed.), The Sage encyclopedia of qualitative research. 
Vol 2. Thousand Oaks, CA: Sage. 
44. Kraemer HC. (2010). Randomized Clinical Trial Design:  50 Years of Learning from Mistakes. New Research 
Approaches for Mental Health Interventions. Boca Raton, FL. 
45. Vogt D, Di Leone BAL, Wang JM, et al. (2014). Endorsed and Anticipated Stigma Inventory (EASI): A Tool 
for assessing beliefs about mental illness and mental health treatment among military personnel and 
Veterans. Psychological Services,11,105-113. 
46. Lachenbruch PA. (2001).Power and sample size requirement for two-part models. Statistics in Medicine, 
20, 1235-1238. 
47. Zickuhr K. (2013). Who’s not online and why. Pew Research Center’s Internet & American Life Project. 
Accessed 01-10-14 from 
pewinternet.org/~/media//Files/Reports/2013/PIP_Offline%20adults_092513_PDF.pdf 
48. Korset 
R. 
(2013). 
World 
Disasters 
Report 
2013. 
Accessed 
on 
06-02-14 
from 
redcross.se/PageFiles/6570/WDR_2013_complete.pdf 
49. Weathers FW, Litz BT, Keane TM, et al. (2013).The PTSD Checklist for DSM-5 (PCL-5). Scale available 
from the National Center for PTSD at ptsd.va.gov. 
50. Ruggiero KJ, Del Ben K, Scotti J, Rabalais A. (2003). Psychometric properties of the PTSD Checklist. J 
Traumatic Stress, 16, 495-502.  
51. Ruggiero KJ, Rheingold AA, Resnick HS, et al. (2006). Comparison of two widely used PTSD‐screening 
instruments: Implications for public mental health planning. J Traumatic Stress, 19, 699-707. 
52. Blanchard EB, Jones-Alexander J, Buckley TC, et al. (1996). Psychometric properties of the PTSD Checklist. 
Beh Res Ther, 34, 669-673. 
53. Forbes D, Creamer M, Biddle D. (2001). The validity of the PCL. BRAT, 39, 977-986. 
54. Weathers FW, Litz BT, Herman DS, et al. (1993). The PTSD checklist: reliability, validity, & diagnostic utility. 
Paper presented at the Annual Meeting of ISTSS, San Antonio, TX, October. 
55. Marx, B. (2013, November). The PTSD Checklist-5 (PCL-5). Pre-meeting institute, International Society on 
Traumatic Stress Studies, Philadelphia. 
56. Kroenke K, Spitzer RL, Williams JB. (2001). The PHQ-9: Validity of a brief depression severity measure. J 
Gen Intern Med, 16, 606-613. 
57. Ferrans C &  Powers M. (1985). Quality of Life Index: Development and psychometric properties. Advances 
in Nursing Science, 8, 15-24. 
58. Ferrans C &  Powers M. (1992). Psychometric assessment of the Quality of Life Index. Research in 

22 
 
 
Nursing and Health, 15, 29-38. 
59. Gibbons RD, Hedeker D, & DuToit S. (2010). Advances in longitudinal analysis. Annu Rev Clin Psychol, 6, 
79-107. 
60.  Fitzmaurice GM, Laird NM, & Ware, JH. (2004). Applied Longitudinal Analysis. New York: John Wiley & 
Sons. 
61. Gebregziabher M, Zhao Y, Dismuke C, et al. (2013). Joint Modeling of longitudinal multiple source cost data 
using multivariate generalized linear mixed models. Health Services and Outcomes Research Methodology, 
13, 39-57. 
62. Gibbons RD, Hedeker D, & Stephen DT. (2010). Advances in the analysis of longitudinal data. Annu Rev 
Clin Psychol, 6, 79-107. 
63. Gebregziabher M, Zhao Y, Dismuke CE, et al. (2013). Joint modeling of multiple longitudinal cost outcomes 
using multivariate generalized linear mixed models. Health Serv Outcomes Res Method, 13, 39–57. 
64. Smith VA, Preisser JS, Neelon B, et al.  (2014). A marginalized two-part model for semi-continuous data. 
Statistics in Medicine, 33 489-490. 
65. Gebregzhiabher M, Voronca D, Teklehaimanot A, et al. (2015).  Weibull mixture regression for marginal 
inference in zero-heavy continuous outcomes. Statistical Methods in Medical Research. Epub. 
66. Fieuws S & Verbeke G. (2004). Joint modelling of multivariate longitudinal profiles: Pitfalls of the random-
effects approach. Statistics in Medicine, 23, 3093–3104. 
67. Hedeker D & Gibbons R. (2006). Longitudinal Data Analysis. John Wiley & Sons, New Jersey. 
68. Shah A, Laird NM, & Schoenfeld D. (1997).  A random effects model with multiple characteristics with 
possibly missing data. J. Am. Stat. Assoc., 92, 775–779. 
69. Rubin, H. & Rubin, I. (2005). Qualitative Interviewing: The Art of Hearing Data (2nd ed.). e[ Thousand 
Oaks, CA: Sage. 
70. NVivo 9 qualitative data analysis software (version 9) [computer program]2010. 
71. Charmaz K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. New York, 
New York: Sage Publications. 
72. Sweeney A, Greenwood KE, Williams S, et al. (2013). Hearing the voices of service user resaerchers in 
collaborative qualiatative data analysis: The case for multiple coding. Health Expect., 16, e89-99.  
73. Grubaugh AL, Tuerk PW, Egede LE, et al. (2012). Perceptions of PTSD research participation among 
patients with severe mental illness. Psychiatry Res., 200, 1071-1073. 
74. Legerski JP & Bunnell SL. (2010). The risks, benefits, and ethics of trauma-focused research 
participation.  Ethics & Behavior, 20, 429-442.