Designing a Mobile App to Support Academic Success for Student Veterans
Phase: NA
Conditions: Mental Health, Depression, Post-Traumatic Stress Disorder, Anxiety, Substance Use Disorder
Sponsor: VA Office of Research and Development

Page 1 of 30 
 
 IRB Protocol Template (3/1919) 
HUMAN SUBJECTS RESEARCH PROTOCOL 
 
 TEMPLATE INSTRUCTIONS 
03/19/2019 
 
 
Project Title: Designing a Mobile App to Support Academic Success for Student Veterans 
 
Protocol Version and Date:  Version 1.0; 4/5/2022 
 
Principal Investigator: Erin D. Reilly, PHD 
 
Co-Investigators: Megan Kelly, PHD; Lisa Mueller, PHD 
 
Institution(s): VA Bedford Healthcare System  
 
1.0 
Objective and Specific Aims: 
In 2018, over 669,922 Veterans used military benefits to pursue higher education, with 75% 
enrolled as full-time students. For these Veterans, advancing their education is an important step toward 
their continued community reintegration, including improved employment opportunities, stable housing, 
family and social roles, and other outcomes. Though Veterans bring multiple strengths to academic 
settings, transitioning to the new culture of higher education may also involve significant challenges that 
impact their academic performance. Specifically, student Veterans experience higher rates of health risk 
behaviors (e.g., substance use), self-reported difficulties adjusting to campus life, and interpersonal 
difficulties with faculty than their civilian peers. These issues can lead to lower enrollment rates and 
slower degree attainment, which seriously impede Veterans’ successful adjustment to civilian life and 
vocational success. 
One particular concern is that Veterans transitioning back to civilian life have diagnosed mental 
health conditions at elevated rates compared to civilian students. As a result, Veterans face difficulties 
related to cognitive (e.g., attention), physical (e.g., pain), or emotional/behavioral (e.g., PTSD, 
depression, substance use) issues. Many Veterans do not report psychiatric problems and do not seek 
treatment, and over half of Veterans who have a mental health issues may not perceive a need for mental 
health. Lack of services and treatment is especially concerning for student Veterans, who have higher 
rates of mental health symptoms and suicide than non-Veteran students, but lower rates of treatment. 
Even for Veterans seeking treatment for academic and psychiatric issues, there can be many barriers 
such as time, transportation, and perhaps most importantly, not understanding the process for becoming 
connected to existing VA and campus services.  
The VA has invested in many in-person resources for academic support and mental health 
treatment, using a catalogue of supported education options for Veterans embedded in Compensated 
Work Therapy (CWT), Veteran Integration to Academic Leadership (VITAL), and mental health clinics. 
Preliminary evidence from supported education pilots such as VITAL- Supported Education (VITAL-
SEd) have not only shown promise in assisting Veterans with their educational success, but have also 
gathered feedback from Veterans on specific areas of treatment that assisted them. However, in-person 
supported education is not available to all – and Veterans across the spectrum of academic and mental 
health concerns could benefit from an asynchronous, mobile application providing a one-stop shop for 
student Veteran support.  

Page 2 of 30 
 
 IRB Protocol Template (3/1919) 
Studies with non-Veteran student populations have shown that targeted mobile platforms to 
orient students to the culture of higher education, while also focusing on their mental health, can 
increase students’ retention and course engagement. Unfortunately, there is no VA mobile app 
specifically aimed toward student Veterans. We plan to develop a personalized academic success app, 
VetEd, with content based in past research, educational literature, and Veteran feedback, to address a 
variety of academic and psychiatric symptom-related educational barriers for student Veterans. 
Specifically, VetEd will provide services to student Veterans who may not yet have access to supported 
education services by 1) promoting the creation and execution of educational planning and goals, 2) 
providing opportunities for Veterans to learn new academic and coping skills such as course 
organization, task completion, and stress management, and 3) connecting Veterans with VA resources 
based off their personal needs, such as information on mental health counseling, existing college 
resources like obtaining educational accommodations, and Post-9/11 GI Bill coverage. 
 
Primary Aims:  
 
Aim 1. Develop a Veteran-centered educational support app by conducting focus groups [(3 focus 
groups, n=8, n=8, n=5, total n= 21)] of student Veterans with mental health disorders to identify their 
perceived academic needs, app preferences, and evaluate Veteran-centered content. 
 
Aim 2. Test and iteratively revise the VetEd app (3 waves of n= 5, total n =15) by assessing app 
software, content, human-computer interface, usability, satisfaction data, and preliminary exploration of 
changes in educational functioning (course activity completion, academic self-efficacy, and retention). 
 
Hypotheses: Although we are underpowered to detect statistically significant changes in 
educational functioning scores, we will utilize non-parametric analyses to assess for potential 
changes from pre- and post-app use related to academic functioning (academic self-efficacy, 
course task completion, and perceived retention/course completion) and community reintegration 
(M2C-Q). 
Secondary Aims: N/A 
 
2.0 
Background and Significance 
 
2.1 Background 
 
 Higher education has a vital, lasting impact on Veteran’s functioning and quality of life. 
As service members transition out of active duty, many decide to pursue higher education as the next 
step toward secure employment and a higher quality of life, with financial support for college often cited 
as a primary motivator for enlisting in the military.1 In 2018, over 669,922 Veterans used military 
benefits to pursue higher education, with 75% enrolled as full-time students.2 Since the Post 9/11 GI Bill 
was introduced in 2009, more than 1.4 million Veterans, service members, and their families have used 
it to fund their education.3 Many Veterans view education as a necessary step toward vocational success, 
as people with higher levels of education are more likely to be employed4 and earn more over their 
lifetime.5 Quality of life is also improved with educational success, as Veterans with at least some 
college show improved employment opportunities, stable housing, and family and social roles.6,7,8  
Student Veterans with mental health disorders need additional resources to meet academic goals. 
While Veterans bring multiple strengths to academic settings, transitioning to the culture of higher education can 
be complicated by challenges impacting daily functioning and academic performance. One particular concern is 

Page 3 of 30 
 
 IRB Protocol Template (3/1919) 
that Veterans have mental health conditions at elevated rates compared to civilians.9 This is particularly 
concerning for student Veterans, who have higher rates of mental health symptoms and suicide than 
non-Veteran students, but lower rates of treatment,10 with up to 50% not engaged in mental health care.11 
Student Veterans with mental health conditions can have additional problems meeting the educational 
goals, as they experience higher rates of health risk behaviors (e.g., substance use), difficulties adjusting 
to campus life,11 and interpersonal concerns with faculty.12 These Veterans can face difficulties related to 
cognitive (e.g., attention), addictive (e.g., alcohol), or emotional/behavioral issues (e.g., PTSD, 
depression), leading to lower enrollment rates13 and slower degree attainment14 which in turn impede 
Veterans’ successful adjustment to civilian life and vocational success. Unfortunately, many Veterans 
with mental health concerns neither report problems nor seek treatment,10 and more than 50% do not 
perceive a need for services.15 Even for Veterans seeking treatment for academic and psychiatric issues, 
there are many barriers such as time, transportation, and perhaps most importantly, not understanding 
how to get connected to existing resources.16  
Though in-person educational services exist to help student Veterans, there are many 
barriers to their use. The VA has invested significant resources to address student success and mental 
health, using a catalogue of in-person treatments options for Veterans. Many of these are based in the 
tenets of Supported Education (SEd), a well-studied and effective approach for students who, because of 
psychiatric disabilities, face challenges in achieving educational goals.17 This framework focuses on 
improving educational competencies related to academic settings (e.g., literacy, study skills, time 
management), successful navigation of the university environment (e.g., financial assistance), and 
improved attitude and motivation. VA resources currently available to student Veterans include 
Veterans Integration To Academic Leadership (VITAL) which provides information regarding 
education benefits, on-campus clinical care coordination, case management, and training; VetSuccess 
On Campus (VSOC) which provides on campus academic services; and Compensated Work Therapy 
(CWT) Supported Education (SEd) program, which provides individualized support for Veterans 
engaged in education and training programs as well as linkages with educational facilities.17,18  
Though these program can be highly successful in assisting student Veterans, they are only 
effective to the extent that student Veterans can access them. Face-to-face resources are often 
unavailable due to geographical issues and a lack of trained supported education staff, or unappealing 
due to the multiple time constraints student Veterans manage.19 Furthermore, there are many reasons 
student Veterans may not seek supported education interventions, including the stigma attached to 
having mental health problems, the desire to be resilient and build on strengths associated with military 
services, and distrust of the Veterans Health Administration (VHA).20,21 Failures and delays in accessing 
or entering needed treatment can result in clinical, financial, and personal losses, as well as negative 
impacts on community reintegration.22, 23 
Mobile application are a promising way to provide support and resources to those unable 
or unwilling to access in-person services. Innovative strategies to reach student Veterans with mental 
health concerns are needed to circumvent inaccessibility and stigma, in order to increase the acquisition 
of coping and academic skills. Although one such intervention, Veterans Integration to Academic 
Leadership-Supported Education (VITAL-SEd; described in further detail below) has shown promise in 
pilots and RCTs, the typical time from standard in-person treatment development to implementation can 
take 20 years.24 Consequently, VA efforts to provide web- and app-based resources for intervention 
programming have increased. Mobile interventions have the potential to reach Veterans who might 
otherwise not have access to VA clinical care, as 81% of adults have access to smartphones.25 Following 
initial technology development, mobile app interventions are relatively easy to implement within clinical 
settings compared to in-person  

Page 4 of 30 
 
 IRB Protocol Template (3/1919) 
interventions, as they do not require training staff.26 Once developed, an app to support Veterans’ 
transition to the student role could provide psychoeducation, skill building, and connection to VA 
services both directly to Veteran users and as a complement to VA-provided services. [However, 
engagement - both initial and sustained - remains a problem for health apps. Studies suggest that 
engagement with behavioral apps can be improved by factors including positive perceptions of 
usefulness, motivation to pursue a goal, and visualizing user data in the app43,44 suggesting the 
importance of Veteran input early in development and testing. Additionally, adoption and engagement 
of apps may be bolstered if the app is recommended by a source the target user trusts and finds credible, 
such as VITAL staff.45 To improve both initial and sustained engagement, both target-use (Veteran) and 
future app referral sources (e.g., VA staff and college staff) should be involved in development, 
assessment, and future dissemination planning. 
Civilian studies have shown that targeted mobile platforms designed to orient students to the 
culture of higher education, while also focusing on their mental health, can increase student retention 
and course engagement.27 However, the majority of publicly-available education apps are extensions of 
learning-management systems for specific courses,28 and do not provide resources that would 
specifically assist Veterans with mental health concerns and academic skills acquisition. Extending what 
current, effective civilian student apps provide (course management options) and adding Veteran-
specific features (academic skill training, stress management exercises, connection to VA services) 
could thus provide an important, easily accessed resource to support Veterans. Unfortunately, there is no 
widely available VHA mobile app for student Veterans, and no accessible targeted intervention for 
student Veterans with mental health disorders - a substantial gap, given the high rate of difficulties these 
Veterans face.  
An academic support app for student Veterans could improve current educational 
functioning and future employment. Readily available, mobile assistance with academic and 
psychiatric issues could provide student Veterans with the knowledge, skills, and resources to help them 
graduate. In the proposed project, we plan to develop a personalized academic success app, VetEd, with 
content based in past research, supported educational literature, and Veteran feedback, to address a 
variety of academic and psychiatric symptom-related educational barriers for student Veterans. 
Specifically, VetEd would provide services to student Veterans who may not have access to VA 
education interventions by 1) promoting the creation and execution of educational planning and goals, 2) 
providing opportunities for Veterans to learn new skill sets like course organization, task completion, 
and stress management, and 3) connecting Veterans with additional VA resources based off their 
personal needs, such as providing information on mental health counseling, obtaining educational 
accommodations, and Post-9/11 GI Bill coverage 
 
2.2 
Preliminary Studies 
 
Our team includes researchers with expertise in student Veterans, technology, and mental health from 
the VISN 1 Mental Illness Research, Education, and Clinical Center (MIRECC) and the Center for 
Healthcare Organization and Implementation Research (CHOIR). The core research team includes PI 
Dr. Erin Reilly, with expertise in educational technologies, student development, and the creation and 
testing of mobile interventions for mental health, insomnia, and chronic pain; Co-I Dr. Lisa Mueller, 
specialist in Veterans’ supported education, career development, and mental health interventions; and 
Co-I Dr. Megan Kelly, with expertise in development and usability testing of mobile programs for 
Veterans with mental health concerns. Our project will draw on the following preliminary studies: 

Page 5 of 30 
 
 IRB Protocol Template (3/1919) 
 
VetSEd: The Veteran Supported Education (VetSEd) manualized treatment29,30 provided 
personalized supported education for Veterans with academic goals. A  small VetSEd RCT pilot was 
conducted with 33 Veterans with PTSD who had higher education goals.31 Despite a small sample and 
matched attention control that could have diluted effects, significant positive differences were found, 
with the intervention group spending greater amounts of time on educational activities (p =.0.002). 
However, the high intervention attrition rate (30%) suggested difficulties accessing the in-person 
program. 
 
VITAL-SEd: The Veterans Integration to Academic Leadership-Supported Education (VITAL-
SEd) program32 was developed using psychiatric rehabilitation and supported education principles. 
Based partially on VetSEd, both the pilot trial and RCT (PI: Dr. Ellison; CO-I’s: Drs. Reilly and 
Mueller) provided critical data on the particular needs of student Veterans with unmet psychiatric issues. 
In the pilot study for VITAL-SED, 8 Veterans with mental health disorders (e.g., PTSD, substance use) 
reported high levels of satisfaction with treatment in relation to perceived academic issues (Table 1).46 
Following this pilot, a larger RCT (n=43) was completed in 2020. Preliminary findings showed that, 
compared to the control group, Veterans in VITAL-SEd reported significant improvement in completing 
academic tasks (p = 0.02), psychosocial functioning (p = 0.04), and a reduction in psychiatric symptoms 
on the PCL-5 (p =0.03) and PHQ-15 (p = 0.02). 
 
2.3 Significance 
 
This development project and exploratory study will combine mobile app technology with empirically-
supported educational interventions for student Veterans struggling with mental health concerns. The 
app will provide opportunities for academic and psychiatric self-management to increase educational 
success for student Veterans, allowing for greater access and virtual care options for Veteran care 
coordination and educational development. Additionally, a resource that improves educational and 
vocational outcomes could help decrease disability payments and increase Veteran lifetime earnings. 
This project will create a resource for student Veterans that 1) provides a novel and accessible 
alternative to in-person education services that could help Veterans complete their academic path, and 2) 
advance our understanding app-delivered educational and psychiatric interventions. In addition, Dr. 
Shana Bakken (National Director, Compensated Work Therapy) and Dr. Khamkay Chitaphong 
(National Director, Veterans Integration To Academic Leadership), our operational partners, have 
dissemination team with several years of experience, which will make it easy to disseminate the VetEd 
app. Successful completion of the project will lead to the development of a Merit Review assessing 
VetEd’s academic impact and effectiveness using a larger scale RCT. 
 
3.0 
Research Design and Methods 
 
3.1 Drug/Device Information 
 
N/A 
 
3.2 Type of Study 
 
This study will be a quantitative and qualitative research study in which we will hold focus groups with 
Veterans and then conduct field-based usability testing of the initial VetEd mobile app. Focus groups 
will be qualitative in nature, and usability testing will be both quantitative and qualitative. 
 

Page 6 of 30 
 
 IRB Protocol Template (3/1919) 
3.3 Study Procedures 
a. Sub-Study Participation 
N/A   
 
b. Study Related Procedures  
Phase 1 Focus Groups and Development (10 months) 
This phase will focus on preliminary app development using focus groups to revise, add, or remove 
potential content already identified as potential areas of app content (focus groups, n=8, n=8, n=5). . 
Following the informed consent process, we will audio record sessions if completed in-person or video-
record if completed via remote teleconferencing, as this will assist with transcription for remote 
sessions. Focus group participants will be provided with drafted versions of potential app content related 
to the content areas below. At a minimum, Veterans in the focus group will be 1) given general 
information about the aims of the VetEd app in terms of its purpose, 2) given the opportunity to discuss 
their own educational experiences related to barriers and facilitators of their academic success, 3) 
presented with a general framework for how the mobile app will work, look, and function, 3) be 
presented with the exercises below as potential options for in-app content. At each stage of this focus 
group, Veterans will be encouraged to both provide feedback on specific content, looks, usability, and 
perceived appropriateness or “fit” with Veteran populations, and 2) to brainstorm potential additional 
content options for future app inclusion, such as resources, exercises, or options for customization. We 
will 
VetEd Content. The VetEd app content provided to focus groups participants for feedback will be 
an adaptation and extension of the successfully piloted VITAL-SEd program) The overarching goals of 
this intervention will be to (1) orient student Veterans with mental health disorders to successfully 
transition to the role of student as defined by their self-created educational roadmap while providing 
services related to (2) academic skills, (3) mental health management, and (4) up-to-date information on 
psychiatric, academic, and financial resources to help them successfully complete courses and increase 
educational and psychiatric functioning. To achieve these goals, the intervention components presented 
to Veterans will cover four specific areas of student development (Table 2). Additionally, to assist with 
app- engagement, we will embed app-notifications prompting users to complete or explore different app 
content. 
 Table 2. Fundamental VetEd Mobile App Areas for Student Development 
Area 
Description 
Potential Content* 
1. Creating & Following 
an Educational Roadmap 
Activities to help orient to the student role, 
reflect on educational and career goals, and 
organize academic choices into a roadmap with 
actionable education goals. 
▪ Education Tactics Plan 
▪ Campus Resources Assessment 
Worksheet 
▪ Creating a School Schedule 
2. Learning Strategies for 
Success 
Short, informational activities and modules 
focused on the skills necessary to effectively 
complete course work. 
▪ Study Strategies 
▪ Time Management Strategies 
▪ Managing Interpersonal Conflict 
3. Coping with Stressors 
Activities and guidance for mental health 
issues, stress management, and coping 
strategies for difficult thoughts and feelings. 
▪ Relaxation Activities 
▪ Dealing with Stressful Thoughts 
▪ Centering Before You Study 
4. Connecting to 
Resources 
Resources and information for academic 
concerns, [mental health issues for multiple 
populations], and VBA/GI-Bill benefits 
information modeled off the VA Inquiry 
Routing & Information System (IRIS).  
▪ Exploring Course Accommodations 
▪ GI Bill Concerns 
▪ Mental Health Services 
▪ Enrolling in the VA  
▪ Center for Women Veterans (CWV) 
  *based on VITAL-SED, consultant, and provider feedback; to be revised as needed using focus group data 

Page 7 of 30 
 
 IRB Protocol Template (3/1919) 
 
 
Focus Group Procedure and Analysis. We will hold 3 focus groups of Veterans with mental 
health disorders (n=21) using a semi-structured interview guide. Our target of 3 groups is based on 
research suggesting that approximately 80% of all themes (i.e., thematic saturation) in focus groups are 
discoverable withing 2 – 3 focus groups47, thus reaching point of “diminishing returns” in discovering 
new insights for preliminary app content.48  Focus groups will either be conducted virtually on in person. 
We will not mix participants in virtual vs. in person focus groups – all participants in virtual groups will 
participate virtually and all participants in in person focus groups will participate in person. Focus 
groups will include Veterans enrolled in at least two courses in a setting of  higher education in the past 
year, whether or not they are currently enrolled students. This will allow for a wider range of feedback 
on potential content and student Veteran needs, which may not be captured if only currently enrolled 
and successful student Veterans are recruited due to survivorship bias.  
Following the informed consent process, participants will complete a 1.5 hour audio (if in 
person) or video-recorded (if virtual) focus group. Dr. Reilly will follow a similar protocol used for 
previous focus groups for educational interventions (PI: Marsha Ellison) and mobile apps (PI: Megan 
Kelly). This will include questions assessing Veterans’ reasons for pursuing higher education, exercises 
they find helpful for learning academic skills (e.g., task management), messages that motivate them to 
complete course work, barriers related to mental health that impede successful course completion, and 
potential app resources to improve their academic functioning. Preliminary app content will be presented 
via PowerPoint and mobile app mock-ups for review and feedback, including discussing additional 
elements Veterans would find helpful in the app. Interviews will be transcribed by the VA Centralized 
Transcription Services Program.  We will use Rapid Assessment Procedures (RAP),33 an effective, 
widely used rapid data collection process. Coding will include the creation of summary points into a 
data matrix that organizes these points for each main topic of inquiry. This will allow the research team 
to systematically assess similarities, differences, and trends in responses across the focus groups. Finally, 
after a discussion of matrix contents, we will create a final memo summarizing findings and noting key 
themes for content. We will use this information to understand factors that that may improve 
acceptability of a VetEd app as well as features to revise or create. 
Phase 2 Pilot Usability Testing (9 months): The goals of this phase are to: 1) gain feedback on 
the VetEd mobile app via Veteran usability testing; 2) assess Veteran interest, satisfaction, and usability 
of VetEd; 3) iteratively address any technological concerns, and 4) collect preliminary information on 
the impact of the VetEd app on academic functioning (retention, perceived course task ability, and 
educational self-efficacy) and community reintegration. We will conduct pilot usability testing with 15 
Veterans who meet eligibility criteria, conducted in three waves. Each usability testing wave will 
include 5 participants, based on the finding that approximately 80% of usability problems can be 
detected using a sample of 4 to 5 participant per wave.34 
 
 
 
 
 
  
Usability testing procedures. Following the consent process, participants will complete a series 
of baselines assessments (approximately 2.5 hours total) related to their academic experiences, 
technology acceptance, mental health functioning and symptomology, and community integration. 
Participants will then be provided with the app and log-in information for VetEd, with instructions on 
how to utilize the app at home over a four-week period. During this Phase, we will access information 
about enrolled participants via their VA Health CPRS and/or VISTA.  
such as diagnoses, progress notes, medications, and specific information concerning substance 
use, to determine how other treatments are influencing outcomes for this study related to their mental 
health and academic functioning. During that time, they will complete at least two in-app tasks within 

Page 8 of 30 
 
 IRB Protocol Template (3/1919) 
per week, with instructions provided specifically about which to access in Weeks 1, 2, 3, and 4. These 
will include setting up an educational roadmap, tracking course assignments, learning about successful 
study strategies, completing a coping exercise, and finding information related to GI Bill benefits. There 
will also be a quick midpoint assessment (45 minutes). Participants will complete surveys following this 
4-week app piloting, and to evaluate the  app in the following areas: (a) app usefulness: whether users 
can successfully complete designated tasks on the app, (b) effectiveness: the ability to accomplish 
assigned educational app tasks quickly and easily, (c) learnability: the ability to meet pre-determined 
app navigation goals (e.g. accessing and inputting information into the education roadmap), (d) 
satisfaction: users’ feelings and opinions about the app related to their educational functioning and 
alignment with mental-health needs, and (e) engagement: initial and sustained interaction between a user 
and a mobile application. Usability measures will include two quantitative system usability scales (SUS; 
AES), task completion (quantitative), app log-in and use information (engagement).  Perceived 
usefulness, engagement, and overall satisfaction will also be assessed through a post-app qualitative 
interview (survey + interview will be 2 hours total). After each round of testing (4-5 participants per 
wave, total n = 15), the investigative team will work with the programming team to improve the app.  
 
 
 
 
 
 
3.4 Data Collection  
 
We will use several qualitative and quantitative assessments during the usability testing of the veteran 
student app, VetEd (Phase 2, see Table 3 below). 
  
Table 3.  Veteran Completed Assessments for the Evaluation of VetEd 
Measure* 
Screen 
Baseline 
 
Midpoint  
Post-App 
Follow-up 
Purpose 
Demographics 
X 
X 
 
 
Eligibility, Sample Description 
SCID-5 
 
X 
 
 
Eligibility, Sample Description 
PCL5 
 
X 
 
X 
Sample Description 
PHQ-9 
 
X 
 
X 
Sample Description 
SUS 
 
 
X 
X 
Usability 
AES 
 
 
X 
X 
Feasibility, Acceptability, Usability 
ASES 
 
X 
X 
X 
Academic Outcome 
CWBS 
 
X 
X 
X 
Academic Outcome 
M2C-Q 
 
X 
 
X 
Community Reintegration Outcome 
Post-App Interview 
 
 
 
x 
Usability Feedback & Academic Outcomes 
*Note. Full names and descriptions of measures are listed below. 
 
Demographics, functional health, and pain measures.  Demographic measures will include age, 
gender, race and ethnicity, and education. These measures will take approximately 10 minutes. 
 
Mobile App Utilization. We will collect information on VetEd usage, including the duration of use 
(minutes), number of logins, completion of in-app activities, and the percentage of participants who 
complete the program.  
 
Mental Health Symptom Measures. To assess for important psychological co-morbidities in 
chronic pain treatment, we will administer the 9 - item Patient Health Questionnaire-9 (PHQ-9)61 to 
assess for depressive symptoms, and 20-item PTSD Checklist-5 (PCL-5).41 These surveys will take 
approximately 7 minutes. 
 
 
Mental Health Diagnostic Measure: The Structured Clinical Interview for the DSM (SCID-5)35 is a 
reliable semi-structured instrument that is the gold standard for diagnosing mental health disorders. The 
utilized SCID modules will take approximately 30 minutes.  

Page 9 of 30 
 
 IRB Protocol Template (3/1919) 
 
System Usability Scale (SUS): The SUS36 is a 10-item measure, scored on a 5-point Likert scale, 
that assesses human-computer interaction. A SUS score above 58 is regarded as above average, and a 
SUS score above 80 is regarded as high and a score above which participants are likely to recommend 
the product to friends. The SUS generates a subjective evaluation score using a globally accepted scale 
and to understand if the system in its current form is sufficiently usable. This survey will take 
approximately 3 minutes 
 
 
 
Acceptability E-Scale (AES): The AES37 is a 5-item validated measure of the acceptability of 
mobile interventions, which has a cut-off of 80% for acceptability covering overall and specific 
component satisfaction. It will take approximately 2 minutes. 
 
Academic Functioning Outcomes: Increased academic function is the primary non-usability related 
goal of the VetEd app. Consequently, preliminary educational functioning information related to 
academic self-efficacy, preparedness for course tasks, and perceived confidence in course 
completion/retention will also be collected and described. In line with educational research suggesting 
non-GPA factors can be stronger predictors of educational success and graduation39, we will specifically 
assess self-reported academic self-efficacy using: 
• The Academic Self-Efficacy Scale40(ASES), an eight-item rapid assessment instrument to assess 
respondent self-efficacy regarding several academic skills related to academic achievement, such 
as time management, taking notes, taking tests, and general academic ability.  
• The Coping With Education Barriers Subscale41(CWBS), a subscale of the full 28 item scale that 
includes 2 subscales, the Coping with Career-Related Barriers (not administered for this study) 
and the Coping with Educational Barriers scale (20 items), assessing participants’ degree of 
confidence in overcoming potential educational barriers (both internal and external).  
• We will also assess self-reported confidence in completing coursework and academic retention 
(qualitative interview). 
 Both quantitative scales will take approximately 15 minutes. The qualitative interview, which will be 
combined with the mobile app feedback interview, will take approximately 30 minutes. 
 
Community Reintegration: The Military to Civilian questionnaire (M2C-Q)49 is a 16-item intended 
to measure dimensions of community reintegration related to productivity (in education, work, and 
domestic life); social relations; community engagement; and perceived meaning in life, self-care, and 
leisure. It will take approximately 5 minutes.  
 
Data Management. Gesiel Software, Inc. will collect app utilization data for all participants. When data 
collection is complete, we will lock the database, conduct quality checks, and perform the data analyses 
described below. Paper research files will be kept at the VA Bedford HCS in a locked cabinet where 
data will be encoded with deidentified ID numbers and on an encrypted, secure VA server space that 
will be used solely for this study.   
 
 
3.5 Analysis Plan 
 
Data analysis plan. The main Phase 2 quantitative measures of acceptability and usability will be the 
Acceptability E-Scale and the System Usability Scale (SUS), respectively. We expect that Acceptability 
E-Scale and SUS ratings will increase across the 3 waves of app revision, and that by the third wave we 
will reach the benchmark of 80% on the Acceptability E-Scale and a 68 on the SUS (good acceptability). 
Additional data will be provided by the qualitative interviews after app use, using RAP processes 
outlined in Phase 1. We will also assess feasibility and engagement by measuring the proportion of 

Page 10 of 30 
 
 IRB Protocol Template (3/1919) 
individuals who successfully complete the 4-week app testing, with at least 5 completed activities (of 
requested 8, 2 per week) or 63% of the program completed, in line with the mean number of completed 
sessions found in previous mobile interventions for chronic conditions.42  
We will also assess academic  functioning (academic self-efficacy, course task completion, and 
perceived retention/course completion) and community reintegration (M2C-Q) pre- and post-app use for 
each student Veteran specifically to 1) characterize the sample in terms of baseline academic functioning 
and community reintegration, 2) assess for possible changes in academic functioning and community 
reintegration, and 3) test the feasibility of these measures in anticipation of a larger, future RCT. This 
assessment will include testing the hypothesis that app use may lead to improvements in academic self-
efficacy (ASES) and ability to complete courses tasks (CWBS), or potentially improvements in 
community reintegration (M2C-Q) mental health symptoms (PCL5, PHQ-9). To evaluate this, we will 
conduct non-parametric. Within subject statistical analyses between pre and post-app use scores on these 
two measures (using Wilcoxon Signed Rank Test), which is appropriate for small sample testing and 
outcomes that may thus not be normally distributed. We will use caution in our interpretation of 
inferential analyses. 
 
4.0 
 Human Subjects 
 
4.1 General Characteristics 
 
Focus Groups. Participants will be 21 Veterans recruited from the Bedford VAMC and the community. 
Veterans at the Bedford VAMC will be identified for recruitment using a CPRS and/or VISTA chart 
review, hearing about the study from providers, or will be self-referred. Potential participants will be 
pre-screened by phone to determine whether eligibility criteria are met. In order to develop a Veteran 
student app that meets the needs of many Veterans with mental health concerns, we will also recruit 
from a diversity of clinics (e.g., VITAL, Women’s Health Clinic, LGBTQ program, MHICM, mental 
health outpatient programs, domiciliary, social work services), and programs that serve different military 
cohorts (e.g., Vietnam Veteran groups, OEF/OIF program). 
 
Usability Testing: Participants will be 15 Veterans recruited from the Bedford VAMC and the 
community. Veterans at the Bedford VAMC will be identified using CPRS chart review and/or VISTA, 
hear about the study from providers, or be self-referred. We will recruit from advertising on the Bedford 
VAMC Facebook page, Bedford VAMC Twitter account, Veteran newsletter, and Craigslist. All 
potential participants will be pre-screened by phone to determine whether basic study eligibility criteria 
are met. 
 
4.2 Inclusion of Vulnerable Subjects 
 
The study population may be considered vulnerable because they may be economically disadvantaged. 
The veteran population includes individuals who are economically or educationally disadvantaged. The 
study keeps payments to participation at a minimum in order to avoid coercion based on economic 
conditions. In addition, the informed consent is written at a grade-school level of education to minimize 
the vulnerability of the educationally disadvantaged. 
 
Vulnerable participants will also include those with diagnosed mental health disorders. These 
individuals may have impaired decision–making abilities. Acute exacerbations of psychiatric illness may 

Page 11 of 30 
 
 IRB Protocol Template (3/1919) 
substantially impair decision-making abilities by decreasing level of understanding and reasoning. 
However, the level of capacity in those that are stabilized is likely to be much higher. This study is not 
focused on treatment for acute exacerbation and thus, plans to include veterans who are not in an acute 
symptomatic exacerbation. Therefore, the decision-making capacity is likely to be intact and will be 
assessed via the planned phone screen. 
 
Another population to be included is that of the Veterans themselves. They are accustomed to taking and 
following direct orders, thus introducing a further need for researchers to prevent coercion either directly 
or indirectly. Veterans will be given ample time to read and consider informed consent, with other 
treatment options presented by investigators. Family members, significant others, and primary treatment 
teams may also be involved in the decision-making process if the Veteran wishes. They are also 
informed that refusal to participate will be accepted without hesitation at any time and will not change 
their eligibility for VA services, treatment, disability payments, or other related VA benefits. Research 
staff will evaluate each subject at the end of the phone screen (focus group and formative evaluation 
phases), focus group, or usability testing session to insure that they have not experienced significant 
distress or exacerbation of their symptoms as a result of their participant. The evaluation will consist of 
asking the subject about (1) feelings of distress and (2) desire for additional support because of distress. 
Anyone experiencing distress and desiring support will be provided with informal support and if 
necessary taken to formal providers (see Protection Against Risk section). 
 
4.3    Inclusion of Pregnant Women 
 
 
Pregnant women will not be excluded because the VetEd app may still be helpful to them, and their 
perspective should have the opportunity to be included in the development of this mobile resource. In 
addition, the risk to the fetus is not greater than minimal and pregnant women may have specific 
academic needs that deserve to be included in this app, per the VA’s mission to provide quality services 
to all who have served, and also aligns with Senate passed legislation The Protecting Moms Who Served 
Act of 2021. Obtaining this information requires that Veteran women who are pregnant or become 
pregnant may provide important feedback on potential app content.  
 
4.3 Inclusion of Incompetent Subjects 
 
We will not enroll incompetent subjects. 
 
4.4 Inclusion/Exclusion Criteria 
 
Phase One: Focus Groups 
Inclusion criteria: 
1) Enrolled in a minimum of at least 2 courses in a higher education setting within the past year  
2) Current DSM-5 mental health disorder  
3) Competent to provide informed consent 
4) Owns a smart phone capable of supporting mobile apps 
5) Are willing and able to read in English 
6) Ages 18 and older 
 
Exclusion Criteria: 

Page 12 of 30 
 
 IRB Protocol Template (3/1919) 
1) Current or recent (within 1 month of study entry) moderate or severe DSM-5 alcohol or drug use 
disorder 
2) Cognitive impairment that would interfere with participation 
3) Severe suicidal ideation or a psychiatric hospitalization within the past month 
 
 
Phase Two: Pilot Usability Testing 
Inclusion criteria: 
1) Currently enrolled in a minimum of at least 2 courses in a higher education setting  
2) Current DSM-5 mental health disorder  
4) Competent to provide informed consent 
5) Owns a smart phone capable of supporting mobile apps 
6) Are willing and able to read in English 
7) Ages 18 and older 
8) Self-report struggling with current course activities  
 
Exclusion Criteria: 
1) Not currently enrolled as a student  
2) Current or recent (within 1 month of study entry) moderate or severe DSM-5 alcohol or drug use 
disorder 
3) Cognitive impairment that would interfere with participation 
4) Severe suicidal ideation or a psychiatric hospitalization within the past month 
 
 
No subjects will be excluded based on their race, religion, ethnicity, gender or HIV status, as applicable. 
 
4.5 Recruitment Procedures 
We will recruit 60 Veterans in total (to meet study goals of max N=36 Veterans in the study) from the 
VA Bedford HCS (focus group=21 and pilot usability testing= 15 
) and the community. Participants will be recruited throughout VA Bedford.  
 
The PI will not be a provider of services to any participant recruited into the study. The PI will not call 
participants prior to enrollment, unless they contact the PI and express interest in the study or they do 
not call after receiving a letter about the study to indicate that they are not interested in the study (see 
below).  
 
The recruitment process will include the following:  
 
1) We will ask providers to mention the study to potential participants if they think it might be of 
interest to the participant, and provide a brochure or flyer if they are interested. We will ask providers 
to tell participants to call us if they are interested. Providers are not recruiting study participants, only 
providing information about the study and referring potential participants to the study flyer for more 
information. Although we will specifically target providers from VITAL, Vocational Rehabilitation, 
Veterans Community Care Center (VCCC), Mental Health Clinic (MHC), the Veterans Mental Health 
and Addictions Program, Primary Care Behavioral Health (PCBH), Veterans Services Coordinators, 

Page 13 of 30 
 
 IRB Protocol Template (3/1919) 
Primary Care, the Community Residential Center (CRC), and the VASH program, we will provide 
information to any clinicians who are interested in learning about it.  
2) We will also provide information to veterans and providers at a Crossroads event at the VA Bedford 
HCS, where information (e.g., flyers) about the present study and other investigators’ studies will be 
distributed at a table in a highly frequented area of the Hospital.  
3) We will also distribute flyers throughout the hospital, with research staff study contact information. 
4) The names of Veterans from a medical record review coming to the following clinics for 
appointments: VITAL, Tobacco Cessation Program, Veterans Community Care Center (VCCC), 
Mental Health Clinic (MHC), the Veterans Mental Health and Addictions Program, Women’s Health 
Clinic, Primary Care, Primary Care Behavioral Health (PCBH),the Compensated Work Therapy 
(WCT) program, the Community Residential Center (CRC), and the VASH program. These names will 
be obtained by searching CPRS and/or VISTA for entry/provider codes associated with these programs. 
a. The research staff will access these patients’ medical records only to determine whether they meet 
the study inclusion criteria and obtain address and phone contact information. We will also access 
SSNs ensure that we are identifying the right patient for contact. We will not open flagged records. 
b. We will then send a letter to the Veteran to invite him/her to learn more about the study (attached to 
this application).  The letter will contain contact information for the research team so the patient can 
contact the research team for more information. It will state that if we don’t hear from the patient within 
two weeks, that we will call them about the study. It will also state that if patients do not want to be 
called, they can call our research staff and let us know. The identifiable information will be used only 
by members of the research team. This information will not be disclosed to anyone outside the VA. 
5. Will obtain a list from the business office? 
 
c. If the potential participant is interested in remote participation in the study after the phone 
conversation, we may use Azure RMS to send them the Informed Consent Information Sheet, study 
flyer, and study information by email.  
 
Psychology menu option 
 
Bed Mh Sw Vitl Cs Mgmt 
BED MH TELE SW VITAL 
BED MH TELE VITAL PSO 1 
BED MH VITL PSO 3 IND 
BED MH VITL PSO GRP 
 
Alissa, christina, kate 
 
 
We will also post our study flyer information on the VA Bedford Facebook page, VA Bedford Twitter 
account, in the VetNetNews newsletter, and Craigslist. 
 
It is expected that recruitment will be enhanced by the VA Bedford’s extensive treatment networks, 
recognition for expertise in vocational rehabilitation, and lack of competing studies, and subject 
payment. If continuous review of recruitment data suggest imbalance in gender distribution, we will 
target our recruitment toward clinics that serve higher numbers of women (e.g., women'’s clinics). 
 

Page 14 of 30 
 
 IRB Protocol Template (3/1919) 
We will call a potential participant three times, and if we do not hear back from them from any of these 
attempts, we will discontinue contact attempts. However, if a potential participant does call back and 
leaves a message, we will reinitiate the three call rule (i.e., call them three more times and discontinue 
if we do not hear from them). If potential participants are not available, research staff will leave a 
message, saying “My name is Dr./Mr./Ms. And I am calling from the Bedford VA Hospital. We are 
calling about a research study and would like to check if you are interested in participating in this 
study. If you would like more information, please give us a call back at 781-687-XXXX. Thank you.” 
If we reach someone by phone who is not the potential participant, we will ask the person to write 
down our contact information for the potential participant (name and number), but will not indicate 
where we are from or the purpose of our call to maintain confidentiality. 
 
4.5.1 Subject Identification and Pre-Enrollment Screening:  
 
We will obtain information through oral communication (telephone screening) with the prospective 
subject. The investigator will also obtain identifiable private information by accessing records (via 
CPRS or VISTA) to determine eligibility. Thus the exception to informed consent applies and no waiver 
of informed consent is necessary. 
 
4.5.1a Use of PHI for Recruitment and/or Screening before consent is obtained:  
 
We will request a waiver of HIPAA authorization for recruitment/screening. 
 
A request to use identifiable information in the conduct of this research study under a waiver of 
authorization for recruitment and screening was submitted with this protocol. The identifiable 
information being requested is:  
 
To use during the recruitment process:  
 
• 
To conduct brief screenings with veterans who express interest in the study after learning about it 
through study presentations, flyers, letters, phone calls, or providers. This will involve asking the 
veterans questions based on study inclusion/exclusion criteria (i.e., academic status, substance 
use history). These are included on the study screening form.  
 
• 
The names of veterans coming to the following clinics for appointments:  Veteran Integration to 
Academic Leadership (VITAL), Veterans Community Care Center (VCCC), Mental Health 
Clinic (MHC), the Community Stabilization Program (CSP), Primary Care Behavioral Health 
(PCBH), Women’s Health Clinic, Primary Care, the Community Residential Center (CRC), the 
Compensated Work Therapy Program (CWT), and the VASH program. These names will be 
obtained by searching CPRS or VISTA for entry/provider codes/clinic codes associated with 
these programs. 
 
o 
The research staff will access these veterans medical records only to determine 
whether they meet the study inclusion criteria and obtain address and phone contact 
information. We will also access SSNs to ensure that we are identifying the right 
patient for contact (e.g., patients with the same name but different SSN). We will not 

Page 15 of 30 
 
 IRB Protocol Template (3/1919) 
open flagged records. We will also obtain email addresses if potential participant 
wants to be sent information through Azure RMS. 
o 
We will then send a letter to the patient to invite him/her to learn more about the 
study (attached to this application).  The letter will contain contact information for the 
research team so the patient can contact the research team for more information. It will 
state that if we don’t hear from the patient within two weeks, that we will call and 
contact them about the study. It will also state that if patients do not want to be called, 
they can call our research staff and let us know. The identifiable information will be 
used only by members of the research team. This information will not be disclosed to 
anyone outside the VA. 
 
The identifiable information will be used or disclosed only by members of the research team.  This 
information will not be disclosed to anyone outside the VA. 
 
The proposed study poses minimal risk to the privacy of the subjects because: 
 
a. 
The identifiable information will be protected from improper use or disclosure by:  
 
• For screening purposes only, this study will maintain a contact log. This log will 
contain the identifying information for veterans who are identified by research 
staff via medical record review or who contact research staff after learning about 
the study. It will contain the patient’s name, social security number, contact 
information, and status (No Response, Screened In, Screened Out, Pending, No-
Show, Chose not to participate). This log will be stored on our designated secure 
server space, accessible only to approved study staff.  This log will be used until 
the end of the recruitment period in order to prevent ineligible or uninterested 
veterans from being contacted again by letters from research staff. All information 
will be kept according to VA Records Retention Policy. 
 
• Study screening forms will be coded, and will not contain the participant’s name, 
social security number, or date of birth. 
 
• All identifying information will be maintained in locked file cabinets in locked 
offices or on a the study’s secure VA server space. 
 
• Only identifying information for candidates who meet the diagnostic eligibility 
criteria will be recorded. 
 
• Identifying information for these patients will be recorded electronically on the 
VA server space, and will be password protected. 
 
• If patients are not interested, we will transfer their information to a log of non-
interested participants which will also be recorded electronically on a secure VA 
server space: \\R04BEDNAS21.v01.med.va.gov 
 
 

Page 16 of 30 
 
 IRB Protocol Template (3/1919) 
o This log will be used until the end of the recruitment period. This will be 
done to prevent uninterested patients from being contacted again by letters 
from research staff. All information will be kept according to VA Records 
Retention Policy. 
 
• This information will be only be used for scientific research purposes. 
 
• Only Bedford VA study team members will have access to this information.  
 
• The study team will not identify, directly or indirectly, any individual participant 
in any report of such research or otherwise disclose participant identities in any 
manner. 
 
 
4.5.1b Consent for Recruitment and/or Screening:  
 
We will conduct brief screenings with interested veterans will enable the research staff to more 
efficiently achieve the enrollment goal. 
 
This recruitment method is the only way we will achieve sample size goals and the goal of a 
representative sample of veterans who are eligible for this study. It is essential that we specifically offer 
the study to all patients who meet criteria for the study in order to obtain a representative sample, rather 
than relying solely on self or provider referrals. If the research team relies on provider identification of 
eligible patients, it is likely that they will fail to identify all patients who meet diagnostic criteria for the 
study. Just relying on provider and self-referrals may result in a biased sample, as it is likely providers 
would only refer veterans to the study who they considered good candidates for this study, rather than all 
candidates who are eligible for the study. By having the research team identify potential candidates 
beforehand and then sending letters to potential candidates to use to contact the patient, and following 
up for those who do not opt out, we can ensure that the largest sample of veterans possible can be 
invited to participate in the study. 
 
The inclusion criteria for the study are very broad and were expressly designed to allow greatest access 
possible to the study. For this study, it is important that we recruit our sample from a broad population 
of patients who smoke, and not just those identified only by providers or themselves as eligible. 
Veterans who decide they want to participate will go through the Informed Consent Information Sheet 
process prior to entering the study. Written information provided to participants and it will provide 
detailed information about study procedures and potential risks and benefits. 
 
4.5.2 Enrollment:  
 
60 participants will potentially be consented in the study to reach our goals for participation in focus 
groups (n = 21) and the formative evaluation, (n=15). A participant will be enrolled in the study 
immediately after the informed consent process. Participants in the focus group will immediately 
participate in the focus group after an informed consent process. For participants in the usability pilot 
testing, participation will take place immediately after an informed consent process. 
 

Page 17 of 30 
 
 IRB Protocol Template (3/1919) 
4.5.2a HIPAA Authorization:  
 
We are requesting a HIPAA waiver for all parts of the project as most participants will be remote and it 
would be impractible to do it through the mail, as consent will be done remotely with a waiver of 
documentation of informed consent. 
 
4.5.2b Informed Consent:  
 
Verbal informed consent will be obtained from each participant prior to entering the study. The 
information sheet provided to Veterans will explain in simple terms, before the patient is entered into the 
study, the risks and benefits to the patient. Verbal consent will be obtained after a thorough explanation 
of the study by the PI or other study personnel (e.g., research assistant, postdoctoral fellow, PI), and an 
opportunity for the participant to ask questions about the study. This discussion will take place under 
conditions in which the participant has adequate time to consider the benefits and risks associated with 
his/her participation in the study. We have decided to allow participants to choose between in-person 
and remote study visits for the focus group and formative evaluation portions of the study. For those that 
choose remote visits, the informed consent process will be done over VA-approved video-conferencing 
software.  
 
After screening, if a Veteran indicates interest in the study, study staff will mail or email the study 
participant the informed consent information sheet if they prefer remote visits, or we will hand the 
Veteran an informed consent information sheet if they prefer in-person visits. A wavier of 
documentation of informed consent will be requested, to accommodate Veterans who complete all 
procedures remotely and for all participants to maintain consistency, as most will be remote. 
 
After the potential research subject has reviewed the informed consent information sheet and indicated 
interest in participating, a study staff member will speak with the Veteran in-person or by phone or using 
VA-approved video-conferencing software to review the informed consent with them. Study staff will 
ask if they have any questions. We will assure the privacy of participants by asking them to use a private 
space if we are doing the consent process remotely, rather than a shared office or common area, to speak 
with us. If they come to the VA to do the process in-person, we will use a private space for the informed 
consent where we cannot be overheard. The informed consent document contain all required elements of 
informed consent and will explain in simple terms, the risks and benefits to the patient and that 
participation is voluntary and the patient is free to withdraw from the study at any time with no 
consequences. Consent will be obtained after a thorough explanation of the study by the PI or other 
study personnel (e.g., research assistant, postdoctoral fellow, PI), and an opportunity for the participant 
to ask questions about the study. This discussion will take place under conditions in which the 
participant has adequate time to consider the benefits and risks associated with his/her participation in 
the study. 
 
Only participants capable of giving informed consent will be admitted into the study. Informed consent 
will be obtained by trained and highly qualified research personnel. The research staff member that 
conducts the informed consent process will ask each participant to verify that the information in the 
consent form is understood. The staff member will review understanding of the consent form, including 
information pertinent to study participation. During the informed consent process, we will assess 
whether the Veteran is competent to provide informed consent. We will determine whether the Veteran 

Page 18 of 30 
 
 IRB Protocol Template (3/1919) 
is oriented to time, place, and person. We will ask questions to understand whether the Veteran 
understands the basics about what the study protocol involves. We will assess whether the consequences 
of participating in the study are understood. We will also assess whether the Veteran is able to clearly 
and voluntarily express his or her preference for participating in the study. If the Veteran is not able to 
do these things, we will determine that the Veteran is not competent to provide informed consent and we 
will end the informed consent process. Only once all questions have been answered and the participant 
understands the purpose of the study and study procedures will the participant give verbal consent, 
which we shall document in a secure excel file recording the participant providing consent and the date 
the provided consent. . A waiver of documentation of informed consent will be requested as individuals 
will be consented verbally.  The informed consent process will be completed prior to the initiation of 
any study procedures 
 
A discussion of the study's purpose, research procedures, risks and potential benefits, and the voluntary 
nature of participation as part of the informed consent process will continue throughout the research 
experience. 
 
It will be the responsibility of the PI to assure that informed consent is obtained from each participant 
prior to the performance of any protocol procedures and in accordance with current state and federal 
regulations.  
 
4.5.2c  Master List 
 
A participant will be added to the master list of subjects when s/he is consented. This master list will be 
kept on the secure VA shared drive (only accessible to study staff) for this study. Only staff that are part 
of the study and have access to the shared drive, and thus will have access to this master list. 
 
4.5.2d VHA Health Records 
 
A VHA health record will not be created and updated for this study. 
 
4.5.2e  Certificate of Confidentiality   
 
We will obtain a certificate of confidentiality for this research study. 
 
4.6 Risk/Benefit Ratio 
4.6.1 Potential Risks and Methods to be Used to Minimize Risks:  
 
1) During the study assessments and treatment, participants may experience some discomfort from 
discussing personal material and completing self-report questionnaires. Likewise, some participants may 
feel uncomfortable about having the study assessments and qualitative interviews related to VetEd app 
usage. 
 
2) There is a risk of breach of confidentiality and loss of privacy, which great care will be taken to 
prevent. As discussed below, we will take precautions to ensure that potential risks are minimized. 
 
 

Page 19 of 30 
 
 IRB Protocol Template (3/1919) 
Protection Against Risk 
 
1) In order to minimize anxiety during the study assessments, focus groups, or usability testing, the PI 
and study staff will make every attempt to help participants feel comfortable when discussing sensitive 
material or talking aloud during focus groups and qualitative interviews. The PI is a licensed clinical 
psychologist with 10 years of clinical experience, particularly with regard to treating student mental 
health concerns, PTSD, and depression. 
 
2) Initial and continuing participation of the participant in these protocols will be strictly voluntary; the 
participant will remain free to terminate participation at any time. 
 
3) Potential study participants will be screened initially over the phone for eligibility. For screening 
purposes only, this study will maintain a contact log. This log will contain the identifying information 
for veterans who are identified by research staff via medical record review or who contact research staff 
after learning about the study. It will contain the veteran’s name, social security number, contact 
information, and status (No Response, Screened In, Screened Out, Pending, Chose not to participate). 
This log will be securely stored on a VA shared drive. This log will be maintained until the end of the 
recruitment period in order to prevent ineligible or uninterested veterans from being contacted again by 
letters from providers or by research staff. The identifiable information will be used only by members of 
the research team. This information will not be disclosed to anyone outside the VA. The proposed study 
poses minimal risk to the privacy of the subjects because: 
 
• Study screening forms will be coded, and will not contain the veteran’s name, social security 
number, or date of birth. 
• All identifying information will be maintained in locked file cabinets in locked offices or on the 
study’s secure VA server space. 
• Only identifying information for candidates who meet the diagnostic eligibility criteria will be 
recorded. 
• Identifying information for these veterans will be recorded electronically on the secure VA 
shared drive. 
• If Veterans are not interested, we will transfer their information to a log of non-interested 
participants which will also be recorded electronically on the study’s secure space on the VA 
server. 
• This information will be only be used for scientific research purposes. 
• Only VA study team members will have access to this information. 
• The study team will not identify, directly or indirectly, any individual participant in any report of 
such research or otherwise disclose participant identities in any manner. 
 
Following this initial screening, potential study candidates will be consented and then participate in a 
detailed psychiatric screening interview at the first study visit with trained project staff who are closely 
supervised by Dr. Reilly. Effective screening will exclude Veterans who would be at greater risk 
because of psychiatric conditions revealed by this process. Dr. Reilly has used the aforementioned 
screening criteria successfully in the past with behavioral intervention studies. If at any point there are  
significant adverse reaction during the screening process, the participant will be removed from the study 
and appropriate treatment implemented. This could include immediate assessment to a local emergency 
room. 

Page 20 of 30 
 
 IRB Protocol Template (3/1919) 
 
4) There is a possibility that study participants will not improve their academic functioning after testing 
the mobile app. All participants who fail to respond to treatment or withdraw prematurely will be 
referred for alternative treatment. If withdrawal from the study is necessary, we will provide appropriate 
alternative treatments (e.g., placing a consult with the VITAL program or Mental Health Clinic). Dr. 
Reilly will discuss the study, alternative treatments, or any concerns about the study with participants if 
requested by the participant or health care provider. 
 
5) Participants may possibly experience clinical deterioration during the study. However, Dr. Reilly will 
be available to participants by phone, or in person if necessary, to discuss any concerns throughout the 
treatment period. Drs. Reilly, Kelly, and Mueller are all licensed psychologists, and will be available to 
study participants in the event of a clinical emergency; participants may also go to Bedford VA walk-in 
clinic if a clinical emergency occurs. This will be clearly communicated orally and in writing to study 
participants. Participants will be withdrawn from the study if their clinical condition deteriorates 
substantially. Participants may also be withdrawn if, in the judgment of the study investigators, 
remaining in the study poses a substantial risk to the participant or a higher level of care is needed. 
 
6) Participants in the study may report suicidal ideation or suicidal intent. Given that some study 
participants may present with numerous vulnerabilities (e.g., history of traumatic brain injury, current or 
past suicidal ideation or behavior), all study staff members will be trained by Dr. Reilly, a licensed 
psychologist, in procedures for conducting risk assessment and crisis interventions. For participants who 
in the PI’s judgment pose a substantial current risk of suicide but do not consent to having a provider 
contacted, we will breach confidentiality and contact their provider while notifying the participant of 
this and explaining our rationale (concern for their safety). At the beginning of the study and each 
subsequent assessment, participants will be told that the information they share is confidential unless the 
participant appears in jeopardy of harming themselves or someone else. For actively suicidal 
participants, or other participants who appear at high risk, who are not currently receiving mental health 
treatment, we will strongly encourage the participant to obtain mental health treatment. In addition, we 
will make every effort to find treatment for the participant and facilitate their obtaining treatment in a 
timely way. 
 
Telephone Risk Assessments. Risk assessment will be conducted with all participants during the initial 
eligibility screening. Elements of risk assessment will include level of suicidal ideation and suicidal 
intent, using questions from the University of Washington Risk Assessment Protocol (UWRAP). If the 
Veteran endorses either of these items, further questions will be asked, including presence and level of 
specificity of a plan, access to weapons and other means, presence of known risk factors (e.g., recent 
loss, substance misuse, chronic pain or other serious medical problems), and presence or absence of 
deterrents. The PI will be notified of all such occurrences immediately. Following the procedures of the 
Bedford VAMC, if a Veteran is at imminent risk of suicide at the time of a phone screen, the RA will 
direct the Veteran to Dr. Reilly or their clinician designee via warm telephone transfer. The psychologist 
will conduct a suicide risk assessment over the telephone in order to determine the Veteran’s level of 
risk. If the psychologist deems the Veteran to be at imminent risk, s/he will call the police and request 
emergency services via a warm transfer. The psychologist will stay on the line with the Veteran until 
s/he makes contact with emergency services and care is transferred. In the unanticipated event that a 
licensed clinical psychologist is not available, and study staff are unable to reach the emergency room at 
the VA, they will conduct a warm transfer to the Veterans’ Crisis Line. If they believe that the Veteran 

Page 21 of 30 
 
 IRB Protocol Template (3/1919) 
is not at imminent risk, but may need a welfare check, s/he will coordinate with VA police and the 
suicide prevention team. We will coordinate with credentialed, clinical research staff at each study site 
to ensure that there is constant coverage by a licensed clinical psychologist. As necessary, the study 
team will collaborate with the local suicide prevention coordinator to facilitate care and clinical follow-
up. If a potential participant is hospitalized, they will remain potentially eligible and will be invited to 
participate during or after their hospitalization. 
 
7) Initial and continuing participation of the participant in these protocols will be strictly voluntary; the 
participant will remain free to terminate participation at any time. 
 
8) All data collected from participants, CPRS/VISTA review, or the mobile app will be used specifically 
for the research purposes described in this application. No identifying information will be associated 
with any information provided by participants as a function of their participation in this study, including 
their names, addresses, or phone numbers. Each participant in the study will be assigned an arbitrary 
study number at random and this number will be used for all data for subsequent analyses. Although the 
consent form will include the name of each participant, it will be filed separately from the actual data 
files for each participant. To ensure protection of confidential information, all data will be coded and 
stored in locked files to ensure confidentiality. Audio recordings of and video recordings (dependent on 
participant’s chosen modality) of usability testing and focus groups will be kept at the data management 
site at the Bedford VA and stored on a secure VA shared drive for this study. Upon completion of the 
study and data entry and analysis, all data and files will be stored and archived indefinitely under lock 
and key. Information about substance use will not be made available to any party without the written 
informed consent of the participant and at the participant’s request. Only the PI and the PI’s research 
team will have access to the computerized data. Names will not be included on videotaped portions of 
the study, in computerized data files, or in any published reports. Case records will be reviewed only by 
study personnel or, if necessary, by institutional, state, or federal regulatory personnel. Research 
assistants and others working on this study will be educated about the importance of strictly protecting 
participants' rights to confidentiality. 
 
9) Numerous safeguards will keep electronic data secure. Each subject will be assigned a code number 
so that should any non-study personnel gain access to the data, they will not know to whom the data 
belong. A secure password is required to access computers, and computer networks are on secured 
servers that meet or exceed federal confidentiality standards. 
 
10) The Bedford VA has institutional policies requiring all clinical investigators and their key personnel 
to undergo mandatory training in human subjects research. The program includes all clinical researchers 
regardless of their source of funding. Researchers typically fulfill this requirement by completing 
Bedford VA research trainings. 
 
4.6.2 Data and Safety Management Plan:   
4.6.2.1 Data Security  
 
Although the consent form will include the name of each subject, it will be filed separately from the 
actual data files for each subject. All consent forms, interview, and questionnaire data will be filed 
securely under lock and key in file cabinets in the PI’s offices at the Bedford VA Hospital (Building 5, 
135B). Local data files, designated by an anonymous subject study number, will be stored directly to the 

Page 22 of 30 
 
 IRB Protocol Template (3/1919) 
secure VA server designated for this study, be securely maintained, and will be accessible only to 
research staff directly involved in this project. Upon completion of the study and data entry and analysis, 
all data and files will be stored and archived indefinitely under lock and key. 
 
Procedures to ensure the privacy of subjects: 
 
Subjects will be asked to come to the PI’s program offices, which are located in Building 5, away from 
busy patient areas. Interviews and questionnaires that take place in person will be completed in the PI’s 
program offices, with the door closed. A white-noise machine will be used to ensure that information 
from the subject will not be heard outside of program offices. Study visits conducted remotely will 
utilize only VA-approved teleconferencing systems using secure meeting rooms. 
 
Procedures to maintain the confidentiality of data/information: 
All data will be used specifically for the research purposes described in this application. No identifying 
information will be associated with any information obtained from questionnaires or interview data, 
including names, addresses, or phone numbers. All data will be kept in accordance with VA regulations. 
Each participant in the study will be assigned an arbitrary study number at random and this number will 
be used for all data for subsequent analyses. All data are filed securely under lock and key in file 
cabinets at the ENRM Veterans Hospital (in Building 5, 135B). Upon completion of the study and data 
entry and analysis, all de-identified data and files will be stored and archived indefinitely under lock and 
key. All computers are password-protected. Information will not be made available to any party without 
the written informed consent of the subject and at the subject’s request. 
 
How/where data will be stored (location with room number): 
Data will be stored in locked file cabinets in Room 135B in Building 5, a Bedford VA building that is 
locked during non-business hours. Computerized data files will be stored in password-protected files on 
a VA serve space designated for this study only. 
 
Who will have access to the data/codes: 
Only Dr. Reilly and authorized study personnel will have access to the data/codes onsite at the Bedford 
VAMC. Codes will be stored on the approved VA server space for this study. Only Bedford study staff 
will have access to one database, linking the code to the Veteran’s name and SSN. This information will 
be stored on the study VA server space and stored separately from other data. Once a research staff 
member is no longer working on this protocol, they will no longer have access to data and codes. 
 
What will happen to the data when the research is complete: 
All data will be kept in accordance with VA regulations. Upon completion of the study and data entry 
and analysis, all de-identified data and files will be stored and archived indefinitely under lock and key. 
 
How data will be archived: 
The data will be archived. Data will be archived on VA authorized storage media and hard copies of 
data (i.e., questionnaires and interview data) will be stored in locked file cabinets in Building 5, Room 
135B. We will retain a complete record of all data obtained during the VA portion of the research in 
accordance with privacy requirements, the Federal Records Act, and VHA Records Control Schedule 
(RCS) 10-1. All disclosures and data transmission will meet privacy and security requirements per VA 
Directive 6500, VA Handbook 6500, and VHA Handbook 1605.1. 

Page 23 of 30 
 
 IRB Protocol Template (3/1919) 
 
Will data with identifiers be released? 
Data with identifiers will not be released, transferred, or shared. 
 
4.6.2.2 Data Safety Monitoring 
 
 
 
A DSMB is not required because multiple clinical sites will not be used, the study is not blinded, and 
interventions are not particularly high risk or vulnerable populations are not included. 
 
Responsibility for Data and Safety Monitoring: Dr. Reilly will have overall responsibility for 
monitoring the integrity of the study data and participant safety. Any difficulties that arise with Veterans 
related to self-disclosed safety issues or concerns will be discussed between the PI and the co-
investigators (Drs. Kelly and Mueller). Additionally, the safety of research participants will be discussed 
in meetings with Drs. Reilly, Kelly, and Mueller, respective to their areas of expertise.   
 
Dr. Reilly will be monitoring adverse events involving data loss and adverse events related to the study. 
Adverse events will be monitored in the following ways: 
Procedures for Monitoring Participant Safety: The PI will implement the following procedures to 
ensure data integrity and the safety of participants during the study: 
 
1) Many elements of the research plan are intended to minimize the risks of study participation. These 
are detailed in the Research Design and Methods section and 2) “Protection Against Risks” in the 
Human Participants section above. For example, the study exclusion criteria exclude Veterans who are 
experiencing clinically significant suicidality or a psychiatric hospitalization within the past month. The 
PI will carefully monitor participants at each study visit to ensure that they do not experience clinically 
significant deterioration; if they do, the PI will reevaluate them and, if indicated, refer them for 
immediate non-study treatment (e.g., pharmacotherapy, counseling, inpatient, or residential treatment). 
The PI (or covering clinician in her absence) will be available to participants in the event of an 
emergency. In the event that serious medical or mental health complications should occur, consultation 
from emergency clinical services is immediately available at the Bedford VA Medical Center. 
 
2) Weekly meetings will be held between Drs. Reilly, Kelly, Mueller, and other study staff. We will 
discuss and resolve any safety issues more frequently if necessary, as such issues arise, possible 
participant withdrawal from the study, or any safety concern. 
 
3) Data integrity and confidentiality will be safeguarded as discussed in the Data Management section of 
the application and under Protection Against Risks above. 
 
Reporting Adverse Events: Reporting of adverse events will occur as follows: 
 
Reporting Adverse Events: Reporting of adverse events will occur as follows: 
1) Any death that is both unanticipated and possibly related to the research will be communicated orally 
to the IRB of the VA Bedford Healthcare System immediately upon becoming aware of the death, and in 

Page 24 of 30 
 
 IRB Protocol Template (3/1919) 
writing within 5 business days.  
 
2) Any Serious Adverse Event (e.g., suicide attempt, hospitalization)  that is both unanticipated and 
possibly related to the research will be reported in writing to the IRB within 5 business days.  Any 
serious adverse event that does not appear to be both unanticipated and related to the research, will be 
recorded and reported at continuing review.  
 
3) Any adverse event that does not appear to be serious, unanticipated or related to the research will be 
summarized in the VA Bedford IRB annual progress reports that are submitted to the IRB for continuing 
review.  
 
 
4.6.3 Potential Benefits:  
 
1) Participants may benefit from a potentially effective mHealth intervention. Veterans may benefit by 
being able to improve their academic functioning and access to VHA resources for student Veterans.  
2) Moreover, the research will help to understand whether mobile, asynchronous interventions for 
academic issues are beneficial for Veterans also managing mental health concerns, as there is still only 
limited research on efficacious mobile app interventions for student Veterans 
3) The information gained from this study will provide direction for future research studies on mHealth 
apps for student Veterans and provide important information on the feasibility and acceptability of the 
VetEd app that can be used for a larger trial. 
 
 
4.6.4 Alternative Procedures:  
 
Participants are free to decline entrance into or withdraw their participation in this study at any time. 
However, if they decline to participate or withdraw from the study, and they indicate that they desire 
assistance with their academic functioning, they will be provided with information about VA supported 
education services and VITAL services. In addition, if Veterans decline to participate or withdraw and 
reported that they would like additional mental health treatments at the VA, they will be referred to 
services such as the mental health clinic, and treatment may include individual and group counseling, 
peer support, and/or medications. Similar treatments are available through programs in the community 
as well. A decision not to participate in this study or to withdraw from this study will not affect the 
participant’s ability to participate in standard VA treatment for educational assistance or mental health 
support. 
 
4.7 Costs and Payments 
 
Focus Groups: We will pay subjects $40 each (in the form of a gift card) for their participation in focus 
groups (1.5 hours). We expect to have a total of 21 Veterans across the three focus groups. 
 
Usability testing: Usability pilot participants will receive $80 for the baseline assessment (2.5 hours), 
$20 for a mid-point check-in to evaluate any potential usability issues (45 minutes), and $90 for follow-
up visit assessments after the pilot testing (2 hours), for a total of $190 per participant. 
 

Page 25 of 30 
 
 IRB Protocol Template (3/1919) 
4.8    Providing for Reuse of Data 
 
Data will not be reused in future studies. 
 
4.9   Creation of a Tissue Bank 
A tissue bank will not be created for this study. 
 
5.0 
Resources 
The PI is the Bedford Education Director of the VISN 1 MIRECC, which provides space and staff for 
this study. The VISN 1 Mental Illness Research, Education, and Clinical Center (MIRECC) involves 
several VA medical centers in the New England network, is centrally administered at the Bedford VA. 
Study activities will take place in the offices of the MIRECC in Buildings 5 and 9. 
 
6.0 
Collaborations 
 
To ensure that the developed VetEd app is responsive to both Veteran-identified needs and VHA 
priorities for Veteran educational supports, a VetEd advisory panel will also be formed, including the 
following stakeholders: VACO consultants Dr. Shana Bakken (National Director, Compensated Work 
Therapy, see support letter) and Dr. Khamkay Chitaphong (National Director, Veterans Integration To 
Academic Leadership), Alisa Bennett (Bedford VAMC VITAL Program Manager), current VITAL-SEd 
providers, and Kristine Babcock (Veterans Services Coordinator, Northshore Community College). This 
will ensure that crucial feedback from senior VACO stakeholders, community partners, and SEd experts 
guide app content and future VetEd app dissemination early in the research process. This group will 
meet with the research staff quarterly, beginning with a project kick-off upon the start of the project. 
These panel members will 1) not be involved in the research study as collaborators or co-investigators, 
2) not have access to research data. The panel members may be provided with options for changes and 
additions to the mobile application as informed by Veteran focus groups, for the purposes of making 
decisions regarding additional app content. This will not include the presentation of Veteran research 
data. 
 
7.0 
Qualifications of the Investigators 
 
Dr. Erin Reilly, Ph.D. is the Education Director, Psychologist, and Investigator with the Mental Illness 
Research, Education, and Clinical Center (MIRECC), working on technology-assisted interventions 
using Acceptance and Commitment Therapy (ACT) with Veterans managing chronic mental and 
physical health concerns. Much of her recent work explores the use of technology to deliver acceptance 
and mindfulness-based interventions supporting psychological and physical functioning concerns for 
Veterans, with a particular emphasis on chronic pain, anxiety, and physical health-related functioning. 
She has also received a Rehabilitation R&D Career Development Award to support the development and 
evaluation of an online intervention for chronic pain management. She also worked with Dr. Lisa 
Mueller on the development and testing of the Veteran Integration to Academic Leadership – Supported 
Education (VITAL-SED) manualized intervention pilot. Her experience with both developing 
interventions for student Veterans and creating online technologies for students in higher education are 
uniquely suited to this proposal for the development and iterative evaluation of the VetEd app. She is the 
named Primary Investigator for the Rehabilitation R&D SPiRE that will fund this development and 
usability pilot project. 
 

Page 26 of 30 
 
 IRB Protocol Template (3/1919) 
Dr. Megan Kelly, Ph.D. is the Co-Director and the Bedford Site Director of the VISN 1 MIRECC. She 
has been involved in the psychosocial treatment research for the past 19 years, with a particular focus on 
the development of behavioral interventions to improve the community reintegration of Veterans with 
PTSD. She has published on trauma and stress responding, the relationship between trauma and barriers 
to VA care, and the effects of stress and trauma on OEF/OIF veterans. She has had five VA and NIH-
funded projects (including three RR&D funded projects) to develop and evaluate new behavioral 
interventions for Veterans focused on the improving the psychological health and community 
reintegration of Veterans. She also has substantial expertise in the development of Veteran-facing 
mobile interventions, including online websites for tobacco cessation, and mobile app development to 
address social support issues for Veterans with PTSD. Her research is focused on social functioning in 
individuals with mental health disorders, and in particular, how social avoidance is an important obstacle 
to positive life functioning. Thus, she has extensive experience with psychosocial treatment research for 
Veterans with PTSD and other mental health disorders. She has a demonstrated record of successful and 
productive projects in the area of individuals with PTSD and her expertise and experience have prepared 
her to serve as a co-Investigator for the proposed project. 
 
Dr. Lisa Mueller, Ph.D. is the Clinical Director of the Bedford VA Vocational program, and has been 
part of a team implementing supported education for the past ten years and am an experienced 
supervisor for this service. She was co-investigator with Dr. Marsha Ellison for her participatory action 
research study that led to the development of the Veterans Supported Education manual (VetSEd). She 
has also worked with Dr. Ellison on a number of supported education studies and recently co-led a work 
group with the VHA Vocational Rehabilitation National Director, Dr. Shana Bakken, for the Office of 
Mental Health and Suicide Prevention to revise the original VetSEd manual to create the CWT SEd 
manual. She has presented on the provision of education services to veterans in local, regional, and 
national presentations for the past eight years and serve as one of the four content experts in the field 
leading a national quarterly technical assistance call for VA Vocational Rehabilitation programs. As an 
investigator for the VISN1 Mental Illness Research Education and Clinical Center (MIRECC), she has 
also conducted qualitative analysis from focus groups and individual interview data on funded research 
projects and received RR&D VA funding to develop and evaluate a motivational enhancement 
intervention for veterans with serious mental illness to increase competitive employment outcomes and 
have contributed to VA funded research in the areas of vocational rehabilitation. 
 
 
 
 
 

Page 27 of 30 
 
 IRB Protocol Template (3/1919) 
REFERENCES: 
 
1. Kleykamp, M. (2013). Unemployment, earnings and enrollment among post 9/11 
veterans. Social science research, 42(3), 836-851. 
2. U.S. Department of Veteran Affairs (2020, June 11). Learn About Student Veterans. VA Mental 
Health. Retrieved from  https://www.mentalhealth.va.gov/student-veteran/learn-about-student-
veterans.asp 
3. Bass, E. (2019). The Post-9/11 GI Bill: Beneficiaries, Choices, and Cost. Congressional Budget 
Office. Retrieved from https://www.cbo.gov/system/files/2019-05/55179-Post911GIBill.pdf 
4. Semega, J. L., Fontenot, K. R., & Kollar, M. A. (2020). Income and poverty in the United States: 
2019. Current Population Reports, P60-259. 
5. Tamborini, C. R., Kim, C., & Sakamoto, A. (2015). Education and lifetime earnings in the 
United States. Demography, 52(4), 1383-1407. 
6. Domenici, P., Armstrong, K., & Best, S. (2013). Courage after fire for parents of service 
members: Strategies for coping when your son or daughter returns from deployment. New 
Harbinger Publications. 
7. Steele, J. L., Salcedo, N., & Coley, J. (2010). Service members in school: Military veterans’ 
experiences using the post-9/11 GI bill and pursuing postsecondary education. Retrieved from 
http://www.rand.org/content/dam/rand/pubs/monographs/2011/RAND_MG1083.pdf 
8. Armstrong, K., Best, S., & Domenici, P. (2013). Courage after fire for parents of service 
members: Strategies for coping when your son or daughter returns from deployment. Oakland, 
CA: New Harbinger. 
9. Brancu, M., Wagner, H. R., Morey, R. A., Beckham, J. C., Calhoun, P. S., Tupler, L. A., ... & 
Fairbank, J. A. (2017). The Post‐Deployment Mental Health (PDMH) study and repository: A 
multi‐site study of US Afghanistan and Iraq era veterans. International Journal of Methods in 
Psychiatric Research, 26(3). 
10. Valenstein, M., Clive, R., Ganoczy, D., Garlick, J., Walters, H. M., West, B. T., ... & Pfeiffer, P. 
N. (2020). A nationally representative sample of veteran and matched non-veteran college 
students: Mental health symptoms, suicidal ideation, and mental health treatment. Journal of 
American college health, 1-10. 
11. Kulesza, M., Pedersen, E. R., Corrigan, P. W., & Marshall, G. N. (2015). Help-seeking stigma 
and mental health treatment seeking among young adult veterans. Military behavioral health, 
3(4), 230-239. 
12. Barry, A. E., Whiteman, S. D., & MacDermid Wadsworth, S. (2014). Student service 
members/veterans in higher education: A systematic review. Journal of Student Affairs Research 
and Practice, 51(1), 30-42. 
13. Barr, A., Bird, K., Castleman, B., & Skimmyhorn, W. (2019). A comparison of postsecondary 
outcomes for Army service members, Veterans, and civilians. EdWorking Paper, (19-50). 
14. Cate, C. A. (2014). Million records project: Research from student veterans of America. Student 
Veterans of America, 33. 
15. National Academies of Sciences, Engineering and Medicine (2018). Evaluation of the 
Department of Veterans Affairs Mental Health Services. Washington, DC: The National 
Academies Press. 

Page 28 of 30 
 
 IRB Protocol Template (3/1919) 
16. Cheney, A. M., Koenig, C. J., Miller, C. J., Zamora, K., Wright, P., Stanley, R., ... & Pyne, J. M. 
(2018). Veteran-centered barriers to VA mental healthcare services use. BMC health services 
research, 18(1), 1 - 14. 
17. Smith-Osborne, A. (2012). Supported education for returning veterans with PTSD and other 
mental disorders. Journal of Rehabilitation, 78(2), 4-12 
U.S. Department of Veterans Affairs [USDVA]. (2021, January 28). VetSuccess on Campus. 
Retrieved from https://www.benefits.va.gov/vocrehab/vsoc.asp  
18. U.S. Department of Veterans Affairs [USDVA]. (2020, July 16). VITAL Program. Retrieved 
from https://www.mentalhealth.va.gov/student-veteran/vital-program.asp 
19. Kukla, M., McGuire, A. B., & Salyers, M. P. (2016). Rural and urban supported employment 
programs in the Veterans Health Administration: Comparison of barriers and facilitators to 
vocational achievement for veterans experiencing mental illnesses. Psychiatric rehabilitation 
journal, 39(2), 129. 
20. Burnam, M. A., Meredith, L. S., Tanielian, T., & Jaycox, L. H. (2009). Mental health care for 
Iraq and Afghanistan war veterans. Health Affairs, 28(3), 771-782. 
21. Kim, P. Y., Britt, T. W., Klocko, R. P., Riviere, L. A., & Adler, A. B. (2011). Stigma, negative 
attitudes about treatment, and utilization of mental health care among soldiers. Military 
Psychology, 23(1), 65–81. 
22. Schnurr, P. P., Lunney, C. A., Bovin, M. J., & Marx, B. P. (2009). Posttraumatic stress disorder 
and quality of life: Extension of findings to veterans of the wars in Iraq and Afghanistan. 
Clinical psychology review, 29(8), 727-735. 
23. Seal, K. H., Metzler, T. J., Gima, K. S., Bertenthal, D., Maguen, S., & Marmar, C. R. (2009). 
Trends and risk factors for mental health diagnoses among Iraq and Afghanistan veterans using 
Department of Veterans Affairs health care, 2002–2008. American Journal of Public Health, 
99(9), 1651-1658. 
24. Rotheram-Borus, M. J., Swendeman, D., & Chorpita, B. F. (2012). Disruptive innovations for 
designing and diffusing evidence-based interventions. American Psychologist, 67(6), 463. 
25. Pew Research Center, Internet and Technology (2019, June 12). Mobile Fact Sheet. Retrieved 
from https://www.pewresearch.org/internet/fact-sheet/mobile/ 
26. Ybarra, M. L., & Eaton, W. W. (2005). Internet-based mental health interventions. Mental health 
services research, 7(2), 75-87. 
27. Masango, M., Muloiwa, T., Wagner, F., & Pinheiro, G. (2020). Design and Implementation of a 
Student Biographical Questionnaire (BQ) Online Platform for Effective Student Success. 
Journal of Student Affairs in Africa, 18(1), 93-110. 
28. Pechenkina, E., Laurence, D., Oates, G., Eldridge, D., & Hunter, D. (2017). Using a gamified 
mobile app to increase student engagement, retention and academic achievement. International 
Journal of Educational Technology in Higher Education, 14(1), 1-12. 
29. Ellison, M. L., Mueller, L., Henze, K., Corrigan, P., Larson, J., Kieval, N. E., . . . Smelson, D. 
(2012). The veteran supported education treatment manual (VetSEd). Bedford, MA: ENRM 
Veterans Hospital, Center for Health Quality, Outcomes. 
30. Ellison, M. L., Mueller, L., Smelson, D., Corrigan, P. W., Torres Stone, R. A., Bokhour, B. G.,... 
Drebing, C. (2012). Supporting the education goals of post-9/11 veterans with self-reported 

Page 29 of 30 
 
 IRB Protocol Template (3/1919) 
PTSD symptoms: A needs assessment. Psychiatric Rehabilitation Journal, 35, 209 –2. 
doi.org/10.2975/35.3.2012.209.217 
31. Ellison, M. L., Reilly, E. D., Mueller, L., Schultz, M. R., & Drebing, C. E. (2018). A supported 
education service pilot for returning veterans with posttraumatic stress disorder. Psychological 
Services, 15(2), 200-207. 
32. Ellison, M.L., Mueller, L., Scoglio, A.J., Crossman, D.M., Calixte, R.M., & Drebing, C.E. 
(2020). The Veterans Integration to Academic Leadership and Supported Education (VITAL-
SEd). Bedford, MA: VA Bedford Healthcare System, Social and Community Reintegration 
Research Program. 
33. Franchi, V., & Swart, T. M. (2003). Rapid assessment procedures: A participatory action 
research approach to field training in community prevention and intervention. Journal of 
Prevention & Intervention in the Community, 25(1), 99-115. 
34. Virzi, R. A. (1992). Refining the test phase of usability evaluation: How many subjects is 
enough?. Human factors, 34(4), 457-468. 
35. First, M. B. (2014). Structured clinical interview for the DSM (SCID). The Encyclopedia of 
Clinical Psychology, 1-6. 
36. Brooke, J. (1996). SUS-A quick and dirty usability scale. Usability Evaluation in Industry, 
189(194), 4-7. 
37. Tariman, J. D., Berry, D. L., Halpenny, B., Wolpin, S., & Schepp, K. (2011). Validation and 
testing of the Acceptability E-scale for web-based patient-reported outcomes in cancer care. 
Applied Nursing Research, 24(1), 53-58. 
38. Selim, A. J., Rogers, W., Fleishman, J. A., Qian, S. X., Fincke, B. G., Rothendler, J. A., & Kazis, 
L. E. (2009). Updated US population standard for the Veterans RAND 12-item Health Survey 
(VR-12). Quality of Life Research, 18(1), 43-52. 
39. Robbins, S. B., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A. (2004). Do 
psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological 
Bulletin, 130(2). 
40. Chemers, M. M., Hu, L. T., & Garcia, B. F. (2001). Academic self-efficacy and first year college 
student performance and adjustment. Journal of Educational psychology, 93(1). 
41. Luzzo, D. A., & McWhirter, E. H. (2001). Sex and ethnic differences in the perception of 
educational and career‐related barriers and levels of coping efficacy. Journal of Counseling & 
Development, 79(1), 61-67. 
42. Martorella, G., Boitor, M., Berube, M., Fredericks, S., Le May, S., & Gélinas, C. (2017). 
Tailored Web-Based Interventions for Pain: Systematic Review and Meta-Analysis. Journal of 
Medical Internet Research, 19(11). 
43. Carroll, J. K., Moorhead, A., Bond, R., LeBlanc, W. G., Petrella, R. J., & Fiscella, K. (2017). 
Who uses mobile phone health apps and does use matter? A secondary data analytics approach. 
Journal of medical Internet research, 19(4), e125.  
44. Zhao, J., Freeman, B., & Li, M. (2016). Can mobile phone apps influence people’s health 
behavior change? An evidence review. Journal of medical Internet research, 18(11), e287. 
45. Horvath, K. J., Alemu, D., Danh, T., Baker, J. V., & Carrico, A. W. (2016). Creating effective 
mobile phone apps to optimize antiretroviral therapy adherence: perspectives from stimulant-
using HIV-positive men who have sex with men. JMIR mHealth and uHealth, 4(2), e5287. 

Page 30 of 30 
 
 IRB Protocol Template (3/1919) 
46. Crossman, D.M., Ahern, M.E., Girouard, C.M,...Ellison, M.L. (in press). Improving Healthcare 
Access and Utilization among Student Veterans through Supported Education: A Case Series. 
Journal of Psychosocial Rehabilitation and Mental Health. 
47. Guest, G., Namey, E., & McKenna, K. (2017). How many focus groups are enough? Building an 
evidence base for nonprobability sample sizes. Field methods, 29(1), 3-22. 
48. Mason, M. (2010). Sample size and saturation in PhD studies using qualitative interviews. 
Qualitative social research, 11(3). 
49. Sayer, N. A., Frazier, P., Orazem, R. J., Murdoch, M., Gravely, A., Carlson, K. F., ... & 
Noorbaloochi, S. (2011). Military to civilian questionnaire: A measure of postdeployment 
community reintegration difficulty among veterans using Department of Veterans Affairs 
medical care. Journal of traumatic stress, 24(6), 660-670.