Work as a Determinant of Health: A Pragmatic Trial of Enhanced Cognitive Behavioral Therapy to Bolster Competitive Work and Wellness in Veterans With Serious Mental Illness (WORKWELL)
Phase: NA
Conditions: Posttraumatic Stress Disorders
Sponsor: VA Office of Research and Development

VA Central IRB
Effective Date: May 4, 2022
Protocol Number: CIRB 20-08 
Work as a determinant of health:
A pragmatic trial of enhanced cognitive behavioral therapy to bolster competitive 
work and wellness in veterans with serious mental illness (WORKWELL)
Funding Agency: HSR&D 
Principal Investigator: Marina Kukla, Ph.D.
Version 4, 6/30/2022
 
VA Central IRB
Effective Date:
July 11, 2022

Abstract
BACKGROUND: Work is a major social determinant of health. In people with serious mental 
illness (SMI), work is associated with better wellbeing, physical and mental health, quality of life, 
and may prevent the onset of disability. Among Veterans with SMI, work is a protective factor 
against suicide.
Most Veterans with SMI are unemployed and suffer substantially worse health and recovery 
across key domains. Despite quality VHA vocational services, such as supported employment 
(SE), two-thirds or more of Veterans who receive these services experience work dysfunction. A 
probable explanation lies in unsolved cognitive and behavioral barriers, such as low work- 
related self-efficacy, ineffective coping skills, little hope that work is attainable, poor work 
motivation and sense of self.
The Cognitive Behavioral Therapy for Work Success (CBTw) intervention was designed to 
target these problems and augment VHA SE services to synergistically improve work, as well 
as health and recovery, in Veterans with SMI. In an open trial pilot, CBTw was associated with 
significant increases in hours worked and wages earned and the majority of CBTw participants 
became steady workers. Veterans also experienced improvements in symptoms, recovery, and 
quality of life.
PROJECT GOALS: Using a randomized controlled trial (RCT) design over a 12-month study 
period at 3 VA SE programs, this project will 1) test the effects of CBTw on competitive work; 2) 
test the effects of CBTw on health and recovery outcomes; 3) Informed by the RE-AIM 
framework, conduct an implementation evaluation of CBTw to inform future wide scale roll-out 
of the intervention across the Veterans Health Administration (VHA).
OBJECTIVES:
Aim
 
  1  : Test the effects of CBTw + SE compared to a control of psychoeducation + SE on work. 
Hypotheses: Participants in the CBTw+ SE arm will work significantly more total weeks in 
competitive jobs (primary study outcome) and will be more likely to become steady workers.
Aim
 
  2  : Test the effects of CBTw + SE on health and recovery.
Hypotheses: Participants in the CBTw + SE arm will have greater improvements on subjective 
recovery and health-related quality of life, and decreases in symptoms, suicidal ideation, and 
inpatient service utilization.
Aim
 
  3  : Conduct an evaluation of the implementation of CBTw using VA supported employment 
staff to deliver the intervention. Implementation evaluation will examine the following RE-AIM 
domains: Reach—number of Veterans who take part and complete CBTw. Effectiveness
 
 — 
examined in Aims 1 and 2. Adoption—factors that influence uptake by SE programs examined 
via interviews and during site visits. Implementation—fidelity to CBT model (collected during 
monthly assessments)-primary Aim 3 outcome; and key barriers and facilitators to CBTw 
implementation gauged via site visits and key informant staff interviews. Maintenance—factors 
that influence continued CBTw use across time examined through key informant staff 
interviews.
METHODS:
WORKWELL is a pragmatic, Hybrid 1 design RCT. CBTw will be tested at 3 SE sites— 
Roudebush VA Medical Center, the Edward J. Hines VA Medical Center, and the VA St. Louis 
Health Care System. 276 unemployed Veterans with SMI will be randomly assigned to receive 
CBTw plus regular supported employment services or a control of psychoeducation plus regular
3/15/2022
VA Central IRB Protocol, Version 3 Page 2 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

supported employment services. Outcomes including total weeks worked in competitive jobs 
(primary study outcome), achievement of steady work, symptoms, recovery, health related 
quality of life, suicidal ideation, and service utilization will be assessed at posttreatment (12 
weeks), 6 months (primary endpoint), and 9 months (to examine sustained effects). Competitive 
work outcomes will also be assessed at 12 months. Aim 1 and Aim 2 work and recovery 
outcomes will be analyzed using multilevel modeling to compare the intervention and control 
groups over time. CBTw implementation planning, training, and consultation will be provided to 
SE staff persons delivering the intervention. CBTw implementation outcomes will be analyzed 
using a rapid mixed methods approach.
3/15/2022
VA Central IRB Protocol, Version 3 Page 3 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

List of Abbreviations
BDI-II: Beck Depression Inventory-II
CAPRI: Compensation and Pension Record Interchange 
CBT: Cognitive Behavioral Therapy
CBTw: Cognitive Behavioral Therapy for Work Success Intervention
COFAM: Dr. Rollins and Dr. Salyers completed HSR&D IIR study, "Comparison of Fidelity 
Assessment Methods"
CDA: Career Development Award 
CDW: Corporate Data Warehouse
CFIR: Consolidated Framework for Implementation Research 
CPRS: Computerized Patient Record System
CTS-R: Cognitive Therapy Scale-Revised 
CWT: Compensated Work Therapy
DSM-5: Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition
ERIC: Expert Recommendations for Implementing Change (resulted in a compilation of 
implementation strategies)
Hines: Edward Hines Jr. Hospital (study site) 
HSR&D: Health Services Research and 
Development ICC: Intraclass Correlation Coefficient
IMR: Illness Management and Recovery
IVIP: Indianapolis Vocational Intervention Program 
MAR: Missing at Random
MCAR: Missing Completely at Random 
MCS: SF-12 Mental Component Summary 
NIMH: National Institute of Mental Health 
PCS: SF-12 Physical Component Summary
PRISM: Expansion of RE-AIM implementation framework including inner and outer settings 
factors
PTSD: Post traumatic stress disorder
QUERI: Quality Enhancement Research Initiative 
RA: Research Assistant
RAIN-MH: Dr. McGuire HSR&D funded study, "Evaluation of Recovery-oriented Acute INpatient 
Mental Healthcare"
RCT: Randomized Controlled Trial
RE-AIM: Implementation science framework intended to translate research findings into 
practice; includes domains of Reach, Effectiveness, Adoption, Implementation, Maintenance
3/15/2022
VA Central IRB Protocol, Version 3 Page 4 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

RLR: Richard L. Roudebush VA Medical Center 
St. Louis: St. Louis VA Health System (study site) 
SE: Supported Employment
SF-12: 12-Item Short Form Health Survey 
SMI: Serious Mental Illness
SWS: Satisfaction with Services Scale
TSES: Therapeutic and Supported Employment Services, VA Central Office 
VA: Veterans Administration
VACO: Veterans Affairs Central Office 
VHA: Veterans Health Administration 
VISN: Veteran Integrated Service Networks
VistA: Veterans Health Information System and Technology Architecture 
VRC: Vocational Rehabilitation Counselors
WORKWELL: Dr. Kukla proposed HSR&D study, “Work as a determinant of health: A pragmatic 
trial of enhanced cognitive behavioral therapy to bolster competitive work and wellness in 
veterans with serious mental illness”
6/30/2022
VA Central IRB Protocol, Version 3 Page 5 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

Contents
Protocol Title..............................................................................................................................................7
1.0
Study Personnel.............................................................................................................................7
2.0
Introduction....................................................................................................................................8
3.0
Objectives....................................................................................................................................10
4.0
Resources and Personnel...........................................................................................................12
5.1
Study Procedures........................................................................................................................17
5.2 Study Design.................................................................................................................................17
5.3 Recruitment Methods...................................................................................................................21
5.4 Informed Consent Procedures.....................................................................................................24
5.5 Inclusion/Exclusion Criteria...........................................................................................................25
5.6 Study Evaluations.........................................................................................................................26
5.7 Data Analysis................................................................................................................................29
5.8 Withdrawal of Subjects.................................................................................................................30
6.0
Reporting.......................................................................................................................................31
7.0
Privacy and Confidentiality..........................................................................................................31
8.0
Communication Plan....................................................................................................................32
9.0
References....................................................................................................................................32
3/15/2022
VA Central IRB Protocol, Version 3 Page 6 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

Protocol Title: Work as a determinant of health: A pragmatic trial of enhanced cognitive 
behavioral therapy to bolster competitive work and wellness in veterans with serious mental 
illness (WORKWELL)
1.0
Study Personnel
1.1. Participating Sites:
Three VA sites, each from a different VISN, will participate in this study: Richard L. Roudebush 
VA Medical Center (RLR), Edward Hines Jr. Hospital (Hines), or the St. Louis VA Health 
System (St. Louis). The study PI will be stationed at RLR, and Local Site Investigators have 
been appointed at Hines and St. Louis. Start-up activities will occur concurrently at all sites.
1.2. Principal Investigator:
Marina Kukla, PhD
Contact Information: marina.kukla2@va.gov 317-988-3330
Facility: Richard L. Roudebush VA Medical Center, 1481 W 10th St, Indianapolis, IN 46202
Employee Status: 8/8th VA
1.3. Co-Investigators:
Alan McGuire, PhD
Contact Information: alan.mcguire@va.gov 317-988-2725
Facility: Richard L. Roudebush VA Medical Center, 1481 W 10th St, Indianapolis, IN 46202
Employee Status: 8/8th VA
Teresa Damush, PhD
Contact Information: teresa.damush@va.gov 317-988-4277
Facility: Richard L. Roudebush VA Medical Center, 1481 W 10th St, Indianapolis, IN 46202
Employee Status: 8/8th VA
Paul Lysaker, PhD
Contact Information: paul.lysaker@va.gov 317-988-2546
Facility: Richard L. Roudebush VA Medical Center, 1481 W 10th St, Indianapolis, IN 46202
Employee Status: 8/8th VA
Angela Rollins, PhD
Contact Information: angela.rollins@va.gov 317-988-4789
Facility: Richard L. Roudebush VA Medical Center, 1481 W 10th St, Indianapolis, IN 46202
Employee Status: 8/8th VA
Wei Wu, PhD
Contact Information: wwu1@iu.edu 317-274-6945
Facility: Richard L. Roudebush VA Medical Center, 1481 W 10th St, Indianapolis, IN 46202
Affiliation Status: IPA
3/15/2022
VA Central IRB Protocol, Version 3 Page 7 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

1.4. Local Site Investigators:
Kenneth Weber, MS, CRC, LCPC
Contact Information: kenneth.weber@va.gov 708-202-7897
Facility: Edward Hines Jr. VA Hospital, 5000 South 5th Avenue, Hines, IL 60141
Employee Status: 8/8th VA
Jessica Hatfield, MS
Contact Information: jessicahatfield@va.gov 314-652-4100 ext 65087
Facility: VA St. Louis HCS, Jefferson Barracks, 1 Jefferson Barracks Drive, St. Louis, MO 63125
Employee Status: 8/8th VA
2.0
Introduction
Among Veterans with serious mental illness (SMI), employment problems are pervasive. 
Over 1 million Veterans have a serious mental illness (SMI)1, formally defined by SAMHSA and 
the VA as the presence of a mental disorder that substantially limits role functioning, including 
work2, and encompasses diagnoses of major depression, bipolar disorder, posttraumatic stress 
disorder (PTSD), and schizophrenia-spectrum disorders2. Compared to unemployment of 
around 4.5% in the general Veteran population3, over 75% of Veterans with SMI are 
unemployed.4 In studies across groups, such as female Veterans5, Veterans of Post-911 
conflicts6, and other combat eras7, SMI is linked with worse work outcomes5. This problem is 
especially troubling because most people with SMI desire to work8; many search for a job with 
little success4. Even among the tens of thousands of Veterans with SMI who receive VHA 
employment services yearly, which focus on helping Veterans obtain community jobs, two-thirds 
or more continue to experience work deficits9.
Employment problems in Veterans with SMI are costly at the personal and societal level.
Unemployment in Veterans with SMI is associated with a higher risk of poverty and greater 
reliance on entitlements and the service system10. Veterans with SMI report greater financial 
troubles, more difficulties covering basic needs, and higher debt11. Overall, unemployment 
creates a deleterious feedback loop of poverty, declining mental status, and related problems, 
such as substance misuse4. Similarly, studies have highlighted that among Veterans with SMI, 
the problem of lack of participation in the workforce has major economic and well-being effects 
that persist into the future, if not adequately addressed12.
Supported by public health research, employment is a major social determinant of 
health13. Employment dysfunction leads to poor health and wellness, including among those 
with SMI receiving quality employment services. In the general population, precarious 
employment has many chronic physical health effects13 and is a major cause of distress and 
poor mental health14. Because people with SMI are particularly susceptible to the negative 
outcomes linked with unemployment, the effects are more damaging and include poor recovery, 
worsening illness trajectory, and more days of psychiatric hospitalization13. In addition, Veterans 
with SMI have more medical conditions15 and experience deteriorating quality of life 16.These 
health problems also persist in unemployed persons with SMI who receive quality employment 
services17.
Sustained work is associated with improvements in health and recovery in SMI, a Health 
Services Research and Development (HSR&D) priority. Across prospective studies, the health 
and wellness benefits of work in people with SMI who are receiving employment support include 
improved self-esteem and life satisfaction, a reduction in psychiatric symptoms, and fewer 
hospitalizations. In addition, studies of Veterans with SMI and broader meta-analysis evidence
3/15/2022
VA Central IRB Protocol, Version 3 Page 8 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

indicates that work leads to major gains across health, quality of life, and wellness areas20. 
These health and recovery benefits are particularly strong when persons engage in steady, 
long-term competitive work17,. Competitive work refers to a regular community job open to 
anyone, and pays at least minimum wage21.
Competitive employment also leads to lower healthcare utilization and lower service 
system costs. A study by Bush and colleagues22 found that people with SMI who had received 
employment services and worked in competitive employment over 10 years had significantly 
reduced service utilization and mental health costs compared with persons who worked little or 
inconsistently. Similarly, young adults with SMI who work use fewer mental health services, take 
fewer medications, and are less reliant on disability entitlements23. Among this group, there is 
evidence that employment may delay or altogether prevent subsequent disability24.
Aligning with a current major VA and HSR&D priority, employment is a protective factor 
against suicide, whereas unemployment leads to increased suicide risk. Meta-analytic evidence 
demonstrates that unemployment is associated with a significantly higher risk of suicide, a 
relationship that is particularly strong among people with mental illness25. A large study found 
that Veterans with recent suicidal ideation were more likely to have PTSD and report 
psychosocial barriers, including work problems. On the other hand, a sense of purpose, such as 
that found through career, was negatively associated with suicidal thoughts26. Similarly in a 
sample of Veterans with depression, compared to their employed counterparts; unemployed 
persons were more likely to have thoughts of self-harm27.
Despite quality VHA employment services, employment dysfunction & poor wellness 
persist. To address work problems in Veterans with SMI, supported employment (SE) services 
have been widely implemented through VHA Therapeutic and Supported Employment Services 
(TSES), including 200 SE programs nationwide. VHA SE services are individualized and include 
a vocational assessment, rapid job search and placement in community jobs, collaboration with 
employers, and ongoing follow-along support after the job has been secured28. Supported by 
robust evidence29, high quality SE services assist Veterans with SMI to find and keep 
competitive jobs in the community. However, SE programs have achieved only partial success 
in improving outcomes29, as 67% or more of Veterans with SMI receiving SE remain 
unemployed. A further challenge is retention, as those with SMI who obtain jobs struggle to 
maintain employment long-term, even in the context of quality SE support31. Moreover, among 
many with SMI receiving SE services, these ongoing struggles to secure competitive work are 
linked with worse health and well-being outcomes across studies e.g., 17.
SE services are not designed to address complex cognitive and behavioral work-related 
barriers. SE services effectively address common barriers such as skills deficits, resume 
building deficiencies, employer-related factors, legal involvement, navigation of job 
accommodations, and job search needs32. However, as demonstrated by an extant of literature 
and our prior work; these services incompletely address many key cognitive and behavioral 
factors that impact work success33. Specifically in people with SMI, including those receiving SE, 
many cognitive and behavioral factors have been identified as work barriers or linked with poor 
work outcomes: low work-related self-efficacy34 and self-esteem35, poor motivation, loss of hope 
that work is attainable 36, interpersonal problems37, difficulties enacting coping on the job38, and 
stress. Further, job tenure has been associated with one’s perception of competence and 
efficacy on the job39. In this regard, work may present a threat to ones sense of self, resulting in 
lowered belief in ones ability to maintain a job39.
In response, cognitive behavioral therapy (CBT) approaches have been suggested to 
augment SE services and address cognitive and behavioral barriers to competitive work 
success. A vast majority of CBT interventions do not focus on functioning as a primary target.
3/15/2022
VA Central IRB Protocol, Version 3 Page 9 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

CBT is an empirically validated, recovery intervention recognized as a best practice in SMI 
care41. CBT is a standard component of VHA mental health services in treating anxiety, 
depression, bipolar disorder, PTSD, and schizophrenia. However, though traditional CBT 
interventions effectively treat psychiatric symptoms42; findings regarding the impact of CBT on 
functioning have been mixed and effect sizes are small42. One potential explanation for the lack 
of findings is that while CBT aims to restructure thoughts about symptoms and increase 
symptom coping; it does not directly address role functioning and hence may have less 
effectiveness in these areas. To address this gap, CBT interventions have begun to arise that 
emphasize correcting unhelpful cognitions and tailoring behavioral strategies directly pertinent 
to deficits in functioning.
Previous interventions combining CBT with rehabilitation practices have had positive 
results on role functioning in Veterans with SMI. A cognitive intervention focused on protected, 
time-limited noncompetitive work experiences had positive effects on work performance in 
people with chronic schizophrenia43. Further, CBT combined with social skills training has 
shown benefits beyond the effects of CBT or social skills training alone, improving skills 
mastery and interpersonal functioning in Veterans with SMI44.
CBT approaches also improve health and wellness in Veterans. Cognitive behavioral 
interventions, either alone or in combination with other approaches, have been shown to 
improve physical health, mental health, quality of life, and related indicators of health in 
Veterans45 and non-Veterans with SMI46. In addition, CBT approaches have shown benefits to 
improve physical health and functioning in Veterans with other chronic conditions, such as 
musculoskeletal pain47.
Moreover, CBT directly builds upon existing VHA SE services. Featuring the provision of 
concrete individualized services to assist Veterans in acquiring strong fitting jobs, SE involves 
job assessment, resume support, rapid job development, collaboration with employers, and 
follow along support28. However, many Veterans with SMI in SE continue to falter in their work 
pursuits, due in part to unsolved personal issues, such as cognitive and behavioral obstacles37. 
The use of an empirically supported, tailored CBT approach tackles these barriers and builds 
upon SE by providing interventions that SE is not designed or intended to provide. CBT is also 
a strong fit with the structure and goals of SE, as demonstrated in past studies and in SE 
stakeholder accounts33.
In summary, an enhanced CBT approach optimized to promote community work and 
wellness has great potential to increase meaningful functioning, improve health and recovery, 
and mitigate risk for negative outcomes, such as suicide, in Veterans with SMI receiving SE 
services.
3.0
Objectives
3.1. Study Purpose.
In partnership with the VA Office of Therapeutic and Supported Employment Services, the 
project is a pragmatic Hybrid 1 randomized controlled trial testing employment and recovery 
outcomes and examining implementation of CBTw at three VA SE programs in three VISNs— 
Richard L. Roudebush VA Medical Center, Edward Hines Jr. VA Hospital, and VA St. Louis 
Health Care System.
3/15/2022
VA Central IRB Protocol, Version 3 Page 10 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

3.2. Specific Aims & Hypotheses.
Aim 1. Test the effects of CBTw + SE compared to a time and format matched active control of 
psychoeducation + SE on competitive work outcomes across the 9 month study period. A total 
of 276 unemployed Veterans with SMI will be randomized to CBTw + SE or the control arm of 
psychoeducation + SE. Assessed at posttreatment (12 weeks), 6 months (primary endpoint), 9 
months and 12 months (to examine sustained effects), the primary outcome will be total weeks 
worked in competitive jobs. A secondary work outcome, achievement of steady competitive 
work (i.e. working at least 50% of the follow-up), will be compared between study arms across 
the 12 month follow up.
Hypothesis
 
  1  : Compared to the control arm, participants in the CBTw + SE arm will work 
significantly more weeks and will be more likely to become steady workers.
Aim
 
  2.
   Test the effects of CBTw + SE compared to the control arm of psychoeducation + SE on 
health and recovery at posttreatment (12 weeks), 6 month and 9 month follow up periods. The 
study will determine the effectiveness of CBTw to improve key health and recovery outcomes 
including health-related quality of life, suicidal ideation, subjective recovery, symptoms, and 
mental health service utilization.
Hypothesis 2a: Compared to controls, participants in the CBTw + SE arm will have 
greater improvement on subjective recovery and health-related quality of life, and 
decreases in mental health symptoms, suicidal ideation, and inpatient service utilization.
Hypothesis
 
  2b
  : Greater participation in competitive work will predict improvements in 
these outcomes over time.
Aim
 
  3.
   Guided by the RE-AIM implementation framework, conduct an implementation evaluation 
of CBTw to inform future wide scale roll-out of the intervention across the VHA. Existing 
Vocational Rehabilitation Counselors (VRCs) at the 3 VA SE programs will deliver CBTw, 
supported by an initial implementation strategy bundle—site tailored implementation planning, 
CBTw training, and weekly consultation including audit and feedback. Informed by the RE-AIM 
implementation framework, data will be collected and utilized to tailor implementation during the 
study, with assessment of the following RE-AIM domains:
(1) Reach—number of Veterans who take part and complete CBTw. (2) Effectiveness— 
examined in Aims 1 and 2. (3) Adoption—factors that influence uptake by SE programs 
examined via interviews and during site visits. (4) a. Implementation—fidelity to CBT model 
(collected during monthly assessments), the primary Aim 3 outcome; and b. key barriers and 
facilitators to CBTw implementation gauged via site visits and interviews. (5) Maintenance— 
factors that influence continued CBTw use across time examined through staff interviews.
3.3. Significance and impact on VA priorities & service mission to veterans.
Addressing national and HSR&D priorities, this project is a critical step toward 
enhancing employment and reducing poor health outcomes experienced by Veterans with 
mental health disorders, including PTSD. As discussed, Veterans with SMI, especially those 
who are unemployed, tend to have poor health and wellness. Targeting work, with its 
therapeutic benefits, is a novel way of improving these outcomes20. Systematic reviews and 
experts have also identified work interventions as an area of future development in order to 
directly bolster health and reduce such inequalities60. The WORKWELL study will fulfill this need 
and result in an intervention and implementation strategy prepared for uptake by SE programs 
across the VHA.
3/15/2022
VA Central IRB Protocol, Version 3 Page 11 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

This study advances chief HSR&D and VA goals in the area of developing novel 
interventions to improve suicide prevention for at-risk Veteran groups. Given the strong links 
between work as both a protective factor among the employed and a risk factor for suicide 
among unemployed Veterans with SMI26, combined with the evidence of the benefits of CBT on 
thoughts and actions of self-harm61; WORKWELL is an encouraging approach to tackle this 
critical issue among Veterans.
Underscored by our strong partnerships with national operational leaders, the CBTw 
intervention has high implementation and scalability potential. Pilot work indicates that CBTw 
appears initially feasible to implement, is acceptable and appropriate for Veterans with SMI, 
and is well supported by VHA SE personnel nationally. Importantly, this research is also 
strongly supported by national leaders in the VHA Therapeutic and Supported Employment 
Services (TSES) office. This project directly aligns with the TSES main priority to develop 
effective and feasible strategies to further enhance competitive work outcomes. This project is 
also a strong fit with the recent TSES transformation to provide services focused on improving 
recovery.
Indeed, if effectiveness is demonstrated and a feasible implementation strategy is finalized in 
WORKWELL, our operational partners are committed to broadly implementing CBTw.
4.0
Resources and Personnel
4.1. Richard L. Roudebush VAMC, Indianapolis, IN
Principal Investigator:
Marina Kukla, PhD
Contact Information: marina.kukla2@va.gov 317-988-3330
Facility: Richard L. Roudebush VAMC, 1481 W 10th St, Indianapolis, IN 46202
Employee Status: 8/8th VA
Dr. Kukla will be responsible for the overall management of the study and will oversee 
delivery of the CBTw intervention and psychoeducation control intervention. Further, she 
will collaborate with study sites and site investigators, train vocational rehabilitation 
counselors delivering CBTw (Aim 3), provide ongoing consultation and implementation 
support throughout the study period, as well as implementation evaluation, e.g., site 
visits (Aim 3). In addition, Dr. Kukla will help oversee the training and supervision of 
research staff, oversee data management and entry, and will lead the research team in 
the preparation of progress reports, final reports, and publications. Lastly, Dr. Kukla will 
work with VA central office vocational leaders and operational partners to disseminate 
study findings and if appropriate, and work with partners to facilitate the a widescale roll- 
out of CBTw at VHA programs nationwide.
Co-Investigators:
Alan McGuire, PhD
Contact Information: alan.mcguire@va.gov 317-988-2725
Facility: Richard L. Roudebush VAMC, 1481 W 10th St, Indianapolis, IN 46202
Employee Status: 8/8th VA
Dr. McGuire will oversee training of SE staff delivering psychoeducation, as well perform 
periodic psychoeducation fidelity assessments and provide psychoeducation
3/15/2022
VA Central IRB Protocol, Version 3 Page 12 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

consultation during the study period. Dr. McGuire will also lend his expertise as part of 
the implementation support team, conducting site visits to provide pre-implementation 
planning and the implementation evaluation component, involving observations of CBTw 
and SE services, and interviews with key stakeholders (Aim 3). In addition, he will take 
part in mixed methods analysis and integration of data collected for the implementation 
evaluation.
Teresa Damush, PhD
Contact Information: teresa.damush@va.gov 317-988-4277
Facility: Richard L. Roudebush VAMC, 1481 W 10th St, Indianapolis, IN 46202
Employee Status: 8/8th VA
As a senior scientist and implementation expert, Dr. Damush will provide guidance to 
refinement of the implementation strategy that will occur throughout the primary study 
period and the Aim 3 implementation evaluation. In addition, she has conducted several 
multi-site studies and thus will provide key guidance on important issues and strategies 
regarding successful management and execution of a multi-site trial.
Paul Lysaker, PhD
Contact Information: paul.lysaker@va.gov 317-988-2546
Facility: Richard L. Roudebush VAMC, 1481 W 10th St, Indianapolis, IN 46202
Employee Status: 8/8th VA
Dr. Lysaker’s role will be three-fold: 1) He will provide subject matter expertise in 
cognitive behavioral interventions tailored to enhance vocational functioning in SMI; 2) 
He will provide guidance and expertise on issues related to clinical assessments in 
veterans with SMI; and 3) Dr. Lysaker will provide expertise as a licensed clinical 
psychologist in accordance with the data safety plan.
Angela Rollins, PhD
Contact Information: angela.rollins@va.gov 317-988-4789
Facility: Richard L. Roudebush VAMC, 1481 W 10th St, Indianapolis, IN 46202
Employee Status: 8/8th VA
Dr. Rollins will coordinate and conduct CBTw fidelity assessment at all sites and work 
with Dr. Kukla to provide feedback to CBTw facilitators during weekly consultation 
meetings in order to ensure full adherence to the CBT model. Drawing from her unique 
background and experience in rapid qualitative and mixed methods analytic approaches, 
Dr. Rollins will also contribute to the Aim 3 implementation evaluation analysis.
Wei Wu, PhD
Contact Information: wwu1@iu.edu 317-274-6945
Facility: Richard L. Roudebush VAMC, 1481 W 10th St, Indianapolis, IN 46202
Affiliation Status: IPA
Dr. Wu will guide the randomization process, oversee and supervise Indianapolis 
research assistants and data manager with regard to basic data management and 
cleaning, as well as rudimentary analysis (e.g., descriptives), conduct all primary 
analyses for Aim 1 and Aim 2, and assist with manuscript preparation.
3/15/2022
VA Central IRB Protocol, Version 3 Page 13 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

Project Staff:
Project Coordinator: Nancy Henry, BA
Contact Information:  nancy.henry2@va.gov 765-688-0677 Facility: Richard L. 
Roudebush VAMC, 1481 W 10th St, Indianapolis, IN 46202 Affiliation Status:  IPA
As project coordinator, Ms. Henry will organize all processes required to receive IRB, 
R&D, and any other regulatory approvals for the project, as well as manage and 
coordinate multiple other research and administrative functions. In addition, she will 
coordinate and oversee all participant recruitment efforts at Roudebush and coordinate 
with the St. Louis and Hines VA sites. She will also supervise and oversee the informed 
consent process and the data collection team, coordinating with research assistants 
located onsite at Roudebush, Hines, and St. Louis. 
Research Assistant (blinded): Miranda Connors, BS
Contact Information: Miranda.Connors@va.gov 317-988-2986
Facility: Richard L. Roudebush VAMC, 1481 W 10th St, Indianapolis, IN 46202
Employee Status: 8/8th VA
Ms. Connors will have the primary responsibility of assisting Ms. Henry with participant 
recruitment and collecting research data from study participants located at the 
Roudebush VAMC, including informed consent procedures, baseline assessments, and 
all follow-up assessments. Second, Ms. Connors will enter all assessment data into the 
dataset. Third, Ms. Connors will perform administrative tasks, coordinate the research 
team and supervision meetings; in years 3 and 4 of the study, she will assist with data 
entry and cleaning. Fourth, as needed, Ms. Connors will assist Ms. Bobiles with 
transcription of staff stakeholder interviews during Year 4. Connors will be supervised by 
Henry for administrative purposes. Connors has a BS degree and has experience 
conducting research within our HSR&D Center and Research Service.
Research Assistant (unblinded): Jessica Carter, BS
Contact Information: Jessica.Carter1996@va.gov 317-988-2074
Facility: Richard L. Roudebush VAMC, 1481 W 10th St, Indianapolis, IN 46202
Employee Status: 8/8th VA
Ms. Carter will be responsible for basic data entry of assessment data into study 
databases and basic data cleaning. She will also be charged with clerical and 
administrative tasks, including reminder calls to participants for assessments and to 
attend (CBTw and psychoeducation, respectively) group sessions. Ms. Carter will also 
manage the Roudebush participant contact tracking sheet including all phone contacts,
3/15/2022
VA Central IRB Protocol, Version 3 Page 14 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

sessions attended, assessment appointments, and contacts with vocational providers 
during the study. Lastly, Ms. Carter will be the primary transcriber of interview recordings 
from Aim 3-formative evaluation.
4.2. Edward Hines Jr. VA Hospital, Hines, IL:
Local Site Investigator: Kenneth Weber, MS, CRC, LCPC
Contact Information: kenneth.weber@va.gov 708-202-7897
Facility: Edward Hines Jr. VA Hospital, 5000 South 5th Avenue, Hines, IL 60141
Employee Status: 8/8th VA
Mr. Weber will serve as site investigator for this study, overseeing all study activities at 
Hines, coordinating with local study personnel (research assistant), and coordinating 
closely with Dr. Kukla and the study team. Further, Mr. Weber will oversee SE staff who 
are delivering CBTw and psychoeducation interventions, will take part in implementation 
support activities (e.g., pre-implementation, consultation, etc.), will assist with ensuring 
that recruitment targets are met, and will participate in the formative evaluation (i.e. 
provide feedback on the fit between CBTw and SE services). He will also utilize his prior 
research experience working with regulatory boards to ensure local study approvals and 
adherence to protocols.
4.3. VA St. Louis Health Care System, St. Louis, MO:
Local Site Investigator: Jessica Hatfield
Contact Information: jessicahatfield@va.gov 314-652-4100 ext 65087
Facility: VA St. Louis HCS, Jefferson Barracks, 1 Jefferson Barracks Drive, St.
Louis, MO 
63125
Employee Status: 8/8th VA
Ms. Hatfield will serve as site investigator, overseeing all study activities at St. Louis, 
coordinating with local study personnel (research assistant), and coordinating closely 
with Dr. Kukla and the study team. Further, Ms. Hatfield will help oversee SE staff who 
are delivering CBTw and psychoeducation interventions, will take part in implementation 
support activities (e.g., pre-implementation, consultation, etc.), will assist with ensuring 
that recruitment targets are met, and will participate in the formative evaluation (i.e. 
provide feedback on the fit between CBTw and SE services). She will also assist with 
the completion of site regulatory requirements and help ensure study approvals and 
proper adherence to protocols.
4.4. Contractors
No services will be performed by contractors
4.5. Data Agreements
3/15/2022
VA Central IRB Protocol, Version 3 Page 15 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

No Memoranda of Understandings (MOUs), Data Use Agreements (DUAs), and/or 
CRADAs that are being entered into for this study.
5.1
Study Procedures
5.2 Study Design
5.2.1 Experimental Design
WORKWELL is a Hybrid 1 design, two arm RCT with 276 unemployed Veterans who 
have SMI and are receiving SE services at one of 3 VA sites: Richard L. Roudebush VA Medical 
Center (RLR), Edward Hines Jr. Hospital (Hines), or the St. Louis VA Health System (St. Louis). 
Participants will be assigned to one of two study arms: 1) the intervention arm consisting of the 
12-week, group-based CBTw plus standard SE services; 2) a time and format matched active 
control arm consisting of psychoeducation plus standard SE services. The intervention period 
will last for 12 weeks after which time Veterans will be followed up for an additional 9 months 
(total of 3 follow-up assessments: posttreatment at 12 weeks, 6 months & 9 months). During the 
study, participants will receive all standard VA services. The primary outcome will be 
competitive work (Aim 1) and secondary outcomes of health and recovery will also be evaluated 
(Aim 2).
SE staff who deliver the CBTw intervention will take part in training led by the study team 
including education on CBTw content, practice of facilitation strategies, and customization of 
content to meet Veterans’ needs. SE staff who deliver the psychoeducation intervention will take 
part in training that parallels this structure. Across the study, staff CBTw facilitators will take part 
in weekly consultation and monthly audit and feedback with study personnel.
Aim 3—Guided by RE-AIM, an evaluation of CBTw implementation will allow for ongoing 
refinement and finalization of an implementation approach (Aim 3). RE-AIM is an 
implementation science framework useful for directing the planning and evaluating of 
interventions, in order to expedite the translation of research into practice49. RE-AIM has been 
successfully utilized across studies49, including multiple HSR&D hybrid 1 intervention studies52.
RE-AIM includes 5 domains: degree of reach to the target population, effectiveness, or 
impact of the intervention, adoption by providers or settings, implementation or fidelity to an 
intervention model, and maintenance of the intervention across time49. WORKWELL will 
primarily focus on effectiveness (Aims 1 & 2) and implementation, assessed via staff fidelity to 
the CBT model (primary Aim 3 outcome). Barriers and facilitators that influence CBTw 
implementation, adoption, and maintenance in SE clinics will be examined via staff interviews.
5.2.2 Control, Intervention, & Standard of Care
Intervention: Cognitive Behavioral Therapy for Work Success (CBTw)
CBTw is a manualized intervention, tailored for unemployed Veterans with SMI who 
have competitive work goals. CBTw will be delivered by local VRCs in 12 weekly, one-hour 
sessions, in a group format with up to 8 participants per group. Once participants have been 
randomized into a group cohort, enrollment into that group is then closed. The cohort goes 
through the entire 12 session CBTw program together, promoting group cohesiveness, trust, 
and unity56. As displayed in Appendix 2, session content falls in five domains: 1) cognitive 
techniques to enhance healthy thinking pertaining to work; 2a) behavioral strategies to increase 
adaptive coping and problem solving in the context of emotions and stressors that impede work
3/15/2022
VA Central IRB Protocol, Version 3 Page 16 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

functioning; 2b) behavioral strategies to facilitate effective interpersonal functioning on the job;
3) narrative exercises to enhance participants’ sense of self as workers; and 4) the formation of 
an individualized work success plan tailored to address work barriers and capitalize on work- 
related strengths to facilitate long-term job success.
During CBTw group sessions, participants check in on their work progress, review 
homework, and review the session agenda. Next, session content is presented in an interactive 
way, allowing participants to apply material through didactic exercises, open discussions of their 
work-related experiences, and via the provision of feedback to one another. Participants are 
provided CBTw manuals containing session content to keep and bring to weekly sessions.
These manuals also guide homework assignments and provide space for further narrative 
reflections, goal setting, application of cognitive restructuring, practice of coping strategies, and 
problem solving. In addition, in Session 6: “Coping with Work Stress and Anxiety,” each 
participant is provided a PlayAway device, a listening device containing audio recordings of 
coping exercises (e.g., breathing); this is intended for use at home and on the job. Veterans will 
keep the PlayAway device after study participation.
CBTw sessions will be audio recorded for the purposes of audit and feedback fidelity 
review (Aim 3). If necessary, group will be held remotely according to local SOPs for remote 
groups, which are now being developed by local Service Line Leadership. In cases where 
group is conducted remotely, Veterans will receive a “Welcome Packet” which will include the 
CBTw Participant Manual (Appendix 2), the Veteran Self-Report Monthly Work Update Form 
(Appendix 8), and a copy of the Study Information Sheet. Veterans will receive the PlayAway 
device shortly before it is needed (utilized in a later module). This Welcome Packet, and the 
PlayAway device, will be mailed directly to the Veteran, or mailed to their local VR office if they 
so prefer or if they do not have an address where they can receive the packet. As described in 
the CBTw Manual, intervention providers will have to option to provide individual booster 
sessions for veterans who miss group sessions.
Control: Psychoeducation
The control arm will receive the 3 psychoeducation modules from the Illness 
Management and Recovery (IMR) program for adults with SMI (Appendix 4). Psychoeducation 
was chosen as the control because it is not work related and potential overlap with CBTw or SE 
services as usual is low. IMR is a well-respected intervention, and perceived credibility is high, 
increasing control group participant engagement and reducing attrition risk.
The control intervention will occur in cohort groups of up to 8 participants, led by local 
VRCs. Each psychoeducation module is presented across 4 sessions (total of 12 weekly group 
sessions, 1 hour in duration, mirroring the CBTw structure). Modules include “Practical Facts 
about Mental Illness,” “The Stress Vulnerability Model,” and “Understanding Relapse.” Review 
evidence indicates that IMR is unrelated to better functioning in SMI77 and, psychoeducation 
alone does not predict improvements in symptoms or functioning78. Psychoeducation training 
will be provided by Dr. McGuire, an IMR expert79. Furthermore, the CTS-R, a highly used fidelity 
measure for CBT interventions, will be used to examine fidelity in both the CBTw and 
psychoeducation conditions. The benchmark for low fidelity on the CTS-R (score of 3 or lower) 
will be used to ensure that drift in the control arm has not occurred. If necessary, group will be 
held remotely according to local SOPs for remote groups, which are now being developed by 
local Service Line Leadership. In cases where group is conducted remotely, Veterans will 
receive a “Welcome Packet” which will include the participant portions of the IMR Manual 
(Appendix 4), the Veteran Self-Report Monthly Work Update Form (Appendix 8), and a copy of 
the Study Information Sheet. This Welcome Packet will be mailed directly to the Veteran, or
3/15/2022
VA Central IRB Protocol, Version 3 Page 17 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

mailed to their local VR office if they so prefer or if they do not have an address where they can 
receive the packet.
Standard Care: VHA Supported Employment (SE) Services
Participants in both study arms will receive standard SE services during the entire study 
period. SE services are individualized; vocational specialists meet one-on-one with Veterans 
and provide job assistance tailored to their needs and preferences. SE services are based on 
the evidence based Individual Placement and Support (IPS) model28 that includes the following 
main tenets: a zero exclusion policy, a brief vocational assessment, rapid job search, job 
development, including dynamic engagement with employers in the community to identify and 
secure jobs, job carving (i.e. working with employers to create jobs based on veteran needs), 
integration with mental health services, benefits counseling, and follow-along support after job 
acquisition 75. Further, VACO does periodic fidelity assessments to ensure quality at the 3 SE 
programs (all have good fidelity to IPS model). VISN learning communities supporting IPS 
implementation are also available.
5.2.3 Risk and Benefit Analysis 
Potential Risks
Veterans
Potential risks for participating in the study are anticipated to be minimal. First, personal 
disclosure is always a part of CBT group interventions. Several prior studies, including our prior 
work developing and testing the CBTw intervention, have found that CBT group sessions are 
well tolerated and accepted by Veterans with SMI. In addition, this could be a potential risk for 
participants randomized to receive psychoeducation, but similarly is expected to be minimal.
Furthermore, research strongly suggests that personal disclosure is an important component of 
the therapeutic group process, allowing for helpful and invaluable feedback from other persons 
in the group. Nonetheless, participants could experience some minor anxiety or distress in 
disclosing work difficulties and experiences in the group context.
Second, the loss of confidentiality is another possible but unlikely risk. Proper privacy 
and security procedures will be followed to protect against this risk, such as the de-identification 
of data and data storage using password protected VA servers; all sites will have access to RLR 
servers behind VA firewall. Any paper data collected will be stored in locked filing drawers in a 
locked office. Moreover, to further ensure the rights of participants and minimize potential risks, 
participants will be clearly informed that they may discontinue study participation at any point in 
time. No pharmacological therapies will be utilized in this study.
Key Informant Staff Persons-Interviews
Potential risks for SE key informants participating in Aim 3 interviews examining CBTw 
implementation may include feeling uncomfortable while talking with study staff. We have 
conducted several prior studies examining the implementation of programs within the VA 
utilizing staff interview methods that have minimized these risks. Key informant staff persons 
will also have the option of ending the interview or not answering any questions as they wish.
Staff delivering CBTw Intervention
SE staff persons who are delivering the CBTw intervention and taking part in the Aim 3 
fidelity audit and feedback process may experience slight discomfort in response to evaluation 
and feedback during the CBTw fidelity assessment process. We have conducted several past
3/15/2022
VA Central IRB Protocol, Version 3 Page 18 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

studies utilizing these implementation and evaluation methods, and our established processes 
have minimized these risks.
Adequacy of Protection From Risk
Protection Against Identified Risks
Prior to the start of research, study investigators will have completed required VA R&D 
committee training (i.e., VA’s Cyber Security Awareness course and Overview of Good Clinical 
Practice and Human Subjects Protection course) as well as other required trainings by IRB. 
Research staff and all site investigators will complete all required VA R&D and IRB research 
trainings, such as Collaborative Institutional Training Initiative courses on responsible practice of 
research and protection against risk.
To protect against the potential loss of confidentiality, all electronic data will be, coded or 
de-identified to the extent possible, and stored on a secure, password-protected VA server. Any 
paper data will be kept in a locked cabinet in secure (locked) office space at each VA site until it 
can be scanned to a digital copy and the hard copy destroyed. Computerized files will be 
protected by the electronic firewall at the Roudebush VAMC and will be password protected 
(Roudebush servers behind VA firewall will be accessible to and used by study personnel at all 
sites, including Hines and St. Louis). Passwords are created according to guidelines set by the 
VA Office of Cyber and Information Security.
To protect against discomfort that may be experienced related to personal disclosure 
during CBTw and psychoeducation group sessions, participants will be explicitly informed that 
personal sharing is not mandatory, and they have the right not to disclose information and/or 
speak during groups at their discretion. This is a standard practice in psychotherapy and 
education-based groups in clinical and research contexts. Second, “group rules” will be laid out 
at the beginning of the CBTw and psychoeducation conditions during the first group session in 
which confidentiality and respect for the privacy of other Veterans will be discussed and agreed 
upon by all participants. Specifically, information discussed in the group will stay in the group 
and is not to be discussed with other persons outside of the group. Forming these “group rules” 
is also standard practice in research and clinical group therapy and psychoeducation settings.
During assessments, all participants will be instructed that they may refuse to answer 
any question that they are uncomfortable answering. In addition, participants may withdraw from 
the study at any time without any repercussions; participants will continue to receive all VA care 
and rehabilitation services (including supported employment) as usual, regardless of their study 
status.
If participants become distressed or experience a clinically significant increase in 
symptoms at any point during the study (e.g., during a CBTw or psychoeducation group session 
or assessment), facilitators will collaborate closely with Dr. Kukla, study PI,, as well as site 
investigators at St. Louis and Hines (SE managers- Ms. Hatfield and Mr. Weber) to quickly refer 
participants to their mental health clinic and mental health treatment coordinating clinician as 
appropriate. Because all Veterans will meet the inclusion criteria of SMI and functional 
impairment, they will have an assigned VA mental health treatment coordinator to arrange and 
directly provide mental health services on a same day basis. This procedure has been used 
extensively across mental health studies and studies of other vulnerable Veteran populations in 
the Roudebush VAMC and is standard across VA studies.
For participants who endorse suicidal thoughts on the Scale for Suicidal Ideation 
(administered at all assessment points—baseline, posttreatment-12 weeks, 6 month follow up, 9 
month follow up), a similar protocol will be followed in which Dr. Kukla will be immediately 
notified and the participant will be connected with their onsite mental health treatment
3/15/2022
VA Central IRB Protocol, Version 3 Page 19 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

coordinator that day. A further VA mandated suicidality screening will be conducted by Dr. 
Kukla (if the onsite mental health treatment coordinator is not immediately available to conduct 
the screening); if plan and/or intention for suicide are endorsed by the participant (indicating 
elevated risk for suicidal behavior), the onsite suicide risk prevention coordinator will be 
informed and connected with the participant immediately. Furthermore, if the participant is at 
imminent risk for self-harm, onsite staff will escort the participant to the medical center’s 
emergency department (if onsite suicide prevention coordinator or mental health treatment 
coordinator are unavailable), or connect the participant with the Veteran’s Crisis Line. We have 
effectively used these procedures in past studies at Roudebush, including studies involving 
multiple off-site clinics (See Drs. Kukla, McGuire, Lysaker, funded VA studies). All study 
personnel will be trained in this protocol.
Aim 3 qualitative interview recordings will be stored on VA approved recording devices 
and immediately uploaded to the VA server (behind VA firewall) and specifically, HSR&D CHIC 
server and the WORKWELL study folder. All study personnel at all sites will have access to the 
CHIC server and study folder.
Similarly, Aim 3 recordings of CBTw sessions for the purpose of fidelity audit will be 
stored on VA approved recording devices and immediately uploaded to the VA server (behind 
VA firewall) and specifically, HSR&D CHIC server and the WORKWELL study folder. All study 
personnel at all sites will have access to the CHIC server and study folder.
Potential Benefits of Research to Subjects and Others
The CBT for Work Success intervention to be rigorously tested in this project has 
potential widespread benefits to Veterans with serious mental illness. Specifically, potential 
benefits to participants include enhanced functioning, recovery and health, and socioeconomic 
outcomes, including increased competitive work success with potential long-term implications, 
such as longer periods worked, higher wages earned, more opportunities for career 
advancement, and a reduction in health disparities. Participants may also experience an 
increased understanding of the impact of their thoughts and feelings on work related behavior, 
greater work-related self-efficacy, more motivation, better self-esteem, and improved problem- 
solving ability and healthy coping. Overwhelmingly, these benefits outweigh any potential 
project risks.
5.3 Recruitment Methods
5.3.1 Number of Subject 
Needed Veterans
276 total participants (138 per study arm) will be needed to ensure > 80% power for this 
two-arm trial, accounting for the clustering effect of CBTw group cohort and potential attrition of 
13%21. The study sample will include around 50% minority participants, and 5% Hispanic; these 
figures are consistent with CBTw pilot studies which included 48% African American 
participants. These are also representative of the mean number of non-white Veterans served at 
RLR, Hines, and St. Louis. In FY18, 48.7% of Veterans at these facilities were non-white, 46.2% 
Caucasian, and 2.7% identified as Hispanic or Latino30. In the CBTw pilot studies, 10% women 
Veterans were recruited, consistent with gender distribution of Veterans served by the 3 study 
facilities (mean of 10.8% of Veterans identify as female in FY18)30. In WORKWELL, comparable 
numbers of women Veterans will be recruited. Data analyses will examine differences in 
outcomes based on gender and ethnicity, although these analyses may be underpowered to
3/15/2022
VA Central IRB Protocol, Version 3 Page 20 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

detect significant differences.
Providers
Key Informant Staff Interviews: At each site, it is anticipated that approximately 5 key 
informal staff persons will participate in interviews occurring onsite or via phone (as needed); we 
will enroll a total of 15 staff for interviews.
Staff Persons Delivering Study Interventions: Up to 4 Vocational Rehabilitation 
Counselors (VRCs) will be enrolled from each study site, 1-2 to deliver the CBTw intervention 
and 1 to 2 to provide the psychoeducation control intervention. We plan to enroll up to 12 
VRCs in total.
5.3.2 Identification and Recruitment of Subjects
Prior to recruitment, we will obtain Central IRB and Local VA R&D Committee 
approval at sites to conduct this proposed research
Veterans
At each site, we will consult with SE program staff and gain permission to recruit 
their patients (Veterans on their caseload who have SMI and are unemployed, those 
who they deem appropriate for the study). SE staff at each site will refer potentially 
eligible participants to the study. Recruitment may also include a review of 
administrative data to identify potentially eligible subjects. Study personnel will pull data 
from the Corporate Data Warehouse (CDW) using stop codes related to vocational 
rehab for each study site (i.e., 568, 574, and 535). Potential participants will be 
monitored for eligibility via chart review for presence of inclusion and exclusion criteria 
by study staff through VHA’s Electronic Health Record (EHR) by way of JLV/CAPRI and/
or CPRS/VistAWeb. If a Veteran is deemed potentially eligible and expresses initial 
interest, they will be given a flyer, or have the flyer read to them by a program staff 
member, about the research study and be contacted by research staff for eligibility 
screening. If the Veteran is not able to be given or read the flyer by a program staff 
member, the research team will send the flyer via mail or email before they are 
contacted by research staff. During the eligibility screening process, study staff will 
confirm the diagnosis of an SMI, enrollment in SE services, and age 18 or older through 
a CAPRI/CPRS/JLV chart review. We used these procedures to successfully confirm 
SMI diagnosis during the CBTw pilot studies and believe it is the most appropriate 
approach for the proposed pragmatic trial. Though participants receiving SE will have a 
competitive work goal, study staff will also verify with the Veteran the presence of such a 
work goal. Participants who confirm that they would like to participate in the study will be 
scheduled for a pre-randomization baseline assessment with a(blinded) research 
assistant. At the start of the assessment, verbal informed consent will be obtained prior 
to further study involvement.
If needed, given the pragmatic study design and consistent with other HSR&D 
studies in our Center for Health Information and Communication; additional recruitment 
methods will be utilized, including self-referral by Veterans receiving SE services (who 
may have indirectly heard about the study).
Providers
Staff Persons Delivering Study Interventions
3/15/2022
VA Central IRB Protocol, Version 3 Page 21 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

During initial start-up, the study team (Kukla and McGuire) will meet with SE staff 
and managers at sites, or remotely, to consult as sites determine the best approach to 
integrating CBTw in work flow, and develop procedures to support implementation (e.g., 
documentation procedures). The study team will also be available for consultation as 
sites select intervention facilitators from available and interested staff persons each SE 
program. Specifically, in consultation with SE program managers, SE staff persons who 
are vocational rehabilitation counselors (i.e. with expertise in rehabilitation counseling 
and in the provision of vocational services) will be chosen as intervention facilitators.
During the study, these staff will participate in the CBTw consultation and audit and 
feedback process. At the beginning of their participation, they will be presented with a 
study information sheet and their consent will be implied by their participation as 
intervention facilitators.
Key Informant Staff Interviews
During day-long site-visits (may be conducted remotely), Drs. Kukla and McGuire 
will interview VRCs delivering CBTw (staff intervention facilitators) as well as SE 
managers, other SE staff, and service line leadership involved in the implementation 
process at their site. The interviews will focus on understanding reasons to uptake and 
continue CBTw, and ongoing barriers and facilitators to implementation (See Appendix 
6).
Other Staff
During site visits, research personnel will observe procedures related to CBTw 
implementation. Staff present on the unit during the site visit may be identified for 
informal interactions to gather research data.
5.3.3 Recruitment Materials
Veterans: Informational Flyer
SE staff at participating sites will provide, or read, this flyer to Veterans who 
express interest in the research study groups, and will include information, such as study 
and PI contact information, that will allow Veterans to verify the authenticity of the 
research project before they are contacted by the study team by phone.
Providers: Site Visits
Ahead of site visits, the research team will provide an email template (Appendix 
11) for SE managers to send to SE staff prior to site visits. These emails will inform staff 
that research personnel will be on site to observe unit operations related to 
implementation and to conduct interviews with available staff involved in the 
implementation process.
Providers: Key Informant Interviews
Ahead of site visits, the research team will provide an email template 
(Appendix 11) for SE managers to send to SE staff prior to site visits. These emails will 
inform staff that research personnel will be on site to observe unit operations related to 
implementation and to conduct interviews with available staff involved in the 
implementation process. A study information sheet regarding the interviews will be 
included for staff to review prior to the visit to ask any questions.
3/15/2022
VA Central IRB Protocol, Version 3 Page 22 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

Staff Persons Delivering Study Interventions
The research team will provide information in person, or by phone/secure video 
conference to VRC staff persons who will be delivering study interventions and taking 
part in training, consultation, and audit and feedback processes involving monthly fidelity 
data collection. As noted, participation as intervention facilitators will be discussed with 
SE managers and potential interested staff, including all relevant implementation 
activities and expectations (facilitation of weekly groups, etc.).
5.3.4 Payments to Subjects
Veterans
Participants in both study arms will receive $20 for each completed outcome 
assessment (baseline, 12-week, 6 & 9 month only). After obtaining informed consent, 
the onsite RA will administer a baseline assessment to gather assessment data (see 
Sections D8a-c, Section E3d). Each assessment will take approximately 60 minutes. 
Participants will not be compensated for attendance in weekly group sessions (CBTw or 
psychoeducation,respectively). Participants will not be compensated for transportation 
costs.
Providers
In accordance with VA policy, VA staff will not receive payment for study 
participation.
5.4 Informed Consent Procedures
Veterans
We are requesting a waiver of informed consent for recruitment. Prior to recruitment, we 
will obtain Central IRB and Local VA R&D Committee approval at sites to conduct this proposed 
research. At each site, we will consult with SE staff and gain permission to recruit their patients 
(Veterans on their caseload who have SMI and are unemployed, those who they deem 
appropriate for the study). Next, SE staff will inform potential Veteran participants of the study. If 
those Veterans express interest, the SE staff person will provide or read the Veteran a study 
flyer.
Next, an onsite research assistant will use the electronic medical record to screen for 
inclusion criteria (SMI diagnosis and age 18 or older) of potential participants who have 
expressed interest in the study. If Veterans meet the SMI diagnosis and age inclusion criteria, 
the research assistant will then call these Veterans and screen for the other inclusion and 
exclusion criteria (confirm unemployed status, goal of competitive work, and no prior 
participation in a intervention CBT tailored for work). If all criteria are met, the RA will ask if the 
Veteran would like to participate in the study. Participants who confirm that they would like to 
participate in the study will be scheduled for a pre-randomization baseline assessment with the 
onsite (blinded) research assistant. At the start of the assessment, verbal informed consent will 
be obtained prior to further study involvement.
RA’s, blinded to which group the participant is randomized to, will obtain verbal informed 
consent. All study group sessions will be audio recorded; all audio recordings will explicitly
3/15/2022
VA Central IRB Protocol, Version 3 Page 23 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

capture the participants consent to be audio recorded. Audio recordings will be transcribed by 
research study personnel and deidentified. All individuals obtaining informed consent (RA’s at 
local sites) and LSI’s will be trained by the PI, Marina Kukla, on appropriate informed consent 
procedures. Travel funds have been designated for Dr. Kukla and other RLR personnel to travel 
to the local sites in order to provide training in informed consent procedures, as well as other 
study procedures. As deemed necessary, these trainings will be conducted virtually to minimize 
non-essential in-person interaction. At regularly scheduled team conference calls, the PI or 
Project Manager will periodically review informed consent procedures and address any 
questions or concerns. Moreover, all individuals obtaining informed consent and LSI’s will have 
completed CITI Human Subjects Trainings for VA Human Subjects Research.
Providers
For all staff participating in the study, we will obtain any necessary approvals for their 
participation from service unions.
Staff Persons Delivering Study Interventions
As described above, in consultation with SE managers at each site, vocational 
rehabilitation counselors with interest in CBTw and psychoeducation will be chosen as 
intervention facilitators (up to 4 staff persons per study condition at each site). Separate staff 
persons will facilitate CBTw and psychoeducation interventions at each site. During the initial 
site visit, research personnel will discuss the role of intervention facilitators. Once consent is 
provided by staff, the initial training and intervention implementation will begin. All study group 
sessions will be audio recorded; all audio recordings will explicitly capture the participants 
consent to be audio recorded. Audio recordings will be transcribed by research study personnel 
and be deidentified.
Key Informant Interviews
Key informants who will take part in interviews (occurring at pre-implementation, early 
implementation, mid implementation, and late implementation) include staff delivering CBTw 
intervention, SE program managers, and other SE staff persons involved in CBTw 
implementation. Ahead of site visits, study personnel will email study information sheets (using 
VA encrypted email). The SIS will be further reviewed onsite (for onsite interviews) or over the 
phone (if interviews occur over the phone) with potential participants. Consent will be implied by 
participant/staff’s participation in the interview. All interview will be audio recorded; all audio 
recordings will explicitly capture the participants consent to be audio recorded. Audio recordings 
will be transcribed by research study personnel and be deidentified.
5.5 Inclusion/Exclusion Criteria
5.5.1 Veterans
Inclusion criteria
1) Unemployment, defined as no current participation in a competitive job; 2) a competitive 
work goal; 3) Presence  of a serious  mental illness  (required  to receive SE  services); 4) 
Receiving SE services; 5) Age 18 or older.
Exclusion criteria
3/15/2022
VA Central IRB Protocol, Version 3 Page 24 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

1) Previous participation in CBT geared toward work; 2) Presence of a severe medical or 
cognitive impairment that will prevent participation in the study.
Details on Veteran inclusion criteria
Employment
 
  Status:
 
  Unemployment is defined as a lack of a competitive job at baseline.
Veterans meeting inclusion criteria who hold noncompetitive jobs (e.g., transitional work 
experiences) are not considered competitively employed and will be eligible to participate in the 
study.
Competitive work denotes a job that it is open and available to everyone, not reserved 
exclusively for persons with disabilities; competitive jobs pay minimum wage or above. Casual 
labor, such as odd jobs, does not constitute competitive work21. Our prior research suggests that 
unemployed Veterans in SE often have competitive work goals that include finding a good job, 
maintaining a job over time, and increasing work hours and wages.
Enrollment
 
  in   SE
   services:
 
  Participants must be formally enrolled in SE services at the 
sites described in Section D2, “Study Sites”. Information regarding nature of these services is 
described in Section D5.
Diagnosis of serious mental illness & work impairment: To be eligible for the study, 
participants must have a lifetime diagnosis of a SMI including: affective disorders including 
major depression and bipolar disorders; schizophrenia spectrum disorders including 
schizophrenia and schizoaffective disorder; and PTSD. In accordance with the standard 
definition of SMI2, no other diagnoses will be included. As noted, all participants will have a 
documented work impairment & SMI diagnosis in order to qualify for SE services at the three 
sites. SMI diagnosis will be further verified during the eligibility screening process.
Including Veterans with SMI
The inclusion of Veterans with SMI was chosen for 3 reasons: 1). These represent the 
average Veterans served nationally in VHA SE services; restricting to a narrow diagnostic group 
would not produce a generalizable understanding of the outcomes of CBTw in regular care72. 2) 
CBTw was designed for Veterans with SMI across the diagnoses that are included in 
WORKWELL. CBTw pilot findings demonstrate that these diagnostic groups did not significantly 
differ on outcomes at the follow-ups. Further, the developmental CBTw pilot found that the 
inclusion of Veterans with various SMI diagnoses was not a barrier; participants reported that 
the experience of vocational dysfunction was unifying56. 3). Maladaptive cognitive and 
behavioral factors that influence work functioning are consistent across SMIs65 and are 
effectively addressed using CBT73.
5.5.2 Staff
Inclusion criteria:
1) Key informants participating in interviews may be supported employment staff persons 
(frontline staff persons, SE manger, administrative staff, etc.) at a participating VA site. Staff 
persons delivering the interventions will be vocational rehabilitation counselors.
Exclusion criteria:
3/15/2022
VA Central IRB Protocol, Version 3 Page 25 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

None
5.6 Study Evaluations
Data Collection Protocol for Study Aims 1 & 2
Conducted by onsite research assistants (RAs) blinded to arm assignment, outcomes 
assessments will take place at four time points: baseline, 12-weeks (posttreatment), 6 months, 
and 9 months. In addition, work outcomes will be collected monthly on Work Update Forms to 
lessen the burden of Veteran recall. This schedule will elucidate the trajectory of treatment 
effects over time, including early response and more sustained impacts on outcomes.
Participants will receive $20 for each completed outcome assessment (baseline, 12-week, 6 & 9 
month only). After obtaining informed consent, the onsite RA will administer a baseline 
assessment to gather assessment data. Each assessment will take approximately 60 minutes. 
All baseline assessments will take place in person; 12 week, 6 month, and 9 month 
assessments may be done in person (first consideration) or by phone.
The primary work outcome, total weeks worked in competitive jobs, will be collected 
monthly and analyzed at all assessment points with the primary endpoint being 6 months to 
allow adequate time for CBT skills to be practiced and solidified. The 12 month endpoint will 
examine whether there are sustained effects of CBTw and determine whether there are 
differences between the arms on achievement of steady competitive work (Aim 1). Health and 
recovery outcomes will be examined between the arms at all assessment points to understand 
the impact of CBTw; the relationship between weeks worked and these outcomes will also be 
examined at all assessment points (Aim 2). Work outcomes may be collected in person or by 
phone.
Aim 1 Competitive Work Outcome Measures: Collected Monthly
The competitive work outcomes are gold standard in VA work studies80. In Aim 1, the 
primary outcome is total weeks worked in competitive jobs. The secondary work outcome is 
steady
 
  competitive
 
  work
 
  status
 
 , defined as working at least 50% of the study period81 (at least
19.5 weeks out of 38 weeks). Also standard procedure in work studies and used in CBTw pilots, 
work outcomes will be collected monthly by onsite RAs via Veteran self-report monthly work 
update form (Appendix 8), and verified with CAPRI/CPRS/JLV SE notes (i.e. SE staff notes will 
be reviewed for data regarding hours and weeks worked) and directly with SE staff using a data 
collection form
(Appendix 9), as needed. Additional verification layers will be used under any of the following 
conditions: monthly work update forms are not fully completed; conflicting or unclear information 
is reported (e.g., Veteran reported only working two days during a month when s/he is 
employed full time); Veterans report having difficulty recalling their work during the past month. 
Work outcomes may be collected in person or by phone.
Aim 2 Health and Recovery Outcomes: Collected at baseline, posttreatment, 6 & 9 months
Used in the CBTw pilot studies and across the SMI literature, subjective recovery 
pertinent to work will be measured using the Recovery Assessment Scale (RAS), a 41 item, 
self-report measure with higher scores indicating stronger recovery attitudes (scored on a 1 to 4 
Likert scale); the RAS was developed and validated in people with SMI82.
3/15/2022
VA Central IRB Protocol, Version 3 Page 26 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

Psychiatric symptoms will be assessed using the adult version DSM-5 Level 1 Cross- 
Cutting Symptom Measure (CCSM), a 23-item, Likert response (scored on a 0 to 4 scale), self- 
rated instrument that assesses 13 domains that are important across SMI diagnoses (e.g., 
depression, anxiety, psychosis, mania, sleep problems, self-harm, substance use, etc.) with 
higher scores indicating more severe total symptoms. The CCSM has been found to have 
strong validity and reliability in SMI samples83.
Health-related quality of life will be examined using the SF-12 comprised of the Physical 
Component Summary (PCS) and Mental Component Summary (MCS)84. The PCS and MCS are 
standardized with a mean of 50 and a standard deviation of 10, with higher scores indicating 
better health-related quality of life. The subscales have good reliability and are predictive of 
health outcomes in Veterans with chronic illness.
Suicidality will be assessed with the 21-item Scale for Suicidal Ideation (SSI)85, an 
interviewer rated measure of a continuum of suicidal ideation, including thoughts, attitudes, 
plans, and behaviors. SSI items are scored on a 0 to 2 Likert scale, with higher scores 
indicating greater intensity of suicidal ideation. The SSI has strong psychometric properties in 
outpatients with psychiatric conditions85.
Collected from a CAPRI/CPRS/JLV data pull, service
 
  utilization
 
  data will include days of 
psychiatric hospitalizations Lead by Mr. Larson, this service utilization data will be extracted 
from the Corporate Data Warehouse.
Other measures and covariates
Sociodemographic and background questionnaire (Appendix 10): Used broadly 
in previous work studies, a standard background and demographic questionnaire will 
probe gender, age, ethnicity, education, medical conditions, substance use83, military 
history, residential status, disability entitlements, and legal history. Given its association 
with future work outcomes found in past studies32, work history will be assessed via self- 
report, CAPRI/VCPRS/JLV chart review, and SE staff report for verification; these 
variables will include longest past competitive job, weeks worked and wages earned in 
the 6 and 9 months preceding baseline. Collected via CAPRI/CPRS/JLV chart review, 
time spent in SE services, operationally defined as the number of days receiving SE 
services, will be examined as a possible covariate. Session attendance for CBTw and 
psychoeducation will also be tracked.
Staff Participation: CBTw Implementation (Aim 3)
In order to ensure successful implementation of CBTw during WORKWELL and wide-scale 
uptake following its completion, an evaluation of this initial implementation strategy and barriers 
and facilitators of implementation will be conducted, guided by the RE-AIM framework.
Implementation Activities
Training (Months 1-3): Kukla, McGuire, and the project manager will conduct onsite or 
virtual trainings. VRCs delivering CBTw will attend a half-day CBTw workshop consisting of: 1) 
Education on CBTw content and application of cognitive, behavioral, and narrative strategies; 2) 
Mock practice of facilitation strategies; 3) Customization of CBTw content to meet unique needs 
of Veterans. Second, led by Dr. McGuire, psychoeducation facilitator training will follow a 
parallel structure. Third, other SE staff will be oriented to study procedures. Fourth, SE 
managers overseeing VRCs will receive training on how to provide supervision to ensure CBTw 
implementation.
3/15/2022
VA Central IRB Protocol, Version 3 Page 27 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

Fidelity monitoring & feedback (~Monthly): Lead by Dr. Rollins, adherence to the CBT 
model will be assessed by the Cognitive Therapy Scale-Revised (CTS-R)87. All CBTw and 
psychoeducation sessions will be audio-recorded to rate fidelity. At each site, 3 sessions from 
each group cohort (i.e. 1 session out of every 4) will be randomly selected for rating (in the 
CBTw and psychoeducation arms). On a monthly basis, consultation lead by Kukla and Rollins, 
via tele or video, will include review of fidelity scores and recommendations to improve as 
needed. Psychoeducation facilitators will also be given fidelity feedback if they begin to drift 
toward CBT elements.
CBTw
 
  consultation
 
  (weekly)
 
  will be led by Kukla and the project manager via tele or 
video with VRC CBTw facilitators and SE program managers. Consultation will feature the 
CBTw Facilitator Guide (Appendix 3) that was piloted in earlier work57. In addition, strength 
based consultation will focus on Veteran case conceptualization, with an emphasis on 
understanding participants’ recovery paths, applying CBTw strategies to fit goals and 
employment trajectories as they evolve over time, and problem solving obstacles.
Implementation Interviews & Site visits:
RE-AIM domains of implementation, adoption, and maintenance will be examined 
through semi-structured key stakeholders’ interviews, lasting approximately 1 hour each. The 
interviews will also focus on understanding reasons to uptake and continue CBTw and 
important issues related to inner context and outer context (See Appendix 6). Interviews will 
also centrally focus on identifying barriers and facilitators to CBTw implementation within SE 
programs.
Interviews will be conducted in-person during periodic site-visits, or via phone/video, by Drs 
Kukla & McGuire. Site visits will take place at pre-implementation (1 month), early 
implementation (~Month 6), mid-implementation (first quarter-Year 2), and later implementation 
(first quarter-Year 3) will be conducted. During day-long visits, Drs. Kukla and McGuire will 
interview VRCs delivering CBTw, SE staff and managers, and make observations of CBTw 
processes. The implementation support team will keep detailed field notes recording activities 
conducted, barriers and facilitators encountered, and strategies utilized to address these. Site 
visitors will produce a written report that will provide a narrative summary of implementation 
strengths and weaknesses.
5.6 Data Analysis 
Sample size and power
276
 
  total
 
  participants
 
  (138
 
  per
 
  study
 
  arm)
 
  will be needed to ensure > 80% power for this two- 
arm trial, accounting for the clustering effect of CBTw group cohort and potential attrition of 
13%21.
Analyses:
Study analyses will be conducted by study personnel associated with the Richard L. Roudebush 
VAMC, as noted in section 4.0, Resources and Personnel.
Preliminary Analyses:
Given the RCT design, characteristics of participants are expected to be balanced 
across the two study arms. Baseline characteristics and baseline clinical outcomes (e.g., 
symptoms) will be compared between the arms using the appropriate test (i.e., t-test, or Chi- 
square test). Variables found to significantly differ will be included as covariates in the primary
3/15/2022
VA Central IRB Protocol, Version 3 Page 28 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

analyses. All primary analyses will also include time receiving SE services and site membership 
as covariates. The potential interaction effects of site with key predictors will also be included as 
predictors to account for the site effect. Lastly, attendance (the number of CBTw or 
psychoeducation sessions attended) and the association with the Aim 1 and Aim 2 outcomes 
will be examined.
Competitive Work Outcomes (Hypothesis 1)
For the outcome, total weeks worked in competitive jobs, three-level linear regression 
models will be used to test the effects of CBTw + SE compared to psychoeducation + SE over 
time, with predictors of study arm, arm by time interaction, site, and interaction between site and 
key predictors, covariates, as well as random effects to accommodate the potential non- 
independence due to repeated measures and group cohort effects87. The time effects could be 
linear or nonlinear; these possibilities will be examined by treating time as continuous or 
categorical. If the outcomes are not normally distributed, robust maximum likelihood estimation 
method will be used to correct the standard error estimates for nonmorality. In the case of 
excess of zeros for this outcome, hurdle negative binomial models, a reliable method to analyze 
zero inflated longitudinal data of this nature, will be utilized. For the binary outcome, steady 
competitive work status across the 12-month study period, two-level logistic regression model 
will be used with the same set of predictors as in the linear models, with random effects to 
accommodate the potential non-independence due to group cohort effects.
Health and Recovery Outcomes (Hypotheses 2a & 2b)
To test Hypothesis 2a, three-level linear regression models (i.e. mixed models) 
accounting for the nested nature of the data will be used to test the effects of CBTw + SE 
services compared to psychoeducation + SE on each of the continuous outcomes including 
recovery (RAS), health-related quality of life (PCS & MCS), symptoms (CCSM), and suicidality 
(SSI) over time. For the count outcome of service utilization, three level Poisson regressions 
will be used to examine group differences over time on number of days of inpatient 
hospitalization . If there is a problem of excess of zeros for utilization (e.g., no hospitalizations), 
hurdle negative binomial models will be used.
To test Hypothesis 2b, three-level linear regression models will examine the effects of 
work on health and recovery outcomes over time, with predictors of competitive weeks worked, 
time, weeks by time interaction, site membership and potential interaction between site and key 
predictors, potential covariates (i.e. controlling for baseline levels of health and recovery), as 
well as random effects to accommodate the potential non-independence due to repeated 
measures and group cohort effects.
Missing data: Aims 1 & 2
Missing data from Aims 1 and 2 will come in two different forms: missing by attrition and 
intermittent missing of observation. Using logistic regression, the effects of missing 
observations due to sample attrition will be examined by investigating the participant 
characteristics associated with dropout. If results reveal that data are missing completely at 
random (MCAR) or due to observed factors (MAR), the data will be analyzed using full 
information maximum likelihood, taking into account potential predictors for missingness. 
However, if the pattern of missing data is nonignorable; pattern-mixture models that includes 
variables defined by the participant’s pattern of missing data will be used89. Sensitivity analyses 
also will be performed to examine how the result will change based on different assumptions 
about the missing data, such as the missing work period having no work, the mean amount of 
work as available data from earlier follow-up periods, or full employment.
3/15/2022
VA Central IRB Protocol, Version 3 Page 29 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

Analyses & Integration of Mixed Methods Data (Aim 3-Implementation)
A deductive-inductive rapid qualitative analysis process will be used to facilitate the 
timely use of qualitative data to refine the implementation strategy89, an approach appropriate 
for the description of implementation processes and environments90. Templated summaries of 
each data collection episode will be created, organizing quotes or observations by pre- 
established codes (the deductive component) pertaining to RE-AIM domains. Emergent themes 
in the data (the inductive component) will be captured through use of a grounded theory 
approach, allowing for discovery of themes not included in a priori codes91. Field notes will be 
written after site visit observations and staff interviews will bolster data not captured in 
immediate data summaries. An iterative process will occur, in which analysis will begin at the 
first observation and will inform direction of future data collection91. The team will then create a 
matrix summarizing data by code and collection episode. This will facilitate examination of 
variation across respondents, identify data gaps, and develop summaries of data mapping onto 
RE-AIM domains, as well as barriers and facilitators encountered92.
5.7 Withdrawal of Subjects
To further ensure the rights of participants and minimize potential risks, participants will 
be clearly informed that they may withdraw from the study at any time without any 
repercussions; participants will continue to receive all VA care and rehabilitation services 
(including supported employment) as usual, regardless of their study status. Withdrawn 
participants will not be able to continue in audio recorded groups conducted for the purposes of 
this study nor complete outstanding assessments. If requested upon withdrawal, the 
participants’ data will be removed from the study to the extent possible. If a participant wishes to 
withdraw, they will be directed to contact the research assistant at their site to provide a reason 
for withdrawal.
If during the course of the study the participant formally dis-enrolls in supported 
employment services, their participation in the study will be terminated. We do not 
anticipate these circumstances occurring often, as they have not occurred in any of our 
previous studies of CBTw.
6.0
Reporting
This study will have a Data Monitoring Committee. The details of this Committee and 
procedures for reporting are described in our Data Analysis Plan (Appendix 7). The 
appropriate local ISO(s) and PO(s) will be notified within one hour of the PI determining 
improper use or disclosure.
7.0
Privacy and Confidentiality
This study will collect and use, but not disclose, subjects’ Protected Health Information 
(PHI). Data will not be banked. Biological specimens are not utilized in this study.
Prior to the start of research, study investigators will have completed required VA R&D 
committee training (i.e., VA’s Cyber Security Awareness course and Overview of Good Clinical 
Practice and Human Subjects Protection course) as well as other required trainings by IRB. 
Research staff and all site investigators will complete all required VA R&D and IRB research 
trainings, such as Collaborative Institutional Training Initiative courses on responsible practice of 
research and protection against risk. If a study member is no longer a part of the research team,
3/15/2022
VA Central IRB Protocol, Version 3 Page 30 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

their access to research study data will be removed.
To protect against the potential loss of confidentiality, all electronic data will be coded, or 
de-identified to the extent possible, and stored on a secure, password-protected VA server. A 
“key” file will connect participant names and contact information with their study ID number; this 
file will be kept on a secured server behind the VA firewall and only be accessible to study 
personnel. Any paper data collected will be kept in a locked cabinet in secure (locked) office 
space at each VA site, until it can be scanned to an electronic file and the hard copy destroyed. 
Computerized files will be protected by the electronic firewall at the Roudebush VAMC and will 
be password protected (Roudebush servers behind VA firewall will be accessible to and used by 
study personnel at all sites, including Hines and St. Louis). Passwords are created according to 
guidelines set by the VA Office of Cyber and Information Security. All research records will be 
retained in accordance with the VHA Records Control Schedule. Study records will be 
destroyed in accordance with VHA Records Control Schedule as written at the time of study 
closure and in accordance with any revision to VA RCS after that time.
To protect against discomfort that may be experienced related to personal disclosure 
during CBTw and psychoeducation group sessions, participants will be explicitly informed that 
personal sharing is not mandatory and they have the right not to disclose information and/or 
speak during groups at their discretion. This is a standard practice in psychotherapy and 
education-based groups in clinical and research contexts. Second, “group rules” will be laid out 
at the beginning of the CBTw and psychoeducation conditions during the first group session in 
which confidentiality and respect for the privacy of other veterans will be discussed and agreed 
upon by all participants. Specifically, information discussed in the group will stay in the group 
and is not to be discussed with other persons outside of the group. Forming these “group rules” 
is also standard practice in research and clinical group therapy and psychoeducation settings.
During assessments, all participants will be instructed that they may refuse to 
answer any question that they are uncomfortable answering. In addition, participants 
may withdraw from the study at any time without any repercussions; participants will 
continue to receive all VA care and rehabilitation services (including supported 
employment) as usual, regardless of their study status.
During key informant staff interviews (Aim 3), all staff participants will be 
instructed that they may refuse to answer any question they are uncomfortable 
answering and they may end the interview at any time.
8.0
Communication Plan
This project has multiple means of ensuring all study sites remain informed of 
current study events and changes. The study will utilize one project manager to oversee 
research assistants, other study personnel, and daily study procedures at all sites. Study 
team meetings will occur on a monthly basis to keep study sites informed of changes to 
the protocol, informed consent, the occurrence of Serious Adverse Events, 
Unanticipated Problems, and interim results. The project manager will lead weekly site 
calls via telephone will occur to discuss progress, plan, and troubleshoot with local site 
investigators and research staff at each site. Weekly consultation calls lead by Kukla 
(PI) will take place with CBTw intervention facilitators at each site. Moreover, the PI will 
make periodic visits to each site for training and data collection. These forms of contact 
will be used to ensure the study is conducted according to the IRB-approved protocol.
3/15/2022
VA Central IRB Protocol, Version 3 Page 31 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

9.0
References
1.
Trivedi RB, Post EP, Sun H, et al. Prevalence, comorbidity, and prognosis of mental 
health among US veterans. American journal of public health. 2015;105(12):2564-2569.
2.
Definitions and terms relating to co-occurring disorders. In: (SAMHSA) SAaMHSA, ed. 
Rockville, MD2006.
3.
USDL-18-0453 Employment Situation of Veterans Summary [press release]. United 
States Department of Labor, 03/22/2018 2018.
4.
Zivin K, Bohnert AS, Mezuk B, et al. Employment status of patients in the VA health 
system: implications for mental health services. Psychiatric Services. 2011;62(1):35-38.
5.
Hamilton AB, Williams L, Washington DL. Military and mental health correlates of 
unemployment in a national sample of women Veterans. Medical care. 2015;53:S32- 
S38.
6.
Strong J, Ray K, Findley PA, Torres R, Pickett L, Byrne RJ. Psychosocial concerns of 
veterans of operation enduring freedom/operation iraqi freedom. Health & social 
work. 2014:hlu002.
7.
Savoca E, Rosenheck R. The civilian labor market experiences of Vietnam‐era 
veterans: the influence of psychiatric disorders. The journal of mental health policy and
economics. 2000;3(4):199-207.
8.
Frounfelker R, Teachout A, Bond GR, Drake RE. Criminal justice involvement of 
individuals with severe mental illness and supported employment outcomes. 
Community Ment Health J. 2011;47(6):737-741.
9.
Abraham KM, Yosef M, Resnick SG, Zivin K. Competitive Employment Outcomes 
Among Veterans in VHA Therapeutic and Supported Employment Services Programs. 
Psychiatric Services. 2017:appi. ps. 201600412.
10.
Danziger S, Frank RG, Meara E. Mental illness, work, and income support programs.
American Journal of Psychiatry. 2009;166(4):398-404.
11.
Elbogen EB, Johnson SC, Wagner HR, Newton VM, Beckham JC. Financial well-being 
and postdeployment adjustment among Iraq and Afghanistan war veterans. Military 
medicine. 2012;177(6):669-675.
12.
Abraham KM, Ganoczy D, Yosef M, Resnick SG, Zivin K. Receipt of employment 
services among Veterans Health Administration users with psychiatric diagnoses. 
Journal of rehabilitation research and development. 2014;51(3):401.
13.
Benach J, Vives A, Amable M, Vanroelen C, Tarafa G, Muntaner C. Precarious 
employment: understanding an emerging social determinant of health. Annual review 
of public health. 2014;35.
14.
Paul KI, Moser K. Unemployment impairs mental health: Meta-analyses. Journal 
of Vocational behavior. 2009;74(3):264-282.
3/15/2022
VA Central IRB Protocol, Version 3 Page 32 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

15.
Kilbourne AM, Cornelius JR, Han X, et al. General-medical conditions in older patients 
with serious mental illness. The American Journal of Geriatric Psychiatry. 
2005;13(3):250-254.
16.
Twamley EW, Narvaez JM, Becker DR, Bartels SJ, Jeste DV. Supported employment 
for middle-aged and older people with schizophrenia. American journal of psychiatric 
rehabilitation. 2008;11(1):76-89.
17.
Kukla M, Bond GR, Xie H. A prospective investigation of work and nonvocational 
outcomes in adults with severe mental illness. The Journal of nervous and 
mental disease. 2012;200(3):214-222.
18.
Luciano A, Bond GR, Drake RE. Does employment alter the course and outcome of 
schizophrenia and other severe mental illnesses? A systematic review of longitudinal 
research. Schizophrenia research. 2014;159(2):312-321.
19.
Luciano A, Metcalfe JD, Bond GR, et al. Hospitalization Risk Before and After 
Employment Among Adults With Schizophrenia, Bipolar Disorder, or Major 
Depression. Psychiatric Services. 2016:appi. ps. 201500343.
20.
Van Rijn RM, Carlier BE, Schuring M, Burdorf A. Work as treatment? The effectiveness 
of re-employment programmes for unemployed persons with severe mental health 
problems on health and quality of life: a systematic review and meta-analysis. Occup 
Environ Med. 2016;73(4):275-279.
21.
Davis LL, Kyriakides TC, Suris A, et al. Veterans Individual Placement and Support 
Towards Advancing Recovery: Methods and Baseline Clinical Characteristics of a 
Multisite Study. 2017.
22.
Bush PW, Drake RE, Xie H, McHugo GJ, Haslett WR. The long-term impact of 
employment on mental health service use and costs for persons with severe mental 
illness. Psychiatric Services. 2015.
23.
Drake RE, Xie H, Bond GR, McHugo GJ, Caton CL. Early psychosis and employment.
Schizophrenia research. 2013;146(1):111-117.
24.
Alvarez-Jimenez M, Gleeson J, Henry L, et al. Road to full recovery: longitudinal 
relationship between symptomatic remission and psychosocial recovery in first-episode 
psychosis over 7.5 years. Psychological medicine. 2012;42(03):595-606.
25.
Milner A, Page A, Lamontagne AD. Cause and effect in studies on unemployment, 
mental health and suicide: a meta-analytic and conceptual review. Psychological 
medicine. 2014;44(5):909-917.
26.
Pietrzak RH, Goldstein MB, Malley JC, Rivers AJ, Johnson DC, Southwick SM. Risk and 
protective factors associated with suicidal ideation in veterans of Operations Enduring 
Freedom and Iraqi Freedom. Journal of affective disorders. 2010;123(1):102-107.
27.
May CN, Overholser JC, Ridley J, Raymond D. Passive suicidal ideation: A 
clinically relevant risk factor for suicide in treatment-seeking veterans. Illness, Crisis 
& Loss. 2015;23(3):261-277.
28.
Bond GR, Drake RE. Making the case for IPS supported employment. 
Administration and policy in mental health and mental health services research. 
2014;41(1):69-73.
29.
Mueser KT, Drake RE, Bond GR. Recent advances in supported employment for people 
with serious mental illness. Current Opinion in Psychiatry. 2016;29(3):196-201.
3/15/2022
VA Central IRB Protocol, Version 3 Page 33 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

30.
The Department of Veterans Affairs Compensated Work Therapy FY15 Interim 
Report: October 1, 2014-July 14, 2015. Northeast Program Evaluation Center;2015.
31.
Bond GR, Kukla M. Is job tenure brief in individual placement and support (IPS) 
employment programs? Psychiatric Services. 2011;62(8):950-953.
32.
Campbell K, Bond GR, Drake RE, McHugo GJ, Xie H. Client predictors of employment 
outcomes in high-fidelity supported employment: a regression analysis. The Journal of 
nervous and mental disease. 2010;198(8):556-563.
33.
Kukla M, McGuire AB, Salyers MP. Barriers and Facilitators Related to Work Success 
for Veterans in Supported Employment: A Nationwide Provider Survey. Psychiatric 
Services. 2016;67:412-417.
34.
Zivin K, Yosef M, Levine DS, et al. Employment status, employment functioning, and 
barriers to employment among VA primary care patients. Journal of affective 
disorders. 2016;193:194-202.
35.
Corbière M, Lanctôt N, Sanquirgo N, Lecomte T. Evaluation of self-esteem as a worker 
for people with severe mental disorders. Journal of Vocational Rehabilitation. 
2009;30(2):87-98.
36.
Waynor WR, Pratt CW. Barriers to vocational effectiveness in ACT: Staff perspectives.
Journal of the American Psychiatric Nurses Association. 2011;17(1):72-79.
37.
Kukla M, Bonfils KA, Salyers MP. Factors impacting work success in Veterans 
with mental health disorders: A Veteran-focused mixed methods pilot study. 
Journal of Vocational Rehabilitation. 2015;43(1):51-66.
38.
Biegel DE, Stevenson LD, Beimers D, Ronis RJ, Boyle P. Predictors of competitive 
employment among consumers with co-occurring mental and substance use disorders. 
Research on Social Work Practice. 2009.
39.
Williams AE, Fossey E, Corbière M, Paluch T, Harvey C. Work participation for 
people with severe mental illnesses: An integrative review of factors impacting job 
tenure. Australian Occupational Therapy Journal. 2016.
40.
Drake RE, Bond GR. IPS supported employment: A 20-year update. American 
Journal of Psychiatric Rehabilitation. 2011;14:155-164.
41.
Kreyenbuhl J, Buchanan RW, Dickerson FB, Dixon LB. The Schizophrenia Patient 
Outcomes Research Team (PORT): updated treatment recommendations 2009. 
Schizophr Bull. 2010;36(1):94-103.
42.
Hofmann SG, Asnaani A, Vonk IJ, Sawyer AT, Fang A. The efficacy of cognitive 
behavioral therapy: a review of meta-analyses. Cognitive therapy and research. 
2012;36(5):427-440.
43.
Lysaker PH, Davis LW, Bryson GJ, Bell MD. Effects of cognitive behavioral therapy on 
work outcomes in vocational rehabilitation for participants with schizophrenia spectrum 
disorders. Schizophr Res. 2009;107(2-3):186-191.
44.
Granholm E, Holden J, Link PC, McQuaid JR. Randomized clinical trial of 
cognitive behavioral social skills training for schizophrenia: Improvement in 
functioning and experiential negative symptoms. Journal of consulting and clinical 
psychology. 2014;82(6):1173.
45.
Galovski TE, Monson C, Bruce SE, Resick PA. Does cognitive–behavioral therapy for 
PTSD improve perceived health and sleep impairment? Journal of Traumatic Stress:
3/15/2022
VA Central IRB Protocol, Version 3 Page 34 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

Official Publication of The International Society for Traumatic Stress Studies.
2009;22(3):197-204.
46.
Hofmann SG, Wu JQ, Boettcher H. Effect of cognitive-behavioral therapy for anxiety 
disorders on quality of life: a meta-analysis. Journal of Consulting and Clinical 
Psychology. 2014;82(3):375.
47.
Stewart MO, Karlin BE, Murphy JL, et al. National dissemination of cognitive-
behavioral therapy for chronic pain in veterans. The Clinical journal of pain. 
2015;31(8):722-729.
48.
Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-
implementation hybrid designs: combining elements of clinical effectiveness and 
implementation research to enhance public health impact. Medical care. 
2012;50(3):217.
49.
Glasgow RE, Harden SM, Gaglio B, et al. RE-AIM Planning and Evaluation 
Framework: Adapting to New Science and Practice with a Twenty-Year Review. 
Frontiers in Public Health. 2019;7:64.
50.
Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for 
specifying and reporting. Implementation Science. 2013;8(1):139.
51.
Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation 
strategies: results from the Expert Recommendations for Implementing Change (ERIC) 
project. Implementation Science. 2015;10(1):21.
52.
Matthias MS, Daggy J, Adams J, et al. Evaluation of a peer coach-led intervention to 
improve pain symptoms (ECLIPSE): Rationale, study design, methods, and sample 
characteristics. Contemporary Clinical Trials. 2019.
53.
Kukla M, Rattray NA, Salyers MP. A mixed methods study examining work 
reintegration experiences from the perspectives of Veterans with mental health 
disorders. Journal of Rehabilitation Research and Development. 2015;52(4):477-490.
54.
Kukla M, McGuire AB, Salyers MP. Rural and urban supported employment programs in 
the Veterans Health Administration: Comparison of barriers and facilitators to vocational 
achievement for veterans experiencing mental illnesses. Psychiatric rehabilitation 
journal. 2016;39(2):129.
55.
Kukla M, Strasburger AM, Lysaker PH. A cognitive behavioral therapy intervention 
targeting competitive work outcomes for persons with mental illness. Psychiatr 
Serv. 2016;67(6):697.
56.
Kukla M, Strasburger AM, Salyers MP, Rattray NA, Lysaker PH. Subjective experiences 
of the benefits and key elements of a cognitive behavioral therapy focused on 
community work outcomes in persons with mental illness. Journal of Nervous and 
Mental Disease. 2017;205(1):66-73.
57.
Kukla M, Strasburger AM, Salyers MP, Rollins AL, Lysaker PH. A Pilot Test of Group 
Based Cognitive Behavioral Therapy to Augment Vocational Services for Persons With 
Serious Mental Illness: Feasibility and Competitive Work Outcomes. The Journal of 
nervous and mental disease. 2018;206(5):310-315.
58.
Kukla M, Salyers MP, Strasburger AM, Johnson-Kwochka A, Amador E, Lysaker PH. 
Work-focused cognitive behavioral therapy to complement vocational services for people 
with mental illness: Pilot study outcomes across a 6-month posttreatment follow-up. 
Psychiatric Rehabilitation Journal. 2019.
 
VA Central IRB
Effective Date:
July 11, 2022

3/15/2022
VA Central IRB Protocol, Version 3 Page 35 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

59.
Kukla M, McGuire AB, Strasburger AM, Belanger E, Bakken SK. Helping veterans 
achieve work: A Veterans Health Administration nationwide survey examining effective 
job development practices in the community. Psychiatric rehabilitation journal. 
2018;41(2):103.
60.
Bambra C, Gibson M, Sowden A, Wright K, Whitehead M, Petticrew M. Tackling the 
wider social determinants of health and health inequalities: evidence from systematic 
reviews. Journal of Epidemiology & Community Health. 2010;64(4):284-291.
61.
Rudd MD, Bryan CJ, Wertenberger EG, et al. Brief cognitive-behavioral therapy effects 
on post-treatment suicide attempts in a military sample: results of a randomized clinical 
trial with 2-year follow-up. American Journal of Psychiatry. 2015;172(5):441-449.
62.
Davis LL, Kyriakides TC, Suris AM, et al. Effect of evidence-based supported 
employment vs transitional work on achieving steady work among veterans with 
posttraumatic stress disorder: a randomized clinical trial. JAMA psychiatry. 
2018;75(4):316-324.
63.
Davidson L, Strauss JS. Sense of self in recovery from severe mental illness. 
British Journal of medical psychology. 1992;65(2):131-145.
64.
Beck AT. Depression: Clinical, experimental, and theoretical aspects. University of Penn. 
Press; 1967.
65.
Beck AT, Clark DA. An information processing model of anxiety: automatic and strategic 
processes. Behav Res Ther. 1997;35(1):49-58.
66.
Beck AT, Rector NA, Stolar N, Grant P. Schizophrenia: Cognitive theory, research, 
and therapy. Guilford Press; 2011.
67.
Skinner BF. Science and human behavior. Simon and Schuster; 1953.
68.
Roberts GA. Narrative and severe mental illness: what place do stories have in an 
evidence-based world? Advances in Psychiatric Treatment. 2000;6(6):432-441.
69.
Gallagher S. Self-narrative in schizophrenia. The self in neuroscience and psychiatry.
2003:336-357.
70.
Lysaker PH, Ringer J, Maxwell C, McGuire A, Lecomte T. Personal narratives and 
recovery from schizophrenia. Schizophr Res. 2010;121(1-3):271-276.
71.
Lysaker PH, Bryson GJ, Bell MD. Insight and work performance in schizophrenia. J 
Nerv Ment Dis. 2002;190(3):142-146.
72.
Campbell K, Bond GR, Drake RE. Who benefits from supported employment: a meta- 
analytic study. Schizophr Bull. 2011;37(2):370-380.
73.
Butler AC, Chapman JE, Forman EM, Beck AT. The empirical status of cognitive- 
behavioral therapy: a review of meta-analyses. Clin Psychol Rev. 2006;26(1):17-31.
74.
Kernan WN, Viscoli CM, Makuch RW, Brass LM, Horwitz RI. Stratified randomization 
for clinical trials. Journal of clinical epidemiology. 1999;52(1):19-26.
75.
Becker DR, Drake RE. A working life: The individual placement and support (IPS) 
program. New Hampshire-Dartmouth Psychiatric Research Center Concord, NH; 1993.
76.
Garber-Epstein P, Zisman-Ilani Y, Levine S, Roe D. Comparative impact of 
professional mental health background on ratings of consumer outcome and fidelity in 
an illness management and recovery program. Psychiatric rehabilitation journal. 
2013;36(4):236.
3/15/2022
VA Central IRB Protocol, Version 3 Page 36 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

77.
McGuire AB, Kukla M, Green A, Gilbride D, Mueser KT, Salyers MP. Illness 
management and recovery: A review of the literature. Psychiatric Services. 2013.
78.
Lincoln TM, Wilhelm K, Nestoriuc Y. Effectiveness of psychoeducation for relapse, 
symptoms, knowledge, adherence and functioning in psychotic disorders: a meta- 
analysis. Schizophrenia research. 2007;96(1-3):232-245.
79.
McGuire AB, White DA, White LM, Salyers MP. Implementation of Illness 
Management and Recovery in the Veterans Administration: An On-Line Survey. Psy 
rehab journal. 2013;36(4):264-271.
80.
Ottomanelli L, Goetz LL, Suris A, et al. Effectiveness of supported employment for 
veterans with spinal cord injuries: results from a randomized multisite study. Archives 
of physical medicine and rehabilitation. 2012;93(5):740-747.
81.
Bond GR, Campbell K, Drake RE. Standardizing measures in four domains of 
employment outcomes for individual placement and support. Psychiatric Services. 2012.
82.
Corrigan PW, Giffort D, Rashid F, Leary M, Okeke I. Recovery as a 
psychological construct. Community mental health journal. 1999;35(3):231-239.
83.
Narrow WE, Clarke DE, Kuramoto SJ, et al. DSM-5 field trials in the United States 
and Canada, Part III: development and reliability testing of a cross-cutting symptom 
assessment for DSM-5. American Journal of Psychiatry. 2013;170(1):71-82.
84.
Ware Jr JE, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of 
scales and preliminary tests of reliability and validity. Medical care. 1996;34(3):220-233.
85.
Beck AT, Brown GK, Steer RA. Psychometric characteristics of the Scale for 
Suicide Ideation with psychiatric outpatients. Behaviour research and therapy. 
1997;35(11):1039-1046.
86.
Muthén LK, Muthén BO. How to use a Monte Carlo study to decide on sample size and 
determine power. Structural equation modeling. 2002;9(4):599-620.
87.
Milne D, Claydon T, Blackburn I-M, James I, Sheikh A. Rationale for a new measure of 
competence in therapy. Behavioural and Cognitive Psychotherapy. 2001;29(01):21-33.
88.
Rapp CA, Goscha RJ, Fukui S. Enhanced consumer goal achievement through 
strengths-based group supervision. Psychiatric rehabilitation journal. 2015;38(3):268.
89.
Beebe J. Rapid assessment process: An introduction. AltaMira Press; 2001.
90.
Hamilton AB, Cohen AN, Glover DL, et al. Implementation of Evidence‐
Based Employment Services in Specialty Mental Health. Health services 
research.
2013;48(6pt2):2224-2244.
91.
Charmaz K. Constructing grounded theory: A practical guide through 
qualitative analysis. London: Sage Publications; 2006.
92.
Averill JB. Matrix analysis as a complementary analytic strategy in qualitative inquiry.
Qualitative Health Research. 2002;12(6):855-866.
93.
Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability 
of new programs and innovations: a review of the empirical literature and 
recommendations for future research. Implementation science. 2012;7(1):17.
94.
Lysaker PH, Davis LW, Bryson GJ, Bell MD. Effects of cognitive behavioral therapy on 
work outcomes in vocational rehabilitation for participants with schizophrenia spectrum 
disorders. Schizophrenia research. 2009;107(2):186-191.
3/15/2022
VA Central IRB Protocol, Version 3 Page 37 of 38
 
VA Central IRB
Effective Date:
July 11, 2022

3/15/2022
VA Central IRB Protocol, Version 3 Page 38 of 38
 
VA Central IRB
Effective Date:
July 11, 2022