Implementation of Tailored Collaborative Care For Women Veterans (CCWV) (QUE 15-272)
Phase: NA
Conditions: Anxiety, Depression, Stress Disorders, Post Traumatic
Sponsor: VA Office of Research and Development

 
 
 
 
 
IMPLEMENTATION OF TAILORED  
COLLABORATIVE CARE FOR WOMEN VETERANS (CCWV) 
 
 An EMPOWER QUERI Project 
Protocol Component A & B 
Funding Agency: QUE 15-272 
Principal Investigator/Study Chair: Alison B. Hamilton, PhD, MPH 
 June 22, 2020 
 
 
 
 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 2 of 45 
 
 
 
1;lip)_--- 
Abstract 
Project Background: Women Veterans are the fastest growing segment of Veterans Health 
Administration (VHA) users. This dramatic growth has created challenges for VHA. Rates of 
depression, anxiety, and mental health comorbidity are disproportionately high among women 
Veterans. Furthermore, a high rate of women Veterans’ attrition from VA care, along with 
organizational barriers to care, substantiate that organizational changes are needed in order to 
engage and retain women Veteran VHA users in evidence-based, patient-centered care. 
Project Objectives: The Enhancing Mental and Physical health of Women through Engagement 
and Retention (EMPOWER) QUERI addresses VHA Blueprint for Excellence Strategy 6, by 
advancing “personalized, proactive, patient-centered” care models, and Transformational 
Strategy 7.2.g by implementation of innovative care models in women Veterans’ health care.” 
The EMPOWER QUERI Program is designed to improve women Veterans’ engagement and 
retention in evidence-based care for mental health. To achieve this impact goal, we propose a 
cohesive portfolio of projects with the following aims: (1) To use an evidence-based 
implementation strategy that emphasizes local tailoring of care models, multilevel stakeholder 
engagement, and systematic evaluation of complex implementation processes in order to enrich 
organizational capacity for innovations in women Veterans’ VHA health care; (2) To implement 
personalized, proactive, patient-centered innovations in VHA women’s health that are 
acceptable, feasible, satisfactory, relevant, and effective for both providers and patients, thereby 
encouraging women Veterans’ engagement and retention and sustainability of the innovations; 
and, (3) To generate implementation “playbooks” for our partners that are scalable and serve as 
guidance for future implementation of a broader array of evidence-based women’s health 
programs and policy.  
Project Methods: “Implementation of Tailored Collaborative Care for Women Veterans” will 
evaluate implementation of an evidence-based collaborative care model tailored to enhance 
provider- and system-level capabilities to address women Veterans' anxiety and depression 
treatment needs, thereby improving organizational primary care-mental health integration (PC-
MHI) effectiveness and women Veterans’ engagement and retention in PC-MHI. The study will 
use a modified stepped wedge design and will apply the evidence-based Replicating Effective 
Programs (REP) implementation strategy. Mixed methods implementation evaluations will focus 
on investigating primary implementation outcomes of adoption, acceptability, feasibility, and 
reach. Multilevel stakeholder engagement will be prioritized. Program-wide organizational-, 
provider-, and patient-level measures and tools will be utilized to enhance synergy, productivity, 
and impact. As a coherent program of women’s health implementation research and quality 
improvement, the proposed EMPOWER QUERI will constitute a major milestone in achieving 
BPE strategies and realizing women Veterans’ engagement and, ultimately, empowerment in 
our VHA system.

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 3 of 45 
 
 
 
1;lip)_--- 
List of Abbreviations 
1. EMPOWER...........................................Enhancing Mental and Physical health of Women 
through Engagement and Retention 
2. PCMHI……………………………………….....……Primary Care-Mental Health Integration  
3. PCP…………………..…………………………………………………..Primary Care Provider 
4. MH…………………….………………………….………………………….…… Mental Health 
5. CCWV………………………………………….……Collaborative Care for Women Veterans 
6. WATCH……………….…….………Women’s Assessment Tool for Comprehensive Health 
7. CALM………………………………………Coordinated Anxiety Learning and Management 
8. PACT…………………………………………..……………VA’s Patient-Aligned Care Teams  
9. WH………………………………………………………………………………Women’s Health 
10. WHS………………………………………………………………….Women’s Health Services 
11. GLA………………………………………………………….……………..Greater Los Angeles 
12. NCP………………………..National Center for Disease Prevention and Health Promotion 
13. BPE…………………………………………………………..………...Blueprint for Excellence 
14. EMR…………………………………………………….…………..Electronic Medical Record 
15. REP…………………………………………………...………..Replicating Effective Programs 
16. TECH……………………………Tool for Evaluating Research Implementation Challenges 
17. WHRN………………………………………..…………..Women’s Health Research Network 
18. PBRN………………………………………………………Practice-Based Research Network 
19. DART……………………………………………………………Data Access Request Tracker  
 
 
 
 
 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 4 of 45 
 
 
 
1;lip)_--- 
Contents 
Protocol Title: ................................................................................................................................................ 5 
1.0 
Study Personnel ............................................................................................................................... 5 
2.0 
Objectives ...................................................................................................................................... 10 
3.0 
Resources and Personnel .............................................................................................................. 12 
4.0 
Study Procedures ........................................................................................................................... 17 
5.1 Study Design ..................................................................................................................................... 17 
Risk to Subjects................................................................................................................................... 25 
Potential Risks..................................................................................................................................... 25 
Adequacy of Protection from Risk ....................................................................................................... 26 
Potential Benefits of the Research to Subjects and Others ................................................................ 27 
5.2 Recruitment Methods ........................................................................................................................ 27 
5.3 Informed Consent Procedures .......................................................................................................... 29 
5.4 Inclusion/Exclusion Criteria ............................................................................................................... 30 
5.5 Study Evaluations ............................................................................................................................. 31 
5.6 Data Analysis .................................................................................................................................... 32 
5.7 Withdrawal of Subjects...................................................................................................................... 34 
5.0 
Reporting ........................................................................................................................................ 35 
6.0 
Privacy and Confidentiality ............................................................................................................. 37 
Data Security ....................................................................................................................................... 38 
Data Collection Procedures ................................................................................................................ 39 
Data Transportation Procedures ......................................................................................................... 40 
Data Transmission Procedures ........................................................................................................... 40 
7.0 
Communication Plan ...................................................................................................................... 41 
References .................................................................................................................................................. 43 
 
 
 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 5 of 45 
 
 
 
1;lip)_--- 
Protocol Title:   
IMPLEMENTATION OF TAILORED COLLABORATIVE CARE FOR 
WOMEN VETERANS (CCWV) 
 
NOTE: Component A (approved 12/27/2016) involves Key Stakeholder VA Staff 
Participants, and will be coordinated by the PI at Greater Los Angeles VA Healthcare 
System.   Component B involves Patient Human Subjects and additional sites outside of 
Greater Los Angeles VA. 
1.0 
Study Personnel 
 
Principal Investigator: Alison B. Hamilton, PhD, MPH 
Greater Los Angeles VA Healthcare System 
11301 Wilshire Blvd, Bldg 206, Los Angeles, CA 90073 
310-478-3711 extension 43785 
Alison.Hamilton@va.gov 
Affiliations:   
1 VA HSRD Center for the Study of Healthcare Innovation, Implementation & Policy 
(7/8ths) 
 
Co-Principal Investigator: Ariel Lang, MPH, PhD 
Veterans Medical Research Foundation 
3350 La Jolla Village Dr. #151ª 
San Diego, CA 92161  
959-642-6297 
Ariel.lang@va.gov 
Affiliations:   
1 VA San Diego Healthcare System (8/8ths) 
 
 
Co-Investigator: Erin Finley, PhD 
VA employee: 8/8 
VA Facility: South Texas Veterans Health Care System 
Phone: (210) 617-5300 
VA email: Erin.Finley@va.gov   
Affiliations 
1 Veterans Evidence-based Research Dissemination and Implementation Center (VERDICT) at 
the South Texas Veterans Health Care System,  
2 Adjunct Assistant Professor with the Division of Clinical Epidemiology, Department of 
Medicine, at the University of Texas Health Science Center at San Antonio. 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 6 of 45 
 
 
 
1;lip)_--- 
Co-Investigator: Catherine Chanfreau, PhD 
VA employee: 8/8 
Phone: 310-478-3711 x 49998 
VA email: Catherine.Chanfreau@va.gov 
VA Facility: VA Salt Lake City Healthcare System 
Affiliations 
1 VA Salt Lake City Healthcare System, Veterans Affairs Informatics Computing Infrastructure  
 
Collaborators include: Alexis Huynh, PhD, MPH; Rachel Kimerling, PhD; Martin Lee, PhD; 
Jessica Moreau, PhD; Anne Sadler, RN, PhD; Jennifer Strauss, PhD; Shannon Wiltsey-Stirman, 
PhD 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 7 of 45 
 
 
 
1;lip)_--- 
Introduction 
BACKGROUND 
Almost half (48%) of women Veterans have at least one mental health (MH) condition,1 with 
depression as the most common MH condition among women Veteran VA users (24% in FY14), 
followed by PTSD (14%) and anxiety (8%). Depression and anxiety are substantially more 
prevalent among women Veterans than men.2  Women Veterans are also more likely than men 
to have comorbid MH conditions, and they are more likely to have experienced military sexual 
trauma (MST)3-4 and intimate partner violence (IPV)5 Like their male counterparts, women 
Veterans tend to have complex mental health needs. However, in contrast with men, a 
significant proportion of women’s symptoms, losses, and treatment contraindications may be 
related to reproductive phases and experiences, gendered trauma histories, and gender role 
responsibilities. 
 
Women Veterans’ distinctive MH needs challenge the organization to appropriately connect 
women to MH providers who understand their unique vulnerabilities, and to support their 
navigation of often fragmented MH care options offered across VA primary care (PC) and MH 
settings. Depending on the VA site, women may have access to integrated MH care in general 
PC clinics, in a separate portion of a general PC clinic set aside for women, or in stand-alone 
women-only clinics, with variable access to women-only care options and same- gender 
providers when preferred. MH clinics may have no special arrangements for women, or may 
have some providers to whom women are preferentially assigned. At a few sites, women-only 
MH clinics may be available, or if the site has a comprehensive women’s PC clinic, full-service 
specialty MH care may be co- located.6 Such highly variable models of MH care delivery in PC 
and MH further challenge the system to engage women in services in a timely manner, and to 
support their retention in care. 
 
Collaborative care models have a strong evidence base for enhancing patient engagement 
and retention in PC-based MH care for depression,7-9 and the VA has extended application of 
collaborative care to many other mild to moderate conditions, including anxiety disorders and 
alcohol misuse. The VA’s approach to integrated MH involves collaborative care with two 
components: co-located MH professionals who are integral components of the PC team, and 
MH care management.10 While co-location has been achieved at many sites, implementation 
of care management has been limited, especially in women’s clinics. The VA endorses three 
collaborative care models that emphasize different elements of care management: Behavioral 
Health Lab,11 White River Junction,12 and Translating Initiatives in Depression into Effective 
Solution (TIDES).13 A recent report on Integrating Mental Health into VA’s Patient-Aligned 
Care Teams (PACT)14 suggests that further work should be directed at understanding the 
advantages and disadvantages of each approach, and suggests moving away from branded 
approaches, toward blended models with local tailoring. 
 
VA policy specifies that MH services must be provided in a manner that recognizes that gender-
specific issues can be important components of care.  With attention to gender-specific 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 8 of 45 
 
 
 
1;lip)_--- 
concerns, collaborative care is a promising model to improve PC-MH integration for women 
Veterans. We propose implementation of tailored “Collaborative Care for Women Veterans” 
(CCW V) that encourages local blending of elements from VA-approved models with a PC-
friendly evidence-based, computer-assisted cognitive behavioral treatment (CBT) platform for 
anxiety and depression.15 
 
 
RATIONALE 
 
The rationale for this project is based on five fundamental premises: 1) PC and MH care for 
women Veterans are complex and diverse (as described above); 2) anxiety and depression 
among women Veterans can and should be treated in PC; 3) improvements are needed in 
PC-MHI for women in these complex care structures and settings; 4) collaborative care 
designed for women Veterans would benefit the VA, providers, and women Veterans 
themselves; and 5) blended models of existing collaborative care models or components, with 
some enhancements, are needed in order to fit with local WH structures, needs, and 
resources. 
 
Anxiety and depression among women Veterans can and should be treated in the PC context. 
Since 2007, Primary Care-Mental Health Integration (PC-MHI) in VHA has evolved into a 
requirement at all VHA medical centers and large community-based outpatient clinics.16  VA WH 
policy now requires that basic mental health services be provided in the same physical location 
as women’s primary care, whether in a gender-neutral primary care clinic, in shared separate 
space in the gender-neutral clinic, or in a separate WH center. In addition, WH clinics must have 
co-located MH and social work services. Assessment and psychosocial treatment need to be 
available in women’s PC for MH disorders, including depression and problem drinking, within the 
parameters of PC-MHI. 
 
Improvements are needed in PC-MHI for women in VA’s complex WH care structures and 
settings. Women’s Assessment Tool for Comprehensive Health (WATCH) data provided to us 
by our WHS primary partners indicates that roughly 63% of VA healthcare settings (including 
medical centers and community-based outpatient clinics) have availability of PC-MHI such that 
a woman can “see a PC-MHI provider when clinically indicated…whenever this clinic is open to 
see patients on a timely, same-day basis.” Unpublished interview data from several of our 
team’s qualitative key informant studies suggest that MH services available in 
women’s PC (or in an adjacent women’s MH clinic) tend to be longer-term specialty MH 
services (as opposed to PC-MHI assessments and brief therapies) as well as women-only 
treatment groups, particularly for MH problems associated with MST experience. Informants 
have tended to report good communication between PC and MH providers, with “warm hand-
offs” and curb-siding easily occurring in the clinic, but without a care management component 
to assess and triage the more and less severe cases to appropriate services. 
 
True integration of MH supports for PC-based management of mild to moderate MH conditions 
may be overshadowed by co-located specialty MH services for more serious conditions. On the 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 9 of 45 
 
 
 
1;lip)_--- 
other hand, when women receive their PC in mixed-gender general PC clinics, women’s 
proportional minority creates challenges to provision of integrated services that can provide 
choice of women-only and same-gender provider services when desired, and may limit access 
to integrated MH providers experienced in working with the unique trauma experiences of 
women in the military. Varying care arrangements and varying needs for gender-specific 
choices make the “glue” of a care management process very important to help women 
Veterans access appropriate services, while also making care management more difficult to 
implement. 
Furthermore, work from the VA Women’s PBRN17 indicates that depression and PTSD are 
associated with women’s strong preferences for designated women’s mental health services, 
as well as treatment within the PC setting. According to a recent national survey of women 
Veterans, access to MH care for these conditions is good, but nearly half of respondents 
reported that VA MH services do not fully meet their needs.17 Our WH, MH, and PC partners 
have urged us to use this QUERI implementation research opportunity to improve PC- MHI for 
women Veterans with depression and anxiety. 
 
Collaborative care designed for women Veterans with depression and anxiety would benefit 
the VA, providers, and women Veterans themselves. Numerous similar models of collaborative 
care have been tested, primarily for care of depression in primary care settings, both outside 
the VA,18-19 and in military20 and VA11, 12, 21 populations.  A 2006 meta-analysis found that 
collaborative care is more effective than standard care in improving depression outcomes in 
the short and longer terms, and concluded that future research needs to address the 
implementation of collaborative care.7 
 
The overall higher MH burden among women Veterans, and greater complexity of 
presentations, adds to the challenges of tailoring care management to meet women’s needs. 
Models based on treating one MH condition, such as depression, tend to exclude patients with 
more complex issues. Our recent work has found that over one-quarter (27%) of women 
Veteran VA users have two or more MH conditions. Though depression is the most common 
MH diagnosis among women Veterans, depression very often presents together with anxiety 
disorders; in fact, depression-plus-anxiety and depression-plus-PTSD are tied for the two most 
prevalent co- occurring MH conditions among women Veterans.1 Therefore, collaborative care 
management models that incorporate PC-friendly transdiagnostic brief treatment options for co-
morbid presentations may be particularly suited to women Veteran populations. In addition, for 
those women whose MH needs exceed what can be offered in PC, strategies to support 
engagement in specialty care (especially when women-only settings are not available), as well 
as strong stepped care and referral management components will be important. 
 
Because of the need for transdiagnostic and flexible models to meet women’s MH needs, we 
propose to enhance existing VA models with elements of another collaborative care 
management model, Coordinated Anxiety Learning and Management, or CALM. CALM is a 
flexible, patient preference-driven model for multiple anxiety disorders (panic, generalized 
anxiety, social anxiety, PTSD) and depression in PC. Evidence for the effectiveness of CALM 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 10 of 45 
 
 
 
1;lip)_--- 
comes from a large (n = 1004) randomized controlled effectiveness trial, in 17 US primary care 
clinics with 71% women.15,22  Women receiving collaborative care showed larger reductions in 
anxiety, greater improvements in mental health functioning, and larger reductions in days of 
restricted activity than women receiving usual care, while men showed no differences in these 
measures. Women receiving the computer-assisted CBT (which used disorder-specific 
treatment modules) attended more sessions of psychotherapy, completed more modules of 
therapy, expressed more commitment, and viewed psychotherapy as more helpful than did 
men.23 Although Veterans were not the focus of the CALM trial, past trials using cognitive 
behavioral strategies for anxiety and depression in VA PC clinics have demonstrated the value 
of the general approach.24 Importantly, the effectiveness of CALM was strongly related to level 
of engagement; outcome was predicted by completing exposures and homework and attending 
more frequently.25   CALM’s care management components closely mirror those emphasized in 
VA-approved models described above, including screening and initial assessments using 
structured instruments; regular monitoring of treatment adherence and side effects using a 
standardized protocol; patient education and activation; decision support using a protocol to 
determine patient preference for medication vs a brief computer-assisted CBT-based therapy 
or both; a stepped care protocol supported by regular brief assessments and a patient tracking 
system; and assistance with referral to specialty care when appropriate. Our ability to 
incorporate elements of CALM for CCWV is strengthened by many key factors: 1) the CCWV 
Co-PI, Dr. Lang, was involved in the original CALM trial and is an expert on the CBT platform; 
2) one of the main CALM developers, Dr. Craske, has guided our study design and is a 
member of our Strategic Advisory Group; 3) CALM is currently being evaluated in a VA HSR&D 
CREATE hybrid type III implementation study, and the research team has agreed to share with 
us their VA-tailored version of the web-based CBT platform as well as their implementation 
results; 4) the lead of the CALM implementation evaluation,150 Dr. Curran, is a nationally 
recognized implementation scientist and will also serve on our Strategic Advisory Group; and 
5) one of our key partners, the PC-MHI National Medical Director, Dr. Post, strongly endorses 
our incorporation of CALM, indicating in his letter that this project will take existing PC-MHI 
programs “to a new level.” 
 
2.0 
Objectives 
 
SPECIFIC AIMS 
 
Our specific aims are to: 
 
1.  In four PBRN sites, blend and tailor existing primary care-based collaborative care 
models with an evidence-based CBT platform to provide evidence-based, patient-
centered collaborative care for women Veterans (CCWV) with anxiety and/or depression; 
2.  Implement the tailored CCWV in the four PBRN sites; and 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 11 of 45 
 
 
 
1;lip)_--- 
3.  Evaluate CCWV implementation using a modified stepped wedge design and 
conduct an implementation-focused evaluation to further refine CCWV to facilitate future 
spread. 
 
Achieving these aims will help achieve VA comprehensive PC policies for women Veterans and 
VA MH Strategic Analytics for Improvement and Learning (SAIL) performance goals of 
population coverage, continuity of care, and experience of care. 
 
Our secondary aims focus on factors that empower Women Veterans to engage in and benefit 
from collaborative care. Specifically, we will evaluate a) effects of patient activation and 
engagement with care on retention in and experiences of CCWV, and b) changes in MH 
symptoms, patient activation, health-related quality of life, and care experiences as a function of 
participation in CCWV. We hypothesize that a) more highly activated women and women with 
greater increases in activation will demonstrate better retention in care and report better care 
experiences, and b) symptoms, patient activation and quality of life will improve as a function of 
CCWV participation. 

3.0 
Resources and Personnel   
  
STAFF 
PROJECT ROLE 
STUDY ACTIVITIES 
Staff 
Degree 
Site 
Role (Level) 
Description 
Access 
to PHI 
Obtain 
Informed 
Consent 
Recruit-
ment 
Conduct 
Interviews 
Perform 
Data 
Analysis 
Hamilton, 
Alison 
PhD, 
MPH 
GLA 
PI 
Ensure that study procedures are followed to collect 
implementation evaluation data that contributes to an 
understanding of the implementation outcomes, as well as 
patient-level data that will inform secondary aims related 
to clinical outcomes. Oversee the study team, 
administrative components, project management, human 
subjects, data collection, data quality, analysis, 
dissemination of findings, and preparation of the final 
report. 
Y 
Y 
Y 
Y 
Y 
Huynh, Alexis 
PhD, 
MPH 
GLA 
Collaborator 
(Implem 
eval/stats) 
Provide both qualitative and quantitative data collection 
support and analysis in the implementation evaluations. 
She will also be the statistical analyst who runs the 
stepped wedge model.  She will contribute to the 
Implementation Core, particularly to cross-project 
analyses and manuscripts. 
Y 
Y 
Y 
Y 
Y 
Moreau, 
Jessica 
PhD 
GLA 
Implem eval 
With mentorship from Dr. Hamilton, Dr. Zuchowski will 
coordinate and lead qualitative data collection and 
analysis. She will ensure that data collection and analysis 
are conducted in a systematic and 
efficient way. In Year 5, she will participate in the cross-
QUERI analyses and in preparation of manuscripts, 
presentations and reports. 
Y 
Y 
Y 
Y 
Y 
Canelo, 
Ismelda 
MPA 
 
GLA 
Admin support 
Manage QUERI project staff and keep the team aware of 
projected timeline milestones, budgetary issues and 
compliance requirements, as well as evaluation activities 
and progress as needed. She will also coordinate the 
scientific activities of the study and provide administrative 
and regulatory support. She will also work to ensure 
the project stays on track according to the Gantt chart 
through effective project coordination, documentation and 
meeting arrangements with the PIs and the SAG. 
N 
N 
N 
N 
Y 
Oberman, 
Rebecca 
MSW, 
MPH 
GLA 
Project Director 
She will meet with leadership and the Implementation 
Team and liaison with the participating sites. Ms. 
Oberman will evaluate implementation progress and 
problem-solve implementation issues with the site staff 
and leadership throughout implementation. She will be 
responsible for the day-to-day central project activities, 
including oversight of research, implementation, and 
interfacing with VHA personnel. At the sites, she will be in 
Y 
Y 
Y 
Y 
Y 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 13 of 45 
 
 
 
1;lip)_--- 
close communication with the care managers and the 
Research Assistants. She will complete research 
assessments as needed. She will collaborate with the PI 
and research team on data analyses, including the 
qualitative data analyses. Ms. Oberman will assist with 
reports and development of the CCWV Implementation 
Playbook. She will work to disseminate project findings 
through relevant VA workgroups. She will manage the 
CIRB and DSMB reports for this multi-site study. She will 
also prepare and collate all study related materials 
(surveys, interview guides). Ms. Oberman will devote 20 
hours per week over the course of the entire QUERI 
Program. 
Jackson, 
LaShawnta 
DrP
H, 
MPH 
GLA 
Project Director 
Dr. Jackson will meet with leadership and the 
Implementation Team and liaison with the participating 
sites. She will evaluate implementation progress and 
problem-solve implementation issues with the site staff 
and leadership throughout implementation. She will be 
responsible for the day-to-day central project activities, 
including oversight of research, implementation, and 
interfacing with VHA personnel. At the sites, she will be in 
close communication with the care managers and the 
Research Assistants. She will complete research 
assessments as needed. She will collaborate with the PI 
and research team on data analyses, including the 
qualitative data analyses. Dr. Jackson will assist with 
reports and development of the CV Toolkit Implementation 
Playbook. 
Y 
Y 
Y 
Y 
Y 
Brunner, 
Julian 
PhD 
GLA 
Stats 
Mr. Brunner will support the stepped wedge analysis. 
Y 
Y 
Y 
Y 
Y 
Than, Claire 
MPH, 
CPhil 
GLA 
Stats 
Ms. Than will support the stepped wedge analysis. 
Y 
Y 
Y 
Y 
Y 
Chrystal, 
Joya 
MSW 
GLA 
Implem. Eval 
Ms. Chrystal will conduct qualitative data collection and 
analysis, and participate in the cross-QUERI analyses and 
in preparation of manuscripts, presentations and reports. 
Y 
Y 
Y 
Y 
Y 
Schweizer, 
Catherine 
Amanda  
PhD 
GLA 
Implem. Eval 
Dr. Schweizer will provide both qualitative and quantitative 
data collection support and analysis in the implementation 
evaluations.  
Y 
Y 
Y 
Y 
Y 
Dyer, Karen 
PhD 
GLA 
Research 
Assoc. 
Dr. Dyer will conduct qualitative data collection and 
analysis, and participate in the cross-QUERI analyses and 
in preparation of manuscripts, presentations and reports. 
Y 
Y 
Y 
Y 
Y 
 
 
 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 14 of 45 
 
 
 
1;lip)_--- 
SITE: Palo Alto 
STAFF 
PROJECT ROLE 
STUDY ACTIVITIES 
Staff 
Degree 
Site 
Role (Level) 
Description 
Access 
to PHI 
Obtain 
Informed 
Consent 
Recruit-
ment 
Conduct 
Interviews 
Perform 
Data 
Analysis 
Kimerling, 
Rachel 
PhD 
Palo 
Alto 
Co-I 
Given her expertise in measurement of women’s 
preferences for care as well as other key measures 
related to patient engagement (e.g., patient activation), 
support the Implementation Core. In Year 1 she will 
contribute to the consolidation of the cross-Program 
measures, ensuring that members of the team are trained 
in the measures and that they are integrated appropriately 
for use in the projects. As the first sites launch, she will 
monitor the data collection activities, ensuring quality and 
consistency and she will conduct preliminary analyses to 
ascertain the integrity of the data. She will continue to 
provide measurement support, and then in Year 5 will 
contribute substantially to cross-Program analyses and 
dissemination of findings. 
N 
N 
N 
N 
Y 
Wiltsey-
Stirman, 
Shannon 
PhD 
Palo 
Alto 
Co-I 
With her strong background and training in implementation 
science, Dr. Wiltsey-Stirman will support the 
implementation evaluation efforts in the CCWV 
implementation research project. Because her focus is on 
implementation of evidence-based treatments, she will 
provide the team with unique expertise regarding 
implementation challenges and solutions, and will 
participate in the use of the TECH. In addition, she will 
provide consistent support to the Implementation Core, 
lending her expertise in sustainability of evidence based 
care models to the team. 
N 
N 
N 
N 
Y 
Carney, 
Diane 
MA 
Palo 
Alto 
PBRN liaison 
Ms. Carney’s responsibilities on this project will be 
consultative in nature. She will serve as the PBRN liaison 
on the Implementation Core. Considering that all projects 
will occur in PBRN sites, it is imperative that we have a 
liaison who can provide contacts, information, and 
connections as well as support for the overall emphasis on 
multilevel stakeholder engagement. 
N 
N 
N 
N 
N 
 
 
 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 15 of 45 
 
 
 
1;lip)_--- 
SITE: San Diego 
STAFF 
PROJECT ROLE 
STUDY ACTIVITIES 
Staff 
Degree 
Site 
Role (Level) 
Description 
Access 
to PHI 
Obtain 
Informed 
Consent 
Recruit-
ment 
Conduct 
Interviews 
Perform 
Data 
Analysis 
Lang, Ariel 
PhD, 
MPH 
SD 
Co-PI 
Co-PI of the CCWV project and will be responsible for 
overseeing implementation at the San Diego PBRN site, 
as well as contributing to the broader scientific effort, in 
particular, overseeing implementation of the collaborative 
care protocol, with expertise in the CALM CBT platform. 
She will provide supervision and training as needed to 
clinical staff who provide the CBT. 
Y 
Y 
Y 
Y 
Y 
Jeanne 
Maglione 
MD 
SD 
Co-I 
Psychiatrist with experience in PCMHI, she will be the San 
Diego site liaison and responsible for facilitating research 
and implementation at the San Diego PBRN site, as well 
as contributing to the broader scientific effort, in particular, 
overseeing implementation of the collaborative care 
protocol. 
 
Y 
Y 
Y 
Y 
Y 
TBN 
 
SD 
Research 
Assistant 
Carry out activities including recruitment, screening, 
consent, survey and qualitative interviews  
Y 
Y 
Y 
Y 
N 
TBN 
 
SD 
Site 
Coordinator 
Manage Local Site CIRB submissions and local site R&D 
submissions.  Assist in research activities as needed 
including recruitment, screening, consent, survey and 
qualitative interviews. 
Y 
Y 
Y 
Y 
N 
 
SITE: San Antonio 
Erin Finley 
PhD, 
MPH 
San 
Antonio 
Implem core 
Oversee the implementation evaluations.  Leading role in 
development, drafting, and dissemination of the 
Implementation Playbook. 
Y 
N 
N 
Y 
Y 
 
SITE: Salt Lake City 
Catherine 
Chanfreau,  
PhD,  
Salt 
Lake 
City 
Implem core 
Dr. Chanfreau will lead the data extraction from CDW, will 
be part of the implementation evaluation team, and will 
oversee the data extraction and measures construction for 
the impact, access, and other measures for 
implementation monitoring and feedback to the sites, as 
well the overall trial. 
Y 
N 
N 
Y 
Y 
 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 16 of 45 
 
 
 
1;lip)_--- 
Co-Investigators involved in design and interpretation and dissemination, accessing only de-identified data, include: Anne 
Sadler, RN, PhD (Iowa City VA) and Jennifer Strauss, PhD (Durham VA). 

4.0 
Study Procedures 
 
This study is part of a larger EMPOWER QUERI project focusing on innovation in care for women 
Veterans at VA. Project stakeholders include staff and leadership as subjects and/or key informants. 
Products include development of an innovative approach for addressing implementation of a care 
model that addresses anxiety and depression for woman Veterans.  
 
5.1 Study Design 
EMPOWER QUERI Conceptual Framework. This QUERI Program focuses on strengthening WH 
organizational capacity for innovation in patient-centered care, using an Implementation Core and 
three projects as a collective platform for examining how particular characteristics of care models 
contribute to providers’ ability to utilize the models and to patients’ engagement and retention in the 
models.  As depicted in our conceptual framework (Fig. 2), the Implementation Core forms the 
backbone of our program, and relies on an evidence-based implementation strategy, Replicating 
Effective Programs (REP), and emphasizes multilevel stakeholder engagement and complexity 
theory. The Core will examine organizational capacity for innovation in the context of local WH care 
arrangements, which we know to be quite diverse, as well as implementation constructs such as 
climate and leadership, and novel constructs such as organizational readiness for patient engagement 
and implementation citizenship. Knowledge of organizational capacity will inform implementation of 
innovative care models that address anxiety and depression. Implementation outcomes will contribute 
to development of Implementation Playbooks—brief, user-friendly summations of implementation 
targets, processes, outcomes, and recommendations for scale-up and spread.26 Together, these 
activities will contribute to our ultimate goal of enriching VHA’s organizational capacity to engage and 
retain women Veterans in patient-centered, proactive, personalized evidence-based care. 
 
 
 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 18 of 45 
 
 
 
1;lip)_--- 
 
EMPOWER QUERI Implementation Strategy: Replicating Effective Programs (REP). We will use 
REP13 across all projects to maximize the implementation science knowledge that will be generated by 
a common approach. Grounded in theories of Diffusion of Innovation and Social Learning, REP was 
selected because of its strong evidence base and application in VHA health services research.27 It 
also provides a phased framework for implementation, with different discrete implementation 
strategies being employed in different phases. REP’s demonstrated effectiveness in promoting uptake 
of evidence-based practices allows us to focus on its application in varied settings and care models. 
Use across all projects will allow for further testing and expansion of the framework, particularly 
through our emphasis on multilevel stakeholder engagement and incorporation of complexity theory.  
The REP framework consists of four phases (Fig. 3): pre-conditions, pre-implementation, 
implementation, and maintenance/evolution. Careful attention is paid to intervention packaging during 
pre- conditions and pre- implementation; training, technical assistance, and fidelity assessment during 
implementation; and recustomizing during maintenance/evolution. 
 
During each phase, local context is paramount, with varying deployment of the intervention depending 
on local priorities, needs, and resources. One of our implementation science goals will be to track the 
relative importance of each discrete strategy in each phase at each site and across sites, as well as in 
each project and across projects, to inform user-friendly, implementation practice “playbooks” for our 
partners. 
 
Enhancing REP with Multilevel Stakeholder Engagement and Complexity Theory. REP was originally 
designed to guide dissemination of evidence-based practices in community-based organizations. 
Kilbourne and colleagues27 note that “it was not designed to address multilevel barriers to 
implementation,” so they enhanced REP with facilitation, using implementation experts as external 
facilitators to provide guidance for overcoming implementation barriers. Interestingly, outcomes were 
favorable for enhanced REP for their primary implementation outcome of uptake (i.e., completed 
contacts with Veterans with serious mental illness who had been lost to care), but not for increased 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 19 of 45 
 
 
 
1;lip)_--- 
utilization of services by patients who had dropped out of care.28 This prompted us to consider 
alternate REP enhancements that are 1) more focused on participatory action29 within complex 
adaptive systems30 in VHA WH clinical settings,31 and 2) potentially more effective in increasing 
patient engagement. Accordingly, we draw on complexity theory and multilevel stakeholder 
engagement (Fig. 3). Complexity theory postulates that outcomes in complex adaptive systems are 
nonlinear and unpredictable; it is a highly relational theory, examining how multiple agents involved in 
implementation interact in complex ways and “make sense” of implementation in different ways.32 This 
emphasis on incorporating multiple perspectives and differing priorities is consistent with our VHA WH 
roadmap to delivering gender-sensitive comprehensive care for women Veterans,33 which suggests 
that multilevel stakeholder engagement is key to improving women Veterans’ health care. 
 
We will operationalize complexity theory through use of the Tool for Evaluating Research 
Implementation Challenges (TECH),34 which was designed to systematically assess impacts of 
implementation challenges and guide potential solutions. Simpson and colleagues recommend this 
tool under three pre-conditions: 1) “implementation adaptations are expected due to the emergent 
nature of complex research settings;” 2) the research environment needs to encourage “spontaneous 
emergent solutions” and creativity; and 3) all research team members must be empowered to 
participate. Our approach, where all stakeholders are integral to achieving the impact goal, supports 
these pre-conditions. 
 
TECH, which has been used successfully in VHA and community studies, is codified into a series 
of interactive steps:  identifying challenges (e.g., through observing day-to-day dynamics, 
listening to complaints, asking questions, etc.), interpreting the challenges in weekly meetings, 
generating and testing solution strategies, and, if necessary, addressing regulatory issues. 
Solution strategies are developed through open dialogue among the team members as well as 
others who might have perspectives on potential solutions. Although applicable to many different 
types of research, TECH is especially useful for research in complex adaptive systems as a tool 
for addressing unexpected challenges in systematic and collaborative ways. 
 
Multilevel Stakeholder Engagement in the EMPOWER QUERI. Our QUERI capitalizes on our 
seasoned team of implementation scientists, health services researchers, and clinicians, supported by 
a stellar Strategic Advisory Group. Furthermore, we have the distinct advantage of being buttressed 
by the VHA HSR&D Women’s Health Research Network (WHRN), which is comprised of the 
Women’s Health Research Consortium and the Women’s Health Practice-Based Research Network 
(PBRN; §3.3.1), and a research aim on multilevel stakeholder engagement. Our entire team is 
associated with the WHRN (with EMPOWER QUERI PI Hamilton also being one of the WHRN PIs). In 
fact, this QUERI was developed with consistent input from the PBRN Site Leads (some of whom are 
on our team: Bean-Mayberry (PI), Mattocks, Sadler), and the projects will all be conducted at PBRN 
sites (Frayne LoS). Moreover, our QUERI proposal has been strengthened from its inception by 
invaluable input of women Veterans themselves. Specifically, a newly forming Women Veteran 
Patient Advisory Council has agreed to serve as an implementation partner for this QUERI Program 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 20 of 45 
 
 
 
1;lip)_--- 
(Gottke LoS); Dr. Hamilton will serve as an advisor to the Council and will use principles of patient 
engagement in research33 to guide collaboration and involvement of the Council. 
 
EMPOWER QUERI Methods 
 
Setting: Women’s Health Practice-Based Research Network (PBRN). The PBRN provides a research 
infrastructure for investigators seeking to increase inclusion of women in VHA research or conduct 
multi-site women’s focused research in VHA. Comprised of 37 VAMCs that see one-third of women 
Veteran VHA users, the PBRN helps investigators overcome the challenges of multi-site studies 
through engagement of Site Leads with established working relationships with local clinicians and 
facility leadership. Site Leads have received implementation training, and have developed across-site 
relationships through regular national calls and technical support organized by the PBRN 
Coordinating Center in Palo Alto (Frayne LoS). 
 
Modified Stepped Wedge Study Design for Implementation Trials. We will use stepped wedge 
designs, which rely on sequential roll-out to participating sites over time, while using other sites as 
controls until they begin implementation. Consistent with our substantial prior experience using these 
designs in VHA and armed with complexity theory’s recognition of nonlinearity of implementation, we 
will use modified stepped wedges (rather than randomized) given their suitability for studying 
implementation. This design acknowledges that sites are heterogeneous, face multiple constraints, 
and, as a result, are ready to adopt interventions at different rates. The modified design explicitly 
considers the timing of implementation spread and addresses the statistical issues introduced by lack 
of randomization in implementation starts and processes. We will analytically compensate for the 
design by collecting patient-, provider-, and site-level data that may be associated with timing of the 
adoption of each intervention. 
 
We will include three levels in our hierarchical stepped wedge models: (1) patient,(2) time of 
intervention (i.e., when a provider starts using the intervention), and (3) site. Outcomes of interest are 
measured for all patients at each site within the given intervention time period. We will include four 
PBRN sites in each of our two implementation trials 
 
Data collection will occur in each of the REP phases (e.g., pre-conditions for 6 months, pre-
implementation for 6 months, implementation for 15 months, and maintenance/evolution for 4 months). 
Similar to previous modified stepped wedge studies, implementation initiates with PCPs. Figure 4 
depicts Site A initiating at Quarter 3, Site B at Quarter 4, and so on, with three PCPs at each site, each 
quarter, for a total of 81 PCPs. The PCPs “turn on” (i.e., use the template in the CV project; refer to 
the care manager in the CCWV project) as they switch from pre-implementation to implementation. 
 
Modified stepped wedge designs make efficient use of all data available for within-site and between-
site comparisons.  For the within-site comparison, sites act as their own controls in an evaluation that 
compares sites pre- versus post-implementation. The comparison examines sites as they cross-over 
from control to intervention states. The between-site comparison evaluates the intervention period for 
a site vs. all other intervention and control periods for all sites. By having these two types of 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 21 of 45 
 
 
 
1;lip)_--- 
comparison, the design improves on the validity of the evaluation of the intervention, by accounting for 
historical time trends that may occur outside of the intervention and for site contextual characteristics 
that may affect site performance. 
 
We will use a modified stepped wedge design to evaluate the implementation in four VA Women’s 
PBRN sites (Frayne LoS). In the context of the modified stepped wedge design, the intervention is 
“turned on” when a PCP at a site makes her/his first referral to the CCWV care manager (see below). 
 
In Year 1, we will begin the REP pre-conditions phase in one VA Women’s PBRN sites. We will 
continue through the phases over the course of four years, with a 15-month implementation phase in 
order to ensure adequate time to evaluate implementation and sufficient numbers of Women Veterans 
being exposed to CCWV. Three additional PBRN sites will be “stepped in” during Year 2at appropriate 
intervals over time. According to the REP framework and complexity theory, all sites will be 
encouraged to adapt and tailor set-up and delivery of CALM according to their local resources and 
care configurations.  
 
REP Implementation Phases 
 
Pre-conditions (Aim 1): The need for CCWV has been established and the review of effective 
interventions has been conducted (see above). To ensure that the care model fits local settings and 
stakeholder priorities, the PI and Co-Investigator Dr. Oishi will visit the sites and meet with the local 
team to discuss local structures, care models and processes, educational needs, as well as elements 
of the proposed CCWV and tailoring options that we have conceptualized (Table 3). Using the TECH, 
the local team will discuss options for CCWV implementation and anticipated barriers to 
implementation. Also during this phase, interviews and surveys will be conducted with all consenting 
key stakeholders, with the exception of patients who will be interviewed and surveyed in the 
implementation phase (see below).

Table 3. Elements of Collaborative Care for Women Veterans (CCWV) 
 
Care management 
Functions 
CCWV Element 
Local Tailoring Options 
Initial MH 
assessment of patients 
referred by PCP 
Care manager conducts baseline MH 
assessments, medication history, and other relevant 
information collection 
Site designates a local MH care manager (RN or 
social worker) who conducts some or all care 
management functions; sites choose from VA-
approved assessment packages 
Women with positive 
assessments offered 
treatment choice 
Treatment choices for anxiety, depression, and PTSD 
are medications, primary care-based CALM CBT, or both 
Site chooses integrated MH providers 
according to local availability (psychologist; 
social worker) for training in CALM CBT 
 
Care management 
Functions 
CCWV Element 
Local Tailoring Options 
Women with severity 
or complexity requiring 
specialty treatment 
assisted to access 
specialty resources 
Care manager inquires about preferences for 
gender-specific services and provider gender; facilitates 
connection to most congruent locally available specialty 
services, and makes a warm handoff if possible; care 
manager follows up with patient to check on any missed 
appointments or group sessions 
Referral can be made to co-located or 
distant specialty services as available at the site; care 
manager is familiar with locally available gender-
specific groups and providers most experienced in 
treating women; same gender provider options 
identified ahead of time 
Systematic 
monitoring of symptoms 
and treatment emergent 
problems 
Care manager uses structured MH assessments 
for depression, anxiety disorders, and alcohol problems as 
well as symptom and side effects checklists to monitor 
treatment response; follows up to assure medication and/or 
CBT session compliance 
Sites choose from VA-approved follow-up 
assessment options; site can choose to involve peer 
support for telephone check-in with patient and in-
person chaperoning and escorting to various services 
Patient education 
and activation 
If the patient has chosen medication, care 
manager provides basic patient education and activation 
relevant to the primary diagnosis; if CBT is chosen the 
care manager facilitates connection to an integrated 
provider trained in CALM 
Site decides which materials from available 
care management programs care manager will offer; if 
in the scope of practice of the locally chosen care 
manger (e.g., MSW) the care manager may be trained 
to provide CALM CBT 
Decision support 
Care manager uses a stepped care protocol 
under supervision of a local psychiatrist to determine when 
decisions to increase or change treatment options are 
needed and to collect information to assist the primary care 
provider in making adjustments 
Decision support templates may be locally 
selected from VA-approved options; sites choose a 
supervising psychiatrist who may be co-located or 
based in specialty mental health depending on local 
circumstances 

Site 1: Because this project’s Co-PI (Dr. Lang) is based at the San Diego PBRN site, we have already 
laid the groundwork for implementation at this site, in collaboration with the PBRN Site Lead, Dr. 
Meredith Barnes, who is the WH Medical Director. VA San Diego is an interesting and complex site at 
which to implement CCWV, because PC is delivered in two locations (La Jolla and Mission Valley) 
that serve both men and women, with 41 designated WH providers across the healthcare system. 
These two locations differ in their PC-MHI teams, with one having a clinical psychologist, a nurse case 
manager, and a part-time (1/8) psychiatrist; and the other having two psychologists, a nurse case 
manager, a part-time (6/8) psychiatrist, and psychology trainee time (approx. 40 hours/week). Unless 
the PCP is prescribing him/herself, most PC patients with MH needs are referred to the Anxiety 
Disorders Clinic or the Mood Clinic, unless the diagnosis is PTSD, in which case women are referred 
to the women’s trauma clinic. These clinics all have waiting lists. MH providers at the two PC sites 
have expressed to the co-PI that CCWV would be an important and needed addition to the services 
currently provided, especially in that, for those who need a higher level of care, it can provide a 
“bridge” to that care and may even circumvent the need for such care if desired outcomes are 
achieved through CCWV. This site has a strong peer support specialist program, led by the Local 
Recovery Coordinator who has expressed her support for this study and will actively “market” the 
availability of female peer support specialists to the WH PCPs and patients. Of note, across the two 
PC clinics from Oct 2014 – Apr 2015, there have been 688 unique encounters in primary care by 
women with anxiety or depression.  This substantiates that we will have sufficient numbers of women 
at this site to ensure that at least 130 will be referred to the CCWV care manager (see power analysis 
section) and will complete the baseline measures. 
 
Pre-implementation (Aim 1 cont.): At each site, a local CCWV champion will be identified by the Site 
Lead when s/he signs onto the project. The CCWV package will be further developed and locally 
tailored, with attention to training and technical assistance needed at each site. Once prepared, the 
package will be piloted locally to ensure that it works as intended with local systems and processes. In 
order to use the CALM CBT platform, we are collaborating with Dr. Curran (see LoS) who will provide 
us with a VA version of the platform when it is ready (their CREATE study). Using the TECH, any 
challenges with pilot implementation will be discussed and addressed by the team. When the team 
determines that challenges have been sufficiently addressed, orientation meetings will be held with 
the broader clinic, where the CCWV package will be distributed, discussed, and marketed. 
 
Implementation (Aim 2): The implementation phase will begin at one PBRN sites and will last a total of 
15 months to ensure adequate time for patient Exposure to the care model as well as implementation 
evaluation and reach. CCWV will spread to three additional PBRN sites in the following years. PBRN 
representatives will be trained in CALM.  Field notes will be taken during the training and minutes will 
be kept of all conference calls and meetings. Notes will be analyzed using ATLAS.ti and will be 
interpreted in conjunction with other evaluation data as described below. During regular 
implementation meetings, TECH will be used to assess and address implementation challenges. 
 
Procedures: Once training is complete and sites are ready for implementation, women Veterans with 
possible depression and/or anxiety disorders will be referred by their PCP (or other referring provider) 
to the locally identified CCWV care manager, who will also conduct local case finding activities to 
identify women Veterans who may benefit from care management. The care manager will then 
complete a clinical evaluation to identify the disorder of primary clinical focus (Panic Disorder with or 
without Agoraphobia, Social Anxiety Disorder, Generalized Anxiety Disorder, Posttraumatic Stress 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 24 of 45 
 
 
 
1;lip)_--- 
Disorder, depression). CCWV is an innovative care model that is proactive, patient-centered and 
personalized to each woman that has an appointment with the care manager. Specifically, the care 
manager assists the patient in deciding which type of treatment she wants to pursue (CBT, 
medication, or both). During or shortly after the initial meeting with the care manager, each woman will 
be asked by the RA if she is willing to participate in our research project, meaning that she will be 
asked to complete a brief written battery of measures (§3.3.3.3) and a short qualitative interview at 
baseline (after the first or second visit with the care manager) and at six-months post-baseline.  
Women who enroll after the baseline window will not be eligible for baseline survey measures; 
however they can participate in interviews. Participants who enroll in research after the baseline 
period, and any women who are otherwise unable or unwilling to complete the written surveys, will be 
invited to consent to participate in other components of the research (i.e., qualitative interviews, , data 
obtained from medical record).     
 
For women who choose CBT, the CALM-trained provider will then meet with the patient weekly to 
review the CALM CBT modules. The program includes five general modules (introduction, 
psychoeducation, self- monitoring, breathing retraining and relapse prevention), and three other 
modules that are customized to each anxiety disorder (cognitive restructuring, in vivo exposure and 
interoceptive exposure), which are tailored to each anxiety disorder. Depression modules are also 
available. The CALM CBT platform is a computer-assisted delivery system. The provider sits with the 
patient in front of the computer and directs the individual to review the appropriate material, clarifying 
and individually tailoring. The program contains educational material as well as skills training and 
exposure exercises. The entire program is typically completed within eight 60-minutes sessions.  
 
Maintenance and Evolution: The last phase in this framework allows us to take the feedback and 
make modifications to our implementation process to enhance CCWV adoption, fidelity to the REP 
framework, and dissemination to future sites. Sharing these details with stakeholders using TECH 
informs the current implementation process and sustainability for the future. The four REP phases 
allow us to account for each step of the guided implementation strategy. Finally, the interchange 
between these phases and the implementation process components provide an in-depth 
understanding of whether CCWV implementation requires further adaptation and customization prior 
to broader VA dissemination. During this phase, the research team will collaborate with the local 
implementation teams to develop points for the CCWV Implementation Playbook. 
 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 25 of 45 
 
 
 
1;lip)_--- 
Risk to Subjects 
This study involves both Veteran (520 total across sites) and VA Staff participants (60 total across 
sites). 
Veteran Involvement and Characteristics  
This 4-site project aims to inform the VA regarding implementation of collaborative care for women 
Veterans (CCWV).  The study is primarily focused on implementation outcomes such as adoption, 
acceptability, feasibility, satisfaction, and reach, as well as patient engagement and retention; patient 
clinical outcomes (symptoms, etc.) though examined, are secondary. After IRB approval, at each 
participating site, we will enroll approximately 130 women Veterans who are referred by their primary 
care provider (PCP) or another referring provider to the CCWV care manager. Any woman referred to 
the care manager (and therefore initially screening positive for anxiety, depression, and/or PTSD) is 
eligible to participate. We will stop consenting enrollees once we have reached up to 80 women who 
have had an encounter with the care manager and who consent participate in the research, in order to 
complete data collection with 65 women at each site.  Our goal is to have 260 women total (across the 
4 sites) engaged in care management over the course of the study.  Participation in CCWV consists of 
completion of the surveys, interviews, and access to relevant medical record data.  
Routine Clinical Care versus Research.   Meeting with the care manager and participating in 
Coordinated Anxiety Learning and Management (CALM), being followed by the care manager for 
medication adherence, psychiatric treatment and medication, or other mental health treatments are 
routine clinical care available to any woman Veteran seen at a participating primary care or women’s 
health clinic.  Consenting to research involves completing the research surveys, research interviews, 
and granting permission to access selected medical record data as part of an evaluation of the 
implementation outcomes and secondary clinical outcomes described above.  Participants who enroll in 
research after the baseline period, and any women who are otherwise unable or unwilling to complete 
the written surveys, will be invited to consent to participate in other components of the research (i.e., 
qualitative interviews, data obtained from medical record).   Patients do not have to consent to 
participating in research in order to receive routine clinical care with the care manager. 
Provider / VA Staff Involvement and Characteristics  
We will recruit approximately 15 VA staff members per site. Staff members from each site will comprise 
primary care providers, care managers, MH providers, social workers, and administrators (e.g., Primary 
Care Chief, MH Chief). Given the diversity of VA staff, this sample will include women, men, and 
members of minority and ethnic groups.  This array of providers and staff will allow varied perspective 
on the CCWV care delivery process at each site. 
Potential Risks 
Risks are minimal.  
For patients, study assessments will include potentially sensitive questions and information about 
mental and/or physical health symptoms, thus there is risk for embarrassment and negative effects on 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 26 of 45 
 
 
 
1;lip)_--- 
patient subjects if sensitive information was improperly disclosed and used in a discriminatory fashion 
against the individual. It is also possible that patients may become upset or triggered when asked about 
mental health concerns and exposures, or that suicidal thoughts may be revealed during questionnaire 
completion or during interviews.  Additionally, individuals’ social security numbers will be collected in 
order to obtain data from VA databases and to collect data from patient medical records.  If disclosed, 
this information could cause negative financial or legal consequences.  Recruitment may also involve 
the risk of coercion if the Veteran perceives that refusal might threaten care or benefits.   
For staff participants, because these studies ask questions about organizational issues at the clinic 
where they are employed, there is risk for embarrassment and negative effects on subjects if sensitive 
information was improperly disclosed and used in a discriminatory fashion against the individual.   
However, given the safeguards against improper disclosure and steps we are taking to ensure 
protection of sensitive information as detailed below, the likelihood of any disclosure is minimal.  
Adequacy of Protection from Risk 
Procedures to safeguard against adverse events. All data collection protocols will include procedures 
whereby research staff members report problems with the data collection, concerns about risk to 
subjects or others, or unusual occurrences during the collection. These allow project leaders an 
opportunity to quickly review and respond to any possible concerns or adverse effects.   
Informed consent. Research participants are advised of the voluntary nature of participation and 
research subjects are informed of their right to withdraw from the project at any time. Each participant 
receives a verbal and written description of the study. The Principal Investigator and delegated 
research team members are available at all points of the study to answer questions and to explain 
assessment procedures, uses to which the data will be put, and confidentiality of data.  
Mandatory reporting safeguards. Procedures to prevent violation of confidentiality are limited by the 
mandatory nature of legal reporting requirements. Subjects are informed in the consent document and 
verbal consent process that their responses to the surveys and interviews are confidential, except: 
“1) If we need to protect you or others from harm. An example of this is if you want to hurt yourself or 
others. In that case, we will give information about you to others that is needed to protect you or others 
from harm.  
2) If you tell us about the abuse of a child or of an elderly person, or if you tell us about being abused 
yourself. In that case, we must report this to a supervisor, who may report it to the authorities.”  
 
Data safeguards. Before analysis of the data, all identifying information will be removed from the data. 
No information that could identify study subjects will be published or presented. Data will be reported 
only in the aggregate. Identifiable data will be stored separately from coded data sets. Access to 
identifiable study data will be restricted to research teams on a need-to-know basis.  
 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 27 of 45 
 
 
 
1;lip)_--- 
The CALM CBT program that will be used by care managers who work with patients choosing this 
option will be accessed from a secure website behind the VA firewall.  No mental health information 
recorded in the program during CALM CBT sessions, including mental health assessments, will be 
accessed for research purposes. We will document which modules were used for each participating 
patient as a way of tracking the most useful and effective content for this population. 
 
 
Discomfort with assessment, intervention procedures or disclosure safeguards. During the course of 
participation in the study, a subject could feel uncomfortable answering the survey or interview 
questions.  All interviewers will be trained in the assessments and with sensitivity to the issues to be 
discussed.  All subjects are informed that they may choose to skip any questions that they find 
uncomfortable or withdraw from the study at any time.  
For patients who consent to be surveyed, the 30-45 minute surveys are handed to the participant by the 
RA, and will be completed in a private room in the VA clinic setting at each site.  The RA will check for 
completeness as well as suicidality items before participant leaves.  Qualitative interviews will be 
conducted in person in a private room or by phone.   
Staff interviews will be conducted over the phone or in person in a private room.   
Response procedures for adverse events are described in Section 6 Reporting (below). 
 
Staff training safeguards. As required, all research staff will be credentialed by their VA Healthcare 
System’s research service, and will successfully complete all required trainings. Clinical staff provide 
routine clinical care.  Research staff will meet with their local study supervisor (the Site Lead or the 
Project Director) on a regular basis for supervision, ongoing monitoring of interactions with subjects, 
and problem solving. Research and clinical staff are trained to identify events that would fall under 
mandatory reporting guidelines, such as harm to self or others. The PI will be available at all times by 
cell phone. When the PI is not available, the PI will delegate this responsibility to another qualified, 
credentialed staff member.  
 
Potential Benefits of the Research to Subjects and Others  
Although individual subjects do not benefit directly, there are potential benefits to human subjects from 
participation. Participants may be able to impact the development and future dissemination of the care 
models for women with anxiety and/or depression at their own or other VA sites. They may also be able 
to provide feedback on care in the clinic where they receive services or are employed, which may affect 
their treatment or working environment in the future if this model is widely adopted.   
 
5.2 Recruitment Methods 
This study involves both Veteran Patients (n=260; 65/site) and VA Staff participants (n=60; ~15/site). 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 28 of 45 
 
 
 
1;lip)_--- 
Patients.  Patient subjects will be recruited after making contact with the CCWV care manager via 
referral from their provider or other staff. The care manager will introduce the research study, and if the 
person wishes to find out more about the study and possibly participate, the care manager will 
introduce the Veteran to the RA (if available), ask the Veteran to complete the Research Candidate 
contact form, add research staff as an additional signer on their CPRS note to automatically notify 
research staff of interested individuals, or contact the RA through another secure HIPAA-compliant 
route such as phone call or encrypted email to relay the Veteran’s referral information.  . The RA will 
check in with the care manager frequently to learn when women Veterans have been referred, as well 
as respond to notifications from the CPRS system, phone calls, and encrypted emails. Using a visit-
based sampling method, patients are eligible during their first or second visit with the care manager 
during the 15-month implementation period. The patient will be informed about the details of research 
participation and asked if they would like to participate in the project.  If they agree, the written or verbal 
(if by phone) informed consent process will be conducted.  Patients that are past the enrollment window 
(i.e., 1st or 2nd visit with care manager), or otherwise unable or unwilling to participate in surveys are still 
eligible to complete qualitative interviews.   
For patients who enroll after the baseline window and will only be invited to consent to participate in 
other components of the research (i.e., qualitative interviews, data obtained from medical record) by 
phone, after receiving information about interested Veterans from the care manager, the RA will mail 
the interested Veteran a written information sheet about the study that contains all the elements for 
informed consent and HIPAA, including information about audio recording.. The RA will wait one week 
before calling the Veteran to check if they have received it. If they have received it, the RA will discuss 
the details of research participation with the Veteran and if the Veteran would like to participate, the RA 
will review the written information sheet and conduct a verbal consent process that includes reviewing 
all the elements of consent and the HIPAA, and verbal consent will be recorded at the time of interview 
for these patients.  
 
Staff.  A total of 60 to 80 VA Staff participants will be recruited from 4 sites. Staff participants will be 
recruited from women’s health clinics that are using a WH PACT model of care.  To obtain the staff 
sample, we will work with each site’s PBRN Site Lead and/or Site Representative to develop a list of 
eligible individuals (and their contact information), who meet the staff categories we would like to 
include. The study PI and/or the Site Lead will present the study to local staff and administrators via 
email and/or phone call.  Staff members will be emailed an invitation to participate in an initial interview, 
which will be collected at the time of the initial interview. This email will also inform the participant that 
their participation includes three semi-structured interviews, either in person or over the phone, that will 
take 30 minutes each, the initial interview in the next few weeks, the second interview 10-12 months 
later and the final interview about a year later (the expected duration of their participation will be a total 
of 3 years).  The recruitment process will include informing potential staff participants about the study, 
who is conducting it, why the individual was selected for recruitment, and how the information gathered 
will be used. Potential staff participants will be informed that their participation is entirely voluntary and 
their decision about participation will not affect their employment, merit, or promotion.  After explaining 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 29 of 45 
 
 
 
1;lip)_--- 
the study, if the staff member is interested, the GLA research staff will schedule an appointment, or 
local staff may help schedule in-person interviews with visiting GLA research staff. We will also inform 
the participant that in 10-12 months, unless they ask us not to, we will contact them again regarding 
participating in a similar follow-up interview to document events across time. 
  
5.3 Informed Consent Procedures 
 
Patients.  For patients who enroll during the baseline period and complete the written surveys, we will 
obtain a signed HIPAA authorization to access patient medical records.  The informed consent process 
will include informing potential subjects about the study, who is conducting it, why the individual was 
selected for recruitment, that their participation in the research will be entered in their medical record, 
how the information gathered will be used, and about the patient participant payments.  Payments 
include $20 voucher for each of two surveys and, $10 voucher for each of the interviews at baseline 
and six months post-baseline.  
Patients who enroll after the baseline window will only be invited to consent to participate in other 
components of the research (i.e., qualitative interviews, data obtained from medical record). We will 
request a waiver of HIPAA in entirety from this subset of patients and a waiver of documentation of 
consent from this subset of patients, as these components may be conducted by phone. After receiving 
information about interested Veterans from the care manager, the RA will mail the interested Veteran a 
written information sheet about the study that contains all the elements for informed consent and 
HIPAA, including information about audio recording. The RA will wait one week before calling the 
Veteran to check if they have received it. If they have received it, the RA will discuss the details of 
research participation with the Veteran and if the Veteran would like to participate, the RA will review 
the written information sheet and conduct a verbal consent process that includes reviewing all the 
elements of consent and the HIPAA, and verbal consent will be recorded at the time of interview for 
these patients.  
 
Potential subjects will be informed that their participation in the research is entirely voluntary, and that if 
they choose not to participate, there will be no effect on the care they receive.  Methods for how 
subjects’ identity and collected data will be protected will be discussed. Patients will be asked if they 
understand the nature of their participation, and will be asked to describe their understanding of 
participation. 
Staff.  A request for waiver of documentation of consent will be requested as affirmative response to 
the email invitation will constitute informed consent and verbal consent will be recorded at the time of 
interview.  After a response has been received, enrolled staff members will be scheduled for an 
interview with a member of the research team.   

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 30 of 45 
 
 
 
1;lip)_--- 
 
5.4 Inclusion/Exclusion Criteria 
 
Staff 
Inclusion Criteria: VA staff affiliated with women’s health care. Given the diversity of VA staff, we expect 
this will include women, men, and members of minority and ethnic groups. 
 
Exclusion criteria: Non-VA staff. 
 
Patients 
Inclusion Criteria: female VA patients who are referred to the care manager for further mental health 
assessment and/or care. 
 
Exclusion criteria: Non-VA patients; less than 18 years of age; anyone who is unable to provide 
consent. 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 31 of 45 
 
 
 
1;lip)_--- 
5.5 Study Evaluations 
 
Qualitative Data Collection:  
 
Staff Interviews:  To understand  clinic organization, provider and staff roles, existing care 
arrangements for women Veterans, and perceived acceptability and feasibility of the CCWV program 
from key participants (VA staff, including WH primary care providers, nurses, clinical staff and health 
coaches); we intend to recruit approximately 15 staff per site who will be asked to complete a brief 
interview. Participation will consist of three 30-minute qualitative interviews (pre-, mid-, and post-
implementation) conducted in-person or over the phone by a research staff member. We will evaluate 
the pre-implementation interviews (approximately five months into the pre-conditions phase); 
perceptions of interim implementation processes and outcomes (approximately six months into the 
implementation phase) and summative perceptions of implementation and potential for spread (last two 
months of the maintenance/evolution phase). The mid-implementation assessment will evaluate the 
progress and implementation barriers and facilitators to CCWV implementation and be used to 
continue to tailor the implementation.  Final interviews will re-evaluate baseline domains, and will 
follow-up on CCWV implementation and sustainability. 
 
 
Ethnographic field notes will be taken by research team members throughout implementation to capture 
aspects of the context of implementation and otherwise unmeasured aspects of usual care. The two 
anthropologists on the QUERI Program (Hamilton and Moreau) will provide training on writing field 
notes. Also, minutes will be recorded for all project meetings (including trainings) and conference calls. 
In addition, substantive emails and other communications regarding implementation will be archived 
and analyzed. 
 
 
Quantitative Data Collection: The measures will be compiled into a written questionnaire for patients.  
Participants will be asked to complete brief questionnaires during or near the same time as the 
qualitative interviews.   
 
Information from and about Veterans:  
Sources of patient data for this project include (1) Data obtained from medical records including 
utilization of MH and CCWV services, psychiatric medications, MH diagnoses, relevant patient history 
(i.e., MST), and symptom measures completed with care manager and obtained from medical record; 
(2) information provided by patient participants via surveys (baseline and six months) and interviews 
(baseline and six months; and (3) CALM modules accessed and/or completed.  ( 
After consent, all enrolled participants in each project who enroll during the baseline window will 
complete a 30-45 minute survey, delivered in-person by a research assistant (RA) at baseline and six-
months post-baseline. The surveys focus on patient activation, health-related quality of life, satisfaction 
with treatment, and project specific measures. In addition, a subset of approximately 45 women 
Veterans in each study will also be asked to complete an additional 30-minute qualitative interview with 
a research staff member at baseline and six months post-baseline.  The participant will be asked to 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 32 of 45 
 
 
 
1;lip)_--- 
complete a hard copy, which the RA will send via fax to a monitored fax machine at GLA to enter into 
the database. Participants who enroll in research after the baseline period, and any women who are 
otherwise unable or unwilling to complete the written surveys, will be invited to consent to participate in 
other components of the research (i.e., qualitative interviews, , data obtained from medical record), 
which may be conducted primarily by phone. 
Patient semi-structured interviews will be coded only with a study ID, digitally audio-recorded, and 
uploaded to a secure VA server behind the VA firewall. If recording consent is denied, the researchers 
will take hand written notes, which will be securely transported and stored as described below until they 
are transcribed into an electronic format. When desired by the patient or the interviewer, or to 
accommodate the cognitive needs of the subject, the survey or interview can be performed at two or 
more sessions. At the end of the implementation period we will obtain utilization history through the 
Data Access Request Tracker (DART) system, conducting chart review in the VA Informatics and 
Computing Infrastructure (VINCI) environment.  
 
5.6 Data Analysis 
 
Qualitative Analysis: All semi-structured interviews will be digitally recorded on an VA-approved audio 
device.   The interview content will be transcribed in order to assure that interviewer notes have 
accurately and fully captured interview content.  Approved staff from the VA Salt Lake City (VASLC) 
will transcribe the CCWV audio files. The VASLC has a Professional Transcription Service available 
to VA sites and monitored by their own IRB. The CCWV] audio recordings to be transcribed by 
VASLC staff will be labeled by the subject's unique alphanumeric code and saved behind the VA 
Firewall in CCWV’s secure shared project folder on GLA VA share drive. The VASLC transcription 
staff will be given access to a sub-folder within CCWV’s secure project folder. Approved study staff 
will place a copy of the audio files in this folder for an approved VASLC transcriptionist to access for 
the purposes of transcription. The VASLC transcriptionist will transcribe each interview verbatim and 
save the completed transcript in the sub-folder using the same alphanumeric code. No data (audio 
files, in process transcripts, or completed transcripts) will leave the GLA VA secure research server. 
As completed transcripts become available, approved study staff will move these files from the 
transcription sub-folder into another sub-folder that is only accessible to study staff, where they will 
be stored and accessed for qualitative analyses. 
 
 
Transcripts will be reviewed, edited for accuracy, and summarized by members of the research team. 
Consistent with our team’s approach across multiple projects, matrix analysis methods35 will be used 
for rapid turn-around of the results36 to share with our Strategic Advisory Group. In-depth analysis of 
the qualitative data will be conducted using ATLAS.ti, a qualitative data analysis software program that 
allows for fluid interaction of data across types and sources. Initially, a top-level codebook will be 
developed for the baseline interviews based on the semi-structured interview guide.37   Using a 
constant comparison analytic approach, this codebook will be elaborated upon based on emergent 
themes, and it will be adjusted as each round of interviews is reviewed. Interviews will be compared 
within each clinic, across clinics, and over time. Additional sources of qualitative data (i.e. meeting 
minutes, field notes, and archival information) will also be included in the data set and will be coded 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 33 of 45 
 
 
 
1;lip)_--- 
separately and in relation to the interview data. These multiple approaches and groupings are easily 
facilitated within the software program, which has the capacity to group data in multiple ways and 
which allows the qualitative researchers maximum flexibility in negotiating a complex narrative 
dataset. Of note, the EMPOWER QUERI qualitative team has been working together for over five 
years; all described analytic procedures are well- established and all members of the team have 
expertise with qualitative data analysis and dissemination of results. 
 
In the pre-conditions transcripts, we will identify clinic organization, provider and staff roles, existing 
care arrangements for women Veterans, and perceived acceptability and feasibility of the CCWV 
program. We will synthesize this information with patient survey data to create baseline summaries of 
care as usual, and to tailor our marketing and implementation strategies for use at the sites.  In mid-
implementation interview data, we will identify factors facilitating and impeding implementation of the 
care model, and strengths and weaknesses of the model as implemented. We will assess the extent to 
which components of the model are being implemented, and which components are efficient and easy 
to incorporate into routine care. We will explore whether particular components appear to be of limited 
value in improving care and examine clinic and provider characteristics associated with varying levels 
of care model implementation and effectiveness. In post-implementation interview data, we will take a 
summative approach to characterizing overall experiences of and perspectives on implementation, 
with a particular focus on recommendations for scale up and spread.55 
 
For providers, we will track changes in perceptions about the utility and impact of CCWV on clinical 
practice from pre-conditions to maintenance. We will also be interested in the experiences of the care 
managers and other stakeholders with regard to working specifically with women, eliciting their 
preferences, providing the CBT, and coordinating with MH providers. For patients, we will compare 
and contrast data from those who selected different treatment options.  In order to evaluate the patient 
impact of CCWV, we will assess changes in patient perceptions of their MH symptoms, self-efficacy, 
activation, health-related quality of life and functioning.  We will also collect information on 
demographics, trauma history, health care, mental health care, and experiences of care. 
 
 
Quantitative Analysis: We will use the modified stepped wedge design to evaluate the implementation 
across the providers at four sites. CCWV will be defined as “turning on” when a provider refers a 
patient to the CCWV care manager. We will model the effect of CCWV implementation on referral 
rates, while controlling for organizational level, provider level and patient level covariates. 
 
In addition to project key personnel, the overall EMPOWER QUERI Implementation Core will be 
involved in data analysis.  The Implementation Core is directed by Dr. Hamilton (EMPOWER QUERI 
PI), an experienced and highly trained implementation scientist. She will be supported by a seasoned 
team of Co- Investigators, many of whom also have implementation research experience. Drs. 
Hamilton and Moreau will lead the qualitative analysis. The implementation core personnel are 
supported by several internationally recognized implementation scientists on the Scientific Advisory 
Group, including Drs. Curran, Mittman, and Yano. 
 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 34 of 45 
 
 
 
1;lip)_--- 
Impact-focused evaluation: Secondary aims for the project focus on factors that empower Women 
Veterans to engage in and benefit from care. We will use generalized linear mixed models (GLM) to 
evaluate a) cross-sectional relationships of patient activation, health-related quality of life, and care 
experiences at enrollment, with provider and site characteristics, adjusting for patient social and 
demographic characteristics; and b) prospective changes in patient activation, health-related quality of 
life, and care experiences from enrollment to 6-month follow-up, adjusting for patient, provider, and 
site characteristics. We will also construct multi-level mediation and moderation models to explore 
whether engagement-related factors such as patient activation, strength of treatment preferences, or 
the communication subscale of the CAHPS are associated with greater benefit from or satisfaction 
with care. 
 
Power Analysis: Our goal with the power analysis was to ascertain the number of providers who would 
need to “turn on” the intervention (i.e., the number of referrals to the CCWV care manager), as well as 
the minimum number of patients that would need to be exposed to the care model. Consistent with 
our modified stepped wedge design, the power analysis presented here is for a 3-level hierarchical 
linear model where patients are clustered within PCPs within sites. In the final analysis structure, 
patients are the Level 1 units, the timing of PCP template use/referrals represent Level 2 units, and 
sites are the Level 3 units. The number of patients who completed the care model sessions out of all 
patients referred). The outcome is binary (referred/retained vs. not), and measured at the patient level 
(Level 1). Treatment effect is measured at Level 2, i.e., timing of when the PCPs turn on. Parameters 
required for calculating power are: alpha (α) = 0.05, sample size of patients clustered within referring 
PCPs (n) = at least 260 (based on prior work), the number of sites (K) = 4, the Plausible for Retention 
among those with low utilization (PI) = 5% to 85%, and Effect Size Variability (ESV) = medium or 0.05. 
Based on the resulting power curve, we need at least 8.78 (rounded to 9) referring PCPs per site to 
achieve power at 0.80. Our power analyses assume that the treatment effect is linear over time, i.e. 
patients that are retained longer have proportionally better outcomes. While selected details of the 
resulting dose response curve may vary, sensitivity analyses demonstrate a reasonable 
approximation enabling detection of treatment effects. 
 
In addition to project key personnel, the overall EMPOWER QUERI Implementation Core will be 
involved in data analysis.  The Implementation Core is directed by Dr. Hamilton (EMPOWER QUERI PI), 
an experienced and highly trained implementation scientist. She will be supported by a seasoned team 
of Co- Investigators, many of whom also have implementation research experience. 
 
 
5.7 Withdrawal of Subjects 
The PI may terminate a participant in the study at any time if the study participant does not meet 
enrollment criteria or does not comply with study requirements, e.g., keeping scheduled appointments 
with the interview team.   
 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 35 of 45 
 
 
 
1;lip)_--- 
Participants may withdraw at any time with no consequences.   
 
Information obtained up to the time of participant withdrawal or termination may be used in the data 
analysis.  
5.0 
Reporting 
 
 
Although we do not expect our project to elicit distress among participants, it is possible that some 
patient participants will experience distress at the recall of certain experiences. Interviewers will be 
trained to respond to indications of distress by redirecting the participant to another area of the 
interview guide. Should the participant continue to indicate distress, the interviewer will ask the 
participant if she/he needs to speak to a mental health professional; local contact information of such 
professionals will be available to all interviewers and will be utilized if necessary.  
 
Under any adverse circumstance, the Coordinating PI will notify local PIs of any actions taken in 
response to the indication of distress, and regarding any unanticipated serious adverse events. In 
addition, the IRB will be notified according to guidelines if there is an adverse event. 
 
The site PI and, in her absence, the Co-Principal Investigator will differentiate serious from non-serious 
adverse events. Upon discovery, all research-related deaths will be immediately and verbally reported 
to the CIRB and the Principal Investigator followed by written notification within 5 working days after 
being made aware of the death.  All local serious unanticipated (unexpected) adverse events that are 
related to the research will be reported to the Central IRB within 5 working days. Unanticipated 
problems that represent a risk to participants and/or others will be reported to the CIRB and the 
coordinating center within 5 working days.  Protocol deviations that substantively affect subjects’ rights 
or safety, or potentially compromise facility human research protection will also be reported to the 
Central IRB within 5 working days. An annual report summarizing all adverse events and unanticipated 
problems/protocol deviations that did not require immediate reporting will be prepared and reported to 
the Central IRB at the time of continuing review. 
 
Response procedures for adverse events 
 
Discomfort with disclosure. Research staff who collect survey and interview data will be trained on how 
to respond to embarrassment or discomfort in an appropriate and compassionate manner. Participants 
will be encouraged to contact the care manager, site PI or other research staff in the event of a 
potential adverse reaction that occurs as a consequence of their participation in the survey or interview. 
They will receive specific written (in the form of a study description and contact information) and verbal 
instructions during the study consent procedures about how to do so, if needed.  
Mandatory reporting. One adversity a participant may encounter is the possibility that staff must report 
to authorities instances of physical abuse or neglect of a child, dependent or elderly person; or threat of 
physical harm to the subject or others. To anticipate these concerns, the study has established 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 36 of 45 
 
 
 
1;lip)_--- 
procedures and guidelines to respond to risk disclosures and crisis situations. Staff will be trained to 
recognize risks or crises that require immediate reporting response.    
We expect that discussion of some of the mental health issues during the course of the research 
assessments may possibly result in some emotional or psychological stress.  If that does occur, the RA 
or interviewer will stop the research process, check on the patient, and inform the patient that she will 
call the PI or designated local clinical staff contact (e.g., care manager) for assistance. 
Each of the possible adverse events is described below, including background, criteria for emergency 
action, and non-emergency action.  
1) Suicidal ideation or intention to harm self  
If a subject endorses thoughts of self-harm when responding to assessments or makes a spontaneous 
disclosure to the research assistant or interviewer at any time, research staff will follow the procedures 
below:  
a. Emergency: in the rare instance that an individual is in imminent danger and needs emergency 
medical or mental health services, research staff will adhere to their medical center’s policies and 
procedures for managing and reporting such events including, if appropriate dialing 911 or the local 
police.   
b. Non-emergency: If a subject spontaneously reports that they are thinking of killing themselves or 
taking their life, for example:  
1. “I’m thinking about suicide/killing myself”  
2. “I feel like I want to kill myself”  
3. “I wish I were dead”  
4. “I wish my life were over”  
5. “I can’t go on” or “I feel I can’t go on”  
The research staff person will say, “You seem to be having a difficult time. If you think this is an 
emergency, we can call 911. If you think you need help with this problem, you can talk to your VA 
clinician or go to the VA emergency room.”  The research staff person will contact the designated VA 
clinician.  The clinician will follow standard clinical care guidelines, assessing a level of suicidality and 
establishing a safety plan as needed 
The research staff person will call either the PI or their delegate within one hour of this incident. They 
will determine what further action is needed.  
2) Intent to harm others  

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 37 of 45 
 
 
 
1;lip)_--- 
Research staff will not be probing for this information and it is not part of any of the research tasks. 
Sometimes threats may be voiced spontaneously. If this is a vague statement about intent to harm 
others and it is not an easily identifiable person, and does not communicate clear intent, capable 
means or immediate intent to do so, the research staff person will redirect the subject to the task at 
hand. However, if the subject makes a specific threat to a reasonably identifiable person, then the 
research staff person will remind the subject about the limits of confidentiality, and follow these 
procedures:  
a. Emergency: In the rare instance that an individual is in imminent danger and needs emergency 
medical or mental health services, the research staff will adhere to their medical center’s policies and 
procedures for managing and reporting such events including, if appropriate dialing 911 or the local 
police.  
b. Non-emergency: If a subject reports intent to commit harm to others that is a non-emergency in 
nature but that still communicates a reasonably identifiable person, clear intent, capable means or an 
immediate intent to do so, the research staff person will contact the PI within one hour and will 
complete an incident report the same day. In addition, the research staff person will talk to the subject 
to express concern and encourage him/her to tell someone. They will determine what further action is 
needed.  
3) Child or dependent abuse/neglect  
Research staff will not be probing for this information and it is not part of any of the research tasks. 
However, if a subject discusses that a child, dependent or elder is being sexually, physically, or verbally 
abused or there is evidence of neglect (e.g., young child left alone for hours), then we will follow these 
procedures:  
a. Emergency: In the rare instance that a child or dependent is in imminent danger, the research staff 
will adhere to their medical center’s policies and procedures for managing and reporting such events 
including, if appropriate dialing 911 or the local police.  
b. Non-emergency: If there is a report or incident of child or elder abuse that is non-emergency in 
nature, the research staff person will consult with the PI or delegate by phone within one hour of the 
incident and will complete a written incident report the same day.  
Consulting with the VA Research Service, and VA legal representatives as needed, we will review what 
is known about each situation and will assess the legal and ethical characteristics of each situation. If it 
is determined that the study has a legal or ethical obligation to report the incident, we will do so.  
Reporting procedures of the VA Research Service will be followed for each incident.  
 
6.0 
Privacy and Confidentiality 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 38 of 45 
 
 
 
1;lip)_--- 
Assurance of Patient Safety 
The local site research staff will be in regular communication with the research team at the main site 
and are responsible for monitoring patient safety at the participating sites.  The Project Director will 
monitor research operations of the project and oversee patient safety.  All research staff will be 
experienced in working with women Veteran patients and will be closely supervised by the study local 
site PI.  Since a Data Safety Monitoring Board (DSMB) is required for multi-site studies, we plan to 
utilize the central HSR&D DSMB for further oversight.  The DSMB provides an ongoing evaluation of 
the study’s progress including patient accrual and retention, monitoring of adverse events, and the 
adequacy and efficiency of the analysis plan to discern outcomes that might require study 
modifications, or result in early cessation of the study due its benefits or harms.  In terms of patient 
safety, the DSMB will evaluate all adverse and serious adverse events, by control and intervention site, 
including all hospitalizations and deaths. 
Data Security 
VA Research staff use VA computers to collect and analyze study data; either desktop or laptop 
computers with the VA image installed and managed by the local Information Resource Management 
department.  All laptops are encrypted with VA-approved FIPS 140-2 certified encryption software to 
ensure that if laptops are ever lost or stolen, no data could be removed.  Laptop computers are never 
removed from VA grounds without supervisory approval and a VA Property Pass approved by the VA 
research service.  Thumb drives are never used as long-term storage of data with identifiers.  If a 
thumb drive is needed to transfer data from one computer to another, only a VA-approved thumb drive 
with the proper encryption software would be used after obtaining authorization from the local ISO. 
VA staff practice regular and secure data backup procedures.  Computer users “lock” their workstations 
when not actively in use, and office doors are closed and locked when staff members are not present.  
Hard copies of sensitive data are stored separately from identifiers in locked filing cabinet in a locked 
office.  All research staff have access to a secure VA server managed by the HSR&D CSHIIP.  This 
server is behind the VA firewall and provides individual storage space for each project, and for each 
member of the study teams.  VA staff regularly backup data stored on computers to this server to 
protect those data in the case of an original data failure. This server is backed up incrementally on a 
daily basis with full backup occurring monthly.  Backups go to a tape library and the tapes are rotated 
on a quarterly basis and stored in the safe located in the ISO’s office at the Sepulveda campus.  
This server also provides a means to transfer data between local and non-local research staff.  Project 
folders can be created where any approved individual on the VA intranet can gain access through their 
approved VA login to a project server folder.  This allows secure data transfers over short and long 
distances with the smallest risk of accidental data disclosure.  Only staff that are listed on delegated 
authorities for the project through the CIRB, are credentialed by Research, have completed all required 
data security training, and are approved by the CSHIIP will be given access to the research project 
folders on the research server.  Staff may only access this folder by using their VA login on a VA 
computer behind the VA firewall. Access to these files is immediately revoked for staff members who 
leave the project.  

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 39 of 45 
 
 
 
1;lip)_--- 
Data Collection Procedures 
Staff data includes a masterlist, consents, and audio and transcription from interviews.  Patient data 
includes a masterlist, consents, surveys, audio and transcription from interviews, CALM web-tool data, 
and data extracted from medical records.    
A series of identification (ID) numbers will be generated for all patient and staff participants. A 4 digit 
study ID code will be generated for each participant and project staff will maintain a link file that 
contains the following for each participant: ID code, gender, name, site, and wave number. This is how 
the 4 digit study ID codes will be generated for each participant: 1st digit=wave #, 2nd digit=site #, Last 
2 digits= random # for unique respondent code.  For staff, the linking file will contain: ID number, 
gender and name.  This link file will be stored on a secure VA server behind the VA firewall.  All 
information linking subject identifiers to their data will be destroyed in accordance with the VA Records 
Control Schedule and guidance from the Central IRB.   
Data from all participants will be collected in a private office at each site or private room by phone by a 
trained member of the project team.  Project staff will maintain a link file that contains for each subject 
or participant: (a) for patients: ID number, subject name, last four of the social security number; (b) for 
staff: ID number and name. This link file will be stored on a secure VA server behind the VA firewall.   
Consents are also collected in paper form and list patient subject's name and social security number 
(no SSNs are collected from staff participants).  The paper consents will be stored in a locked file 
cabinet within a secured room at the local site.  A limited number of authorized project staff have 
access to these consent forms.  The study master list has the subject ID number, SSN, full name and 
contact information. This master list is only accessible to the PI, co-PI, and limited project staff.  The 
master list is stored on a VA desktop or encrypted laptop and a backup copy is stored on a VA server 
behind the VA firewall. 
 
Social security numbers are not collected from staff participants; we are only collecting their VA contact 
information. 
 
Patient survey responses will be coded with the study ID and recorded directly into a survey program 
running on a VA computer and stored in a project-specific folder on the CSHIIP secure server. 
The VA CALM web-tool stores information entered by the patient during CALM CBT sessions (e.g., 
behavioral goals) and also tracks scores on mental health assessments such as the PHQ-9 that may 
be completed during sessions for follow-up over the course of participation.  The VA version of CALM 
was developed for the on-going CREATE CALM project (PI, Dr. Michael Cucciare, Little Rock VA) and 
have been made available to the CCWV project on a secure VA server in Little Rock for use by care 
managers during CALM CBT sessions with patients.    
 
Patient information collected in the CALM web-tool is considered part of routine clinical care, and will be 
stored on a single-secure VA server at North Little Rock VA. 
 
Audio and Transcription from Interviews.  All staff participants and a subset of patient participants 
will also complete a semi-structured interview with a research staff member. With the participants’ 
permission, the interviews will be recorded on VA-approved devices and uploaded to the project-

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 40 of 45 
 
 
 
1;lip)_--- 
specific folders.  Data on patient participants extracted from VA data systems, will also be stored in 
project folders. Paper data collected will be entered and verified, and securely stored in a locked 
cabinet in a locked office at the local HSR&D site.   
Any paper data collected from participants will have only the subject’s ID number and no identifying 
information.  No data will be collected outside the VA. Paper data containing PHI is never left out on 
unattended desks. 
Data Transportation Procedures  
If paper data and audio recording devices need to be transported between sites they will be securely 
carried in VA locked bags. Once on site, they will be secured in a locked file cabinet in a secure room.  
Data Transmission Procedures 
Paper surveys from local sites may be faxed to GLA.  Local staff will coordinate with GLA staff to 
arrange a time when the fax machine can be monitored by GLA project staff to secure the data.  The 
qualitative interviews will generate digital voice files, which are considered identifiable. To minimize risk, 
during recorded interviews subjects will only be identified using their project ID number, and subjects 
are instructed not to state any names or other identifying information in their interviews.  These audio 
files will be uploaded from recorders to the secure server The transcription company will be informed 
not to transcribe any identifying information accidentally disclosed in interviews. Any identifying 
information found in voice files from the qualitative interviews is accidental and rare; no identifying 
information will be found in transcripts. The study team downloads the transcription files to the project-
specific folders on the VA server behind the VA firewall. Once they have been downloaded and verified, 
the transcription company destroys the copies on their own server.  
 
Access, Storage and Handling of Secondary Data  
In addition to surveys and interviews, other outcome data will be extracted from national VA data 
systems, and patient medical record charts.  Data from VA electronic data systems will initially include 
patients’ social security numbers (SSN) along with any other identifiers, since these are used to assure 
that the data matches the project participants.  We will merge the extracted data with our existing 
research data. Once the data have been checked to ensure the merge has been successful, within the 
shortest possible time period, the SSNs will be removed from the analytic data file and stored in a link 
file kept in a separate, high security folder on secure VA server.   Each subject will already have been 
assigned a project ID number that does not contain any identifiers. The subject’s ID number will replace 
the SSN in the files.  All information linking subject identifiers to their data will be destroyed in 
accordance with the VA Records Control Schedule and guidance from the CIRB. 
VA records data will be obtained from CPRS, DSS or CDW.  In order to obtain data, the project team 
will work with a list of SSNs of patients consented and enrolled in the study. Project or VA-certified 
database analysts will be given this list of SSNs and will then run a query to extract the needed data, 
either working in the VINCI platform or downloading the data to a temporary folder on the CSHIIP 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 41 of 45 
 
 
 
1;lip)_--- 
secure server.  Extracted data will be placed into our secure folder on the local CSHIIP server and 
coded for analysis. This data will always remain behind the VA firewall.   
All the above data are stored and backed up on a secure VA server.  Computer users “lock” their 
workstations when not actively in use and office doors are locked when staff members are not present.  
Hard copies of sensitive data are stored in a separate locked filing cabinet in a locked office.   
Special consideration and precautions will be taken with regard to sharing information provided by 
individual employees with supervisors and co-workers.  While general information derived from 
employee interviews related to relevant clinical practices, resources, quality improvement activities, and 
barriers/facilitators will be shared with other site participants in aggregate or synthesized form, detailed 
information that would allow identification of individual employee responses will not be shared with 
supervisors of the employees or their co-workers.   
 
 
 
7.0 
Communication Plan 
 
The CCWV team is comprised of a PI (Hamilton) with strong implementation science and mixed 
methods expertise, a Co-PI (Lang) with an exceptional background in CALM, CBT, and trial research, 
and several Co- Investigators who are subject matter experts in both women Veterans’ MH and 
implementation science. Of note, one of our Co-Investigators, Dr. Strauss, is also the National 
Women’s Mental Health Program Manager, thus providing our project with a direct and consistent link 
to one of our key partners (WMHS; McCutcheon LoS). Drs. McCutcheon (WMHS) and Post (PC-MHI) 
are highly supportive of our approach to further developing collaborative care for women Veterans. Dr. 
Post, in particular, has guided our team in our conceptualization of blending models and incorporating 
elements of CALM. Reflecting their strong commitment to the proposed work and important policy-
relevant information to be gained, these partners have worked closely with us to develop this study 
and will remain active partners throughout the course of this work. They are eager for the CCWV 
Implementation Playbook to guide future PC-MHI efforts. 
 
The project headquarters will be at the VISN 22 HSR&D CSHIIP offices in Los Angeles.  Alison 
Hamilton, PhD, MPH, is a medical anthropologist at the Greater Los Angeles VA, and Associate 
Research Anthropologist at UCLA.  She will provide leadership of this study as the coordinating PI and 
will be responsible for all aspects of this project.   
 
The Coordinating Principal Investigator will be responsible for informing local sites of protocol changes, 
reporting compliance issues related to CIRB and VA research service requirements, for reporting 

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 42 of 45 
 
 
 
1;lip)_--- 
progress of research at least annually, and for reporting any injuries or other unanticipated problems 
involving participants or others.   
 
Local Site leads are responsible for obtaining local approvals, complying with local policies, including 
notifying local facility directors of research engagement.   
 
Each local site lead will ensure that the study is being conducted according to the IRB-approved 
protocol.  Issues will be discussed with the Corresponding PI. 
 
Each local site will hold its own regular meeting.  Site leads will be responsible for tracking local events, 
complying with local reporting requirements, and reporting all events to the Coordinating PI.   
 
On a regular basis, Dr. Hamilton will meet with Local Site leads.  On a monthly basis, calls will be held 
with all leads to ensure cross-level synergies and consistency in qualitative approaches to data 
collection and analysis.  Enrollment, data collection, and administrative issues will also be discussed.  
Issues may include IRB, protocol changes, informed consent, and HIPAA.   
 
Any events that may impact the conduct of the study or the safety of participants will be reported by 
local staff to the Coordinating PI within 1 business day.  This includes any SAEs, Unanticipated 
Problems, concerns about protocol deviations, and any interim results that may impact the conduct of 
the study. 
 
Although we do not anticipate any SAEs/UAPs if any safety issues (SAEs or UAPs) arise, they will be 
reported to the Coordinating PI within 1 business day.  The PI will follow CIRB reporting guidelines and 
collect relevant information.  Data collection occurs at one point in time; therefore, it is unlikely we will 
become aware of any SAEs.  If we become aware of any SAEs, we will collect the information by 
phone.  Issues or interim results that may impact the conduct of the study will be reported by the 
Coordinating PI to Local Site PIs on weekly calls. 
 
Sites will be informed when the study is over and they are no longer engaged in research.    The 
coordinating PI will report to CIRB.  Local Site PIs are responsible for reporting to local authorities.   

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 43 of 45 
 
 
 
1;lip)_--- 
References 
 
1. Hamilton A, NC M, Oishi S, et al. Mental health needs and diagnoses among women Veterans 
receiving Veterans Health Administration health care. VA Women's Health Services Research 
Conference on Enhancing Partnerships for Research and Care for Women; 2014.  
2. Maguen S, Ren L, Bosch JO, Marmar CR, Seal KH. Gender differences in mental health diagnoses 
among Iraq and Afghanistan veterans enrolled in veterans affairs health care. Am J Public Health. 
2010;100(12):2450-2456.  
3. Kimerling R, Gima K, Smith MW, Street A, Frayne S. The Veterans Health Administration and 
military sexual trauma. Am J Public Health. 2007;97(12):2160-2166.  
4. Valdez C, Kimerling R, Hyun JK, Mark HF, Saweikis M, Pavao J. Veterans Health Administration  
mental health treatment settings of patients who report military sexual trauma. J Trauma Dissociation. 
2011;12(3):232-243.  
5. Dichter ME, Cerulli C, Bossarte RM. Intimate partner violence victimization among women veterans 
and associated heart health risks. Womens Health Issues. 2011;21(4 Suppl):S190-194.  
6. Oishi SM, Rose DE, Washington DL, MacGregor C, Bean-Mayberry B, Yano EM. National variations 
in VA mental health care for women veterans. Womens Health Issues. 2011;21(4 Suppl):S130-137.  
7. Gilbody S, Bower P, Fletcher J, Richards D, Sutton AJ. Collaborative care for depression: a 
cumulative meta-analysis and review of longer-term outcomes. Arch Intern Med. 2006;166(21):2314-
2321.  
8.  Adli M, Bauer M, Rush AJ. Algorithms and collaborative-care systems for depression: are they 
effective and why? A systematic review. Biol Psychiatry. 2006;59(11):1029-1038.  
9. Thota AB, Sipe TA, Byard GJ, et al. Collaborative care to improve the management of depressive 
disorders: a community guide systematic review and meta-analysis. Am J Prev Med. 2012;42(5):525-
538.  
10. VHA Handbook. 1160.01.(2008). Uniform mental health services in VA Medical Centers and Clinics. 
Washington, DC: Department of Veterans Affairs, Veterans Health Administration.  
11. Tew J, Klaus J, Oslin DW. The Behavioral Health Laboratory: building a stronger foundation for the 
patient-centered medical home. Fam Syst Health. 2010;28(2):130-145.  
12. Pomerantz AS, Shiner B, Watts BV, et al. The White River model of colocated collaborative care: A 
platform for mental and behavioral health care in the medical home. Fam Syst Health. 2010;28(2):114-
129. 
13. Chaney EF, Rubenstein LV, Liu CF, et al. Implementing collaborative care for depression treatment 
in primary care: a cluster randomized evaluation of a quality improvement practice redesign. Implement 
Sci. 2011;6:121.  
14. Integrating Mental Health Into PACT (IMHIP). Report on Integrating Mental Health Into PACT in the 
VA.  VA Health Services Research & Development, VA Office of Patient Care Services;2013.  

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 44 of 45 
 
 
 
1;lip)_--- 
15. Roy-Byrne P, Craske MG, Sullivan G, et al. Delivery of evidence-based treatment for multiple 
anxiety disorders in primary care: a randomized controlled trial. JAMA. 2010;303(19):1921-1928.  
16. Management of Post-Traumatic Stress Working Group. VA/DoD clinical practice guideline for 
management of post-traumatic stress. Version. 2010;2:149-150.  
17. Kimerling R, Pavao J, Greene L, et al. Access to mental health care among women Veterans: is VA 
meeting women's needs? Med Care. 2015;53(4 Suppl 1):S97-S104.  
18. Wells KB, Sherbourne C, Schoenbaum M, et al. Impact of disseminating quality improvement 
programs for depression in managed primary care: a randomized controlled trial. JAMA. 
2000;283(2):212-220.  
19. Unutzer J, Katon W, Callahan CM, et al. Collaborative care management of late-life depression in 
the primary care setting: a randomized controlled trial. JAMA. 2002;288(22):2836-2845.  
20. Engel CC, Oxman T, Yamamoto C, et al. RESPECT-Mil: feasibility of a systems-level collaborative 
care approach to depression and post-traumatic stress disorder in military primary care. Mil Med. 
2008;173(10):935-940.  
21. Post EP, Metzger M, Dumas P, Lehmann L. Integrating mental health into primary care within the 
Veterans Health Administration. Fam Syst Health. 2010;28(2):83-90.  
22. Craske MG, Stein MB, Sullivan G, et al. Disorder-specific impact of coordinated anxiety learning 
and management treatment for anxiety disorders in primary care. Arch Gen Psychiatry. 2011;68(4):378-
388.  
23. Grubbs KM, Cheney AM, Fortney JC, et al. The role of gender in moderating treatment outcome in 
collaborative care for anxiety. Psychiatric services (Washington, D.C.). 2015;66(3):265-271.  
24. Lang AJ, Norman GJ, Casmar PV. A randomized trial of a brief mental health intervention for 
primary care patients. J Consult Clin Psychol. 2006;74(6):1173-1179.  
25. Glenn D, Golinelli D, Rose RD, et al. Who gets the most out of cognitive behavioral therapy for 
anxiety disorders? The role of treatment dose and patient engagement. J Consult Clin Psychol. 
2013;81(4):639-649.  
26. Sharp A, Nguyen H, Hahn E, et al. The Just Do It Playbook for Implementation Science. HMO 
Research Network Conference; April 1-3, 2014, 2014; Phoenix, Arizona.  
27. Kilbourne AM, Abraham KM, Goodrich DE, et al. Cluster randomized adaptive implementation trial 
comparing a standard versus enhanced implementation intervention to improve uptake of an effective 
re-engagement program for patients with serious mental illness. Implement Sci. 2013;8:136.  
28. Kilbourne AM, Almirall D, Goodrich DE, et al. Enhancing outreach for persons with serious mental 
illness: 12-month results from a cluster randomized trial of an adaptive implementation strategy. 
Implement Sci. 2014;9:163.  
29. Leykum LK, Pugh JA, Lanham HJ, Harmon J, McDaniel RR, Jr. Implementation research design: 
integrating participatory action research into randomized controlled trials. Implement Sci. 2009;4:69.  

Version 4 (06/22/2020) 
VA Central IRB Protocol Template 
Page 45 of 45 
 
 
 
1;lip)_--- 
30. Jordon M, Lanham HJ, Anderson RA, McDaniel RR, Jr. Implications of complex adaptive systems 
theory for interpreting research about health care organizations. J Eval Clin Pract. 2010;16(1):228-231.  
31. Yano EM, Haskell S, Hayes P. Delivery of gender-sensitive comprehensive primary care to women 
veterans: implications for VA Patient Aligned Care Teams. J Gen Intern Med. 2014;29 Suppl 2:S703-
707.  
32. McDaniel RR, Jr., Lanham HJ, Anderson RA. Implications of complex adaptive systems theory for 
the design of research on health care organizations. Health Care Manage Rev. 2009;34(2):191-199.  
33. deKleijn M, Lagro-Janssen AL, Canelo I, Yano EM. Creating a roadmap for delivering gender-
sensitive comprehensive care for women Veterans: results of a national expert panel. Med Care. 
2015;53(4 Suppl 1):S156-164.  
34. Simpson KM, Porter K, McConnell ES, et al. Tool for evaluating research implementation 
challenges: a sense-making protocol for addressing implementation challenges in complex research 
settings. Implement Sci. 2013;8:2.  
35. Averill JB. Matrix analysis as a complementary analytic strategy in qualitative inquiry. Qual Health 
Res. 2002;12(6):855-866.  
36. Hamilton A. Qualitative methods in rapid turn-around health services research. Spotlight on 
Women’s Health. VA HSR&D National Cyberseminar Series December 11, 2013 2013.  
37. Glaser BG, Strauss AL. The Discovery of Grounded Theory: Strategies for Qualitative Research. 
Chicago: Aldine Publishing Company; 1967.