Improving PACT Coordination Across Settings and Services (QUE 15-276)
Phase: NA
Conditions: Coordination, Administrative
Sponsor: VA Office of Research and Development

Official Title of Project: Improving PACT Coordination across Settings and Services
NCT number: 03063294 
Document date: May 26, 2015 
In lieu of a formal study protocol for this project, we are providing the original project proposal from 2015, 
with redaction of staff names from the proposal for privacy reasons. The original project proposal is 
excerpted from its parent VA Care Coordination QUERI Program proposal: “Improving Patient-Centered Care 
Coordination for High Risk Veterans in PACT” (QUE 15-276). In the attached proposal document, the 
Coordination Toolkit and Coaching project is referred to under its official title of “Improving PACT 
Coordination across Settings and Services.” 
Since the project proposal included here was written, changes have been made throughout the five-year 
project period. Accurate and updated documentation of the project has been entered and kept current in the 
ClinicalTrials.gov Protocol Registration.  All trial conduct, amendments, and procedures were tracked in the 
Protocol Registration section of the study record. This project was determined to be non-research by the VA 
Greater Los Angeles Healthcare System’s Institutional Review Board and therefore did not include a formal 
study protocol or statistical analysis plan document. We also did not have an informed consent document, as 
we did not have any human subjects.  
David Ganz, MD, PhD 
Principal Investigator 

 
2 
 
2.0 Implementation Project (IP) #1: Improving PACT Coordination across Settings and 
Services  
2.1 IP#1 Specific Aims  
1) Develop an evidence-based, user-friendly Care Coordination Toolkit and a Care Coordination 
Distance Coaching Manual to improve care for high-risk Veterans.  The Manual will be used as the 
basis for training care coordination coaches. Both the Toolkit and Manual will be based primarily on 
previously tested tools. 
2) Pilot the Care Coordination Toolkit and Distance Coaching Manual at one site and engage 
participating VISNs, medical centers, and clinics. 
3) Use a cluster-randomized evaluation design across three VISNs, randomized at the PACT clinic 
level, to compare the effectiveness of the Care Coordination Toolkit alone to the combination of the 
Care Coordination Toolkit plus distance coaching for improving Veteran experience of care, 
communication and coordination between PACT providers and specialists, and utilization of acute 
care services. 
2.2 IP #1 Background and Rationale 
Failure to coordinate care well is a source of waste in health care ($25-$45 billion in 2011).10 
Excellent coordination of care remains a central tenet and expected benefit of a patient-centered 
medical home, and improvement in coordination of care was one of the motivations behind the 
introduction of VA’s PACT model in 2010.11  Care coordination is particularly critical for Veterans 
seeking care in VA, given that these Veterans have a high number of comorbid conditions and 
higher mental health burden.12  While VA providers benefit from a shared electronic health record in 
an integrated health system, care coordination has been identified as a challenge for VA patients 
with multimorbidity13 and for primary care providers.14,15  Other recent work has identified primary 
care providers’ communication with specialists, a major facilitator of care coordination, as a prevalent 
challenge.16 The passage of the Veterans’ Choice Act has further increased the need to focus on 
care coordination, given the increased involvement of non-VA providers in delivering care to 
Veterans. 
VA Care Coordination Setting:  In VA, most high-risk Veterans are managed in primary care rather 
than a specialty service.17  PACT was expected to improve care coordination by creating the care 
manager role for the PACT teamlet nurse.  However, there have been significant challenges in 
implementing the care manager role as intended. Many of the care coordination challenges involve 
the “medical neighborhood” outside of PACT.9 Care coordination remains an inherently human 
endeavor, consisting of communication between a variety of different individuals who share 
responsibility for the patient, including the patient and caregivers. Technology can support human 
social systems, but those human systems need to be functioning well in order to make best use of 
technological supports. Care coordination challenges of various types have resulted in a wide variety 
of relevant tools. No integrated toolkit that organizes these tools around the overall concept of care 
coordination, however, has yet been developed. The Care Coordination Toolkit will thus consist 
primarily of previously tested tools, but will assemble the tools around key elements of coordination 
that support both interpersonal and technological aspects of care coordination improvement. 
VA initiatives have implemented a number of technological supports with the aim of improving care 
coordination through bi-directional primary care provider-specialist communication.18 A major 
facilitator of care coordination, for example, has been electronic consults that allow a primary care 
provider to query a specialist for guidance without necessitating the patient see the specialist in 
person. The Specialty Care Access Network-Extension for Community Healthcare Outcomes 
(SCAN-ECHO) also allows providers and primary care providers to interact via 
videoteleconferencing. However, adoption of these innovations is dependent on specialist and PACT 

 
3 
 
team participation, and implementation rates have varied among different specialties and across 
sites.19  In addition, these innovations have not been designed or tested specifically around the 
concept of care coordination as an outcome. 
Conceptual Framework: The proposed work is guided by the Framework for Organizational 
Transformation, which identifies five key features of successful, sustained implementation of quality 
improvement activities in organizations: “(1) Impetus to transform; (2) Leadership commitment to 
quality; (3) Improvement initiatives that actively engage staff in meaningful problem solving; (4) 
Alignment to achieve consistency of organization goals with resource allocation and actions at all 
levels of the organization; and (5) Integration to bridge traditional intra-organizational boundaries 
among individual components.”20 Features 1, 2, and 3 are important determinants of success for 
quality improvement programs; features 4 and 5 (alignment and integration) are particularly relevant 
to PACT teams, which need to integrate themselves into the organizational landscape of their 
respective VA healthcare systems and with non-VA care. Later, the Framework for Organizational 
Transformation was distilled into a six-step process used in toolkits for the prevention of pressure 
ulcers21 and falls22 in hospitals: 1) leadership engagement, 2) change management, 3) selecting 
intervention components, 4) implementing the intervention, 5) measuring for improvement, and 6) 
sustainability. The toolkits were designed for interdisciplinary teams at all stages of organizational 
readiness. We will use the six-step approach to systematize our toolkit improvement strategies.  The 
proposed project follows the later framework. 
To address leadership engagement, the project will engage VISN, medical center, and local PACT 
leaders in an assessment and planning process using previously developed EBQI principles. The 
need to engage leadership is critical regardless of strategy, given the repeated finding of leadership 
involvement as a facilitator in implementing any initiative.23 Our leadership engagement, toolkits and 
coaching strategies will each include a focus on change management and intervention component 
selection. One change management approach will be to provide intervention sites with feedback on 
their baseline organizational readiness for care coordination, using the tool developed by the 
Implementation Core.  Given that specific gaps in care coordination are likely to vary locally, we do 
not propose a “one-size-fits-all” intervention here. Rather, each of our two implementation strategies 
will emphasize matching particular coordination solutions to the local coordination problems being 
experienced. To study the process of intervention implementation, we will compare toolkit to toolkit 
plus coaching strategies, and will measure improvement and sustainability based on patient and 
provider experience measures as well as patient utilization of acute care. 
2.3 IP #1 Procedures 
2.3.1 Proposed IP #1 Improvement Strategies  
Toolkit and Coaching Implementation Strategies: Both toolkit and distance coaching strategies have 
been used in a wide variety of VA quality improvement initiatives, including by the National Center 
for Patient Safety, but, to our knowledge, have not formally been compared, although toolkits and 
distance coaching have been compared to other alternatives. Kilbourne and colleagues compared a 
standard package of technical assistance to reengage patients with serious mental illness in VA care 
to an individualized distance coaching strategy focused on sites not responsive to the standard 
package, using an adaptive design.24 Distance coaching resulted in enhanced program uptake by 
sites, but no change in patient-level utilization of mental health services, for reasons that may have 
been related to the patient population under study and the statistically conservative nature of the 
study design. Speroff compared toolkit and collaborative approaches outside VA, and found that a 
collaborative was better at changing processes of care, but neither approach was successful in 
changing outcomes in the intensive care unit setting.25 Both the Kilbourne and Speroff studies differ 
substantially from ours in terms of the problem being investigated and the patients involved, so 
results cannot be generalized to the PACT setting. 

 
4 
 
A toolkit strategy has many aspects in its favor – toolkit deployment is relatively inexpensive and 
makes tools nationally available and accessible over a relatively short time horizon. A potential 
limitation of this strategy, however, is the lack of interaction with an expert to tailor the tools provided 
to the needs and local organizational context of the user, potentially limiting the level of engagement 
with the tools. This limitation is suggested by preliminary PACT Toolkit data from Fiscal Years 2011-
2014 (IP#1 staff member, unpublished data), showing variable uptake of the toolkit by VISN (mean 
number of unique users per VISN ± standard deviation, 731 ± 298; range 302-1386), despite 
aggressive marketing and a PACT Collaborative.26,27 We hypothesize that toolkit uptake would be 
greater overall, and less variable, if individual PACT clinic champions had personal access to an 
expert coach to guide them through use of the tools, allowing these champions to become 
comfortable with the tools and share them with colleagues. Although the original PACT collaborative 
did include coaches, the amount and intensity of coaching was low (five industrial engineers and five 
coaches for the entire national collaborative, with coaching activities performed as a collateral 
duty26). This hypothesis will be tested by adding coaching to the Care Coordination Toolkit. Our 
distance coaching approach will be based on prior tested facilitation and coaching materials and use 
a variety of communication methods (videoconferencing, teleconferencing, instant messaging and 
other means) to provide personalized technical assistance to PACT teams for improving care 
coordination. A distance coaching strategy has the advantage of being easily scaled and sustained 
while making economical use of limited resources.  
An additional principle that is implicit in implementation of successful care coordination improvement 
strategies is that of creating routine systems for communication and organizational alignment that 
connect units that frequently need to coordinate care. 6  The Care Coordination Toolkit will focus on 
creating effective routine systems. For example, in VA, studies have found that care coordination 
agreements were associated with better care coordination experiences for primary care providers 
(manuscript in preparation). However, it matters how these agreements are implemented; in 
interviews, some specialists indicated that these agreements are not treated as binding in everyday 
use (unpublished data). Any gap between appearances and reality suggests a potential 
implementation problem that needs to be tackled in order for care to improve. 
In summary, we will compare the use of a novel Care Coordination Toolkit alone with Toolkit use 
plus facilitation by a remotely-based coaching team. For both strategies being compared, we will 
systematically engage VISN and facility leaders as partners in testing these approaches, using 
principles of EBQI and CPPR.8,28 
2.3.2 IP #1 Study Design 
Specific Aim 1:  Develop a Care Coordination Toolkit and Distance Coaching Manual 
Aim 1 will span October 2015 – September 2016. Development will begin with an environmental 
scan to collect and use relevant tested care coordination tools and coaching materials from existing 
sources.  We will then conduct approximately twenty one-on-one interviews with potential users of 
the identified tools and coaching materials to assess their reactions to the tools. We will also assess 
the utility of harvested tools using the Agency for Healthcare Research and Quality (AHRQ) Toolkit 
Guidance,29 which addresses tool content, usability, organization, design, and language.  Finally, we 
will compare the set of promising tools against the six-step approach (leadership engagement, 
change management, selecting intervention components, implementing the intervention, measuring 
for improvement, and sustainability), to determine where voids in coverage exist. At this phase, new 
tools and coaching content will be created to fill these voids. All materials will then be adapted and 
formatted to create a common look and feel. The draft toolkit will then be deployed by VA-CASE.  
VA Care Coordination Toolkit: An initial review shows several relevant tools in the PACT Toolkit.  
These include the readmission risk calculator (tool 5), PACT Teamlet checklist (tool 20), VA 
Geriatrics Resource Guide (tool 21), PACT Roll-out kit for providers (tool 24), huddle checklist (tool 

 
5 
 
68) and co-managed care toolkit (tool 75). Additionally, the Specialty Care Toolkit’s care coordination 
agreement guide (tool 17) and E-consult guides (tool 28) are relevant. However, all of these tools 
need to be updated given new developments, including the CAN score, the Patient Care 
Assessment System (PCAS), and the Veteran’s Choice Act. In addition, the currently available 
PACT and Specialty Care Toolkits do not include many tools on general management principles and 
approaches, which is a strength of the AHRQ toolkits, including the fall prevention toolkit created by 
Dr. Ganz. These general management tools include obtaining leadership buy-in, assessing available 
resources, team formation and management, audit and feedback, and sustainability. New tools to 
engage patients in improving ambulatory care also need to be added, and could be adapted from the 
Aligning Forces for Quality Initiative.30 Finally, we will identify and adapt tested tools from successful 
projects that have included elements of care coordination, such as those focused on improving 
hospital discharge or care for specific chronic conditions.  
Care Coordination Distance Coaching Manual: Candidate coaching materials include Mental Health 
QUERI’s facilitation guide,31 and the materials hosted on VA’s Transformational Coach SharePoint 
Site. The coaching team that will help develop and then use the manual consists of individuals who 
have experience in working with PACT teamlets, are facile with data, and are expert at building 
relationships via non-face-to-face means. The coach will serve as an external facilitator, working 
closely with an internal facilitator or “champion” at each PACT clinic.31 Two project advisors have 
special expertise in this area and will consult on manual development. 
Specific Aim 2:  Pilot Testing and Site Engagement 
Aim 2 spans October 2016 – March 2017. During this period, the Care Coordination Toolkit and 
Coaching Manual will be pilot tested at the South Texas Veterans Affairs Healthcare System and 
three of its PACT clinics. VISNs, medical centers, and clinics for the cluster randomized evaluation 
will also be identified. 
Pilot Testing: We will pilot test our engagement strategies, toolkits, and coaching approaches. 
Engaged clinics will identify care coordination improvement aims and tools, as well as QI measures. 
Assessment of pilot results will include a usability assessment of chosen tools based on a user focus 
group, and qualitative interviews of project participants from each site aimed at identifying what tools 
or coaching strategies were used and on obtaining feedback on how the approaches could be made 
more helpful. We will also test the coaching manual to see whether it is sufficiently prescriptive to 
actually result in a standardized coaching approach.  Finally, we will conduct cognitive interviews to 
determine whether the implementation core’s readiness assessment instrument can distinguish 
clinics in terms of their readiness to improve care coordination, specifically around whether the 
clinics’ care coordination problems are more pervasive (e.g., insufficient staffing) versus more 
bounded (e.g., inability to get patients in to a particular specialty clinic). The readiness assessment 
will be refined based on the cognitive interviews. Findings from the readiness assessment will be 
used to update toolkit and coaching materials. 
Site Engagement: The two evaluation strategies, once pilot tested, will be implemented and 
evaluated in PACT clinics within three VA VISNs. In parallel with pilot testing, therefore, the site 
engagement process will begin. The engagement process will take account of the VA VISN, medical 
center, and local site contexts, thus recognizing the critical role that network and facility leadership 
plays in facilitating any new VA endeavor aimed at changing routine care. As a first step, the Chief 
Consultant for the Office of Primary Care and the senior project PI, who are experienced in VISN 
engagement through EBQI,7 will identify potential participating VISNs.  As in other EBQI efforts, the 
project PI will then approach the VISN Director and Chief Medical Officer regarding their willingness 
to participate.  Once a VISN agrees to participate, the project team will prepare VISN-specific 
feedback reports based on PACT Compass and Survey of Healthcare Experiences of Veterans 
(SHEP) Patient-Centered Medical Home data relevant to care coordination.  Reports will focus on 

 
6 
 
showing clinic level data for each medium-sized (> 5000 patients) or larger non-contract PACT 
clinics, whether community based or medical center based, in each potentially participating VISN. At 
a videoconference meeting with VISN leaders, the two study PIs will present the data and consider 
with them which medical centers and clinics would be best to engage in care coordination 
improvement.  The PIs will then work with medical center and PACT leaders from identified clinics on 
engaging the individual PACT sites, including provision for needed release time for those at each 
site who will be participating in a care coordination QI team. Physician, nurse, and administrative 
leaders from each clinic will then complete the implementation core’s readiness assessment 
(organizational readiness for care coordination), to assess problems with care coordination and 
readiness to address those problems.  
Specific Aim 3:  Cluster Randomized Evaluation Comparing Two Implementation Strategies 
Aim 3 spans April 2017-September 2018.  During this period, we will conduct a cluster randomized 
evaluation of using prior evidence to improve care coordination via a toolkit alone (six clinics) versus 
the combination of the care coordination toolkit plus individualized distance coaching (six clinics).  
Implementation sites will be distributed across the three VISNs such that each VISN includes two 
clinics for each strategy for a total of four per VISN, or 12 altogether. The care coordination toolkit 
will be marketed to all sites, but only the six clinics randomly assigned to also receive distance 
coaching will receive this supplemental support. 
Initiating the Implementation Strategies: We will introduce the Toolkit to QI teams at each of 12 
PACT clinics using a variety of modalities including email and webinars. For the six clinics randomly 
assigned to receive coaching in addition to the toolkit, the coach will be available on demand, via 
videoconference, webinar, telephone, and instant messaging, depending on the communication 
preferences of the individual clinic champions. Scheduled coaching activities will include an 
introductory coaching call with teams from each of the six clinics, with subsequent meetings initially 
scheduled on a weekly basis. The introductory coaching call will include a presentation about high-
risk patients’ challenges and needs for care coordination and an overview of the toolkit. Subsequent 
meetings will include case reviews, where teams discuss how they would use the toolkit to address 
different types of patient needs. We have budgeted a limited number of site visits to the six coaching 
strategy clinics to augment the distance coaching model; site visits will occur at the discretion of site 
leaders and can be scheduled at any time during the implementation period as needed.  
2.3.3 Key Outcomes and Analyses  
Measuring and Analyzing Care Coordination Outcomes: We will assess each of our two evidence-
based implementation strategies for care coordination (toolkit and toolkit plus coaching) through a 
patient survey, electronic data, historical document review, and key stakeholder interviews.  Through 
these data sources, we will evaluate both processes and outcomes relevant to the two strategies. 
Our primary outcome is patient experience of care coordination, measured based on the 
Multimorbidity Hassles Scale (measured at baseline and follow-up).33 Secondary outcomes include 
provider perceptions of the quality of coordination between primary and other services (measured six 
months in to the implementation via interviews), and electronic administrative data-based measures 
of acute care utilization (ED and hospital visits, and length of stay for hospitalized patients) and of 
patient care experiences.  Due to the continuous nature of the administrative data, we will compare 
all PACT patients at each participating site based on study year one results (baseline) compared to 
study year three results (post-implementation). 
Patient Survey on Experiences of Care Coordination:  The primary outcome (Multimorbidity Hassles 
Scale) will be compared between the two groups (toolkit and combined toolkit/coaching), with a 
minimum of 564 patients per group. The Multimorbidity Hassles Scale was specifically designed and 
tested for VA patients with multiple chronic conditions.33 This 16-item scale asks patients questions 

 
7 
 
such as whether they are getting their medications refilled on time, whether they had information 
about why they were referred to a specialist, whether there has been poor communication between 
different doctors or clinics, or disagreements between doctors about the patient’s diagnosis or the 
best treatment for the patient. Prior work has shown that better scores were associated with better 
ratings of the usual care physician on communication and coordination of care, that patients with 
multimorbidity have higher hassles than patients with a single chronic illness, and that all items load 
onto a single factor when subjected to factor analysis, thus capturing a unified concept. The 
Multimorbidity Hassles Scale will be measured on all patients with CAN score > 90 in participating 
clinics. We will use VA administrative data from the Corporate Data Warehouse (CDW) to determine 
assignment to each clinic and the relevant CAN score for each patient. The scale will be 
administered via computer-assisted telephone interviews from a survey contractor. 
Administrative Data on Patient Experiences of Care Coordination and on Acute Care Utilization:  
Secondary outcomes will include clinics’ overall performance on items routinely collected by VA, 
including coordination of care items found in the SHEP Patient-Centered Medical Home survey, 
which derive from the Consumer Assessment of Healthcare Providers and Systems (CAHPS) 
Patient-Centered Medical Home survey, and PACT compass metrics related to care coordination.  
Although we will identify medium-sized or larger clinics, we may have a limited number of CAHPS 
surveys available.  To mitigate this limitation, we will use control charts to plot change over time, 
because CAHPS is administered throughout the year.  Our team has expertise in this approach 
using CAHPS (IP#1 data analyst, manuscript submitted for publication). 
Key Stakeholder Interviews: Key stakeholder interviews will assess both user ratings of the utility 
and usability of the two strategies as a secondary outcome, and the process of implementation of 
each strategy.  We will conduct semi-structured key stakeholder interviews across all 12 sites 
approximately 6 months after the toolkit is made available. For all sites, we will assess toolkit 
adoption and perceived usefulness, methods used to implement tools (e.g., PDSA cycles), resources 
needed, challenges and facilitators, and perceived impact of tool implementation.  In addition, we will 
ask about other ongoing efforts to improve care coordination, outside of the Care Coordination 
Toolkit. For sites receiving coaching, the interviews will also assess the perceived quality of coaching 
and participation in the coaching calls/activities. All key informants will also be asked about their 
perceptions of the quality of care coordination at their site. Key stakeholders will include the site 
quality improvement teams and leads (up to 5 interviews per site). We will adapt the interview 
protocol used by the National PACT Toolkit team. 
Document Review: In addition to key stakeholder interviews, we will abstract information from 
coaching call minutes to assess participation by the 6 coached sites receiving coaching. Specifically, 
we will collect information on who participated, frequency of coaching calls, and topics discussed. 
From this information we will construct variables to measure “dose” of coaching, such as number of 
calls each site participated in, and how many quality improvement team members participated (on 
average). 
Power and sample size calculations:  With participating clinics required to have a minimum of four 
teamlets, an average PACT teamlet panel of 1200, 10% of that panel defined as “high risk” by CAN 
score criteria (CAN score > 90), and a survey response rate of 40% (the most recently available 
response rate for SHEP’s patient-centered medical home survey was 44.3%35), then we project at 
least 192 survey responders per clinic. For our primary outcome, the Multimorbidity Hassles Scale 
has a mean of 17.7 (standard deviation 14.4) in a sample of patients with multiple chronic illnesses 
(n=227).33 In a design with six clinics assigned to the toolkit group and six clinics to the combined 
toolkit/coaching group, a sample size of 94 responders per clinic is needed to provide 80% power to 
detect a small to moderate effect size of 0.3 standard deviations, assuming an intracluster 
correlation coefficient of 0.01, a type I error rate of 5%, and an attrition rate of 40% at 18 months. 

 
8 
 
Thus, we have sufficient sample to survey a total of 1128 patients (94x6=564 in the toolkit group and 
94x6=564 in the toolkit/coaching group) in the current evaluation.  
Key Covariates. In our analyses of the Hassles Scale (patient survey) as an outcome we will control 
for clustering at the PACT site level and include VISN as a covariate.  We will also include patient-
level factors: gender, age, number of providers longitudinally involved in patients’ care, use of non-
VA care (e.g., fee-basis), and distance to travel for primary care based on the patient’s address of 
record. In our secondary outcome analyses based on electronic data we will use a similar approach 
though only some of the above variables will be available electronically. 
Limitations. Although the proposed design is cluster-randomized, the total number of clusters (six 
clinics per implementation strategy) is small. Thus, quantitative assessment of the effect of clinic-
level factors, such as baseline organizational readiness to coordinate care, on the effectiveness of 
each implementation strategy will not be possible. Second, our PACT sites do not include very small 
clinics (with <5000 patients) and our results cannot be generalized to these.  Our power calculations, 
however, indicate we will be able to rigorously assess our primary outcome. 
2.4 IP #1 Impact  
If successful, the project will improve care coordination in twelve sites in three VISNs, and will 
provide readily-applicable methods for spreading these results throughout VA.  In addition, the 
project will indirectly inform VA regarding factors, including the availability of distance coaching, that 
may influence the utility of toolkits. 
2.5 IP #1 Timeline 
Year 
FY16 
FY17 
FY18 
FY19 
Quarter 
1 
2 
3 
4 
1 
2 
3 
4 
1 
2 
3 
4 
1 
2 
3 
4 
Aim 1: Develop online toolkit and 
coaching manual 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Develop readiness assessment 
X X 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Develop and post online toolkit 
 
X X X 
 
 
 
 
 
 
 
 
 
 
 
 
Develop standardized coaching 
strategy/manual 
 
X X X 
 
 
 
 
 
 
 
 
 
 
 
 
Aim 2: Pilot toolkit and coaching 
strategy and engage VISNs in site 
selection 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Pilot readiness assessment  
 
 
 
 
X 
 
 
 
 
 
 
 
 
 
 
 
Pilot standardized coaching manual 
 
 
 
 
X X 
 
 
 
 
 
 
 
 
 
 
Tool usability focus groups with pilot 
sites 
 
 
 
 
X X 
 
 
 
 
 
 
 
 
 
 
Refine readiness assessment, 
coaching manual, and online toolkit 
per pilot tests 
 
 
 
 
X X 
 
 
 
 
 
 
 
 
 
 
Identify participating VISNs, facilities 
and clinics 
 
 
 
 
X X 
 
 
 
 
 
 
 
 
 
 
Aim 3: Deploy toolkit and 
combined toolkit/coaching 
strategies 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Conduct readiness assessments 
 
 
 
 
 
 
X 
 
 
 
 
 
 
 
 
 
Launch toolkit marketing campaign 
 
 
 
 
 
 
X X X 
 
 
 
 
 
 
 
Implement coaching strategy 
 
 
 
 
 
 
X X X X X X 
 
 
 
 
Analyses 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
9 
 
Field patient experience of care 
survey  
 
 
 
 
 
 
 
X 
 
 
 
X 
 
 
 
 
Conduct interim evaluation of 
implementation and provider 
perceptions of care coordination 
 
 
 
 
 
 
 
 
X 
 
 
 
 
 
 
 
Analysis and final report 
 
 
 
 
 
 
 
 
 
 
 
 
X X X X 
 
Bibliography & References 
1. 
Wang L, Porter B, Maynard C, et al. Predicting risk of hospitalization or death among patients 
receiving primary care in the Veterans Health Administration. Medical Care. Apr 
2013;51(4):368-373. 
2. 
Hong CS, Hwang AS, Ferris TG. California Healthcare Foundation. Finding a Match: How 
Successful Complex Care Programs Identify Patients [online]. 2015. Accessed 5/15/2015. 
3. 
Zulman DM, Ezeji-Okoye SC, Shaw JG, et al. Partnered Research in Healthcare Delivery 
Redesign for High-Need, High-Cost Patients: Development and Feasibility of an Intensive 
Management Patient-Aligned Care Team (ImPACT). Journal of General Internal Medicine. 
Oct 30 2014. 
4. 
Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health affairs. 
May-Jun 2008;27(3):759-769. 
5. 
Agency for Healthcare Research and Quality, Rockville, MD. Chapter 2. What is Care 
Coordination?: Care Coordination Measures Atlas Update [online]. Publication #14-0037-EF. 
2014. Accessed 5/15/2015. 
6. 
Davies GP, Williams AM, Larsen K, Perkins D, Roland M, Harris MF. Coordinating primary 
health care: an analysis of the outcomes of a systematic review. Medical Journal of Australia. 
2008;188(8):S65-68. 
7. 
Rubenstein LV, Stockdale SE, Sapir N, et al. A patient-centered primary care practice 
approach using evidence-based quality improvement: rationale, methods, and early 
assessment of implementation. Journal of general internal medicine. 2014;29(2):589-597. 
8. 
Jones L, Wells K. Strategies for academic and clinician engagement in community-
participatory partnered research. Jama. 2007;297(4):407-410. 
9. 
Taylor EF, Lake T, Nysenbaum J, Peterson G, Meyers D. Coordinating care in the medical 
neighborhood: critical components and available mechanisms. White Paper (Prepared by 
Mathematica Policy Research under Contract No. HHSA290200900019I TO2). AHRQ 
Publication No. 11-0064. Rockville, MD: Agency for Healthcare Research and Quality; 2011. 
10. 
Berwick DM, Hackbarth AD. Eliminating waste in US health care. Jama. 2012;307(14):1513-
1516. 
11. 
Patient Centered Primary Care Implementation Work Group. Patient Centered Medical Home 
Model Concept Paper: Veterans Health Administration Primary Care Services. 
12. 
Yoon J, Scott JY, Phibbs CS, Wagner TH. Recent trends in Veterans Affairs chronic 
condition spending. Population health management. 2011;14(6):293-298. 
13. 
Noel PH, Frueh BC, Larme AC, Pugh JA. Collaborative care needs and preferences of 
primary care patients with multimorbidity. Health Expect. Mar 2005;8(1):54-63. 
14. 
Yano EM, Fleming B, Canelo I, et al. National Survey Results for the Primary Care Director 
Module of the VHA Clinical Practice Organizational Survey. Technical Monograph #08-
CPOS02. Sepulveda, CA: VA HSR&D Center for the Study of Healthcare Provider Behavior; 
2008. 
15. 
VHA Primary Care Survey: National Survey Results. Sepulveda, CA: VA HSR&D Center of 
Excellence for the Study of Healthcare Provider Behavior; 2009. 

 
10 
 
16. 
Zuchowski JL, Rose DE, Hamilton AB, et al. Challenges in Referral Communication Between 
VHA Primary Care and Specialty Care. J Gen Intern Med. Mar 2015;30(3):305-311. 
17. 
Chang ET, Wang M, Kirsh S, Rubenstein LV. VA High-Risk Populations in Primary Care. 
Mineapolis, MN: Accepted for presentation at AcademyHealth Annual Research Meeting; 
2015. 
18. 
Foy R, Hempel S, Rubenstein L, et al. Meta-analysis: effect of interactive communication 
between collaborating primary care physicians and specialists. Ann Intern Med. Feb 16 
2010;152(4):247-258. 
19. 
Cordasco KM, Zuchowski JL, Hamilton AB, et al. Early lessons learned in implementing a 
women's health educational and virtual consultation program in VA. Med Care. Apr 
2015;53(4 Suppl 1):S88-92. 
20. 
Lukas CV, Holmes SK, Cohen AB, et al. Transformational change in health care systems: an 
organizational model. Health Care Manage Rev. Oct-Dec 2007;32(4):309-320. 
21. 
Berlowitz D, VanDeusen Lukas C, Parker V, et al. Agency for Healthcare Research and 
Quality, Rockville, MD. Preventing Pressure Ulcers in Hospitals: A Toolkit for Improving 
Quality of Care [online]. Publication # 11-0053-EF 2014. Accessed 5/15/2015. 
22. 
Ganz DA, Huang C, Saliba D, et al. Agency for Healthcare Research and Quality, Rockvill, 
MD. Preventing Falls in Hospitals: A Toolkit for Improving Quality of Care [online]. Publication 
# 13-0015-EF. 2013. Accessed 5/15/2015. 
23. 
Alexander JA, Hearld LR. The science of quality improvement implementation: developing 
capacity to make a difference. Medical care. 2011;49:S6-S20. 
24. 
Kilbourne AM, Almirall D, Goodrich DE, et al. Enhancing outreach for persons with serious 
mental illness: 12-month results from a cluster randomized trial of an adaptive 
implementation strategy. Implementation Science. 2014;9(1):163. 
25. 
Speroff T, Ely EW, Greevy R, et al. Quality improvement projects targeting health care-
associated infections: comparing Virtual Collaborative and Toolkit approaches. J Hosp Med. 
May 2011;6(5):271-278. 
26. 
Bidassie B, Davies ML, Stark R, Boushon B. VA experience in implementing Patient-
Centered Medical Home using a breakthrough series collaborative. J Gen Intern Med. Jul 
2014;29 Suppl 2:S563-571. 
27. 
Luck J, Bowman C, York L, et al. Multimethod Evaluation of the VA’s Peer-to-Peer Toolkit for 
Patient-Centered Medical Home Implementation. Journal of general internal medicine. 
2014;29(2):572-578. 
28. 
Rubenstein LV, Meredith LS, Parker LE, et al. Impacts of Evidence-Based Quality 
Improvement on Depression in Primary Care: A Randomized Experiment. Journal of general 
internal medicine. 2006;21(10):1027-1035. 
29. 
Agency for Healthcare Research and Quality, Rockville, MD. Section 6: Toolkit Guidance: 
AHRQ Publishing and Communications Guidelines [online]. Publication # 13-0009-EF. 2013. 
Accessed 5/15/2015. 
30. 
Aligning Forces for Quality. Robert Wood Johnson Foundation, Princeton, NJ. Engaging 
Patients in Improving Ambulatory Care: A Compendium of Tools for Maine, Oregon, and 
Humboldt County, California [online]. 2013. Accessed 5/18/2015. 
31. 
Kirchner JE, Ritchie M, Dollar K, Gundlach P, Smith J. Implementation Facilitation Training 
Manual: Using External and Internal Facilitation to Improve Care in the Veterans Health 
Administration. Version 1 [online]. Accessed 5/15/2015. 
32. 
Veterans Health Administration Employee Education System. T-21 Integrated Curriculum 
Workspace: Transformational Coaching Work Area [online]. Accessed 5/18/2015. Last 
updated 5/7/2015. 
33. 
Parchman ML, Noel PH, Lee S. Primary care attributes, health care system hassles, and 
chronic illness. Med Care. Nov 2005;43(11):1123-1129. 

 
11 
 
34. 
Agency for Healthcare Research and Quality, Rockville, MD. CAHPS Clinician & Group 
Patient-Centered Medical Home Survey [online]. Accessed 5/15/2015. 
35. 
VHA Office of Analytics and Business Intelligence, Washington, D.C. Patient Expereices 
Dimensions of Care, Outpatient Reports [online, VA intranet]. 2013. Accessed 5/15/2015. 
36. 
Kripalani S, Jackson AT, Schnipper JL, Coleman EA. Promoting effective transitions of care 
at hospital discharge: a review of key issues for hospitalists. J Hosp Med. Sep 2007;2(5):314-
323. 
37. 
Moore C, Wisnivesky J, Williams S, McGinn T. Medical errors related to discontinuity of care 
from an inpatient to an outpatient setting. Journal of general internal medicine. 
2003;18(8):646-651. 
38. 
Nunez S, Hexdall A, Aguirre-Jaime A. Unscheduled returns to the emergency department: an 
outcome of medical errors? Qual Saf Health Care. Apr 2006;15(2):102-108. 
39. 
Kelly AM, Chirnside AM, Curry CH. An analysis of unscheduled return visits to an urban 
emergency department. N Z Med J. Aug 11 1993;106(961):334-336. 
40. 
Pierce JM, Kellerman AL, Oster C. "Bounces": an analysis of short-term return visits to a 
public hospital emergency department. Ann Emerg Med. Jul 1990;19(7):752-757. 
41. 
Vukmir RB, Kremen R, Ellis GL, DeHart DA, Plewa MC, Menegazzi J. Compliance with 
emergency department referral: the effect of computerized discharge instructions. Ann Emerg 
Med. May 1993;22(5):819-823. 
42. 
Hastings SN, Smith VA, Weinberger M, Schmader KE, Olsen MK, Oddone EZ. Emergency 
department visits in Veterans Affairs medical facilities. The American journal of managed 
care. 2011;17(6 Spec No.):e215-223. 
43. 
McDonald KM, Sundaram V, Bravata DM, et al. AHRQ Technical Reviews. Closing the 
Quality Gap: A Critical Analysis of Quality Improvement Strategies (Vol. 7: Care 
Coordination). Rockville (MD): Agency for Healthcare Research and Quality (US); 2007. 
44. 
Rogers EM. Diffusion of Innovations, 5th Edition: Free Press; 2003. 
45. 
Martino S, Ball SA, Nich C, Canning-Ball M, Rounsaville BJ, Carroll KM. Teaching community 
program clinicians motivational interviewing using expert and train-the-trainer strategies. 
Addiction. 2011;106(2):428-441. 
46. 
Gadomski AM, Wolff D, Tripp M, Lewis C, Short LM. Changes in health care providers' 
knowledge, attitudes, beliefs, and behaviors regarding domestic violence, following a 
multifaceted intervention. Acad Med. Oct 2001;76(10):1045-1052. 
47. 
Solberg LI, Kottke TE, Brekke ML, Calomeni CA, Conn SA, Davidson G. Using continuous 
quality improvement to increase preventive services in clinical practice—going beyond 
guidelines. Preventive medicine. 1996;25(3):259-267. 
48. 
Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual 
distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. Mar 
2011;38(2):65-76. 
49. 
Bell K, McKinsey K, Namboothiri M, et al. VISN 22 Veterans Assessment Improvement 
Laboratory for Patient-Centered Care. Super Huddles: Addressing High Utilizers Toolkit 
[online, VA intranet]. Accessed 5/18/2015. 
50. 
Thor J, Lundberg J, Ask J, et al. Application of statistical process control in healthcare 
improvement: systematic review. Quality and Safety in Health Care. 2007;16(5):387-399. 
51. 
Kangovi S, Barg FK, Carter T, et al. Challenges faced by patients with low socioeconomic 
status during the post-hospital transition. J Gen Intern Med. Feb 2014;29(2):283-289. 
52. 
Kangovi S, Levy K, Barg FK, Carter T, Long JA, Grande D. Perspectives of older adults of 
low socioeconomic status on the post-hospital transition. J Health Care Poor Underserved. 
May 2014;25(2):746-756. 
53. 
Stephens C, Sackett N, Pierce R, et al. Transitional care challenges of rehospitalized 
veterans: listening to patients and providers. Popul Health Manag. Oct 2013;16(5):326-331. 

 
12 
 
54. 
Kangovi S, Mitra N, Grande D, et al. Patient-centered community health worker intervention 
to improve posthospital outcomes: a randomized clinical trial. JAMA Intern Med. Apr 
2014;174(4):535-543. 
55. 
Balaban RB, Galbraith AA, Burns ME, Vialle-Valentin CE, Larochelle MR, Ross-Degnan D. A 
Patient Navigator Intervention to Reduce Hospital Readmissions among High-Risk Safety-
Net Patients: A Randomized Controlled Trial. J Gen Intern Med. Jan 24 2015. 
56. 
Bodenheimer T. Unlicensed Health Care Personnel and Patient Outcomes. J Gen Intern 
Med. Mar 28 2015. 
57. 
Wells KB, Jones L, Chung B, et al. Community-partnered cluster-randomized comparative 
effectiveness trial of community engagement and planning or resources for services to 
address depression disparities. Journal of general internal medicine. 2013;28(10):1268-1278. 
58. 
Chung B, Ong M, Ettner SL, et al. 12-month outcomes of community engagement versus 
technical assistance to implement depression collaborative care: a partnered, cluster, 
randomized, comparative effectiveness trial. Ann Intern Med. Nov 18 2014;161(10 
Suppl):S23-34. 
59. 
Kenrick DT, Griskevicius V, Neuberg SL, Schaller M. Renovating the Pyramid of Needs: 
Contemporary Extensions Built Upon Ancient Foundations. Perspectives on psychological 
science: a journal of the Association for Psychological Science. 2010;5(3):292. 
60. 
Goldsteen RL, Goldsteen K, Dwelle T. Chapter 8: Building Healthy Communities. Introduction 
to Public Health: Promises and Practice. Second ed. New York, NY: Springer Publishing 
Company; 2014. 
61. 
Coleman K, Austin BT, Brach C, Wagner EH. Evidence on the Chronic Care Model in the 
new millennium. Health Aff (Millwood). Jan-Feb 2009;28(1):75-85. 
62. 
Improving Chronic Illness Care, Seattle, WA. The Chronic Care Model [online]. Accessed 
5/18/2015. 
63. 
Logan J, Graham ID. Toward a comprehensive interdisciplinary model of health care 
research use. Science communication. 1998;20(2):227-246. 
64. 
Graham ID, Logan J. Innovations in knowledge transfer and continuity of care. CJNR 
(Canadian Journal of Nursing Research). 2004;36(2):89-103. 
65. 
Balasubramanian BA, Cohen DJ, Davis MM, et al. Learning Evaluation: blending quality 
improvement and implementation research methods to study healthcare innovations. 
Implementation Science. 2015;10(1):31. 
66. 
True G, Butler AE, Lamparska BG, et al. Open access in the patient-centered medical home: 
lessons from the Veterans Health Administration. J Gen Intern Med. Apr 2013;28(4):539-545. 
67. 
Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: 
results from the Expert Recommendations for Implementing Change (ERIC) project. 
Implement Sci. 2015;10:21. 
68. 
Weiner B, Amick H, Lee S-Y. Conceptualization and measurement of organizational 
readiness for change: a review of the literature in health services research and other fields. 
Medical Care Research and Review. 2008. 
69. 
Weiner BJ. A theory of organizational readiness for change. Implement Sci. 2009;4(1):67. 
70. 
Rubenstein LV, Danz MS, Crain AL, Glasgow RE, Whitebird RR, Solberg LI. Assessing 
organizational readiness for depression care quality improvement: relative commitment and 
implementation capability. Implementation Science. 2014;9(1):173. 
71. 
Nelson KM, Helfrich C, Sun H, et al. Implementation of the patient-centered medical home in 
the Veterans Health Administration: associations with patient satisfaction, quality of care, 
staff burnout, and hospital and emergency department use. JAMA internal medicine. 
2014;174(8):1350-1358.