PACT Training Programs 

for PACT Teams

PACT Demonstration Lab Cyberseminar 

March 20, 2013 

Carole Warde MD

VISN 22 Veterans Assessment and Improvement 

Laboratory

Anneliese Buttler MSW

VISN 4 Center for Evaluation of PACT
What aspects of PACT training would 

you like to learn from this 

Cyberseminar? 

a) Curricular content to train teamlets in PACT 

principles

b) Strategies to help teamlet members work 

together as a team

c) Innovative models for PACT training and spread

d) Frontline staff perspectives on PACT training 

(effectiveness, suggested improvements)

e) General knowledge of how other VAs are 

training for PACT
What is your perspective for learning?

a) Clinical Content – what is in the curriculum?

b) Operations – how to put on a PACT training 

seminar?

c) Research – how to study the effectiveness of 

a PACT training seminar?
Team Communication Exercise

“TEX”

Development of an Intervention to Improve and 

Teach Team Function

Carole Warde MD and Marjorie Pearson PhD

PACT Demonstration Lab Cyberseiminar 

March 20, 2013 

 
 

 

TEX Aim 

We aimed to develop an interactive 

practice-based coaching program to 

assist PACT teamlets with trainees to 

learn, practice and coach effective 

team function.
TEX Presentation Outline

1. The Conceptual Model

2. The Pilot Curriculum Planning

3. The Pilot TEX Intervention

a) Sample Session Outline 

b) Coaching Methods

4. Formative Evaluation Plan

5. Lessons Learned: Curriculum

6. Lessons Learned: Implementation

7. Key Recommendations to Improve TEX

8. Conclusions
1. TEX Conceptual Model

Salas 2009. Team Effectiveness in Complex Organizations

Team Function

TEAM PROCESSES:

Meetings 

1. Follow ground rules

2. Clarify meeting aims

3. Assign meeting roles

4. Negotiate agenda/time

5. Work through agenda 

by discussing information

6. Review meeting actions

7. Plan the next steps

8. Evaluate the meeting

TASK WORK:

Quality Improvement

 

1. Set aims

2. Identify problems

3. Establish measures 

4. Select changes

5. Test changes using 

PDSA cycles

6. Implement changes

TEAM WORK:

Relationships

1. Listen actively

2. Build trust

3. Manage emotions

4. Make decisions by 

consensus

5. Define roles

6. Hold accountable

7. Give feedback

8. Receive feedback

9. Support change in 

others
2. Pilot TEX Curriculum Planning 

Tools Were Developed for

Coaching and Evaluation in 3 Curricular Areas 

• Needs assessment tools 

• Post-intervention evaluation tools 

• Teaching Materials 

– QI and Team Meeting Process handouts 

– Communication behavior pocket cards

– Interactive learning models and exercises 
3. Pilot TEX Intervention 

 

• Two volunteer teamlets with trainees at two VA 

sites in Southern California

• Two coaches facilitated the learning sessions 

• Five onsite workshops with all teamlet members

– 1 Engagement Session (Administrators also attended)

– 1 Observed Needs Assessment 

– 2 Coached Interactive Workshops

– Final Discussion and Evaluation

• 4 coaching calls with teamlet leaders only
Sample Workshop Outline: Overview

Objective: Identify the 3 aspects of team function 

• Task work related to Quality Improvement (QI)

– Rationale for QI 

– IHI QI Process

• Team Processes – Effective Team Meetings

– Ground Rules

– Guide for team meetings

– Member roles during team meetings

• Team Work – Relationship-centered team communication 

– Lessons learned from the NDP 
Coaching Practices Emphasized 

Motivation, Education and 

Consultation 

• Mindfulness practices

• Appreciative reflection

• Active listening exercises

• Emotional management skills

• Stress-management awareness and management

• Paired discussions

• Facilitated large-group discussions and team meetings

• Role-modeling of relationship behaviors

• Role-plays

• Didactic presentations of relevant models and skills
4. Formative Evaluation to 

Assess and Guide TEX Improvement

• Participants and Coaches assessed: 

– Reactions to TEX curriculum and implementation 

– Behavior change in each of the three areas 

• We collected evaluation data in several ways: 

– Immediate participant feedback at the end of each session 

– Immediate coaches’ discussion and documentation 

– Reflective coach observation summary and recommendations

– Participant end of course survey assessment 

– Coach group discussion and consensus to determine lessons 

learned and recommendations 
5. Lessons Learned: TEX Curriculum 

• The Triple Focus worked

– Teamlets had needs and improved in all 3 areas

• Team Meeting Processes were essential

– Mastery of these first facilitates learning in other areas

• Team Communication Skills needed more

– More time and practice is needed in all behaviors

• Quality Improvement was a struggle

– A challenge from mindset to measurement

– Work-flow redesign pushes teams to improve team 

function 
6. Lessons Learned: TEX 

Implementation 

• Coaching was appreciated 

• Teamlets responded to coaching in all three areas 

• A number of coaching methods were successful 

• Teamlet dysfunction needs immediate response 

• The pilot structure worked

– The onsite, teamlet directed course was well-received

– Both the interactive sessions and coaching calls were useful 

• Including trainees was a “win-win” 

– Trainees benefited from active involvement in TEX

– Teamlets benefited from the trainees’ time, curiosity and 

fresh skills 
7. Key Recommendations to Improve 

TEX 

Curriculum

• Emphasize relationship-centered meeting processes early 

• Allow more time for coached communication and QI skills

• Involve administrators to motivate and support QI

• Introduce ideas for change involving work-flow re-design 

Implementation

• Alternate sessions and calls every 2 – 3 weeks

• Emphasize ongoing needs assessment and feedback 

• Intensify coaching in trust-building, identification and 

management of teamlet dysfunction, and teamlet leadership
8. Conclusions

• The Triple-Focus model is useful to improve 

and teach teamwork

• The interactive TEX curriculum helped PACT 

teamlets with trainees improve team function

• Our assessment tools and processes provided 

meaningful feedback to direct future sessions

• We expect the recommended revisions to 

further strengthen the TEX innovation 
Acknowledgement

• TEX was funded through an Innovation Grant 

from the Veterans Administration and 

Improvement Laboratory

• Special thanks to the following individuals 

who made this pilot project possible:

– Lisa Rubenstein, Lisa Altman, Susan Stockdale

– Art Gomez

– Christina Huang and Lopmurda Das
VISN 4 Virtual PACT Collaborative 1.0: 

Successes, Challenges, & Lessons Learned

Anneliese Butler, MSW

March 20, 2013
Project Team:

• Anneliese Butler, MSW

• Anne Canamucio, MS

• Michele Lempa, DrPH

• Mary Pelak, MSW

• Gala True, PhD, Co-Director 

Qualitative Evaluation Core

Partners: 

• David Macpherson, CMO 

• Jennifer Skoko, AO to CMO

• VISN 4 Virtual Collaborative 

Steering Committee

• Virtual Collaborative 

participants and coaches

With the support of:

Rachel Werner, MD, PhD, CEPACT Director

Judith Long, MD, Associate Director

Judy Shea, PhD, Director of Qualitative Evaluation Core
• Focus & approach:

1. Implementation process 

• What did it take to implement the model?

• How did implementation vary between sites? 

• How did the model evolve over time?

• Methods: process diary and interviews with key contacts 

2. Learners’ experiences 

• Was the VC successful from the perspective of the teams? 

• What worked, for whom, under what conditions?

• Methods: anonymous survey, key contact interviews
Why another PACT training?

Aug 2010SepOctNovDecJan 2011FebMarAprilMayJuneJulyAugSepOctNovDecJan 2012FebMarAprilMayJuneJulyAugSepOctNovDec

LS1 LS2 LS3 LS4 LS5 LS6

Regional PACT Learning Collaborative

6 x 3-day Learning Sessions (pilot teams)

PACT Learning Centers

Twice/month 1-hr virtual sessions (all) VISN 4 Virtual Collaborative

V1. One-time 3-day basic 

training (non-pilot teams)

PACT 

101 

→ 2x/mo virtual sessions (all teams) 
VISN 4

Coatesville, 

PA

Erie, PA

Butler, PA

Pittsburgh, PA

VISN 4 Office

Clarksburg, WV

Altoona, 

PA

Wilkes-Barre, PA

Lebanon, 

PA

Philadelphia, 

PA

Wilmington, DE

10 facilities, 45 CBOCs

~350 primary care teams

>300,000 Veterans per yr
VC model ~ online elements

Coatesville, 

PA

Erie, PA

Butler, PA

Pittsburgh, PA

VISN 4 Office

Clarksburg, WV

Wilkes-Barre, PA

Lebanon, 

PA

Philadelphia, 

PA

Wilmington, DE

Virtual learning sessions 

- via Live Meeting 2x/month, 12-1pm

-Schedules blocked per VISN directive

VC SharePoint

- VC archives

- Recorded sessions

- Tools & resources

Altoona, 

PA
Coatesville, 

PA

Erie, PA

Butler, PA

Pittsburgh, PA

VISN 4 Office

Clarksburg, WV

Altoona, 

PA

Wilkes-Barre, PA

Lebanon, 

PA

Philadelphia, 

PA

Wilmington, DE

VC model ~ “offline” elements

On-site coaching

Team 

assignments

Team meetings

≥ 1 hr every 2 weeks, 

per VISN directive
VC Schedule/Curriculum (first 9 months)

Jan 5 & 19 PACT 101 (for teams with no prior PACT training)

Feb 2 Introduction to VC and details of PACT metrics

Feb 16 The inpatient outpatient transition

Mar 1 Chronic Pain Management (providers only)

Mar 15 “Hot spots” – Patients at high risk for admission 

Apr 5 Advanced clinic access

Apr 19 Diabetes Management (providers only)

May 3 Telephone care

June 7 Update in General Internal Medicine (providers only)

June 21 Team Care – working in a team

July 5 Where do we stand? Current status of PACT implementation

Aug 2 Team Presentations – 4 teams present frontline perspective

Aug 16 PACT team roles – top of license and new roles

Sep 6 Pain Management - caring for opioid misuse (providers only)

Sept 20 Palliative Care

8 
 Anonymous, brief (5 min), 38 items (incl. 4 open-ended)

 Target group: Core team members (PCPs, RNs, Clinical & Clerical Associates)

 Fielded twice during first 9 months (April & September)
1. Determine extent of participation in VC activities (virtual 

learning sessions, team meetings, etc.). 

2. Assess learners’ perspectives on the usefulness and value

of each VC activity/component and of the VC as a whole. 

3. Gauge learners’ perceptions of VC’s overall impact on 

PACT-related knowledge and access to resources. 

4. Assess acceptability of the VC format relative to prior PACT 

training initiatives. 

...and identify differences by role, site, facility type, prior exposure to 

PACT trainings, and extent of participation in VC (for aims 2-4) 
Round 1 Round 2

Survey Response rate: 41% (335/819) 43% (353/820)

Role:

PCPs 34% 39%

RN/Care Managers 28% 25%

Clinical Associates 18% 17%

Clerical Associates 7% 5%

Facility type:

VAMC 52% 48% 

Training exposure:

Had prior PACT training 33% 35%
Participation/Engagement: 

Good attendance of virtual sessions; team meetings 

shorter than intended, possible erosion over time

Perceived usefulness & impact: 

Virtual sessions & team meetings valued; “HW” unpopular

Benefits: Increased knowledge, perspective, access to 

tools & resources, learning from peers across VISN

Factors associated with perceived benefits: 

Exposure to prior PACT training 

Full participation in all VC activities
“What was useful?”

“I like to see what others 

are doing so we know 

other ways to implement 

and use PACT.” (Nurse 

Manager, VAMC) 

“It helps to understand 

how the PACT teams are 

functioning both at our 

facility and in VISN in 

general.” (PCP, VAMC) 

“increased understanding 

of how individual factors 

come together to form one 

big picture” (Clinical 

Associate, CBOC) 

“It gives me the

perspective that none of 

us are having an easy time 

putting this into practice.” 

(Clinical Associate, VAMC) 

“reduction of travel 

makes it much less of a 

negative impact on group 

and team productivity” 

(PCP, VAMC)
Constructive criticism: VC Content

“It seems like the majority of them were more directed towards the 

provider and not the rest of the PACT team.” (Clinical Associate, VAMC) 

“I think having a training on PACT, to get a basic understanding prior to 

the VC, would have been very helpful. All these months into it, we are 

just now finding out about tools to help with this process. I feel that we 

are embarking on a quest without having the basic fundamentals in 

place first.” (Clinical Associate, CBOC) 

“I am not yet fully PACT as we are quite low in RNs... The presenters of 

the collaborative could give alternative ways of doing things for 

teams like ours that have 1 RN for 8 providers ...instead of making 

suggestions for the ideal theoretical team that we currently don't have 

yet.” (PCP, CBOC)
Benefits limited by familiar challenges

“...Only my RN comes to the collaborative, no one else from my team ... has 

time or has been released from work tasks. I think if we value something then 

we make it a priority so if the PACT is truly a priority, set aside time away 

from work to let people attend.” (PCP, VAMC)

“We are so severely understaffed that going to these sessions is like sending 

a diabetic to lunch at a candy store. What's the use when you can't avail 

yourself of such wonderful 'theoretical’ concepts. PACT is flying overhead and 

we're still in the bunkers.” (PCP, VAMC)

“At the team level there are very few changes that can be made to implement 

PACT to the best of our ability – our hands are tied by the administration!” 

(PCP, CBOC)

“going to the [VC] is torture ... I go learn about things other facilities try and 

do [that] we aren't allowed [to do] ... I get to hear things that the National 

and VISN recommend but they are ignored at this facility.” (PCP, VAMC)
Lessons learned & takeaways

• VC holds promise

• Insist on truly protected time

• Consider needs of all team members

• Match content to audience 

• Speak to teams’ challenges 

• Confront leadership issues

• Actively seek input from “the trenches”
• Practice Improvement Series 

– expanded focus (beyond Compass metrics)

– emphasis on team-based QI, less on “PACT”

– more team autonomy
For more information: 

anneliese.butler@va.gov

VISN 4 Center for Evaluation of PACT 

(CEPACT)

Philadelphia VA Medical Center

3900 Woodland Ave, Bldg 4100

Philadelphia, PA 19104

http://vaww.visn4.va.gov/CEPACT/ 
Virtual Collaborative Survey of 

Learner Experiences – Final Report 

January 2013

VISN 4 Center for Evaluation of Patient Aligned Care Teams 

Anneliese Butler, MSS 

Anne Canamucio, MS 

Mary Pelak, MSW 

Gala True, PhD1

 

1

 Readers are invited to submit questions and comments about the contents of this report by e-mailing 

Anneliese.Butler@va.gov. 
Page | 2 

Contents

I. ACKNOWLEDGEMENTS ..................................................................................................................................................... 3

II. EXECUTIVE SUMMARY ..................................................................................................................................................... 4

III. BACKGROUND ................................................................................................................................................................ 6

IV. DESIGN & METHODS ...................................................................................................................................................... 7

V. RESULTS .......................................................................................................................................................................... 8

VI. DISCUSSION ..................................................................................................................................................................13

LIMITATIONS .............................................................................................................................................................................13

KEY TRENDS ..............................................................................................................................................................................13

OPPORTUNITIES FOR IMPROVEMENT ..............................................................................................................................................16

VII. TABLES ........................................................................................................................................................................18

VIII. ADDITIONAL QUOTES .................................................................................................................................................25

PRAISE – WHAT’S BEEN USEFUL.....................................................................................................................................................25

CONSTRUCTIVE CRITICISM ............................................................................................................................................................26
Page | 3 

I. ACKNOWLEDGEMENTS 

This work could not have been done without the support of: 

David Macpherson, MD, MPH 

Jennifer Skoko, MHA 

Virtual Collaborative Planning Committee and Coaches 

CEPACT leadership and core team 

We are especially grateful to our key contacts and to all VISN 4 PACT team members, whose generosity makes 

this work possible. 

Suggested Citation: 

Anneliese Butler, Anne Canamucio, Mary Pelak, Gala True. 2013. Virtual Collaborative Survey of Learner 

Experiences: Final Report. VISN 4 Center for Evaluation of Patient Aligned Care Teams; Philadelphia VA Medical 

Center. Department of Veterans Affairs.

Table of Contents
Page | 4 

II. EXECUTIVE SUMMARY 

The VISN 4 Virtual Collaborative (VC) was launched by Dr. David Macpherson, Chief Medical Officer for VISN 4, in 

January 2012 as a central part of the VISN’s plan to advance Patient Aligned Care Team (PACT) implementation 

and spread. The Center for Evaluation of Patient Aligned Care Teams (CEPACT) partnered with Dr. Macpherson 

and the VC Planning Committee to assist in evaluating the VC. One component of the evaluation is a survey to 

assess the experiences and perceptions of VC participants at midpoint (April 2012) and again at the end of the 

first 9-month phase (September 2012). This report presents results and findings from Round 2 of the survey, and 

reflects on trends over the first year of the VC and implications for improving learners’ experiences as the VC 

enters its next phase. Significant results for each survey aim as well as key findings are summarized below. 

Survey Sample: 

Forty-three percent of VC participants (n=353) responded to the second round of the survey. As with Round 1 of 

the survey, respondents to Round 2 were more likely to be Primary Care Providers (39%) or RN Care Managers 

(25%) than Clinical or Clerical Associates (17% and 5%, respectively). About one-half of respondents in each 

survey were located at a VA Medical Center, and about one-third had attended some form of PACT training prior 

to entering the VC.

Aim 1: Extent of Participation in Virtual Collaborative Activities

Participation in Live Meeting sessions was comparable to participation at midpoint, with a majority of 

respondents reporting that they had attended all or most Live Meeting sessions. Respondents in Round 2 

were more likely to have attended sessions alone rather than with others compared with respondents to 

Round 1. 

Participation did not vary significantly by facility type or exposure to prior PACT training. PCPs were more 

likely to have attended most Live Meeting sessions compared with RNs.2

 

Compared to Round 1, respondents in Round 2 were less likely to report that their teams had been meeting 

at the expected frequency (at least once every two weeks). Exposure to prior PACT training and working at 

a VAMC were associated with higher team meeting frequency. 

Similar to Round 1, most respondents indicated that team meetings were shorter than originally envisioned, 

with only 15% stating that their teams met for the recommended hour. Respondents in Round 2 were less 

likely to have completed all homework assignments compared with Round 1 respondents. About one-third 

of respondents said their team had completed all homework assignments since the midway point, while 

almost half completed some or most of the assignments. Approximately half said that homework was 

completed as a team. 

One-fifth of respondents were considered to have participated fully in the VC between April and September; 

those at a VAMC and those with prior PACT training were more likely than those at a CBOC or those with no 

prior training to be getting the full VC experience. 

Aim 2: Perceived Usefulness and Value of the Virtual Collaborative 

Over half of survey respondents found the Live Meeting sessions to be of use, especially the stories and 

examples from other teams. RN Care Managers were more likely to find the sessions useful, as were those 

who had prior PACT training and those who were highly engaged in the VC. 

A majority of respondents agreed that team meetings were useful. Team meetings were valued more highly 

by respondents who were stationed at a VAMC, had prior PACT training, and/or were fully engaged in the 

VC. 

 

2

 We surmise that we would have seen similar differences between PCPs and both clinical and clerical associates if the latter 

two roles had been more strongly represented in the overall survey sample. 

Table of Contents
Page | 5 

As in Round 1, homework was the least popular aspect of the VC; however respondents who were fully 

engaged in the VC were more likely to agree that the assignments were useful in making PACT related 

changes 

Of those respondents who used available resources (coaches, SharePoint, VC coordinator), those 

participating fully in VC activities were also more likely to rate these resources as useful. 

Aim 3: Perceived Impact of the Virtual Collaborative

A majority of survey respondents agreed that participation in the VC has positively impacted their 

knowledge about, awareness of, and access to further resources related to PACT. Close to one-half of 

respondents felt that participation in the VC was worthwhile for someone in their role, and a plurality 

agreed that the VC had increased communication with other teams and helped their team progress with 

PACT implementation. 

Respondents with prior PACT training and those who were highly engaged in the VC were considerably more 

likely to perceive a positive impact. 

Aim 4: Acceptability of the Virtual Collaborative Format

Over half of respondents felt the VC format was an acceptable alternative to off-site training and liked 

receiving training in “small doses” spread over time. Respondents to this survey were more likely than 

Round 1 respondents to feel that the VC offered a comparable level of training to an off-site format. 

However, 42% indicated that participation in the VC interfered with their day-to-day work. 

Respondents with prior PACT training and those who were highly engaged in the VC also tended to find the 

format more acceptable. 

Key Trends 

Survey results suggest that participation in the VC has benefited front-line staff by increasing knowledge 

about PACT, access to PACT-related resources, and peer-to-peer exchange of ideas. However, these benefits 

were not equally distributed, and several opportunities for greater impact remain. (Read more) 

The potential benefits of the VC are hampered by challenges related to time and staffing that mirror barriers 

reported to impede PACT implementation on a broader scale. Although it is beyond the scope of the VC 

alone to solve these challenges, acknowledging and addressing these constraints in the context of the VC 

may mitigate their impact and facilitate movement toward their resolution. (Read more) 

Perceived lack of leadership support at certain facilities poses an unaddressed barrier to achieving and 

sustaining teams’ engagement and optimism about PACT. (Read more) 

Opportunities for Improvement

 Prompt presenters to craft presentations that speak to the needs of key target groups (e.g., clinical and 

clerical support staff; CBOC teams; “atypically” staffed teams) and that address the challenges of 

implementing recommended practices in the context of persistent resource constraints. 

 Offer newer PACTs (particularly those at CBOCs) additional support around team-building and basic team 

processes (e.g., via more intensive coaching, more opportunities for contact with experienced teams). 

 Ensure that ALL team members have truly protected time to participate in practice improvement work; 

continue to emphasize the importance of maintaining such protected time at all organizational levels. 

 Equip coaches with resources to help teams problem-solve around ways to adapt PACT principles and 

strategies to local circumstances, and to engage more passive or resistant team members. Facilitate contact 

between coaches at different facilities to encourage information sharing and collective problem-solving. 

 Teams at some sites may need support from outside the facility to address larger systemic issues. Leaders at 

certain sites may benefit from targeted education about PACT concepts and processes in order to ensure 

that teams have the resources (staff, time) and autonomy needed to realize the goals of PACT.
Page | 6 

Key components of the Virtual Collaborative model

• Live Meeting sessions: VISN-wide didactic sessions conducted 

via Live Meeting twice a month from noon-1:00pm on 

Thursdays

• Team meetings: Dedicated time for teams to meet outside of 

the Live Meeting sessions to work on PACT implementation

• Homework: Assignments for teams to complete between Live 

Meeting sessions, designed to promote hands-on application of 

Live Meeting content

• Additional resources and supports for teams, including on-site 

coaches and a central SharePoint site maintained by the VC 

coordinator that draws together VC- and PACT-related 

resources from various sources and also houses recordings of all 

Live Meeting sessions.

III. BACKGROUND 

The VISN 4 Virtual Learning Collaborative was launched by Dr. David Macpherson, Chief Medical Officer for VISN 

4, in January 2012 as a central part of the VISN’s plan to advance PACT implementation and spread. It aims to 

address needs not met by earlier training and spread initiatives (i.e., Regional PACT Collaboratives and PACT 

Learning Centers) while at the same time testing an innovative method for encouraging and sustaining 

organizational change. 

It is important to note that the VC model as it was originally conceived assumed that teams would enter the VC 

with at least some foundational knowledge of PACT, acquired through a 3-day Learning Center training 

coordinated by the Regional Center of Excellence. However, these introductory trainings were suspended 

shortly before the VC was to launch. To compensate for this unforeseen turn of events, the VC model was 

revised so as to have all teams join at the same time, regardless of prior training exposure. 

In partnership with the Virtual Collaborative (VC) Planning Committee, the Center for Evaluation of Patient 

Aligned Care Teams (CEPACT) developed The Survey of Learner Experiences as part of a larger multi-method 

evaluation of the VC. The overall goals of this evaluation are to capture and assess (a) the evolution of the VC 

model during the first 9 months of its implementation, and (b) the experiences and perceptions of VC 

participants. 

The Survey of Learner Experiences was fielded twice, once at mid-point (April 2012) and again at the 9-month 

mark (September 2012). The present report focuses on results3 and findings from Round 2 of the survey, reflects 

on trends over the first year of the VC, and presents opportunities for improving learner experiences in the 

future. 

 

3

 Section VII lists selected tables only; readers may request additional tables by e-mailing Anneliese.Butler@va.gov. 

Table of Contents
Page | 7 

IV. DESIGN & METHODS 

Survey design and methods are summarized in our Midpoint Report.4 Details specific to the approach taken for 

Round 2 are highlighted below. 

Data Collection and Analysis 

As in Round 1, the second survey was announced and endorsed by Dr. Macpherson during the 9/20/12 Live 

Meeting session, and an email invitation containing a Web Link to the survey was sent to all registered VC 

participants immediately following this session. Survey responses were collected anonymously in order to 

ensure VA staff would feel comfortable in giving honest feedback. A limitation of collecting survey data 

anonymously is that we do not know the extent to which there was an overlap between respondents to Rounds 

1 and 2. For this reason, we must treat these two respondent groups as independent samples. 

Modification: The survey period for Round 2 was extended from two to five weeks in order to offset the effects 

of survey fatigue generated by the volume of surveys directed at PACT teams (especially providers) over the 

summer. 

As before, residents and those with no involvement in the VC to date were excluded from analysis. Survey 

responses were dichotomized (e.g., agree and strongly agree versus other) and all comparisons were made using 

Chi square tests. Multiple comparisons (i.e., between different roles) were done using Bonferroni adjustment. 

Responses to open-ended survey items were reviewed and summarized using ATLAS.ti to identify positive 

feedback as well as opportunities for improvement (at the local and VISN level) from the perspective of VC 

participants.

 

4

 A copy of the Midpoint Report is available upon request and can also be accessed on the VC SharePoint site. 

Survey Aims

1. To determine the extent of participation in VC activities, including Live Meeting 

sessions, team meetings for planning, and homework assignments.

1.a. To identify differences in extent of participation by role, site, facility type, and 

prior exposure to PACT trainings (i.e., NE PACT Learning Collaborative, Center of 

Excellence/Learning Center training).

2. To assess learners’ perspectives on the usefulness and value of each VC 

activity/component (including Resources) and of the VC as a whole. 

2.a. To identify differences in perceived usefulness/value by role, site, facility type, 

and prior exposure to PACT trainings. 

3. To gauge learners’ perceptions of the overall impact of the VC (as a whole), in terms 

of increasing PACT-related knowledge and access to resources. 

3.a. To identify differences in perceived impact by role, site, facility type, prior 

exposure to PACT trainings, and extent of participation in VC.

4. To assess the acceptability of the VC format relative to prior PACT training 

initiatives, which were off-site and more concentrated/time-intensive. 

4.a. To identify differences in perceived acceptability of the VC format by role, site, 

facility type, prior exposure to PACT trainings, and extent of participation in VC.

Table of Contents
Page | 8 

Aim 1 Summary: Participation in VC Activities

Participation in Live Meeting sessions was comparable to 

participation at midpoint, with a majority of respondents 

reporting that they had attended all or most Live Meeting 

sessions. Respondents in Round 2 were also more likely to 

have attended sessions alone rather than with others 

compared with respondents to Round 1.

Participation did not vary significantly by facility type or 

exposure to prior PACT training; however, RNs were less likely 

than PCPs to have attended most sessions. 

Compared to Round 1, respondents in Round 2 were less 

likely to report that their teams had been meeting at the 

expected frequency. Exposure to prior PACT training and 

working at a VAMC were associated with higher team meeting 

frequency. 

Similar to Round 1, most respondents indicated that team 

meetings were shorter than originally envisioned.

(continued on next page)

V. RESULTS 

Please follow hyperlinks to see relevant Tables for each section.

Survey Sample (Table) 

Of the 820 participants invited to complete the second survey, we received 371 responses. Of these, 18 were 

disqualified (17 who reported having no involvement with the VC to date, and 1 who self-identified as a 

resident). The total remaining sample size for Round 2 was 353 for a response rate of 43%, slightly higher than 

the response rate for Round 1 (41%, n=335).

Those who answered the second round were similar to those who answered the first round with respect to role, 

primary site, whether or not they had prior PACT training and if they attended 5-7 sessions. In both survey 

rounds, Primary Care Providers (PCPs) were most strongly represented (34% and 39% for Rounds 1 and 2, 

respectively), followed by RN Care Managers (28% and 25%), Clinical Associates (18% and 17%), and Clerical 

Associates (7% and 5%). About one-half of respondents in each survey were located at a VA Medical Center 

(VAMC), and about one-third had attended some form of prior PACT training. VAMC respondents were 

considerably more likely than their CBOC peers to have attended PACT training prior to entering the VC (50% vs. 

29%, p=0.0002). 

Aim 1: Extent of Participation in Virtual Collaborative Activities (Tables) 

Participation in Live Meeting Sessions

Three hundred twenty-eight (328) respondents (93% of the sample) reported that they had attended 1 or more 

of the 7 Live Meeting sessions that took place between April and September 2012. Of these, 116 (35%) said they 

attended with other members of their team, 

and 130 (40%) said they attended with a group 

of teams. A higher proportion relative to Round 

1 reported attending the Live Meeting sessions 

on their own (23% vs. 9%). 

A majority of respondents (57%) attended 5 to 

7 sessions during this period, nearly the same 

proportion as for Round 1 (61%). RNs in this 

sample were significantly less likely to attend 

most/all the Live Meeting sessions than PCPs 

(45% vs. 71%, p=0.00). 

Participation in Team Meetings 

Just under half (49%) of respondents in Round 

2 reported that they met with their teams at 

least every two weeks in the period from April 

to September 2012. Respondents at VAMCs 

were more likely than those at CBOCs to report 

meeting every 2 weeks or more (55% vs. 44%, 

p=0.05), as were those who had participated in previous PACT training compared to those with no prior training 

(60% vs. 42%, p=0.00). Compared to respondents in Round 1 of the survey, Round 2 respondents were less likely 

to have met every 2 weeks or more (49% vs. 63%, p=0.00). 

As in Round 1, comparatively few respondents (15%) reported that their teams met for 60 minutes or more, and 

this did not vary by facility type or participation in prior PACT training. 

Table of Contents
Page | 9 

Aim 1 Summary: Participation in VC Activities (continued)

Respondents in Round 2 were less likely to have completed all 

homework assignments compared with Round 1 respondents. 

About one-third of respondents said their team had completed 

all assignments since the midway point, while almost half 

completed some or most of the assignments. Approximately 

half said that homework was completed as a team. 

One-fifth of respondents were considered to have participated 

fully in the VC between April and September; those at a VAMC 

and those with prior PACT training were more likely than those 

at a CBOC or those with no prior training to be getting the full 

VC experience.

Participation in Homework

Homework has been a less prominent feature of the VC since June 2012, with only 3 assignments occurring after 

the mid-point mark (in late-April, May, and June). Results to survey items regarding VC homework should be 

interpreted with this in mind. 

About one-third of respondents (31%) said 

their team completed all homework 

assignments, and slightly less than one-half 

(47%) said their team completed most or some 

assignments. Of those who completed any 

homework, a slight majority (54%) said that it 

was a team activity. Compared to Round 1 

respondents, Round 2 respondents were less 

likely to have completed all homework 

assigned during the corresponding period (31% 

vs. 51%, p=0.00). 

Overall Engagement

As in Round 1, we used two measures to summarize overall engagement in the VC. The first of these was a 

“meeting dose” variable based on frequency and duration of team meetings. Teams who spent more time 

together (met frequently and for longer durations) were said to have a “high dose,” and those who met less 

frequently and for shorter durations were said to have either a “medium” or a “low dose.” Of the 268 

respondents who had responses for the contributing variables, 84 (31%) were in the high dose group, nearly the 

same proportion of respondents as in Round 1 (34%). Respondents from VAMCs were more likely to have a high 

dose than those from CBOCs (40% vs. 21%, p=0.00). 

Another measure of overall engagement was a dichotomous variable that took into account whether or not the 

respondent could be considered to have had the “full VC experience,” defined as having attended at least 4 Live 

Meeting sessions between April and September 2012 and having a high team meeting dose.5 According to these 

specifications, 70 respondents (20%) were considered to have had the full VC experience. Respondents who did 

not meet the criteria or who failed to answer any of the contributing questions were not considered to have had 

the full VC experience. Respondents from VAMCs were more likely to have had the full VC experience than those 

from CBOCs (30% vs. 13%, p=0.00), as were those who had participated in prior PACT training (29% vs. 15%, 

p=0.00). 

Aim 2: Perceived Usefulness and Value of the Virtual Collaborative (Tables)

Usefulness of Live Meeting Sessions 

 Over half of respondents (54%) agreed that the presentations made during Live Meeting sessions were useful, 

and nearly two-thirds (61%) found the stories and examples from other teams to be useful. Just under half of 

respondents endorsed the polls (49%) and felt that the Live Meeting sessions were a good use of their time 

(45%). Round 2 responses did not differ significantly from Round 1 responses for these items. 

Comparisons by role revealed that RN Care Managers were significantly more likely than either Clinical 

Associates (70% vs. 50%, p=0.05) or PCPs (70% vs. 47%, p=0.00) to find Live Meeting presentations useful. RNs 

were also more likely than PCPs to say that team stories/examples were useful (75% vs. 53%, p=0.01). 

Comparisons by facility type revealed no significant differences between VAMC and CBOC respondents for these 

items. By contrast, those respondents who had prior PACT training, spent more time with their team (“high 

 

5 We revised the original definition of the “full experience” variable used in Round 1 analyses in order to reflect the ways in

which the VC evolved over time (i.e., homework was less central to the VC experience). Thus, while the “full experience” 

variable included items pertaining to homework completion for Round 1, those items were omitted for Round 2. 
Page | 10 

Aim 2 Summary: Perceived Usefulness

Over half of survey respondents found the Live Meeting 

sessions to be of use, especially the stories and examples from 

other teams. RN Care Managers were more likely to find the 

sessions useful, as were those who had prior PACT training and 

those who were highly engaged in the VC. 

A majority of respondents agreed that team meetings were 

useful. Team meetings were valued more highly by 

respondents who were stationed at a VAMC, had prior PACT 

training, and/or were fully engaged in the VC. 

As in Round 1, homework was the least popular aspect of the 

VC; respondents who were fully engaged in the VC were more 

likely to agree that the assignments were useful in making 

PACT related changes 

Of those respondents who used available resources (coaches, 

SharePoint, VC coordinator), those participating fully in VC 

activities were also more likely to rate these resources as 

useful. 

dose”), and/or got the full VC experience were significantly more likely to perceive Live Meeting presentations 

and team stories as useful and to feel that the sessions were worth their time. 

Usefulness of Team Meetings

As in Round 1, a majority of respondents (58%) agreed that team meetings helped with PACT implementation. 

We found no significant differences by role; however, respondents stationed at a VAMC were more likely than 

those at CBOCs to agree with this statement 

(65% vs. 50%, p=0.02). More striking 

differences were found for those with prior 

PACT training compared to those without 

(71% vs. 48%, p=0.00), those with a high team 

meeting “dose” (85% vs. 46%, p=0.00), and 

those getting the full VC experience (89% vs. 

47%, p=0.00). 

Similarly, most respondents (60%) felt that 

team meetings were a good use of time. 

Again, there was significantly stronger 

agreement among respondents stationed at a 

VAMC (66% vs. 52%, p=0.02), those with prior 

PACT training (74% vs. 49%, p=0.00), those 

with a high team meeting “dose” (85% vs. 

48%, p=0.00), and those who participated fully 

in the VC (87% vs. 50%, p=0.00). 

Usefulness of Homework

Less than one-quarter (22%) of respondents in Round 2 agreed that the VC homework assignments had helped 

with PACT implementation, and they were less likely to agree with this statement than Round 1 respondents 

(22% vs. 30%, p=0.05). Only about one-fifth (19%) of Round 2 respondents agreed that the homework was a 

good use of their time. 

Responses to both homework-related items differed significantly by measures of overall engagement: of 

respondents with a high team meeting “dose” and those who were getting the full VC experience, just over onethird agreed that the homework had helped with PACT implementation (38% and 37%, respectively), and nearly 

one-third agreed that the homework assignments were a good use of their time (32% and 30%, respectively); by 

contrast, rates of agreement were roughly half as high for respondents who did not meet these criteria (see 

details). 

Usefulness of VC Resources

Of those using available resources (coaches, SharePoint, VC Coordinator), just under half of respondents felt that 

the resources were useful, roughly the same proportion as in Round 1. RNs found the SharePoint site more 

useful than PCPs did (62% vs. 37%, p=0.01). Respondents with previous PACT training were significantly more 

likely to endorse the usefulness of both the SharePoint site (55% vs. 43%, p=0.05) and the VC Coordinator (62% 

vs. 36%, p=0.00), compared to those without such training. As with all other VC components, the highly engaged 

group (full experience) was more likely to perceive all VC resources as useful compared with those who were 

less engaged. 

Aim 3: Perceived Impact of the Virtual Collaborative (Tables) 

A number of survey items assessed participants’ views about the VC’s impact, including its impact on their 

knowledge about PACT, on the general awareness of PACT at their facility, on the degree of communication with 

other PACT teams, on their access to resources and experts who could answer questions about PACT, as well as 
Page | 11 

Aim 3 Summary: Perceived Impact

A majority of survey respondents agreed that participation in 

the VC has positively impacted their knowledge about, 

awareness of, and access to further resources related to PACT. 

Close to one-half of respondents felt that participation in the 

VC was worthwhile for someone in their role; a plurality agreed 

that the VC had increased communication with other teams and 

helped their team progress with PACT implementation.

Respondents with prior PACT training and those who were 

highly engaged in the VC were considerably more likely to 

perceive a positive impact.

Aim 4 Summary: Acceptability

Over half of respondents felt the VC format was an acceptable 

alternative to off-site training and liked receiving training in 

“small doses” spread over time. Respondents to this survey 

were more likely than Round 1 respondents to feel that the VC 

offered a comparable level of training to an off-site format. 

However, 42% indicated that participation in the VC interfered 

with their day-to-day work. 

Respondents with prior PACT training and those who were 

highly engaged in the VC also tended to find the format more 

acceptable. 

whether they felt that participating in the VC had value for someone in their role and was helping their PACT 

move forward with implementation. 

A majority of respondents agreed or strongly 

agreed that the VC increased their knowledge 

of PACT and the general awareness of PACT at 

their facility (64% and 52%, respectively), and 

that it had improved access to PACT experts 

and other PACT-related resources (50% and 

58%, respectively). Somewhat fewer 

respondents agreed that participation in the 

VC had increased their communication with 

other primary care teams (44%), was 

worthwhile for someone in their role (47%), 

or was helping their team put PACT concepts 

into action (40%). Round 2 responses did not differ significantly from those obtained in Round 1. 

Both RN Care Managers and Clinical Associates were significantly more likely than PCPs to agree that 

participation in the VC had increased their access to people who could answer PACT-related questions (59% vs. 

40%, p=0.03, and 61% vs. 40%, p=0.04, respectively). 

Those with prior PACT training were significantly more likely to agree that participating in the VC had increased 

communication with other teams (54% vs. 37%, p=0.00), access to experts (61% vs. 43%, p=0.00), and access to 

PACT-related resources (70% vs. 50%, p=0.00). They were also more likely to agree that VC participation was 

worthwhile for someone in their role (58% vs. 40%, p=0.00), and that it was helping their team to put PACT 

concepts into action (50% vs. 33%, p=0.00). 

Respondents who met criteria for high overall engagement (high team meeting dose, full VC experience) 

consistently voiced stronger agreement with items assessing the VC’s impact. Differences were substantial and 

significant across all of these items (see details), but the differences in perceived impact on PACT 

implementation were most striking: approximately two-thirds of those in the highly engaged group agreed that 

participating in the VC was helping their team to implement PACT, compared to only one-third of all remaining 

respondents. 

Aim 4: Acceptability of the Virtual Collaborative Format (Tables) 

Respondents were asked to assess the acceptability of the VC in comparison to a more traditional, off-site 

training format, in terms of the level of training received and the acceptability of integrating training sessions 

into the regular work day. 

Over half of all respondents felt the VC format 

was an acceptable alternative to off-site 

training and said they liked that the VC 

provides PACT training in “small doses” over 

several months (54% and 53%, respectively). 

Forty-two percent agreed that they were 

getting the same training through the VC as 

they would from an off-site training, and they 

were more likely to agree with this than 

respondents in Round 1 (42% vs. 33%, p=0.03). 

About one-third (35%) agreed that 

participating in the VC had not interfered with 

their day-to-day work, while 42% disagreed with this statement. 
Page | 12 

Those with previous PACT training were more likely than those without to agree that the VC offered the same 

level of training as an off-site training would (49% vs. 37%, p=0.04), and that participating in the VC had not 

interfered with their work (42% vs. 30%, p=0.04). They were also more enthusiastic about getting training in 

“small doses” (61% vs. 48%, p=0.03). Respondents who spent more time meeting with their teams were also 

more apt to agree that the VC offered equivalent training (57% vs. 37%, p=0.00) and that their participation had 

not interfered with their day-to-day work (48% vs. 30%, p=0.00). Similarly, those who met criteria for high 

engagement in the VC showed significantly higher agreement on all measures of acceptability (see details). 
Page | 13 

“[Some VC sessions] were helpful. It seems like the majority of them were more directed towards the provider and 

not the rest of the PACT team.” (Clinical Associate, VAMC)

“I think having a training on PACT, to get a basic understanding prior to the VC, would have been very helpful. All 

these months into it, we are just now finding out about tools to help with this process. I feel that we are 

embarking on a quest without having the basic fundamentals in place first.” (Clinical Assoc, CBOC)

“I am not yet fully PACT as we are quite low in RNs... The presenters of the collaborative could give alternative 

ways of doing things for teams like ours that have 1 RN for 8 providers....instead of making suggestions for the 

ideal theoretical team that we currently don't have yet.” (PCP, CBOC) (See more quotes)

VI. DISCUSSION 

After noting the limitations of the study, we highlight and expand upon key trends that emerged from our 

analyses of rounds 1 and 2 of the Survey of Learner Experiences by drawing on qualitative data gathered since 

the VC was launched (i.e., write-in responses to open-ended survey items, interviews with key contacts at each 

facility). We conclude with some reflections on opportunities for improving learner experiences in the months 

ahead. 

Limitations 

Certain limitations must be kept in mind when reading and interpreting the results reported here: 

Our survey methods do not permit us to generalize results, as we do not know whether the learner 

experiences of survey respondents are representative of all VC participants. 

The survey design assumes that all participants are members of defined and fully activated PACTs; however, 

we know from survey comments and interviews with key contacts that some PACT teams are not yet truly 

functioning as such due to several interrelated barriers around staffing, workload, and time constraints. 

Furthermore, the VC registration list included some individuals who were not in a core team role (e.g., 

managers or supervisors). Because the wording of several survey items assumes membership on an active 

PACT team, it is possible that some respondents dropped out before completing the survey, or else selected 

the best available response option, even if it did not capture their experience accurately. 

In addition, the inclusion of a “neutral” response category on most survey items limits what can be learned 

from this survey, because for several survey items, a sizeable proportion of respondents chose to remain 

neutral and it is impossible to know whether rates of agreement would have increased or decreased had 

respondents been “forced” to choose. 

Key Trends 

I. Survey results suggest that participation in the VC has benefited front-line staff by increasing knowledge 

about PACT, access to PACT-related resources, and peer-to-peer exchange of ideas (see quotes). However, 

these benefits were not equally distributed, and opportunities remain for improving the VC’s impact on 

teams’ capacity to make PACT-related changes. 

Across both surveys, the VC appears to have been most useful to those participants who had prior 

PACT training and those who were able to fully participate in VC activities. In addition, participants 

at VAMCs were considerably more likely to have had prior PACT training and to be fully engaged in 

VC activities than were those at CBOCs, indicating that VAMC participants have had more overall 

exposure to PACT training of any kind. Further, RNs report receiving greater benefit from the VC 

compared with other team members. 

Together, these results suggest that the VC has been less successful at engaging certain subgroups of 

learners most in need of greater inclusion and training in PACT, namely primary care staff at CBOCs, newly 

formed PACTs, and support staff. While available data do not provide a definitive explanation for this 

finding, they do point to a number of possible explanatory factors: 

Table of Contents
Page | 14 

Those with greater overall exposure to PACT training are in many ways better positioned to perceive a 

benefit from the VC because they are starting from a comparatively stronger base: they are familiar with 

the basic concepts and have a scaffold on which to build; they and their team members likely have more 

experiential (as opposed to purely conceptual) knowledge of PACT; and they may feel greater 

motivation and perceived capacity to attempt PACT-related changes by virtue of their longer immersion 

in “PACT culture.”

It is not surprising that those who lacked prior training derived less benefit from the VC when one 

considers the circumstances under which the VC was launched. As noted earlier, the VC model as it was 

originally conceived assumed that learners entering the VC would have at least some foundational 

knowledge of PACT. When the Center of Excellence trainings were suspended shortly before the VC 

launch date, organizers had to decide whether to start all teams at the same time despite differences in 

basic knowledge, or to proceed as planned. They opted for greater inclusion but had little time to adjust 

the model to compensate for this unexpected change in circumstances. This may account for at least 

some of the difference in perceived benefit between participants with versus those without prior 

training. 

Teams at CBOCs may experience greater tension between clinic demands and expectations for VC 

participation, as they often have less back-up and are thus more vulnerable to staff absences. Many 

CBOC teams also have less frequent and intensive contact with local coaches and other PACT experts 

due to their geographic distance from a VAMC. 

Whether at CBOCs or VAMCs, teams at an earlier stage of team development (i.e., recently formed 

PACTs) are at a disadvantage compared to more mature PACTs, in that they must learn what it means to 

operate as a PACT even as they are asked to engage in practice improvement work as if they were 

seasoned PACTs. 

A number of participants commented that the content of the VC was not relevant to their needs. Clerical 

and clinical associates in particular voiced a wish for more content specific to their role functions under 

PACT, and newer teams may need more content on team-building than has been offered so far. 

In terms of both content and timing, the VC has been shaped to meet the needs of providers; however, 

less attention has been given to other team members, especially clerical associates, who remain the 

least involved group.

II. The potential benefits of the VC are hampered by challenges related to time and staffing that 

mirror barriers reported to impede PACT implementation on a broader scale. Acknowledging and 

addressing these constraints in the context of the VC may improve learner engagement and 

motivation for change. 

“...Only my RN comes to the collaborative, no one else from my team (LPN, clerical staff) has time or has been 

released from work tasks. I think if we value something then we make it a priority so if the PACT is truly a priority, 

set aside time away from work to let people attend.” (PCP, VAMC)

“...many people are filling multiple roles [on] PACT teams [and] in other areas of the clinic as well. PACT is a good 

concept in theory- however it cannot be implemented to its full potential if we are not given the time and 

resources to do it. We no longer have the time for team huddles; team meetings due to short staffing” (RN, CBOC)

“We are so severely understaffed that going to these sessions is like sending a diabetic to lunch at a candy store. 

What's the use when you can't avail yourself of such wonderful 'theoretical’ concepts. PACT is flying overhead 

and we're still in the bunkers.” (PCP, VAMC)

“...not enough people work at my CBOC to form a PACT. There is no team. So, since there is no team and no PACT 

at my CBOC, having to participate in the Virtual collaborative is like teaching a computer course to someone 

without a computer. Worthless.” (PCP, CBOC) (See more quotes)
Page | 15 

Dr. Macpherson has estimated that teams need a minimum of one hour a week set aside to 

dedicate to practice improvement work. Round 2 results underscored the ongoing challenge of 

creating and preserving time for team-based practice improvement work in the face of competing 

priorities and the need for teams and facilities as a whole to adapt to ongoing resource constraints 

(e.g., staff absences, loss of staff, increased panel sizes). 

Protected time is still not universally available, and where it is, there are signs of erosion: 

The percentage of respondents who reported that their teams met at least every two weeks was 

significantly lower in Round 2 than in Round 1. While we cannot infer from this that team meeting 

frequency has decreased overall, since we assume the two groups are independent, qualitative data 

lend support to this possibility. 

Protected time is not equally available to all team members. Qualitative data suggests that, as with 

content and timing, local VC implementation has prioritized the needs of providers, rather than those of 

the entire team. In some cases, there is an expectation that all team members will participate but clinic 

schedules have not been restructured in such a way to make this consistently feasible. In other cases, 

support staff are no longer expected to participate in any regular way. 

In addition to issues of time, challenges related to staffing levels persist: 

Fluctuations in available staff on any given day hamper VC participation, as clinic demands and patient 

care will always take priority over training activities. While this is true for all teams, regardless of staffing 

levels, clinics with fewer available staff to help cover for absent team members are particularly hard hit. 

Many survey respondents and key contacts described team configurations that do not correspond to the 

ideal team model at the heart of PACT (i.e., 1 PCP, 1 RN, 1 LPN, 1 clerk), and which is the sole model 

reflected in all PACT training materials. While pilot teams either were built or were already staffed in 

accordance with this official model, more recently formed PACTs are often configured according to 

some “hybrid” model or lack key roles on the team. Write in responses indicate that many frontline staff 

do not feel that the VC “speaks to” the particular realities and needs of their team, and many conclude 

that they cannot implement PACT until their team matches the ideal model. While this may be a 

misperception in some cases, in certain cases there is a real need for additional staff in order for PACT to 

move forward. It may be worth looking more closely at how thinly spread people are, to ensure that 

expectations set for teams are realistic. 

III. Perceived lack of leadership support at certain facilities poses an unaddressed barrier to achieving and 

sustaining teams’ engagement and optimism about PACT. 

Open-ended survey responses convey considerable frustration from staff at facilities where there is a 

perceived lack of support and engagement by executive leadership. Many comments express varying 

degrees of skepticism regarding local leaders’ understanding of and commitment to PACT: 

“I am not sure that our ACOS fully supports PACT; trusts us; wants to empower patients; values the effort I put 

into building relationships with my patients.” (PCP, VAMC)

“The PACT concept is good but our suggestions are not taken. Management wants to ‘get the numbers up’ and 

have threatened the providers with being fired.” (Clinical Associate, VAMC)

“[Changes at the team level are hampered by] administrative red tape ... [Let] us make changes on the lowest 

level - everything does not need to come from the top down - if we had some independence to arrange our 

scheduling, I think there would be less overtime, more quality care delivered and more employee satisfaction.” 

(PCP, CBOC)

“[The VC] means nothing [because] things you talk about are NEVER implemented at [our] facility. ... going to the 

collaborative is torture ... I go learn about things other facilities try and do [that] we aren't allowed [to do] ... I get 

to hear things that the National and VISN recommend but they are ignored at this facility ... I am looking for 

someone above me to finally care and listen to what I am saying. ... Don't just go to the senior management here 

... Come see the people that try to implement PACT [and] hear the truth.” (PCP, VAMC) (See more quotes)
Page | 16 

Some respondents are doubtful about leadership’s investment in PACT in comparison with other 

competing priorities. 

Others feel that the PACT Compass metrics are poor indicators of meaningful change and are subject 

both to misinterpretation and manipulation by executive leadership. A number of survey respondents 

and key informants spoke of a discrepancy between performance on the Compass measures and the 

actual situation on the ground; a few believed that local leadership was not only aware of this 

discrepancy but was also exploiting it to create the impression of compliance without actually 

supporting needed changes. 

In addition, frontline staff at certain sites reportedly have little or no input into planning and decisions 

around PACT. Survey and interview data describe situations marked by excessive “administrative red 

tape” and a lack of autonomy at the team level. 

Finally, we have heard isolated reports of teams who feel they are being “punished” for high 

performance by having vital resources withdrawn. For example, one high-functioning multi-provider 

team that lost a PCP was informed that the PCP would not be replaced since the team had been doing so 

well. 

Opportunities for Improvement 

Several opportunities exist to enhance the VC experience for those who have benefited less to date. In 

particular: 

 Presenters could be explicitly prompted to consider the particular needs of key target groups when 

developing their presentations. Survey results suggest that the role functions of clinical and clerical 

associates as well as CBOC-specific circumstances and implementation challenges deserve special attention. 

 Newer PACTs, particularly those at CBOCs, could benefit from additional support around team-building and 

basic team processes, for instance through more intensive coaching and greater opportunities for contact 

with more seasoned teams. 

 Homework, while unpopular, can be a useful catalyst for change provided that it is truly a team activity. As 

the VC moves into its next phase of more team-driven practice improvement work, coaches should assess 

not only whether but also how and by whom assignments are completed. 

The changes needed to fully resolve persisting barriers around time and staffing are beyond the VC’s sphere of 

influence. Nevertheless, there are ways in which the VC might mitigate the impact of these challenges and 

facilitate movement toward their resolution: 

 ALL team members need protected time to attend VC sessions and participate in team meetings, especially 

those without dedicated administrative time. Without such protected time, it will be hard for them to 

engage in, feel engaged by, and move forward with practice improvement work, and the VC may 

inadvertently contribute to staff burnout. The importance of maintaining protected time should be reemphasized frequently, at all organizational levels. 

 Future VC content should reflect and speak to the actual conditions in which teams are working, including 

the reality of persistent resource constraints and sites’ adaptations to those (e.g., alternate staffing models). 

 Coaches should be equipped with resources and tools to help teams (including those that are not yet 

structured or operating as PACTs) problem-solve around ways to adapt PACT principles and strategies to 

their specific local conditions. Coaches may also need tools to engage more passive or resistant team 

members, and to help teams through their “growing pains.” VC organizers could support coaches (and 

ultimately teams) by facilitating some form of structured virtual forum (e.g., a monthly telephone meeting, 

an online forum), which would help coaches network, share information and ideas, engage in collective 

problem-solving, and ask questions of PACT experts around the VISN. 

 Teams at some sites may need support from outside the facility to address larger systemic issues. 

Intercession at the executive leadership level might be warranted in cases where local facility leadership 

appears to be working at cross-purposes to PACT (e.g., reducing staff on high-performing teams, scrutinizing 
Page | 17 

open slots on clinic schedules, inhibiting team-level autonomy to innovate). Leaders at certain sites may 

benefit from targeted education about PACT concepts and processes in order to ensure that teams have the 

resources (staff, time) and autonomy needed to realize the goals of PACT. 

 In some cases, there may be a real need for additional staff in order for PACT to move forward. It may be 

worth looking more closely at how thinly spread people are, to ensure that expectations set for teams are 

realistic.
Page | 18 

VII. TABLES 

Survey Sample (Return to text) 

Table 1. Characteristics of study sample 

Response Rate

Surveys sent 820

Responses 371

Disqualified 18

Total Valid Responses 353

Survey Response Rate 43%

Respondent characteristics (n=353) (% of total)*

Role:

PCPs 136 (39%)

RN/Care Managers 87 (25%)

Clinical Associates 61 (17%)

Clerical Associates 17 (5%)

Facility type:

Primary Site is VAMC 168 (48%)

Primary Site is CBOC 142 (40%)

Training exposure

Had prior PACT training 125 (35%)

Attended 5-7 Live Meeting sessions (since April 2012) 191 (54%)

Parent facility:

Altoona 22 (6%)

Butler 24 (7%)

Clarksburg 23 (7%)

Coatesville 11 (3%)

Erie 28 (8%)

Lebanon 51 (14%)

Philadelphia 56 (16%)

Pittsburgh 33 (9%)

Wilkes-Barre 34 (10%)

Wilmington 28 (8%)

 * Percentages may not total 100%, as some respondents chose not to answer these items.

Table of Contents
Page | 19 

Extent of Participation (Return to text) 

Table 2.1. Mode of Live Meeting participation 

Attended at least one Live Meeting session 328 (93%)

Attended with other team members 116 (35%)

Attended with other teams 130 (40%)

Attended on own 75 (23%)

Table 2.2. Participation in VC activities 

Attended 5-7 sessions 191 (57%)

Met every 2 weeks or more 158 (49%)

Met for 60 min or more 39 (15%)

Did all homework 98 (31%)

HW done as team 128 (54%)

High Dose 84 (31%)

Full Experience670 (20%)

Table 2.2.1. Comparisons by VAMC vs. CBOC 

VAMC CBOC p-value

Attended 5-7 sessions 56% 62% 0.28

Met every 2 weeks or more 55% 44% 0.05

Met for 60 min or more 16% 12% 0.32

Did all homework 27% 35% 0.18

HW done as team 54% 53% 0.94

High Dose 40% 21% 0.00

Full Experience 30% 13% 0.00

Table 2.2.2. Comparisons by Role 

Clerical 

Associate

Clinical 

Associate

RN/ Care 

Manager

PCP p-value

Attended 5-7 sessions 41% 59% 45% 71% 0.00

Met every 2 weeks or more 53% 54% 52% 45% 0.59

Met for 60 min or more 15% 26% 8% 13% 0.04

Did all homework 35% 36% 33% 26% 0.49

HW done as team 45% 61% 59% 47% 0.27

High Dose 38% 38% 31% 27% 0.54

Full Experience 18% 31% 21% 19% 0.27

Table 2.2.3. Comparisons by prior PACT training 

Prior training No Prior training p-value

Attended 5-7 sessions 56% 58% 0.77

Met every 2 weeks or more 60% 42% 0.00

Met for 60 min or more 15% 14% 0.73

Did all homework 32% 30% 0.65

HW done as team 57% 51% 0.40

High Dose 36% 28% 0.16

Full Experience 29% 15% 0.00

 

6

 “Full experience” defined as attending ≥4 sessions and having a high team meeting “dose.”
Page | 20 

Usefulness and Value of VC (Return to text)

Table 3. Usefulness (% who agree or strongly agree) 

Live meetings are a good use of time 144 (45%)

LM presentations useful 175 (54%)

LM stories and examples useful 195 (61%)

LM polls useful 156 (49%)

Team meetings help with PACT implementation 156 (58%)

Team meetings a good use of time 160 (60%)

HW helps with PACT implementation 55 (22%)

HW a good use of time 46 (19%)

Coaches are useful resource 113 (47%)

SharePoint is useful resource 125 (48%)

VC Coordinator is useful resource 114 (47%)

Table 3.1. Usefulness by VAMC vs. CBOC (% who agree or strongly agree) 

VAMC CBOC p-value

Live meetings are a good use of time 47% 41% 0.27

LM presentations useful 58% 49% 0.11

LM stories and examples useful 61% 61% 0.99

LM polls useful 49% 48% 0.88

Team meetings help with PACT implementation 65% 50% 0.02

Team meetings a good use of time 66% 52% 0.02

HW helps with PACT implementation 20% 25% 0.40

HW a good use of time 17% 21% 0.53

Coaches are useful resource 51% 42% 0.21

SharePoint is useful resource 48% 48% 0.92

VC Coordinator is useful resource 45% 49% 0.49

Table 3.2. Usefulness by Role 

Clerical 

Associate

Clinical 

Associate

RN/ Care 

Manager

PCP p-value

Live meetings are a good use of time 50% 43% 51% 40% 0.44

LM presentations useful 50% 48% 70% 47% 0.00

LM stories and examples useful 63% 60% 75% 53% 0.02

LM polls useful 56% 45% 60% 44% 0.10

Team meetings help with PACT 

implementation

38% 58% 64% 58% 0.39

Team meetings a good use of time 54% 58% 64% 60% 0.88

HW helps with PACT implementation 36% 34% 23% 16% 0.06

HW a good use of time 36% 30% 18% 12% 0.03

Coaches are useful resource 64% 54% 55% 38% 0.06

SharePoint is useful resource 54% 49% 62% 37% 0.01

VC Coordinator is useful resource 64% 43% 56% 40% 0.12
Page | 21 

Table 3.3. Usefulness by prior PACT training 

Prior 

training

No Prior 

training

p-value

Live meetings are a good use of time 55% 39% 0.00

LM presentations useful 65% 48% 0.00

LM stories and examples useful 70% 55% 0.01

LM polls useful 57% 44% 0.03

Team meetings help with PACT implementation 71% 48% 0.00

Team meetings a good use of time 74% 49% 0.00

HW helps with PACT implementation 25% 20% 0.42

HW a good use of time 23% 15% 0.14

Coaches are useful resource 51% 44% 0.23

SharePoint is useful resource 55% 43% 0.05

VC Coordinator is useful resource 62% 36% 0.00

Table 3.4. Usefulness by measures of overall engagement 

High Dose Low or 

Med Dose

p-value

Live meetings are a good use of time 60% 41% 0.01

LM presentations useful 73% 50% 0.00

LM stories and examples useful 74% 59% 0.02

LM polls useful 59% 46% 0.06

Team meetings help with PACT implementation 85% 46% 0.00

Team meetings a good use of time 85% 48% 0.00

HW helps with PACT implementation 38% 15% 0.00

HW a good use of time 32% 14% 0.00

Coaches are useful resource 67% 42% 0.00

SharePoint is useful resource 68% 45% 0.00

VC Coordinator is useful resource 63% 48% 0.04

Full 

Experience7

Not Full 

Experience

p-value

Live meetings are a good use of time 61% 40% 0.00

LM presentations useful 77% 48% 0.00

LM stories and examples useful 76% 57% 0.00

LM polls useful 60% 45% 0.03

Team meetings help with PACT implementation 89% 47% 0.00

Team meetings a good use of time 87% 50% 0.00

HW helps with PACT implementation 37% 17% 0.00

HW a good use of time 30% 14% 0.01

Coaches are useful resource 66% 40% 0.00

SharePoint is useful resource 65% 43% 0.00

VC Coordinator is useful resource 61% 42% 0.00

 

7

“Full experience” defined as attending ≥4 sessions and having a high team meeting “dose.”
Page | 22 

Impact of the VC (Return to text)

Table 4. Impact (% who agree or strongly agree) 

Increased knowledge of PACT 202 (64%)

Increased awareness of PACT at facility 165 (52%)

Increased communication with other PC teams 138 (44%)

Increased access to people who can answer PACT questions 158 (50%)

Increased access to resources related to PACT 183 (58%)

Think participating in VC is worthwhile for someone in role 148 (47%)

Participating is helping team put PACT concepts into action 126 (40%)

Table 4.1. Impact by VAMC vs. CBOC (% who agree or strongly agree) 

VAMC CBOC p-value

Increased knowledge of PACT 65% 62% 0.52

Increased awareness of PACT at facility 51% 54% 0.68

Increased communication with other PC teams 48% 39% 0.15

Increased access to people who can answer PACT questions 53% 46% 0.25

Increased access to resources related to PACT 61% 53% 0.13

Think participating in VC is worthwhile for someone in role 48% 46% 0.76

Participating is helping team put PACT concepts into action 40% 41% 0.95

Table 4.2. Impact by Role 

Clerical 

Associate

Clinical 

Associate

RN/ Care 

Manager

PCP p-value

Increased knowledge of PACT 76% 61% 67% 62% 0.57

Increased awareness of PACT at facility 65% 57% 56% 46% 0.20

Increased communication with other PC teams 41% 51% 52% 37% 0.10

Increased access to people who can answer PACT 

questions

47% 61% 59% 40% 0.01

Increased access to resources related to PACT 59% 61% 67% 50% 0.09

Think participating in VC is worthwhile for 

someone in role

41% 41% 60% 43% 0.05

Participating is helping team put PACT concepts 

into action

41% 49% 45% 35% 0.21

Table 4.3. Impact by prior PACT training 

Prior 

training

No Prior 

training

p-value

Increased knowledge of PACT 66% 63% 0.62

Increased awareness of PACT at facility 58% 49% 0.12

Increased communication with other PC teams 54% 37% 0.00

Increased access to people who can answer PACT questions 61% 43% 0.00

Increased access to resources related to PACT 70% 50% 0.00

Think participating in VC is worthwhile for someone in role 58% 40% 0.00

Participating is helping team put PACT concepts into action 50% 33% 0.00
Page | 23 

Table 4.4. Impact by measures of overall engagement 

High Dose Low or 

Med Dose

p-value

Increased knowledge of PACT 78% 60% 0.00

Increased awareness of PACT at facility 72% 49% 0.00

Increased communication with other PC teams 61% 41% 0.00

Increased access to people who can answer PACT 

questions

75% 44% 0.00

Increased access to resources related to PACT 81% 54% 0.00

Think participating in VC is worthwhile for someone in role 60% 45% 0.02

Participating is helping team put PACT concepts into action 66% 32% 0.00

Full 

Experience

Not Full 

Experience

p-value

Increased knowledge of PACT 83% 59% 0.00

Increased awareness of PACT at facility 75% 46% 0.00

Increased communication with other PC teams 64% 38% 0.00

Increased access to people who can answer PACT 

questions

80% 42% 0.00

Increased access to resources related to PACT 84% 51% 0.00

Think participating in VC is worthwhile for someone in role 64% 42% 0.00

Participating is helping team put PACT concepts into action 70% 32% 0.00

Acceptability of the VC (Return to text)

Table 5. Acceptability (% who agree or strongly agree) 

VC format is an acceptable alternative to off-site training 168 (54%)

Feel I am getting the same training through VC as through off-site 131 (42%)

Participation in VC has not interfered with my work 109 (35%)

I like that the VC provides training in small doses 167 (53%)

Table 5.1. Acceptability by VAMC vs. CBOC (% who agree or strongly agree) 

VAMC CBOC p-value

VC format is an acceptable alternative to off-site training 53% 54% 0.83

Feel I am getting the same training through VC as through off-site 39% 45% 0.30

Participation in VC has not interfered with my work 35% 35% 0.99

I like that the VC provides training in small doses 54% 54% 0.91

Table 5.2. Acceptability by Role 

Clerical 

Associate

Clinical 

Associate

RN/ Care 

Manager

PCP p-value

VC format is an acceptable alternative to off-site 

training

65% 44% 57% 55% 0.31

Feel I am getting the same training through VC as 

through off-site

47% 36% 47% 43% 0.59

Participation in VC has not interfered with my 

work

47% 39% 41% 28% 0.11

I like that the VC provides training in small doses 59% 41% 63% 52% 0.06
Page | 24 

Table 5.3. Acceptability by prior PACT training 

Prior 

training

No Prior 

training

p-value

VC format is an acceptable alternative to off-site training 58% 50% 0.16

Feel I am getting the same training through VC as through off-site 49% 37% 0.04

Participation in VC has not interfered with my work 42% 30% 0.04

I like that the VC provides training in small doses 61% 48% 0.03

Table 5.4. Acceptability by measures of overall engagement 

High Dose Low or 

Med Dose

p-value

VC format is an acceptable alternative to off-site training 63% 51% 0.09

Feel I am getting the same training through VC as through off-site 57% 37% 0.00

Participation in VC has not interfered with my work 48% 30% 0.00

I like that the VC provides training in small doses 63% 53% 0.15

Full 

Experience

Not Full 

Experience

p-value

VC format is an acceptable alternative to off-site training 65% 50% 0.03

Feel I am getting the same training through VC as through off-site 61% 36% 0.00

Participation in VC has not interfered with my work 52% 30% 0.00

I like that the VC provides training in small doses 65% 50% 0.02
Page | 25 

VIII. ADDITIONAL QUOTES 

The following pages list a selection of verbatim quotes from VC participants’ write-in responses to open-ended 

survey questions. These quotes reflect the experiences and perspectives of particular individuals, in particular 

roles, working under particular circumstances. In listing them here, our intent is to convey the range of learners’ 

experiences, and we caution against generalization. This being said, it is clear that at least some participants at 

some sites are experiencing significant challenges; regardless of the accuracy of their perceptions, they raise 

important questions for further exploration and possible intervention. 

Praise – What’s been useful (Return to text)

Content 

“Having an ongoing discussion of VA objectives that sometimes seem meaningless to the front line staff [is 

useful]. Truly appreciate Dr Macpherson’s personal commitment to this.” (PCP, VAMC) 

“Increased understanding of how individual factors come together to form one big picture” (Clinical Assoc, 

CBOC) 

“getting a more comprehensive operational understanding of PACT principles” (Clinical Assoc, CBOC) 

“Definitive accurate up to date information” (Clerical Assoc, CBOC)

“looking at case studies in the [VISN] that’s the most helpful, they need to focus on that more” (PCP, CBOC)

“Great concepts. Interesting clinical topics. Innovative, cutting edge, and forward thinking. Thank you for your 

efforts. Impressive use of new communication and leadership tools.” (PCP, VAMC) 

“when a specific topic is presented like palliative care, pain management, etc.” (RN, VAMC)

“The disease specific ones help to dx, tx, and effectively assist the patients in a patient centric environment” 

(RN, VAMC) 

“Evidence based clinical training, especially the pain/opioids session” (PCP, VAMC)

Perspective 

“The information about what other teams are doing (nationally, locally, within the VISN) is good information to 

know” (PCP, VAMC) 

“Knowing that other facilities have the same issues.” (PCP, VAMC)

“It gives me the perspective that none of us are having an easy time putting this into practice” (Clinical Assoc, 

VAMC) 

Contact with Peers 

“increased communication among team members” (Clinical Assoc, CBOC) 

“Getting our teams all together” (Clinical Assoc, CBOC)

“Getting together and sharing information” (Clinical Assoc, CBOC) 

“seeing how other team[s] here work together” (Clinical Assoc, CBOC) 

“Learning how other teams handle different experiences within PACT.” (Clinical Assoc, VAMC)

“I like to see what others are doing so we know other ways to implement and use PACT. It is a very good 

resource.” (Nurse Manager, VAMC) 

“It helps to understand how the PACT teams are functioning both at our facility and in VISN in general.” (PCP, 

VAMC) 

Table of Contents
Page | 26 

Format 

“several teamlets get training at the same time, rather than one group at a time” (Clinical Assoc, VAMC)

“that many people are receiving information at the same time” (PCP, VAMC) 

“Polls help give great insight into clinical thought processes and how you align with the group” (PCP, VAMC) 

“training in small doses” (PCP, VAMC)

“small steps approach [although it] does feel disconnected at times from what is actually occurring at the local 

or clinic setting” (PCP, CBOC)

“saves time” (PCP, CBOC) 

“reduction of travel makes it much less of a negative impact on group and team productivity” (PCP, VAMC) 

“not having to attend off-site conferences [has been useful]; would appreciate continuing VC over off-site 

training” (Clinical Assoc, VAMC) 

Overall 

“Revamping how things are done is a good thing. Killing institutional complacency is a good thing. Getting 

providers and nurses out of their comfort zone in the name of making the product and process better is a good 

thing. It's like changing coaches on a team. An apt analogy for local Steelers fans is Ben Roethlisberger; he has a 

new coach, he can't run around and improvise like it's his playground, he's throwing 3-4 touchdowns a game. He 

was forced out of his comfort zone in order to accomplish something more. I may not always like homework, 

meetings, and perceived busywork; I greatly appreciate the room for more autonomy with my patients and 

schedule. Keep it up.” (PCP, VAMC) 

Constructive criticism 

Content (Return to text)

“[To improve the VC] present info on implementing PACT in CBOCs and the particular challenges in doing this.” 

(RN, CBOC) 

“I got no information as a clerk how to do the daily items. Scheduling PACT calls. Scrubbing schedule concepts.” 

(Clerical Assoc, CBOC) 

“I would like for these meetings to describe what we are supposed to be doing in the clinic setting. The only 

difference our PACT team is doing is scrubbing the schedule and making telephone visits. There is no time to use 

the TEACH training I received. We are tired of the statistics which do not benefit us.” (Clinical Assoc, CBOC) 

“All they do is give you statistics which can be manipulated the way you want them. There is no training on how 

to explain to a patient that he's doing well, a face to face is not needed. To many of these patients it is a ‘social 

time’ for them. The only time they get out of the house and the travel pay is quite lucrative.” (Clinical Assoc, 

VAMC) 

“Segment it for different team members to collaborate with each other. Like just for RNs, just for LPNs and have 

focused sessions. Or have more time included for teams in the same building [to] discuss ideas and changes.” 

(Clinical Assoc, CBOC) 

“I would like to see it a little more geared to nursing and front line staff” (Nurse Manager, VAMC) 

“It could be useful if we were allowed time for discussions, both within our group (that's connected by one 

computer), and if we were able to talk with other people. The problem becomes, everyone is at different stages 

of implementation and have different levels of staffing that it is hard to compare notes outside of the facility.” 

(PCP, VAMC) 

“VAs across the country are at different stages. Going over information about Telephone grids and team huddles 

when our team did not have the grids or time for huddles is not helpful. Education would be better if closer to 

the time of starting the PACT in our CBOC” (PCP, CBOC) 
Page | 27 

Time (Return to text)

“always good info, wish I could prioritize it more” (RN, VAMC)

“Having time freed up to actually attend the sessions and for my team to meet at all [would improve my 

experience]. I have to miss most sessions due to clinical duties, and my team has almost never been able to 

meet.” (PCP, VAMC)

“I think it will be very useful, but we are working very hard to even finish patient notes daily. [I suggest that] on 

the days there is the collaborative we should have lighter amounts of patients to be seen, so the whole team can 

be part of the collaborative. We really need more time to be able to finish our daily work.” (PCP, VAMC)

“[Having] a one day seminar away from the workplace would be much more productive, useful, and would give 

everyone a chance to participate...I think the format is not useful or helpful. More than half the time, my clinics 

have not been cancelled or they have only been partially cancelled so I am eating lunch during the lecture. Only 

my RN comes to the collaborative, no one else from my team (LPN, clerical staff) has time or has been released 

from work tasks. I think if we value something then we make it a priority so if the PACT is truly a priority, set 

aside time away from work to let people attend.” (PCP, VAMC) 

“At this point, these mandatory meetings have to go away. Teams have received the information, but they do 

not have time to get together on their own to work on their individual plans.” (PCP, VAMC) 

“The PACT is being pushed administratively but not being practiced by the majority of the teams because we 

don't have time, it hasn't been made a serious priority in actuality (in theory it has), team members haven't been 

able to attend, we are getting the information in a piecemeal fashion that hasn't really taken hold.” (PCP, VAMC) 

Staffing 

“I have been rarely able to participate due to my workload. It is a nice concept.” (RN, VAMC)

“I agree with PACT team philosophies and appreciate the Virtual Learning events. However, our site does not 

have even minimal support staff to implement this program. We had one LPN for 4 providers, and now she has 

been pulled to manage [another] clinic. We do not have adequate SW, or Behavioral Health support. The PACT 

concept is good, the implementation and commitment to the underlying structure that is required has been 

poor.” (PCP, CBOC) 

“The Virtual Collaborative would be great if all facilities were able to implement it. Unfortunately, resistance 

from our local leadership to make changes to help us implement PACT has been a MAJOR issue and being 

extremely short staffed has been another barrier. Unfortunately, we have no LPNs or Health Techs and the RNs 

do all the triage, phone calls, run all the specialty clinics, do all the dressings, nursing visits, etc. At times, [we 

have] 2 providers sharing 1 RN with no additional help. This makes it quite difficult to implement any PACT 

concepts and basically we are NOT doing PACT. It [may] look like it on paper but we are not doing it because of 

the stated barriers. It would be a great concept otherwise.” (PCP, CBOC) 

“[Our facility’s] PACT model states we should be afforded one RN, one clerical and one clinical per PCP. The 

clerical and clinical person is the same, meaning their time is split between two separate jobs, but more 

importantly, there is not enough staff to properly implement PACT. We do not even have the ability to do 

huddles, which when we began this process, we could and it was very useful. Now our clerical staff are assigned 

multiple providers a day, and are not able to participate because they are not familiar with the team or patients 

anymore. ... [At this] VA, the PACT model is just a slogan, therefore the VC is not of use.” (RN, VAMC) 

“Virtual collaborative is fine and I agree with PACT concepts, but we are still not staffed for PACT and this leads 

to low morale and a significant barrier to achieving PACT metrics. Burnout is increasing due to increased staffing 

burdens and pressures to perform with inadequate staffing...and I'm not sure the Virtual Collaborative can help 

with this in the current fiscal/FTE climate.” (PCP, VAMC) 

“A hybrid concept was introduced to have 1/2 RN per panel. Try doing that for a clinic with 2.4 providers. It 

doesn't work... The concepts presented were great. The attempt by management to implement the concepts 

without the agreed upon definition of a PACT team is morale deflating.” (RN, CBOC) 
Page | 28 

Leadership (Return to text)

“Nothing done in the virtual collaboratives is going to help if we don't have someone enforcing our facility to 

participate in pact the way they are supposed to. Our facility can make things look good on paper but not in 

practice; PACT needs to be uniform in the facilities otherwise leadership can make things look like they are 

happening when it really isn't. PACT has done nothing but increase my work load as a primary care provider.” 

(PCP, VAMC) 

“I do like the small steps approach but it does feel disconnected at times from what is actually occurring at the 

local or clinic setting. [Suggest reducing] the administrative red tape – there are several ways that I think my 

team could function better but as providers/PACT teams we cannot adjust or rearrange our own schedules, 

nursing cannot be utilized as case managers because we have no one else to ‘room’ patients – they cannot 

provide the care they would like, also pts are not as agreeable as I thought to less visits and more telephone care 

– they WANT to come in to see us! Let us made changes on the lowest level – everything does not need to come 

from the top down – if we had some independence to arrange our scheduling, I think there would be less 

overtime, more quality care delivered and more employee satisfaction. At the team level there are very few 

changes that can be made to implement PACT to the best of our ability – our hands are tied by the 

administration!” (PCP, CBOC)

“MANAGEMENT is the MAJOR BARRIER of PACT. They refuse to let anyone work to the top of their license...treat 

nurses like they are 2nd rate health techs [and] won't allow the teamlet to work together at all!! If we start 

changing a schedule management comes in and threatens the clerks. Our management has put in a so called 

supervisor clerk to watch everything the clerks, nurses, and providers do and reports back. Our [pilot] team was 

[supposed to try] new things [to] figure out how to make things work with the personnel we were given but [is] 

reprimanded for trying anything. Nursing is really strapped [constrained in what they are permitted to do 

independently]. I am not asking for anyone to even spend more money at this time I just want someone to allow 

us to work for the Veteran.” (PCP, VAMC) 

“Upper management selectively chooses what parts of PACT we will apply and makes dramatic about faces on 

policy, making it difficult to now from day to day what we are doing. This also effects the patients’ ability to 

understand what is going on... The front line staff feel that PACT is a very effective process, but we have no 

input into the actual workings here. We do not even have the opportunity to "buy in" to the process before it is 

changed without warning [and] we do not have adequate staffing to implement the policies and procedures 

effectively. Managers and supervisors are figurehead with little or no training or experience and so they 

become mouthpieces for management instead of a conduit of information and data both ways.” (Clinical 

Associate, VAMC) 

“I lost a lot of motivation when I found out that my incentive pay was tied to PACT outcomes TWO MONTHS 

before the end of the evaluation period. It’s not that I need incentive pay to be motivated to implement PACT. 

The basic concepts of patient-centered care make a lot of sense. I like the idea of PACT. But when the higher ups 

change the work conditions and priorities seemingly at random and without notice or input from those of us ‘in 

the trenches,’ I start to wonder if PACT is really a priority for the institution or just some passing management 

fad. I’m burning out.” (PCP, VAMC) 

“[Our] institution is resilient to implement any useful PACT changes ...stop [the VC and come] to this facility and 

check directly what was implemented at levels of LPN, RN care coordinator, clinical pharmacist, dietician, social 

work. Discuss directly with PACT providers and team members and not with service chiefs, which do not 

practice, and have no clues of pt care and PACT model of care.” (PCP, VAMC) 

“Getting together and sharing information is good. Except at [this] VA, it appears to be management making 

PACT look good, even though they don't implement the ideas or listen to suggestions. [This] VA is not 

implementing PACT in a useful way. It makes things SOUND good, but if anyone would really look in to the 

details, they would see that [this] VA is moving away from PACT (while still collecting the money to implement 

it) in MANY ways. ... PLEASE, PLEASE send an objective observer to the [VAMC] Primary Care dept and see how 

far away from PACT we are getting!” (Clinical/Clerical Assoc, CBOC) 
10#30#12

Team Communication Exercise (TEX) Executive Summary

Carole'Warde'MD, Marjorie'Pearson PhD

Christina Huang MPH, Lopamurda Das MPH

November 5, 2012

INTRODUCTION: Why did we start?

As part of the VA'effort to build a stronger national health'care system, primary care has been'

reorganized into a patientGcentered medical home model called Patient Aligned Care Teams (PACT).

Health professionals and support staff have been'regrouped'into'multidisciplinary units that

include a primary care professional (MD or NP), a RN Care Manager, a LVN, and a medical service

assistant (MSA). These PACT'teamlet members are now required'to integrate'their work to better

deliver excellent care to Veterans, and to continuously improve it. A wellGfunctioning care team is

THE'essential ingredient in a better primary care system.' However,'these newly formed teamlets

have little or n training in'how to'work together as an interdisciplinary'unit.' Additionally,'health

professional trainees are often'present and they must know how to work'in'wellGfunctioning teams.

Our VAIL innovation aimed to develop an interactive practiceGbased educational program'to assist

PACT'teamlets with'trainees to learn,'improve and coach effective'team function.' In this report,'we

describe how we planned, pilot tested'and'assessed'our educational program and'make

recommendations'to improve future teamGbased coaching programs.

INNOVATION DESIGN:% What were the innovation plans?

We'focused our innovation on improving team function. Our conceptual model included an

adaptation of a widely used framework conceptual model of team function from the business

literature that emphasizes the three domains below (1):

•	 Task work is the work that teams perform.' It involves understanding the'nature'of the'task

and the skills individual team members need to'perform the work. In our program,'we

chose the task work to be a teamlet’s quality'improvement practices rather than their

patient care because it involved a definable knowledge and'skill set and because the

teamlets initial task of integrating their work as a team is an improvement process at

heart.'

1
10#30#12

•	 Team work involves the steps or processes that team members use to coordinate their

actions. We defined this aspect'of team function as specific interpersonal communication

behaviors used by teamlet members in'order to perform their work'together.

•	 Team processes are the structures'that'enable team members to coordinate their work. In

our case, this meant protected time for team meetings and effective meeting processes.

For the content of the three domains,'our conceptual model drew on the literature on quality

improvement (2), relationshipGcentered communication (3, 4) and effective meeting practices (5G

9). We chose practice3coaching methods to deliver the innovation to the participating teams (10,

11).

The intervention planning,'tool development,'piloting,'and refining was planned to take place over

8 months. Curricular planning and tool development corresponded to key behaviors we hoped our

learners would acquire in each of'the three areas of'our curriculum:

•	 Six'QI behaviors G setting aims, identifying problems, establishing measures of

improvement, selecting changes,'testing changes by using PDSA cycles,'and implementing

changes.

•	 Nine team communication behaviors G active listening,'trust building,'managing emotions,'

making decisions by consensus, role definition, holding members accountable, giving

feedback, receiving feedback, and supporting change in others.

•	 Eight team meeting processes G setting and following ground rules,'clarifying meeting aims,'

reviewing and assigning meeting roles, negotiating agenda and time allotments, working

through agenda by discussing information, reviewing the meeting actions, planning the next

actions, evaluating'the meeting

The intervention'was structured to include five inGperson'sessions, each two hours in'length, with

the teamlet'at'its facility:' one for engagement,'another to'perform the needs assessment; two for

coaching; and one at the end'for summary'and'evaluation. Four conference calls (one hour in

length) with the teamlet leaders occurred after the teamlet coaching sessions and these were

completed before the final evaluation'session.

Key intervention activities included:

•	 Engagement of participating teamlet members and their respective administrative

supervisors for motivation building and understanding of the innovation.

2
10#30#12

•	 An'assessment of team function'needs for each teamlet at the beginning;

•	 Collaborative discussion with'each teamlet of the'assessment findings and corresponding

learning goals for the intervention;'

•	 Coaching'for knowledge and skill building in communication,'meeting,'and QI practices

tailored to each teamlet’s identified needs;'

•	 Subsequent handGoff of the PACT coaching'to'the teamlet leaders with monthly conference

calls to support the PACT teamlet leaders in leadership and coaching skills to continue team

function improvement.'

Prior to these'activities and with intervention faculty'assistance, the'senior administrator at each of

the 3 VAILGPCC demonstration'sites was asked'to recruit one volunteer PACT'teamlet and to assure

that'a oneGhour time slot for weekly team meetings would be protected from other patientGcare

duties and dedicated'to'QI.

EVALUATION DESIGN: What was the evaluation plan?

We designed'a formative evaluation'with'the purpose of providing information that would guide

program improvement to program developers and administrators (12). We documented, assessed,

and reassessed the innovation and its implementation throughout'the course of the intervention,

including the contextual challenges, adaptations, and responses. Since the focus was on the early

stages'of program'development and the participant samples were small; the evaluation was not'

intended as summative, but we did monitor the effectiveness and impact within these initial

samples. To monitor'effectiveness, we followed a wellGestablished education evaluation framework

(13) that'focused on four levels of effectiveness:' 1) the reactions of the trainees to the program;'2)

their learning (knowledge and skills);'3) change in their behavior; and 4) results in terms of

programGrelated outcomes. We also'focused the formative evaluation on key aspects of'the

curricular design (the triple focus on team relationships, team meeting processes, and quality

improvement and the coaching strategy) and the implementation approach (protected time,

delivery structure,'leadership,'team composition,'and contextual strengths and challenges.)

The evaluation'planned to collect and triangulate both quantitative and qualitative data, including

selfGreported data collected from the participants'(teamlet members'and administrators)'through

interviews, questionnaires, and course evaluation forms;'coach assessments synthesized from data

collected through visit and conference call observation; and results from the relational coordination

survey (RCS)'administered to teamlet members before'and after the'intervention. The RCS is a

validated 7Gquestion'survey that measures team communication'and'relationships (14).' It has

3
10#30#12

been'used to measure improvement in'team coordination'and has been'associated with positive

changes in quality of care, lower costs, and improved satisfaction among patients and clinicians

(15).

INNOVATION IMPLEMENTATION:% What did we do?

Implementation of the team function intervention, including development, piloting, and refining,

took 11 months to complete (3 months longer than originally planned).' The implementation

closely followed the structure that was planned. For each site, the coaches provided five inGperson'

coaching sessions to the teamlet and four coaching calls to the teamlet leaders.' The inGperson'

sessions'included:

•	 One engagement meeting to explain and discuss the intervention,'identify an area of'

improvement, and to solicit input and support of the teamlet members and site leadership,

•	 One needs assessment visit to get a “snapshot “ of each'teamlet’s functioning needs through'

observation of a teamlet QI meeting, interviews with'teamlet members, and'administration

of the Relational Coordination Survey,

•	 session in which a summary of the results of the needs assessment was presented to the'

teamlet'and learning needs were collaboratively identified during the first'half of the

session.' Coaching of specific skills'in communication, meeting, and QI were initiated in the

second half of the session to address'these needs.

•	 Another'coaching session targeted to the reminder of the group’s learning needs.

•	 final session that included observation and interviews to get a followGup'“snapshot” of the

teamlet’s functioning and a joint'evaluation and feedback discussion.

The engagement session was'one hour'in length and the other'sessions'were two hours. Multiple

tools (e.g., handouts, exercises, pocket'guides)'were provided throughout the intervention.

The four coaching calls began'by asking the team leaders to provide an'update on'the teamlet’s

meetings, relationships, and QI activities and to specify what they would like to work on. The

coaches then responded to these requests. The calls were one hour in length at one site and a halfG

hour at the other site.

INNOVATION EVALUATION:% What did we%find? What changes do we

recommend?

4
10#30#12

Our innovation pilot was intensive and small. We present our key observations based'on'this

experience, and discuss their possible implications for future efforts within the VA to'facilitate team

communication and improve team care within PACT. Detailed curricular recommendations are also'

are provided. Neither the'participant numbers (2 teamlets with 6 members each) nor the'

evaluation design (preGpost with no comparison'group),'however, allows us to generalize to other

VA teamlets with any degree of confidence.

Was it effective?

Evaluation'findings at the two participating sites suggest that the TE intervention'was effective in'

improving team function in some, but not all, respects. summary of our findings follows.'

Participant Reactions: In this section of the evaluation, participants assessed the acceptability of

course content and structure and the effectiveness of the coaching practices. Most teamlet

participants in'the TE interventions were highly satisfied with the quality of the course and found

the content'useful and interesting. One teamlet consistently had more positive reactions, but both

generally'were positive. Participants liked that the coaches came to them (onGsite coaching). They

rated all coaching activities as useful, very useful or extremely useful. The general coaching'

practices that'participants found most helpful'were those that enhanced motivation, provided

education and offered encouragement. Other coaching activities valued highly by either site were:'

learning QI skills and processes, facilitation of'discussion to elicit ideas for change; encouragement

of goals; and help to'set priorities. The least effective aspect of coaching was in providing links to

external sources and tools.

Participant Learning: Teamlet members generally exhibited low levels of knowledge and skills in

all three areas of team functioning'(team meetings, relationships, and QI processes) when TEX

began. At the'end, they'reported that they'had gained knowledge'and skills, but the'specifics varied

among'the different respondents. They'particularly'reported increased awareness and attention to'

interpersonal dynamics including:' 1) being mindful of'others; 2) emotion'management; 3) active

listening to team members; and 4) communication during clinical'days and at other times to share

goals,'successes, and frustrations.' The participants’'discussion of lessons learned suggested that

their mindset'had'progressed'from “individual responsibilities” to “team responsibilities.”

Regarding QI processes, site 1 teamlet’s discussion suggested that'they did not'come to understand

or accept the ongoing nature of QI,'but rather retained the view of it as oneGtime project that they

had'completed'at the point the TEX intervention'ended. The other teamlet, o the other hand,

5
10#30#12

expressed clear recognition that they'would continue'to have'work to do and room for

improvement.

Behavior Change: Some behavior change was evident.' The RC survey results suggested a trend

towards improvement'in relational coordination, particularly at'site 2. The coaches also observed

positive behavior change at this site. Improvements were noted in'the teamlet’s ability to 1) make

decisions collaboratively; 2) receive feedback; 3) manage emotions; and'4) support change in'

others. In the their final assessment, both'teamlets reported'improvement in their team

relationship behaviors, team meeting behaviors, and QI behaviors,'with site 2 consistently

reporting higher'increases. By coach assessment, however, only site 2 changed how they

conducted meetings; site 1 never coalesced as a team. Performance of QI processes proved difficult

for both teams. One team (site'2) did make'headway'but not across all QI processes. The'other

team made little headway.

QI Results: Structural or process redesigns resulted, with some reported improvement in

workflow. At site 1, the structural redesigns that'did not'require much teamlet participation or

decision'making (new electronic consult'and schedule of availability)'appeared easier'to implement'

than the workflow'process changes at site 2 that necessitated collaboration among multiple team

members and new ways of relating to each other (new care algorithm).

Was it$feasible?

While the original plans called for three teamlets, it only was possible to recruit two.'

Administrators at one of the VAILGPCC Demonstration'sites were unable to designate a team or

assure protected time. Teamlet uptake of the intervention'took'longer than'expected'due to clinical

demands, vacations and'possibly motivation.

Implementation of this coaching intervention proved feasible in the two participating sites.'

Facilitation in the application of all three'coaching components (motivation, education and

consultation) was essential for the success of this intervention. Other necessary coaching skills

were in team relationship behaviors, especially to identify and address factors indicative of teamlet

dysfunction.

Site and teamlet leadership and staff motivation proved to'be implementation facilitators in some

instances. Some of the challenges to smoother'implementation included the ongoing time pressures'

on the teamlet members (even with'protected'time), staffing disruptions,'absence of organizational

6
10#30#12

rewards'for'team performance, minimal job requirements'for'teamwork skills, variation in job

accountability'(different silos), variation in leadership interest and monitoring, and lack of

mechanisms to support teams. Support from site administrators, especially'to'assist teamlets with

crossGdepartmental role, staffing, and'motivationGrelated problems'was'needed.

How could the innovation be improved?

In addition to effectiveness and feasibility, we examined how the curricular and delivery elements

of the innovation performed'in this pilot test. Based'on these findings and'observations, we offer a

number of recommendations for innovation'improvement. These recommendations, along with the

corresponding findings and observations, are presented in Table 1. The key findings and

observations are derived'from the interviews, survey, course evaluation forms, coach'assessments

synthesized from data collected through visit'and conference call observation. The coaches

independently developed the observations and recommendations and then discussed and arrived

at the final lists in Table 1 by consensus.

revised curriculum, based on these recommendations, follows'in Table 2.

7
10#30#12

Table&1. Key Findings and Recommendations for Future Curriculum and Implementation Approach

TEX Design

Characteristic:

Key Findings and&Observations Key Recommendations

CURRICULUM

Triple focus • The needs assessment indicated&that teamlet had&

needs in&all 3 areas of team function

• The teamlet responded positively to&the course

and indicated improvement in all areas

• Mastery of meeting processes was beneficial&to

learning QI&and team communication

• Simultaneous learning of team relationship and QI

behaviors were complimentary

• Continue with&triple focus

• Emphasize mastery of team

meeting processes first

Team Meeting& • The site that mastered the team meeting process • Introduce the importance of

Processes exhibited more&success in coalescing&as team

• Following ground rules does not&come naturally

meeting practices early

• Emphasize continued application

of ground&rules during all&teamlet

activities

Team • While the teamlets showed improvements in many • Incorporate more time for

Relationships relationship skills, they needed&more time and&

practice

• Communication&between&teamlet members is

often&absent and&sometimes actively resisted&

• Especially difficult communication behaviors&were:

trust&building, defining roles, collaborative

decision#making, giving&and receiving&feedback,

and holding teamlet members accountable

• Teamlets did¬ understand&or believe the

feedback from the RC scale

teamlet&members to practice the

relationship skills we identified as

most difficult.

• Increase time for&communication

skills&facilitation and modeling

• Expand the relationship skills

exercises to address all&9

behaviors

• Eliminate use of the RC scale for&

feedback

Quality • The mindset of continuous improvement is lacking • Encourage administrators to:

Improvement and even resisted, seemingly due&to lack of

knowledge of the QI process, time pressures of

clinical work&and resistance to change.

• External motivating factors were lacking for QI

• Teamlet ha a poor grasp&of how to&conduct QI

processes

• They avoided selecting and using measures of

improvement

• Improving work flow processes among all teamlet

member called for high levels of relationship

skills improvement efforts that only involved 1 or

members side#steppe the team#building focus

• Teamlets struggled with the ongoing nature of QI,

as opposed to it being one#time project&

• Resolving barriers as they emerged&during the QI

process required&perseverance, but did&lead&to

successful tests of&change

• Teamlets assessed their own QI skills higher that

a) Set the&tone: “From now on,

we’re going to fix problems not

ignore them; and

b) Set expectation that problem#

solving barriers&is&part of QI.

c) Identify timelines and venues

for&teamlets to share QI work

• Include a brainstorm session to

identify problems and solutions

early on to motivate teamlet&

members and engage&them in QI

• Encourage QI focus on work

flow redesign that involves the

whole team

• Create exercises/tools to practice

each PDSA step

8
10#30#12

did&the coaches

Coaching • All coaching components&(motivational, • Continue to&use all three

educational and consultative) are&useful. coaching components&

• Both&teamlets required extensive follow#up (motivation, education and

coaching to make timely progress on their QI& consultation)

activities and relationship building. • Intensify and sequence coaching:

• Factors indicative of teamlet dysfunction were 1) Introduce trust#building

1) Lack&of commitment activities early and continue&

2) Teams not meeting regularly them

3) Interpersonal relationship&tension 2) Emphasize meeting&processes

4) Resistance to&examining work processes early especially ground rules,&

5) Resistance to&receiving&feedback active&listening, role definitions

6) Lack&of response to coaches’ suggestions 3) Emphasize QI skills once

• Successful coaching methods included: relationship#building is

1) Small group interactive&workshop format underway

2) Modeling • Coaches should&b on the&

3) Handouts # QI and Team Meeting Processes lookout for signs&of teamlet

4) Post#it exercises (equalized member input) dysfunction&and&should address

5) Coach&modeling of behaviors in&all 3 areas them by modeling the difficult

6) Facilitated team meetings communication skills we

7) Teamlet member participation in agenda setting identified above

• Continue successful coaching

methods

IMPLEMENTATION

Protected time • Protected time&is difficult to obtain but necessary • Administrative support is needed&

• Effective use of&protected&time required to ensure that&protected time is:

facilitation and monitoring. 1) Granted (including for&the

• The boundary spanners (pharmacists and& boundary spanners)

psychologists) could not&always participate due to& 2) Effectively used

lack of protected time

Structure • The faculty found the tools in the three areas • Continue to&use the behavior#

useful for needs assessment and&evaluation focused tools in the future • Participants had positive&feedback on course&

structure, particularly: • Continue both&the in#person&

1) Location&at the clinical practice sessions&with the teamlet and

2) Small group workshop format phone calls with&the leaders and

3) Monthly coaching calls with leaders alternate&them

4) Opportunity to apply lessons during clinical work& • Schedule&sessions every to 3

• Onsite sessions allowed coaches to observe and& weeks

engage&teamlet members’ interactions • Engage the team leader early to

• The coaches used in#person&sessions primarily to initiate leadership skill&learning

motivate and educat and the&calls to provide& an assessment of the teamlet’s

consultation and leadership education to leaders current relationships, meeting

• The needs assessment session was resource processes, and&QI experience

intensive for all&parties. Teamlets found it difficult • Incorporate needs assessment

to prioritize the results and feedback into all sessions

• Workshops and calls spaced few weeks apart and eliminate&the&targeted needs

were more animated and productive; interest and assessment session

9
10#30#12

involvement waned&when&spacing was&longer • Se revised curriculum below for&

the next pilot.

Administrative • The teamlets required&administrative&support in • Ad a pre#course session with

Leadership the areas of team function. It would have been administrators – see Table 2

helpful for the administrators to have more

background&o the intervention.

• Expectations and incentives for presentation and

sharing of results were lacking

• Loss of a teamlet member hindered the progress

of member relationships

Teamlet • Co#leadership with PCP&and other health& • Encourage paired teamlet

Leadership professional provided built#in leadership support, leadership.&

and helped to share&the&burden of leading and • Develop and coach a specific

organizing the team. curriculum for teamlet leaders&

• The role of team leader seemed to be new and based&o the needs we observed&

undefined. • Continue the coaching calls

• Team leaders required support for their new role • Encourage teamlet leaders to

• The team leaders had particularly difficult time address dysfunctional team

managing dysfunctional team factors (see above) factors and work with their&

• Teamlet leaders needed&specific skills&to perform administrators to problem solve.

the following leadership activities:

1) Define common goals, roles and approaches;

2) Focus and coordinate team activities;

3) Assess team performance giv frequent&

feedback and provide mentoring;

4) Motivate team members for improvement;

5) Establish positive team atmosphere;

6) Ask for support and resources from supervisors

7) Model good relationship skills

Teamlet • The trainee who was embedded in all teamlet • Include trainees in&future

Trainees activities had a positive learning experience, training sessions when possible

demonstrated&learning in&all 3 team functions, • Select trainees who are&active&

and served as role&model for the teamlet. members of practicing teamlets

• Trainees facilitated positive relationships and& • Take&advantage&of opportunities

learning through curiosity and lack of judgment. for&trainees to lead QI&efforts,&

• Trainees have energy and time for&QI work and motivate other team members

often&led&these efforts. and serve&as role models

Context • We observed that a number of contextual • Encourage administrators to&

challenges&to team function were evident: better support team function by

1) Tension between&standardizatio of member addressing the barriers in&the

roles and the need to be flexible to&team needs work environment that&we

2) Lack&of cross#disciplinary accountability observed.

3) Lack&of team incentives and rewards

4) Workload stress is apparent in teamlet

members, but may not be to administrators

• Psychological safety may be&an issue&for some

• Administrators’ view of team culture were more

rosy than frontline teamlet&members’

10
10#30#12

Table 2. Revised Curriculum Based on Recommendations

All sessions will be 2 hours. They will occur every 2 – 3 weeks to allow the teamlets to apply lessons

Pr

and

eQ

p

co

rac

ur

t

se

ice

S

s

e

k

ssi

ills

o

d

n

u

t

r

o

in

E

g'

n

p

g

at

ag

ie

e

n

T

t

e

car

am

e

l

an

et

d

Su

at

p

t

e

e

r

am

viso

m

r

e

s/

et

A

in

d

gs

ministrators: – This will be a session to

motivate, engage and prepare site administrators for their role in teamlet learning and improvement.

The

•

ir role to be discussed interactively:

•

Get buyGin for the intervention plan, especially the protected meeting time

•

Explain'the intervention, especially the triple focus of the curriculum;

•

Set expectations in all 3 areas and followGu to assure teamlet accountability

•

Stay'abreast of progress in all 3 areas and problem solve with teamlet leaders as needed.

Session

As

1

s

:

ur

M

e

o

co

ck

ns

T

t

eam

ancy

M

of

eet

tea

i

m

ng,

co

C

m

ou

po

rs

s

e%

iti

I

o

n

n

tr

a

o

n

d

d

u

re

be organized as if it were a team meeting. After the

c

s

t

p

io

la

n

ce

an

te

d

am

Fac

m

u

e

l

m

ty%

b

O

er

b

s

s

w

er

it

vat

hou

io

t

n

delay

– the session would

ession, the faculty will assess the team’s meeting

and relationship skills using'developed checklists. The “meeting” agenda'would include:

1. Assign'Team Meeting Roles

2. Introdu

o

ction to'the Course – Overview,'motivation,'TEX structure.' Objectives:'

o

Learn how the course will proceed

o

Learn general concepts of the CCM and'QI models, why'they'are important in PACT

o

Learn and'practice a model for relationshipGcentered team meetings

Identify how our relationships affect'the quality of our teamwork.

3. Participant Introductions and'Learning Goals

4. What are the “Curve Balls” of Team Meetings?

5. Ground'Rules

6. Brief Review of a Model for RelationshipGCentered'Team Meetings

7. Team Meeting Role Play

Coa

8.

chi

D

ng

ebr

Ca

ief

ll

the

1:

Role Play and'Discuss Lessons Learned'

This will be with the team leader(s) to debrief,'identify how the team currently

fu

•

nctions, brainstorm ideas for how to proceed as leader, and review concepts of PACT leadership:

•

Define common goals, roles and approaches to the work;

Focus and'coordinate team activities

11
10#30#12

•	 

•	 

•	 

•	 

Motivate team'members for improvement;

•	 

Establish a positive team atmosphere;

•	 

Seek support and resources from supervisors when needed

•	 

Model good relationship skills

Sessio

Ho

n 2

m

:

e

C

w

h

o

r

r

o

k

n

–

i

read CCM articles

The teamlet will have

com

•	 

pleted the ACIC

c

e

C

v

ar

alu

e%

a

M

ti

an

on

agem

and re

en

tu

t

r

I

n

an

ed

d

it

P

to

at

t

i

h

en

e f

t

a

QCe

cul

nt

ty

e

b

r

e

e

f

d%

or

Ca

e t

r

h

e

e

Q

meeting. Goals of the session:

•	 

Understand the Chronic Care Management Model and how it applies to your'care of patients'

•	 

Define patient selfGmanagement

•	 

Learn knowledge and'practice communication skills to'support collaborative decisionGmaking

•	 

Practice how to assess and'document patients’ selfGmanagement needs

Session

Un

3

d

:

e

C

r

h

st

r

a

o

n

n

d

ic

ho

C

w

ar

t

e%

o

M

in

an

vo

agem

lve pa

en

tie

t

n

I

t

I

s

–

and families in the care

•	 

Goals of the session:

•	 

Define the components of delivery system design for CCM: preGvisit assessment skills and tools

•	 

Learn a model for “Planned'Diabetes Visits”

•	 

Learn and'plan for “Group Diabetes Visits”

•	 

Brainstorm: How can we apply the CCM to our care of patients with Diabetes?

Session

Ho

4:

m

Q

e

u

w

al

or

it

k

y%

:

I

A

m

t

p

te

r

a

o

m

vem

me

en

eti

t

ngs develop a process map and prioritize initial actions

•	 

– Goals of the session:

Adopt the mindset of continuous quality improvement Specify the central role of Quality

•	 

Improvement'in Primary Care reform and PACT

•	 

Review your targeted areas for improvement for CCM

Lear

o

n

	 

QI Skills to'Improve your PACT work:

o	 

IHI'Model for improvement

o	 

Practice writing an'SMMART'aim statement

o	 

Brainstorm measures – existing ones and easy ones to collect

o	 

Review process map

	 

Promote individual and'group'accountability by frequent assessment and feedback

Assess team member needs and teach/mentor them accordingly;

Brainstorm reasons for failure and create a tally sheet to monitor

o Design a small test of change and apply before next session

12
10#30#12

o	 

Coaching Call 2: This will be with the team leader(s) to debrief,'identify how the team currently

Ses

func

s

t

i

io

on

ns

5

,

:

br

R

a

el

in

at

st

i

o

o

r

n

m

sh

id

ip

ea

QCe

s fo

nt

r

e

h

r

o

e

w

d%

t

Com

o pro

m

ce

uni

ed

c

a

a

s

t

le

ion%

ad

I

e

–

r, and review concepts of'PACT leadership

•	 

Goals of the session:

•	 

Learn the model of RelationshipGcentered Care

•	 

Understand the links between patientGcentered communication and team function

•	 

Understand the imperative for selfGcare an stress management in'PACT

•	 

Practice mindful interpersonal communication'strategies with'team members

Define team member CCM roles for Diabetic patients using relationshipGcentered

Coaching

com

Ca

m

ll

uni

3:

cation strategies and collaborative decisionGmaking

This will be with the team leader(s) to debrief,'identify how the team currently

Ses

func

s

t

i

io

on

ns

6

,

De

:

br

R

a

el

in

at

st

i

o

o

r

n

m

scribe a pa

s

t

h

id

ie

i

n

p

ea

QCe

s fo

nt

r

e

h

r

o

e

w

d%

t

Com

o pro

m

ce

uni

ed

c

a

a

s

t

le

ion%

ad

I

e

I

r

–

, and review concepts of'PACT leadership.

•	 

Goals of the session:

•	 

tGcentered care environment

•	 

Gain insight into personal communication strengths and challenges

Lear

o

n

	 

and'practice communication skills relevant to'patients and'team members

o	 

Respond to emotions

o	 

Explore values, beliefs and fears

o	 

Giving and receiving feedback

Coa

func

ching Call

Ho

4:

lding others accountable to role responsibilities

This will be with the team leader(s) to debrief,'identify how the team currently

Sess

t

i

io

on

ns

7

,

:

br

Sh

ain

ar

s

i

t

n

or

g%

m

Bes

ide

t

a

P

s

r

fo

ac

r how to proceed as leader, and review concepts of PACT leadership

•	 

Goals of the session:

•	 

Learn and'practice prep

ti

ar

ces

ing'

–

presentation of PDSA improvement process

•	 

Practice delivering a presentation

Homework: RelationshipGcentered communication processes

Review guidelines for publication of QI results

13
10#30#12

We

ACKN

would

OW

like

LEDG

to acknowledge

EMENTS

Arthur Gomez for his wisdom'and guidance as we'

conceptualized'this'project and Hector'Rodriguez for his insight and editorial suggestions.'

REFEREN

1.

CES:

2.

Salas

Psychol

E,

ogy'Press.

Goodwin GF,

Institute for Health

New

Burke CS.

Care

York

Improvement.

2009

Team'Effectiveness

(pages

The

384

Model

G408)

in Complex Organizations.

for Improvement,

3.

http://www.ihi.org/knowledge/Pages/HowtoImprove/ScienceofImprovementHow

Suchman

toImprove.aspx

AL, Sluyter

. Accessed

DJ, Williamson

10/29/2012

PR. Leading Change in Healthcare:

4.

Relati

Levinson

Transforming

onshipGcent

Organizations

ered'Care.

Using

W, Lesser CS, Epstein

Radcl

RM.

iffe'Pub

Complexity,

patientGcentered Affairs

Developing

lishing.'

Positive

2010; 7:1310G

physician

New York

Psychology and

care. Health 1318

communication

2011

skills for

5.

Giving

Fisher R

In.

, Pa

H

t

oughton

ton BM, Ury

BL,

Mifflin

WL.

Company,

Getting To

6. Scholtes PR, Joiner Streibel BJ. The

NY

Y

Team'H

1991.

es:' Negotiating Agreement Without

Incorporated,'Madison,'WI 2003.

andbook: 3rd Edition.' Oriel'

7. AL,

8.

Suchman Risdon

Lencion

Teams.

i,

www.d

P. The

oc.com

C,

.

Neuwirth

Five Dysfunctions

Accessed

Z,

7/08.

Gorosh MR. Communication on Healthcare

2002.

of a Team: A Leadership Fable. Jossey Bass

9. Nel

Quali

son'

ty'by

EC,'

D

B

e

a

sign.'

talden'

Jossey

PB,'Godfrey M

Goodwin

,'San

M.' Cha

10. MA, Zyzanski SJ,

G

Zronek

Bass

S, et

Fr

al.

anci

p

s

t

co

er'

2

1

00

G

7

Effective meeting skills I in

preventive'service'd

Am

eliver

J Prev%Med

y.'The Study'to

A

Enhance

clinical

Pr

trial

evention'by

of tailored

11.

Pract UP).'

Und

office systems for

ice'(STEPG . 2001;21(1):20–28.

erstanding'

Coleman

http://www.ahrq.gov/populations/businessstrategies/coachingmanual.pdf.

the Safety

K,

Net:

Pearson

A Practice

M, Wu

Coaching

S. Integrating

Manual.

Chronic

2009.

C

Available

are and Business

at:

Strategies in

12. Rossi

Sage Publications,

PH, Freeman

Thousand

HE, Lipsey MW. Evaluation: A Systematic Approach, 6th edition.'

13.

Revisiting

Kirkpatrick'D. Great Ideas Revisited.

Oaks, CA,

Techniques

1999.

Kirkpatrick's'FourGLevel Model. Training

for

and

Evaluating

14.

(1

Development

Training Programs.

);54G59

1996: 50

Gittell

Achieve

JH

Quality,

. High Performance

Efficiency and

Healthcare:

Resilience.

Using

McGraw

the

G

Power

Hill 2009

of Relationships to

15. Gittell

of care,'post

JH, Fairfield

Goperat

K,

ive'

Bierbaum'B,

pain and'fun

et

ct

al.

ion

Impact

ing, and'

of

len

relational

gth'of sta

coordination

y:'a nineGhospit

on quality

al

study'of surgical patients.' Medical'Care 2001; 38: 807G819.

14