**Transcript of Cyberseminar**

**Patient-Aligned Care Teams**

**PACT Training Programs for PACT Teams**

**Presenters:  Carole Warde, MD; Anneliese Buttler, MSW**

**March 20, 2013**

**PACT Training Programs for PACT Teams**

**March 20, 2013**

Molly:As we are approaching go time, I would like to introduce our two presenters for today. Speaking first, we have Dr. Carole Warde. She is the Director of Education for PACT Implementation at the Greater Los Angeles VA Healthcare System, and a visiting Associate Professor of Medicine at David Geffen School of Medicine at UCLA. Joining her today is Anneliese Buttler, master of social work. She’s a Program Specialist at VISN 4 Center For Evaluation of PACT known as CEPACT at the Philadelphia VA Medical Center. I’d like to thank them both for sharing their expertise with us today and at this point, I’d like to turn it over to you, Dr. Warde.

Carole Warde, MD:Thank you. Good morning, everybody. Thank you for coming to this cyberseminar today. On behalf of Anneliese and myself, we’re really excited to share with you what we’ve learned about training PACT teams. 

So in the first half of today’s talk, I’m going to describe an innovation project from VISN 22 Veterans Assessment and Improvement Laboratory called TEX for Team Communication Exercise. We’ve developed this program to teach and improve team function through better communication. I will cover the development, curriculum, implementation, lessons learned, and recommendations for others interested in using this model to train PACT teamlets [00:01:26]. 

In the second half of today’s talk, Anneliese Butler will describe VISN 4 Virtual PACT Collaborative and present key findings from a survey of learner experiences that her demo lab conducted as part of a larger formative evaluation.

Now, to kind of get an assessment of who you all are, I see there are a hundred and fourteen of you out there, we just wanted to ask you a couple of questions to determine what your learning needs might be. So Molly, do you want to take it from here? 

Molly:Thank you. So audience members, I have launched a poll question. Please click the circle next to your answer so we can get a better idea of what you’d like to get out of this cyberseminar. So the question is, “What aspects of PACT training would you like to learn from this cyberseminar?” The first option, Curricular content to train teamlets in PACT principles or Strategies to help teamlet members work together as a team, Innovative models for PACT training and spread, Frontline staff perspectives on PACT training (effectiveness, etc.), General knowledge of how other VAs are training for PACT. So we will give everyone just a few more seconds to answer. We’ve had about fifty percent of the audience answer thus far. So Carole, if you’d like to make any comments while we wait, we can do that.

Carole Warde, MD:So both of our talks can emphasize different pieces, depending on what your needs are so we’re looking forward to getting your answers so we know what to concentrate on. [Interruption] questions.

Molly:Oops, go ahead.

Carole Warde, MD:Go on, I’m done.

Molly:I was just going to say we have had everyone answer, so if you want to talk through the results real quick, go ahead.

Carole Warde, MD:I can’t see the results.

Molly:Oh, okay, I can read through them.

Carole Warde, MD:Oh, yes, I can. Yes, I can. Okay. So it looks like nine percent of you are interested in the curricular content so we won’t emphasize that too much. It looks like the majority of you are interested in strategies to help team members work together as a team. Some of you are interested in innovation models for PACT training and spread. Frontline perspectives on PACT training is eighteen percent, and just general knowledge about how other VAs are training in PACT. So it looks like the top two are really the strategies to work together and general knowledge.

Okay. So I think we’re ready to go to the next question. 

Molly:Excellent. We have launched the next question, which is, What is your perspective for learning? So please select one option. Clinical content – what is the curriculum? Operations – how to put on a PACT training seminar? Or Research – how to study the effectiveness of a PACT training? And the answers are streaming in. It looks like we’ve had just about half of our audience answer already. So we’ll give people about ten more seconds to select their response. We do appreciate your feedback, as Dr. Warde was saying, because it does help us gear the talk towards the audience players. 

Alright, and it looks like the answers have stopped streaming in so I’m going to go ahead and close the poll and I will share the results. Can you see those, Dr. Warde?

Carole Warde, MD:Yes, I can, thank you. So it looks like at least half or more than half of you are coming from the clinical content perspective. Fourteen from actually how to put on something like this, and then thirty-one percent of you are from a research perspective, how to study the effectiveness. So Annaliese, is that good with you? [Pause] Okay, sorry about that.

Molly:Anneliese, I do believe you’re still muted.

Anneliese Butler:Yeah, that sounds fine.

Carole Warde, MD:Okay, great. So I think we’ll go ahead and get going. I’ll go ahead and get going with my portion. Okay, so just getting here. 

So as our primary care system transforms, for teams to make meaningful and innovative changes in the clinical setting, team members need relationship centered communication skills. These skills are also essential for medical teams to work and learn together, which is key if we’re going to transform to PACT and deliver patient centered care.

So it’s pretty clear that a well-functioning team is really the key ingredients to a better primary care system. However, many VA teams have little or no training in how to actually work together as an interdisciplinary unit, rather as individuals. Teamwork training has lagged far behind practice changes, and there’s a gap between what’s needed in practice and what helps professionals’ and their trainees’ gap. A clearly defined practical intervention is needed to help primary care teams learn how to function as a team.

Recognizing this need, the VA Assessing and  Improvement Laboratory for Patient Centered Care provided funding to develop a prototype for an intervention. A small design was asked to work with two or three newly formed care teams to develop, pilot, and refine the intervention. Our objective was to assemble innovative ideas into a coherent innovation model, refine this model, and test specific curricular and implementation methods through applying it in a few settings and then finally, refine it on the basis of this preliminary field work. So our aim of PACT was to develop an interactive practice-based coaching program to assist PACT teamlets with trainees to learn, practice, and coach effective team function. 

Just to give you a brief overview of what we’re going to cover, I’ve kind of organized the presentation by numbers. We’ll go through the model, and we planned the pilot, the intervention, the performance evaluation, what we learned in terms of both curriculum and implementation, and key recommendations to improve PACT for anybody else who’d like to use it and finally, our conclusion.

So on this next slide, this is our TEX conceptual model. We based our conceptual model on the work by Eduardo Salas on team effectiveness, and it’s called Team Function. There are three interrelated aspects to our model in each of the boxes here. So on the first one on the last Task Work is the work that teams perform. It involves each team member understanding the nature of the task, having the right attitude, knowledge, and skills to do their work. In our model, the task work that we emphasized was a quality improvement project. We used the IHI model for improvement as our framework, and we identified six specific behaviors in quality improvement that you see there on the screen.

Team Work involves how team members communicate to coordinate their actions. Here, we identified nine specific communication behaviors used by teamlet members during their work. These nine behaviors derived from the literature in patient-centered communication and relationship centered administration. We can see that it involves active listening, trust building, emotional management, consensus decision making, role definition, accountability, feedback, and support. 

In the third box, the Team Processes are the structures that enable team members to come together to coordinate their work. For our innovation, we’ve selected team meetings as our focus, and we’ve targeted eight specific relationship-centered meeting processes that was a combination of the work of five other leaders in training medical teams. We based this intervention on this particular model because effective team building skills are best learned in the context in which they’re used. This tells us that for patient-centered medical home teams to learn to work together, they really have to be doing meaningful work, communicate effectively, and have the time and space to reflect and plan. So that was our model.

So next, our curriculum planning, we developed coaching and evaluation tools in each of the three curricular areas based on the behaviors I’ve just described. These included needs assessment surveys and checklists, post-intervention evaluation tools, and our teaching materials. The materials that we used during the intervention included quality improvement and team meeting process handouts – give a one-page handout – communication behavior pocket cards for the behaviors I described before, and then interactive learning models and exercises that we used during the session. 

So now, I’ll describe what our pilot looked like. So we had two volunteer teamlets with trainees at two different VA sites in Southern California. There were two to three coaches that facilitated and observed the learning sessions. There were five onsite workshops in which the whole teamlet attended, and I really want to emphasize this was an intervention targeted for all the teamlet members – so the RNs, the LDNs, the clerks, and then the primary care providers. They were spaced over eight to ten months at each site. We began with a one-hour engagement session, in which teamlet members and their administrators from the involved disciplines were present. We then went out to each site and observed a team meeting to assess behaviors in the three areas of team function. From our observations, we developed two two-hour coached interactive workshops that included group feedback, reflection exercises, mock team meetings, facilitated team meetings, and then specific exercises to learn the communication behaviors. The last two-hour session included time for participants to complete their post-course evaluation surveys and discuss areas of success, needed improvements, effective coaching methods, and next steps.

Finally, after the sessions were completed, we had four one-hour coaching calls with teamlet members only. These were spaced two to four weeks apart. The intention of these calls was to support and empower the teamlet leaders to continue their efforts in team building improvement and teaching.

So here’s just a sample outline of the first overview session. And in each of these sessions, we tried to cover task work, team processes, and team work. If you want more of this, I can certainly send this to you in a handout. 

Molly:Dr. Warde, I apologize for interrupting. Can you just speak a little bit closer to the microphone? You’re kind of fading in and out, or maybe just a little louder. Thank you.

 Carole Warde, MD:Okay, okay, thank you. Okay, so on this slide, I’m showing coaching practices that emphasize motivation, education, and consultation. The coaching methods were relationship centered. We figured that we needed to be relationship centered and model these behaviors if that was the focus of our intervention. We purposely wanted our teamlet members to be in touch with their own feelings and thoughts, since this affects what one says and what one does with others when they’re working in a team. We also wanted to emphasize the positive first as a place of strength from which to build learning, enhance our inclusion of appreciate reflection. We went on to use other relationship centered coaching strategies including emotional management skills, stress management awareness and management, paired discussions, facilitated large group discussions and team meetings. As I said, we’ve tried to role model these behaviors. We used role plays. And then we sparingly used didactic presentations but when we did use them, it was to present the models and the specific skills and steps to building those skills.

So our formative evaluation – and I want to emphasize this was a formative evaluation, not a formulative one – so that we could really learn how to improve the curriculum and the implementation plan. So we collected evaluation data in several ways. There was immediate participant feedback that was solicited at the end of each session and then, the coaches discussed what worked and didn’t work and what they would do differently. After a few days, the coaches wrote a reflective summary of their experience with each session and these results were discussed and tabulated with all the coaches present. At the end of the course, participants completed a survey to evaluate their satisfaction with the course contents and methods. Participants also completed a three-part survey in which they evaluated their improvement in the behaviors related to each of those three domains. And finally, the coaches reflected on all of these sources, integrated it into tables, and met to come to a consensus in the lessons learned and recommendations.

So finally, what did we learn? So to start off, the triple focus worked. The teamlets had needs and improved in all three areas. Next, for the team meeting processes, we found that mastery of how to have a successful meeting fostered learning in quality improvement and better communication. Third, team communication skills exercises required more time and practice. We found that participants had ongoing needs in six of the nine communication skills. And these areas of continued need were trust building, role definition, collaborative decision making, giving and receiving feedback, and holding team members accountable. These six behaviors were particularly difficult for the team members, and probably understandably. 

Quality improvement was a struggle from mindset to ongoing continuous improvement. Identifying simple measures for improvement and implementing them was especially difficult for our participants. The quality improvement project that involved workflow redesign seemed to push the teams to really improve their team function.

Okay, in terms of implementation, first we learned that the coaching was really appreciated. The teamlets responded well to it in all three areas of the curriculum. We identified successful methods and these included role modeling, mock team meetings, and exercises that equalized member input. We observed several signs of teamlet dysfunction that needed to be addressed early on by the coaches, and these skills taught to team leaders. These included teamwork that did not meet regularly, interpersonal relationship tension, resistant to examining work processes, and resistance to giving and receiving feedback. In retrospect, we identified these areas as signs of teamlet dysfunction that we wished we’d addressed early on.

So the structure of the pilot really works. The teamlets liked the coaches coming to them and the fact that the whole team was participating in the sessions and they were learning together. They found that the interactive sessions and coaching calls were also very useful. 

Finally, including trainees was a win-win situation for both the trainees and the teamlets. The trainees learned about PACT teamwork through their active involvement and the teamlets benefited from the infusion of energy, curiosity, and the fresh skills that the trainees were able to impart to the teamlet members. In both cases, we were surprised to see that the trainees played a very substantial role in the activities related to all three areas. They really stepped up to the plate as the worker bees and allowed a lot of the QI activities to progress as quickly as they did.

So our key recommendations to improve TEX. So for the curriculum next time around, we would emphasize relationship-centered meeting processes early in the course. We would plan more time for coached communication and QI skills. We will involve administrators more in the intervention with the hope of increasing motivation and support for QI, and support for addressing some of the dysfunction behaviors. We will introduce ideas for change involving workflow redesign early on. For instance, we would probably familiarize teamlets with the chronic care model since there are many opportunities for workflow redesign in this model that are directly relevant for PACT. Our teamlet members seem to be really lacking this chronic care infrastructure. 

Regarding implementation, we would alternate sessions and calls every two to three weeks. We wouldn’t separate them, we would alternate the sessions with the coaching calls. We would emphasize an ongoing needs assessment and feedback rather than taking up the whole session at the beginning specifically for needs assessments. We would intensify coaching in trust building, identification, and management of teamlet dysfunction and teamlet leadership skills. We found our teamlet leaders to be really in need of that.

Okay. In conclusion, the Triple-Focus model is useful to improve and teach teamwork. The interactive TEX curriculum helped PACT teamlets with trainees improve team function and our assessment tools and processes provided meaningful feedback to direct future sessions. We expect the recommended revisions to further strengthen the TEX innovation. 

And finally, I would like to acknowledge that TEX was funded through an Innovation Grant from the Veterans Assessment and Improvement Laboratory. Special thanks to the following individuals who made this pilot project possible: Lisa Rubenstein, Lisa Altman, Susan Stockdale, Art Gomez, and our worker bees, Christina Huang and Lopmurda Das, in addition to Marge Pearson and my collaborators. Thank you.

Molly:Thank you very much. And now, I’m going to turn it over to Anneliese. Just go ahead and click “Show my Screen” and we’ll be all set.

Anneliese Butler:Alright, one second. There I go. 

Molly:Great, thank you.

Anneliese Butler:Is that good? Alright.

Molly:Yeah, perfect.

Anneliese Butler:Thanks so much. So good afternoon or good morning, depending on your time zone. My name’s Anneliese Butler and I’m a Project Manager at the VISN 4 Center for Evaluation of Patient Aligned Care Teams, better known as CEPACT, where I’m a member of the Qualitative Evaluation Team. Our demo lab is tasked with evaluating the PACT rollout across VISN 4. And as part of our larger formulative evaluation, we’ve been following the major training initiatives that have been deployed to spread PACT since about August, 2010, including a very innovative training endeavor that’s been targeted specifically to teams in our VISN. 

The VISN 4 virtual PACT collaborative launched in January of 2012, and it represents a core element of the VISN strategy to advance PACT implementation and spread across the network. Unlike the TEX study that Dr. Warde just described, the Virtual Collaborative was not an intervention designed and implemented by our demo lab. Rather, CEPACT’s primary role has been to help evaluate the Virtual Collaboratives and to provide actionable feedback to the planning committee.

I want to just take a moment to acknowledge my fellow team members and for all of our partners in this work, most especially our VISN’s Chief Medical Officer, Dr. David Macpherson, who conceived of the Virtual Collaborative and his Administrative Officer, Jennifer Skoko, who is really the nerve center for the collaborative.

Our evaluation activities around the Virtual Collaborative is concentrated on two main areas. The implementation process, so what does it take to implement a Virtual Collaborative, and how the Virtual Collaborative is experienced by learners. My focus today will be on the second area; specifically, on findings from a survey that we developed to assess the experiences of team members who are participating in the Virtual Collaborative. 

But before I discuss the survey, I want to give you a brief overview of the Collaborative itself. So, why the need for another PACT training? What you see here is a timeline showing Virtual Collaborative in relation to the two major training initiatives that were part of the original implementation plan for PACT at the national level. The blue arrow here at the bottom depicts the Regional Learning Collaborative that started in August of 2010, and ran for eighteen months. These were targeted to pilot teams from each facility. The red arrow in the middle represents the less intensive three-day learning centers that were run by the Centers of Excellence and were designed to be a kind of basic training for other non-pilot PACT teams. They started in April of 2011, and were supposed to continue through 2013. However, they were suspended after about six months due to national restrictions on travel funding. They have since been revised and the revised Learning Centers started up again late last year.

So this is how our leadership recognized that these two training vehicles alone would not be enough to truly spread PACT to all teams, especially teams located in remote CBOCs, many of whom found it difficult to access training and were feeling largely out of the loop about PACT. Initially, the Virtual Collaborative was designed to pick up where the Learning Center’s sessions left off, meaning that teams would first attend a Learning Center to get a basic understanding of PACT and then join the Virtual Collaborative for further training. However, when the Learning Centers were suspended, the planning committee for the Collaborative modified their plans last minute and opted to have all PACT’s teams join the Virtual Collaborative at the same time, including those without even a basic familiarity with PACT.

To give you a sense of the scope of this undertaking, here’s a map of VISN 4, which is home to ten medical centers and forty-five community-based outpatient clinics – CBOCs – which provide care to over three hundred thousand veterans each year. All primary care teams across the VISN are required to participate in the Virtual Collaborative, and this translates to approximately three hundred and fifty teams, give or take. 

So how do you do that? What exactly is a Virtual Collaborative anyway? The Virtual Collaborative model replicates the more traditional learning collaborative model where large group learning sessions that are held offsite usually last about three days are interspersed with action periods during which teams complete projects and receive further coaching at their home facilities. The main difference here is that learning sessions are conducted virtually and via LiveMeeting instead of requiring teams to travel to an offsite location. Dr. Macpherson knew that it was a challenge for many teams to find time to learn about and work on PACT so he therefore explicitly instructed facility directors to block clinic schedules so that teams would be able to tune in to the learning sessions. The VISN further instructed facilities to give teams protected time to meet between the virtual sessions in order to apply what they learned to their own practices, to complete exercises, or team homework assignments, and to meet with their coaches as needed. And the minimum expected for these team meeting times was one hour every two weeks with a preference for one hour per week. 

I’m going to move on to the survey design now. But for anyone who’s curious about the curriculum, your handout does include this slide, which lists the topics that were covered during the first nine months of the Collaborative. 

As I mentioned earlier, the Survey of Learner Experiences was one part of a larger multi method evaluation of the Virtual Collaborative. CEPACT developed this survey in consultation with the Virtual Collaborative Planning Committee and we pilot tested it with frontline primary care staff. This was an anonymous survey and it was administered by Survey Monkey. It comprised thirty-eight IMs, including four open-ended writing questions. We fielded it twice during the first nine months of the collaborative – once in April of 2012 and again in September. Anyone registered for the Collaborative was eligible to participate but the survey was chiefly aimed at core PACT team members, since they are the primary target audience for the Virtual Collaborative, so your primary care providers, nurses, and clerical support staff. 

The specific aims of the survey were first, to assess the extent of participation in various aspects of the Collaborative and then, to elicit individual learner’s perspectives about the Collaborative in terms of its effectiveness and its acceptability as a means of facilitating practice change. 

Just over eight hundred participants receive a survey invitation in each round and the response rate was about forty percent both times. In both survey rounds, primary care providers were most strongly represented while clerical staff comprised the smallest group of respondents. About one-half of respondents in each survey were located at a medical center and about one-third had attended one or both of the previous PACT training initiatives that I mentioned earlier. 

The open-ended survey items also generated a large number of written comments, which provided vivid details about respondents’ experiences of, and thoughts about, the Virtual Collaborative, as well as the PACT initiative more broadly. And their comments shed light on patterns that we saw in our statistical analyses. 

In the interest of time, I’m going to skip past details about how we analyzed our data. But if you’d like more details about our analytic approach or any other aspect of our evaluation that I’m not covering today, please feel free to send me an email.

So I’d like to move now to what we learned from the survey about how participants experienced Virtual Collaborative and in particular, why some participants benefitted from it less, with a focus on factors that have implications for other training efforts geared toward PACT teams. 

I want to tell you up front, I won’t be showing you a lot of statistics today, although I’ll be glad to send you our complete survey report, which summarizes the quantitative results in detail. Instead, I’d like to tell the story of what we learned using participants’ own words because their comments really speak more eloquently to the challenges that they face than any charts or tables that I might show you really could. That said, let me just briefly highlight a few of the key findings from our statistical analyses.

In terms of participation, the majority of respondents to both surveys said that they attended most or all of the virtual sessions. And in both rounds, most of them said that team meetings were shorter in duration than originally envisioned. The frequency of team meetings was lower in Round 2 than it was in Round 1. However, because the two survey samples were not identical because it was an anonymous survey, we can’t conclude from the quantitative data that there was, indeed, a downward trend. That said, our qualitative data does suggest that meeting frequency decreased over time.

Respondents at medical centers were considerably more likely than those at CBOCs to have attended prior PACT training. They were also more likely to be what we considered fully engaged in the Virtual Collaborative, meaning that they attended most virtual sessions, met regularly with their teams, and participated in team assignments. 

Most respondents felt that the virtual sessions and the team meetings were valuable, while team assignments – which at the time were called homework – were the least popular aspect of the Collaborative in both rounds of the survey.

In terms of specific benefits, the Virtual Collaborative benefitted participants by increasing their knowledge of PACT’s concepts and strategies by providing a broader perspective in terms of the larger PACT vision, as well as challenges that they shared in common with other teams. By improving access to tools and resources, and especially by facilitating peer-to-peer exchanges of ideas, both with other teams at their own site, as well as with teams at sites across the VISN.

And finally, respondents who had prior PACT training and/or who were fully engaged in the Virtual Collaborative were consistently more likely to feel that the Virtual Collaborative had benefitted them.

I’m just going to leave this slide up for a moment. The quotes shown here give you a taste of some of the many positive comments that were submitted by respondents. Since we’re pressed for time, I’ll let you read those in your handout at your own leisure and move on now to the constructive criticism that respondents offered.

So why did some participants benefit less? One critique was that the content of the Virtual Collaborative did not match the learning needs of certain groups; notably, participants without previous PACT training, those in support staff roles, and those on teams that deviated from the recommended three-to-one staffing ratio for PACT teams. As I mentioned earlier, the original vision was that learners would join the Virtual Collaborative with at least some foundational knowledge of PACT. Because the Learning Centers were suspended shortly before the Virtual Collaborative was set to launch, there was little time to revise the content to meet the needs of an audience that now included both advanced teams and teams that were only just forming and had no previous training in PACT. The following quote illustrates how this affected the experience for  some participants. “I think having a training on PACT, to get a basic understanding prior to

the Virtual Collaborative would have been very helpful. All these months into it, we are just now finding out about tools to help with this process. I feel that we are embarking on a quest without having the basic fundamentals in place first.” 

A number of participants commented that the content of the Virtual Collaborative was not relevant to their specific role; clerical and clinical associates, in particular, expressed a wish for more content specific to their functions under PACT. As one person said, “ It seems like the majority of them – meaning the sessions – were more directed towards the provider and not the rest of the PACT team.” 

A further problem is that the Virtual Collaborative, like other PACT trainings before it, has really emphasized a team model that remains out of reach for many sites. Many PACT teams are configured according to some hybrid model or else they’re still missing key goals on the team. As a result, many participants felt that the Virtual Collaborative did not speak to the particular realities and needs of their team. As one person put it, “I’m not yet fully PACT as we are quite low in RNs. The presenters of the Collaborative could give alternative ways of doing things for teams like ours that have one RN for eight providers instead of making suggestions for the ideal theoretical team that we currently don’t have yet.” 

The other major reason why some participants benefitted less from the Virtual Collaborative has to do with the challenges that have hampered PACT implementation as a whole. We were struck by the number and intensity of survey comments that described issues around time, staffing, and local leadership. Even though facilities were mandated to give teams dedicated time to participate in Virtual Collaborative activities, nine months into the Virtual Collaborative, protected time was still not universally available. Where it was, there were signs of erosion as local enforcement relaxed and other priorities took over. The following two quotes were echoed by several respondents to the survey. Only my RN comes to the collaborative. No one else from my team has time or has been released from work tasks. I think if we value something then we make it a priority so if PACT is truly a priority, then set aside time away from work to let people attend.” 

And:  “We are so severely understaffed that going to these sessions is like sending a diabetic to lunch at a candy store. What's the use when you can't avail yourself of such wonderful theoretical concepts. PACT is flying overhead and we're still in the bunkers.”

These comments also hint at the third major barrier that was highlighted by the survey; namely, perceived lack of support from local leadership. The next two quotes express this frustration more directly. “Going to the Virtual Collaborative is torture. I go learn about things that other facilities try and do that we aren't allowed to so. I get to hear things that the National and the VISN recommend but that are ignored at this facility.”

Another person said, “At the team level, there are very few changes that can be made to implement PACT to the best of our ability. Our hands are tied by the administration.”

I should point out that most comments of this kind came from respondents at the same handful of sites. Nevertheless, they do indicate that leadership at some sites may be working across purposes both to the Virtual Collaborative and to PACT, more generally. Whether accurate or not, these kinds of beliefs about leadership undermine team engagement and morale and therefore, warrant serious attention. 

So what lessons can we draw from these results? First, the Virtual Collaborative does hold promise. Although there are many opportunities for improvement, our data do indicated that the Virtual Collaborative was perceived as beneficial by the audience that it was intended for. And they also indicated that the model holds promise as a strategy for training primary care teams impact provided that local conditions support full engagement by staff. No training endeavor can tack the challenges needed to fully resolve persisting barriers around time and staffing. 

Nevertheless, there are ways in which the Virtual Collaborative and other similar efforts might mitigate the impact of these challenges and facilitate movement toward their resolution. First, insist on truly protected time. All team members need protected time to participate in training sessions and team meetings. Without such protect time, it will be hard for them to engage in, feel engaged by, and move forward with practice improvement. And training efforts may inadvertently contribute to staff burnout. This message should be emphasized repeatedly and frequently at all organizational levels. 

Next, match the content to your audience and consider the needs of all team members. Presenters should be exclusively prompted to consider the particular needs of key target groups when developing their presentations or curricula. Based on what we’ve seen across all PACT training initiatives so far, including the Virtual Collaborative, we recommend special attention to the role functions of clinical and clerical associates, as well as implementation challenges that are unique to community-based outpatient clinics. 

Speak to team challenges. Training content should reflect and speak to the actual conditions in which teams are working, including the reality of persistent resource constraints and the need to find creative adaptations to those. 

Confront leadership issues. Leaders at certain sites may benefit from targeted education about PACT concepts and processes in order to ensure that teams have the resources and the autonomy that they need to be successful. Teams at some sites may need support from outside the facility to address larger systemic issues. Inner session at the executive leadership level might be warranted in cases where local facility leaders seem to be working at cross-purposes to PACT. 

And finally, actively seek out input from the trenches. Our findings highlight the value of seeking constructive criticism from those working at or near the frontlines of patient care. But perspective is necessary for effective problem solving, and they will also be more engaged and motivated if they have a say in proposed changes and feel that their concerns are taken seriously. 

So I thought you might like to know where the Virtual Collaborative is now. One year into the process, it is now firmly established as a key driver of PACT implementation and spread in VISN 4, providing a unified vision and sense of direction while also supporting innovation and autonomy at the team level. The Virtual Collaborative has continued to evolve in response to feedback about what works, what doesn’t and what more is needed. This January, it entered a new phase – one with a broader focus than just on meeting the five required PACT performance goals. The curriculum has shifted from teaching PACT concepts and strategies to really getting teams more actively involved in developing their own practice improvement projects. The long-term vision is to cultivate a culture of continuous quality improvement where these sorts of team-based improvement projects are part and parcel of a team’s every day practice. 

So I’m going to conclude here so that we have enough time to answer some of your questions. But again, please do feel free to contact me about anything discussed or referenced today. You’ll also find my contact information in your handout. Thank you very much.

Molly:Excellent. Thank both of you very much. We do have some pending questions. I just want to mention to those of you that joined after the top of the hour, if you’d like to submit a question or a comment for one of the presenters, just open up the GoToWebinar dashboard on the right-hand side of your screen and just type your question into the question section and press “send.” And we’ll get right to those now. Also, I noticed some people raising their hand. Please do not do that, I can’t unmute you. You have to type your question in, thank you.

So the first question came in during Carole’s presentation. What is the PDSA cycle?

Carole Warde, MD:Okay, the PDSA cycle is Plan, Do, Study, Act. And it’s the steps for initiating and performing small steps of change. It’s part of the IHI model for quality improvement. Is that good?

Molly:Thank you very much. Sounds good. They can write in if they need further detail. The next question we have, How were you able to regularly involve the same trainees with their complicated critical schedules?

Carole Warde, MD:Well, the trainees that we chose at each site were there for two... One of them was there for two full days out of the week and the other one was there all week. So one of them was a psychology fellow and the other one was a nurse practitioner student. They even, the nurse practitioner student, after a while, loved being so much a part of the sessions that they even came in on their days off. We’ve subsequently involved residents that are here doing ambulatory blocks. We have a primary care resident here for three months at a time, so she’s actually involved in it now and is actually teaching some of the stuff.

Molly:Thank you for that response. The next question we have, Is TEX available now for use at other VAs? If so, how do we access it?

Carole Warde, MD:I’m happy to share what we have. We’ve developed a revised curriculum that we would really be interested in partnering with others and doing a nice evaluation of it to see what works and what doesn’t. You know, we’re really open for suggestions depending on what people would like. Maybe we can develop some collaborative training and introduction, whatever people are interested in. We’re really interested in learning what people’s needs are.

Molly:Thank you very much. The next question, or it’s more of a request. I would like to receive a copy of the VC PACT report and results. Is that an option?

Anneliese Butler:It absolutely is. If you could just send me an email, I’d be happy to send it to you directly.

Molly:Great.

Anneliese Butler:I can’t see who submitted that question.

Molly:No problem. They’re still on the call and they can contact you offline.

Anneliese Butler:Great, thanks.

Molly:The next question. Can you post an executive summary of your findings? The slides are very nice but a more succinct summary would complement them for us to tell your story to important colleagues.

Anneliese Butler:So I’m not sure if that’s directed towards me or Carole or both of us perhaps, but I’d be happy to share, actually, if you want, both executive summaries from... We’ve published two reports. One was midpoint and the other one was after the final survey, and they will provide the kind of detail that might be helpful for you. So again, if you don’t mind, please, just sending me an email. I’d be happy to send that to you unless Molly, you’d like to post something as part of the archive. We can do that, too.

Molly:Yeah, they’re ready for general consumption. Feel free to email them to me and I can upload them to the archive page along with the handouts.

Anneliese Butler:Okay, great. 

Carole Warde, MD:And we also have an executive summary that I can share with anyone, and I’m happy to respond to questions offline. And we’re also in the process of writing a paper on our results so hopefully, you’ll be able to read it in text soon. But you certainly can have the executive summary. 

Molly:Great. Thanks for making yourself available offline. Next question we have, Any particular findings related to the home-based PACT model, which is a little different?

Anneliese Butler:This is Anneliese. Not on our end, so I can’t speak to that.

Molly:Carole, do you have any input?

Carole Warde, MD:No, I don’t know what the home-based PACT model is so I can’t respond to that either, I’m sorry. 

Molly:Not a problem, thank you. Okay, next question. Why was the homework looked upon as a negative component?

Anneliese Butler:Well, as you might imagine, it was either very busy teams and also they’re adults, and they didn’t appreciate the use of the term “homework.” Unfortunately, once the word was out there, even though they changed it, people still kind of associated with it. They didn’t always see the connection between the exercises they were being asked to do and the benefits that might be derived from them. However, that was usually an initial kind of resistance to just taking the time to do it. The teams that did complete the homework, at least from the feedback that we got from coaches and other key contacts, they actually were surprised that it did benefit them. They learned a lot and they had these “aha” moments where what they were learning about on the virtual learning session suddenly clicked for them.

So based on feedback not just from the survey but from other folks at the sites who were helping to implement the Virtual Collaborative locally, we concluded that the homework can be actually a really useful tool. It should be framed differently, but it’s helpful in really prompting teams to take that next step of applying the knowledge that they’ve acquired. But yeah, I think the primary reason was it was another thing in their very busy days.

Molly:Thank you for that report.

Carole Warde, MD:This is Carole. We, in regards to homework, we didn’t call it homework but these were quality improvement processes that they take back or that actually derived from the practice. And we’ve tried to make them be small, we tried to help them identify measures that were really easy to collect that didn’t take a lot of work. And we also tried to help them select changes that were going to be directly beneficial and make their life easier. 

So they were really involved in choosing those things upfront. I mean, certainly we guided them in terms of particular content areas. But the actual process they chose to improve was directly related to them.

Having said that, that’s where the trainees came in really, really helpful. They were like the energy bunnies behind it and kind of helped the clerks do this, would remind them. So they really kind of got things going. I have to say that doing it from what we’ve just learned in other sessions, that doing it on a short timeframe from one week to the next is really the best because then they don’t get too upset about making this big huge project. It’s really small [inaudible] PBSAs that they can see results, even if was just qualitative results. It was kind of got excited about it, it was almost infectious.

Anneliese Butler:Yeah, I would second that. It’s actually the direction that the Virtual Collaborative has moved in in its new phase, to have teams really design their projects based on what makes the most sense for them, I mean, with a lot of guidance about how to select outcome measures and so forth. But yeah, that makes a lot of sense, Carole.

Carole Warde, MD:The other thing that my co-facilitator was very, very helpful in, she kept correcting me when I called them projects. And the reason was we would call them processes because projects denotes a bigger kind of project. And the process is much more small and less... it doesn’t create so much resistance. 

Molly:Thank you for those replies. We are down... Yes, we have one more pending and we’ll get to it now. How is the TEX training going to be sustained and spread?

Carole Warde, MD:Oh, boy, that’s the million-dollar question. Well, right now, it’s been infused in other sessions not completely as it is, as we designed it. That’s where it really depends on the site where you’re at. I think it’s, as a process, because of its emphasis on relationships, it’s pretty unique. I think for TEX to go on, I think there really needs to be... the coaches need to be trained in communication facilitation and relationship-centered processes themselves. And that can certainly... I mean, that really has to be done onsite. I think it’s difficult to do that virtually. I think the coaches on the ground, or the way they’re trained that way to really help the team learn their communication skills, get out of difficult interpersonal kinds of tensions with the third person around. But I think real training in the three areas of team function by coaches is probably the way to go.

The other thing is, it’s just like the informed intervention. The whole team was there. If you don’t have the whole team there, there’s really just no sense in a PACT training. My personal opinion, what I’m seeing in terms of other sites. You’ve just got to train the whole team. It’s based on inter-professional learning series. You’ve just got to train the team together. So I think trained coaches or a team who has whole teams, it is the way to implement TEX.

Molly:Thank you very much for that reply. That is the final pending question we have at this time. I do want to give both of you a chance to make some concluding comments. I also want to make the announcement to our attendees that as you exit today’s webinar, please wait just a second and a survey will appear on your screen. We do really appreciate your feedback, as the topics you suggest and the feedback you provide is what guides our program.

So with that, Dr. Warde, would you like to make any concluding comments?

Carole Warde, MD:Well, I would just like to thank you for coming and thank you for your interest in questions. I am available by email or to set up phone calls to discuss any of this with anyone. I will also be presenting the steps by a general internal medicine meeting as a poster, so maybe we could even meet together. That’s in Denver at the end of April. So thank you, thank you very much for your interest and questions.

Molly:Excellent. And Anneliese, would you like to add anything?

Anneliese Butler:Again, just thank you for your attention. Please contact me if you want any further materials. And I just want to express my appreciation if there are any PACT team members out there, especially from VISN 4. Thank you so much. Without you, none of our work would be possible. So I just wanted to acknowledge you, as well. Thanks.

Carole Warde, MD:And thank you, Molly.

Molly:Not a problem. I’m very happy to help. I want to thank both of you for sharing your expertise with the field and I want to thank our attendees for joining us. And please do remember that we have a PACT session every third Wednesday of the month at noon Eastern, and we do hope that you will join us for the next one. So with that, this does conclude today’s cyberseminar and I hope everybody has a wonderful day. Thank you.

Carole Warde, MD:Thank you.

Page 1 of 16

Page 16 of 16