**Transcript of Cyberseminar**

**Patient-Aligned Care teams Demonstration Labs**

**Creating, Implementing and Assessing a PACT Training Program within the Minneapolis Health Care System**

**Presented by: Craig Roth MD, and Gregory Stewart, PhD**

**December 18, 2013**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact**Craig.Roth@va.gov**or**Gregory.Stewart2@va.gov**.*

*Dr. Roth:*—30 minutes I want to tell you a story.  The story takes place not far from Lake Wobegone.  It tells how the Primary Care Serve Line has approached the challenge of training its large, and mostly above average clinical staff, to redesign and transform their practice to a more team based, and patient centered, and coordinated clinical operation, or referred to as a PACT or patient-centered medical home.

I’ll be telling this story of our planning process, and what training’s been delivered.  The training’ll be given in broad strokes as I describe it.  Because there won’t be enough time to give details about the curricular content, or the teaching method.  However, if you want those details, links to many of the documents and resources are going to be available at the end of the presentation.  

I’ll conclude with some lessons learned, which might be helpful for some people who are either on a similar journey, or contemplating one.  I’m going to skip by this slide, because this is an overview of mostly what I just said.  We’re going to start with an audience poll.  I would be interested in knowing who in the audience is a member of a PACT, and who might be involved with teaching in a PACT.

*Moderator:*Thank you Doctor Roth.  We do have that poll slide up now.  It looks like our attendees are streaming in their responses.  We’ll give people some more time to get those in.  It looks like we have a pretty good response rate for our audience.  Thank you we do appreciate you taking the time to answer this.  It looks like about 26 percent of our audience are members of a PACT.  About 74 percent are not.  Thank you again to our respondents.  

I think we have one more after this.  We’ll go ahead and open that right up.  This question asks, do you teach PACT concepts or skills to others?  It looks like we’re hovering right around half-and-half for each one.  46 percent say yes, and about 54 percent say no.  Once again thank you for your responses.

*Dr. Roth:*Yeah, thank you.  I find that very useful.  This story begins, for me and for our training efforts, back in April of 2010 when the central office launched, I think on a large scale, the plans for transforming care to The Patient Aligned Care Team Model in the VA system.  For those of you weren’t there, it was a two day conference.  Every VA medical center in the country was represented.  

There were just many people there from leadership talking about their vision, and also lots and lots of hands on workshops to learn about these principals.  For people like myself and a number of the people who—there were probably about eight of us who went from here in Minneapolis.  This was mostly new, except for our—we had two pilot teams who had been learning about this a little bit before, but that was only a very small group compared to a lot of us who knew nothing about this, and so it was all new.  That’s when this started.  

Then just by way of a time line here that I’ll set up.  Not long after we came back, our chief of the Primary Care Service Line made it clear to us that this was something that we were going to embrace, and that we needed to think about how to roll out the training.  He was an educator, and so he was interested in this and charged a group, which we called the PACT Education Group.  

The people who were put onto this group was the co-chief, the other co-chief of general medicine Ken Engelhart, and the clinical director Lori Pawelski, who oversees all of the nursing in all of our primary care clinics, at least in Minneapolis.  We also had a similar persons who were the head of the community based outpatient clinics, and the clinic director RN for all the nurses in the outpatient clinics.  

Then there was a health behavior coordinator.  A highly skilled person, Craig Helbok, who’s a health psychologist and a Ph.D.  Who’s had a lot of training, and who has been very instrumental in a lot of the training that we’ve done.  Two RN educators who had a lot of experience as educators in our organization, and were familiar with resources.  A pharmacist.  We have residents here at who train and they do a lot of training and educating of the pharmacy members here.  

RN case managers, three of them from the clinics.  We had the supervisor of all the medical support assistance.  Then two frontline physicians, myself and Peter Weissmann, who both had a lot of experience with teaching, mostly at the resident level, it’s a medical school, and then also some specialized training in small group facilitation, and some team training.  That was our group.  

Then shortly after we were put together we had to hammer out a vision and a mission.  We created a charter for our group.  Our vision was that every PACT team would be exceptionally prepared to transform the delivery of primary care.  Kind of a big vision.  Our mission was to develop a comprehensive standardized curriculum to train Minneapolis staff about The Patient Aligned Care Team Model.

*Moderator:*Sorry for interrupting Doctor Roth.

*Dr. Roth:*Yes.

*Moderator:*Can I ask you to increase the volume on your telephone a little bit, or speak up?

*Dr. Roth:*I’ll try to.

*Moderator:*Thank you.

*Dr. Roth:*How’s that?  How’s that—I might have moved away.  How am I doing now?

*Moderator:*A little bit better.  Thank you.

*Dr. Roth:*Is that a little better?  Okay, sorry.  This headsets been a problem before.  Then we looked at our teamlets, which hadn’t existed before.  For those of you who may not be familiar with this, a teamlet, for our purposes of discussion, would be a physician, an RN case manager, a LPN, and a MSA.  By dividing up into teams in Minneapolis, we ended up with 24 teamlets in Minneapolis, and 21 in our community based outpatient clinics in our area.  That was a total of approximately 230 people.  

Then we had to decide on our curricular content.  We drew heavily on what came from VACO, and the resources that were available.  A lot of them from that conference in Las Vegas, and from the pilot—from the regional collaboratives that were already underway.  There were a lot of nice resources, and already some pilot teams had assembled some valuable training information, which we drew on heavily.  

Then we also wanted to tap into what the values, and where the energy was among our own staff to change things.  We had a session where we got together all of our physicians in general medicine and asked them to answer some questions.  Including one like what could we do around here?  What could you imagine doing in your practice that would be more patient centered, or veteran centered?  

They generated a list of really valuable topics, many of which lined up with the goals for the PACT initiative.  We felt that this was a little more bottom up, rather than top down.  We’d like to try to build on that in our training design.  We also, because we were aware of all these—we started to get aware of all these resources, we created a share point.  Which was a great repository for lots of information.  I think it was valuable for us to keep everything together as we worked along.  

It took us a while before we finally actually started doing some training.  In late 2011 we started some training.  I’m going to talk a little bit about what that is in the next set of slides.  I just wanted you to see how our group had worked together in a timeline there.  By the way, our team, PACT Education Group, was meeting weekly, because we felt a sense of urgency to get things moving a long, and needed to work as much as we could together.  We spent about an hour or 90 minutes in there every week.  

We also, at this time, recognized that there were some training available for teams, and for coaches of teams.  It was serendipity, but the VISN offered a training program called Team Effectiveness Training, which was based on the team development measure, which PeaceHealth had developed.  They anticipated we were going to be doing more teamwork. 

It was good timing, because we were interested in trying to figure out how to work together more as a team.  These team coaches we were—the committee was very supportive of that, and sent people off for training.  I can talk more about that a little bit later.  

Then the last thing is we’re up to 2013.  It’s just a relatively recent development that we’ve had coaches that we are assigning to each teamlet, and so there’s roughly one coach for every three teamlets.

We’ll stop there for a minute.  That’s the timeline of our group that was overseeing this, the design and the implementation of this curriculum.  We’re going to talk about the curriculum in a minute, but maybe we can stop and take a poll question.

*Moderator:*Thank you.  We have that next poll question up.  The question is have you personally participated in a PACT training activity?  Answers are still streaming in.  It looks like we’ve got about a 50 percent response rate so far, so we’ll give people some more time to get their responses in.  

All right, we seem to be hovering right around 54 percent to say yes they have been in a PACT training activity.  About 45 percent say no they have not.  Thank you for those responses.

*Dr. Roth:*Yeah.  Thank you.  This is the conceptual design that we came up with when we thought about the content and how we might approach it, just so we could keep our eyes on what was going on.  We thought that the foundational work was teamwork, and that we also needed leadership training, because the teams were being led by mostly by physicians.  

There were some physician’s assistants and nurse practitioners who were leading the teams, but as they were conceived they were initially just—it was just the doctors on the teams that were given that role.  Most of them had no formal leadership training, and as we found out some of them weren’t particularly keen to be leaders.  *[Chuckle]*  We felt that we needed to give some training in that area.

Then of course there are these principals of the PACT model which—like practice redesign, and care management, and coordination, and access which some of which—many of which were foreign to people.  We lumped them into these three large areas, and thought about our curricular design and our timelines based on this concept. 

What I’m going to do, I’m going to skip ahead real quick and just give you a broad overview of what’s going to happen, what I’m going to talk about.  There are three—I’ve got the teamwork column here, and the PACT principals here on a timeline, and leadership on a timeline.  I’m going to go back and talk about each one of them a little bit more in detail.  

Let’s start with the teamwork.  In 2011 we—as I told you earlier, we trained some coaches for Team Effectiveness Training.  This is something that just fell into line for us.  With the plan that they were going to go out and help all the teams start to—their training was to deliver a one day eight hour workshop to every one of our teams.  We wanted to do this as early as possible, because we felt this was foundational work.  

They were trained in 2011.  It wasn’t until later in the year that they were actually able to start working with each of the teams.  In the meantime, in the middle, there was some Center of Excellence Training, which was a national—the support for that came from a national level.  In each of the VISN there was training offered where people came in from around the country who had expertise in this.  We could send our teams to these Centers of Excellence in various places.  We had some in Minneapolis, but some were in other parts of our VISN where we sent some of our teams.  

We managed to send about half of our teams there.  Then the funding stream for that didn’t—either the funding, or the content was felt to not be quite what they wanted it to be and they stopped doing them.  That was back to us to come up with something more.  We had to keep pushing ahead to try to decide what we wanted to do.  All the while keeping our eyes on what might be out there to take advantage of that was supported from the VISN, or the national level.  

We went ahead and did the Foundations of Team Effectiveness Training, an eight-hour workshop.  Where actually for many of the teams it was the first time they’d ever actually physically sat together and saw themselves as a team.  We found this to be a very helpful day together.  They were relieved of clinic and came together.  This was a facilitated eight-hour session.  

They learned about elements of successful teams.  Then they learned about the team development measure.  Which is a tool that they can use to periodically assess their progress and their development.  

The next thing that happened was up in early in 2012 there was a PACT Essentials Course that was a homegrown four-hour curriculum that we put together for all of our teams.  The teams came to that.  Although there was some teamwork in that four-hour curriculum, most of that was about the nuts and bolts.  We felt it was important, because the teams were working together.  

In 2013 the VISN has been putting together a really nice training program, set of trainings that are—we’ve divided up into five sessions.  That we’ve done the first three in 2013 that are listed here.  Again, the details of all this is beyond the time that we have here.  Much of the activities that took place in each one of these sessions are highly team based and interactive.  The teams really got to come together, and work together, and share what they were learning and where they were.  

By the way, at the same time in 2013, as I mentioned we’ve got coaches assigned to the teamlets, the coaches have started meeting monthly here in Minneapolis.  We found that to be very valuable.  We actually have a facilitator, who facilitates the discussion of the coaches who have been attending as many of the weekly meetings that the teams have as possible.  They talk about the more challenging situations that come up in the teams.  The coaches have also been trying to increase their understanding of teamwork by reading books that we have been providing them.  

The next column is the PACT principals, which is this portion of it.  What we did very early on—the first thing we did was to develop an online course called PACT 101, which took about an hour.  Then you could go into the Learning Management System and get credit for that.  We made it clear that we wanted everyone involved in these teams to take this.  We had really good participation in that.  I think it was a helpful introduction.  

Then, as I mentioned, we had that Centers of Excellence Training for about 15 of our team.  They got exposed to some PACT principals there to be sure.  This NCP initiative TEACH, which I’m—for those of you who might know about it, is an eight-hour interactive activity that has very patients—promotes very patient centered, value based communication with patients, and is really—lines up very well with the spirit of PACT, which is here.  That was offered.  Many people took advantage of it, but a lot of people didn’t, but we offered it.  

Then also motivational interviewing was made available to people.  It was strongly encouraged that people take that.  Then also we had a one-hour session on secure messaging.  You could take it on, I think, the TMS system.  You could take it as an online course, or we offered some of those as live one-hour sessions.  I told you about that PACT Essentials.  That was talking about some nuts and bolts of PACT like access, and scrubbing your work schedules, and also some other practice redesign strategies.  

Then the three hours, the three training sessions that I mentioned before with, when I was talking about teamwork.  The last was the leadership, this part of it here.  We were a little bit later.  The other things came along.  The leadership was the last thing.  Actually we got a charge from our primary care chief, who sat all of the leaders down together of the teams and said, “It’s your responsibility to really be actively involved in leading these teams forward.”  

Shortly after that, we knew at that time that we were going to have Doctor Stewart help us with the Theories of Training sessions for the leaders.  He was strongly encouraging them to attend that.  We had one session in late 2012 that was a six-hour live session, where we had all the people from—all the leaders of the PACT teams from Minneapolis, St. Paul, and from the CBOCs and came together for that session.  

The leaders got a book on—Bodenheimer’s book on Improving Primary Care.  The coaches got some training.  The coaches joined us at that session as well.  They got some books, and got to know the teams a little better there.  Then two subsequent series were done by—for the Minneapolis people.  They were done live, but they were done by V-Tel for the community based outpatient clinics.  

Those were two-hour sessions, session two, and session three.  The content of these is made available in a PDF for you if you’re interested in it.  Then Greg is going to—Doctor Stewart’s going to talk about this a little bit later.

The last thing we’ve done, that’s in process right now, is a job description for leaders.  It seems like you would think that that should have been done first, but we realized that we hadn’t actually written a job description for the team leaders.  We’re working on that now, and hope that that will be helpful.  We do have a job description for the coaches by the way.  

This is the overview of the whole training again.  Because Greg’s going to talk about the evaluation—just so you can book mark this that the evaluation occurred at the end of 2012.  Where there was an evaluation of all of the members of the PACT teams that happened here.  The demonstration lab gathered that information.  Then I think Greg’s going to talk a little bit more about the leadership sessions.  At the end of these three sessions there was an evaluation of the people who—the leaders who participated in this.  

We have this thing we’ve patched together that I’ve shared with you.  It did come together.  It wasn’t entirely linear, like that diagram might make you think, but we did have some success.  I thought I would—as I reflected on this—think about the lessons that we’ve learned, and share those with those of you who might be thinking about doing the same thing.  

The first lesson was to create a clear vision and some urgency.  Our leaders of this group did that.  The leaders of our primary care section did that.  We took small steps forward, but I think that that was really important for us to have that vision.

The second lesson was to assemble and support the right people.  I think we’ve stayed together for three and a half years, the committee has.  I think that says something about how we like being with each other, and also about the talent that’s in the group.  I think that the expertise that’s been brought to this group has lent to our success.  We have people who are talented, and comfortable educating, and comfortable working in small groups.  Comfortable handling conflict and the growing pains that occur with a big change like this, and who are very serious about their work.  

They’re also getting some protected time, which I—this is the people who are in the group where it’s recognized that they have conflicting interests, or competing interests for their time.  They have been able to get salary support to deal with this, or to handle their responsibilities.  We can talk a little more about that in the question and answer if people are interested. 

As I said earlier, the relationships in the group, in the PACT Education Group, were really important, and continue to be that way.  We like being with each other.  Our discussions are very lively.  People are comfortable with dissenting.  The leadership has been very open to talking with people about doing things a different way.  It’s been very collaborative and not at all top down.  I think that’s really important.  

We spent a lot of time doing that.  We worked on ground rules.  We were very intentional about being relationship based in the way we set up our steering group, and continue to do that.  We even have check-ins in our meetings, and like being with each other, like I said.  I think it’s—we learned that it was really helpful to tap into the existing PACT education resources that were out there.  There are a lot of them now at a national level.  You can tap into that, and a number of the VISNs have their own share points too.  

We were very grateful to all of the work that people have done for us, including, as I mentioned earlier, what’s been going on at the VISN.  Jaime Bland at the VISN 23, she’s created some—she and the people who are in the videos that they have with that, and the handouts, and the exercises are very learner center—or very adult learning—based on adult learning theory.  They’re very lively and interactive, and I think they’ve been very useful for us.  Those are available if you want to tap into them.  I encourage you to do that.  

The other thing I think is the leadership in this group was keeping plugged into what was going on.  Looking for what was out there from VACO and the VISN regarding PACT.  Where the support was going to be for it, where the winds were blowing if you will.  Where the opportunities were, and also where the gaps were, and trying to make sure that we didn’t lose momentum if there was a pause in terms of support, like the learning collaborative.  

We took some initiative, took some risks, and created our own opportunities by creating some curricular material.  Some of it worked, some of it didn’t.  I think that we pushed ahead, because we knew that this was going to be something that was important, and was going to last, and that we didn’t want to get behind.  Some of our efforts just required pushing forward.

Another lesson was to do what we could to encourage the teachers, which—and the team members.  By way of the teachers we—over here, we protected some of their time.  Same with the coaches.  For delivering the—for the teachers delivering the content, for the teams, for their meetings, and for the training.  Making sure that they had plenty of clinic time cancelled to do that.  

We also, for the members of the teams, we built in, for the physicians anyway, into their performance pay things that were part of the PACT model such as access.  Then recognizing the entire teams as much as possible for achieving their—what the goals were for the training, and for the PACT metrics.

The last one was to manage the process effectively.  I do think the leadership of this group, Ken Engelhart and Lori Pawelski, managed this process very effectively.  Making sure that we had weekly meetings, kept our momentum, that we had a timeline and a schedule, and that they were comfortable delegating their responsibility, and tapped into all the different resources that were in our group.

For us success was, as I said, wasn’t exactly linear.  It really looked a little bit more like this.  It still does.  Those are some lessons that we’ve learned.  There’ve been a lot of challenges.  I think that, not surprisingly, buy-in for the staff has been a challenge.  There’s been some—we certainly had some resistance early on.  It seems to be breaking down slowly.  

Turn over’s a real problem, because of our training.  We have people who come and go, particularly at the MSA level.  We’ve not been quite sure how to handle that problem.  Getting the teams to meet was initially a problem, but that’s slowly getting better, especially with the coaches.  The leaders, as I mentioned, had a little resistance to the buy-in, and felt that they weren’t skilled in this area.  That seems to be changing a little bit as well as time goes on.  

Then we have had some problems—one of the things I would say we’ve had a problem with is getting patient input.  We initially tried to figure out a way to get a patient involved in our group, but we couldn’t.  We just couldn’t figure out a way to get a patient in our group.  I think that’s a missing part that we would be open for.  If people have figured out how to do that at other places, we’d love to hear how you managed to do that, but we have not done that yet.  

I think this is the last slide.  We were trying to think about how do we know if any of this has made any difference or not.  A somewhat crude metric that we might look to is PACT recognition.  For those of you who don’t know the PACT recognition is something that central office—they look at various metrics that have added up to qualify for—it’s like in the patient-centered medical home model outside of the VA.  

If your practice has certain characteristics, and meets certain benchmarks, you are recognized as a certified performing team or patient-centered medical home.  The VAs PACT recognition scoring includes things like reaching a certain benchmark for continuity, a continuity metric for a certain amount of home care, for reaching out to patients after they’ve been discharged from the hospital, and creating same day access.  

Then also doing a survey.  Each PACT team completes a survey to look at how patient centered they are as a team.  We didn’t have very many people, very many teams that reached that metric in 2011, only four.  By 2012 we’d gone up to 20 that had reached VACO—their standard.  We actually created an internal PACT recognition, because we had a lot of teams here who were very close, and a lot of them who had improved quite a bit.  

We created our own PACT recognition, and had the director of the medical center and all the teams come together.  We had plaques that were given out for the teams.  I think that was a nice ceremony for people.  There are some references, and PDF files, and hyperlinks that I have made available to Molly.  She can make those available to you.  I think that’s where I will stop.  Should we ...

*Moderator:*Thank you Doctor Roth.  Doctor Stewart would you like to take over?

*Dr. Stewart:*Yeah.  If we wanted to just go ahead and answer that poll question.

*Moderator:*All right.  Let me go ahead and pull that up.  There we go.  The poll question is, what would you consider your level of knowledge of the PACT practice model to be?  Answer choices are none or very little, moderate, or a great deal.  

The answers are still streaming in, but it looks like we’re hovering right around 12 percent no none or very little.  48 percent responded with moderate.  38 percent responded with a great deal.  Thank you.

*Dr. Stewart:*Thanks.  I think that’s interesting, and shows us that we’ve made a lot of progress in the last three years or so to get a better sense of the PACT process model.  

I am with the VISN 23 Demonstration Laboratory.  I want to spend just a few minutes sharing with you some of the ways that we’ve partnered with this.  As Doctor Roth has shared with you, in Minneapolis they have this training program that has developed over time and looked at facilitating the skills and the knowledge of people that implement PACT.  

Early on we were doing a parallel project in the demonstration lab where we were working with learning collaborative and tracking 22 early adopting teams, and trying to find out barriers and things that kept them back from fully implementing the PACT model.  One of the things that we found was that there was a leadership deficit.  People really needed to get a better handle on how they led the PACT teams.  Particularly those physicians, or other primary care providers that were in charge of these teams, and had really received no training in their professional schooling or other places, and needed some skills and training.  

About that same time we partnered up with—our demonstration lab partnered up with the Minneapolis Hospital to create this leadership-training program.  For us this has been a really important and a very good partnership.  We’re able to bring some perspective from a broader area of leadership.  That’s where some of my research is.  

I’m affiliated—I’m a professor in The College of Business here at the University of Iowa, and so bring some of the leadership and broader areas we have and combine them with Doctor Roth and some others with a very frontline perspective to develop a training program.  I think we both found that to be a beneficial illustration of the way that operations and research can partner and improve the way that we do things.  

Part of what we’ve done is assess the training to try and figure out optimal ways of delivering the training.  I’ll share a little bit about that with you.  We’ve had this ongoing evaluation of the intervention to make sure that we’re doing things that are actually impacting on the job behavior to the best degree that we can.  Getting that training transferred back to the everyday activities of the people that are members of these PACT teams.  

Of course as we do that, we find out that there are often times some barriers, some difficulties, but trying to identify those and then use our knowledge of those to improve.  One of the things that we’ve been able to do in our research operations partnership is figure out ways to continually improve.  

We call what we do formative evaluation, which suggests that we shouldn’t wait until things are done, and then as researchers feedback what we learned, but it’s this ongoing partnership.  We’re constantly assessing things.  Working together to find new ways to do things, and to deliver better training in this case.  

Let me just share with you a couple of the things that we found.  One of the things that we’re able to do in our training program is compare—as Doctor Roth said, we had a face-to-face meeting when everybody was able to come together into a common place and we conducted our training there.  At another training session that we did later on, where about a third of the participants met in a common location.  Others were spread throughout the different CBOCs and other areas, and were in by V-Tel.  

We tried to make those training programs as similar as they could, except for the delivery method, which was one face-to-face and virtual.  Part of our ongoing training then allowed us to see what people thought about that.  We asked them several questions at the end of their training about how satisfied they were, and whether they liked the training.  

One of the things that we found is that the face-to-face is definitely something that is preferred over the virtual training.  Where there are obviously resource constraints and other issues.  We’re looking into some different ways to maybe present that virtual training.  One of the things that we learned is that does make it more difficult, and as we do that we struggle with that a little bit.  

This is just an example, I think, of how a systematic evaluation helps us pick up on things since part of our demonstration lab side of this we’re able to see really a natural experiment almost here.  Where we could compare these training modules, and get a sense of how things work.  Then now begin to say, “Okay we know that virtual training didn’t work quite as well as the face-to-face.  What might we do, and how might we improve that as we go forward?  

Another thing that we did is we measured training in a couple of areas.  Some of you might be familiar with this.  We talked about training assessment and some levels.  Sometimes we call it level one assessment, did they enjoy the training?  Did people, when they walk out, say, “Oh yeah this was good.  I learned something.”  

There’s a level two assessment that would take it a little beyond that and say, “Did they actually get new skills?”  We might test them and look at their new skills.  Then there’s a level three kind of assessment that says, “Did they take those skills and apply them back on the job?”  Then there would be a level four training assessment that would say, “Did the outcomes change?”  

Well, one of the things we built in here is we had that level one assessment, which I label on this slide, concurrent participant reactions.  These are reactions of people as they went through the training at the end.  You’ll notice those hover around four to four four, so they’re the mid fours with their participant reactions.  

We also had what we might call a level three evaluation, which asks them to reflect back on the job and say, “After a period of time have we been able to apply those principals?”  You’ll notice that there’s quite a bit of we might call creep, or there’s—it’s less clear that what we do in our training interventions actually gets transferred back on the job.  This is a finding that we might expect.  This is not something that’s unique to the VA or the training program that we have.  We find it difficult to transfer the training principals and get them back to on the job behavior.  

As part of our assessment we’re able to capture this and know that yeah they’re training, they’re generally receiving it well, but they’re finding some difficulties as they try and apply those principals back.  We were able to do a few things in part of our evaluation to do that.  Doctor Roth talked about their first two training programs that were the PACT Essentials, or kind of learning about PACT skills.  Another one was teamwork.  

At the end of that we did an assessment where we asked them some items.  We also conducted some focus groups to ask people specificially what it is that they found helped, or made it difficult for them to transfer these principals back to on the job behavior.  Then in our leadership training, as we concluded that training, we actually asked them to reflect back again, after a period of time, and identify how well they’d been able to apply these principals.  

As we went through that data we found that were kind of two big things that were barriers that kept these training principals from being applied as fully as they might like.  One is the staffing and employee turnover.  This was captured by a quote “For us we have new staff, new physicians, new nurses.  That’s another reason we don’t have an effective group.”  One of the real barriers to PACT training is the stability of the group and having everybody in the positions that are needed.  

We go through and we do the training.  People walk out of that training experience pretty positive, but as they get back and over time try to apply those principals, one of the things that we found was really difficult for them is doing it in this context where their team may not be fully staffed or where people were coming and going.  

As Doctor Roth alluded to earlier, this is particularly a problem in the MSA position.  What we sometimes call the clerical associate of that PACT.  There’s just a great deal of turnover.  Because this a team based issue, if you have certain people on the team who aren’t trained it can be detrimental to everyone on the team.  It spreads beyond just that one individual.  

The other barrier that we identified is what we call leadership harmony.  This is captured by a quote from one of our training participants who said, “I really wish that nursing and provider directors had this training to understand that we are accomplish.  They tend to clash trying to attain goals in the old model versus what we are trying to accomplish in the new model.”  I think this is a really good illustration of a principal that is important for us to think about as we conduct training.

That we’re able to go in and work clearly with these people who are frontline leaders in the PACT teams.  As they went back and applied it sometimes they found frustration with the supervisors above them having different goals.  We called this a harmonized program.  Where your leadership again has to come kind of at all levels.  Make sure that the principals that you’re teaching are being reinforced and supported as people go back and interact with their leaders.  

What might we conclude, or some of our main takeaways from the evaluation of what we’ve done in this training program.  One is there’s clearly a preference for face-to-face training.  That’s probably not surprising.  I think it leads us to think about the way that we deliver this model, and think about ways that we might improve some of our virtual interactions.  How we might, going forward, use that technology and take advantage of it better.  

We also identified the fact that getting these behaviors down to on the job is much more difficult obviously than just teaching them.  A couple of the things that really make that problematic is this turnover.  That’s not surprising again given that we’re doing team based training.  That’s what makes part of this PACT model even more difficult.  If you’re going to a training program, and you’re trained in a skill that you do as an individual you can take that back.  You can apply that on the job more easily than you can many of these team based principals that you have to take back and implement as a group and as a collective.  

The other kind of broad takeaway that I might have is that this is for us as an illustration of a really beneficial operations research partnership.  From our side of the demonstration lab it’s helped us really figure out what’s going on on the frontlines.  Being sure that our work is relevant, and partnering, and giving us an opportunity to try and test some of the interventions and things, in this case around leadership, that are really critical to us.  

I think from the operation side it’s also been really helpful too to give some feedback.  To not just go out and do the training, but to do some assessment as far as to be able to give them feedback.  We continue to refine this model.  The partnership that we’ve had has been over a couple of years now, and we see that hopefully going forward.  Where we continue to look at this model, refine it and then be able to find better ways to train.  

Not only do it here in Minneapolis, but to take those lessons now and disseminate them out through other *[inaudible 42:59]* and to the broader health care world.  I think this is a nice illustration of where research and operations is able to partner, and both of us benefit from things that we’re doing together.

Why don’t we go ahead and stop here now, and see if people have some questions for us.  We’ve come to the point where we’ve laid out this program and talked a little about the evaluation.  Anybody have some questions or comments?  We’d be interested to hear other people’s experiences.  If you have anything that you would share that would be similar or different from what we’ve talked about today, or specific questions about content of what we’ve done in the training or evaluation.

*Moderator:*Great.  Thank you.  Yeah, we do have some questions and comments that have come in.  For anybody looking to submit one of their own, simply use the Q&A box.  You can submit questions and comments through there.  The first one that came in, “Is PACT 101 still available on TMS?”

*Dr. Roth:*Good question.  I don’t know the answer.  I’m willing to try to find it out, if that person wants to give me an address.  How would you—if I find that out what would you recommend Molly is the best way to go here?

*Moderator:*That person has your contact information from the slides, so I will ask her to contact you directly.

*Dr. Roth:*Please contact me, and I will be able to tell you.  The content of it is currently in a PDF.  That is one of the things I sent to you—some materials I sent to you Molly.  If that person contacts me directly, I’ll find out if it’s still available.  I’ll look it up—

*Moderator:*Great.  Thank you.  That was Craig Roth speaking, so it’s Craig Roth—craig.Roth@VA.gov.  Yes.  You did give me some supplemental materials, which will be posted in the archive catalog.  You’ll all receive a link to that in the next few days. 

The next question that came in, “Can you expand on why you think buy-in was a challenge, and what you consider sources of resistance—and what you consider the sources of resistances might have been.  Any strategies your team used to develop more buy-in and reduce resistance?”

*Dr. Roth:*The sources of resistance—I think I’ll address—this is Craig Roth talking.  I’ll address the experience with the physicians.  I think that’s where we met the biggest resistance actually, that I’m remembering.  It was because people had—I think it was the idea of sharing the responsibility.  I think that people were uncomfortable with—at the team level, of imagining that someone else, for instance like a pharmacist, could adjust blood pressure medication.  That was a foreign concept to many of them actually.  

Of course we subsequently learned that here, a lot of us have, that if you listen to what you find out about the skills that other people have on the team, they can very much—they can do a better job than we can.  There was resistance to this idea of sharing that responsibility.  That’s one example.  

Another would be resistance to having—to putting patients off a little bit.  Not bringing them back as often, or giving responsibility for following diabetes and blood pressure with a nurse visit rather than a physician visit.  Initially that was a little bit—people didn’t want to give that up.  That was some resistance in terms of sources.  

I think the way we helped break that down was in some of our training sessions we had people bring stories.  We had some wonderful stories.  I have a DVD of a couple stories, that I think we could share if you want, that we showed at one of our sessions.  Where there were lots of groups there, where people told stories about where this new model of care had really helped the patients.  It had really helped the members of the team.  

For instance, one I’m remembering is where an LPN had talked about how she really for the first time felt like she had had been a member of the team, and had really helped a patient, and very favorably influenced the outcome, because of some things that were being done with the new PACT model.  That’s just one example that comes to mind to bring beyond resistance.  I think hearing about success from their colleagues.  

We tried as much as possible to share success stories.  I think stories with—real examples in stories are compelling.  That would be my suggestion to bring down resistance.  That’s just one thing.  Given the time limitations I won’t say more.

*Moderator:*Thank you for that reply.  We do have a comment that came in.  “We made sure to ask our staff, who are also veterans, to provide us feedback from the perspective of both hats, an employee and a patient.“

*Dr. Roth:*That’s wonderful.  Actually that is as close as we’ve been able to get to patients being involved in the process.  I’m pleased to say on our team we have an RN who’s a veteran, we have a—our mental health Ph.D. is a veteran.  We have a couple people who have spouses who are veterans, who get their care here.  That has really been helpful for us.  

*Moderator:*Great.  Thank you.  Someone else writes, “Great presentation.  Did you discover the PACT tool kit hosted on the national systems redesign?”  They’ve also provided the link.  You can get that under the Q&A section.  “Examples have come from the field, and some of your ideas would make a great addition.  I’m on the team that is still collecting tools to be shared widely with a list serve audience of over 3,000 who get periodic updates when new tools are added.”

*Dr. Roth:*This is Craig Roth and I assume that’s directed at me.  I am aware of that tool kit.  We’ve used it, and are grateful for it, so thank you.  If you think there is something here that’s not—that you heard today, that is not there that you think should be, I’d love to hear that and to communicate with you a bit more about it.  We feel very open to sharing what we’ve done here and any of it.

*Moderator:*Thank you.  Somebody just wrote in that the PACT 101 module is still available in TMS.  Thank you for that information.

*Dr. Roth:*Oh great.

*Moderator:*This person writes, “We are about to launch a monthly PACT training for all teams.  However, we’re having problems with logistics of getting all team members together at the same time.  Do you have any recommendations for freeing up teams from clinic work and phones to get together?”

*Dr. Roth:*That is a big challenge.  The way we dealt with it most head on was to offer the same training on several different days.  Usually we would do it back to back.  We’d have a Tuesday and a Wednesday, so that not everybody was out of clinic, because we know that that’s really hard to bring the operations to a standstill.  We tried to pair them like that.  That seemed to help.

*Moderator:*Thank you for that.  We have a comment that came in.  “I wanted to point out that the PEG team, P E G, consisted of volunteers who were working on the project together, as well as working in their full time jobs.  It’s a great group.”  Thank you for that.

*Dr. Roth:*Yes.  Thank you.  We’ve had over the years had some quite a lot of support, and absolutely people who weren’t getting supported who’ve contributed greatly.  

*Moderator:*“What other duties do your coaches have?  At our facility, coaches have other ideas”—I mean, sorry, “have other duties, for instance assigned to PACT, leadership over PACT, etcetera.”

*Dr. Roth:*As it turns out the people who are doing the coaching in our group, there are—at least here in Minneapolis, and I’m afraid I don’t—I can’t speak for the CBOCs, but in Minneapolis the group people have clinical assignments.  The two physicians have clinical assignments.  They’re spending about—I think, they have about five to ten percent of their time protected for coaching.  The rest of the time they’re doing clinical duties in clinical areas.  I don't know if that answered the question?

*Moderator:*Thank you.  They’re always welcome to write in for further clarification.  This next person writes, “I am new to PACT work as of August 2013.  It seems that staff turnover is particularly challenging to the PACT model of teamlets and team practice.  I have worked at a group health in Seattle where team practice is more easily implemented as it is a smaller organization.  

Are there members of PACT teams or leadership that are exploring how to train and implement the PACT model in settings that have high staff turnover and many part time staffers?  For instance, there may be an eye to modifying the training and expectations of teams.”

*Dr. Stewart:*This is Greg Stewart, and I’ll maybe take this one.  This person I think—we identify this all the time.  The movement to PACT and the VA is much more complex than in many of the places where it’s been tried.  The system wide idea is a lot bigger in scope than a smaller practice that’s much more localized.  The issue that you bring up is right on.  We hear this a lot in terms of one of the issues.  

Are people working on it?  Yeah, our demonstration lab and a lot of other people are working on identifying practices and ways.  We find places that have developed some ongoing training programs with this notion that people are always going to be turning over.  As part of their initial introduction to the organization to include some PACT training, and help teams to continually adapt and train.  

Where it becomes not a one shot training program, where we go in and assume that it’s been done, but it’s this ongoing process.  The implementation is an ongoing process, really a new way of delivering care.  That we have to think about what we do very differently.  It’s a complex thing, and that’s why it’s taking some time to unfold and to see how it’s doing.  

Some of our survey work that we’ve done in the demonstration lab would say that the first two years are particularly difficult in terms of transition and getting people into these new roles.  Many of our measures look significantly better in the last year.  This idea that we’re kinda have turned a corner, and just people are working through a lot of that frustration.  Their roles are becoming more clear.  The training has had an impact, and a lot of what people are doing now is improved upon where we’ve been over the couple years.  I think that’s really encouraging.

*Moderator:*Thank you for that reply.  Let’s see.  The next question, “After three years I feel that PACT has plateaued.  How do you reenergize the staff again to this model?”

*Dr. Stewart:*Maybe I’ll start again, and then Craig you can too.  I think that that’s a typical thing that we go through.  There’s this initial work.  Again, I think the answer is that we think of PACT in terms of a bigger picture of delivering effective care.  There are principals and kind of an emphasis that continue to come forward, and helping people see a bigger picture of where we can go and how it can get even better.  

Doctor Roth talked about some of these—importance of sharing stories, and helping people identify things that have done better.  I think that’s part of creating this vision, and helping continue to push it forward.  I don't know if you have anything else to add Doctor Roth?

*Dr. Roth:*I don’t really.  I think that’s—I think those stories are compelling.  Also hearing what other teams are doing has been really valuable.  When we get together we have always made sure that a few people, a few teams get a chance to showcase something that they tried that really made a difference.  It seems to—people resonate with that and think, “Huh, maybe we should do that.”  

I think the face-to-face get-togethers have been really helpful.  To bring six, eight teams together live and have some of this conceptual work together.  Making sure that the facilitators in those groups create opportunities for people to share their energy, and what’s been working for them, and what’s been a benefit for them.  

That group energy has been really useful.  I would strongly encourage you to do that periodically.  I think that’s a shot in the arm for a lot of people, a lot of teams.

*Moderator:*Thank you both for those replies.  The next question, “How did you go about getting approval for protecting ten percent of time for coaches?”

*Dr. Roth:*I’m not the best person to ask about that, because I’m not in the leadership role for that.  If you want to send me an email, I will tell you how that worked.  *[Chuckle]*  I will find it out.  

*Moderator:*Great.  Thank you.  “Do you have any suggestions that might improve quality of virtual PACT training?  It would be good to see what we can do to get widespread distributions of training that can be done via online training.”

*Dr. Stewart:*Yeah.  There are resources and people that are working on this.  I’d say one of the things that is important is to try and make it as interactive as you can.  Obviously there’s this two-way interaction that can be done with V-Tel.  I think that’s an important thing to build that in.  I think it’s important to—often times in these remote sites, have somebody who’s facilitating and working there to make sure that things go.  

Let me just give you an example.  You’re doing a group exercise and you may say, “Okay, go back and work on it as a group.”  Well, if you’re in a site that’s not physically located with them, I think it’s probably helpful to have somebody back at that site that will facilitate that.  Another thing that we found similar to that that’s important, is to have the remote sites report back on some of these issues and try to incorporate them in.  

Another suggestion I might have is to keep your material in kind of short segments.  If you’re doing an hour, hour and a half long presentation, and there’s no stopping or allowing for a group presentation, people quickly zone out more easily than on some of the virtual training.  Those would be two or three suggestions I might have.

*Moderator:*Thank you.  Somebody wrote in with regards to getting protected time for the coaches.  They state that they brought their request for protected time to the PACT steering committee, and told them of the importance of protected time.  That’s how they were able to get it secured.  Thank you for that.  The final question we have, “How many clinics does your nurse manager mange?  Do they have one clinic, two, three, four?”

*Dr. Roth:*The RN?  I’m not quite sure which person they’re referring to there?

*Moderator:*The nurse manager.  

*Dr. Roth:*Nurse manager.

*Moderator:*How many clinics does your nurse manager manage?

*Dr. Roth:*Oh the nurse manager.  I think there are—well, at Minneapolis if I’m identifying the right person, I think they have about 25 PACT teams that are spread out over five clinic areas in our medical center building.  

Then the CBOC clinic nurse manger oversees 21 PACT teams in about—I apologize if I get this wrong, but I think there are about 7 or 8 of the community based outpatient clinics.  There are 21 teams in those—spread out over those sites.  There’s one manager who’s over that group.  Does that answer—

*Moderator:*Great.  Thank you for that reply.  We’ve gotten a lot of people writing in expressing their appreciation for this session.  Saying it was very well presented.  Thank you both for lending your expertise in the field.  

I’m going to ask our participants to hang on for just a second.  I am going to put up our feedback survey.  We would appreciate your feedback, as it’s your opinions that help guide which sessions we have presented.  Thank you very much.  Thank you to our attendees.  Thank you to our speakers.  

Please join us for the next PACT cyber seminar.  It will be taking place on January 29th.  You will receive an email regarding that.  Have a wonderful day everyone.

*[End of Audio]*

Page 1 of 19