**Transcript of Cyberseminar**

**Patient Aligned Care Teams**

**PACT: The Culture of Quality Improvement**

**Developing an Infrastructure to Support PACT Culture of QI**

**Presenter: Susan E. Stockdale, PhD; Devan Kansagara, MD; Anaïs Tuepker, PhD MPH**

**April 17, 2013**

**PACT: The Culture of Quality Improvement**

**April 17, 2013**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at http://www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact:**devan.kansagara@va.gov*

Moderator:It looks like we are just at the top of the hour here; I would like to introduce today’s presenters. Our first presenter is Susan Stockdale; she is Project Director the Veterans Assessment and Improvement Laboratory VISN 22 Research Health Scientist, greater Los Angeles VA HRS&D Center of Excellence, Assistant Research Sociologist at the UCLA Neuropsychiatric Institute Health Services Research Center and Lecturer in the UCLA Departments of Sociology. She will be followed by Devin Kansagara, he is the Co-PI of the VISN 20 Pact demonstrations lab, staff physician at the Portland VA Medical Center and Assistant Professor of Medicine in the division of General Internal Medicine at Oregon Health and Science University. He is joined by Anna Tuepker, HSR&D investigator at the Portland VA Medical Center and Research Assistant Professor in the division of Internal Medicine and Geriatrics at Oregon Health and Science University. With that Susan can I turn things over to you?

Susan Stockdale: Yes, okay well good morning everybody, can everybody hear me okay? I am going to assume you can hear me unless I hear otherwise. In my talk today, I am going to share with you some of our efforts here at the VISN 22 Pact demonstration Lab to develop and infrastructure to support PACT quality improvements. We wanted to start you with a poll question to get some sense of our audience and your involvement with PACT throughout sites so Heidi can you do that for us.

Moderator:Yes I am pulling that up right now okay. And Susan while people are responding here we have gotten a couple comments here if you could speak up a little more or pull that microphone a little closer to you, you should have that boom microphone on that headset. A lot of times, I really have to pull it pretty close to my mouth that would be great thank-you.

Susan Stockdale: Okay.

Moderator:And we will give everyone just a few more seconds to respond here and then I will show the results. Okay there we go.

Susan Stockdale: So it looks like about a quarter of our audience is a part of a PACT teamlet or a PACT Team and then about a quarter of our audience are researchers and then the rest are involved in either administrators or part of a clinical care team outside of primary care so that is good. Alright so I have control again, alright my objective for my talk today are to describe the Vail demonstration lab and quality improvement intervention and to present some of the results from out key stakeholder interviews that show how we did in the first phase of rolling out the infrastructure to support quality improvements PACT. And these results are going are going to focus on our first three demonstration lab sites, I am going to present some of the key function of the infrastructure that we developed as well as some of the barriers and facilitators for the functioning of the infrastructure. And then finally towards the end of the time after our co-presenters go, I am going to present some recommendations for how we can sustain our approach. So first to describe what Vail is, Vail is the Veterans Assessment and Improvement Laboratory for Patient Centers here. The VISN 22 PACT demonstration lab one of five national demonstration labs is funded by the Optimization Care Services we have an evaluation and an intervention component in our labs. Three of the five VAMC’s in our VISN are participating in the Lab and if it can probably best be described as a clinical research partnership that spans off multiple levels from the VISN all the way down to the primary care practices. The focus of our intervention arm is on facilitating evidence based quality improvements for PACT implementation.

This slide provides a little bit of background on the motivation prevalence and the problems that Vail needs to address. First is the lax coordination for quality improvement and innovation across the VISN. Our experience has shown that local facilities and VISN’s often lack structures and processes necessary for conducting QI and in particular what is missing is this ability to adapt effective and interventions to local physicians and to obtain and use data from the VA system to tailor those interventions and then track progress over time as the interventions are rolled out. We like to use this analogy of the sandbox here to demonstrate that national, regional and medical center leadership establishes the sandbox. They provide the resources and the people to do that job, they set the goals and priorities, and they determine the rules and develop the incentive structures. But it is really at the practice level where change happens and at the practice level, what we often see is a lack of interdisciplinary innovation leadership for PACT. And also a patient center focus so it is at the practice level where clinic leaders and the PACT teams identify the problem and often have ideas about how to resolve them. And Vail’s focus is to link these two processes, link the top down and bottom up process change. So the Vail intervention component has two main pieces to it, the first is a multilevel QI infrastructure that we developed in the first phase of Vail. The second is a facilitated EBQI process, this slide here describes the infrastructure that we have develop for Vail. The two main pieces of the infrastructure are first of all our VISN level steering committee, which is composed of VISN and medical center leadership. Their role is to select, demonstrate like practices from the three medical centers that are participating in Vail. So in the first phase of Vail we selected one practice from each medical center and the second phase of Vail which we are wrapping up now. We selected an additional three sites from those three medical centers. The steering committee also sets the priorities for PACT innovation and they review QI innovation project proposals that come in from our side and select which projects will move forward.

The other major piece of the infrastructure is the quality councils and these are the medical center and demonstration type practices. They are basically QI committees that include leaders from administration, nursing, and medicine services, so the three services involved in PACT as well as some of the leaders from the PACT neighborhood like pharmacy and behavioral medicine. In our vision of these quality councils, there should also be a patient representative involved to represent important patient concerns for patient centered care. This slide described the facilitative EBQI that Vail has tried to develop and the main piece of this are locally initiated EBQI projects that come from the demonstration practices so they are developed at the demonstration type practices and implemented there by local clinicians. We call these EBQI projects because they are informed by data, the teams use data to identify data and other evidence to identify the problem and to develop interventions as well as to monitor progress over time and see if what they are doing is working. The innovation piece also used QI methods such as slow mapping, planned, and study act cycles as they are implementing their projects. The Vail demonstration labs provide support for these projects in terms of providing a little bit of seed funding for the projects as well as logistical support and expertise from our health services researchers. In particular we have a couple of work groups that have been very helpful including the measures work groups that help the teams to figure out how they can get data to monitor how they rare doing and an evidence review worker which provides rapid evidence reviews of the literature for the innovation teams. 

Vail also supports the salaries for quality council coordinators, we have one fulltime quality council coordinator at each of the three medical centers that are participating in Vail. So right now as of phase two, these quality council coordinators are each supporting two-demonstration sites because each of the three medical centers at this point in time has two demonstration sites. This is a master’s level person who provides support for meeting administration as well as project management for their QI project including obtaining data and analyzing data. As part of the facilitative EBQI Vail also has learning collaborative which includes biweekly EBQI leadership calls with the innovation leads and the other site leaders and quality council leaders? And then there is also a separate group of calls for the quality council coordinators themselves. These calls are meant to be an opportunity for cross-site sharing as well as to get some support and input from the health services researchers involved in Vail. The collaborative also has biannual conferences with further opportunities to cross-site sharing of their projects and what they are doing at their sites as well as VISN level participation. Vail has also developed SharePoint sites for the teams to use in their tool development and cross-site sharing.

The rest of my talk is going to focus on results from our implementation evaluation that described how we did in terms of implementing the infrastructure of our intervention. So that includes the steering committee and the quality council but I am going to focus on the quality council because these have turned out to be very important at these sites for moving the quality improvement innovation project forward. So the results are from some key stakeholder interviews that we did in the first phase of Vail at our first three demonstration site practices. We did conduct 58 interviews with Vail leadership as well as VISN, VAMC and site leaders and some other partners at our sites that were important for the innovation such as clinical champions and other leaders on the innovation teams. We conducted thematic analysis of summaries of these interviews focusing on the themes of quality council and leadership and then also trying to identify what are some of the barriers of facilitators for the effective functioning of these quality councils and what did some of the key stakeholders say about spread and sustainability of this approach. Alright, so the quality councils, we found at the site that two different models of quality council developed in the first phase of VAIL. In the first model which represented two of our sites, there was an overarching quality council at the healthcare system level and a smaller quality council at the demonstration site practice. The overarching quality council compose of leaderships at the medical center which included primary care leadership as well as system redesign and perhaps some of the executive leadership and related disciplines like myHealtheVet center. At the demonstration site practice itself the smaller quality council was composed mostly of just the people who are working on the innovation project themselves and so this was more of an operations group. And the people that were included in that varied depending on the project that they were doing.

At our third site what we saw was just a quality council at the demonstration site practice itself and this quality council was larger and more inclusive so it included a lot of the site leaders from behavioral medicine and the site manager as well as the primary care leads and the innovation team lead. All of the quality council had some interdisciplinary participation, two of them had patient representatives. There were some differences across the quality council in terms of the regularity of their meetings and also some differences in terms of their regular engagement and inclusion of nursing and administration services. In terms of the key function of these quality councils, most of them focused on the VAIL EBQI innovation project that were approved by the steering committee but one site had an early focus on PACT implementation, later shifting to the VAIL innovation. The quality councils were important for networking and communication, especially across the services involved in primary care. This is a place  where people knew that they could find each other and talk about some important things going on. Not just within VAIL but also around PACT and at the site. The quality councils were important for the ownership of PACT implementation and EBQI so as the quality council developed and matured, they took more and more ownership of EBQI. They decided which innovation projects would move forward, they vetted the project proposal before they went to the steering committee, and they also were involved in allocating time and resources, the innovation team doing the projects. And the quality councils were important for dissemination of effective innovation within and across the sites. So to move the innovation project from the pilot phase where it just included a few providers or a few teamlettes to disseminating it on a wider scale either within the site or within CBOC within the same medical center, the quality council played an important role there.

The next couple slides are going to present some of the facilitators and barriers that we saw from these key stakeholder interviews and in the interest of time I want to leave time for questions and comments at the end so I am not going to read through all these quotes, but there are here on the slide for people to download and to read later if they want. One of the most important things that we saw in terms of facilitators was the existence of interdisciplinary relationship, there is some sort of interdisciplinary infrastructure prior to VAIL implementation. And this quote here demonstrates that one of our types had a strong PACT system redesign group and the VAIL quality council had merged with that group and so that was important for them to be able to move their project forward. Another universally important facilitator was the availablility of quality council coordinators, this person was very important in terms of organizing the meetings and keeping things on track as well as making sure that the innovation project moved forward. Another thing we thought too of our sites was that the level of research or QI expertise of the individuals on the steering committee really also helped move the project forward and to take ownership of the quality improvement at the site. And as this quote demonstrates at one of our quality, councils there were a couple of very strong research level people on the quality council to help move it forward. And another important facilitator was the support of the VAIL researchers and staff so the innovation team that tapped into that support tended to be able to move their project along a little faster and a little bit better.

Universally we thought that having easy access to realtime data and performance measures for quality improvement was also important. Some of the teams knew how to go to VSS and download the site level performance measures for PACT but they did not find these to be terrible useful for conducting QI, they wanted to know how they were doing right now and that was what was needed and that was what VAIL helped them to get. Also the opportunities for cross-site sharing were important, especially the collaborating and participating in the biweekly calls. It was important for the groups to see what each other were doing, they were really learning a lot from each other, and it was motivating to see what other sites were doing. In terms of barriers, one thing that we saw across all the sites and something that we are still trying to figure out how to address and a challenge is the lack of engagement of key leadership across the primary care services. And this has been challenging on a couple of different levels because on the one hand it has been a challenge to figure out which levels of leadership need to be involved. And this may be site specific to some extent and project specific as well and then it is difficult to determine at what level of engagement from these leaders. Certainly the primary care leadership at the facility and at the site level need to be involved and it may depend on the complexity of the site and how many layers of leadership there are or what other leaders need to be involved. One of the consequences of not engaging the right level of leadership is that VAIL was seen early on at some of the sites being kind of a separate thing that was going on over there and not well integrated in the PACT structures. And activities and the concern here is of course that it may impact the sustainability of this approach over time.

These quotes here demonstrate the lack of leadership engagement, these two primary care leaders towards the end of the first phase of VAIL really did not know that much about what was happening and they thought of the problem moving forward. As I said the leadership and figuring out leadership engagement was difficult but we also found that in some cases that the patient of local type leaders to be a barrier in terms of people feeling uncomfortable voicing their opinions in front of their bosses. So this is part of the whole level of engagement that people need to participate at. We found in some instances that the local leadership actually derailed the agenda of the quality council by bringing non-PACT and non-VAIL issues to the quality council. On the one hand, it is an indicator of how important they thought that these bodies could be, but on the other hand, it kind of detracted a little bit from PACT implementation. Another issue that we heard about from one of our key stakeholders was personality problems or the inability of leadership to reach agreement about priorities. And additional barrier was that the quality council in the early phase of VAIL was not really sure what their role was and they were not taking ownership of quality improvement as well thought that they could. These quotes here demonstrate the barriers to the quality council role and this first quote is innovation lead is expressing his perception that the quality council was not sure what their role was and that the person thought that they really could have helped to move the project forward if they would have coordinated better across service leaders.

Finally, a few other barriers that we saw at the sites were a lack of engagement of front line providers. The front line providers did not have enough protected time to participate in the meetings and activities. Also, we heard about lack of accountability mechanisms and the problem with this especially early on before the quality councils really understood what their role was, the lack of accountability mechanisms resulted in the lack of product coming out of the innovation project. And then just generally a lack of communication between VAIL and primary care leadership within and among the primary leadership and service leadership themselves and then between VAIL and the front line providers and staff. So to summarize these results we feel pretty optimistic actually about moving forward. We feel that the goals that we set out to do in our first phase of VAIL have been met. We have planted to seed for quality and for a culture to develop at multiple levels across our VISN. We found that the quality councils do play an important role in quality improvement for PACT, in particular having multi level interdisciplinary leadership has been necessary for linking the top down mandates that come from the VISN with the bottom up change process that happens in clinics. The quality council coordinators are a key piece of this as its access to realtime data for conducting local QI. Where we can improve, we are still working on the engaging leadership piece of this and also moving forward into phase three, as they move into phase three, we will need to do a better job of integrating balance to the structures and activity that are already existing at the site so that can be sustained for the long run. And this slide here just with my contact information as well as my collaborators and co0authors and with that, I will turn it over to Deven and Anna.

Anais:So thank you and this is Anna speaking, hello everyone thank you so much for being with us today. We are going to pick up and continue on the same theme of how we build the culture of quality improvement and we are going to focus today on talking about Portland’s VA Medical Center’s Primary Care Workgroup Initiative. 

Moderator:Anna I think we just lost your audio. Okay I apologize to the audience, I do not think they realize, while I am trying to get them back on the audio Susan I have questions here if you could possibly handle that while I am trying to get them back on. The question is could you talked more about realtime data and why PACT metrics were not seen as useful, does not the compass measures reflect important PACT processes for EBQI?

Susan:Sure I am happy to talk about that actually and we did use our, are Anna and Deven back on?

Deven:Yes sorry about that, we had technical difficulties on our end.

Anais:We were trying to improve the sound quality and instead we just made it obviously much worse.

Moderator:Well I am glad you guys were able to get back so if is okay Susan we will hold that question until the end and let Deven and Anais speak now.

Susan:Sure yes.

Anais:Okay thank you again, apologies for that so, let us try and make up for lost time but we do want to start by acknowledging our whole team. We are part of the VISN 20 demo lab and Susan already gave some background on the demo lab, so we want to thank everyone on our team and especially our partners in implementation within primary care division. I am going to say we a lot in this presentation and I really cannot take a lot of personal credit for some of the ideas behind this workgroup initiative that really came from our primary care leadership. And I also want to thank the workgroup initiate participants themselves; there were about 60 people from across the primary care division to participate in this. So it is a lot of people and they put in their time. So we have a poll question, is there a, I am not sure what you are seeing on your screen right now? 

Moderator:It just takes me a second to pull it up there so the audience should have it right about now.

Anais:So the question we wanted to know was how much experience do you have with quality important projects and your choices are lots of experience, you have worked on multiple projects. Some experience you have completed a project, or you are doing your first project right now, or you have no past experience with doing a QI project.

Moderator:And we will give just a few more seconds for responses to come in there, and there you go.

Anais:So I am not actually seeing the results, can you tell us what the results are?

Moderator:I am sorry; I thought you were seeing those, about twenty-nine percent say lots of experience. Twenty seven percent say some experience, eleven percent doing their first project right now and thirty three percent no experience with a QI project.

Anais:So we have a huge range which is interesting and questions from people everywhere along that spectrum will be good. So to give some background why workgroups, but also what are these workgroups, so let me just say briefly what they are in terms of who was on them. So the workgroups were an initiative and I am going to say more about how they came into being but essentially each workgroup was about ten people, they were drawn from primary care staff in all roles on the PACT teamlettes that we tried to get primary care providers, nurse care managers, clerical and clinical associates on these teams. They came very deliberately from different community based outpatient clinics and then each group was co-lead by clinic level management which in our system means for each CBOC there is a group practice manager and an operations manager and each workgroup had a group practice manager and an operations manager from two different CBOC;s leading a group. And then each group also had someone from our demonstration lab, in fact it usually had an investigator level person and a research assistant available to assist or facilitate the group in whichever way they wanted to make use of that person. So when we talk about a workgroup those are the people on the workgroup. So what was the thinking behind these workgroups? Well within primary care and the context of implementing PACT, there was an emphasis on wanting to find solutions to some common challenges from the ground up to identify as Susan stressed as well, the need to find solutions among people really working on these issues on a day to day kind of way.

There was a desire to build staff content knowledge around particular problem areas and to build their collaboration skills. There was also a recognition that leadership abilities of clinic managers were going to be very important to success in PACT implementation and we wanted to have an arena to further development those leadership abilities. And then finally there was a deliberate thinking that the interdisciplinary kind of team approaches of PACT in order for PACT as an initiative to succeed ultimately that kind of thinking needs to expand way beyond the teamlettes. And so here, we were trying to find a way to make that happen in a very tangible way beyond the teamlettes. So I will talk some more about how we tried to do that but a little bit more context. So this was staged to happen in year two of the PACT rollout. The first year there were so many changes going on and indeed making that new team based approach to care really the norm within the clinics that was the focus. So this initiative was deliberately staged to happen in year two and although there were, many areas that could have been focused on, primary care leadership deliberately decided to limit this to five groups. And these groups were prioritized to focus on critical performance areas that had already been identified by both primary care leadership and facility leadership as key challenges. So the workgroup topics were for chronic pain management, congestive heart failure management, preoperative management, referral management and then team formation and function all of which with the possible exception of team formation and function which clearly is an important PACT theme. All of the other workgroups really are focused on care coordination not just within primary care but also with other services and divisions within the VA.

I just want to mention a parallel level effort going on at the same time, the patient flow collaborative which in some ways is very similar to the workgroups but meaning it involved staff from primary care in an attempt to find solutions and innovation. But it came about in a different way. So I am not really going to talk about it today but our evaluation at the demo lab is going to be looking at the differences in how these efforts came about and seeing if there is anything, we can learn about what makes these kinds of efforts successful. So one last point quickly just on this is interesting I think that this approach both assumed a certain level of strength at the clinic level in terms of leadership and facilitation skills, and also intended to build on that. And I think that that is an interesting thing to think about both for our project and also in light of what Susan said about how important the leadership and ownership of these projects really is in their success. So just to reiterate again one of the key things I think I these groups was the deliberate attempt to build relationships and build teams that went beyond the clinics. So if yo think about our organization as starting with the patient and their PACT team, and then the next level out in relationships and patient care is the clinic or the CBOC, then most of the sites participating in our demo lab are community based outpatient clinics. And then those are all together under primary care division which itself is one division among many in the medical center. These are the many levels that people need to be able to work comfortably and have strong working relationships across and the workgroups because of the topics they were charged with, it was clearly envisioned that they would first of all extent relationships outside of the clinic because of who was on the workgroups. But then also build relationships outside of primary care division and with the medical center.

And this is the model that also went into thinking about what we wanted these workgroup to accomplish so in a way they mirrored the PACT team because the themes that they are trying to achieve as a process are very similar. So developing content expertise around common challenges in the delivery of care, there is the hope that they will encourage the respect and collaboration across the disciplines and rules by having people work together in an interdisciplinary environment that these groups will also help people develop effective communication skills and problem solving skills. And whereas all of these elements come into play for a PACT team, in order to find individual team solutions, what is going to work for our team to try and fix whatever the problem is or to address whatever the challenge is. The idea with the workgroups was that they would develop and deploy these skills in order to develop system solutions and so that ties in very much with the idea that they are working across clinics and indeed across the facility and really thinking with the systems perspective.. So I am going to give you one example of these groups, it is the group that I was actually part of os I know it firsthand. This is the referral management group which was charged with identifying ways to improve the process of patient referral from primary care to specialty care. And the way this workgroup operated all of these groups met over quite a long period, they were initially given 12 months and then that timeline was extended to 18 months to develop their recommendation or product that would address the challenge they have been charged with. So this group met sometimes monthly, sometimes every other week, predominantly in face to face meetings which are quite hard to organize but they did pull it off and we will talk more about how they did it. At the beginning they held a retreat actually to establish both relationships between people on the workgroups that did not always know each other and then also to set their agenda. 

Very early on in the process they pulled data on referrals within our system in order to understand better where the most referrals were, where were patients facing the biggest delays in getting into specialty care, in order to be able to answer those questions with data and the demo lab helped with that. And then this group organized dialogue sessions with speciality services to invite them to the table to together discuss what works well in the process for you, what does not. And through that process they met with 13 different specialty services and the end result was a report which had 31 quite specific recommendations some of which were internal to primary care division and some which were across the Portland Medical Center. These recommendations were sometimes technical in the sense of suggesting changes to our CPRS template that are used in referrals and sometime more cultural such as suggestions for how staff should be mentored or trained at the very beginning of their service. As well as refreshers, but finding ways to make sure that people are aware of changes in practice and ways to improve cross service and also for patients. So those were the kinds of recommendations this group came up with, now this evaluation phase which has really just starting as the workgroups presented their product about a month ago to primary care leadership. And so now we are entering an evaluation phase and of course, a key piece of that evaluation will be to track which recommendations have actually been implemented. And I think another piece that is going to be really crucial to see how well this process works not just in terms of achieving its process outcome but it is finding actual solutions. I am going to turn it over now to Deven.

Deven:Alright, hello so I am just going to talk a little bit about some preliminary evaluation we have done so Anais described one of the workgroups and there were five workgroups in all and actually a sixth that came about in a different way. So whenever you have multiple workgroups and multiple processes going on there is inevitable variability in the success with which workgroups were able to meet, accomplish goals and essentially be productive. And with this variability, there is an opportunity to examine what some of the facilitators and barriers were for quality improvement in this context. So, so far we feel that surveys were sent out to 61 participants, we have received 25 back and we are in the process of getting more. Two workgroups were the most widely represented amongst these surveys. Anais had mentioned that one of the goals of the workgroups was to really foster a sense of interdisciplinary communication and a sense of empowerment with doing this kind of work. And many participants did not feel that they gained these skills, sixty percent felt they gained these skills, two thirds felt that their contributions were valued, sixty percent felt the group had achieved its goals. On a different note, only two respondents actually felt that their work had received by the end of the process, any recognition or credit. We also had comments on each of these surveys and the comments also showed that overall staff valued to workgroup process and meeting with others, building relationships with people from other disciplines and from other clinics and also from other services. But participants also had a number of specific recommendations for how the process of QI could work better in the context of PACT.

So one that came up a number of times was that it was important that the initial goals of the workgroup were clearer as some participants felt unsure of what their charge was. Other participants had acknowledged that it might be important that participants want to be there. So as Anais mentioned the way the workgroups came about, participation in some cases was assigned and for some workgroups and in other cases I think participating felt that having been assigned to these positions that they felt more distance from the subject and were not as engaged. There are several suggestions that were of a logistic sort in a quite mundane but quite important, one of these was that phone meetings were not felt to be that effective and face-to-face meetings were where the rubber met the road. Creating regular meeting times was important because just the chaos of scheduling workgroup meeting was a big barrier. Susan had mentioned this also, so blocking out clinic time for people to attend was important, many acknowledged that having a data analyst involved through the process and I think this is something that the VAIL Lab has been able to do, would have been an important thing. I believe this also echoes some things that Susan had said including high-level leaders not only in the final reporting process but also through the process of doing the work would be important. So in other words having key operational stakeholders involved from the outset would help move things forward in a way where you might increase the chance of having an actionable product.

Many participants felt that they were curious to know what happened to all the work that they had done and had not heard back on what happened with their work. Some groups but not all so many groups did quite well in meeting and all participants participated equally but some groups felt that they might have benefitted from facilitated discussion so that all members were heard from and that there was a variety of opinions expressed. So Anais had mentioned demo lab participants were able to longitudinally observe the workgroup process so at the end of these initiatives we gathered the demo lab participants that have been observing the workgroups and conducted a focus group with them. They had several observations that echoed some of the participant observations these included that they also observed that non-providers, or non-physician workgroup members faced even greater barriers to attendance, it was much harder for them to block clinic time. They also found that the clarity of purpose while very well articulated in some groups was less well articulated in other groups and they found that the actual work that had gotten done often happened during actual sessions. So Anais had mentioned the referral management group and they were able to get a lot done in these retreat type of formats and the process of trying to do work over the phone and then via email did not seem to be quite as productive. We are still in the process of evaluating this so we are collecting more surveys and Anais had mentioned we will also track the number of recommendations that ultimately get implemented and do some more qualitative analysis in the future.

So we and Susan will sum up by trying to pull out some practical recommendations that we can offer both from the VAIL experience perspective and from our experience. So we will start here, one of these is that it is important to choose topics with strong relevance for participants. So I think in the VAIL model having the quality council prioritize topics is one way of doing that but also identifying participants to do the actual QI work they have to feel close to the topic and engaged in it. Again the logistical issues end up being really key so blocking out time for staff participation and not just the PCP’s is really important and ideally scheduling actual three or four hour blocks of time to do the work. Again it is really important to set clear goals and expectations up front to understand how success will be measured and to make sure that there is an attainable goal. I think participants really wanted acknowledgement for the work they were doing here, they were already doing busy clinical work but this was additional work in addition to their clinical responsibilities. So it might be useful to incorporate QI work into things like performance evaluations or to have something like a QI research day type activity for frontline participants to show off their work. Closing the circle so it is important to feedback what happens to QI efforts so once the product does get developed, what has happened to it once it gets disseminated feeding back that broader experience to the initial participants. And as I think, Susan also has alluded to and we will also allude to is that I think to insure consistency in goal setting and understanding how to use metrics in doing QI work and so forth. It is important to have members that either have members that have experience doing QI or to have the facility to introduce QI expertise into the mix. Some groups might benefit form a facilitated discussion and then obviously using data for QI is important. And with that, we will turn it over to Susan.

Susan:Hopefully everybody can hear me okay, alright well in addition to the recommendations that Deven made, all those recommendations certainly hold true for the VAIL quality councils as well. But from our perspective moving forward if we want the VAIL approach to be sustainable in the future, we recommend it is important for VAIL to try to perhaps the quality council is not necessary. A separate quality council might not be necessary but at least having some sort of interdisciplinary leadership structures possibly something that is already existing at the medical center to take on this QI function that the quality councils are doing in VAIL. From what we have seen so far this may require a level of autonomy for primary care that is not currently the norm for most sites and it would require identifying some key site champions to keep the momentum going. And it might take some time if these relationships do not already exist it is going to take some time to build them and for this group to really feel like it owns QI and give them momentum working together. From what we saw also having the quality council coordinator or a QI coordinator for PACT facilitated introducing these QI efforts into the PACT changes. 

The quality I cannot emphasize that enough, how important these quality council’s coordinators became not just for the VAIL projects but also they are really PACT facilitators at the site. They are going to the BFOC, pulling down the data, and putting it in some sort of adjustable format and they are talking to the teamlettes about how they are doing. They are very important for quality improvement around PACT and as part of that having access to data or some sort of measures for reporting IT infrastructure at the facility is necessary because the stuff that you can get off of the VSSC and be partially addressing this persons question. The data you get off of VSSC is helpful but the teams really want to see how they are doing now. They want to address things that they are doing wrong now. I can give an example of that with our efforts around improving continuity, the continuity measure that you get off the VFC is a 12 month average so that is on average how you have done over the past 12 months. They what to know how they did in the last two weeks so that they can do somethings in the next two weeks to improve that measure for the month and that is what the VAIL researchers were able to do by having access to an IT infrastructure to pull out the data that they need. And then of course to do all of this facility senior administrations support or it would buy in is important in order to obtain the resources and to get help from the knowledge management department and then whatever IT infrastructure they have as well as engaging VISN leadership. Especially in terms of knowing what the priorities are for VISN leadership and making sure that the resources are there. I am not sure if I have control of the slides, I do not think I do. I guess I do, so that is it for my recommendations and so I guess now we can open it up for questions.

Moderator:Great thank you we do have several pending questions right now so I am just going to start at the top and work our way down. Can you give an example of what a VAIL innovation might be?

Susan:An example of a VAIL innovation, one of our innovations that we now have packaged up as a toolkit was an innovation to enroll veterans in My Health Vet at the point of care and actually get them authenticated at the point of care. Because what happens is, the veterans have to go to the business office to fill out some forms so they can opt in to getting messages from their provider and for seeing some of their health records. So one of our groups had a process in the clinic where there was a clinical reminder that popped up and there LVN would ask them do you want to be authenticated today for My Healthy Vet and she had the forms right there for them to sign. That was just basically one stop shopping so they could get signed up for My Healthy Vet.

Moderator:Great thank you, and Susan this is another question that came in while you were speaking. Did you find any methods and solutions to overcome barriers?

Susan:I am trying to think of an example of that. I think we are still working on that, I mean some of the things around data we have found some solutions and figured out whom to work with. Some of it we are still trying to figure out such as engaging leadership and that certainly is going to be what we are doing going forward figuring out how to overcome some of these barriers to quality improvement. There are things like not having dedicated time to do the quality improvement project some of that was at least partially alleviated by VAIL requesting time for innovation leads and others working on the teams to have release time. VAIL has also been involved with VISN leadership to try to address some of the challenges overall with PACT and it will soon be trying VISN wide innovation around PACT in terms of giving clinic leaders more autonomy over PACT implementation of their site.

Moderator:Great thank you and this is the question I started asking earlier, if you could talk more about realtime data and why PACT metrics were not seen as useful. Does the compass not measures reflect important PACT processes for EBQI?

Susan:Yes and I do have to correct myself a little bit there because the PACT compass has been useful and the teams have been using it for the quality council coordinators have been going to the PACT compass getting data for the teamlettes level and putting it into a digestible format. They can see at least on the five main PACT measures how they are doing, but as I said if they want to make improvements this month to change their performance measure for this month, the compass data is from last month or two months ago and does not really help them with this month. So they need the data to show them how they are doing yesterday and the day before and more aspecific data. Sometimes at the patient level to see what went wrong, why did this person go to the ED or could we have prevented this person from going to the ED instead of coming into primary care.

Moderator:Great thank you, next question this is also for you Susan, how do you figure that you have planted the seeds successfully given all the barriers presented. It seems like there was a lot to overcome.

Susan:Yes, there is a lot to overcome, but we feel that we do have the quality council established at the sites now, we are working on integrating them into the existing structures so they will be sustainable going forward. And we have also given them the basics in terms of QI mentorship and QI tools that they can use to take overt QI at their sites and we are starting to see signs of that now in phase two that they are really starting to take ownership of quality improvements. They are very actively, I mean some of them did not even know how to VSSC before, some of the site leaders and now they are going there and pulling the data off to see how they are doing. So just instilling in them the values of quality improvement and using quality improvement in their day to day process and making that a part of their daily routine. We are starting to see some of that in phase two although I did not present any of those results, we are just now starting to look at the results from our data from phase two.

Moderator:Great thank-you, and I just want to throw it out there we are at the top of the hour here, we do still have a few pending questions. I just want to double check with the presenters that you are okay staying on a little while to get through some of these questions.

Susan:Yes I am fine.

Deven:Sure.

Anais:We are fine.

Moderator:Okay great, the next question here came in while Anais was talking. The next question we have here, were there any baseline quality measurements?

Anais:Well in terms of the skills or the feelings of the workgroup members that we were talking about in our evaluation no. I think each workgroup depending on their topic might be able to point to the baseline data available when they started, but it would vary by workgroup and our evaluation did not collect baseline data no. if I am not understanding that question, try to clarify it and I will try again.

Moderator:Okay great, thank you, the next question I have here, was VA TMMCF the official VA process improvement handbook followed?

Anais:So for the workgroups I would have to say no, if the question is directed at us no, and I think that that is part of what the presentation as a whole is getting at is that there is both an implementation right in the clinics that are doing PACT level of trying to build this culture. And then there is a systems piece and I think one of the potential weaknesses of our initiative is just that in some ways it is not well connected to the system, whether that is through VA guidance that is already available on how to conduct quality improvements. And so I think putting what we have looked at together with what Susan and the people at VAIL have looked at which is really much more of a systems piece, we hope that the picture that has come across is you need to have the culture needs to be built at many levels. Certainly one piece of that culture is tying into existing guidelines, which I have to say the workgroups did not do.

Susan:yes, this is Susan and I can say that at one of our sites we did as I mentioned see a very strong systems redesign element and I believe the VA TMMCF sort of the system redesign groups perspective. At the other two sites, we did not see a whole lot of involvement of system redesign with PACT itself, with the PACT implementation themselves. They were doing things that were half on the margins of PACT like doing flow mapping of the patients flow and that kind of thing but we did not see a whole lot of on the ground interaction between system redesign and the frontline clinicians in terms of trying to improve quality. So I am not sure how much they were actually following the handbook.

Moderator:We did get a comment back that VA TMMCF piece actually addresses culture.

Susan:This is Susan, I cannot say that we heard any references or at least not in our key stakeholder interviews, we did not see any references to that, to the TMMCF handbook or any of that method.

Anais:Yes, I think within our workgroups we had a similar experience that that is not a resource that people are really familiar with.

Moderator:Okay, sounds good we will move on to the next question then. The next question I have were any processes studied improved and were any of them connected to actual patient care outcomes?

Deven:So this is Deven, in the workgroups some of them like the referral management one that Anais is talking about is now fielding patient surveys to try and answer that question because if what they are trying to improve is the patient experience as they go through the referral process. One of the ways to answer that is through surveys that the workgroup is actually with the demo lab help fielding these surveys. The ED primary care collaborative workgroup that Anais had mentioned before also is tracking patient experience as well and there is obviously ER utilization that can be tracked but it is probably too early to see an impact on that.

Anais:I think just to add to that because it might also be relevant to the question if the question is were any of these recommendations actually implemented, at this point very few because we as I said the reports of the groups were just made about a month ago. However, I know some of the actions that could be taken by these groups, some of their recommendations especially the ones that were easier to implement did happen along the way. So just to give one example I am familiar with  within the referral management group, there were the data analysis early on identified that referrals were still being made for some services that patients could actually, they did not need a referral for. They could directly contact that clinic and schedule care themselves and so one of the changes that came out of that was people from the group going back to their clinics and raising the issue. There was already I believe a letter to say to patients you can do this yourself but making sure that that letter was actually being used and getting out to patients. Obviously one of the things to track in our followup is as well has that really changed, are more patients getting in quickly to care by bypassing the sometimes slow referral process where they can. But there are some small changes like that that I think did already happen but we need a bit more time to see which ones are going to be implemented and stay.

Deven:I think this larger question on impact on health outcomes and other metrics is an important one and will take time to answer.

Moderator:We did receive a followup from the questioner here, it would be nice to know if any of the processes relate for instance to hospital admission, readmission, and then if admissions and readmissions were measured at baselines and following improvement.

Deven:Yes, the work I can say from our perspective, five of the workgroups were not really doing things that would impact either of those metrics in the near term. The sixth one that would be primary care one is designed to improve communication between the emergency room and primary care. And so one of the things they have created is something called the Monday Manifesto where every Monday a list is sent out to each teamlettes identifying their patients from their panel that have been to the emergency room. And then this will prompt an ED followup phone call similar to the post discharge followup calls and then the idea would be to track ER utilization thereafter. So again it is still too early in process but there are plans to do that. There is a, not part of the workgroup initiative but there is also a parallel process ongoing on post discharge followup call and these are initiated from the PACT teams and at pretty high penetrants now and we are conducting a time series analysis to examine the impact of those on readmission rates.

Moderator:Okay great, thank you, the next question here, is that My Healthy Vet reminder in CPRS now?

Susan:It is in CPRS now, we have it in VISN 22, we have a couple of different versions of it and if people are interested, they could contact me and I can point them to the people who have that reminder dialogue.

Moderator:Great sounds good, and your contact information is on the screen right now so people have that available right there. And that actually does wrap up all of the questions that we have submitted at this time so with that I would like to thank our presenters Susan, Deven, and Anais, thank you very much for taking the time to put together these presentations and present for our audience. I know our PACT audience finds so much value from these sessions; they really appreciate all the time that you put into this. For our audience thank you very much for joining us today, as you exit the session today you will be prompted with a feedback survey, if you could take a few moments to fill that out we would all very much appreciate getting feedback. We definitely do read through and respond to much of it. Thank you everyone for joining us fort today’s PACT cyber seminar, we hope to see you at a future session. Thank you this does conclude today’s PACT cyber seminar.

Page 1 of 15

Page 15 of 15