**Transcript of Cyberseminar**

**Timely Topics of Interest**

**Evaluation of Improvement Capability Grants: Lessons from the Field, Part 2**

**Presenters: Carol VanDeusen Lukas, EdD; Martin P. Charns, DBA**

**February 2013**

**Evaluation of Improvement Capability Grants: Lessons from the Field, Part 2**

**February 2013**

Molly:We are at the top of the hour, so at this time I would like to introduce our two speakers. We have Dr. Marty Charns. He is the Director of the Center for Organization, Leadership & Management Research, known as COLMR, a VA HSR&D Center of Excellence in the VA Boston Healthcare System. He is also Professor of Health Policy & Management and Director of Program on Healthcare Organization Studies at the Boston University School of Public Health. Joining him today is Dr. Carol VanDeusen Lukas. She also works for the Center for Organization, Leadership & Management Research at the VA Boston Healthcare System and in the Department of Health and Policy Management at Boston University School of Public Health. So I would like to thank them for presenting for us today and I would like to turn it over to you at this time.

Dr. Carol VanDeusen Lukas:Thank you, Molly. This is Carol. I will kick off. And of course, I have to ...

Dr. Martin P. Charns:Get the screen.

Dr. Carol VanDeusen Lukas:... get the screen going. And I apologize if you are seeing my reminder for the Cyber seminar. As Molly said, this is what we consider Part 2 of a session we started last time. Recognizing that some of you may not have been on last time’s call and recognizing that even for those of you who were, a lot of things have happened in between, we just want to begin with a quick recap. 

And I think the important thing to begin with is reminding us all why we are interested in improving capability. This is certainly a priority of Dr. Petzel, our under secretary. It also is a high priority with our system leaders in the field and network directors, the medical center directors. And one of their explicit performance objectives is to lead change and particularly create a culture of continuous improvement and learning. And the reason we are concerned with this—we and the royal we is the organization—is that Veterans Health Administration is always striving, as we know, to move to higher levels of performance. And the way to do this is to create a system where we can, on an ongoing basis, change and improve.

Last time we asked a number of questions, interactive questions. We are going to cut down on those this time, but we do want to begin with a question of, who is on the phone? And if you could answer this poll—Molly is going to take it over and since this is a poll question, just give us a sense and give everybody a sense of who is participating.

Molly:Thank you very much. We have launched the poll, so the question is, in the past year have you been involved in a team to improve work processes or outcomes? No. Yes, as a team member. Yes, as a team leader. Yes, as an improvement advisor/facilitator/coach. 

Looks like we have had about two-thirds of our audience answer. We will give people a few more seconds to select their option. Okay. And the responses have stopped streaming in. So at this time I am going to close the poll and share the results. Carol, you should be able to see those on your screen now.

Dr. Carol VanDeusen Lukas:Yes, thank you. So it looks like a small group has not been involved in any organized effort to improve work processes in the last year; 37 percent have been involved as team members; 10 percent as team leaders; and 39 percent as improvement advisors, coaches or facilitators. So we have a group on the phone that is really experienced with these efforts. And hopefully during the question period and the comment period, you can contribute to the discussion.

So again to continue the recap, we at the Center for Organization, Leadership & Management Research did – were asked by the Office of System Redesign to evaluate their improvement capability grants. And these grants were awarded competitively with a very broad RFP that encouraged local sites to build on their – to develop their own strategies to build improvement capability.

There were ten grants awarded in 2009, 20 grants awarded in 2010 to both medical centers and network centers. These were three-year grants and all of them are now completed.

And it ... does not want to advance.

Molly:Just click anywhere on the slide and it will move forward.

Dr. Carol VanDeusen Lukas:There we go. One of the questions from last time was, which medical centers are participating. There had also been a request for all the projects they are doing and that was a little much to fit in a slide show. So we have listed the 2009 and the 2010 grantees. As I said, there are 30 of them.

And as we also highlighted last time and consistent with the system redesign’s initiative to have these be locally developed strategies, they are really multiple types of activities funded with these grants. So there is a small number that has funded infrastructure improvement such as creating a registry, such as creating a learning center, an education resource center.

Many sites used training as their focus and these projects included very often initially having trainers come into the sites. I know in at least one case they consulted – they contracted with a private consulting firm. That was not particularly successful because it was a group that was not used to working with healthcare organizations. It was not used to working with the VA.

The more successful training tended to be from our VERCs, our Midwest Mountain VERC and our VA Case VERC. And I think that that has become a more frequent option as the activities have gone along. 

Also, as the grants have ended and as sites are maturing in the capability for training, more are starting to do training internally.

A second category was projects that just did clinical improvements, and the notion was by doing these clinical improvements not only would they make the improvement, but that they would also learn improvement skills that could be applied to other projects. But that was not always a formal part of their activity.

So as examples of clinical projects that were included, one site focused on surgical specialty clinics, maximizing efficiency there through engineering simulation. Another site looked at ambulatory care optimization in their CBOCs. Another introduced an intensive clinical care management program for diabetes. So those are just examples of the projects.

Then we had some places that combined training and clinical projects explicitly in their grant. So in one of these the initial clinical focus was on primary care. This was before PACTs had started, so this was an important focus in that medical center. At the same time, that was a VISN grantee, did training throughout the VISN in improvement techniques, and they also created in each medical center what I will call an Improvement Resource Team. So this included the system redesign coordinator, a clinical person to be the contents sponsor, a clinical applications coordinator to serve as the data resource, and then a data analyst. So that was a multi-pronged initiative.

So that is just a quick picture of some of the activities that these sites undertook with their grants. As I mentioned, all the grants are now completed, but a lot of those activities are continuing.

So. At COLMR we have the privilege of following these sites for the duration of their three-year grants, and have now followed up on the 2009 grants for a followup year. The work that we are going to talk about today is based primarily on those visits. So again, I am almost done with summarizing last time. 

We talked about the majority of the sites by our rating really showed some good progress. So 74 percent fully or mostly—not perfect but a lot of the pieces in place—met their grant objectives;

another 65 percent spread their grant activities; 68 percent planned to sustain their grant activities after the funding ends. I think most of us recognized that it is more challenging to spread and sustain activities than it is to meet the initial objectives, as challenging as those are.

So also to recap last time, we talked about an improvement culture really having three main components, and this is, of course, an oversimplification. But the first is having engaged staff both to again help solve the problems because they are going to be the ones that on the front line are the most closely involved with the issues in care delivery with the administrative areas that we want to improve. But it is also a way to get them motivated to do other improvement, give them those improvement skills that can carry forward in other projects.

We talked last time about an aspect of improvement culture having an infrastructure where you do not rely necessarily on a single office or a single individual to have those improvement skills. You try to build those in throughout the organization. There are several models for doing that, which we can come back to if there is time.

And most important—I should not say most important, but equally important, senior leaders are really engaged in system improvement. And we focus here on senior leaders in the organization because they are the ones that set the vision. They are also the ones that create the structures and the processes and bring people together, get the resources and the accountability to be able to work on improvement in an ongoing way.

So with that recap, we want to turn to the second question for you. This is again a question that we asked last time. This time we are reframing it a little bit. How would you describe the improvement culture in your organization? And I will turn it to Molly at that point.

Molly:Thank you very much. And the options are, improvement work throughout the organization, scattered pockets of improvement work, minimal improvement activities. And we have had about 40 percent of our audience vote, so we will give people more time to select their option. Okay. It looks like about two-thirds of our audience have voted and the answers have stopped streaming in. So I am going to go ahead and close the poll at this time and I will share the results.

Dr. Carol VanDeusen Lukas:Okay. So this is good. We have got a quarter of the respondents saying there is improvement works throughout their organization. Two-thirds of you said that you have got scattered pockets of improvement, and 11 percent said minimal improvement. I think this fits with last time. We had asked about minimal developing and mature and do not know of the categories, and two-thirds of the response last time said that their organizations they would rate in the developing stage of improvement culture. 

Okay. So that helps. We are getting there but we are not completely there yet.

So last time we identified seven factors in developing improvement. The first is the improvement training is linked to the application with the improvement projects. We talked about the fact that if you have training and do not have a way to use it when you get back, people get discouraged. Those improvement skills dry up. But this has turned out to be – and it is also the reverse. If you are doing an improvement project but you do not have – you are not learning skills of how to do another project, that sort of has limited application.

In terms of improvement training being linked to application, this is an area where a lot of medical centers have challenges. It is not easy if you send a large group of people for training to bring them back. And I think we will talk a little bit more about that as we go on.

We need data and skills to analyze. And we need the strong improvement infrastructure. I think what we have emphasized here and focused on is improvement infrastructure within the medical center. 

But we also wanted to highlight that the VISNs have an important role in this, and in several of our VISN grantees, they had monthly calls with their system redesign coordinators. Sometimes those were linked with their quality management officers to really not only share information down from the VISN but also provide a venue for exchange among the system redesign coordinators. And that exchange not only happened on the calls, but began to create and fostered a social network that then the system redesign coordinators could call one another when they came up against a problem, needed advice, needed a sounding board, started to venture into a new area. And the social networks turned out to be, or are turning out to be, very valuable to the participants.

So this week we wanted to focus on four new areas. You can see them here, the front-line staff engagement, the middle manager engagement, the senior leadership engagement and the strategic alignment with organizational priorities. And I think what Marty and I feel is different about this group that builds on the last is this starts to get into more of the culture change, the organization change. So they are a little harder to just set up structures, tell people to do them.

One of the questions last time was, do we have a conceptual model that we are grounding this work in, and we mentioned the organizational transformation model. For those of you who are interested, we are highlighting it here. We have got references at the end of the slide show where you can find a broader discussion. 

But hopefully you will see a consistency between these five elements and the key factors in change that we just showed. So what we feel is important in transforming an organization. And Marty and I worked with others in a private-sector project to develop this model. We have since then tested it in VA and shown that it also applies not only when you think of transforming the whole organization, but also when you are working to implement evidence-based practices.

So first of all, you need an impetus to transform. You need the leadership commitment to quality and to the particular improvement efforts. You need what we have called improvements initiatives, but the idea is really activities that can engage staff in a real problem. And again, this is both to get good solutions to the problems but also to motivate them to do more improvements, show them that it can be successful, that they can make a difference. 

The fourth factor is alignment to achieve consistency of goals, and we will come back to that today in a little more detail about not wanting these improvement initiatives to be sort of floating out on their own.

And then the last one and what is sometimes the most difficult is integrating activities so that you can work across organizational boundaries. And this both ranges from the clinical level, where I think places are already pretty good at case management. But then as you move up to moving across departments, it sometimes becomes more challenging.

So at this point, Marty is going to take over and may have more comments on these marks.

Dr. Martin P. Charns:Right. So I am going to just pick up with a couple comments on the model and then we will go to the next slide in a minute.

So I just want to start by reinforcing the orange box there in the lower right-hand corner that we have labeled “Improvement Initiatives,” to which Carol has referred. And these are the improvement projects . And again, to reinforce the concept here, they are so critical in the cultural transformation because they are the key opportunity for engaging staff. And it is not only for improving particular processes, which of course have value in their own right. There was a phrase that we heard: “you could do 100 projects and still not change the culture.” So yes, those 100 projects are important, but using those projects as a way of getting the staff engaged, motivated and giving them opportunities to develop and apply the skills are the additional, critical parts of using the improvement activities. 

And for the staff working on these specific projects, the projects must be meaningful, concrete and tangible for them, and that is very important in addition to the more abstract statement of the vision for the culture change. 

So going to the next slide, we are going to start with front-line engagement and talk about that middle manager and senior leader engagement.

So having staff identify the projects that should be conducted versus having them coming from the top down is a way of tapping into their motivation and getting their engagement. We saw many examples of this across the improvement capability grant sites. One nice example of this is a facility where a nurse manager had gone for training. When she came back to the unit, she encouraged one of the staff nurses to do a 5-S project around some issues that the staff had identified. 

The project was a huge success on the unit by improving efficiency and engaging other staff in the work. Then the nurse manager was able to, and purposely, allocated time for this staff nurse to work with the Office of Process Improvement in that medical center to expand that project to other units within the medical center. So it really was very motivating as well as having the additional opportunity for skill development of the staff.

Looking ahead a little bit, I do want to note, however, that we have to be careful about too much of a good thing. Everything in organizations, and I know that is a generalization, is probably best in balance. We want to make sure that we balance the locally-initiated projects that are coming from the front line together with organizational priorities, which we will talk about a bit more under the term “alignment” in this presentation.

One organization was so conscious of this that they were particularly working to balance the projects about 50-50. So they were getting both the ideas and getting the staff engaged in work from the front-line staff and at the same time organizing projects that were mapped against the organization’s strategy.

So some challenges around front-line engagement. Again, we will reinforce, as we talked about last week because the finding has just stood out so much for us, that even though the staff may be motivated to engage in particular projects, often, especially early in an effort at developing a culture of improvement, they lack systematic improvement methods. Or the staff has been sent for training without the opportunity to apply that training. And there is where the two pieces, the training and the application to the projects, really address these challenges.

I must again note, as I did last week, that sometimes staff will do improvement projects but not really use improvement tools. And so without the appropriate coaching to help them do that, they are really not learning the skills that can be generalized and extended to other projects. So it is critical that the staff has the skills and has the time for the improvement that is built into their schedules, and also another form of encouragement is having improvement activities as part of performance goals.

We are going to shift now to talk about middle managers. Obviously, middle managers play critical roles in the organization. It may be useful to say our definition of middle managers was everything from the front-line supervisor up through nurse managers, service chiefs and service-line managers if that position was present in the medical center.

Again, there were a number of challenges in getting middle managers engaged. As with others throughout the organization, there was an issue of getting the skills in the improvement methods and making time available. Sometimes they had skepticism and so did front-line staff. They worried about things like “flavor of the month.” 

I think this is the longest-staying initiative that I have seen in VA. But for staff, even if we are trying very similar things under different names, for middle managers, sometimes they get confused: is this a new program? So that does present challenges.

And sometimes we forget middle managers. Many people have heard the importance of engaging the front line, and of course that is critically important. But sometimes we forget about getting the middle managers engaged because of the effort that we are putting in and the focus we are putting in on the front line. So I am not saying, do not attend to the engagement of front-line. But do not forget about middle managers.

Some ways to engage the middle managers. Think about targeting them for redesign training. Certainly, we do not want to leave them out of training activities, but often we do see that happening in our medical centers. There is coaching and support in doing projects, structures and holding them accountable as the culture changes. And this can also be reinforced that it becomes a sign of status for middle managers to lead redesign teams, to coach, to sit in and provide support for teams.

One example of this in our medical centers where we heard about the most buy-in from middle managers and front-line was when the leaders were talking about improvement in ways that conveyed their passion to the front-line and middle managers. So it usually starts with the senior management conveying that passion, but then you want to see it being conveyed also by the middle managers.

One place had started with middle manager engagement in the executive teams for the projects, and over time they transitioned this responsibility to the middle managers. And then they held the middle managers accountable by having them report to the quadrad on the progress of their projects.

So one of the things that you are seeing by this example of the transition of responsibility to the middle managers is that some of the actions that are appropriate is that different actions are appropriate at different points in the maturation of the development of the culture of improvement. So whereas senior managers need to do some things hands-on in the beginning, it is important to get the middle managers involved and taking on those roles as the effort matures.

Finally, I will turn to senior managers and senior management engagement. There is probably nothing in the organizational change literature on implementation more prominent than the importance of leadership support. Part of that is the vision. And if you go back to our model a few slides ago and we think about the impetus for change, the vision statement is something that contributes to that impetus for change—getting people excited about a better way of delivering care then the current state represents. 

It is important for senior leaders to be knowledgeable about improvements and to model the improvement principles. So of course, they need to have the improvement skills themselves. Not necessarily at black belt level, although we have seen some in the quadrad that are black belts, but more of being the role models for the rest of the organization.

Also critical is that they foster the collaboration and establish a psychologically safe environment.

A good example that we have seen in terms of the role modeling in a number of medical centers is where the members of the quadrad attend the training with other staff. They are showing the other staff their commitment. They are showing the other staff the importance of the training. And that, then, it is surprising but we have seen how the stories about the senior leaders were right there. They rolled up their sleeves. They were engaged. That, then, goes throughout the medical center and people know about the commitment of the senior leaders to the improvement activities.

And one thing I would note: we are trying to change culture. When you change culture, one of the things that you find in organizations is that a lot of the cultural change is transmitted by stories. Opportunities for having those stories or creating those messages is an important role of the senior leaders.

Of course, it does have challenges. The first thing, and we touched on this last week also, is when the different members of the quadrad  do not have a shared vision about the role of improvement in the organization. The other thing that we encountered in a number of places among our 30 sites was changes in leadership in the quadrad. In a few cases, those changes were positive for the organization because the incoming member brought with him or her skills and commitment and they were a better member of the team than the previous person in that position. Or in some cases the position had been vacant for so long that it was welcomed to have a new member.

But there are those situations where the constancy of purpose was not shown and where people in the organization knew that the whole quadrad was not behind the effort and this had very negative effects on building the culture and getting people engaged, because it really does transmit throughout the organization.

Some strategies. One is building on the Leading Organization improvement Workshops or LOIWs. Another thing to be done is to have communication campaigns so that people learned, heard of successes in the organization. Often projects or activities are not known. Showing the example of economies that were developed saving money, how that could be reinvested, are very positive ways of contributing to the momentum of change.

One other thing for the quadrad members is to have system redesign peer mentors. So where does a quadrad member turn for support? Sometimes it is the VISN and there are some very good VISN system redesign coordinators who have played that role, the VISN leadership, the VISN director or CMO working with quadrad members. But it is a critical element and often a major challenge to moving forward.

So now we are going to ask you to do another little poll, and this is on how engaged is your senior management team, i.e. the quadrad or the pentad in those few organizations that have five people in the leadership, is engaged in improvement work in your organization? And if you base that on what I have just talked about as examples and think about these three possible responses, we would like you to do the poll, please.

Molly:Thank you, Dr. Charns. While our attendees are filling out the poll question, can you explain real quick what quadrad is?

Dr. Martin P. Charns:I am sorry, Molly. Would I say what a quadrad is?

Molly:Yeah. Somebody has asked that question.

Dr. Martin P. Charns:Oh. What is a quadrad? Okay. I thought that was a national VA term. So it is often used to refer to the four top leaders in a VA medical center: the director, the chief of staff, i.e. chief medical officer, associate director for nursing, and associate director. Sometimes there is a pentad, which typically is an assistant medical center director. But that is the usual composition. Did that sound clear enough?

Molly:It does. We have had about 60 percent of our audience vote, so we will give people some more time to complete that.

Dr. Martin P. Charns:Good. And I am going to turn the phone over to Carol, who is going to read the results and go on from here.

Molly:Great. Well, it looks like the majority of our audience has voted, about 60 percent, so I am going to go ahead and close the poll and share the results.

Dr. Carol VanDeusen Lukas:Okay. So this is a mixed picture, which is from our work not surprising. So a quarter of you say that the members of your quadrad are all knowledgeable and regularly supportive, which is terrific. A little over half said some are knowledgeable and supportive, others are not. And with 20 percent, 21 percent, we have got a ways to go.

So I wanted to talk now about coming back to this notion of—if we can have the screen back, Molly ...

Molly:Yep, one second here. There you go. You should see the popup now.

Dr. Carol VanDeusen Lukas:Great, thanks. Okay. So. We wanted to come back now to this notion of aligning the improvement with the organizational priorities. And this steps aside a little from the conversation. But just the history of quality improvement is often probably great enthusiasm among individual clinicians, individual staff members for making  a quality improvement in their own area. Some organizations have lots of these scattered improvement projects. 

The problems are when these are done in isolation, first if they feel that in order to really solve the problem they are tackling they need to involve people from departments that are not on the original team, they need support from IT, if they are acting on their own, they do not have access to those. They run into problems when they come up against their organizational boundaries.

Another issue is that they do not have the resources they need, and sometimes these resources may just be staff time to work on it because it is not a high priority. And this is not to go against what Marty was saying earlier about a balance between front-line organizationally initiated improvement and top-down improvement; but sometimes the history is people have been trying to tackle bigger issues than just in their local units. And with their isolated teams they have done anything.

And it also comes back to they may reach project success on their pilot unit with their grant funding, but then when the grant funding goes away, when they want to spread it, the whole initiative more or less dies.

So to get around that, it is really important that the improvement structure and the improvement processes be aligned with the organizational priorities. 

And so one aspect of this is really selecting projects to support business and strategic directions. And this can happen at multiple levels of the organization. So in some places, quality improvement system redesign, professionals are included in the strategic planning for the organization. Out of that strategic planning comes some high-level, high-priority improvement projects that people see as really critical to furthering their strategic direction.

The same thinking, the same philosophy applies when individual departments are held accountable for improvement projects. And this is something that I think is becoming increasingly common, and department chairs are held accountable for service line managers, division heads, that they need to have a couple improvement projects going on that are going to further their priorities for their individual departments. And then ideally these will flow up to the larger priorities of the organization.

Second, one of the strategies that some of the medical centers are using is to have formal tools that help them review and evaluate candidates for projects. And again, these would be projects that are crossing division lines or unit lines. Some formal tools that help them rate what impacts a project might have, what resources it will take, and those tools, help them set priorities and strategically select improvement projects. And this, again, is helping them limit those more complex priorities to a limited set.

Another area that is really important is really ensuring accountability. Having redesign teams, improvement teams—have a place where they report in the organizational structure. Whether it is the department leads, whether as in a larger level, is up to the performance improvement committee. Many organizations are now doing this. And the challenge is that not only should the projects be on a dashboard or a reporting system, but when they do report, if they are not doing well, there needs to be a way to hold them accountable and to support them in moving forward.

We have heard stories of projects that sort of go on for many months. Membership in the committee and the improvement group would fall off because they were not making any progress, and there was no one holding their feet to the fire, as opposed to other places that have monthly reports, weekly reports; and if they are not meeting their milestones, then they are asked to come back with an action plan.

The other part of this is allocating resources. So the resources may be, again as I mentioned a few minutes ago, as simple as making sure that staff are freed up from their day-to-work. And in many nursing services they build into their monthly calendar of nursing scheduling opportunities for staff to participate on committees and improvement teams.

Then the last point here is developing those systems for monitoring and sustaining improvement results. As I said a minute ago, too often there is a history of projects meeting their initial goals and then that performance falling off. 

And all this needs to be embedded in an infrastructure that is going to support alignment.

One of the things that we want to highlight—and we are almost done so we can open it up for discussion—is that this is an area that has been maturing since the Improvement Capability Grants were awarded. So there are a lot of other things going on through the Office of System Redesign and other parts of our agency to support this notion of building improvement capability.

So the first is one that we have mentioned already, leading organizational improvement workshops (LOIWs). I know here they were completed in fiscal 12. I should clarify that, that the initial round, which was conducted with every VISN as the nexus of those workshops, those are now completed. But any medical center can request a local LOIW at their facility, fortunately it is at their expense, too, and for that if you are interested, you should contact julie_morgan@vacase. M-O-R-G-A-N. I think she is on Outlook.

The second area that has really grown in the last few years is our Veterans Engineering Resource Centers, the VERCs, and there are four VERCs across the country. Several of them are providing direct support to redesign efforts working with medical centers in their facility and working with leaders at the national level. They also, particularly VA-CASE and Midwest Mountain VERC, are offering extensive training in Lean methods at different levels in the Lean terms of the yellow belt and the green belt and the black belt.

They have also started some targeted middle manager training, which, as we mentioned a while ago, we think is a key ingredient here.

And there is also underway a Lean deployment pilot. This is four or five sites that really want to delve into bringing Lean not just as a couple tools but really as a way of running their organization, and the results of those pilots can be really important.

There is also an improvement culture guidebook that is being pilot tested and a field-based analytics workshop that is being done with the Midwest Mountain VERC in collaboration with the Office of Analytics that is under the Quality Safety and Value group in Central Office.

So I have run through these very quickly. You will see that I have given Ro Hurley’s contact information at the bottom. So if you want more information on this, Ro has agreed to have her name on the slide and I am sure would be happy to talk with you about any of these opportunities.

Then this is not going forward again ... there we go.

Again, I will not dwell on this; but just again to emphasize these pieces that we think are important; and I think that the goal of these efforts is to really get to the point where improvement is fully embedded in the way the medical center does its business, across all levels, across all services. And we are really aiming to have improvement be part of the work, not an extra activity that is put aside when resources are tight and people are busy.

We want to get to the point where these improvement methods are seen not as an extra burden, but as our path to solving the problems that we are all facing today with increasing complexity, increasing numbers of Veterans and limited dollars. And we also want to get to the point in developing improvement capability where if the leadership changes in the organization, we can still continue. 

These are ideals. I think a lot of places look to experiential site visits with ThedaCare and Virginia Mason. Those are health effects models. But it is important to remember that they took a very long time to get there. None of this stuff is easy and none of it happened over night.

So just in closing, here is some contact information. Marty and I have been doing all of the talking. Lauren Babich is our project manager for this and is very knowledgeable about all of this. And again, Ro Hurley has information of the system redesign.

The other information that is on the handout slides that I did not get added to this, and I apologize, is references for the OTM articles that we mentioned earlier. So I would like to now turn it to Molly to open for questions if anyone has them.

Molly:Thanks and thank you very much. We do have several questions that have come in, so we will go ahead and jump right into those. The first question: did you assess the relative complexity of grant objectives as part of your evaluation?

Dr. Carol VanDeusen Lukas:And could you repeat that, Molly? We were switching phones.

Molly:No problem. Did you assess the relative complexity of grant objectives as part of your evaluation?

Dr. Carol VanDeusen Lukas:Definitely qualitatively. This is a qualitative evaluation and we try to summarize with some quantitative numbers. But we do have a sense of the complexity, but it is not statistically assessed.

Dr. Martin P. Charns:There also is an elaboration on this. It is clear that some of the original grant objectives were limited either in the scope of what they were – what they proposed to accomplish or the amount of the organization that it affected. So some might be only in primary care for the grant funding, or only in the mental health service line. And others ranged to literally training everybody in the organization and changing the culture of the whole medical center. So yes, that variation was there in the applications, in the efforts and we are taking that into account as we are trying to understand the spread and the sustainability of the efforts.

Dr. Carol VanDeusen Lukas:The other thing we are focused on understanding, particularly as we get to the end of the grant site visits for 2010 and the followup year visits with 2009 grantees, is how the improvement capability activities fit into, have been influenced by, and influence the larger structure of improvement in the organization. Very often the grants were complimentary to other things going on. And so we are very conscious of that, too.

Molly:Thank you both for those replies. The next question: which was the VISN that did the VISN-wide training in improvement skills?

Dr. Carol VanDeusen Lukas:Several have done that, so ...

Dr. Martin P. Charns:But it is not everybody and every medical center in the VISN.

Dr. Carol VanDeusen Lukas:Right, right. But they affected all ...

Dr. Martin P. Charns:All medical centers.

Dr. Carol VanDeusen Lukas:... so both VISN 12 and VISN 7 did VISN-wide improvement. I think 22 and 17 did also.

Dr. Martin P. Charns:Right. And the focus of the project in 6, which was the fifth grantee, that is at the VISN level, was in primary care. And it included training for a representative group of the primary care in their medical centers.

Molly:Thank you both for those replies. Next question: were there any best practices to publicize and share improvement projects/information to inform all levels of staff at the facility to allow the staff to see what improvement efforts are being pursued and to engage the staff and keep them informed?

Dr. Carol VanDeusen Lukas:A couple if we can interpret that. One way is, were the medical centers sharing their successes? And I think that that is an area that varied, but I think that many medical centers hold quality fairs where they share their local successes. Many have newsletters that go out regularly from the system redesign and/or quality improvement office. Several sites have expanded their morning meeting to include not only the core leaders and the service chiefs, but down to front-line supervisors. And very often have a regular day where improvement projects are reported, and at least one site makes a point of not having the service chief do the reporting, but the front-line team that was involved.

Dr. Martin P. Charns:Right. You could also put into that category having recognition, formal recognition of achievement, which serves multiple purposes including communication about successes. And then when we see leaders who are regularly doing walk-arounds, often that also leads to their transmitting the examples that they see to others in the organization as concrete examples.

Molly:Thank you very much. Can I ask you to refresh the screen real quick?

Dr. Carol VanDeusen Lukas:Um, sorry.

Molly:No problem. Thinking that. [Laughter] For middle manager engagement for those middle managers who had negative experience with improvement, for instance did not meet goals or expectations, was there any messages to reengage them, to support improvement activities within their service?

Dr. Martin P. Charns:Well, I guess a couple things I would say. One is to bring in the relationship part of leading and how the senior managers then interacted with the middle managers to reengage them. And this is the message they had: this is the way we are moving forward and I am counting on you to be an important part of our team in going forward, then reinforcing that by having accountability in terms of performance goals and presentations of results. So it is both the relationship and the more transactional part of the reward system and so on. 

And we have seen places where that turnaround of individuals did happen. And I believe it includes a confrontation of the fact that I understand this did not work well and let us look at how we can do it better.

Dr. Carol VanDeusen Lukas:And I think that it has a number of dimensions. So one is the notion that you have gone ahead with the project, you have gone ahead with the improvement area and it did not have the results you expected. That is a different kind of failure than your team sort of fell apart because it could never get its charge together or never get its effort together.

And so in the former where it did not have the expected results, I think that this is where it comes back to having a psychologically safe environment. We all know or we all talk about, but it is sometimes not carried out, that in order to break new ground or in order to solve problems you have got to take risks. Although by definition a risk is not always going to be successful.

The other part of it, though, is this notion that the team is floundering. And I think in those cases, hopefully it is before it has become a disaster. But I think this comes back to why it is important for there to be the infrastructure around teams so that before the team becomes a disaster—and that is a strong word, I apologize. I am using too strong of a word. But before then, there is a resource within the organization, whether it is the system redesign coordinator or a green belt advisor that the team can really turn to, the middle manager who is trying to lead the team can turn to, to help figure out why things are not going right. 

The last thing that sometimes happens is that in the larger accountability system, it is important to just say, this not working; we should stop; we should rethink. Have we got the right definition of the problem? Have we got the right people on the team?

And in one place that I can think of, one of the reasons that several teams kept floundering, just meeting and meeting and not getting anywhere is because they did not have the personal skills. They did not have the resources to call on to help them.

Dr. Martin P. Charns:Right. We can think of several examples where sites did that reassessment and said, how do we readjust or how do we stop this activity? One thing I would also use this opportunity to say is, it is really important to think about the choice of projects especially early on. And taking on something that is too big or too politically charged or does not have the appropriate leadership for it when the culture has not yet developed and the skills have not yet developed to be able to successfully do that project, it is inviting the failure. So that project selection really has multiple facets to it that ought to be considered, including the timing and where there is the capability to lead the project in what parts of the organization are involved.

Molly:Thank you both for those replies. The next question: everything you said today rings true based on what our group has seen and our evaluation of the PACT rule _____ [01:03:04]. In disseminating your findings and recommendations, what kind of response have you encountered from VA leadership?

Dr. Carol VanDeusen Lukas:We have not presented it to the top-top leadership. We have presented it to system redesign leadership, the system redesign leadership subcommittee and the overall system redesign committee and they have been very receptive. They are working with Ro Hurley and Sharon Quasney and David Voss [PH] to work on action items and thinking about how they can, particularly at the leadership level, dovetail some of the things that we are finding and some of the activities that they want to do as a result with other things going on in the organization. 

Dr. Martin P. Charns:Right. Also say that early on in the evaluation, we saw that there were fragmented efforts at the medical center level in many places where system redesign and patient safety and quality improvement and perhaps even a separate process improvement office were all operating independently. We fed that back in a national report. We also fed that back locally and a number of medical centers reorganized and were responsive to that recommendation.

Dr. Carol VanDeusen Lukas:Of course, that is also One Team, One Dream.

Dr. Martin P. Charns:Yes, that is true. They had the One Team, One Dream Conference. But we will take credit for it. That is okay.

Molly:Thank you. Next question: I liked the earlier point about ideas for process improvement coming from front-line staff, recognizing they have to fit with organizational priorities. Any suggestion for how to get these front-line suggestions for prioritization?

Dr. Martin P. Charns:Well, a couple things. One is the development of an infrastructure on patient care units and the individual services so that there is a participative mechanism that includes both the leaders in the service or unit and the front-line staff in their own prioritization. You start to get into bigger issues around alignment when projects cross internal organizational boundaries. So you need another service to be involved. Some of that, then, could be negotiated informally; but eventually it needs to fit into the organization’s infrastructure around prioritizing and commissioning projects and allocating resources to it. So it really is a multi-faceted approach that we have seen and would recommend.

Dr. Carol VanDeusen Lukas:And I think for those ideas that need that cross-boundary involvement, the couple ways that we have seen are, as I think I mentioned earlier, they have got formal tools for scoring projects in terms of their impact, in terms of the resources needed and in terms of the alignment with priorities. These things help them select those more high-level projects that will need more support. The second way that this is done, not inconsistent but it could be a companion piece, is to have a performance improvement committee that is responsible for handling that process and formally chartering the high-level team.

Dr. Martin P. Charns:Yeah. One thing else that I do not think we have mentioned at all today. It is the involvement of Veterans and families and in a number of sites they have had committees that included Veteran representatives. Typically what we hear when that is done is the staff will say, we thought there were some things that were really important to the Veterans. But when we asked them, it turned out they were not so important, and of course the converse. So sometimes it is difficult to get the involvement of Veterans. But it also is important to do in these activities including in some of the project selection.

Molly:Great. Thank you. Do you have any suggested interventions for environments that are not psychologically safe? Failures may be viewed as performance issues resulting in reprimands.

Dr. Martin P. Charns:Well, we are looking at each other. This is really tough because  the senior leadership has so much influence over the culture including psychological safety that if we do not have their active involvement in trying to change that culture and consistently working at it, I am afraid it is not going to happen.

Dr. Carol VanDeusen Lukas:Yeah. I think that this is something that at least one of the networks that we have worked with pays close attention to the medical center leadership. We have been talking with them about it in terms of support for the improvement capability development. But I think also that is one strategy, and again this is not for those of us in the middle of the organization to try and do. It really has to come from the top in the way that I think leaders are selected and trained and the expectations of them.

Dr. Martin P. Charns:Right, right.

Molly:Thank you both. That was our final question from the audience. Would either of you like to give any concluding comments?

Dr. Carol VanDeusen Lukas:Just that we did not directly answer that last question. That is a tough one.

Dr. Martin P. Charns:Yeah, and I have to say just like we cannot cure every disease, on the organizational level are some problems that are intractable; and this one really does depend on having the right leadership. I think one overview comment is that we really have seen a lot of progress among our 30 grantees. There are some that have really taken off, others that have moved at a slower pace. There has been some good sharing across sites. Some of the learnings have spread from the individual grantees to other medical centers, typically in their own VISN. We are really pleased to have seen that and are appreciative of the opportunity and the openness from the staff as we have come in and asked questions of them every six months to try to track this work in a real-time way.

Dr. Carol VanDeusen Lukas:And we want to thank those of you who have stuck with the presentation and joined from the beginning.

Dr. Martin P. Charns:Yeah, and we are really quite open and would love to have correspondence around this both through the system redesign, the share point site, email with us—you have our contact information. And we could even go to the old-fashioned way of a phone call. So thank you.

Molly:Thank you both for sharing your expertise with the field and thank you for our attendees who joined us. And to exit today’s session, please just wait one moment while the feedback survey populates on your screen. Thank you all for joining us and have a great rest of the day.

Page 1 of 17

Page 17 of 17