**Transcript of Cyberseminar**

**Patient Aligned Care Teams (PACT) Demonstration Labs**

**Provider and Staff Experience with PACT: Results and Recommendations from National and Regional Primary Care Surveys**

**Presenters: Christian Helfrich, PhD; Sandra Joos, PhD; Lisa Meredith, PhD**

**February 20, 2013**

**Provider and Staff Experience with PACT: Results and Recommendations from National and Regional Primary Care Surveys**

**February 20, 2013**

Dr. Helfrich:Hello everybody. My name is Christian and today’s presentation is actually part of two sets of presentations on findings on the PACT initiative from primary care personnel surveys. The first presentation we did last month on January 16th was focused on measures of PACT implementation and changes in primary care. My co-presenter at that time was Michele Lempa from the VISN 4 PACT Demonstration Lab. Today we are going to be talking about provider and staff experience with PACT and specifically about burnout and some of the factors that might be contributing to burnout.

Just real quickly before starting I wanted to ask a quick poll question and get some background on our audience and their experience with PACT. 

Mollie:Thank you Dr. Helfrich. So attendees you will see on your screen a poll question at this time. The question is, do you have any involvement in PACT. The answers are no and virtually know nothing about it; no but have some knowledge about the initiative; yes I am involved in researching or evaluating it; yes I am clinician, staff member or administrator involved in implementing PACT; or, yes I am involved in PACT in another capacity. So it looks the answers are streaming in. We have had about two-thirds of our audience respond so far so we will give people a few more seconds to get their answers in. We thank you for your participation as it does help the presenters gear the presentation more towards their audience. Okay, it looks like we have reached about 80 percent response rate so I am going to go ahead, close the poll, and share the results now. Dr. Helfrich, do you see those results?

Dr. Helfrich:Actually, I am not seeing the results.

Mollie:Okay, just one second. Now if you press okay you should be able to see them.

Dr. Helfrich:Oh yes, sure. Fantastic. Okay so it looks like almost half of folks, 46 percent are involved in implementing PACT as clinicians, staff, and administrators. It looks like another 18 percent are involved in researching it and 20 percent know about the initiative but are not involved and 14 percent involved in some other capacity. Just 2 percent do not know much about it. Can I close this window now? Okay, great. There we go.

So I am going to give just a very brief bit of background but given the vast majority of folks do know about the initiative I am going to skim over this fairly quickly. The Patient Aligned Care Team initiative is VHA’s patient-centered medical home model. It was formally launched in April of 2010 and it includes a variety of components although perhaps the central one is a move to team-based care with the formation of what have been termed teamlets with primary care providers, nurse care managers and...folks if you could mute your lines there is a little bit of background noise. Thank you. So primary care teamlets composed of primary care providers, nurse care managers, clinical associate and clerical associates. The idea is that they share responsibility for the defined panel of patients that the primary care provider cares for. 

There is also a move to alternative visits; electronic and telephone visits and the use of the nurse care managers and additional health promotion to support staff to do care management, proactive care management and patient support.

There were resources that were devoted to PACT implementation nationally. There was funding to support the expanded staffing model sent out by the VISNs. There was training, most notably regional learning collaboratives that were designed with the VA Office of Systems Redesign. Then there were finally PACT demonstration laboratories in VISNs 4, 11, 20, 22 and 23 that focus on certain elements of PACT and do more in depth formative evaluation of the PACT initiative. So studying the initiative as it rolls out, feeding those findings back to the clinical leadership teams to try to better understand how to make this model work. 

Then there was the formation of the National Demo Lab Coordinating Center and that is whom I work for. And the Coordinating Center is working with the Demonstration Labs to standardize approaches where it makes sense. So for example the way that certain factors are measured such as burnout. Also, the National Demo Lab Coordinating Center was tasked with conducting an evaluation of the implementation of the PACT initiative and the outcomes.

The part of the presentation that I am going to do today is from one piece of this national evaluation. That is the results of a primary care personnel survey that we fielded this past summer with the Healthcare Analysis and Information Group within the VA. This work again was part of my role with the Demonstration Lab Coordinating Center and a working group that we call the Organizational Function Working Group. It comprises members of the Demo Labs including my co-presenters today, external consultants who study and work on patient-centered medical home initiatives outside of the VA and also some organizational change in education evaluation experts within the VA. 

This survey again was part of the national evaluation. It was fielded in May of 2012 and the goal was to help, not completely answer, but help answer three broad questions that the evaluation was trying to address. And that is, overtime, to what extent has PACT been implemented? What is the progress on the implementation of this initiative? What are the factors that foster or hinder PACT implementation, including which resources seem to be associated with greater implementation of PACT? And then of course we are very interested in knowing what the effect of PACT is overall. To what extent does it achieve the outcomes we are interested in including our employees; the employee experience, most notably burnout and job satisfaction.  

One of the reasons that is of critical importance to us is that primary care staff, we know from the literature that there is a growing level of burnout and dissatisfaction in primary care and that the roles of primary care providers have been declining. There are fewer medical students who are going into family medicine and primary care residencies at the same time that we see growing demand for primary care. One of the more intriguing early findings from patient-centered medical home literature was a demonstration project at the group health cooperative which is here in Washington state where I am situated. They found when they did their demonstration project, they were experiencing very high rates of burnout; approximately 30 percent in their primary care providers. A year into their patient-centered medical home initiative they saw that burnout rate at the demonstration site had dropped to 10 percent while it had remained at about 30 percent at the control clinics. So that has been an intriguing finding and something that we want to monitor carefully. Not least because of organization change often itself is stressful. So launching a big initiative can itself introduce stress.

We completed a web-based survey, this again was with the help of the Healthcare Analysis and Information Group. It was fielded via email so an email link sent out through the Office of the Deputy Undersecretary for Management and Operations. It went through the clinical leadership in primary care, nursing, pharmacy, social work and nutrition. We gathered a total of 6,476 respondents, 5,400 of which self-reported as being in the occupations of the comprised teamlets. So primary care providers; that is M.D.s, nurse practitioners, physician assistants, nurse care managers, clinical associates such has licensed practical nurses or medical technicians and then clerical associates. We do not have, because of the way we fielded the survey, we do not have a firm denominator. We do not know the exact number of primary care personnel that should have received the survey. Based on estimates looking at response rates from the All Employee Survey and numbers of respondents we believe that we achieved approximately a 30% response rate.

What I will do is describe some respondent demographics and then I would like to launch into the burnout, the rates of burnout that we saw among our respondents and then talk about some of the factors that appear to be associated with the burnout rates. Overall, almost half of the respondents of this survey were 50 years or older. Over half of them had been in the VA for greater than five years; five years or more. One-third of them had been with the VA for over ten years. Almost two-thirds had no supervisory responsibility and again 78 percent of them were in one of the four PACT teamlet occupations; PCP, RN, care management clinical associate or administration associate.

The table here shows three different sets of burnout rates. The left hand column are the burnout rates for primary care personnel from our 2012 PACT survey. It is based on a measure taken from the physician work life study. It is a single item measure of five categories and it has the respondent defined burnouts for themselves. It goes from the lowest category, not experiences any burnouts to the highest category, experiencing burnout and not sure if I can go on. Three or higher is defined as showing signs of burnout and these are the proportions of respondents are scoring three or higher so indicating some signs of burnout.

I’m sorry, did somebody have a question? Again, there are some.

Mollie:I took care of that.

Dr. Helfrich: Okay, great. Thank you.

So the left hand column are the burnout rates from our survey in the red. The blue column are from the All Employee Survey which also included the single item burnout measure and was fielded just prior to our survey. The blue column has the burnout rates from the All Employee Survey just for primary care personnel. The black, the right hand columns in the black text, those are the burnout rates for all of VA. The first row you can see are providers. The next row is nurse care managers. The nurse care managers were not identified expressly distinctly in the All Employee Survey. Clinical associates, administrative associates and then nurses and other RNs other than care manager roles are on the second to last line and then the total burnout rate on the last line. So what you can see in our survey is almost exactly one-third of respondents, 36.6 percent, reported signs of burnout.

In the All Employee Survey, primary care personnel that responded to the All Employee Survey, it was slightly lower at 31 percent. That compares to 28.1 percent for VA overall. That is all respondents to the VA All Employee Survey. Essentially, across the board, the burnout rates that we saw to respondents to our survey were slightly higher than among primary care personnel in the All Employee Survey. So 45 percent, that top line, 45 percent of providers in our survey reporting signs of burnout compared to 40 percent of primary care providers in the All Employee Survey. We saw again almost 40 percent of nurse care managers; nurse care managers were not expressly identified in the All Employee Survey. We say 30 percent of clinical associates and 37 percent of administrative associates reporting signs of burnout. Again, higher than in the All Employee Survey. 

So overall, we saw a higher rate of burnout in respondents to our survey than among primary care respondents to the All Employee Survey suggesting that we did get more, that our respondents, we probably had a response bias where folks experiencing burnout were more likely to respond to our survey. However, that being said, we see very high rates of burnout. So virtually one out of three respondents to both of these surveys is reporting signs of burnout.

Now when we look at how those individuals who report signs of burnout compare to those who do not, and these are figures that are restricted just to the four occupations that comprise teamlets; primary care provider, nurse care manager, clinical associate or administrative associate. Here we see the average number of hours they report spending on a typical day in these four different activities. Teamlet huddles, which are about planning for patient care on a given day, face to face with patients, on the telephone with patients and electronic communication with patients. We see that respondents who report no signs of burnout they report spending about 0.15 hours more a day in teamlet huddles and little bit less, about 0.3 hours a day less in face to face care with patients. Likewise, just a little bit less in those reporting signs of burnout in telephone contact with patients.

When we look at that last line this is the percentage reporting being on a teamlet that is staffed to the recommended 3:0 ratio meaning three fulltime equivalent support staff for each fulltime equivalent primary care provider. Perhaps not surprisingly, 57 percent of those who do not report signs of burnout say that they are on a teamlet that is staffed to ratio, whereas 38 percent of those showing signs of burnout reporting signs of burnout say that they are on a teamlet staffed to the recommended ratio.  

We also asked a series of questions to try to get at how much of their time they spend on work that is well suited to their training or what we have called, Gordon Schectman I think coined the term, working to the top of competency. I will just draw attention to the last line. We asked what proportion of their time they spent on work that could be done by someone else with less training, with what proportion of time they spent well matched to their training and what proportion of the time they spent on work for which they had too-little training. That is that last row right here at the top of the table. Those not showing signs of burnout 45 percent of them said that they spent less than a quarter of their time, that was the lowest category, less than a quarter of their time on work for which they had too little training. That was as compared to 30 percent, 29.8 percent, for those who report signs of burnout.

We also asked a serious of questions about team functioning, about communication around difficult topics or conflict. Participatory decision making on their team, how stressful or chaotic the teamwork environment was and whether or not they felt that their team had a history of, their clinic had a history of successfully making changes together. We actually saw really no difference between those showing signs of burnout and those not reporting signs of burnout, so no difference in communication and decision-making. No difference in how stressful or chaotic they reported the work environment, with a very small difference in...

Mollie:Christian?

Dr. Helfrich: Yes.

Mollie:I am sorry to interrupt. Can I ask you to speak up a little bit?

Dr. Helfrich:Oh, absolutely. Thank you.

Mollie:Thank you.

Dr. Helfrich:  Is that better?

Mollie:Much better. Thanks.

Dr. Helfrich:Okay, great.

...and a very small difference in the score on history of change. These questions were scored on a like-it scale of 1 to 5.  

We also asked a series of questions about to what extent primary care providers were relying on their teamlets to complete a series of clinical tasks. This is the proportion of respondents who indicated, for primary care providers, it was the proportion who said that they relied on their teamlet a great deal to complete these tasks. For the teamlet members that are in care managers, clinical associates and administrative associates it was the percentage who said that they were relied on a great deal to complete these tasks. You can see where there are small differences. Where there are differences are generally in activities around encouraging lifestyle modifications, educating patients, assessing lifestyle factors, gathering preventive services history, education about medications. It is about supporting the patient. It is about care management supporting the patient.

Because of the time constraints and I want to make sure Sandy and Lisa have enough time to present, we also have these figures broken down for PCPs and nurse care managers. Those tables are in the slides that are available with this talk. I would be happy to answer questions for folks about those. I am just going to skip ahead to the conclusions. One thing with these findings, some important limitations, as I noted with the denominator, we do not have a true denominator for this. We do not know who actually had an opportunity to complete the survey and there is most definitely a danger of response bias. It is not possible to link these observations to specific teamlets, only at the clinic level. Because to that, that limits our ability to triangulate these findings at the team level with other data and again just makes it a little more difficult for us to triangulate and be confident with conclusions about relationships among variables such as staffing teamlets, staffing and burnout.  

The final thing, and possibly most importantly, these are cross-sectional associations and they are also gathered from the same instrument. So we asked people about burnout and we asked people about delegation of activities and their staffing level in the same survey. One of the dangers is something called method bias. Where the mood that the person is in at the time they fill out the survey or the way the survey is framed might introduce correlations among the items that have nothing to do with staffing or burnout, but have everything to do with the state of mind the person was in at the time they filled out that survey. So those are important limitations to keep in mind when considering our findings.  

That being said, what we find is relatively high levels of burnout, 36 percent, so a third of primary care respondents to our survey reported signs of burnout. That is slightly higher than data available from the All Employee Survey but not much higher, so high levels of burnout. We see the PACT staffing the PACT team that is being staffed to the 3:0 ratio in slides that I did not get to but we saw some evidence that the training support for those that found the training support helpful that that was negatively associated with burnout. We see that delegation of clinical responsibilities is negatively associated with burnout for PCPs that, again I apologize for the slides today that I was not able to get to, but for nurses the relationship with delegation of clinical responsibilities depends on the activity for the nurses. For nurses when delegation is related to activities related to patient education, care management, that is associated with lower burnout rates for nurses. Team functioning which surprised me, we did not see an association with burnout. Although, my colleague, Sandy, is going to show some data that comes to a different conclusion. 

Again, just a final caution, these results should be viewed as descriptive and highly preliminary at this point. With that I am going to hand it over.

Mollie:  Thank you very much. I am going to turn it over to you now Sandy. Just click the popup Show My Screen. Great. We can see your screen if you just want to go up into slideshow mode. You can do that in the bottom right hand corner if you want, the bottom right hand corner, just over to the left a little bit, the third one to the right. There you go. Thank you. Go ahead Sandy.

Dr. Joos:Can you hear me now. Okay. So I am going to talk about what we saw with our employee survey, specifically with regard to staff experiences and associations with burnout because that is the theme today. I want to acknowledge my colleagues with whom I have worked on the employee survey project. We opted out of the national survey because we felt like we could get a better response rate and because we also wanted to do some...Am I okay Mollie?

Mollie:Yes.

Dr. Joos:Okay. We wanted to do some modifications so that the content made better sense locally. We distributed the surveys during an on sight visit and we used mail and email reminders. The first round of the survey distribution and collection was between August 2010 and May 2012. We are in the midst of the second round of surveying but I am only going to focus on the first round today. 

The clinics that were in this first round were very diverse in terms of their size, their location, their distance from and access to both VA and community healthcare facilities. Our surveys were anonymous. Our overall response rate was 72 percent but that ranged in the individual clinics from 58 percent to 100 percent.

This slide summarizes respondent characteristics by their role in the clinic. I am going to go through this pretty quickly because it is similar to what Christian showed for the national surveys. But, it is divided up by roles in the clinic for the first four rows are the teamlet roles; the PCPs, the nurse care managers, the clinical associates and the clerical associates. Then the other clinical staff in the clinic which would be like social workers, pharmacists, mental health providers and then other clerical or administrative people in the clinic. We had a total number of surveys returned was 226 and in this column you can see that the response rates were high for all groups except for clerical associates, particularly high for nurse care managers but particular low for clerical associates, unfortunately. Respondents were mostly female. About a third of the respondents had been with the VA for less than three years. The only exception was within the other clinical staff, social workers, pharmacists, mental health personnel. Overall, a little more than a third of employees have been with the VA for more than eleven years. 

Finally, this last column here, the majority of respondents were full time in a primary care clinic, about 90 percent of respondents were full time in a primary care clinic but many of these respondents said that they worked more than 40 hours a week. That perhaps is not surprising for primary care providers but it is somewhat unexpected for the people that are not in a primary care provider role.

So with that background, the remainder of the presentation will focus on the 193 respondents that are in the teamlet roles. In this top row we can see that almost all of the primary care providers, the nurse care managers and the clinical associates said that they were assigned to a teamlet. But only about three-quarters of the clerical associates said they were assigned to a teamlet and a number of them said that they did not know if they were assigned to a teamlet. So I do not know if this may suggest that those in the clerical roles were not as well integrated into the teamlets at the time we did the survey. 

We also asked if their teamlets were staffed to the recommended ratio and a small proportion of primary care providers and nurse care managers said that their teamlets were staffed to the recommended ratio as compared to clinical and clerical associates. 

The amount of time that respondents said that they spent on various activities in a typical day differed by role. In this slide I presented the median amounts of time rather than means but the patterns are the same for median and means. So the median response across all roles was to spend about a half an hour or less on teamlet huddles. The difference is the cross-roles were not significant.

Looking at the three activities that involve interactions with patients, the face to face with patients, telephone with patients and electronic communications with patients, compared to those in other roles the nurse care managers reported spending significantly less time in face to face interaction with patients. The median number of hours they reported being 1.25 versus 4.6 in the other roles. The nurse care managers also reported spending more time in telephone interactions with patients; a median of 4.8 hours versus 1 to 3 hours in the other roles. There was relatively little time spent by any team members in electronic communications with patients, that is this row here. PCPs and nurse care managers reported spending more time in patient care activities that do not involve direct communication with the patients. So I am calling that electronic not with patients but what that consists of is electronic review of charts and entering notes and orders and responding to electronic alerts and those sorts of things. Again, PCPs and nurse care managers said they spent much more time doing those types of activities than clinical and clerical associates.

Finally, this bottom row is, the question was, what percent of your time do you spend doing work that is well matched to your training. What we saw was that almost all groups said that they spent most of their time doing work that was well matched to their training. The exception was with nurse care managers who actually spent a relatively very smaller proportion of their time doing what they said doing work well matched to their training. That was a pretty dramatic difference.

Okay, I am going to turn factors associated with burnout and job satisfaction among teamlet members. As Christian already has emphasized, these are associations we cannot defer causation. I am going to go through this slide pretty quickly because Christian already described the single item level of burnout so I am not going to belabor that. I will just say that we had a single item level of burnout. We also had a nine item Maslach burnout inventory. The single item measure corresponds very, very closely to the longer Maslach inventory, especially with the emotional exhaustion subscale and the same association was seen with both the single item and the Maslach measure. 

So this slide shows the percent who endorse each level of burnout by role. There are really no significant differences, unlike what they found in the national survey, by role in terms of the proportion of people in each role who report burnout. Although, there was a tendency for nurse care managers to report somewhat higher levels of burnout especially at the level of 4 and 5 and for clinical associates to report somewhat lower levels of burnout.

This slide shows the proportions of employees by roles who were somewhat or very satisfied with various aspects of their job or their work environment. There was a tendency for fewer nurse care managers and more clinical associates to be report being satisfied but there were few statistically significant differences between the roles and the various aspects of job satisfaction. Basically across all the roles employees were most satisfied with the type of work that they did, their relationships with coworkers and their jobs overall. That is where we saw the highest proportion saying they were somewhat or very satisfied. Then, lower proportions who said that they were satisfied with the amount of work, working conditions or opportunities for promotion. 

Also, compared to what their satisfaction level was two years ago, that was one of the questions asked about that so this is just people who had been with the VA for at least two years, 38 to 49 percent reported a somewhat or much less overall level of satisfaction with their jobs. So they are saying that their satisfaction with their jobs have declined over the past two years. Then, in this last row, 28 to 41 percent agreed with the statement, if I were able I would leave my current job because I am dissatisfied. Again, there were not statistically significant differences in this.

This slide shows associations between several aspects of job satisfaction and symptoms of burnout. I am going to go through this pretty quickly. It is just basically we show that symptoms of burnout are much more likely to be, I am sorry let me...that those who do not have symptoms of burnout are much more likely to be satisfied with all the various aspects of the job and work environment compared to those who do have symptoms of burnout. So those who have burnout are less likely to be satisfied with their type of work, amount of work, working conditions, etcetera than those who do not report symptoms of burnout. Those were pretty strongly statistically significant. 

In the remainder of slides, I am just going to show associations with burnout but I just wanted to say that the pattern seen with job satisfaction are the same or sometimes even stronger than those seen with burnout in terms of their associations with other variables that we looked at. So this slide looks at the associations between time spent on daily activities staffing to the recommended ratio and burnout. In this slide I presented means of standard deviations but again the findings are the same if you use medians.

Those who reported symptoms of burnout said they spent less time in huddles and more time on all other activities but the differences were only significant for electronic communication with patients and patient care that does not involve direct communication with the patient. Among those who did not have symptoms of burnout a greater proportion reported that their teamlets were staffed with recommended ratio but unlike in the national survey this was not statistically significant out of 0.05 level although the trend was in the same direction.

Then finally, the 62 percent of those with no burnout said they spend at least three-quarters of their time doing work that is well matched to their training versus only 47 percent of those with symptoms of burnout.

This slide shows the association between perceptions of barriers to delivering patient-centered care and burnout. Those with symptoms of burnout were significantly more likely to report that these factors were ones that limited their ability to deliver patient-centered are a great deal. So again, the percentages are the proportion of people who said that each of these factors limited their ability to present patient-centered care by a great deal. They were all statistically significant except this bottom one and I will explain that in just a minute. But we did see that the extent to which respondents said that these factors limited their ability to provide care differed somewhat by role. So nurse care managers were much more likely to say that lack of support from clinical leadership and lack of my teams responsiveness to my requests for assistance were factors that limited their ability to deliver care a great deal. Whereas primary care providers were more likely to cite lack of control of their schedule and patient panels were too large. These differences were significant for primary care providers. The patient panel size was a significant factor for primary care providers. Then, there were not for the other two items, the time per patient counseling or education and inadequate time for follow-up care really did not differ across roles.

This slide shows perceptions regarding PACT varied by level of burnout. Again, except for saying whether or not they understand PACT very well, there were significant differences with those being more burned out; less likely to be confident that they could implement PACT; that their team could implement PACT; that PACT would improve care; and that they had seen any improvements in care since they first became aware of PACT.  

Finally, perceptions of team functioning in the work place. Again, as Christian eluded to, they did not see a difference in these measures in the national survey but we did see differences in these measures in our survey in our respondents. Again, on just about everything single measure that has to do with team functioning and work place environments and people who were burned out perceived those conditions more negatively than people who did not have any symptoms of burn out. 

So just quickly summarizing, the symptoms of burnout were also present in a relatively high proportion of employees in our survey. There was more dissatisfaction with the amount of work and work conditions than with the type of work, relationships with coworkers or the job overall. Burnout, jog satisfaction are strongly associated and burnout and satisfaction were strongly associated with perception of these barriers to providing patient care, perceptions of PACT itself and perceptions of team functioning and the workplace environment. 

We cannot draw any firm conclusions about causation from one survey but these findings do suggest areas for further exploration and I think we can expect to gain more insight with repeated surveys. So that is the end of my section here.

Mollie:Thank you very much. At this time I am going to turn it over to Dr. Meredith.

Dr. Meredith:Okay. Thank you very much. Can you hear me okay?

Mollie:If you could pick up your handset or speak a little louder that would be helpful.

Dr. Meredith:Okay, let me first get my slides pulled up and I apologize to coming to the party a little late. I had put this down on my calendar at the wrong time. Okay. Can you see my slides?

Mollie:Not yet. There should be a button for you to click that says Show My Screen.

Dr. Meredith:Okay.

Mollie:There you go. Now we just need you to go into your slides.

Dr. Meredith:Okay. I am going to take you to speaker. Does that work better for you?

Mollie:Much better, thank you.

Dr. Meredith: Okay. Excellent. So I am going to turn to focusing on the survey that we administered for VISN 22. So like Sandy, let see, the slides are not advancing. 

Mollie:Just click under the arrow.

Dr. Meredith:There we go. Okay. So we are moving from the national data, again to the focus on VISN 22 findings. I first want to point out that the VAIL evaluation which stands for Veteran’s Assessment and Improvement Laboratory is the PACT Demo Lab there and there are a lot of components like with many of the Demo Labs. I am just going to be focusing on the clinician and staff survey components. A lot of people on the VAIL team contributed to this survey. I just want to make sure I acknowledge everybody who is involved. Just a little bit of background on our approach.

So the main purpose obviously of these surveys is to track changes over time and add it to the experiences of the clinicians and staff who work with primary care clinicians with regard to implementing PACT and VAIL which is kind of an enhanced PACT version. So it is important to understand all of this because we will learn how we can aid the design of better care processes and identify best practices. 

So we fielded our survey a little later than the national survey and also I guess Sandy’s VISN 20 survey. We did the first wave in the winter of 2011 to 2012 and we plan to field a second survey about 18 months after the first wave. So we are going to try to get that going this late spring, early summer. Like VISN 20, we also opted out of the national survey. We have a lot of the same items that overlap across these surveys so you will see some redundancy as well as confirmation or validation of similar patterns here.

We are ultimately planning to look at pre-post changes over time and kind of the differences in differences. Analysis for VISN 22 where we have some sites in sort of regular PACT and we have some sites that are implementing a more enhanced PACT. So we will be comparing those two types of interventions over time. Today I am obviously reporting on just the first wave surveys.

This slide gives you a sense of our response rates which were actually quite good.

Mollie:Dr. Meredith.

Dr. Meredith:Yes.

Mollie:I am sorry to interrupt. You slides actually are not advancing. Can you click on the actual slides? Click anywhere on it or you can use the arrows in the lower left hand corner.

Dr. Meredith:I was using the arrows on the left and they are changing for me. I should be on slide six. What slide are you seeing.

Mollie:We are still seeing the original one. So I am going to take control real quick and then I will turn it back over to you and hopefully this will help. You should see the popup that says, Show My Screen again. Go ahead and click on that box that says Show My Screen. Okay. Now just go back up into Slideshow Mode. 

Dr. Meredith:There we go. Okay. So are you seeing slide six at the moment?

Mollie:Actually, you are not up in Slideshow mode yet so we are not seeing that. I will go ahead and advance through your slides for you.

Dr. Meredith:Do you want to go back to the front. I have already briefed the first five. Just so people can see them.

Mollie:Yes. No problem. 

Dr. Meredith:I wonder why it is not working.

Mollie:Just one second here. Thank you for your patience. Okay. I will go ahead and advance your slides for you. So you should be able to see your slides now.

Dr. Meredith:Yes. So are you on slide six?

Mollie:I am.

Dr. Meredith:So should we just start from here?

Mollie:Yes. I think we can do that because everybody has a copy.

Dr. Meredith:Okay. So this, actually we should be on slide five. I am sorry. We are not quite at slide six. Could you go back one please?

Mollie:Yes. We are all set.

Dr. Meredith:See I am showing slide six. That is strange. Okay. I guess I am not connected to you so I have to advance my own to stay with you. 

Okay. Looking at slide five, this is the response rates by the five different healthcare systems that are part of this study. You can see that we have fairly good response rates with the highest rates being at greater L.A. and also Long Beach but even the lowest rate was 55 percent which is not bad for a provider survey.

The next slide. You know I cannot tell, I cannot see what you are advancing.

Mollie:It is okay. As soon as you say next slide we are on the next one. 

Dr. Meredith:Then I will just follow you with my paper version here. Okay. So this also shows the response rates for the non-demonstration sites relative to the phase one roll out of three of the sites that are participating in VAIL or the enhanced pack and then more recently the sites that rolled in in the second and final phase of VAIL. So phase two really was happening simultaneously with the survey so we did not expect much difference. But we did get a slightly higher response rate among those three.

Next slide. These demographics are the sample of respondents. We have 515 providers or staff employees who responded. You can see, well first I should say, our missing data is less than 10 percent overall here. While the primary care clinician sample is slightly older and has longer tenure at the current clinic, there is a lot more racial diversity in the staff sample. The physicians are predominantly male whereas nurses are predominantly female. That is probably no big surprise. I do not have a slide for this but relative to the national survey and also to Sandy’s VISN 20 also, I should not that at that later point in time nearly 100 percent of the respondents told us that they were part of a teamlet so we did not break anything out in that way.

Next slide. Since we are focusing on burnout I thought I would show you this comic that relates to PACT very much in that it suggests that this employee would probably rather work alone than on a team. It is possible to imagine that somebody like that might be suffering from some symptoms of burnout.

Moving on to talking a little bit about burnout, I know that the previous presenters have already briefed this but basically as Christian mentioned this recent article by Shanafelt and colleagues in the Archives of Internal Medicine has found that burnout is much more common among physicians than among other types of U.S. workers. Physicians who are in the frontline of care tend to be at the greatest risk. There is a good amount of evidence in the previous literature that links burnout to a number of different adverse outcomes and that includes: Poor mental and physical health as well as reduced performance at work, lower organizational commitment and even intentions to quit. That is a little background.

We can go on to the next slide here. I will just go a little further and say that you can imagine that being not satisfied with your job would be linked to burnout. We have seen that in the other data. Not being in a supportive environment that allows people to learn and innovate can sometimes contribute to burnout. Lack of time and not feeling respected, poor communication, encounters with patients that are more difficult to treat like high utilizers or the complainers have also been shown in the literature to lead to provider burnout. So that is just a little background.

We can go to the next slide. This is just an overview of the different components of burnout. To point out what items we had in our VISN 22 scale we did create an overall scale, though that is not always recommended in the literature to use but it had a very high alpha. We also created three subscales where we have all nine of the Maslach emotional exhaustion items and then we have three of the cynicism items and we have three of the professional efficacy items. The items in italic I believe are the items that are similarly on the national survey.

We can go to the next slide. I am finally going to show you some data. This slide shows, first of all these descriptive slides are not adjusted. Though we did create weights that we have used for our multivariate analyses. I will explain that later. So these are just raw distributions of average burnout scores based on the overall burnout scale. You can see that there is really no difference for the VAIL versus the non-VAIL or the plain PACT sites here but we do see a trend for differences in burnout by job type with primary care clinicians in the middle set of bars being slightly more burned out. 

We can go to the next slide now. This shows that burnout does differ by job type with pharmacists having the highest levels of burnout followed by primary care physicians. Then physician’s assistants and nurse practitioners and also registered nurses. Dieticians and social workers tend to be much lower as well as the health med-techs.  

So the next slide. I am going to show you a series here of slides for the burnout scale using the Maslach cutoffs that are in the literature for low, medium and high. Here you can see that the green portion is the normed data for folks that scored in the low burnout range. The orange is medium and the high is the red. You can see that there is some variation here with PAs and NPs as well as MDs and pharmacists having more people in the medium to high burnout range.

So we will go to the next slide. This is a similar pattern for the cynicism subscale. And again we sees some higher rates of burnout among the dieticians and nutritionists when we use the cutoffs.  

Next slide. So this is also consistent with the previous slides. The low burnout category on professional efficacy which was the burnout scale here tends to be much higher for the mental health workers. We saw that earlier. So just giving you an idea of where people fallout in terms of those cutoffs.

Then I will show you I think just two more in this pattern. The next one is showing burnout with the cut points by the medical center. Here you can see that San Diego and southern Nevada tend to report higher burnout rather than the other sites. 

Then the last slide of this family, this is comparing VISN 22 data from our survey with the national sample in Shanafelt. We have broken it down by our sample of primary care clinicians. So in this case it does have the PAs and NPs included in there. Then the lower bar is for other staff. Interestingly our sample, even though it is not really comparing apples to apples because the national sample of Shanafelt had all different types of physicians including specialists and surgeons whereas we have only primary care. But the pattern is not that different, maybe a less burnout than the national sample. The other non-primary care provider sample is even less burned out than the primary care provider samples.

Okay, so how am I on time. I am going to go very quickly here. This is just an overview of some of the organizational functioning measures that we are looking at in relationship to burnout. So I will let you all just kind of take a look at that. We have combined a few of them for our purposes because of the high correlation so we have combined the psychological safety and the openness to new ideas scales. Then we combined both sets of items for communication into one scale to make the analysis a little more parsimonious.  

So we can go to the next slide. This shows correlations that are based on waiting on nonresponse and we used this nonresponse or we created them stratified by site and job type. So the weighted data are not particularly different from the unweighted data I should note. But the main point here is that all of the correlations are significantly, all of these factors, satisfaction, organizational, functioning measures, are highly correlated with burnout. Whether it is overall or one of the subscales at p<0.01.

I should note that we also ran this separately for primary care clinicians and staff and the patterns were nearly identical. So this is just more data to support what we have already heard from Christian and Sandy.

So the next slide is a summary of our multivariable regression models. I focused on the emotional exhaustion scale here which is the strongest scale but conceptually in terms of representing burnout and also empirically because it is a nine item scale. So these weighted data were entered in four sequential steps. So we first looked at the contribution of demographic variables to emotional exhaustion and you can see that being male and being Latino and also being a non-primary care provider, all of those factors are associated with lower burnouts. Those factors hold pretty much across the board even after we control for the addition of clinic characteristics, which is in the second model, the second column of numbers. When we enter the clinic characteristics in the model, there is really a very small effect that smaller sized clinics have providers that are less burned out. That affect is attenuated once we control for other perceptions and experiences of the number of providers. Then when we add it in the organizational climate variables, and these are various perceptions about organizational functioning, leadership, communication and some of those things I showed you in the previous chart. There we see there is not much happening. There is a small affect for organizational communication but that is a pretty low magnitude. That drops out once we add in individual experiences. There, as you might expect because of how highly correlated it is, job satisfaction has the biggest affect in terms of being more satisfied as associated with lower burnout.

The only other thing I wanted to point out is that these models explain a pretty decent amount of variation with the third model explaining 27 percent which is a pretty big jump from 9 percent when we had only demographics and clinic characteristics. When we add everything else in, it explains 40 percent.

So I have kind of run out of time due to technical difficulties but the next slide is basically just more data showing how closely linked burnout is to perceived barriers. You can just kind of take a look quickly at that busy slide but the pattern is very consistent.

Going to the next slide, we can see that burnout scores among respondents who said that they were more exposed and found these different PACT activities helpful were also less likely to be burned out. Obviously the main implications here are that if we can take a look at some of these other factors early on in the implementation of practice change intervention, it is possible that some of these barriers could be minimized and some of the experiences focused on to make the changes a lot easier and ultimately keep burnout at lower rates. 

Since we are at the end of the time I will just go to the last slide and I guess we are open for questions if we have time.

Mollie:Great. Thank you Dr. Meredith. If you three are able to stay on we can run through those. There are just five short questions.

Dr. Helfrich:Yes, absolutely.

Dr. Joos:I can stay.

Dr. Meredith:Yes.

Mollie:Excellent. Thank you. We do apologize for the technical difficulty and appreciate everybody that is still with us.

The first question came in during Dr. Helfrich’s presentation. What kind of variation did you see across the system?

Dr. Helfrich:That is a really good question and it is a little tricky. That actually could be a much longer discussion but I will say this. We see a lot of variation across the system. However, the amount of variation of burnout explained by clinics, so this is like a given CBAC, a community based assessment center or hospital based clinic, it is only about 4 percent. So at least across the system, that does not mean that there are not specific clinics that have either very low levels of burnout because the things they do at those clinics or a small number of clinics with very high levels of burnout. It is just that across the system a clinic is not the same. The differences among the clinics are not what is driving the overall burnout rate across our system, at least from our data.

Mollie:Thank you. 

This question came in during Dr. Joos’ presentation. On teamlet role characteristic slides, any idea on the content of NCM calls? Are these administrative conversations about scheduling or more clinical or checking in on follow-up to recent visits.

Dr. Joos:  I am probably not the best person to answer that question but in the focus group data and also written comments on the surveys, nurse care managers would say things like I am on the phone all day or I spend all my time on the phone. I think it is a combination of responding to calls from patients, answering phones and following up with patients who leave messages. It seems like they do a lot of the telephone contact with patients. But I am sorry, I really cannot provide any more information than that. I am just not sure.

Mollie:Okay. Thank you.

I believe this one also came during Christian’s presentation. Regarding slide, burnout among PACT compared to All Employees Survey, is it correct that RNs in primary care were selected to act as care managers? Therefore, it can be said that burnout among RNs increased substantially with the reframing of their role. Is there focused research underway on RN care manager role?

Dr. Helfrich:That is a great question. It is difficult to know from these data, it is very difficult to draw causal inferences from these data. That the RN care manager role, that transformation is what is leading to burnout. However, it certainly looks like that RN care manager role has significantly higher rates of burnout than RNs in other roles. There is some work underway on this. A couple of our colleagues, Bonnie Wakefield, who is in the VISN 23 Demo-Lab and Gala True who is with the VISN 4 Demo-Lab are actually spearheading some work to look at that nurse care manager role and kind of what has happened with the transformation of that role and specifically then the association with burnout. That work, we are really just kind of planning that. So we do not have any findings to date but that is something most definitely of interest to us.

Mollie:Thank you for that reply. We just have a few more questions.

Did you look at the impact of “walk-ins” to burnout above and beyond the scheduled clinic patients on any given day?

Dr. Helfrich:Great question. We have not. Sandy, Lisa, is that something that your demo-labs have been able to look at?

Dr. Meredith:Well the VISN 22 we are trying to do some work on patient experiences at the patient level but we really, and we are doing stuff on cost and utilization that Jean Yoon is doing up in the Palo Alto VA. I think Jean might be the person who might have been looking at that data but certainly not through the provider surveys.

Dr. Joos:This is Sandy. We did not look specifically at walk-ins in the provider surveys but anecdotally walk-ins certainly are a problem for clinics but I am not sure to what extent that accounts for burnout per se.

Dr. Helfrich:Yes. That is a great question. Mollie, was there another question?

Mollie:Sorry about that. Yes. I just wanted to mention real quick that Nancy Sharp did mention regarding the last question that VISN 23 Lab is exploring RN roles in depth. So they are taking care of that.

Dr. Helfrich:Yes. Thank you for that.

Mollie:So it seems many of our providers want the NCM doing more of the hands on triage instead of more proactive telephone contact. How would you suggest to improve the utilization of the RN in preventative contact with patients?

Dr. Helfrich:Boy, that is also an excellent question and a tough question. Sandy, Lisa, I do not know if you guys have, I hesitate because I do not feel like I am in a position to make recommendations about how to better utilize those roles.

Dr. Meredith:No I would not be in a position either but what I would say is that it really seems that implementation at the specific team level in terms of the teams deciding how to best use folks in their roles because it is not really a one size fits all. That is my guess.

Dr. Joos:I would agree with that.

Dr. Helfrich:One thing that that immediately makes me think of is, I think it was last year at the HSR&D, the VA Health Services Research and Development Conference, there was a PACT Demo-Lab presentation. I remember Greg Stewart talking about some of the qualitative interviews they had done. One of the observations they made was that with the primary care providers had said, Greg said, their team members, this was not just the RN care managers but their team members in general were not sort of demonstrating an initiative in taking on roles. Conversely, the RN care managers and I think it was even more so for the clinical associates, LPNs and Med-Techs, certainly the PCPs were delegating. So it was like each of their roles were sort of waiting for the other to initiate. The nurses were waiting for the provider to say, this is what I need you to do or this is what I think would be a helpful role. Providers were waiting for the nurses to step in and kind of take things. That might be a point of fristration. I am not a clinician and from survey data would not tell people how best to manage their clinical team. That discussion might be an important point of departure, just expectations of that. Who is figuring out the roles and what the most helpful roles are for each of the team members.

Dr. Meredith:I will just add that because I have attended some of the collaborative meetings for our region and a lot of the discussion at the last one was about, is there a way to provide better guidance for roles across teams. I think that what I heard out of that was that I do not think you can really come up with guidelines. We also heard that in the nursing cyberseminar that it is just really hard to, not everybody has the same situation or even the same kinds of people on their teams or the same patient flow. So it has got to be a decision at the local level.

 Mollie:Thank you all for that input.

The final question we have is, given that this is all supposed to be patient-centered, is there any evidence patients like PACT better than the old way? I think PCPs are burned out to a great extent because we do not have the resources to meet patient’s needs. This has not changed because of PACT.

Dr. Helfrich:Boy that again is an excellent question and a very, very critical question. That is one that we are also intensely interested in. On the national level we worked with, now I am going to have to space on the office that is doing this. It is part of the office of information but we have worked on updating the SHEP survey with some, that is the survey of health experience of patients and we have worked with SHEP to update that survey to add some questions and get it the patient-centered medical home experience, the patient-centered experience. I do not know, actually I do not know this off the top of my head. Sandy, Lisa, are either of your Demo Labs doing patient interviews or patient surveys?

Dr. Meredith:That was one piece of our budget cutting that we had to surgically remove from the VAIL demo but we are like you, we are trying to come up with add on items to get at this issue through the existing VA surveys like SHEPS. So Jill Darling, on our team, is actually leading that component.

Dr. Joos:And in VISN 20 we are also have done over samples, had SHEP do over samples for us. We had plans to conduct interviews and surveys with patients directly but because of the OMB process we are severely held up in that. We are formulating some plans now going forward but that ironically was stymied by the whole OMB process and we have not been able to do as much as we would have liked from the patient’s perception.

Dr. Meredith:This is my opinion, I do not know, I have not worked with these data before but I cannot help but wonder if patients would be sufficiently sensitive to really even noticing the differences and whether or not analyses of those kinds of data would be able to detect any changes over time. But I guess we will find out.

Dr. Helfrich:That is a great point. I was going to mention just briefly for folks who do not know, OMB is the Office of Management and Budget and they have very stringent requirements about surveying the public or talking to the public and patients for their purposes are defined as the public. So it can be very onerous to get any sort of new data collection of patients. It is interesting, that is one thing that Lisa your point is really an important one and one thing that we knew coming into this was that the SHEP measures, the patient satisfaction measures that the VA has been collecting for years and years now, they move very little. They are very stable when you roll them up to like a clinic level.

Dr. Meredith:Yes.

Dr. Helfrich:So I think that you are, I suspect that you are quite right that those are things that we are not going to see a whole lot of movement on. I think what would really be helpful and what the person asking the question was just, do patients like that experience better. It would be really helpful to know if patients at places where the PACT teamlets are up and functioning and where you have an RN care manager who is maybe doing some proactive outreach, is that experiences for patients any different. My suspicion is that the way we would get at that is actually by just getting, sitting down with some patients or getting them on the phone an after an encounter and finding out what that experience was like. And unfortunately, at least at an national level, that is not something that we are able to do.

Dr. Meredith:Very good, good point. I agree that that would be the way to do it.

Mollie:Thank you all for that input. That was our final pending question. So if anyone would like to give any concluding comments feel free to go ahead.

Dr. Helfrich:Well, thanks so much. Yes, Sandy, Lisa, any parting comments?

Dr. Meredith:I will just say that it is actually interesting to see data from multiple locations. It kind of creates a consistent pattern. That is always a good thing. There is so much data available to do analyses with. You can see that each of us has picked pieces but there is a lot more data there. We will be continuing to do more work with these data and kind of honing in on what really is important. So this is just the beginning. I appreciate everybody’s interest.

Dr. Joos:I do echo that. This is the first pass and we all have additional surveys planned and so I think there will be a lot more interesting information coming out in the future.

Dr. Helfrich:Yes. I totally agree. I think that one of the things, just on that sort of, I think one of the big takeaways from this is that primary care, we are seeing high burnout rates. This is definitely a broad challenge and I think the good news is that this process, this whole initiative, even though it has been going on since 2010, this is a big change. It is still actually rolling out. It is still happening now. This is not the end of the story. Also, the national leadership is aware of these challenges and I think the task for us involved in the PACT Demo-Labs and the coordinating center is to continue analyzing these data and trying to find those things that we can feedback that are actionable. Just like Lisa and Sandy said, that is exactly what we are trying to do. We have got a lot of, this is the point of departure not the final word.

Mollie:Great. We all look forward to more updates. I would like to thank our three presenters for sharing their expertise with the field. I would like to thank our audience members for sticking with us through this extra-long presentation. As you exit today’s session, please wait for a feedback survey to come up on your screen. It is just a few feedback questions. It does impact which presentations we select. Also please join us next month on March 20th for another PACT session. This will be on training programs for PACT teams. So we look forward to you joining us for that. So once again, thank you everyone and have a wonderful day.

Dr. Helfrich:Thanks so much everybody.

Dr. Meredith:Thank you Mollie. Thanks everyone.

[End of audio] 

Page 1 of 20

Page **19** of **19**