**Transcript of Cyberseminar**

**Patient Aligned Care Teams (PACT) Demonstration Labs**

**PACT Implementation and Provider Job Turnover**

**Presenter: Philip Sylling, MA**

**February 19, 2014**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact philip.sylling@va.gov*

*Moderator:*We are at the top of the hour, so I’d like to introduce our presenter at this time.  We have Philip Sylling.  He’s a statistician at the PACT Demo Lab Coordinating Center for the Office of Analytics and Business Intelligence.  I’d like to thank Philip for joining us today.  I’ll turn it over to you.  

*Philip Sylling:*Hi, everyone.  Thanks to you all for being here today.  I really appreciate the opportunity just to share with you some of our recent work, taking a look at VA primary care provider job turnover.  The Demo Lab Coordinating Center has tried to leverage VA administrative data to evaluate PACT nationally.  This analysis of job turnover is one project to come out of that work.  I’ll be presenting today, but this project is the product of many collaborators here at the Coordinating Center.

I’d like to start with an audience poll, so hopefully I can get a better feel for how you’re involved with PACT.  

*Moderator:*Thank you.  For audience members, you can see the survey that’s up on your screen.  Go ahead and click the circle next to your answer.  It looks like the responses are streaming in.  We’ve got about a 50-percent response rate so far.  We’ll give people a little bit longer to get those answers in.  So far, it looks like no one is clicking “I’m unfamiliar with PACT,” so that’s a good start.  

We have right around 11 percent that are PACT teamlet member, around 38 percent that are researching or evaluating PACT, about 25 percent each for administrator or implementing PACT or involved in PACT in another capacity.  Thank you to our respondents.  

*Philip Sylling:*Great.  Thanks, everyone, for responding.  That really gives me an idea of how I can maybe tailor the presentation.  This is what I’m going to talk about today.  I’d like to briefly mention some previous PACT seminars that are relevant to primary care providers, and then talk about our motivations for studying PCP turnover.  I’ll go through where our data come from, and explain how we defined turnover.  

I’m first going to share with you some initial descriptives from our data sets, and then go to some regression models that take a look at how turnover changed nationally after PACT implementation.  Then, we’ll look at some provider and job characteristics that are associated with turnover.  Finally, we’ll break down the PACT turnover association by PCP characteristic groups.  Finally, I’ll talk about the limitations of this work and draw some conclusions.  Then, I really look forward to any questions that you may have.  

For me, it was really helpful to see a little bit of where we’ve been in PACT evaluation.  I’ve gathered here’s a few previous seminars from this series that include findings on the experiences of PACT teamlet members.  Just to refresh, we’ve seen findings from surveys, or interviews, or both, from all of the Demo Labs and from the national survey from the Coordinating Center here.  

We have some idea about providers perspectives as to what makes PACT easier to implement or more difficult to implement, and also about their attitudes regarding burnout and job satisfaction.  I think studying provider job turnover is a logical next step.  It’s kind of a nice complement to this prior work, since now we’re trying to measure and explain providers’ actual behavior.  

I tried to pull what I think are a few recurring themes from PACT surveys and interviews that are relevant to PCP turnover.  First, providers often report that inadequate staffing is a significant barrier to implementing PACT.  They also report relatively high levels of burnout.  Also, teamlet members indicate that the role shifts that are associated with team-based care are often difficult and stressful.  

I think, on the positive side, I got the sense that there was support for PACT when teamlets function well together, and also, when they receive supports like protected teamlet huddle times.  I think these points are relevant for turnover because one, burnout might be a precursor to later turnover, and two, also because team-based care probably takes some time to develop.  Any relationship between PACT and job turnover might be very different in the short-term and in the long-term.  I just want to say that, here, we’re really just looking at initial changes in turnover after PACT.  

Why do we want to study turnover as a part of PACT evaluation?  I think, firstly, there’s kind of a circular argument, as far as implementation.  Organizational shifts and role changes might influence turnover, at least in the short-term.  On the other hand, turnover might also impact staffing as an implementation barrier.  

Then, as far as PACT principles, turnover directly affects the patient-provider continuity, since if a provider leaves the VA, then obviously that ends the patient-provider relationship.  Turnover might also influence coordination of a patient’s care, perhaps with a mental health or a specialty care provider.  I think this topic is important to administrators if there are significant predictors of turnover.  Also, because turnover entails substantial recruitment costs.   

Finally, just methodologically, I think this work complements previous surveys of provider attitudes by trying to use administrative data to attempt to study provider behavior.  

We thought that PACT’s organizational changes might be associated with turnover nationally, since it’s a system-wide implementation.  We measured this, using an interrupted time-series analysis, which I’m going to describe in more detail coming up.  Since our unit of analysis is at the provider level and job turnover is ultimately an individual-level decision, we also wanted to adjust for individual-level and local-level economic incentives that might be changing over time, at the same time that PACT is being implemented.  These things might also influence turnover.  

I’m going to briefly talk about where our data come from.  We identified VA primary care providers from 2003 to 2012, using the patient providers table in the CDW, with a couple of restrictions on provider role and team purpose.  Then what we did is used the field’s relationship start date and relationship end date just to establish a PCP’s panel of patients in a given quarter.  Basically, when a provider had no patient relationships in a given quarter, using these fields—relationship start date and relationship end date—then the PCP was assumed to not work in primary care in that quarter.  

Then, we drew some provider-level covariates using an extract from the VA’s personnel and accounting integrated data—or it’s called PAID data—such as FTE status, and the start of VA employment, and a few other things, which I’m going to describe in the next slide.  

We used PAID data to establish that the PCPs that we identified in the CDW actually had hours of works during a given fiscal quarter.  I’ve provided a link here to a PAID data guidebook, which gives a really nice description from the HERC page. 

Just a couple of other data sources.  We wanted to try to account for changes in federal pay schedules relative to private-sector pay, in the years before and after PACT.  We used what’s called the Medicare Wage Index for VA Facilities, which is also available on the HERC website.  We did this just to inflation-adjust VA provider salaries.  The salaries were drawn from the PAID data, which I just talked about.  The idea was that, if non-VA salary growth exceeds that for a VA provider since 2010, then VA PCPs could’ve had a greater incentive to leave VA primary care.  We wanted to account for this, so we adjusted all of the salary rates for PAID back to 2003 dollars.  

Lastly, market area unemployment rates were used as a proxy for providers’ non-VA employment opportunities.  The planning system support group in the VA divides the 23 VISNs into 81 market areas.  The unemployment rate for each market area is just an average of the constituent’s county unemployment rates.  These are weighted by veteran population.  This is an imperfect proxy, but we ultimately decided to use overall unemployment at a more local level—in this case, market areas—rather than using national level healthcare-specific unemployment, because we wanted to capture differences in economic conditions around the country.  

I’m just going to summarize the covariates that we pulled from these data sources that we used in our modeling.  We have end of quarter full-time equivalent salary rates for providers, adjusted to 2003 dollars.  Now, in HR terms, some positions have a lower priority, in the event of a layoff—or sometimes called a reduction in force, in HR parlance.  This is denoted “tenure group equals zero.”  We wanted to control for these positions ending either involuntarily, and also because providers might have an incentive to perhaps leave the VA to seek a more secure position.  We wanted to control for tenure group.  Tenure group equals one, that just means those providers have a priority, in the event of a layoff.  

We controlled for whether providers are employed full-time or part-time by the VA.  Now, this is based only on pay.  A PCP might be a full-time employee but just work half-time in primary care.  This doesn’t have anything to do with the amount of time that the PCP actually works in primary care.  They could be a researcher part of the time, or work in primary care the other part of the time.  We did this because we just want to control for part-time employees perhaps having incentive to seek full-time employment outside of the VA.  Positions might be at a medical center or at a CBOC.  

At the provider level, we control for gender, whether the provider is an MD, a nurse practitioner, or a physician assistant, and their age at the end of each quarter, and also their VA experience at the end of each quarter.  This is just the end-of-quarter date minus their VA start date.  We do this because this may influence their retirement incentives.  We use VA experience, rather than trying to measure their amount of experience in a certain position.  

As mentioned before, we’ve included market area unemployment.  We also include fixed effects at the parent station level.  Anything that might influence turnover at a particular station that stays constant over the whole sample period, from 2003 to 2012, would be accounted for by those fixed effects.

Just to give you a brief description of how we constructed our data sample.  We were able to collect PCP data from FY2003-Quarter 2 to FY2012-Q4.  This is at a quarterly frequency.  Now, since we’re studying PACT at the national level, we just assumed that PACT is initiated, system-wide, in April 2010 or quarter 3 of FY2010.  I think this makes sense because, although realistically, PACT was surely implemented gradually, the organizational changes which might be associated with turnover could probably be felt more quickly.  We’re not really measuring the association of turnover with a fully implemented PACT.  We want to look at turnover as PACT is being implemented.  

As I mentioned earlier, we consider a PCP to be in the VA primary care labor force during a given quarter if the provider has assigned patients, using the patient providers table, and has hours worked in PAID data.  Probably not surprisingly, we found that residents and also PCPs who were classified as intermittent in PAID data had markedly higher turnover compared to nonresidents and regular providers.  We excluded these PCPs from analysis.  

Lastly, about 11 percent of providers hold multiple positions at different facilities during the same quarter, in at least one quarter of the time that they work at the VA.  In this case, what we did is we included only the PCP’s longest-held position.  If they worked for ten years in a certain position, but maybe for two of those years they also worked in another location, we just include the ten-year position, and we exclude the two-year position.  There are a few PCPs for which we couldn’t get any covariate data, so we excluded these, as well. 

I’m going to talk about how we defined the dependent variable.  Basically, for each quarter, if a PCP is in our data set—that means they met the previous inclusion criteria—then otherwise, they are not in the data.  Basically, turnover was defined by providers dropping out of our sample for two or more consecutive quarters.  This is actually just to allow for leaves of absence.  This actually allows for a fairly long leave of absence, because you might work in January and then be absent from February to August, let’s say, and then work again in September.  You’re working the first month of fiscal quarter two, and the last month of fiscal quarter four.  The PCP is only completely absent from the data for one fiscal quarter.  We don’t consider that turnover.  You could have an absence of up to seven months.

Transfers between facilities we did not consider turnover, since PACT was implemented system-wide.  PCPs presumably couldn’t avoid PACT or opt into PACT by changing facilities.  If a PCP transferred within the VA to a non-primary care position, then we did consider that turnover, and we assumed that would show up in our data by that provider’s panel of patients ending in the *[fading voice 21:45]*.  We allowed PCPs to turnover and to later reenter the sample.  We don’t consider—we don’t really know the reason for turnover here.  It could be retirement, or someone could resign and leave the VA.  We don’t know the reason for turnover.  I just wanted to give you a little example to help you visualize this.  

We have a fictitious PCP who is missing from the data after FY7-Q1.  We consider that PCP to have turned over in FY7-Q1, because they’re not in the data after that point.  Another PCP leaves the labor force in FY6-Q4.  Then they’re missing from the data for four quarters, and then they reenter our sample in FY8-Q1.  The last PCP is only missing from the data for one quarter, so we consider that that PCP didn’t have any turnover events in our data.  

Now that you understand how we defined turnover, I want to share with you a few descriptive statistics.  In the second column here, we’ve aggregated over all observations in the pre-PACT period.  In the third column, we aggregated over all the post-PACT observations.  In the fourth column, there’s a P-Value testing the equality of the pre- and post-PACT periods.  

The quarterly turnover rate is typically between three and four percent.  To get an annual rate, you could just multiply by four.  Turnover is significantly greater after PACT for all PCPs.  These are just unadjusted differences.  If we break this down by provider type, turnover is higher after PACT only for MDs who are full-time employees and in tenure group one, which is the vast majority of MDs.  There are slightly more female providers after PACT.  The distribution of provider type is pretty stable across the two periods, with about 70 percent MDs, 20 percent nurse practitioners, and 10 percent PAs.   

We did find that the provider workforce is getting older over our sample period.  Here, I’ve broken up the PCPs into age: those aged under 45, 45 to 55, and over 55.  You can see that the over-55 group is about one-third larger in the post-PACT period.  

Providers have about nine years experience.  We’ve set our inflation-adjusted salary rate, so the mean is equal to 1 in 2003.  Then we can calculate that, compared to that salary index of 1 in 2003, the average for the pre-PACT period is about 1.036, and after PACT, about 1.108.  VA PCP salaries are rising about seven percentage points faster than the Medicare wage index, across these two periods.  The vast majority of PCPs are in tenure group one, and are full-time employees.  

Lastly, medical centers employ relatively fewer PCPs after PACT, which probably just reflects growth in the number of CBOCs.  As we all know, the unemployment rate is much greater in the last several years, compared to the early 2000s, so about 50 percent higher.  Our sample contains about 9800 unique PCPs—I’m sorry.  Contains about 11,300 unique PCPs.  You can see the number of observations we have in the pre- and post-PACT period.  Our unit of observation is the provider quarter level.  

I want to just talk a little bit about our interrupted time series methodology.  Basically, what we just looked at is a pre-post comparison with no control group.  We can do a little bit better than this, if we take into account that turnover may have already been trending up or down before PACT.  We can account for this using an interrupted time series analysis.  

I’ve just included this figure so you can visualize how we’re assuming turnover evolves over time.  Basically, we assume the rate of turnover follows one of these three paths.  We allow for a secular trend in the turnover rate, which might be upward, downward, or maybe no trend at all.  Then, we allow for a discrete change in the turnover rate after PACT.  Basically, we do this with an indicator variable that’s zero for all observations that are before FY10-Q3, and a one for all observations after that period.  When we talk about the association between PACT and turnover, we’re really talking about this PACT indicator variable.  Then, we also allow for some seasonality in turnover, using a quarter-of-year indicator.   

Just a few statistical notes.  We’re modeling the quarterly probability of turnover using a logistic regression model.  We use robust standard errors, which account for clustering at the provider level, since we have repeated observations.  With a model like this, there is a problem if we include dummy variables at the person level, and if we only have a few observations for each person.  I think, in this case, we avoid this problem when we include dummy variables for each parent station.  We have a lot of observations for parent stations.  We have over 500 observations for all but 6 stations.  

I just wanted to mention that some researchers are probably more familiar with using odds ratios from logistic regressions.  We are going to present the association of our explanatory variable with turnover in terms of average marginal effects.  This will make the interpretation much easier when we look at the PACT turnover association for distinct demographic groups.

Basically, the average marginal effect is the change in probability that’s associated with a one-unit change in an explanatory variable.  If provider type were to change from MD to a nurse practitioner, and if we hold everything else constant, and the probability of turnover then changes from, say, four-and-a-half percent to five percent, then the average marginal effect for nurse practitioner is just a half a percent, or .005.  You can see in this table that, if you want to think in terms of relative risk, then if the average marginal effect is .005, that’s somewhere between a 13- and 20-percent increase in the relative risk of turnover.  

Let’s look at some results, finally.  The association between PACT and turnover, we measured this in a few models, since we wanted to see how stable it was when we controlled for different covariates.  In model one, we only include the secular trend in turnover and seasonality.  Then, in model two, we add all of the PCP and job covariates.  Then, in model three, we add just—in addition, we add just the unemployment rate.  

You can see that there’s always a significant—it’s a small but statistically significant association of PACT and turnover.  I just included the baseline turnover in blue here.  I assumed a baseline turnover of three percent, just so you can see the relative size of the marginal effect of PACT relative to baseline turnover. 

Now, personally, I thought it was interesting that, when you add the unemployment rate to the model, it bumps up the association of PACT and turnover to .004.  I just wanted to add a little bit of context there, and also mention that, all the results from here on out, we’re just going to be talking about this model three.  

I just wanted to say that economic conditions might influence the conclusions that we draw from these kinds of policy evaluations.  If you think about the first blue arrow on the left here, this just signifies that we observed that there’s elevated unemployment, which is roughly coincident with PACT implementation after 2010.  If we suppose that there’s a small increase in turnover after PACT, that’s what we observe.  We know that—or, we hypothesize that—historically, the relationship between unemployment and turnover is a negative relationship.  When we see higher unemployment, we would expect there to be lower turnover, which is indicated by this red arrow.  Then, in our models, we will probably find a greater estimated association between PACT and turnover.  

Now, let’s take a look at some of the factors that predict baseline turnover.  Since our sample’s really large, it’s pretty easy to find statistically significant differences.  I think some of these associations are not trivial.  We find that female PCPs, which made up just over half of the workforce, had slightly lower turnover.  Compared to MDs, we found that nurse practitioners and physician assistants both had significantly higher turnover.  

I thought this was an interesting finding because, in the studies that I looked at from the academic literature, I couldn’t find consistent differences in job satisfaction or job stress among MDs, NPs, and PAs.  I think, also, in the PACT surveys, there didn’t appear to be consistent differences in job satisfaction and burnout between these types of providers.  I thought this result was interesting.  

Now, there’s a U-shaped relationship between age and turnover.  Since the under-45 age category is our holdout group—so we assume it has an effect of zero—and then turnover is lower as age goes into the 45-to-55-year age group, and then it’s higher for providers over age 55, so probably a retirement effect there.  The fact that turnover is higher for the youngest PCPs is interesting, when we keep in mind that we’ve excluded residents for our sample.  It’s not due to a residents’ effect.  We found that turnover in CBOCs is lower than in medical centers.

I’ve included the other covariates that we included in our model here, in this table, mostly for reference.  I’d just like to point out that we did find a slightly upward trend in the turnover rates from 2003 to 2012.  This means that the interpretation of our previous results is that we find an increase in turnover associated with PACT, after accounting for an existing upward secular trend.  

Maybe notably, in the second row here and the third row, we find that VA experience and inflation-adjusted salary are not strongly associated with turnover.  I think this might be because salary schedules are based on other covariates that we’ve already accounted for here, such as provider type, experience level, regional differences—which would be captured by the Sta3n dummy variables.  

Now, we’ve looked at the association between PACT and turnover at an overall level for all PCPs.  We assumed that the change in turnover was the same for all PCPs, down a little bit, and examined differences in this relationship across provider groups.  Basically, in the regression model, we interacted the PACT indicator with some of the key demographic variables.  Then, we can calculate a distinct average marginal effects for the PACT turnover association for each of these PCP characteristic groups.  

If you look at gender, the increase in turnover that’s associated with PACT was directionally greater for male PCPs, although this is not a significant difference.  Likewise, the increase in turnover associated with PACT was higher for MDs, compared to NPs and PAs, just directionally.  Again, this was not a significant difference.  That’s probably due to the fact that we don’t have a large sample size for NPs and PAs.  Also, we only have eight quarters of post-PACT data.  We just don’t really have a large enough sample size to tease out these effects across provider groups.  

Then, we also looked at the interaction between the PACT indicator variable and the provider age group and years of VA experience.  If you look at the second column of average marginal effects, for the youngest PCPs, in the second column the AME close to zero, and it’s close to zero for the least experienced PCPs.  In fact, if you look at the 95 percent confidence intervals, we could say that, since they both include zero, that there actually is no association between turnover and PACT for the youngest PCPs and for the least experienced PCPs.  

As you look down the column of marginal effects for the PACT turnover association, you can see that they’re directionally increasing in both age and in VA experience.  We are able to actually find some significant differences here.  The AME of PACT is significantly greater for the middle age group, 45 to 55, and also for the over-55 group, compared to the under-45 group.  If we look at VA experience, the PCPs with 20 years of VA experience have a higher association of PACT and turnover, compared to those with 5 years of experience. 

I think these results motivate the question of whether there might be a retirement motivation at play here as a plausible explanation for these results.  It’d be really great, in the future, to be able to tie in a reason for turnover.  That would be really helpful.  

Now, so far, we still only looked at the national-level picture.  I just wanted to briefly zoom in a little bit.  In this figure, what I’ve done is aggregated turnover across only the eight post-PACT quarters.  Then, all facilities are rolled up to the parent stations, Sta3n-level.  This does include CBOCs, which are aggregated up to the Sta3n-level.  Then, I’ve just grouped these stations by their post-PACT turnover rate.  

You can see that there’s a fair amount of variation at this level.  From these data, for example, the 13 stations at the far left, they have an annualized turnover rate of only about 5 percent, while the 24 stations in the highest groups, they have an annualized turnover rate of over 20 percent.  We can see that these facilities might have a significant challenge to staffing stability. 

I want to explain several limitations to our study.  Since PACT was implemented system-wide, we don’t have any good control group of VA facilities with which to compare the turnover rate.  Our interrupted time series analysis tries to address this by establishing the turnover trajectory or growth rate within the VA, prior to PACT.  

Second, we have to infer turnover from providers dropping out of our sample, so our derived turnover rate might be inaccurate.  I think the fact that we use a consistent definition gives us some more confidence that we can identify changes in turnover across time, even if we don’t exactly or accurately estimate the exact turnover rate.  We’d like to ultimately be able to incorporate turnover reason into our analysis, but we don’t have that, at this stage.  

Finally, accounting for PCPs outside the VA job opportunities using the overall unemployment rate is not optimal, although we did find the expected relationship for this variable.  We did find that higher unemployment was associated with lower turnover.  That was actually in the other covariates table, which I should’ve mentioned.  

Finally, I’d just like to wrap up with a few conclusions.  From a high-level view, our results suggest that PACT was associated with slightly higher provider turnover, at least at the beginning of implementation, since we’ve only considered two years of data after PACT.  Just to add a little context, our model would predict that, if there were about 7,000 VA PCPs, then a marginal increase in turnover probability of .004 times 8 post-PACT quarters would apply additional turnover of exactly 224 additional PCPs.  

When we drilled down, we found that the PACT turnover association was significantly greater for PCPs over age 45, and also for those with 20 years of experience.  Our findings identified several significant predictors of turnover.  For example, we found higher turnover for nurse practitioners and physician assistants, compared to MDs.  Then, finally, at the Sta3n-level, it does appear that the burden of provider turnover is not spread evenly across parent stations, which is probably a big challenge for some primary care administrators.

Now, that’s what I have.  I’d like to open it up and hopefully get some questions from you guys. 

*Moderator:*Excellent.  Thank you so much.  We do have a number of good questions that have come in, so we’ll jump right into it.  I just want to make a quick announcement.  A couple people let me know that the handouts link at the beginning wasn’t correct.  Thank you for bringing that to my attention.  If you do want the handout link, just go ahead and write into the Q and A box, requesting it, and I will get it over to you.  Also, Philip, it turns out that the HERC link might’ve been misdirected, as well, so we will fix that before we post it in the archive catalog.  

With that, we’ll go ahead and get going.  Given the increase of age of the providers, to what extent is the turnover increase merely being driven by retirement and not PACT?  I think you may have touched on that.  That came in pretty early on.  

*Philip Sylling:*Right.  That’s a great question.  That’s something that, at this stage, we can’t really answer very well because we don’t know the reason for turnover.  That’s something that I think we might be able to—that’s data we might be able to acquire.  It would be really interesting to be able to directly measure that.  I guess the only thing I would say is that it could be that, if retirement—if we find that the association that we found was driven mostly by retirements, it’s still kind of hard to pinpoint any influence of PACT because some retirements could be somewhat voluntary in the short-term.  Of course, all turnover is involuntary in the long run, and there’s 100 percent long-run turnover.  In the short run, even with retirements, there could be some wiggle room for there to be some influence of organizational things.  

*Moderator:*Thank you for that reply.  Regarding slide 23—I can pull that up real quick—is this the unemployment rate for other PCPs only?  

*Philip Sylling:*We just used basically the overall unemployment rate that you might read in the newspaper.  Right now, the overall unemployment rate is, I don't know, something around seven percent.  That’s the unemployment rate that we’re talking about.  It is at a level of granularity.  We’re at the market level.  We have variation in the unemployment rate across the country, and also across time.  

We used this as a really rough way to measure providers’ ability maybe to leave the VA and secure outside employment.  It’s really imperfect because it’s overall unemployment, rather than healthcare-specific unemployment, but we just don’t have a good healthcare unemployment rate at a local level.  We decided to use overall unemployment.  What we did find the expected relationship, where areas of higher unemployment saw a lower provider turnover.  

*Moderator:*Thank you for that reply.  The next question we have: what is the overlap between female providers, nurse practitioners, and PAs?  These are not isolated parameters.

*Philip Sylling:*That’s a great question.  Yes, I’m pretty sure that there are more female nurse practitioners.  They’re definitely not independent.  In our regression model, since we include all three factors, we’re really able to measure the association with turnover of the individual components.  Even though there’s a correlation between being a female provider and perhaps being a nurse practitioner, our regression still allows us to estimate the association with turnover of just being a female provider, or just being a nurse practitioner.  

*Moderator:*Thank you for that reply.  The next question we have: was there a correlation between discipline and age?  For example, were NPs a younger group?  In my experience, NPs often get first job experience in VA, and then move to the private sector.

*Philip Sylling:*That’s a great question.  I don’t recall that I looked at that, specifically, so I can’t say, off the top of my head.  That’s a really great question.  That’s something that we could look at.  

*Moderator:*Thank you.  Next question: do you have enough old but inexperienced PCPs to tease apart age and experience?  

*Philip Sylling:*That’s a great question.  I think there are a number of PCPs who enter VA primary care at an older age.  We are actually able to measure the effects of age and experience.  Often, in a model, one can only include age or experience, because they might be just perfectly related.  In this case, I think the correlation between age and experience was only maybe .4 or .5 or something like that.  There was still, I think, enough variation to be able to estimate individual effects for age and experience.  

*Moderator:*Thank you for that reply.  Somebody did comment that they agree with your retirement explanation.  The next question: how does this compare with turnover in VHA staff, overall?  We know that there is a large number of retirement-eligible staff throughout the VHA.

*Philip Sylling:*Right.  That’s a great question.  I really have no good estimate of non-PCP or overall VA turnover.  I’m really not sure whether provider turnover is higher or lower than turnover overall in the VA.  

*Moderator:*Thank you.  Will you or have you looked at turnover rates for the rest of the teamlet members?

*Philip Sylling:*That’s another great idea.  I guess we started with providers.  I think it would be a little bit more difficult with measuring turnover for the other teamlet members because one way that we measured turnover was to use patient assignments to a provider.  I’m not sure, for example, with an LPN or a clerical assistant, I’m not sure where we could find data so that we could measure turnover.  I’m sure, in some HR data, it’s probably possible, but nothing that we have ready access to, at this point.

*Moderator:*Thank you.  The next question: did you look at whether panel size affected turnover?

*Philip Sylling:*That’s a great question.  At this stage, we haven’t looked at panel size.  That’s for a couple of reasons.  First, I think part of PACT is to define panel size as, as a teamlet has a panel of about 1200 patients, and we have the 3.0-staffing ratio.  We didn’t do it at this stage because panel size is sort of a part of PACT implementation.  Also, on just a data level, a practical level, there’s sort of—you can often see a reverse causation between panel size and turnover.  I think that providers don’t just immediately one day just say, “I’m leaving.”  There’s some advance notice.  We see, often, that providers’ panel sizes are reduced right before they leave the VA.  At this point, I haven’t found a good way of getting at the relationship between panel size and turnover.  That’s something that we’d really like to do.

*Moderator:*Thank you for that response.  It looks like you may have already touched on this, but I’ll read the question.  Do you plan to look at any other PACT members, such as nurses and pharmacists?

*Philip Sylling:*At this point, we don’t have a plan to do that.  It would be really interesting to look at turnover, for example, for nurse practitioners and other members of the teamlets.  We’re just not sure about where to proceed, as far as data for that question.  

*Moderator:*Someone wrote in a general question.  What is proposed that we are to do with this data?  There are so many areas of analysis that would be needed for decision-making on future course for PACT, as well as VHA.

*Philip Sylling:*Absolutely.  That’s a really great comment.  It’s just meant as a first look at an overall level of turnover in the VA and turnover after PACT.  I don’t think—I think it’s too early to give specific recommendations.  It’s kind of a first look.  Hopefully, we can drill down to a deeper level.  We’d really like to look, rather than just at the national level, look at specific characteristics of facilities that are implementing PACT, and look at turnover in those facilities.  That’s something that’s still in the future.

*Moderator:*Thank you.  Someone did look up the Bureau of Labor statistics, and they don’t have healthcare-specific unemployment, but they do have good local-area unemployment that may refine your analysis.  I sent you that link offline.  

*Philip Sylling:*Okay.  Great, thank you.  I’ll take a look at that.

*Moderator:*We do have several people that wrote in and wanted to thank you for making the methods understandable and going through this approach step-by-step.  It’s much appreciated.  

*Philip Sylling:*Wonderful.

*Moderator:*I’m going to ask our attendees to hold on for just a sec.  I’m going to close the meeting up.  We do want you to fill out our brief survey.  It looks like somebody else wrote in.  HRSA does publish healthcare-shortage areas.  Thank you for writing that in.

*Philip Sylling:*Okay.  That’s interesting.  

*Moderator:*Another resource.  Would you like to give any concluding comments, Philip?

*Philip Sylling:*No.  I’d just like to thank everybody for joining in.  I hope you found it informative.  I really appreciate all of your feedback.  I just thank you for listening. 

*Moderator:*Great.  I want to thank you for sharing your expertise with our attendees, and thank our audience for joining us today.  As I mentioned, I’m going to shut down the meeting, and you will be redirected to a brief survey.  Please do give your input, as it helps guide the direction of our program.  Thanks again, and have a great day, everybody.  

*[End of Audio]*

Page 1 of 13