HMCS (HERC Health Economics Seminar)- 17 -Department of Veterans Affairs  HMCS-051612

**Department of Veterans Affairs**

**HMCS (HERC Health Economics Seminar)**

**Outpatient Waiting Time Measures and Patient Satisfaction**

**Julia C. Prentice**

**Steven D. Pizer**

**May 16, 2012**

Moderator:  I wanted to welcome everybody to this month’s HERC Cyber Seminar.  We have Julia Prentice whose going to be presenting today.  She received her PhD in Community Health Sciences in 2004 from UCLA, and then joined the VA in Boston, where she’s been ever since.

So she’s been doing a lot of work looking at wait time measures and patient satisfaction.  These are performance measures that are doled out to facilities, and you’ve probably heard in the news that there is a lot of interest in trying to improve wait times and making sure that we’re minimizing wait times.  So it’s with great pleasure that I give the floor to Julia.  Thanks Julia.

Julia Prentice:  I’m happy to be here today to discuss how the various measures of wait times affect patient satisfaction.  But before I get started, I want to acknowledge that this work is done in collaboration with Steve Pizer, who is also online, or should be.  And also, I need to acknowledge Dr. Michael Davies, who is the Director of Systems Redesign in the Central Office.  Systems Redesign is actually sending this work, and he’s been really great about giving us his historical overview of how the different wait time measures have evolved in the VA.

So wait times have been a main policy focus for over a decade.  Before 1999, there were reports that wait times were probably too long in the VA, but the VA wasn’t actually systematically collecting data on wait times.  So Congress, in response to pressure from veterans who were complaining, requested that the VA start systematically measuring wait time data and reporting on these measures.

At the same time, in order to decrease wait times the VA implemented a variety of different interventions to minimize wait times.  VA facilities and VA managers are now subjected to a variety of performance measures that they must get patients in for care in a certain amount of time.  As well, they implemented the Advanced Clinic Access Initiative back in 2000 in six different target clinics, such as primary care.

The Advanced Clinic Access measure essentially has clinics look at supply and demand, and then changes how clinics are scheduling their appointments to balance that supply and demand to ensure that they’re spending more time actually providing care to patients, versus triaging patients, and that decreases wait times overall.

The VA also implemented primary care panel sizes for their primary care physicians so no physician is supposed to have too many patients, to help ensure that veterans get in right away.  And they limited enrollment to new priority seven and eight veterans between 2003 and 2009 to help balance supply with demand.

So overall, this is the wait times for the first next available appointment for new patients this graph is showing you.  And I’ll explain that measure in a moment.  But it overall just shows you that wait times have substantially decreased over time.  The mean wait time was about 50 days back in ’02, and it has steadily decreased since then to about 20 days in 2010.

Despite that, there is still quite a bit of variation between the VA facilities and how long individuals are waiting.  In 2010, 10 percent of the facilities had VA wait times that were about 25 days, and 10 percent of the VA facilities had wait times that were about two days.

And as we were discussing, there still remains a lot of concern about wait time.  The VA OIG just recently has given another report out on wait time policies and access to mental health care that just came out in April of 2012.  There have been a variety of Congressional hearings on access in the last year.  The most recent was the Veteran’s Affairs Committee had a Congressional hearing on access in April of 2012, and the House Veteran’s Affairs Committee had a hearing on access just last week.  So especially for accessing mental health care, wait times have repeatedly come up lately.

So the VA has used a variety of wait time measures over the years, but the overall reliability of these wait times is largely unknown.  Even though they’ve implemented a variety of initiatives to decrease wait times, very little research has actually used wait time measures to predict outcomes.  And so this study is a means to fill this knowledge gap.  And today I’m going to focus on our results for patient satisfaction only, but our future analysis are going to focus on linking the different wait time measures to other health outcomes, such as mortality or preventable hospitalizations.

So overall, the VA has used several different wait time measures that I’m going to explain in detail.  The first that they started with are is what’s known as a capacity measure, and it’s the first next available appointment.  Now as far as we know, when the private sector is measuring wait times they solely rely on capacity measures, and haven’t moved beyond those.

The times the VA has moved due to limitations of the capacity measures, the VA has moved on to a retrospective time stamp measure, which I’ll define in a moment, and then they have also used a prospective access list measure.  So these latter two measures are two different dates the VA uses to calculate wait times; a create date and a desired date.  And I’ll explain all of this in detail.

So first we’re going to start with the capacity measure, which is the first next available appointment.  So suppose that new patient A requests to be seen as soon as possible on January 5, 2010?  The scheduler simply looks into the scheduling system and finds that the first next available appointment is January 10, and so a wait time of five days is assigned; the tenth minus the fifth.

This measure, however, is measuring simply overall supply in the system.  It doesn’t take into account at all whether or not the patient is available to take that appointment, whether or not that patient prefers to take that appointment.  And so it may not be at all reflective of how long this individual actually ends up waiting.

It also requires the schedules to distinguish between follow-up and urgent care appointments, and this is especially problematic for established patients versus new patients.  So if a veteran is told to come back in three months from their provider, the scheduler needs to enter their appointment request as follow-up versus urgent care, and if it inappropriately gets entered as, “urgent,” then (inaudible) artificially links in the wait time feature.

The other problem with the first next available appointment types is that there are multiple appointment types in the system.  So for example, physicals are often longer, are maybe like, 45 minute appointment slots versus 20 minute appointment slots.  And so if the first next available appointment type is for a physical and the patient doesn’t want that, then that appointment type isn’t actually going to fulfill his needs.   And we’ve also heard that physicians often are practicing in multiple different clinics.  For example, you may have a physician that is practicing in primary care but also in the cardiology clinic.

The scheduling grid can’t consult all of the different scheduling profiles for the same physician, and so it may show false availability for that physician when that physician isn’t actually available.

However, it is the first next available appointment that has been used in any of the previous work that’s been linked to poorer health outcomes.  Now this has largely work that Steve Pizer and I have conducted over the last few years, and we started with a sample of geriatric veterans, or veterans who are visiting the geriatric clinic.  This was a very old and a very vulnerable population.

We found that these veterans who are visiting VA facilities with longer waits were at risk for higher mortality rates and higher preventable hospitalization rates.  And the preventable hospitalization is simply an arc, a safety indicator that says that in their conditions, that if you are receiving timely outpatient care you shouldn’t actually be hospitalized for those conditions.

We then, in a subsequent study moved on, and we took a sample of veterans who were diagnosed with diabetes.  We began to look at wait times using the first next available measure, and examined the fact whether veterans who were visiting facilities with longer waits were more likely to have higher rates of mortality, preventable hospitalizations, heart attacks, stroke, and higher hemoglobin A1C levels.  We did find that especially for veterans who were over aged 70 and who had greater comorbidities at baseline, there was a significant relationship between visiting a facility with longer waits and poorer health outcomes.

However, the FNN measure has several limitations; one of the most important being that it does not actually measure whether or not the patient is available to take the appointment that they are offered and whether they want that appointment.  So the VA managers then decided to explore other options which specifically take these preferences into account.

So the first thing they tried was the create date time stamp calculation.  Now this is a retrospective measure.  And assume that new patient A is requesting to be seen once again, as soon as possible on January 5.  They cannot take the January 10 appointment, and so the appointment is scheduled for January 21.  The wait time ends up being simply 16 days; January 21 minus January 5.

The advantage of this measure is that it requires very little information from the scheduling clerk.  The appointment request is when the patient is actually requesting an appointment, that automatically gets entered into date, automatically gets entered into the system and then the date of the appointment is entered into the system as well.

However, this measure is based on completed appointments.  So it excludes any patient no-shows or any cancellations that were not rescheduled.  And this is an important distinction when we start talking about the access list and prospective measures.

The other problem with this measure is that different VA facilities are scheduling their follow-up appointments in different ways.  So for example, if a veteran is told to come back in six months for a check-up, some VA facilities, it goes out to the scheduling clerk and the VA facility will schedule that appointment right then and there, and so it looks like that wait time is six months.

In contrast, other VA facilities will say, “Okay.  You need to be back in six months, so make a note of that.”  And a month before that veteran is supposed to come back, they contact him and they schedule his appointment.  So it looks like he has a wait time of about one month.  So once again, this is more problematic for established versus new patients.

But it was this limitation that led the VA to start considering a desired date calculation.  So suppose established patient B requested an April 5, 2010 follow-up appointment back in January.  The appointment actually ends up getting scheduled for April 20, and so the wait time ends up being 15 days because they’re using that April 5 desired date instead of when the appointment was originally requested.

In 2010, the VA has shifted entirely to this desired date measure because it’s not influenced by the use of re-call systems or by how the different VA facilities are scheduling follow-up appointments, and it takes into account specific patient preferences, which VA managers really like.  That being said, the schedulers have to correctly enter the desired date.  And so the original desired date must be kept when negotiating an appointment.

So for example, if the provider said, “I want to see you back here on May 1,” and the patient goes out to the scheduling clerk and they say, “I need an appointment on May 1,” and the scheduling clerk brings up the schedule and says, “The earliest we can get you in is May 5.  Does that work?  Do you like that?”  They will say—And the patient says, “Yes.”  Some of the scheduling clerks will go ahead and enter May 5 as the desired date instead of May 1.  It should be May 1.

The VA has tried to address this with extensive training of the schedulers and recent audits from systems redesign have found that desired date is entered correctly about 90 percent of the time.

So the time stamp measures I have just been discussing are retrospective, and they only include a completed appointment.  So if a patient actually never shows up for the appointment, that appointment doesn’t get included in the wait time measure.  And if the patient or the clinic cancels an appointment and never reschedules it, those appointments are not included.

So the other way you can measure wait time is in a prospective measure.  So these are the accessless measures.  You calculation—And they—These measures we’re calculating ways (inaudible) pending appointments.  And because you do not know who actually isn’t going to show up or who may not cancel later on, this includes all appointments that are actually scheduled, regardless of whether or not they actually happen.

So the access list create date calculation is—Suppose that new patient A requests a (inaudible) appointment as soon as possible on January 5, and that appointment is actually not scheduled until February 10, 2010.  The access list, or the (inaudible) runs bimonthly report dates.  So on the first and the fifteenth of each month, they’re pulling a list of all pending appointments.  And this is how they’re calculating their wait times.

The appointment is not actually eligible for calculation until the create date is equal to or before the report date.  So for the January 1 report, since the appointment was not actually requested until January 5, the appointment won’t be included.

However, for the January 15 report, the wait time is 10 days.  It’s the report date, so January 15, minus the date that the patient requested the appointment, which was January 5.  And then since the appointment hasn’t happened yet, it will all show up on the—It will also show up on the February 1 report date, and it will be assigned a wait time of 26 days.

Now the purpose of the access list is basically to make sure that VA facilities are processing their appointments through in a timely manner.  And so the performance measure is the percent of appointments that have less than a 14 day wait.  So for example, if a VA facility is looking, and in January 95 percent of their appointments do have less than a 14 day wait, but in February, when they’re pulling these access list reports and they find that 85 percent of their appointments have less than a 14 day wait, then they know that there is something about the demand or supply that’s changing and they need to start addressing wait times.

For our purposes, we wanted to make the access list measure comparable to the other measures and so we averaged the wait times together.  We’re averaging these access list waits together.  Again, the advantage of—The disadvantage of the create date version of this measure is that it is once again influenced by how follow-up appointments are scheduled.

So there is also a desired date version of this calculation, and it follows the same logic.  An appointment—If established patient B requests an April 5, 2010 follow-up of appointment and that appointment actually gets scheduled for May 5, then it’s going to show up on the April 15 report date in 10 days, and on the May 1 report, with a wait time of 25 days.

And again, they’re using a performance measure of a percentage of appointments that have less than a 14 day wait, which we—And to make it comparable to our other measures, we’re actually averaging that wait time.

And it has the same limitations as the other desired date measure, which is that schedulers must correctly enter desired dates.  So this slide just gives you a summary of the different wait time measures that I’ve just discussed.  For each measure, they’re calculated for new versus established patients separately.  The time stamp measure, which includes both create date and desired date to calculate the wait is retrospective, and the first next available appointment and the access list measures are prospective, and the access measures include a create date version and a desired date version.

Moderator:  Can I ask some questions about this if that’s okay, Julia?

Julia Prentice:  Sure.

Moderator:  I guess the first question that I have is, is this for every clinic, or is this just for general medical outpatient care?  Are we able to separate clinics here?

Julia Prentice:  We are, and I will discuss that in a moment.  They do keep wait times for every clinic.  However, there are 50 clinics that are specifically targeted for performance measures, and these 50 clinics are high volume clinic stops and they also are the ones that are most likely to capture patient/provider interactions.  So for things like labs or telephone interactions where you’re not really scheduling appointments, those don’t get included.  And they also (inaudible) most of the major medical subspecialties.  So mental health, cardiology, dermatology, etc.

Moderator:  Great.  And then you had mentioned at one point audits.  So I guess one of the questions is do you have information on how valid these systems are, and are all the clerks actually using the system?  I’ve heard rumors that patients are getting locked over from one clinic to another clinic because it circumvents this whole wait time system and I wasn’t sure if your audit data has anything to add to that?

Julia Prentice:  I think that that—Steve might have more to add on this if he’s on the line.  But it’s an ongoing and valid question, how well, how valid the data are.  Because as soon as you measure things, there is a potential to gain that (inaudible).  That said, systems redesign is doing a lot of work to try to minimize the (inaudible).

Steve Pizer:  So let me add in—This is Steve.  There is concern, there has been concern that’s been raised by the inspector general and various people about whether scheduling clerks are doing what they’re supposed to be doing.  And early on, after the adoption of desired data in particular, there was some internal audits done by systems redesign that found some problems.  And they instituted a lot of training and did some follow-up on it and found evidence to suggest that the clerks were doing what they were supposed to be doing.

This is still a matter of some disagreement.  You know, the IG—This is the subject of some of what the IG is talking about and some of what the Congressional hearings have been about.  But there has been a lot of effort to try to institute training and audits to make sure that the clerks are doing what they’re supposed to be doing.

I think as researchers, we can say that this is a matter of some ongoing controversy and that the central management at VA is doing a lot to try to get data that is as reliable as they can.

Moderator:  Thanks, Steve.  We actually had a question that someone wrote in.  Let me read it and then we can see if this helps.  The access measure does not list an appointment against the access percentage until the patient has already waited 14 days.  Whereas the future pending appointment report will include all appointments scheduled greater than 14 days from DD, the current Q, regardless as to whether they have already waited 14 days or not.

Additionally, the future pending appointment list shows the true percentage within clinics of appointments that are scheduled greater than 14 days from the desired date.  And this person suggested it may be helpful to clarify these differences for the listeners.

Now, if it’s easier to read the question Steve?  I don’t know if Heidi has given you access to the questions, because you might also be able to read while he presents and answer some of these better than I can read them to you.

Steve Pizer:  That might be helpful.  I can say—

Heidi:  Steve, I just gave you access.  I’ll come down and tell you where it is.

Steve Pizer:  Thanks, Heidi.  One of the things that I think is important for us to say is that what we’re presenting here is research using selected measures.  And in particular, I think it’s easy to get confused about some of the access list measures.

People in the field may be familiar, and particularly people in operations, familiar with some of the reports that come off of some of the data that is often called up (inaudible) the access list data.  We’re not using those same reports that list the percentages waiting more than 14 days, and Julia, maybe you haven’t quite gotten to that, yet?  Is that right?  Or maybe you—I’m not sure whether you did or not.  So we’re—

Julia Prentice:  (Inaudible).

Steve Pizer:  The research that we’re doing is using data from that same source but we’re creating measures that are a little different from the standard dashboard measures that people might be seeing.

Julia Prentice:  Okay.  Should I go on?

Moderator:  Yes.  Please do.

Julia Prentice:  Okay.

Moderator:  Thanks.

Julia Prentice:  So the main research question we’re addressing today is how well do these alternative measures and wait times predict patient satisfaction?  And so when surveys, research (inaudible) surveys on patient satisfaction have often found that access is a key component to satisfaction.  Patients find it difficult to judge the technical quality of their health care providers, so they don’t know, for example, if a physician is ordering the correct diagnostic test, but they are able to judge the practical aspects of their health care experience.  So did they feel like they sat in the waiting room too long?  Did they feel like when they called to get an appointment they couldn’t get in right away?  And those aspects tend to lead to lower satisfaction.

Our satisfaction data comes from the 2010 survey of healthcare experiences of patients, which is managed by the Office of Quality and Performance, and it’s modeled after the consumer assessment of health care providers and systems in the—Which is used extensively the private sector.

It’s a simple random sample of patients who have a completed appointment in each month.  And the visit date of that completed appointment is recorded in the survey data.  Our overall sample is about 220,000 individuals.  And we looked at a variety of satisfaction measures.  (Inaudible) were specifically focused on access.  The first is how often were you able to get an appointment as soon as you wanted, which we are going to refer to as timely appointment.

Respondents were also asked how easy was it to get a test or treatment in the VA in the last 12 months, and how easy was it to access a specialist visit in the last 12 months?  So all of these satisfaction measures are asked for the last 12 months, but when the respondents are filling out the survey, we suspect that they’re actually keeping their most recent visits in mind.

And the response options given for these measures was always, usually, sometimes or never.  So we end up dichotomizing these responses into those who said, “Always and usually,” versus, “Sometimes or never.”

We also looked at two or more general satisfaction measures.  So respondents are asked to rate VA healthcare in the last 12 months on a scale of one to 10, with 10 being the best.  We compared those who answered nine or 10 to those who answered less than or equal to eight.  And respondents were asked about their satisfaction with the VA at their most recent visit.  That’s a (inaudible) scale of one to 7, the 7 being the most satisfied.  We compared those who answered six or seven versus those less than or equal to five.

So since all of the outcome measures are dichotomized, we ended up using a (inaudible) progression predicting patient satisfaction.  And again, our main explanatory variable of interest is the wait time measures.

So the wait time measures, as I said, are based on the 50 clinic stops that are used for performance measures.  These are high volume clinic stops that do, they capture patient provider interactions and cover all major medical specialties.

The wait times for facilities—All the wait times for the 50 clinic stops are averaged together for a month, and that includes the access measures.  So we’re not doing the percent of appointments that we’re seeing in 14 days.  And the wait time that’s assigned to each individual is matched to the visit date when the respondent was selected for the sample.

For ease of interpretation, we went ahead and we categorized all of the wait time measures into quartiles.  So for each measure, we are comparing the VA facilities that have the longest waits compared to the VA facilities, the 25 percent of VA facilities that have the shortest waits.

We also include some basic risk adjustors (inaudible), basic demographics such as sex and age.  Also, how often they used health care in the last 12 months, and self-reported health status, because those in poorer health may be less satisfied overall.

And so this slide is just giving you the overall descriptive statistics of the sample, and it shows that it basically follows the VA population.  Ninety-five percent of them are male.  Nearly a third of them had more than five visits to doctor’s offices in the last 12 months, so they’re heavy health care users, and they also have fairly poor health.  Seventy-five percent of them reported that their health was fair or poor in the last 12 months.

Despite that, they’re overall pretty satisfied with the VA.  Eighty-three percent said that they always or usually could get an appointment as soon as they wanted; 85 percent and 82 percent said that they were able to access the treatment, or test or a specialist as soon as they wanted or usually in the last 12 months.  And some nearly 80 percent, around 80 percent gave very highest ratings when it came to satisfaction with VA care.

So this slide just gives you the mean wait times on the different measures for new versus established patients.  For new patients, the first next available wait time and the time stamped create date, which is the retrospective measure and the access create date measure, all track fairly closely to each other, ranging from 20 days for first next available, to 16 days for the access create date measure.

The desired date measures are a lot lower.  The time stamped version of this desired date is about five days, and the accessed desired date version is 2.5 days.  For established patients, the create date versions are long, covering for both access and retrospective and prospective measure as hovering around 30 days.  The first next available is only about eight days for established patients, and that matches the prospective version of desired dates.  And then the time stamped version of desired date is even lower.  It’s two and a half days for established patients.  So there is a wide variety in these calculated wait times across measures.

So first I’m going to talk about the new patient results.  And we’re hypothesizing that longer waits are going to predict lower patient satisfaction.  So this slide just gives you the odds ratios for the first outcome which was how often were the individuals able to get an appointment as soon as they wanted?  And it gives the results for the first next available measure, the time stamp date date measure, and the access create date measure.

And the reference group is the VA facilities that had the shortest, the 25 percent of VA facilities with the shortest waits.  And as you can see that as individuals who are visiting these facilities but wait longer, their satisfaction decreases.  It’s a significant decrease, and it’s consistent across all three of these measures.

When you look at the desired date versions of these measures however, the story is (inaudible).  And for time stamped desired dates, there is not a clear relationship.  In fact it looks like individuals who end up visiting VA facilities with longer wait times are more satisfied, and for the access desired date version the results jump around, but again, those who are waiting longer appear to be more satisfied.

So when you look at all five outcomes for first next available time, the retrospective version of create date and the access version of create date for new patients, the same story that was found for timely appointment is found across the other four measures.  There seems to be a significant relationship between individuals who are using VA facilities where the wait time is longer.  They are significantly less satisfied compared to individuals who visit facilities where their wait times are in the shortest 25 percent.  That is true on each of these outcomes for each of these measures.

And once again when you look at the other measures from the desired date versions for new patients, the results are either significant in the wrong direction, or they’re telling, or they’re not significant at all.  So there is not a clear consensus of the effect of the desired date on these access measures for new patients.

So the first next available and the create date measures appear to be most reliable because they do predict satisfaction in the expected direction for all five satisfaction measures.  This concurs with the theory that new patients probably want to be seen as soon as possible.  So dates that an appointment request is originally made is probably pretty reliable when calculating wait times.

I’m now going to talk about the established patient results and again, I’m going to be hypothesizing that the longer waits are predicting lower satisfaction.  Again, this is showing you the results for the timely visits outcomes, and it’s showing you the first next available measure of the time stamp create date measure and the access create date measure.  And here the story isn’t as clear, as well.  The first next available for veterans who are visiting VA facilities with the longest, and the top 25 percent of facilities with the longest wait are significantly less satisfied compared to veterans who are visiting VA facilities with the shortest waits.

There is some evidence that for time stamped create dates you get lower satisfaction levels with longer waits, but it’s not a consistent story, and it certainly doesn’t hold up when you are looking at veterans who are visiting the VA facilities, 25 percent of VA facilities with the longest waits, and the same is true for the access, create date measure.

Moderator:  Julia, this is Todd.  We actually have a fair number of questions, and I don’t know if Steve is monitoring those, if this is a good point to jump in with questions or if you want to wait a bit?

Julia Prentice:  Can we wait just one more slide or two?

Moderator:  Sure.  Please.

Julia Prentice:  Two more slides, and we’ll—Three more slides and then we’ll do it.  So the desired date version for established patients, the time stamp desired date once again doesn’t tell the most consistent story, but the access desired date version is in the direction that was originally hypothesized in the expected.  There is a linear trend (inaudible) as veterans who visit VA facilities with longer waits are significantly less satisfied.

And I just want to show that (inaudible) results that are found for the timely appointment are holding up for all of the other outcomes.  So again, for first next available you have those veterans who are visiting the top 25 percent of facilities with the longest waits are significantly less satisfied than those who are visiting VA facilities with the shortest waits.  There is some evidence that the time stamp create date measure is leading to lower satisfaction, but it’s not consistent.  And that’s also true for the access create date measures.  They’re either not significant or it’s not a consistent relationship.

However, when you look at the access desired date version on all of the measures once again, you get this linear relationship where veterans who have long—Who are visiting those VA facilities with longer waits are significantly less satisfied for established patients.

So on the established patients it appears that this access list desired date is the most reliable.  And remember, this is the prospective version that includes no shows and cancellations.  And so since it’s unknown when patients, which patients are actually not going to show up in the end, or which appointments are later going to get canceled, we’re hypothesizing that the difference between the access list desired date versus the retrospective desired date version, the reason that the access list is more reliable is because it’s probably a more accurate measure of supply in the system when the veterans are actually requesting their appointments.

The results also suggest that the desired date reflects established patient preferences because the first next available and the create date measures were not consistent, were noisy in the consistent stories.  I can take questions.  That would be good now.

Moderator:  Steve, are you able to see the questions too?

Steve Pizer:  Yeah.  I’m able to see the questions.  I wasn’t sure about responding.  So the two of us can chime in at this point.

Moderator:  Okay.  Sounds great.  So the first question I’m going to raise is two questions.   And as people type these in, it’s very helpful for me as I read these if you could type in complete sentences.  Otherwise, I run the risk of reading too quickly and botching them.  So I apologize if I botch them ahead of time.

The data linking access to outcomes are primary care, specialty care or both?  And also, was there some threshold of access beyond which there was no further benefit on outcomes?

Julia Prentice:  I would type—For patient satisfaction outcomes, with primary care and specialty care, both.  And no.

Moderator:  And then a threshold?

Steve Pizer:  Yeah.  I think the idea of a threshold is a good question.  So we’re looking at quintiles here, right?

Julia Prentice:  Quartiles.

Steve Pizer:  Quartiles or quintiles?

Julia Prentice:  Quartiles.

Steve Pizer:  Quartiles, yeah.  So that is a real descriptive look and you can see some of the figures that Leah (phonetic) put up and begin to answer that question for yourself.  In some cases the relationship was pretty smooth, in other cases it’s jumping around.  I think at this stage of the research, we are looking at the different measures and trying to see some indication that there is a relationship that we would expect.  That is, we would strongly accept that people that have to wait longer are less satisfied.  And if we don’t see that, then we worry that the measure of wait time is (inaudible).  But we hope it does.

So that exercises (inaudible).  We’re not trying to (inaudible) yet.

Moderator:  Great.  The next question has to do with many VA MCs are using Press Ganey to collect more real time, more detailed information about patient satisfaction.  Why go with SHEPP (phonetic) if it’s only a yearly thing, when you can get more real time data and potentially link it to the patient level?

Steve Pizer:  This is interesting.

Julia Prentice:  Yes.

Steve Pizer:  I’m not aware of this.

Julia Prentice:  Of that.

Steve Pizer:  Julia, do you know more about it?

Julia Prentice:  No.  I don’t know about that.

Steve Pizer:  Yeah.  So you know, the advantage of SHEPP  is that it’s been around a long time, we know what’s in it, it’s national in scope, we can link it to the waiting time data.  And we can look at waiting times that are located in time approximately when the questions were asked.   We just don’t know about this other source of data.  I’d be interested in finding out more.

Moderator:  Yeah.  Heidi, I wasn’t sure if the person who asked that question has the ability to ask it over the phone or clarify over the phone?  I’m not familiar with what Press Ganey is either, so I would love more clarification on what that means?  But maybe Heidi is not able to see that.

Heidi:  Sorry.  I stepped away for a second.  What was it that you were looking for?

Moderator:  The question, the person asked about the Press Ganey.  I wasn’t sure if that person has a phone headset access so they could clarify their question?  They talk about the difference between using SHEPP versus using Press Ganey.  And I apologize, but the three of us, Julia, Steve and myself are not familiar with Press Ganey.  So we wanted to learn more about that.

Heidi:  Okay.  Sylvia, I’m going to unmute your phone.

Sylvia:  Hi, there.  This is Sylvia Hypher (phonetic) from Houston VA.  The Houston VA I know uses Press Ganey.  Press Ganey is actually a private company that provides—That’s one of the sort of main products that they have is actually doing a—They provide monthly reporting, but essentially their methodology is to send—They send a questionnaire to the patient, I think it’s something like within two weeks of their patient visit.  And I know (inaudible) as well, is also, has used Press Ganey and basically they provide monthly reporting of patient satisfaction information and they’ve got multiple—Their questionnaire has multiple sub-scales again, sort of including—Some of the things are kind of pretty basic, like sort of the cleanliness of the facility and things like that.  But certainly I know that wait time is one of the, is one of the factors, as well as sort of the relationship with your provider.  (Inaudible) some data that sort of looks at sort of the overall relationship between patient satisfaction and sort of various subscales.  And at least in the data set that I looked at sort of the driving force is actually the relationship with the providers—The driving force of overall patient satisfaction primarily is sort of their satisfaction with their provider.  That’s not to say that access was certainly not an issue.

Moderator:  Thanks for adding that.  I really—

Julia Prentice:  (Inaudible).

Moderator:  Yeah.  Thanks for adding that.  That’s very helpful.  I wasn’t aware of that data set or data source.  I don’t know if that’s helpful for you, Steve and Julia.

Steve Pizer:  I think we’ll ask about it where it’s potentially very interesting, especially if it’s done widely around the country.

Sylvia:  And a lot of other private, you know, like private sector  hospitals rely on Press Ganey also for that similar kind of data.  Thank you.

Moderator:  Yeah, thanks.  The next question is how are new and established patients differentiated?

Julia Prentice:  A new patient is defined as—This is also a new standard definition that is used for the performance measure.  So if a patient has not been to a clinic stop.  For instance, they have not been to primary care in the previous 24 months, they are defined as a new patient.

Moderator:  And then the next question has to do with the facility in your regression models and whether you control for it, and then if you do, what form you control for it, using either a random effect, a fixed affect, things like that?

Julia Prentice:  It was a cross section of the survey so we did not control for facility affects in these analysis.  We could not.

Moderator:  Is there any data that changing wait times affects patient satisfaction?  I mean, you sort of just answered that.

Julia Prentice:  Yeah.

Moderator:  (Inaudible) data (inaudible) person for example, or within facilities that would give you better information on that?

Steve Pizer:  We can’t do changes since it’s cross-sectional.  We don’t have—It’s not—We don’t have longitudinal data here.

Julia Prentice:  In other work, in our previous work that was linking the first next available measure to health outcomes, we consistently do it for the facility (inaudible) those measures, in that research.

Steve Pizer:  It’s a very sensible question and we do more longitudinal analysis when we can.  I think Julia is probably going to talk a little bit later about what the next steps are and one of the next steps is to look at some other outcomes other than patient satisfaction, where we can use administrative data to get a longitudinal view.

Moderator:  Sounds good.  There’s a couple of questions that we have about the model itself.  And so one model—Have you really tried—You started off here with dichotomizing use of logistic regression.  Have you thought about other models for possibly fractional (inaudible) or OLS that might be useful for (inaudible)?

Steve Pizer:  So correct me if I’m wrong Julia, but I think we’re following the lead of OQP here in dichotomizing the results from the satisfaction surveys.  That’s the way they told us they thought it was most appropriate to analyze these data, and we didn’t want to make them mad because we got the data under the conditions that we take their advice.  So we are.

Julia Prentice:  And they’re also—The variations in the outcome measures is such that dichotomizing—

Steve Pizer:  I’m sorry.  Say that again, Julia?  I didn’t quite catch it.

Julia Prentice:  The variation in the outcome measures is such that it made the most sense to dichotomize it anyway.  Because 80 percent are reporting they’re satisfied in the top two categories.

Steve Pizer:  Right.

Julia Prentice:  On each outcome measure.

Moderator:  Sounds good.  There’s a couple of questions that we have about model fit.  And can you speak to anything about your model fit and sort of what you sense here?  I don’t know if you’ve been able to do sort of the standard, either Pregabon (phonetic) or Hosmer-Lemeshaw type model fits, but there a questions about that.

Julia Prentice:  We have not done that.

Steve Pizer:  Right.  So there—

Moderator:  All right.

Julia Prentice:  Go ahead.

Steve Pizer:  So this gets back a little bit to what we’re trying to accomplish with the models.  Unlike a lot of other work that you might see, this isn’t a risk adjustment model or a classic outcome  model, and we’re not sure that we’re actually going to be able to explain much of what’s going on here.

Julia Prentice:  Right.

Steve Pizer:  So what we’re trying to do is find one relationship that we have a good conceptual reasons for expecting that we would find, and we’re having some difficulty finding it.  But we are finding it in most places.  So the bottom line is that the models don’t fit very well and in particular, in models where there is no relationship or not much of a relationship between the measured waiting time and satisfaction, there is not much about—That model is not fitting at all.  And that’s sort of the point of the exercise, is to try to find which ones (inaudible) at least a little bit.

Moderator:  (Inaudible).  Do you have any sense on whether the access issues and wait times are differential for people in their distance to the VA?

Julia Prentice:  We, in our previous work thank links the first next available measure to health outcomes, we do include distance, and that ends up being a non-significant.  It’s not significant in those models.  Distance doesn’t have an affect.  The priority status has an affect, has a large affect, because again, the official policy for a long time, and I think it still is, is that if you’re receiving a service for a 50 percent or more service-connected disability, you’re supposed to be able to get in right away.

Moderator:  Yeah, it’s interesting.  Because the different wait time models for distance might matter, especially if you think about the do not shows.

Julia Prentice:  Right.

Moderator:  So the people who live a long ways away might be much more likely to schedule the first one but not show.

Julia Prentice:  Right.

Moderator:  And so you could think about having a different effect that way.

Julia Prentice:  Right.

Moderator:  There is a question about why did you choose the first next available and not the third next available?  I thought the second was more of a true access measure.

Julia Prentice:  Yes, that’s an ongoing—In the private sector they definitely use their next available.  And the reason we used the first next available was that when we started this work several years ago, the VA had first next available, but not (inaudible) next available.  And so we did the—We did the initial work using first next available, and then we were trying to compare the first next available to the other measures in this project.  So we wanted to stick with something that we had tried, we had tested, and were familiar with.  But it is a common—It is a common understanding that third next available is a little more reliable.

Moderator:  Another asker.  Just curious.  Were there any facilities that looked abysmal by one measure of access and not much better by a different measure?  Or did the facility ranking stay pretty constant no matter which measure you used?

Julia Prentice:  I don’t—

Steve Pizer:  We’re still looking for some measures that we have, that we can establish some validity on other than—I mean, they all have face validity, but we’re trying to establish some outcome validity before we look at rankings.  Other interesting questions are, you know, can we learn something from the contrast between different measures?  But that’s all future.  That’s all down the road a little bit for us.

Moderator:  As I assume that linking wait times to fee basis status, I don’t know if you’re planning to do that, but one would expect that if you’ve got a long wait time you’re much more likely to use another provider, whether it’s fee basis or Medicare.

Steve Pizer:  Yes.  You’d expect that to be true.  We haven’t tried to figure that out yet.  Well actually, we do have some other papers on related questions and we have found some evidence that when wait times are long, people use more Medicare services.  But the affect wasn’t necessarily as huge as you might think.

Moderator:  You said there are only moderate substitutes, if you will?

Julia Prentice:  Yes.

Steve Pizer:  Yeah.  Actually, what we found was that people used more Medicare services but they didn’t necessarily use less VA services, which suggests that maybe if you can’t get in, you go elsewhere, but then you still have to come back and you end up possibly with more duplication.

Moderator:  Great.  I have to admit that we’ve rarely had a cyber seminar with this vocal an audience, so it’s great to have this many questions.  I think it just speaks to sort of how important this topic is, and I think everybody is very interested in it.  So I think that’s all the questions we have for right now.

Julia Prentice:  So I’ll just quickly finish up, then.  So I’ll—

Moderator:  Julia, if you could speak a little bit louder?  There are some people who are chiming in saying it’s a little bit hard, that you’re going in and out.  And I know that some people have voice access and so it makes it a little bit challenging, but if you could speak clear, that would be helpful.

Julia Prentice:  Okay.  How is this?  Better?

Moderator:  Yes.

Julia Prentice:  So as far as policy implications go,  for as much as we know right now, and we say this with knowing that there is still a lot to be done, you may need multiple measures of wait times, especially for new versus established patients, because the needs of those two subsets of populations is different, and when they want their appointments is different.

Survey data has confirmed that new patients want to be seen right away.  Usually it’s because they have a change in health status, and so they’re concerned about that change in health status so they want to get in as soon as possible.  So your capacity and your create date measures may be better for new patients.

Established patients though, the survey data suggests that they don’t necessarily prioritize how long they’re waiting.  They may care more about getting the provider that they normally see, or being able to schedule an appointment at a convenient time for them.  And so the VA is really the leader in recognizing this new versus established patient complexity.  Because as I said before, the private sector has, if they measure wait times at all, they focus (inaudible).  They focus on the capacity measures, and they don’t think about these other issues.

But this is just—We were noting, this is just one outcome of satisfaction, and our future work that’s going to go on and predict a variety of health outcomes we’re going to look at stroke, heart attack, mortality, preventable hospitalization and A1C, hemoglobin A1C levels, with the hope that some of these measures will consistently pop up again and again as being a reliable predictor of a wide variety of health outcomes, and that would better—That would help us set better policies on what measures to use.

And so we’ve been taking questions, but this just gives you the papers that we have published on the topic.  The one in the Journal of Health Economics is the one that looked at the substitution between waiting for the VA and whether or not you’re more likely to use Medicare.  And we can take other questions.

Moderator:  That’s all the questions that people have right now, Julia.  So I think we probably interrupted you to answer a ton of questions, but—

Julia Prentice:  That’s fine.

Moderator:  Just to let people know, we had almost 100 people on the phone, so other people might be typing questions now.  I would encourage that.  We have a few more minutes, but in the meantime, I just want to thank you so much for a great presentation.

Julia Prentice:  Thank you.

Moderator:  And thanks Steve, for joining in and answering.  And then there’s been people asking about slide availability.  Heidi, do you just want to say a moment about, there is a slide availability?

Heidi:  Yes.  Slides are available.  I did include a link in the reminder that was sent out this morning.  So I know most of the people received that, so you can go back out to that reminder.  Just click on the links.  They’ll pop right up.  Or if you can’t find that, just send it into the Q&A, I’ll give you the link there, or mail it into the Cyber Seminar box and we can always get them out to you there.

Moderator:  I think that’s it, Julia.  Thank you again, and thank you, Steve.  Thanks Heidi.

Heidi:  Thank you.  Thank you, Steve.

**[End of Recording]**

Cambridge Transcriptionswww.ctran.com

Cambridge Transcriptionswww.ctran.com