**Transcript of Cyberseminar**

**HERC Cost Effectiveness Analysis Course**

**Introduction to Effectiveness, Patient Preferences and Utilities**

**Presenter: Patsi Sinnott, PT, Ph.D., MPH**

**October 31, 2012**

**Introduction to Effectiveness, Patient Preferences and Utilities**

**October 31, 2012**

Dr. Patsi Sinnott:So good morning – good afternoon on Halloween and the large parade that is going to be held in downtown San Francisco for the fabulous World Series champions San Francisco Giants. Welcome to our course and our Introduction to QALYs and Preference Measurement. Let me see.

I am going to go to slide show, I hope.

Heidi:Yep, you can do it there or there is a button at the bottom of the pane, the button next to the – where it says 68 percentile also. Get down a little bit further. So you can move that out of the way. Go down to the bottom of your screen. [Laughter] Just to the left and down a little bit to the left. Right. Right there. Yep. Just click on that.

Dr. Patsi Sinnott:I did. 

Heidi:Give it another try.

Dr. Patsi Sinnott:There we go!

Heidi:[Laughter]

Dr. Patsi Sinnott:Okey-dokey. And so, I am sorry. I am forwarding the slides. Page down. Ah. Here we go. So, I am going to do a brief review of a cost effectiveness analysis, and basically, the concept of the ICER, talk a little bit about outcomes in this cost effectiveness analysis and the concept of QALYs, a general overview of how to estimate QALYs, and some guidelines on selecting measures.

Just a – uh. So you remember that the cost effectiveness analysis compares the outcomes and costs of two or more interventions where the difference in cost is measured and divided by the difference in outcomes to define the incremental cost effectiveness ratio.

Generally, the outcomes are measured in natural units and if you do not have natural units or choose not to use natural units, then you will need something else for the outcomes that captures the effect of the intervention.

Additionally, if you need to compare this incremental cost effectiveness ratio across programs for policy purposes, you will need to do the analysis from the societal perspective.

So from the policy perspective and really who – what is the focus of your cost effectiveness analysis, you need to define who your audience is and how the results will be used, and then define from that what perspective you should use.

And you know the Gold book, which is our textbook for this class, recommends that we use the societal perspective; and that is so that interventions and policy decisions can be compared across different interventions that have different effects. And you need to be able to standardize the outcomes to something that is useful across several or many programs.

The cost effectiveness analysis compares the outcomes of two or more interventions. The outcome is defined by the health benefit and the outcomes are quantified in a single scale. So for example, if your perspective is your hospital system, you might be talking about bed days of care as your outcome. In other words, is there a difference in bed days of care by adopting the intervention? Is the cost of adopting the intervention worth it in savings of bed days of care?

But if you are trying to compare bed days of care and immunization, then you recognize that you have a problem comparing bed days of care to, for example, flu cases avoided.

So let us consider a scenario, which is to reduce or eliminate post-op infections in patients following hip fracture. What are the costs you are going to include in this evaluation? And this is a bit of a review. You might put your answers in the – is it the question list, Heidi?

Heidi: In the – yep, in the Questions portion and those will pop up and Jean will be able to read those on the call. For those of you wondering, the Questions pane is on that dashboard on the right-hand side of your screen. Just click on that orange arrow at the upper right-hand corner of your screen to open it up and you can type in right in the Questions portion.

Dr. Patsi Sinnott:And if you go ahead ...

Jean:We do not have any responses yet.

Dr. Patsi Sinnott:Okay? So again we are looking at what costs to include in this scenario, where you have an intervention to reduce or eliminate post-op infections following hip fracture repair.

Jean:Still no responses. So somebody wrote in prescription drug costs.

Dr. Patsi Sinnott:Drug costs, yep.

Jean:Somebody else wrote medication, surgical equipment, and manpower.

Dr. Patsi Sinnott:Right.

Jean:Another response is cost of intervention per se including drugs, length of stay.

Dr. Patsi Sinnott:Right.

Jean:Another response says additional supplies for infection control. Lost work time. Antibiotic stewardship. I am not sure what that is. Followup visit costs.

Dr. Patsi Sinnott:And I might throw out there post-acute hospitalization for a [inaudible] or long-term care. I might also add differences in DME, for example, home care visits, things of that nature. And again, remember, if you are looking at an intervention from the societal perspective, you are also going to be looking at differences in caregiver time and travel time and expense in your cost analysis. Now generally we do not include lost work time in the costs following the recommendations—I am sorry—in the outcomes because following the recommendations from the Gold panel.

And then the question is, what outcomes are you going to use? How are you going to measure the outcome? And let us throw out some ideas about what the outcomes are that you might be measuring with this scenario. For example, you might count re-hospitalization. You might count repeat repairs. What else might you use as an outcome? And again let us put those in the question.

Jean:Somebody responded, avoided hospital-acquired infection.

Dr. Patsi Sinnott:Avoided infections. Correct.

Jean:Delays in commencement of post-op rehab therapy.

Dr. Patsi Sinnott:Right.

Jean:There are not any other responses right now.

Dr. Patsi Sinnott:You might also look at mortality and the duration of life. You might also look at pneumonia events and ... I think I said repeat repairs. 

Jean:Somebody ...

Dr. Patsi Sinnott:Mm hm?

Jean:... so a bunch of responses that just came in. People wrote in mobility, physical function, bed days, quality of life, early mobility, removal of urinary catheter within 24 hours, and follow-up wound care.

Dr. Patsi Sinnott:Great, great. So if we are looking at outcomes for the cost effectiveness analysis from the perspective of the payer or the hospital system, you might use those natural units to value and to estimate the incremental cost effectiveness ratio in let us say cost per repeated repair saved, or cost per pneumonia cases avoided.

But if you are taking an intervention to reduce post-op infections in patients following hip repair and comparing it to an intervention that, let us say, reduces hospitalization due to diabetes, then you can see how difficult it might be to compare outcomes across these scenarios. And you understand that a simple outcome like these natural units is not going to solve the problem or help you analyze the problem because your outcome is particular to the condition of interest, or the conditions of interest that you are trying to compare will not be standard. Or will not become.

So again the – your outcomes for the cost effectiveness—you have the payer system, the societal ... now suddenly I cannot get it to move. I see. So now I am trying to go forward. Okay. And from the societal perspective again comparing across programs, you need to have a measure that quantifies the length of the illness or the length of life and the quality of that life. 

And that is where the QALY comes in, the quality-adjusted life year. This describes the duration of illness or years of survival adjusted for the quality of life experienced during that survival.

The QALY can range from 0 to 1 where one is perceived as the worst possible health state or death, and one is a year in perfect health.

So the next question is how to quantify the quality or outcome of interest. So let us consider. You have one year in perfect health, therefore your QALY is 1 and I have one year in good health at .80 QALYs. Then you and I compared have a difference in QALYs of .20 QALYs.

This is a straightforward question, but most interventions do not have these simple effects on patients, for example with cancer treatments and joint replacements.

So a QALY requires a description of the health states experienced by the patients and the subjects in the trial, an estimation of the duration of each health state, and a comparison to or assessment of individual or community preferences for each health state. And that is a weighting of each health state based on a community response to a description of this health state.

So here is an example. We have a new cancer treatment versus standard of care. The weights range from 0 to 1. With the new treatment in the first six months, the patient feels pretty good at 0.9 QALYs. The next six months in the depths of treatment, they feel horrible, so their QALYs are 0.3. The next six months they feel about 50-50 and the last six months they feel about a quarter of perfect health.

And you see that in this scenario, on the average the patients who received the new treatment lived two years. And so the calculation for the total QALYs experienced during that time is the calculation of .9 times a half a year, .3 times a half a year, .7 times a half a year, and .25 times a half a year. I am not sure where the .7 came from. Let us assume that it is correct. And the total QALYs for that period of time are .268.

Now the usual care patients live a year and a half, but the QALYs experienced over the two years are calculated, and you see that the QALYs for the patient who receives the new treatment are greater than the QALYs over two years experienced by the usual care patients.

So then you calculate your ICER and we are assuming the new treatment is $10,000, the usual care treatment is zero. The calculation of the QALYs is .268 - .2065 and you see that your calculation per QALY is $162,000. So that is the general concept of calculating a QALY.

So the basic methodology for deriving preferences or utilities or QALYs for various health states are that individuals provide a personal reflection on the relative value of different health states experienced or described. These values or preferences can be provided by patients, by providers, or a community sample.

There are three general methods to derive preferences: off the shelf, direct, and indirect.

Off the shelf values use preference weights determined in previous studies for the health state of interest and these are useful in decision modeling. And we had a class several weeks ago about decision modeling.

And this, for example, is relatively straightforward using diabetes health states, but not all health states have been characterized.

In a direct method, individuals are asked to choose between their current state and alternate health status scenarios. And they make it based on their own experience at any one time. And generally if these methods are used during a study, they are asked multiple times throughout the study.

So a health state in the aggregate would ask – you might describe that you are able to see but you need help of another person and a cane to get around. Unlike anyone else, you are occasionally angry, irritable, anxious, and depressed. You are able to learn and remember normally. You are able to eat, bathe, dress, and use the toilet. And you are free of pain. So in this particular person requires the help of another person to get around and a cane and occasionally gets cranky and depressed. 

So let us say that Todd and I both were responding to this scenario. Todd might be extremely annoyed because he needed the help of another person and a cane to walk around. And I, having been a physical therapist, would regard that as perfectly acceptable. So my response to this might be that I am – I feel about .8 of my 100 percent, but Todd might feel .65, for example. 

So this is just how to express how people respond differently.

So in these direct methods, the standard gamble presents a question to an individual in the study, and the question is an alternative of living the rest of your life in your current state of health or take a pill to be restored to perfect health. And the question really is, how much chance of dying or risk of death would you be willing to take to be restored to perfect health?

In the previous example, Todd might be willing to risk a 40 percent chance of dying and I would only be willing to risk a 5 percent.

So this is how the questions are presented to patients; and they are presented to patients repeatedly until the patient agrees that the chance of perfect health equals the chance of dying at this time.

So really, the question is, how much would you be willing to take to be restored to your perfect health at any one time? Would you be willing to take a risk of death of 10 percent or 20 percent or actually no percent? And how might your responses be different from other people, your friends, spouse or partner, or your parents and someone who is 60 versus someone who is 80? You can see how this would – responses would be different based on a number of characteristics of the respondents.

Another direct method is the Time Trade-Off and rather than a risk of death, what you are presented with is an alternative between a number of years of life—in other words, this period from t1 to t2, how much of your current health life would you be willing to give up to be restored to 100 percent of your health? Would you be willing to give up five years, ten years or no years? And again, would your responses be different from your spouse, partner, your parents, et cetera?

You may need to use the direct methods, the Standard Gamble and Time Trade-Off, if the effects of the intervention are complex and they affect multiple domains, domains meaning physical, mental, emotional, and the effects are not captured in indirect or disease-specific interventions.

Direct methods have high variance in the estimates from patients in a trial and they reflect risk aversion, feeling about disability, for example. What is the difference between Todd and me and the pain. 

And because of high variance you really need a large sample size to reduce that variance.

And it is not the “community value” specified by Gold.

In indirect methods, basically you use survey instruments that have been developed specifically for this purpose and then values or weights have been assessed using community samples in various parts of the world. The surveys include multiple domains of health and the community sample is presented with composite descriptions of health stati. And then they value it and then the values are assigned to those health states. And then when the respondents in a study use the indirect methods, the respondents’ responses are linked to the community responses.

So here is another question and these are questions from the EQ-5D, and the EQ-5D is one of the indirect methods and is very commonly used in cost effectiveness analysis. It is very simple to use. 

There are five domains: mobility, pain, anxiety/depression, self-care, and usual activity. And there are only three answers. You have no problems, some problems, or extreme problems with your mobility today. 

And so let us go ahead and have you respond in the answer area. How do you feel in mobility? Pain? Anxiety and depression? Self-care and usual activities?

Jean:Would you want them to write in one of the three responses?

Dr. Patsi Sinnott:Right.

Jean:Of no problems, some problems, extreme problems?

Dr. Patsi Sinnott:Right. For mobility first.

Jean:And somebody asked, can you please name the instrument again for the indirect method.

Dr. Patsi Sinnott:I am going to name all of them in a minute. But this is just a sample question for you guys to respond to. I will outline all of them in a minute.

Jean:Like I have about ten responses that say no problem.

Dr. Patsi Sinnott:Okay. And what about pain? Anybody with pain?

Jean:The first ones are saying no problem. Some problems.

Dr. Patsi Sinnott:Right. Extreme problems.

Jean:Nobody said extreme.

Dr. Patsi Sinnott:So if you look at this, the question is really you can see that someone who is recovering from an intervention or a surgery or has mental health problems, they will respond to these questions differently than most of us will, and that there is a chance for many, many different health states described. In other words, someone could have no problems with mobility or pain but be extremely anxious, unable to do self-care, and have some problems with usual activity.

And the various combinations tell you how many health states that a measure might be able to provide for you.

So the indirect measures that are currently used are the HUI, the Health Utility Index; the EuroQol EQ-5D—and those are the questions from the EQ-5D that we just looked at; the Quality of Well-Being Scale, the QWB; and also the QWBSA, which means self-administered; and SF-6D.

These indirect measures vary in their dimensions, in other words their mobility, anxiety, self-care domains. And the size of the sample and the nationality of the sample who were – the community sample who established the weights—they vary in the kind of health states and the number of health states defined by the survey, and how the summary score is calculated. In other words, what I am telling you is that you are not likely to get the same answer from these measures across indirect measures.

So these are widely used. You want to pick one that has been used in the literature in your population of interest. But you may also find that let us say that the population was a group of people who had spinal cord injury and it was in a work-related or a job-training program. You can see that the EQ-5D may or may not be responsive in that population, so you really want to go back to the literature and find out what measure is used in the population of interest.

You might also find that if you were working on an intervention that was going to increase functional performance of the upper extremity, the EQ-5D also might not be responsive in your population.

So a brief overview of the various indirect measures. HUI has 41 questions. There are eight domains of health and 972,000 health states. And the basis of the domain weights was a community sample from Canada and they used utility theory to define the preferences. And depending on the version, patients can be asked to consider their health over the past two, three, or four weeks.

The EQ-5D has five questions, 245 health states, in both British and U.S. weights. And the EQ-5D asks participants to consider to consider their health during the day of the interview.

The QWB has two versions. The SA is more feasible – the self-administered is more feasible. They are “only” 76 questions. This is still very time-consuming; 1,215 health states defined, symptoms, mobility, physical activity, and social activity. The domain weights were established with a community sample from primary care patients in San Diego.

And the SF-6D converts SF-36 or SF-12 to utilities. There are six health domains in the SF-6D. There are 18,000 health states and the basis of the domain weights for the SF-6D is a British community sample. But we also have access to the VR-36 and now a translation to the VR-6D, which is the veteran adaptation to the SF-36. So this would be weighting appropriate for use on a veteran population.

You can also use disease-specific surveys. Now just to remind you that these disease-specific surveys like the FACP or the Oswestry back pain inventories or any of the other disease-specific measures are designed to measure changes in the condition and the consequences of the condition for the patient. But they are not designed to be used in economic evaluation. So the question is, when are you going to use them? And generally these are used alongside other economic tools, the HUI and the QWB.

And if you want to then translate the health statuses defined by your population using this disease-specific instrument, you would then—sorry about that—you would then take the health states defined by your population and have a community sample then assess the value or preference for the health states defined in the health states for your disease-specific. But you understand that that is extremely time-consuming and expensive.

So ... mm hm?

Jean:How does one convert the SF-12 to utilities?

Dr. Patsi Sinnott:There is an algorithm to convert it to the SF-6D. It is available on our website. Not the algorithm, but the link. And there are published methods by Brazier to do that translation. And then we also have a new translation, VR-12 to the VR-6D from Lew Kazis and his group. Again, the disease-specific surveys, they are often used in addition to generic measures.

So the question then is which method to use? There is this trade-off between what is sensitive or responsive and both patient and staff burden in terms of time. You can imagine that if the survey is very time-consuming that patients with repeated surveys would become less interested in participating. So you really, as I said before, start with a literature search re the condition of interest.

And I just want to mention that in the Preference Measurement Guidebook on the HERC website, we have a summary through I think 2006 of the instruments and the conditions that they are reflecting. You can start there and then do a more recent search to look at who has been using what instruments with what conditions since then.

In terms of a hierarchy of methods, the easiest thing to use and the least burdensome is the off-the-shelf utility values. But again I mention that these are not available for all conditions. 

The indirect measures are very easy to deploy in a study. Disease-specific surveys during a trial and translation to preferences using a community sample is complicated and time-consuming and direct measures, the standard gamble and the time trade-off, are very burdensome both for patients and for the staff. And if you are going to use software, for example, on a laptop for the patients to complete these surveys, you have to think in your study about the life of that laptop and how you might have a problem if the laptop decides that after three to four years it really does not want to do this anymore. Because that is what laptops do.

So here are some important resources. The Tufts Center for Risk Assessment has a resource to look at cost per QALY as background again to see what kind of research has been done on your condition of interest. This review of the use of health status measures in economic evaluation from Brazier and Deverill at the Health Technology Assessment program in the United Kingdom is a very extensive resource and gives a lot of information that will be helpful for you.

There is also a table of published utility weights at the Tufts website and also we have a guidebook at HERC that provides a summary of the research that has been done using preference measurement in CSP trials, a summary of the literature by condition, and a summary of the responsiveness, the test/re-test validity, and then the ceiling and floor effects as reported in the literature through 2007. So that is also a good place to start and again that is on our website.

So here we are. Do we have questions?

Jean:Not right now.

Dr. Patsi Sinnott:So again, HERC has a guidebook in publications on our website that you can download. It is a public site. It is not a private VA site.

Jean:Somebody just posted a question. They want to know, can we get an example of the off-the-shelf? They have off-the-shelf utility.

Dr. Patsi Sinnott:Yeah. Just a sec. I have one on my [inaudible]. So if you go to RTI Press, you have a – there is a validation of the CDC RTI Diabetes Cost Effectiveness model. And basically they have established the utility weights for each level of condition associated with diabetes, for example renal disease, neuropathy, retinopathy, cardiovascular disease, and mortality. So these values for patients with diabetes can be used in a decision model. And I think if you go back to the presentation on decision modeling, he also has looked at the resources. Basically, what you are doing is looking in the literature to see if there is a valid report of the preference weight or the QALYs experienced by patients with a particular level of the condition of interest. So patients with diabetes with neuropathy. So that would be used in your decision model. 

Any other questions?

Jean:Somebody made a comment: the links are not working.

Dr. Patsi Sinnott:On the slides?

Jean:I think that is what he was referring to.

Dr. Patsi Sinnott:Sorry. I have ...

Jean:So we will have to double-check that.

Dr. Patsi Sinnott:We will have to check that.

Jean:Yeah.

Dr. Patsi Sinnott:Yeah. Sorry about that.

Jean:Someone else asked: I am on the HERC website. I cannot find the description of instruments for different diseases. Can you be more specific?

Dr. Patsi Sinnott:So go into Publications and then Guidebooks and then find the Guidebook Preference Measurement in Economic Analysis ... and ... go to page 19 and 20 and you will see first of all the test/re-test reliability. And then on Table 4, page 22, use of multi-attribute health status classification systems by clinical category and condition.

Jean:Okay. Great. Another few questions in here. So somebody else asked, can you use the utility weights for different states, and this is not an analytic model, from different studies using different instruments?

Dr. Patsi Sinnott:I would say no. But I would rely on people who do decision modeling a lot more than I do, which is ... I have not.

Jean:Okay.

Dr. Patsi Sinnott:But I think that the issues is that if you used the five indirect methods at any one time, all at the same time, you would get a different score on each of them from a single patient. So you would want to look at the utility scores in a single instrument across the duration of the patient’s life for a decision model. Or you would want to be able to do a valid cross-link between them. And I am not familiar with doing that.

Jean:Okay. The next question asks: What are the presenter’s thoughts on use of stated preference methods, contingent evaluation, contingent behavior, choice experiments versus standard gamble [overlapping voice] ...

Dr. Patsi Sinnott:I have not used it.

Jean:... and time trade-off?

Dr. Patsi Sinnott:I have not used them. I am sorry. This is ...

Jean:The next question asks: What is the justification for using a community sample preference in a population that may have adapted to the disability and might reveal different preferences?

Dr. Patsi Sinnott:Well, in a societal evaluation, what you are trying to do is use the perspective of a standard group of people across all conditions. So you would not want to use a population who have accommodated to their condition in particular for a policy purpose. Let us say that you have a population of people with spinal cord injury and their spinal cord injury is at least five years old. And in general this population has accommodated somewhat to their injury. And their improvement, due to an intervention, might be small because it does not take into effect the fact that they are already disabled and mobility-impaired. And if you try to compare an intervention for a population with spinal cord injury to a population of people with diabetes, if the changes in quality of life are not measured by a constant sample, you are going to get very different evaluations of improvement due to the intervention. So you use a community sample to standardize the weights and then link patient responses to the changes inQALYs and the weights of the community sample. I hope that answered the question.

Jean:Yeah, I think so. The next question asks: We are planning on having a journal club to review and critique articles on cost effectiveness analysis. Any food for thought on what we need to be focusing on when we are viewing and critiquing articles for journal club discussion?

Dr. Patsi Sinnott:Hmm. Well. There are—let me see if I can remember the paper. There is a series of papers that defines a quality of a cost effectiveness analysis paper. It started with I think BMJ and then I think it was JAMA and they both created through normal processes a way to evaluate the quality of a cost effectiveness analysis manuscript. And I am sorry; I do not have this on the tip of my tongue. I have used it in the past. But I would be happy to send you the link. That information from those two groups has been synthesized into a field agreement on what is the quality of a cost effectiveness analysis. I can tell you and maybe Jean can chime in—it gives you the areas to discuss in evaluating a cost effectiveness analysis. I am not sure it is very helpful to actually rank or score them. Jean and I were involved in a project that was trying to score economics—cost effectiveness analysis papers in physical therapy and we were not finding a lot of difference between the papers when you used scoring mechanisms. But I think that the areas for scoring are a usable way to lead or to direct your discussion, and I would be happy to send you that info if you send me a message. And I will send you the link back.

Jean:Yep. So he actually did respond with his email address. 

Dr. Patsi Sinnott:Great.

Jean:And somebody else responded that ISPOR SMDM, which I think is the Society for Medical Decision-Making, released a series of papers for good practices research.

Dr. Patsi Sinnott:Mm hm.

Jean:And he also provided a website: www.ispor.org.

Dr. Patsi Sinnott:Correct.

Jean:For his articles. Okay, it looks like there is one last question. It says: When we interpret a utility score, are we assuming this is for one year?

Dr. Patsi Sinnott:Yes. In other words, what you do is you standardize to a year. So it is a quality of life year that you are looking at. Quality-adjusted life year.

Jean:It looks like that is all the questions.

Dr. Patsi Sinnott:Okay. Thanks very much and the next talk is Vilija Joyce, who will be presenting on Measuring QALYS over Longitudinally, although I do not have the title completely perfect.

Heidi:Oh, the title I have here, Modeling Health-Related Quality of Life over Time.

Dr. Patsi Sinnott:Mm hm.

Heidi:I think that is it. Fantastic. And for our audience, we will be sending you registration information for that session in about a week. Keep an eye on your email and we hope to see you at that future HSR&D Cyber seminar. Thank you for joining today. And Patsi and Jean, thank you so much for taking the time to put this together and present today. We really appreciate all the time you put into it.

Dr. Patsi Sinnott:Well, thank you for managing all of this.

Heidi:Of course. I am happy to do it. Thank you all. We will see you at a future HSR&D Cyber Seminar. Thank you.

Page 12 of 12