12/14/2011  Introduction to Effectiveness, Patient Preferences and Utilities HERC Cost Effectiveness Analysis Course Sinnott, Patsi

>> Great. Well welcome to everyone. I appreciate your interest in the topic. You'll notice that I have changed the title a little bit from the beginning. But that just reflects an update on the topics. And a firmer -- I mean a further consolidation of the discussion.

>> I am going to warn you in advance, that I have a cold and a cough. And if I think I am going to cough, rather than cough in your face, I will go to mute. So if I seem to disappear, that is what is going on. 

>> Just a quick overview about what we are doing. We are going to talk a little bit -- as a reminder -- about cost effectiveness analysis and what the outcomes can be, how to estimate QALYs. And this is an introduction to estimating QALYs. And then a more in depth discussion of preference and utility measurement for estimating QALYs. What the most frequently used systems are and some guidelines we developed at HERC about selecting measures for your study. 

>> I just want to make a note here that I am going to try to use the terminology from economics and cost effectiveness analysis very specifically. And that may be different from what you more casually would understand these words to use. So I just wanted to warn you. 

>> So the big challenge in cost effectiveness analysis kind of in the 30,000 foot view is, what are the costs and how to estimate the costs. What are the outcomes that you are trying to measure and how do you measure them. And then, what are the policy questions that are going to be addressed with the cost effectiveness analysis. What questions are you trying to answer for policy people. 

>> And it obviously is based on the fact that everything has limited resources. And policy people need to decide which programs to use in the most rational way but 

>> So the cost -- you are looking at both the cost of the intervention, but also the downstream benefits of the intervention. Your outcomes are things that can be measured in the period of time that the study goes on. And you have to capture them throughout the duration in which the outcomes will occur. And the outcomes may seem to vary through the duration of the intervention or the duration of the follow-up. In other words -- sometimes the intervention patient might be -- feel terrible, but in a short period of time would feel better. And those of the things that you need to capture in the cost effectiveness analysis. 

>> So just as a reminder -- the cost effective analysis compares the effectiveness and cost of two or more interventions. Using the incremental cost effectiveness ratio calculation. Anf that cost effectiveness analysis or cost utility analysis are done primarily from a societal perspective. And that is so that you can compare outcomes, compare programs with different outcomes. Again, back to the resource and allocation questions. 

>> So the benefit, or the effectiveness is defined by the benefit, or outcome. And these benefits are measured in a single scale in cost effectiveness analysis. A day in the hospital, or an infection avoided. A day free of anginal pain and a day in of improved quality of life. So obviously these outcome are more or less easy to measure. But if you have two programs -- one that measures a day in the hospital avoided and one that measures a day free of anginal pain -- it would be very difficult to compare these across the two programs. And that is really where QALYs come in. 

>> What outcomes can be used in the cost effectiveness analysis? Costs or cost savings. Hospital days or hospital days avoided. The VR 36. QALYs . Or infections avoided. 

>> We talked a little bit about hospital days avoided and possibly infections avoided. QALYs is the most often used measure across Programs. The costs or cost savings really are not part of a cost effectiveness analysis. The specifics of a cost or cost savings would be in a cost benefit analysis. And the VR 36 -- unless you transformed it to an economic tool, could not be used in a cost effectiveness analysis because it would be too complicated and it's missing the preference weight to it as it stands alone. 

>> So the cost or cost savings is really -- as I said -- a cost benefit analysis but when the costs or cost savings are on -- the costs are on both the numerator and denominator of the equation. Hospital days, yes. TVR36, no. QALYs, yes  and infections avoided, yes. 

>> So effectiveness is measured in natural units. Which is what we talked about -- the infection, anginal pain.  The natural units that we talked about but as I suggested earlier, there are more complicated outcomes than a simple hospital day avoided. And effectiveness can also be measured as a summary measure. That includes quality of life, quantity of life, and then weighted by societal preference for that quality of life- and this is the QALY. And I will talk a little bit more about the societal preference in a few slides.

>> So the QALY describes years of survival, adjusted for quality of life or preference. The QALY  can range from 0 to 1. Where 0 is perceived as the worst possible state of health or death and 1 equals a year in perfect health. So the next questions is how do you quantify the QALY. Let's say that you have one year in perfect health and means you have 1 QALY. And I have one year in good health -- that means I have a .8 QALYs. So the difference is .20 QALYs. You experience .20 more QALYs than I did last year, but this is a straightforward question. And not complicated and most interventions do not have simple effects on patients. For example, cancer treatments. And joint replacements. 

>> So let me just talk through the calculation of a QALY. I am using an example of a new cancer treatment versus the standard of care. Weights range from 0 to 1. And this reflects a two-year studywhere QALYs are measured every six months and this is a simple introductory calculation but you can see that both parties -- both groups started off feeling pretty good. The new treatment group felt horrible in the next six months while that group was undergoing cancer treatment. The usual care group felt about 50/50. And they -- and they are not getting any treatment. And the third six months, the intervention group is feeling better but the usual care group continues to feel worse. And in the last six months of the two years, the group getting the new treatments feels about a quarter -- feels the same way that the usual care group did in the last six months of their life. In the last six months in the two-year period the usual care group has passed away. 

>> So what we are doing in the calculations here, at the bottom, is calculating the QALYs -- the QALYs for half a year and then we are standardizing to one year. And we show that in the treatment group, they experience or they gained .268 QALYs in that year but and the usual care group experience .2065 QALYs in that year. 

>> So then we are going to take the next step. Is to calculate that ICER -- that is the cost of intervention minus the cost of usual care. And again here -- I have given you a hypothetical that all other costs are equal, which is not usually the case. And these costs are divided by the change or difference of quality of life experienced by the two groups. So this is a -- this comes out to $162,602 per QALY gained, just as a reminder the standard or the usual accepted measure of a cost effective intervention is that it costs somewhere between $50,000 and $100,000 per QALY gained. So I just wanted to review that measure before we go but forward. 

>> So then, we need a way to estimate these QALYs. And that's where reference measurement comes in. This estimation requires that you know all of the health states or qualitites of life that a patient might experience. You need to estimate how desirable these health states are and you need to know the duration of each health state. So in this way, you need to be able to define the stages of health that are related to the quality of life that each participant in the study experiences. And finally, you need to be able to take this description of the health states and in the duration of each health state and weight them by community preferences for each health state. In other words, how the regular community might perceive and value that health state at any one time. 

>> So why do we use community or individual preferences? I want to take you back to the cost effectiveness analysis and societal perspective. Again, we're trying to measure potentially an intervention versus usual care or an intervention -- or two interventions in order to compare the resource allocation purposes. And that means that we really need the communities perspective on the quality of life experience for these two or three scenarios. The only way we can do that is if we take a program or a measurement instrument that gives us a community perspective or has been evaluated by the community. 

>> So the basic methodology for deriving preferences, which are also called utilities for health states, is that the individuals -- and they might be patients, providers or community sample- and I will talk a little more about that -- they provide a personal reflection on the relative value of the different health states experienced. Basically it is a comparative ranking. And these comparative rankings are elicited through various tools. There are two methods -- direct and indirect. And these are methods to derive preferences. The direct method -- individuals are asked to choose or declare their preferences between their current health state and an alternative health scenario. They make their choices base on their own comprehensive health state, or the composite described to them. 

>> So for example, let's say this is you today. You are able to see, hear and speak normally. You require the help of another person to walk or get around, and require mechanical equipment as well. You are occasionally angry, iiratable, anxious and depressed. But mentally you are fine, you are able to eat, bathe, dress and use the toilet normally and you're free of pain and discomfort. So this health state is how a patient might feel on any one day. And if we can think about this a little bit, let's say you hate the idea of being dependent on another person to walk or get around, or require mechanical equipment. This is just a horrible kind of life outcome to you. But for me, I used to be a physical therapist and I don't think that is such a bad thing. So I might kind of rate my quality of life today at .95 out of 1. But you might rated at .80 today. 

>> And that is how differences in quality of life are measured. People are either -- they either experienced this in a study -- and I will show you how that works or you might -- if you are a community sample, be given this scenario to rate. For your own -- from your own perspective. So one direct method is the standard gamble. And you are given the choice of living the rest of your life in your current health state. Let's consider the previous health state that was described. Or you could take a pill that has risks. To be restored to perfect health. And then this scale represents the risk of death you are willing to bear in order to be restored to full health.  

>> So what happens is, the individual doing the standard gamble, is asked, how much risk of dying are you willing to take on to be restored to perfect health? Is a 10%? Is it 20 percent? Would you take on a 20% risk of death in order to be immediately restored to perfect health? That is how the standard gamble works. And each individual scores each health state. Either the one that they are experiencing or the one that they are reviewing on that basis. 

>> The other direct measure is the time trade-off. It is similar but you are being asked how much life are you willing to give up in order to be restored today to perfect health. And again, there is a balancing question about how much life you are willing to give up. And patients are guided to make this choice. And that becomes their score for the time trade-off. 

>> Indirect methods are used -- that individuals are asked to rate their preferences for separate domains of health states. scores are aggregated to create a composite health state. So in indirect methods, patients in a trial for example are responding to these questions. Physical functions. Social functioning. Mental health, etc... And by answering questions to these questions, a composite of their health state is recorded. And then these composites have been previously weighted by community samples to give them a preference rate. 

>> So which statement best describe you today? And if you think about your answer choices, you have a variation on the no problems, some problems and extreme problems. And this is from the indirect preference measurement estimate call the EQ-5D. 

>> So in each of these questions, let's have you answer -- how you are doing today. Are you having any problems with mobility? Do you have any pain? Are you feeling no problems, some problems or extreme problems with anxiety or depression? With self-care or usual activity? You can imagine that -- I can't remember right now how many different health states the EQ-5D measures or produces. But you could imagine that there are quite a few different combinations but there are 245 health states that are described with this particular instrument. The combination of no problems for mobility, some problems in pain, etc. etc. etc. So these composites then produce the preference weight for the quality of life measure. So when you look at these various instruments, these utility or preference measurement instruments vary in what dimensions or attributes are included. The size and nationality -- needed to establish weights -- although this  for folks in the United States, this is changing very rapidly. Most of these instruments have a sample weight from a US sample. 

>> The health states defined by the survey and also how the summary score is calculated. So the question for you is really what method to use -- a direct or indirect. Or some other method. And again there's this trade-off between sensitivity and burden. The sensitivity will be greatest in direct measurements and the burden will be greatest in the direct measurement and that burden is both with for patients as well as for the providers who has to administer the test. 

>> So your first step in choosing the methods will be to do a literature search about your condition of interest to find out what instruments have been used in the past. From the most burdensome to the least burdensome. I'm sorry -- the least to the most. Off the shelf utility values can be used from the literature. You can use indirect measures -- and these abbreviations reflect the four most commonly used instruments. You can use disease specific surveys during the trial and then transform them later to preferences. And then you can use direct measurement. 

>> Off the shelf weights are from the literature and most useful in decision modeling when you don't actually have patient experience in the model or in the trial to use in your model. The indirect measures are standard surveys. You want to be able to look at the studies in the population of interest but the drawbacks is that they may lack responsiveness. And I am going to go backwards a bit about that poll. And again, here are the five questions -- the five domains that the EQ-5D measures if you're interested an intervention that will have a significant effect on urinary incontinence. You might not pick up any changes using this instrument and that's one of the issues that you have to really consider when you're looking at the indirect measures, is how well your condition, the changes in the patient's experience are going to be captured in the instrument you choose. 

>> Went using disease specific quality of life instruments -- if you really are convinced there's no way you are going to capture change in the patient's quality of life with generic measures I just described, you might use the disease specific instrument alongside the generic measure. And if you want to use that to then extrapolate the QALYs, you would then take the health states created in the -- with the disease specific measure and use a community sample to value or declare your preferences for the various health states that you have experienced. 

>> There are some very important issues to attend to in managing the trial. Is often very difficult to describe the health states to a community respondent. And in any instrument that measures quality of life, there are a large number of possible health states and it is very expensive and time-consuming to establish the values. But again, for a condition for example like prostate cancer that has very specific quality-of-life impacts that are probably not captured with the generic, you want to do this disease specific instrument alongside the generic measure. And then the direct methods, the standard gamble and the time trade-off -- if the effects are complex and and effect multiple domains, and they are not captured in indirect and disease specific instruments, then you really need to go to direct methods. But again, some caveats. There is a high variance in the estimates. And it's not the community value specified by the book Introduction to health economics by gold etc. etc. And I am not giving you that title correctly. 

>> So these indirect methods -- these are the most commonly used. And you can see my -- the initials. The SF-6D is from the SF-36 and the VR-36 has been extrapolated to the VR-6D -- actually through the VR-12 to the VR-60 to capture the differences in those two measures. 

>> And -- let me tell you a little bit about each of these instruments. The HUI -- health utility Index has 41 questions, there are eight domains and 972,000 health states. And the basic -- basis of the domain weights originally were Canadian samples. And I am not positive right now if there is a US sample. But again, this is the scenario where the hypothetical health states that are created by combining elements of the eight domains of health are valued or are preferences declared by aCanadian community samples. And depending on the version, a patient can be asked to consider their health over the past two, three or four weeks. 

>> The EQ-5D has five domains. And you saw only three answers for each domain, there are 245 health states and there are US weight recently published. And EQ-5D asks patients to consider their health of the day of the interview or the day that they complete the instrument. 

>> The QWB, or quality of well-being scale -- there are two versions. The self-administered is more feasible. It 136 questions and 1215 health states defined. The community sample were primary care patients in San Diego. And this instrument asks respondents to consider their health over the past three days. 

>> And then the SF-6D is converted from the SF-36 or SF-12. There are six domains, there are 18,000 health states and again there is a new paper from Lou Casis and Aurthor Selem about translating the VR-12 into the VR-6D so they can be used in economic analysis. 

>> So here are a couple of important resources. In particular, Brazier et al. this is a terrific website for getting summary information about both indirect and indirect measures and another resource is the tufts CEA registry. Which gives you reports of the cost per QALY and the quality-of-life reported in various studies. It is also a summary of evidence that is of high quality, so you can go there and see if the condition you are interested in has been measured. And accepted essentially into this registry. And you can review the instruments that have been used for those trials listed there. And then we have a guidebook on our website that gives you again some more in-depth description of preference measurements. And also provides a summary of what had happened to 2007 in CSP trials. What utility measures have been used. And also gives you a summary of the conditions at that time and what instrument -- what conditions have been measured with what instruments. 

>> So Pattsi there was a request that you display the first set of references again. I believe -- 

>> And these are on the handouts as well. 

>> Great. And there was a question about how -- about the book that you referenced. And that's -- I believe -- 

>> It is the course text book. And I fumbled it -- it is the text developed by the US preventive services task force on cost effectiveness analysis. From the US public health service it is called cost effectiveness in health and medicine by Martha Gold, Joanna Seagle, Louise Russel and Milton Weinstein. And I would say again -- this is supposed to be your resource text for the entire course. Let me read it again -- it is cost effectiveness in health and medicine, by Martha R. Gold. From Oxford press. 

>> We have another question. Can you talk about measuring lost productivity and cost effectiveness analysis? And whether these measures are sensitive to it. And there will was a suggestion that perhaps you could talk about the SCIVIP project. 

>> So, in general, the recommendation from the Gold book, et al and that committee, is that QALYs are supposed to include lost productivity. So they do not calculate lost productivity in the benefit or the effectiveness, or the outcome, of the patient. So that is pretty straightforward. And in that particular study, we have not included lost productivity for that reason. 

>> Okay. I don't see anymore questions coming up. 

>> Jane Garvin has a question. 

>> I don't see that one right now but there is a follow-up to the question about lost productivity. But are these measures sensitive to changes in employment? 

>> Well, in this particular study, that a colleague is referencing, we measured QALYs in two groups of patients. One is a group of veterans with spinal cord injury who -- I'm sorry. Let me start over. There are two groups of veterans with spinal cord injuries. They both were recruited to participate in this study which provided what's called supported employment. Which is a quite enhanced vocational rehab program. To assist the veterans to find work. And the findings of the effectiveness study found that the people who received supported employment were either three or four times more likely to get a new job. Or get a job after the intervention than the group that did not receive the intervention. So one would expect that quality-of-life might be very different it is two groups. One group that went to work and one that didn't. 

>> But what we found is that there are no differences in quality of life as reported. And no difference in QALYs estimated between the two groups. Which make us think  -- and I think the question is, what is it we think Is happening? Is it that the quality-of-life really is not different for the patients? Or the veterans who started a new job? Or is it that maybe the instrument was not sensitive enough. To detect the kinds of changes that we might expect to see. If you think about a veteran with a spinal cord injury, the changes you would expect are primarily mental health. But you might also see some in ability to interact with other people. And again, we are talking about psychosocial issues associated with getting out and getting a job. Well these should be captured in the instrument that we used which was the VR-36 to capture quality of life which was then transformed to the VR-6D to estimate QALYs. So this is kind of a conundrum that we cannot really answer. Except to assume on one hand that there is not much changing quality-of-life, or QALYs. Or that the instrument was not sensitive enough to detect the changes. 

>> There are two questions here. Are the QALYs  differences confirmed with statistical tests or just nominal differences? 

>> Statistical tests. 

>> And are geographic benefit plan treatment duration factors -- are these used in cost effectiveness analysis? 

>> Would you repeat that again. 

>> Sure. Are geographic benefit plan or treatment duration factors used in cost effectiveness analysis?

>> Well they certainly are in the cost of the care. But I'm not sure I understand what you mean about the qualities of the preference measurement. Maybe that person could raise her hand and we could open that microphone. 

>> It depends whether they are using the computer or phone. 

>> So do you have any thoughts on the question? So let me talk out loud a little bit. So that question sounds like people are concerned about variation in the non-VA coverage. Outside of the VA. And how would you capture differences in the benefits that are available to an individual in order to estimate the cost. And obviously is a big concern outside the VA because there is a lot of variation. I hope I have answered that question. Are there any more? 

>> I don't see any more. 

>> Okay. I am happy to answer any were questions but either directed by e-mail.