**Transript of Cyberseminar**

**HERC Cost Effectiveness Analysis Course**

**Introduction to Effectiveness, Patient Preferences, and Utilities**

**Presenters: Patsi Sinnott, PT, PhD, MPH**

**May 14, 2014**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact herc@va.gov.*

*Patsi Sinnott:*Good morning and good afternoon, wherever you are.  We're here in California and it’s 95 degrees, which is really unusual.  I wanted to welcome you to the cyber course from HERC and the lecture introducing QALYs and preference measurement in cost effectiveness analysis.

I'm sorry, how do I do next slide?

*Moderator:*At the lower left hand corner of your screen.

*Patsi Sinnott:*Oh, I see, yes.  Perfect.  So we're going to talk about the ICER, just do a brief review, talk a little bit about outcomes and the concept of outcomes in QALY measurement, how to estimate QALYs and provide some guidelines in selecting measures and some important references that will be helpful to you as you move forward.

I just want to emphasize, this is an introduction and overview so that, if you were looking for something more complicated, this is not the course.

You remember the ICER.  In a cost effectiveness analysis, you compare the outcomes and costs of two or more interventions.  If you can’t measure the outcomes in natural units, which is what you might do, then you will need to use some other measure that captures the effect of the intervention.  If you need to measure across several programs where the outcomes are different, you'll need to find a measure that provides a common benefit across the program.

As you consider your cost effectiveness analysis and your QALY and outcome measurement, you have to keep in mind the policy implications of your study.  Who is your audience?  How will the results be used and what perspective should you use?  If this is a resource allocation question at Vaco, then you need the societal perspective.  If this is a question at the facility level, then you need to be using the facility perspective.

The outcome here is, it provides you with a health benefit, you have to be able to measure it, and they're measured and quantified in a single scale.  You need to be able to measure, for example, if you have an initiative that’s supposed to detect bone mineral loss.  You need to compare the cost of the intervention to the cost of usual care and compare the outcomes; again, back to the ICER equation.

Let’s consider a scenario where you have, you are going to be testing an intervention, a medication that’s going to reduce or eliminate postop infections in patients following hip fracture repair.  Now, you all have talked already about the costs to include, and I wonder if we might post these on this slide.  Is that possible, Heidi?

*Moderator:*Well, what I can do is—I can’t do it on this slide, but I can bring up a wipe board right here.  Let me give people access.  Now everyone is able to type on here—I'm sorry, Patsi, I can’t remember what the question was.

*Patsi Sinnott:*The question is, what are the costs you might include in your cost effectiveness analysis for this scenario, where you are producing, using an antibiotic, presumably, to reduce or eliminate postop infections in patients following hip fracture repair.

*Moderator:*At the top of the screen, there is—yep, like on the T at the top, and then just go down to the screen and you can type right on there.  A little bit easier than using the pencil tool.

*Patsi Sinnott:*For example, you would be interested in the difference in costs of the drug that you're using to treat patients.  Very good—medical costs, pharmacy costs, patient costs, including the home care, caretaker costs, transportation, staff training costs, direct medical costs, travel facilities.  I think we have all of those.  We might be looking at the costs in hospital care and in follow up visits, which might include physical therapy, and you would be comparing the costs in the two groups—those who received the intervention and those who did not.

Cost of decolonization preoperatively—that assumes I believe that people were colonized preoperatively.  Again, what we're looking at is the stream of costs that are experienced or produced by the patients in a control arm compared to the patients who received the intervention.  

If we clear this wipe board or go to a new one, what kinds of outcomes can we be using to look at natural units when we're comparing two interventions, the usual care, and a prophylactic antibiotic?  In this scenario, you might be concerned if the patient’s—yes, pain— that the patients might be re- hospitalized because of infections.  Patients might need more home care, adverse drug events, all right.  Dementia, drug resistance, requiring repeat repair, and then again avoided infections, delays to rehab which might involve your duration of care in the acute hospital, but also affect your rehab hospital costs, and we have someone who’s included the QALY.

In these kind of outcomes, what you're looking to compare is the effect of the intervention compared to control, but recognize that the cost effectiveness analysis is going to give you a cost per days to mortality or a cost per day of care in the inpatient facility.  But if you're also trying—if you're in a resource allocation position where you're trying to decide whether you should include the preventive drug for these patients who've had surgery or going to have surgery compared to providing vaccinations, you have a disconnect between your outcomes.  Because in one case, you have a cost per, let’s say, a cost per day in the inpatient hospital and, on the other hand, you have a cost per case of flu avoided if you have, you're comparing it to a flu vaccination.

That’s why you need the QALY, of which we'll give you a standardized outcome across both arms, both types of studies so that you can make resource allocation decisions.  Can I go back to the slides?  Great—and the next one.

We're back to this intervention to reduce the postop infections in patients with hip fracture repair.  Your first question, really, is what is your perspective?  Is it the hospital system, or are you trying to make those policy questions?  If it’s the hospital system, then you want to be able to use natural units that are present in both groups that you're comparing.  If you're comparing those people who received hip fractures in two groups, then you can use those natural units, but if you want to compare, as I said, these hip fracture patients to costs for vaccinations, you have a different question, a different problem.

If you are using a societal perspective, which is much more the Vaco perspective, the policy decision makers, you need a tool that is going to help you quantify the length of the illness or the duration of the illness and the length of life, the quality of life during the follow up period.  It’s measured in both groups and comparable across programs.

That is your QALY.  A QALY describes the duration of illness, years of survival, adjusted for quality of life experience during that survival.  A QALY can range from 0 to 1, where 0 is perceived as the worst possible health state or death, and 1 equals a year in perfect health.  

The question is, how do you quantify the quality?  This overview shows you that if you have one year in perfect health and I have one year in good health at 0.8 QALYs, there’s a 0.20 QALYs difference in how we experienced the last year.  But this is a very straightforward question, and most interventions don’t have simple effects on patients—for example, cancer treatments and joint replacements and other complex medical care that our patients receive.

The estimation of the QALYs requires that you be able to describe the health states experienced by patients during the trial and afterwards, estimation of the duration of each health state, comparison to or assessment of individual or community preferences for each health state.  You need to know all the health states experienced during that period of follow up.  

Let’s give you an example.  In our hip fracture repair example, we have prophylactic antibiotic provided to the patient versus the standard of care, where there are no prophylactic antibiotic treatments provided.  The new treatment—and so what I've done is provided a quarterly assessment of the QALYs experience by the patient.  This is entirely hypothetical.

Let’s say, in the first three months after the surgery, both patient groups, new treatment and usual care, both feel about 50 percent of their usual, and in the second quarter of the year, because the patient received the new treatment, they're feeling a little better.  They have no infections, they've had no re-hospitalizations, they're starting to be able to walk in the community.  But the usual care population, a portion of them have gotten infected and a portion of them have been re-hospitalized.  If we assume this is just two people and the usual care patient got an infection and a new treatment person did not, then you can see how the QALYs increase across the year for the patient who received the new treatment and there’s a dip and a delay from the first quarter to the second quarter to the third quarter to the final quarter.  

They end up at the same place, but their experience during the year is different.  The new treatment patient gains 0.125 QALYs in the first quarter.  I hope it’s clear what I'm doing, here.  In the second quarter, it’s 0.15, in the third quarter 0.2, and the fourth quarter 0.2 QALYs, so that the total QALYs for this patient during the one year is 0.675.  In the usual care, they start off at the same place, but they don’t feel well, and so they are feeling about 35 percent of normal in the second quarter.  Then they kind of recover back to how they felt postop at 50 percent of their usual.  Then at the end of the study period or the one year follow up there, they feel about the same as the patients who received the treatment, but they're just starting to feel that way, so that the QALYs gained by this population or this person who has received usual care is 0.5375.  This is the cumulative QALYs gained over the period of time of the study.

Let’s go to the next slide.  We're now going to calculate this cost per QALY.  Again, we're back to the ICER, we have the new treatment versus standard care.  We have assumed that the new, in this hypothetical, the new treatment costs $10,000.00 and all other costs are the same, recognizing that, in fact, if the usual care patient went back in the hospital, the costs would not be the same—but, for this exercise, we're assuming they're the same.

The difference in cost is $10,000.00, and the difference in QALYs is 0.1374, so you have an estimate of $72,000.00 per QALY gained.  This would be considered possibly in the range of cost effective, depending on who your reference is, because the references vary from about $50,000.00 per QALY to $100,000.00 per QALY.

How do you get these QALY weights?  How do you get these measurements?  The basic methodology is to provide a personal reflection on the relative value or preference weight of different health states.  These are done by patients, by providers, and by a community sample.  Basically, it’s comparative ranking of preference for health states, and they're elicited through various tools.  There are three methods to derive preferences.  You can use off the shelf ones, direct measurements, or indirect measurement.

Off the shelf values are those that you find in the literature, and hopefully you can find weights or previous studies that report weights for your condition of interest, but not all health states have been characterized, and they may not be specific to your problem that you're studying.  They are useful when you're doing decision modeling, they're useful when you're trying to do a lifetime follow up from a particular intervention, but again, not all health states have been characterized.

Direct methods—individuals are asked to choose or, again, to clear their preferences between their current health state and alternative health status scenarios.  They make these choices based on their own comprehensive health states at the time of measurement, and they do this several times—multiple times during a study.

Here is kind of the global perspective on what the domains are for a health state.  It’s how you can see and hear and speak.  Do you need help to get around?  Can you walk independently, can you use various important rooms in your house, are you irritable or cranky?  Are you cognitive stable and functioning well?  Can you take care of yourself and toilet normally and you're free of pain or discomfort?  

These things will be different for different people.  For example, Todd, who is our backup here, might be really annoyed if he needed the assistance of another person to get around—whereas I might not, but we both would be very frustrated if we couldn’t speak normally or learn and remember normally.  You can see how these would vary across individuals.

One of the direct measurement methodologies is the standard gamble, and this essentially, you are asked a question or the respondent is asked a question.  If you could take a pill to be restored to perfect health, how much, what percent chance of dying would you be willing to risk by taking a pill to live the rest of your life in perfect health?  Let’s assume that you are postop this hip fracture, but you cannot, you have to use a walking device and a raised toilet seat and you can’t drive your car.  The question is, if you could take a pill that would restore you to perfect health, but there’s a chance of dying as a result of taking that pill, what is the chance of dying you would be willing to risk in order to be restored to your perfect health?

Again, we will have different responses.  I might be willing to risk only 5 percent chance of dying if my physical scenario was as I described, that I couldn’t drive, I needed—because I'm a physical therapist, I can figure out how to work around those problems, where Todd, who wants to bicycle 200 or 300 miles a week would not be willing.  He might be willing to risk a 40 percent chance of dying to return to normal health.  Again, the question is, how much are you willing to risk to return to normal health?  How might this vary between you and your friends, between someone who is 60 and someone who is 80?  These are all things you will take into consideration when you're planning your study.

The second direct method is called the time tradeoff.  In this question set, you are asked how many years of life are you willing to give up to move from your current health to perfect health?  In this picture, perfect health would be the light blue, and the question is, how much time between time one and time two would you be willing to give up to move from your current state of health to perfect health.  Again, how many years?  Again, how might this be different if you were 25 versus 75?

You use these methods, which are very time consuming, if the effects of the intervention are complex.  There are multiple domains affected by the intervention and the effects are not captured in indirect or disease specific instruments, and this is an important consideration, the concept of the effects being captured.

In the direct methods, there’s high variance in estimates from patients in trial, and that reflects our individual risk aversion, our feelings about disability.  You need, because there’s high variance, you need a very large sample size.  Additionally, it’s not the community value specified by goals at all in the standard book and our reference book for the course.  These are individual measures, and they will reflect the variation of the individuals in your sample rather than a community value that’s observing from the outside.

The next method is our indirect methods.  Again, they are standardized, generic, health related quality of life surveys that patients take during a study.  They involve multiple domains of health in that those multiple domains of health describe a composite health state of the respondent at the moment, and the composite is linked to community results or weights that have been previously studied.

For example, here is the EQ-5D, which is one of the indirect methods, which has basically five questions that you can answer in three dimensions.  In mobility, you have no problems, some problems, or extreme problems.  In the area of pain, you have no problems, some problems, extreme problems.  In the concept of anxiety and depression, et cetera; no problems, some, or extreme.  You can see that these are very general questions, and would not capture an effect that you might be interested in if the intervention you're studying is very specific to a particular task.  For example, let’s say you broke your finger or there was some treatment for finger fracture.  The recovery—I'm not suggesting you would do a cost effectiveness analysis for finger fracture treatment, but I'm just using that as a model.  That you're not going to see if the finger is healed, you're not going to see a change in mobility.  You may see some change in pain, but possibly not very much.  Your anxiety, depression—maybe, maybe they'll be in effect.  You may have some self-care effects, and you may have some usual activity effects; for example, if you knitted a lot.

The question is, in these indirect methods, how many health states are there?  Because if you go back, so we have, I think, 250 different health states for the EQ-5D, but we'll talk about that in a minute, because of the way these answers can match up together.  The indirect methods have, there are four basic ones that are used in cost effectiveness analysis.  We're just going to talk about these at this point, although there are more being developed, and I have a reference for you.  These are generic health status measures, and they vary in the dimensions or attributes that they have, the size and nationality of the sample population that was used to establish weights, the health states that are defined by the survey, and how the summary score is calculated.

In these indirect measures, what’s happened is, the tool has been developed and then taken to a community sample of people who are not disabled, who do not have the conditions of interest, and the various health states that might be described by the health utility index are then given to this community sample, and that community sample tells you, through whatever tool is being used, either the standard gamble to the time tradeoff, what their preference is for that health state.  You can see, in the health utility index, there are 41 questions and 8 domains of health.  There are 900, almost 1,000,000 health states that can be described with these 41 questions.  The community sample that established the weights were Canadian, and it’s based on utility theory, which we're not going to describe here.

The EuroQol, or the EQ-5D—oops, sorry—has 245 health states.  The five domains that we described with three possible answers for each of the domains, the original weights were established by a British community sample, and—but no U.S. weights have been done, meaning that the survey and the health states have been then given to a community sample who established their preferences.

The quality of well-being instrument has two versions.  The SA, which is self-administered, has 76 questions, and 1,215 health states.  The primary care patients in San Diego were the community sample that established the various weights.

The SF-6D comes from the SF-36, or SF-12 scores.  It has six domains, defines 18,000 health states.  The original basis of domain weights is a British community sample, but they are also—you can also use the VR-36, which is the Veterans RAND 36, and use the VR-12 scores to estimate the VR-60, the Veterans 60, for use in economic studies.  Again, the VR-36 and the VR-60 are specifically for, were developed from the SF-36 for use with veterans and an older population.

Then there are disease specific surveys, so you have these generic instruments, but you don’t think you're going to capture the quality of life effect in the generic instrument that you think will really tell you whether or not the quality of life is affected by the intervention.  Increasingly, people use disease specific surveys that describe functional change alongside a generic instrument, and then extrapolate weights from the disease specific surveys and the generic weights.

These are very expensive to do, because not only do you have to do both surveys, but you have to then do the post talk extrapolation—but they're potentially more sensitive to the variations in the quality of life with this disease, so there’s a trade-off for using these in your study.  Again, there’s a trade-off between sensitivity and burden.  What you would want to do is start with your, a literature search that is about QALY adjusted like your measures, that have been used in the past in the condition of interest that you're studying in a similar population and that describes the outcomes of interest.  I think, increasingly, people are finding that they're—certainly in musculoskeletal problems, the quality adjusted life year measures are not sensitive enough to the changes in physical function that you might want to be estimating and consequently, if you see no change compared to usual care, then what will happen is, you will not have a cost effective intervention, because there’s no difference in the outcomes for the patient.

In terms of a hierarchy, you might use off the shelf utility values if they match, again, your condition, your population, and your outcome.  You would then select an indirect measure and possibly a disease specific survey during the trial, and then if really, your problem is really complex, then you would go to the direct measures.  These are really the least burdensome to the patient, to the most burdensome, not only to the patient but also to staff managing this study.

I want to provide some important resources, and these, I hope, will be linkable from the slides.  Tufts Center for Risk Assessment, you can see lots of utilities and preference weights that have been already measured.  They capture these and keep them in a log or table.  The National Institute for Health Research is in the U.K. and they are doing an extensive amount of work on QALY measurement, transforming disease specific measures into preference measures.  I think here are two resources from the National Institute for Health Research, a review of the health status measures from 1999, and then developing testing methods for driving the preference measures of health from condition specific measures.  These are two terrific resources.  This was the table of utility weight for preferences at Tufts, and here is our guide book on preference measurement at HERC, which is available online on our website.

I wonder, have we any questions?

*Todd Wagner:*At this point we don’t have any questions.  We can hang out for a few minutes and see if other people will type in questions.

*Moderator:*To type in questions, just use that Q&A box in the lower right hand corner of your screen.  We have a great opportunity here.  We have some time for questions for Patsi, so please feel free to type those in.

*Todd Wagner:*All right, they're starting to flow in, so this is great.

*Patsi Sinnott:*The first question is, “Can you explain the hierarchy?”  What we're describing here is just the least burdensome to the most burdensome.  Off the shelf utility values, you would use, they're already in the literature.  Direct measures at the bottom are the most complicated to administer and they're the most complex and time-consuming for your patients.

I don’t see the end of Willy’s—let me see if I can do this.

*Todd Wagner:*One answer, another question for you, Patsi, is imagine you're doing a study, a clinical trial on HIV prevention or treatment, I should say.  You see that other people have done extensive work linking utility values to CD4 counts, such that as your CD4 count gets worse, your utility gets worse, you might say that that is sufficient for urinalysis and that you can just use the off the shelf.  Otherwise, you might dive into these other things about, “Let’s actually collect the data from the patients themselves.”  You're right that indirect measures are often used in multi-site trials just because they're relatively straightforward and quick to do in multi-site trials, whereas the standard gamble is quite burdensome and gets quite tricky and time-consuming, actually.

*Patsi Sinnott:*Well, and expensive, because most—the direct measures, at this point, are administered through laptops, and so over time, let’s say you have a four or five year study, the laptops are going to break down, the laptops are going to disappear, the laptops are going to need updates and they're going to need to be replaced, and so you have to build that into your budget and your concept of what you're going to need to do these kinds of methods.

*Todd Wagner:*The next question asks about, you've talked about this ICER, this incremental cost effectiveness ratio, and most of it you focused on the effectiveness component.  This person was asking, “Is there more on the cost?”  I guess my answer is, most of the other slides are actually—or not the other slides, most of the other talks in the cyber course are on the cost component, trying to make sure that you do an accurate job capturing medical care costs.  The last course a week ago, we talked about DSS and HERC.  That’s probably the best place is to check back on those other courses.

The next question is, “Do you have any indirect measures for the pediatric patient population?”

*Patsi Sinnott:*Hmm.  Since the patient, presumably—it depends on how young these patients are—there are quite a few pediatric questions here; young infants, specific for pediatrics.  Very young children obviously can’t complete the surveys themselves.  I think you would have to rely on parental responses, but you're going to lose some accuracy.  Have any ideas, Todd?  *[Laughs]*

*Todd Wagner:*The one other thing to keep in mind is self-reported accuracy, and 9 is the general cutoff.  Most people just won’t try to capture self-report with patients under age 9.  A lot of these indirect measures actually allow for proxies, and what they would say is, “Be consistent in how you measure it.”  Let’s say you had a group of patients who ranged between 7 and 10, they would say, “Use a single method across all of it.  Don’t try to split your methods by your age groups.  If you're going to use proxies, use proxies for everybody.”  But there’s actually a fairly big literature on the use of proxies.

*Patsi Sinnott:*Down below, Ben Huang has said, “EQ-5D has a youth version, and a HUI2 can be used for children.”

*Todd Wagner:*Great comment.  The next question, “Is there a great place to find weighted factors for various health care measurements?”  I'm assuming that they mean the utility weights, and I think that I would turn you to Peter Neumann’s work at Tufts as well as the U.K. link that was provided, but obviously the literature is robust and growing, there.

*Patsi Sinnott:*Here on this side, the table of the published utility weights or preferences for different health states and also for different conditions.  If you—I'm sorry?

*Todd Wagner:*No, I was going to go onto the next question.

*Patsi Sinnott:*Just that it’s here.  Okay.

*Todd Wagner:*Yeah, so one of the participants notes, Dennis Fryback, who’s at the University of Wisconsin, showed that different, indirect measures were highly correlated cross-sectionally, but changed over time and did not agree.  Another question came in, which was very related, is, “How do you reconcile the different utilities for the same health states in different studies?”

People struggle with this, and I'll turn it back to you, Patsi, but is there any correct—

*Patsi Sinnott:*Only that that’s true.

*Todd Wagner:*Yeah, I think that’s true, yeah.

*Patsi Sinnott:*Mm-hmm, and that you can’t predict one from the other.

*Todd Wagner:*I think some of the work that I've most followed is Sherbourne and Schoenbaum’s work with depression.  Depression poses a number of unique circumstances in measuring preferences and utilities; specifically if you're going to do it around standard gamble where you're implicitly trading off risk.  The idea, from the psychologist’s perspective, is that when you're depressed, you're not willing to have much risk.  When you get healthier, you're willing to trade things off in risk.  When you might be actually getting healthier, it might look like your preferences are getting work.  You're right that this is just a challenge for the scientific community here.

All right, so we have another question for you, Patsi Sinnott:  “How do you account for changes in QALYs that may not be attributed to the intervention?”  The classic one is a job loss, right?

*Patsi Sinnott:*Mm-hmm, right, but you assume that those changes are equally distributed across both groups, right?

*Todd Wagner:*Should you be doing a randomized trial?

*Patsi Sinnott:*Right.

# Todd Wagner:Otherwise, you're into the sort of econometric literature of, how do you adjust these things?  You're right, or this question is right, that these things are often sensitive.  There are other quality of life measures.  There’s one from the World Health Organization called WHOQOL that’s measuring all these other aspects, religiosity and your family community and so forth.  Those things can have a profound impact on your quality of life, but not at all health related, necessarily.

*Patsi Sinnott:*Right, and just as an aside, we—in a study that I worked on, where job acquisition was the outcome of interest, we saw no difference in QALYs gained over the follow up period, the first year follow up of the study, but the job acquisition was 18 percent higher—well, it was significantly higher in the intervention group compared to the control group.  You can have a very different outcome and if you do the cost effectiveness analysis using, for example, in this case, jobs obtained, the intervention would be cost effective, but if you use it, estimate it using QALYs, there’s no difference in the QALYs, and so it’s not cost effective.

*Todd Wagner:*I would follow up with that.  If you're interested in sort of the nuances here, *Medical Decision Making* is a fascinating journal just to follow, also the work of Peter Ubel.  There’s a lot of work that shows that, for example, if you randomly leave a quarter on the Xerox machine, it changes people’s sort of valuation of their day.  Weather can do that.  Statistically, what that typically means is, it ends up in the error term, because it’s not something we observe.  “Did someone find a quarter?”—you don’t ask that in the survey.  Often what we find is that there’s a lot of variation in these measures.

*Patsi Sinnott:*Right.

*Todd Wagner:*The next question for you, Patsi, “Is there any preference for different preference measures, especially indirect approaches?”  Among the HUI, EQ-5D, and the QWB, and there’s even one from Australia called the AQoL.  Do you find any of these that are, is there a preference among these?

*Patsi Sinnott:*Right, so remember, what you're trying to do is use an instrument that’s going to give you, that’s going to measure your outcome of interest.  For example, you had a project, Todd, where the outcome was—one of the outcomes was dexterity, right?

*Todd Wagner:*Correct, yeah.  It was a stroke rehab study.

*Patsi Sinnott:*And so you used the QWB, right?

*Todd Wagner:*HUI, Health Utility—they have a specific question on that, but we chose that one.

*Patsi Sinnott:*It’s really going to depend on what you're trying to measure, and I would just caution that—and this comes from my own personal experience—number one that your proposal includes that you're going to use a natural outcome as well as a QALY outcome, so that you don’t get told by a reviewer that, since you didn't propose this in the original study, you can’t publish the results.  That you get, you capture the outcome that you're interested in, and that you run the disease specific, if there’s something that’s very sensitive, and then try to extrapolate it to a preference measurement, to preference weights.

You can’t choose one, because the conditions are too different, and what you're trying to measure too different. 

*Todd Wagner:*In randomized clinical trials, I will say that there is a bit of nationality pride, here.  The EQ-5D, being created in Europe, is frequently used in the European trials.  It’s gaining more acceptance in the U.S. because the weight is now in the U.S., but I'll speak, because it’s so short.

*Patsi Sinnott:*And simple.  If you go to our guide book, Vilija [Joyce] has done a summary to date, and I think this is as of about 2006 or ’07, of all the different conditions that were measured with these four instruments.  That would give you some background to choose your measure.

*Todd Wagner:*The next question is asked—can you recommend a report using the methods you described as the best captures of value of this kind of analysis?  Personally, I’d have to think about that, if there’s a good time to get back to you.

*Patsi Sinnott:[Laughs]*  Yeah.

*Todd Wagner:*Besides the ones that were just listed there.  I mean, there’s seminal articles by Milt Weinstein, but—I'm sort of pulling off the top of my head right now.

*Patsi Sinnott:*Yeah.  You know—

*Todd Wagner:*Let’s go onto the next one, and we can follow up with that person if we can think of it.

*Patsi Sinnott:*Right.  What I would suggest is, starting with the Gold book as an introduction to the methodologies.  I think there’s a revision coming out shortly; isn’t there, Todd?

*Todd Wagner:*I know that Doug Owens was on that committee.  I don't know if they're publishing a book or what the report is going to look like.

*Patsi Sinnott:*Okay.

*Todd Wagner:*There is another question that says, “Note that a lot of these utility values were derived 20 or 30 years ago,” which—and I was alive when those were being derived, and I remember seeing this published, so I guess I'm feeling very old all of a sudden.

*Patsi Sinnott:[Laughter]*

*Todd Wagner:*It then says, “Are these utility values still valid?”  I hope I'm still valid; I don't know about the utility weights.  I'll let you answer that one.

*Patsi Sinnott:*Well, you bring up an interesting question, which is—or issue, which is, have relative values changed, have preference weights changed?  Of course, we don’t know, unless we compare and repeat.

*Todd Wagner:*Here’s a follow up question, an excellent question that we often struggle with.  Often, you're doing a study—and let’s imagine it’s spinal cord injury, and you have two choices between weights.  You can have weights that are identified from community people who do not have spinal cord injury, versus those who have a spinal cord injury and perhaps are recovering.  Which—do we have thoughts about which to use, the community preference, or the patient’s preference?

*Patsi Sinnott:*Well, the recommendation to use community preference comes from the standard recommendations from Gold, et al.  That is that the community preference is what the rest of society thinks about the condition and thinks about the limitations associated with the condition.

You can imagine that someone with a spinal cord injury who was, let’s say, four or five years out from the original injury, might regard their limitation in ambulation, for example, much less significantly than someone from the community would imagine how limiting that would be, or their preference for that condition.  Over time, someone who—the patients accommodate to their disability and therefore are less likely to—first of all, they're not going to reflect the community value, and they're going to be less impacted by changes to those particular conditions; for example, walking or transfers.  Whereas the community will imagine what it would be like to have a spinal cord injury—but they're more sensitive to the limitations.  Does that make sense?

*Todd Wagner:*It makes sense.  I think, again, we're at the frontier of research here, and I understand the tax payers are paying for the health care that you might want to use community wide, but it is a challenge, especially when you talk about something that has such a, maybe stigma is too strong a word, but the idea of spinal cord injury.

*Patsi Sinnott:*Right, right.

*Todd Wagner:*We think of that as being something so traumatic.

*Patsi Sinnott:*You know, the literature recommends that you do not use providers, and just as I said earlier, so I'm trained as a physical therapist, and I can imagine how easy or not it is to make accommodations for someone with physical limitations to get around, to accomplish their task, to live their life.  My perspective might be quite different from Todd’s, who has, lives a very active life and hasn’t worked with patients.

They really generally—so if a provider is asked to establish weights or establish preference, they're going to be imagining what it’s like to be a patient, possibly their patient, based on their experience with patients similar to the patient or the condition at hand.  I think that the community value really is there to address how society should value these improvements, these changes, these disabilities for—well, I think that’s the sentence.  You really want the community value, so that it’s standard across all conditions, theoretically.

*Todd Wagner:*All right, so we have a number of questions left and I know that we're running out of time, so we should try to address these as quickly as possible, Patsi.  To follow up that excellent question with another excellent question, costs that are in the future are discounted to net present value—what about QALYs?  Imagine you're doing a study on prevention and the real sort of benefit of the current treatments, let’s say it’s cancer screening, is the prevention of cancer in 20 years.  What do you do about the QALYs?

*Patsi Sinnott:*You discount them.

*Todd Wagner:*Correct.  That’s the general approach.  Okay, so the next question—most ADHD scales use parental responses.  I guess that was a comment about, “Can you ask about utilities from young adults?  There are theoretical, empirical, and practical challenges to the use of monetized qualities in BCA.”

*Patsi Sinnott:*I'm sorry.

*Todd Wagner:*I'm not sure I fully understand the question.  I think the person is—BCA being benefit cost analysis—there are theoretical, empirical, and practical challenges to monetize QALYs.  I would agree with that, so if you were trying to estimate sort of the cost of life and put it in a dollar value—that’s why a lot of people like the idea of measuring QALYs.

*Patsi Sinnott:*Right, and there’s been a lot of work done on—not to say it’s not challenged—on estimating the value of a life and mortality.  It’s much, much, much more difficult to kind of monetize the morbidity, the sickness, and the illness, and the disability.

*Todd Wagner:*The next question is, “Can we say if we focus on precision of preference measures, we should reverse the order of the hierarchy for methods in the slide?”

*Patsi Sinnott:*Yes.

*Todd Wagner:*Yeah, and keep in mind that precision is often a function of your sample size, so it’s both not just an issue of doing standard gamble or time trade-off, but it’s also how many patients you're going to run through your elicitation.

*Patsi Sinnott:*Right.

*Todd Wagner:*“Is there a literature regarding QALY measures available for military personnel and mental health states?”  That’s a great question.

*Patsi Sinnott:*I don't know an answer.

*Todd Wagner:*Yeah, I think we can follow up with this person.  My general, and I haven’t kept up with the literature on the mental health states, and I know it’s blossomed because of the idea of depressed realism, what I was talking about earlier with Cathy Sherbourne and Michael Schoenbaum’s work on depression.

*Patsi Sinnott:*Mm-hmm.  Paula might have some ideas.

*Todd Wagner:*Paula Schnurr, yeah.

*Patsi Sinnott:*Mm-hmm.

*Todd Wagner:*We'll follow up with that person online.  Here’s another question, we have just a few questions left.  “An example of a change that may not be attributed to a QALY, maybe a patient with a form of disability, adjusting to their disability over time.  I think that the spinal cord injury, that’s often why we use it, because they accommodate or believe they accommodate and realize that it’s not as bad as their first ideas.”

Using the QALY example on slide 15—can you go back to slide 15?

*Patsi Sinnott:*Mm-hmm.  Yeah.

*Todd Wagner:*“Is QALY typically considered as a crude or unadjusted estimate or are these adjusted for potential confounders that may be different in the two groups?”

*Patsi Sinnott:*These are averages across the groups.

*Todd Wagner:*I would say typically these are unadjusted and you would then have to, in your QALY, your ICER, build in a cost effectiveness region, usually through bootstrapping.  In theory, though, depending on how one does this, one could try to figure out and do different ways.  One could do a stratified analysis, perhaps.  You have different severity of illness, and you wanted to check the severity of illness, you could stratify your analysis.  In theory, you could do predicted analyses or adjusted models, too.  Use your regression to compute the adjusted probability of a QALY.

All right, I think we're both running out of questions and running out of time, here.  I think we've done a pretty good job.  I apologize—some of the questions were repetitive on the young infants and kids, so hopefully we address those.  Then again if we haven’t followed up and answered your questions specifically or you have a follow up to your question, please just let us know, and we can respond to you.  Thank you, Patsi.

*Moderator:*Thank you.  Can I put a feedback form up for everyone, if you could take just a few moments to fill this out, we really do read through all of your feedback.  We have made many changes to the program, really directly responding to the feedback that we have received.  Patsi, Todd, thank you both very much for taking the time to prepare and present for today’s session.  We very much appreciate that.  The next session in this course is scheduled for May 28th and Jeremy Goldhaber-Fiebert will be presenting “Medical Decision Making and Decision Analysis.”  I know most of you are registered for that course, but if you are not, we will be sending out registration information for that in about a week, so keep an eye on your e-mail and you will receive that out there.

*Todd Wagner:*Just to push that one, Jeremy is one of our colleagues at Stanford, trained in decision modeling at Harvard, and worked a lot with Sue Goldie and then has been doing a lot of work here at Stanford on decision modeling.  He’s just a true star, so I hope you'll join us for that course.

*Moderator:*Fantastic, and thank you, everyone, for joining us for today’s HSR&D cyber seminar, and we do hope to see you at a future session.  Thank you.

*Todd Wagner:*Thanks, Heidi.

*Moderator:*Thanks.

*[End of Audio]*

Page 15 of 15