**Transcript of Cyberseminar**

**HERC Cost Effectiveness Analysis**

**An Overview of Decision Analysis**

**Presenter: Risha Gidwani, DrPH**

**April 9, 2014**

*Paul Barnett:*Yeah, this is Paul Barnett.  I’m Director of the Health Economics Research Center.  Welcome to our course on Cost-Effectiveness Analysis.  This is a new and improved version of the course, and the improvements largely have to do with today’s speaker, Risha Gidwani.  She’s a health economist at the HERC— Health Economics Center—and also an investigator with the COIN.  It’s at Palo Alto—the Center for Innovation to Implementation. 

She’s a consulting assistant professor at Stanford, and she does research focused on the intersection of cost and outcomes with expertise in areas of cost-effectiveness analysis, health economics and measurement of the quality of care.  She obtained her Doctorate of Public Health from the UCLA School of Public Health, Department of Health Services.  

Prior to coming to VA she was at Stanford and also at a Boston consulting firm where she prepared some cost-effectiveness analyses and health economic models that have been submitted to the regulatory authorities in UK and Canada, US and other countries.  Risha, please give us our introduction.

*Dr. Gidwani:*Thank you, Paul.  I’m very pleased to kick off our cost-effectiveness analysis cyber course with an overview of decision analysis.  Please let me know if you’re not able to hear me.  I am on speakerphone, but I can change that if that’s a problem.  Let me get started here by talking about what I’m hoping you’re going to take away from this lecture.  First I wanted to just give an understanding of why to even engage in doing decision analyses.  

We’ll then talk about the different types of decision analysis you can operationalize.  One of my goals today is to be able to turn jargon that’s often used in this field into definitions.  One of the most specific definitions I’d like to provide is on the difference between cost-effective and cost-saving, which are often used interchangeably when speaking about this field colloquially, but have really specific and different meanings as they relate to health economics that I will go into.

In terms of why to engage in decision analysis, it’s usually because you have to choose between funding different interventions, and there’s not enough money to be able to allocate in order to fund each intervention that you think might have some health benefits.  There is usually no clear right answer of which intervention is the best one to fund.  That’s because each intervention is going to have its own pros and cons.  Decision analysis therefore gives us a logical, transparent and quantitative way to evaluate the pros and cons of each intervention, and therefore, make an informed decision for funding allocation.  

When we weigh the pros and cons of the decision, we know that they are not all created equal.  There’s going to be differences amongst these pros and cons in terms of the importance of their consequences, and there’s also going to be variations in terms of their probabilities.  Then because of a number of different reasons that we’ll touch on later, we know that there’s going to be some uncertainty or variation in our estimates of the probabilities of these pros and cons.  

Here’s an example of how one might think about pros and cons when doing a decision analysis.  Let’s say here that we have: Option A.  Maybe that’s a drug, and it has an 80 percent probability of cure, and a 2 percent probability of having a serious adverse event, so we have a pro and a con noted here.  We also have a second treatment option: Option B.  That drug has a 90 percent probability of cure, so a higher likelihood of cure—of pro—but also has a higher likelihood of con. 

It has a five percent probability of serious adverse event.  Now to complicate things even further, we have a third option: Option C.  Option C has the highest likelihood of pro.  It has a 98 percent probability of cure, but it has a lower probability of con than Option B, but some of these cons are very serious.  With Option C we have a one percent probability of treatment-related death, which of course is the most serious thing that could occur, and a one percent probability of having a minor adverse event.  We need to figure out a way to choose between one of these three options.

What we can do is plug all three options into a decision analysis to compare them against one another.  Here I’ve noted three options, but when you are doing a decision analysis you can have as many options as you want assuming that you have good data for each one of these options.  You don’t need to limit them to two options.  In our next lecture given by Dr. Ciaran Phibbs he’ll talk about how to actually deal with a situation where you have multiple options that you’re evaluating and how you are able to whittle them down in order to understand the relative value of one option to another.

When we think about choosing one option for funding an intervention, that means that we’re going to forego other options.  That could be either because we don’t have enough money to fund everything or because we have resource constraints.  Meaning that we can only focus our attention on implementing one intervention.  There’s many examples of this: for example, we could have a Department of Public Health that has to decide between funding a directly observed therapy program for patients with tuberculosis or a community health worker promatora-based breast-feeding campaign.  

That’s because there’s finite man power.  This is a resource constraint requiring us to choose one option versus another, or it could be something that’s at a very large level.  For example, when we’re doing decision analysis in the environmental fields you might be interested in understanding the effect of a cap-and-trade program versus a carbon tax on reducing greenhouse emissions.  The regulatory burden of each approach means that you have to prioritize one over another.

Decision analysis also allows us to accommodate variation, so in medicine and health care we have a lot of variation that can stem from a number of sources.  If we have a non-pharmacological intervention, then we could have variation in the way that the intervention is actually operationalized.  Maybe we’re interested in comparing a disease management program to a drug, but when we’re looking at multiple disease management programs each one of them has been implemented in a way that’s culturally specific.  Therefore across the disease management programs there’s variation.  

There can be variation in adherence to an intervention, and that can be both across patients as well as within patients over time.  For example, after a certain time period of taking a medication a patient may drop off in his or her adherence.  There can also of course be real variation in response to an intervention that could occur perhaps on a subgroup basis.  Maybe people that are less sick are more likely to respond to the intervention versus the people that are more sick.  

Then we also know that we have variation that comes from sampling error or uncertainty because we’re not studying the population usually when we’re doing health care research or medical research.  We’re studying a sample of the population.  To recap why we should use decision analysis.  We have situations all the time where we have to allocate limited resources, and we are trying to choose then between funding one intervention and funding other—another intervention.  Each intervention is going to have its pros and cons, and each intervention can be different in terms of, let’s say, the condition or population that it affects.  

It’s definitely going to be different in terms of the cost of each intervention, and each intervention could have differential effects on health outcome.  To complicate matters further we know that there’s uncertainty around the estimates that we’re using about the pros and cons of the intervention and the cost and the health outcomes that are affected by the intervention.  Decision analysis has a multitude of advantages then.  One of the big advantages is that we can evaluate different interventions because we’re using the same measure in order to do—to compare those different interventions.

When we compare results using the same metric, that could be a number of different things that we use as our metric.  We could just look only at costs that are affected by each one of the interventions.  We can look at costs per life year saved, or we can get even more specific and look at cost per quality-adjusted life year.  Decision analysis can be applied to really almost anything you can think of as long as you can get good data for it.  Often times it’s used to evaluate drugs, so that might be a new therapy that comes onto the market versus existing treatments.  

It can be applied to different procedures such as surgical procedures, for example.  Different health programs.  Maybe a health education program.  Disease management program.  It can be applied to screening.  When I say screening, that could be both the types of screening that is done as well as the frequency of screening or the population for whom screening represents the best value.  We can look at decision analysis as applied to vaccines, reimbursement decisions, health policy decisions.  As long as you can measure it and find good quality data, you can really do a decision analysis for any topic.

For example when I came to the VA and I was trying to choose amongst the different health insurance options that were available to me, I built a decision model in order to figure out which type of health insurance I should purchase.  Really if you can think of it, you can measure it.  You can build a decision analysis around it.  Now that we’ve talked about why to even—why to engage in decision analysis, let’s shift over to the different types of decision analyses you can engage in.  

I’m gonna talk about the major forms of decision analysis today: those being cost-effectiveness analysis, cost-benefit analysis, and budget impact analysis.  These types of decision analyses could be applied to really any industry like finance, environment to economics, health care.  Today I’m really just going to focus on applications to health care, but I do want to point out that many fields have been doing this for longer than health care, specifically environmental economics.  A lot of the techniques that we use have been developed by that other field.

One of the things—before I get in—more into cost-effectiveness analysis, I want to point out that these different—the cost-effectiveness analyses, the cost-benefit analyses are comparative evaluations.  They’re going to evaluate one intervention relative to another intervention.  In health care the intervention could be a standard of care, or it could be a do-nothing approach, but when we do include a do-nothing approach when we’re doing these cost-effectiveness analyses or cost-benefit analyses or even budget impact analyses, we need to exclusively include the downstream costs associated with the do-nothing approach.  

Meaning that there are probably some health consequences that will result in utilization of the medical care system that we need to quantify.  Let’s get into cost-effectiveness analysis.  The results of a cost-effectiveness analysis are cost relative to health effect.  You’re explicitly considering the cost of an intervention relative to the health benefit you get from that intervention.  Those health effects can really be anything.  They could be life-years saved, cases of cancer avoided, a number of infections treated, really any sort of health effect you can think of that would be of interest to you can be an outcome in the cost-effectiveness analysis.  

Again, as long as you can measure it.  When we do cost-effectiveness analyses we’re comparing the impact of two or more interventions.  This is again a comparative or a relative effect that we’re getting.  The result from our cost-effectiveness analysis is an incremental cost-effectiveness ratio or what we call an ICER.  What that ICER does is it looks at the cost of an intervention relative to the cost of another intervention and looks at that quantity compared to the difference in health effects amongst those two interventions. 

Cost-utility analysis is a particular form of cost-effectiveness analysis, and we can think of cost-effectiveness analysis being the umbrella term under which cost-utility analysis sits.  In a cost-utility analysis the health effect is very specific.  It’s a quality-adjusted life year or a QALY.  Now the QALY is derived from the utility, and so because the QALY is calculated for utilities our approach is called cost-utility analysis.  This is just a quick summary of what a cost-effectiveness analysis is versus a cost-utility analysis.  

Both of these types of techniques are going to compare two or more interventions.  In cost-effectiveness analysis our outcome is a change in cost relative to a change in health effect.  When we do a cost-utility analysis we have a very specific health effect we’re looking at, and therefore our outcome is a change in cost relative to a change in quality adjusted life year.  I said before that the QALY is a function of the utility.  Specifically it’s a function of the number of years of life that somebody lives times the utility of that life.  

Here’s a quick example of how we get a—of how we would calculate a QALY.  Let’s say that somebody has a utility of 0.8, and they live for five years.  Our QALY is 0.8 times 5 resulting in a QALY of 4.0.  Now utilities represent a preference for health.  They are not simply a measure of health.  What they actually are doing are combining information about the health state that a person is in with a valuation of that health state.  Utilities conventionally range from zero to one, where zero represents death and 1.0 represents perfect health.  

I’m going to talk a little bit about utilities in the upcoming slides, but I also want to mention that Dr. Patsy Sinnott will be giving a lecture on this later on in our cyber seminars.  I would strongly encourage you to attend that, because utility and quality calculations can be very complicated and certainly merit a deep understanding if you’re going to operationalize a cost-utility analysis.  Here’s an example of how one would calculate a utility.  Here we have two people.  We have Jane and we have Joe, and you can see that their health - their health is a function of four variables.

Activities of daily living, exercise, mental clarity and emotional well-being.  I’m not sure—can you all see my pointer on here?  

*Moderator:*No, but if you go to the top of the screen there is an arrow that points down to the lower right.  If you click on that—yep, you’ll get a big green arrow right there.

*Dr. Gidwani:*Right.  Okay.  Here we have our four variables that constitute health, and you can see that Jane and Joe have the exact same functional status here.  They’re equally able to perform activities of daily living.  Equally able to exercise.  Have equal mental clarity as seen by the 0.4.  Equal emotional well being.  Where they do differ is in terms of their valuations.  Here we’ve asked Jane and Joe to tell us how important they think each one of these four components is in determining health, and you can see here that Joe feels that activities of daily living represent the most important part of health.  

Jane feels that exercise and mental clarity for her are equally important and most important in representing health.  When we are calculating utilities we’re looking at both the health state that somebody is in—so here are the activities of daily living of 0.8 of Jane—as well as her valuation of that health state.  Here we see that Jane values activities of daily living as being 15 percent of her entire valuation of health.  What we do is we multiply her health status times her valuation of that health status.  

Thumb across each one of these different sub-types of health and add them together in order to get her utility score, which here is 0.405.  For Joe, it’s 0.655, so Jane and Joe have the same health status, but because they value these health statuses differently, they have a different overall utility.  Now I want to point out that in reality we’re not going to ask Jane and Joe to give us this valuation.  We’re actually going to get this valuation from the community sample and apply it to both Jane and Joe.  I’m just using this example for illustration of what health versus the valuation of health means.

Oh, I see.  Looks like we had some formatting issues.  Okay.  Moving on.  We have Jane and Joe’s utility, and now we need to actually derive a QALY or a quality adjusted life year from that utility.  We know from the previous slide that Jane’s utility is 0.405, and Joe’s utility is 0.655.  Here’s an example of Jane living for ten years, and if she lives for ten years, then she has 4.05 QALYs.  If Joe lives for ten years he has 6.55 QALYs.  Now what happens is Jane actually ends up living for 12 years we find out, and so therefore her QALYs are 4.86.  

Joe only ends up living for five years, and therefore his QALYs are 3.275.  People can get different QALYs in many different ways, so even though Jane had a lower utility than Joe, she ended up getting a greater number of QALYs than Joe did because she ended up living for a greater number of years.  The advantage of using these utilities and then deriving QALYs from them is that they incorporate morbidity and mortality into a single measure.  The utility’s going to give us the estimate of the morbidity, and the number of life years is going to give us the mortality component.  

We combine the two together to get the QALY.  When we use the QALY we can compare really different interventions.  Let’s say that we are a health care system, and we have enough money to either fund newborn screening or prostate cancer treatment.  We need to decide between the two which one represents the better value for money.  Obviously these are treating extremely different populations.  The outcomes of newborn screening may be something like the ability of the child to learn or engage—develop properly.  The outcomes of prostate cancer treatment would be maybe the number of cases of cancer that have been properly treated or gone into remission.

Very different outcomes, but if we use the QALY—the quality adjusted life year—we actually have a singular common metric that we can use to compare these very disparate populations.  That could happen across any number of interventions.  It could be a non-medical related intervention.  It could be something like early childhood education versus community health centers—and which one represents the greatest number of quality adjusted life years if we fund one intervention versus the other.  When we’re thinking about cost-utility analysis we also have a result in the form of an ICER.  

The difference between the ICER I showed you before is in the denominator.  The numerator’s the same.  It’s the incremental cost.  The denominator is an incremental health effect where the health effect is specifically in the form of QALYs.  In a cost-utility analysis if we have an ICER that’s less than $50,000.00 per quality adjusted life year, it’s often considered cost-effective.  We’ll chat a little bit more about this later.  Here’s just an example of ICERs in cost-utility analysis.  Again, the ICER is going to be the difference in costs versus the difference in QALYs across the interventions.

Here let’s say we have two interventions, and one of them is a mobile text messaging application for medication adherence in diabetic patients.  The other intervention is a diabetes care coordinator.  We know that the mobile text messaging application costs $40,000.00, and the diabetes care coordinator costs $150,000.00.  Let’s say we’re studying 50 patients, and we see that of these 50 patients that were given—50 patients each that were given these interventions—we see that the mobile text messaging application yields a QALY of 25.  The diabetes care coordinator yields a QALY OF 35.  

Here we know just from this table that the diabetes care coordinator costs more money and provides more health effect.  What we’re interested in understanding is whether the additional money we’re spending is worth it.  Here, in order to calculate the ICER, we would look at $150,000.00 minus $40,000.00 in the numerator.  It would be 35 minus 25 in the denominator.  That would yield $110,000.00 over 10 or an ICER of $11,000.00.  Like I said before if that ICER is below—around below $50,000.00 for quality adjusted life year—it’s often considered cost-effective.  Here we would say that, yes, this is cost-effective.

Now before I talk about the threshold of $50,000.00 for quality adjusted life year, let’s talk about this cost-effective versus cost-savings situation.

*Paul Barnett:*Risha, this is Paul.  I was wondering if you could back up to that last line for a second.

*Dr. Gidwani:*Sure.

*Paul Barnett:*A question—some things that I’ve had people say, “Well, so Program A—it cost $40,000.00—has 25 QALYs.  Isn’t its cost-effectiveness $40,000.00 divided by 25?

*Dr. Gidwani:*We’d have to figure out what that is in relation to before we come to that conclusion.  We can either say that Program A is cost-effective relative to a standard of care, which may just be providing medications to patients.  It could be cost-effective relative to a do-nothing approach.  Remember I mentioned before that that do-nothing approach is still going to have cost consequences to it, even though it may not have any cost associated with the intervention.  If you do nothing these diabetic patients are going to get sicker over time.  

Therefore they’re going to be using the health care system, and you need to accommodate those costs, so it’s not going to be $40,000.00 relative to zero dollars.  It’s going to be $40,000.00 relative to whatever the diabetes related costs are of the do-nothing approach.

*Paul Barnett:*Great.  Thank you.

*Dr. Gidwani:*Okay, so cost-saving versus cost-effective.  This is I think one of my biggest pet peeves.  If you come away with nothing else today, I will very much hope that you really understand that when you use these terms they mean disparate things.  Here’s a table that will summarize what these terms mean.  Cost-saving means that a new intervention costs less than the old intervention, and it provides greater health.  In this situation it’s a win-win.  We’re spending less money, and we’re getting more health.  That’s fantastic.  What is more frequently occurring is that things are cost-effective.  Meaning that they—meaning one of two things.  

Either the new intervention costs more than the old intervention, but it provides proportionally more health, or the new intervention costs less than the old intervention, and it provides proportionally less health.  If the new intervention costs more money and provides greater health than the old intervention, it’s an increase in spending that we’re willing to accept if we say that it’s cost-effective.  In the second situation, if it costs less and provides proportionally less health, it has to be a reduction in health that we are willing to accept.  

What we most often see in health care is this situation where the new intervention costs more, and it provides that greater health that we think is proportional.  Cost-effective Program B costs more than Program A, but Program B provides proportionally more health benefit than Program A.  What do I mean by proportional?  That means that the ICER is less than a willingness to pay threshold.  In the United States a willingness to pay threshold is often considered $50,000.00 per quality adjusted life year.  Sometimes you’ll see $100,000.00 per quality adjusted life year.

That means that we are willing to pay up to $50,000.00 for one additional quality adjusted life year.  This is often what you’ll see bandied around in the literature, but it’s actually a pretty arbitrary threshold, and it’s been heavily criticized.  One thing to keep in mind is that this is not an empirically derived threshold.  This $50,000.00 per quality adjusted life year was first published in the literature of 1992 and began to be used more frequently around 1996.  A few papers used this threshold.  Even though they themselves said in their papers that this was an arbitrary threshold, it gained traction in the literature and began being used.

It’s something that you often see today.  It has not been updated since 1996 even though we know that inflation and medical inflation has risen since then.  One of the things to do when you do a cost-effectiveness analysis is to think about varying this willingness to pay threshold, because it is not empirically derived.  There are people that use different types of thresholds when you go outside of the United States.  For example, the World Health Organization uses—says—recommends that you should use one to three times the GDP per capita as a willingness to pay, but this also doesn’t have any theoretical justification.

There is a panel on cost-effectiveness in health and medicine that published gold standard recommendations for cost-effectiveness analysis in the 90s.  Dr. Phibbs is going to go into these gold standard recommendations in his next lecture, which I believe is next week.  In this book this panel did not actually endorse any explicit willingness to pay threshold.  We know that in the UK the National Institute for Clinical Effectiveness or NICE—which does use cost-effectiveness analysis when they are making market access and reimbursement decisions for new technologies—NICE in the UK does not have an explicit threshold for reimbursement.  

They do recommend that results are presented using a willingness to pay threshold of £20,000.00 to £30,000.00 per quality adjusted life year, but even they themselves don’t use any hard and fast rule of a willingness to pay threshold.  In actually my—the last lecture that I give—there is—we’re going to talk about sensitivity analyses for cost-effectiveness analyses and talk about how to vary this willingness to pay threshold and see how your results change in response to varying the threshold.  We’ve chatted about cost-effectiveness—

*Paul Barnett:*There are—

*Dr. Gidwani:*- analysis, and now I’d like to turn our attention— 

*Paul Barnett:*- there are a couple of questions—

*Dr. Gidwani:*-  to cost benefit analysis.

*Paul Barnett:*- sorry, Risha.  There are a couple of questions that have accumulated.

*Dr. Gidwani:*Okay.

*Paul Barnett:*Maybe we can—and so one of them was just about—it said, “Who decides the $50,000.00 per K threshold?”

*Dr. Gidwani:*Like I said nobody has actually—no agency or organization has put their stamp of approval on this threshold.  It’s, again, just something that you’ll often see in the literature that people conclude something is cost-effective because it’s met this threshold.  That really—there’s an interesting article by Scott Gross that assesses the cost-effectiveness—the history of this quality threshold.  Essentially he follows how this has come about.

Some people have said, “Oh, this comes from looking at the cost of end stage renal disease and dialysis for these people from the 1980s and Medicare thinking—Medicare reimbursing that.  Therefore it’s Medicare, which is societally funded money, is reimbursing dialysis.  This threshold.  Anything else should also be considered cost-effective if it meets this threshold.  That’s actually not where this threshold came from.  It’s really just something arbitrary that popped up in the literature and that the folks that presented it first said is arbitrary.  

We’re just choosing this cut-off point, and it’s ended up just gaining traction.  I think the real takeaway from this is that even though you’re often going to see this willingness to pay threshold considered cost-effective, you really want to vary this.  A lot of people nowadays will present results—they’ll just present the ICER.  Then you yourself can conclude whether you think it’s cost-effective, or they’ll say, “Okay, this is cost-effective at a threshold of $50,000.00 per quality adjusted life year.”  

It may not be cost-effective at a threshold of $100,000.00 per quality adjusted life year.  Again it’s an arbitrary threshold that has not actually been determined by any particular organization or empirically derived analysis.

*Paul Barnett:*There is that World Health Organization yardstick if you will. 

*Dr. Gidwani:*Right.  Right, but like I said that also is not—that does not have an empiric justification behind that either.

*Paul Barnett:*No, but I think it’s worth noting.  The World Health Organization says—it’s roughly—the threshold is per capita GDP—Gross Domestic Product.  I think that the thinking there is—well, I think the significance of that is just to say that it varies across countries, and that in some countries the thresholds are actually quite low in poor countries.

*Dr. Gidwani:*Mm-hmm.  It can be.  Yeah.  Then they also do recommend that it can be between one to three times per capita GDP.  That can of course vary quite a bit within country as you go from one to three times that GDP per capita.

*Paul Barnett:*Then there was one other question which I think would be worth talking about here is basically—does VA use QALYs in making its decisions?

*Dr. Gidwani:*Well, I think cost-effectiveness analysis is something that the VA is interested in.  I’m afraid I don’t know enough to speak on behalf of certain VA decision makers of what their willingness to pay threshold is.  That’s not something that I’m aware of or privy to.

*Paul Barnett:*I think the answer to that is two answers: one is sometimes.  The other answer is—I think the last lecture in the series is just about who uses cost-effectiveness analysis and how can we increase its acceptability to decision makers.

*Dr. Gidwani:*Okay.  Alright, so let me move on to cost-benefit analysis now.  In cost-benefit analysis we are looking at costs and health effects as well, but the big difference is that we are expressing those health effects entirely in dollar terms.  We convert all the health effects that we measure into costs.  What we do is we look at the incremental benefit in the form of costs minus the incremental costs of an intervention in order to understand the net social benefit.  If the net social benefit is a positive number, then we consider the program to be worthwhile.  

What we’re doing here in cost-benefit analysis is that we’re putting a dollar value on health.  For this reason it’s used less frequently in health care as people are uncomfortable with placing a dollar value on human lives and on the quality of life.  It’s more often used in environmental economics.  I do want to point out that cost-benefit analysis is more than just a return on investment analysis.  A return on investment analysis would evaluate the intervention—financial input versus an intervention’s financial output.  

A cost-benefit analysis can go beyond this including information about the quality of the health effect or safety issues or externalities that go beyond the intervention.  For that reason it’s more comprehensive than a return on investment analysis.  One thing to keep in mind here is even though we’re looking at cost versus benefit, we’re not going to actually express this as a ratio of cost to benefit.  We’d rather express the results of the net social benefit.  

The reason for this is because if we express the outcome as a cost-benefit ratio, it can be confusing because cost can be inconsistently placed in the numerator or the denominator.  An example of this would be the cost of the disease avoided could be put as a benefit, or it could be considered a negative cost.  It could be in either the numerator or the denominator.  Depending on where you place it, it can lead to different results.  If we calculate the net social benefit we avoid this problem, and if the incremental benefit is greater than the incremental costs, then we consider the program to be beneficial.  

Of course cost-benefit analysis yields this large problem of how do we actually assign this dollar value to life.  There are two large schools of thought about the way in which this can occur.  The first is using willingness to pay, and the second uses a human capital approach.  In a willingness to pay you can either examine revealed willingness to pay, or you can elicit willingness to pay.  An example of revealed willingness to pay would be seeing, for example, how much people will pay out of pocket for lasik eye surgery, which is not covered by insurance.

We’re actually observing how much people spend on a particular service and consider that to be their willingness to pay.  One thing to keep in mind though is this would be an upper bound estimate in the example of lasik, because the people who would not—who would be willing to pay a lower amount wouldn’t actually be able to purchase this service.  If we want elicit willingness to pay, we can ask people to choose between different scenarios and state their probability of achieving a desired outcome with or without the intervention.  Then ask for their willingness to pay for the intervention.  

We’re asking them to respond to a hypothetical scenario on how much they would be willing to pay in order to achieve a certain health outcome.  Whether you use revealed willingness to pay or you elicit willingness to pay, the idea is that you aggregate willingness to pay across individuals in order to have a societal value of willingness to pay.  There are issues with willingness to pay.  There’s issues of framing and, specifically, loss aversion.  People would rather avoid a negative outcome than get a positive outcome even when the expected value of those two outcomes is the same.  

There can be age-related effects.  People that are younger may have different attitudes towards spending money than people that are older.  Then of course the attitudes or ability for the elicit willingness to pay or the revealed willingness to pay can differ with differing levels of disposable income.  There’s—willingness to pay has some operationalization issues that one needs to consider.  The human capital approach is different.  What it does is it eases projected future earnings in order to value a life.  

You might go to, let’s say, the Bureau of Labor Statistics data and understand how much—what the average salary is for a person.  Then multiply that times the number of years that that person is expected to live, and then that’s an estimation of the value of that person’s life.  Some concerns that have been brought up with that is that this of course assumes that an individual’s value is entirely measured by formal employment.  Therefore we run into problems when we’re evaluating interventions for children or retired people who are of course not formally employed.  

Then we know that there are pay differentials between men and women that have the same education and the same job.  We know that there are pay differentials across different races as well, so those are some sticky points that need to be considered using a human capital approach.  In health care and medicine—like I said before—we don’t often use cost-benefit analysis.  People struggle to assign dollar values to life.  We touched just on the previous slide on some of the issues that come into play in doing that, and then of course there’s also these ethical concerns as well—of explicitly assigning a dollar value to life.  

Then of course there’s issues with evaluating quality of life.  You can have two people that have the same willingness to pay or have the same valuation of future earnings based on the human capital approach, but if they’re in very different illness levels, then their quality of life is going to be very different.  These approaches are not going to capture that.  I’m going to move on now to budget impact analysis, which we see more frequently used in health care.

*Paul Barnett:*Before you move on—

*Dr. Gidwani:*Budget impact analysis—

*Paul Barnett:*- sorry, Risha, to interrupt.  There was a question about human capital approach, which was how would a dollar amount for the human capital approach or willingness to pay be calculated for the disabled vet?

*Dr. Gidwani:*- well, so, I mean, this is essentially a situation in which you’d have a very difficult problem if this person is not—for the human capital approach if they’re not formally employed by society you’d run into some issue.  You’d have to proxy that and try to find maybe somebody—maybe you’d look actually at disability payments could be one approach to doing that.  I am not an expert in the human capital approach, so I don’t—I don’t want to pretend that I am, but you would—yes.  I think if we’re just looking at the amount that they’re going to be earning, that earning would be probably a disability payment.  

In the willingness to pay you could certainly ask them and elicit their willingness to pay for achieving a certain health status.  Again just keeping in mind that they may have a low level of disposable income that could influence their willingness to pay.

*Paul Barnett:*I think this is—this gets at the whole issue.  Why do we use QALYs and why we don’t value lives in dollars based on human capital approach—is just because health care analysts make this assumption that every life year and quality adjusted life year is equivalent regardless of who enjoys it.

*Dr. Gidwani:*Mm-hmm.  Right, and the human capital approach of course would consider somebody that earns a lot of money to have a higher value life than somebody that is disabled and not earning as much money.  Okay, so moving on to budget impact analysis.   A budget impact analysis estimates the financial consequences of adopting a new intervention.  You can estimate those financial consequences both on a global budget and on sub budgets.  For this reason it’s often welcomed by operational folks who have to make the real world decisions of deciding how exactly an intervention is going to be funded.  

A budget impact analysis is usually performed in addition to a cost-effectiveness analysis, so the complement to cost-effectiveness analyses rather than a substitute.  The cost-effectiveness analysis answered the question, “Does the intervention provide good value?”  Whereas the budget impact analysis answered the question, “Can we afford it?”  Here’s an example.  We have a situation where Drug A from a cost-effectiveness analysis has an ICER of $28,000 per QALY compared with Drug B.  We’re going to consider Drug A to be cost-effective.  

We know Drug B costs $70,000.  Therefore in order to gain an additional ICER, you are paying $98,000 for Drug A.  Now we know that there are 10,000 people eligible for Drug A.  That means that the total cost of providing Drug A to our population is $980 million.  That’s a lot of money, and we need to figure out whether we as a health care system can afford it.  That’s what the budget impact analysis tells us.  It tells us the true cost of an intervention.  We’re looking at—the cost of the intervention is a function of the unit cost of the intervention.  

Maybe the cost per person of an intervention times the number of people that are affected by the intervention.  That tells us the total budget that’s going to be required in order to fund this intervention.  In terms of cost-effectiveness analysis versus budget impact analysis, the purpose of the cost-effectiveness analysis is saying, “Does this new intervention provide higher value compared to the old intervention?”  The budget impact analysis asks, “Can we afford the new intervention even if it does provide higher value?”  The outcome of a cost-effectiveness analysis is in the form of cost relative to health outcomes.  

The budget impact analysis only looks at the cost of the intervention, because we’ve already decided that the cost relative to the health outcome is favorable.  Now we just need to know the true costs when we apply this intervention to everybody.  Since we’re going to apply this intervention to everybody, we explicitly consider the size of the population in the budget impact analysis, which we don’t do in a cost-effectiveness analysis.  There’s going to be more information about budget impact analysis.  Dr. Patsy Sinnott is giving an upcoming lecture in this cyber course about it.

This is just a brief overview, but in order to really understand the specifics of budget impact analysis and how to operationalize it, I’ll encourage you to attend her upcoming lecture.  We talked about cost-effectiveness analysis and cost-utility analysis, cost-benefit analysis and budget impact analysis.  Now I want to talk about approaches that we can use to operationalize these different types of decision analysis techniques.  There are two large categories of methods that we use to operationalize the decision analysis.  

They fall into modeling or measurement alongside a critical trial.  In a cost-effectiveness analysis or a cost-utility analysis, you can use both approaches as you can also do in cost-benefit analysis.  A budget impact analysis however is relegated to only using modeling when you’re operationalizing it, and the reason is because you’re projecting costs for your population and a critical trial will never be conducted for an entire population.  When we do measurement alongside a clinical trial, we are simply piggy-backing our health economic analysis onto an existing randomized control trial.  

The trial is going on, and they’re collecting information about efficacy and adverse events, and we’re just collecting extra information on the same patients enrolled in this trial.  We’re collecting information on costs and utilities.  In terms of costs usually what we’re doing is measuring health care utilization, and then translating that into costs, and those are considered direct medical costs.  Depending on the information you collect you may also be considering information like care giver costs, which are direct non-medical costs.  In terms of utilities you would query the patients at baseline and then also at any follow up time in the clinical trial.  

When we do modeling as opposed to doing measurement alongside a clinical trial, we’re doing so because there is no real world experiment that exists.  That could be because there is no randomized control trial that’s evaluating your interventions of interest, or it can be that it’s unethical to create a randomized control trial.  For example, we’d never randomize people to smoking versus not smoking.  If we wanted to understand the impact of smoking and different frequencies of smoking on costs and health outcomes, we would have to build a model, which is a mathematical framework, in order to understand the relationship between inputs and outputs.  

When we do a model what we do is we build a model structure in the software.  We populate it with inputs from the literature, and then we’ll run the model to derive our output.  Because you yourself are creating this model from start to finish, you’re not depending on the existence of a randomized controlled trial to piggyback your data analysis onto.  Rather in your modeling you decide on the boundaries of the analysis, so you can look at whatever interventions are of interest to you.  You can decide on the timeframe and the population.  

There are a multitude of models that one can build.  You could do a decision tree.  A state transition model.  A micro simulation.  These are going to be discussed later in a modeling lecture that’s given by Dr. Jeremy Goldhaber-Fiebert.  Let’s summarize modeling versus measurement in a single slide. When we’re doing measurement alongside a clinical trial, we can only consider the treatments that are being evaluated in the randomized controlled trial.  Randomized controlled trials, especially those that are conducted in the United States, are often going to include placebos.

That’s something that limits the utility—the usability of a randomized controlled trial when doing, let’s say, cost-effectiveness analysis.  In a cost-effectiveness analysis you’re trying to understand which intervention would best help patients, and the choice is not going to be between an active treatment and a placebo.  It’s going to be between one active treatment and another active treatment.  Because randomized controlled trials in the United States often do not compare two active treatments in a head-to-head randomized controlled trial, we oftentimes have to use modeling instead in order to understand the relative value of two active treatments.

In modeling we can consider any treatment of interest.  It’s not that we don’t used randomized controlled trial data.  We do, but we would just treat it as a model input as opposed to building our entire data collection activities around a single randomized controlled trial that’s being conducted at the same time.  One of the advantages of measurement alongside a clinical trial is that you are able to collect a lot of rich information.  You can help design the case-report forms.  In terms of, let’s say, health care utilization, you’d be able to collect a lot of specific information that you think is of interest.  

You also are going to have the individual patient level data.  Therefore you’re able to do whatever analyses you might think are important such as subgroup analyses.  If you collect utilities yourself when you’re doing measurement alongside a clinical trial, you have both advantages and disadvantages to doing this.  The advantages of collecting utilities is that they can be more accurate.  Because they’re not just going to be health condition specific as they would be if you had plucked them from the literature, but they can also be treatment specific.  

If a particular treatment has really terrible side effects that are also affecting somebody’s health related quality of life, when you elicit utilities yourself and measure them alongside a clinical trial, you’re going to be able to pick up on those treatment related side effects that could also be of interest.  An advantage of modeling—I know I’ve only listed one here, but this is such a huge advantage that it trumps a lot of other things.  That’s that you don’t need to wait for a trial to be funded to do your analysis.  Research funding is getting tighter and tighter.

The ability to answer your questions of interest from a model that you build and populate entirely from the literature can be a big advantage.  Meaning that you can actually get it done in a reasonable time frame.  Disadvantages to measurement alongside a clinical trial is that clinical trials do not last forever.  Patients are only going to be enrolled for a particular, relatively short time horizon, and best practices for a cost-effectiveness analysis as we’ll see in our next lecture by Dr. Phibbs is that you should really incorporate a lifetime time horizon for your patients.

Because the patients in the clinical trial may only be followed for, let’s say, two to three years, you’re still going to have to project data beyond the trial.  The measurement alongside the clinical trials is also not going to provide every single one of the data inputs that you need in order to truly understand, let’s say, the cost-effectiveness of one treatment versus another.  You’re still going to have to go to the literature for some other information.  Here’s the disadvantage of utilities.  I previously mentioned an advantage of collecting utilities when you’re doing measurement alongside of clinical trial.  

The disadvantage here is that utilities are going to come from a patient perspective rather than a community perspective.  Now Dr. Sinnott will get more into this in her utility lecture, but you can essentially elicit utilities or preferences from health from people that have the condition, or you can elicit them from a representative sample of the community.  Where people that have the condition will be—if you’ve done the sampling properly, the people that have the condition will be represented in a frequency of proportion to what we see in the population.

What we do know is that when we query patients about their utilities, we end up with higher utilities than if we had presented a hypothetical situation to people in the community.  Let’s say, for example, people in a wheel chair will give us a higher estimate of utility if we ask them directly what their utility is versus if we present a hypothetical situation about being in a wheelchair to a sample of the community.  They are going to give us lower utility estimates than the people that actually are in the wheelchair from living with that condition.  

*Paul Barnett:*Risha, this is Paul.  Can I just interject something here because there was just a question about this that I answered, and we gave different answers *[laughter].*In a trial sometimes we use these off the shelf measures that are surveys, and the person just answers objective questions about their health status.  There is a scoring system that uses community ratings of that health status.  What you said is true if we were to use a direct measure like standard gamble or time trade-off, but as a practical matter those are hard to apply in a trial.

We use a survey like the health utility index or the euro qual or the quality well being scale.  That allows us to get around this problem of patient versus community rating.  We can gather information in a trial that applies community preferences, community ratings and utility.  If we use one of those off the shelf surveys. 

*Dr. Gidwani:*That’s correct, so I should be careful here.  There’s direct and indirect elicitation of utilities.  The indirect is what Paul is referring to.  If we give the patients in the trial a survey, and then the—and so from the patients in the trial we get an estimate of their functional status as related to health.  Or their health status I should say.  From the community we get a valuation of each one of those particular health states, and we combine the two.   That would mean that we—

*Paul Barnett:*Those community ratings are just—they come with a survey.  You don’t have to do that survey.  It’s been done for you.  They’ve already surveyed several thousand people in the community about what they think about the status of being in a wheelchair.  The status of being blind or whatever the functional limitation is.

*Dr. Gidwani:*- right.  Right.  That’s correct.  If you do a direct elicitation of utility from the people in your randomized controlled trial, which you can do using a time trade-off, a standard gamble or the left well respected visual analogue scale, then it will be entirely from the patient perspective rather than from a combined patient community perspective.  Those are often frowned upon.  For example, NICE in the UK only want to see cost-effectiveness analyses where the utilities come from the community sample.

You couldn’t just directly elicit utility from patients in your randomized controlled trial and consider that to be an adequate input for a model that you’re submitting to them.

*Paul Barnett:*We should acknowledge—that went by very fast, and there’s a whole lecture on this, so *[laughter].*

*Dr. Gidwani:*Think of this as an issue of breadth versus depth.  I only have seven minutes left, and I still have a few more slides to get into, so I think like I mentioned before it’s a very vast literature.  It certainly merits further understanding if you’re going to operationalize a cost-effectiveness analysis.  Dr. Sinnott has a very nice lecture that will be upcoming in a few weeks about this where she’ll get more into it.  Alright, just to quickly move on.  When we are doing modeling the disadvantage—again I’ve only noted one here, but it is its own large disadvantage.  

The disadvantage of modeling is that you have inputs that are coming from a number of different sources.  You have a multitude of model inputs, and you’ve got to be able to populate each one of those.  You go to the literature in order to derive each one of the—in order to obtain each one of those inputs.  The problem with doing that is that if you go to different studies in the literature there —may have really different populations or different follow up times, or different interventions.  You need to select studies from the literature for model input where the studies are as similar as possible to your question and population of interest.  That can at times be a very challenging process.

Okay, in the few—oops—sorry, it looks like we have some formatting issues here.  In the few minutes I have left I just want to briefly talk about how cost-effectiveness is viewed for resource allocation and take a step back and think about the practicality of this work in decision making in the real world.  Ex-US cost-effectiveness analysis is pretty frequently used for regulatory and market access purposes for new technologies, which are often new medications, new pharmaceuticals or new biologics.  NICE in the UK, PBAC in Australia, CADTH in Canada regularly use these for regulatory purposes.  

You’ll find payers in regulatory groups in a number of European, especially western European countries, that are looking at results from cost-effectiveness analyses when they are deciding what to allow onto their markets or how to fund what is allowed onto their markets.  I’m sorry, how to reimburse what’s allowed onto their markets.  In the United States even though a lot of the methods for cost-effectiveness analysis come from the United States and the US academic world, US health care doesn’t really use cost-effectiveness analysis to make health care decisions like other countries do.  

Medicare has historically not used cost-effectiveness analysis to derive coverage decisions.  Actually with the passage of the Affordable Care Act, it has been explicitly prohibited.  Here is some text from the Affordable Care Act, and you can see that circled here, “The Secretary shall not utilize such an adjusted life year (or such a similar measure) as a threshold to determine coverage, reimbursement, or incentive programs under title XVIII.”  Title XVIII is the Act that specifies Medicare, so that’s essentially outlawing cost per quality adjusted life years as a threshold to determine coverage reimbursement or incentives in the Medicare program.  

In terms of who does use cost-effectiveness analysis in the United States, pharmaceutical companies are using this.  Many pharmaceutical companies are conducting their randomized controlled trials about their drug in the United States because the FDA only requires that their drug be compared to placebo.  There’s an incentive to conduct your trial in the United States, because you have a most likely ineffective comparison that you’re—an ineffective treatment that you’re comparing your pharmaceutical against.  The pharmaceutical companies are interested in getting their drugs onto international markets.

They need to present cost-effectiveness analyses to these ex-US audiences, but they’ll oftentimes contract the work in the US.  Academics of course use it, as well as the VA, which is very interested in cost-effectiveness analysis.  Again, it’s not used by Medicare, and it’s not used by the FDA.  In summary there are three major types of decision analysis.  There’s budget impact analysis, cost-benefit analysis, and cost-effectiveness analysis.  Cost-utility analysis is a particular sub form of cost-effectiveness analysis where the health effect that we’re measuring is the quality adjusted life year.  

That quality adjusted life year has an advantage of giving us a measure of both morbidity and mortality simultaneously.  If we want to operationalize the decision analysis we can do so either using measurement alongside a clinical trial or by engaging in modeling.  To emphasize, cost-effective and cost-savings are very different things.  Cost-saving means that a new intervention both costs less and provides more health benefit relative to an existing intervention.  Cost-effectiveness means that the—there is greater costs and a greater health benefit for the new intervention relative to the old, or that there is a lower cost and a lower health benefit of the new intervention relative to the old.  

The cost difference is proportional to the difference in health benefit that we’re realizing.  If you are interested in finding out more about decision analysis and cost-effectiveness analysis, there are three textbooks that I would recommend.  The first is the Gold et al. Panel on Cost-effectiveness in Health and Medicine.  Dr. Phibbs will be going more into the recommendations from this book in the next lecture, but if you are going to operationalize a cost-effectiveness analysis, these are the gold standard recommendations that you should follow.  

The next two books: Hunink and Muennig are just fantastic references, and I’d strongly recommend reading both of them if you’re interested in working more in this field.  With that, I will open it up to questions.  I know we only have a minute left, but hopefully we can get a couple of them at least.

*Paul Barnett:*We’ve had a couple requests for speaker notes, and I have just noted that the complete transcript of the—session will be on the HSR&D cyber seminar website, and it’s also possible to see it again.  There’s an archive version or to download copies of the slides themselves.  Someone asked could you please share a copy of the slides with us, and again—Heidi, I don’t know if you have something to add about that.

*Moderator:*The slides are available.  Unfortunately the entire VA website is down right now, so they will be available when the VA website is back up.  I will put the link up here.  Everybody should have just received that.  I’m not sure when it will be back up but hopefully by tomorrow morning at the very latest.

*Paul Barnett:*Well, but you did just put a link about where to get the slides—

*Moderator:*I did.  Yeah.

*Paul Barnett:*- in the— 

*Moderator:*In the Q and A box.  Yes, exactly.

*Paul Barnett:*- but the link is dead right this minute.

*Moderator:*The link is dead right now.  Yes *[laughter],* but it should be back up soon.  

*Paul Barnett:*That’s—it’s a snafu as they say.

*Moderator:*Yeah.  Now for those of you who are still with us, I just put up a feedback form on your screen.  If you could take a few moments to fill that out before you leave today, we would very much appreciate getting your feedback.  I know here at *[inaudible 59:43]* we read through it, and at HERC I know you read through this feedback that we receive for every session.  We use that for our current and upcoming sessions.  We really do react and respond to the feedback that you leave for us.  

It looks like we are just about at the top of the hour.  Risha, Paul— do either of you have anything—any final remarks before we wrap things up?

*Interviewer:*Just to remind people when Ciaran’s talk is.  

*Moderator:*Ciaran [Phibbs] is scheduled to present on April 23rd at 2 p.m.   I know a lot of you are registered for that session, but we will be sending additional registration information out on April 16th for those of you who have not registered.  Keep an eye on your email for that.

*Paul Barnett:*It’s two weeks from today, but it will start at one hour earlier than today’s talk did.

*Moderator:*Yeah, exactly.  Yeah.

*Paul Barnett:*The rest of the seminar will be at that earlier time—

*Moderator:*Yes.

*Paul Barnett:*- and on Wednesdays.  They’re basically every other Wednesday, but sometimes we’re—we break that pattern.

*Moderator:*Well, between the HERC monthly session and the cost-effectiveness, I think you have it going for 15 weeks straight.  Tune in here every Wednesday at 2 p.m. for the next talk*.*

*Paul Barnett:*Yes, but some of them are outside speakers, and the others are the course itself.

*Moderator:*Exactly.  Risha, did you have any final remarks before we wrap up?

*Interviewee:*No, I don’t.  Thanks very much, Heidi.

*Moderator:*Okay, fantastic.  Thank you so much for presenting today.  Paul, thank you very much for supporting Risha today, and for the audience—thank you for joining us.  We hope we’ll see—I’m sorry, Paul?

*Paul Barnett:*Lots of good questions there *[music playing].*

*Moderator:*Yes, a lot of great questions.  Thank you everyone for joining us for today’s HSR&D cyber seminar, and we hope to see you at a future session.  Thank you.

*[End of Audio]*

**Page 1 of 22**