Cost Effective Analysis- 23 -Department of Veteran Affairs

February 29, 2012

# Department of Veteran Affairs

**Cyberseminar Transcript**

**How Can CEA Analysis Be Made More Relevant to U.S. Health Care?**

**February 29, 2012**

Paul G. Barnett, PhD:  Glad that people could join us today for this talk on...   because this is the last session in our cost effectiveness course and directed at trying to figure out how cost effectiveness analyses that we do could be made more relevant to the U.S. health care system generally, VA specifically.  So the overview of the talk is just to briefly review what we mean by cost effectiveness analysis. 

And those of you that have been with us through the course should know this is the adjusted review, talk about how cost effectiveness analysis is being used in both the United States and in other countries, some of the barriers that people have identified to using cost effectiveness analysis to making health care decisions, and some things we can do to overcome those barriers, and then also to talk briefly about the distinction between cost effectiveness analysis and comparative effective analysis and even in a world where we only want to do comparative effective analysis how some of the tools that we have for cost effectiveness analysis can be helpful. 

So this is the review part.  So as you learned in the course when we do cost effectiveness analysis we are usually trying to identify the incremental effect of some innovative treatment and compare that to standard care.  We measure all the cost using the societal perspective and practical implications of the societal perspective as we want to be sure to include in the cost that the patient has incurred, travel or caregiver time, those sorts of things we want to include those because we wouldn’t want to favor an intervention that just shifts cost from the payer to the caregiver or to the patient and ignore those costs.  

We want to account for them too.  So we measure—we also identify all outcomes and express them in terms of quality adjusted life years, that is length of life adjusted for the health related quality of life, and of course with those specific tools that allow us to translate or measure them in a single scale of quality adjusted life years so that we can trade off quality of life with length of life.  

And then we do a lifetime look at it over the patient’s lifetime because we know that some interventions that are benefit them that may be years down the road, something like preventive services will incur the costs upfront, but not generate the benefits for many years later.  And then we discount both the cost and the outcomes to reflect the lower value associated with delay.  

We would rather have an improvement in our quality of life now than five years from now.  We would rather have a dollar market today than five years from today.  So we need to discount consistently both cost and outcomes.  

Once we have gathered all this cost and outcomes data then we test for dominance.  And that is just one of the interventions, both less costly and more effective whether it wins and we adopt it.  Or it could be weakly dominant, that is if the interventions are at equal cost but one is more effective or vice versa, and they’re equally effective and it’s less costly.  Then we choose that one.   

But it is often the case in health care is we are looking at something that is both more expensive and delivers some greater value and we have to decide whether that value is sufficient to justify the cost.  So we do that by calculating the incremental cost effectiveness ratio which is the next slide.  

We are looking at the increase of cost caused by the experimental condition compared to the increase in quality adjusted life years, the value that is generated.  And so we are basically looking at this kind of efficiency ratio.  What does it cost to get an additional quality?  What is our— and we have some critical threshold that the decision maker will look at.  So if an intervention costs $5,000 per quality that might be considered a good deal.  A million dollars per quality may be too little value to justify or certainly too little value to justify the intervention.  

Typically in this country people use these thresholds in the range of $50,000 to $100,000 per quality and other countries probably less, a lower threshold to justify, judge something that is cost effective.  And it has been observed by some that the critical threshold is approximately, in many countries anyway, the average per capita income.   

So now we turn out so what is the practical application of this?  Where can cost effectiveness analysis be applied?  So to a certain extent this can influence the decisions of both physician and patient, but we are really thinking cost effectiveness matter is in system wide decisions.  So we make decisions about what is the scope of benefits of the coverage?   

What drugs are in the formulary or what devices are approved for use in the plan?  What procedures are considered useful, and also in terms of guidelines for what is the optimal practice, especially in screening but also treatment.  We look towards cost effectiveness analysis to decide what are the screening treatment and management strategies that should be used.  

And so in other countries there are agencies that do this technology assessment and apply it to their health system.  So in Canada there is an agency for drug and technologies, a federal agency, and there are also prevention organizations that do this work.  And they advise the Canadian health plans in each province.  

The United Kingdom has its National Institute on Clinical Effectiveness and it advises the National Health Service.  In other countries cost effectiveness is being used to set the formulary to decide which drugs should be used in their health plan, so in Sweden, Australia and the Netherlands they have this kind of formulary rule.  

Germany has this relatively new Institute of Quality and Efficiency that is doing these kinds of assessments applying cost effectiveness and analysis methods.  And then France also does pharmaceutical reviews, but updates them periodically so in this respect I think it’s fairly unique.  

In some, in most developed countries health plans are applying cost effectiveness analysis.  It is used in coverage decisions for drugs and technology.  It doesn’t mean that the cost effectiveness findings are always followed and sometimes they are not, even despite something being shown as yielding relatively low value for the cost it may still be adopted.  And I will talk about that a little bit later on in the talk.  

And there aren’t so many examples we can show that things have been outright rejected based on cost.  More often it is just that they are restricted to certain patients.  There are some things that have been ruled out by application of cost effectiveness analysis.  

And also the last time I looked at this, and I haven’t updated this I must admit in the last year, but there haven’t been very much in the way of formal evaluations of how technology assessment has impacted health care.  Just some studies that looked, compared how well the decisions are correlated at different countries.  There is a paper looking at Canada, Australia and the U.K. drug decisions and find that they are not always coming to the same conclusion.  There is some correlation but not perfect correlation.  

So in the United States we have what we use cost effectiveness analysis to a much lesser extent.  So we have a history that in 1989 Medicare proposed using cost effectiveness as part of the criteria that they would use to make coverage decisions.  And those regulations precipitated about a decade of very contentious debate and they were regulations, proposed regulations were withdrawn.  And so currently the Medicare Coverage Advisory Commission really doesn’t have a mechanism to consider either cost or value when it makes its coverage decisions.  It just considers effectiveness.  Does this treatment work?  But there are some subtle ways in which cost of value do enter into their decision making.  

Another example of application of cost effectiveness in the U.S. is the Oregon Medicaid program.  And this is a somewhat famous example that nearly twenty years ago, actually maybe more than twenty years ago now they attempted to restrict expensive treatments and basically Oregon set a priority list of things that they funded and said these are things that are very high value for their cost.  These are things that are low value for the cost and they tried to draw a line and say we are not going to do some of these low value things.  

And it precipitated a great political controversy and they backed down from it at the time, but I observe now that Oregon Medicaid programs still prepare such a list.  And I understand from folks who are involved with that is that the managed care organizations who provide contract managed care services for Medicaid in Oregon do consult that list and it does influence their coverage decisions.  

So even though the Oregon experiment is generally regarded as not having panned out with using cost effectiveness it is actually still has some impact of the cost effectiveness considerations on how Medicaid spends, how the Oregon program spends its resources.  There also was a survey done on managed care plans in the U.S. that indicate that they consider cost in making coverage decisions and that many managed care programs used some sort of formal cost effectiveness analysis in at least is making some of their decisions.  

Q:  Paul,— 

Paul Barnett:  Yes. 

Q:  — I just wanted to add that most managed care organizations actually have contract language that allows them to use it that is in their policies, but that doesn’t mean that they use it.  

Paul Barnett:  So people they seem to be aware of it and to a greater and lesser degrees thinking about it.  So I had this up for discussion.  Unfortunately we don’t have the discussion, but we were thinking about what are some of the potential objections that have people observed for using cost effectiveness analysis.  We don’t have a white board anymore do we, Heidi?  

Q1:  We don’t.  But if we would like to have some form of discussion people who are on their phones we can have those lines un-muted.  And people who are just using their computer audio we could have them type into their Q&A any comments that they have here.  So we do have— 

Paul Barnett:  Well so maybe the Q&A would be a good way to see if we could get people thinking about what are some potential objections to using cost effectiveness analysis.  

Q1:  And while people are starting to type that into, use the Q&A screen on the right-hand side of your screen to type that in, Paul, we did get a comment in.  I am not sure if you are using your handset on your phone or if you have a headset, but we are getting some comments that it’s a little hard to hear you where you are kind of fading in and out a little bit.

Paul Barnett:  Okay.   

Q1:  Okay.  We’ve gotten one—we are getting a couple of responses here versus political, the whole [rasseting] controversy, limited data to make good analyses about effectiveness, defining costs and benefits, fairness.

Paul Barnett:  Fairness?  

Q1:  Fairness.  

Paul Barnett:  Yeah, so is it fair to certain groups, disadvantaged groups perhaps?  Yes.  These are very good observations, yes.

Q1:  Worker productivity showing men and youth with higher value.  

Paul Barnett:  Yeah.  So that’s, yeah so both of those have to do with the question do we regard all quality-adjusted life here as just equal I think.  I think that’s what those are driving at.  

Q:  Defining what is a reasonable ICER?  

Paul Barnett:  So that is what is that do we have a good threshold for deciding what is enough value to justify the cost.  These are all great comments.  Whoops.  So do we have any others to add?  

Q:  There is one more that says limitations on the scope of factors considered in the CEA.  

Paul Barnett:  So some thought that maybe we don’t have—so that could be maybe we haven’t measured all the value that is generated, and maybe there are just some deficiencies in how we estimate our incremental cost effectiveness ratio otherwise, something about the bottle or the assumptions that sort of thing it could be.  

Well let me show some information about what decision makers say about their objections.  So the last time I looked at this there were at least sixteen different surveys of decision maker attitudes about health economic studies, specifically about cost effectiveness analysis.  And these I will just characterize what they say about the concerns.  

One is just simply is that they don’t understand what cost effectiveness analysis is or what it is trying to accomplish, but even but there are decision makers who are actually quite sophisticated in their understanding, but don’t trust the methods that they don’t have confidence that quality-adjusted life years or qualities of measuring what they report to, and that they lack confidence in the models that are used to extrapolate cost and benefits into the future.   

And then another concern is that the information in cost effectiveness analysis is just simply insufficient because it doesn’t really take decision makers’ own context, which is they are concerned about the very short term, what’s going to happen in the next year or two, and how is it going to affect my particular budget as a health plan of my particular health plan?  So they are not so interested in what costs are being shifted to the patient or what is going to happen in the out years.  

And there is also a concern about some studies having bias imposed by the sponsor.  And I think some of the classic, some of the countries that require drug companies to submit cost effectiveness analysis, part of the formulary decisions are certainly worried that they are not getting an objective statement of what cost effectiveness is, but that extends to this country.  

And I would add to this is that also sometimes we see studies where someone has developed some sort of innovative care and it’s really someone who is a proponent of that innovation that is doing the cost effectiveness analysis that there’s not the dispassionate evaluation that is really needed to make it credable.  So those are all some of the concerns the decision makers have expressed.  

People alluded to some of these other concerns about political attitudes.  I think one is that we have this American attitude that we really don’t want to have “government decision makers or corporations making decisions about health care.” Somehow this is a fundamental human right, and unwillingness to really concede that resources are limited that we ought to be able to get any health care that is effective.   

And I should add to this one on distrust.  I think there is also a distrust of experts that of making decisions lieu of the patient or the doctor.  So I think these are other concerns that are expressed about cost effectiveness analysis.

So how do we address these barriers and improve acceptance?  And here are some thoughts that have been expressed about that, first from the International Society on Pharmacoeconomics and Outcomes Research, ISPOR.  

They had a task force just considering this very problem.  How do we get better acceptance of cost effectiveness analysis.  And they have some recommendations about what we as scientists are doing in these and actually do to improve the acceptability.  

One is if we describe how many people are involved, what is the relative population that would get this intervention, and how many folks, and who are they and then to do a budgetary impact analysis.  And that should include exactly whose budgets will be affected as a health plan.  Is it the patient?  So we want to not know not just that we are buying a quality for $10,000, but we want to know how many qualities we have to buy.  

ISPOR group also recommended that we disaggregate cost and outcomes.  And so by this aggregated costs we mean so perhaps what is going to happen is that if we dropped a drug we will reduce inpatient hospitalization costs.  So disaggregated means what is the cost of the drug, what is the cost of the hospital’s ambulatory care cost.   

We are not just simply interested in the total cost.  Disaggregated outcomes we might want to know is this involved, not just simply qualities, but how do the, why do the qualities change?  Is it because life years are extended, or quality of life is increased or are we trading off one for the other?  So it’s this disaggregation that we mentioned too.  

And then another key question is for is to look at different subgroups and what is cost effectiveness in different groups of patients because the cost effectiveness, incremental cost effectiveness ratio may not be the same in all groups.  And that is important information for the decision makers to have.

And then the final point here is just transparency.  The decision makers need to know what the assumptions, the data sources and also have confidence that the analysis has been looked at for whether the results are sensitive to any of those assumptions or data sources. So essentially the analysis would say which assumptions are likely to matter and which assumptions aren’t going to drive results.  That’s so especially important in modeling.  

And I think there is just a kind of a common sense thought about how do we improve acceptance of our cost effectiveness studies.  And I think one is just to be doing the right studies in the first place that we look at interventions that are expensive because those are the ones that are really of the greatest concern.  

And so that’s a little tricky to estimate sometimes a priori.  It might seem just like the big ticket item.  It costs a lot to provide it to one patient, but it also really is reflected on the frequency.  How many people did we provide it to?  So you might think cholesterol tests for instance really screening for cholesterol is not very expensive one by one, but when we do it for millions of people it becomes a big ticket item.

The other thing about choosing the right analysis is to involve the decision maker actually at the outset.  And in other countries the cost effectiveness analyses are actually commissioned by the decision makers.  They are anxious for the results.  

And so we should think about that as about as a sort of dreaming of what we want to study.  We ought to be in contact with the people who are making the coverage decisions and asking them what do they want to know about?  What is coming down the pipeline of new technologies that they are going to have to make choices about?

And that gets to the issue of the next slide which is we want to be timely.  And the real goal of cost effectiveness analysis would be to effect the adoption of the coverage decision and not to be doing it post hoc.  And we have some examples where it is very hard to withdraw some technology that is already widely disseminated.  And it’s a kind of truism in health care that it is easier to not apply care then it is to withdraw it once it has begun.  

And the other thing about being timely I think is to actually have anticipated maybe not a specific technology, but at least that there will be a technology or innovations in a particular area.  So right now we are working on some work on hepatitis C drugs and working on models to look at cost effectiveness for the two drugs that are now coming into play, but we realize that this is going to be important for future years because there is many more agents that are in the development pipeline that are going to need to be evaluated.  So we really need to kind of preposition our resources to be ready to do some of these cost-effective analyses so that when the next drug, or the next device or procedure comes out we will have already had a decision model for similar treatment that we can use to estimate cost effectiveness.  And that way hopefully we will be more times. 

Now I am going to flip back here a little bit to say so what do we do now to make coverage decisions in the U.S.?  And this is really what harkens to some of the papers that Alan Garber has written about his experiences both in Blue Cross Shield and in Medicare that coverage decisions are largely based on effectiveness, but and the questions that the decision makers are making is, is the effect big and is the strength of the evidence good?  

And there is some sort of implicit cost effectiveness analysis being done here by thinking about strong evidence and big effect which is his point is that if treatments are very expensive then they want very strong evidence or very big effects.  And so there is sort of a kind of an implicit cost effectiveness impact or cost effectiveness analysis in that not necessarily good or accurate, but somehow they are thinking about value and that there should be if it is an expensive intervention some strong evidence of value.  

Cost effectiveness analysis does have in the U.S. some role in the recommendations at the Preventive Service Task Force as the screening and Peter Neumann also notes that the Managed Care Pharmacy society, the American Managed Care Pharmacy Group has developed its guidelines for formulary and he talks about how cost effectiveness analysis should be used for making formulary decisions.  And they are being used by these pharmacy benefits management programs. 

Now finally I wanted to turn to the issue of cost effectiveness analysis and comparative effectiveness research.  So we’ve seen in the last several years that really is comparative effectiveness research that has been put forward.  And of course the—we have a new institute of to do comparative effectiveness research.

And one reason I think it’s generally acknowledged that comparative we’re doing, talking about comparative effectiveness research rather than cost effectiveness research is because cost effectiveness research is too—it’s seen as too controversial, this whole political question about considering costs and the health care decisions.  And so this is sort of an idea that came that well we’ll just kind of finesse this and that we need this comparative effectiveness information too, and this is important. 

And so kind of the classic issue is that where comparative effectiveness is important is we require for licensing pharmaceuticals is shown to be more effective than placebo, but what we really are interested in is are they more effective than the drug that’s already being used that is standard of care, but just saying the cost effectiveness is concerned about, but comparative effectiveness is saying well I don’t care about new drug compared to placebo.  I care about new drug compared to old drug, and in the same way we want to compare all sorts of innovative treatments in this way. 

And I think that’s kind of the essence of the comparative effectiveness question.  Now there are some limits to this.  That’s important information to have should we adopt the new drug instead of the old drug, but what happens if say we have a new treatment that has a better outcome, it’s better, it’s more effective in terms of preventing the thing that we or the disease or slowing the progress of the disease, but it also implies some increase of side effects.  

Comparative effectiveness it’s very difficult for it to say how do we trade off the bad effects of the side effects with the value of getting more effective treatment.  And then the same problem, another problem with comparative effectiveness is what do we—how do we look at the tradeoff between some sort of long-term and short-term benefits?  

We may undergo some sort of screening test for a disease and has its risks because we want to get some sort of long-term benefit in terms of preventing bad outcomes down the road.  Or the short-term effect may be that we’ll have a false positive in the go you’ll have unnecessary biopsy or something like this.  So there’s another hazard versus benefit tradeoff. 

So these are all things that are hard to gauge in the comparative effectiveness world.  And so in this way cost effectiveness methods actually can be useful in comparative effectiveness research.  And I really owe a debt to Louise Russell making this, putting this through, and I refer you to her paper which just have all the sites by the way are last on the last slide, fixed beside it.  

So Dr. Russell’s point is is that if we have a situation in which we’re doing a comparative effectiveness study and the innovation is both has benefits, but increases risks or increases side effects, we can do like we do at cost effectiveness analysis or convert to qualities and find the net benefit just to find out what treatment is the most effective.  So that’s we’re not doing cost effectiveness analysis, but we are using the tool that helps us answer this comparative effectiveness question.  In the same way when we have to get beyond the short-term effectiveness and think about those tradeoffs over the long run, the decision models that we use in cost effectiveness now can actually be very useful compared to effectiveness research.  

So the other criticism of just ignoring the cost and looking at comparative effectiveness is what Alan Garber calls a menu that’s as if we went into a restaurant and there was a menu with no prices.  So if we go into the restaurant, sit down, we’ll just order the truffles, and the caviar and the champagne and not worry about the bill, but of course had we known what it would cost at the outset we would have ordered something, ordered the pizza instead.  And that would have been fine. 

So I think that is his analogy.  I hope it’s not too overstretched.  I think the point is clear that costs somewhere have to enter into the assessment value. 

And it is interesting that even in the when the Institute of Medicine was asked by Congress to set priorities for comparative effectiveness research that it’s gone into this new comparative effectiveness institute and also the topics that are funded by the economic stimulus bill, projects now winding up.  You look at their report and you see they say these words that cost effectiveness analysis is useful.  

And if you actually look at the hundred priorities that they came up with the word cost appeared in thirteen out of a hundred of them.  So even when they were basically asked to only look at comparative effectiveness, not cost effectiveness they had trouble looking away from it and found that still kind of important.  

Now I wanted to turn a little bit to that whole question we talked about where cost effectiveness analysis might actually even though it says one thing we may end up doing something differently making our health care decisions.  And this is based partly on the experience in the U.K. and other places that have this too where their situations where some treatment has a very high ICER or very high cost effectiveness ratio.  Let’s say it’s above $50,000 or I think it’s GBP20,000 per quality or even well above it that the intervention will get approved anyway. 

And I think this comes down to this idea that there are certain groups where that some priority should be given that all qualities are not equal.  And I’ve put down here life threatening conditions, but I think also we should think about the first indication, the first treatment for a disease. 

So if some disease was formerly hopeless that there was no effective treatment for it and now we have one, we will give the benefit of the doubt to those patients because we see there’s some value without giving them set hope, and that people who, so children are seen to be a special group in consideration that we want that there seems to be the error on the side of inclusion and interventions for them, and also for people who have, are endowed with few qualities, and so this idea of doing more for the disabled. 

And this gets at this point that maybe not all qualities are equal.  They are the people that somehow in society we think that the quality just of life is more valuable if we’re giving it to somebody who doesn’t have very many left to them, and less value to those who are relatively healthy and richly endowed. 

And I think that when we talk about VA we could add to this list some things that are real connected to our mission that is the treatment for the continuous injuries or illnesses that are related to military service.  So I doubt that we will ever be in a situation of using cost effectiveness criteria to judge whether effective interventions for things like traumatic brain injury, or polytrauma or PTSD should be adopted or not.  I think they’re just so central to our mission and it’s back to that kind of special groups that we will probably not apply cost effectiveness, the criteria to those kinds of treatments. 

Now how do you make these value judgments?  And I think one way to think about this that the quality is, and I think and one, and someone indicated that this got at this as a question what are the barriers to adoption of cost effectiveness balances?  

Is it maybe we don’t capture all the value of on a scale of qualities?  And so one idea is that there are some extra value whether it’s that we’re getting as a health care system from treatments for traumatic brain injury just beyond qualities, has to do with our mission, organizational reputation.  There’s other kinds of values that we might incorporate like patient satisfaction that may or may not be captured in quality. 

So one way in which this has been incorporated into the decision making bouquet is to use a citizen council at the National Institution of Clinical Effectiveness advisors.  And so that is kind of the route in which they gathered this initial information as something the formal cost effectiveness of findings. 

I think another interesting experiment with public involvement is something that Martha Gold did, which she recruited people from the New York state juror pool, provided them with examples of coverage decisions and asked them to kind of rank which things we should cover in a health plan.  Then she did a little portion on the basics of cost effectiveness and had them rank them again and found that they were actually influenced by a cost effectiveness analysis. 

So it does rate that imply that if we just had a better job of getting our message out there about what cost effectiveness analysis is and really it’s all about getting the best possible value from available health care dollars, then we actually could generate more public support for this and it’s partly a political problem.  Maybe it’s partly just a public understanding of science problem. 

Now I will say and here’s just some thoughts of my own about that maybe VA has a unique role in showing how cost effectiveness analysis can be applied in the U.S.  We are a globally budgeted organization, so we have basically a fixed pot of money and we’re trying to get the most value we can out of it.  And so that’s a situation.

If we adopt new technologies it’s not like we can build the insurance plan for more or increase the premiums where we’re stuck with that global budget so that there’s some incentive there to make careful coverage decisions.  The other interesting area is is that since we have a dedicated research unit within VA that we have some opportunity to identify decision makers who are making coverage and guideline decisions and in collaborating and to find out what their needs are and work to meet those needs. 

And then we also have that identified constituency talking about how the public can be involved in decision making.  And really I think the experience [inaudible]  suggested they really must be involved in that application across effectiveness analysis to make health plan choices.  

So just and kind of sum up the brief recap here what I think are the central points of the talk are is as what do you do as a researcher that wants to do the work in this area?  I think one set of questions is, or one set of maybe lessons, well take-home message, whatever is that if when choosing a topic for cost effectiveness analysis is if you involve the decision maker right at the outset and ensure that you’ve got a customer who’s interested that certainly is a path to having your study be relevant when it’s finished, and then also just to choose the kind of interventions that are where the just cost effectiveness designing is likely to be relevant. 

So is it an expensive treatment and virtually you wouldn’t probably want to do something that is a treatment for one of the exceptional groups that we think that cost effectiveness analysis may be less informative because there’s other value being generated or that we don’t want to make that assumption that all qualities are equal.  I think there are some lessons about what should we study. 

And then the question, then some lessons from just looking at the literature and thinking about it about what we do as analysts when we prepare our cost effectiveness analysis.  I think the key one is transparency and this is a way to get the best possible acceptance from decision makers.  If they understand exactly what you’ve done in terms of especially if there are any analytic assumptions, but what is the source of your cost effectiveness data, yeah, whether any of your results are sensitive to assumptions you’ve may have made in modeling them. 

And that transparency lesson I think is really clear that that’s very important in the acceptability of analyses overseas and the countries where they use cost effectiveness analysis to make their decisions about their health plan. This idea from the ISPOR task force also then we provide disaggregated costs to sort of aggregated outcomes that it’s not just enough to today dollars for quality, but we want to know how, what specific costs will change to what subcategories of costs, what subcategories of outcomes, and also for different subgroups. 

I think this is actually a very interesting question because if you look at the application of the NICE findings it’s often the case that they will, and also true in Canada that things don’t get rejected outright, but they may be just restricted to some small subgroup where the cost effectiveness ratio is the lowest.  And I think this is an interesting question of how cost effectiveness analysis can have an impact.  

So there is some evidence that there is considerable evidence really that about thirty percent of what we do in health care  are services that don’t really yield much if any value.  And this is from the small area variation analysis.  There’s also some evidence of this by comparing the different lands within Rand health experiment, other data like this. 

So if thirty percent is unnecessary why is that so?  And so part of it has to do with the indication creep, the idea that we show something is effective or cost effective in some group and then the health plan, the health care system slowly rolls it out to a bunch of other subgroups where it was never shown to work, was never shown to be cost effective.  So the idea that we look at subgroups right at the outset could be very important to preventing the indication creeping, having the cost effectiveness [inaudible].  

And then finally I think the important thing if we’re going to prepare cost effectiveness analysis I think that the budget impact analysis has increasingly got to be an essential part of our economics plan.  So as I said it’s not enough for the decision maker to know that it costs only $15,000 per quality if they implement this drug, or device, or treatment or screening procedure.  They really need to know how many qualities they’re committing to buy and what that cost is not just from a societal perspective, but what will it cost the health plan real dollars in the next year or two because that’s the—that’s what they need to manage to. 

So these are the— I think some of the key readings.  Drummond’s paper is on the question of what are the—excuse me, this first one is the ISPOR Task Force.  And the Garber paper is looking at his coverage experiences, and it’s a person who’s been on coverage panels, mention Martha Gold’s public involvement, time to ask the taxpayers it’s called, and Peter Neumann’s surveying that landscape about what are the barriers to using cost effectiveness analysis, and then Louise Russell’s insights on how the cost effectiveness methods can be used in comparative effectiveness even when we’re just limiting ourselves in comparative [inaudible]

So happy to entertain any questions or discussions at this point, got a few more minutes left. 

Q:  Oh there was one question about where the slides are and where they might be available.  

Q1:  We will have them available for download on our catalogue archive.  And everyone should have been sent a link to that in their reminder that was sent out this morning.  If they can’t find that they can send us an e-mail at cyberseminar@va.gov and we’ll get that right out to them.  

Paul Barnett:  And they should also be able to find it on the HERC website before too long at www.herc.research.va.gov.  

Q:  I have a couple of questions here.  CEA for genetic testing have you any information about that?  

Paul Barnett:  So I don’t, but the—I’ve not looked at that personally, but if you’re looking for to see if a cost effectiveness analysis has been done on any particular type there is a registry that has been created by Peter Neumann’s group down at Tufts and there is a link to that on the HERC website or just e-mail me and we’ll send you that link.  

And so you can search for a particular topic and we’ll give you that information; that is it will tell you who has done a study that’s estimated a cost effectiveness ratio.  And that’s one place they keep it pretty up to date.  And so genetic screening you think about is just like any other screening.  You’ve got—it’s got to have some sort of decision model to look at that what are the consequences of the false positives and false negatives?  And what is the cost?  What are the payoffs?  

And I don’t know so many examples of payoffs yet out of some of the at least in adults, but I could be wrong about that.  So it would interesting to look.  It would be interesting to find out. 

Q:  There’s another question about where can I learn more about the societal perspective questions and the value of responses that are typically included in a CEA?

Paul Barnett:  Yeah.  So the societal cost perspective so basically all of the questions about cost effectiveness methodologies is there are two books that are available that are kind of the Bibles of this if you will, or the really the what the public health task force has developed our guidelines in the U.S., also reference case.  And so there is a book by Gold, et al, and called “Cost Effectiveness in Health Medicine,” which is basically the findings of the panel on cost effectiveness and we came up with that reference case. 

And so they definitely would go into the what is what do we mean by societal costs.  And there’s another book by Mike Drummond and some of the Canadian folks who’ve worked in this area which is very consistent with the Gold task force, although I must say the Drummond book has now been updated in the third edition so it has some advantages, but it’s for the economic evaluation health care programs.  So both of those are cited on the HERC website and if you have trouble finding those let us know. 

So societal perspective is this idea of a long-term cost from the perspective of society so which we think of as sort of the average cost to the payer or opportunity cost and also the costs incurred by the patient.  And so what was the second part of that than, Patsy, so I didn’t quite understand that?

Q:  Here we go.  And the value of responses that are typically included in the CEA.  So I’m not sure—maybe....

Paul Barnett:  Maybe have you—have that questioner elaborate a little bit because I’m not quite sure what....

Q:  Nancy, can you, or Heidi can you un-mute this person who has asked the question, which is Nancy Lightner?

Q1:  Yep.  Nancy, I’m going to un-mute your phone line so you’ll be able to talk on the call.  You’re un-muted. 

Q2:  Okay.  I was just wondering more about the societal perspective and how you place the value on the different responses that are given to some of the questions.  Like you could extend a life, a quality a life a year or two on average. Then how is that—I guess I’m really looking for the formulas that are used.  

Paul Barnett:  Right.  Well do the value side or the benefit side,  there are some techniques.  And I don’t know whether you had a chance to participate in that course, but that would be on the archive, but the idea is we have some sort of standard way of rating quality of life on a scale of zero, representing death, to one representing perfect health.  

And the attributes of that utility scale or preference based health related quality of life is that you can trade off quantity of life with quality of life.  So the notion is to say you have measured this person’s quality of life with the standard instrument and it comes out to 0.50.  Well that is equivalent of two years of life with that 0.50 reading would be regarded as equivalent of one year in perfect health.  

So that is a cardinal attribute of preferences.  So what are the standard instruments?  There are some [insulin] paper instruments like the health utilities index or the Euroqual or quality of well being scale, basically those three.  And there are two more complicated instruments that could be used called the standard gamble and time trade off that usually require some sort of software, computer media thing and measure a slightly different construct.  

If you are interested in how to measure preferences HERC has a guidebook on its website that Patsy is the first offer of it that describes some of that and how you can figure out what’s for your particular disease what would be the best way to go.  There is also the possibility of taking some sort of disease specific measure of health outcomes and turning it into a preference weight, or that probably won’t work if there is side effects are important if you need to have something that is going to incorporate treatment side effects, or treatment harms or risks, that sort of thing into your assessment, so it would be better to use one of these specific preference measures.

Q:  Right. 

Paul Barnett:  Anything you want to add, Patsy? 

Q1:  No.  This is my first exposure to this and I am fascinated.  Is the course available individually? 

Paul Barnett:  So we have an archive of all the lectures that have been given.  And those are on that HSR&D site.  And if you want to look at the slides they are both there and you can either look at the slides or the whole lectures.  

There are also three articles that were in JAMA sometime around 1996 that summarize what’s in that book by Martha Gold.  I am sure we could find a way to make sure that you if not get the sites, get the articles themselves, so they would give you kind of a summary about the whole what the course covered.    

Q1:  Thank you so much.    

Q:  I have another question.  Can you comment about decision analysts sharing their model so that others can modify them for their specific settings?    

Paul Barnett:  Yeah, that’s an interesting question.  I so they do become so... what typically happens when a model comes out is there some sort of say you get your decision, your results of your decision out of your cost effectiveness analysis published in some journal, and usually details available as an appendix.  And now the modern way is if that appendix is posted on the journal’s website and you can see all the parameters that go into the model, but I don’t know that people actually share the specific soft copy that is the file that is used in the model whether because it becomes sort of a researcher’s product.    

Q:  Intellectual property.  

Paul Barnett:  Intellectual property as it were.  Now I do think though that the government agencies like they do require actually to be able to have the model and be able to manipulate it.  So in some countries they do require that as part of the submission.  

So in theory if the model is thoroughly described in that appendix you ought to be able to recreate it in any software.  There are two principle software packages that are used for modeling.    One is a TreeAge, which is a proprietary thing, somewhat expensive and there is another public domain one.  

And then some people build models just in Excel or lots of other ways to do it.  So like even if you did have that it may not be so translatable to a new investigator and a new setting.  

So I don’t know to the extent that those sorts of things are actually being shared and you could piggyback on someone else’s work in exactly that way, but the parameters that they have and the assumptions they are making about what is the effectiveness or the meta analysis they use about effectiveness and long-term course of the disease those should be clearly articulated in that supplemental material so that you can really get the complete picture of the model.  That’s the standard.  

Q:  Paul, I just want to mention a couple of people have their hands raised in the attendee list.  And I don’t know that they have actually asked a question.  So I just want to suggest they go down to the question area and write out the question if we have not gotten to it so far.  I can’t quite tell if these are still pending questions or there is a problem.

Paul Barnett:  So I am also interested in whether there is anybody who is like in the position of rather being a creator of cost effectiveness analysis that they are more of a utilizer of cost effectiveness findings if they have any questions or comments that they would like to make as really this is all about how do we make these things more useful to them.  So no further questions or we are waiting to see? 

Q:  Nobody from CMS who is a user or CDC? 

Paul Barnett:  Well I think that we’re also we were thinking that there were some, there may be some PA decision makers here.    

Q:  Yes.  And their e-mails are a little hard to discriminate where exactly they work.  So I don’t see any more.

Paul Barnett:  So, Heidi, I think I’ll surrender— 

Q:  Oh, wait a minute.  I’m sorry.  Here is a question.  Has anyone used discrete cost in outcome categories rather than continuous variables? 

Paul Barnett:  Well, not usually, but in the case of dominance that is sort of all that matters is that the cost are less and the outcomes are better, so in that sense yes, but usually we like to gather these as continuous variables because we didn’t get into this whole issue of certainty, but there is uncertainty about the cost.  There’s uncertainty about effectiveness.  We only know how much more effective one of the interventions is to some degree of statistical precision.  

And so if we were to dichotomize that then we are giving up a lot of our power to detect those differences.  So we really want to know something beyond just that it’s better, but we want to have some idea of the effects on it, back to that thing about coverage designs, coverage decisions, bigger effects, stronger eminence is better.  

It favors the decision and the same thing is true when we apply that into cost effectiveness analysis and the consideration of cost in that way.  And so we didn’t get into it here, but the decision makers’ choice of about a threshold...  there is a difference statistical precision associated with each threshold and what best practice is for expressing cost effectiveness analysis is to say and each choice of threshold what is the probability that the intervention is cost effective.  

So for instance, and so this is a p value or what is the chance of this occurring just by grand chance alone, and you want a very small p value and can be sure you are making the right choice.   So typically you will see this plot of the $10,000 in p values 0.01 and $20,000 per quality to p values perhaps we would expect it to be even more certain if it is et cetera, so that the decision maker says okay I know what I am willing to pay for quality now.  What is the level of statistical precision which you can assure me that this really is cost effectiveness at my particular threshold?

 And so there is a method of doing that.  There is actually a couple of different methods for doing that.  And if a decision model is involved then we also want that information to reflect the uncertainty that is attributable to the lack of precision about the parameters that go in the model.    That is called the Monte Carlo analysis.  I think we covered that precision model lecture.    

Q:  So, Paul, there is also the question of whether you are using quality versus the natural unit as your outcome in the cost effectiveness analysis.  So for example you might use employee or not as your outcome as a result of the intervention.    

Paul Barnett:  Right.  And so then you would end up with dollars cost per additional person employed.  We could also think about dollars cost for addition smoker who has quit, or a dollars cost for I would say day free from angina, so all these are in natural units.  Some of them could be discreet.  

So the problem with having those results is we don’t have any cut point.  We don’t have any threshold.  How much is the society will to pay for a day free of angina.  How much is society willing to pay to get someone to achieve an abstinence from tobacco or to get someone reemployed?  

We just don’t have... and even if we did know that we can’t really say is that a good trade off compared?  We can’t compare those industry we spend the money on smoking cessation or should we spend it on cardiac intervention, which if they are delineated in different units.  

Now if we know that the cardiac intervention is $20,000 for quality and the smoking cessation intervention is $5,000 for quality, maybe we would get more value  if we had to choose between doing the smoking cessation, but if they are enumerated in whiffs and angina days then we can’t compare them.  So that is the disadvantage of that and we have no threshold too apply.  

Now I can’t say like on the smoking one I do know that people have estimated that if someone quit smoking, a middle aged smoker has quit smoking you are going to gain, oh, one to three qualities depending on how people model it but that about that much.  So that right there gives us some idea of what we should be willing to spend on smoking cessation.

Q:  We have one participant who says in my work at Intermountain, my modeling efforts have been focused on models that answer questions asked by decision makers which lead to decisions about services, just that we do have a payer decision.    

Paul Barnett:  You should give this lecture next time.

[Laughter]

Paul Barnett:  You should give this lecture next time.

Q:  I’ll make sure you know who it is.    

Paul Barnett:  We’ll make— we learn from them.  So it would be very interesting, very interesting to have them tell us what they are doing and how they— 

Q:  So, Jim, you are on the short list here. The next question what do you do when there is a limited effectiveness data for your model?

Paul Barnett:  Yes.  So then you would simply end up with greater imprecision.  And sometimes that is okay. So I have been involved in a couple of clinical trials where we have not, where we found that the more expensive innovation, and the innovation is more expensive and we don’t really have anything to say about its effectiveness.  

And so when we do the incremental cost effectiveness ratio and we plot it’s uncertainty.  We can say, gee, the decision maker can reject this intervention because we do know that it is more costly.  We can’t really say much about its benefits.  So it doesn’t look like it is a good purchase.  So sometimes it’s okay to have if to have some imprecision about effectiveness, but obviously the less precision you have then the less able you are to help the decision maker out.

Q:  Okay.  And the next, I think and final one, is how to models account for cost then for ratios for different types of conditions, e.g., perhaps evidence of a small effect might be more beneficial for certain conditions than others.

Paul Barnett:  Yes.  So from the cost effectiveness framework we’re... so we are looking at dollars per quality, right, change in costs over change in qualities.  And we don’t usually call that cost benefits.  That’s cost benefits is sort dollars cost over dollars benefits.  And that tends to be used more in environmental development economics and that sort of— so in health we talk about cost utility or cost effectiveness analysis.  So it’s dollars per quality and so it doesn’t really matter in our decision whether those— we are just really interested in that incremental ratio.  And it could turn out that you could have a very low cost divided by a very low quality and you could buy, so you could be buying just a little tiny increment of qualities by doing something modest.  

Or you can have a very high cost and have a very profound in treatment.  And you could end up with very similar cost effectiveness ratios.  It could be that the low cost intervention like say a vaccine is highly cost effective even though it is not very expensive just because it gained so many qualities and et cetera.  So it really doesn’t matter the size of the increments because this is a ratio.

Q:  Okeydoke.  If we are going to go on we have a question about: can you comment on different checklists for rating the quality of CEAs and our participant, Jim, is already his hand’s up and he’s winking.  He is ready to give the course.    

Paul Barnett:  Okay, good.  So....

Q:  Checklist.    

Paul Barnett:  Yes, quality of cost effectiveness analogy.  So that is very interesting work.  People have starting with Mike Drummond but many others have done these papers on just exactly that.  They go in a particular clinical area and say how good are the evaluations that have been done and what are their major flaws?  

And there are actually, as a result, have been created a checklist and Drummond has one, but the others [inaudible]. 

Q:  [Inaudible].  Doesn’t BMJ have the original one?

Paul Barnett:  I think that’s Drummond’s, exactly. 

Q:  Uh-huh.  And then there is kind of a summary one as well. 

Paul Barnett:  In fact I think there are bunches of them.  And my recollection is that there’s not only a lot of these.  There are so many of these evaluations that have been done that there’s actually a meta analysis of these evaluations that was in JAMA a few years back.  It’s like at 40 evaluations of how well cost effectiveness analyses were compared. 

Q:  Right. 

Paul Barnett:  So and many checklists and not content with just that many, I sort of offered and some other criteria about costing is and how to the costing part of it which is special supplement that National Cancer Institute put in Medical Care I think it was two years ago.  So we can certainly follow up on that, but I think those BMJ criteria are are a great place to start.  And I think those the checklist is also in Drummond’s...  in the Blue Book which is the U.K., Canadian guide to how to do cost effectiveness analysis. 

Q:  Okeydoke.  We’re at 12:06.  

Paul Barnett:  So, Heidi, do you want to take the com back and do your magic with the survey?

Q:  Well actually the magic with the survey is really magic now because everyone gets it when they leave the session. 

Paul Barnett:  Okay.  

Q:  That’s pretty cool. 

Paul Barnett:  Yeah, very good. 

Q:  So, yep, it’s very high tech with.  That’s the hope.  We’re seeing a pretty good response so people are—

Paul Barnett:  Well we really do appreciate the feedback and try to improve the course based on really appreciate Heidi’s efforts in getting that for us. 

Q:  Yes.  For those of you leaving the session we really do appreciate the feedback that you give to us.  Please take just a moment and fill that out.  We will definitely take it into consideration for our future series.  

Paul and Patsy, I want to thank both of you for taking the time to join us for today’s session.  We really do appreciate the time that you put in today. 

Paul Barnett:  And do we have the next HERC seminar I know is scheduled for March, but I’m afraid I don’t have the information at my fingertips. 

Q:  The next HERC session is scheduled for March 21st.  And Jason Hockenberry will be presenting Cost of Readmission in the VAHA, Implications for Reimbursement Policy.  Today is the last in the Cost Effectiveness Analysis course, so the next—

Paul Barnett:  And we’re going to be—

Q:  I’m sorry. What was that?

Paul Barnett:  I was saying we’re starting another course up.  I think it is in the fall.  I’m not correct on the exact dates, but on Econometric Analysis. 

Q:  Good.  I’ll be a harassing Todd and Angela for that tomorrow.  

Paul Barnett:  Right.  So we have I think that Todd and Angela have come up with some dates for the next course.  It’s not soon, but not too far off so that’s the statistical economic methods for statistical analysis, econometrics.  And I think one reason we made that a little late is we’re trying to make it a little bit more driven by some examples to change it, try to improve it a bit.

Q:  Perfect.  Everyone will love to hear that, so fantastic.  So you guys can look forward to putting that on your calendars this fall.  We’ll be sending out the registration information as soon as we have that available, fantastic.  Paul, any last concluding remarks for the series?

Paul Barnett:  I think we’ve done it.  

Q:  Perfect.  Okay, this formally concludes today’s HSR&D cyberseminar.  Thank you, everyone, for joining us.  

**[End of Recording]**

Cambridge Transcriptionswww.ctran.com