HERC Cost Effectiveness Analysis Course- 13 -Department of Veterans Affairs

Sept. 5, 2012

**Department of Veterans Affairs**

**HERC Cost Effectiveness Analysis Course**

**Elements of a Complete Cost Effective Analysis**

**Ciaran S. Phibbs, Ph.D**

**September 5, 2012**

Moderator:  We are now at the top of the hour and so I’d like to go ahead and introduce our speaker.  We have Dr. Ciaran Phibbs presenting for us and he is a health economist at HERC located at VA Palo Alto and at this time I would like to turn it over to you Dr. Phibbs.  Are you ready to share your screen?

Dr. Phibbs:  Yes.  Thank you for attending.  This is the start... this is the first lecture in a series of cyberseminars that HERC has given for several years now.  It is an introduction to cost effectiveness analysis.  You can think of it as a quick course almost.  There’s about eight or ten lectures.  I’m not sure the exact number, but it’s somewhere in that range.  It’s not a full course in cost effective analysis, but provides a reasonable overview and outlines some the key issues and in terms of the data and in terms of some of the other lectures a lot of it are really focused on how to do this within VA in terms of... a fair bit of focus on specific VA data issues.  Operationally in terms of this as Molly mentioned there are so many attendees we can’t have the lines open.  If you have a question, type the question in.  Todd Wagner is monitoring those questions.  If it’s just a question that should just be answered to the individual he will answer that.  Otherwise, he will stop me.  There’s sort of two levels of stopping.  One is something where’s it’s not clear where I should just — he'll just jump in, say "Ciaran, you need to clarify that," or whatever, or if it’s something we need to cycle back to at an appropriate time he will raise the question.  We will, time permitting, we’ll get all of your questions answered.  We tend to answer some of the residual at the end.  It just depends on how the question fits into the flow in terms of Todd’s judgment.  

Moderator:  Dr. Phibbs, I apologize for interrupting, can I ask you to speak up a little bit please.

Dr. Phibbs:  Okay. I will move the speaker closer.  Is that better?

Moderator:  Yes.  Thank you.

Dr. Phibbs:  I moved the mouthpiece closer to my mouth.  Today’s lecture is “Recommendations for Conducting Cost Effective Analysis.”  The title is really the elements of the reference case and what is this reference case?  Well, how do I get these to, here we go— This really refers to the public health service, now over 15 years ago put together an expert panel that published a book on “Cost Effectiveness in Health and Medicine.”  The first author of this was Gold and it sort of double standard in that our use and that this is also referred to—when it first came out was the Gold standard, the measure of units for doing cost-effectiveness analysis and some of the things have evolved a little bit.  There are some elements that are a little bit out date, but it is a very good starting point in terms of the issues of doing cost-effectiveness analysis.  I will also note that summary of these recommendations were published in series of three JAMA articles in 1976.  Also that is a more concise way of accessing the key elements of these recommendations and those are readily available.  Before we go into this, I’m going to ask... we have a poll coming up here, Molly.  Molly’s going to pre-empt me now.  There’s the question, just how many of you have every conducted a cost-effectiveness analysis where the possible answers are no, one study, and more than one study, and that poll should be opening soon.  

Moderator:  Thank you Dr. Phibbs.  Yes the poll is open now and the answers are steaming in so for those of you who will be answering just simply click the circle next to the answer that is correct for you and it looks like we had two thirds of the audience vote already.  So we give them a few more seconds.

Dr. Phibbs:  So how do I see the— How do I see it?

Moderator:  As soon as I close the— I will share the results with you in just one moment and then you’ll be able to see them.  Right now we have 75% voted so I’ll leave it open for about 5 more seconds and then I’ll show you the results.  It looks like answers have stopped coming in so I’m going to go ahead and share those.  You should be able to see those now.  Do you see the results Dr. Phibbs?

Dr. Phibbs:  Yeah. So two thirds of you haven’t and some you have done some studies  Now I can close that.  

Moderator:  I’ve turned it back over to you.  Go ahead and accept.  

Dr. Phibbs:  I didn’t see—Where is the accept?  

Moderator:  Let me turn it over to you one more time.  One second.

Dr. Phibbs:.  Okay.  Got it.  There you go.  I need to—so that’s... So about a third of you have some prior experience and 

Moderator:  I apologize.  We actually can’t see your slides right now.  Let me turn it over to you one more time.  Wait just one moment and you’re going to see the pop-up.  Right now click “share my screen.”  

Dr. Phibbs:  That’s not showing up.  I think I’m not seeing any “share my screen” show up.

Moderator:  One second.  I think I know how to fix this.  

Dr. Phibbs:  You’re a presenter.  Okay, “show my screen,” there we go.

Moderator:  There we go.

Dr. Phibbs:  Now let me get the—

Moderator:  I apologize for that.  Thank you .  

Dr. Phibbs:  Getting back.  Why do we need cost-effectiveness analysis?  What it really boils down to is in health care is a big complex industry if you will.  There are many different interventions that affect outcomes in many different ways and the underlying ideas of cost-effectiveness analysis is to allow a common metric to compare across the diverse diseases, conditions, and patient populations.  Think of it as comparing the value of an invention for PTSD versus coronary artery disease.  PTSD is treating a very serious mental health condition with consequences in terms of behavior, coronary artery disease if it’s serious, you’re preventing—you’re preventing death or serious morbidity.  There’s very different types of health outcomes and you’re looking at how do you make a comparative value of these?  The idea that underlies cost-effectiveness analysis is which intervention is the highest value in terms of getting return for the money invested.  This is especially important in the VA as we are a capitated six budget system.  So we have a fixed pot of money that we have to allocate across all Veterans and how do we do that so that we can get the most value for the money that government is spending in terms of care for Veterans.  

What is cost-effectiveness analysis?  It is a tool for making decisions.  It is a common metric to compare diverse interventions I just alluded to.  To do this, you need to find both the costs of the intervention and to assign values to the outcomes.  To make this feasible across these diverse things, you have to have all the outcomes need to be measured on a single scale.  That standard is the “Quality Adjusted Life Year.”  What this QUALY, what you may or may not have heard of is the idea that in additional year of life is not the same for everybody because your quality of life matters.  It’s not just are you breathing.  The quality of life in persistent vegetative state is certainly less than for someone that’s fully healthy.  The question is how much?  There are standardized ways of measuring this and this is the standard internationally that has been adopted in terms of measuring cost-effectiveness analysis and there are various ways of doing it.  I’m just going to allude to qualities right now.  There are subsequent talks that will go in to the detail of how you actually get into measuring a QUALY.  

Why do we really need this idea of QUALY?  Well essentially it is that if outcomes aren’t measured in qualities you don’t have a way of valuing that.  You can say “well we gained something, but we can’t really compare if you don’t have a method of valuing these outcomes where you may pay more for them.  You can only deterministically say that something is favored if it is more effective and it cost less.   The problem is this is really rare.  It doesn’t happen that often that you get an intervention in health care that not only improves outcomes in some dimensions, but also reduces expenditures at the same time.  When you get it it’s a no brainer from a policy perspective and you can have what’s called extended dominance where A is clearly more effective and cost less than B and B is less costly and more effective  than C.  So A dominates C as well.  That’s the idea of extended dominance.  It’s useful to consider here that if we have this simple chart has on the vertical axis it’s measuring the change in cost for more expensive/less expensive and here we’re measuring on the horizontal axis we’re measuring changes in effectiveness, less effective/more effective.  

So if you have something that cost more and doesn’t work as well it’s clear that that intervention is not worth doing.  Similarly if you have something here that’s what we call the dominance—it’s more effective and it cost less down here in the lower right that’s what we have the strong dominance type of intervention when you’re testing a new intervention or comparing two interventions.  So that’s in this lower right corner.  That’s a no brainer, but what we have is on the diagonal elements, you have something that cost less, but it’s less effective or cost more and it’s more effective and the question is “how much more does it cost and what are we getting for our money, is it worth it.”  That’s where cost-effectiveness comes in, you spend ten million dollars to have some marginal changing quality of life for a few extra days of life, that’s clearly not going to be worth it.  Spending a dollar—to spending a few dollars to extend life for ten years with high quality of life is something that is very cost-effective and you’re certainly going to do it.  So the question is where on that in this quadrant does those interventions fall?  I’m going to use this—deviates from VA patients, but it provides a good example of some of the key issues in terms of dominance and why dominance is not only hard to obtain, but some of the issues that one needs to think about and this is the introduction of neonatal surfactant replacement therapy in 1990 and it reduced the reduction from expiratory distress syndrome in infants by fifty percent.  Dramatic effect on mortality in a relatively high mortality condition, mortality went from about 25% down to about 12 or 13 percent in one year for patients with this condition, dramatic effect.  

So in many cases, and this is especially true here, the reduced mortality increased cost and this is especially true for these small premature infants.  What happens is if they die, almost all of them die in the first couple days and if they survive they’re facing two to three months of intensive care which is very expensive.  So one would think that okay and that is frequently the case because in a lot of conditions the deaths are less expensive if they occur rapidly and so when you increase survival you are going to increase cost.  That’s certainly the case here, but this was a sort of an interesting case in the surfactant replacement therapy was treating the underlying cause of the lung disease so it was dramatically reducing the treatment intensity of those who—and the length of stay of those who would have survived anyway and you ended up netting out with lower mortality and lower cost.  

This was a rare confluence of events.  There are either some vaccines that are especially lower cost vaccines pan-out this way, but besides from that, if one looks at health care interventions in the United States or developing worlds or even in across.,..it is really rare to have something that has this type of strong dominance with better outcomes and lower costs.   The point I really want to make here is “oh well we got lower mortality,” but the lower mortality might cause increase cost, but then there was this off-setting treatment severity and that’s how it all—so the point is that the inner actions between how the treatment affects resource consumption can be quite complex and not what one wants to see them.  So that’s by way of background.  Why am I—okay— 

So absence of strong dominance, what we do and what we move towards calculating is something called the incremental cost-effectiveness ratio.  This is when we have a situation where one intervention is more costly and more effective.  What we do is we take—without examining the cost of experimental therapy of for trying and testing a new therapy compared to the control which should be usual care and one could talk later about the idea well a lot of drug trials just look at placebos so you just know it does it work, but how does it work compared to other therapies not the fact that the FDA only requires comparison placebo control trials.  Means that you have to play some games to look at is new drug A better than new drug B.  They often don’t do that direct comparison.  The second thing that you want—so that goes into the numerator.  You compare the cost and then you compare the difference in outcomes.  Where outcomes are measured in quality adjusted life years and this becomes the incremental cost-effectiveness ratio or ICER  This is how the recommended and standardly accepted way of valuing interventions.  

So we get the standard that has been applied to this country is that something is cost-effective if the cost per QUALY is less than $50,000.00 a year.  I will note that that is a arbitrary number it was reached by determining when—there was a policy decision made to extend dialysis to anybody who needed it in this county and they figured that it was estimated that that cost per QUALY for that was 50,000 and since this was a tax payer financed decision they set that threshold.  There has been no explicit—so some people now say we have to adjust for inflation and that number should really be closer to $100,000 per QUALY.  Others say well, but there’s no firm definition.  That’s sort of a commonly excepted benchmark when you have thing that are very low that are obviously going to be regarded as cost-effective.  If you get something down at $5,000.00 per QUALY, that’s regarded as a very good buy.  Ten million dollars per QUALY is something that is clearly not cost-effective.  The idea is when you do your intervention, if the intervention has a change of effectiveness relative to its cost that has a cost per QUALY below $50,000.00, the intervention is universally accepted as cost-effective and the intervention would be preferred.  If not even though the intervention might be effective, it’s not cost-effective.  One can argue what the actual numbers should be, but $50,000.00 is the standard.  

This lecture is about the reference case.  What is the reference case?  The reference case is a set of methods and assumptions to serves as a point of comparison across studies.  This is actually very important because there are many different assumptions, methods, and perspectives that can affect the outcomes of a cost-effectiveness analysis.  Without any kind of standardization, it wouldn’t not always be able to compare the results across studies.  By standardizing, we greatly increases the policy value of cost-effectiveness analysis because that allows us to compare interventions from different studies, either in the same field comparing two different methods of treating coronary artery disease or two different methods of treating PTSD or to compare treating coronary artery disease with PTSD or whatever condition one want to apply—consider.  

The point is that by having a common set of standards that get applied we can compare studies from the whole body of literature.  Now in reality, despite a lot of effort, this doesn’t always hold and people have looked at this in terms of the standard of economic analysis.  There’s been studies published that show a great deal of variability in terms of the QUALY of the cost-effectiveness studies, but the one thing—one lesson that I would like everybody to take from this is that if you are involved in a cost-effectiveness study, do try to adhere to this standard because it greatly increases the value of these studies.  Especially as we move to the world of comparatively effectiveness research which is not supposed to include cost, but everybody knows that it will.  Doing meta analyses, you need to be able to do this.  It’s especially—what happens if there’s a bunch of small studies, none are very conclusive, but you can pool them in a  meta analysis, in terms of doing that, you’ll need to have methods that are similar enough so that they can be combined.  

The key elements of the PHS recommendations are; to adopt a perspective of society; to measure all cost, where that includes the direct cost of the intervention, all health care expenditures and patient incurred cost; and to express outcomes as QALY.  

Todd Wagner:  Hey Ciaran, this is Todd, we have a question that comes up that’s probably worth discussing a little bit which is “Are QUALYs adjusted across different cultures”?  So do QUALYs take into account cultural values?  

Dr. Phibbs:  The way that you should do that is it depends on what method you use obviously.  The standard recommendation which is something we’re going to actually get to, but I’ll just jump ahead is—here I will—where is that slide.  I’m looking ahead of my slides.  I don’t—, but basically the recommendation is that you use societal—general society type weights to get your QUALY.  So the idea is that you don’t query people who have a specific disease.  You get it from general weighting of different health states to get it.  Then you’re doing it for the society if you will or the population that you’re looking—so that—and there is some difference in quality weights if you look at—if you try to elicit preferences in the United States versus Western Europe versus Central Africa.  You’re going to get some differences in terms of the values those societies place.  So you can reflect that by getting a weight where the intervention was done, but that eliminates the standard.  There are some standard instruments which have sort of North American weights or European weights that are standardly applied as instruments to get QUALYs.  Those reflect those biases so they won’t fully reflect the preferences of somebody in Central Africa to continue the example unless there’s also direct utility measurement.  Again, that depends so the cultural biases will depend on how you get your utility weights.  Todd do you want to add anything to that?

Todd Wagner:  The one thing I would add to that is that for some of the methods of collecting quality of life years or quality of life, based as you mentioned, that they are based on societal values.  One in particular that has taken a lot of publicity is the EQ5V which was credited by Euro- QAL and it was based on European weights and has been re-estimated with American weights in theory because Americans have different values of quality of life.  So that could take into account that one could also try measure more directly utility states using type trade-off for standard gamble.  Specifically using a culture if you believe that it’s related to you specific area that you’re studying.  You could do that.  It just might make it much more complex.  

Dr. Phibbs:  The issue—the point of this lecture is not how to measure quality, but the one thing that happens here is that how the qualities are measured and there may be differences in how qualities are measured.  Qualities are measured between studies and there may be difference depending on both the instrument used and the population sample.  So that’s a weakness one could factor something into that in terms of sensitivity analysis. When we get to sensitivity analysis, if I don’t bring it up, remind me Todd and I’ll roll it back in.  So basically, for the recommendations is if all health effects go into the denominator and the numerator is supposed to capture the changes in resource consumption or cost and both cost and outcomes are discounted at 3%.  I’m going to note in terms of the discount rate, if you have interventions that have particularly long time horizons either in terms of their cost or their benefits the discount rates can matter a lot and why is 3% appropriate and that is because that is essentially the long term historic return on capital.  It also allows you to not have to worry about—it’s a real interest rate not adjusted for inflation and if you use higher interest rates that has an inflation premium built into it and that means you have to add factors for inflation into your analysis and it greatly complicates things.  It is much better to do everything using—just ignoring inflation and using real term so you don’t assume inflation.  

Other key elements of the PHS recommendation is you do a clinical trial...  you may need to model the effects of the intervention that are not fully realized during the study period.  You do an intervention to treat a condition and you do a trial and you have data on the patients during the course of the trial.  When that trial is over, those patients or those that are still alive will continue to receive benefits from the intervention or may and you have to project those over the rest their life time.  Therefore, it may be necessary to build some sort of a model to incorporate those.  You need to conduct sensitivity analysis.  Sensitivity analysis are important because it tells you—in any study you are going to make a lot of assumptions and you need to find out and report on which of those assumptions results are sensitive because that—and how sensitive they are.  So that you can—the analysis will present those.  What you really need to do is find out which analysis—which in terms of the point of the sensitivity analysis is really—which of your findings are sensitive to assumptions and what assumptions and what happens to the findings as you varies those assumptions.  

So that doing so greatly increases the value of the analysis for a (inaudible) perspective because then it is much easier to say—let’s just take a North American example.  In the United States our health care system is real terms of producing the same service, has to be significantly more expensive than it is in Europe, by doing a very clear sensitivity analysis on the cost of the intervention if you use lower cost, then how do things change.  That provides information that the Europeans can use and you can work in the opposite direction.  You need to task the statistical significance of the cost-effectiveness findings.  There are standards for reporting cost-effectiveness analysis that clearly outline both the in the public health services report and in that JAMA paper that they published and there other places where standards for cost-effectiveness analysis are available.  They tend to be fairly similar across all these different standards and in terms of the standards reporting.  So that’s saying what is the check list of all the things that should be included when you report your findings so that people can utilize the studies effectively in meta-analysis and so on.  

In general, things are slowly getting better, but in general not everybody is reporting everything that the VHS recommends would be a short summary of that.  The societal perspective, we eluded to that before, and that is the results can be affected by the perspective you take.  The society perspective is considering all the cost and burdens imposed on society as a whole.  Think about an insurance company may have a very different perspective because they’re concerned about the money that’s coming in and out that they’re paying out and so they may not be concerned if the patient has to bear some of the cost or the patient’s family or some government program.  The patient has a very different perspective.  The patient cares about the benefit to the patient and the cost.  Other payers, thinking about the VA, the VA perspective is very different because “oh Medicaid’s going to pay for that or Medicare’s going to pay for that or other insurance is going to pay for that” so we’re not concerned about that.  There are the perspectives of the other individuals.  There’s also the perspective of the employers.  

In terms of the VA perspective this is a quick sideline.  We’ve separately have had seminars on some budget impact analysis which is the idea of doing the analysis from the perspective of just the VA.  In terms of the increased emphasis or relative in terms to management within VA research, if you’re doing a cost-effectiveness analysis you may want to consider a VA perspective budge impact analysis because that tells the VA “what is the effect of the VA as opposed to this broader societal effective?”  It may well be useful information that VA managers can use to make policy decisions.  I will note that in terms of budget impact analysis that if you are doing a proper cost-effectiveness analysis in a VA study, you will have all the information you need to do a VA focus or budget impact analysis from the VA perspective as essentially a sub-study.  You have all the information you will need to it and you can just do it.  As I know it will be covered in a later lecture.  

Todd Wagner:  We’re getting some questions that come in that might be useful to handle at this point.  So the first one has to do with—I think it relates to the common cut-off of fifty-thousand per QUALY.  So is it related to GDP per capita.  

Dr. Phibbs:  That means that fifty-thousand has nothing to do with GDP per capita.  That was as I eluded.  That was an arbitrary number based on a one policy decision made in the United States and it sort of then been commonly adopted.  The one adjustment to that is some people have adjusted it for inflation.  In some of the other countries they—Australia has a specific cost per QUALY standard per approval of new drugs.  A couple of other countries may have similar standards for that and you have an interesting phenomenon where in Australia you have to have a cost per QUALY of X thousand Australian dollars.  The pharmaceutical companies price their products according so they come in just underneath that threshold.  People respond to incentives, but a little much lower income country may have a different perspective so that if something can be called cost-effective but if you have a fixed amount of resources to allocate and you are nowhere that cost per QUALY, but you can still use cost-effectiveness analysis even if you only have enough money to fund up to five thousand of QUALY. Cost-effectiveness analysis will tell you how you are best spending you money.  

Todd Wagner:  I’ve heard some people talk about two GDP’s per capita as your cost cut off, but I don’t I think there’s any gold standard.  I think it still works out to what is useful for the country as you pointed out, fifty thousand’s sort of what’s been floated in this county. Clearly sub-Saharan Africa can’t afford that as their cut off.  

Dr. Phibbs:  Yeah.  I think the real point is that this is some arbitrary line for what’s is considered cost-effectiveness, but the real value of cost-effectiveness analysis is a way of comparing widely diverse and different interventions to tell you which ones are of more value.  That can be used—and if you don’t have the money to reach the $50,000 threshold they can still help with resource allocation at a lower threshold.  

Todd Wagner:  So the next question is, but what is the time horizon commonly used in cost-effectiveness analysis?

Dr. Phibbs:  It’s supposed to be a lifetime.  You are supposed to carry it out to a life-time and I eluded to the fact that’s why you may have to model things past the end of the trial.  

Todd Wagner:  The next question and I think you’re going to touch on this at this end of the talk which is; “Why should I use ICER’s instead of incremental net benefits” and maybe you can focus on the acceptability of QUALYs over measuring benefits or dollars and life years.  

Dr. Phibbs:  Okay why don’t we defer that one?  

Todd Wagner:  Okay.

Dr. Phibbs:  Remind me.  All right is there another question?

Todd Wagner:  That’s it for right now.

Dr. Phibbs:  So I’m going to come back in terms of the denominator versus the numerator.  The idea is that all health effects are in the denominator and expressed as QUALY and the numerators the cost-effectiveness ratio captures the changes in resources consumption associated with the interventions.  I will note that there are some potential gray areas and the most important thing is to avoid double counting.  Now we have a poll, Molly?

Moderator:  Yes we do.  Thank you.  I’ll go ahead and show it right now.

Dr. Phibbs:  To think about it the poll there’s 40 yes or no questions as to which of these belong in the numerator so the cost part of the ICER.

Moderator:  Thank you.  The way we have it set up is to please select all of these that apply if you believe that they belong in the numerator of the ICER and we have had a small percentage of our attendees vote so we will give people some more time to think this one through.  It looks like a third of our audience has voted, but answers are still streaming in so we will continue to give people more time.  Again, please just select all that apply “which of these belong in the numerator of the ICER” and your options are health care cost associated with the intervention, length of stay, cost of patient time and the value of lost productivity.  We’ve had about half of our audience respond.  We’ll give them just a few more seconds.  Okay, the responses have stopped streaming in.  So I’m going to go ahead and close it and show the results.  Dr. Phibbs would you like to talk to those real quick?

Dr. Phibbs:  Yes.  Can you?  We have here health care cost associated with the intervention.  Most of you got that right.  Yes, that’s a cost associated with the intervention and absolutely has to be in the denominator.  Length of stay; ten percent said yes and that’s a fuzzy one and it really depends on how you’re measuring cost.  You have to convert that to a cost and it needs to be—it should be imbedded in the first one, but you need to make sure that it is.  To reiterate the point, do you need to capture all of the cost and—

Todd Wagner:  Hey Ciaran, there was a problem with the poll in that it only allowed respondents to pick one so don’t focus too much on this.  Sort of summarize—

Dr. Phibbs:  Okay. Don’t focus on it.  Okay, All right, never mind.  Then the next thing is cost of patient times.  I have no idea how many people included that, but absolutely, cost of patient time should be included because time has value.  The value of lost productivity, this one is controversial.  The sort of dogma or standard accepted dogma is that the value of the lost of productivity should be captured in the QUALY and it should not be in the numerator.  That said, if you are doing a study, it has a particular large effect on productivity.  I strongly encourage you to try to get some measure of the productivity effect because the QUALY will not fully happen, realize this.  So if you’re doing an intervention on something where the patient has the condition and can’t work, and all of a sudden they can work and work productively, if you’re just using standard measure quality like a health utilities index, you will underestimate the value of that intervention to society and to employers.  

The idea is that this is supposed to be captured in the thing, but it’s not. In terms of the generalized instruments—like the EQ5D and the HUI and the various sources of the SF36-6-10, whatever.  You will not fully capture it and it is something that you should measure even though it doesn’t officially belong in the numerator it’s something you need to address in the sensitivity analysis.  Can we move back here?  Okay show my screen.  So we’ve covered those so to go over those in detail the components that belong in the numerator; the cost of health care services; the cost of patient time; and the reason Russell had a nice paper a couple of years ago that showed that failure to capture patient time versus including it can have huge effects on the measured cost-effectiveness.  The cost of care giving, paid and unpaid, so you need to include part of the cost....  that there’s burden on other family members that’s unpaid.  That’s a cost that’s being born that needs to be included. There’s other cost such as travel time and you should measure the—as I mentioned before—cost should be measured in cost of dollars that way you won’t have to worry about projecting inflation and you just discount at 3% and use wage rates to value time cost.  The other thing to remember there is we’re doing a societal perspective so it should be average wages not the actual wage so you don’t impose a bias in terms of what the wage of those types of patients is.

Elements that belong in the numerator are the non-health care cost.  So and again, this is one of those gray areas ones.  Non-health care criminal health care cost, education and environments, cost imposed on others; employers.  I worked on a study a long while ago that was looking at treatment on substance injection drug users and one of the—by getting them into treatment we found that there was a significant reduction in the cost to the criminal justice system, not born by the health care system, but as a society as a whole we were having huge effects.  An intervention to prevent neurologic damage to infants can have huge benefits to the education system in terms of avoided special education costs.  Those are all direct cost that belong in the numerator.  Again, do not include productivity as it results in double counting because the improved health that allows the higher productivity is measured in the QUALY again as I noted in with caveat that you need to make sure that it’s fully captured.  

Components belonging in the numerator; the health care cost that result from living longer.  This is one where the direction by the public health service and standards and what people think starts to get a little gray.  It is clear that you include the cost of the intervention related disease then the original expected life years span and for the added years of costs.  You should include the cost of treating adverse events, but the idea is that you should exclude unrelated health care cost and non-health care cost from a when in the original life span.  So before the intervention the person was expected to live to age 67 and they were going to have other health care cost s that were not related to the disease you were looking at.  That should be excluded and you should exclude the non-health care cost of the not added years of life.  No recommendation was made for unrelated health care costs—cost of care not related to the intervention for the added years of life.  So this is the official recommendations.  There are serious operational problems with this recommendation because it becomes very difficult to determine what was a health care cost related to the intervention versus for the intervention related condition versus not.  

Just to give you an example, heart disease and diabetes, they have synergistic effects on the cost of either.  So if you approve the management of diabetes, you’re also going to affect the cardiac costs.  Should you be including that, and you shouldn’t be able to include that, but that’s a not a related disease or it’s not a directly related disease and how do you start parsing that out?  People with mental health conditions tend to have higher medical care cost.  How do you parse that out.  So many studies do not do this simply because it’s not tractable and it depends on the intervention.  There are interventions where the difference is clear cut.  It only has affect on the effected condition.  It doesn’t have any spillover health care cost and you can clearly do this distinction.  Those are relatively rare so this becomes very difficult.  This is the recommendation, but operationally is a whole lot more difficult than it says just in terms of checking off on this slot.  Many studies do not—will include all health care cost simply because it is not tractable to do otherwise.  

Components belonging in the denominator and I know we’re going long so I’m going to speed up; measure the health effectiveness of qualities;  qualities should be preference based; the weights should be based on community preferences; and use a generic health-state classification as opposed to disease specific; and use age and sex-specific health related quality of life to value of the gains and losses.  Remember that there are perspective changes over a life-time.  We talked some about that previously and there is a separate lecture on measuring qualities.  Components belonging in the denominator continue measure.  Modeling may be necessary.  Most clinical trials don’t cover the full time horizon and you need to use modeling and or data from other sources.  The one firm recommendation here is use data in modeling, don’t use expert judgment unless it’s absolutely necessary because that’s full of all kinds of potential bias.  

I talk about discounting before, I just reiterate, use 3%.  Sensitivity analysis, you need to conduct extensive sensitivity analysis.  One way sensitivity analysis is vary this assumption and see what happens.  Figure out what are the assumptions you made or that are key to the analysis.  This can be both in terms of how the costs are changing, how the qualities are changing, how the outcomes change things.  So you’re doing a study looking at the effect on a vaccine and how effective it is.  Well vary that see how it changes the analysis.  In addition to one-way analysis under overall uncertainty, you should also conduct to the extent possible, multivariate sensitivity analysis.  That gets more complicated.  Some of the decision modeling tools are pretty good in terms of making it feasible to do a lot of sensitivity analysis.  

You should bootstrap your cost effectiveness region.  So the way—for those who aren’t familiar with the idea of bootstrapping is you take your data and you sample the size—so you have a study with a 1,000 patients— a different number...  you have a study with 500 patients so you’re going to sample 500 patients from your data, but with replacements.  So in each of these samples you do, you couldn’t say observations could it be in multiple times or not be in at all and repeat this a 1,000 times.  Then you get this picture that looks like this, where this is just sample n replacement and so this shows—and then how many of them show up in this interval here which is cost-effective in terms of improved survival and reduce cost and on 95% in this preferred thing and you can conduct a confidence region, if you will.  Where do the 95% of your observations fall and where does that fall in terms of the axis.  The percentage of the replications that are not cost-effective represents the P value.  

So looking down here, the percentage of the observation that are not in the lower right quadrants, you calculate that.  That gives you your P value for your cost-effectiveness.  One thing I want to note is that the—and this is something that Paul Barnett did this is from a study he did that shows that the P value could vary with his cost-effectiveness threshold.  So if he varied how he defined something to be cost-effective it affected the P value of his analysis.   It’s just something you should—50,000, 20,000, 10,000 whatever, affected the P value.  Again, I alluded to this, the JAMA article—one of the JAMA articles that came out of the public health reports efforts through the public health services committee effort was a list of information that should be included. It’s simply a check list, all the items that should be included in the cost-effectiveness analysis because this allows comparison in the cross studies and I want to enter it—emphasize that this is very important from a policy perspective.  One other thing I want to mention and Todd sort of mentioned this is that there are now some alternatives to the idea of an ICER.  There’s something called a net benefit regression papered by (Jeffrey Hoch et al.) and I will also note if you want to learn more about that, that there was a HERC Cyberseminar, I listed it here it was August of 2006.  All the cyber seminars are archived so you can go back and in addition to reading the paper that was published in 2002 in Health Economics you can actually see a seminar on that.  It’s a regression so in that it’s somewhat different, but it gets out of the same things.  It’s a different method.  I’m going to go into details.  I just want to note that it exist and in terms of the slides that have been distributed,  I did put out a list here of some useful references.  There’s a book by Drummond et al. about the Methods of Economic Evaluations of Health Care that’s been widely available, it’s been updated.  There’s a couple of articles that I mention in here and then the ISPOR which is the****International Society for Pharmacoeconomics and Outcomes Research, Todd?

Todd Wagner:  Yes.

Dr. Phibbs:  (ISPOR) had a task force on cost-effectiveness in clinical trials and they had a paper available here that’s also a useful reference to check.  There was something I was going to come back to Todd, what was that?  

Todd Wagner:  You mentioned that it was the Jeff Hoch paper the 2002, about the net benefits.  We have two questions, one of them I responded privately, but we can also touch on it briefly.  If there’s in-kind contributions, whether they’re labor or non-labor, I sort of responded talking about care giving, how do you value these in-kind contributions?  

Dr. Phibbs:  Time has value.  Care giving is a good example.  You track the amount of unpaid caregiver time and you value that at an average wage rate for society.  

Todd Wagner:  The next question, in the case for effectiveness is multidimensional including survival cure of disease, can I do a ICER plane using bootstrap?  

Dr. Phibbs:  Well it’s multidimensional, but that’s the whole idea.  QUALY gets that multidimensional aspect of quality or effectiveness.  That should be embedded right in the QUALY.  

Todd Wagner:  The whole idea of the QUALY is to turn that multidimensionality into a single dimensional.

Dr. Phibbs:  Yeah.

Todd Wagner:  So you’re measuring the quality of life by mortality, in some sense.  

Dr. Phibbs:  So what happens in terms of the QUALY is measured on a zero to one scale where one is perfect health for a year and zero is dead for the year.  Then as you live with lower quality of life, you’re discounting that additional year.  That incorporates both mortality and all kinds of different measures of how it affects quality of life.  What is actually captured depends on the instrument—the method used to capture the QUALY.  

Todd Wagner:  I know we’re right at the top of the hour.  Just to think Ciaran for his presentation and all.  I also want to mention three weeks from this date, we’ll be talking about estimating the cost of the intervention and I’ll be giving that lecture.  

Dr. Phibbs:  Got it covered Todd.  It’s the last slide.

Todd Wagner:  And well be talking a lot about how to value labor as well—I’ll also walk through example of how to value to a capital investment as well. 

Dr. Phibbs:  So I just want to close and note that we covered a whole lot of different topics and now what’s going to happen for the rest of the course is we’re going to come back and deal with various elements of these in detail. Are there other questions or we’ve reached the hour?

Moderator:  We have reached the top of the hour.

Todd Wagner:  There’s one final question which is, are there peer review system dynamic model that calculate qualities and take into account non-linear effects?  I think we’ll have to respond to that one separately.  

Dr. Phibbs:  Yeah.

Moderator:  Well thank you to our attendees and thank you Todd and Ciaran for joining us and providing us your expertise and please do join us on September 26 for the next session on cost-effectiveness analysis.  Have a nice day everyone.

(End of recording.)