**Transcript of Cyberseminar**

**HERC Cost Effectiveness Analysis Course**

**Medical Decision Making and Decision Analysis**

**Presenter: Jeremy Goldhaber-Fiebert, PhD**

**May 28, 2014**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact herc@va.gov.*

*Moderator:*  I’ll just provide a brief introduction to Jeremy and then I’ll hand the reins over to him. Jeremy Goldhaber-Fiebert is a PhD; he is an assistant professor of medicine, a core faculty member at the Center for Health Policy and Primary Care and Outcomes Research at Stanford. He is also a faculty affiliate at the Stanford Center on Longevity and the Stanford Center for International Development as well as the affiliated researcher with VA Paola Alto. His research focuses on complex policy decision surrounding the prevention and management of increasingly common chronic diseases and the life course impacted exposure to risk factors. So I’ve had the pleasure of working with Jeremy for a little while and I can say he’s one of the most capable cost effective researcher I’ve met and we are extremely pleased to have someone of his stature and skill set presenting for us today. And with that Jeremy; I’ll hand it over to you.

*Jeremy Goldhaber-Fiebert:* Thank you very much. So I’m going to give a talk today entitled modeling and medical decision analysis. I know that there are more slides than fit in the time and I hope that people will potentially turn back to some of the ones that I don’t cover in detail as reference at some point in the future if they refer back to the slides. So without further ado let me switch over to agenda. We’ll talk a little bit about decision analysis and cost effectiveness analysis and then we’ll talk about some modeling techniques that are used to perform these two types of analysis. 

So the first question of course is what is a decision analysis? And there are many definitions. This is a definition that I tend to like and use when I teach about it. A decision analysis is a quantitative method for evaluating decisions between multiple alternatives in situations of uncertainty. So let’s unpack this a little bit and I apologize the underline looks like it’s shifted a little bit. So first let’s focus on evaluating decisions between multiple alternatives. This is really the core here. We have to allocate resources to one alternative and not another, do something and not do another. There’s no decision without an alternative and the key element of a decision analysis is about making a choice. Now in order to make that choice we need to evaluate it and in this case we’re going to evaluate it in a quantitative sense and obviously there are other ways of evaluating choices, ethical ways, and impressionistic ways and qualitative ways of evaluating choices, but here we’re going to evaluate it in a quantitative way. That involves gathering information, looking at consequences and outcomes from each alternative, clarifying the dynamics and trade offs that happen when you have multiple outcomes or multiple objectives and selecting an outcome that gives us the best chance of getting most of what we want and the least of what we don’t want. 

So we’re going to employ some sort of model in general, often a probabilistic model since everything isn’t certain in order to quantitatively evaluate these multiple alternatives and we’re going to talk a lot about modeling in just a few minutes. So let’s talk a little bit about the steps that one undertakes when one is performing a decision analysis. And I actually have started to design a new analysis. I will literally write out these steps in some sort of document and then I will fill in details about each one as I’m going about planning what I’m going to do and that becomes both sort of a design document, the analysis or proposal as well as a skeleton in some sense for a method section for a paper that flows from central analysis. 

So I’m going to enumerate all the relevant alternatives; these are the treatments or strategies or options people use various sorts of words but what I could do versus what else I could do versus what else I could so. Then I’m going to identify important outcomes. So there are lots of outcomes of anything that we do and the key word here is important and important is what’s relevant in the policy making context or the clinical context that we’re discussing, what’s relevant to the patient, what’s relevant to the payer; so that could be a variety of things. 

Then we’re going to determine the relevant uncertain factors that might change our chances of having various outcomes occur given what decision that we make. We’re going to encode probabilities for these uncertain factors so this might be the chance of dying or the chance of treatment success or the chance of various types of side effects and we’re going to quantify those chances in terms of probabilities. We’re going to specify a value for each -- of each outcome and we’ll talk about this a little bit more later. And then we’re going to combine these elements to analyze the decision using some sort of model. 

The decision trees would be one type of model that we’ll use and we’ll talk about first and then there are more advanced related models that will enable us to do that. So let’s talk first about identifying important outcomes. In a decision analysis we don’t need to necessarily consider cost; we could look for example at the probability of surviving 30 days after a procedure or life expectancy or even quality adjusted life expectancy, which I’ll talk a little bit about in awhile. 

A cost effectiveness analysis is sometimes is just a type of decision analysis that includes cost as one of its outcomes and then we need to talk about the trade off between benefits and costs. So what is a cost effectiveness analysis?  In the context of health and medicine a cost effectiveness analysis is a method for evaluating trade offs between health benefits and costs resulting from alternative courses of action. So a CEA is an important tool but it supports decision makers; it’s not the complete resource allocation procedure itself, meaning there are other considerations that clinicians or policy makers, payers and patients make when deciding what to do, not just whether something is good value for money. 

So the core element or outcome of a cost effectiveness analysis is something called a cost effectiveness ration and here really what I mean is an incremental cost effectiveness ration often abbreviated ICER. So there’s two parts to this ration, there’s the numerator and the denominator. The numerator is the difference between the cost of the intervention and the cost of the alternative under study. So what are these costs? Well let’s suppose we’re talking about one drug versus another. Well one part of this cost is the cost of giving that one drug versus getting that other drug. But let’s say one drug is more effective so that we avoid some downstream complications or bad outcomes that would result in rehospitalization. So those avoiding or differential avoidance costs also go into the numerator. What I’m talking about here is the total expected lifetime costs; the net if you will of everything you have to spend to deliver that intervention and everything that you may or may not face relative to the alternative. Likewise the denominator in this ratio is the difference between the health outcomes, the effectiveness or benefit of the intervention and the health outcomes of all the alternatives. So this could be, again these avoided bad events in the future as well as surgical mortality; so we have to net out those two things and that gives us our denominator. 

So as I say here the incremental resources required by the intervention relative to the alternative and incremental health effects are gained or losses the intervention with respect to the alternative. So really what we’re going to use a model for is to try to quantify Ci, Calt, Ei and Ealt in the context of a CEA and that’s really where the modeling comes in. If we had some other sort of direct data collection of some sort of large longitudinal study we might be able to make estimates of these things even in the absence of a model. But most of the time we need a model. 

So what is a model? A decision model and what I mean by a model is a schematic representation of all critically and policy relevant features of the decision problem. I’m going to walk through building it to sort of unpack this a little bit further. So it’s including the following things in its structure. Decision alternatives, the strategies or interventions that we’re going to do, the treatments if you will. The clinical and policy relevant outcomes and the sequence of events which may or may not be certain; some events don’t happen with certainty even if we take a given action. 

The model -- the purpose of the model is that it enables us to integrate knowledge about the decision problems for many sources, the probabilities which might come from epidemiologic studies, the values, the cost which might come from economic studies or from the elicitation from people’s preferences for one health state versus another. And so the model of the structure by which we take all this information from all these various sources and combine it to help inform our decision. 

The model in this case computes an expected outcome because we have probabilities; so we’re going to average across these uncertainties and say on average we expect decision alternative A to give us this and alternative B to give us that. There is a chance of course that it might give us more or less but given the fact that there is uncertainty we’re going to try to do our best on expectation. The nice thing about models by the way is that you can also use it to quantify uncertainty and to explore a bunch of alternative assumptions iteratively. 

So let’s talk a little bit about building decision analytic model now. I apologize I have allergies; so I’m sorry for my coughing. All right, so we’re going to build a decision analytic model so first we’re going to need to define the model structure then we’re going to assign probabilities to all the chance events in that structure. We’re going to assign values like utilities for various health states to all of the outcomes included in the structure, we’re going to evaluate the expected utility of each decision alternative, we’re going to perform sensitivity analyses which is very, very, very important. And the goal of our model is to make it simple enough to be understood but complex enough to capture the problems, elements in a convincing way. 

So when you submit some of these models to journals you will often have clinician reviewers who know very well all of the subtleties that they face when helping patients to make decisions. And if your model over simplifies it’s not convincing to them; that’s what I mean by complex enough to capture the important convincing elements. But you want it simple enough because the point is this is a computer model, this isn’t reality. Everything is an abstraction to a certain level and we want to make the model tractable to that it can be looked at and viewed and have some face validity. 

A quote that I like to use, especially because I’m a modeler is all models are wrong but some models are useful. So the simplifications are part of what make models wrong, right? Because they’re more elements in the world than the model fully represents. But the model can still be very useful as long as we include all the important relevant complexities and so that balancing, that tension is what we’re sort of striving for as we’re building our model.

So let’s dive in a little bit more and talk about how we might try and define a model structure. So let’s talk about what a decision trees structures, what the elements are. So the first thing is we’re going to have a decision node within a decision tree; this is a place in the tree where we’re going to make a choice between several alternatives. So here is an example that I’m showing; I’m going to make a choice between surgery or medical treatment. This is a choice that I might make either as a patient or a clinician or whatever, but it’s a certain choice. I will choose either A or B, either surgery or medical treatment and I’m going to choose it with certainty. 

Now in the example I have a choice between two options, but of course you can have a choice between many options. The point is that you have to choose between one or the other; you can’t choose two options. So the way you specify the options has to be mutually exclusive. If I choose one of them I am not choosing the other one. So if I could do for example both medicine and surgery I would need to have another arm in here that said, “Medicine and surgery”. 

The next element of a decision tree is a chance node, which is a determined and outcome based upon a probability. So there might be a chance node denoted here in the green -- where the green circle that a patient experiences either no complication or dies. There is some probability that the patient does and there’s some probability that the patient experiences no complication. I can’t choose that; there’s a chance that one of those things might happen. And again you can have more than two outcomes on a probability node. Again we want to make sure that they are mutually exclusive and collectively exhaustive. That means one definitely happens and only one happens. So for example a patient experiences complications and then dies that would have to be an additional chance branch off this chance node. 

So let’s talk a little bit more about mutually exclusive and collectively exhaustive, which I think we have a sense of but let’s really define them -- they’re very important. Mutually exclusive means only one alternative can be chosen or only one event can happen; collectively exhaustive means at least one event occurs, one of the possibilities must happen, taken together the possibilities make up the entire range of outcomes that could happen.

And finally we have terminal nodes; they’re the final outcome associated with each pathway of choices and chances. It’s denoted with this triangle and after the triangle we typically put some sort of value; so the -- outcome must be valued in relevant terms, cases of disease, life years, quality adjusted life years, cost so that we can use them for comparison. So for example in this case life expectancy if you get to this point in the tree would be 30 years. 

So in summary a decision tree is completely specified by decision nodes, chance nodes and terminal nodes. Decision nodes enumerate choices between alternatives that a decision maker can choose; chance nodes enumerate possible events determined by chance and probability which are conditioned on the alternatives that we’ve chosen and terminal nodes describe the outcomes associated with a given pathway of choices and chances. 

So let’s build a decision tree. So we’re going to consider an example; I’m a PhD and not an MD so many of you are probably MD’s so please don’t jump on me for the terribly incorrect clinical presentation that I’m about to give. The point here is to talk about the decision tree itself. So in this decision tree, it’s a decision problem, it’s a patient who presents with a set of symptoms which is likely a serious disease but it’s unknown whether the patient has the disease or what policy the disease is without some sort of treatment. There are two treatments that are available. There is surgery which is potentially risky and medical management which has the low success rate, but which is less risky. With surgery once the patient is on a table one must assess the extent of the disease and decide whether to perform a curative procedure or just to use palliation. And the goal here in this particular case is to maximize life expectancy for the patient.

So let’s determine the structure. So in the initial decision between surgery and medical management denoted with this blue square if you choose medical management there’s a chance that the disease is present and there’s a chance that the disease is absent; that’s not caused by medical management, that’s just the prevalence of the disease. If the disease is present and medical management were given there’s a chance that medical management results in cure and there’s a chance that it’s not effective and doesn’t result in cure. 

If surgery is given there’s a chance that the disease is present and a chance that the disease is absent; so remember this is prevalent, it’s not determined by the treatment so that probability that disease is present in the top is the same as the probability that it’s present in the bottom. If a disease is absent there’s a chance that a patient lives and there’s a chance that the surgery causes surgical death. If the disease is present there’s a decision now between trying to cure the disease or to use palliative surgery only.  There’s a chance of surgical death with attempted curative surgery and it with palliation. And if the patient does not die from surgical death there’s a chance that there’s a cure or no cure and there’s even a chance of a cure with palliation. 

All right so this is our complete tree structure. So now what’s the next thing that we need to do? The next thing we need to do is put probabilities on our chances. So this would be a path; I choose surgery, there’s a probability that the disease is present. I then choose to try to cure the patient, there’s a probability that the patient lives and does not have surgical death and given that the patient lives there’s a probability that they’re cured. That’s what this says. 

So we’re not going to add probability into the tree. So as you can see for example curative surgery has a high chance of cure, a much higher chance of cure than palliative surgery. Palliative surgery basically has the same chance of cure as medical management. The disease prevalence is the same for surgery medical management because it’s not caused by the alternative that we’ve chosen and then we have the problem of surgical death for palliation and much higher chance of surgical death for cure. So let’s keep going with our example. 

Now we’re going to add outcomes. The outcomes that we’re going to add are life expectancy, conditional upon each of our pathways. So if you die you have no additional life expectancy, zero years. If you are not cured you have two years of life expectancy. If you are cured you have 20 years of life expectancy. All right, we have now specified our tree. We have our alternatives, we have our events and we have our outcomes. 

So the next thing we want to do is to figure out on expectations what decision is the best decision; what decision maximizes the outcome that we’re looking to maximize. So this is -- we’ll use a process called averaging out and folding back. So that chance nodes we’re going to average across the probabilities, we’re going to multiply 10% by 20 years and 90% by two years and then the expected life -- the life expectancy for disease present averaging across cure versus non cure for medical management is whatever that result says; so that’s what that box says. And we can replace that little sub tree with 3.8 years expected for the disease present given that we were treated with medical management.

Again we can average out and fold back here. We’re going to get -- do the same calculation that the same little sub tree in this particular case. Again 3.8 years expected and we’ll do again, now a medical management we’re going to average out -- average out again 10%, 3.8 years, 90% 20 years. Medical management on expectation gives us an 18.38 year of life expectancy; so that’s as good as we’re going to do with medical management on expectation. So now we need to do the same for the surgical sub tree. So we’re going to average out and fold back for palliative surgery, 3.72 years. And we’re going to do the same here average out and fold back for curative surgery 90% times 20 plus 10% times two, 18.2 years. And now we’re going to average out and fold back between surgical and living after curative surgery, 18.2 times 90% plus zero times 10%, 16.38. 

Now we’ve gotten to a very important part of the tree. We have a decision node. At the decision node we do not average out and fold back. At the decision node we choose the alternative that has the highest expected values and so what we see is that curative surgery looks better than palliative surgery and so we’re going to choose curative surgery. We’re going to fold back the alternative we don’t like and only keep the one that we do like. 

So now we continue to average out and fold back 19.8 years of the disease is absent and now .9 times 19.8 plus .1 times 16.38 and surgery delivers 19.46 years. So again we’re at a decision node; we’re not averaging out, we’re folding back. We’re going to choose the alternative that gives us the best expected value. And we look and we take the difference between those two. We see that surgery is the best and gives us the highest value and incrementally it gives us 1.08 years of life expectancy increase relative to medical management. And so the option that we -- the recommendation that we would make would be two surgery with a tri-cure surgical option to try to get on expectation 19.46 years.

All right, so that is a decision tree. We could do the same thing if we also had costs; so we do the same averaging out and folding back with a second outcome at each of our terminal nodes and if we did that let’s say for cost effectiveness analysis and it might be the case that surgery costs $10,000.00 and medical management costs $100.00. So incrementally we would gain 1.08 years of life expectancy as we said before and the difference in expected costs would be 10,000 x 100.00 so $9,900.00 and then the incremental cost effectiveness ratio between surgery and medical management would be $9,167.00 per year of life gained. And so if we’d be willing to pay at least $9,167.00 per year of life gained we’d choose surgery with tri-cure otherwise we’d choose medical management. 

All right, so now I know that sensitivity analysis is way down in our list of things to do when we’re doing this analysis but in some ways sensitivity analysis is the most important part of a decision analysis. It’s the most important part of a decision analysis because it is very rare that we know all of our probabilities and all of our outcomes values for certain an so what we want to do is say “Our decision, our recommendation is either robust or not so robust to various uncertainties that we have about what we know about the state of the world”. So I’m going to talk about some of the few exemplar sensitivity analyses. There are also more advanced techniques which I’ll mention briefly afterwards but first we’re going to start with the simplest sort of sensitivity analysis called a one way sensitivity analysis or sometimes also called a threshold analysis. 

So sensitivity analysis is systematically asking what if questions. We’re going to look at how robust a decision is by varying one of our parameters, one of our probabilities in this case and seeing at which point we flip and all of a sudden we prefer medical management instead of surgery. So we’re going to look at this probability of surgical deaths given that we try curative surgery, which is that 10% in our base case, in our main analysis. So we’re going to ask the question how high does it have to be before we -- how risky would the surgery have to be before we would choose medical management instead. So we will change that probability and we will see the blue line is the outcome; so our base case is here, denoted in the purple -- I’m sorry let me use the pointer, denoted with the purple line that’s 10%, right? Or 10% probability, surgery the blue line is much higher than medical management the red line and we’re going to ask the question at one point, at what value of -- how likely would surgical death have to be before we would be wanting the medical management instead of surgery? Notice since this is a probability that only affects surgical death, medical management is flat regardless of what the probability is and we’re going to choose the alternative that has the highest expected value. Surgery until we get to this point then medical management.

So if the probability of surgical death was very high we would choose medical management, that’s the threshold, 70% in this particular case or roughly 70% in this particular case. And so it may be the case that our certainty about surgical death is not very good and we have a wide range of plausible values. And in fact it could be a plausible value might even be up to 80% in which case our decision might be somewhat uncertain, our choice between medical management and surgery wouldn’t be clear and certain one at least for this probability. Or it might be the case that we noted that surgical mortality is essentially a no center that performs the surgery higher than 20% in which case our decision is very robust to our uncertainty about this single parameter. 

But we can perform two-way sensitivity analyses. Here we are going to vary the probability of surgical curative death again between zero and one where our base case is 10% and we’re going to vary the probability of disease, which remember effects both arms of our tree. And what we see is that unless the probability of disease, the prevalence of disease is very high and the probability of surgical death is -- I’m sorry, if the probability of disease is very low or the probability of surgical death is very high we prefer surgery across a wide range of uncertainty. And then again in this two-way sensitivity analysis our decision is quite robust to our uncertainty about these two parameters. Our base case is here, we would have to be think that surgery is much more risky or that the prevalence is substantially lower. So it maybe the case that in some populations the prevalence is much lower and so we would prefer medical management. And in other populations the prevalence is very high such that even the surgical prevalence -- probability of surgical death, 50% would still prefer surgery. 

Now I’m only going to mention this more advanced technique doing sensitivity analyses which is called probabilistic sensitivity analysis. So there are articles and chapters of books written about this. The basic idea is that the sensitivity analysis as you saw in our tree we have I don’t know 15 or 20 values varying them all simultaneously leads to many, many, many, many combination and it’s very hard to do eight way sensitivity analyses and make sense of things. I have trouble graphing anything more than two dimensions. So probabilistic sensitivity analysis involves putting uncertainty distribution on each of the parameters, sampling from those distributions, running our model, figuring out which alternative we prefer and doing that again, and again and asking what the percentage of the time that we prefer surgery to medical management and giving that as a summary measure of how uncertain or robust our decision is. And there are -- as I said chapters and articles written about this, some journals require it for publication so it’s an important thing but it’s beyond what will cover in our time together today.

*Moderator:*  Jeremy I’ll just note that we are going to be doing a session specifically on sensitivity analysis in a few weeks; folks should look out for that and sign up if interested. 

*Jeremy Goldhaber-Fiebert:* Great, definitely attend that one. So now I’m going to talk a little bit about one other sort of modeling techniques that sort of go beyond or decision trees. And sort of the rationale for using them and they -- they are sort of mark off models are very, very commonly used for decision and cost effectiveness analyses as actually published in applied publications in major biomedical journals. So let’s take a look at Markov’s models. And essentially the idea about why you might want to choose a Markov model instead of just using a decision tree for a problem is when there’s a possibility of repeated events or decisions that especially when those events might occur at non-regular intervals. So if you have a decision about a one time immediate action that has sort of acute sense of outcomes and then people either walk away from that event or action and sort of live out the remainder of their life and there aren’t affected and other sort of complicated downstream ways then the decision tree is very important. And maybe an intervention changes and increases your probability of having a good outcome relative to a bad outcome. However if a decision is about a repeated action screening every five years or every 10 years for colorectal cancer or screening for cervical cancer or some sort of repeated treatment that’s given at various points in time that into a time dependent event it becomes very hard to model that in a tree structure. The tree becomes very, very long. It -- and I’ll show you kind of the illustrate, kind of how the tree starts to unfold in this unmanageable recursive sort of way. So here we have somebody who is healthy, they either can become sick at a given time point or remain healthy. If they remain healthy at the time point they can either remain healthy or they can become sick. If they happen to become sick they -- they could either get healthy again or they could have some bad events. Likewise if they become sick they can get better, they can stay sick or they can get very sick. And you can imagine how the tree structure sort of blows up over time for sort of this recursive expanding pattern in the tree that becomes unmanageable. 

So for situations like these we typically use things like state transition Markov model which is why I’m going to talk about Markov models now. So what is a Markov model? A Markov model is a master medical modeling technique from matrix algebra that describes transition, the transition a cohort of patients makes between a number of mutually exclusive and collectively exhaustive state during a series of short intervals or cycles. 

So there are certain properties of a Markov model. Individuals are always in one the finite number of health states. Events are modeled as transitions between one state and another; so typically we’re going to model discreet Markov models, use discreet Markov models. The time spent in each health state and certain events that occurred determine the overall expected outcomes, living longer without disease you’ll yield higher life expectation and quality adjusted life expectation. And during each cycle of the model individuals may make the transition from one state to another or remain in their current state. 

So we’re going to construct a Markov model and define our health states, we’re going to determine possible transitions between these health states which we call state transitions or transition which occur with transition probabilities and we’re going to determine clinically -- a clinically valid cycle length. And I’ll say something about that in a second. So the cycle length we want it to be short enough that for a given disease or a condition such that true events or transitions occurring within one cycle is very unlikely. 

So for a disease that has a rapid set of transitions between important health states we want a relatively short cycle whereas some other diseases you might be able to get away with a longer cycle, so typically a monthly cycle is used for many diseases. Some things like stays in the ICU are modeled with very short cycle length.

So we’re going to define let’s say the natural history of the disease for the hypothetical disease which is very simple, we’re going to have three health states. I’m either healthy or sick or dead and mutually exclusive and collectively exhaustive, one and only one of them and it’s best to define health states in terms of their biology or pathophysiology not in terms of test results. We’re going to make a set of Markovian assumptions. Markovian assumptions are homogeneity the, that means everybody who is in a given health state had the same probability of transition and the same rewards for being in that health state. If past history makes that not true then we need to have a stratification to multiple health state. And memorilessness, the current state determines all future risks. There are other advanced topics which I’m not going to cover today about how you stratify models and how you add tunnel states to ensure Markov assumption is whole. Let’s just stick with a simple example where we have healthy sick and dead and our Markovian assumption are pregnant.

So now we need a set of transitions. So I’ve drawn a set of transitions -- they’re denoted by these blue arrows and so a healthy person can die, they can be hit by a bus and a healthy person can become sick, a healthy person -- and those are the two -- or a healthy person which I didn’t draw here can remain healthy. A sick person can either remain sick, can become well again or can die; either because they’re hit by a bus or because their illness or disease causes them to die. And a dead person of course remains dead. So there are no transitions out of the dead state, it’s also typically called an absorbing state because people who enter do not leave.

You need to have a risk of death from all time in all states, no health state protects you from dying and death is the absorbing state in our biologically plausible model. So for -- we typically use software like trees or excel or other programs to represent these Markov models, but if we’re doing it as sort of a matrix algebra we’d represent our transitions in a matrix like this shown that the bottom and the notation is here is PHH, which means the probability of starting healthy and ending up healthy. The probability of starting sick and ending healthy and the probability of starting dead and ending healthy, which is zero because -- so the columns represent the transitions out of the first one is out of healthy, the second one is out of sick, and the third one is out of dead. So the one means that if you’re dead you stay dead and you don’t have any chance of travestying out of death and we know that the columns add up to one because probability you must go somewhere. So these probabilities might be estimated from epidemiologic longitudinal studies. 

Now at times zero or times we need to know the proportion of people that each of the health states; so lets say the prevalence of the disease, the baseline for maybe we start everybody as healthy if we’re starting model for babies or something like that. And each month or each cycle length we take our matrix and we multiply it by these proportions to get the proportions at the next time step, the time step key plus one. So the transition probabilities can be time dependent as well; right now we’re not going to show that. So the way you do this matrix multiplication if you guys remember from junior high school or whatever you were introduced to this is your multiplying rows times columns summing and then doing that again for the next row and the next row and that gives -- so the probability from healthy to healthy, from sick to healthy and from dead to healthy get times the proportion in those states gives us the proportion or healthy at the next time step.

So how does this model we’re going to get these proportions in each of the states at each of the times and if we think about doing that for many time steps we will get something called the Markov trait. So this -- so the green line for example gives us the proportion for healthy as a function of model time. The orange gives us the proportion that are sick as the function of model time and the red gives us the proportion of dead as a function of model time and what we see is as we run the model for longer period of time eventually everybody dies from old age if not from disease. 

So the proportion that are in these health states is not the prevalence that’s because we want the proportion divided by the people who are not dead, prevalence is the number of people in the state divided by the number of people who are not dead. And the model time is not the age unless you start the model at age zero but you can use if you know the certain age of the cohort you can use model times to compute the age. So when you’re using the software you’ll capture these things, remember these questions, they’re important for when you’re going to produce epidemiologic meaningful and clinically meaningful outcomes from your model. Another way to think of the traits is that we have the proportion in each of the states at a given time step or model stage is the initial stage -- this is each stage of the model and we’re going to run the model until everybody is dead or until we have run you know the model for a long time and this column just is one minus the probability of being dead so that we could compute the prevalence of healthy or the prevalence of sick.

So each of these health states -- living these health states I prefer to be healthy to being sick and I prefer being healthy or sick to being dead. So we can value these using quality of life weights that are derived from studies so we can compute quality adjusted life years. So we need to place a value on living a time step let’s say for the simple example in each of these states. So healthy is as good as it gets, I give that a one. Dead is as bad as it gets in this example, I give that a zero and sick is somewhere in between and these numbers come from some sort of study that uses standard gambles or time trade offs or some other elicitation methods to get how much we like one state relative to another. 

*Moderator:*  Sorry, we are going to be presenting another seminar that talks about how to actually derive these values in a couple of weeks, the --

*Jeremy Goldhaber-Fiebert:* Beautiful. So let’s say you have derived them using the lecture that you’ll attend. If you want to compute the expected qualities you multiply the proportion in the given state at a given time by the quality weights for being in that state and sum that with the proportion in the other state by the quality weight for that other state and of course the proportion dead times zero since our quality weight for dead is zero an we sum up. Now that’s the expected qualities; that’s not the expected discounted qualities which is the typical measure that’s used. But discounting is basically a way of expressing time preference and I’m not going to cover it here today. You can -- there should be a standard formula that you find in any decision analysis or cost effectiveness book which would be a slight modification to this formula. But the idea is we’re going to weight these outcomes and if we had quality weights of one for all non-dead health states, what we get is the life years or life expectancies. If we put weights between zero and one then we get the quality adjusted life expectancy. 

And cost we can do in a very similar way; the proportion in that state times the cost of living in that state times the cost of living in that state; the proportion living in another states times the cost of living in that other state. And additionally onto cost would be any event cost like if we provided treatment in that -- in some transition or other. And again we can add discounting. 

So interventions -- the typical way we think of interventions is that they’re going to modify some transition probability. I’m not going to walk through this example, I’m sorry because of time but the idea in the example that I’ve given is that the probability say of transitioning from healthy to sick might be modified by some streaming and treatment intervention where sensitivity and specificity determine our detection of people who are sick and our treatment effectiveness determines their likelihood of not transitioning to sick but rather to staying healthy; or if they are sick of transitioning back to healthy. So what this does essentially is it increases our probability of transitioning back to healthy; so we’re going to live more time in healthy, less time in sick and we’re going to have lower probabilities of dying in a given -- in a given cycle. And so here is our probability -- is the proportion of our population who is not dead, right? The proportion not dead from our trace before as a function of model time. And here the increase on top of that is the increase due to this intervention that I’ve modeled and the difference in the area between these two sets is the gain in qualities, the gain in life expectancy from the intervention.

So this is again I’m not going to walk through this because of the -- of our time but this is the model structure for that intervention with drawn in a Markov modeling diagram; so you can see that is very similar -- it’s very similar to how we diagram our decision trees but we have this special node, this Markov node so a healthy person either gets -- can either get sick and stay healthy or can die and if they die they transition to the dead state; if they get sick they transition to the sick state, if they get sick -- they stay healthy they stay in the healthy state and then we repeat the cycle at this Markov node.

And here is the same thing with -- for the intervention version and we deciding between intervention and no intervention where we either flow into the Markov model for the natural history and the absence of intervention or we flow into a Markov model where we would have the intervention. 

So the last thing I want to mention briefly because you may see it; not because we’ll be able to walk through it is that there -- in addition to state transition Markov models which I sort of talked about today, also called Markov cohort models there are another type of model called a micro simulation or an individual Markov model or first order Monte Carlo model there are a variety of different names. The difference is that as we saw just up above our cohort Markov models, the ultimate proportions of a very large or infinite cohort flowing between various states and our models. And the individual model we’re going to model individual people having chance probabilities in our transition probability of flowing through the model, getting sick, staying sick, getting healthy, getting sick again and dying. And we’re going to run lots and lots of people through the model and some people are going to get lucky and only be sick for a little while before they die and some people are going to get very unlucky and they’re going to die right away. And if we average across a large number of paths we would get proportions of those little individual people that we simulated that closely approximate our Markov cohort model. Now you might ask why would you go through doing individual level simulation if you could just do it with a Markov cohort model. And what this line basically says is that we could approximate this but we have to simulate a lot of people to get a very close approximately. So why would I do this?

I might do it even though this is more advanced and more complicated because a Markov cohort model when we need to do all this stratification by different people’s characteristics and different paths clinical history, the size of the Markov cohort model grows. The number of states that we have to have in the model grows very rapidly and the Markov model, just like the decision tree becomes entrap table to actually code up in a software language. And so -- and so the micro simulation you can essentially use something like a risk equation to talk about individual risk and so it’s much more easy to code -- to avoid the issue of state explosion. So that’s the general intuition about why you might want to choose that whereas encourage you to do is that if you felt that you might need to have a micro simulation for decision problem that you were doing talk with somebody who is modeling expert and really talk through it before going that route because it’s much more complicated. But that’s what a micro simulation is in a nutshell.

So just to wrap up with a couple pieces of advice and then I’m really happy to take questions so people who my mentors have told me know what the information your consumers need, know the outcomes that they value, the probabilities that they’re concerned about, the clinical details that are relevant to them and make sure that you understand those and that you either include them into your model, you have very goo reason for not doing so. Take a model that is as simple as possible, don’t start out with the most complicated model that you could possibly think of because you’re going to have to find probabilities for all of those things, you’re going to have to code it up, you’re going to have to debug it, you’re going to have to do sensitivity analyses with it and you need it to be understandable to people, you need it to be practical for yourself and you need it to be explainable. Also know the limits of what your model does; those decisions you made about what not to include in it. Make statements about what the model does in terms of the outcomes within those limits. All research has limitations both modeling research and other research. And that’s okay, know what those are and don’t over claim with your model and you will have you and your model will have a lot of faith for validity.

So in summary clearly define alternatives, events and outcomes. Formalize your methods for combining the evidence, a decision analysis can prioritize information acquisition, additional information for things that are uncertain, things that your model conclusions are sensitive to and decision analysis and healthcare providers and policy makers make medical decisions under conditions of uncertainty. Helpful today, I’ve included some standard references including a link to the society for medical decision making website which has a series of best practice guidelines that they published year or year and a half ago which updates on a lot of these -- of these publications that I’m showing here. So thank you very much and I’m happy to take questions.

*Moderator:*  Great thank you Jeremy. So right now we don’t actually have any questions but please submit those as we do have a couple minutes and we are -- we do have the advantage of having someone very well versed in this topic here at hand. But while we wait for some more questions to come in Jeremy I’m wondering if you can maybe speak a little bit more about how you decide on what health states to include in your model and what cycle needs to use for your model?

*Jeremy Goldhaber-Fiebert:* Yeah sure, so as I said health states I’m really trying to focus on the underlying -- in general the underlying disease natural history and biology. So an example is that let’s take cervical cancer screening and PAP smear screening. So there is a set of test results that are based upon examining cellular abnormalities and that might -- that might result. And you could think of modeling a disease in terms of being test positive or some extreme test result positive or something like that. The problem there is that if you then institute a second test that uses some other means of the testing, the underlying disease or condition let’s say the presence of human papilloma virus DNA that model structure doesn’t have -- it doesn’t have an obvious meaning whereas if you’re thinking about your model structure in terms of the presence or absence of various viral types and various sort of histological changes to the uterine/cervix that structure can be mapped via you know test characteristics to any type of test in sort of more robust to defining underlying biology; so that’s the kind of thing that I try to do in HIV you know people are modeling things in terms of vial load and CD4 count typically. In diabetes people are typically modeling things in terms of plasma glucose levels or HBA1C levels, which sound like a test result but which are actually sort of underlying biological kind of measures. In addition to the levels of changes that might -- that people want to experience in terms of kidney function, in terms of neurological damage, retinopathies etc.; so they’re representing a state that defines the -- all of the organ systems affected by the disease.

So -- and in terms of cycle length where I said before is that I don’t want it to be the case that I can have many transitions occur within a single cycle because then the set of transitions that I have to have, the transition from state A, to state B to state C all within a cycle become very complicated and so I want to choose a shorter cycle length so it’s very unlikely that I’ll experience more than one transition. I will either stay in that same space or at most I would transition just to the next kind of adjacent state if you will; not sort of big jump through a whole variety of states and just because it makes the modeling simpler. 

So you know if you’re talking about the Peds ICU where the health status of children -- babies can change very, very rapidly the cycle length may be very, very short minutes or hours or something like that. Whereas you know for -- we have a model of hepatitis C and in fact the disease itself can be represented fairly well with a cycle length of a month however treatment for hepatitis C is on the order of weeks. The treatment is changing a run in period of a few weeks of a different drug and then a different drug and various sort of check points about when biologic response is assessed and so we’ve actually opted to model our current iteration of our hepatitis C model as a model of  -- in terms of weeks.

*Moderator:*  Okay so a combination of both the disease characteristics as well as the treatment characteristics?

*Jeremy Goldhaber-Fiebert:* That’s right.

*Moderator:*  Okay so a few more questions have come in. First can you clarify the difference between first order and second order Monte Carlo simulation?

*Jeremy Goldhaber-Fiebert:* I’m sorry?

*Moderator:*  The difference between first order and second order simulations?

*Jeremy Goldhaber-Fiebert:* Yes. So a first order of Monte Carlo simulation is I will actually just go back to the picture for that; I think it will be helpful. Is just -- is saying that I k now the transition probability essentially with certainty but there -- but I’m going to run individual people. So the first order means I’m going to run individual people; so my uncertainty is because I have a finite sample of people that I simulated. Fine example of simulated individuals; that’s a first order of Monte Carlo simulation also called a micro simulation and the point of having such a simulation is because it enables modeling diseases where there’s too much explosion for state transition model. Second order of Monte Carlo simulation relates to probabilistic sensitivity analysis. I do not let a cohort model in general, I can actually also do a micro simulation that’s very advanced and there what I’m saying is I’m certain about the transition probabilities -- I’m sorry let me go back to -- yes I’m uncertain about the transition probabilities and the value that I find and so I’m going to sample from their uncertainty distribution. So for example I’m going to have a longitudinal study that looks at people who are sick at baseline and looks at their probability of returning to healthy. So that’s a study let’s say with 1,000 people so  my transition probability is pretty well estimated but there’s still some uncertainty around it, there’s still confidence interval around that and that confidence interval is essentially defines an uncertainty distribution that I’m going to sample from and I’m going to run my model, my cohort model many times in order to effect the certainty of my decision given my uncertainty about the parameters. That’s second order Monte Carlo simulation also known as probabilistic sensitivity analysis. 

*Moderator:*  There’s also a very nice explanation of this for folks that like to see it in written format in the Society for Medical Decision Making Recommendations their modeling chapter. And they do a nice job of explaining this as well and the way that they mentioned is a micro simulation is sort of -- let’s say that you have a 40% probability of something happening; that doesn’t mean every single person has a 40% probability of something happening. With each individual the events could occur, zero or it could -- or it could not occur, zero or it could occur, one and so that micro simulation helps you to model sort of that individual. And then even if we know for a cohort that the probability X is 40% it’s going to be some sampling error, some uncertainty because we’re getting that estimate from a cohort rather than from a population. But we -- that’s the recommendation also do lay that out for folks that are interested further.

Another question Jeremy is can you complete the system analysis using empirical data from the trial without engaging in modeling?

*Jeremy Goldhaber-Fiebert:* So the answer is yes but qualified. So the issue is trials in general do not follow people from their state now until they die. And so if you are interested in looking at the life expectancy or the quality addressed life expectancy or the total life times cost you would essentially need to have a trial at a very, very long follow up period. 

But in principle there’s no reason that you couldn’t. There’s also no reason that you couldn’t if the outcome that was of great interest was infection averted in the period of three years. And so where I would say there is still what you would essentially -- you essentially have a model in some -- in terms of statistical model in order to estimate the outcomes that you’re comparing in your incremental cost ration or your comparison of outcomes across the trial but there’s no reason why  you could not do that decision analysis just within the context of the trial and in fact there were a number of people I believe Drummond -- I believe Louise Russell (ph) might be on the call so she may actually know the citation better than I do but I believe that Drummond and Bernie O’Brien and a number of other people were involved in developing methods around sort of trial based decision analyses, so you can. And often times they are helpful especially if the situation -- the clinical situation you’re looking at is sort of acute, more acute or shorter term.

The other thing I would say though is that unless you think that you’re study sample is unbelievably representative of the larger population of patients that you’re going to deliver this intervention in you’d probably want to do sensitivity analyses and in which case if you’re even looking at sort of primarily a trial you might want to model in order to assess your uncertainty in terms of -- in terms of how representative your study population is -- your study cohort is for your overall patient population.

So I tend to think of these as complimentary methods and that’s why I engage in a lot of empirical -- data analytic work as well as modeling base work. I certainly don’t think that models replace trials and I certainly think that trials and observational studies can benefit from having modeling linked to them.

*Moderator:*  All right, okay. So we’re a couple minutes over. I’m just going to pose one last question and it is an important one and that is there are limits to the number of health states and Markov model handle which makes simulating individuals crafted -- is there any practical limits to the number of risks function that can be used in individual simulation?

*Jeremy Goldhaber-Fiebert:* I mean not -- so you can answer this in a couple of different ways. So first you have to estimate those risks some how so there’s a natural limit in terms of the number of things measured on enough people longitudinally to be able to estimate those risks equations appropriately that in terms of how they compete or interact with each other. Let’s say that you could generate a lot of that -- let’s say you have the access to the complete Danish following everybody registry or something like that; so you really, really, really estimate a lot of those risk equations and then basically it’s a computational size times sort of issue. You can code up a very, very complex detailed micro accumulation model that gets into a lot of the health details of people multiple conditions etc. and now it’s just about how long and how many computers you have available to you and with sort of the advantage of computing Amazon web services and Google, compute and you know the clusters that are available at many universities and NSF etc. You can simulate very, very complicated things. Remember in worlds way outside of health people are doing simulations, physics simulations of formation of clusters and galaxies and black holes and these sorts of things and the number of elements and the number of time steps that they’re including equations are enormous and the data that they generate are enormous and the whole situation -- the limiting thing on size is it really computational power, it’s the amount of data -- the amount of knowledge that we have about how a complete human’s health physiology, pathophysiology, interactions with environment, other people, interventions, health systems, how all that plays out, that’s actually what limits us in terms of the complexity of our simulation. 

*Moderator:*  Great, well thank you Jeremy. This is a fascinating presentation and I hope that our listeners found it to be as festinating as I did. We do have a few more cost effective analyses courses coming up so please keep your eyes out for those. The next one is going to be on transition probability, next Wednesday to save time. And thank you Jeremy for presenting to us.

*Jeremy Goldhaber-Fiebert:* My pleasure, take care. Have a good day.

*Moderator:*  Thank you. So Heidi I think you’ve got a poll you’re going to be talking about?

*Moderator 2:*Yes I just put up a feedback form if the audience can take just a few moments to fill that out we really do read through all the feedback that you send in; we use it for our current and upcoming sessions and it will just take a few moments. There is not a submit button, as soon as you click a button we will get that data. I want to thank everyone for joining us for today’s cyber seminar. We did send registration information out this morning for our session next week. I know you’re presenting, what was the title again?

*Moderator:*  Next week I’m presenting on Estimating Transition Probabilities for a Decision Model.

*Moderator 2:* Fantastic, I hope all of you can join us for that session. And thank you every one for joining us for today’s cyber seminar. Thank you.

Page 1 of 16