**Transcript of Cyberseminar**

**HERC Cost Effectiveness Analysis Course**

**Estimating Transition Probabilities for a Model**

**Presenter: Risha Gidwani, PhD**

**June 4, 2014**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact herc@va.gov*

Risha Gidwani:Good afternoon everybody or good morning as the case may be. I am Risha Gidwani, and I will be presenting today on how to Derive Transition Probabilities for Decision Models. Before I begin, a few reminders. This is about modeling, rather than measurement onset of clinical trials, so this is about probabilities for inputs in a model that you create yourself, which we talked about a little bit in this course. Dr. Jeremy Goldhaber-Fiebert did go over it in last week’s seminar. We are going to have an interactive example today so I will ask that you please get your calculators out or pull up an excel spreadsheet because in a few slides we are going to be working through an example together. 

With that, let us begin. We are interested in transition probabilities because they are the engine of our decision model. We can have a number of different types of decision models. 

[Cross talk]

Risha Gidwani:You can have a state transition model which looks at people moving from one health state to another. In this case, the transition probability would be the probability of moving from one health state to another. That could be something like moving from a cancer health state to remission. If we were doing a discreet event simulation model, we would be looking at the probability of experiencing an event. And here our transition probability may be something like the probability of experiencing an acute myocardial infarction. 

Because transition probabilities are the engine to our decision models, we are going to need to be able to derive inputs so that we can include them in our models. Oftentimes we will go to the literature in order to derive probabilities that we use as inputs and today we are going to learn when we can do this and how exactly we can do this. 

Before I go further, I would like to acknowledge Dr. Rita Popat who is an epidemiologist over at Stanford University whose feedback was very helpful in creating this lecture. Dr. Popat and I are currently working on some of these methods to derive transition probabilities and will be sharing them with a larger audience once they are available. 

This is a schematic of what a decision model may look like. Here we are looking at diabetic patients and we are hypothetically evaluating the cost effectiveness of Drug A versus Drug B. To do that we need to know how effective each one of these drugs is in achieving a controlled state of diabetes. This is the structure tour decision model. , now we need inputs for our model, those are represented in the blue boxes, and you can see here that right now we have no transition probabilities. 

I do want to point out that when we are building a decision model whether it is a cost effectiveness model or some other type of model we do not have to compare two drugs; we can compare any two strategies like we spoke about in the Overview to Decision Analysis Lecture that kicked off this cybercourse. Here we are looking at diabetic patients and they can either take a drug to control their diabetes or we can route them to a diet, exercise and telehealth monitoring program. Here we like to see the cost effectiveness of each one of these strategies. We still need to derive transition probabilities. Here, the probability for the drug would be the probability of achieving controlled diabetes if you have taken a drug and for the second treatment strategy it would be the probability of achieving controlled diabetes if you were engaged in diet, exercise and telehealth program.

I filled in here some hypothetical examples of what probability inputs might look like. Here we might find for example that eighty-four percent of patients on a drug achieved controlled diabetes versus seventy-six percent of patients in a diet, exercise and telehealth monitoring program achieving controlled diabetes. Now we would want to figure out what the cost effectiveness was of each one of these strategies, which depends not only on these probability inputs, but also the cost as well as the health effect associated with each strategy. 

To derive these model inputs that you saw in the blue squares we can use one of two major strategies. We can attain existing data form a single study or if there are multiple studies that exist, we can synthesize those existing data from multiple studies and do so for the mechanisms as a meta-analysis and mixed treatment comparisons or a meta-regression. 

In today’s seminar, we are going to talk about how to attain existing data from a single study and next week I am going to talk about how you can synthesize existing data from multiple studies. 

So let us begin with how we derive existing data from a single study. If you are very, very lucky, you will find a journal article that will have exactly the type of information you need. For this example, we might find the probability of achieving controlled diabetes at three months for people that engaged in a diet, exercise and telehealth monitoring program. However, the vast majority of people are not extremely lucky and most people need to modify existing literature in order to derive model inputs. 

There are many different types of inputs that are available from the literature. The ones that I bolded here are the ones that you are most likely to see and those could be probabilities, mortality rates, relative risks, odds radios, means and medians. The statistics that are above the dotted line represent data for binary outcomes and the specifics that are below the dotted line represent data for continuously distributed outcomes. 

When we are using inputs in a decision model, we need data in the form of probabilities. The biggest probabilities are used for binary or yes/no outcomes. What we have in a decision model are binary outcomes. The probability of moving from one health state to another is a binary outcome and that is why we are interested in probabilities. 

So there are a lot of different statistics that you will see in the literature and this table summarizes what exactly they mean and what their ranges are. In the interest of time, I am not going to go into explaining every single one of these but I hope that you can use this in the future as a reference. What I will point out right now is the difference between a probability and a rate. Let me see if I can use this arrow here, it looks like it came up. When we are looking at a probability, which is sometimes called a risk, we are looking at the number of events that occur in a time period divided by the number of people followed for that time period. Conversely, when we are looking at a rate, we are looking at the same numerator, the number of events that occurred in a time period but now we have a different denominator. Our denominator is the total time period that is experienced by all subjects that are followed. 

When we are using data as input in a decision model, we need non-comparative data and here I have now listed what types of data each one of these statistics are and whether they are non-comparative or comparative. Inputs for a decision model are going to require non-comparative data and by that I mean, using an example of comparing Drug A and Drug B, what we want is the probability of controlled diabetes with Drug A being our first input into the decision model. Our second input would be the probability of controlled diabetes with Drug B. Each one of these is just looking at probability in one group or non-comparative data. 

There are some statistics that you can transform the non-comparative data if they are comparative data and we will talk about those here. Here you will notice that with the odds ratio I have a dotted line, same that we can transform the comparative data to non-comparative data because in some situations you will be able to do this and other situations you will not be able to do this and we will talk about those different situations. 

When we use probabilities that exist in the literature, so you go to a journal article and they do not report a relative risk or odds ration or a means, they actually report a probability itself then you have at least the data in the statistic that you are interested in of probability. However, that literature based probability may not exist for your timeframe of interest. Dr. Goldhaber-Fiebert spoke last week about cycling for models, each decision model is going to have its cycling. We need to make sure that the probabilities we are using as inputs into the model are relevant to the cycling to that model.  If your literature based probability it not relevant for the cycling of your model, you can transform that probability into a timeframe that is. 

For example, I may find for my hypothetical decision model that a six month probability of controlled diabetes is reported in the literature. But my model has a three month cycling; I would therefore need a three month probability. 

Probabilities cannot be manipulated easily. You cannot multiply or divide probabilities. I cannot take a six month probability in the literature divided by two and then get a three month probability.  So another example is that a hundred percent probability of five years does not mean a twenty percent probability at one year. Keep that in mind you can work backwards and it will show you that there is a ludicrous result and it will help you remember this. For example, thirty year probability at one year will not mean a hundred and twenty percent probability at four years. You of course cannot get above a hundred percent probability. This is a good way to keep in mind that you really cannot multiply or divide probabilities and sort of use these examples as a check to remind yourself of that rule. 

What you can do is transform a probability to a rate in order to change it to a timeframe of interest. You can do that because rates can be mathematically manipulated, they can be added or multiplied even though probabilities cannot. In order to change the timeframe of a probability if I wanted to get to a six month, I wanted to change a six month probability into a three month probability what I could do is change that probability into a rate, change that rate back into a probability and then I would have a probability relevant for my timeframe of interest. One thing to keep in mind though is that this calculation does not assume that the event occurs at a constant rate over the time period of interest. If that is not the case, then you cannot use this conversion. 

Before I get into how to actually do the rate to probability conversion, let us step back and talk a little bit more about rates versus probabilities and how exactly they are calculated. As a reminder, a probability is the number of events that occur in a time period divided by the number of people that are followed for that time period. A rate is the number of events that occur in a time period, divided by the total time period experience by all subjects followed. 

We can go through an example here and see what that means. So here, I have a situation in which I have four people that are followed and they are being followed until they die. Here I have a person number one who died at three years; person number two did not die and lived for the entire timeframe; person number three died at one year and person number four died at two years. When I do my calculations, I find that my rate of death is three divided by ten and the ten is obtained by looking at the number of years for which people are followed. Because of person number one died at year three, that person is only followed for three years; person number four did not die and therefore is followed for four years or the entire time of the study. Person number three died at one year and person number four died at two years. Therefore, my denominator or the number of the time period experience by all subjects followed is three plus four plus one plus two, which sums to ten and my rate of death is three divided by ten or 0.3 per person year. My probability is the number of events that occurred in a time period divided by the total number of people followed for that time period. Because I have three deaths and four people followed, my probability of death is seventy five percent. 

Now this is a different example and it will show us that in rates, we care when the event happens because it changes the rate, but it does not change the probability. Here on the left hand side, I have the exact same example that I had on a previous slide. On the right hand side now, things have changed. Person number one instead of dying at three years is dying at 0.5 years; person number two has the exact same experience; person number three has the exact same experience and now person number four instead of dying at two years, is now dying at 0.5 years. My rate is again the number of events that occur in a time period divided by the number of, I am sorry, the time period experience by all subjects followed because people are now dying earlier in my right hand side, they are experiencing less time that they are followed. My denominator, my numerator actually stays the same; there are still pre-events that occurred, three deaths that occurred. My denominator however changes; my denominator is now 0.5 plus four plus one plus 0.5, which sums to six. My rate is three divided by six and now it becomes 0.5 per person year compared to the 0.3 per person year that it was on the previous slide. My probability of death remains the same because I still have three events and four people that are followed. The time period at which an event happens will change the rate, it will not change the probability. 

Now that we have understood a little bit more about the difference between rates and probability is I will show you the equation that you can use to convert from probability to rates. If you want to get a 

rate from a probability, a rate is calculated as a negative natural log of one minus the probability divided by the time. If we want to change a rate to a probability, we do so by looking at the fact that a probability equals one minus the exponentiated value of the negative rate times the time. So these are a lot of equations, let us put some numbers in here as an example so that we can get a more concrete understanding of what this is doing. 

Here I have an example of the three year probability of controlled diabetes being sixty percent. What I want to know is what is the one year probability of having controlled diabetes. I am going to assume that the incidence rate of constant over three years and by doing so I am able to employ my rate to probability conversion equations. My rate is again the negative natural log of one minus the probability divided by the time period. Here, that becomes the negative natural log of one minus 0.6 divided by three and that is 0.304. Now here I have divided by three because I am taking a three year probability into a one year probability. When I want to take this rate and convert it to a probability I plug this value into my probability equation. Now I have one minus the exponentiated value of negative 0.3054 times one and that gives me 0.2632 or twenty-six percent. Notice what I did here is first transform three-year probability to a one year rate and then I transform that one year rate to a one year probability. You could also take a different approach in which you transform the three year probability to a three year rate and then the three year rate to a one year probability. In this case, the time would be one-third instead of the one that you see here. There are just different approaches each one of them is going to give you the same response. To reiterate here what I did was transform the three year probability to a one year rate by dividing by three and then once I have that one year rate it is a one year probability and that is why I have a one year. We have a hypothetical data that thirty percent of people have controlled diabetes at one year and this was let us say was published in a journal article. We have one year probability or one year cycle length, excuse me, for our decision model so we need to know what is the one year probability of having controlled diabetes. 

I would like you to go ahead and use the equations that you see on your screen and calculate what is the one year probability of controlled diabetes based off of this information. Let us take a few moments, about thirty seconds and let us see what sorts of answers pop up.

Moderator:Do you want to put the answers up here; do you want me to bring up a fresh white board?

Risha Gidwani:If you can bring up a fresh white board that would be great. 

Moderator:Okay. 

Risha Gidwani:Although actually it looks like then people do not get to see the equation. 

Moderator:They do not get to see the equation. Sorry, let us go back to the equation. 

Risha Gidwani:That is fine. 

Moderator:And how can we have these answers pop up?

Risha Gidwani:What I am going to do I just need to change the permissions here and I will let the audience have the ability, this is going to get a little messy everyone, but we are just going to bear with it. I am going to give everyone the ability to write on the slide. 

Moderator:Lovely. Looks good, let us give about thirty second for this and I am not sure if you have a watch. 

Risha Gidwani:Actually, I have a clock here so I am going to start this and we will...

Paul Barnett:Submit it Risha, we are not quite that fast. 

Moderator:It is actually not letting me do it so, it is wanting me to do, it is not liking the additional. It will let me do a white board. 

Risha Gidwani:Okay in that case not a problem I would rather have them see the equations. I will give them my answer and hopefully they can check it out. 

Moderator:We do have the Q&A screen if people want to type their answer in there and we will be able to see that if that would work. 

Risha Gidwani:Sure. 

Moderator:Okay, so for everyone if there is that Q&A screen just type your answer in there and we will go through the answers that we receive. 

Risha Gidwani:Right. Alright we will give just about twenty more seconds here, it looks like there are a few answers that have popped up. If you are still working on it, just finish up and then we will reconvene. Okay great, I am seeing a bunch of answers pop up including a number of correct answers. It does look like a few folks have not done the complete conversion, but they got the first correct answer and now just need to get to the second. The correct answer is 6.89, which many of you have noted which is great. To just work through this, as a group what we have done here is we have taken our rate which is the negative natural log of one minus 0.30 divided by five and that gives us a value of 0.0713, which I do see some folks have noted up here in the Q&A. So that is great, what you have done here is you have taken a five year probability and converted it into a one year rate. Now we need to take that one year rate and convert it back to a one year probability so we use our second equation which is one minus exponentiated value of negative 0.0713 times one that gives us a value of 6.89 percent which I see most of you obtained. Great work. 

We have talked about probabilities and rates and converting them so can we convert a probability to a probability? Yes, it already is one, but the important thing to remember is that you need to use rates to convert the time period to which they apply. A rate can be converted to a probability as we just saw in the previous equation. 

Now I am going to go through a number of different statistics and talk about how we can convert these probabilities. I am going to next speak about relative risks, odds and odds ratios, which are oftentimes going to see in the literature. 

Paul Barnett:Risha, I am sorry that I neglected to note a point that one of our or question I guess that was asked here before we move on it might be good to address. It is asked – are some statistics that are described as rates really probabilities. For example, the number of infant deaths per thousand live births is described as an infant mortality rate. 

Risha Gidwani:yeah they certainly can be and this is a problem that we often see in the literature and people confusing rates and probabilities. For this reason when you see something that is described as a rate you really need to see how it is calculated and think critically about it. The denominator of a probability is a number of people fall into that time period so the number of live births is an indication of the number of people. A rate has a denominator as the total time period experienced by all subjects followed so if a thousand people are followed for one year, then your denominator would be a thousand person years. So you certainly do need to think critically about what exactly that statistic is giving you, if the denominator is a number of people followed, it is a probability. If the denominator is really the total time period experienced by the people followed, then it is a rate. 

Paul Barnett:Someone made the same point about smoking quit rate is another kind of misuse of the term. 

Risha Gidwani:Okay. We have a number of statistics reported in the literature, rates, probabilities, rates that are really probabilities, you definitely do need to sort of think critically about this and not just accept what is published in a journal article even if it looks to be published in a high quality journal. Unfortunately, you can still see some things that get mixed up here so be a wise consumer of information that is published. 

Okay so get back to this slide, you can see here that I have noted mean difference and standardized mean differences as potentially being able to be converted to probabilities. These topics are beyond the scope of this seminar, I am just noting them here to indicate to you that if you do find a mean difference or standardized mean difference in the literature, all hope is not lost, you can potentially convert these to probabilities but they will require the involvement of a Ph.D. statistician. So do not try this at home by yourself, but try to get somebody on board that has some very strong statistical and mathematical knowledge and you may be able to use these. 

Well you are most often going to see that it is a relative risk to odds and odds ratios and we are going to talk now about how to convert those to probabilities. In the beginning, there were two by two tables and anybody that has taken an epidemiology class will be familiar with these two by two tables, which essentially are looking at people that do and do not have an outcome and people that are and are not exposed to an intervention. I should not say intervention; it could be any sort of health process whether it is being exposed to surgery, being exposed to smoking or being exposed to a drug. 

You can see here that we can calculate the probability of outcome and expose which is usually what we are interested in for our decision model directly from this table. That is A divided by A plus B. I am going to try to draw here so we have A, I guess that did not work, it could be my arrow here. 

Moderator:You just need to click the arrow off and then you will be able to draw. 

Risha Gidwani:Okay great thank you. So the probability up from the exposed is A divided by A plus B. The odds ratio is AD divided by BC and the relative risk is the probability of outcome and exposed divided by the probability of outcome and unexposed. We do not need to go through each one of these equations in detail, but I just want to point out that the probability of the odds ratio and the relative risk are all coming from the same source of data. If you have the two by two table, you are set you can just calculate the probability from this itself. Most of the time in a journal article you are not actually going to have the two by two table, you are going to have to have the summary statistic of the odds ratio or the relative risk reported. But knowing that these all come from the same source, you may be able to then convert these odds ratios or relative risks into probabilities. 

I am sorry, I skipped a slide, let us go back here. This is the same slide, but what I have done here is I have now made this pertinent for our example of controlled diabetes and uncontrolled diabetes. Here we may have an example of a journal article, which looked at Drug A versus placebo and how well each did in achieving controlled diabetes. The probability here would be the probability of controlled diabetes with Drug A. The odds ratio would be the odds of achieving controlled diabetes with Drug A versus the odds of achieving controlled diabetes with placebo. The relative risk would be the probability of achieving controlled diabetes with Drug A versus the probability of achieving controlled diabetes with placebo. 

Let us speak a little bit more about this odds ratio versus relative risk. The relative risk probability of outcome in exposed versus probability of outcome in unexposed is different than the odds ratio which is the odds of the outcome in exposed versus the odds of the outcome in unexposed and the odds themselves are the probability of outcome divided by one minus the probability of outcomes. Because the relative risk is dealing with two probabilities, it is easier to interpret than the odds ratio, which is dealing with two odds. But the odds ratio has better statistical properties. The odds ratio of harm is the inverse of the odds ratio of benefit; we cannot say the same about the relative risk. The relative risk of harm is not the inverse of the relative risk of benefit. So if you were really interested in understanding the relative risk of benefits, but what was reported in the literature was a relative risk of harm, you would not be able to invert that in order to get the statics that you are interested in. 

Much of the data though that is reported in the literature is going to be reported as odds ratios. And oftentimes that is because those are the default output of a logistic regression, the relative risk will not be the default output of logistic regression so most people are going to run a logistic regression and report the odds ratio.

Let us talk about how we can actually derive probability from a relative risk. Relative risk again probability in exposed divided by probability in unexposed. What we are interested in for our decision model input is the probability of an outcome amongst people that were exposed. Here it would be the probability of controlled diabetes amongst people that were exposed to Drug A. In order to derive this we would take the relative risk and multiply that times the probability in the unexposed. So let us see why this works. The relative risk is a probability of exposed divided by the probability in unexposed. When we complete this equation, we see then that the probability in exposed is equal to probability in exposed divided by the probability in unexposed, times the probability in unexposed. The probability in unexposed of course cancels each other out and then we see that the probability in exposed equals the probability in exposed. 

This does of course require that you are able to find the probability of unexposed in the journal article which most of the time you will be able to do. So let us go through another example of this using actual numbers. Let us say from the literature we see a relative risk of 2.37 and that journal article also told us that the probability in unexposed was 0.17. In order to derive our probability in exposed which is what we are interested in, we will multiply this 2.37 times 0.17. That gives us a probability of 0.403 or 40.3 percent. One thing to keep in mind though is that this is the probability in exposed over the entire study period. If this relative risk is being recorded for a two year follow up and you are really interested in the one year probability you are going to have to take this 40.3 percent that applies to two years and use that probability to rate back to probability conversion that we went through earlier to derive the one year probability in the exposed. 

There is a caveat when you are using this type of conversion. The probability I am sorry the relative risk if it is the result of regression has been adjusted for covariates. Most of the time the probability in the unexposed that is reported in the journal article will be unadjusted. This causes a bit of problems. We have our probability in exposed equals the relative risk times the probability of unexposed, which is the same equation we saw before, I just now note that the probability in unexposed is unadjusted. 

When we use our equation we see again the same thing that we did before, now what I have just done is I have noted what data are adjusted for covariates and what data are unadjusted for covariates. Here are probability in exposed, equals a probability in exposed adjusted divided by the probability in unexposed adjusted, that is our adjusted relative risk and we multiply that quantity times the probability in unexposed, which most likely is unadjusted and reported that way in the literature. Therefore, our probability in unexposed unadjusted does not really cancel out our probability in unexposed adjusted. There is going to be some bias here if this is the way that you derived your probability estimate. Now not all hope is lost, you just have to know that you have some degree of inaccuracy in your derived probability in exposed from this relative risk calculation. So you are going to have to make sure that when you do your decision modeling you vary this input extensively in your sensitivity analyses. Then we will speak more about sensitivity analyses in another lecture that is coming up in a few weeks. 

So relative risks are nice they are easy to interpret and we can in many cases derive probability in exposed from them, but odds ratios are more likely to be reported in the literature than relative risks because as I mentioned before they are the default output from a logistic regression. 

How do we derive probability from an odds ratio? If the outcome is rare, meaning it occurs ten percent of the time or less, then you can assume this odds ratio approximates relative risk. And many of you may remember this from your epidemiology classes. If the outcome is not rare, if it occurs ten percent or more of the time, this is an advanced topic and you should not proceed without consulting a statistician. 

Here is a really nice schematic that comes from a Zhang and Yu article that was reported in *JAMA* in 1998 and it shows you the relationship between a relative risk and the odds ratio. Here you can see that when we have an incidence that is, I am having trouble with this arrow, here we go. When you have an incidence, that is ten percent or less, meaning that the outcome is rare, the relative risk pretty well approximates the odds ratios. Odds ratios seen on the left axis the relative risks are these lines that are going across the Y axis. You can see that once we get above an incidence of ten percent that the relative risk starts to divert from the odds ratio. When the odds ratio is greater than one, the relative risk will over, I am sorry, the odds ratio will overestimate the relative risk when it is greater than one and underestimate the relative risk when it is less than one. 

This is a good thing to keep in mind if you have an incidence that is ten percent or less you can feel pretty confident that the odds ratio will approximate the relative risk. Also, you will notice that when we have odds ratios that are closer to one, we can have a higher incidence rate before the relative risk and the odds ratio start diverging. When you have more extreme odds ratios the relative risks and odds ratios do start diverging pretty quickly. So really when you are trying to figure out whether your odds ratio should approximate your relative risk, you want to look both at the actually value of the odds ratio as well as the incidence rate. So again, odds ratio is a good measure of the relative risk and the outcome is rare. 

So let us calculate a probability from an odds ratio when we have a rare outcome. Here we have our equation again that the probability in exposed equals the relative risk times the probability in unexposed. We have a rare outcome so we are going to assume that the odds ratio approximates that relative risk and then we are just going to plug in the same equation. Here we have an example of where we have an odds ratio of 1.57 and the journal article has reported for us that the probability of outcome in unexposed is eight percent that is less than ten percent so we consider this to be a rare outcome and we can assume the odds ratio approximate to relative risk. Therefore, down here below, I am going to implement the same equation and I am going to derive the probability in exposed which is 1.57 times 0.08 and that gives me a value of 12.56 percent. Here is actually the two by two tables for which I derive these data. Now this two by two table is not going to be reported in your journal article. If it is you can just get the probability from that itself, but I am showing it here because you can see that if we look at the underlying two by two table data, that the probability in exposed is twelve divided by twelve plus eighty-eight or twelve divided by one hundred meaning it is twelve percent. Our value of 12.56 percent pretty closely approximates the twelve percent that we would have gotten had we been able to go to the underlying data themselves and calculate the probability from them. 

So this probability in unexposed is a key variable, it tells us whether we can assume that the odds ratio approximates the relative risk. It should be available in a journal article from which you are getting the odds ratio, if it is not, you can try going to the literature to try to find the value first and work with the patient. 

So we talked about probability from relative risk and probability from odds ratios, now let us talk about probability from odds, which is one of the easiest things you can do. Here you can see that equations that the odds equal probability divided by one minus probability, therefore the probability equals the odds divided by one plus the odds. If we have an odds of one-seventh that is going to give us a probability of 0.125 and we derive that by plugging in the odds in the numerator, one plus the odds in the denominator and that gives us this value of 0.125 or 12.5 percent. 

If you have data yourself and you are running let us say logistic regression you can actually get the probability from a statistical program yourself and you can do so using the margins command. The logistic command in data is going to give you the odds ratio and you would normally type in logistic Y X. Here what we would do in order to get the probability is we are going to type in logistic Y I.X and what is I.X does is that it dummies the X variable. You do not often do this when you have a variable with two levels like male/female or yes/no, but you have to do this even if your variable has two levels if you want to be able to get the predicted probability. So you would type in let us say logistic controlled diabetes I.Drug A and then margins I.Drug A and that will give us the predicted probabilities of X given that Y equals one.

We have gone through now and we talked about probabilities, rates, odds, odds ratios and relative risks. To summarize you can convert and odds to a probability. You can convert an odds ratio to a probability if the outcome is rare and you have the probability in unexposed and you can convert a relative risk to a probability if you have the probability in unexposed. 

Now we are going to talk about risk differences. A risk difference is the risk in one group minus the risk in the other. Some people use the word risk, risk really just means probability. If we want to calculate a risk difference, we can calculate the probability in one group minus the probability in another. Here for our hypothetical example we have the risk difference as being the probability of controlled diabetes with Drug A minus the probability of controlled diabetes with placebo. This would be very hypothetically; these are all made up examples. This could hypothetically be 0.84 minus 0.17, which gives us a risk difference of 0.67. This is the change in risk or the change in probability that is due to treatment. If the treatment has a lower risk than control then the risk difference is going to be negative. If the treatment has a higher risk than control then the risk difference will be positive. 

To continue more with the risk difference in how we get probabilities from it, risk difference is a probability of treatment minus the probability in control. If the article gives you the risk difference it will oftentimes give you one of the other probabilities. If it gives me the probability of treatment, that is fine you use that directly, that is the input you want into your decision model. If the article gives you the probability of control as well as the risk difference, you use that to drive the probability of treatment so pretty straightforward here. 

We just have gone through now risk differences and risk differences X minus Y equals Z or probability in treatment minus probability in control equals the risk difference. You can derive a probability I should say if the paper reports X or Y in addition to Z. 

Now that we have covered those, let us go on to the survival data. When we talked about probabilities in this lecture, we assumed that they are constant throughout the model and they do not have to but they can be. So I could say that the probability of achieving controlled diabetes is going to be the same whether I met cycle number one in my model or cycle number one hundred in my model. I could also change that, it is not set in stone, I could certainly make probabilities change over time or I could keep them constant. The one probability that I should never keep constant over time is that of survival or inverse mortality. We know that the risk of death is going to increase as people age and as the model cycles continue, people are going to get older and older. Therefore, survival should not assume to be constant over time. You are going to therefore have multiple probabilities for deaths in your model, one for each time period of interest whatever that time period might be. 

There are a few different types of survival data that you can use in your model and they are going to have generally different sources. The first type of data is all-cause mortality so this would be somebody dying from getting hit by a bus, dying from a co-morbidity, dying from really any cause that is why it is called the all-cause mortality. You can get this oftentimes from the CDC or from other purveyors of these mortality rates. These are oftentimes if you get them from the CDC they are going to be age and sex adjusted so all-cause mortalities usually reported as a rate. You are also going to have survival data that is specific to your disease or potentially to your treatment. Say if your treatment is surgery and that disease or treatment specific data is most likely going to be reported as a probability of death at a particular time. Oftentimes these data are going to come from survival curve. 

Here is an example of some rate data, mortality rate data that are reported by the CDC for 2010. Let us say that I have a model and I am interested in looking at people age seventy-five and older who are males. I would take these data that are highlighted in this blue box from the CDC and I would convert these, I know that they are per one hundred thousand and I would convert these into probabilities using the rate to probability conversions that we talked about before and the equation of which is noted here on your screen. On the left hand side column, I have just denoted what the CDC numbers are for each age range. We know that they are per one hundred thousand because that is what the CDC told us and from that we get a rate and I convert this rate to a probability of death using this equation. What I then do is I know that these rates are specific to age categories. So when I enter these rates into my decision model this is how I would do it. I would do so in the form of a table, rather than entering a singular point as estimate as the probability of death, I would create a table in which the probability of death is dependent upon the age that somebody is experienced in the model. In my model, everybody starts off at age seventy-five and that is why in cycle zero people are seventy-five years old. I have a hypothetical one year cycle length, therefore in cycle one or the second year of my model, people have now aged and they are seventy-six years old. What I have done here is I have taken the probability of death that is specific to this age range and I have included it here in my table so that people age seventy-five to seventy-nine have the probability of death at seventy-six percent per year. Once they jump to age eighty their probability of death jumps to 0.82 and it stays 0.82 until they reach age eighty-five until which it jumps to ninety-four percent.  This is one way of doing it; you centrally do not need to do it this way. Another potentially even better option would be actually taking this value that we get for the probability of death and applying it to the mid-point of the age category. So here it would be 0.76 for people that are seventy-seven years old; 0.82 for people that are eighty-two years old and 0.94 for people that are 86.5 years old for example. And then what we could do is we could take the values that are associated with the midpoint of this age range and linearly interpolate between them for the age ranges that fall in between. Either option is fine, the second one actually may be a better way to do it. 

That is getting rates, probability of death from rate data. Now we will talk about how you get probability of death from disease specific probabilities that are reported in the literate through survival curves. There are two big categories of survival curves that are going to be reported in the data and the first one is the Kaplan-Meier Curve and the second is the Cox Proportional Hazards Curve. 

The Kaplan-Meier Curve is oftentimes used with randomized control trial data. Through the magic of randomization if it is done appropriately, people are going to be bound to the baseline for their covariates and therefore it is unadjusted data. The Cox Proportional Hazards data are used with observational data and they are adjusted for covariates because usually any data this collected on two groups of people through observational purposes is going to have imbalance in covariates at baseline. 

This is a survival curve that has come from the literature and I am sorry it looks like the citation has been cut off. This is a paper that comes from *The Lancet* and it is looking at two groups of people that had cardiac admissions [dead audio], ablation together with the control group and you will notice here that this has some dots in it. These dots are denoting censoring or lots of follow up. You can see on the left hand side that we have the percentage of people who have lived. And so from this curve what we could do is we can focus in on the time period of interest. For us it might be month eighteen and we can see here the probability of living at month eighteen is about forty-four percent from the control group. And from this curve, we could get the probability of surviving at different time periods. Now this is not accounting just getting these data from the survival curve is not accounting for the censoring, but it is a very sort of quick and dirty way to get probability of survival or probability of death from data that are reported in curves. 

When we have data that comes from a continuous distribution and we want to get probabilities from that, we run into more difficulties. So let us say that we will have data that are reported as mean, meaning that they are coming from a continuous distribution. If we want to convert these to probabilities, we need first a validated way to generate a binary variable from a continuous distribution. This is often present in the literature. So for example, there may be some agreement that controlled diabetes is indicated by a hemoglobin A1C of less than seven. If a journal article had and I know that there are some contention around the level of hemoglobin A17 that is considered controlled diabetes but maybe we can get an expert panel may stop for our model purposes, a level of 0.7 with an appropriate threshold. We have a journal article with reported data in terms of the mean hemoglobin A1C of patients. If we have a threshold that has been agreed upon, here we are going to pretend that it is seven, we could take the mean data and assign controlled versus uncontrolled diabetes using this threshold. 

However in order to do this, we not only need this threshold, but we need an estimate of variation that has been reported around the point estimate. If we have mean data, we need the standard deviation or the variance. If we have median data, we need an interquartile range or a range. This can be a cumbersome process. I have been through it before and it can involve a lot of steps, but it is possible however, I would recommend that you involved a statistician if you are trying to derive probability data from continuously distributed data. If you do not have the ability to involve a statistician a quick and very dirty way to do this would be to use the mean and variation to plot a distribution and then estimate how many people fall below a cutoff. But this is definitely a dirty way to do this, as you are assuming that y our distribution is perfectly normally distributed. This is why it is much harder to do if you have the median because you can assume you know that your data are not normally distributed. But if you have median and you ideally have the minimum, the maximum, the twenty-fifth percentile and the seventy-fifth percentile, if you involve a statistician you should be able to estimate a probability from the median in the types of measures of spread. I do want to emphasize that what I am saying here is really just a very quick and dirty recommendation if you do have continuous data, please, please involve a statistician if you can. 

Getting back to our table here, we have gone through all the different statistics, probabilities, rate, odds, odds ratios, relative risk, risk differences. We just talked about survival curve data and know that we can get probabilities from these data, but we have to remember that these probabilities are going to be conditional and can change with each time period. We may be able to convert mean data to probability so we have an estimate of variation, but hopefully you will have the involvement of a statistician when doing this. 

One thing that I do want to point out is that we still need to derive estimates of variation around whatever point estimates we derive for our probabilities. This is going to be necessary for sensitivity analyses. It is more of an advanced topic and beyond the scope of this seminar, but it is something that I want you all to keep in mind. 

Before I wrap up, I do want to point out that when you are taking inputs, I am sorry you are taking literature and using that literature to derive your model inputs, the quality of the literature that you look at is going to matter greatly in the quality of the input that you derive. Here is our example of our diabetic patients that could be routed to a drug or have diet and exercise recommendations plus telehealth monitoring. In this situation if we were deriving probabilities from literature based inputs we would ideally want these two treatments to have been studied in a head to head randomized control trial and for the data for each one of these arms to have been reported separately in the journal article. However, much of the time you are at the treatments of interest are not going to have been studied in a head to head randomized control trial. In this case, which is more common we would then want the drug to be compared to placebo in a randomized control trial and diet, exercise and telehealth to be compared to placebo in another randomized control trial and we would want these two randomized control trails to have enrolled similar patients. If you have randomized control trials that enroll dissimilar patients then you have a problem. Let us say that you have a drug study that enrolls sicker patients in the diet, exercise and telehealth study and will help your patients, then you are running up against some obstacles. There is a potential solution to this in that you can do a network meta-regression which we will briefly touch on the next lecture but that has its own limitations because it can only adjust for differences at the study level rather than the individual level. We will go a little more into that next week. 

In summary, you need to transform data that are reported in the literature into probabilities for use in your decision model. The easiest way to do that is that you have a rate that is reported an odds ratio to outcome is less than ten percent or relative risk or if you have survival data. If you have continuous data with an estimate of variation, it is more difficult to do this, but it is still possible. Very advanced topics include odds ratio when the outcome is greater than ten percent and mean difference of a standardized mean difference in which case you cannot proceed without consulting a TSG statistician. 

Probabilities apply to particular lengths of time and if you want to change, the length of time to which probability applies you can do so through a rate conversion in which you would change that probability to a rate, change that rate back into a probability. 

If you are interested in learning about these topics there are a few journal articles that you can look at, one by Miller and Korman, which is about determining transition probabilities and another one by Naglie and colleagues, which is a Primer on Medical Decision Analysis, both, are published in the *Journal of Medical Decision Making*. 

With that, I will open it up to any questions. 

Paul Barnett:How many people can enter questions in the Q&A box?

Moderator:Everyone, we are keeping the full lines muted, but if you do have a question, please type that into the Q&A box. We do have a few minutes here for questions, we would love to see a few that have come in here. 

Paul Barnett:Katie Sudo asked, I need to, wait a minute, I think she is just giving you kudos Risha.

Moderator:We are getting a lot of compliments but no questions. 

Risha Gidwani:Well thank you. I know that this is a broad topic with a lot of nuances so I listed my email address here if folks are interested in sending any questions that may pop up in the future. 

Paul Barnett:I had a question about the life tables that you showed there. The mortality rate seemed quite high in those older folks. That surprised me a little bit. 

Risha Gidwani:Yeah I am taking the directly from CDC. Yeah it is high it is based on that 2010 census. 

Paul Barnett:SO this is saying that in any given year, those numbers are quite small that are highlighted here right. Rates per one hundred thousand so that is the number that died in a given year right. 

Risha Gidwani:Yes. 

Paul Barnett:So if you go to the next page, where you calculate that, so it is a hundred and forty-three thousand died per hundred thousand, but isn’t that really, so how is that possible, more than the number that are at risk died, I do not quite understand that or am I missing something. 

Risha Gidwani:I think this situation of rates are that the total time period experience by the people that were followed. This not saying that there were a hundred thousand people that were followed, but rather a hundred thousand person years’ experience by these people. So people essentially must have died very soon after the start of that year and therefore, did not accrue an entire person year but it accrued a portion of a person year. 

Paul Barnett:Right, so that makes for these high probabilities, okay. 

Risha Gidwani:So this is a good example of what we were talking about earlier which is what are they really reporting. So even here, unfortunately, they are saying rates per one hundred thousand population but really, it is rates per one hundred thousand person years. Thankfully, we can be confident in the CDC because this is one of the big things that they estimate. But Paul I think you are correct in thinking well how can the numerator exceed the denominator in a probability that would not be able to occur and a rate that could occur because of the fact that not everybody is going to be followed for the same period of time. 

Paul Barnett:Well I do not think we still do not have any other questions. 

Moderator:Paul we have three questions come in here. 

Paul Barnett:Well that is funny because I am not seeing them. 

Moderator:Okay let me try to get through these quick here then.

Paul Barnett:Wait a minute here, they rolled it down. 

Moderator:Do you see them?

Paul Barnett:How does the hazard ratio relate to the relative risk ratio?

Risha Gidwani:So the hazard ratio is a hazard of an event occurring and it is generally used for survival data rather than non-survival data and usually the relative risk is going to be used for non-survival data. I will not pretend to be a survival data expert but one of the things that you do need to keep in mind the survival data is that it can be censored which means people are lost to follow up. Yeah the hazard ratios, the hazard of an event occurring and it is a function of the survival time. 

Paul Barnett:But aren’t they really just different names for the same thing, hazard of an event, risk of an event. 

Risha Gidwani:It is a good question.  I am trying to think back and I am thinking about S of T and F of T. So they are survival time and failure time and I believe if I remember correctly but again I am not an expert on the survival data component of things but I believe the time to an event is also included in the hazard ratio. I would have to defer that question to somebody that is a little bit more familiar with the survival type of data. 

Paul Barnett:So then, somebody had a very specific question. If a person’s lifetime risk of active being and I am not quite sure what that word means, with TB at six percent, and that is reduced, this is kind of scrolling off the page for me, I am trying. That is reduced to ninety-sever percent what are the steps to get the annual risk of activation given forty-two more years of life expectancy. Lifetime risk is six percent and that is reduced by ninety-seven percent, let me just kind of set this up and not necessarily do any calculations. We know their lifetime risk of activating and so they want to know the annual risk of activation. There are two situations there, with and without reduction. If you know the lifetime risk is six percent and they have forty-two more years of life expectancy then you have an answer there don’t you based on what you did before. 

Risha Gidwani:Right, yeah you would just have to think about whether there was a constant risk or whether the risk increase with age. 

Paul Barnett:Right. And then the other question and if the risk is reduced by ninety-seven percent, so what do you multiply that ninety-seven percent times?

Risha Gidwani:So you have your initial risk and that is reduced by ninety-seven percent you would multiply the initial risk by ninety-seven percent and then you would get your....

Paul Barnett:Is the risk of activation the risk of an event, that is it the rate or the probability that you are multiplying by?

Risha Gidwani:Well you cannot multiply the probabilities you would have to multiply the rates. So if you have a rate and a probability you would have to convert that probability to a rate and then do your multiplication. 

Paul Barnett:And then someone asked how do you convert a five year mortality rate to the annual probability of death?

Risha Gidwani:So we would take that mortality rate for five years and convert it not a one year probability where the timeframe in that probability calculation would be one-fifth. 

Paul Barnett:And then someone pointed out that the instantaneous risk is the hazard ratio. 

Risha Gidwani:Great, thank you I am not an expert on the hazard ratio so thank you to folks that are answering those questions. 

Paul Barnett:I think the data and table they are the number of deaths and not rates, rates are not showing in this table. I think that is what you said too, is basically the problem is the denominators not quite articulated. 

Risha Gidwani:Okay. 

Paul Barnett:Here are some more, with the denominator for the seventy-five to seventy-nine rates we are going back to what is on the screen, the five times one hundred thousand  because it is five years. 

Risha Gidwani:These are annual rates that are reported. This is saying this is the number of people in the age group that died per year. 

Paul Barnett:And the five year although the age group is bounded by five years it is not that it is five years’ experience it is one years’ experience. 

Risha Gidwani:Yeah, and you can see that it says its rate is per one hundred thousand population in the specific group. So it is a hundred thousand person years’ experience by all people in that five year age category. 

Paul Barnett:It does seem as though they are adding together male and female, it is a very tiny type here. 

Risha Gidwani:We just looked at the ones for male here, but you could do that for both sexes as well. 

Paul Barnett:Yeah but why would the male and female be added in, maybe, we will think about this offline a little bit. There is someone who says I am the Deputy Chief Actuary, any mortality calculations could be directed to me. Well there is a generous offer from I hate to try to pronounce the name, but it looks like G-u-o, Guo, Li Xia Guo, we will maybe if Dr. Guo would send us some contact information and anybody wants to follow up with them we will be glad to. Heidi can we forward that email address.

Moderator:What I can do is I am going to go to the end here and I am actually just going to type that on the screen here. 

Paul Barnett:Great. Someone just added the comment they would like to see the follow up seminar with the measure of variation in the probabilities, which is a bit harder to calculate. 

Risha Gidwani:It is, and so Dr. Popat and I are doing some work in that field right now actually and as soon as we have results ready for prime time we will share them. 

Paul Barnett:So it obviously must depend on the underlying source of data like how many observations were used to calculate the relative risk or the difference in probability. Very small study then that must be the confidence [indiscernible] is quite large. 

Risha Gidwani:That is correct. Yes, it should be. Then there is also going to be of course this issue that you are going to have not only the estimate of variation around the probability that would occur through the sampling error that you know Paul, but also there is going to be greater estimates of variation because you are deriving a probability. So there is going to be some inaccuracies with the fact that you are deriving the estimates of variation rather than calculating the estimates of variation from your data itself. So two factors will result in variation around your probability estimate. One of them is the same factor you would see in any research study of sampling error and the other one is new to this type of approach and is specific to inaccuracies that come when you are deriving estimates. 

Paul Barnett:And Heidi, we also had someone who asked if they would be able to access the slides and I think at the outset we had a way to download them. 

Moderator:I am sending the link out to everyone right now so everyone that is a live link, feel free to click on that and those will pop right up. We are just past the top of the hour, Risha, Paul do either o you want to make any final remarks before we close things out here. 

Risha Gidwani:I just want to remind folks that next week I will be speaking about how to derive model Inputs when you have data from multiple studies rather than just a single studies we talked about today. 

Moderator:Fantastic, I did just send registration information out on that a few hours ago,  you should all have that in your email so take a moment if you have not registered take a moment and find that and we will get a confirmation out to you. I did just put a feedback form up on the screen for everyone if you could take just a few moments and fill this out we really do read through all of your feedback but please send that in. There is not a submit button on the screen, once you click the button we do have that information aggregated on the back end so please do not worry about a submit button there. Risha, Paul I want to thank both of you very much for Risha taking the time to present and both of you for being here today for the live session. For the audience we thank everyone for joining us today and staying a little bit beyond the top of the hour for the Q&A portion and we do hope to see all of you next week for the next session in this course. 

Paul Barnett:If I could just add Heidi a second, your recommendation or request that people fill this out unfortunately we do not have a good way to interact when there is today almost a hundred people participating in the call, it is really not possible to get feedback except through the ways that we do with questions and answers. This also represents really important feedback to the faculty, the fact that you fill out this form and if you have some specific areas for suggestions for topics or areas for improvements for this talk that is very valuable to us. 

Moderator:Really, it is not ignored, we really do read through all of the feedbacks but please take the time we really do appreciate everything that is sent in. And if that is all we have today, I want to thank everyone for joining us at today’s HSR&D cyberseminar and we hope to see you at a future session. Thank you. 

Paul Barnett:Thanks Heidi. 

Page **1** of **20**