**Transcript of Cyberseminar**

**HERC Cost Effectiveness Analysis Course**

**Modeling Health-Related Quality of Life over Time**

**Presenter:  Vilija Joyce, MS**

**November 14, 2012**

**Modeling Health-Related Quality of Life over Time**

**November 14, 2012**

Vilija Joyce:Okay, good morning and good afternoon to everyone. Today I will be presenting on Modeling Health-Related Quality of Life over Time. Let us see. There we go. In our last session, Patsi Sinnott defined the Incremental Cost Effectiveness Ratio or ICER, which is at the core of cost effectiveness analysis, again the ICER is the difference in cost divided by the difference in outcomes. These outcomes, shown here as the denominator and highlighted, are often expressed as quality adjusted life years or QALYs, which take into account both quantity and quality of life generated by a given intervention. So, essentially, it is the amount of time in a given health state, weighted by quality of life during that health state. As Patsi discussed in the last session, there are several different methods and instruments used to generate these quality of life valuations. These are also known as utilities or preference based health-related quality of life. So for example, the EQ-5D and the health utilities index are instruments that are often used to measure and value health-related quality of life. 

In today’s talk, I will describe how to analyze these weights, these preference based health-related quality of life data, that are used to generate QALYs. These data are often gathered at multiple points in time, sometimes with great variability in the actual timing of the assessments. There can also be problems with missing data, which we will touch upon. 

So that said, I thought it would be helpful to first review the different types of longitudinal studies and models available for your use in analyzing these data. This is the first part of this talk. The second half applies one of these types of models to a real world example. This would be a study of interventions for patients with advanced HIV. In general, there are three common links between all longitudinal studies including those that collect utilities. These are helpful to keep in mind as you are designing your quality of life study. First, there must be multiple waves of data, meaning that the study must collect repeated observations of the same items over a period of time. I wanted to note that the more waves, the better. A study that collects, say, only two waves of data does not provide any shape and we are left wondering whether all of the change occurred immediately after the first assessment or throughout the course of the study. It is also hard to distinguish between true change and measurement error. Also, with more waves of data, you can use more flexible models with less restrictive assumptions. 

The second feature is you will need a sensible metric for time, for example, days for randomization. There is no need for equal spacing. So for example, if you expect rapid change in quality of life during a particular time period, say when a patient is undergoing chemotherapy, you could collect more data during this time. Depending on the type of longitudinal model that you use, you will be able to deal with different time collection schedules and unbalanced data sets. As long as these unbalanced data sets are associated with random attrition, you can run certain types of longitudinal models. We will definitely go over this in the second part of today’s talk. 

Finally, outcomes must change systematically over time and this works in several ways. You must verify that the value of an outcome on any occasion must represent the same amount of the outcome on every occasion. The second bullet point, outcomes should also be equally valid across all measurement occasions. An example of when this might be violated is when a study subject, say after the fourth or fifth time of taking your quality of life survey, he learns to game the survey in order to complete it more quickly. Lastly, try to preserve precision, which means that you should work to minimize errors caused by instrument administration. 

With that said, we can move on to the first type of model I will talk about today, Repeated Measures Models. Repeated Measures Models are often used for studies where the subjects are all experiencing the same condition. For example, they are all undergoing the same treatment therapy. They also work best for studies with a limited number of assessments. So you can think about a study conducted in just three phases: before, during, and after treatment therapy. So as an example, here is a figure that displays data from a breast cancer quality of life study. Each horizontal row corresponds to a study subject. The X axis is weeks post randomization. The top half of the figure is the treatment group. The bottom half is the control group. You can see by the clustering that the data is collected at only three points in time: before treatment around week 0, during treatment, and then four months after therapy. The Repeated Measures Model works well for this type of study. 

The trouble is assessments may not take place when they are supposed to be scheduled and this is particularly true in studies with more frequent assessments. Once you are dealing with more than, say, four assessments, it is increasingly difficult to define the windows of time for each planned assessment. An example comes from the study I will be discussing in detail in the second half of this presentation. This graph shows quality of life data from the Optima Trial, a trial of interventions for patients with advanced HIV. The graph shows 11 out of 368 HIV patients enrolled and it is limited to assessments conducted during the first year of the trial. Each row represents a randomized patient. As you can see by the vertical red lines, during the first year, quality of life data were scheduled to be collected at week 0, 6, and then through week 48. At baseline, all of the assessments fall right on schedule, but starting with week 6, it is more difficult to identify which assessment belongs to which week and it would really depend on how narrow or wide your windows are. If you were to use a Repeated Measures Model, you might end up having to throw out observations which may lead to biased estimates and reduced statistical power. Repeated Measures Model may not be the best choice for this type of quality of life study. 

This is where a different model, the Growth Curve Model, comes into play. They surfaced in the ‘80s and they are known by a variety of names including individual growth models, random coefficient models, mixed models. The actual term “growth curve” comes from early studies that looked at changes in height and weight as a function of age in children. So now, something to think about is why bother using the Growth Curve Model at all? Why not just use an Ordinary Least Squares, an OLS model? 

I am hoping for some feedback here. So take the next few seconds to use your Q&A tool to let me know. Paul, it would be great if you could help me out here, if you could read aloud some of the answers. 

Heidi:So for our audience, just use your Q&A screen on that dashboard on the right hand side of your screen. You can open that using that orange arrow at the upper right hand corner and just type in “Why not use OLS?” into the Q&A portion. 

Paul:So we have some responses here. Observations are not necessarily independently distributed, but Ordinary Least Squares assumes they are. 

Vilija:That is true. 

Paul:Ordinary least squares cannot partition between versus within subject variance. So that is sort of the same point, is not it?

Vilija:That is right.

Paul:Multiple observations nested with individuals. So we are saying those serial correlations for observations on the same individual and it violates the independence assumption among observations, observations correlated over time.

Vilija:Exactly, so it seems like everybody is right on track here. Ordinary Least Squares assumes that observations are independent. If you run an OLS using a longitudinal data set where subjects have these repeated assessments and correlated areas, you run into the problem of biased standard errors and inflated type one errors, meaning that you overstate the significance of your coefficients. So Growth Curve Models are set up to handle this problem. Thank you, Paul, for reading those. 

So that said, let us move on to the definition of the Growth Curve Model. A Growth Curve Model measures change over time in a phenomenon of interest, for example quality of life, at both the individual and aggregate levels. A way to understand this definition is to break the model down into its two submodels. The Level 1 submodel, the Within-Person Change, captures how individuals change over time. Within the Level 1 submodel, you can include time-varying predictors such as days since randomization. The Level 2 submodel, or the Between-Person Change submodel, captures how changes vary across individuals. Here, you can insert time-invariant predictors, for example, the randomization group assigned at baseline. 

Let us take a look at these two submodels individually and then combined. Here I first show the Level 1 submodel, the Within-Person Change. This looks a lot like an OLS, but again, we are set up to handle the problem of correlated error terms. To the left of the equal sign is your outcome of interest Yij for subject i and time j. The outcome is a function of everything to the right of the equal sign. π0i denotes the intercept or subject i’s true value of the outcome at baseline. π1i denotes the slope and this is the subject i’s rate of change and the true outcome. This can be daily, weekly, whatever your rate of change is. Finally, εij denotes random measurement error on the Level 1 residual. So embedded within the Level 1 model are the Level 2 submodels. The Level 1 model that I just discussed is highlighted at the top in yellow. The Level 2 submodels are highlighted below in pink. So for the Level 2 submodels, we assume that subjects are randomized to either an intervention or control group. I have abbreviated that as INTVN here. I will circle that. So consider the intercept, or π0i outlined in red. This can also be written as a function of population intercept or γ00 plus the subject’s deviation from the population intercept or γ01 plus the residual. I should say the subscript 0i. Similarly, subject slope or π0i outlined in blue can also be written as a function of population slope or γ01 plus the subject’s deviation from the population slope or γ11 plus a residual. 

You can collapse these sets of models by substituting the Level 2 submodels into the Level 1 model and after you multiply out and rearrange the terms, you can form an integrated model, which I show here in the green box. The integrated model can guide you in constructing a model using statistical software. Know that the integrated model is made up of two parts denoted by the first and second set of brackets. In the first set of brackets, you will find a Fixed Effects Model of the average outcome. The second set of brackets, which contain the random effects, model the variation among the subjects relative to the average. 

So the advantages of the Growth Curve Model are data can be modeled at the individual level, meaning that the model allows for individual variability in the intercepts and rates of change. Another advantage is that time can be treated continuously and subjects can be observed at different time points. Another bonus is that Growth Curve Models will retain participants with missing data at one or more time points. Finally, I will not go into too much detail here, but Growth Curve Models can be easily generalized to include additional levels of nesting such as patients within different hospitals, which I believe was mentioned in a Q&A session a few minutes ago. 

So now that we have covered these types of longitudinal studies and models, let us apply what we have learned to a real world study in the area of HIV and quality of life, the Optima Study. A little bit of background, since the mid ‘90s, effective antiretroviral or ART therapy has improved survival in HIV infected patients. However, the optimal management strategy for advanced HIV patients at the time of the study was unclear. VACSP trial 512 Optima, which stands for Options in Management with Antiretrovirals was designed to clarify which strategy would be best. Optima was a 2x2 open factorial trial that randomized advanced HIV patients to either a three month therapy interruption or no interruption and to treatment intensification or standard treatments. For the purposes of today’s presentation, I will only be discussing the intensification as the intervention. 

Treatment intensification involved five or more drugs versus standard treatment, which is four or fewer. The study took place in the UK, Canada, and U.S. It began in 2001 and concluded in 2007. The study team enrolled and randomized 368 patients. The primary outcome in the Optima trial was time to first AIDS-defining event or death. The secondary outcome was time to first serious adverse event. The trial found no significant differences for the primary outcome or in the secondary outcomes among the management strategy groups. We also collected other sociodemographic and clinical data including age, sex, and serious adverse events. Quality of life was also an important outcome and was measured at baseline, week 6, 12, 24, and every 12 weeks thereafter. We used several instruments, which I listed here such as the Health Utilities Index, the EQ-5D, the Visual Analog Scale, the MOS HIV, and the Standard Gamble and Time Trade-Off methods to measure quality of life. We ended up collecting over 5,000 quality of life assessments over six and a quarter years of follow-up with the median being closer to three years of follow-up. For today’s exercise, I will focus on the Health Utilities Index. 

As I mentioned earlier, the Health Utilities Index, or HUI3, is often used to generate quality of life weights used in calculating QALYs. The HUI3 contains 17 questions related to quality of life with eight attributes, each with five to six levels, for a total of 972,000 possible health states. Weights are estimated with data from a sample of Canadian adults and utilities are continuous, ranging from death to perfect health, scale of -0.36 to 1. For the purposes of this presentation, our research questions will be “What is the effect of treatment intensification on quality of life in advanced HIV patients?” and 2) “What is the effect of serious adverse events on quality of life?”  

I have given you an overview of the Optima Study and our research questions. It is now time to run some exploratory analyses. As I mentioned earlier, quality of life data is often plagued by missing data. It is a good idea to familiarize yourself with your data’s particular patterns of missingness. So I have another question for you to answer in the Q&A box: why is missing data a problem?

Paul:Well people are typing, here we go now all of a sudden a flurry. Leads to measurement error, missing data can bias results, loss of power, potential bias reduces power. Loss of statistical power—and here is a good one I think, missing may not be random, can be systematic. So I guess that is an effect of the bias. So I think people have got two ideas here. One is that you are losing power and the other is that it is you might be missing at random. You may be missing in some systematic way.

Vilija:That is right, which is exactly what I have on my slide. So thank you to everyone. I appreciate everyone’s responses. Missing data is a problem in that with fewer and fewer observations, you lose power to detect clinically meaningful differences in quality of life. This may affect your QALY calculations and ultimately your incremental cost-effectiveness ratio. The second problem is biased estimates. So for example, a patient who is experiencing poor quality of life because of an adverse event might be less likely to come into clinic and complete his quality of life assessment in your study. So in this case, quality of life may be overestimated. On the flip side, a patient that feels well might skip his visit and so your quality of life estimates might be underestimated. 

Within Optima at baseline we found that 4% of our Health Utilities Index quality of life assessments were missing, which is low, but we were still concerned about missing data in the remaining assessments from week 6 and on. To help us understand our particular patterns of missingness, we created a series of plots including average quality of life scores by time to drop out, average scores by time to death, and average scores by percent missing over time. I will show one of these plots here, this graph shows the main quality of life score by visit week when patients are grouped by time of drop out. The Y axis is the main quality of life score ranging from 0 to 0.8 in this case. The X axis is the visit week ranging from baseline to week 312. Patients were grouped into one of five categories depending on when they were lost to follow-up. The arrow indicates the patient group that was lost to follow-up early on, sometime between baseline and week 48. You can see that at baseline, this patient group had a lower average quality of life compared to the groups that were lost to follow-up later in the trial. 

In addition to the plots, we ran models to try to further understand these patterns and mechanisms of drop out. We wondered if baseline characteristics predicted drop out. So we ran a Proportional Hazards Model to see if certain characteristics were related. We used the PROC PHREG and SAS statement to do so. We also checked to see if “skippers” were different from other patients. By “skippers,” I mean patients with intermittent quality of life assessments. We were interested in finding out whether these “skippers” were more likely to be IV drug users, unemployed, and so forth. So to that end, we ran a series of regressions using PROC PHREG in SAS with percent skips as the independent variable and the baseline characteristic in question as the independent variable. 

Finally, we wondered if certain clinical characteristics such as a recent clinical event were associated with missing quality of life assessments. We used a Generalized Linear Mixed Model, the PROC GLIMMIX procedure in SAS. After running the plot for models in our exploratory analysis of missing data, we did uncover some interesting patterns. For one, quality of life assessments linked to one or more previous serious adverse events were significantly more likely to be missing. However, in the main clinical trial, these events were distributed equally among the randomization groups. So we made the choice not to impute missing data and we left the data set as is. 

In your quality of life study, you might be concerned about missing data that are not ignorable, as with the case in Optima. In that case, imputation, where you substitute some value for a missing data point, may be a useful tool for you to consider as part of your sensitivity analysis. Dr. Diane Fairclough has a very informative chapter on multiple imputation techniques in her 2010 book and I will be providing a reference to her book at the end of the talk. 

Moving on in our exploratory analysis, for the type of analysis we are about to do, it is important to reorganize your data if you have not already done so into a person-period format or a long format. Here is an example of what a person-period data set or long data set looks like. Here I’ve displayed records for just two subjects in our HIV study, subject 003 and 004. Let us take a closer look at patient 003, which is outlined in the red box. Rows 4970 through 4990 indicate that there are multiple records for this patient. The first column is the patient’s ID number. The second column is our outcome variable, quality of life as measured by Health Utilities Index. The third column is the first of our two predictor variables of interest. This is an indicator for whether or not the patient was assigned to the intensification group. As you note that patient 003 was not randomized to receive intensification. The fourth column is our time indicator, which is time in years. This particular patient begins at time 0 through time 5.23, indicating that he has over five years’ worth of follow-up data. Finally, the last column is a time dependent predictor, an indicator of whether or not the patient was experiencing a serious adverse event. Patient 003 did not have any SAEs. However, you will see that patient 004 had several visits that overlapped with ongoing SAEs. 

Having taken a peek at the data set, let us go back to our exploratory analysis and start off with trying to visualize that Level 1 model, the Within Person Change over Time by plotting several subjects’ quality of life scores over time. Here are a set of empirical growth plots for a subset of Optima patients. The Y axis is the HUI3 quality of life score. The X axis is weeks post randomization. Again, I only present one year’s worth of data here. Each plot corresponds to an individual patient, dots are actual data points. I have also added a non-parametric trajectory. This is the blue line in each plot. This is all done in Stata. When I am looking at these graphs, I am observing the elevation, the shape, the tilt. You will notice that most people are changing over time and that the rate of change is different across people. So for example, when you look at the graph, you will see that relative to baseline, some patient scores decline. That is in the red box, here. Some rise. Some remain almost the same. Ideally, you would want to run plots for everyone, but if you have a very large data set, you might want to try a random sample or perhaps you might try subsetting values of important predictors. 

As in the previous slide, let us keep this Growth Curve Model in mind as we run these exploratory analyses. In this case, let us visualize the Level 2 submodels which help to tell us if subjects change in the same way or in different ways. Here are two graphs. On the left is the control group. On the right is the intervention group, the intensification group. The Y axis is a HUI3 quality of life score. The X axis is time in years. For clarity, I have only plotted 19 patients; ten are the control and nine in the treatment intensification group. You could, however, run these graphs for all patients if you wished. Each graph depicts the entire set of smoothed, individual trajectories by whether or not the patient received the intervention. Unlike the previous slide in which we used empirical growth plots, this slide depicts separate parametric models, the thin blue lines fit to each person’s data. So essentially, they are a series of little OLS models for each of the random 19 patients. The graph also shows a bold red line, which is the average trajectory for each subgroup. We cannot draw any definitive conclusions given that we are looking at just a few patients. However, if this were the full data set, we might say that the average observed trajectory of the bold red lines look similar between groups in terms of intercept and slope. If, on the other hand, the intervention group’s average curve had a steeper slope than that of the control group as I have shown here with the green curve, it would indicate that those randomized to receive treatment intensification boost their quality of life more rapidly than those randomized to the control group. 

We are done with our exploratory analysis. We can move on to the final topic, which is a pair of modeling examples using data from the Optima HIV study. Here are our research questions again. Let us tackle the first one. “Are there differences in quality of life over time by intervention?”  We know that our model will be a Growth Curve Model and here it is again. Let us plug in the information from our research question into the model. I have also listed the SAS code for this model, which you can modify to run in Stata, SPSS or [inaudible]. So first off, within SAS, we must use a mixed procedure if we want to estimate a Growth Curve Model. The default estimation method is restricted maximum likelihood. We have also identified QoL as a data set. 

On the left hand side of the model, we have a dependent variable, which in this case is quality of life as measured by the HUI3 instrument. The intercept is presented by γ00. The SAS model includes the intercept by default. Time is incorporated and the intervention which is whether the patient received this treatment intensification is noted as γ01. We also created an interaction variable between time and the intensification indicator. You will note that you can produce interaction terms on the fly in SAS PROC mixed. That is here. Here, we are requesting significance tests for all fixed effects and we specified the degrees of freedom as calculated using the KR or the Kenwood Roger Method. Finally, we add inthe random effects. The model has a random intercept. That is here. It also has a random slope for each individual as is typical for many longitudinal studies. ζi can be interpreted roughly as the average difference between a patient’s response and a mean response. ζ1i  allows for variation in the rate of change over time among subjects. εij stands for the residual errors. Here, we have assumed a simple homogenous structure. Know that we have also specified an unstructured covariance of random effects, that is indicated here, covariant structures are outside the realm of this talk, but again, Dr. Fairclough’s book, chapter 3, specifically covers this topic in detail. 

This slide shows selected output from the submitted SAS code.

Paul:If I might just insert there, so it might just be helpful to explain why what is the significant of un, there, the unstructured. I guess there are two points to be made. One is you have made few assumptions, right, the fewest assumptions possible.

Vilija:That is correct.

Paul:  Then the other is that, that has an implication in terms of your ability to estimate. It may be harder for the model to work. If you had less data, it would be harder to use it unstructured. In other words, you might need to make some assumptions and use more structure.

Vilija:That is true, yes. She actually goes over the different options that you have there. Yes, Paul, that is correct.

Paul:I just wanted to say that, not to bypass it entirely, but to say what you have done here is like the best possible thing, but sometimes you need to make stronger assumptions.

Vilija:That is correct. Okay, so returning to the results, once we have run the SAS model, this is part of the output. We see in the red box the variance covariance estimates for the intercept, slope, and the intercept and slope, as well as the level one residual. We also see some measures of model fit. I will very briefly cover how to compare models using these statistics in a future slide. 

Paul:There is a question here and I am not sure I understand the question, but it is this: what was the reason for using the specific degrees of freedom? Do we specify degrees of freedom here?

Vilija:Yes, the Kenwood Roger, I just returned to that slide. Here we go. We specified here. This was actually suggested to me by the—I had taken a class from the SAS Institute and they suggested that we use this, but I would have to review my notes and let everyone know why exactly that would be. If this person wants to contact me individually, I can give them some more information on it. My contact details will be on the last slide. 

I am just going to fast forward here a little bit. Back to our results, this is some additional output from running the model in SAS. I have also added the model back up at the top of the slide. These are the solutions for the fixed effects estimates. As you can see, none of the parameters of interest are significant, but I would still like to step through each one so you understand what is going on here. 

The intercept represents the average HUI3 quality of life score at baseline for the control group. The time term represents the rate of change in HUI3 score for an average control subject per year. You see that it appears to be decreasing very slightly, but again, this result does not meet the 0.05 level of significance. The intensified term represents the baseline difference in HUI3 scores between the intensification and control groups, again, not significant. The interaction term represents the difference in the rate of change in quality of life per year for the intensification versus the control groups and once again, not significant. Overall, there were no sustained differences in HUI3 quality of life scores between the two groups and over time. These results reflect our earlier plots with the bold red lines, the average trajectories. As you recall, the slopes were nearly flat and they were very similar between the two groups. 

For our second research question, we asked, “What is the effect of ongoing serious adverse events on quality of life?”  The same as with the previous model, we know that our model will be a Growth Curve Model, as listed again, along with the corresponding SAS code. The model is very similar to the previous ones so I will not take too much time going over it, but I did want to note that conceptually you do not need any special strategies to include a time dependent predictor such as serious adverse events in a mixed model. In fact, the only difference here is that I have changed the subscripts to signify the time varying nature of this particular variable. Again, our dependent variable is HUI3. On the right hand side is our intercept and our predictor is time, an indicator of whether the patient had an ongoing serious adverse event at that time and interaction between time and serious adverse events allowing the trajectory slope to vary by whether or not the patient had an ongoing SAE. 

Here the fixed effects estimates pulled from our SAS output, we definitely have something to talk about here. Again, the intercept represents the average quality of life score at baseline for the control group. However, in this case, these are patients without an ongoing SAE. The time term represents the rate of change for the average control patient in HUI3 score per year. Once again, it is decreasing. This time it is significant. The next term reveals there is no baseline difference in HUI3 scores between patients with ongoing SAEs and those without. The interaction term is significant and represents the difference in the rate of change in quality of life per year for those with and without ongoing SAEs. Overall, both those with and without ongoing SAEs have small, but significant within subject changes and HUI3 over time. Also, quality of life is decreasing at different rates with the more rapid decline in patients with these ongoing serious adverse events. That is quantified as -0.009 per year in the control group versus -0.04 per year in patients with ongoing SAEs. 

So just a few final notes, the first is centering. When applying mixed models, it is often helpful to center your predictor variables to improve interpretation of your parameter estimates. As an example, say you have a 2x2 trial where patients receive treatment A, treatment B, or both. This is actually the case in Optima. Rather than represent the values for these indicators as ones and zeros, you could subtract each value by a constant, in this case 0.5. Those assigned to both treatments, A and B, would have it centered, treatment A indicator of 0.5, a centered treatment B indicator of 0.5, and a both indicator of 0.25. 

I also wanted to briefly mention how to determine model fit of these types of mixed models. The first method is the deviance statistic, which compares a log likelihood statistics for two models. You can find this in the SAS output under the fit statistics section. But to compare deviance statistics, your models, at a minimum must be estimated using identical data, which means you must delete any record in the data set that is missing for any variable in either model. They also must be nested within one another. So a reduced model is nested within a full model, if every parameter in the reduced model also appears in the full model. If you want to compare models that are not nested, you can use two ad hoc criteria, the Akaike Information Criterion, AIC, and the Bayesian Information Criterion, BIC. These are also found under the fit statistic section of the SAS output. Similar to the deviance statistic, the AIC and BIC are also based on the log likelihood statistic, but each penalizes the log likelihood according to certain criteria. You want to be sure that the pair of models you are intending to compare are fit to the identical set of data. The model with the smaller information criterion fits better and Raftery in his 1995 paper suggested that the difference in BIC criterion of ten and over to be very strong. 

To wrap up today, we have learned about how to analyze an important component of quality adjusted life years, the utility weight. We have covered a brief intro to a model that works well in analyzing these type of data, Growth Curve Model. We have also applied the model to utility data from Optima which is a VACSP sponsored HIV clinical trial. To that end, we have discussed an overview of the Optima trial. We ran some exploratory analyses and I have presented two research questions and built Growth Curve Models based upon these questions. Finally, we have interpreted SAS output. 

So here are some suggested references that I have mentioned throughout the presentation. The Singer, Willett book, this is the second bullet, also includes a companion. They also reference a companion website. It is hosted at UCLA with data sets and code in several programming languages. The UCLA site in general is a great place to find other statistical code such as reshaping your data from a wide to long format, which you will need to do for these mixed models.

So thanks very much for attending today’s seminar. The next talk will take place on November 28. Patsi Sinnott will be covering Budget Impact Analysis. We can open to questions.

Paul:Before we do all that, we have got some questions. Before we convince people that it is all over. So someone wrote could you please explain the serious adverse event variable in more detail? If a patient has one, are they listed as, and then the number one as the indicator takes a value of true, in the data set in all observations after that event or do they go back to a zero?

Vilija:So if I recall correctly, it can revert back to a zero. It is just an indicator if they had at least one ongoing serious adverse event at that particular point in time. If the SAE resolved, it would then revert back to a zero. 

Paul:There are a couple more questions here, two questions. First, how do you analyze if the observations were not the repeated measures? Let me see if I understand this, e.g., the data from population surveys randomly drawn at each time point, should we use the Growth Curve Model when we only have data from two time points?

Vilija:I think in that case that rather than using the Growth Curve Model, you would use the Repeated Measures Model. The reference I provide in the previous slide—

Paul:Excuse me, so you are saying if there are only two time points, you would use a Repeated Measure and not a Growth Curve Model?

Vilija:That is correct. I think that would be more suitable.

Paul:Their question was worded a little bit differently, but yes, that is right. So you would use Repeated Measures rather than Growth Curve when you have only two time points?

Vilija:Yes, that is correct. Dr. Fairclough, this is the first bullet here that I present, her 2010 book, chapter three goes into detail about these Repeated Measures Models. There are different versions of it. Reference Group Models, Reference Cell Models, Center Point Models. She provides the appropriate SAS code to go along with these. I find that to be very helpful.

Paul:I guess one interesting thing is that data in theory, we would want to gather like I think in Optima was it every ten weeks we were getting a follow-up?

Vilija:Every 12 weeks.

Paul:Every 12 weeks, yeah, and of course it never happened on any sort of schedule at all. [Laughs]

Vilija:That is right. [Laughs]

Paul:Because it is when the patients are available, etc. 

Vilija:That is right. With the Repeated Measures Model, we would have really missed out. We would have had to drop a lot of these patients, but the Growth Curve Model allows you to use subjects measured at different time points.

Paul:Then it says, with the previous example, I think they mean interaction, is not significant, would you drop the interaction term and then rerun and just use the main effects model instead?

Vilija:I am going to return back to that slide. Let us see here. I am assuming that the person is stating that we would be dropping this term. 

Paul:Right.

Vilija:I would assume so.

Paul:I think that is always a safe idea when you are estimating models is if the main effects are significant, then go ahead and estimate an interaction term. In this case, once we put the interaction term, then the main effects were no longer significant, but it is not terrible either. I mean a T of 1.5 is getting up there, so it is something. It is part of what we really wanted to know about was whether people were getting worse as the study went along.

Vilija:That is correct. We do see that with this first estimate here. 

Paul:Right. Then someone asked if you had missing data on both the dependent and the independent variables, how would go about imputing for the independent variable?

Vilija:For both the dependent and the independent variables. I believe it is the same way. I believe multiple imputation would apply. Paul, correct me if I am wrong.

Paul:Yeah, you always want to do multiple imputation for the following reason: you do not really know what the value is when you have imputed. So there is some sort of error in imputation and by doing multiple imputation, you are accounting for that contribution to the variance. It turns out you only have to impute like a half dozen times to account for that. Robbins is the classic statistician who wrote kind of the early work on this that really explains that well. There are lots of ways to go about multiple imputation and, in fact, I think that is a whole course, let alone a lecture, let alone answer to a question. I guess in the software, it has new and better ways to do this as time goes on. I understand that the newest version of SAS supports some better ways of doing multiple imputation. I do not know. I think that is probably getting beyond the scope of the class. Basically, you are taking all of the information you do have to synthesize the information you do not have. There are different statistical techniques for doing it. The more modern methods make fewer assumptions about how to do it. Do not parameterize it, etc. That is probably as far as we ought to go on that. 

Could have we answered both research questions by using one model is the next question.

Vilija:I am not sure. That would be very difficult to interpret. 

Paul:I think the answer is no. So there were two questions. One is what is the effect of treatment group assignment on health-related quality of life? The other question is what is the effect of an adverse event on quality of life?

Vilija:Yes.

Paul:So we would not want to know the effective treatment group assignment while controlling for adverse events because that would be throwing out the information that we had. Say the treatment group assignment led to more adverse events. Well, we would not want a control for the number of adverse events. So really they are two separate questions, two separate models are required to answer. I think that is the answer. Does that make sense?

Vilija:Yes, thanks, Paul.

Paul:Then in general, how many assessments would you need per person to model changes in health over time? I am working with a data set in which only two time points are available.

Vilija:That is right, so again you will have to work with the Repeated Measures Model. As I mentioned in one of the earlier slides, that it is best at the beginning of your study. It is difficult to measure change with just two time points. You do not know if all of the change is occurring immediately after the first one or somewhere in the middle. But now that you have the two time points, I think a Repeated Measures Model is your best bet.

Paul:Well, I think the other thing to think about here is over what period of time? If those two time points are 12 weeks apart, well, that would be no better or worse than what we did in Optima.

Vilija:That is true.

Paul:Where we followed people for an average of what, five years, something like that.

Vilija:Yes, the maximum was six and on average about three years, that is correct.

Paul:Every 12 weeks, so two points would be fine in a certain sense. If the time frame were ten years, maybe two points would probably not capture them. I think it is a function of how long the follow-up is. Then the other interesting thing is sometimes people use other measures to capture. For instance, like what you are doing now, Vilija, is doing some work about well, what if we only have some sort of disease specific measure that talks about you can only ask a few questions that had particular to do with their disease and then you can estimate what the quality of life based on their answers to those questions. Or even if you only asked them one question like, “What is the state of your health: excellent, good, fair, poor?”  That might explain an awful lot if you had that additional information.

Vilija:That is correct, yes. If you had some sort of other measure in between those two time periods and then you mapped to this preference based quality of life, that would be very helpful and may allow you to use a Growth Curve Model or just in general give you more information about what is going on in terms of quality of life in your study.

Paul:It says the magnitude of the change in quality weight seems very small. Could you please comment on policy significance?

 Vilija:Well, in this case, again in the clinical trial, there was no difference between the groups. So in some sense, we knew that it was very possible that we would not see a quality of life difference either. In terms of policy, I am not sure. Paul, we did not end up running the cost effectiveness analysis for one thing. 

Paul:On this particular, slide 37, we are saying that being randomized to ART intensification yields 0.03 QALYs, right, but it is not statistically significant. So that is no significance at all. There was no benefit from the strategy. What about the other one?

Vilija:The serious adverse events, there was a decrement in SAEs, in quality of life.

Paul:But it is basically you lose about 0.03 QALYs for each year you go into the study. It might be more interpretable. By five years, at the end of five years, if somebody has a serious adverse event, they are getting a 0.15 hit—isn’t that what this is saying—which is a pretty serious decrement later on.

Vilija:Yes, this is yearly, that is correct. 

Paul:Because we would say the time year, times SAE indicator, so year five times SAE, that variable is going to have a value of five. We multiply that times -0.03. That is actually pretty big. Just in terms of clinical significance, if you will, I have heard Doug Owens say that a 0.03 is about the minimum that anyone would care about. You have to have at least that much before anybody is going to pay too much attention to you. 

On the other hand, if you look at the cost effectiveness literature, you often see cost effectiveness results coming out of very small incremental differences and very small cost. You have to put it in the context of what does it cost to get you this. So if you can buy 0.02 QALYs for a dollar, well that is a great bargain. The fact that it is a very small amount, if you have measured it with precision it is still worth knowing about if you can get it at a low cost. Is it important or not depends on what it cost you to achieve it.

In these models, patients can have different slopes, but how to model changes in individual’s slopes?

Vilija:I believe that is all within the model. This is part of those submodels I had showed earlier. 

Paul:In essence, you are controlling for that. You are allowing everybody to have a different slope, but what you really care about are the parameters that apply to the whole study, that is what is your treatment group assignment or did the person have an adverse event. You control for the fact that a person could have a different slope. 

Vilija:Yes, I have. The model definitely allows for variability in the intercepts and individual intercepts and rates of change, intercepts and slopes. 

Paul: I guess the answer is how to model for changes in individual slopes, that is the specification that includes time as a random effect. You have got your program there. 

Vilija:Let me move back a slide here. Right in here, specifies intercept and time as random effects, yes.

Paul:So that random statement is what allows each individual to be modeled with a different slope?

Vilija:That is correct.

Paul:That is the answer to the question, I think. That is how you do it. I am not sure why controlling for treatment will not answer the side effects, in fact could not the interaction between the two be fitted? I think this is the follow-up to the question about could you use one model to estimate everything? I think it is kind of a conceptual question. If we have treatment, we want to know what the effect of treatment group assignment is. That, we would have one model where we say is treatment group assignment associated with better quality of life or worse quality of life? Then the thing is the treatment group assignment, say that being randomized to intensification of antiretroviral treatment avoids serious adverse events, well that is a good thing and you would expect that to increase their quality of life had that happened. Now, if we run a model again where we say what is the effect of treatment group assignment? We control for how many adverse events we have. We are basically throwing out the information that we have on what is the benefit of the treatment group assignment. That is why we would want to include both in there. I hope that is clear to the questioner. Did I explain it well, Vilija?

Vilija:Yes, definitely.

Paul:Of course, I sign her performance appraisal so she may not be totally objective here.

Vilija:I wonder if you might include intensification and an SAE interaction. Again, I would have difficulty interpreting that type of model. 

Paul:Then you do not know the effects of intensification because randomized intensification. Presumably, if it had some benefit, one of the potential benefits is that it will avoid adverse events.

Vilija:That is correct.

Paul:How can you get an effects size odds ratio for the fixed effect parameters? These models are basically linear models, right?

Vilija:Yes.

Paul:They can just be interpreted as intercept is 0.6, right that you have up here. An odds ratio is something that you would do more with a kind of a nonlinear or a survival or maybe a logistic regression, something like that. That would be like an odds rates. I do not think it makes sense in a linear model to find an odds ratio. The danger of giving these talks is there is always someone who is attending the talk who knows a lot more than you do. If that person wants to raise their hand now or send us an email and correct us, that would be great, but I do not think we can apply an odds ratio to a linear case. I think that is the last question.

Vilija:Last question, wonderful. 

Paul:So you have that last slide which tells about the next talk.

Vilija:Yes, I will go ahead and move forward to that. The next talk, again, Patsi Sinnott, Budget Impact Analysis is in two weeks’ time.

Paul:I am just going to give a little promotional idea about it. We have been talking a lot about cost effectiveness analysis. That has to do with are you willing to purchase the QALY by adopting this intervention. Budget impact analysis is yeah, but if I do agree that the QALYs are worth purchasing at this rate, how many of them am I obligated to buy. How is this going to affect my health plan? Is it ten people that I am going to have to provide intervention to or is it 100,000? That matters a lot to the decision maker. Those two interventions might have the same cost effectiveness ratio, but budget impacts might be quite different. Now we are beginning to think budget impact analysis is an essential component of an economic evaluation. We will see people in two weeks. Heidi, someone is interested in the copy of the slides. I think they can find that on the first slide. 

Heidi:  That is actually in my opening slide. Let me get those back open here and I will display that. For those of you who are interested in the Budget Impact Analysis session, I know a lot of you already are registered, but we will be sending out additional registration information in about a week so just keep an eye in your email. We will get that out everyone. As soon as this slide gets opened, I will, right here is the link. It is a tiny URL, so you would need to type that into your browser window and that will get you right to where we have today’s handouts available. I want to thank Vilija and Paul, both of you for preparing and presenting for today’s session. We really appreciate the time you put into it. Thank you to our audience for joining us today and we hope to see you at a future HSR&D cyber seminar. Thank you. 

[End of audio]  

Page 16 of 16