**Modeling Utilities over Time**

Vilija R. Joyce, MS

1/11/2012

Transcript of Captioning

>>Well, my name Vilija Joyce I'm a research associate here at HERC, the Health Economic Resource Center in Menlo Park, good morning and good afternoon to everyone. In our last course session, Patsy Sinnott, one of PIs here the HERC introduced us to cost effective analysis, and use of quality of adjusted life years and methods to assess a very specific type of health related quality of life measure, preference-based quality of life or utilities. Today we'll talk about how to analyze data that you’ve gathered with multiple observations over time. In the specific case we'll be looking at health related quality of life data, which is often used for cost effectiveness analysis. However, many of these techniques can be applied to any type of longitudinal data. 

>>Today's course is broken up into two parts. The first is an introduction to the different types of longitudinal studies and models, the second is applying one of these types of models to a real world example, which would be a study of interventions for patients with advanced HIV. In general there are three common features of all longitudinal studies. First off there must be multiple waves of data, meaning that the study must collect repeated observations of the same items whether that's quality of life scores, BMI, some other type of outcome, over a period of time. And also the more waves, the better. A study that collects only two waves, doesn’t present any shape and we're left wondering if, say, all the change occurred immediately after the first assessment or was it throughout the course of the study? It's also hard to distinguish between true change and measurement error. Plus with more waves of data, you can use more flexible models. It's less restrictive assumptions.

>>The second feature is that you need a sensible metric for time. So for example, you can use days for randomization, months from surgery, even age. And also there's no need for equal spacing. If you expect rapid change in quality of life during a particular time period, such as when a patient's undergoing chemo therapy, you could collect more data during this time, and depending on the type of longitudinal model that you use, you may be able to deal with different time collection schedules and unbalanced data sets where say one patient has five quality of life assessments and the other has only two. As long as the unbalanced data sets are associated with random attrition you can run certain type of longitudinal models and we'll go over this in more detail.

>>Finally outcomes that change systematically over time. This is in a different way. First you must verify that the value of the outcome on any occasion represents the same amount of the outcome on every occasion. Outcomes should be equally valid across all measurement occasions. An example of when this might be violated is when a study subject who, say, after the fifth time taking a quality of life survey learns to gain the survey in order to complete the study more quickly.  And last, you should try to preserve precision, which means you should work to minimize errors caused by instrument administration. 

>>With that said, we can move onto the first type of model I will talk about today, repeated measures models. These types of models are often used for studies where the subjects are all experiencing the same condition. For example, they're all undergoing the same treatment therapy. They also request for studies with a limited number of assessments, so think about study conducted in just three phases, before, during and after treatment therapy. Here's a figure that displays data from a breast cancer quality of life study. Each horizontal row corresponds to a study subject. The X axis is weeks post randomization. The top half of the figure is the treatment group. The bottom half is the control group. And you can see by the clustering that data is collected only at three points in time before treatment, which is around week 0, during treatment, which is around week 13 and then four months after therapy. And the repeated measures model work well for this type of study. The trouble is assessments may not take place when they're supposed to be scheduled, and this is particularly true in studies with more frequent assessments. Once you're dealing with more than say, four assessments, it becomes increasingly difficult to define the windows of time for each planned assessment. The perfect example comes from a study I'll be discussing in detail in a second half of this presentation. This graph shows quality of life data from the Optima trial, a trial of interventions with patients with advanced HIV. The graph shows 11 out of 368 HIV patients enrolled, and is limited to assessments conducted during the first year of the trial. Each row represents a randomized patient. As you can see by the vertical red lines during the first year, quality of life data were scheduled to be collected at baseline, week 6, 12, 24, 36 and 48. And that baseline all of the assessments fall right on schedule. But starting with week six, it becomes more difficult to identify which assessment belongs to which week, and it will really depend on how wide or narrow your windows are. When you use a repeated measures model you might end up having to fill out observations which may lead to biased estimates and reduced statistical power. Now imagine the problem if I were to show you, not just these 11 patients, but rather all 368 patients over the entire course of the trial, which for some is over six-year's worth of data. A repeated measures model is not the best type of model for this study. 

>>This is where the second type of model that I'll discuss today comes into play. The growth curve model. Growth curve models which surfaced in 1980s are known by a variety of names including individual growth curve models, random co-efficient models, mixed models, etc. The actual term growth term comes from early studies that looked at changes in height and weight as a function of age in children.

>>So now something to think about. Why bother using a growth curve model at all? Why not use an ordinary least squares model? The answer is: OLS regression assumes that observations are independent. If you run an OLS using a longitudinal data set where subjects have repeated assessments and correlated errors, you run into the problem of biased standard errors and inflated type one errors, meaning that you overstate the significance of your coefficients. Growth curve models are set up to handle the problem of correlated error terms. With that said let move onto a definition of a growth curve model. A growth curve model measures change over time in a phenomenon of interest, for example, quality of life at both the individual and aggregate levels. And a good way to understand this definition is to break the model down into its two sub models. The level one sub model within person change captures how individuals change over time. Within the level one sub model you can include time varying predictors such as days and randomization. The level two sub model or the between-person change sub model captures how changes vary cross individuals and here is where you can invert time-invariant predictors, such as the randomization group. So let's take a look at these two sub models individually and then combined. So here is the level one sub model within-person change. This looks a lot like OLS but here again we're set up to handle the problem of correlated error terms. So the left of the equal sign is your outcome of interest. Y for subject I and time J, the outcome is a function of everything to the right of the equal sign. π*0i*denotes the intercept or subject I true valve of the outcome at baseline. π*1i* denotes the slope. This is subject I's rate of change in true outcome, and this can be a daily, weekly, annual, or other rate of change. And note that the 0 and the 1 subscripts for the two π variables help us to distinguish between the two variables but they don't represent anything more than that. 

>>Finally ε*ij* denotes random measurement error or the level one residual. 

>>Embedded within the level one model are the level two sub models. Here the level one model that we just discussed is highlighted at the top in yellow and the level two sub models are highlighted below in pink. Looking at the level two sub models and assuming that subjects were randomized to either an intervention or control group and here I have abbreviated that as I-N-T-V-N, intervention, consider the intercept or π*0i*outlined in red. This can also be written as the function of a population intercept, or γ00   Plus the subject's deviation from the population intercept or γ01 plus the residual, which is ζ0i. Similarly a subject slope or π1i outlined in blue can also be written as a function of population slope. Or γ1, plus the subject deviation from the population slope, or γ11 plus the residual. 

>>You can collapse these sets of modeling by substituting the level two submodels into the level one model and after multiplying out and rearranging the terms you can form an integrated model shown here at the bottom in the green box. This integrated model can guide you using a model in statistical software. Notice the integrated model is made up of two parts denoted by the first and second set of brackets. In the first set of brackets you'll find the fixed effects, fixed effects model the average outcome. The second set of brackets, which contain the random effects, model variation among the subjects relative to the average. 

>>The advantages of the growth curve models are: They can be modeled at the individual level, meaning the model allows for individual variability in the intersects and rates of change. Another advantage is that time can be treated continuously and subjects can be observed at different time points. Another plus is that growth curve models will retain participants with missing data at one or more time points. We'll talk more about missing date in a future slide. And finally although I won't go into too much detail here, growth curve model can be easily generalized to include additional levels of nesting such as patient within different clinics or hospitals. 

>>So now that we've covered the types of studies and models. Let's apply what we've learned to a real world longitudinal data set in the area of HIV and quality of life. The OPTIMA study. So first some background. Since the mid '90s effective antiretroviral drugs have improved survival in HIV infected patients. However the optimal management strategy for advanced patients is unclear. A CSP trial 512 OPTIMA, which stands for Options in Management with Antiretrovirals was designed to clarify which strategy would be best. OPTIMA was a two by two open factorial trial that randomized HIV patient to either a three-month treatment interruption or no interruption, and to treatment intensification or standard treatment. And for the purposes of this presentation, I'll only be discussing the treatment intensification as the intervention. Treatment intensification involves five or more antiretroviral drugs versus four or fewer antiretrovirals. The study took place in the UK, Canada and US. It began in 2001 with follow-up concluding in December 2007. And we ended up randomizing 368 patients. 

[Off-topic discussion about audio quality]

>>So moving on the primary outcome in the OPTIMA trial was time to first AIDS defining event or death. The secondary outcome was time to first serious adverse event. The trial found no significant differences for the primary outcome or the secondary outcome among the groups. 

>>We also collected other associated demographic and clinical data including age, sex, serious adverse events and so on. Quality of life, of course, was an important outcome and we measured that at baseline, week 6, 12, 24, and every 12 weeks thereafter. 

>>We used several instruments, which I've listed here. We ended up collecting over 5,000 quality of life assessments over a maximum of six and a quarter years follow-up with the median being closer to three years of follow-up. And for today's modeling exercise I'll focus on one of the quality of life instruments as an outcome, that's the Health Utility Index Mark 3 or HUI3. 

>>HUI3 is a special quality of life instrument called a preference based or utility based instrument. And unlike descriptive instrument such as the SS36 which measure symptom severity, a utility base instrument can assess the strength of a patients preference for an outcome. Utilities are essential inputs into cost effectiveness analysis. The HUI3 contain 17 questions related to quality of life, with eight attributes. Each with 5 or 6 levels for a total of 972,000 possible health states. Weights are estimated with data from a sample of Canadian adults and utilities are continuous ranging from death to perfect health, a negative 0.36 to 1. 

>>For the purpose of this presentation, our research questions will be, what is the effect of intensification on quality of life in advanced HIV patients? And also what is the effect of serious adverse events on quality of life? 

>>So now that I've given you an overview of the OPTIMA study and our research questions it’s time to run some exploratory analysis. 

>>It's a good idea to familiarize yourself with your data, particular patterns of missing data and mechanisms of missing data. Missing data is a problem in that with fewer and fewer observations that you lose power to detect meaningful differences in quality of life. The second problem is biased estimates. So for example, a patient experiencing poorer quality of life because of an adverse event might be less likely to come into clinic to complete his quality of life assessment. In this case, quality of life is over estimated. On the other hand a patient that feels well might also skip his visit and thus your quality of life estimates may be underestimated. Within the OPTIMA trial at baseline we found that 4% of HUI3 quality of life assessments were missing. This is a lower proportion of missing assessments but we were still concerned about missing data from weeks six and on, and to that end we constructed a series of plots to describe the quality of life data in terms of missingness. Some the plots we ran were, average quality of life scores by time of drop out, average scores by time to death, and average scores by % missing over time. So I show one of the plots here. This graph depicts the mean HUI3 quality of life score by visit week when patients are grouped by when they were lost to follow up. The Y axis is the mean HUI3 quality of life score. On the X axis the visit week ranging from baseline to week 312 and we group patient in one of five categories depending when they were lost to follow up. In red you'll see that a patient group that we lost to follow up early on. This is sometime between baseline and week 48. You can see that at baseline, this patient group has a lower average quality of life compared to the groups that were lost to follow up later in the trial. In addition to the plots, we ran models to try to further understand the patterns and mechanisms of drop out. We wondered if baseline characteristics predicted drop out, so we ran a proportional hazards model to see if certain characteristics were related to drop out. We use the PROCPHREG statement in SASS to do so. 

>>We also checked to see if skippers were different from other patients, and by skippers, I mean patients with intermittent quality of life assessments. We were interested in finding out whether these skippers were more likely to say, unemployed, drug users, so forth, and to that end we ran a series of regressions using PROC REG in SASS with% skipped as the variable and the baseline characteristic in question as the independent variable. 

>>Finally we wondered if certain clinical characteristics such as a recent clinical event were associated with missing quality of life assessments. We use the generalized linear mixed model the PROC GLIMMIX procedure in SASS and this is a variant of the growth curve model. 

>>After running the plots and the models in our exploratory analysis of missing data we uncovered some interesting results. First quality of life assessment linked to one or more previous serious adverse events were significantly more likely to be missing. However, in the main clinical trial, these events were distributed equally among the randomization groups, and as a result we chose not to impute missing data. We left the data set as is. In your quality of life study, you may be concerned about missing data that are not ignorable. In that case, imputation, which is where you substitute some value for a missing data point may be a useful tool for you to consider. Dr. Diane Fairclough has a very informative chapter on multiple imputation techniques in her 2010 book, and I will provide a reference to her book at the end of the presentation. So moving on in our exploratory analysis,  for the type of analysis we're about to it's important to re-organize your data into a long format, otherwise known as a person period format. Here's an example of what a long format data set looks like. [Displays records for two subjects. Subjects 003 and 4.] Let's take a closer look at patient three, which I just outlined with the red box. Rows 4970 through 4990 indicate that there are multiple records for this patient. The first column is the patient's ID number. The second column is their outcome variable, quality of life as measured by the HUI3. The third column is the first of our two predictor variables of interest during today's talk. This is the indicator of whether or not the patient was assigned to the intensification group. You will see that patient three was not randomized to receive intensification while patient four was. The fourth column is our time indicator, which is time in years. And this particular patient begins with time of 0, which is baseline. Through time 5.23 or so. Indicating that he was over five-year's worth of follow-up data. And finally the last column is another predictor. This time at time dependent one, an indicator of whether or not the patient was experiencing a serious adverse event during a given visit. And patient three didn't have any SAE's. However, you'll see that patient four had several visits that overlapped with ongoing SAEs. 

>>So now that we've taken a sneak peak at the data set let's get back to the exploratory analysis. And we'll start off by trying to visualize the level one model, the within person change over time by plotting several subjects quality of life scores over time. Here are, some empirical growth plots for a subset of OPTIMA patients. The Y axis is the HUI3 quality of life scores; the X axis is the week post randomization. I only present one year's worth of data here. Each plot corresponds to an individual patient. And the dots are actual data points and I've also added a non-pair metric trajectory. That's the blue line in each plot. These plots were done in STADA [?]. When I'm looking at these graphs, what I'm looking for is changes in elevation, shape and tilt. You'll notice that most people change over time as the rate of change is different across people. I'm going to highlight a few of these differences. When looking at the graph you'll see that relative to baseline, some patients scores decline. Some rise, and some remain the same. Ideally, you would want to run plots for everyone. If your data set is very large you might try a random sample and perhaps subset by the values of important predictors. 

>>So as in the previous slide, let's keep the growth curve model in mind as we run these exploratory analyses. In this case let's visualize the level two sub models which help to tell us if subjects change the same way or a different ways. So here are two graphs. One for the control group on the left, and one for the intervention group on the right. The Y axis again is HUI3 quality of life score and the X axis is time, this time time in years. And for clarity I've decided to plot data for only 19 patients. Ten in the control and nine in the treatment intensification group. Each graph depicts the entire set of smoothed individual trajectories by whether or not the patient received the intervention. And unlike the previous slide in which we used empirical growth plots, this slide depicts separate parametric fit to each person's data, essentially what that means is that there are a series of little OLS models for each of the random 19 patients. The graphs also show a bold red line which is the average trajectory for each sub group. And now we can't draw any definitive conclusions given that we're looking at just a few patients. If this were the full data set, we might say that the average observed trajectories, the bold red lines look similar between groups in terms of interceptive slope. If on the other hand the intervention groups average curve had a steeper slope As you can see with this green line, that I've presented, it would indicate that those randomized to receive treatment intensification boost their quality of life more rapidly than those randomized to the control group. So we're done with the exploratory analysis we can move onto the final topic. It is a pair of modeling examples using data from the OPTIMA HIV study. And as a refresher, here are our research questions again. Let's tackle the first question. Are there difference of quality of life over time by intervention? We know that our model will be a growth curve model and here it is again. So let's go ahead and plug in the information from our research question into the model. I've also listed the SASS code for this model, but of course, you can run this model in STATA, SPSF or R as well. First off within SASS you invoke the mixed procedure if you want to estimate a growth curve model. The default estimation method is restricted maximum likelihood. We also QoL as a dataset that we'll be using today. 

>>On the left-hand side of the model we have the dependent variable, the Health Utilities Index. Data intercept is represented by γ00 and the SASS model includes the intercept by default. Time is incorporated as γ10. The intervention which is whether the patient received treatment intensification is noted as γ01. We also created in interaction variable between time and the intensification indicator and note that you can produce interaction terms on the fly in the mixed procedure in SAS. 

>>Here we're requesting significance tests for all fixed effects. And specify the degrees of freedom as calculated using the KR or the Ken ward-Roger method. And finally we add in the random effects. This model has a random intercept, random slope for each individual which is typical for many longitudinal studies. 0i can be interpreted roughly as the average difference between a patient’s response and the mean response. 1i allows for variation in the rate of change over time among subjects, and ij stands for residual errors. We’ve assumed uncorrelated homoskedastic residual errors. And we’ve also specified an unstructured covariant on random effects. And that's the type equals UN statement in the SASS code. Covariant structures are outside the realm of this talk, but again, Dr. Fairclough's book covers this topic in detail. 

>>[Paul Barnett] The idea here is that just by using the unstructured is that you’re making as few assumptions as possible about how the errors are correlated. 

>>[Vilija R. Joyce] That's correct. Thank you for clarifying. 

>>This slide shows selected output from the submitted SASS code, we can see the variants. Co-variants estimate for the intercept slope and intercept and slope, as well as the level one residual. We can also see some measures of model fits and I'll briefly cover how to compare models using these deviant statistics in a future slide. Here's some additional output from running the model in SASS I've also added the model back up to the top of the slide. These are the fixed effects estimates. As you can see none of the parameters of interest are significant, but let's step through each one to understand what's going on. The intercept represents the average HUI3 quality life score at baseline for the control group. The time term represents the rate of change in HUI3 score per year, you can see that it appears to be decreasing slightly, but that doesn't quite meet the 0.5 level of significance. The intensify term represents the baseline difference in HUI3 scores between the intensification and control groups. And the interaction term represents the difference in the rate of change in HUI3 per year for the intensification versus the control groups. Again not significant. Overall -- 

>>[Paul Barnett] I was going to say, that the time years parameter, the one that is gamma one zero. That would really, if I'm understanding right, that's the trend for the control group right, and the trend for the intensification group, that is when I said the control group... people that didn't get intense antivirals, people that got standard antiretrovirals, then we would add time years plus the coefficient for time years times intensify to find the trend for the intensification group; is that right? 

>>[Vilija R. Joyce] Yes. That's right and actually, when I present the next research question, the model and the results, I'll go over that in detail, but that's exactly correct. 

>>So overall there were no sustained differences in HUI3 quality of life scores between the two groups this we've looked at here. And these results reflect our earlier thoughts where we displayed the smooth individual trajectories and those bold red line the average trajectories and as you recall, the slopes were nearly flat, very similar between the intervention and control groups. 

>>So moving on, our second research question, will hopefully prove to be more compelling. We asked what the effect, what is the effect of ongoing serious adverse events on health-related quality of life? And the same as the previous model we know that our model will be a growth curve model and I've listed it here along with the corresponding SASS code. The model is very similar to the previously one I wouldn't take up too much time going over it. I just want to note that conceptually you don't need any special strategies to include a time dependent predictor such as SAE's in the mixed model. The only difference is that I've changed the subscripts to signify the time varying nature of this particular variable. Again, our dependent variable is HUI3, on the right hand side is our intercept, and our predictors, time and indicator of whether or not the patient had an ongoing serious adverse event at that particular point in time, and an interaction terms between time and SAE. 

>>So here are the fixed effects estimates. Pulled from our SASS output and we definitely have something to talk about here. The intercept represents the average HUI3 quality of life score at baseline for the control group. 0.61, which in this case are patients without an ongoing SAE. Again the time term represents the rate of change in HUI3 score per year and it's decreasing and unlike the previous model, we see here that it's significant. The next term reveals that there's no baseline difference in HUI3 scores between patients with ongoing SAE's and those without. And the interaction term is significant. So overall both those with and without ongoing SAE's have small but significant within-subject changes with HUI3 over time. Quality of life is decreasing at a different rate, though, with more rapid decline observed in patient with ongoing SAE's, and that's quantified as a -0.009 per year in the control group, the patient without ongoing SAE's versus -0.04 per year in patients with ongoing SAE's, and the -0.04 per year as Paul alluded to is just some of the estimates for the time and the interaction.

>>So I wanted to touch upon a few final notes. The first centering. When applying mixed model it's often helpful to center your predictor variable to improve the interpretation of your parameter estimates. So as an example, this is actually pulled from the OPTIMA trial, a two by two trial. Patients received either treatment A, treatment B, or both. So rather than represent the value for these indicators as ones and zeros, you would subtract each value by a constant in this case we use the 0.5. This time to both treatments A and B would have a center treatment A indicator of 0.5, a center treatment B indicator value of 0.5, and a value for the both indicator of 0.25. 

>>[Paul Barnett] One way to think about, I think... express this might be think about expressing it as a deviation from the mean, right? that's what you're really ... Adding a constant to the variables so that they're all expressed, so that at the mean value for the independent variable becomes 0, so then the intercept has a... if the interpretation which is its mean value dependent variable with every independent variable at its mean, so I think that's, that's part of the conveniences of this centering or expressing your independent variables a deviation from the mean, including the indicators in this case, [INDISCERNIBLE] but continuous variable obviously [INDISCERNIBLE]

>>[Vilija R. Joyce] As well. Yes. Thank you. Thank you for explaining that. Also I wanted to briefly mention how to determine model fit for these types of mixed models. So the first method is, a deviance statistic, which compares the log likelihood statistics for two models if you can find this in the SASSPROC mixed output under the fit statistics section. However in order to compare deviant statistics your models at a minimum must be estimated using identical data, which means that you must eliminate any record in the data set that is missing for any variable in either model. The models must also be nested within one another, so a reduced model is nested within a full model if every parameter in the reduced model also appears in the full model.  If you want to compare models that are not nested you can use two ad hoc criteria. The Akaike, this is very difficult to pronounce, Akaike, I think that's right, information criterion, AIC and the Bayesian information criterion, these are also found under the fit statistics section of the SASS output. And similar to the deviance statistic, the AIC and BIC are also based on  a log likelihood statistic but each one penalizes the log likelihood statistic accord to certain criteria. You'll have to be sure that the pair of models that you want to compare are fit to the identical set of data. And the model with the smaller information criterion fits better. Raftery suggested that differences in BIC criteria of ten and over to be very strong. And to wrap up, today we've covered a brief introduction to growth curve modeling including its level 1 and 2 sub models and its advantages over repeated measures models. We've also applied the growth curve model to the longitudinal quality of life data from OPTIMA. A CSP-sponsored HIV clinical trial. To that end we discussed an overview of the OPTIMA trial. We ran some exploratory analyses, I've presented the two research questions and built a growth curve models based on these questions and finally we interpreted SASS output and I hope that I've convinced that growth curve modeling is an extremely powerful and flexible model for analyzing quality of life or any longitudinal data. I encourage you to learn more about these models and consider them for your future work and here are the references that I mentioned earlier in the presentation. I wanted to mention that the Singer Willet book has a companion website hosted the UCLA with data sets in code in several programming languages. 

>>I want to thank everyone very much for attending today's seminar, and I'm open to questions. >>. "

[OPERATOR] To submit questions you can submit them through the dashboard on your screen. It's on the right hand side of your screen. Just type your questions into the question pane there. I'm sorry. Paul, what were you saying? 

>>[PAUL BARNETT] I was just saying, Vilija, the other things that I noticed that I think you got some benefit out of the SASS mixed modeling course. 

>>[Vilija R. Joyce] Yes. 

>>[PAUL BARNETT] A web course? 

>>[Vilija R. Joyce] It is. So the SASS organization offers a series of courses, they're often in person but some are available on-line. I took the on-line version a few years back, called mixed models analyses using the SASS system. I can't remember. It was over, oh, gosh, at least three days worth of course material. Maybe two days. And I found it to be very helpful. You are on the phone and you're simultaneously coding and looking at slides on your computer and I would highly recommend it. 

>>[Paul Barnett] Yeah. I think it's helpful, he especially if you're running in SASS because sometimes you're not quite sure what SASS is doing, and having, and these were not people from SASS but somebody from a university researcher as I recall, faculty. 

>>[Vilija R. Joyce] Yes, that's correct. 

>>[PAUL BARNETT] Someone asked about how to get handouts, right Heidi? 

>>[OPERATOR]Yes, we are no longer able to upload handouts into the program here. We sent a link out to everyone in the reminder this morning so the direct link for the handouts is there. For if you're unable to find that just shoot me a question at the question pane or at cyberseminar@va.gov and we will get that link to you. 

>>[PAUL BARNETT] We have a complementary comment but no other questions. One thing maybe to go back to is that you mentioned about imputation. Now you didn't do any imputation of missing values here, and you mentioned that it said multiple I am pew take, maybe you can explain when you meant, mean by that. 

>>[Vilija R. Joyce] Sure. So multiple imputation is in contrast to something like at the last value carried forward method of imputing values. So the reason why you might not want to do something like last value carried forward is that it's a biased assessment. It's a mean value for quality of life. It assumes that quality of life for the mixed assessment is the same as quality of life for the previous assessment but the reason the person fails to show up for their appointment might be actually related to treatment, so if that's the case, the values are not missing at random, so in contrast to last value carried forward. Multiple imputation imputes several sets of value for the missing data, and is able to incorporate variability and uncertainty. It's just a better choice and I believe the procedure in SASS is PROCMI. 

>>[paul Barnett] Yes. So I think the ideas are some sort of model that predicts the missingness and then you sample from the data set of people that have similar characteristics to the...  that have data but are similar in terms of the propensity to be missing, and the problem is if you impute just once, that overstates your certainty about what the missing value would have been, and so you have to impute a couple times and then take that multiple imputation into account with estimating the parameters. In other words, there's some uncertainty associated with imputation. And it turns out you don't have to impute a whole bunch of times. Usually I think people impute five times something like that, five values, that sounds out to be sufficient to account for that uncertainty about imputation, so the multiple is that you don't just impute one missing value, you impute a couple, and the imputations strategy is somehow based on the observations propensity to be missing and that you substitute or you impute, drawing from somebody else's very much like that observation in the data set that has data.  So it's illness that's right, maybe they had a serious adverse event means the next time they are missing you select somebody who's like that... has a serious adverse event. 

>>[Vilija R. Joyce] Okay. Thank you. 

>>[PAUL BARNETT] So somebody wrote...  Heidi do you want to do the question?

>>[OPERATOR] I can read it. The comment that we received, I think the terminology growth curve modeling is frequently referred to instruction equational modeling which is a latent approach. The longitudinal modeling you just mentioned is actually mixed effects or random effects modeling. 

>>[Vilija R. Joyce] Okay. I mentioned earlier I presented a list of different terminology and I find that they're used interchangeably but I'm still new to this and still learning so I appreciate the clarification, and, yes, the term I would use daily would actually be mixed effects, so. 

>>[PAUL BARNETT] It is interesting that in different disciplines we use different jargon to refer to the same thing, and it's an interesting question whether the jargon...it’s not consistent. I don't know the definitive answer to that, but it would be interesting in learning more about, from Dr. Wong, about this particular use of  that term growth term modeling. So I think the economist called it random effects. Who call it, biostatisticians?... mixed effects. 

>>[Vilija R. Joyce] Mixed effects. Yeah. 

>>[OPERATOR] That is it for the pending questions for the moment. 

>>[PAUL BARNETT] So you have a survey you want to do for us, is that? 

>>[OPERATOR] And actually with the new GoTo Webinar, it pops up when people leave the meeting. So everyone gets it whether they've already left or whether they're still here. 

>>[PAUL BARNETT] Well, we encourage you to give us a rating so we know how we're doing. 

>>[OPERATOR] Okay. We did just get another question in here. I only saw the fixed effects coefficient do you ever look at the random co-efficient? 

>>[Vilija R. Joyce] Let me see if I can move back a few slides. This is the last model we looked at let me blow this up. Let me see if I can use this highlighter tool. That's the other, can you see that, the red spotlight?   Okay. Great, so the fixed effects is denoted by this first bracket here. The random effects are denoted by the second set of brackets, and they're incorporated into this part of the SASS code. I hope that explains anything. 

>>[PAUL BARNETT] The question is once you estimate them is there anything to be done with the random effects coefficients or the random effects estimates.  It's usually the fixed effects are sort of like in this case once the effect of adverse event on quality of life, so it does present you with a fixed effects, I mean, with the random effects, so what does that tell you? Does that tell you about correlation within observations, that sort of things? 

>>[Vilija R. Joyce] That's right. I'd have to sort of sit down and really work through that. I don't have an answer offhand. 

>>[PAUL BARNETT] Usually I think it's the fixed effects that are the scientific hypothesis and we just are interested in estimating the random effects to make sure that our standard errors and the fixed effects are appropriately adjusted. I think that random effects parameters can tell you about, to the degree to which observations that are, say, from the same person, are correlated. But usually we don't. That's not of so much interest but I know in educational literature sometimes people are interested in that. You know, like students in the same school. 

That sort of things. But it's usually the fixed effects we care about, I think. 

>> [PAUL BARNETT] And then there's a couple more questions now. 

>>[OPERATOR] The next question we received, I understand why you would center a continuous variable. Can you explain why you will want to center a categorical variable again? 

>>[Vilija R. Joyce] Okay. Centering a categorical variable. So for instance, centering the, the SAE, whether or not a patient has an ongoing SAE. Again, I believe it's because we're looking at, we're interested in representing the average so... 

>>[PAUL BARNETT] I guess one answer would be then the intercept has this natural interpretation, which is it's the mean value with all the variables held at their mean, including the categorical variable. In the special case that we had with this trial where we used those, you remember the, we had a factorial trial, so everybody was randomized to two different interventions, and by centering the variables, we were, the parameters have the interpretation for the intervention group of being the, what's the incremental effect of being randomized to this particular treatment while controlling for the assignment to the other treatment, so that in, and the special case of a factorial trial where you randomize people with more than one intervention it turns out to be incredibly useful to have parameters that have natural interpretation, and that interpretation is what's the effect of the intervention on the outcome? So, so that's probably the more compelling reason. And the other, which is just you know, have that intercept be, represent the mean. But it's handy especially when you have interactive terms. You know all the parameters become much easier to interpret when they're centered that way. 

>>[OPERATOR] We did receive a couple comments from the asker of that question. While you were talking there. Choices of variants co-variants. Variants co-variants matrix can be optimized when estimating the random effects, and the unstructured may not always be the best choice of the matrix. 

>>[PAUL BARNETT] Right. I think that's right. Do you want to say something more about that Vilija. 

>>[Vilija R. Joyce] No, I don't have anything to add. 

>>[PAUL BARNETT] So you can make other assumptions about correlation in time and within subject over time that by making those somewhat modest assumptions gain a lot of more statistical power. And sometimes, you probably run into this, if you use unstructured specification the model does not run. 

>>[Vilija R. Joyce] Exactly because it fails to converge. 

>>[PAUL BARNETT] Because there's just not enough information to... you have to make some sort of assumption to get the model estimated, some sort of modest assumption about the appropriate error structure. So I think that's, that's a good point. And there are some, you know, assumptions about how the matrices symmetric, but you know I'm not familiar enough or at least not remembering right now enough about how... what the options are in SASS but there are 3 or 4 options about, at least that you can make about those, those specifications and it's basically from weaker to stronger assumptions, so -- if it doesn't affect your results, that's great. If it does, you have to think about whether or not those assumptions are broken.]

>>[OPERATOR] Okay.  Thank you. We do have a couple other questions here. The next question we received, how are time-varying co-variants handled in growth curve models? 

>>[Vilija R. Joyce] Okay. So I'm still on the slide here. Where we inserted a time varying co-variants. SAE is whether or not there is an ongoing SAE at a particular assessment time is a time varying co-variant. They're handled much the same way as time invariant co-variant. As I mentioned earlier there's some minor changes in notation. I'm not sure if that's, I have much else to add but they're handled very similarly. 

>>[OPERATOR] Okay. That's fine. We do have one last question here. Can you please talk about the intercept in this model? You indicated its average score is baseline for CNT group, but our group variants is not included in the model. I know I'm missing something here? 

>>[Vilija R. Joyce] Can you repeat that question? I'm sorry. 

>>[PAUL BARNETT] So it's, it's, is the intercept the average score baseline for the control group? And that was true when we were in the prior model, not the one that's on the screen now. So in this case it, this...  The model that you have now, what's the interpretation of the intercept? 

>>It's the average. 

>>[Vilija R. Joyce] Oh, the variables are not centered here. So -- 

>>[MULTIPLE SPEAKERS] 

>>Yeah. 

>>Oh, sorry. If it were? 

>>[PAUL BARNETT] Yeah, I guess we have to think about what we meant by, by time, yes. In this case. So you know, I don't think we did centered time in these models. 

>>[Vilija R. Joyce] No, I didn't include a centered. 

>>[PAUL BARNETT] So in the one that's on the screen, right now it's just the baseline value, right? 

>>[Vilija R. Joyce] Yes.  The intercept -- 

>>[PAUL BARNETT] So the, I think the take home message here rather than get too fussy about what these models are, is that a common things that people do is, is center variables, that is express this deviation in mean because it helps you begin to interpret what the marginal effects are, when all variables are set to their mean because basically you're, that's what the intercept represents the mean value while everything else is set to, while all the independent variables are set to the mean. That's a kind of a hard thing to say in words but easier to see if you just imagine, you know we had Y equals alpha plus beta X, and X is expressed as a deviation from the mean. And obviously we set X to the mean, beta times 0 is 0 and so alpha is just the mean value of Y so that carries out  a lot of interactive terms or some useful interpretation there's kind of a convenience for interpreting parameters or interpreting marginal effects and I think the reason why we introduced this here was because of that factorial trial issue where it becomes a credible use where the only way to makes sense of the incremental effects when  we have the two treatments.   Maybe we’ve gone off a little bit in the woods on this. 

>>[OPERATOR] We did get a comment. While you were talking,  the intercept is the average baseline value for the group coded 0 in the interaction term. 

>>[PAUL BARNETT] Yes, that's true ids we weren’t centering the interaction term. Right. So. We may have muddied the waters here. Gone off in the woods, whatever. Whatever it is. Maybe that’s a suggestion for next time how to make this clearer. 

>>[Vilija R. Joyce] More information on centering, yes. 

>>[PAUL BARNETT] Yeah. Or less. 

>>[LAUGHING] 

>>[OPERATOR] Well, we are at the top of the hour. We have gone through all of the questions that we have pending. Either of you have any concluding comments that you'd like to make before we wrap things up here? 

>>[PAUL BARNETT] So these are real simple from versions of the papers that Vilija has authored and have been in a couple different journals and I'm sure she’ll be glad to send you the papers and either of us, you know, send us an E-Mail if there's some... if we've managed to mystify you entirely. I have certainly in my answers to questions. But we'd be glad to follow up by E-Mail. Anything you want to say, Vilija? 

>>[Vilija R. Joyce] Just to echo your comment. I'd be happy to send any papers out and also, I'm not sure if Heidi mentioned the next presentation, I'll just pull it up here. By Patsi on February 1st on budget impact analysis. 

>>[OPERATOR] And we have sent out one round of announcements for that, but everybody should be seeing another one next week, so keep an eye on your E-Mail boxes. 

>>[PAUL BARNETT] And then, we do have another presentation scheduled for later in February, right? Which was going to be on the 22 and we're thinking we're going to switch it to a weekday to the 29. I think we actually definitively will switch that.  So I sent you a message about that Heidi, but that's, this is the course if people present just let them know. That is going to be on, what is that topic going to be about? And I'm giving it. 

>>[OPERATOR] Paul Barnett will be presenting on how can cost effective analysis be made more relevant to US healthcare. So hopefully we'll be able to see a lot of people at those two upcoming sessions. So I want to thank he both of you, Vilija, for both of you to take the time to be at the session. We very much appreciate the time that both of you put into this session. 

>>[EVENT CONCLUDED]