Exploratory Analysis of Practical Trial Data for Informing the Individualization Of Treatment

Objective: Practical randomized trials produce rich data that allow comparison of the effectiveness of several different treatments on heterogeneous participants. They also enable investigation of how relative effectiveness of different treatments varies with patient variables like age or symptom severity. The authors present exploratory data analysis tools that can be used to reveal relationships between prescriptive variables and the best choice of treatment, in order to suggest interesting hypotheses for future investigation. Method: Characteristics of practical clinical trial data are reviewed, and the use of such data for informing individualization of treatment is motivated. A method for suggesting potential strategies for treatment individualization is proposed, and its use is illustrated using data from the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) study. The authors present an analysis that uses the self-reported Quick Inventory of Depressive Symptomatology (QIDS-SR) score to predict the best subsequent treatment. Results: The proposed exploratory data analysis method provides a measure of confidence that estimates, for a given value of a prescriptive variable, the probabilities that each treatment would be selected to be the best if the trial and analyses were repeated. The authors relate the meaning of this quantity to confidence intervals for the standardized effect size, making the method a useful alternative interpretation of more traditional analyses. Conclusion: Exploratory analyses of practical trial data as presented here can yield insights that suggest avenues for future research and eventually inform clinical practice. Exploratory Analysis 3 Exploratory Analysis of Practical Trial Data for Informing the Individualization Of Treatment Historically, the majority of randomized clinical trials have been designed to establish that a new treatment is efficacious. Numerous treatments have been vetted in this way, most commonly by comparison against standard care and against placebo (Dawson & Lavori, 2004). This practice has provided clinicians with an extensive menu of options for treating the patients in their care, but has seldom offered guidance on how to select a treatment for an individual, in large part because such trials typically neglect the consideration of external validity (Rothwell, 2005a). This lack of guidance has persisted despite a clear desire on the part of clinicians to tailor treatments to individuals; for example, clinical experts have come together to develop guidelines for predicting which class of antidepressant would most likely benefit a particular patient (Stern, Rush, & Mendels, 1980). Since the discovery and approval of modern neurotransmitter reuptake inhibitors, the opportunity for tailored treatment of depression has increased even further. Currently, there is significant interest in implementing large practical clinical trials designed to simultaneously investigate the comparative effectiveness of a set of treatments. For example, the STAR*D and CATIE (Clinical Antipsychotic Trials in Intervention Effectiveness) (Stroup et al., 2003) trials were designed to study the effect of a variety of appropriate treatments on a population similar to the population of patients a clinician encounters in practice (Tunis, Stryer, & Clancy, 2003). The usefulness of these trials stems from two important characteristics: First, they assess the comparative effectiveness of a variety of standard treatments. Second, the trials use broad inclusion criteria and thus acquire data on a heterogeneous group of patients who exhibit a range of treatment responses. The data produced by these trials are therefore useful for investigating the effect of treatments that are clinically relevant on the patients who are are likely to present in practice. Exploratory Analysis 4 Primary analysis of these trials commonly consists of two-way comparisons between treatment groups. These comparisons use standard statistical analyses, e.g. hypothesis tests and confidence intervals, to convey the statistical significance of the observed difference in the outcome of each group. Note that these comparisons average over outcomes of all patients who fall in each group—populations that, as we have mentioned, can be quite diverse in practical trials. In this work, we focus on analyses that take advantage of this diversity to look inside treatment groups and investigate relationships between patient variables and outcomes that can inform the tailoring of treatment to individual patients. Our goal is to augment established sources of clinical knowledge using exploratory data analysis methods in a way that is useful for directing future scientific investigation in the service of informing clinical decisions. Exploratory analysis of the data from a randomized trial is useful way of generating hypotheses about how treatment should be tailored; however, any conclusions drawn should not be attributed the same validity as the conclusion of the primary analysis for which the trial was designed. Rothwell (Rothwell, 2005b) states, “The best test of validity of subgroup-treatment effect interactions is their reproducibility in other trials. ” We are in agreement. We therefore produce exploratory analyses with a measure of confidence that is relevant to future investigations by asking: “How much evidence is there in our existing data that an observed effect will replicate in a future trial? ” To begin, we review the concepts of qualitative and non-qualitative interactions and illustrate how their presence or absence can be estimated using regression. We then present our measure of confidence, which we term “selection probability,” and present two analysis examples, one using synthetic data and one using a subset of the STAR*D trial data. We describe how selection probability is related to the standardized effect size, and how a confidence interval for one relates to a confidence interval for the other. Exploratory Analysis 5 Qualitative Interactions and Best Treatment The usefulness of a patient variable for tailoring treatment is determined by whether or not there is an “interaction” between that variable and treatment outcomes. For example, the patient variable could be a baseline variable like age or gender, or it could measure a patient’s response to a previous treatment. We say a “qualitative interaction” is present when the best treatment differs depending on the value of a patient variable; such a variable is termed “prescriptive. ” The “best” treatment among those under consideration is the one that minimizes1 a chosen outcome. One example of a qualitative interaction between QIDS-SR and best treatment might be “Patients with lower QIDS-SR scores respond better to treatment A, while patients with higher QIDS-SR scores respond better to treatment B. ” We say that a “non-qualitative interaction” is present when the best treatment is the same for all values of the variable, but the magnitude of its advantage over other treatments changes depending on the specific value of the variable. An example of a non-qualitative interaction would be “Patients respond better to B than they do to A. Patients with high QIDS-SR scores respond much better to B than to A, while patients with low QIDS-SR scores respond moderately better. ” In the most extreme instance of a non-qualitative interaction, a treatment may be clearly superior for some values of a prescriptive variable, but essentially equivalent to its competitors for other values. Knowledge of a qualitative interaction between a prescriptive variable and best treatment is useful because it can inform the choice of treatment for future patients. For example, if trial data indicate an interaction between baseline QIDS-SR score and best treatment, we may decide that in future states of clinical equipoise we should measure the baseline QIDS-SR of an incoming patient and, all else being equal, select the treatment that is expected to result in the best response for the measured value of QIDS-SR. Knowledge of a non-qualitative interaction can be useful as well; suppose for example that Exploratory Analysis 6 overall, patients respond better to B, but patients with low QIDS-SR scores respond equally well to B and A. Then for patients with low QIDS-SR, we may be more inclined to select treatment based on other desiderata of the patient and physician, such as avoiding certain side effects, despite the fact that B appears to be superior on average. For patients with higher QIDS-SR, we would eliminate treatment A from consideration. Interactions can also reveal scientific insights and generate hypotheses about the relationship between patient characteristics, a treatment’s mechanism of action, and disease progression. For example, suppose we offer a first-line treatment T to a patient, and monitor their disease progression. After a time, if results are unsatisfactory, we consider two follow-up treatments A and B. Suppose A and T are considered similar, and B and T are considered dissimilar in terms of mechanism of action. We might therefore expect B to be better overall for patients who have failed T, but qualitative interactions could reveal prescriptive variables that describe the circumstances in which A is in fact a better follow-up choice. Discovery of such interactions can be a first step toward identifying the populations in which A and T act differently, and spur future scientific investigation into the reason for this difference. Interactions and Confidence We now introduce methodology for investigating the presence of qualitative interactions of prescriptive variables with treatment outcomes using data generated from practical clinical trials. For simplicity of exposition, we describe the method for the case where both the prescriptive variable and the patient outcome are continuous variables. This setting allows the use of an intuitive, regression-based approach