Prentice et al. Respond to “How Far Can Epidemiologists Get with Statistical Adjustment?”

We thank Drs. Petitti and Freedman for their invited commentary (1) on our paper (2) that brought together data from the Women’s Health Initiative randomized controlled trial and cohort study in an attempt to understand the apparent discrepancy between results from trials and observational studies on the topic of combined estrogen-plus-progestin postmenopausal hormone therapy and cardiovascular disease. We agree with the substance of Petitti and Freedman’s commentary and write only to add additional perspective on a couple of points. First, the main finding of our analysis is that if study subjects in a cohort begin to be followed some years after exposure initiation, then variations in hazard ratio over time can strongly influence a summary hazard ratio. This issue may best be dealt with through study design by following persons from the initiation of exposure, but even then, average hazard ratio estimates may need to be described as a function of follow-up duration. If not dealt with at the design stage, some more complex analyses may be needed to estimate an average hazard ratio or other suitable summary measure. This issue seemed quite influential in our analyses, for both coronary heart disease and venous thromboembolism, for which hazard ratios were elevated early and then declined. Second, we endorse Petitti and Freedman’s closing sentence: ‘‘However, observational studies are not a substitute for clinical trials no matter how sophisticated the statistical adjustments may seem’’ (1, p. 417). In fact, our own closing sentence reads similarly: ‘‘The inability of those factors [confounding and time from combined hormone therapy initiation] to provide a full explanation for differences between [clinical trial and observational study] stroke hazard ratios reinforces the importance of randomized controlled trial evidence, especially when public health implications are great’’ (2, p. 412). We also agree with Petitti and Freedman when they write the following: ‘‘Without the clinical trial, it would not be clear when to stop adjusting or whether prior adjustments were successful’’ (1, p. 416). This set of circumstances implies that there is an important opportunity when good quality observational study data and clinical trial data happen to be available for a treatment of interest from comparable populations. The clinical trial data can complement the observational study analysis to gain insights concerning biases and potential design and analysis improvements. Moreover, though not illustrated by our paper (2), joint analysis of clinical trial and observational study data may be able to extend the implications of a clinical trial in a fairly reliable manner, though less so than would be the case with a larger, longer, or more comprehensive clinical trial. For example, as with the Women’s Health Initiative hormone therapy trials, it may not be possible to extend the intervention and follow-up long enough in a trial to answer questions about the benefits and risks of long-term use, but joint analyses that are able to ‘‘align’’ the results from the two sources, while making some provision for observational study residual confounding, may enjoy considerably greater reliability than would observational study analyses alone. Other extensions may pertain to more precise evaluations in important subsets of a study population and even consideration of doses, schedules, routes of administration, and agents beyond those specifically studied in a clinical trial. Though there is little experience to date with joint observational study and clinical trial data sources for this type of purpose, we have carried out such analyses for both estrogen-alone and combined postmenopausal hormone therapy preparations and plan to present these analyses for publication in the near future.