Reply to ‘The potential and perils of observational studies’ by M. Buyse et al.
Authors: S. Delaloge, D. Pérol, M. Robain, C. Cailliot
Journal: Annals of Oncology (2016)

We thank Dr Buyse and Dr Vansteelandt for their constructive comments and suggestions [1.Buyse M. Vansteelandt SP. The potential and perils of observational studies.Ann Oncol. 2017; 28: 182Abstract Full Text Full Text PDF PubMed Scopus (3) Google Scholar] regarding our recently published evaluation from the ESME program of the real life outcomes of two large cohorts of Her2-negative metastatic breast cancer patients treated with paclitaxel ± bevacizumab as first-line chemotherapy [2.Delaloge S. Pérol D. Courtinard C. et al.Paclitaxel plus bevacizumab or paclitaxel as first-line treatment for HER2-negative metastatic breast cancer in a multicenter national observational study.Ann Oncol. 2016; 27: 1725-1732Abstract Full Text Full Text PDF PubMed Scopus (57) Google Scholar]. Indeed, although the need for observational data to complement those obtained by randomized clinical trials (RCTs) is now widely acknowledged, we agree with our colleagues that no perfect methodology has yet been identified to analyze them. In particular, we recognize that observational studies may substantially bias treatment effects on mortality outcomes despite the use of rigorous standard screening procedures, effective methods for matching patient data across disparate systems and propensity scores to address confounding biases [3.Hemkens L.G. Contopoulos-Ioannidis D.G. Ioannidis JP. Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey.Br Med J. 2016; 352: i493.Crossref PubMed Scopus (125) Google Scholar]. Recent large international group discussions have attempted to define the best approaches in this setting [4.Chavez-MacGregor M. Giordano SH. Randomized clinical trials and observational studies: is there a battle?.J Clin Oncol. 2016; 34: 772-773Crossref PubMed Scopus (36) Google Scholar]. Our survival findings are not widely discrepant with those of RCTs: the observed progression-free survival rates in both groups are close to those described in RCTs, and overall survival (OS) in patients receiving paclitaxel + bevacizumab was exactly as that expected from RCTs (median 27.7 months). However, OS observed in patients receiving paclitaxel alone was much lower than that observed in RCTs (19 months) [5.Miller K. Wang M. Gralow J. et al.Paclitaxel plus bevacizumab versus paclitaxel alone for metastatic breast cancer.N Engl J Med. 2007; 357: 2666-2676Crossref PubMed Scopus (2712) Google Scholar]. Interestingly, a major difference with RCTs is that the cross-over rate appears very low in ESME: only 5% of patients treated in the paclitaxel alone group received bevacizumab subsequently (when compared with 40% and 49%, respectively, in two of the three RCTs in the same setting) [6.Miles D.W. Chan A. Dirix L.Y. et al.Phase III study of bevacizumab plus docetaxel compared with placebo plus docetaxel for the first-line treatment of human epidermal growth factor receptor 2-negative metastatic breast cancer.J Clin Oncol. 2010; 28: 3239-3247Crossref PubMed Scopus (791) Google Scholar, 7.Robert N.J. Diéras V. Glaspy J. et al.RIBBON-1: randomized, double-blind, placebo-controlled, phase III trial of chemotherapy with or without bevacizumab for first-line treatment of human epidermal growth factor receptor 2-negative, locally recurrent or metastatic breast cancer.J Clin Oncol. 2011; 29: 1252-1260Crossref PubMed Scopus (803) Google Scholar]. In sensitivity analyses, we showed that the OS differential between treatment groups was confirmed in patients not enrolled in clinical trials (Table 2) (HR for OS 0.59). Using an additional sensitivity analysis, as proposed by Dr Buyse et al., we compared OS in ‘RCT-type’ patients, who have been included in any clinical trial during the course of their disease and received paclitaxel ± bevacizumab as sole first-line therapy. From the total, 257 such patients received the combined treatment, whereas 79 received paclitaxel alone. Crude median OS in these two groups are, respectively, 31.6 months [95 CI 27.9–37.8] and 27.7 months [95 CI 18.0–35.2]. We also explored the possibility of taking into account results between centers or geographic areas of prescriptions according to the instrumental variables principle. However, these variables had a direct effect on OS, and not only indirectly through the treatment and therefore could not be considered as relevant instrumental variables to remove the effects of hidden bias. Although we performed the best possible sensitivity analyses and obtained concordant results, we regret that biases still cannot be totally eliminated from this work. Notwithstanding, initiatives such as ESME should help clinicians and researchers realize the need to agree on high quality data collection and analytical standards in real life studies, as well as to better understand the contributions of real-world evidence to causal reasoning. The authors thank Lorna Saint Ange for editing the manuscript. ESME database and research, including the present work, are supported by R&D UNICANCER and industrial fundings by Roche, Pierre Fabre, AstraZeneca and Pfizer.