Prediction of Prognosis and Treatment Benefit by the Adjuvant! Web-Based Tool: Calibration and Discriminatory Accuracy in 5380 Dutch Breast Cancer Patients.
Authors: Schmidt M. , Mook S. , Rutgers E. , van de Velde T. , Visser O. , Armstrong N. , Ravdin P. , Van't Veer L. 
Published: December 2010

Abstract
Adjuvant! online (AOL) is a web-based tool that calculates individualized 10-year survival probabilities and predicted benefit of adjuvant systemic therapy. The AOL model was constructed using data from breast cancer patients recorded in the US population-based SEER registry and has not yet been validated in any large European series. European patients may differ from those in the US with regard to prognostic factors and intrinsic tumor characteristics. Therefore the aim of our study was to validate AOL in Dutch patients, investigating both its calibration and discriminatory accuracy.Patients who were treated at the Netherlands Cancer Institute for breast cancer between 1987 and 1998 were selected according to the following criteria: pT1-3, pN0-3, M0, definitive primary surgery, and complete axillary staging. Median follow-up was 11.7 years; &gt;96% of patients had more than 10 years follow-up. Clinicopathological characteristics and adjuvant treatment data were entered into the AOL version 8.0 batch processer, with blinding to outcome.For all 5380 patients included, the 10-year observed overall survival (69.0%) and breast cancer-specific survival (78.6%) rates as compared to the 10-year rates predicted by AOL (69.1% and 77.8%, respectively) were within 1% and not statistically different. Moreover, differences between predicted and observed outcomes were within 2% for most relevant clinicopathological subgroups. Subsets of patients for which there was a discrepancy between the predicted outcomes by AOL and the actual observed outcomes included patients under 40 years, for which both the predicted overall and breast cancer-specific survival were overly optimistic (Pred – Obs: by 4.2% and 4.7% respectively; p&lt;0.05).The concordance(c)-index, which indicates discriminatory accuracy at the individual level, was 0.71 for BCSS in the entire cohort, indicating that AOL performs better than chance. The c-index for a multivariate Cox regression model fitted to the 5380 patients using a backward approach including clinicopathological variables was also 0.71 for BCSS, suggesting AOL optimally incorporates the information in these variables.In order to validate the AOL model, we evaluated both its calibration (goodness of fit) and discriminatory accuracy. Although the latter is rarely examined, it is of paramount importance to justify the use of prognostic models for clinical outcome prediction. We showed that in addition to good calibration, the model was capable of separating individuals with a poor outcome from those with a good outcome with moderate power. In conclusion, AOL accurately predicted 10-year outcomes in this first large scale European validation study and is of use for adjuvant treatment-decision making. Additional inclusion of biological markers in AOL in the future is expected to lead to increased discriminatory accuracy.
Citation Information: Cancer Res 2009;69(24 Suppl):Abstract nr 4031.