Are Risk Stratification Tables the Best Way to Evaluate Model Performance?
Authors: Holly Janes
Journal: Annals of Internal Medicine (2009)

Letters17 March 2009Are Risk Stratification Tables the Best Way to Evaluate Model Performance?Holly Janes, PhD, Margaret S. Pepe, PhD, and Wen Gu, MSHolly Janes, PhDFrom Fred Hutchinson Cancer Research Center, Seattle, WA 98109, and University of Washington, Seattle, WA 98195.Search for more papers by this author, Margaret S. Pepe, PhDFrom Fred Hutchinson Cancer Research Center, Seattle, WA 98109, and University of Washington, Seattle, WA 98195.Search for more papers by this author, and Wen Gu, MSFrom Fred Hutchinson Cancer Research Center, Seattle, WA 98109, and University of Washington, Seattle, WA 98195.Search for more papers by this authorAuthor, Article, and Disclosure Informationhttps://doi.org/10.7326/0003-4819-150-6-200903170-00022 SectionsAboutFull TextPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinkedInRedditEmail IN RESPONSE:We thank Drs. Stern and Smith for their thoughts on improving methods for the evaluation of risk prediction models. We agree wholeheartedly that the distribution of risks predicted by the risk prediction model is key for evaluating model performance. In the statistical literature, this has been called the predictiveness curve, and we have advocated strongly for its use (1, 2). In fact, the margins of a risk stratification table display exactly this: the population distribution of risk according to the 2 models, albeit by using discrete categories. Because the main goal of our article is to emphasize that ...References1. Pepe MS, Feng Z, Huang Y, Longton G, Prentice R, Thompson IM, et al. Integrating the predictiveness of a marker with its performance as a classifier. Am J Epidemiol. 2008;167:362-8. [PMID: 17982157] CrossrefMedlineGoogle Scholar2. Huang Y, Sullivan Pepe M, Feng Z. Evaluating the predictiveness of a continuous marker. Biometrics. 2007;63:1181-8. [PMID: 17489968] CrossrefMedlineGoogle Scholar3. Pepe MS, Janes H, Gu JW. Letter by Pepe et al regarding article, “Use and misuse of the receiver operating characteristic curve in risk prediction” [Letter]. Circulation. 2007;116:e132; author reply e134. [PMID: 17679623] Google Scholar4. Pepe MS, Janes HE. Gauging the performance of SNPs, biomarkers, and clinical factors for predicting risk of breast cancer [Editorial]. J Natl Cancer Inst. 2008;100:978-9. [PMID: 18612128] CrossrefMedlineGoogle Scholar5. Huang Y, Pepe MS. A parametric ROC model based approach for evaluating the predictiveness of continuous markers in case-control studies. Biometrics. 2009. [Forthcoming]. Google Scholar6. Pepe MS, Feng Z, Gu JW. Comments on ‘Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond’ by M. J. Pencina et al., Statistics in Medicine (DOI: 10.1002/sim.2929). Stat Med. 2008;27:173-81. [PMID: 17671958] Google Scholar Author, Article, and Disclosure InformationAuthors: Holly Janes, PhD; Margaret S. Pepe, PhD; Wen Gu, MSAffiliations: From Fred Hutchinson Cancer Research Center, Seattle, WA 98109, and University of Washington, Seattle, WA 98195.Disclosures: None disclosed. PreviousarticleNextarticle Advertisement FiguresReferencesRelatedDetailsSee AlsoAssessing the Value of Risk Predictions by Using Risk Stratification Tables Holly Janes , Margaret S. Pepe , and Wen Gu Are Risk Stratification Tables the Best Way to Evaluate Model Performance? Ralph H. Stern and Dean E. Smith Metrics Cited byUse of reclassification for assessment of improved prediction: an empirical evaluation 17 March 2009Volume 150, Issue 6Page: 428KeywordsCellsConflicts of interestReceiver operating characteristic curveStatistical distributions ePublished: 17 March 2009 Issue Published: 17 March 2009 Copyright & PermissionsCopyright © 2009 by American College of Physicians. All Rights Reserved.PDF downloadLoading ...