Reclassification Calculations for Persons With Incomplete Follow-up
Authors: Ewout W. Steyerberg
Journal: Annals of Internal Medicine (2010)

Letters2 February 2010Reclassification Calculations for Persons With Incomplete Follow-upEwout W. Steyerberg, PhD and Michael J. Pencina, PhDEwout W. Steyerberg, PhDFrom Erasmus MC University Medical Center Rotterdam, Rotterdam 3015, the Netherlands; and Boston University, Boston, MA 02215. and Michael J. Pencina, PhDFrom Erasmus MC University Medical Center Rotterdam, Rotterdam 3015, the Netherlands; and Boston University, Boston, MA 02215.Author, Article, and Disclosure Informationhttps://doi.org/10.7326/0003-4819-152-3-201002020-00019 SectionsAboutFull TextPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinkedInRedditEmail TO THE EDITOR:We applaud Cook and Ridker (1) for their clear discussion of reclassification measures and recent developments in judging the incremental value of a biomarker for prediction of outcome. A key area of application is in cardiovascular disease, in which the time horizon is typically 10 years. One important problem recognized by Cook and Ridker is that not all persons will have follow-up completed until 10 years. Kaplan–Meier curves and Cox regression analysis were introduced long ago to deal with such censored observations. Reclassification measures, such as the net reclassification index (NRI) (2), have been proposed for binary ...References1. Cook NR, Ridker PM. Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures. Ann Intern Med. 2009;150:795-802. [PMID: 19487714] LinkGoogle Scholar2. Pencina MJ, D'Agostino RB Sr, D'Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27:157-72; discussion 207-12. [PMID: 17569110] Google Scholar3. Viallon V, Ragusa S, Clavel-Chapelon F, Bénichou J. How to evaluate the calibration of a disease risk prediction tool. Stat Med. 2009;28:901-16. [PMID: 19156698] CrossrefMedlineGoogle Scholar4. Pepe MS, Feng Z, Gu JW. Comments on 'Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond' by M. J. Pencina et al., Statistics in Medicine (DOI: 10.1002/sim.2929). Stat Med. 2008;27:173-81. [PMID: 17671958] CrossrefMedlineGoogle Scholar Author, Article, and Disclosure InformationAuthors: Ewout W. Steyerberg, PhD; Michael J. Pencina, PhDAffiliations: From Erasmus MC University Medical Center Rotterdam, Rotterdam 3015, the Netherlands; and Boston University, Boston, MA 02215.Disclosures: None disclosed. PreviousarticleNextarticle Advertisement FiguresReferencesRelatedDetailsSee AlsoAdvances in Measuring the Effect of Individual Predictors of Cardiovascular Risk: The Role of Reclassification Measures Nancy R. Cook and Paul M Ridker Reclassification Calculations for Persons With Incomplete Follow-up Nancy R. Cook and Paul M. Ridker Metrics Cited byImpact of calcification on clinical outcomes after drug‐coated balloon angioplasty for superficial femoral artery disease: Assessment using the peripheral artery calcification scoring systemImpact of High Lipoprotein(a) Levels on Clinical Outcomes Following Peripheral Endovascular TherapyLa albuminuria y la enfermedad polivascular mejoran la capacidad predictiva de los modelos multivariados después de un evento cardiovascular agudo. Cohorte AIRVAGAlbuminuria and polyvascular disease improve multivariate predictive models after an acute cardiovascular event. The AIRVAG cohortPredicting 2‐year all‐cause mortality after contemporary PCI : Updating the logistic clinical SYNTAX scoreRefining prediction of major bleeding on antiplatelet treatment after transient ischaemic attack or ischaemic strokeMeasures for evaluating risk prediction models: a reviewPredictive Model for Mortality Risk Including the Wound, Ischemia, Foot Infection Classification in Patients Undergoing Revascularization for Critical Limb IschemiaWeighted Multi-marker Genetic Risk Scores for Incident Coronary Heart Disease among Individuals of African, Latino and East-Asian AncestryA methodology review on the incremental prognostic value of computed tomography biomarkers in addition to Framingham risk score in predicting cardiovascular disease: the use of association, discrimination and reclassificationQuantifying the added value of new biomarkers: how and how notAnalysis of Plasma Albumin, Vitamin D, and Apolipoproteins A and B as Predictive Coronary Risk Biomarkers in the REGICOR StudyValor predictivo de la albúmina plasmática, la vitamina D y las apolipoproteínas A y B como biomarcadores de riesgo coronario en el estudio REGICORClinical Utility of Carotid Ultrasonography in the Prediction of Cardiovascular Events in Patients with Diabetes: A Combined Analysis of Data Obtained in Five Longitudinal StudiesEvent rate net reclassification index and the integrated discrimination improvement for studying incremental value of risk markersAssessment of long-term mortality in patients with complex coronary artery disease undergoing percutaneous intervention: comparison of multiple anatomical and clinical prognostic risk scoresA clinical-genetic approach to assessing cardiovascular risk in patients with CKDUpdated risk factors should be used to predict development of diabetesClinical Utility of Multimarker Genetic Risk Scores for Prediction of Incident Coronary Heart DiseaseExercise and Bayes' Theorem: Some things never go out of styleIncremental prognostic value of coronary computed tomographic angiography high-risk plaque characteristics in newly symptomatic patientsEvaluating risk-prediction models using data from electronic health recordsCarotid Stiffness Is Associated With Incident StrokeAdding low ankle brachial index to classical risk factors improves the prediction of major cardiovascular events. The REGICOR studyGraphical assessment of incremental value of novel markers in prediction models: From statistical to decision analytical perspectivesRanking of psychosocial and traditional risk factors by importance for coronary heart disease: the Copenhagen City Heart StudyMultidimensional Prognostic Index Predicts Mortality and Length of Stay During Hospitalization in the Older Patients: A Multicenter Prospective StudyIncremental prognostic utility of coronary CT angiography for asymptomatic patients based upon extent and severity of coronary artery calcium: results from the COronary CT Angiography EvaluatioN For Clinical Outcomes InteRnational Multicenter (CONFIRM) StudyLong-Term Prognostic Effect of Coronary Atherosclerotic BurdenSelf‐Reported Cardiorespiratory Fitness: Prediction and Classification of Risk of Cardiovascular Disease Mortality and Longevity—A Prospective Investigation in the Copenhagen City Heart StudyProspective study of cognitive decline assessed using the mini-mental state examination and the risk of incident long-term care insurance among community-dwelling older JapanesePrimary Gleason Grade 4 at the Positive Margin Is Associated with Metastasis and Death Among Patients with Gleason 7 Prostate Cancer Undergoing Radical ProstatectomyAn assessment of the relationship between clinical utility and predictive ability measures and the impact of mean risk in the populationIs pathology necessary to predict mortality among men with prostate-cancer?Charting a Roadmap for Heart Failure Biomarker StudiesComparison of Tumor Regression Grade Systems for Locally Advanced Rectal Cancer After Multimodality TreatmentHow to interpret a small increase in AUC with an additional risk prediction marker: decision analysis comes throughPrognostic Value of Site SYNTAX Score and Rationale for Combining Anatomic and Clinical Factors in Decision MakingThe Predictive Ability of the CHADS2 and CHA2DS2-VASc Scores for Bleeding Risk in Atrial Fibrillation: The MAQI2 ExperienceUrinary Biomarkers of AKI and Mortality 3 Years after Cardiac SurgeryNet Reclassification Improvement: Computation, Interpretation, and Controversies A Literature Review and Clinician's GuideMaarten J.G. Leening, MD, MSc, Moniek M. Vedder, MSc, Jacqueline C.M. Witteman, PhD, Michael J. Pencina, PhD, and Ewout W. Steyerberg, PhDPredictive Performance of SYNTAX Score II in Patients With Left Main and Multivessel Coronary Artery DiseaseDevelopment and Validation of a Predicting Model of All-Cause Mortality in Patients With Type 2 DiabetesUsefulness of Exercise Test and Myocardial Perfusion–Gated Single Photon Emission Computed Tomography to Improve the Prediction of Major EventsDevelopment and validation of a prediction rule for recurrent vascular events based on a cohort study of patients with arterial disease: the SMART risk scorePrognostic Interplay of Coronary Artery Calcification and Underlying Vascular Dysfunction in Patients With Suspected Coronary Artery DiseaseEvaluation of Markers and Risk Prediction ModelsAssociation Between Coronary Vascular Dysfunction and Cardiac Mortality in Patients With and Without Diabetes MellitusDevelopment and Validation of a Coronary Risk Prediction Model for Older U.S. and European Persons in the Cardiovascular Health Study and the Rotterdam StudyMichael T. Koller, MD, MSc, Maarten J.G. Leening, MD, MSc, Marcel Wolbers, PhD, Ewout W. Steyerberg, PhD, M.G. Myriam Hunink, MD, PhD, Rotraut Schoop, PhD, Albert Hofman, MD, PhD, Heiner C. Bucher, MD, MPH, Bruce M. Psaty, MD, PhD, Donald M. Lloyd-Jones, MD, ScM, and Jacqueline C.M. Witteman, PhDCoronary Calcification and the Risk of Heart Failure in the ElderlyRisk prediction of incident coronary heart disease in the Netherlands: re-estimation and improvement of the SCORE risk functionCommon carotid intima-media thickness in cardiovascular risk stratification of older people: the Rotterdam StudyCoronary Computed Tomographic Angiography and Risk of All-Cause Mortality and Nonfatal Myocardial Infarction in Subjects Without Chest Pain Syndrome From the CONFIRM Registry (Coronary CT Angiography Evaluation for Clinical Outcomes: An International Multicenter Registry)Assessment of the value of a genetic risk score in improving the estimation of coronary riskEvaluation of Newer Risk Markers for Coronary Heart Disease Risk Classification A Cohort StudyMaryam Kavousi, MD, MSc, Suzette Elias-Smale, MD, PhD, Joost H.W. Rutten, MD, PhD, Maarten J.G. Leening, MD, MSc, Rozemarijn Vliegenthart, MD, PhD, Germaine C. Verwoert, MSc, Gabriel P. Krestin, MD, PhD, Matthijs Oudkerk, MD, PhD, Moniek P.M. de Maat, PhD, Frank W.G. Leebeek, PhD, Francesco U.S. Mattace-Raso, MD, PhD, Jan Lindemans, PhD, Albert Hofman, MD, PhD, Ewout W. Steyerberg, PhD, Aad van der Lugt, MD, PhD, Anton H. van den Meiracker, MD, PhD, and Jacqueline C.M. Witteman, PhDMultiple Biomarkers for Risk Prediction in Chronic Heart FailureAdded predictive ability of the information on job strain beyond the standard Framingham risk scoreDoes aortic stiffness improve the prediction of coronary heart disease in elderly? The Rotterdam StudyImproved Cardiac Risk Assessment With Noninvasive Measures of Coronary Flow ReserveFree Protein S Level as a Risk Factor for Coronary Heart Disease and Stroke in a Prospective Cohort Study of Healthy United Kingdom MenMethods for Assessing Fracture Risk Prediction Models: Experience With FRAX in a Large Integrated Health Care Delivery SystemPerformance Measures for Prediction Models and Markers: Evaluation of Predictions and ClassificationsMedidas del rendimiento de modelos de predicción y marcadores pronósticos: evaluación de las predicciones y clasificacionesUse of reclassification for assessment of improved prediction: an empirical evaluationUsing Additional Information on Working Hours to Predict Coronary Heart Disease A Cohort StudyMika Kivimäki, PhD, G. David Batty, PhD, Mark Hamer, PhD, Jane E. Ferrie, PhD, Jussi Vahtera, MD, PhD, Marianna Virtanen, PhD, Michael G. Marmot, MD, PhD, Archana Singh-Manoux, PhD, and Martin J. Shipley, MScHigh-Sensitivity ST2 for Prediction of Adverse Outcomes in Chronic Heart FailureSeveral methods to assess improvement in risk prediction models: Extension to survival analysisExtensions of net reclassification improvement calculations to measure usefulness of new biomarkersMethods for evaluating novel biomarkers - a new paradigmA multilocus genetic risk score for coronary heart disease: case-control and prospective cohort analysesCoronary Calcium Score Improves Classification of Coronary Heart Disease Risk in the Elderly 2 February 2010Volume 152, Issue 3Page: 195-196KeywordsCardiovascular diseasesConflicts of interestForecastingPerformance measuresRegression analysisSurvival analysis ePublished: 2 February 2010 Issue Published: 2 February 2010 Copyright & PermissionsCopyright © 2010 by American College of Physicians. All Rights Reserved.PDF downloadLoading ...