Evaluating Performance and Agreement of Coronary Heart Disease Polygenic Risk Scores
Authors: Abramowitz Sarah A. , Boulier Kristin , Keat Karl , Cardone Katie M. , Shivakumar Manu , DePaolo John , Judy Renae , Bermudez Francisca , Mimouni Nour , Neylan Christopher , Kim Dokyoon , Rader Daniel J. , Ritchie Marylyn D. , Voight Benjamin F. , Pasaniuc Bogdan , Levin Michael G. , Damrauer Scott M. ,   Penn Medicine BioBank, Rader Daniel J , Ritchie Marylyn D , Weaver JoEllen , Naseer Nawar , Sirugo Giorgio , Poindexter Afiya , Ko Yi-An , Nerz Kyle P. , Livingstone Meghan , Vadivieso Fred , DerOhannessian Stephanie , Tran Teo , Stephanowski Julia , Santos Salma , Haubein Ned , Dunn Joseph , Verma Anurag , Kripke Colleen M. , Risman Marjorie , Judy Renae , Wollack Colin , Verma Shefali S. , Damrauer Scott M , Bradford Yuki , Dudek Scott , Drivas Theodore 
Published: November 2024

Importance

Polygenic risk scores (PRSs) for coronary heart disease (CHD) are a growing clinical and commercial reality. Whether existing scores provide similar individual-level assessments of disease susceptibility remains incompletely characterized.

Objective

To characterize the individual-level agreement of CHD PRSs that perform similarly at the population level.

Design, Setting, and Participants

Cross-sectional study of participants from diverse backgrounds enrolled in the All of Us Research Program (AOU), Penn Medicine BioBank (PMBB), and University of California, Los Angeles (UCLA) ATLAS Precision Health Biobank with electronic health record and genotyping data.

Exposures

Polygenic risk for CHD from published PRSs and new PRSs developed separately from testing samples.

Main Outcomes and Measures

PRSs that performed population-level prediction similarly were identified by comparing calibration and discrimination of models of prevalent CHD. Individual-level agreement was tested with intraclass correlation coefficient (ICC) and Light κ.

Results

A total of 48 PRSs were calculated for 171 095 AOU participants. The mean (SD) age was 56.4 (16.8) years. A total of 104 947 participants (61.3%) were female. A total of 35 590 participants (20.8%) were most genetically similar to an African reference population, 29 801 (17.4%) to an admixed American reference population, 100 493 (58.7%) to a European reference population, and the remaining to Central/South Asian, East Asian, and Middle Eastern reference populations. There were 17 589 participants (10.3%) with and 153 506 participants without (89.7%) CHD. When included in a model of prevalent CHD, 46 scores had practically equivalent Brier scores and area under the receiver operator curves (region of practical equivalence ±0.02). Twenty percent of participants had at least 1 score in both the top and bottom 5% of risk. Continuous agreement of individual predictions was poor (ICC, 0.373 [95% CI, 0.372-0.375]). Light κ, used to evaluate consistency of risk assignment, did not exceed 0.56. Analysis among 41 193 PMBB and 53 092 ATLAS participants yielded different sets of equivalent scores, which also lacked individual-level agreement.

Conclusions and Relevance

CHD PRSs that performed similarly at the population level demonstrated highly variable individual-level estimates of risk. Recognizing that CHD PRSs may generate incongruent individual-level risk estimates, effective clinical implementation will require refined statistical methods to quantify uncertainty and new strategies to communicate this uncertainty to patients and clinicians.