An external validation of the QCovid risk prediction algorithm for risk of mortality from COVID-19 in adults: national validation cohort study in England
Authors: Nafilyan Vahe , Humberstone Ben , Mehta Nisha , Diamond Ian , Coupland Carol , Lorenzi Luke , Pawelek Piotr , Schofield Ryan , Morgan Jasper , Brown Paul , Lyons Ronan , Sheikh Aziz , Hippisley-Cox Julia 
Published: January 2021

Background

To externally validate a risk prediction algorithm (QCovid) to estimate mortality outcomes from COVID-19 in adults in England.

Methods

Population-based cohort study using the ONS Public Health Linked Data Asset, a cohort based on the 2011 Census linked to Hospital Episode Statistics, the General Practice Extraction Service Data for pandemic planning and research, radiotherapy and systemic chemotherapy records. The primary outcome was time to COVID-19 death, defined as confirmed or suspected COVID-19 death as per death certification. Two time periods were used: (a) 24thJanuary to 30thApril 2020; and (b) 1stMay to 28thJuly 2020. We evaluated the performance of the QCovid algorithms using measures of discrimination and calibration for each validation time period.

Findings

The study comprises 34,897,648 adults aged 19-100 years resident in England. There were 26,985 COVID-19 deaths during the first time-period and 13,177 during the second. The algorithms had good calibration in the validation cohort in both time periods with close correspondence of observed and predicted risks. They explained 77.1% (95% CI: 76.9% to 77.4%) of the variation in time to death in men in the first time-period (R2); the D statistic was 3.76 (95% CI: 3.73 to 3.79); Harrell’s C was 0.935 (0.933 to 0.937). Similar results were obtained for women, and in the second time-period. In the top 5% of patients with the highest predicted risks of death, the sensitivity for identifying deaths in the first time period was 65.9% for men and 71.7% for women. People in the top 20% of predicted risks of death accounted for 90.8% of all COVID-19 deaths for men and 93.0% for women.

Interpretation

The QCovid population-based risk algorithm performed well, showing very high levels of discrimination for COVID-19 deaths in men and women for both time periods. It has the potential to be dynamically updated as the pandemic evolves and therefore, has potential use in guiding national policy.

Funding

National Institute of Health Research

RESEARCH IN CONTEXT

Evidence before this study

Public policy measures and clinical risk assessment relevant to COVID-19 need to be aided by rigorously developed and validated risk prediction models. A recent living systematic review of published risk prediction models for COVID-19 found most models are subject to a high risk of bias with optimistic reported performance, raising concern that these models may be unreliable when applied in practice. A population-based risk prediction model, QCovid risk prediction algorithm, has recently been developed to identify adults at high risk of serious COVID-19 outcomes, which overcome many of the limitations of previous tools.

Added value of this study

Commissioned by the Chief Medical Officer for England, we validated the novel clinical risk prediction model (QCovid) to identify risks of short-term severe outcomes due to COVID-19. We used national linked datasets from general practice, death registry and hospital episode data for a population-representative sample of over 34 million adults. The risk models have excellent discrimination in men and women (Harrell’s C statistic>0.9) and are well calibrated. QCovid represents a new, evidence-based opportunity for population risk-stratification.

Implications of all the available evidence

QCovid has the potential to support public health policy, from enabling shared decision making between clinicians and patients in relation to health and work risks, to targeted recruitment for clinical trials, and prioritisation of vaccination, for example.