Development and external validation of prognostic models for COVID-19 to support risk stratification in secondary care
Authors: Adderley Nicola J , Taverner Thomas , Price Malcolm , Sainsbury Christopher , Greenwood David , Chandan Joht Singh , Takwoingi Yemisi , Haniffa Rashan , Hosier Isaac , Welch Carly , Parekh Dhruv , Gallier Suzy , Gokhale Krishna , Denniston Alastair K , Sapey Elizabeth , Nirantharakumar Krishnarajah 
Published: January 2021

Objectives

Existing UK prognostic models for patients admitted to hospital with COVID-19 are limited by reliance on comorbidities, which are under-recorded in secondary care, and lack of imaging data among the candidate predictors. Our aims were to develop and externally validate novel prognostic models for adverse outcomes (death, intensive therapy unit (ITU) admission) in UK secondary care; and externally validate the existing 4C score.

Design

Candidate predictors included demographic variables, symptoms, physiological measures, imaging, laboratory tests. Final models used logistic regression with stepwise selection.

Setting

Model development was performed in data from University Hospitals Birmingham (UHB). External validation was performed in the CovidCollab dataset.

Participants

Patients with COVID-19 admitted to UHB January-August 2020 were included.

Main outcome measures

Death and ITU admission within 28 days of admission.

Results

1040 patients with COVID-19 were included in the derivation cohort; 288 (28%) died and 183 (18%) were admitted to ITU within 28 days of admission. Area under the receiver operating curve (AUROC) for mortality was 0.791 (95%CI 0.761-0.822) in UHB and 0.767 (95%CI 0.754-0.780) in CovidCollab; AUROC for ITU admission was 0.906 (95%CI 0.883-0.929) in UHB and 0.811 (95%CI 0.795-0.828) in CovidCollab. Models showed good calibration. Addition of comorbidities to candidate predictors did not improve model performance. AUROC for the 4C score in the UHB dataset was 0.754 (95%CI 0.721-0.786).

Conclusions

The novel prognostic models showed good discrimination and calibration in derivation and external validation datasets, and outperformed the existing 4C score. The models can be integrated into electronic medical records systems to calculate each individual patient’s probability of death or ITU admission at the time of hospital admission. Implementation of the models and clinical utility should be evaluated.

Article Summary

Strengths and limitations of this study

We developed novel prognostic models predicting mortality and ITU admission within 28 days of admission for patients hospitalised with COVID-19, using a large routinely collected dataset gathered at admission with a wide range of possible predictors (demographic variables, symptoms, physiological measures, imaging, laboratory test results).

These novel models showed good discrimination and calibration in both derivation and external validation cohorts, and outperformed the existing ISARIC model and 4C score in the derivation dataset. We found that addition of comorbidities to the set of candidate predictors included in model derivation did not improve model performance.

If integrated into hospital electronic medical records systems, the model algorithms will provide a predicted probability of mortality or ITU admission for each patient based on their individual data at, or close to, the time of admission, which will support clinicians’ decision making with regard to appropriate patient care pathways and triage. This information might also assist clinicians in explaining complex prognostic assessments and decisions to patients and their relatives.

A limitation of the study was that in the external validation cohort we were unable to examine all of the predictors included in the original full UHB model due to only a reduced set of candidate predictors being available in CovidCollab. Nevertheless, the reduced model performed well and the results suggest it may be applicable in a wide range of datasets where only a reduced set of predictor variables is available.

Furthermore, it was not possible to carry out stratified analysis by ethnicity as the UHB dataset contained too few patients in most of the strata, and no ethnicity data was available in the CovidCollab dataset.