Improving clinical management of COVID-19: the role of prediction models
Authors: Laure Wynants, Giovanni Sotgiu
Journal: The Lancet Respiratory Medicine (2021)

A year after the identification of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and its related disease, COVID-19, in the Chinese province of Hubei, one of the most difficult-to-manage modern health-care crises has unfolded worldwide. WHO declared COVID-19 a pandemic in March, 2020, and the epidemiological severity has since been shown by the high incidence of infections, critical cases, and deaths from the disease and, indirectly, by tragic socioeconomic disruption. The long wait to confirm the epidemiological effectiveness of immunisation against COVID-19, based on a confident estimation of vaccine efficacy and safety, should be accompanied by the retention of preventive interventions (ie, physical distancing, hand hygiene, face masks, ventilation), which have been the only effective measures available until now. In addition, improvements in the diagnostic and therapeutic management of patients with COVID-19 are needed, including rapid, accurate, point-of-care diagnostic platforms and new effective and safe antiviral and anti-inflammatory drugs. These improvements present a scientifically fascinating challenge given the current scenario of a poor therapeutic armamentarium and suboptimal sensitive molecular tests. Good clinical management further relies on accurate assessment of patients' prognoses, for which reliable predictive models can be a good scientific base. In The Lancet Respiratory Medicine, Gupta and colleagues describe the 4C Deterioration model, which can be used at hospital admission to estimate the risk of death or requirement for ventilatory support or critical care.1Gupta RK Harrison EM Ho A et al.Development and validation of the ISARIC 4C Deterioration model for adults hospitalised with COVID-19: a prospective cohort study.Lancet Respir Med. 2021; (published online Jan 11.)https://doi.org/10.1016/S2213-2600(20)30559-2Crossref Google Scholar The modelling effort by Gupta and colleagues follows an earlier study that tested 22 existing prognostic models and showed that none was of sufficient quality to inform the management of patients with COVID-19.2Gupta RK Marks M Samuels THA et al.Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: an observational cohort study.Eur Respir J. 2020; 562003498Crossref PubMed Scopus (124) Google Scholar Multivariable prognostic models combine and give appropriate weights to diverse prognostic factors to calculate risks for individual patients, and therefore usually outperform individual risk factors. Surprisingly, the tested models were not more clinically useful than oxygen saturation on room air alone to predict deterioration, or than age alone to predict mortality. The most likely causes for the disappointing predictions are serious methodological flaws and shortcomings of the model development studies, as flagged by a living systematic review of COVID-19 models.3Wynants L Van Calster B Collins GS et al.Prediction models for diagnosis and prognosis of COVID-19 infection: systematic review and critical appraisal.BMJ. 2020; 369m1328Crossref PubMed Scopus (1901) Google Scholar The 4C Deterioration model is of high quality compared with earlier models.2Gupta RK Marks M Samuels THA et al.Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: an observational cohort study.Eur Respir J. 2020; 562003498Crossref PubMed Scopus (124) Google Scholar, 3Wynants L Van Calster B Collins GS et al.Prediction models for diagnosis and prognosis of COVID-19 infection: systematic review and critical appraisal.BMJ. 2020; 369m1328Crossref PubMed Scopus (1901) Google Scholar A key strength is that the model has been developed and validated with a large, multicentre database of consecutive patients (over 66 000 patients for model development and 8000 patients for external validation), and that the authors transparently reported on their analysis process following the TRIPOD guidelines.4Moons KG Altman DG Reitsma JB et al.Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration.Ann Intern Med. 2015; 162: W1-W73Crossref PubMed Scopus (1644) Google Scholar The model's reported C-statistic of 0·77 indicates that it can discriminate at hospital admission between patients likely and unlikely to deteriorate. Crucially, the authors also illustrate that the model's predicted risks are well calibrated (eg, among 100 patients with a deterioration risk of 10%, ten will deteriorate),5Van Calster B Nieboer D Vergouwe Y De Cock B Pencina MJ Steyerberg EW A calibration hierarchy for risk models was defined: from utopia to empirical data.J Clin Epidemiol. 2016; 74: 167-176Summary Full Text Full Text PDF PubMed Scopus (415) Google Scholar and that the model has utility for decision making in clinical practice.6Van Calster B Wynants L Verbeek JFM et al.Reporting and interpreting decision curve analysis: a guide for investigators.Eur Urol. 2018; 74: 796-804Summary Full Text Full Text PDF PubMed Scopus (505) Google Scholar The model works well in each of nine National Health Service (NHS) regions across England, Wales, and Scotland after updating to accommodate regional differences.7Riley RD Ensor J Snell KI et al.External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis: opportunities and challenges.BMJ. 2016; 353i3140Crossref PubMed Scopus (308) Google Scholar The careful analysis and good results from external validation suggest that the 4C Deterioration model could be used to support clinicians in the decision to keep a patient in the hospital, admit a patient to critical care, or initiate therapy. However, care must be taken when interpreting the predicted risk: it does not reflect the risk of deterioration in the absence of such an intervention or the risk with an optimal intervention. Although it would be ideal, it is not straightforward to predict the risk of deterioration with and without intervention.8van Geloven N Swanson SA Ramspek CL et al.Prediction meets causal inference: the role of treatment in clinical prediction models.Eur J Epidemiol. 2020; 35: 619-630Crossref PubMed Scopus (43) Google Scholar Readers and users should keep in mind that the risks predicted by the 4C Deterioration model reflect the probability of deterioration of a patient receiving similar care to that of patients in the development cohort, who were hospitalised in the UK between February and August, 2020. It follows that the risk model is likely to need updates as changes in the infected population and care occur over time. In addition, differences with respect to patient case-mix and care might necessitate local updates before the model can be transported to another setting. Further external validation of the published risk equation in other countries should be done to assess the inter-country variability. The main clinical advantage of this predictive model is its predictors, which can be easily collected as part of daily routine care and inform stratification of patients on the basis of clinical severity. The 4C Deterioration and Mortality models could be combined and included in the programmatic standard of care adopted by hospitals to better identify the most appropriate clinical pathways for patients with COVID-19. Reliable predictive models can be a means to improve clinical management and, consequently, to better allocate human and economic resources. We declare no competing interests. Development and validation of the ISARIC 4C Deterioration model for adults hospitalised with COVID-19: a prospective cohort studyThe 4C Deterioration model has strong potential for clinical utility and generalisability to predict clinical deterioration and inform decision making among adults hospitalised with COVID-19. Full-Text PDF Open Access