VALIDATION OF A SEVERITY SCORE TO IDENTIFY PATIENTS ADMITTED FOR COVID-19 PNEUMONIA AT HIGH RISK FOR AN INTENSIVE APPROACH
Conditions: COVID, Pneumonia
Sponsor: Fondazione Policlinico Universitario Agostino Gemelli IRCCS

 
 
Versione del 17 aprile 2020  
1 
 
 
 
 
VALIDATION OF A SEVERITY SCORE TO IDENTIFY PATIENTS 
ADMITTED FOR COVID-19 PNEUMONIA AT HIGH RISK FOR AN 
INTENSIVE APPROACH  
 
 
Versione del 17 aprile 2020 
 
 
Massimo Montalto, Antonella Gallo, Enrica Porceddu, Antonio Liguori, Erika Pero, Simona 
Panunzi, Geltrude Mingrone, Antonio Gasbarrini, Raffaele Landolfi,  
Unità di Ricerca Covid, Fondazione Policlinico Gemelli, IRCCS,  
Catholic University Rome, Italy 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
Versione del 17 aprile 2020  
2 
 
 
 
 
 
INTRODUCTION 
 
The outbreak of  the coronavirus disease 2019 (COVID-19), first merged in China in December 2019,  
is now becoming a  Public Health Emergency, recently confirmed as a pandemic disease by the World 
Health Organization (WHO) (1).  It has been estimated that the worldwide number of confirmed cases 
nearly amount to 1 million with more than 40000 deaths (1).  
In particular, since February 2020, a rapidly growing number of cases has been identified in Italy; 
current data reveal a number of infected over 110000 subjects, with more than 13000 deaths and 4000 
hospitalization in intensive care units (2).   
The clinical picture of ranges from asymptomatic cases, mild upper respiratory tract infections to 
severe pneumonia with respiratory failure and death (3). In most severe cases, COVID-19 disease 
may be complicated by acute respiratory distress syndrome (ARDS), septic shock and multiorgan 
failure (4). 
Since now, more than 30 agents have been tested for efficacy against SARS-CoV-2 and safety, with 
promising but still not conclusive evidences (5).  
Therefore, it results fundamental to early identify those subjects who rapidly may worsen their clinical 
status, often requiring an intensive care unit (ICU) admission.  
 
It has been showed that, mainly in more severe forms of SARS-Cov-2 disease, there is the 
development of an hyperinflammatory status resembling a cytokine storm syndrome (6), as already 
reported in SARS patients (7).   
A recent study by Haung et al. reported that patients with COVID-19 infection showed high amounts 
of IL1B, IFN-gamma, IP10 and MCP1, probably linked to activated T-helper1 (Th1) cell responses. 

 
 
Versione del 17 aprile 2020  
3 
 
Those requiring ICU admission had higher levels of cytokines than those subjects not requiring ICU 
admission, thus suggesting that cytokine storm was associated with disease severity (6).  
A similarity between cytokine profile of COVID-19 disease and secondary haemophagocytic 
syndrome (sHLH) has been reported (4). Therefore, it was suggested to screen all patients with severe 
COVID-19 infection both for hyperinflammatory markers (like ferritin), and the HScore commonly 
used to generate a probability for diagnosis of sHLH (8), which includes some laboratory parameters 
like triglycerides, fibrinogen, ferritin,  serum aspartate aminostransferase (8). 
A recent retrospective analysis on 487 COVID-19 Chinese patients admitted to Hospital, showed that  
the 49 (10.1%) severe cases at admission, was elderly, more commonly male and had a higher 
incidence of hypertension. Therefore, the Authors defined a host risk score on the basis of the three 
risk factors, aimed to assess the intrinsic host susceptibility to develop severe cases of COVID-19. 
This score was also validated in 15 out of 66 patients with a mild to moderate presentation of disease 
at admission, who showed a progression of disease during hospital stay within a median follow-up 
time of 15 days (9).  
Another work by Zhou et al. on 191 Chinese COVID-19 patients showed that older age, D-dimer 
levels greater than 1 μg/mL, and higher SOFA score on admission were associated with higher odds 
of in-hospital death (10). Moreover, subjects with most severe forms of COVID-19 illness commonly 
showed high levels of blood IL-6, high-sensitivity cardiac troponin I, lactate dehydrogenase and 
lymphopenia (10).  
Based on our previous experience with a large number of patients affected by pneumonia from 
Covid19, we have observed that those subjects with a more severe prognosis might have some 
predictive markers. We intend to verify if these markers can identify those subjects with Covid19 
infection who need a more intensive therapy and to find a prognosis score. 
 
 
 

 
 
Versione del 17 aprile 2020  
4 
 
 
 
 
 
Methods 
 
Before starting the study, the protocol will be submitted to and approved by the local Ethical 
Committees at the Fondazione Policlinico Universitario A. Gemelli IRCCS, Catholic University, 
Rome, Italy. Before enrollment each subject will sign the informed consent. 
Inclusion criteria: subjects of both sexes aged 18 years or older consecutively hospitalized with 
diagnosis of pneumonia, confirmed by chest imaging, Covid-19 test positive, given informed consent 
to data collection from the patient. 
Exclusion criteria: age lower than 18 years, pregnancy or breast-feeding. 
Nasopharyngeal swab samples will be taken for quantitative real-time polymerase chain reaction to 
make diagnosis of Covid19 (2 repeated tests). 
Data collected include time of type of symptoms (cough, fever, dyspnea, conjunctivitis, diarrhea, 
asthenia, arthralgia) and duration, comorbidities (presence of diabetes, hypertension, cardiovascular 
disease),  PaO2/FiO2 ratio, age, sex, height, weight, triglycerides, complete blood count, D-dimer, 
lactic acid dehydrogenase (LDH), high-sensitivity C-reactive protein (hs-CRP), albumin, and ferritin.    
The duration of the follow-up will be 28 days from the patient’s hospitalization or until his/her 
discharge or death. 
Primary end-point  
The primary end-point is to identify the best predictors of critical coronavirus pneumonia and to 
realize a simple severity score able to early classify high-risk individuals admitted to Internal 
Medicine Department for COVID-19 disease, needing an intensive approach. 
 
 

 
 
Versione del 17 aprile 2020  
5 
 
The evaluated predictors for the score assessment are the following: symptoms duration, type of 
symptoms (cough, fever and dyspnea), comorbidities (presence of diabetes, hypertension, ischemic 
heart disease), PaO2/FiO2 ratio, age, sex, body mass index (BMI), triglycerides, white cell count, 
lymphocyte count, D-dimer, lactic acid dehydrogenase (LDH), high-sensitivity C-reactive protein 
(hs-CRP), albumin, and ferritin. These parameters were collected at admission.  
 
Sample size calculation 
The sample size determination is based on the following considerations:  
1) 
the objective of the study is that of finding the best predictors of severe coronavirus pneumonia 
outcome; 
2) 
the study sample will be constituted by inpatients cases of COVID-19 admitted to Internal 
Medicine Department of Fondazione Policlinico A. Gemelli IRCCS; 
3) 
in order to test the predictive ability of the developed score the study sample will be split into 
a fitting population and a validation population by using the internal/validation/data-splitting method  
4) 
the following rule-of-thumb will be considered: the power for a test of a medium-sized partial 
correlation between y and a predictor, holding all other predictors constant, is approximately .80 if N 
= 104 + m, where m is the number of predictors (11). 
 
Starting from the above considerations, and from the hypothesis that about 20 predictors will be 
evaluated for development of a risk score able to predict the severe coronavirus pneumonia outcome, 
the population size of the fitting population is set to N = 104+20=124 patients. In the procedure of 
splitting the data set into a fitting population and a validation population, the 50% of the entire sample 
will be used for the fitting step and the remaining 50% will be used as a validation population. A 
sample size of 248 patients will be therefore necessary. 
 
Statistical Analysis  
Primary Analysis 
The data set will be split into a fitting population and a validation population by using the 
internal/validation/data-splitting method. The internal/validation/data-splitting method is chosen to 
limit the possible increase in bias that can occur with other techniques for internal validation such as 

 
 
Versione del 17 aprile 2020  
6 
 
Bootstrapping technique if the original study population contains biases. The portion of the 
population used as fitting population for model development will be chosen randomly. The model 
will be then tested, in terms of its performance on the remaining validation population. Computation 
of the C statistic will be used to indicate the discrimination performance of the model. The C statistic 
represents a measure of the model's performance in terms of the model discrimination ability to 
separate subjects with different outcomes, and for the logistic model, it is equivalent to the area under 
the curve of the receiver operating characteristic (ROC) curve associated to the model (12) Values of 
C between 0.8 and 0.9 will be considered to give an excellent discrimination.  
Computation of the probabilities: For each subject, both from the fitting population and the validation 
population, the probability of the studied outcome will be computed on the basis of the patient's 
covariate values. Each subject will be therefore characterized by the values of the covariate set, the 
outcome (coronavirus-severity, 0/1) and by the estimated probability of the outcome. 
Computation of the C statistics: For both the fitting and the validation populations, the C statistics 
will be computed. The accuracy (percentage of true positive and negative on the total subjects) of the 
prediction model will be also computed. 
Procedure repetition: The entire procedure (data splitting, model fitting, probability computation) will 
be repeated 200 times. 
Model fitting onto the whole population: The model will then be fitted using all the original data. 
Results analysis: At the end of the procedure, the distribution of different indicators will be computed: 
for each variable, frequency histograms of the estimated coefficients along with their associated 
standard errors and levels of significance will be plotted. The importance index of each variable 
computed from each data splitting and model fitting will be also plotted. 
The estimated coefficients from the whole population will be compared with the respective 
distributions (they should be centred around the estimates obtained on the whole sample); the 
distribution of the variability of the model coefficients indicates to which extent the results are 
dependent from the subpopulation used; comparison between the distribution of the C statistic 
computed on the fitting and the validation sample, along with the distribution of their Delta provide 
indication of the discrimination power of the model. It is expected that the C statistic is larger for the 
fitting sample than for the validation sample. A model with a high discrimination power presents 
small drops in the values of the C statistic when computed on the validation sample with respect to 

 
 
Versione del 17 aprile 2020  
7 
 
values computed on the fitting sample for which the values are expected to be larger (C Delta 
distribution centred around the zero). 
On the basis of the obtained results on the full model, the entire procedure is repeated on the subset 
of the most important variables to test and validate the final reduced model including the most 
meaningful predictors. Once the model validation is assessed the fitted coefficients used to compute 
the probability of the outcome on a generic patient will be those obtained from the “reduced model” 
fitting on the whole population sample. 
Secondary analyses 
Secondary analyses include the use of other classification procedures to compare results from the 
primary analysis. One of the classification procedures that will be used to classify patients with severe 
and non-severe coronavirus pneumonia outcome, on the basis of the collected variables, will be the 
Partial Least-Squares Discriminant Analysis (PLS-DA). A cross validation approach will be followed 
and the optimal model with the optimal number of PLS components will be derived. The Q2 value, 
the ROC AUC and the derived misclassification error, by the classification Youden criterium, will be 
used to quantify the classification goodness. A permutation approach, including 1000 random 
permutations, will be used in order to associate a P level to the obtained Q2 value. Random forest 
classification procedure, which is based on learning algorithms, will be also tested following the same 
procedure of splitting the entire sample into a training set and a test set. Univariate analyses studying 
the association between severity and predictors will be conducting using Chi-Squared tests or Fisher 
exact tests when appropriate or univariate logistic regression models for quantitative variables. 
Secondary analyses will have only an explorative purpose.  
 
REFERENCES 
 
1) Remuzzi A, Remuzzi G. COVID-19 and Italy: what next? Lancet 2020 Mar 13 (doi: 
10.1016/S0140-6736(20)30627-9) 
2) Livingston E, Bucher K. Coronavirus Disease 2019 (COVID-19) in Italy. JAMA, March 17, 2020 
(doi: 10.1001/jama.2020.4344) 

 
 
Versione del 17 aprile 2020  
8 
 
3) Wang D, Hu B, Hu C, et al. Clinical characteristics of 138 hospitalized patients with 2019 novel 
coronavirus-infected pneumonia in Wuhan, China. JAMA 2020; published online Feb 7. 
DOI:10.1001/jama.2020.1585 
4) Yang X, Yu Y, Xu J et al. Clinical course and outcomes of critically ill patients with SARS-CoV-
2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study. Lancet Respir 
Med. 2020; (published online Feb 24) https://doi.org/10.1016/S2213-2600(20)30079-5 
5) Dong L et al. Discovering drugs to treat Coronavirs Disease 2019 (COVID-19). Drug Discov Ther 
2020 
6) Huang C, Wang Y, Li X et al. Clinical features of patients infected with 2019 novel coronavirus 
in Wuhan, China. Lancet 2020; 395: 497-506 
7)Womg CK, et al. Plasma inflammatory cytokines and chemokines in severe acute respiraoty 
syndrome. Clin Exp Immunol 2004;  
8) Fardet L, Galicier L, Lambotte O, et al. Development and validation of the HScore, a score for the 
diagnosis of reactive hemophagocytic syndrome. Arthritis Rheumatol 2014; 66: 2613-2620 
9) Shi YU, et al. Host susceptibility to severe COVID-19 and establishment of a host risk score: 
findings of 487 cases outside Wuhan. Crit Care 2020, 24:108 https://doi.org/10.1186/s13054-020-
2833-7 
10) Zhou F, Yu T, Du R et al. Clinical course and risk factors for mortality of adult inpatients with 
COVID-19 
in 
Wuhan, 
China: 
a 
retrospective 
cohort 
study. 
Lancet. 2020; https://doi.org/10.1016/S0140-6736(20)30566-3 
11) Green SB. How many subjects does it take to do a regression analysis. Multivariate Behav 
Res 1991;26:499–510.   
12) Harell FE. Regression Modeling Strategies: With Applications to Linear Models, Logistic 
Regression, and Survival Analysis. New York: Springer; 2001.