# Adaptive Stacking Ensemble Techniques for Early Severity Classification of COVID-19 Patients
## 3 Proposed Method
### Stacking Ensemble Model Construction
Unlike traditional methods, where the user manually selects the classifiers that will be integrated into the stacking ensemble, our approach automatically selects the base classifiers using a greedy algorithm. Algorithm 4 describes the procedure for constructing the adaptive stacking ensemble model. ```
0:\(N\) selected classifiers for composite stacking ensemble, \(x\) is the oversampled train dataset, \(y\) is the test dataset
0:Final F1_score final_f1
```
1:define model dictionary \(N\) (set of selected candidate base classifiers)
2:initialize sort models by F1 and AUC scores \(S_{m}\)
3:initialize last best F1 score lbf1 as 0.0
4:initialize\(X_{next\_layer\_train}\) as \(X_{train}\)
5:initialize\(X_{next\_layer\_test}\) as \(X_{test}\)
6:\(S_{m}\leftarrow\)sort \(N\) selected candidate classifiers by \(F1\_score\) and AUC_score
7:randomly split train dataset \(x\) and test dataset \(y\) into (\(X_{train}\), \(Y_{test}\)), (\(X_{train}\), \(Y_{test}\))
8:function train_layer (Models, \(X_{train}\), \(Y_{train}\)):
9:initialize layer_output as empty list
10:for each model in Models do
11:\(tm\leftarrow\) train each model using datasets \(X_{train}\) and \(Y_{train}\)
12: add \(tm\) to layer_output
13:endfor
14:return layer_output
15:endfunction
16:initialize patience_counter as 0
17:while true:
18:initialize current best models \(cbm\) as an empty list
19:initialize current best f1 \(cbf1\) as lbf1
20:initialize interim models \(im\) as an empty list
21:for each model in \(S_{m}\)do:
22: add model to \(im\)
23:\(interim\_X_{train}\leftarrow\) call function train_layer (\(tm\), \(X_{next\_layer\_train}\), \(Y_{train}\))
24:initialize meta_model as LogisticRegression
25: train _meta_model using \(interim\_X_{train}\) and \(Y_{train}\)
26:\(interim\_f1\leftarrow\)evaluate meta_model using \(X_{next\_layer\_test}\) and \(Y_{test}\)
27:if interim_f1 > cbf1:
28:\(cbf1\leftarrow\) interim_f1
29:\(cbm\leftarrow\) copy interim models \(im\)
30:endfor
31:if cbf1 \(\leq\) lbf1:
32:\(patience\_counter\) += 1
33:if patience_counter \(\geq\) max_patience:
34: break
35:else:
36: lbf1 = cbf1
37:\(X_{next\_layer\_train}\leftarrow\) call function train_layer (\(cbm\), \(X_{next\_layer\_train}\), \(Y_{train}\))
38:\(X_{next\_layer\_test}\leftarrow\) call function train_layer (\(X_{next\_layer\_test}\), \(Y_{test}\))
39:\(patience\_counter\) to 0
40:endwhile
41:initialize final_meta_model as LogisticRegression
42: train final_meta_model with \(X_{next\_layer\_train}\) and \(Y_{train}\)
43:final_f1 \(\leftarrow\) evaluate final meta_model using \(X_{next\_layer\_test}\) and \(Y_{test}\),
44:return final_f1 ```

**Algorithm 4** Adaptive stacking ensemble model construction

We constructed an adaptive stacking model using 14 candidate base classifiers, including linear, nonlinear, distance-based, probabilistic, tree-based, and ensemble techniques. To effectively calibrate the parameters and ensure the optimal selection of hyperparameters for each classifier, we employed GridSearchCV in our hyperparameter tuning process. We systematically explored a wide range of hyperparameter combinations for each classifier,within the context of a five-fold cross-validation process repeated 10 times. Table 4 lists the 14 classifiers we utilized, along with their respective hyperparameter settings. As depicted in Figures 7-9, the adaptive stacking ensemble learning process selects a subset of suitable classifiers from 14 base classifiers. This selection is based on a data-driven strategy defined by the groups of clinical variables for severity classification and the types of severity classes (i.e., \(CSS\)). To ensure robust performance, the selected classifiers were structured in a multi-layered architecture for prediction. Figure 7 shows the structure of the Type 1 prediction, which comprises Groups 1-5 and predicts only two severity classes using the data-driven strategy. For the optimal combination of base classifiers, 8 out of 14 base classifiers (i.e., LGBM, XGB, DT, CAT, LR, RF, GNB, and MLP) were selected.