# Adaptive Stacking Ensemble Techniques for Early Severity Classification of COVID-19 Patients
## 3 Proposed Method
### Stacking Ensemble Model Construction
The trained classifiers are then evaluated using five-fold stratified cross-validation to measure their average F1 and AUC scores. Based on a predefined threshold, only \(k\) candidate classifiers with an F1 score or AUC score of 0.7 or above are selected for inclusion. Subsequently, the correlations between the predicted outcomes of the selected classifiers are calculated. To reduce model bias and create a diverse set of predictions, classifiers with low prediction correlations are chosen. Classifier combinations with a correlation coefficient exceeding 0.8 are excluded, and models with poor performance  
Figure 6: Model construction process of the proposed adaptive stacking ensemble technique. for certain pairs are removed. The equation used to calculate the correlation coefficient between two classifiers is defined as follows:  
\[M_{corr}(M_{x},M_{y})=\frac{\sum_{i=1}^{n}\left(M_{x}-\overline{M_{x}}\right) \left(M_{y}-\overline{M_{y}}\right)}{\sqrt{\sum_{i=1}^{n}\left(M_{x}-\overline{ M_{x}}\right)^{2}\left(M_{y}-\overline{M_{y}}\right)^{2}}} \tag{3}\]  
where \(M_{x}\) and \(M_{y}\) represent the prediction outcomes from the individual classifiers \(x\) and \(y\), respectively, and \(\overline{M_{x}}\) and \(\overline{M_{y}}\) represent the mean values of the predicted outcomes of these classifiers. Algorithm 3 describes the procedure for selecting the candidate base classifiers. ```
0:\(x\) is the oversampled train dataset, \(y\) is the test dataset
0:\(N\) selected classifiers for composite stacking ensemble
```
1:define model dictionary \(M\) (set of \(m\) single classifiers)
2:initialize model's evaluation score \(F_{m}\) as a dictionary
3:initialize selected classifiers \(N\) as an empty list
4:for each model in \(M\)do:
5:\(tm\leftarrow\) train each model using dataset \(x\)
6:\(F1\_score\), \(AUC\_score\leftarrow\) evaluate \(tm\) with dataset \(y\) using 5-fold stratified cross-validation
7:\(F_{m}[model]\leftarrow\) (\(F1\_score\), \(AUC\_score\))
8:endfor
9:for each model, (\(F1\_score\), \(AUC\_score\)) in \(F_{m}\)do:
10:if either \(F1\) score or \(AUC\_score\geq\) threshold:
11: add model to select classifiers \(N\)
12:endfor
13:for each model \(M_{i}\) in \(N\)do:
14:for each model \(M_{j}\) in \(N\)do:
15:if model \(M_{i}\)!= model \(M_{j}\)
16:\(M_{corr}\left(M_{i},M_{j}\right)\leftarrow\) calculate correlation coefficient between \(M_{i}\) and \(M_{j}\)
17:if\(M_{corr}\left(M_{i},M_{j}\right)\geq\) corr_threshold:
18:if\(F_{m}[model\ M_{i}]>F_{m}[model\ M_{j}]\):
19: mark \(M_{j}\) for removal
20:else:
21: mark \(M_{i}\) for removal
22:endfor
23:endfor
24:remove marked models from selected classifiers \(N\)
25:return\(N\)
```

**Algorithm 3**Selection of candidate base classifiers

Our adaptive stacking ensemble model combines the individual base classifiers to learn the best possible prediction combination of various base classifiers and is structured as a multilayer stacking ensemble model that outperforms the individual models. In the _'model construction module'_, the \(N\) candidate base classifiers selected from the previous module are first sorted in descending order based on their F1 and AUC scores. The training and testing datasets are split to facilitate the construction of a multilayered structure. A greedy algorithm utilizing an internal function called the _train_layer_ is used to progressively add the classifiers based on their performance and evaluate the efficiency of each combination. If adding more classifiers does not enhance the performance, the current combination is considered the optimal single-layer configuration and the process moves to the next layer. Thus, the classifier combinations in the previous layer serve as inputs to the next layer. This iterative process of adding classifiers and layers is continued until no further performance improvement is observed. Due to the expected high complexity of the multilayer stack ensemble model, Logistic Regression was chosen as the meta-learner to minimize the risk of data overfitting.