# Adaptive Stacking Ensemble Techniques for Early Severity Classification of COVID-19 Patients
## 3 Proposed Method
### Stacking Ensemble Model Construction
All candidate base classifier models were structured in a two-layer architecture; RF, CAT, and LGBM were frequently selected. Additionally, for Groups 3 and 5, which included underlying diseases, the first layer consisted of three models. \begin{table}
\begin{tabular}{c c c} \hline
**Type** & **Classifier Name** & **Parameter Setting** \\ \hline \multirow{2}{*}{Linear} & Logistic Regression (LR) & C: 0.01, solver: newton-cg, Max\_iter: 1000, Penalty: 12 \\ \cline{2-3}  & Support Vector Machine 1 (SVM 1) & Kernel: Linear, C: 0.025 \\ \cline{2-3}  & Stochastic Gradient Descent (SGD) & Penalty: 12, Max\_iter: 1000 \\ \hline \multirow{3}{*}{Non-linear} & Support Vector Machine 2 (SVM2) & Kernel: RBF, Gamma: 2, C: 1 \\ \cline{2-3}  & Multi-Layer Perceptron (MLP) & N\_layers: 5 (including input, 3 FC hidden, and output layers) \\ \cline{2-3}  & Hidden\_layer\_size: (100), alpha: 1.0, Max\_iter: 1000 \\ \hline Distance-based & K-Nearest Neighbor (KNN) & N\_neighbors: 2, weight: distance \\ \hline Probabilistic & Gaussian Naive Bayes (GNB) & Var\_smoothing: 1 \(\times\) 10\({}^{-9}\) \\ \hline \multirow{3}{*}{Tree-based} & Extra Trees Classifier (ExTree) & Criterion: entropy, Max\_depth: 20, \\ \cline{2-3}  & Decision Tree (DT) & Min\_sample\_split: 2 \\ \hline \multirow{3}{*}{Ensemble} & XGBoost (XGB) & N\_estimators: 340, Max\_depth: 2, \\  & Learning\_rate: 0.0628, Gamma: 1.17 \\ \cline{2-3}  & Light Gradient Boosting Machine & N\_estimators: 440, Max\_depth: 10, \\  & (LGBM) & Learning\_rate: 0.0496, Max\_depth: 6 \\ \cline{2-3}  & AdaBoost (ADA) & N\_estimators: 320, Learning\_rate: 0.0726 \\ \cline{2-3}  & Random Forest (RF) & N\_estimators: 600, Max\_depth: 8, \\  & N\_jobs: -1, Min\_sample\_split: 2 \\ \cline{2-3}  & CatBoost (CAT) & Learning\_rate: 0.7462 \\ \hline \end{tabular}
\end{table} 
Table 4: Candidate base classifiers and hyperparameter settings. Figure 7: Optimal combinations of base classifiers in the adaptive stacking ensemble models for CSS Type 1. Figure 8 shows the structure of Type 2 prediction, which encompasses Groups 1-5 and predicts three severity classes. This type expands the prediction scope when compared with that of Type 1, with three severity classes instead of two. For the optimal com-bination of base classifiers, 7 out of 14 classifiers (i.e., LGBM, ADA, RF, GNB, SVM2, SGD, and DT) were selected for Type 2. Except for the Group 1 model, which had fewer features (i.e., 9 clinical variables), the majority of the Type 2 models comprised the first layer. LGBM and RF were frequently selected as the base classifiers, whereas the remaining base classifiers were evenly incorporated into the combinations. Figure 9 depicts the structure of Type 3 prediction, which is composed of Groups 1-5 and predicts five severity classes. Considering the complexity of predicting the five severity classes, the base classifiers were structured with three layers for Groups 1-3, whereas Groups 4 and 5, which included the blood test results, were structured with two layers. For the optimal combination of the base classifiers, 7 out of 14 classifiers (i.e., LGBM, GNB, LR, MLP, RF, ADA, and SVM2) were selected. MLP emerged as the most frequently selected base classifier across all group combinations for Type 3, and LGBM and LR were utilized in all groups except Group 3. ## 4 Results

In this section, we present a detailed description of the performance evaluation of our adaptive stacking ensemble technique. We first introduce the metrics used to evaluate our

Figure 8: Optimal combinations of base classifiers in the adaptive stacking ensemble models for CSS Type 2. Figure 9: Optimal combinations of base classifiers in the adaptive stacking ensemble models for CSS Type 3.

technique and discuss the results of the performance evaluations. The performance of our technique was evaluated through three experiments: (1) the performance of the prediction models, (2) the effectiveness of the data-driven strategy and oversampling, and (3) the effect of the feature selection algorithm on the performance.