# Adaptive Stacking Ensemble Techniques for Early Severity Classification of COVID-19 Patients
## 3 Proposed Method
### Stacking Ensemble Model Construction
\begin{table}
\begin{tabular}{c c c c c c c} \hline  & \multicolumn{4}{c}{**Before**} & \multicolumn{4}{c}{**After Oversampling**} \\ \multirow{2}{*}{**Type**} & \multirow{2}{*}{**Model**} & \multirow{2}{*}{**F1 Score**} & \multirow{2}{*}{**AUC**} & \multicolumn{2}{c}{**SMOTE**} & \multicolumn{2}{c}{**ADASYN**} \\ \cline{5-8}  & & & & & **F1 Score** & **AUC** & **F1 Score** & **AUC** \\ \hline \multirow{8}{*}{Type 1} & Logistic Regression (LR) & 0.8739 & 0.9128 & 0.9059 & 0.9562 & 0.9166 & 0.9407 \\ \cline{2-8}  & XGBoost (XGB) & 0.9182 & 0.9163 & 0.9340 & 0.9348 & 0.9454 & 0.9460 \\ \cline{2-8}  & PyCaret (CatBoost (CAT)) & 0.9071 & 0.8962 & 0.9183 & 0.9011 & 0.9246 & 0.9024 \\ \cline{2-8}  & H2O.ai (Stacking: 1 st layer: 1 GBM, 1 GLM, 2 DRF, Meta: GLM) & 0.8626 & 0.8126 & 0.9031 & 0.8387 & 0.9267 & 0.8433 \\ \cline{2-8}  & \multicolumn{4}{c}{**Proposed Model (Group 5-CSS Type 1)**} & \multirow{2}{*}{(Stacking: 1 st layer: RF, MLP, SGD 2 nd layer: RF, SGD, Meta: LR)} & **0.9315** & **0.9336** & **0.9623** & **0.9853** & **0.9904** & **0.9877** \\ \hline \multirow{8}{*}{Type 2} & Logistic Regression (LR) & 0.7734 & 0.8316 & 0.8046 & 0.8599 & 0.8136 & 0.8654 \\ \cline{2-8}  & XGBoost (XGB) & 0.8590 & 0.8925 & 0.8594 & 0.9052 & 0.8682 & 0.9062 \\ \cline{2-8}  & PyCaret (CatBoost (CAT)) & 0.8192 & 0.8214 & 0.8199 & 0.8214 & 0.8210 & 0.8295 \\ \cline{2-8}  & H2O.ai (Generalized Linear Modeling (GLM)) & 0.8013 & 0.7928 & 0.8312 & 0.8045 & 0.8339 & 0.8084 \\ \cline{2-8}  & \multicolumn{4}{c}{**Proposed Model (Group 5-CSS Type 2)**} & \multirow{2}{*}{(Stacking: 1 st layer: LGBM, DT, SGD, Meta: LR)} & **0.8972** & **0.8957** & **0.9158** & **0.9112** & **0.9318** & **0.9121** \\ \hline \multirow{8}{*}{Type 3 (Group 5)} & Logistic Regression (LR) & 0.6211 & 0.7724 & 0.6514 & 0.7935 & 0.6967 & 0.8079 \\ \cline{2-8}  & XGBoost (XGB) & 0.7882 & 0.8325 & 0.7892 & 0.8527 & 0.7905 & 0.8658 \\ \cline{2-8}  & PyCaret (CatBoost (CAT)) & 0.7495 & 0.7312 & 0.7503 & 0.7341 & 0.7543 & 0.7362 \\ \cline{2-8}  & H2O.ai (Gradient Boosting Machine (GBM)) & 0.7366 & 0.7097 & 0.7399 & 0.7113 & 0.7451 & 0.7187 \\ \cline{2-8}  & \multicolumn{4}{c}{**Proposed Model (Group 5-CSS Type 3)**} & \multirow{2}{*}{(Stacking: 1 st layer: LGBM, SVM, 2 nd layer: MLP, LR, Meta: LR)} & **0.7921** & **0.8572** & **0.8524** & **0.8811** & **0.8754** & **0.9115** \\ \hline \end{tabular}
\end{table} 
Table 13: Performance results before and after applying oversampling. Figure 11: Comparison of performances before and after applying oversampling. the AUC score improved by 5.17% with SMOTE and 5.41% with ADASYN. For Type 2, the improvements were more moderate than those for Type 1. The F1 score increased by 1.86% for SMOTE and 3.46% for ADASYN, whereas the AUC score increased by 1.55% for SMOTE and 1.64% for ADASYN. Finally, for Type 3, there was a notable increase in the F1 score by 5.33% with SMOTE and 7.63% with ADASYN, and the AUC score increased by 3.39% and 6.43%, respectively. In the case of XGBoost, PyCaret (CatBoost), and H2O.ai (GBM), oversampling did not lead to significant improvements across any of the types. This lack of improvement was attributed to the nature of the boosting algorithms. For instance, XGBoost and CatBoost construct new trees by correcting the errors in the previously created, weaker decision trees. Due to this approach, the oversampled data did not significantly contribute to the results. Consequently, the models in Type 2, especially those using boosting ensemble models such as LGBM and RF, as well as those utilizing AutoML approaches based on boosting techniques, did not show substantial performance improvements after oversampling. However, the proposed combination of models in Type 3, which included a variety of models such as MLP and SVM, demonstrated the most remarkable performance improvement.