# Ensemble Deep Learning for the Detection of COVID-19 in Unbalanced Chest X-ray Dataset
## 4 Experimental Setting, Results and Discussions
### Experimental Results and Discussions
\begin{table}
\begin{tabular}{p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt}} \hline \hline
**Authors** & **Classification Methods** & **Classification Strategy** & **Data** & **Performance** \\ \hline \multirow{4}{*}{[PERSON] et al. [16]} & & Two staged binary classification: & (i) Healthy: 3520 lung diseases: 3003 & Accuracy 96\% \\  & VGG16 based transfer learning & & & \\ \cline{3-5}  & & (i) healthy and lung diseases & (ii) COVID-19 & 250 lung diseases: 2753 & Accuracy 98\% \\  & & (ii) & COVID-19 and lung diseases & & \\ \cline{3-5}  & & & diseases & \\ \hline \multirow{4}{*}{[PERSON] et al. [17]} & A custom deep learning model with multiple feature extraction layers & Multi-classification of normal, COVID-19 and pneumonia & COVID-19: 238 Normal: 8851 Pneumonia: 6045 & Accuracy 97.48\% \\ \cline{3-5}  & & Step by step binary classification: & Normal: 558 Abnormal: 558 & Accuracy 98\% \\ \cline{3-5}  & Dee learning based decision tree classifier & (i) Normal and abnormal & TB: 492 non TB: 492 & Accuracy 80\% \\  & & (ii) TB or non TB & COVID-19 & COVID-19: 142 TB: 142 & Accuracy 95\% \\ \hline \multirow{4}{*}{[PERSON] et al. [19]} & & Binary classification of non-findings and COVID-19 & COVID-19: 125 Normal: 500 & Accuracy 98.08\% \\ \cline{3-5}  & & Multi classification of normal, COVID-19 and pneumonia & COVID-19 and Pneumonia: 500 & Accuracy 87.02\% \\ \hline \multirow{4}{*}{[PERSON] et al. [20]} & 16 state-of-the-art CNNs model and ensemble of those models using voting & Multi-classification of normal, COVID-19, and pneumonia & COVID-19: 237 Normal: 1338 Pneumonia: 1336 & Accuracy 99.31 \% \\ \cline{3-5}  & & Three-staged classification & Normal: 526 Pneumonia: 605 TB: 382 & Accuracy 95.9\% \\ \cline{3-5}  & & & TB & COVID-19: 735 Pneumonia: 650 & Accuracy 98.9\% \\ \cline{3-5}  & & (ii) COVID-19 and non COVID-19 pneumonia & Midi: 80 Security76 & Accuracy 76\% \\ \hline \multirow{4}{*}{[PERSON] et al. [22]} & A patch-based ResNet18 model & Multi-classification of normal, COVID-19, pneumonia and TB & COVID-19: 180 Normal: 191 TB: 57 & Sensitivity 92.5\% \\ \cline{3-5}  & Ensemble of iteratively pruned deep learning models & Multi-classification of normal, COVID-19 and pneumonia & COVID-19: 313 Normal: 7595 Pneumonia: 8792 & Accuracy 99.01\% \\ \cline{3-5}  & & Binary classification of non-findings and COVID-19 & AUC 82\% \\ \cline{3-5}  & & Multi classification of normal, pneumonia and TB & Pneumonia: 5689 & AUC 73.1\% \\ \hline \multirow{4}{*}{[PERSON] et al. [25]} & & Transfer learning of VGG-16 with a combination of data augmentation methods & Multi-classification of healthy, COVID-19 pneumonia, and non-COVID-19 pneumonia & COVID-19: 215 Healthy: 500 Pneumonia: 533 & Accuracy 83.6\% \\ \cline{3-5}  & & & non-COVID-19 pneumonia & Pneumonia: 533 & Accuracy 99.23\% \\ \hline \multirow{4}{*}{Our proposed study} & \(\bullet\) Here, 11 deep learning models with five different techniques to handle the imbalanced class distribution & Multi-classification of normal, COVID-19, and pneumonia & COVID-19: 3616 Pneumonia: 10,192 Pneumonia: 1345 & Accuracy 99.27\% \\ \cline{3-5}  & & Ensemble classifier of best-performing models using voting strategies & & \\ \hline \hline \end{tabular}
\end{table} 
Table 12: _Cont._tion problem and provided promising results, with 99.23% accuracy and a 99.97% AUC. However, there is more room for improvement, and some limitations, which should be targeted in future research. Most of the fundamental limitations arise from the dataset. The proposed models are developed and validated using the same public dataset. Characteristics of our dataset may vary from that of the real-time clinical data or other datasets. Although the proposed models achieved higher accuracy on the testing set of the related dataset, they might not provide sustained performance when externally validate using the data from different unrelated datasets. In this case, overfitting may have occurred in external validation.