# Ensemble Deep Learning for the Detection of COVID-19 in Unbalanced Chest X-ray Dataset
## 4 Experimental Setting, Results and Discussions
### Experimental Results and Discussions

Table 6: Classification results of normal, pneumonia and COVID-19 using cropped lungs and 11 CNNs with Image Augmentation (Approach 2). \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Deep CNNs** & **Accuracy** & **Sensitivity** & **Specificity** & **AUC** & **F-Measure** \\ \hline XceptionNet & 97.80\% & 97.93\% & 97.87\% & 99.77\% & 96.075 \\ InceptionV3 & 96.63\% & 93.13\% & 94.13\% & 99.37\% & 93.93\% \\ VGG-16 & 95.57\% & 95.50\% & 96.77\% & 99.07\% & 92.27\% \\ VGG-19 & 94.67\% & 93.77\% & 95.37\% & 98.70\% & 90.60\% \\ ResNet50 & 97.77\% & 97.80\% & 98.13\% & 99.83\% & 96.33\% \\ ResNet152 & 96.67\% & 96.67\% & 97.60\% & 99.60\% & 93.43\% \\ MobileNetV2 & 96.33\% & 94.97\% & 96.07\% & 99.30\% & 91.63\% \\ DenseNet201 & 96.33\% & 97.17\% & 96.73\% & 99.57\% & 93.87\% \\ InceptionResNetV2 & 96.77\% & 96.67\% & 97.17\% & 98.50\% & 94.73\% \\ EfficientNetB7 & 97.57\% & 95.67\% & 96.20\% & 99.57\% & 95.83\% \\ NasNetMobile & 97.40\% & 95.47\% & 97.17\% & 99.50\% & 95.50\% \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Classification results of normal and pneumonia and COVID-19 using cropped lungs and 11 CNNs with Undersampling (Approach 3). \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Deep CNNs** & **Accuracy** & **Sensitivity** & **Specificity** & **AUC** & **F-Measure** \\ \hline XceptionNet & 98.63\% & 97.57\% & 98.23\% & 99.90\% & 97.47\% \\ InceptionV3 & 96.27\% & 97.57\% & 96.43\% & 99.53\% & 93.43\% \\ VGG-16 & 96.27\% & 96.20\% & 96.83\% & 99.37\% & 93.73\% \\ VGG-19 & 92.93\% & 94.20\% & 95.00\% & 98.70\% & 88.77\% \\ ResNet50 & 97.77\% & 97.27\% & 97.70\% & 99.63\% & 96.17\% \\ ResNet152 & 97.37\% & 97.77\% & 97.73\% & 99.77\% & 95.43\% \\ MobileNetV2 & 97.90\% & 96.10\% & 97.40\% & 99.77\% & 95.83\% \\ DenseNet201 & 98.20\% & 98.07\% & 97.23\% & 99.77\% & 97.27\% \\ InceptionResNetV2 & 98.67\% & 98.17\% & 98.23\% & 99.90\% & 97.63\% \\ EfficientNetB7 & 89.03\% & 90.00\% & 92.07\% & 98.97\% & 84.90\% \\ NasNetMobile & 97.80\% & 98.03\% & 99.83\% & 99.83\% & 96.17\% \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Classification results of normal, pneumonia and COVID-19 using cropped lungs and 11 CNNs with Oversampling (Approach 4). \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Deep CNNs** & **Voting Strategy** & **Accuracy** & **Sensitivity** & **Specificity** & **AUC** & **F-Measure** \\ \hline Ensemble of & Soft Voting & 97.93\% & 98.43\% & 98.40\% & 99.90\% & 96.70\% \\ Approach 0 & Hard Voting & 96.63\% & 97.97\% & 98.44\% & N/A & 96.62\% \\ \hline Ensemble of & Soft Voting & 99.03\% & 98.97\% & 99.20\% & 99.97\% & 98.23\% \\ Approach 1 & Hard Voting & 98.35\% & 98.66\% & 99.11\% & N/A & 97.88\% \\ \hline Ensemble of & Soft Voting & 99.23\% & 99.27\% & 99.27\% & 99.97\% & 98.30\% \\ Approach 2 & Hard Voting & 98.84\% & 99.05\% & 99.38\% & N/A & 98.41\% \\ \hline Ensemble of & Soft Voting & 98.00\% & 98.17\% & 98.37\% & 99.87\% & 96.53\% \\ Approach 3 & Hard Voting & 96.53\% & 97.63\% & 98.33\% & N/A & 96.13\% \\ \hline Ensemble of & Soft Voting & 99.17\% & 98.87\% & 98.80\% & 99.93\% & 98.57\% \\ Approach 4 & Hard Voting & 98.61\% & 98.28\% & 98.90\% & N/A & 98.43\% \\ \hline Ensemble of & Soft Voting & 98.00\% & 99.33\% & 95.93\% & 99.90\% & 96.63\% \\ Approach 5 & Hard Voting & 98.75\% & 98.41\% & 99.07\% & N/A & 98.59\% \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Ensemble of top five models from each approach using majority soft voting and hard voting. Figure 16: ROC curves (true positive rate vs. false positive rate) of each ensemble classifier.