# COVID-19 Diagnosis in Chest X-rays Using Deep Learning and Majority Voting
## 3 Results
### Discussion
Although the top-performing algorithm is different from \begin{table}
\begin{tabular}{c c c c} \hline \hline
**ALGORITHM USED** & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline EfficientNet-B0 & 0.88316 & 0.44681 & 0.91304 & 0.6 \\ \hline EfficientNet-B6 & 0.93471 & 0.6875 & 0.95652 & 0.8 \\ \hline EfficientNet-B4 & 0.94502 & 0.7931 & 1 & 0.88462 \\ \hline EfficientNet-B3 & 0.95189 & 0.82143 & 1 & 0.90196 \\ \hline EfficientNet-B2 & 0.95189 & 0.74194 & 1 & 0.85185 \\ \hline EfficientNet-B1 & 0.95876 & 0.88462 & 1 & 0.93878 \\ \hline InceptionV3 & 0.96564 & 1 & 1 & 1 \\ \hline ResNet50V1 & 0.96564 & 0.88 & 0.95652 & 0.91667 \\ \hline EfficientNet-B5 & 0.96564 & 1 & 1 & 1 \\ \hline VGG16 & 0.96907 & 0.95833 & 1 & 0.97872 \\ \hline ResNet50V2 & 0.97595 & 95652 & 0.95652 & 0.95652 \\ \hline MobileNetV2 & 0.97595 & 0.92 & 1 & 0.95833 \\ \hline Xception & 0.97945 & 1 & 0.95652 & 97778 \\ \hline DenseNet-201 & 0.98625 & 1 & 1 & 1 \\ \hline ResNet11 & 0.99313 & 1 & 1 & 1 \\ \hline EfficientNet-B7 & 0.99313 & 1 & 1 & 1 \\ \hline Majority Voting (hard approach) & 0.99313 & 1 & 1 & 1 \\ \hline Majority Voting (soft approach) & 0.99313 & 1 & 1 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Evaluation metrics of the algorithms on the validation dataset. \begin{table}
\begin{tabular}{c c} \hline \hline
**Algorithm** & **Accuracy** \\ \hline
3D CNN (ResNet18) ([PERSON] et al., 2012) & 86.7\% \\ VBNet ([PERSON] et al., 2012) & 91.6\% \\ COVID-Net ([PERSON] et al., 2012) & 93.3\% \\ ResNet50V2 ([PERSON] et al., 2012) & 96.23\% \\
4 CNN models ([PERSON] et al., 2016) & 89.5\% \\ ResNet50V2 & 99.315\% \\ MobileNetV2 & 99.658\% \\ Majority Voting & 99.315\% \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Comparative results for each model on test Accuracy. test to validation datasets. The best classifier in the test set was MobileNetV2, with an accuracy of 0.99658. But it gave lesser performance when tested on the validation dataset: 0.97595. However, the Majority Voting approach gave in the validation set the best accuracy: 0.99313. Going deeper, we studied the average overall accuracy of every algorithm on both sets (the test set and the validation set). The results are provided in Table 11. The results are ordered by descending order following the average accuracy calculated by averaging every algorithm's accuracy on the test and the validation sets. Based on the results provided in Table 11, we note that the best method to use for the problem treated in this paper is the Majority Voting method based on combinating results from 5 classifiers: MobileNetV2, ResNet50V2, ResNet50V1, DenseNet-201, and ResNet11. In fact, this method gave us the best average accuracy on both test and validation sets. Moreover, it gives us 100% accuracy regarding the class COVID on both validation and test sets (100% precision, 100% recall, and 100% F1 score on both sets). The average accuracy of the Majority Voting approach is remarkably better than any other classifier tested on the Chest X-ray Dataset (even the top selected classifiers in the test set). Going deeper, we decided to test the statistical significance of our introduced approach. In other words, we need to statistically reject the hypothesis that assumes that the Majority Voting superiority came only by chance in our experiments. So, we decided first to define explicitly the Null Hypothesis \(H_{0}\) that we want to reject by calculating the \(p\)-value. As we need to statistically quantify the superiority of one algorithm over the other, we considered \(H_{0}\) as the hypothesis that the classification Method does not belong to the top 5% margin of the difference between the top and the least performing algorithm on a selected dataset. The Alternate Hypothesis \(H_{1}\) will be then defined as the hypothesis that the classification method belongs to the top 5% margin of the difference between the top and the least performing algorithms on a selected dataset.