# COVID-19 Diagnosis in Chest X-rays Using Deep Learning and Majority Voting
## 3 Results
### Ensemble Voting Approaches
The first is the OVO (One versus One) approach, in which we divided the dataset into multiple sub \begin{table}
\begin{tabular}{l c c} \hline \hline
**ALGORITHM** & **Accuracy in the** & **Accuracy in the** \\
**USED** & **Validation Set** & **Test Set** \\ \hline Weighted Voting using Neural Networks & 0.99658 & 0.98630 \\ \hline SVM-based voting & 0.99658 & 0.98973 \\ \hline Random Forests-based voting & 0.99315 & 0.98630 \\ \hline Mean & 0.99543 & 0.98744 \\ \hline SD & 0.00198 & 0.00198 \\ \hline Confidence Level (95.0\%) & 0.00491 & 0.00491 \\ \hline Confidence Interval (95.0\%) & 0.99051–1 & 0.98252–0.99236 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Evaluation metrics of 3 ensemble classification algorithms (Neural Networks, SVM and Random Forests on the test and the validation dataset. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**ALGORITHM USED** & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline EfficientNet-B5 & 0.97603 & 1 & 1 & 1 \\ \hline EfficientNet-B6 & 0.97603 & 0.88889 & 1 & 0.94118 \\ \hline EfficientNet-B7 & 0.97603 & 0.96 & 1 & 0.97959 \\ \hline Majority Voting (hard approach) & 0.98630 & 0.96 & 1 & 0.97959 \\ \hline Majority Voting (soft approach) & 0.98973 & 1 & 1 & 1 \\ \hline Mean & 0.98082 & 0.96177 & 1 & 0.98007 \\ \hline SD & 0.00667 & 0.04538 & 0 & 0.02401 \\ \hline Confidence Level (95.0\%) & 0.00828 & 0.05635 & 0 & 0.02982 \\ \hline Confidence Interval (95.0\%) & 0.97253–0.98911 & 0.90541–1.00000 & 1–1 & 0.95025–1 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Comparison between the soft and the hard approaches for Majority voting implemented on three classifiers (EfficientNet-B5, B6 and B7). datasets. In each one of them, we only consider only one class versus another. The second approach used is the OVR (One versus the Rest). In OVR, we split the dataset into multiple sub-datasets where we consider only one class versus all the rest. Then, for every one of the two approaches, we averaged the different obtained scores using two methods: macro average score and prevalence weighted average. The scores are calculated for the top selected classifiers (DenseNet-201, ResNet50V1, ResNet50V2, MobileNetV2, and ResNet11) and the Soft Majority Approach. We cannot measure the AUC score for the Hard Majority Approach because the predicted label in this case is not obtained from a probability. The Results are presented in Table 8.  
We can see there that the Soft Majority Voting and the MobileNetV2 have the best AUC scores than all the others. The margin between these two methods is not statistically significant compared to their margin with the others classifiers. In fact, this margin is only 0.2% of the total margin between the best and the least performing algorithms. Hence, this confirms the validity of choosing the Majority Voting approach in general cases to avoid performance variability of the classifiers among different test sets. In fact, the performance of MobileNetV2 was remarkably lower than the Soft Majority Approach on another test set (see Table 11), while the Soft Majority Approach was always the best performing algorithm on it.