# Pneumonia Detection on Chest X-ray Images Using Ensemble of Deep Convolutional Neural Networks
## 4 Experimental Study
### Results and Analysis  
Initially, we chose the pretrained models from the previous research, as in [26, 52]. The best three methods were selected after comparing their accuracy to other methods using the previously mentioned free dataset of chest X-ray images. In terms of testing accuracy, the MobileNetV2, VIT, and DenseNet169 models performed best, as shown in Table 1. Their characteristics are described above in Section 3.1, which also includes a summary of their structures.  
Figure 2: Samples of chest X-ray for pneumonia classification task from the selected database. The above row shows the normal images, and the bottom row shows the pneumonia images.  
The performance of MobileNetV2, DenseNet169, Vision Transformer (VIT), and the proposed ensemble learning (EL) method on training and validation losses and accuracy is compared in Figure 3. The proposed method reduced verification loss, as shown in the figure, which improved the accuracy results. DenseNet169's training loss is 0.1664, training accuracy is 0.9319, validation loss is 0.2408, and validation accuracy is 0.9103. Additionally, MobileNetV2 achieved a training accuracy of 0.9122, a training loss of 0.2096, a validation loss of 0.2072, and a validation accuracy of 0.9087. The VIT had a training loss of 0.1503, a training accuracy of 0.9421, a validation loss of 0.2071, and a validation accuracy of 0.9215. The proposed ensemble method had a training loss of 0.1361, a training accuracy of 0.9525, a validation loss of 0.0421, and a validation accuracy of 1.0.  
The test set fulfills the requirements too. In reality, we obtain better accuracy results and a lower loss on the test set with the proposed method, illustrating a more reliable and robust approach. The accuracy of the test set can be seen in Table 2.  
A comparison of true and predicted labels can be seen in the confusion matrices for the proposed Ensemble Learning (EL) method, Vision Transformer (VIT), MobileNetV2, and DenseNet169, as shown in Figure 4, to better understand how the four approaches did in these binary classifications. This is done primarily to understand what a good classification approach should be, as well as how it could be enhanced while trying to deal with the diagnosis of diseases, which is often critical to a patient's survival. The confusion matrices for the MobileNetV2, DenseNet169, VIT, and the proposed EL method are shown in Figure 4. The confusion matrices in the figure contain actual and predicted labels for both normal (234) and pneumonia (390) on the chest X-ray images.  
\begin{table}
\begin{tabular}{c c c c c} \hline
**Model** & **Precision** & **Recall** & **F1-Score** & **Accuracy** \\ \hline Xception & 0.7971 & 0.7676 & 0.7713 & 0.7676 \\ VGG16 & 0.8126 & 0.8103 & 0.8087 & 0.8103 \\ MobileNetV2 & 0.9003 & 0.9073 & 0.9034 & 0.9087 \\ InceptionV3 & 0.8897 & 0.8871 & 0.8854 & 0.8871 \\ ResNet50 & 0.8233 & 0.8222 & 0.8226 & 0.8222 \\ DenseNet169 & 0.9133 & 0.9009 & 0.9063 & 0.9135 \\ ResNet152V2 & 0.8702 & 0.8687 & 0.8673 & 0.8687 \\ DenseNet121 & 0.8927 & 0.8922 & 0.8911 & 0.8922 \\ VIT & 0.9245 & 0.9247 & 0.9244 & 0.9247 \\ \hline \end{tabular}
\end{table}
Table 1: The results of well-known CNN models.  
\begin{table}
\begin{tabular}{c c c c c} \hline
**Model** & **Precision (\%)** & **Recall (\%)** & **F1-score (\%)** & **Accuracy (\%)** \\ \hline DenseNet169 & 91.33 & 90.09 & 90.63 & 91.35 \\ MobileNetV2 & 90.03 & 90.73 & 90.34 & 90.87 \\ VIT & 92.45 & 92.47 & 92.44 & 92.47 \\ \hline EL (Our) & **93.96** & **92.99** & **93.43** & **93.91** \\ \hline \end{tabular}
\end{table}
Table 2: Comparison of testing data results among the proposed Ensemble Learning (EL) and three well-known CNN models.  
Figure 3: The plots of the loss and accuracy on the training and validation sets of the chest X-ray images.