# Prediction of COVID-19 from Chest CT Images Using an Ensemble of Deep Learning Models
## 5 Experimental Results and Discussion
Xception models are also expensive to train due to their modified depth-wise separable convolution as compared to inception models. \begin{table}
\begin{tabular}{c c c c c c} \hline
**Model** & **Accuracy (\%)** & **Precision** & **Recall** & \(F_{1}\)**-Score** & **AUC Score** \\ \hline VGG-16 & 98.39 & 0.9839 & 0.9839 & 0.9839 & 0.9839 \\ \hline ResNet50 & 95.17 & 0.9517 & 0.9517 & 0.9517 & 0.9515 \\ \hline Xception & 94.57 & 0.9457 & 0.9457 & 0.9456 \\ \hline \end{tabular}
\end{table} 
Table 1: Details of the performance of the models for the prediction of COVID-19 on the SARS-CoV-2 CT scan dataset. We have also stacked the different combinations of these three models for the improvement of the model for the prediction of COVID-19. Table 2 reflects the outcomes observed for different combinations of the used models. After analyzing Table 2, it can be said that the ensembling of models (any combination) produces better recognition accuracy than the individual one. The best recognition accuracy of 98.79% was observed when all the three models were ensembled. For this model, the observed classification accuracy, precision, recall, and \(F_{1}\)-score are 98.79%, 0.99, 0.99, and 0.99, respectively. It can be seen from Table 1 that Xception does not perform well considering all the metrics when compared to the other two base models, namely VGG-16 and ResNet50. However, the stacking ensemble consisting of all three base models outperforms all other combinations. We used a fully connected neural network as a meta-learner to provide flexibility to the stacked model and decrease the generalization error. Thus, the neural network combines the predictions obtained from all the base models in such a way that the performance of the overall model improves. The neural network is tuned so that it can ignore the wrong predictions made by the base models and utilizes only those predictions that help improve the classification score. Different CNN models commit errors on different samples; thus, aggregating those together helps to achieve a better accuracy on the test set. Image index 165 belongs to a test set and was originally a COVID image. It is misclassified by VGG-16 as a non-COVID image but properly classified by ResNet50 and Xception. It is also misclassified by both VGG-16+ResNet50 and VGG-16+Xception combinations, but in the stacked model, it is properly classified. Similarly, image index 170 belongs to the test set and is originally a COVID +ve CT-scan, which is misclassified by Xception but accurately recognized by VGG-16 and ResNet50. As all three models are included in the final ensemble architecture, it gets correctly classified by the stacking ensemble model. The aforementioned images can be seen in Figure 5. The complete test set contains 497 samples. The pronounced difference in the performance of the said models on the test set can be visualized from confusion matrices shown in Figure 6. In the final confusion matrix of the stacking ensemble model that can be seen in Figure 7, only four COVID +ve images are incorrectly classified as non-COVID, and two non-COVID samples are recognized as COVID samples. This can be considered as a significant improvement compared to other models. Figure 4: ROC curve obtained for VGG-16, ResNet50, and Xception model when considering the SARS-CoV-2 CT scan dataset. Figure 5: Images misclassified by the base models but accurately recognized by the proposed ensemble framework. Figure 6: Confusion matrices of base models and different combinations of the ensemble model. In medical research, especially for critical diseases like COVID-19, it is significantly important to reduce the FP and FN rate as much as possible when designing the prediction model. FN should be as low as possible, because if COVID-19 (+ve) patients get wrongly classified as a COVID-19 (-ve), it may cause otherwise avoidable deaths again, and misclassification of COVID-19 (-ve) cases with COVID-19 (+ve) (FP) may lead to unnecessary emotional disruption for an individual.