# A Novel Transfer Learning Based Approach for Pneumonia Detection in Chest X-ray Images
## 3 Results  
### Results  
The primary goal of our transferring learning approach was to correctly diagnose pneumonia among normal chest X-ray images. For this we prepared all of the models as shown above and trained them separately. We performed training and testing using a computer with Intel(R) Core (TM) i7-6700 (Intel Corporation, Santa Clara, CA, USA) with 3.30 GHz CPU, NVIDIA GeForce GTX 1070 8 GB GPU (NVIDIA Corporation, Santa Clara, CA, USA) and 48 GB of RAM. For training, we used the Adam optimizer [63] and the cross-entropy loss function. The learning rate is started from the value of 0.001 and is reduced by 2 after every three epochs. AlexNet was trained for 200 iterations at learning rate of 0.001 and then trained at very low learning rate of 0.00001; it achieved a test accuracy of 92.86% and a train accuracy of 93.0%. The AUC value was 97.83%. ResNet18 did better than AlexNet and other models; it achieved an area under the ROC curve as 99.36% and test accuracy of 94.23%.  
The average computational time for all models on CPU is 0.332 s and for GPU it is 0.043 s, whereas the ensemble model took 0.161 s for computation.  
Figure 11 shows plots of accuracy and loss against epochs. The best results were obtained by the ResNet18 network both in terms of accuracy and loss values.  
Figure 11: Accuracy against epoch (**a**) and cross-entropy loss against epoch (**b**) for all trained models. DenseNet121 and InceptionV3 were trained for 100 epochs, and GoogLeNet was trained for 50 epochs. This was done to prevent overfitting and get a better generalization. Plots are drawn for the training set.  
Figure 12 shows the plot of sensitivity against specificity for each model and the ensemble model on the testing set. Note that the ensemble model has performed better.  
Figure 13 shows the activation map in early convolutional layers for each of the models. Note that the networks can capture and learn similar activation maps, albeit at different layers. For example, the Conv1_4 layer activation map of ResNet18 is very similar to the Conv1_1 layer activation map of Inception V3. Note that activations maps show that the network in most cases correctly differentiates between bacterial pneumonia, which is characterized by activations in the regions of lobar consolidations, and viral pneumonia, which typically has diffuse interstitial patterns across the lungs.  
Figure 12: Sensitivity and specificity for each model and the ensemble model on the testing set.  
Figure 13: Plots of activation maps for each model in early convolutional layers. Activation maps are calculated for the positive class and are shown for the last layer of each convolutional block before pooling.  
Table 2 shows the results for each neural network model. After analyzing the obtaining of the predictions of all models, we combined the results of 5 models: AlexNet, DenseNet121, InceptionV3, GoogLeNet, and ResNet18, and predicted the class which was most frequently predicted by all models. This combination gave the remarkably best performance and achieved a test accuracy of 96.39%, with an area under the ROC curve of 99.34%, and the sensitivity as 99.62%.