# Attention-Based Transfer Learning for Efficient Pneumonia Detection in Chest X-ray Images
## 4 Discussion  
It is widely known that as the model gets deeper, the number of parameters increases. Thus, the flexibility of the model increases and the training error decreases. However, the generalization error increases due to over-fitting to the test data. Sufficient data are needed for the model to classify pneumonia and normal. Distinguishing between normal organs and pathological signs requires medical professionals that have the relevant knowledge. Therefore, large medical datasets with correct answer labels are expensive. To solve this problem, we used three pre-trained models from different domains (ImageNet, Chest X-ray14, Custom dataset). DenseNetl21 was trained to predict 14 anomalies including pneumonia on chest X-ray images. Among the above pre-trained models, it was confirmed that using the model based on DenseNetl21 had the highest accuracy. The domain of the test dataset is similar to that of the chest X-ray14 data set. When combining the models with  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **Recall (\%)** & **Precision (\%)** & **AUC (\%)** & **F-Score** & **Accuracy (\%)** \\ \hline [PERSON] et al. [32] & 93.2 & - & 96.8 & - & 92.8 \\ [PERSON] et al. [34] & - & - & 98.4 & - & - \\ [PERSON] et al. [35] & 96.2 & **97.7** & **99.3** & 0.970 & 96.2 \\ [PERSON] et al. [24] & 87.22 & - & - & - & 94.18 \\ [PERSON] et al. [36] & 94.85 & 95.72 & - & 0.953 & 95.07 \\ [PERSON] and [PERSON] [37] & 82 & - & - & - & 87 \\ [PERSON] et al. [38] & - & - & - & - & 90.68 \\ \hline
**Our model(SE-Attention)** & **98.46** & 96.24 & 96.03 & **0.973** & **96.63** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Comparative results for other models on same test dataset. Bold numbers indicate best performance.  
an attention mechanism, it means that the final model will also show good performance if the combined transfer learning model is pre-trained with data from a domain similar to the domain of the target task. The test data for this paper was 5,856 images, which is insufficient to train a model that used over a million images such as ResNet and DenseNet. With data preprocessing, our model can train with low-quality input images and reach good performance. However, good quality input images as input can improve the performance of our model. There are limitations where there may not be a pre-trained model using data from domains similar to our target task. In this case, it is expected that the performance can be improved either by using pre-trained models trained on various datasets, or by using more models. Future work on our study will add a study related to domain adaptation that can achieve domain changes by maintaining high performance in the original domain. We will also have to study how many pre-trained models are desirable to use. Our framework is not the only one used to diagnose pneumonia. Simply, our framework is to look at chest X-rays and classify them as pneumonia or normal with the hope that it could be used as an aid to medical expert diagnosis. At the same time, our framework is expected to add credibility to the diagnosis of pneumonia.