## 4 Results and Discussion
### Performance Evaluation of Customized CNNs
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Task** & **Data** & **Models** & **Accuracy** & **AUC** & **Precision** & **Recall** & **Specificity** & **F-Score** & **MCC** \\ \hline \multirow{8}{*}{Normal vs. Pneumonia} & \multicolumn{3}{c}{Customized} & \multirow{2}{*}{**0.957**} & \multirow{2}{*}{**0.990**} & \multirow{2}{*}{**0.951**} & \multirow{2}{*}{**0.983**} & \multirow{2}{*}{**0.915**} & \multirow{2}{*}{**0.967**} & \multirow{2}{*}{**0.908**} \\  & & VGG16 & & & & & & & \\  & & Sequential & 0.943 & 0.983 & 0.920 & 0.980 & 0.855 & 0.957 & 0.878 \\  & & Residual & 0.910 & 0.967 & 0.908 & 0.954 & 0.838 & 0.931 & 0.806 \\  & & Inception & 0.886 & 0.922 & 0.887 & 0.939 & 0.800 & 0.913 & 0.755 \\ \cline{2-9}  & \multirow{4}{*}{Cropped ROI} & Customized & \multirow{2}{*}{**0.962**} & \multirow{2}{*}{**0.993**} & \multirow{2}{*}{**0.977**} & \multirow{2}{*}{0.962} & \multirow{2}{*}{**0.962**} & \multirow{2}{*}{**0.970**} & \multirow{2}{*}{**0.918**} \\  & & VGG16 & & & & & & & \\  & & Sequential & 0.941 & 0.984 & 0.930 & **0.995** & 0.877 & 0.955 & 0.873 \\  & & Residual & 0.917 & 0.971 & 0.913 & 0.959 & 0.847 & 0.936 & 0.820 \\  & & Inception & 0.897 & 0.932 & 0.896 & 0.947 & 0.817 & 0.921 & 0.778 \\ \hline \multirow{8}{*}{Bacterial vs. Viral Pneumonia} & \multicolumn{3}{c}{Customized} & \multirow{2}{*}{**0.936**} & \multirow{2}{*}{**0.962**} & \multirow{2}{*}{**0.920**} & \multirow{2}{*}{**0.984**} & \multirow{2}{*}{**0.860**} & \multirow{2}{*}{**0.951**} & \multirow{2}{*}{**0.862**} \\  & & VGG16 & & & & & & & \\  & & Sequential & 0.928 & 0.954 & 0.909 & **0.984** & 0.838 & 0.946 & 0.848 \\  & & Residual & 0.897 & 0.921 & 0.880 & 0.967 & 0.784 & 0.922 & 0.780 \\  & & Inception & 0.854 & 0.901 & 0.841 & 0.934 & 0.714 & 0.886 & 0.675 \\ \cline{2-9}  & \multirow{4}{*}{Cropped ROI} & Customized & \multirow{2}{*}{**0.936**} & \multirow{2}{*}{**0.962**} & \multirow{2}{*}{**0.920**} & \multirow{2}{*}{**0.984**} & \multirow{2}{*}{**0.860**} & \multirow{2}{*}{**0.951**} & \multirow{2}{*}{**0.862**} \\  & & VGG16 & & & & & & & \\  & & Sequential & 0.928 & 0.956 & 0.909 & **0.984** & 0.838 & 0.946 & 0.848 \\  & & Residual & 0.908 & 0.933 & 0.888 & 0.976 & 0.798 & 0.930 & 0.802 \\  & & Inception & 0.872 & 0.919 & 0.853 & 0.959 & 0.730 & 0.903 & 0.725 \\ \hline \multirow{8}{*}{Normal vs. Bacterial vs. Viral Pneumonia} & \multicolumn{3}{c}{Customized} & \multirow{2}{*}{**0.917**} & \multirow{2}{*}{**0.938**} & \multirow{2}{*}{**0.917**} & \multirow{2}{*}{**0.905**} & \multirow{2}{*}{**0.958**} & \multirow{2}{*}{**0.911**} & \multirow{2}{*}{**0.873**} \\  & & VGG16 & & & & & & & \\  & & Sequential & 0.896 & 0.922 & 0.888 & 0.885 & 0.948 & 0.887 & 0.841 \\ \cline{1-1}  & & Residual & 0.861 & 0.887 & 0.868 & 0.882 & 0.933 & 0.875 & 0.809 \\ \cline{1-1}  & & Inception & 0.809 & 0.846 & 0.753 & 0.848 & 0.861 & 0.798 & 0.688 \\ \cline{1-1} \cline{2-9}  & \multirow{4}{*}{Customized VGG16} & Customized & \multirow{2}{*}{**0.918**} & \multirow{2}{*}{**0.939**} & \multirow{2}{*}{**0.920**} & \multirow{2}{*}{**0.900**} & \multirow{2}{*}{**0.960**} & \multirow{2}{*}{**0.910**} & \multirow{2}{*}{**0.876**} \\  & & VGG16 & & & & & & \\ \cline{1-1}  & & Sequential & 0.897 & 0.923 & 0.898 & 0.898 & 0.949 & 0.898 & 0.844 \\ \cline{1-1}  & & Residual & 0.879 & 0.909 & 0.883 & 0.890 & 0.941 & 0.887 & 0.825 \\ \cline{1-1}  & & Inception & 0.821 & 0.865 & 0.778 & 0.855 & 0.878 & 0.815 & 0.714 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Performance of customized CNNs with baseline and cropped ROI data. and AUC achieved with the customized VGG16 model are shown in Figures 8-10. We observed that the training metrics are poor compared to test accuracy. This is due to the fact that noisy images are included in the training data to reduce bias, overfitting, and improve model generalization. We compared the performance of the customized VGG16 model trained with the cropped ROI, to the state-of-the-art.