# Classifying Breast Density from Mammogram with Pretrained CNNs and Weighted Average Ensembles
## 4 Results and Discussion  
The experiments were started with applying pre-trained CNN models that used in the related works. Then, we investigated and tested recent and efficient pre-trained CNNs supported by Keras. These models included VGG19, InceptionResNetV2, EfficientNetV2, and Xception. Some models required a significant amount of memory, such as NasNet; therefore, we could not use them. The training/validation behavior based on accuracy and loss for each model is illustrated in Figures 11-19. Moreover, Tables 4-12 report the performance of each model. Figure 11 explaining the behavior of the DenseNet based model. It can be notice that, the validation loss increase in oscillatory manner after epoch 40. The performance of DenseNet model, was low in terms of overall accuracy and F1 of each class, as shown in Table 4. Moreover, MobileNet model was the worst model in classifying breast density. We can notice this from training behavior in Figure 12, and testing results in  
Figure 10: Weighted average ensembles. Figure 9: Average ensembles. Table 5. The loss value of ResNet50, increasing dramatically after epoch 80, as shown in Figure 13. Both ResNet50 and InceptionV3, have overall accuracy exceed 70%. However, the F1 of each class was better with ResNet50 model comparing with InceptionV3, as shown in Tables 6 and 7. The training behaviors of VGG16 and VGG19, are very oscillate as illustrated in Figures 15 and 16. Meanwhile, the performance of VGG16 and VGG19 shown in Tables 8 and 9 were low, the overall accuracy and F1 score were under 65%. Comparing with ResNet50 and Inception V3, the InceptionResNetV2 model was worse, the behavior of the model and its performance present in Figure 17 and Table 10. The training behavior of the model based on Xception is shown in Figure 18, and the model's performance illustrated in Table 11. Finally, the training behavior and testing performance of EfficientNetV2 is shown in Figure 19 and Table 12, respectively. Figure 14: The training behavior of the model based on InceptionV3. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Confusion Matrix**} & \multicolumn{1}{c}{**Precision**} & \multicolumn{1}{c}{**Recall**} & \multicolumn{1}{c}{**F1**} \\  & **A** & **B** & **C** & **D** & **Precision** & **Recall** & **F1** \\ \hline
**A** & 114 & 16 & 2 & 1 & 0.66 & 0.86 & 0.75 \\ \hline
**B** & 51 & 134 & 34 & 8 & 0.73 & 0.59 & 0.65 \\ \hline
**C** & 8 & 34 & 164 & 10 & 0.76 & 0.76 & 0.76 \\ \hline
**D** & 0 & 0 & 15 & 26 & 0.58 & 0.63 & 0.60 \\ \hline \multicolumn{3}{c}{**Accuracy**} & \multicolumn{1}{c}{0.71} & \multicolumn{1}{c}{} \\ \hline \multicolumn{3}{c}{**Overall F1 score**} & \multicolumn{1}{c}{0.71} & \multicolumn{1}{c}{} \\ \hline \hline \end{tabular}
\end{table} 
Table 6: ResNet50. Figure 13: The training behavior of the model based on ResNet50. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Confusion Matrix**} & \multicolumn{3}{c}{**Precision**} & \multicolumn{1}{c}{**Recall**} & \multicolumn{1}{c}{**F1**} \\  & **A** & **B** & **C** & **D** & & & & \\ \hline
**A** & 48 & 84 & 1 & 0 & 0.69 & 0.36 & 0.47 \\ \hline
**B** & 21 & 153 & 53 & 0 & 0.53 & 0.67 & 0.60 \\ \hline
**C** & 1 & 49 & 166 & 0 & 0.64 & 0.77 & 0.70 \\ \hline
**D** & 0 & 1 & 38 & 2 & 1.00 & 0.05 & 0.09 \\ \hline  & \multicolumn{3}{c}{**Accuracy**} & \multicolumn{3}{c}{0.60} \\ \hline  & \multicolumn{3}{c}{**Overall F1 score**} & \multicolumn{3}{c}{0.57} \\ \hline \hline \end{tabular}
\end{table} 
Table 8: VGG16. Figure 16: The training behavior of the model based on VGG19. Figure 15: The training behavior of the model based on VGG16. Figure 17: The training behavior of the model based on InceptionResNetV2.