## 4 Experimental Results
### Classification Results
#### 4.2.2 Bagging Ensemble Classification Results  
Three fusion methods were used in the bagging ensemble experiments, including majority voting, output average, and performance weighting. In addition, the same backbone and a different backbone network were tried separately, and three CNN models and two CNN models were separately fused during the prediction fusion stage. Results from the bagging ensemble with the majority voting fusion method are shown in Table 5. During the experiment, the same CNN models were selected as the backbone, and the hybrid CNN models were also included. Notably, the fused hybrid CNN model results were better than the experimental results using the same CNN models. Compared with the best accuracy from the same CNN models, the hybrid CNN model's best accuracy improved by 5%. The best bagging ensemble experimental results were achieved with the following hybrid CNN models: ResNet-34, ResNet-18, and MobileNet-V2. The best bagging ensemble results  
Figure 8: This figure shows the result of division. For each class, how many cases are in each fold. The total number of benign cases is 10; after division, there are 2 cases in each part. The total number of malignant cases is 59; after division, there are 12 cases in each part, except for the last part. with the majority voting fusion method were a 0.70 accuracy, 0.63 F1-Score, 0.70 AUC, 0.83 PPV, 0.68 NPV, 0.58 sensitivity, and 0.82 specificity. Table 6 presents the bagging ensemble with output average fusion method results. Concerning the CNN model selection stage, it cannot be determined whether fusing two or three CNN models can achieve better results without actually conducting the experiment. Therefore, this study fused the same CNN models and hybrid CNN models. A ResNet-34, ResNet-18, and MobileNet-v2 backbone obtained the best results for hybrid CNN model fusion: 0.70 accuracy, 0.63 F1-Score, 0.70 AUC, 0.84 PPV, 0.68 NPV, 0.57 sensitivity, and 0.84 specificity (Table 6). Interestingly, the best result from fusing the same CNN models almost matched the best hybrid CNN model fusion results: 0.70 accuracy, 0.63 F1-Score, 0.70 AUC, 0.87 PPV, 0.67 NPV, 0.56 sensitivity, and 0.84 specificity. Results from the bagging ensemble with the performance weighting fusion method are shown in Table 7. In this experiment, a two- and three-CNN model fusion were attempted. Similarly, the same CNN and hybrid CNN model fusions were also performed. The best hybrid CNN model fusion results included ResNet-34, ResNet-18, and MobileNet-v2 as the backbone: 0.70 accuracy, 0.63 F1-Score, 0.75 AUC, 0.84 PPV, 0.68 NPV, 0.56 sensitivity, and 0.85 specificity. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Model** & **Accuracy** & **F1-Score** & **AUC** & **PPV** & **NPV** & **Sensitivity** & **Specificity** \\ \hline ResNet-18 & 0.59 & 0.60 & 0.61 & 0.65 & 0.50 & 0.60 & 0.59 \\ \hline ResNet-34 & 0.53 & 0.59 & 0.64 & 0.54 & 0.43 & 0.68 & 0.35 \\ \hline DenseNet-121 & 0.62 & 0.66 & 0.66 & 0.68 & 0.54 & 0.68 & 0.54 \\ \hline DenseNet-169 & 0.62 & 0.61 & 0.62 & 0.70 & 0.60 & 0.58 & 0.67 \\ \hline MobileNet-V2 & 0.63 & 0.62 & 0.62 & 0.76 & 0.59 & 0.50 & 0.78 \\ \hline ShuffleNet-V2 & 0.58 & 0.63 & 0.57 & 0.62 & 0.52 & 0.67 & 0.47 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: This table shows the experimental result of the baseline. Data augmentation is utilized, but without using downsampling and bagging ensemble learning when training the CNN models. The experimental result of each model is the average result after five-fold cross-validation.