## 4 Experimental Results
### Classification Results
#### 4.2.2 Bagging Ensemble Classification Results
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline
**Bagging Ensemble** & **Model Name** & **Model Numbers** & **Accuracy** & **F1-Score** & **AUC** & **PPV** & **NPV** & **Sensitivity** & **Specificity** \\ \hline \multirow{4}{*}{Performance} & ResNet-18 & 3 & 0.68 & 0.56 & 0.67 & 0.87 & 0.63 & 0.46 & 0.91 \\ \cline{2-9}  & ResNet-34 & 2 & 0.68 & 0.61 & 0.71 & 0.82 & 0.67 & 0.56 & 0.80 \\ \cline{2-9}  & DenseNet-121 & 2 & 0.68 & 0.64 & 0.69 & 0.81 & 0.69 & 0.60 & 0.76 \\ \cline{2-9}  & MobileNet-V2 & 2 & 0.67 & 0.58 & 0.69 & 0.87 & 0.62 & 0.49 & 0.86 \\ \cline{2-9}  & ResNet-18+ & & & & & & & & \\ \cline{2-9}  & DenseNet-169+ & 3 & 0.67 & 0.61 & 0.73 & 0.83 & 0.63 & 0.54 & 0.80 \\ \cline{2-9}  & MobileNet-V2 & & & & & & & & \\ \hline \multirow{4}{*}{Weighting} & ResNet-34+ & & & & & & & & \\ \cline{2-9}  & ResNet-18+ & 3 & 0.69 & 0.60 & 0.72 & 0.82 & 0.66 & 0.53 & 0.85 \\ \cline{2-9}  & ResNet-18 & & & & & & & & \\ \cline{2-9}  & MobileNet-V2+ & & & & & & & & \\ \cline{2-9}  & DenseNet-121+ & 3 & 0.65 & 0.60 & 0.68 & 0.76 & 0.64 & 0.56 & 0.74 \\ \cline{2-9}  & DenseNet-169 & & & & & & & & \\ \hline \multirow{4}{*}{ResNet-34+ & & & & & & & & & \\ \cline{2-9}  & ResNet-34+ & & & & & & & & \\  
Our investigation adopted a rigorous five-fold cross-validation methodology. Specifically, we showcased the confusion matrices corresponding to the third fold, utilizing the ResNet34 architecture as the backbone for the baseline model and our novel CAD system. This technique enabled us to comprehensively evaluate and juxtapose these two models' performances. Figure 9 depicts the confusion matrix for the baseline and our proposed CAD system when using ResNet34 as the backbone. There were 100 EBUS images in the test dataset, comprising 50 benign and 50 malignant cases. In Figure 9, seven malignant images were erroneously classified as benign, whereas all 50 benign lesions were mistakenly identified as malignant. The number of malignant cases within our dataset significantly outweighed the number of benign cases. Despite employing various data augmentation techniques to mitigate this data imbalance, the model's performance remains inadequate in distinguishing between benign and malignant instances. As evident from the confusion matrix, the model erroneously predicted all benign images. However, as shown in Figure 9, incorporating our proposed CAD framework greatly alleviated the data imbalance impact. The confusion matrix indicates that nearly all 50 benign EBUS images and approximately 80% of malignant EBUS images were accurately identified. Compared to the baseline's confusion matrix, although there was a false negative (FN) increase, the false positives (FP) significantly decreased. This comparative analysis substantiates that our proposed CAD framework effectively mitigates the data imbalance impact. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multicolumn{2}{c}{\
\begin{tabular}{c} **Model Name** \\ **or Fusion Name** \\ \end{tabular} } & **Accuracy** & **F1-Score** & **AUC** & **PPV** & **NPV** & **Sensitivity** & **Specificity** \\ \hline \multirow{8}{*}{Baseline} & ResNet-18 & 0.59 & 0.60 & 0.61 & 0.65 & 0.50 & 0.60 & 0.59 \\ \cline{2-8}  & ResNet-34 & 0.53 & 0.59 & 0.64 & 0.54 & 0.43 & 0.68 & 0.35 \\ \cline{2-8}  & DenseNet-121 & 0.62 & 0.66 & 0.66 & 0.68 & 0.54 & 0.68 & 0.54 \\ \cline{2-8}  & DenseNet-169 & 0.62 & 0.61 & 0.62 & 0.70 & 0.60 & 0.58 & 0.67 \\ \cline{2-8}  & MobileNet-V2 & 0.63 & 0.56 & 0.62 & 0.76 & 0.59 & 0.50 & 0.78 \\ \cline{2-8}  & ShuffleNet-V2 & 0.58 & 0.63 & 0.57 & 0.62 & 0.52 & 0.67 & 0.47 \\ \hline \multirow{2}{*}{
\begin{tabular}{c} Bagging \\ Ensemble \\ \end{tabular} } & Majority Voting & 0.70 & 0.63 & 0.70 & 0.83 & 0.68 & 0.58 & 0.82 \\ \cline{2-8}  & Output Average & 0.70 & 0.63 & 0.70 & 0.87 & 0.67 & 0.56 & 0.84 \\ \cline{2-8}  & Performance Weighting & 0.70 & 0.63 & 0.75 & 0.84 & 0.68 & 0.56 & 0.85 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: This table shows comparison results between ensemble results and baseline.