## 4 Results
For the competition, the AUC score was calculated as the average of the public AUC score, which was based on the public test set, and the private AUC score, which was based on the test set. It can be seen that 1D CNN achieves the highest AUC on both the public dataset and private dataset. However, the methods proposed in this study achieve higher AUC values than those reported in [31] for the classification of BD and SCH. ResNet + Grad-CAM + FS + BO-DNN achieves the highest mean AUC, outperforming all other configurations. Additionally, this model attains the highest maximum AUC and the highest minimum AUC values in a test subset. On the other hand, ResNet + Grad-CAM + FS + XGBoost and ResNet + Grad-CAM + FS + AdaBoost also demonstrate competitive performance in terms of AUC. Figure 8: Comparative ROC curves of the models. The ResNet + Grad-CAM + FS + BO-DNN model exhibited a statistically significant improvement over the ResNet baseline, with a paired t-statistic of \(-3.4337\) and a \(p\)-value of \(0.0089\). The mean difference between these models was \(0.0245\) in favor of BO-DNN, and the Cohen's d value of \(1.1446\) indicates a very large effect size, signifying a substantial performance advantage of this model over the baseline. On the other hand, the other models did not show statistically significant differences when compared to ResNet. For instance, ResNet + Grad-CAM + FS + SVM reported a paired t-statistic of \(1.2078\) and a \(p\)-value of \(0.2616\). Similarly, ResNet + Grad-CAM + FS + XGBoost yielded a paired t-statistic of \(-0.1551\) and a \(p\)-value of \(0.8806\), which also implies no meaningful improvement. Additionally, the paired t-statistics for these models ResNet + Grad-CAM + FS + AdaBoost and their associated \(p\)-values ResNet + Grad-Cam + FS + RF indicate that their performance differences relative to ResNet are not practically meaningful. As shown in Table 3, the Bayesian optimized model, ResNet + Grad-CAM + FS + BO-DNN, is the only model to show statistically and practically significant improvements over the ResNet baseline. The statistical results given in Table 3 are visualized in Figure 9. These visualizations emphasize the significant distinction of the ResNet + Grad-CAM + FS + BO-DNN model compared to the other models. Figure 9 shows the paired t-statistics for all models compared to the ResNet model. The t-statistics are plotted with their corresponding \(p\)-values to show the statistical significance. The ResNet + Grad-CAM + FS + BO-DNN model demonstrates the largest negative t-statistic, which is associated with a statistically significant \(p\)-value of \(0.0089\). \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multicolumn{1}{c}{**Model**} & \multicolumn{1}{c}{**Data**} & \multicolumn{1}{c}{**AUC Public**} & \multicolumn{1}{c}{**AUC Private**} & \multicolumn{1}{c}{**Mean**} & \multicolumn{1}{c}{**AUC**} & \\ \hline
1D CNN & & 70.50 & 63.70 & - & - & - \\ LR & & 67.80 & 58.90 & - & - & - \\ SVM & & 67.50 & 51.00 & - & - & - \\ LDA & rsfMRI [31] & 68.60 & 58.90 & - & - & - \\ GNB & 62.40 & 54.70 & - & - & - \\ KNN & & 54.90 & 50.20 & - & - & - \\ DT & & 50.00 & 50.00 & - & - & - \\ RF & & 65.80 & 54.00 & - & - & - \\ \hline ResNet + Grad-CAM + FS + SVM & & - & - & 81.03 & 62.86 & 98.67 \\ ResNet + Grad-CAM + FS + XGBoost & - & - & 81.36 & 58.09 & 98.05 \\ ResNet + Grad-CAM + FS + AdaBoost & \multirow{2}{*}{sMRI} & \multirow{2}{*}{sMRI} & \multirow{2}{*}{-} & 81.88 & 61.68 & 97.57 \\ ResNet + Grad-CAM + FS + RF & & - & - & 81.91 & 61.32 & 97.5 \\ ResNet + Grad-CAM + FS + BO-DNN & & - & - & **83.35** & **65.0** & 98.48 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Classification performance of model in terms of AUC.