## 5 Experimental Design
### Parameter Optimization
#### 5.4.1 Results of CIFAR-10 Dataset  
Table 9 presents the final experimental results, with accuracy representing the overall accuracy and the remaining metrics evaluating the performance of minority class samples. Based on the analysis of the experimental results, we observed that modifying the kernel size and learning rate had a significant impact on the CNN model's performance, especially with imbalanced datasets. Model 1, serving as the baseline model, showed lower metrics, particularly recall and F1 score, at 0.22 and 0.36, respectively, indicating poor performance on minority class samples. Although Model 2's overall accuracy decreased to 0.49, its recall significantly improved to 0.16, demonstrating an enhanced ability to recognize minority class samples. Model 3, with the kernel size adjusted to \(5\times 5\), significantly improved the overall accuracy, recall, F1 score, and G-mean, reaching 0.65, 0.33, 0.72, and 0.78, respectively, indicating a substantial improvement in recognition and accuracy for the minority class samples. Model 4 adjusted only the learning rate, achieving a 0.67 overall accuracy and improved performance on minority class samples, particularly in F1 score and G-mean, with results of 0.71 and 0.76, respectively. Model 5, which combined adjustments to the number of kernels, kernel size, and learning rate, achieved performance metrics close to those of Model 4 and slightly better than those of Model 3, with a recall of 30.96% and an F1 score of 0.71. The results demonstrate that simultaneously adjusting multiple parameters can yield better performance, as shown in Figure 11.  
Figure 11: Performance metrics of minority class samples across the five models for the CIFAR-10 dataset. Based on the analysis of variance and tuning process described above, the optimal configuration for a dataset with an imbalance ratio of 25:1 was confirmed to be (0.013, 0.2, 0.2, 3 \(\times\) 3). Table 10 presents the performance metrics of the parameter-optimized CNN model, Model 1 with undersampling, and Model 1 with SMOTE for the CIFAR-10 dataset for each class. Compared to the standard data-processing methods, the parameter-optimized model demonstrated superior performance in most categories in terms of recall, F1 score, G-mean, and P-mean, especially for the minority class. In this study, the minority class corresponds to the automobile category. For the minority class, the recall, F1 score, G-mean, and P-mean of Model 5 reached 0.31, 0.71, 0.76, and 0.94, respectively, whereas Model 1 with undersampling only achieved 0.23, 0.37, 0.48, and 0.52, respectively. The results indicate that, under a 25:1 imbalance ratio, optimizing the parameter selection improves the classification performance of the minority class, highlighting the importance of optimizing parameter selection in handling imbalanced data scenarios. Compared to undersampling, Model 1 with SMOTE showed mixed results for the minority class. The SMOTE resulted in a lower recall than undersampling, only reaching 0.12, though it slightly improved the F1 score to 0.22. Despite the slight improvement in F1 score, the G-mean and P-mean values for the SMOTE were still significantly lower than those of the parameter-optimized Model 5. This indicates that, while the SMOTE's synthetic samples provide some improvements in certain metrics, its effectiveness in enhancing recall for the minority class is limited. This further underscores that, in cases of extreme imbalance, optimizing model parameters is more effective than solely relying on SMOTE oversampling. In terms of the G-mean and P-mean metrics, Model 5 demonstrated a more balanced and consistent performance. For other class data, Model 5 achieved a recall rate of 0.90 for the airplane category, compared to 0.87 achieved by Model 1 with undersampling. In the frog category, Model 5 had an F1 score of 0.77, while Model 1 had an undersampling score of 0.70.