## 3 Experimental Results and Discussion
### Multilayer CNN-Based Classifiers' Training and Validation
For the training dataset with 500 feature patterns (290 tumor-free patterns and 210 tumor patterns), Figure 8a,b showed the training history curves for accuracy and loss value for 1000 training epochs in the training stage, as a blue real-line for the training curve and an orange real-line for the validation curve. It can be seen that the training history curve reached saturation over the 400 training epochs in the training stage; thus, a classification accuracy of 97% was obtained and was guaranteed to gradually reach the convergence condition. Finally, the results of the training convergence curve converged, and the value of loss function was 0.091, as seen in the training convergence curve in Figure 8b. In the recalling stage, for the 500 feature patterns, the experimental results of the classifier produced a visual confusion matrix for testing results, with the abnormal pattern yields TP (true positive) = 203 and FP (false positive) = 7 and the normal pattern yields TN (true negative) = 282 and FN (false negative) = 8 in Figure 9, which were used to compute the four evaluation indices of the classifier, including precision (%) = 96.70%, recall (%)  
Figure 8: Training history curves for 2D fractional-order CNN-based classifier. (**a**) Training history of classifier’s accuracy for 1000 training epochs; (**b**) Training history of loss value for 1000 training epochs. = 96.20%, accuracy (%) = 97.00%, and F1 score = 0.9640, respectively. Hence, the above criteria were evaluated to quantify the classifiers' performance. For small-scaled databases, the cross-validation method was used in ML and DL for improving the model's classification performances when we did not have enough datasets to split the training, validation, and testing; through 10-fold (\(K_{f}\) = 10) cross-validation tests, for each fold test, we randomly selected 200 feature patterns from datasets for training the both classifiers, and another 200 feature patterns for validating the classifier's performance. The experimental results of the 2D fractional-order CNN-based classifier were shown in Table 3, with an average precision of 95.90% (as the positive predictive value, PPV) and an average recall of 96.10% for identifying the feature patterns for tumor cases (B and M) and also accurately identifying the abnormality (TP), respectively; an average accuracy of 96.00% for correctly identifying the tumor-free feature patterns and tumor feature patterns; and an average F1 score of 0.9599 for evaluating the classifier's performance for accurately separating the normality from abnormality. For each fold test, the classifier's computations took an average of 330.0 s of CPU time to complete the tasks, including the training and testing stages. For the same cross-validation tests with 1D feature signals, as seen in Figure 10 (100 Nors, 50 Bs and 50 Ms), with the 2D spatial and 1D CNN-based classifier, the experimental results were shown in Table 4, with an average precision of 96.70% and an average recall of 96.13% for accurately identifying the tumor cases (TPs); an average accuracy of 96.40% for correctly identifying the normality and abnormality; and an average F1 score of 0.9641 was also greater than 0.9000, the higher the better, which indicated the classifier had a great potential prediction capability for quantifying the classifier model. In addition, the recall (%) as the index of PPV was also greater than 80.00%, which indicated the classifier had a predictive performance for identifying the abnormality (TP). Its pattern recognition scheme took an average of 0.6025 s of CPU time to identify the possible breast lesions. Hence, we recommend the use of the 2D spatial and 1D CNN-based classifier to automatically screen the presence of breast lesions on mammographic images in clinical applications. Figure 9: Confusion matrix for classifier’s testing results and formulas for the evaluation criteria.