# A Grey Wolf-Based Method for Mammographic Mass Classification
## 4 Experimental Results
### Results and Discussion  
As presented in Figure 1, there are two variants of our method. One aims to classify ROI images into normal and abnormal, i.e., detecting abnormal breast cases. The second variant is to detect benign or malignant cases from the ROI images directly, i.e., diagnosing whether an ROI image is benign or malignant. It is notable that the features used in both variants are the same, but the type of features would be different based on the BGWO selection. There are two main scenarios (without using BGWO and with using BGWO) in which the random forest algorithm is used to achieve the classification task. \begin{table}
\begin{tabular}{l c c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multicolumn{2}{c}{**Abnormal ROI**} & \multirow{2}{*}{**Normal ROI**} \\ \cline{2-3}  & **Benign** & & **Malignant** \\ \hline Detection (Normal-Abnormal) & 100 & 100 & 200 \\ Diagnosis (Benign-Malignant) & 100 & 100 & - \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The used ROI datasets for proposed method assessment. Figure 5: Confusion matrix, where \(TP=\) true positives, \(TN=\) true negatives, \(FP=\) false positives and \(FN=\) false negatives. Without using BGWO as a feature selection technique: The first scenario is designed to understand the performance of the feature sets (curvelet, wavelet and combining both curvelet and wavelet) without using the BGWO algorithm as a feature selection technique. The summary of the results of this scenario is shown in Table 2. From the obtained results, the following notes can be mentioned. Firstly, in case of detecting normal or abnormal cases, the best results are achieved when the fusion of both curvelet and wavelet was used with accuracy 99.0%, precision 99.0%, recall 99.0% and ROC area 1.00. These results would make sense as the fused features contain the most discriminative features from each of the wavelet and curvelet. Secondly, in case of diagnosing a raw ROI image, whether it is a benign or malignant case, the fused features and curvelet-based features gave the best results with very close performance in terms of accuracy 74.0%, precision 74.0%, recall 74.0% and ROC area 0.871. The main difference between used features and curvelet-based features is the number of features where the curvelet only needed 810 instead of 970 for the fused one. However, the results of the second scenario are not as good as in detecting normal or abnormal cases. With using BGWO as a feature selection technique: In the second scenario, as in the first one, the same feature sets (i.e., the main features of curvelet (810), wavelet (160) and fused curvelet with wavelet (970)) will be fed to the BGWO algorithm to select the best feature set that would discriminate between different classes (normal, abnormal, benign or malignant). This scenario is further divided into different sub-scenarios based on the feature type as shown below. Before describing the results of these two scenarios, it is worth mentioning that BGWO algorithm is a stochastic technique. [PERSON] et al. [31] reported that the stochastic algorithms need to be run at least 10 times to produce meaningful results. Usually, the average of these results is used to report the performance of these types of heuristic algorithms. In this paper, we followed the same approach where the proposed BGWO-based method was run 20 times with each run, one set of features was produced and then fed to the classifier (random forest) to evaluate whether this set would give the best results in terms of accuracy, precision, recall and ROC. Tables 3-8 present a summary of these results where the best result of these 20 sets is highlighted in each table. Specifically, Tables 3-5 summarize the results of using BGWO to detect normal or abnormal cases while Tables 6-8 summarize the results of using BGWO to detect benign and malignant cases.