# Enhanced Feature Selection via Hierarchical Concept Modeling
## 5 Results and Discussion
### The Performances of Prior Feature Selection Approaches  
This section evaluates the classification performance of feature selection using IG, CS, FS, and BS. IG and CS algorithms function as filter methods, while FS and BS algorithms operate as wrappers. These algorithms were implemented in the RapidMiner Studio program, representing traditional feature selection methods for comparison with our proposed model. The 10 datasets listed in Table 2 were employed for training and testing through a 10-fold cross-validation approach. The experimental results, presented in Table 7, present the use of DT, SVM, and ANN as classifiers, with the \"_No._\" column indicating the number of selected attributes. The classification accuracies reported in this table will be further compared in the subsequent section. To improve clarity and facilitate comparisons, we provide a color-coded heatmap visualization for Table 7, shown in Figure 10. This heatmap represents the classification accuracy results across various feature selection methods (IG, CS, FS, BS) and classifiers (DT, SVM, ANN) for each dataset. The color intensity in each cell corresponds to the accuracy level, with darker shades indicating higher accuracy. Figure 10 illustrates the classification performance for each method and classifier on different datasets. This visual format allows for quick identification of the most effective feature selection method and classifier combination for each dataset. The heatmap makes it easier to compare performance across different configurations, addressing a reviewer's suggestion to include enhanced data interpretation. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multicolumn{1}{c}{**Methods**} & **Level** & \(\mathbf{x\ \pm SD}\) & **Paired \(t\)-Test** & **Sig.** \\ \hline \multirow{6}{*}{DT} & LV.1 & \(86.69\pm 8.39\) & 2.97 & **0.016** \\ \cline{2-5}  & LV2 & \(89.14\pm 6.74\) & 2.55 & **0.031** \\ \cline{2-5}  & LV3 & \(91.00\pm 7.13\) & 1.59 & 0.146 \\ \cline{2-5}  & LV4 & \(92.43\pm 3.07\) & 2.22 & 0.053 \\ \cline{2-5}  & LV5 & \(92.51\pm 2.17\) & 3.03 & 0.014 \\ \hline \multirow{6}{*}{SVM} & LV.1 & \(79.34\pm 3.12\) & 4.88 & **0.001** \\ \cline{2-5}  & LV2 & \(77.70\pm 4.07\) & 4.15 & **0.002** \\ \cline{2-5}  & LV3 & \(84.54\pm 3.52\) & 2.85 & **0.019** \\ \cline{2-5}  & LV4 & \(92.27\pm 0.94\) & 2.46 & **0.036** \\ \cline{2-5}  & LV5 & \(93.21\pm 0.88\) & 1.56 & 0.153 \\ \hline \multirow{6}{*}{ANN} & LV.1 & \(86.28\pm 12.67\) & 2.55 & **0.031** \\ \cline{2-5}  & LV2 & \(88.00\pm 6.50\) & 3.20 & **0.011** \\ \cline{1-1} \cline{2-5}  & LV3 & \(89.51\pm 6.20\) & 2.59 & **0.029** \\ \cline{1-1} \cline{2-5}  & LV4 & \(92.02\pm 3.45\) & 2.35 & **0.043** \\ \cline{1-1} \cline{2-5}  & LV5 & \(92.85\pm 3.33\) & 1.65 & 0.133 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The paired-samples \(t\)-test results. As shown in the results, the IG and CS methods prioritize features with strong discriminatory power, which directly boosts classifier performance, particularly in datasets with well-defined feature relevance, such as Dermatology and Glass. IG, in particular, enhances the performance of DT and SVM by focusing on features that most impact classification decisions. Similarly, FS and BS methods refine the feature space by iteratively selecting or removing features based on their importance. This targeted approach leads to high accuracy in datasets like Pageblocks and Segmentation, where complex feature relationships are effectively reduced to only the most critical attributes for classification. Feature importance also varies across classifiers. For instance, DT benefits from feature selection methods that retain features with hierarchical significance, aligning well with its split-based structure. In contrast, SVM performs best with methods that maintain a broad set of influential features, such as those identified by CS. ANN, on the other hand, benefits from a more nuanced feature selection approach, as seen in the results for Soybean and Wine.