# Enhanced Feature Selection via Hierarchical Concept Modeling
## 5 Results and Discussion
### Comparison of Performances
To highlight the comparative advantages of the hierarchical concept model (using FCA and DT), we compared its performance with traditional feature selection methods, such as flat filters and wrappers, across multiple datasets. Our results reveal specific cases where the hierarchical model demonstrates notable benefits over traditional approaches. The hierarchical concept model's structured approach is particularly advantageous in datasets with layered feature relationships and complex class structures. Hierarchical methods like FCA and DT can capture multi-level dependencies between features. For instance, in datasets where features have a natural hierarchy or are interdependent, these methods outperform traditional flat methods by preserving relational structures within the data. This leads to a more meaningful reduction in feature space and improved classification accuracy. In datasets with many classes or intricate class relationships, the hierarchical approach organizes features according to relevance at each level, leading to better feature prioritization and reducing the computational load. For example, in the Soybean dataset, FCA's concept lattice helped isolate critical features while minimizing noise, thereby enhancing performance compared to non-hierarchical methods. Considering the count of selected features, the results show clearly that the DT retained fewer features than the other methods, as shown in Figure 13. This figure will illustrate the number of selected attributes at each level from 1 to 3 in comparison with other methods that exhibit classification performance as shown in Figures 6 and 7 mentioned earlier. Figure 13a displays the count of selected attributes from the decision tree structure, while Figure 13b shows the count of selected attributes from the FCA structure. From as the results, the  
Figure 10: The classification performance comparison in Table 7.  
hierarchical structure using decision tree method can be applied to select the feature for data dimension reduction yields favorable outcomes. This is due to the decision tree's method, which involves a top-down data entry process using specific filtering and selecting root node. However, FCA produces less satisfactory results due to the creation of a layer-wise structure designed for bidirectional ordering--both from top to bottom and from bottom to top. This may lead to effect of feature selection. The experimental results confirm that FCA's concept lattice can achieve dimensionality reduction without compromising classification accuracy. By systematically identifying and retaining only the most informative features, FCA produces a reduced feature set that achieves comparable accuracy to the original, full feature set. This performance validates FCA's effectiveness in filtering out features that are irrelevant or redundant while preserving those that contribute to robust classification outcomes. Compared to traditional feature selection methods, FCA's lattice structure organizes attributes in a way that maintains high classification reliability while minimizing the feature set. This approach optimally balances computational efficiency and model interpretability with performance, making FCA an ideal choice for applications that require both robust classification and efficient feature reduction. While the hierarchical feature selection model demonstrated robust performance with DT, SVM, and ANN classifiers, certain conditions could further enhance its results. For instance, tuning the parameters for each classifier, such as adjusting hyperparameters in SVM or layer configurations in ANN, could improve classification accuracy. Moreover, increasing the hierarchical levels of feature selection allows finer distinctions among features, which can be beneficial when working with high-dimensional datasets. This model can be adapted to other classification algorithms, such as k-nearest neighbors (k-NN) and random forest, by modifying the selection criteria within the hierarchical structure.