# Enhanced Feature Selection via Hierarchical Concept Modeling
## 2 Related Works
Conventional feature selection methods, like filter, wrapper, and embedded approaches, often struggle to capture the complex, layered interactions between features in datasets where hierarchical relationships exist. This has led to the development of hierarchical feature selection techniques, among which FCA and DT have been widely applied. While FCA and DT form a foundational basis for hierarchical selection, other methods have emerged, such as fuzzy rough sets and multi-granularity clustering structures, which provide additional insights. For instance, the fuzzy rough sets method combines the robustness of rough set theory with the flexibility of fuzzy logic, allowing for the handling of uncertain or imprecise data in a hierarchical format. Studies show that fuzzy rough sets are particularly effective for data with varying levels of uncertainty across different hierarchical layers, making them well suited for classification tasks where feature relevance may be ambiguous or context-dependent [5]. The multi-granularity clustering structures approach organizes features at different granularity levels, allowing for a flexible clustering of features based on varying levels of detail. Multi-granularity structures can adapt to both broad and specific feature relationships, providing a more nuanced hierarchy than flat clustering methods. They have shown promise in applications requiring both high-level summaries and detailed feature differentiation. These alternative methodologies highlight the flexibility of hierarchical models in capturing feature dependencies across multiple layers, further emphasizing the limitations of traditional, non-hierarchical methods in structured classification tasks [12]. Recent advancements have explored hybrid methodologies that combine hierarchical approaches with other selection techniques to enhance the adaptability and performance of feature selection [16]. The hybrid hierarchical-filter methods such as information gain or chi-square allow for the quick preliminary reduction of features before building a hierarchical structure. This hybrid approach leverages the strengths of each method, achieving both computational efficiency and structured feature prioritization. Hybrid methods incorporating fuzzy logic into hierarchical models, such as fuzzy FCA, enhance flexibility in handling complex and imprecise feature relationships. This advancement allows for more nuanced feature selection in datasets where feature relevance is context-dependent or fluctuates across hierarchical levels. All the above-mentioned studies assumed specific relationships between categories for hierarchical regularization. However, most of the existing hierarchical feature selection methods are not robust when dealing with the inevitable data outliers, resulting in a serious inter-level error propagation problem in the classification process that follows. Thus, FCA is applied in this current study because the previous experimental studies have shown that FCA is not sensitive to outliers [7]. FCA provides a well-defined mathematical framework to discover implicit and explicit knowledge in an easily understood format by using formal context and a Hasse diagram that clearly represents the concepts' generalization/specialization relationships. In addition, FCA can construct an informative concept hierarchy providing valuable information on various specific domains. To support this work, [PERSON] et al. [15] also applied the FCA method to present a new filter for feature selection, called H-Ratio, which can identify pertinent features from data based on the Shannon entropy, also known as the diversity Shannon index, which reflects how many different types there are in a dataset to select features without considering classification accuracy. However, the classification accuracy is an important aspect of selection. Thus, this work applied the classification accuracy to formulate for feature selection from concept lattice.