# Enhanced Feature Selection via Hierarchical Concept Modeling
## 3 The Proposed Models for Feature Selection  
FCA uses a concept lattice structure to systematically organize features based on their generalization and specialization relationships. By identifying clusters of features, FCA can pinpoint attributes that contribute significantly to classification accuracy. In this hierarchical structure, each level of the concept lattice corresponds to a specific set of feature combinations that add distinct value to the classification task. Features that do not contribute to these essential relationships are pruned, allowing FCA to retain a streamlined set of features that still preserves the necessary classification information. This lattice-based approach reduces the dimensionality of the feature set without compromising accuracy, as FCA naturally filters out features that may add noise or redundancy to the model. The retained features are those that contribute most to accurate classification outcomes, ensuring that the reduced feature set is both efficient and effective. The proposed approaches based on hierarchical models focused on two methods, namely FCA and DT. FCA and DT were applied to build the hierarchical structures for feature selection from data so that each node in the structure represented a feature (attribute). The choice of FCA and DT for hierarchical feature selection was motivated by their ability to structure features in a multi-level hierarchy. Unlike traditional flat feature selection methods, which may overlook complex relationships between features, FCA and DT enable the preservation of feature dependencies across levels. This hierarchical organization allows for more informed feature selection, enhancing the model's classification accuracy and interpretability, especially in datasets with intricate class structures. An overview of the proposed models is depicted in Figure 1.  
In developing the hierarchical models for feature selection using FCA and DT, we make the following assumptions: (1) We assume that the dataset contains hierarchical relationships among features that can be effectively captured through a concept lattice or decision tree structure. This structure is assumed to reflect generalization and specialization among features, allowing us to reduce feature dimensionality meaningfully. (2) Each level in the hierarchical model is assumed to contain features that independently contribute to classification accuracy. Redundant features within a level are minimized under the assumption that the FCA and DT structures accurately filter irrelevant or less useful attributes. (3) It is assumed that the feature contributions to classification accuracy remain consistent across the hierarchy. This means that a feature's impact at one level should either be retained or improved at higher levels. This allows us to optimize feature selection sequentially through the hierarchy. Initially, the hierarchical structure form was built, where the top node represents general knowledge, whereas the bottom represents specific knowledge. Next, the top node of structure was chosen by considering each level of structure. In this study, we selected a  
Figure 1: A conceptual overview of the proposed models. feature for each level of the structure, and then the selected feature was used to represent data for classification in the next step of evaluating the performance. Afterward, the relationship between the number count of features (attributes) and classification accuracy was considered. We found that the number of attributes for each level affects classification accuracy at some level. We attempted to optimize the feature selection from the hierarchical structure. Thus, the paired-samples \(t\)-test was applied to choose appropriate features for each level. This approach has advantages, including the estimation of mean differences between paired observations and the identification of statistically significant changes in two related measurements.