# Enhanced Feature Selection via Hierarchical Concept Modeling
## 6 Conclusions  
Feature selection methods are used in a wide variety of applications. Such methods proceed to eliminate unfavorable features that may be noisy, redundant or irrelevant, as these can penalize the performance of a classifier. Feature selection reduces the dimensionality of data and restricts the inputs to a subset of the original, but the features can have missing values. In this paper, we propose a new feature selection method based on a hierarchical concept model to improve the performance of supervised learning. We applied a decision tree and FCA to build the hierarchical structures from data to generate knowledge-based systems. In these hierarchical structures that embed knowledge, the top and bottom nodes represent general and specific knowledge, respectively. For this reason, we selected the top node of each structure to be representative of the dataset. However, the proper selection of relevant attributes leads to an efficient classification. Thus, we also used paired-samples \(t\)-tests to optimize the selected levels in the hierarchical structures. In this study, numerical experiments used available software, RapidMiner Studio v. 9.2.0 and ConExp v. 1.3 programs, to generate the hierarchical structures for decision tree and FCA, respectively. Afterward, the attributes in top node of the obtained structure were selected to use in learning classifier models. Three popular classifier algorithms were used in this study: decision trees, SVM, and ANN. To compare the performances of our feature selection methods with available alternatives: all original features (without using feature selection) and the prior feature selection methods. We used 10 datasets from the UCI Machine Learning Repository for classification. The prior feature selection methods tested were IG, CS, FS, and BS. The results show clearly that the proposed models with DT and FCA gave smaller set of retained attributes than the other methods, while still classification performance remained comparable. In other words, the new approach provided superior data reduction to the least dimensionality without performance sacrifice. In summary, this study demonstrates the strengths of hierarchical feature selection methods, specifically FCA and DT, over traditional flat methods in scenarios involving complex feature interdependencies or layered class structures. The hierarchical model's ability to retain structured relationships and prioritize relevant features enables it to outperform conventional approaches in both feature reduction and classification accuracy for multi-level datasets. This makes hierarchical methods a valuable approach for feature selection in data-intensive applications requiring nuanced feature analysis. Future work could focus on refining the hierarchical feature selection model by exploring more dynamic methods of defining hierarchical relationships among features. Integrating adaptive methods, such as reinforcement learning, to determine optimal hierarchical levels could further improve model efficiency and adaptability to diverse datasets. Additionally, investigating other classifiers, including ensemble models like gradient boosting machines (GBMs) and random forest, could provide insights into the model's adaptability across various machine learning paradigms. Figure 13: The number of selected features by **(a)** DT and **(b)** FCA methods in each level, compared to the other selection methods. **Author Contributions:** Conceptualization, J.M. ; Methodology, J.M. and B.C. ; Software, P.W. and J.S. ; Validation, A.W., A.I., J.S., L.B. and B.C. ; Formal analysis, A.W., L.B. and B.C. ; Writing - original draft, J.M. ; Writing--eview & editing, P.W., A.W., A.I., J.S., L.B. and B.C. ; Visualization, A.I. ; Supervision, J.M. ; Funding acquisition, J.M. All authors have read and agreed to the published version of the manuscript. This research was supported by National Science, Research and Innovation Fund (NSRF) and Prince of Songkla University (Ref. No. (SIT6601016S)). Not applicable.