# Enhanced Feature Selection via Hierarchical Concept Modeling
## 5 Results and Discussion
### The Hierarchical Concept Model Performance
#### 5.2.3 An Example of Feature Selection Optimization  
In our feature selection, we select the top node of structure to be representative of dataset. However, the proper selection of relevant attribute leads to an efficient classification. For example, in Tables 4 and 5, we chose datasets (Dermatology, Glass, and Iris) and applied three classifiers (DT, SVM, and ANN) to illustrate a line plot of classification accuracy across hierarchical levels (Levels 1 to 5), as shown in Figure 8. This visualization demonstrates how classification performance improves with higher hierarchical levels, providing insight into the incremental impact of feature selection at each level for each classifier and dataset. Both graphs show that as the hierarchical level increases, accuracy gradually improves. However, after reaching a certain level, the accuracy plateaus and remains constant. By systematically adjusting the number of hierarchical levels (from LV.1 to LV.5) and observing corresponding accuracy changes, we gained insights into the optimal level configurations for each dataset. Specifically, we used paired-samples statistical tests to compare accuracy across levels, ensuring that each selected level added meaningful improvement without redundant features. Thus, we experimented with using the paired-samples \(t\)-test to optimize the selected level. We use the experimental results from Table 4 to demonstrate the proper selection of the node attribute in each level. The Dermatology dataset was investigated to show the calculation for selecting attribute in each level of the tree. We assume the hierarchical structure of the Dermatology dataset, and the classification accuracies are shown in Figure 9.  
From Figure 9, we can apply the proposed feature selection in an example calculation. The initial attribute node (or root) is selected with an accuracy of 64.94%. Next, we consider the next step for selection or not by using the Equation (2) where \(n\)= 366, \(\hat{p}\) = \(\frac{\sum_{i=1}^{n}Acc_{(i)}+\sum_{j=1}^{n}Acc_{(i)-1}}{n+n}=\frac{245+270}{3 66+366}=0.704\), \(\hat{q}=1-\hat{p}=0.296\), thus  
\[Z_{|J-1}=\sqrt{\frac{366}{22*0.704*0.296}}(73.77-64.94)=42.18.\]  
Figure 8: The classification accuracy across hierarchical levels for selected datasets. (**a**) Feature selection using decision tree. (**b**) Feature selection using FCA.  
Figure 9: An example hierarchical structure using decision tree and the accuracies with selected features.  
From Table 1, if we consider the level of significance 0.01, if \(|Z_{l,l-1}|\leq 2.576\), then we select level 1 (LV.1 in Figure 9). Next, we still consider the next level (LV.2). We calculate \(z_{2,1}=-1.61\) and find that this value is less than \(z\) in Table 1. Thus, LV.2 is not selected, and we stop the process. Thus, in this structure, we choose attributes from the structure in LV.0 to LV.2.  
In addition, we estimate for all datasets which levels should be chosen. We applied the statistical paired-samples \(t\)-test to estimate this across all the datasets shown in Table 6, where \(x\) is the mean of the accuracy values of the data set for each level and \(SD\) is the standard deviation. We find that the feature selection from each level depends on the classifier model. Namely, we can use two levels in a decision tree classifier, whereas we should select four levels for using SVM or ANN classifiers.