# Enhanced Feature Selection via Hierarchical Concept Modeling
## 5 Results and Discussion
### The Hierarchical Concept Model Performance
#### 5.2.2 Feature Selection Using FCA
This improvement across levels aligns with FCA's role in selecting more specific feature combinations as the hierarchy expands, allowing for more refined data representation. For datasets like Dermatology and Zoo, accuracy remains relatively stable across all levels, with minor fluctuations. This stability indicates that FCA-based selection effectively identifies a core set of important features early on, and additional levels do not significantly alter the classification performance. This result suggests that, for these datasets, FCA's early levels capture the most relevant information, making deeper feature selection levels less impactful. The heatmap highlights variability in performance across different classifiers. For instance, ANN achieves high accuracy on datasets such as Segmentation and Soybean at all levels, while SVM and DT sometimes lag in accuracy. This variability underscores the adaptability of FCA-based selection: certain classifiers, especially those more sensitive to specific features or complex interactions (like ANN), benefit more from the hierarchical structure than others. In some cases, specific levels offer optimal feature sets. For example, Glass and Iris datasets achieve high accuracy with minimal levels (LV.1 or LV.2), indicating that these datasets require only a few select features for effective classification. This observation supports the advantage of FCA-based selection in balancing dimensionality reduction with performance, especially for simpler datasets where adding more features would not yield substantial accuracy gains. A few datasets, such as Lung-cancer and Zoo, demonstrate lower accuracy across most levels, particularly for certain classifiers. This pattern may reflect inherent complexities in these datasets that are not fully captured by FCA-selected features or limitations in the classifiers used. It suggests that alternative selection strategies or more complex models may be required to handle such cases effectively. Likewise, we compared classification accuracy of our feature selection using FCA to the benchmark original features (without using feature selection at all) in Section 5.1, as shown in Figure 6.  
From Figure 6, we see that the classification accuracy is not different between the original and the proposed model at each level. Indeed, if we add the number of attributes from each level, the trend in accuracy is increasing. From the testing above, we find that the classification accuracy is not different between the original and the proposed model for each level. Thus, we can compare using paired-samples \(t\)-tests as mentioned in Section 3. On considering the relationship between the number of selected attributes and the average classification accuracy for datasets, as shown in Figure 7, we found that the proposed model using a decision tree gave a better classification result than the model using FCA. \begin{table}
\begin{tabular}{l c c c c c c c c c c c c c c c c c c c c c c} \hline \hline  & & \multicolumn{10}{c}{**The Classification Accuracy \(\varphi_{3}\) Using PCA-Based Feature Selection**} \\ \cline{2-13}
**Dataset** & \multicolumn{4}{c}{**LV1**} & \multicolumn{4}{c}{**LV2**} & \multicolumn{4}{c}{**LV3**} & \multicolumn{4}{c}{**LV4**} & \multicolumn{4}{c}{**LV5**} \\ \cline{2-13}  & \multicolumn{2}{c}{**No. **} & \multicolumn{2}{c}{**MT**} & \multicolumn{2}{c}{**SVM**} & \multicolumn{2}{c}{**ANN**} & \multicolumn{2}{c}{**No. **} & \multicolumn{2}{c}{**Attribute**} & \multicolumn{2}{c}{**DT**} & \multicolumn{2}{c}{**SVM**} & \multicolumn{2}{c}{**ANN**} & \multicolumn{2}{c}{**No. **} & \multicolumn{2}{c}{**HT**} & \multicolumn{2}{c}{**SVM**} & \multicolumn{2}{c}{**ANN**} & \multicolumn{2}{c}{**No. **} & \multicolumn{2}{c}{**Attribute**} & \multicolumn{2}{c}{**DT**} & \multicolumn{2}{c}{**SVM**} & \multicolumn{2}{c}{**ANN**} \\ \cline{2-13}  
Figure 5: The classification accuracy using PCA-based feature selection across levels and classifiers (Table 5).