## 5 Discussion  
In many cases, the data available in databases or algorithms that extract information from 1D and 2D signals or sensors are correlated with each other, often providing much less new information compared to uncorrelated data. Therefore, in such cases--especially in a classification problem where the number of available examples is small compared to the amount of information per example, and even more so when there are many classes with an uneven distribution of examples--we are faced with a complex classification problem. For this reason, we can often achieve comparable or even improved classification rates in much less time by first applying a feature selection step. Dedicated feature selection methods exist for this purpose, falling into two main categories (due to being excessively time consuming, wrapper methods have not been considered):  
1. Filter methods, which evaluate each feature independently of the classification model, using statistical measures to determine its relevance;
2. Embedded methods, which perform feature selection during model training, simultaneously optimizing the model's coefficients. Filter Methods Performance The Kruskal-Wallis method achieved the best classification rate among the tested methods, reaching 75.9% for 150 features, which is even higher than the original dataset's performance (74.8%) using all 279 features. This indicates that removing irrelevant or redundant features can improve model accuracy. Additionally, with just 50 features, the Kruskal-Wallis test still achieved 75.7%, demonstrating its effectiveness in maintaining a strong performance with a reduced feature set. The MRMR method, known for selecting relevant features while minimizing redundancy, achieved 75% accuracy with 125 features, comparable to the original dataset, and 74.6% with 100 features. In contrast, the chi-square (CHI2) and ANOVA tests did not perform as well, showing a more significant performance drop when the number of features was reduced. For example:  
* CHI2 with 100 features: 73.2% accuracy;
* ANOVA with 100 features: 72.6% accuracy. Among all tested methods, the least squares (LS) and ReliefF methods performed the worst, achieving only 65.3% and 64.6% accuracy, respectively, for 50 features. Although their performance improved with more features, neither method came close to matching the performance of the full dataset. \begin{table}
\begin{tabular}{l l l} \hline \multicolumn{3}{l}{**Onological Databases—216 Instances Grouped into 2 Classes (Normal and**} \\
**Patological), with 4000 Features for Each Instance** \\ \hline \hline \multicolumn{3}{l}{**No. of Features**} & \multicolumn{1}{l}{**Classification Rate**} \\ \hline \hline
**ORIGNAL DATA\({}^{\star}\)** & **4000 features** & **98.6\%** \\ CS\_Random Projections & 100 features & 98.1\% \\ Lasso & **203 features** & **100\%** \\ GroupLasso & **91 features** & **100\%** \\ OMP with no feature setting & **400 features** & **100\%** \\
**BP\_ADMM\({}^{\star}\)** & **107 features** & **100\%** \\ \hline \hline \end{tabular} \({}^{\star}\)The results for original data and the best results for feature selections are written in bold.  
\end{table} 
Table 5: Classification results for oncological databases with all data and feature selection obtained using standard methods in APP Machine Learning. Embedded Methods Performance  
Unlike the faster but simpler filter methods, embedded methods are more advanced, relying on optimization techniques and penalized regression. From our analysis of two datasets, the BP_ADMM method (basis pursuit using alternating direction method of multipliers) achieved the highest classification rate, reaching 77% accuracy with 248 features. This outperformed the original dataset's performance, suggesting that careful feature selection can improve classification results. The LASSO method, which applies L1 regularization to eliminate less relevant features, achieved 73.2% accuracy with just 72 features--very close to the original dataset's performance but with a significantly reduced feature set.