SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We report dataset-wise downstream performance based on the proposed benchmark, as well as overall rank of the methods in Tables 1, 2, and 3 and we include results with standard errors computed across seeds in Tables 16, 17, 18. While we did not discover a universally optimal feature selection method that performed the best across all datasets and extra feature types, we did identify several noteworthy patterns, which we will discuss below.
Random Features. In the easy scenario where extraneous features are Gaussian noise, we observe in Table 1 that XGBoost, Random Forest, univariate statistical test and Deep Lasso perform on par for MLP downstream models, while for FT-Transformer downstream models, Random Forest and XGBoost outperform other methods. Conversely, Lasso, 1L Lasso, AGL, LassoNet and Attention Map Importance are less competitive. These findings align with the results depicted in Figure 2, which highlight the high similarity in importance rank between the methods and the greater tendency of Lasso based methods to assign higher ranks to random features. In addition to downstream performance, we report ROC-AUC and precision scores in Tables 11, 12.
Corrupted Features. In a more challenging scenario involving corrupted extra features, both Deep Lasso and XGBoost significantly outperform the other feature selection methods. Specifically, Deep Lasso exhibits superior performance for the MLP downstream model, while XGBoost performs slightly better for the FT-Transformer downstream model, see Table 2.
Second-Order Features. Finally, in the most challenging scenario of choosing among original and second-order features, Deep Lasso demonstrates a significant performance advantage over the other methods, see Table 3. Interestingly, we discover that the relative rank of Deep Lasso is lower when  $75\%$  of all features were generated, indicating that Deep Lasso excels in more challenging feature
selection problems involving a significant proportion of spurious or redundant features, as observed in both corrupted and second-order extra features.
7 Similarity between Feature Selection Methods
In this section we analyze which feature selection algorithms are more similar to each other in terms of feature ranking. In Figure 3, we present a heatmap of pair-wise Spearman correlation between feature rankings of different FS methods averaged across datasets. We focus on the setup involving second-order features, as we discovered the lowest level of agreement between methods within this configuration. It is worth noting that we provide results for methods fine-tuned for both MLP and FT-Transformer models, ensuring a comprehensive comparison.
We find related similarity patterns for both downstream models. In particular, we observe that Random Forest and XGBoost rankings exhibit high correlation, indicating a strong agreement between these both tree-based methods. We find that First-Layer Lasso is highly correlated with Adaptive Group Lasso, which is an extension of First-Layer Lasso. Additionally, Random Forest results are also correlated with results from the Univariate Test. Deep Lasso is most correlated with Lasso-based methods, such as classical Lasso, First-Layer Lasso and Adaptive Group Lasso and with Attention Map Importance.
8 Discussion
In this paper, we present a challenging feature selection benchmark for deep tabular models, addressing the need for more realistic scenarios in feature selection evaluation. Our benchmark encompasses real-world datasets with extraneous uninformative, corrupted, and redundant features. Through comprehensive experimentation, we compare various feature selection methods on our proposed benchmark. Our findings reveal that while classical feature selection methods, including tree-based
Figure 3: Similarity between feature selection methods tuned for downstream MLP and FT-Transformer models. Heatmap depicts pair-wise Spearman correlation of feature importances for the setup with second-order features. Correlations are averaged across the datasets.
algorithms perform competitively in the random and corrupted setups, specialized deep tabular feature selection methods, like the proposed Deep Lasso outperform other methods in selecting from second-order features. This indicates the benefits of neural network inductive biases in feature selection algorithms. Overall, our study contributes a systematic new benchmark with analysis, a new feature selection method, and insights into improving the performance and robustness of deep tabular models. The benchmark code and Deep Lasso implementation are made available to facilitate reproducibility and practical usage.
This work was made possible by the ONR MURI program and the AFOSR MURI program. Commercial support was provided by Capital One Bank, the Amazon Research Award program, and Open Philanthropy. Further support was provided by the National Science Foundation (IIS-2212182), and by the NSF TRAILS Institute (2229885).
R. Agarwal, A. Muralidhar, A. Som, and H. Kowshik. Self-supervised representation learning across sequential and tabular features using transformers. In NeurIPS 2022 First Table Representation Workshop, 2022. URL https://openreview.net/forum?id=wIJI1mr1Dsk.
T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2623-2631, 2019.
S. O. Arik and T. Pfister. Tabnet: Attentive interpretable tabular learning. In AAAI, volume 35, pages 6679-6687, 2021.
S. Badirli, X. Liu, Z. Xing, A. Bhowmik, K. Doan, and S. S. Keerthi. Gradient boosting neural networks: Grownet. arXiv preprint arXiv:2002.07971, 2020.
P. Baldi, P. Sadowski, and D. Whiteson. Searching for exotic particles in high-energy physics with deep learning. Nature communications, 5(1):4308, 2014.
M. F. Balin, A. Abid, and J. Zou. Concrete autoencoders: Differentiable feature selection and reconstruction. In International conference on machine learning, pages 444-453. PMLR, 2019.
B. Zhu, X. Shi, N. Erickson, M. Li, G. Karypis, and M. Shoaran. Xtab: Cross-table pretraining for tabular transformers. arXiv preprint arXiv:2305.06090, 2023.
Z. Zhu, Y. Fan, Y. Kong, J. Lv, and F. Sun. Deep learning inference using knockoffs with applications to genomics. Proceedings of the National Academy of Sciences, 118(36):e2104683118, 2021.
A Limitations
In this section we discuss limitations of our work. We note that Deep Lasso feature selection is more computationally demanding than using GBDT-based feature selection methods, especially when large-scale hyperparameter optimization is performed. In fact, for this work we ran over 80 thousand experiments. Combined with computationally intense methods, this may have resulted in excessive carbon emissions. In addition, while we introduce challenging feature selection tasks and use sizeable datasets, the scale of these datasets is still smaller than truly large-scale industrial tabular data with billions of samples.
B Benchmark Detail
Our benchmark comprises of datasets used in [Gorishniy et al., 2021, 2022]. We select datasets with up to 150 numerical features and no categorical features.
- ALOI [Geusebroek et al., 2005] (AL, image data)
- California Housing [Pace and Barry, 1997] (CH, real estate data)
- Covertype [Blackard and Dean, 1999] (CO, forest characteristics)
- Eye Movements [Salojärvi et al., 2005] (EY, eye movement trajectories)
- Gesture Phase Prediction [Madeo et al., 2013] (GE, gesture phase segmentation)
- Helena [Guyon et al., 2019] (HE, anonymized)
- Higgs 98 [Baldi et al., 2014] (HI, simulated physical particles)
House  $16\mathrm{H}^2$  (HO, house pricing)
- Jannis [Guyon et al., 2019] (JA, anonymized)
- Microsoft [Qin and Liu, 2013] (MI, search queries)
- Otto  ${}^{3}$  (OT,product categories)
- Year [Bertin-Mahieux et al., 2011] (YE, audio features)
Table 4: Dataset Details. Number of classes is reported for classification problems.
AL	CH	CO	EY	GE	HE	HI	HO	JA	MI	OT	YE
# samples	108000	20640	581012	10936	9873	65196	98050	22784	83733	1200192	61878	515345
# features	128	8	54	26	32	27	28	16	54	136	93	90
# classes	1000	-	7	3	5	100	2	-	4	-	9	-
B.2 Extraneous Features
In the random features setup we draw extraneous features from the Gaussian distribution. After preprocessing the original features with quantiler transformation, both original and extraneous features follow normal distribution. In the corrupted features setup we randomly sample  $n$  original features and corrupt them with Gaussian noise with the same standard deviation  $x_{c} = 0.5x + 0.5\epsilon, \epsilon \sim \mathcal{N}(0,\sigma)$ . For generating second-order features we randomly sample two subsets of original features and compute their products.
$^{2}$ https://www.openml.org/search?type=data&sort=runs&id=574
<sup>3</sup>https://www.kaggle.com/c/otto-group-product-classification-challenge/data
B.3 Data Preprocessing
We follow [Gorishniy et al., 2021] and apply quantiler transformation to normalize numerical features in all datasets, except for Helena and ALOI, where standardization is applied instead. For the regression tasks we also standardize the target variable. Due to limited computational resources, we restrict the number of samples in the Microsoft dataset to 500,000. We reserve  $20\%$  of samples for test set and  $15\%$  of samples as a validation set for tuning the hyperparameters of feature selection and downstream models.
B.4 Code
We include the benchmark code as well as instructions for reproducing our results in the supplementary. We will release our code as a GitHub Repository under the MIT License.
C Experimental Detail
C.1 Hardware
We ran our experiments on NVIDIA GeForce RTX 2080 Ti machines. Each hyperparameter tuning experiment took from 1 to 72 hours depending on the model and the dataset size. Overall we ran 756 experiments, each with hyperparameter tuning for 100 trials and 10 subsequent seed runs, which resulted in  $\sim 83160$  training runs.
C.2 Implementation Licenses
For the model implementations we adapt the code from the following publicly available repositories and libraries:
- MLP and FT-Transformer models from RTDL repository  ${}^{4}$  under MIT License
- GBDT model from XGBoost library  ${}^{5}$  under Apache License
- Random Forest, Lasso and Univariate statistical test models from Scikit-Learn library  ${}^{6}$  under BSD 3-Clause License.
C.3 Training Details
We train all deep tabular models for 200 epochs with early stopping after 20 epochs, meaning that training stops if the validation accuracy does not improve after 20 consecutive epochs. XGBoost models are trained with a maximum of 2000 estimators and 50 early stopping rounds. We utilize the AdamW optimizer and apply a Linear learning rate scheduler after epochs 40 and 80. The batch size for all experiments is set to 512. Other training parameters, including learning rate and weight decay, are determined through hyperparameter tuning.
C.4 Statistical Significance
We run each experiment with 10 random model initializations (seeds) after we find the optimal hyperparameters for feature selection and downstream models. We report average test metrics in Tables 1 2 3 as well as model ranks. When calculating the average ranks, we consider the statistical significance of performance differences among the models. Specifically, we assign rank 1 to the top-performing models that do not exhibit statistically significant differences. We determine statistical significance with the one-sided Wilcoxon Rank-Sum test Wilcoxon [1945], Mann and Whitney [1947] with  $p = 0.05$ .
<sup>4</sup>https://github.com/Yura52/tabular-dl-revisiting-models
<https://xgboost.ai/
<sup>6</sup>https://scikit-learn.org/stable/
D Hyperparameter Tuning
We carefully tune the parameters of all models for each experiment using the Bayesian hyperparameter optimization library Optuna[Akiba et al., 2019]. For the feature selection experiments we tune parameters for both feature selection and downstream models simultaneously to optimize for downstream validation accuracy. We run 100 trials to identify the optimal parameters for each experiment. We adapt the hyperparameter search spaces from the original papers with slight modifications, which are provided below.
- Hyperparameters for the MLP model are provided in Table 5
- Hyperparameters for the FT-Transformer model are provided in Table 6
- Hyperparameters for the XGBoost feature selection model are provided in Table 7
- Hyperparameters for the Random Forest feature selection model are provided in Table 8
- Hyperparameters for feature selection regularizers (Lasso, Deep Lasso, First-Layer Lasso, Adaptive Group Lasso) are provided in Table 9
Table 5: Optuna hyperparameter search space for MLP
Parameter	Search Space
Number of layers	UniformInt[1, 8]
Layer size	UniformInt[1, 512]
Dropout	{0, Uniform[0, 0.5]}
Learning rate	LogUniform[1e-5, 1e-2]
Weight decay	{0, LogUniform[1e-6, 1e-3]}
Table 6: Optuna hyperparameter search space and default configuration for FT-Transformer
Parameter	Search Space	Default
Number of layers	UniformInt[1, 4]	3
Residual dropout	{0, Uniform[0, 0.2]}	0.0
Attention dropout	Uniform[0, 0.5]	0.2
FFN dropout	Uniform[0, 0.5]	0.1
FFN factor	Uniform[2/3, 8/3]	4/3
Learning rate	LogUniform[1e-5, 1e-3]	1e-3
Weight decay	LogUniform[1e-6, 1e-3]	2e-4
Table 7: Optuna hyperparameter search space for XGBoost
Parameter	Search Space
Max depth	UniformInt[3, 10]
Min child weight	LogUniform[1e-8, 1e5]
Subsample	Uniform[0.5, 1]
Learning rate	LogUniform[1e-5, 1]
Col sample by tree	Uniform[0.5, 1]
Gamma	{0, LogUniform[1e-8, 1e2]}
Lambda	{0, LogUniform[1e-8, 1e2]}
Table 8: Optuna hyperparameter search space for Random Forest
Parameter	Search Space
Num estimators	UniformInt[10, 2000]
Max depth	UniformInt[3, 10]
Table 9: Optuna hyperparameter search space for Feature Selection regularizers penalty weights
Feature Selection	Penalty Search Space
Lasso	Uniform [1e-3,5e-1]
Deep Lasso	LogUniform[1e-2,5e-1]
First-Layer Lasso	LogUniform[1e-2,5e-1]
Adaptive Group Lasso	LogUniform[1e-3,5e-1]
For certain datasets training XGBoost (ALOI dataset) and FT-Transformer (Microsoft, Year, ALOI, Covtype) models can be excessively time-consuming. Due to these time constraints, we do not perform parameter tuning for these particular pairs of models and datasets and instead utilize the default configurations of the models. We also note that the computational complexity of attention mechanisms grows quadratically as the number of features in the dataset increases. Finally, LassoNet implementation performs hyperparameter tuning automatically, therefore we do not use Optuna to tune LassoNet model, and only tune parameters of the downstream models.
E Deep Lasso for Linear Regression
Consider the case of linear regression, where the model is
$$
f _ {w, b} (X) = X w ^ {T} + b.
$$
We follow the PyTorch-style notation and  $w$  is a row vector with  $m$  entries where  $m$  is the number of features - columns of  $X$ . We show that Deep Lasso applied to the model output (rather than loss) is equivalent to regular Lasso and First-Layer Lasso.
Deep Lasso applied to the model output is given by:
$$
\min  _ {\theta} \alpha \mathcal {L} _ {\theta} (X, Y) + (1 - \alpha) \sum_ {j = 1} ^ {m} \left\| \frac {\partial f _ {w , b} (X)}{\partial X ^ {(j)}} \right\| _ {F}. \tag {3}
$$
Now, for  $j$ -th feature (column of  $X$ ), we have
$$
\frac {\partial f _ {w , b} (X)}{\partial X ^ {(j)}} = \frac {\partial (X w ^ {T} + b)}{\partial X ^ {(j)}} = w _ {j} \mathcal {I} _ {n}.
$$
That is, we get:
$$
\sum_ {j = 1} ^ {m} \left\| \frac {\partial f _ {w , b} (X)}{\partial X ^ {(j)}} \right\| _ {F} = \sum_ {j = 1} ^ {m} \| w _ {j} \mathcal {I} _ {n} \| _ {F} = \sum_ {j = 1} ^ {m} | w _ {j} | \| \mathcal {I} _ {n} \| _ {F} = \| w \| _ {1} \sqrt {n}, \tag {4}
$$
Finally, we get the equivalence of Deep Lasso to regular Lasso (with the adjustment of the Lasso penalty by  $\sqrt{n}$ ):
$$
\left. \min  _ {\theta} \alpha \mathcal {L} _ {\theta} (X, Y) + (1 - \alpha) \sum_ {j = 1} ^ {m} \left\| \frac {\partial f _ {w , b} (X)}{\partial X ^ {(j)}} \right\| _ {F} = \min  _ {\theta} \alpha \mathcal {L} _ {\theta} (X, Y) + (1 - \alpha) \sqrt {n} \| w \| _ {1}. \right. \tag {5}
$$
Additionally, we show the equivalence to First-Layer Lasso too. First-Layer Lasso applies a Group Lasso penalty to the weights of the first layer parameters:
where  $W^{(j)}$  is the  $j$ -th column of weight matrix of the first hidden layer corresponding to the  $j$ -th feature. However, in the case of linear regression, the first layer output coincides with the model
Table 10: Benchmarking feature selection methods for MLP and FT-Transformer downstream models on datasets with corrupted with Laplace noise extra features.
%	FS method + Model	AL	CH	EY	GE	HE	HI	HO	JA	MI	OT	YE	rank
50	No FS + MLP	0.942	-0.471	0.546	0.525	0.372	0.804	-0.616	0.704	-0.909	0.781	-0.797	8.27
50	Univariate + MLP	0.955	-0.448	0.532	0.531	0.345	0.810	-0.627	0.717	-0.920	0.791	-0.827	7.00
50	Lasso + MLP	0.953	-0.456	0.560	0.507	0.374	0.810	-0.594	0.718	-0.902	0.797	-0.789	6.55
50	1L Lasso + MLP	0.955	-0.445	0.594	0.521	0.378	0.816	-0.572	0.721	-0.902	0.798	-0.776	3.09
50	AGL + MLP	0.955	-0.443	0.570	0.518	0.384	0.810	-0.577	0.721	-0.900	0.8	-0.782	3.36
50	LassoNet + MLP	0.951	-0.451	0.554	0.551	0.387	0.812	-0.55	0.716	-0.904	0.797	-0.778	4.45
50	AM + MLP	0.953	-0.453	0.593	0.526	0.387	0.813	-0.560	0.719	-0.905	0.795	-0.778	3.91
50	RF + MLP	0.953	-0.449	0.563	0.558	0.379	0.810	-0.583	0.723	-0.905	0.789	-0.787	5.00
50	XGBoost + MLP	0.956	-0.454	0.570	0.6	0.385	0.813	-0.560	0.720	-0.894	0.8	-0.787	2.64
50	Deep Lasso + MLP	0.957	-0.441	0.589	0.539	0.383	0.816	-0.556	0.720	-0.900	0.801	-0.776	1.91
75	No FS + MLP	0.919	-0.504	0.505	0.514	0.357	0.786	-0.645	0.687	-0.913	0.765	-0.808	7.73
75	Univariate + MLP	0.950	-0.612	0.515	0.493	0.338	0.739	-0.621	0.688	-0.921	0.779	-0.838	7.64
75	Lasso + MLP	0.952	-0.447	0.545	0.482	0.376	0.811	-0.568	0.714	-0.902	0.796	-0.794	4.36
75	1L Lasso + MLP	0.955	-0.447	0.543	0.519	0.372	0.814	-0.596	0.714	-0.902	0.791	-0.781	3.73
75	AGL + MLP	0.954	-0.443	0.550	0.519	0.382	0.810	-0.566	0.722	-0.904	0.792	-0.784	3.09
75	LassoNet + MLP	0.947	-0.460	0.548	0.543	0.375	0.807	-0.575	0.712	-0.909	0.791	-0.787	5.09
75	RF + MLP	0.951	-0.450	0.564	0.524	0.367	0.805	-0.561	0.716	-0.902	0.770	-0.790	4.45
75	XGBoost + MLP	0.956	-0.454	0.571	0.558	0.380	0.810	-0.555	0.721	-0.896	0.797	-0.787	2.00
75	Deep Lasso + MLP	0.953	-0.443	0.552	0.527	0.377	0.812	-0.557	0.72	-0.898	0.798	-0.78	2.00
output and has only a single logit. That is, the weight matrix  $W$  is in fact a single row vector  $w$  with  $m$  entries giving us the equivalence between the First-Layer Lasso and regular Lasso, and therefore between First-Layer Lasso and Deep Lasso too in the case of linear regression:
$$
\min  _ {\theta} \alpha \mathcal {L} _ {\theta} (X, Y) + (1 - \alpha) \sum_ {j = 1} ^ {m} | | w ^ {(j)} | | _ {2} = \min  _ {\theta} \alpha \mathcal {L} _ {\theta} (X, Y) + (1 - \alpha) \| w \| _ {1}.
$$
F More Results
Tables 16, 17, 18 are similar to the tables in the main body, but they include the standard deviations computed across seeds for each experiment. We also report performance of XGBoost baseline model for comparison, however we exclude it from rank computations.
F.1 Corrupting features with Laplace noise
In addition to the experiments with Gaussian noise corruption, we conduct experiments with corrupting features with Laplace noise. In particular, we sample additional features from the original ones and corrupt them with Laplace noise with the standard deviation of original features:
$$
x _ {c} = 0. 5 x + 0. 5 \sigma \epsilon_ {l}, \epsilon_ {l} \sim L a p l a c e (0, \frac {1}{\sqrt {2}})
$$
We include results for this setup in Table 10. Similarly to the experiments with Gaussian noise corruption we observe that Deep Lasso along with XGBoost feature selection achieve the best downstream performance in terms of average rank.
F.2 ROC-AUC, Precision and Recall Metrics
One common way of measuring performance of feature selection methods is using ROC-AUC, precision and recall scores, which indicate how often these methods choose "correct" and "incorrect" features. We instead focus on the downstream performance metric since in the case of corrupted and more prominently second-order extraneous features, it is not possible to treat feature selection as binary classification problems because additional features may contain useful information. However in this section we report precision, recall and ROC-AUC scores for the setup with random features, which can be reasonably treated as a binary classification problem. Tables 11 12 present ROC-AUC, and precision scores correspondingly with ranks computed taking importance into account. We note
Table 11: ROC-AUC scores for feature selection methods trained on datasets with random extraneous features Bold font indicates the best numbers dataset-wise and lower rank indicates better overall result.
%	FS method	AL	CH	CO	EY	GE	HE	HI	HO	JA	MI	OT	YE	rank
50	Univariate	1.0	1.0	1.0	0.916	0.926	0.982	0.866	1.0	0.706	0.993	1.0	0.990	2.50
50	Lasso	0.988	1.0	1.000	0.795	0.371	0.976	0.847	0.920	0.735	0.596	0.991	0.965	5.00
50	1L Lasso	0.785	0.995	0.969	0.789	0.513	0.980	0.888	0.879	0.716	0.947	0.952	0.998	4.58
50	AGL	0.972	0.827	0.862	0.870	0.471	0.999	0.786	1.0	0.746	0.925	0.931	0.999	4.25
50	LassoNet	1.0	0.999	1.0	0.681	0.664	0.998	0.862	0.979	0.674	0.630	0.722	0.966	4.83
50	AM	0.848	0.992	0.752	0.765	0.770	1.0	0.876	0.980	0.759	0.704	0.972	0.988	4.33
50	RF	0.906	1.0	0.898	0.884	0.998	1.0	0.874	0.934	0.729	0.559	0.995	0.891	4.17
50	XGBoost	0.945	1.0	0.944	0.993	0.943	1.0	0.890	1.0	0.75	0.968	0.992	0.988	2.33
50	Deep Lasso	1.0	0.998	0.981	0.927	0.482	1.0	0.922	0.993	0.699	0.957	1.0	1.0	2.33
75	Univariate	1.0	1.0	1.0	0.918	0.872	0.986	0.846	1.0	0.731	0.993	1.0	0.991	2.08
75	Lasso	1.0	1.0	0.976	0.771	0.781	0.976	0.837	0.858	0.77	0.597	0.991	0.943	4.08
75	1L Lasso	0.979	0.998	0.932	0.841	0.477	0.983	0.784	0.910	0.739	0.830	0.963	1.0	4.33
75	AGL	0.922	1.0	0.862	0.815	0.492	0.986	0.791	0.908	0.726	0.891	0.831	1.000	4.92
75	LassoNet	1.0	1.0	0.998	0.582	0.531	0.965	0.821	0.974	0.613	0.448	0.589	0.931	5.08
75	Forest	0.927	1.0	0.957	0.916	1.0	1.0	0.88	0.990	0.730	0.700	0.998	0.942	3.50
75	XGBoost	0.945	1.0	0.948	0.946	1.0	1.0	0.700	0.997	0.737	1.0	0.981	0.999	2.92
75	Deep Lasso	1.000	0.994	0.985	0.919	0.273	1.0	0.895	0.995	0.762	0.959	1.000	1.0	2.25
Table 12: Precision scores for feature selection methods trained on datasets with random extraneous features Bold font indicates the best numbers dataset-wise and lower rank indicates better overall result.
%	FS method	AL	CH	CO	EY	GE	HE	HI	HO	JA	MI	OT	YE	rank
50	Univariate	1.0	1.0	1.0	0.808	0.844	0.963	0.786	1.0	0.648	0.986	1.0	0.967	2.75
50	Lasso	0.987	1.0	0.989	0.704	0.403	0.963	0.750	0.875	0.661	0.588	0.969	0.922	5.00
50	1L Lasso	0.776	0.988	0.954	0.719	0.497	0.937	0.811	0.856	0.644	0.882	0.915	0.988	4.67
50	AGL	0.946	0.738	0.794	0.777	0.469	0.989	0.704	1.0	0.665	0.870	0.875	0.987	4.50
50	LassoNet	1.0	0.988	1.0	0.619	0.625	0.985	0.771	0.931	0.644	0.582	0.692	0.908	5.00
50	AM	0.768	0.962	0.715	0.692	0.684	1.0	0.789	0.938	0.676	0.646	0.920	0.953	4.58
50	RF	0.901	1.0	0.893	0.885	0.984	1.0	0.807	0.875	0.661	0.553	0.985	0.809	4.25
50	XGBoost	0.945	1.0	0.944	0.969	0.894	1.0	0.818	1.0	0.981	0.956	0.992	0.977	2.17
50	Deep Lasso	1.0	0.988	0.974	0.835	0.488	1.0	0.861	0.969	0.641	0.950	1.0	1.0	2.67
75	Univariate	1.0	1.0	1.0	0.769	0.688	0.963	0.643	1.0	0.537	0.987	1.0	0.967	2.67
75	Lasso	1.0	1.0	0.957	0.608	0.766	0.963	0.679	0.812	0.622	0.401	0.963	0.878	3.75
75	1L Lasso	0.882	0.975	0.909	0.665	0.231	0.948	0.632	0.881	0.531	0.721	0.933	0.996	5.17
75	AGL	0.856	1.0	0.743	0.685	0.253	0.952	0.654	0.869	0.541	0.775	0.704	0.987	4.75
75	LassoNet	1.0	1.0	0.976	0.458	0.488	0.919	0.650	0.912	0.415	0.331	0.524	0.770	5.08
75	RF	0.918	1.0	0.943	0.888	1.0	1.0	0.789	0.912	0.559	0.664	0.995	0.836	3.17
75	XGBoost	0.938	1.0	0.946	0.873	0.994	1.0	0.643	0.938	0.567	0.993	0.974	0.981	2.67
75	Deep Lasso	0.990	0.95	0.957	0.777	0.262	1.0	0.775	0.956	0.576	0.946	0.992	1.0	2.17