Table 2: Benchmarking feature selection methods for MLP and FT-Transformer downstream models on datasets with corrupted extra features. We report performance of models trained on features selected by different FS algorithms in terms of accuracy for classification and negative RMSE for regression problems. % refers to percent of extra features in the dataset: either  $50\%$  or  $75\%$  features are corrupted. Bold font indicates the best numbers dataset-wise and lower rank indicates better overall performance.
%	FS method + Model	AL	CH	CO	EY	GE	HE	HI	HO	JA	MI	OT	YE	rank
50	No FS + MLP	0.946	-0.475	0.965	0.557	0.525	0.370	0.802	-0.607	0.703	-0.909	0.778	-0.797	8.00
50	Univariate + MLP	0.955	-0.451	0.966	0.556	0.514	0.346	0.810	-0.620	0.717	-0.920	0.795	-0.828	7.33
50	Lasso + MLP	0.955	-0.449	0.968	0.548	0.512	0.382	0.813	-0.602	0.713	-0.903	0.796	-0.795	5.42
50	1L Lasso + MLP	0.955	-0.447	0.968	0.566	0.515	0.382	0.812	-0.581	0.718	-0.902	0.795	-0.780	4.75
50	AGL + MLP	0.953	-0.450	0.968	0.588	0.538	0.386	0.813	-0.561	0.722	-0.902	0.796	-0.780	3.00
50	LassoNet + MLP	0.955	-0.452	0.969	0.570	0.556	0.382	0.811	-0.551	0.719	-0.905	0.795	-0.777	3.83
50	AM + MLP	0.955	-0.449	0.967	0.583	0.527	0.381	0.814	-0.555	0.722	-0.905	0.797	-0.780	3.58
50	RF + MLP	0.951	-0.453	0.967	0.574	0.568	0.383	0.810	-0.565	0.724	-0.904	0.788	-0.786	4.67
50	XGBoost + MLP	0.954	-0.454	0.969	0.583	0.510	0.385	0.815	-0.553	0.722	-0.892	0.803	-0.779	2.67
50	Deep Lasso + MLP	0.955	-0.447	0.968	0.577	0.525	0.388	0.815	-0.567	0.721	-0.895	0.801	-0.776	2.58
75	No FS + MLP	0.921	-0.516	0.956	0.518	0.503	0.356	0.788	-0.632	0.686	-0.913	0.762	-0.808	7.58
75	Univariate + MLP	0.955	-0.569	0.941	0.510	0.495	0.347	0.742	-0.620	0.686	-0.921	0.779	-0.838	7.50
75	Lasso + MLP	0.948	-0.454	0.963	0.565	0.490	0.373	0.810	-0.593	0.717	-0.903	0.795	-0.791	5.00
75	1L Lasso + MLP	0.955	-0.444	0.967	0.549	0.495	0.380	0.811	-0.576	0.715	-0.903	0.797	-0.779	3.25
75	AGL + MLP	0.928	-0.566	0.967	0.548	0.490	0.382	0.811	-0.574	0.714	-0.904	0.788	-0.780	4.92
75	LassoNet + MLP	0.947	-0.452	0.969	0.539	0.533	0.383	0.805	-0.572	0.708	-0.908	0.791	-0.785	4.33
75	RF + MLP	0.952	-0.450	0.963	0.547	0.533	0.372	0.805	-0.573	0.716	-0.903	0.765	-0.788	4.92
75	XGBoost + MLP	0.954	-0.515	0.968	0.571	0.53	0.381	0.811	-0.571	0.721	-0.895	0.800	-0.784	2.58
75	Deep Lasso + MLP	0.959	-0.441	0.968	0.554	0.517	0.386	0.813	-0.563	0.718	-0.898	0.804	-0.778	1.42
50	No FS + FT	0.960	-0.430	0.967	0.686	0.576	0.386	0.818	-0.574	0.731	-0.901	0.809	-0.793	6.08
50	Univariate + FT	0.963	-0.422	0.965	0.681	0.574	0.345	0.812	-0.628	0.733	-0.920	0.812	-0.826	6.17
50	Lasso + FT	0.952	-0.422	0.936	0.697	0.556	0.387	0.820	-0.586	0.732	-0.937	0.812	-0.915	6.42
50	1L Lasso + FT	0.962	-0.419	0.969	0.718	0.571	0.389	0.820	-0.570	0.731	-0.899	0.816	-0.795	3.50
50	AGL + FT	0.906	-0.426	0.969	0.697	0.591	0.392	0.820	-0.552	0.735	-0.914	0.816	-0.830	4.08
50	LassoNet + FT	0.962	-0.426	0.97	0.679	0.578	0.393	0.814	-0.572	0.736	-0.903	0.813	-0.79	3.92
50	AM + FT	0.962	-0.424	0.969	0.680	0.572	0.392	0.820	-0.549	0.734	-0.901	0.817	-0.790	3.42
50	RF + FT	0.962	-0.422	0.969	0.711	0.6	0.387	0.819	-0.557	0.735	-0.898	0.806	-0.793	3.42
50	XGBoost + FT	0.963	-0.422	0.97	0.706	0.564	0.392	0.821	-0.548	0.735	-0.897	0.816	-0.790	2.42
50	Deep Lasso + FT	0.961	-0.422	0.968	0.725	0.577	0.393	0.821	-0.561	0.736	-0.898	0.809	-0.788	2.67