SlowGuess/NeurIPS_2023_OCR huggingface dataset:
(c) Our
Figure S2: Visualization with t-SNE, where the first row and the second row correspond to COIL-20 and Jaffe, respectively. Different classes are indicated with different colors. Intra-class connections are indicated in red and inter-class connections are indicated in blue. For each sample, we only present the connections to the 5 nearest neighbors.
S6.5 Extended Results for Ablation Study
Table S4 and Table S5 present the clustering result and the reconstruction result of the ablation study on DGL in Section 4.4:
Table S4: Clustering results without DGL.
Method	Madelon	PCMAC	Jaffe	PIX10	GLIOMA	PROSTATE
w/o DGL	0.52±0.01	0.51±0.01	0.81±0.06	0.78±0.06	0.65±0.08	0.57±0.05
Our	0.60±0.01	0.53±0.02	0.87±0.06	0.87±0.07	0.75±0.08	0.68±0.09
Table S5: Reconstruction results without DGL.
Method	Madelon	PCMAC	Jaffe	PIX10	GLIOMA	PROSTATE
w/o DGL	0.99±0.00	1.07±0.10	0.26±0.05	0.45±0.02	0.74±0.03	0.79±0.08
Our	0.98±0.00	1.30±0.22	0.22±0.01	0.39±0.03	0.69±0.04	0.73±0.13
S6.6 Additional Experiment: Parameter Sensitivity Analysis
In this section, we analyze the effect of the parameters of our method, including the learning rate, the number of nearest neighbors  $k$ , the hyperparameter  $\gamma$  in the entropy regularized OT problem 14, and the selected feature number  $m$ . We use four real-world datasets, including a text dataset PCMAC, an artificial dataset Madelon, an image dataset Jaffe, and a biological dataset PROSTATE.
The analysis is based on the optimal parameter obtained in Section 4.3. For each dataset, we fix the values of the remaining parameters and vary the value of one parameter at a time. We retrain our method using the updated parameter combination and evaluate the corresponding FS result with the random forest. This allows us to observe the impact of different parameters on the performance of our method. For example, to analyze the effect of the learning rate on Madelon, we keep  $k$ ,  $\gamma$ , and  $m$  at their optimal values, then we vary the learning rate in  $\{10^{-4}, 10^{-3}, 10^{-2}, 10^{-1}, 10^{0}, 10^{1}\}$  (the range as we described in Appendix S6.1.2), and evaluate their corresponding performance in the classification task with the random forest. The overall results are shown in Fig. S3, where the stars represent the results using optimal parameters.
One the one hand, we observe that the variations in the learning rate,  $k$ ,  $\gamma$  have little impact on the performance of our method across different datasets. This suggests that we can set a value within a proper range for these parameters, without the need to determine their values on different datasets. On the other hand, the most sensitive parameter is  $m$ , where a higher number of features contributes to better results, aligning with intuition and observations from existing literature. However, it is important to emphasize that more features are not always better. As demonstrated in Section 4.3, the FS methods consistently outperform AllFea in most classification and clustering tasks. Fewer features not only result in lower computational costs but also contribute to faster learning speeds. This suggests the need to adjust the value of  $m$ , for example, starting with a relatively small value and gradually increasing it until the model performance begins to decline.
(a) Madelon
(b) PCMAC
Figure S3: Parameter sensitivity analysis using the random forest, where the starred point denotes the performance on the optimal parameter combination.
(c) Jaffe
(d) PROSTATE