SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 8: Statistics for unsupervised learning TU-datasets.
DATASETS	CATEGORY	GRAPHS#	AVG. N#	AVG. DEGREE
NCI1	BIOCHEMICAL MOLECULES	4,110	29.87	1.08
PROTEINS	BIOCHEMICAL MOLECULES	1,113	39.06	1.86
DD	BIOCHEMICAL MOLECULES	1,178	284.32	715.66
MUTAG	BIOCHEMICAL MOLECULES	188	17.93	19.79
COLLAB	SOCIAL NETWORKS	5,000	74.49	32.99
RDT-B	SOCIAL NETWORKS	2,000	429.63	1.15
RDT-M	SOCIAL NETWORKS	2,000	429.63	497.75
IMDB-B	SOCIAL NETWORKS	1,000	19.77	96.53
Table 9: hyperparameters setting on graph CL. Note that  $\{\cdot\}$  indicates the range of hyperparameters that we tune and the numbers in bold are the final settings.
DATASETS	BEST τ	μ	σ
NCI1	{0.05, 0.10, 0.15, 0.20, 0.25}	{0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}	{0.5, 1.0}
PROTEINS	{0.05, 0.10, 0.15, 0.20, 0.25}	{0.5, 1.0, 1.5, 2.0}	{0.5, 1.0}
DD	{0.05, 0.10, 0.15, 0.20, 0.25}	{0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}	{0.5, 1.0}
MUTAG	{0.05, 0.10, 0.15, 0.20, 0.25}	{0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}	{0.5, 1.0}
COLLAB	{0.05, 0.10, 0.15, 0.20, 0.25}	{0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}	{0.5, 1.0}
RDT-B	{0.05, 0.10, 0.15, 0.20, 0.25}	{0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}	{0.5, 1.0}
RDT-M	{0.05, 0.10, 0.15, 0.20, 0.25}	{0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}	{0.5, 1.0}
IMDB-B	{0.10, 0.20, 0.30, 0.40, 0.50}	{0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}	{0.5, 1.0}
Table 10: Unsupervised representation learning classification accuracy (%) on TU datasets. The compared numbers are from except AD-GCL, whose statistics are reproduced on our platform. Bold indicates the best performance while underline indicates the second best on each dataset.
DATASET	NCI1	PROTEINS	DD	MUTAG	COLLAB	RDT-B	RDT-M5K	IMDB-B	AVG.
NO PRE-TRAIN	65.40±0.17	72.73±0.51	75.67±0.29	87.39±1.09	65.29±0.16	76.86±0.25	48.48±0.28	69.37±0.37	70.15
INFOGRAPH	76.20±1.06	74.44±0.31	72.85±1.78	89.01±1.13	70.05±1.13	82.50±1.42	53.46±1.03	73.03±0.87	74.02
GRAPHCL	77.87±0.41	74.39±0.45	78.62±0.40	86.80±1.34	71.36±1.15	89.53±0.84	55.99±0.28	71.14±0.44	75.71
AD-GCL	73.91±0.77	73.28±0.46	75.79±0.87	88.74±1.85	72.02±0.56	90.07±0.85	54.33±0.32	70.21±0.68	74.79
RGCL	78.14±1.08	75.03±0.43	78.86±0.48	87.66±1.01	70.92±0.65	90.34±0.58	56.38±0.40	71.85±0.84	76.15
ADNCE	79.30±0.67	75.10±0.25	79.23±0.59	89.04±1.30	72.26±1.10	91.39±0.31	56.01±0.35	71.58±0.72	76.74
B.4 Additional ablation study
The primary motivation behind ADNCE is to mitigate the issues of over-conservatism and sensitivity to outliers. These limitations stem from the unreasonable worst-case distribution, which assigns excessive weights to the hardnest negative samples. Consequently, any weighting strategy capable of modulating the worst-case distribution to focus more on the informative region holds promising. In our ADNCE, we opted for Gaussian-like weights due to its flexibility and unimodal nature. However, alternative weighting strategies such as Gamma, Rayleigh or Chi-squared could also be employed. The following experiment demonstrates that these alternative weighting strategies can yield comparable results to Gaussian-like weights. The above table showcases the comparison of the
Table 11: Alternative weighting strategies for ADNCE.
WEIGHT STRATEGY	PROBABILITY DENSITY FUNCTION	CIFAR10
GAMMA-LIKE	w(x,m,n) = 1/Γ(m)n^m x^{m-1}e^{-\frac{x}{n}}	91.74
RAYLEIGH-LIKE	w(x,m) = \frac{x}{m^2} e^{-\frac{x^2}{2m^2}}	91.73
CHI-SQUARED-LIKE	w(x,m) = \frac{1}{2^m/2\Gamma(m/2)} x^{m/2-1}e^{-x/2}	91.99
GAUSSIAN-LIKE (ADNCE)	w(x,m,n) = \frac{1}{n\sqrt{2\pi}} e^{-\frac{1}{2}\left(\frac{x-m}{n}\right)^2}	91.88
TOP-1 accuracy performance on CIFAR10 dataset under different weight strategies. The parameters  $m$  and  $n$  are utilized to denote the parameters within their corresponding probability density function, while the variable  $x$  represents a random variable. It is important to note that, due to the domain definition of some PDFs being  $(0, +\infty)$ , we need to set  $x = \text{prediction\_score} + 1$ .
Table 12: Sensitivity analysis of  $\mu$  .
μ	0.1	0.3	0.5	0.7	0.9
CIFAR10	91.3	91.66	91.9	91.77	92.25
STL10	87.84	88.22	87.56	88.48	88.45
CIFAR100	69.34	69.31	68.70	69.24	68.95
Table 13: Sensitivity analysis of  $\sigma$  .
σ	0.2	0.4	0.6	0.8	1	1.5	2
CIFAR10	90.07	91.85	92.02	91.77	91.72	91.69	91.94
STL10	86.54	88.30	88.10	87.54	88.95	88.12	88.40
CIFAR100	67.38	69.36	69.52	69.01	68.70	69.24	69.42
B.5 Sensitivity analysis of  $\mu$  and  $\sigma$
The above tables showcase the comparisons of the TOP-1 accuracy performance on three datasets under different  $\mu$  and  $\sigma$ . As can be seen, changing the parameters  $\mu$  and  $\sigma$  would impact the model performance, but not as dramatical as tuning the parameter  $\tau$ . (For example, on STL10  $\sigma$  from 1.0 to 0.2 just brings  $2.7\%$  performance drops, while changing  $\tau$  brings  $7.6\%$  performance gap.) This outcome indicates that tuning  $\mu$  and  $\sigma$  is not a significant burden. In most scenarios, it may suffice to set  $\sigma = 1$ , requiring only the tuning of  $\mu$  within the range of 0.1 to 0.9 (can refer to Table 6, 7, 9 in Appendix B for more details).