SlowGuess/NeurIPS_2023_OCR huggingface dataset:
- We plot the average value of  $p(j|(u,i))$  across one batch, as depicted in Figure 4a. The figure reveals that AdvInfoNCE learns a skewed negative sampling distribution, mirroring the trend observed in the Tencent dataset. Such a distribution places more emphasis on popular negative items and reduces the difficulty of unpopular negative items, which have a higher probability of being false negatives.
- To examine the effect of fine-grained hardness, we conduct experiments with four different hardness learning strategies, including AdvInfoNCE, InfoNCE, InfoNCE-Rand, and AdvInfoNCE-Reverse. AdvInfoNCE-Reverse refers to a strategy where hardness is learned by minimizing, rather than maximizing, the loss function. This inversion results in what we term 'reversed hardness', in contrast to the approach of our AdvInfoNCE method. InfoNCE-Rand denotes the assignment of uniformly random hardness for each negative item. We conduct 5-fold experiments with different random seeds for each strategy and report the mean value with standard error in Figure 4c. As the result shows, AdvInfoNCE yields consistent improvements over the other three different hardness strategies during the training phase. In contrast, the performance of AdvInfoNCE-Reverse drops rapidly as continuously training the reversed hardness. The sustained superior performance of AdvInfoNCE indicates that it effectively promotes the generalization ability of the CF model by automatically distinguishing false negatives and hard negatives.
- Figure 4b illustrates the uniform and align loss during the training phase following the initial warm-up epochs. As demonstrated, after the warm-up phase, both InfoNCE and InfoNCE-Rand exhibit a slight increase in align loss, while their uniform loss maintains a stable level. In contrast, AdvInfoNCE significantly improves uniformity at an acceptable cost of increasing align loss. On the other hand, employing reversed hardness (as in AdvInfoNCE-Reverse) appears to have a negative impact on representation uniformity. These findings underscore the importance of fine-grained hardness in AdvInfoNCE, suggesting that AdvInfoNCE learns more generalized representations.
D.3 Effect of the Adversarial Training Epochs on KuaiRec
In this section, we conduct experiments on KuaiRec, where AdvInfoNCE is trained for varying numbers of adversarial epochs. We plot performance metrics (Recall@20 in Figure 5a and NDCG@20 in Figure 5b) on the test set, which represents out-of-distribution data, throughout the training phase. The green stars mark the corresponding endpoint of adversarial training. As illustrated in Figure 5a and 5b, both Recall@20 and NDCG@20 show a proportional trend with the number of adversarial training epochs, up to a certain threshold. However, it is worth noting that in the extreme condition when the number of adversarial training epochs exceeds the threshold, performance on out-of-distribution sharply declines. This indicates a need to strike a balance when determining the appropriate number of adversarial training epochs for AdvInfoNCE.
Table 9: Training cost on Tencent (seconds per epoch/in total).
+InfoNCE	+SGL	+NCL	+XSimGCL	+CCL	+BC loss	+Adap-τ	+AdvInfoNCE
MF	16.8 / 7,123	-	-	-	17.2 / 4,111	19.1 / 6,751	22.3 / 7,694	21.6 / 11,534
LightGCN	41.2 / 21,177	82.5 / 4,868	54.9 / 5,161	42.1 / 842	42.1 / 11,114	43.5 / 23,664	55.6 / 17,236	44.6 / 21,586
Algorithm 1 AdvInfoNCE
Input: observed interactions  $\mathcal{O}^+$ , unobserved interactions  $\mathcal{O}^-$ , learning rate of adversarial training  $lr_{adv}$ , maximum adversarial training epochs  $E_{adv}$ , adversarial training intervals  $T_{adv}$ , parameters of the CF model  $\theta$ , parameters of the hardness evaluation models  $\theta_{adv}$ , weighting parameter  $K$   
Output:  $\theta$
Initialize: Initialize  $\theta$  and  $\theta_{adv}$ ,  $e \gets 1$ ,  $e_{adv} \gets 1$
Freeze parameters of the hardness evaluation model  $\theta_{adv}$
Randomly sample  $N$  negative items from  $\mathcal{I}_u^-$  for each interaction within a batch
Compute  $s(u,i),\delta_j^{(u,i)}$  with  $\theta$  and  $\theta_{adv}$  , respectively
Compute  $\mathcal{L}_{\mathrm{Adv}}(u,i) = -\log \frac{\exp(s(u,i))}{\exp(s(u,i)) + K\sum_{j = 1}^{N}\exp(\delta_j^{(u,i)})\exp(s(u,j))}$
Update  $\theta$  by minimizing  $\mathcal{L}_{\mathrm{Adv}}(u,i)$
if  $e\bmod T_{adv} == 0$  &  $e_{adv}\leq E_{adv}$  then
Freeze parameters of the CF model  $\theta$
Update  $\theta_{adv}$  by maximizing  $\mathcal{L}_{\mathrm{Adv}}(u,i)$
$$
e _ {a d v} \leftarrow e _ {a d v} + 1
$$
$$
e \leftarrow e + 1
$$
until CF model converges
D.4 Hardness Learning Strategy
To accurately evaluate the hardness of each negative instance, we need to establish a mapping from unobserved user-item pairs to their corresponding hardness values, and this mapping mechanism can be diversified. Generally, the hardness learning strategy can be formulated as:
$$
\begin{array}{l} \delta_ {j} ^ {(u, i)} \doteq \log \left(\left| \mathcal {N} _ {u} \right| \cdot p (j | (u, i))\right) (28) \\ \stackrel {\cdot} {=} \log \left(| \mathcal {N} _ {u} | \cdot \frac {\exp \left(g _ {\theta_ {a d v}} (u , j)\right)}{\sum_ {k = 1} ^ {| \mathcal {N} _ {u} |} \exp \left(g _ {\theta_ {a d v}} (u , k)\right)}\right), (29) \\ \end{array}
$$
where  $g_{\theta_{adv}}(u,j)$  is a raw hardness score function for the unobserved user-item pair  $(u,j)$ , and  $p(j|(u,i))$  is the probability of sampling the negative instance  $j$ , which is calculated by normalizing  $g_{\theta_{adv}}(u,j)$ . In this paper, we proposed two specific mapping methods: embedding-based (i.e., AdvInfoNCE-embed) mapping and multilayer perceptron-based (i.e., AdvInfoNCE-mlp) mapping. It should be noted that all the results reported in the main text of this paper are based on the implementation of the AdvInfoNCE-embed version.
AdvInfoNCE-embed. The hardness computation process in AdvInfoNCE-embed follows a similar protocol as CF models. We directly map the index of users and items into its corresponding hardness embedding and calculate the hardness of each user-item pair through a score function. Specifically, this process involves a user hardness encoder  $\psi_{\theta_{adv}}(\cdot): \mathbb{U} \to \mathbb{R}^d$  and an item hardness encoder  $\phi_{\theta_{adv}}(\cdot): \mathbb{I} \to \mathbb{R}^d$ , where  $\theta_{adv}$  denotes all the trainable parameters of the hardness learning model. In our experiments, we adopt the same embedding dimension as the CF models for hardness learning. For the score function, we define  $g_{\theta_{adv}}(u,j) = \psi_{\theta_{adv}}(u)^{\top} \cdot \phi_{\theta_{adv}}(j)$ .
AdvInfoNCE-mlp. Unlike AdvInfoNCE-embed, AdvInfoNCE-mlp maps the embeddings of items and users from the CF model into another latent space with two multilayer perceptrons (MLPs) and calculates the hardness on this latent space. By respectively defining the MLPs for users and items as  $\mathrm{MLP_u}$  and  $\mathrm{MLP_v}$ , the score function for hardness calculation is defined as  $g_{\theta_{adv}}(u,j) = \mathrm{MLP_u}(\psi_\theta(u))^{\top} \cdot \mathrm{MLP_v}(\phi_\theta(j))$ . In our experiment, we simply employ one-layer MLPs and set the dimension of latent space as four. It's worth noting that this MLP-based implementation of
(a) Distribution of hardness
Figure 4: Study of hardness. (4a) Illustration of hardness i.e., the probability of negative sampling  $(p(j|(u,i)))$  learned by AdvInfoNCE w.r.t. item popularity on KuaiRec. The dashed line represents the uniform distribution. (4b) Performance comparisons with varying hardness learning strategies on KuaiRec. (4c) The trajectories of align loss and uniform loss during training progress. Lower values indicate better performance. Arrows denote the losses' changing directions.
(a) Recall@20
(b) NDCG@20
Figure 5: The performance on KuaiRec with different numbers of adversarial training epochs. (5a) Performance comparisons w.r.t. Recall@20 on KuaiRec. (5b) Performance comparisons w.r.t. NDCG@20 on KuaiRec.
AdvInfoNCE may also be adaptable for handling out-of-distribution tasks in other fields, such as computer vision (CV) and natural language processing (NLP).
As reported in Table 10, both AdvInfoNCE-embed and AdvInfoNCE-mlp yield significant improvements over InfoNCE. Moreover, AdvInfoNCE-embed generally outperforms AdvInfoNCE-mlp.
D.5 The Intuitive Understanding of AdvInfoNCE
In this section, we aim to understand the mechanism of AdvInfoNCE intuitively, from the perspective of false negative identification.
Figure 6 illustrates the changes of the out-of-distribution performance and FN identification rate during the training process on Tencent. Here, the FN identification rate indicates the proportion of false negatives with negative  $\delta_{j}$ . It can be observed that the out-of-distribution performance exhibits a rising trend along with the FN identification rate. Meanwhile, the out-of-distribution performance of InfoNCE remains relatively low. This indicates that AdvInfoNCE enhances the generalization ability of the CF model by identifying false negatives.
Figure 7 illustrates how AdvInfoNCE adjusts the scores and rankings of sampled negative items, by identifying the false negatives and true negatives. We retrieve the negative items sampled during training. If a sampled negative item appears in the test set, it is labeled as a false negative (FN); otherwise. In Figure 7a and 7b, the leftmost item represents a false negative, while the other two items on the right are negatives. The bar charts in blue and red depict the cosine similarity scores of sampled negative items measured by InfoNCE and AdvInfoNCE respectively. The rankings of sampled negative items are annotated above the bars. The line graph illustrates the hardness  $\delta_{j}$  computed by AdvInfoNCE, measured on the right axis. It can be observed that when the hardness  $\delta_{j}$
Table 10: The performance comparison between AdvInfoNCE-embed and AdvInfoNCE-mlp over the LightGCN backbone.
KuaiRec	Yahoo!R3	Coat	Tencent (γ = 200)	Tencent (γ = 10)	Tencent (γ = 2)
Recall	NDCG	Recall	NDCG	Recall	NDCG	Recall	NDCG	Recall	NDCG	Recall	NDCG
InfoNCE	0.1800	0.4529	0.1475	0.0698	0.2689	0.1882	0.0540	0.0320	0.0332	0.0195	0.0242	0.0145
AdvInfoNCE-embed	0.1979	0.4697	0.1527	0.0718	0.2846	0.2026	0.0594	0.0356	0.0403	0.0243	0.0295	0.0180
Imp.% over InfoNCE	9.94%	3.71%	3.53%	2.87%	5.84%	7.65%	10.00%	11.25%	21.39%	24.62%	21.90%	24.14%
AdvInfoNCE-mlp	0.1851	0.4579	0.1545	0.0724	0.2843	0.2002	0.0567	0.0339	0.0364	0.0221	0.0260	0.0160
Imp.% over InfoNCE	2.83%	1.10%	4.75%	3.72%	5.73%	6.38%	5.00%	5.94%	9.64%	13.33%	7.44%	10.34%
Figure 6: FN identification rate and NDCG@20 during training on Tencent, where FN identification rate indicates the proportion of false negatives (FN) with negative  $\delta_{j}$  and NDCG@20 shows the out-of-distribution performance. As training proceeds, AdvInfoNCE' FN identification rate increases, capping at nearly  $70\%$ . This reveals AdvInfoNCE's capability to identify approximately  $70\%$  of false negatives in the test set. We attribute the superior recommendation performance of AdvInfoNCE over InfoNCE to this gradual identification.
is negative (i.e., indicating that the item is identified as a false negative), the cosine similarity score improves. Conversely, if the hardness  $\delta_{j}$  is positive, the score decreases.
E Hyperparameter Settings
For a fair comparison, we conduct all the experiments in PyTorch with a single Tesla V100-SXM3-32GB GPU and an Intel(R) Xeon(R) Gold 6248R CPU. We optimize all methods with the Adam optimizer and set the layer numbers of LightGCN by default at 2, with the embedding size as 64 and the weighting parameter  $K$  as 64. We search for hyperparameters within the range provided by the corresponding references. For AdvInfoNCE, we search  $lr_{adv}$  in [1e-1, 1e-4],  $E_{adv}$  in [1, 30] and  $T_{adv}$  in {5, 10, 15, 20}. We adopt the early stop strategy that stops training if Recall@20 on the validation set does not increase for 20 successive evaluations. It's worth noting that AdvInfoNCE inherits the hyperparameter sensitivity property of adversarial learning, therefore it's necessary to choose proper hyperparameters for different datasets. We suggest selecting a suitable adversarial learning rate  $lr_{adv}$  first and then increasing the number of adversarial training epochs  $E_{adv}$  gradually until AdvInfoNCE reaches a relatively stable performance. We report the effect of changing the number of negative sampling in Table 13, where  $N$  is the number of negative sampling.
(a) User 46767
(b) User 54944
Figure 7: Case studies of refining the item ranking. With two randomly sampled users along with their sampled negative items, we subsequently retrieve their associated  $\delta$  values, ranking positions, and cosine similarities. Here FN denotes false negative (i.e., interactions unobserved during training but present in testing). The bar charts demonstrate the cosine similarity scores of these sampled negative items as gauged by both InfoNCE and AdvInfoNCE. Their rankings are annotated atop the bars. An accompanying line illustrates the hardness  $\delta_{j}$  derived by AdvInfoNCE (measured on the right axis). Notably, when  $\delta < 0$ , AdvInfoNCE identifies and elevates an FN; conversely, for a potentially true negative, AdvInfoNCE leans towards a positive  $\delta$  and declines its rank.
Table 11: Hyperparameters search spaces for baselines.
Hyperparameter space
MF & LightGCN	lr ~ {1e-5, 3e-5, 5e-5, 1e-4, 3e-4, 5e-4, 1e-3}, batch size ~ {64, 128, 256, 512, 1024, 2048} No. negative samples ~ {64, 128, 256, 512}
SSM	τ ~ [0.05, 3]
CCL	w ~ {1, 2, 5, 10, 50, 100, 200}, m ~ {0.2, 0.4, 0.6, 0.8, 1}
BC Loss	τ1 ~ [0.05, 3], τ2 ~ [0.05, 3]
Adap-τ	warm_up_epochs ~ {10, 20, 50, 100}
SGL	τ ~ [0.05, 3], λ1 ~ {0.005, 0.01, 0.05, 0.1, 0.5, 1.0}, ρ ~ {0, 0.1, 0.2, 0.3, 0.4, 0.5}
NCL	τ ~ [0.05, 3], λ1 ~ [1e-10, 1e-6], λ2 ~ [1e-10, 1e-6], k ~ [5, 10000]
XSimGCL	τ ~ [0.05, 3], ε ~ {0.01, 0.05, 0.1, 0.2, 0.5, 1.0}, λ ~ {0.005, 0.01, 0.05, 0.1, 0.5, 1.0}, l* = 1
Table 12: Model architectures and hyperparameters for AdvInfoNCE.
Hyperparameters of AdvInfoNCE
lradv	Eadv	Tadv	τ	lr	batch size	No. negative samples
LightGCN
Tencent	5e-5	7	5	0.09	1e-3	2048	128
KuaiRec	5e-5	12	5	2	3e-5	2048	128
Yahoo!R3	1e-4	13	5	0.28	5e-4	1024	64
Coat	1e-2	20	15	0.75	1e-3	1024	64
MF
Tencent	5e-5	8	5	0.09	1e-3	2048	128
Yahoo!R3	1e-4	12	5	0.28	5e-4	1024	64
Coat	1e-2	18	15	0.75	1e-3	1024	64
Table 13: Varying number of negative sampling on Tencent
N	γ = 200	γ = 10	γ = 2	Validation NDCG
HR	Recall	NDCG	HR	Recall	NDCG	HR	Recall	NDCG
64	0.1513	0.0563	0.0333	0.1006	0.0373	0.0225	0.0708	0.0269	0.0164	0.0854
128	0.1600	0.0594	0.0356	0.1087	0.0403	0.0243	0.0774	0.0295	0.0180	0.0879
256	0.1642	0.0609	0.0367	0.1125	0.0419	0.0253	0.0815	0.0310	0.0189	0.0889