SlowGuess/NeurIPS_2023_OCR huggingface dataset:
An Zhang $^{1*}$  Leheng Sheng $^{2*}$  Zhibo Cai $^{3\dagger}$  Xiang Wang $^{4}$  Tat-Seng Chua $^{1}$
$^{2}$ Tsinghua University
<sup>3</sup>Center for Applied Statistics and School of Statistics, Renmin University of China
<sup>4</sup>University of Science and Technology of China
anzhang@u.nus.edu, chenglh22@mails.tsinghua.edu.cn,
caizhibo@ruc.edu.cn, xiangwang1223@gmail.com, dcscts@nus.edu.sg
Contrastive Learning (CL) has achieved impressive performance in self-supervised learning tasks, showing superior generalization ability. Inspired by the success, adopting CL into collaborative filtering (CF) is prevailing in semi-supervised top- $K$  recommendations. The basic idea is to routinely conduct heuristic-based data augmentation and apply contrastive losses (e.g., InfoNCE) on the augmented views. Yet, some CF-tailored challenges make this adoption suboptimal, such as the issue of out-of-distribution, the risk of false negatives, and the nature of top- $K$  evaluation. They necessitate the CL-based CF scheme to focus more on mining hard negatives and distinguishing false negatives from the vast unlabeled user-item interactions, for informative contrast signals. Worse still, there is limited understanding of contrastive loss in CF methods, especially w.r.t. its generalization ability. To bridge the gap, we delve into the reasons underpinning the success of contrastive loss in CF, and propose a principled Adversarial InfoNCE loss (AdvInfoNCE), which is a variant of InfoNCE, specially tailored for CF methods. AdvInfoNCE adaptively explores and assigns hardness to each negative instance in an adversarial fashion and further utilizes a fine-grained hardness-aware ranking criterion to empower the recommender's generalization ability. Training CF models with AdvInfoNCE, we validate the effectiveness of AdvInfoNCE on both synthetic and real-world benchmark datasets, thus showing its generalization ability to mitigate out-of-distribution problems. Given the theoretical guarantees and empirical superiority of AdvInfoNCE over most contrastive loss functions, we advocate its adoption as a standard loss in recommender systems, particularly for the out-of-distribution tasks. Codes are available at https://github.com/LehengTHU/AdvInfoNCE.
Contrastive Learning (CL) has emerged as a potent tool in self-supervised learning tasks [1-4], given its superior generalization ability. By simultaneously pulling positive pairs close together while pushing apart negative pairs in the feature space [5], CL has demonstrated the ability to extract general features from limited signals. This promising result has propelled research interest in leveraging CL for Collaborative Filtering (CF) in top- $K$  recommendation tasks, leading to a marked enhancement in performance and generalization ability [6-12]. Specifically, the prevalent paradigm in CL-based CF methods is to routinely adopt heuristic-based data augmentation for user-item bipartite graphs
*An Zhang and Leheng Sheng contribute equally to this work.
†Zhibo Cai is the corresponding author.
(a)  $\forall j,s(u,j) - s(u,i)\leq 0$
(b)  $s(u,j_1) - s(u,i)\leq -\delta_{j_1}$
(c)  $s(u,j_2) - s(u,i)\leq -\delta_{j_2}$
Figure 1: Comparative geometric interpretation of ranking criteria based on similarity measure  $s(\cdot, \cdot)$  for InfoNCE (1a) and our proposed AdvInfoNCE (1b-1c). The figures depict the user  $u$  as a red dot, with items  $i$  (positive),  $j_1$  (true negative), and  $j_2$  (false negative) depicted as points with varying shades of blue. The color gradient reflects the underlying similarity between user  $u$  and the items, while the shaded region delineates the feasible zone for its corresponding negative item.
[13-15], coupled with the indiscriminate application of contrastive losses such as InfoNCE [16]. Wherein, a common assumption underlying these methods is considering all unobserved interactions as negative signals [17, 18].
Despite the empirical success of CL-based CF methods, two critical limitations have been observed that potentially impede further advancements in this research direction:
- Lack of considering the tailored inductive bias for CF. As a standard approach for semi-supervised top- $K$  recommendation with implicit feedback, CF methods face unique challenges. In most cases, the majority of user-item interactions remain unobserved and unlabelled, from which CF methods directly draw negative instances. However, indiscriminately treating these unobserved interactions as negative signals overlooks the risk of false negatives, thus failing to provide reliable contrastive signals [19]. This issue is known as the exposure bias [20, 21] in recommender systems. Unfortunately, this inherent inductive bias is rarely considered in current CL-based CF methods, resulting in suboptimal recommendation quality and generalization ability [22].
- Limited theoretical understanding for the generalization ability of CL-based CF methods. While data augmentation, as a key factor for generalization ability, is pivotal in computer vision [23, 24], CL-based CF models exhibit insensitivity to perturbations in the user-item bipartite graph [25]. Recent studies [26-28] empirically reveal that contrastive loss plays a more crucial role in boosting performance in CF than heuristic-based graph augmentation. However, an in-depth theoretical understanding of contrastive loss in top- $K$  recommendation that could shed light on the generalization ability of CL-based CF methods remains largely under-explored.
To better understand and reshape the contrastive loss specially tailored for top- $K$  recommendation, we aim to automatically and adversarially assign hardness to each negative user-item pair, which de facto concurrently enlarges the similarity discrepancy for hard negatives and loosens the ranking constraint for false negatives. To be specific, the fundamental idea revolves around discerning a fine-grained degree of hardness for negative interactions, thereby yielding more profound insight into the underlying ranking criterion for top- $K$  recommendation, as depicted in Figure 1. Notably, for a given user  $u$ ,  $\delta_{j}$  refers to the hardness of item  $j$ . When  $\delta_{j_1} > 0$  (as seen in Figure 1b), item  $j_1$  is identified as a hard negative, resulting in a significant contraction of the feasible representation space. Conversely, when  $\delta_{j_2} < 0$  (as seen in Figure 1c), the ranking criterion is relaxed compared to the constraint of InfoNCE, allowing the recommender to mitigate the impact of false negative noise. Nevertheless, the tasks of effectively distinguishing between the hard and false negatives, and of learning a proper fine-grained ranking of interactions continue to present significant challenges [22].
To this end, we incorporate a fine-grained hardness-aware ranking criterion and devise a slightly altered version of InfoNCE loss through adversarial training. This modified contrastive loss coined AdvInfoNCE, is explicitly designed for robust top-  $K$  recommendation. Specifically, we frame hardness learning as an adversarial optimization problem by designing a hardness mapping from interactions to hardness and iterating between hardness evaluation and CF recommender refinement.
Benefiting from this adversarial framework, our AdvInfoNCE endows two appealing properties. On the one hand, it serves as a specialized contrastive loss for top- $K$  collaborative filtering that acknowledges the risk of false negatives while utilizing hard negative mining. Additionally, we theoretically demonstrate through adaptive gradients that AdvInfoNCE subtly employs informative negative sampling (cf. Appendix B.3). On the other hand, we bridge the adversarial hardness learned by AdvInfoNCE with the ambiguity set in distributionally robust optimization (DRO), thereby naturally demonstrating its generalization ability and assuring its robust performance against the noise of false negatives (cf. Theorem 3.1). This furnishes a potential theoretical rationalization for the exceptional robustness observed in contrastive learning. Extensive experiments on the out-of-distribution benchmark datasets in top- $K$  recommendation tasks further highlight the capability of AdvInfoNCE in addressing the issue of biased observations. We advocate for AdvInfoNCE to be considered as a significant reference loss for future contrastive learning research in CF and recommend its adoption as a standard loss in recommender systems.
2 Preliminary of Contrastive Collaborative Filtering (CF)
Task Formulation. Personalized recommendations aim to retrieve a small subset of items from a large catalog to align with the user preference. Here we focus on a typical setting, collaborative filtering (CF) with implicit feedback (e.g., click, purchase, view times, etc.), which can be framed as a semi-supervised top- $K$  recommendation problem [29]. Wherein, the majority of all possible user-item interactions are unobserved, thus previous studies typically assign them with negative labels [29, 30]. Let  $\mathcal{O}^{+} = \{(u,i)|y_{ui} = 1\}$  ( $\mathcal{O}^{-} = \{(u,i)|y_{ui} = 0\}$ ) be the observed (unobserved) interactions between users  $\mathcal{U}$  and items  $\mathcal{I}$ , where  $y_{ui} = 1$  ( $y_{ui} = 0$ ) indicates that user  $u \in \mathcal{U}$  has (has not) interacted with item  $i \in \mathcal{I}$ . For convenience, let  $\mathcal{I}_u^+ = \{i|y_{ui} = 1\}$  ( $\mathcal{I}_u^- = \{j|y_{uj} = 0\}$ ) denote the set of items that user  $u$  has (has not) adopted before. To overcome the distribution shifts in real-world scenarios [31, 10], the long-acting goal is to optimize a robust CF model  $\hat{y}: \mathbb{U} \times \mathbb{I} \to \mathbb{R}$  capable of distilling the true preferences of users towards items.
Modeling Strategy. Leading CF models [32-35] involve three main modules: a user behavior encoder  $\psi_{\theta}(\cdot):\mathbb{U}\to \mathbb{R}^{d}$ , an item encoder  $\phi_{\theta}(\cdot):\mathbb{I}\rightarrow \mathbb{R}^{d}$ , and a predefined similarity function  $s(\cdot ,\cdot):\mathbb{R}^d\times \mathbb{R}^d\to \mathbb{R}$ , where  $\theta$  is the set of all trainable parameters. The encoders transfer the user and item into  $d$ -dimensional representations. The similarity function measures the similarity between the user and item in the representation space, whose widely-used implementations include dot product [30], cosine similarity [36], and neural networks [37]. For the sake of simplicity and better interpretation, we set the similarity function in our paper as:
$$
s (u, i) = \frac {1}{\tau} \cdot \frac {\psi_ {\theta} (u) ^ {\top} \phi_ {\theta} (i)}{\| \psi_ {\theta} (u) \| \cdot \| \phi_ {\theta} (i) \|}, \tag {1}
$$
in which  $\tau$  is the hyper-parameter known as temperature [38].
Loss Function. Point- and pair-wise loss functions are widely used to optimize the parameters  $\theta$ :
- Point-wise losses (e.g., binary cross-entropy [39, 40], mean square error [32]) typically treat observed interactions  $\mathcal{O}^+$  as positive instances and all unobserved interactions  $\mathcal{O}^-$  as negatives. CF methods equipped with point-wise loss naturally cast the problem of item recommendations into a binary classification or regression task. However, due to the large scale of indiscriminative unobserved interactions, this type of loss function fails to effectively consider the ranking criterion nor efficiently handle false negative instances [41].
- Pairwise loss functions (e.g., BPR [30], WARP [42], pairwise hinge loss [43]) aim to differentiate items in a specific relative order. For instance, BPR assumes that the user prefers the positive item over unobserved items. Although effective, these loss functions share a common limitation that lacks sensitivity to label noise and false negatives [22].
Recent CF methods, inspired by the success of contrastive learning (CL) in self-supervised learning tasks, show a surge of interest in contrastive loss:
- Contrastive losses (e.g., InfoNCE [16], InfoL1O [44], DCL [45], SupCon [46]) enforce the agreement between positive instances and the discrepancy between negative instances in the representation space. Separately treating the observed and unobserved interactions as positive and negative
instances allows us to get rid of data augmentation and directly apply these losses in CF. Here we focus mainly on InfoNCE (also well-known as softmax loss in CF [29]), and our findings could be easily extended to other contrastive losses. Specifically, given a user  $u$  with a positive item  $i$  and the set of items  $\mathcal{I}_u^-$  that  $u$  has not interacted with, InfoNCE essentially encourages the CF model to satisfy the following ranking criterion:
$$
\forall j \in \mathcal {I} _ {u} ^ {-}, \quad i > _ {u} j, \tag {2}
$$
where  $>_u$  represents the personalized ranking order of user  $u$ . We equivalently reformulate this ranking criterion in the semantic similarity space, as depicted in Figure 1a:
$$
\forall j \in \mathcal {I} _ {u} ^ {-}, \quad s (u, j) - s (u, i) \leq 0. \tag {3}
$$
Overall, the objective function of personalized recommendation with InfoNCE loss can be formulated as follows:
$$
\mathcal {L} _ {\text {I n f o N C E}} = - \sum_ {(u, i) \in \mathcal {O} ^ {+}} \log \frac {\exp (s (u , i))}{\exp (s (u , i)) + \sum_ {j \in \mathcal {N} _ {u}} \exp (s (u , j))}, \tag {4}
$$
where  $(u,i)\in \mathcal{O}^{+}$  is an observed interaction between user  $u$  and item  $i$ . In most cases, the number of unobserved items, i.e.,  $|\mathcal{I}_u^- |$ , can be extremely large, thus negative sampling becomes a prevalent solution. The standard process of negative sampling involves uniformly drawing a subset of negative instances, denoted by  $\mathcal{N}_u$ , from the unobserved item set  $\mathcal{I}_u^-$ .
Discussion. Scrutinizing the ranking criterion in Equation (3), we can easily find that InfoNCE cares about the positive item's relative orders with the negative items holistically, while ignoring the ranking relationships among these negatives. Moreover, it simply treats all unobserved interactions as negative items, thus easily overlooking the risk of false negatives [29, 30]. Such oversights impede the performance and generalization ability of InfoNCE-empowered CF models. To bridge the gap, we aim to devise a variant of InfoNCE by considering these factors.
3 Methodology of AdvInfoNCE
On the foundation of InfoNCE loss, we first devise AdvInfoNCE, a robust contrastive loss tailor-made for Top-  $K$  recommendation scenarios. Then we present its desirable properties that underscore the efficacy of negative sampling and generalization ability across distribution shifts. Additionally, we delve into the hard negative mining mechanism of AdvInfoNCE and its alignment with Top-  $K$  ranking evaluation metrics in Appendix B.3 and B.4, respectively.
3.1 Fine-grained Ranking Criterion
Instead of relying on the coarse relative ranking criterion presented in Equation (3), we learn a fine-grained ranking criterion in Top-  $K$  recommendation by incorporating varying degrees of user preferences for different items into the loss function. In other words, we need to treat the positive item as an anchor, quantify the relative hardness of each item as compared with the anchor, and differentiate between false negatives and hard negatives. Here the hardness of a negative item is defined as its minimal similarity distance towards the positive anchor.
For simplicity, we consider a single observed user-item pair  $(u,i)\in \mathcal{O}^{+}$  from now on. Let us assume that for  $(u,i)$ , we have access to the oracle hardness  $\delta_{j}$  of each negative item  $j\in \mathcal{N}_u$ . By assigning distinct hardness scores to different negatives, we reframe a hardness-aware fine-grained ranking criterion in the semantic similarity space as follows:
$$
\forall j \in \mathcal {N} _ {u}, \quad s (u, j) - s (u, i) + \delta_ {j} <   0. \tag {5}
$$
Incorporating the hardness scores enables us to exhibit the ranking among negative items (i.e., the larger hardness score  $\delta_{j}$  the item  $j$  holds, the less likely the user  $u$  would adopt it) and endow with better flexibility, compared with InfoNCE in Figure 1a. Specifically, as Figure 1b shows, if  $\delta_{j_1} > 0$ ,  $j_1$  is identified as a hard negative item with a contracted feasible zone, which indicates a stringent constraint prompting the model to focus more on  $j_1$ . Meanwhile, as Figure 1c depicts, if  $\delta_{j_2} < 0$ ,  $j_2$  tends to be a false negative item and its feasible zone expands, which characterizes a more relaxed constraint. Clearly, the hardness degree  $\delta_{j}$  encapsulates a fine-grained ranking criterion for user  $u$ , thereby facilitating more accurate recommendation.
3.2 Derivation of AdvInfoNCE
The fine-grained ranking criterion for a single positive interaction  $(u,i)$ , as defined in Equation (5), can be seamlessly transformed into the main part of our AdvInfoNCE:
$$
\underbrace {\operatorname* {m a x} \left\{0 , \left\{s (u , j) - s (u , i) + \delta_ {j} \right\} _ {j \in \mathcal {N} _ {u}} \right\}} _ {\text {F i n e - g r a i n e d r a n k i n g c r i t e r i o n}} \approx \underbrace {- \log \left\{\frac {\exp (s (u , i))}{\exp (s (u , i)) + \sum_ {j = 1} ^ {| \mathcal {N} _ {u} |} \exp (\delta_ {j}) \exp (s (u , j))} \right\}} _ {\text {A d v I n f o N C E}} \tag {6}
$$
InfoNCE is a special case of AdvInfoNCE, when  $\forall j, \delta_j = 0$ . We briefly outline the main steps of derivation (see Appendix B.1 for the complete derivation):
Derivation Outline. Based on the LogSumExp operator, i.e.,  $\max(x_1, \ldots, x_n) \approx \log(\sum_i \exp(x_i))$ , the fine-grained ranking criterion could be approximated in a different form as  $\log(\exp(0) + \sum_{j=1}^{|N_u|} \exp(s(u, j) - s(u, i) + \delta_j))$ . Consequently, the right-hand side of Equation (6) can be derived by factoring out the negative sign of the logarithm and reorganizing the term.
We would like to highlight the straightforward nature of this derivation, which is achieved without the need for assumptions, but the widely-used LogSumExp operator [47].
However, the oracle hardness  $\delta_{j}$  is not accessible or measurable in real-world scenarios. The challenge, hence, comes to how to automatically learn the appropriate hardness for each negative instance. Inspired by the adversarial training [48], we exploit a min-max game, which allows to train the model alternatively between the prediction of hardness and the refinement of CF model. Formally, taking into account all observed interactions, we cast the AdvInfoNCE learning framework as the following optimization problem:
$$
\min  _ {\theta} \mathcal {L} _ {\text {A d v I n f o N C E}} = \min  _ {\theta} \max  _ {\Delta \in \mathbb {C} (\eta)} - \sum_ {(u, i) \in \mathcal {O} ^ {+}} \log \frac {\exp (s (u , i))}{\exp (s (u , i)) + \sum_ {j = 1} ^ {| \mathcal {N} _ {u} |} \exp (\delta_ {j} ^ {(u , i)}) \exp (s (u , j))} \tag {7}
$$
Here, with a slight abuse of notation, let  $\delta_j^{(u,i)}$  denote the relative hardness of negative item  $j$  in contrast with interaction  $(u,i)$ , and  $\Delta$  represent the collective set of all  $\delta_j^{(u,i)}$ . The definition and enlightening explanation of  $\mathbb{C}(\eta)$  will be presented in Section 3.3.
3.3 In-depth Analysis of AdvInfoNCE
In spite of the adversarial training framework of AdvInfoNCE, the theoretical understanding in terms of its inherent generalization ability remains untouched. In this section, we study it through the lens of distributionally robust optimization (DRO) [49] over negative sampling. From this perspective, AdvInfoNCE focuses on the worst-case distribution over high-quality hard negative sampling.
Theorem 3.1. We define  $\delta_j^{(u,i)}\doteq \log (|\mathcal{N}_u|\cdot p(j|(u,i))$  where  $p(j|(u,i))$  is the probability of sampling negative item  $j$  for observed interaction  $(u,i)$ . Then, optimizing AdvInfoNCE loss is equivalent to solving Kullback-Leibler (KL) divergence-constrained distributionally robust optimization (DRO) problems over negative sampling:
$$
\min  _ {\theta} \mathcal {L} _ {\text {A d v I n f o N C E}} \Longleftrightarrow \min  _ {\theta} \max  _ {p (j | (u, i)) \in \mathbb {P}} \mathbb {E} _ {P} [ \exp (s (u, j) - s (u, i)): \mathcal {D} _ {K L} (P _ {0} | | P) \leq \eta ] \tag {8}
$$
where  $P_0$  stands for the distribution of uniformly drawn negative samples, i.e.,  $p_0(j|(u,i)) = \frac{1}{|\mathcal{N}_u|}$ ;  $P$  denotes the distribution of negative sampling  $p(j|(u,i))$ .
For brevity, we present a proof sketch summarizing the key idea here and delegate the full proof to Appendix B.2.
Proof. By defining  $\delta_j^{(u,i)}\doteq \log (|\mathcal{N}_u|\cdot p(j|(u,i))$ ), we have  $\frac{1}{|\mathcal{N}_u|}\sum_{j = 1}^{| \mathcal{N}_u|}\delta_l^{(u,i)} = -\mathcal{D}_{KL}(P_0||P)$ . Therefore, by rearranging the terms, optimizing AdvInfoNCE can be restated as a standard DRO
problem:
$$
\begin{array}{l} \min  _ {\theta} \mathcal {L} _ {\text {A d v I n f o N C E}} = \min  _ {\theta} \max  _ {\Delta \in \mathcal {C} (\eta)} \sum_ {(u, i) \in \mathcal {O} ^ {+}} \log (1 + | \mathcal {N} _ {u} | \sum_ {j = 1} ^ {| \mathcal {N} _ {u} |} p (j | (u, i)) \exp (s (u, j) - s (u, i))) \\ \Longleftrightarrow \min  _ {\theta} \sum_ {(u, i) \in \mathcal {O} ^ {+}} \sup  _ {p (j | (u, i)) \in \mathbb {P}} \mathbb {E} _ {P} [ \exp (s (u, j) - s (u, i)) ], \text {s . t .} \mathcal {D} _ {K L} (P _ {0} | | P) \leq \eta \\ \end{array}
$$
We now turn our attention to the role of  $\mathcal{C}(\eta)$  and its relationship with the ambiguity set  $\mathbb{P}$  in DRO. The ambiguity set  $\mathbb{P}$  is formulated by requiring the distribution of negative sampling to fall within a certain  $\eta$  distance from the uniform distribution, as defined below:
$$
\mathbb {P} = \left\{p (j | (u, i)) \in \left(\frac {1}{| \mathcal {N} _ {u} |} - \epsilon , \frac {1}{| \mathcal {N} _ {u} |} + \epsilon\right): \mathcal {D} _ {K L} \left(P _ {0} \| P\right) \leq \eta = - \log \left(1 - \frac {\epsilon^ {2}}{| \mathcal {N} _ {u} |}\right) \right\} \tag {9}
$$
where  $\epsilon$  serves as the hyperparameter to regulate the deviation of the negative sampling distribution from the uniform distribution. In terms of implementation,  $\epsilon$  is controlled by the number of adversarial training epochs (see Algorithm 1). Clearly,  $\mathcal{C}(\eta)$  is an equal counterpart to the ambiguity set  $\mathbb{P}$ .
Discussion. Grounded by theoretical proof, we understand the generalization ability of our AdvInfoNCE through the lens of DRO over informative negative sampling. In essence, by learning the hardness in an adversarial manner, AdvInfoNCE effectively enhances the robustness of the CF recommender. Moreover, apart from the robustness of AdvInfoNCE (see Section 3.3), we also identify its hard negative mining mechanism through gradients analysis (see Appendix B.3) and its alignment with top- $K$  ranking evaluation metrics (see Appendix B.4). Specifically, the gradients concerning negative samples are proportional to the hardness term  $\exp(\delta_j)$ , indicating that AdvInfoNCE is a hardness-aware loss. To some extent, our AdvInfoNCE is equivalent to the widely-adopted Discounted Cumulative Grain (DCG) ranking metric.
**Limitation.** AdvInfoNCE employs end-to-end negative sampling in an adversarial manner, compelling the recommender to ensure robustness from the worst-case scenario. Despite its empirical success and desirable properties, AdvInfoNCE is not immune to the limitations of adversarial training, which is well-known for its potential training instability.
We aim to answer the following research questions:
- RQ1: How does AdvInfoNCE perform compared with other CL-based CF methods?
- RQ2: Can AdvInfoNCE effectively learn the fine-grained ranking criterion? What are the impacts of the component  $\epsilon$  (i.e., the number of adversarial training epochs) on AdvInfoNCE?