Semantic Retrieval Augmented Contrastive Learning for Sequential Recommendation
Authors: Ziqiang Cui
Yunpeng Weng
Xing Tang
Xiaokun Zhang
Shiwei Li
Peiyang Liu
Bowei He
Dugang Liu
Weihong Luo
xiuqiang He
Chen Ma
Venue: NeurIPS 2025 poster ()
Keywords: Large Language Models
Recommender Systems
Contrastive Learning
Primary area: applications
TL;DR: This paper proposes a novel framework that generates better contrastive pairs for contrastive learning by integrating LLM-based semantic retrieval with a learnable sample synthesizer.

Abstract: Contrastive learning has shown effectiveness in improving sequential recommendation models. However, existing methods still face challenges in generating high-quality contrastive pairs: they either rely on random perturbations that corrupt user preference patterns or depend on sparse collaborative data that generates unreliable contrastive pairs. Furthermore, existing approaches typically require predefined selection rules that impose strong assumptions, limiting the model's ability to autonomously learn optimal contrastive pairs. To address these limitations, we propose a novel approach named Semantic Retrieval Augmented Contrastive Learning (SRA-CL). SRA-CL leverages the semantic understanding and reasoning capabilities of LLMs to generate expressive embeddings that capture both user preferences and item characteristics. These semantic embeddings enable the construction of candidate pools for inter-user and intra-user contrastive learning through semantic-based retrieval. To further enhance the quality of the contrastive samples, we introduce a learnable sample synthesizer that optimizes the contrastive sample generation process during model training. SRA-CL adopts a plug-and-play design, enabling seamless integration with existing sequential recommendation architectures. Extensive experiments on four public datasets demonstrate the effectiveness and model-agnostic nature of our approach. 
Our code is available at https://github.com/ziqiangcui/SRA-CL

Semantic Retrieval Augmented Contrastive Learning for Sequential Recommendation
Ziqiang Cui!, Yunpeng Weng? Xing Tang", Xiaokun Zhang', Shiwei Li², Peiyang Liu
Bowei He², Dugang Liu", Weihong Luo', Xiuqiang He*, Chen Ma!,
City University Hong Kong HHuazhong University of Science Technology
Tencent Shenzhen Technology University Pekknng University "Shenzhen University ziqiang cui @ww.eemmmm.m.uuu.tt.e (wengyp. lishiwei) @hust.edu.cn, xing.tang hotmail. @ @kku.edu.coo boweihe2-cee letiiee-c@mmcciiuu.ceoee my.cityu.edu.hk. lobby66@ 163.com, hexiuqiang @ eztu.edu.cn, chenma cityu.cdu.hk
[dawnkun] 1993, dugang. @ gom, liupeiyang
Abstract
Contrastive learning has shown effectiveness in improving sequential recommen- dation models. However, existing methods still face challenges in generating high-quality contrastive pairs: they either rely on random perturbations that corrupt preference patterns depend sparse collaborative data that generates unreliable contrastive pairs. Furthermore, existing approaches typically require predefined selection rules that impose strong assumptions, limiting the model's '' abil- ity to autonomously learn optimal contrastive pairs. To address these limitations, we propose novel approach named Semantic Retrieval Augmented Contrastive Learning (SRA-CL.). SRA-CL leverages the semantic understanding and reason- ing capabilities of LLMs to generate expressive embeddings that capture both user preferences and item characteristics. These semantic embeddings enable the construction of candidate pools for inter-user and intra-user contrastive learning through semantic-basse retrieval. To further enhance the quality of the contrastive samples, we introduce learnable sample synthesizer that optimizes the contrastive sample generation process during model training. BRA-CL adopts plug-and and-play design, enabling seamless integration with existing sequential recommendation architectures. Extensive experiments on four public datasets demonstrate the ef- fectiveness and model-agnostic nature of our approach. Our code is available htttps///gtthhn cow wwtp///gagngguuu/sRA-------
1 Introduction
Sequential recommendation aims to model user preferences based on historical behavior sequences, task of significant value for online platforms like YouTube and Amazon. However, accurate preference modeling faces fundamental challenge: data sparsity, as most users have only limited interaction records and most items receive little attention. To address this issue, numerous self- supervised learning techniques [39 46 34) have been proposed, leveraging auxiliary tasks to improve data utilization efficiency, Among these, contrastive learning has emerged as predominant approach due to its conceptual simplicity and proven effectiveness [34 24 3 [2 47]. Typically, it constructs positive sample pairs from the data and maximizes their agreement the representation space (2]
As illustrated in Figure [ existing contrastive learning approaches for sequential recommendation can be broadly classified into two categories: (1) inter-user contrastive learning, which contrasts sequences
"Equal contribution. *Corespooning authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Recommendation Task Loss
Inter-User Contrastive Learnnng
Intra-User Contrastive Learning
T
K-Means
Soquence Encoder
Maxirnize
Maximize Agrecment
Agreement
Sequence Encoder
Sequence Encoder
Mask
Mork
Figure I: Illustration of existing contrastive learning methods in sequential recommendation, catego- rized into two main types: (1) inter-user contrastive learning and (2) intra-user contrastive learning.
from different users, and (2) intra-user contrastive learning, which contrasts different augmented views of single user's sequence. In the inter-user paradigm, user sequence representations clustered using K-means, and users within the same cluster are treated as positive samples for each other B 23). In the intra-user paradigm, perturbations are applied to user's sequence to generate augmented views, and the similarity between these views is maximized (34 21 [24, 22).
These contrastive learning methods are typically employed auxiliary tasks alongside the primary recommendation objective and have been demonstrated to enhance recommendation performance by
improving user representation learning [4 36).
Despite their empirical success, existing methods suffer from several limitations in contrastive pair construction, which may undermine their effectiveness in recommendation scenarios. 1) Semantic Divergence. Many existing methods construct contrastive pairs through random augmentation opera- tions such random masking (34 21 and Dropout (24). However, in sequential recommendation where data is inherently sparse and exhibits sequential patterns, such random operations may lead a complete change in the sequence's semantics (i.e., user preferences). Bringing semantically different sequences closer together in embedding space may diminish the model's ability to discrim- inate among distinct user preferences. Additionally, some methods determine contrastive pairs by clustering user representations derived from collaborative signals [3] 23). where users within the same cluster are considered positive pairs. However, the sparse ID signals can lead to low-quality representations and inaccurate clustering results. 2) Unlearnability. Existing methods rely on predefined rules to construct positive pairs, such as directly selecting users from the same cluster
23 21. or treating sequences sharing the same next item as positive pairs [24 23 These rigid heuristies impose strong assumptions that constrain models from autonomously learning optimal contrastive pairs. Moreover, the approach of using sequences with identical next items as positive pairs essentially replicates the recommendation objective (i.e., next-item prediction), providing no additional information gain. Therefore, the suboptimal construction of contrastive pairs in existing methods limits their effectiveness and hinders contrastive learning's full potential.
Given these limitations, constructing high-quality contrastive samples remains a critical challenge. Semantic information, which readily available in textual data such as product categories, brands, and descriptions, provides a promising solution. Unlike sparse behavior signals, semantic data maintains validity regardless of data volume or training dynamics, as it derives from structured knowledge rather than co-occurrence patterns [66]. Additionally, semantic features offer complementary information beyond collaborative signals. Motivated by these advantages, we propose leveraging semantic information to construct superior contrastive pairs. However, accurately capturing user preferences requires models with powerful understanding and reasoning capabilities. Recent research has shown that large language models (LLMM)) can effectively understand user preferences and achieve competitive performance on sequential recommendation tasks [35]. Inspired by this, we propose to enhance contrastive learning through LLM- LLW-po semantic retrieval.
In this paper, we propose SRA-CL (Semantic Netrieval-Augmentt Contrastive Learning), a novel framework with two key innovations: 1) Semantic-based Retrieval. We develop a semantic- based retrieval mechanism that operates at both inter-user and intra-user levels. For inter-user contrastive learning, we leverage LLMs to process sequential user interaction histories. Each sequence is fed to the LLM in chronological order of item interactions, where each item consists of both its attributes and textual description, enabling the model to generate preference-aware semantic embeddings through comprehensive understanding of user behavior patterns. For intra-user contrastive learning, enhance item understanding by providing LLMs with both item attributes and their contextual

---

sequence information, producing context-aware semantic embeddings that capture both intrinsic item properties and their relevance within the recommendation context. Subsequently, we leverage the semantic embeddings to retrieve the top-k most similar users and items, constructing candidate positive sample pools for contrastive learning. 2) Learnable Sample Synthesis. To construct more effective contrastive samples, our framework incorporates learnable sample synthesizer. For inter-user contrastive learning, the synthesizer dynamically generates positive samples for each user sequence by selectively combining elements from the candidate pool. This generation process is jointly optimized with the model training, ensuring the synthesized samples effectively improve representation learning.
Our main contributions are summarized as follows.
We propose model-agnostic framework, SRA-CL, which leverages semantic information and the capabilities of LLMs to construct better contrastive pairs, thereby improving the contrastive learning in sequential recommendation.
We propose semantic-basee retrieval approach for contrastive pair construction that integrates dual retrieval mechanisms: user retrieval for inter-user contrastive learning and item retrieval for intra-user contrastive learning, with each mechanism maintaining its dedicated candidate pool. To further enhance this framework, we introduce a learnable sample synthesizer that optimizes the contrastive sample generation process during model training.
We conduct extensive experiments on four public datasets to validate the superiority and model- agnostic nature of our approach, as well as to confirm the efficacy each module.
Preliminary
2.1
Sequential Recommendation Task
We denote the of users and items by U and respectively. Each user u U has chronological sequence of interacted items S, [u 12 ",), where vi" indicates the item that interacted with step t, and n is the predefined maximum sequence length. For user sequences longer than n, we retain only the most recent items. The goal of sequential recommendation is to predict the next item t according S,, which can be formulated as:
arg max P(o+ u vlS.), "EV
where the probability P represents the likelihood of item e being the next item, conditioned on S..
2.2 Sequential Recommendation Backbone
Our method model-agnostic and can integrated with various sequential recommendation models, as demonstrated in Section To facilitate the introduction of our approach, we adopt the transformer architecture (28) as the backbone recommendation model following previous studies (22-24).
Embedding Layer. We initialize embedding matrix M R to encode item IDs, where [V represents the size of the item set and d denotes the dimensionality of the latent space. Given a user interaction sequence S,, we obtain item embeddings E,, e x and position embeddings P E gaxd Consequently, the input sequence S, can be represented H., E., P.
Sequence Encoder. The representation of the input sequence is then fed into L Transformer layers 28 to capture complex sequential patterns, which can be defined as follows:
(oansformerH..). h = H
Here, h, E d represents the last position of H and selected as the final representation of S.,. Prediction and Objective Function. During prediction, we calculate the probability of each item using ý = softmax h, where ÿ e RD' and te represents the likelihood of item being the next item. For training, we adopt the same cross-entropy loss function our baseline methods 22 24 to ensure fairness, where + denotes the ground truth item for user u.
LRse log(
(3)
3

---

Inter-User Contrastive Learning via User Semantic Retrieval
Synthexival Pavitive Sample
DA
Overs
User Semambic Embedkings
Loss
Seq Encoder
Attestiin
Contrastive e Learning via Retrieval
LIM
Finheddings
Contrasive
Loss
Encodereh
Share
Pronce
Nare, Brank Compary Description
Retheve
Retrivve
Saquinoc
PA
Figure 2: Overview of the proposed SRA-CL Framework.
3. The Framework of SRA-CL
In this section, we provide detailed introduction to SRA-CL, which is shown in Figure SRA-CL integrates inter-user contrastive learning via user semantic retrieval and intra-user contrastive learning via item semantic retrieval. To further enhance the framework, introduce learnable sample synthesizer that optimizes the contrastive sample generation process during model training.
3.1 Inter-User Contrastive Learning via User Semantic Retrieval
SRA-CL employs semantic retrieval generate reliable supervision signals for inter-user contrastive learning. Leveraging the advanced reasoning capabilities of LLMs, we first derive comprehensive representation of user preferences, which are then encoded as semantic embeddings. Based on the similarity of these embeddings, we introduce a semantic- based retrieval mechanism to construct candidate sample pool. Subsequently, learnalee contrastive sample synthesis method is employed to generate effective contrastive pairs.
User Preference Understanding with LLMs. Textual data (e.g.. product categories, brands, and descriptions) plays a pivotal role recommender systems by encoding rich semantic signals that reflect user preferences. Given user u's interaction sequence S,, we extract textual attributes for each item in chronological order, preserving both content and sequential context. These features structured into prompt P.. where item attributes and their order explicitly guide the LLM in inferring user preferences A, = LLM(Pw). The prompt template is detailed in Figure Next, we employ pretrained text embedding model M to extract and convert the semantic informa- tion contained in the textual responses of L.LMSs into embeddings, which formatted as:
h M(A)),
where hi E dd represents the semantic embedding of user preferences and d is the embedding size of the text embedding model M. Specifically. M indicates SimCSE-RoBERTE to its open-souree availability and excellent sentence semantic extraction capabilities. The generated semantic embeddings are cached and remain fixed throughout the whole training process.
in this paper due
Semantic- based User Retrieval. Once the semantic embeddings of user sequences are obtained, similar users can be retrieved based on semantic similarity. For given user sequence S,, we calculate the cosine similarity between its semantic embedding h, and the semantic embeddings other users. Users are then ranked in descending order according to the computed semantic similarity. The top k users are retrieved to construct the homogeneous user pool for user u, denoted as No..
Nr. {u' E 24 {u} rank(cosine) Lank(cssie_ssimilartyy (h.,, h.))) S k).
(5)
where H \\ {u} denotes the set of all users except

---

Learnable Contrastive Sample Synthesis. Sole reliance on hard rules, such as selecting user from the current user's dedicated candidate pool as the positive sample, often yields suboptimal solutions (as shown in Table 2). To enhance contrastive sample construction, we introduce a learnable sample synthesizer that optimizes the contrastive sample generation process during model training. Specifically, we first map the semantic representations of user sequences through a learnable adapter. Then, in the mapped space, we employ an attention mechanism, where the current user serves as a query to compute the probability Pu,w' that each candidate user u' E Ni. is suitable as the positive sample for the current user u. This process is formulated as:
Wa LeakyReLU(a [Wh, Wh
(6)
exp(Wuu.k
Pu,a' softmax, (oftmax./(ww.))) En,EN. exp( (xp(Wu,)))
(7)
where W E IRdxd is learnable weight matrix, and || denotes the concatenation operation. a E R2 represents single-layer neural network used to generate the attention score, with the LeakyReLU activation function adopted [29]. The softmax function is employed to transform the coefficients into probabilities. Based on this, generate the composite positive contrastive sample h for h, by:
h = E.wex. Pu,u Pu,cha's ha',
(8)
where has E IR.d is the recommendation model's output sequence representation for v', as defined in Equation (2. This operation enables a fine-grained learnable selection of contrastive samples.
Inter-User Contrastive Loss. For each user u. h, the sequence representation obtained from the recommendation model. The synthetic representation h is regarded as the positive sample for h.. while the remaining N 1 synthetic representations within the same batch are treated as negative samples for h,, where N is the batch size. We compute the inter-user contrastive loss Les as follows:
exp (h, h
Lcs = - loog exp (h. h ) En,eh, exp (h. h.
where (.) represents the inner product operation, H denotes the set of negative samples for ha-
3.2 Intra-User Contrastive Learning via Item Semantic Retrieval
For intra-user contrastive learning, most existing methods apply predefined random perturbations to the original sequence to generate augmented views, which are treated as a pair of positive samples 34 24 A significant limitation of them is the introduction of considerable uncertainty in the semantic similarity between positive samples. This substantial variation in user sequence semantics among positive samples undermines the reliability of the contrastive learning process. To address this issue, we leverage a comprehensive understanding of both the semantic information of the item itself and the typical contexts in which the item appears. Based on this understanding, we replace certain items in the sequence with similar ones, resulting in semantic- cemantic-consistent positive samples.
Item Understanding with LLMs. To enhance the LLM's comprehension of items, provide two types of input information: (1) textual altributes of the item, including category, brand, and description, which supply fundamental information and enable the LLM perform a -oarse-grained assessment of item similarity; and (2) user sequences containing the given item. By analyzing the typical contexts in which an item appears, the LLM can infer the characteristics of its potential audience. This methodology facilitates more accurate evaluations of the relationships between items in the context of sequential recommendation. Given the token limit for LLM input, have constrained the maximum number of item-related sequences in the prompt to 10, leaving the exploration of this value for future research. Next, these two types of information for item are integrated into a structured prompt Pe.. which is processed by the language model to generate the item summary A. = LLM(P.).. The detailed prompt template illustrated in Figure 6 Then, the pretrained text embedding model M is used to convert the textual responses of LLMs into embeddings: é, M(A)).
Semantic-based Item Retrieval. Similar to user retrieval, we compute the cosine similarity between the semantic embedding of an item and those of other items. Next, Top-k most semantically relevant items for item u are retrieved, which is formulated as:
No = {o' E VV (v) rauX(osiin__smiilatiyee... ē.))) S k),
(10)
5

---

Contrastive Sample Selection. For intra-user contrastive learning, generating two semantic- consistent augmented views of the same user sequence is crucial. Here, we employ semantic-based item substitution approach. Specifically, for each sequence S,, we randomly select 20% of the items. For each selected item v, we substitute with a semantically similar item sampled from its candidate .N. This operation yields an augmented sequence Sy derived from the original S,. By repeating this process, we obtain two augmented views, denoted as St and S", which form a positive sample pair. Critically, the substitution is not entirely random but is guided by semantic similarity, which accounts for both item attribute similarity and contextual relevance in the recommendation scenario. This reduces uncertainty and enhances semantic consistency between augmented views.
Our preliminary experiments also explored the use of learnable synthesizers (analogous inter- contrastive learning approaches) for generating substitute items, yet yielded no measurable perfor- mance improvements (shown in Table This can be attributed the inherently higher interpretability and quantifiability of item semantics relative to user preferences. Therefore, directly identifying appropriate substitutes from semantically similar candidate pools simpler and more reliable com- pared to matching users with analogous preference patterns. A more detailed analysis is provided in Appendix C.T]
Intra-User Contrastive Loss. For the two augmented sequences S' and S we obtain their hidden vectors h' and h" using the sequence encoder defined in Equation Then the intra-used user contrastive loss can be calculated as:
exp (h", hw)
log Lis
(11)
exp (h,, bm) + 2 2x EH exp (h', mey
In batch with a size of N, we have 2N augmented sequences. Among these, h', and h are positive samples of each other and are interchangeable. The remaining 2(N 1) samples excluding h and h" are considered negative samples H
neg
Algorithm Training for SRA-CL
Require: Training data } } for all E Lt; hyperparameters a, 3, k 1: Obtain user semantic embeddings (h,} for all E Ui; obtain item semantic embeddings (ē.)
for all E V.
2: Freeze the embeddings {h.} and {ē,), and initialize the model parameters.
3: for each iteration do
4: Compute h, using Equation
5: Calculate ý = softmax
6:
Compute Lree using Equation
7:
Retrieve N₁, for each u E 24 using Equation
8:
Synthesize ht for each using Equations (6 and 8
9:
Compute Lcs using Equation (9).
10:
Retrieve N₂ for each
V using Equation (TO).
11:
Generate S, and S. along with the corresponding h', and h".
Compute Lis using Equation (II)
13: Calculate the total loss L LRee + aLcs + BLs.
Update the model parameters using the gradient of L.
15:
end for
Return the final model parameters
Algorithm 2 Inference for SRA-CL
Require: Trained model parameters 0; test data {Sw}
1: for each user sequence in test data do
2: Compute h. using Equation 2
3: Calculate the predicted scores y softmax (h,
4: Obtain the top-k items with the highest scores y.
5: end for
6: Return the recommended items for all users.

---

3.3 Training and Inference
During the training phase, all semantic embeddings are fixed. The training objective consists of three components: the loss of the recommendation model LRee, which serves the main loss, and the inter-user contrastive loss Les and intra-user contrastive loss Lis, which act as regularization terms.
L LR aLcs BLis.
(12)
where a and 3 are hyperparameters.
During inference, only the recommendation backhone is utilized. The contrastive learning tasks and I.LMs' semantic embeddings are not involved in the inference process. This implies that our framework can be deployed in real-world applications without incurring any additional inference latency from incorporating LLMs. The training and inference processes are detailed in Algorithm) respectively.
and Algorithm 2
4 Experiments
4.1 Experimental Settings
conducted experiments on four public real- world datasets: Yelp, Amazon Sports, Beauty, and Office. The statistics for these datasets are presented in Table More details about the datasets are shown in Appendix [B.]] [B.1]
Datasets. Following previous studies 21 34 22],
Evaluation Metrics. To evaluate the performance of the models, we use widely recognized evaluation metrics: Hit Rate (HR) and Normalized Discounted Cumulative Gain (NDCG), follow previous studies 41 30 9 [2). The leave-one-out strategy employed, where the last interaction is used for testing, the second-to-last interaction for validation, and the remaining interactions for training. To ensure an unbiased evaluation, we rank the prediction on the whole item set without sampling.
Baseline Methods. We compare our method with 13 baseline methods, categorized into three groups:
1) classical methods (GRU4Rec [0], SASRec [12). BERT4Rec (27). 2) contrastive learning-baat based
methods -S-RRec [46). CL4SRec (34), CoSeRec (21). ICLRee 3 DuoRec (24). MCLRee [22].
ICSRec 23 and 3) ALM-based methods (LRD RLMRec 25 LLM-ESR (20)).
Implementation Details. All experiments are conducted with single 32G V100 GPU. The em-
bedding size is set to 64. We adopt the batch size of 256 and employ the Adam optimizer with learning rate of 0.001. The dropout rate is set 0.5 across all datasets. Following previous studies [35]. we set the maximum sequence length to 20. The early stopping is applied if the metrics on validation set do not improve over 10 consecutive epochs. For LLM, we use DeepSeek- -V3 by invoking its API. We set the LLM's temperature to O and top-p to 0.001. For the text embedding model M. we use the pre-trained RoBERTa from Hugging Face. Note that identical settings are adopted for our method and baselines that involve LLMs and text embeddings to ensure fairness. More implementation details can be found in Appendix B.3
4.2 Comparison Results with Baselines
The comparison results are presented in Table 1 Each experiment conducted five times, and the average results are reported. SRA-CL consistently outperforms all baseline methods across all datasets, achieving performance improvements of up 11.82% The improvements are also confirmed by paired t-test with a significance level of 0.01. Contrastive learning- methods generally surpass traditional methods (GRU4Rec, SASRec, BERT4Rec). Among the contrastive learning baselines, MCLRec and ICSRec demonstrate superior performance. However, both methods underperform compared SRA-CL, as they fail to control the quality of contrastive samples. SRA- retrieval augmentation, thereby improving
CL mitigates this issue by introducing semantic-basee the quality of contrastive samples and enhancing the overall effectiveness of contrastive learning. Regarding LLM- -nh-nnne baselines, they demonstrate superior results compared to classical methods. However, our proposed SRA-CL achieves significant improvements over these LLM approaches. Un- like existing M-Lee methods, SRA-CL. is fundamentally different in motivation- specifically addresses the limitations in contrastive learning through enhanced construction of positive sample pairs using semantic information.
7

---

Table 1: Performance comparison of different methods on four datasets. Bold font indicates the best performance, while underlined values represent the second-best. SRA-CL achieves state-of-the-ae results among all methods, confirmed by paired t-test with significance level of 0.01. Due to space constraints, additional metrics (HR @ 10 and NDCG@ 10) are provided in Appendix C.2
Yelp
Sports
Beauty
Office
Model
HR@ 20
NDCG@20
HR
NDCG@20
HR
NDCG@
HR@20
NDCG@20
GRU4Rec
0.0639
0.0243
0.0325
0.0129
0.0488
0,0189
0.0956
0.0361
SASRec
0,0899
0.0390
0,0498
0.0216
0.0887
0,0382
0.1329
0.0482
BERT4Rec
0.0913
0.0394
0,0578
0.0241
0.0933
0,0399
0,1436
0.0520
R*e
0.0964
0.0443
0.0607
0.0262
0.0994
0.0414
0.1568
0.0571
CL4SRec
0.0923
0.0395
0,0562
0.0235
0.0980
0.0416
0.1297
0.0488
CoSeRec
0.0984
0.0404
0.0638
0.0293
0.14
0.0487
0.1354
0.05 16
ICLRec
0.0974
0,0432
0.0636
0.0284
0.1056
0,0482
0,1513
0.0559
DunRec
0.1173
0.0493
0.0706
0.0302
0.1224
0.0535
0,1549
0.0653
MCLRec
0.1150
0,0486
0.0736
0.0318
0.1239
0.0536
0,1629
0.0684
ICSRec
0.1165
0.0495
0.0728
0.0304
0.1205
0.0528
0.1643
0.0690
LRD
0.1082
0.0455
0.0589
0.0257
0.0931
0.0402
0.1468
0.0577
RLMRec
0,1125
0.0478
0.0664
0.0298
0.1
0.0521
0.1532
0.0613
LLM-ESR
0.1061
0.0451
0.0638
0.0277
0.1064
0.0515
0,1425
0.0602
SRA-CL
0,1282
0.0533
0.0823
0.0347
0.1314
0.0568
0.1702
0.0725
Improvement
9.29%
7.68%
11.82%
9.12%
6.05%
5.97%
3,59%
5.07%
Yolo
Sports
Beauty
Office
0.15
+10.4%
6:10
9.16
0:00
-11.0%
$2.8%
5:08
6.06
+27.2%
0.16
+16.5%
5.04
2%
D06
90
GRUdRRo GRLARRSO
BASffer
BusRan DuaRomk
Ed0
GRVARIO GRLARREn
BAuRbe
BAUR
Dinffee Dusiter
QutRee Rasitenn
GRUATIe
GRLARRSe
Darieed Aaiitin
Yelp
Sports
Beauty
Office
PD 2%
0000
NDC10g000
ND00320
0.22
+11.8% 8%
NDco-g20
2.04
1002%
GRUMdfeed ORldfee
Storie Sestee
Dopfie Boaflack
GRUMPee
Dowffe Dusffane
GRLA
Reeffe Chaffenk
GRUatee GRuifiee
SeoPpe
Qunfiee
Figure 3: Experimental results demonstrating the model-agnostic nature and strong generalization capability of SRA-CL. "+" indicates the addition of SRA-CL to different recommendation models.
4.3 Validation of Model-Agnostic Characteristic
In this section, we validate the model-agnostic nature of our method. We select three classic recommendation models (GRU4Rec, SASRec, DuoRec) as the backbone and integrate SRA-CL to observe performance changes. We retain the original loss functions of the backbones and introduce our contrastive loss Lcs and Lis during training. The results are shown in Figure which indicate that for all three backbone methods, the versions enhanced with SRA-CL ("+") consistently outperform the original versions. Specifically, HR @ 20 improves by 8.3% to 27.3%, and NDCG@22 20 increases by 9.7% 25.5%. These findings validate that SRA-CL can robustly improve the performance of various recommendation models.
4.4 Ablation Study
In this section, we evaluate the effectiveness of each component in SRA-CL. The results, presented in Table demonstrate the impact of removing individual modules. Overall, the results show that removing any component degrades model performance, confirming the necessity of each module. Specifically, the variants "w/o Lcs" and "w/o Lis" exhibit significant performance drops, highlight- ing the importance of both inter-user and intra-user contrastive learning objectives. The "w/o CL" variant suffers a more severe performance decline than those removing only one contrastive objective, suggesting that these two types of objectives complement each other. Additionally, the "w/o learn." variant also leads to reduced performance, indicating that a learning- sample synthesizer more effective than random selection for inter-user contrastive learning. Furthermore, removing
8

---

Table 2: Ablation study on all datasets.
w/o CL
w/o Las
w/o Lis
w/o learn.
w/o semantic
w/o
Metric
Ours
LLM
11@20
0.1101
0.1203
0.1228
0.1253
0.1187
0.1190
0.1282
Yelp
N@20
0.0473
0.0504
0.0519
0.0520
0.0495
0.0501
0.0533
H@
0.0745
0.0780
0.0795
0.0792
0.0772
0,0781
0.0823
Sports
N@20
0.0296
0.0315
0.0332
0.0336
0.0311
0.0314
0.0347
H@
0.1206
0,1273
0.1279
0.1273
0.1265
0.1259
0.1314
Beauty
N@-0
0.0518
0.0546
0.0545
0.0551
0.0532
0.0537
0.0568
1 20
0.1476
0.1621
0.1619
0.1617
0.1624
0,1643
0.1702
Office
@0
0.0599
0.0691
0.0689
0.0681
0.0673
0.0692
0.0725
Sports
Office
0.150
150
HR@20
HR@20
HR@22
0.125
0 125
0.5
0.100
0-100
0,075
0.075
10.3
10-2
56-2
10-1
Se-1
16-3
10-2
50-2
50-1
20
50
100
8
Figure 4: Hyperparameter experiments on the weight of Los (a), the weight of Lis (3), and the number retrieved users/items (k).
semantic information and relying solely on collaborative signals for retrieval ('w/o semantic") results in notable performance decline, underscoring the importance of semantic information in construct- ing high-quality contrastive samples. This finding aligns with our initial motivation. Similarly, the absence of LLM-based text processing ("w/o LLM"") also results in performance degradation, demonstrating that utilizing the LLM's ability to understand and reason about user preferences is erucial.
4.5 Hyperparameter Study
In this section, we investigate the impacts of three key hyperparameters, a, 3, and k. Here, a and B are the weights of Los and Lıs, respectively, while i denotes the number of retrieved users/items. From Figure 4 we observe that as both a and / increase. the model's 's performance initially improves slightly and then decreases marginally. Empirically, the optimal range for a and [3 is between 0.05 and 0.1. This is reasonable as contrastive learning loss acts as regularization term. As the value of k increases, the performance initially improves and then declines, with the optimal value around 10. As k increases, the semantic relevance of retrieved neighbors decreases and randomness increases. A very small i results in a candidate set that too small without diversity. Conversely, very large k loses semantic relevance, thereby degrading the effectiveness of contrastive learning. Note that NDCG@ 20 results are provided in Figure 7 due to space limitation.
4.6 Contrastive Learning in Sparse Data: Analyzing SRA-CL's Superiority
To further examine SRA-CL's capability in mit- igating the issue of low-quality contrastive sam- ples data-sparse scenarios,
Beauty
Office
categorize
0 1.1
MCLREs SPA/CL
MCLRas
user sequences into three groups based on their length and compare the evaluation results of different methods. Due to space limitation, we present the experimental results for Beauty and Office, as shown in Figure 5 By com- paring SRA-CL with the two strongest con- trastive learning baselines (MCLRec and IC- SRec), we observe that SRA-CL consistently outperforms them across all user groups. No- tably, our method achieves greater improvements in sparser user groups (e.g., those with fewer than 7 or 7-10 interactions). This result further validates our motivation: while MCLRec and ICSRec
0.200
S1M4-CH OL
0.150
0.100
0.075
7-10
>10
7-10 lnntth
Sequence ength
Figure 5:
Performance comparison on different user groups among MCLRee, ICSRec and Ours.

---

construct contrastive sample pairs based on collaborative signals, their performance degrades in data-sparse scenarios due the diminished quality of contrastive samples. In contrast, our method significantly enhances the quality of contrastive pairs by incorporating semantic information, leading to superior performance under sparse data conditions.
5 Related Work
Contrastive Learning in Sequential Recommendation. Contrastive learning has been successfully used to enhance sequential recommendation [40] [1 38 31
33 37 4 5 23]. In terms of the composition of contrastive samples, we categorize existing methods into two types: (1) Inter- user. This involves generating contrastive samples from different user sequences. For example, ICLRec (3] clusters user interests into distinct categories by K-Means and brings the representations of users with similar interests closer. ICSRec (23] further segments a user's behavior sequence into multiple subsequences generate finer-grained user intentions for contrastive learning. These methods generate contrastive supervision signals based on collaborative signals. However, the sparsity of the co-occurrence pattern leads unreliable clustering results, which in turn affects the performance of contrastive learning. (2) Intra-user. This involves applying perturbations to the original sequence to generate augmented views. The two views of the same sequence are treated as a pair of positive samples. For example, CL4SRec (34] employs three data-level augmentation operators: Cropping, Masking, and Reordering, to create contrastive pairs. CoSeRee [2T] introduces two additional informative augmentation operators, building upon the foundation of CL4SRec. In addition, some methods generate augmented views from the model's hidden layers. notable example is DuoRec (24]. which creates positive pairs by forward -pssing a sequence representation twice with different dropout masks. MCLRec [22 further combines data-level and model-level augmentation. Despite their effectiveness, they employ random operators, introducing significant uncertainty and potentially generating unreasonable positive samples for contrastive learning.
Sequential Recommendation with LLMs. Building upon foundations laid by traditional recom- mender systems 9 15 16 recent studies have successfully integrated LLMs into the recommender 14 6. Overall, LLMs are employed either as direct recommenders or tools for extracting semantic information [13 19 42 26]. In the former approach, all inputs are converted into textual format, and the LLM generates recommendations based on its pre-trained knowledge or after undergoing supervised fine-tuning. Representative examples include LC-Rec [44], LLM-TRSR [45), and CALRec [18]. However, these methods rely the inference process of large language models to generate recommendation results, which is computationally expensive and often challenging deploy in practical scenarios. Another line of research [26 30 20
paradigm [32
43
35| 42 [
leverages LLMS to process semantic information and incorporates it into traditional ID-based models. For example. SLIM (30] distills knowledge from large-scale LLMs into a smaller student LLM to improve the recommendation model. LLM-ESR (20) addresses the long-tail problem by leveraging collaborative signals and semantic information through dual-view modeling and self-distillation. LRD (35] utilizes the LLM to explore potential relations between items and reconstructs one item based on its relation another. Unlike the aforementioned methods, our approach, grounded in the essence of contrastive learning, aims to construct more effective contrastive pairs with LLMSs.
6 Conclusion
In this paper, we analyze the limitations of contrastive learning in sequential recommendation, namely Semantic Divergence and Unlearnability. To address these issues, we propose SRA-CL, novel framework that enhances contrastive sample construction by integrating LLM-based semantic retrieval with learnable sample synthesizer. SRA-CL leverages the capabilitics of LLMs without increasing the inference time of the recommendation model, making it practical for large-scale real-wor real-world applications. Through comprehensive experiments, we demonstrate that LLM-based semantic-guii -uided contrastive sample construction improves the contrastive learning, and we validate the effectiveness of the learnable sample synthesis mechanism. Furthermore, experiments with different recommendation model backbones confirm the model-agnostic nature of our approach.
7 Acknowledgements
This work was supported by the Early Career Scheme (No. CityU 21219323) and the General Research Fund (No. CityU 11220324) of the University Grants Committee (UGC), the NSFC Young Scientists Fund (No. 9240127), and the Donation for Research Projects (No. 9220187 and No. 9229164).
10

---

References
[1] Artun Boz, Wouter Zorgdrager, Zoe Kotti, Jesse Harte, Panos Louridas, Dietmar Jannach, and Marios Fragkoulis. Improving sequential recommendations with 1lms. arXiv preprint arXiv:2402.01339, 2024.
[2] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597-1607. PMLR, 2020.
[3] Yongjun Chen, Zhiwei Liu, Jia Li, Julian McAuley, and Caiming Xiong. Intent contrastive learning for sequential recommendation. Proceedings of the ACM Web Conference 2022, pages 2172-2182, 2022.
[4] Ziqiang Cui, Yixin Su, Fangquan Lin, Cheng Yang. Hanwei Zhang, and Jihai Zhang. Dual disentangled attention for mulli-information utilization in sequential recommendation. In 2022 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE, 2022.
[5] Ziqiang Cui, Haolun Wu, Bowei He, Ji Cheng, and Chen Ma. Context matters: Enhancing sequential recommendation with context-aware diffusion- contrastive learning. Proceed- ings of the 33rd ACM International Conference on Information and Knowledge Management, pages 414, 2024.
[6] Ziqiang Cui, Yunpeng Weng, Xing Tang, Fuyuan Lyu, Dugang Liu, Xiugiang He, and Chen Ma. Comprchending knowledge graphs with large language models for recommender systems. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1229-1239, 2025.
[7] Jacob Devlin, Ming-Wor Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
[8] Tianyu Gao, Xingcheng Yao, and Danqi Chen. SimCSE: Simple contrastive learning of sentence embeddings. In Empirical Methods in Natural Language Processing (EMNLP), 2021.
[9] Xiangman He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. Lightgen: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages 639-648, 2020.
[10] Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.000 2015.
[11] Yangqin Jiang, Yuhao Yang, Lianghao Xia, and Chao Huang. Diffkg: Knowledge graph diffusion model for recommendation. arXiv preprint arXiv:2312.66890 2023.
[12] Wang-Cheng Kang and Julian McAuley. Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM), pages 197-206. IEEE, 2018.
[13] Lei Li, Yongfeng Zhang, and Li Chen. Prompt distillation for efficient Ilm-based recommenda- tion. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 1348-1357, 2023.
[14] Lei Li, Yongfeng Zhang, Dugang Liu, and Li Chen. Large language models for generative recommendation: A survey and visionary discussions. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 10146-10159, 2024.
[15] Shiwei Li, Huifeng Guo, Hou, Wei Zhang, Xing Tang, Ruiming Tang, Rui Zhang, and Ruixuan Li. Adaptive low-precision training for embeddings in click-through rate prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 4435 4443, 2023.
11

---

[16] Shiwei Li, Zhuoqi Hu, Xing Tang, Haozhao Wang, Shijie Xu, Weihong Luo, Yuhua Li, Xiugiang He, and Ruixuan Li. Mixed-precisiie embeddings for large-scale recommendation models. arXiv preprini urXiv:2409.20305. 2024.
[17] Xuewei Li, Aitong Sun, Mankun Zhao, Jian Yu, Kun Zhu, Di Jin, Mei Yu, and Ruiguo Yu. Multi intintioteniion oriented contrastive learning for sequential recommendation. In Proceedings of the sixteenth ACM international conference on web search and data mining, pages 411-419, 2023.
[18] Yaoyiran Li, Xiang Zhai, Moustafa Alzantot, Keyi Yu, Ivan Vulić, Anna Korhonen, and Mohamed Hammad. Calrec: Contrastive alignment of generative Ilms for sequential recommen- dation. In Proceedings of the 18th ACM Conference on Recommender Systems, pages 422-432, 2024.
[19] Jiayi Liao, Sihang Li, Zhengyi Yang, Jiancan Wu, Yancheng Yuan, Xiang Wang, and Xiangnan He. Llara: Large language assistant. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1785- 1795, 2024.
[20] Qidong Liu, Xian Wu, Yejing Wang, Zijian Zhang, Feng Tian, Yefeng Zheng, and Xiangyu Zhao. Lim-esr: Large language models enhancement for long-tailed sequential recommendation. In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024.
[21] Zhiwei Liu, Yongjun Chen, Jia Li, Philip S Yu, Julian McAuley, and Caiming Xiong. Con- trastive sule-supe sequential recommendation with robust augmentation. arXiv preprint arXiv:2108.00 06479, 2021.
[22] Xiuyuan Qin, Huanhuan Yuan, Pengpeng Zhao, Junhua Fang. Fuzhen Zhuang, Guanfeng Liu, Yanchi Liu, and Victor Sheng. Meta-optimized contrastive learning for sequential recommen- dation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, page 89 99, 2023.
[23] Xiuyuan Qin, Huanhuan Yuan, Pengpeng Zhao, Guanfeng Liu, Fuzhen Zhuang, and Victor S Sheng. Intent contrastive learning with cross subsequences for sequential recommendation. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 548- 556,,55,222 2024.
[24] Ruihong Qiu, Zi Huang, Hongzhi Yin, and Zijian Wang. Contrastive learning for representation degeneration problem in sequential recommendation. Proceedings of the fifteenth ACM international conference web search and data mining, pages 813-823, 2022.
[25] Xubin Ren, Wei Wei, Lianghao Xia, Lixin Su, Suqi Cheng, Junfeng Wang, Dawei Yin, and Chao Huang. Representation learning with large language models for recommendation. In Proceedings of the ACM Web Conference 2024, pages 3464-3475, 2024.
[26] Yankun Ren, Zhongde Chen, Xinxing Yang. Longfei Li, Cong Jiang, Lei Cheng, Bo Zhang, Linjian Mo, and Jun Zhou. Enhancing sequential recommenders with augmented knowledge from aligned large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 345-354, 2024.
[27] Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In Pro- ceedings of the 28th ACM international conference on information and knowledge management, pages 1441-1450, 2019.
[28] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
[29] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint
12

---

[30] Yuling Wang, Changxin Tian, Binbin Hu, Yanhua Yu, Ziqi Liu, Zhiqiang Zhang, Jun Zhou, Liang Pang, and Xiao Wang. Can small language models be good reasoners for sequential recommendation? In Proceedings of the ACM on Web Conference 2024, pages 3876-3887, 2024.
[31] Jiancan Wu, Xiang Wang. Fuli Feng, Xiangnan He, Liang Chen. Jianxun Lian, and Xing Xie. Self-supervised graph learning for recommendation. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pages 726-735, 2021.
[32] Likang Wu, Zhi Zheng, Zhaopeng Qiu, Hao Wang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, et al. A survey on large language models for recommendation. World Wide Web, 27,5):60, 2024.
[33] Lianghao Xia, Chao Huang, Chunzhen Huang, Kangyi Lin, Tao Yu, and Ben Kao. Automated self-supervised learning for recommendation. In Proceedings of the ACM Web Conference 2023, pages 992-1002, 2023.
[34] Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Jiandong Zhang, Bolin Ding, and Bin Cui. Contrastive learning for sequential recommendation. In 2022 IEEE 38th international conference on data engineering (ICDE), pages 1259-1273. IEFF, 2022.
[35] Shenghao Yang, Weizhi Ma, Peijie Sun, Qingyao Ai, Yiqun Liu, Mingehen Cai, and Min Zhang. Sequential recommendation with latent relations based on large language model. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 335-344 2024.
[36] Yuhao Yang, Chao Huang, Lianghao Xia, Chunzhen Huang, Da Luo, and Kangyi Lin. Debiased contrastive learning for sequential recommendation. In Proceedings of the ACM Web Conference 2023, pages 1063-1073, 2023.
[37] Yihang Yin, Qingzhong Wang, Siyu Huang, Haoyi Xiong, and Xiang Zhang. Autogel: Auto- mated graph contrastive learning via learnable view generators. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 8892-8900, 2022.
[38] Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Lizhen Cui, and Quoc Viet Hung Nguyen. Are graph augmentations necessary? simple graph contrastive learning for recommendation. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval, pages 1294-1303, 2022.
[39] Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Jundong Li, and Zi Huang. Self-supervised learning for recommender systems: A survey. IEEF Transactions on Knowledge and Data Engineering, 36(1):335-355, 2023.
[40] Dan Zhang, Yangliao Geng, Wenwen Gong, Zhongang Qi, Zhiyu Chen, Xing Tang, Ying Shan, Yuxiao Dong, and Jie Tang. Reedel: Dual contrastive learning for recommendation. In Proceedings of the ACM on Web Conference 2024, pages 3655-360000 3356-3666,71 2024.
[41] Xiaokun Zhang, Bo Xu, Youlin Wu, Yuan Zhong, Hongfei Lin, and Fenglong Ma. Finerec: Exploring fine-grainee sequential recommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1599- 1608, 2024.
[42] Yuyue Zhao, Jiancan Wu, Xiang Wang, Wei Tang, Dingxian Wang, and Maarten de Rijke. Let do it for you: Towards 11m empowered recommendation via tool learning. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1796-18066 2024.
[43] Zihuai Zhao, Wenqi Fan, Jiatong Li, Yunqing Liu, Xiaowei Mei, Yiqi Wang, Zhen Wen, Fei Wang, Xiangyu Zhao, Jiliang Tang, al. Recommender systems in the era of large language models (Ilms), arXiv preprint arXiv 2023.
13

---

[44] Bowen Zheng, Yupeng Hou, Hongyu Lu, Yu Chen, Wayne Xin Zhao, Ming Chen, and Ji- Ji-Rong Wen. Adapting large language models by integrating collaborative semanties for recommen- dation. In 2024 IEEE 40th International Conference on Data Engineering (ICDE), pages 1448-1448. IEBE, 2024.
[45] Zhi Zheng, Wenshuo Chao, Zhaopeng Qiu, Hengshu Zhu, and Hui Xiong. Harnessing large language models for text-rich sequential recommendation. In Proceedings of the ACM Web Conference 2024, pages 3207-3216, 2024.
[46] Kun Zhou, Hui Wang. Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang. Zhongyuan Wang, and Ji-Rong Wen. S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM international conference on information & knowledge management, pages 1893-1902, 2020.
[47] Peilin Zhou, Jingqi Gao, Yueqi Xie, Qichen Ye, Yining Hua, Jaeboum Kim, Shoujin Wang. and Sunghun Kim. Equivariant contrastive learning for sequential recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems, pages 129-140, 2023.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction provide concise yet comprehensive overview of our core innovation, key methodological contributions, and the specific research problem addressed in this work.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the work in the appendix.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer:
[NA]
Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification:
This paper provides full disclosure of all information necessary to reproduce our key experimental results, and provides source code on an anonymized GitHub repository.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
16

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The four datasets used in this paper are all public, and we offer their links. We also provide our source code on the anonymized GitHub repository.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: We specify all training and test details, including data splits, hyperparameters, how they were chosen, type of optimizer, etc. necessary to understand the results in the experiment section and the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The experimental results in this paper are confirmed by paired t-test with significance level of 0.01.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provides sufficient information the computer resources needed reproduce the experiments.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't fidn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.c http:////eeaaapppp.ccpppbbb/heaa inn
Answer: [Yes]
Justification: The research conducted in the paper conforms, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discussed both potential positive societal impacts and negative societal impacts in the appendix.
Guidelines:
18

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negatíve societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: There no risk of misuse of the proposed method and the datasets used the paper are open-sourced.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All third-party assets are properly credited through citations and in-text acknowledgments.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
19

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
We communicate the details of the dataset/coddemmodel as part of our submission with an anonymized github URL..
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
20

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy hatt:p///earrrppccc/ooooeereeee/ /ct/////ooeeeeecc//22////.. for what should or should not be described.
21

---

A Technical Supplement to SRA-CL
A.1 Prompt Template
In this section, we provide detailed description of the LLM prompt templates employed in our study. Specifically, enhance the model's ability comprehend user preferences and items, have meticulously designed specialized prompts, as illustrated in Figure
Prompt Template
are a recommender system expert. A user's historical interactions are provided below in chronological order:
<product name brand I, category __,, description |>;
User
<product name 2, brand 2, category 2, deseription 2>;
<product_ saaicet_nmm__u__ n, brand n, category_t description_i
_
Please analyze both the textual information and sequential patterns, summarize the user preferences with no more than 200 words.
Assume you recommender system expert. Below is the information of specifie product.
The product name <name?; category <cate>; brand <brand>; description < description?, The historical sequences of users who have interacted with this product: [sequencel]; {sequence2];
Item
[sequence10).
Please describe the given item and potential audience according to its attributes context, Your response should not exceed 100 words.
Figure 6: Prompt Template.
A.2 Efficiency Analysis
During inference, only the recommendation backbone is utilized. The contrastive learning tasks and the LLMs' semantic embeddings are not involved in the inference process. This ensures that framework can be deployed in real-world applications without incurring any additional inference latency from incorporating LLMs.
Inference Efficiency.
Training Efficiency. The training process of method consists of two stages: In the first stage, we use an LLM API to obtain semantic information and convert it into embeddings, which are then cached construct contrastive sample indices. The primary time cost in this stage comes from the API calls. However, by employing asynchronous concurrency, this step can be completed within a few hours. Crucially, this stage is performed once and requires repetition during model training. In the second stage, we use the pre-constructed contrastive sample index to train the recommendation model. Regarding the computational complexity of this stage, method maintains comparable time complexity to general ID-based contrastive recommendation approaches. The only additional overhead during training compared conventional contrastive recommendation models comes from the lightweight learnable sample synthesis module whose parameter size is negligible compared to that of the main recommendation model.
B Experimental Setting Details
B.1 Datasets
We conducted experiments on four public real-world datasets: Yelp, Sports, Beauty, and Office. The statistics for these datasets presented in Table 3 These datasets cover diverse range of
22

---

Table 3: Dataset statistics.
Datasets
#Users
#Items
#Actions
Avg. Length
Density
Yelp
19,936
14,587
207,952
10.4
0.07%
Sports 35,598
18,357
296,337
8.3
0.05%
Beauty
22,363
12,101
198,502
8.8
0.07%
Office
4,905
2,420
53,258
10.9
0.45%
application scenarios. The Yelp datase is widely used for business recommendations The Sports, Beauty, and Office datasets sourced from Amazor one of the largest e-commerce platforms. Following previous studies [21 22], the users and items that have fewer than five interactions are removed.
B.2 Baseline Methods
To ensure comprehensive assessment, we compare our method with 12 baseline methods, categorized into three groups: classical methods (GRU4Rec, SASRec, BERT4Rec), contrastive learning- based methods --eRc CL4SRec, CoSeRec, ICLRec, DuoRec, MCLRec, ICSRec), and LLM-enhanced methods (LRD, LLM-ESR).
GRU4Rec 10
applies recurrent neural networks (RNN) to sequential recommendation.
SASRec
12
is the first work to utilize the self-attention mechanism for sequential recommenda-
tion.
BERT4Rec 27
employs the BERT framework to capture the context information of user
behaviors.
S Ree [66 leverages four self-supervised objectives uncover the inherent correlations within the data.
CL4SRec 34] proposes three random augmentation operators to generate positive samples for
contrastive learning.
CoSeRee (21] introduces two additional informative augmentation operators, building upon the foundation of CLASRec.
ICLRec 3 clusters user interests into distinct categories and brings the representations of users with similar interests closer together.
DuoRec [24] combines a model-level dropout augmentation and sampling strategy for choosing hard positive samples.
[ integrates CL4SRec's random data augmentation for the input sequence and employs MLP layers for model-level augmentation.
MCLRee
ICSRee is an improvement on ICLRec, further segmenting a user's 's sequential behaviors into multiple subsequences to generate finer-graines user intentions for contrastive learning.
LRD 35 is an LLM-based method. It leverages LLMs to discover new relations between items and reconstructs one item based on the relation and another item.
RLMRec 25 utilizes LLMs generate text profiles and combine their semantic embeddings with recommendation models.
LLM-ESR 20 also LLM-based method. addresses the long-tail problem by simultaneously leveraging collaborative signals and semantic information through the dual-view modeling and self-distillation.
B.3 Implementation Details
All experiments are conducted with a single V100 GPU. The embedding size for all methods is set for fair comparison. We use training batch size of 256 and employ the Adam optimizer
hhtps://www.yylp.comlll com/ataatttc/a bbttp://jmcaully.uss/ uesd
edw/data/ammazo// http://maaaaaa.....eeeelaaaaaaa
23

---

with a learning rate of 0.001. The dropout rate is set to 0.5 for both the embedding layer and the hidden layers across all datasets. Following previous studies [35, we set the maximum sequence length to 20 for all datasets. The early stopping is applied if the metrics on the validation set do not improve over consecutive epochs. Our method model-agnostic and can be applied to any sequential recommendation model. The transformer backbone mentioned in Sec. [2.2 comprises two layers, each with two attention heads. For the LLM, we select DeepSeek- V3, a robust large language model that demonstrates exceptional performance on both standard benchmarks and open- ended generation evaluations. For detailed information about DeepSeek, please refer to their official website Specifically, utilize DeepSeek-VV V3 by invoking its AP To reduce text randomness of the LLM, we set the temperature to 0 and the top-p to 0.001. For the text embedding model M, we the pre-trained that involve L.MSs and text embeddings to ensure fairness.
from Hugging Face. Identical settings used for baselines
C Additional Results & Analysis
C.1 Discussion on Learnable Sample Synthesis
Inter-User Contrastive Learning. User preferences exhibit significant heterogeneity across individ- uals. Sole reliance on hard rules, such as selecting a user from the current user's dedicated candidate pool as the positive sample, may yield suboptimal solutions. Our experiments (as shown in Table "w/o learn.") validated this. To enhance contrastive sample construction, we introduce a learnable sample synthesizer that optimizes the contrastive sample generation process during model training for inter-user contrastive learning.
Intra-User Contrastive Learning. Our preliminary experiments also explored the use of learnable synthesizers (analogous to inter-contrastive learning approaches) for generating substitute items, yet yielded measurable performance improvements (shown in Table 4 Our analysis suggests this results from the inherent nature of item semantics being more readily interpretable and quantifiable than user preferences. Therefore, directly identifying appropriate substitutes from semantically similar candidate pools is simpler and more reliable compared to matching users with analogous preference patterns.
Table 4: Performance impact of learnable versus non non-learablle sample synthesis strategies in intra- user contrastive learning.
Yelp
Sports
Beauty
Office
HR@20 NDCG@20
HR@20
NDCG@ 20
@2 NDCG@20
@2 NDCG@2 20
Learnable 0.1276
0.0531 0.0533
0.0825 0.0823
0.0344 0.0347
0.1309 0.1314
0.0561
0.1706 0.1702
0.0722 0.0725
Unlearnable 0.1282
0.0568
C.2
Additional Comparison Results
We provide additional comparison results (HR @ 10 and NDCG 10) of different methods Table 5 The experimental results demonstrate that our method outperforms all baselines across all datasets, further validating its superiority.
C,3 Additional Results for Hyperparameter Experiments
Due to space constraints, we only present HR 20 in Figure Of the main text for hyperparameter study. Here, we additionally report the NDCG@2 20 evaluation results in Figure 7 providing comple- mentary performance metrics for comprehensive analysis. As shown, the trend in NDCG@20 20 closely aligns with that HR @0..
"haps://apiidoossdeee wewpseseeeeeeeeeeeceeooc
24

---

Table 5: Additional comparison results for HR @ 10 and NDCG@ 10. Bold font indicates the best performance, while underlined values represent the second-bes best. "ND" represents for "NDCG". Our method SRA-CL achieves state-of-the-art results among all methods, as confirmed by paired t-test with significance level of 0.01.
Yelp
Sports
Beauty
Office
Model
HR@ 10
ND@ 10
HR @ 10
ND@ 10
HR @ 10
ND@ 10
HR@ 10
ND@@ 10
GRU4Rec
0.0362
0.0173
0.0193
0.0096
0.0279
0.0137
0.0540
0.0260
SASRec
0.0572
0.0308
0.0304
0.0157
0.0612
0.0336
0.0791
0.0348
BERT4Rec
0.0582
0.0311
0.0349
0.0189
0.0628
0.0352
0.0821
0.0376
Sd-Res
0.0612
0.0339
0.0385
0.0204
0.0647
0.0327
0.0931
0.0426
CL4SRec
0.0583
0.0315
0.0358
0.0189
0.0649
0.0329
0.0695
0.0322
CoSeRee
0.0607
0.0309
0.0439
0.0244
0.0725
0.0410
0.0782
0.0412
ICLRec
0.0598
0.0328
0.0428
0.0235
0.0713
0.0396
0.0922
0.0411
DuoRec
0.0747
0.0380
0.0474
0.0242
0.0841
0.0443
0.1015
0.0519
MCLRec
0.0721
0.0378
0.0498
0.0257
0.0870
0.0442
0.1036
0.0538
ICSRec
0.0738
0.0380
0.0487
0.0243
0.0844
0.0437
0.1034
0.0540
LRD
0.0693
0.0357
0.0376
0.0191
0.0620
0.0294
0.0887
0.0431
RLMRec
0.0709
0.0371
0.0426
0.0238
0.0764
0.0439
0.0927
0.0496
LLM-ESR
0.0669
0.0353
0.0415
0.0221
0.0750
0.0435
0.0889
0.0468
SRA-CL
0.0817
0.0419
0.0539
0.0274
0.0924
0.0469
0.1111
0.0575
Improvement
9.37%
10.26%
8.23%
6.61%
6.21 %
6.11%
7.24%
6.48%
Ye
Sports
Sports
Yelo
Sports
0.07
NDC0800
0.08
NDD68200
0.00
NDC64220
0.08
0.05
0.05
0.04
0.05
1012
50
5e-1
Figure 7: Hyperparameter experiments on the weight of Los (a), the weight of Lıs (9), and the number retrieved users/items (k) (NDCG results).
D
Other Discussions
D.1 Limitation
Considering computational budgets and resource limitations, we specifically analyzed how two selected LLMs (DeepSeek and Qwen) affect our framework's effectiveness. While more LLMs might yield different results, our study focused on these representative models.
D.2 Broader Impacts
SRA-CL demonstrates significant improvements in sequential recommendation accuracy (positive impact), with potential applicability to real-world platforms. Like all recommendation systems, its personalized nature may occasionally limit content diversity, though this effect is inherent to the recommendation paradigm rather than unique to our method.
25


Reviews:
[{"review_id": "nlxg69KYB7", "reviewer": "Reviewer_rXdN", "summary": "This paper proposes Semantic Retrieval Augmented Contrastive Learning (SRA-CL) to enhance the performance of classical sequential recommendation models. Specifically, SRA-CL explores inter-user contrastive learning as well intra-user contrastive learning paradigms, where positive and negative pairs are constructed according to semantic similarity derived by LLM. The proposed framework is model-agnostic and can be applied to a lot of sequential recommendation models using contrastive learning. Enhanced by SRA-CL, the performance of sequential recommendation models improves by ~10% compared with existing sequential recommendations models aided by contrastive learning.", "questions": "Please refer to the weakness I listed above.", "limitations": "Limitations are discussed. One other limitation could be applicability to large-scale data.", "rating": 2, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength:\n1. I think the question this paper tries to study is valid and the proposed framework seems like a reasonable approach to solve this problem (i.e., improving the quality of negative/positive pairs in contrastive learning for sequential recommendation). \n2. the paper is nicely written and easy to follow. \n\nWeakness:\n1. While the question is valid, as a practitioner in sequential recommendation I don't find it to be very interesting as the proposal doesn't really spur too much intellectual thinking. I feel like this paper is better positioned at an IR conference rather than NeurIPS that emphasizes a lot on understanding/learning. \n2. Efficiency and costs should be discussed -- I think it is trivial to leverage LLM to improve the performance of classical recommendation models. What is the trade-off for using LLMs to help recsys.\n3. Following up on 2, there are a lot of new methods under the umbrella of generative recommendation, another totally different horizontal of using LLMs for sequential recommendation. How does the proposal compare against those (e.g., performance-wise, efficiency-wise, cost-wise, deployment difficulty-wise, etc).", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer's valuable time and insightful comments. Below are our point-by-point responses to the questions raised. We look forward to any additional feedback the reviewer may provide.\n\n# Response to W1: About the Contribution and Positioning\n### **1) The Contribution and Significance**: \n- Our work provides an in-depth analysis and novel exploration of contrastive learning in recommender systems. Unlike CV or NLP, contrastive learning in recommender systems presents unique challenges that render direct adoption of methods from other domains ineffective.\n- Furthermore, while existing contrastive learning approaches in sequential recommendation have predominantly relied on ID-based representations, we critically examine their limitations and, for the first time, propose leveraging semantic information to construct higher-quality learnable contrastive pairs, which is achieved by harnessing the understanding capabilities of LLMs.\n- We believe our work offers a significant contribution by redefining the understanding and application of contrastive learning in recommender systems. Our approach represents a meaningful and successful innovation, as mentioned by Reviewer 6Nm1: \"The innovative use of LLMs to uncover semantically rich representations offers a promising new direction for further research and real-world application.\"\n\n### **2) On the Positioning**:\n- Our paper aligns with NeurIPS's scope. Regarding the comment of “*this paper is better positioned at an IR conference rather than NeurIPS*”, we respectfully clarify that while specialized domains have dedicated venues (e.g., CVPR for computer vision, ACL for NLP, SIGIR for IR), NeurIPS serves as a premier interdisciplinary platform that explicitly welcomes cross-domain contributions—including research bridging information retrieval and machine learning. \n- Critically, our work focuses on user/item **understanding** and contrastive **learning**—precisely the areas highlighted by the reviewer as \"***NeurIPS's emphasis on understanding/learning.***\" \n- Moreover, the acceptance of **21 papers** at **NeurIPS 2024** featuring the terms \"**recommendation**\" or \"**recommender**\" in their titles further validates the suitability of our work for this venue. Representative examples include:\n  - Q. Liu et al. \"LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation\"  \n  - Y. Liu et al. \"End-to-End Learnable Clustering for Intent Learning in Recommendation\"  \n\n# Response to W2: About the Efficiency\nWe appreciate the reviewer’s attention to efficiency. Actually, we have clarified the inference efficiency in Section 3.3 (Lines 220–224 of Page 6).\n- **Inference Efficiency**: During inference, only the lightweight backbone recommendation model is executed, while contrastive learning tasks and semantic embeddings are entirely disabled. This design ensures no additional computational overhead from LLM integration, preserving the same inference latency as the original recommendation backbone. Thus, our approach remains deployment-friendly and scalable for real-world systems, as it introduces no runtime cost to the production pipeline.\n- **Training Efficiency & Cost**: The training process of our method consists of two stages:\nIn the first stage, we use an LLM API to obtain semantic information and convert it into embeddings, which are then cached to construct contrastive sample indices. The primary time cost in this stage comes from the API calls. However, by employing asynchronous concurrency, this step can be completed within a few hours. Crucially, this stage is performed once and requires no repetition during model training. Regarding the monetary cost of API usage, if using the DeepSeek API, all the experiments in this work can be covered within a budget of $50. Alternatively, if using an open-source model like Qwen-32B, this cost is reduced to zero, as it can be run locally without incurring API fees.\nIn the second stage, we use the pre-constructed contrastive sample index to train the recommendation model. Regarding the computational complexity of this stage, our method maintains comparable time complexity to general ID-based contrastive recommendation approaches. The only additional overhead during training compared to conventional contrastive recommendation models comes from the learnable sample synthesis module—an adapter and a single attention layer—whose parameter size is negligible compared to that of the main recommendation model. Specifically, each experiment was conducted on a single 32GB V100 GPU and completed within 3 hours of training.\n\n# Response to W3: Discussion of LLM-based Methods\nOur paper has provided a comprehensive comparison and discussion of state-of-the-art LLM-based sequential recommendation methods. This is demonstrated through: \n1) **Experiments**: Our experimental design includes three SOTA LLM-based sequential recommendation methods as baselines, all of which were recently published in top-tier conferences such as NeurIPS and SIGIR. Additionally, we have compared against seven contrastive learning approaches and three classical recommendation methods, resulting in a total of 13 baseline methods. We believe this constitutes a thorough comparative studies in this domain as the primary objective of our work is to enhance existing contrastive learning methods in sequential recommendation using LLMs. \n2) **Related Work**: In the related work section (Section 5), we have systematically reviewed the latest advancements in the topic of LLMs for sequential recommendation and clearly delineated the distinctions between our work and these existing approaches. \n\nBelow is the summarization of the comparison in terms of performance, efficiency, cost, and deployment difficulty:\n- **Performance**: Our approach demonstrates consistently better recommendation accuracy across multiple benchmarks, validating its effectiveness.\n- **Computational Efficiency and Cost**: 1) Inference Efficiency: Our method eliminates the need for LLM involvement or semantic processing during inference, making it as efficient as traditional recommendation models. 2) Training Cost: While leveraging semantic information, our framework maintains comparable training costs to other LLM-based methods.\n- **Scalability & Deployability**: Due to its lightweight inference mechanism, our approach is highly scalable and readily applicable to real-world large-scale recommendation systems.\n\n\n# Response to Reviewer's Comment Regarding Large-scale Data Applicability\nOur method is **deployment-friendly** and **scalable for real-world applications**, as it introduces no additional inference latency from incorporating LLMs. As discussed in Line 220-224 of Page 6, during inference, only the recommendation backbone is utilized. The contrastive learning tasks and LLMs’ semantic embeddings are not involved in the inference process. This design ensures no additional computational overhead from LLM integration, preserving the same inference latency as the original recommendation backbone."}, {"review_id": "L0JyX9rKDr", "reviewer": "Reviewer_6Nm1", "summary": "This paper proposes Semantic Retrieval Augmented Contrastive Learning (SRA-CL), which aims to address the challenge of data sparsity in recommendation systems. By leveraging large language models (LLMs) to generate semantic embeddings of user preferences and items, the authors enhance the reliability of contrastive learning samples, thereby improving the accuracy and robustness of the recommendation outcomes. A key innovation of the proposed approach lies in the utilization of LLMs as semantic retrieval engines to augment the contrastive learning process. Furthermore, experiments on public datasets against baselines show significant improvements in HR@20 and NDCG@20. Ablation studies and hyperparameter analyses further validate the effectiveness and LLM-agnostic nature of each module.", "questions": "See Weaknesses", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThis paper addresses a critical data sparsity challenge in recommender systems by identifying higher-quality positive behavior sequences for contrastive learning, thereby strengthening sequence modeling capabilities. The technical exposition is thorough and logically structured, and the authors back up their claims with extensive experiments and in-depth analyses. Moreover, the innovative use of LLMs to uncover semantically rich representations offers a promising new direction for further research and real-world application.\n\nWeaknesses:\n\n1. Both Qwen2.5-32B-Instruct and DeepSeek-V3 backbones yield similar HR and NDCG scores. Have the authors explored smaller versions of LLMs (e.g., 3B or 0.5B parameters)? Smaller models could significantly reduce inference latency and cost, making the approach more practical at scale.\n2. Would directly encoding user semantics and collaborative signals using a pre-trained SimCSE-RoBERTa model produce comparable results to those of LLM-generated text outputs? A comparison could help clarify the added value of incorporating the LLM step.\n3. Has the method been evaluated in scenarios where item or user text metadata is sparse or even completely omitted during LLM encoding?\n4. The conclusion is relatively brief. It would benefit from a more detailed summary of the findings and a dedicated discussion of potential future work.", "was_revised": true, "final_justification": "During the rebuttal, the authors provided further clarification on the LLM model selection,  application scenarios, and conclusion section, which partially addressed my concerns. Since these points do not significantly affect the main quality of the paper, I will maintain my original rating.", "rebuttal": "We sincerely appreciate the reviewer's valuable time and insightful comments. Below are our point-by-point responses to the questions raised. We look forward to any additional feedback the reviewer may provide.\n\n\n# Response to Weakness 1\nWe sincerely appreciate the reviewer’s valuable suggestion. Below, we answer this question in two aspects:\n- Our method does not rely on LLM inference during the prediction phase; instead, it only involves the inference of the backbone recommendation model. Therefore, our framework can be deployed in real-world large-scale applications without introducing additional latency from LLM integration. This also explains why our initial experiments did not explore smaller LLMs.\n- We fully acknowledge the reviewer’s insightful suggestion regarding smaller LLMs, as they could reduce training time and computational costs. Accordingly, we have conducted additional experiments using Llama3.2-3B-Instruct, and the results are presented below.\n\n|             |   Yelp  |         | Sports |         | Beauty |         | Office |         |\n|-------------|:-------:|:-------:|:------:|:-------:|:------:|:-------:|:------:|:-------:|\n|             | HR@20   | NDCG@20 | HR@20  | NDCG@20 | HR@20  | NDCG@20 | HR@20  | NDCG@20 |\n| DeepSeek-V3 | 0.1282  | 0.0533  | 0.0823 | 0.0347  | 0.1314 | 0.0568  | 0.1702 | 0.0725  |\n| Qwen2.5-32B | 0.1275  | 0.0526  | 0.0815 | 0.0338  | 0.1306 | 0.0559  | 0.1701 | 0.0720  |\n| Llama3.2-3B | 0.1258  | 0.0510  | 0.0801 | 0.0322  | 0.1288 | 0.0545  | 0.1680 | 0.0708  |\n\nThe experimental results indicate that the 3B LLM underperforms compared to larger models. We attribute this to the fact that reasoning and summarizing user preferences require advanced inferential and abstraction capabilities, which smaller models tend to exhibit to a lesser degree than their larger counterparts. Consequently, the quality of generated user preference descriptions may be comparatively lower. In contrast, Qwen-32B, despite having fewer parameters than DeepSeek-V3, still maintains a sufficiently large scale to preserve most of these capabilities, thus showing less significant performance degradation. These observations suggest a potential trade-off between model scale and performance - smaller models may offer improved training efficiency, but correlate with diminished effectiveness in downstream tasks. We will include the experimental results and analysis in the appendix of the final version.\n\n\n# Response to Weakness 2\n- In fact, we had already conducted the experiment mentioned by the reviewer, and the results are presented in Table 2 of the Ablation Study section in the original paper. The \"w/o LLM\" setting denotes the scenario where the LLM is not used for text comprehension and processing—that is, the SimCSE-RoBERTa model directly encodes user and item texts, as the reviewer proposed. \n- The results in Table 2 indicates directly applying the text embedding model without LLM-based processing leads to performance degradation. This confirms that leveraging the LLM’s ability to understand and reason about user preferences is crucial. Without secondary processing by the LLM, the textual data from user historical behaviors becomes excessively verbose, with key information obscured by irrelevant details. Such lengthy and noisy text poses significant challenges for the embedding model, resulting in lower-quality embeddings. The LLM’s reasoning and summarization capabilities effectively mitigate this issue.\n\n\n# Response to Weakness 3\nWe sincerely thank the reviewer for their insightful question and suggestion. The primary application scenario of this study focuses on cases where textual attributes of items are available, consistent with prior work [1, 2]. The metadata in the datasets used in our experiments exhibits a certain degree of sparsity, typically below 20%. We noticed that most real-world applications, such as Amazon, Taobao, Netflix and YouTube, generally provide and maintain high-quality textual metadata. For scenarios where text metadata is extremely sparse or entirely unavailable, such cases pose significant challenges for all LLM-based recommendation methods. However, addressing these challenges is beyond the scope of this paper due to space and time constraints. We plan to explore this direction further in future work. Once again, we deeply appreciate the reviewer's valuable suggestion.\n\n[1] Ren, Xubin, et al. Representation learning with large language models for recommendation. WWW' 24\n\n[2] Liu, Qidong, et al. Llm-esr: Large language models enhancement for long-tailed sequential recommendation. NeurIPS' 24\n\n\n# Response to Weakness 4\nWe sincerely thank the reviewer for their suggestion regarding the Conclusion section. Your feedback will help improve our paper further. In the final version, we will expand the Conclusion and include a discussion on future work. The revised Conclusion is as follows: \n\n\"*In this paper, we analyze the limitations of contrastive learning in sequential recommendation, namely Semantic Divergence and Unlearnability. To address these issues, we propose SRA-CL, a novel framework that enhances contrastive sample construction by integrating LLM-based semantic retrieval with a learnable sample synthesizer. SRA-CL leverages the capabilities of LLMs without increasing the inference time of the recommendation model, making it practical for large-scale real-world applications. Through comprehensive experiments, we demonstrate that LLM-based semantic-guided contrastive sample construction improves the contrastive learning, and we validate the effectiveness of the learnable sample synthesis mechanism. Furthermore, experiments with different recommendation model backbones confirm the model-agnostic nature of our approach. Looking forward, there are potential directions for future work: (1) optimizing the computational overhead of LLMs during the training process, and (2) exploring how to adapt our method to scenarios with scarce textual information.*\""}, {"review_id": "IqLqNR76nH", "reviewer": "Reviewer_CCcq", "summary": "This paper identifies two limitations in existing contrastive learning frameworks—semantic divergence and unlearnability, and proposes SRA-CL to address them. The proposed method harnesses the semantic reasoning capabilities of LLMs to mitigate semantic divergence while introducing a learnable sample synthesizer that dynamically optimizes contrastive sample generation during training. Extensive experiments validate SRA-CL’s effectiveness and demonstrate its robust generalization across diverse recommendation models. Overall, this paper presents a well-motivated approach supported by comprehensive experimental validation.", "questions": "•\tCould the authors explain why the Learnable Sample Synthesis is only applied to user-level contrastive learning but not extended to the item-level?\n•\tCould the authors analyze the training/inference efficiency of the proposed method, as well as the cost for API calls? \n•\tWhy is cosine similarity preferred over dot product for semantic retrieval?", "limitations": "NA", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n•\tSemantic divergence is a key challenge in contrastive learning for sequential recommendation. This work effectively addresses this issue by utilizing LLMs' semantic understanding and reasoning capabilities to retrieve semantic-consistent users/items, which is a well-motivated solution.\n•\tThis paper proposes a learnable sample synthesizer to optimize contrastive sample generation during recommendation model training. This trainable approach helps ensure that the synthesized samples effectively enhance end-task performance.\n•\tThis paper includes thorough experiments comparing different types of baselines and provides detailed ablation analysis. In addition, the generalization tests across various recommendation models and LLMs demonstrate the method's model-agnostic characteristic.\nWeaknesses:\n•\tThe paper does not discuss the training efficiency of the proposed method, despite its high inference efficiency. In addition, it doesn't provide cost analysis for API calls. \n•\tThe Learnable Sample Synthesis proposed in this paper is only applied to user-level contrastive learning and not utilized at the item-level, with no specific explanation provided.", "was_revised": true, "final_justification": "I have read the author's responses to my concerns in their rebuttal. Thank you for the efforts of the authors and the AC, which have helped me further reflect on the score evaluation.", "rebuttal": "We sincerely appreciate the reviewer's valuable time and insightful comments. Below are our point-by-point responses to the questions raised. We look forward to any additional feedback the reviewer may provide.\n\n\n# Response to W1 & Q2 \nWe sincerely thank the reviewer for recognizing the high inference efficiency of our method. In the main text, we have discussed inference efficiency, and we have included the training and inference algorithm workflows in Algorithm 1 and Algorithm 2 in the appendix. Here, we provide a detailed explanation of the inference and training costs.\n- **Inference Efficiency**: During inference, only the recommendation backbone is utilized. The contrastive learning tasks and the LLMs’ semantic embeddings are not involved in the inference process. This ensures that our framework can be deployed in real-world applications without incurring any additional inference latency from incorporating LLMs.\n- **Training Efficiency & Cost**: The training process of our method consists of two stages:\nIn the first stage, we use an LLM API to obtain semantic information and convert it into embeddings, which are then cached to construct contrastive sample indices. The primary time cost in this stage comes from the API calls. However, by employing asynchronous concurrency, this step can be completed within a few hours. Crucially, this stage is performed once and requires no repetition during model training. Regarding the monetary cost of API usage, if using the DeepSeek API, all the experiments in this work can be covered within a budget of $50. Alternatively, if using an open-source model like Qwen-32B, this cost is reduced to zero, as it can be run locally without incurring API fees.\nIn the second stage, we use the pre-constructed contrastive sample index to train the recommendation model. Regarding the computational complexity of this stage, our method maintains comparable time complexity to general ID-based contrastive recommendation approaches. The only additional overhead during training compared to conventional contrastive recommendation models comes from the learnable sample synthesis module—a single attention layer—whose parameter size is negligible compared to that of the main recommendation model.  \n\n\n# Response to W2 & Q1\n- We sincerely apologize for any confusion caused. In fact, we have already addressed this question in the original paper. Specifically, Lines 202–208 on Page 6 clarify why the Learnable Sample Synthesis is applied only to user-level contrastive learning and not extended to the item level. Due to space constraints, we provided further elaboration in Appendix C.1, along with experimental comparisons in Table 4 of the appendix.\n- To reiterate, our preliminary experiments explored employing learnable synthesizers at the item level (analogous to inter-user contrastive learning) to generate substitute items. However, this approach yielded no measurable performance improvements (see Table 4, Appendix). We attribute this outcome to the inherent nature of item semantics, which are more straightforward to interpret and quantify than user preferences. Consequently, directly retrieving substitutes from semantically similar candidate pools proves both simpler and more reliable than attempting to match users with analogous preference patterns.  \n\n\n# Response to Q3\nWe appreciate the insightful question regarding our choice of cosine similarity for semantic retrieval. Our decision was based on the following considerations:\n- **Normalization robustness**: Unlike the dot product, cosine similarity inherently normalizes similarity scores by accounting for vector magnitudes. This property makes it more robust to scale inconsistencies in semantic embeddings, particularly since we do not apply explicit normalization to our embeddings.\n- **Alignment with the text embedding model’s pretraining objective**: The text embedding model we employed, SimCSE-RoBERTa, was explicitly trained using cosine similarity as its semantic similarity metric. By adopting the same measure for retrieval, we ensure consistency with the model’s original training objective, thereby enhancing the reliability of user similarity computations."}, {"review_id": "VIaairYfyy", "reviewer": "Reviewer_UspK", "summary": "This paper leverages the semantic understanding capabilities of LLMs to enhance contrastive learning in sequential recommendation systems. It proposes a semantic similarity-based contrastive pair construction framework called SRA-CL. The core idea is to utilize LLM-generated semantic embeddings to construct positive contrastive pairs through semantic-based retrieval, followed by a learnable sample synthesizer that optimizes the contrastive sample generation process during model training. \n\nThe motivation behind using semantic information instead of random perturbations or sparse collaborative signals is to address the semantic divergence problem and the learnable sample synthesis component aims to overcome the unlearnability issue by enabling models to autonomously learn optimal contrastive pairs rather than relying on predefined heuristic rules. SRA-CL adopts a plug-and-play design that can seamlessly integrate with existing sequential recommendation architectures, demonstrating both effectiveness and model-agnostic nature across multiple datasets.", "questions": "Q1: Experimental Validation of the Learnable Sample Synthesizer's Effectiveness\n\nAuthors propose a Learnable Contrastive Sample module to address user preference heterogeneity, but the experiments lack direct evidence that current positive pair construction methods suffer from this problem. Please provide:\n- Visualization of selection probabilities for positive samples generated by the learnable synthesizer\n- Comparative analysis showing how the synthesizer's selections differ from baseline methods\n\nQ2:Comparison with State-of-the-Art LLM-Based Embedding Methods.\n\n\n\nif authors could partially solve my concerns during the rebuttal period, I am pleased to raise my score.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "S1: Problem Scope\nThis paper aims to the construction of high-quality contrastive pairs for self-supervised learning by leveraging LLM semantic understanding.\n\nS2: Experimental Validation\nThe authors conduct experiments across four public datasets to validate both the effectiveness and model-agnostic nature of the approach. The experimental design includes ablation studies to confirm the efficacy of individual components, providing evidence for the contribution of each proposed module to the overall performance improvement.\n\nW1: Insufficient Justification of Motivation.\n\nThe authors claim that existing methods for constructing contrastive pairs may lead to a complete change in sequence semantics (i.e., user preferences). However, the approach of selecting positive pairs based on LLM preference summarization and semantic embedding retrieval still exhibits a significant gap between LLM-generated preferences and actual user preferences [1,2]. The proposed method does not adequately address this fundamental motivation. This limitation is further evidenced in Figure 4, where both small and large numbers of retrieved semantically-related users/items as contrastive learning samples result in suboptimal experimental results.\n\n[1] Ren, Xubin, et al. \"Representation learning with large language models for recommendation.\" Proceedings of the ACM Web Conference 2024. 2024.\n[2] Sheng, Leheng, et al. \"Language Representations Can be What Recommenders Need: Findings and Potentials.\" arXiv preprint arXiv:2407.05441 (2024).\n\n\nW2: Lack of Experimental Validation for Proposed Method's Motivation.\n\nThe authors propose a Learnable Contrastive Sample module to address the issue that \"User preferences exhibit significant heterogeneity across individuals.\" However, the experiments fail to demonstrate that current positive pair construction methods actually suffer from this problem. The authors should provide visualization of the selection probabilities for positive samples generated by this module to further substantiate their claims.\n\n\nW3: Experimental Design Fails to Incorporate Recent Advances\nThe paper's embedding retrieval still relies on early small-scale embedding models. Recent research has demonstrated that LLM-based embeddings, such as LLaMA [4], can significantly enhance model performance, as evidenced in LLM-ESR [3] and AlphaRec [2]. The authors are recommended to supplement their work with relevant experiments using state-of-the-art embedding approaches.\n\n[3] Liu, Qidong, et al. \"Llm-esr: Large language models enhancement for long-tailed sequential recommendation.\" Advances in Neural Information Processing Systems 37 (2024): 26701-26727.\n[4] Touvron, Hugo, et al. \"Llama: Open and efficient foundation language models.\" arXiv preprint arXiv:2302.13971 (2023).", "was_revised": true, "final_justification": "I appreciate the authors' efforts during the rebuttal period. The newly added experiments better support the paper’s motivation, and the inclusion of results with different embedding models further demonstrates the effectiveness of SRA-CL. These clarifications have resolved most of my earlier concerns. Accordingly, I am raising my score to borderline accept.", "rebuttal": "We sincerely appreciate the reviewer's valuable time and insightful comments. Below are our point-by-point responses to the questions raised. We look forward to any additional feedback the reviewer may provide.\n\n\n# Response to W1: Justification of Motivation\nWe sincerely apologize for any misunderstanding, and we appreciate this opportunity to clarify our motivation. \n- As presented in Page 2, lines 41–44, we posit: \"*Many existing methods construct contrastive pairs through random augmentation operations such as random masking and Dropout. However, in sequential recommendation where data is inherently sparse and exhibits sequential patterns, such random operations may lead to a complete change in the sequence’s semantics*.\" Specifically, our critique focuses on random data augmentation methods, wherein contrastive samples are constructed in an entirely arbitrary fashion, frequently compromising the semantic of the original sequence. In contrast, our method is based on LLM's semantic comprehension. It stands to reason that our approach would yield more reliable results. This claim has been empirically validated through both ablation studies (Table 2  \"w/o semantic\") and comparative experiments (Table 1).\n\n- To further mitigate the potential gap between LLM-summarized user preferences and genuine user interests, we avoid naively adopting LLM-identified \"most relevant\" users as positive contrastive samples. Instead, we propose a Learnable Sample Synthesis approach, where the LLM only performs initial screening to generate a candidate set of potential positive samples, and then a trainable synthesis mechanism is used to dynamically generate informative samples from the candidate set, which is optimized end-to-end based on the final recommendation performance. This ensures the synthesized positive samples enhance recommendation metrics.\n\n- Explanation of **Figure 4**. The figure illustrates the impact of k on model performance. When k=1, the method simply adopts the top-1 most relevant sample selected by the LLM as the contrastive sample, bypassing our learnable synthesis module. The suboptimal performance at this setting indicates a potential gap between the LLM's inferred user preferences and actual user preferences, underscoring that leveraging LLMs to guide contrastive sample construction is indeed nontrivial. As k increases (e.g., to k=10), the LLM performs preliminary screening to select k potentially relevant candidates for the sample synthesis module. Through end-to-end learning optimization, our module synthesizes an optimal contrastive sample from these candidates. The performance improvement indicates that the learnable synthesizer effectively mitigates the semantic gap between LLM outputs and real user preferences, thereby generating superior contrastive samples to improve the performance. Excessively large k values (k=100) relax the LLM's semantic similarity threshold, introducing substantial noise, which compromises the learning process and degrades performance. Overall, this figure highlights our key contribution: developing an effective learnable approach to bridge the semantic gap between LLM outputs and genuine user preferences.\n\n\n# Response to W2 & Q1:  Experimental Validation of the Learnable Sample Synthesizer\n## 1) Clarification of Motivations\n- The statement, \"User preferences exhibit significant heterogeneity across individuals,\" highlights the diversity of user interests. The primary goal of recommendation models is to effectively model this diverse preferences. However, existing methods construct contrastive pairs through random augmentation or predefined hard rules, which significantly alter the user preferences, potentially leading to substantial differences in preferences between two users treated as positive samples. Pulling users with differing interests closer in the latent space leads to representation homogenization, which contradicts the core objective of recommendation.\n\n- Our method leverages LLM-based semantic information to perform an initial screening and generate a candidate set that aligns with each user's preference-related semantics, which helps alleviate the aforementioned issue. However, directly using the top-1 sample as the positive sample for contrastive learning may lead to a potential problem: “There may exist a gap between the LLM’s interpretation of preferences and the user’s actual preferences.” To address this issue, we propose the Learnable Contrastive Sample Synthesis module. \n\n## 2) Experimental Validation\nIn the ablation study section of our original paper (Table 2), we reported the experimental results for the \"w/o learn.\" setting. The observed performance degradation validates the effectiveness of the proposed Learnable Contrastive Sample Synthesis module. \n\nWe greatly appreciate the reviewer’s suggestion. In response, we conducted two additional experiments to validate the motivation, which will be added to our final version.\n\n- To validate whether there is a gap between the LLM's understanding of user preferences and the actual user preferences, we randomly selected 100 users and, for each user, observed and compared the semantic similarity score distribution of the users in the candidate set (as derived from LLM-based relevance) with the weight distribution learned by the proposed learnable module for these users. The results show that the ranking results differ in 99% of the cases. Due to space limitations, we provide an example of the selection probabilities comparison as follows:\n\n|                                           | u1    | u2    | u3    | u4    | u5    | u6    | u7    | u8    | u9    | u10   |\n|-------------------------------------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|\n| similarity scores wihout learn.           | 0.882 | 0.856 | 0.854 | 0.819 | 0.798 | 0.785 | 0.763 | 0.744 | 0.740 | 0.719 |\n| attention scores with learn. (normalized) | 0.051 | 0.099 | 0.198 | 0.152 | 0.067 | 0.101 | 0.120 | 0.072 | 0.061 | 0.079 |\n\nFrom the results, we can observe that there are clear differences between the two distributions, which validates the rationale behind employing the proposed learnable module. Our learnable module leverages a non-linear adapter and an attention mechanism to automatically assign weights to the samples in the LLM-filtered candidate set, which makes that the selection probabilities of candidate samples are no longer solely based on the original semantic embeddings but are instead learned and optimized end-to-end according to the objective of the recommendation model. \n\n- To validate the impact of the learnable module on representation heterogenization, we visualized the user sequence representations learned by our method with and without the learnable module using t-SNE. Due to rebuttal guidelines prohibiting the inclusion of images or links, we provide a brief summary of the results here. The embeddings with the learnable module are more dispersed, indicating greater heterogeneity in the representations, while the embeddings without the module are relatively compact. The results suggest that the learnable module effectively mitigates the risk of user representation homogenization or even collapse, preserving the heterogeneous diversity of user interests.\n\n\n# Response to W3 & Q2: SOTA LLM-based Text Embedding Models\n\nWe sincerely appreciate the reviewer's suggestions regarding the text embedding model. We selected sup-simcse-roberta-large for two reasons: 1) It is a widely adopted model for extracting sentence-level semantic embeddings, and 2) Since this component is not our primary innovation focus, using a general embedding model better demonstrates that our method's performance does not rely on advanced text embedding models.\n\nWe fully agree that comparing LLM-based text embedding models would yield valuable insights. In response, we have extended our evaluation to incorporate two additional SOTA embedding models: 1) OpenAI’s **text-ada-embedding-002**, and 2) **Qwen3-Embedding-8B**. The results are shown as the following table:\n\n|                                        |  Yelp  |         | Sports |         | Beauty |         | Office |         |\n|----------------------------------------|:------:|:-------:|:------:|:-------:|:------:|:-------:|:------:|:-------:|\n|  | HR@20  | NDCG@20 | HR@20  | NDCG@20 | HR@20  | NDCG@20 | HR@20  | NDCG@20 |\n| simcse-roberta-large | 0.1282 | 0.0533  | 0.0823 | 0.0347  | 0.1314 | 0.0568  | 0.1702 | 0.0725  |\n| OpenAI/text-ada-embedding-002          | 0.1296 | 0.0540  | 0.0835 | 0.0354  | 0.1331 | 0.0576  | 0.1718 | 0.0730  |\n| Qwen/Qwen3-Embedding-8B                | 0.1327 | 0.0556  | 0.0858 | 0.0375  | 0.1362 | 0.0593  | 0.1746 | 0.0748  |\n\n\nOur results show that advanced text embedding models can boost the performance of our method: 1) text-ada-embedding-002 (1B params, 1536D) outperforms SimCSE-RoBERTa-large (335M params, 1024D) across all datasets, thanks to its greater capacity for semantic representation. 2) Qwen3-Embedding-8B (8B params, 4096D) achieves SOTA results, surpassing both models in preference modeling accuracy.\n\nFurthermore, to demonstrate that our method consistently outperforms other LLM-based approaches across different text embedding models, we compared our method with the strongest LLM baseline, RLMRec, under the best-performing Qwen3-Embedding-8B. \n\n|                                    |  Yelp  |         | Sports |         | Beauty |         | Office |         |\n|------------------------------------|:------:|:-------:|:------:|:-------:|:------:|:-------:|:------:|:-------:|\n|                                    | HR@20  | NDCG@20 | HR@20  | NDCG@20 | HR@20  | NDCG@20 | HR@20  | NDCG@20 |\n| RLMRec with Qwen3-Embedding-8B     | 0.1146 | 0.0492  | 0.0690 | 0.0319  | 0.1244 | 0.0545  | 0.1571 | 0.0652  |\n| Our method with Qwen3-Embedding-8B | 0.1327 | 0.0556  | 0.0858 | 0.0375  | 0.1362 | 0.0593  | 0.1746 | 0.0748  |"}]

Author rebuttal:

Meta-review: The paper presents an LLM-based approach to improve contrastive learning for sequential recommender systems. The proposed framework utilizes LLM as a semantic retrieval engine to augment the contrastive learning process and achieves better performance in practice. Ablation studies are also sufficient. 

Most of the concerns raised by the reviewers are well addressed in the rebuttal, including efficiency and results with smaller LLMs. The only remaining concern is about the venue fit (raised by Reviewer rXdN). However, the AC thinks that developing an improved algorithm for sequential recommendation is a good fit for NeurIPS, so we recommend acceptance for this paper.
Decision: Accept (poster)