LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation
Authors: Qidong Liu
Xian Wu
Yejing Wang
Zijian Zhang
Feng Tian
Yefeng Zheng
Xiangyu Zhao
Venue: NeurIPS 2024 spotlight (2024)
Keywords: Large Language Models
Recommender Systems
Sequential Recommendation
Long-tail
Primary area: machine_learning_for_other_sciences_and_fields

Abstract: Sequential recommender systems (SRS) aim to predict users' subsequent choices based on their historical interactions and have found applications in diverse fields such as e-commerce and social media. However, in real-world systems, most users interact with only a handful of items, while the majority of items are seldom consumed. These two issues, known as the long-tail user and long-tail item challenges, often pose difficulties for existing SRS. These challenges can adversely affect user experience and seller benefits, making them crucial to address. Though a few works have addressed the challenges, they still struggle with the seesaw or noisy issues due to the intrinsic scarcity of interactions. The advancements in large language models (LLMs) present a promising solution to these problems from a semantic perspective. As one of the pioneers in this field, we propose the Large Language Models Enhancement framework for Sequential Recommendation (LLM-ESR). This framework utilizes semantic embeddings derived from LLMs to enhance SRS without adding extra inference load. To address the long-tail item challenge, we design a dual-view modeling framework that combines semantics from LLMs and collaborative signals from conventional SRS. For the long-tail user challenge, we propose a retrieval augmented self-distillation method to enhance user preference representation using more informative interactions from similar users. To verify the effectiveness and versatility of our proposed enhancement framework, we conduct extensive experiments on three real-world datasets using three popular SRS models. The results consistently show that our method surpasses existing baselines. The implementation code is available in Supplementary Material.

LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation
Qidong Liu!? Xian Wu'; Yejing Wang', Zijian Zhang*+,
Feng Tian Yefeng Zheng*: Xiangyu Zhao²
School of Auto. Science & Engincering. MOEKLINNS Lab, Xi' an Jiaotong University City University of Hong Kong
Jarvis Research Center, Tencent YouTu Lab, Jilin University
School of Comp. Science & Technology, MOEKLINNS Lab, Xi'an Jiaotong University Medical Artificial Intelligence Lab, Westlake University
liuqidong@stu. xjtu. edu. cn, (kevinxwu, yefengzheng))teencem com, yejing wangāmy cityu. edu hk, changzijiange lu edu en, eejtu.ddu...m edu. en, xianzhao@cittyo tyu. edu. hk
fengtiandmail. xjtu.
en,
Abstract
Sequential recommender systems (SRS) aim predict users' subsequent choices based on their historical interactions and have found applications in diverse fields such as e-commerce and social media. However, real-world systems, most users interact with only handful of items, while the majority of items are seldom con- sumed. These two issues, known as the long-tail user and long-tail item challenges, often pose difficulties for existing SRS. These challenges experience and seller benefits, making them crucial to address. Though few works have addressed the challenges, they still struggle with the seesaw or noisy issues due to the intrinsic scarcity of interactions. The advancements large language models (LLMs) present promising solution to these problems from semantic perspective. As one of the pioneers in this field, we propose the Large Language Models Enhancement framework for Sequential Recommendation (LLM-ESR). This framework utilizes semantic embeddings derived from LLMs to enhance SRS without adding extra inference load from I.LMs. To address the long-tail item chal- lenge, we design a dual-view modeling framework that combines semantics from LLMs and collaborative signals from conventional SRS. For the long-tail user chal- lenge, we propose a retrieval augmented self-distillation method to enhance user preference representation using more informative interactions from similar To verify the effectiveness and versatility of our proposed enhancement framework, we conduct extensive experiments on three real- real-world datasets using three popular SRS models. The results show that our method surpasses existing baselines consis- tently, and benefits long-tail users and items especially. The implementation code is available at https://gtthbbbc hwtphh//wwwo//ppieieeeceeeeeeeee
adversely affect user
Introduction
The objective of sequential recommendation is to predict the next likely item for users based on their historical records 7 (4) Owing to wide-ranging applicability various domains such as commerce [48) and social media [L sequential recommendation has garnered considerable attention recent years. Given that the essence of sequential recommendation revolves around extracting user preferences from their interaction records, several innovative architectures have been proposed. For
"Corresponding authors: Xian Wu, Feng Tian and Xiangyu Zhao
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

0.55
25k
1.0
25k
0.52
20k
0.8
20k
HR@10
0.49
15k
#serss
HR@10
0.6
15k
items
SASRec
SASRec
0.46
10k
0.4
10k
5k
1-4
5-9
10-14
15-19
20+
1-9
10-29
30-59
60-99
100+
User Group
Item Group
Long-tail User Challenge
(b) Long-tail Item Challenge
Figure 1: The preliminary experiments of SASRec on Beauty dataset.
instance, SASRec 18
applies the self-attention technique to capture the users' long-term preference, introduces pure MLP architecture to identify dynamics in users' preference.
while FMLPRec 24
Despite significant advancements in sequential recommendation, the long-tail challenges continue to undermine its practical utility. Generally, these challenges can be categorized into two types, affecting either the user or the item side. To illustrate, we present the performance of well-known SRS model, SASRec [18] on the Amazon Beauty dataset, along with its statistics in Figure [ i) Long-tail User Challenge: Figure (a), we note that above 80% users have interacted with fewer than 10 items (i.e.. long-tai tail users), and SASRec's performance is subpar for these users compared to those with more interaction records. This suggests that the majority of users receive less than optimal recommendation services. ii) Long-tail Item Challenge: Figure (b) demonstrates that SASRec performs significantly better on more popular items. However, the histogram indicates that around 71.4% items own more than 30 interaction records, meaning they are less frequently consumed. Addressing these long-tail challenges crucial for elevating user experience and seller benefits.
To tackle the long-tail item challenge, existing studies [1 20 examine the co-occurrence pattern between popular and long-tail items, aiming enrich the representation of long-tail items with that of popular ones. Nevertheless, ignorance of the true relationship between items may cause a seesaw problem (36] As for the long-tail user challenge, existing research [37 explores the interaction history of all users, attempting to augment pseudo items for tail users. However, these approaches still only rely on collaborative information, which inclines to generate noisy items due to inaccurate similarity between users 34 At this time, superb semantic relations between users or items can make an effect, which indicates the potential of utilizing semantics face long-tail challenges.
Recent advancements in large language models (LLMs) offer promise for alleviating long-tail chal- lenges from a semantic perspective. However, LLMs initially designed for natural language have made efforts to adapt, but two problems still exist. i) Inefficient Integration: Recent research has explored de- riving informative prompts to activate ChatGPT (55) I0] or modifying the tokenization method of LLaMA (25 27 59 for sequential recommendation. Despite their impressive performance. these approaches are challenging to apply in industrial settings. This is because recommender systems typically require low latency for online deployment, whereas LLMs often entail high inference costs 11. ii) Deficiency of Semantic Information: Several recent works [3] [6] propose utilizing embeddings derived from LLMs initialize the item embedding layer of sequential recommendation models, thereby integrating semantic information. However, the fine-tuning process, if not done without freezing the embedding layer, may erode original semantic relationships between items. Additionally, these approaches focus solely on the item side, neglecting the potential benefits of incorporating semantic information on the user side which could aid the sequence encoder of an SRS.
processing tasks but not for recommendation ones. Some works 66 43
In this paper, to better integrate LLMs into SRS for addressing tail challenges, we design Large Langauge Models Enhancement framework for Sequential Recommendation (LLM-ESS)) Firstly, we derive the semantic embeddings of items and users by encoding prompt texts from LLMs. Since these embeddings can be cached advance, our integration does not impose any extra inference burden from LLMs.. To tackle the long-tai tail item challenge, we devise dual-view modeling framework that combines semantic and collaborative information. Specifically, the embeddings derived from LLMs are frozen to avoid deficiency of semantics. Next, we propose retrieval augmented self- distillation method to enhance the sequence encoder of an SRS model using similar users. The similarity between users is measured by the user representations from LLMs. Finally, it is important

---

Dual-view Modeling
Retrieval Augmented
Self-Distillation
Updated
Po
LDD
Item
Sementic
Sequence Encoder fu(-)
Embedding
S
800
BB
Collaborative
Embedding
Cress attention
Cross attention
coo
LLMs Embedding
1/
Jm
61
Retrieve
Semontic User
Base Van
Adapter
Sementic Embedding Layer E
PCA
Collaborative Embedding Layer B
hnae,e braid. price....
historical
S
A
u
Figure 2: The overview of the proposed LLM-ESR framework.
to note that the proposed framework is model-agnostic allowing be adapted any sequential recommendation model. The contributions of this paper are as follows:
We propose a large language models enhancement framework, which can alleviate both long-tail user and item challenges for SRS by introducing semantic information from LLMs.
To avoid the inference burden of LLMs, design embedding based enhancement method. Besides, the derived embeddings are utilized directly to retain the original semantic relations.
We conduct extensive experiments on three real-world datasets with three backbone SRS models to validate the effectiveness and flexibility of LLM-ESR.
2 Problem Definition
The goal of the sequential recommendation is to give out the next item that users possible to interact with based on their interaction records. The set of users and items are denoted as U 4= --1,..... ...... Wiul and {vi,.... Ug,..... .......2. v respectively, where [24] and | the number of users and items. Each user has an interaction sequence, which arranges the interacted items by timeline, denoted as S, {o(()) ....., ((()) ..... (u)) n, represents the interaction number of user u. For simplicity, we omit the superscript (u) in the following sections. Then, the problem of sequential recommendation can be defined follows:
arg max v.EV
P( U:((..)
we can split the users and items into tail and head groups. Let n, and p, denote the length of the user's interaction sequence and the popularity of the item v (i.e., the total interaction number). Firstly, we sort the users and items by the values of nt and p. in descending order. Then, take out the top 20% users and items as head user and head item according to Pareto principle [4) denoted as Uhead and Vhead. The rest of the users and items are the tail user and tail item, iee, Mrail = l \ Uhead and Viait = V Vread- To alleviate the long-tail tail challenges, we aim elevate the recommending performance for Utat and Vieu-
Following the existing works related to long-tailed SRS 17 20

---

3 LLM-ESR
3.1 Overview
The overview of the proposed LLM-ESR is shown in Figure 2 To acquire the semantic information, we adopt LLMs to encode textual users' historical interactions and items' attributes into LLMs user embedding and LLMs item embedding. Then, two modules are proposed to augment long-tail items and long-tail users, respectively, i.e. Dual-view Modeling and Retrieval Augmented Self- Distillation. i) Dual-view Modeling: This module consists of two branches. One is semantic-view modeling, which aims extract the semantic information from the user's interaction sequence. first utilizes the semantic embedding layer. derived from LLMs item embedding, to encode the items. Then, an adapter is designed for dimension adaptation and space transformation. The output item embedding sequence will be fed into cross-altention for fusion and then sequence encoder to get the user representation in semantic view. The other branch is collaborative-viey modeling, which transforms the interaction sequence into an embedding one by a collaborative embedding layer. Next, followed by a cross-attention and the sequence encoder, the collaborative user preference is obtained. At the end of this module, the user representations in the two views will be fused for the final recommendations. ii) Retrieval Augmented self-Distillation: This module expects to enhance long-tail users through informative interactions of similar users. First, the derived LLMs user embedding is considered as a semantic user base for retrieving similar users. Then, similar users are fed into dual-view modeling get their user representations, which are the guide signal for self-distillation. Finally, the derived distillation loss will be utilized as an auxiliary loss for training.
3.2 Dual-view Modeling
The traditional SRS models are skilled in capturing collaborative signals, which can recommend for popular items well 20 [7]. However, they compromise on long-tail items due to the lack of semantics 2 Therefore, we model the preferences of users from the dual views to cover all items simultaneously. Besides, we propose a two-level fusion to better combine the benefits from both two.
Semantic-view Modeling. In general, the attributes and descriptions of items contain abundant se- mantics. To utilize the powerful semantic understanding abilities LLMs, we organize the attributes and descriptions into textual prompts (the template of prompts can be found in Appendix I.. Then, in avoid of possible inference burden brought by LLMs, we cache the embeddings derived from LLMs for usage. In specific, the embeddings can be obtained by taking out the last hidden state of open-sourced LLMs, such as LLaMA [51], or the public API, such as wweee-ede-edndddddeeeeeeedde ada--0ad -02 We adopt the latter one in this paper. Let Egc E IR V|xdam denotes the LLMs embedding of all items, where dum is dimension of LLMS embedding. Then, the semantic embedding layer E,e from LLMs can be used for semantic-view modeling to enhance long-tail tail items. However, previous works 13 [6] often adapt it as the initialization of the item embedding layer, which may ruin the original semantic relations during fine-tuning. In order to retain the semantics, we freeze the Ese and propose an adapter to transform the raw semantic space into the recommending space. For each item i, we can get its LLMs embedding clim by taking the i-th row of E Ese. Then, it will be fed into the tunable adapter to get the semantic embedding:
WW !im + bb)) + b
(2)
where W IR dilm dim Wa Rdx سسيك and by E R
سسي by Rdx1 are the weight matrices and bias of adapter. Following this process, we can obtain the item embedding sequence of the user's interaction records, denoted S [ef ...., e. Similar to a general SRS model, we employ a
sequence encoder fe (e.g., self-attention layers [52]
for SASRec (18) to get the representation of
user preference in semantic view as follows:
u fS
where T dx is the user preference representation in semantic view and denotes the parameters of sequence encoder in an SRS model.
Collaborative-view Modeling. To utilize the collaborative information, adopt trainable item embedding layer and supervised update it by interaction data. Let Eco E R denotes the

---

collaborative embedding layer of the item. Then, the item embedding sequence S [ei ..... e. acquired by extracting the corresponding rows from Ec.. To get the user preference n in the collaborative view, we input embedding sequence to sequence encoder, i.e., u (o() It is worth noting that, the sequence encoder fo is the same one both semantic and collaborative views for the shared sequential pattern and higher efficiency [46]. Besides, the embedding layers in the two views are in unbalanced training stages (one is pretrained, while the other is from scratch), which may lead to optimization difficulty (.. To handle such a problem, we initialize the Eco by dimension-reducee Ese. The Principal Component Analysis (PCA) is used as the dimension reduction method in this paper.
Two-level Fusion.
The effective integration of both semantic-view and collaborative view is essential to absorb the benefits of these two. However, the direct merge of the user representations in dual views may overlook the nuanced inter-relationship between item sequences. Thus, we design a two-level fusion method for the dual-view modeling module, i.e., sequence-level and logit-level. The former aims to implicitly capture the mutual relationships between the item sequences of dual views, while the latter explicitly targets the combination of recommending abilities. In specific, we propose a tross-attention mechanism for sequence-level fusion. To simplify the description, we only take the semantic view interacting with the collaborative view for illustration, and the other view is the same. Specifically, S is considered as the query, and S as the key and value in attention mechanism. Let Q S K S V = S where W W W R are weight matrices. Then, the interacted collaborative embedding sequence can be formulated as follows:
QK T
Sco = Softmax
JV
(4)
Following the same process of cross-attention, we can also get the corresponding semantic embedding sequence S=e Finally, S S are substituted by Sse She to be fed into fa(-). As for logit-level fusion, we concatenate the two-view user and item embeddings for recommendation. The probability score of recommending item j for the user u is therefore calculated as:
P 2y (1...)
lej"
"ef"" [u" .u"
(5)
where ":" denotes the concatenation operation of two vectors. Based on the probability score, we adopt the pairwise ranking loss to train the framework:
Thu loga (P( = [111a)- - P(v [ui:k))
L Rank WELL A=1
(6)
where vit and 1 are the ground-truth item and paired negative item. It is worth noting that the ranking loss may differ little according to different backbone SRS models, e.g,, sequence-to-one pairwise loss for GRU4Rec [1].
3.3 Retrieval Augmented Self- deff-Distillation
The long-tail user problem originates from the lack of enough interactions for the sequence encoder in SRS to capture users' preferences. Thus, propose self distillaaitn method to augment the extraction capacity of the sequence encoder. Self-distillation [12 61 is a type of knowledge distillation that considers one model as both the student and teacher for model enhancement. As for the SRS, since multiple similar users have more informative interactions, it is promising to transfer their knowledge to the target user for strengthening. Thereafter, there two key challenges for such knowledge transfer, iee, how to retrieve similar users and how to transfer the knowledge.
Retrieve Similar Users. Previous works have confirmed that LLMS can understand the semantic meanings of textual user interaction records for recommendation 225 I0] Based on their observation, we organize the item's title that interacted by users into the textual prompts (the template of prompts Then, similar to the derivation of LLMs item embedding Eze, we
can be found in Appendix[A.] can obtain and save the L.LMs user embedding, denoted as Unm E R244
is also dubbed as the semantic user base in this paper, because the semantic relations are encoded in it. For each target user h. we can retrieve the similar user set Us as follows:
Us Top((cos(ur Im .u, ( )}==1, N)
(7)
5

---

where c.s(.,)) is the cosine similarity function to measure the distance between two vectors. NV represents the size of similar user sets, which a hyper-parameter.
Self-Distillation. As mentioned before, we design the self-distillation to transfer the knowledge from several similar users the target user. Since the representation of user preference, i.e, u and u encode the comprehensive knowledge of the user, we configure such representation as the mediator for the distillation. To get the teacher mediator, we first utilize the dual-view modeling framework (Section 3.2) to get the user representation for each similar user, denoted as {ufe, ( Jj=1- Then, the teacher mediator is calculated by mean pooling, as the following formula:
[Md
T Mean__Mollliee Pooling([[u]) =ug))
(8)
The student mediator is the representation of target user k. i.e., [ug student mediators, the self-distillation loss can formulated as:
uga). Based on the teacher and
1 [2
CSD
|
u
ju
u
Note that the gradients of uzt and uff are stopped, because they only provide the guidance signal instead of optimizing the model.
3.4 Train and Inference
Train. Based on the illustration in Section 3.2 and Section 3.3 we only update the collaborative embedding layer, adapter, roos-aatenntion and sequence encoder during the training, while freezing the semantic embedding layer and semantic user base. Since the original LLMs embeddings E.c and Unm are frozen, the original semantic relations get preserved well. The training loss for optimization is the combination of pairwise ranking loss and self-distillation loss, which can be written as follows:
L = LRank + a LSD
(10)
where ar is hyper-parameter to adjust the magnitude of self-distillation.
Inference. During the inference process of the LLM-ESR, the retrieval augmented self-distillation module is exempted due to no need for the auxiliary loss. Thus, we follow the dual-view modeling process for the final recommendation by Equation (5). Besides, since the semantic embedding layer can be cached in advance, the call for LLMs is avoided, which prevents the extra inference costs. Due to the limited space, the algorithm lies in Appendix A.2 for more clarity.
4 Experiment
4.1 Experimental Settings
Dataset. There are three real-world datasets applied for evaluation, i.e., Yelp, Amazon Fashion and Amazon Beauty. We follow the previous SRS works [8 50 for preprocessing and data split. More details about the datasets and preprocessing can be seen Appendix B.I]
Baselines. To validate the flexibility, we combine the competing baselines and LLM-ESR with three well-known backbone SRS models: GRU4Ree [14), Bert4Rec [49] and SASRec (18). Then, two groups of baselines are compared in the experiments, One group is the traditional enhancement framework for the long-tailed sequential recommendation, including CITIES (17) and MELT (20).
The other group is the L-sM-based enhancement framework, which contains RLMRec 45 and The more details about baselines are put into Appendix B.2
LLMInit 13 [6].
Implementation Details. The hardware used in all experiments is an Intel Xcon Gold 6133
platform with Tesla V100 32G GPUs, while the basic software requirements are Python 3.9.5 and Py Torch 1.12... The hyper-parameters {1. 0.5, 0.1, 0.05, 0.01}. More details about the implementation details are in Appendix B.3 The implementation code is available at https://github... com/ haamm//ppieedddee Saapled-Machiieeee-. Machine- Maahhne--Larring... Lab/LLLLLLaaLLLMM
and a are searched from f2.6. 10, 14, 18 and
Evaluation Metrics. In the experiments, we adopt the metrics of Top-10 10 list for evaluation. Specifi- cally, the Hi Rate (H@ 10) and Normalized Discounted Cumulative Gain (N@ 10) are used. Follow- ing [18).
we randomly sample 100 items that the user has not interacted with as the negatives paired

---

Table 1: The overall results of competing baselines and our LLM-ESR. The boldface refers to
the highest score and the underline indicates the next best result of the models. *n indicates the statistically significant improvements (L.e., two-sided t-test with p < 0.05) over the best baseline.
Overall
Tail Item
Head Item
Tail User
Head User
Dataset
Model
H@ 10
Nē 10
H@ 10
N@
H@
N@10
H@
N@
H@
N@
GRU4Rec
0,4879
0.2751
0.0171
0.0059
0.6265
0.3544
0.4919
0.2777
0.4726
0.2653
CITIES
0.4898
0.2749
0.0134
0.0051
0.6301
0.3543
0.4936
0.2783
0.4756
0.2618
-ELTT
0.4985
0.2825
0.0201
0.0079
0.6393
0.3633
0.5046
0.2865
0,4750
0.2671
RLMRec
0.4886
0.2777
0.0188
0.0067
0.6269
0.3574
0.4920
0.2804
0.4756
0.2671
0,4872
0.2749
0.0201
0.0072
0.6246
0.3537
0.4908
0.2775
0,4732
0.2647
L.M--ES
0
0_3413*
0.0763
0.0318
0.7184
0.4324
0.5782
0.3456
0
0.3247
Bett4Rec
0.5307
0,3035
0,0115
0.0044
0.6836
0.3916
0.5325
0.3047
0,5241
0.2988
Yelp
-CITIES
0,5249
0,3015
0.0041
0.0014
0,6783
0.3899
0.5274
0.3032
0.5155
0.2954
-MELT
0.6206
0.3770
0.0429
0.0149
0.7907
0.4836
0.6210
0,3780
0.6191
0.3733
-RLMRec
0,5306
0.3039
0.0104
0.0040
0,6938
0.3922
0
0.3065
0.5137
0.2936
-LLMInit
0.6199
0.3781
0.0874
0,0330
0.7766
0.4797
0.6204
0.3796
0.6178
0.3723
LLM-ESR
0.6623
0.4222
0.1227
0,0500
0.8212
0.5318
0.6637*
0.4247
0.6571
0.4127
SASRec
0.5940
0.3597
0.1
0.0495
0.7353
0.4511
0.5893
0.3578
0.6122
0.3672
-CITIES
0,5828
0.3540
0.1532
0.0700
0,7093
0,4376
0.5785
0.3511
0,5994
0.3649
MELT
0.6257
0.3791
0.1015
0.0371
0.7801
0.4799
0.6246
0.3804
0.6299
0.3744
RLMRec
0.5990
0.3623
0.0953
0.0412
0,7474
0,4568
0.5966
0,3613
0.6084
3.3658
L.LMIni
0.6415
0.3997
0.1760
0.0789
0,7785
0.4941
0.6403
0.4010
0.6462
0. 1.39988
LLM-ESR
0.6673
0,4208
0.1893*
0.0845
0. 0.8080
0.5199
0.6685
0.4229
0.6627
0.4128
GRU4Rec
0,4798
0,3809
0.0257
0.0101
0.6606
0.5285
0.3781
0.2577
0.6118
0.5408
CITIES
0.4762
0.3743
0,0252
0.0103
0.6557
0.5191
0.3729
0,2501
0.6103
0.5354
-METT
0,4884
0.3975
0.0291
0.0112
0.6712
0,5513
0.3890
0,2770
0.6173
0.5538
RLMRec
0.4795
0,3808
0.0255
0.0105
0.6603
0.5282
0.3773
0,2577
0.6120
0.5405
LLMMait
0.4864
0,4095
0.0250
0.0104
0.6702
0.5684
0,3852
0.2973
0.6177
0.5550
LLM--SR
0,5409
0.4567
0
0.0384
0.7242
0.6233
0.4560
0.3568
0.6512
0.5864
Fashion
Bett&Rev
0,4668
0,3613
0.0142
0.0067
0.6470
0.5024
0.3500
0.2344
0.6183
0.5258
CITTIS
0.4926
0.4090
0.0223
0.0099
0.6799
0.5679
0.3952
0.2975
0.6190
0.5535
-METT
0,4897
0,3810
0.0059
0.0019
0.6823
0.5319
0.3842
0,2514
0.6266
0.5491
RI MLM
0.4744
0.3567
0.0044
0.0015
0.6615
0.4981
0.3626
0.2268
0.6194
0.5251
LLLMlii
0,4854
0.4035
0.0328
0.0161
0,6655
0.5577
0.3773
0,2846
0.6255
0.5578
L.M--ES
0.5487
0.4529*
0.0525
0.0225
0.7462
0.6243
0.4629
0.3460
0.6599
0.5916
SASRee
0,4956
0.4429
0,0454
0.0235
0.6748
0.6099
0.3967
0.3390
0.6239
0.5777
CITIE
0.4923
0.4423
0.0407
0.0214
0.6721
0.6098
0.3936
0,3392
0.6203
0.5760
-METT
0,4875
0.4150
0.0368
0.0144
0.6670
0.5745
0.3792
0.2933
0.6280
0.5729
RLMKec
0.4982
0.4457
0.0410
0.0223
0.6803
0.6143
0.3990
0.3415
0.6270
0.5808
-LMMlit
0,5119
0.4492
0.0396
0.0305
0.6920
0.6159
0.4184
0,3501
0.6332
0.5777
LLM-ESR
0.5619
0.4743
0.1095
0.0520
0.7420
0.6424
0
0.3769
0.6668
0.6005
GRU4Rec
0.3683
0.2276
0.0796
0.0567
0.4371
0.2683
0.3584
0.2191
0.4135
0,2663
CITTES
0,2456
0,1400
0.1122
0.0760
0,2774
0.1552
0,2382
0.1346
0.2795
0.1645
-EETT
0.3702
0.2161
0.0009
0.0003
0.4582
0.2675
0.3637
0.21
0,3997
0.2365
RLMRee
0,3668
0,2278
0.0780
0.0560
0.4357
0.2688
0.3576
0,2202
0.4089
0.2626
-L.MInit
0.4151
0.2713
0.0896
0.0637
0.4928
0.3208
0.4059
0.2621
0.4571
0.3133
LLM--SRR
0.4917
0.3140
0.1547
0.0 0.0801
0
0.3698
0
0.3079
0.5220
0.3420
Bett4Rec
0,3984
0.2367
0.0101
0.0038
0.4910
0.2922
0.3851
0.2272
0.4593
0,2801
Beauty
-CITIES
0,3961
0.2339
0.0023
0.0008
0.4900
0.2895
0.3832
0.2250
0.4551
0.2746
-MELT
0.4716
0.2965
0.0709
0.0291
0.5671
0.3603
0.4596
0,2865
0,5263
0.3423
RLMRecc
0,3977
0.2365
0.0090
0.0032
0.4903
0.2921
0.3853
0.2277
0.4539
0.2765
-LLMInit
0,5029
0.3209
0.0927
0.0451
0.6007
0.3867
0.4919
0,31
0,5530
0. 0.3632
-LM--SR
0.5393
0,3590
0.1379
0.0745
0.6350
0.4269
0.5295
0.3507
0.5839*
0,3972
SASRec
0.4388
0.3030
0.0870
0.0649
0,5227
0.3598
0.4270
0.2941
0.4926
0.3438
CITTIS
0.2256
0.1413
0.1363
0.0897
0.2468
0.1536
0.2215
0416
0.2441
MELT
0.4334
0.2775
0.0460
0.0172
0,5258
0.3995
0.4233
0.2673
0.4796
0.3241
RLMRec
0,4460
0.3075
0.0924
0.0658
0.5303
0.3652
0.4365
0.3016
0,4892
0.3345
L.LMlnit
0.5455
0.3656
0.1714
0.0965
0.6347
0.4298
0.5355
0.3592
0,5893
0.3948
L.I.-ESR
0.5672
0.3713*
0.2257
0.1
0.6486
0.4334
0.5581
0.3643
0.6087
0.40322
with the ground truth for calculation of the metrics. To guarantee the robustness of the experimental results, we report the average results of the triplicate test with random seeds {42, 43, 44).
4.2 Overall Performance
To validate the effectiveness and flexibility of the proposed LLM-ESR, we show the overall, tail and head performance on three datasets in Table T At glance, we find that the proposed LLM-ESR can outperform all competing baselines with all SRS models across all user or item groups, which verifies the usefulness of our framework. Then, we probe more conclusions by the following analysis.
7

---

Table 2: The ablation study on the Yelp dataset with SASRec as the backbone SRS model. The
boldface refers to the highest score and the underline indicates the next best result of the models.
Overall
Tail Item
Head Item
Tail User
Head User
Model
H@10
N@10 10
H@
N@10
H@
N@10
H@ 10
N@10
H@10 10
N@10
LLM--SSR
0.6673
0,4208
0.1893
0,0845
0.8080
0.5199
0.6685
0,4229
0.6627
0.4128
-wo Co-view
0.6320
0.3816
0.1898
0.0856
0.7621
0,4687
0.6318
0,3823
0.6325
0.3787
-wo Se-view
0.6468
0.4038
0.1105
0,0460
0.8047
0,5091
0.6459
0.4043
0.6501
0,4018
w/o SD
0.6572
0.4121
0.2003
0.0898
0.7911
0,5071
0.6566
0.4130
0.6574
0,4091
w/o Share
0.6595
0,4158
0.1728
0.0783
0.8027
0,5152
0.6606
0.4186
0.6552
0,4055
-w/o CA
0.66444
0.4160
0.1850
0.0803
0.8004
0.5119
0.6652
0.4175
0.6616
0.4105
1-layer Adapter
0.6108
0.3713
107
0.0469
0.7580
0,4668
0.6065
0.3702
0.6269
0,3754
Random Init
0.6440
0.3984
0.1899
0,0839
0.7777
0,4910
0.6454
0.4018
0.6388
0,3853
Overall Comparison. From the results, we observe that the proposed LLM-ESR leads the overall performance under both two metrics, which indicates better-enhancing effects. LLMInit is often the secondary. This phenomenon shows that the injection of semantics from LLMs actually augments the SRS. However, RLMRee often underperforms compared with other LLM-based methods, because it is devised for collaborative filtering algorithms, incompatible with SRS. As for the traditional baselines, MELT stays ahead in most cases. The reason lies in that it addresses the long-tail user and long-tail item challenges simultaneously. By comparison, CITIES is even sometimes inferior to the backbone SRS model due to the seesaw problem, i.e., drastic drops for popular items.
Long-tail Item and User Comparison. According to the split method illustrated Section 2 the items are grouped into Tail Item and Head Item. From T we observe that our LLM-ESR not only achieves the best on the tail item group but also gets the first place on the head item group. Such performance comparison highlights the combination of semantics and collaborative signals by our dual-view modeling. LLMInit leads the tail group across all baselines, which suggests that semantic information benefit long-tail items. It is worth noting that CITIES sometimes perform better for the tail group but harm those popular items, which means it has a seesaw problem. Additionally, the results illustrate that MELT, LLMInit and LLM-ESR can augment the tail user group markedly. MELT is devised to enhance tail user, but underperforms our method because of its limitations to collaborative perspective. Though LLMInit can also benefit tail users by introducing semantics, ignores the utilization of LLMs from the user side.
Flexibility. shows that the proposed framework can get the largest performance improvements on all three backbonn SRS models, which indicates the flexibility of LLM-ESR. By comparison, the other baselines incline to depend on the type of SRS. The traditional method, i.e., CITIES and MELT, tend to perform better for GRU4Rec, while LLMInit is more beneficial to Bert4Rec and SASRec.
4.3 Ablation Study
The results of the ablation study are shown in Table 2 Firstly, we remove the collaborative view or semantic view to investigate the dual-view modeling, denoted as w/o Co-view and w/o Se-view. The results show that w/o Co-view downgrades performance dramatically on the head group, while w/o Se-view harms tail items evidently. Such changes indicate the distinct specialty of collaborative and semantic information, highlighting the combination of both. w/o SD means dropping self-distillation, which shows performance drops for long-tail users. It suggests the effects of the proposed retrieval augmented self-distillation. The results of these three variants validate the motivation for designing each component for LLM-ESR. w/o Share and w/ CA represent using split sequence encoder and removing cross- aots-ntttntion. The decrease in performance of these two illustrates the effectiveness of the sharing design and sequence-leve fusion. More results can be seen in Appendix C.I
Furthermore, we have two designs to ease the optimization of the entire LLM-ESR framework. One that we use dimension-reduus reduced LLM item embeddings to initialize the collaborative embedding layer instead of random initialization. On the other hand, we propose two-layer adapter to fill the large dimension gap between LLM embeddings and item embeddings. To illustrate the effectiveness of these two designs, we compare l-layer Adapter and Random Init variants of LLM-ESR. The results, shown in Table 2 indicate that both variants underperform the original L.M--SS, verifying the success of our special designs.
8

---

0.668
0.422
0.668
0.4210
Ho
0.664
NDCG@10
0.418
0.686
NDCG@10
0,4205
0.660
0.414
80.664
0.656
0.410
0.662
20.419
50
Nio
0.652
5e-1
le
30
le-2
0.406
5e-1
1e-1
5e.2
1e.2
0.680
10
14
18
0.4190
10
14
18
N
N
Figure 3: The paraeeerr experiments on weight of self-distillation loss a and the number of retrieved similar users N. The results are based the Yelp dataset with the SASRcc model.
SASRec
MELT
LLMInit
Goo
LLM-ESR
0.7
1.0
0.8
0.6
HR@10
HR@10
0.6
0.5
0.4
0.4
0.2
0.3
0.0
1-4
5-9
10-14
15-19
20+
1-9
10-29
30-5 59 60-99
100
(a) User Group
(b) Item Group
Figure 4: The results of the proposed LLM-ESR and competing baselines in meticulous user and item groups. The results are based on the Beauty dataset with the SASRec model.
4.4 Hlyper-parmmee Analysis
To investigate the effects of the hyper-parameters in LLM-ESR, we show the performance trend along with their changes in Figure 3 The hyper-paramene a controls to what extent the designed self- distillation affects the optimization. With ranging from 1 to 0.01, the recommending accuracy rises first and drops then. The reason for the compromised performance of large a lies in that overemphasis self-distillation will affect the convergence of ranking loss. Smaller also downgrades the performance, which indicates the usefulness of the designed self-distillation. As for the number of retrieved users N, the best is 10. The reason is that more users provide more informative interactions. However, too large N may decrease the relatedness of the retrieved users.
4.5
Group Analysis
For more meticulous analysis, we split the users and items into 5 groups according to sequence length Nu and popularity Ps, and show the performance of each group in Figure From the results, we observe that LLM-based frameworks derive increases every user and item group, while MELT has positive effect on some specific groups. It reflects the seesaw problem of MLET and reveals the benefit making use of semantic embeddings from LLMs. Comparing LLMInit with LLM-ESR, LLM-ESR can get more increments on the long-tail groups (e.g., 1-4 user group and 1-9 item group), which proves the better reservation of semantic information from LLMs by our framework. The group analysis of Bert4Rec and GRU4Rec as backbones are shown in Appendix C.3
5
Related Works
5.1
Sequential Recommendation
The core of sequential recommendation refers capturing the sequence pattern for the next likely
item 229
39
38
60
24
23
26 40). Thus, the early stage, researchers focus fabricating the

---

architecture to improve model capacity. GRU4Rec 14 and Caser [50] apply RNNs and CNNs [21 for sequence modeling. Later, inspired by the great success of self-attention [22] in natural language processing, SASRec [8] and Bert4Rec [4 verify its potential in SRS. Also, Zhou et al. [66] proposes pure MI.P architecture, achieving similar accuracy but higher efficiency compared with SASRec. Despite the great progress in SRS, long-tail problems are still underexplored. As for the long-tail item problem, CITIES [17 designs an embedding inference function for those long-tail items specially. In terms of the long-tail user problem, data augmentation is the main way [37 34). Only one work, MELT [20], addresses both two problems simultaneously but still sticks to a collaborative perspective. By comparison, the proposed LLM-ESR handles both the two long-tail tail problems better from a semantic view by introducing LLMs.
5.2
LLMs for Recommendation
have attracted widespread attention due to their powerful abilities in semantic understanding. Recently, There emerge several works to explore how to utilize LLMs in recommender systems (RS) [64 28 22 57 58 65
Large language models 63 43
32
30 35 which can be categorized into
two lines, i.e., LLMs as RS and LLMs enhancing RS. The first line of research aims to complete recommendation tasks by LLMs directly. At the early stage, researchers tend to fabricate the prompt templates to stimulate the recommending ability of LLMs by dialogues. For example, ChatRec [10] proposes dialogue process to complete recommendation tasks step by step. DRDT (55) integrates retrieval-based dynamic reflection process for SRS by in-context learning [6. LLMRerank [9] and
UniLLMRec 62) fabricate the chain-of-thought prompts to target the reranking stage and whole
ommendation process, respectively. Besides, some other researchers explore fine-tuning open-sourced is the first one, which fine-tunes LLaMA-7B by parametter-efficient fine-tuning techniques [5] 33 Some following works, including E4SRec [251. LLaRA (27] and RecInterpreter [59] target combining collaborative signals into LLMs by modifying the tokenization. However, this line of work faces the challenge of high inference costs. Another line, LLMs enhancing RS, is more practical, because they avoid the use of LLMs while recommending. For instance, RLMRec [45] aligns with LLMs by an auxiliary loss. AlphaRec [47] adopts LLMs embedding to enhance the collaborative filtering models. On the other hand, LLM4MSR (56) and Uni-CTR [8] propose to utilize LLMs to augment the multi multi-domain recommendation models. As for LLMs enhancing sequential recommendation, Harte et al. [13 and Hu et al. [16] adopt LLMs embedding as the initialization for the traditional models. The proposed LLM-ESR belongs to the latter category but further alleviates the problem of defect of semantic information.
LLMs for RS. TALLRec 2
Conclusion
In this paper, we propose a large language model enhancement framework for sequential recommen- dation (LLM-ESR) to handle the long-tail user and long-tail item challenges. Firstly, we acquire and cache the semantic embeddings derived from LLMs, which is for inference efficiency. Then, view modeling framework is proposed to combine the semantics from LLMs and collaborative signals contained in the traditional model. It can help augment the long-tail items in SRS. Next, design the retrieval augmented self-distillation to alleviate the long-tail user challenge. Through the comprehensive experiments, we verify the effectiveness and flexibility of our LLM-ESR.
7
Acknowledgements
This research was partially supported by National Key Research and Development Program of China (2022YFC3303600) National Natural Science Foundation of China (No. NNo.61197781. 67772877 No.62293551, 60.62277 No.62137002, No.6172 1002, No.6193700) No. No.62377038), Project of China Knowledge Centre for Engineering Science and Technology, "LENOVO-XITU" Intelligent Industry Joint Laboratory Project, Research Impact Fund (No.R 1No.R1015-22 1015-23). Collaborative Research
Fund (No.C 110.31044,,, APRC UityU New Research Initiatives (No. (No.9610565 Start-up Grant for New Faculty of UityU), CityU- HKIDS Early Career Research Grant (No.9360163), Hong Kong ITC Innovation and Technology Fund Midstream Research Programme for Universities Project (No. Mo//TS//3//2MM)) Hong Kong Environmental and Conservation Fund (No. 88/2022), SIRG CityU Strategic Interdisciplinary Research Grant (No. 70204620,,) and Tencent (CCF-Tencent Open Fund, Tencent Rhino- Bird Focused Research Program).
10

---

References
[1] I. Amos, J. Berant, and A. Gupta. Never train from scratch: Fair comparison of long-sequene models requires data-driven priors. In The Twelfth International Conference Learning Representations, 2023.
[2] K. Bao, J. Zhang. Y. Zhang, W. Wang. F. Feng, and X. He. Tallrec: An effective and efficient tuning framework to align large language model with recommendation, In Proceedings of the 17th ACM Conference Recommender Systems, pages 1007-1014, 2023.
[3] BehnamGhader, V. Adlakha, M. Mosbach, D. Bahdanau, N. Chapados, and S. Reddy. Llm2vec: Large language models are secretly powerful text encoders. arXiv preprint arXiv:2404.05961 2024.
[4] G. E. Box and R. D. Meyer. An analysis for unreplicated fractional factorials, Technometrics, 28(1):11-18, 1986.
[5] J. Chang, C. Gao, Y. Zheng, Y. Hui, Y. Niu, Y. Song, D. Jin, and Y. Li. Sequential recom- mendation with graph neural networks. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pages 378-387, 2021.
[6] Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, and Z. Sui. A survey in-context learning, arXiv preprint arXiv:2301.002 ..00344 2022.
[7] H. Fang, D. Zhang, Y. Shu, and G. Guo. Deep learning for sequential recommendation: Algorithms, influential factors, and evaluations. ACM Transactions on Information Systems (TOIS), 39(1):1-41 2020.
[8] Z. Fu, X. Li, C. Wu, Wang. Dong, Zhao, M. Zhao, H. Guo, and R. Tang. A unified framework for multi-domain ctr prediction via large language models. ACM Transactions on Information Systems, 2023.
[9] J. Gao, B. Chen, X. Zhao, W. Liu, X. Li, Y. Wang. Z. Zhang, W. Wang. Y. Ye, S. Lin, et al. L Lim-enhanced reranking in recommender systems. arXiv preprint arXiv:2406. /2433, 2024.
[10] Y. Gao, T. Sheng, Y. Xiang, Y. Xiong, H. Wang, and J. Zhang. Chat-rec: Towards interactive and explainable llms-augmented recommender system. arXiv preprint vrXiv:2303./4522 2023.
[11] B. Geng, Z. Huan, X. Zhang, Y. He, L. Zhang, F. Yuan, J. Zhou, and L. Mo. Breaking the length barrier: Lim-enhanced ctr prediction in long textual user behaviors. arXiv preprint a403 19347, 2024.
[12] J. Gou, B. Yu, S. J. Maybank, and D). Tao. Knowledge distillation: A survey. International Journal of Computer Vision, 1789-1118 2021.
[13] J. Harte, W. Zorgdrager, P. Louridas, A. Katsifodimos, D. Jannach, and M. Fragkoulis. Leverag- ing large language models for sequential recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems, pages 1096-1102, 2023.
[14] B. Hidasi, A. Karatzoglou, L.. Baltrunas, and D. Tikk. Session-based recommendations with recurrent neural networks. In The International Conference on Learning Representations, 2016.
[15] E. J. Hu, P. Wallis, Z. Allen-Zhu Y. Li. S. Wang, L. Wang, W. Chen, et al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2021.
[16] J. Hu, W. Xia, X. Zhang, C. Fu, W. Wu, Z. Huan, A. Li, Z. Tang, and J. Zhou. Enhancing sequen- tial recommendation via lim-dased semantic embedding learning. In Companion Proceedings of the ACM on Web Conference 2024, pages 103-111, 2024.
[17] S. Jang, H. Lee, H. Cho, and S. Chung. Cities: Contextual inference of tail-item embeddings for sequential recommendation. In 2020 IEFE International Conference on Data Mining (ICDM), pages 202-211. IEEE, 2020.

---

[18] W.-C. Kang and J. McAuley, Self-attentive sequential recommendation. In 2018 IEEE interna- tional conference on data mining (ICDM), pages 197-206. IEEE, 2018.
[19] D. M.-W. C. Kenton and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171-4186, 2019.
[20] K. Kim, D. Hyun, S. Yun, and C. Park. Melt: Mutual enhancement of long-tailed user and item for sequential recommendation. In Proceedings of the 46th international ACM SIGIR conference on Research and development in information retrieval, pages 68-77, 2023.
[21] Y. LeCun, I.. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied document recognition. Proceedings of the IEEE, 86(11):2278-2333 23244, 1998.
[22] L. Li, Y. Zhang. D. Liu, and L. Chen. Large language models for generative recommendation: A survey and visionary discussions. arXiv preprint arXiv:2309.01 0157 2023.
[23] M. Li, Z. Zhang, X. Zhao, W. Wang, M. Zhao, R. Wu, and R. Guo. Automlp: Automated mlp for sequential recommendations. In Proceedings of the ACM Web Conference 2023, pages 1190-1198, 2023.
[24] M. Li, X. Zhao, C. Lyu, M. Zhao, R. Wu, and Guo. MIpArec: pure mlp architecture for sequential recommendations. In 31st International Joint Conference on Artificial Intelligence and the 25th European Conference on Artificial Intelligence (ICAA-ECCA 2022), pages 2138- 2144. International Joint Conferences on Artificial Intelligence, 2022.
[25] X. Li. C. Chen, X. Zhao, Y. Zhang, and C. Xing. E4srec: An elegant effective efficient extensible solution of large language models for sequential recommendation. arXiv preprint arXiv:3312.02443, 2023.
[26] J. Liang, X. Zhao, M. Li, Z. Zhang, W. Wang, H. Liu, and Z. Liu. Mmmlp: Multi-modal multilayer perceptron for sequential recommendations. In Proceedings of the ACM Web Conference 2023, pages 111-, 2023.
[27] J. Liao, S. Li, Z. Yang, J. Wu, Y. Yuan, X. Wang, and X. He. Llara: Aligning large language models with sequential recommenders. arXiv preprint arXiv:2312.02445. 2023.
[28] J. Lin, X. Dai, Y. Xi, W. Liu, B. Chen, X. Li, C. Zhu, H. Guo, Y. Yu, R. Tang, et al. How can recommender systems benefit from large language models: survey, arXiv preprint arXiv:2306.05817 2023.
[29] L. Liu, L. Cai, C. Zhang, X. Zhao, J. Gao, W. Wang, Y. Lv, W. Fan, Y. Wang, M. He, et al. Linrec: Linear attention mechanism for long-term sequential recommender systems. In Proceedings of the 46th International ACM SIGIR Conference Research and Development in Information Retrieval, pages 289, 2023.
[30] Q Liu, J. Hu, Y. Xiao, X. Zhao, J. Gao, W. Wang, Q. Li, and Tang. Multimodal recommender systems: A survey. ACM Computing Surveys, 57(2):1-17, 2024.
[31] Q. Liu, F. Tian, Q. Zheng, and Q. Wang. Disentangling interest and conformity for eliminat- ing popularity bias in dession-based recommendation. Knowledge and Information Systems, 65(6):2645 2664) 2023.
[32] Q. Liu, X. Wu, W. Wang, Y. Wang, Y. Zhu, X. Zhao, F. Tian, and Y. Zheng. Large lan- guage model empowered embedding generator for sequential recommendation. arXiv preprint a40iv:2409. 19925, 2024.
[33] Q. Liu, X. Wu. X. Zhao, Y. Zhu. D. Xu. F. Tian, and Zheng. When moe meets 1lms: Parameter efficient fine-tuning for multi-task medical applications. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1104-1114, 2024.
[34] Q. Liu, F. Yan, X. Zhao, Z. Du, H. Guo, R. Tang, and F. Tian. Diffusion augmentation for sequential recommendation. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 1576-1586, 2023.
12

---

[35] Q. Liu, X. Zhao, Y. Wang, Y. Wang, Z. Zhang, Y. Sun, X. Li, M. Wang, P. Jia, C. Chen, et al. Large language model enhanced recommender systems: Taxonomy, trend, application and future. arXiv preprint arXiv:2412. 13432, 2024.
[36] S. Liu and Zheng. Long-tail session- recommendation. Proceedings of the 14th ACM Conference on Recommender Systems, pages 509-514, 2020.
[37] Z. Liu, Fan, Y. Wang. and P. S. Yu. Augmenting sequential recommendation with pseudo- prior items via reversely pre- pre-training transformer. In Proceedings of the 44th international ACM SIGIR conference on Research and development information retrieval, pages 1608-1612, 2021.
[38] Liu, Q. Liu, Y. Wang. W. Wang, Jia, M. Wang. Liu, Y. Chang, and Zhao. Bidirectional gated mamba for sequential recommendation. arXiv preprint arXiv:2408. 1451, 2024.
[39] Z. Liu, S. Liu, Z. Zhang. Q. Cai, X. Zhao, K. Zhao, L. Hu, Jiang, and K. Gai. Sequential rec- ommendation for optimizing both immediate feedback and long-term retention. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 117221882, 2024.
[40] Z. Liu, Tian, Q. Cai, X. Zhao, Gao, S. Liu, Chen, He, D. Zheng. Jiang, al. Multi-task recommendations with reinforcement learning. In Proceedings of the ACM Web Conference 2023, pages 1273-1282, 2023.
[41] S. Luo, Y. Yao, B. He, Y. Huang, A. Zhou, X. Zhang, Y. Xiao, M. Zhan, and L. Song. Integrating large language models into recommendation via mutual augmentation and adaptive aggregation. arXiv preprint arXiv:2401.13877 13870, 2024.
[42] McAuley, C. Targett, Q. Shi, and A. Van Den Hengel. Image- Image-based based recommendations on styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pages 43-52, 2015.
[43] B. Min, H. Ross, E. Sulem, A. P. B. Veyseh, T. H. Nguyen, O. Sainz, E. Agirre, I. Heintz, and D. Roth. Recent advances in natural language processing via large pre-trained language models: A survey, ACM Computing Surveys, 56(2):1-40) 2023.
[44] K. Pearson. Liii. on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin philosophical magazine and journal of science, 2(11):559-572, 1901.
[45] X. Ren, W. Wei, L. Xia, L. Su, Cheng, J. Wang, D. Yin, and C. Huang. Representation learning with large language models for recommendation. In Proceedings of the ACM Web Conference 2024, page 3464-3475, 2024.
[46] J. Shang, T. Ma, C. Xiao, and J. Sun. Pre-training of graph augmented transformers for medication recommendation. 28th International Joint Conference on Artificial Intelligence, IJCAI 2019, pages 5953-5959, International Joint Conferences on Artificial Intelligence, 2019.
[47] L. Sheng, A. Zhang, Y. Zhang, Y. Chen, X. Wang, and T.-S. Chua. Language models encode collaborative signals in recommendation. arXiv preprint arXiv:2407.0544 2024.
[48] U. Singer, H. Roitman, Y. Eshel, A. Nus, Guy, O. Levi, Hasson, and Kiperwasser. Sequential modeling with multiple attributes for watchlist recommendation in e-commerce. In Proceedings of the Fifleenth ACM International Conference on Web Search and Data Mining, pages 937-946, 2022.
[49] Sun, J. Liu, Wu, C. Pei, X. Lin, Ou, and P. Jiang. Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management, pages 1441-1450 2019.
[50] J. Tang and K. Wang. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the eleventh ACM international conference on web search and data mining, pages 565-573, 2018
13

---

[51] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, Joulin, E. Grave, and G. Lample. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 13971, 2023.
[52] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. Advances neural information processing systems, 30, 2017.
[53] L. Wang, N. Yang. X. Huang, L. Yang. R. Majumder, and F. Wei. Improving text embeddings with large language models. arXiv preprint arXiv:2401. 2000::2401.0036,, 2023.
[54] S. Wang, L. Hu, Y. Wang. L. Cao, Q. Z. Sheng, and M. Orgun. Sequential recommender systems: challenges, progress and prospects. In 28th International Joint Conference on Artificial Intelligence, IJCAI 2019, pages 6332 6338. International Joint Conferences on Artificial Intelligence, 2019.
[55] Y. Wang, Liu, J. Zhang, W. Yao, S. Heinecke, and P. S. Yu. Drdt: Dynamic reflection with divergent thinking for Ilm-based sequential recommendation. arXiv preprint arXiv:2312.11336, 2023.
[56] Y. Wang, Y. Wang, Z. Fu, X. Li, X. Zhao, H. Guo, and R. Tang. Llm4msr: An llm-enhanced paradigm for multi-scenario recommendation. arXiv preprint 22XXv:2406.12229 2024.
[57] L.. Wu, Z. Qiu, Z. Zheng. H. Zhu, and E. Chen. Exploring large language model for graph data understanding in online job recommendations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 9178-91888 2024.
[58] L. Wu, Z. Zheng, Z. Qiu, H. Wang, H. Gu, Shen, Qin, C. Zhu, H. Zhu, Q. Liu, et al. A survey on large language models for recommendation. World Wide Web, 27(5):60, 2024.
[59] Z. Yang, J. Wu, Y. Luo, J. Zhang, Y. Yuan, A. Zhang, X. Wang, and X. He. Large language model can interpret latent space of sequential recommender. arXiv preprint arXiv:2310.20487, 2023.
[60] C. Zhang, Q. Han, R. Chen, X. Zhao, P. Tang, and H. Song. Ssdrec: Self-augmented sequence denoising for sequential recommendation. arxiv preprint arXiv:2403.04278, 2024.
[61] L. Zhang, J. Song, A. Gao, J. Chen, C. Bao, and K. Ma. Be your own teacher: Improve the performance of convolutional neural networks via self distillation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3713-3722, 2019.
[62] W. Zhang, X. Li, Y. Wang, K. Dong. Y. Wang, X. Dai, X. Zhao, H. Guo, R. Tang, et al. Tired of plugins? large language models can be end-to-end recommenders. arXiv preprint urXiv:2404.000 00702, 2024.
[63] W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang. Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong, et al. A survey of large language models. arXiv preprint Xi..:23333.122.. 2023.
[64] Z. Zhao, W. Fan, J. Li, Y. Liu, X. Mei, Y. Wang, Z. Wen, F. Wang. X. Zhao, J. Tang, et al. Recommender systems in the era of large language models (Ilms). IEEE Transactions on Knowledge and Data Engineering, 2024.
[65] Z. Zheng, W. Chao, Z. Qiu, H. Zhu, and H. Xiong, Harnessing large language models for text-rich sequential recommendation. In Proceedings of the ACM on Web Conference 2024, pages 3207-3216, 2024.
[66] K. Zhou, H. Yu, W. X. Zhao, and J.-R. Wen. Filter-enhaceed mlp is all you need for sequential recommendation. In Proceedings of the ACM web conference 2022, pages 2388-2399, 2022.
14

---

A Supplement to Method
In this section, the details of prompt design and the procedures of LLM-ESR are addressed.
A.1 Prompt Design
In Section 3.2 and Section 3.3 we format the attributes of items and historical interactions of users into textual prompts, for their semantic embeddings by LLMs. During the process of constructing prompts, the templates play vital role. Here, the templates are listed as follows.
Item Prompt Template. The templates mainly organize the attributes and descriptions of items, which vary across distinct datasets due to different recorded attributes. In the following templates, the words underlined are the corresponding attributes that will be filled in.
Item Prompt Template (Yelp)
The point of interest has the following attributes:
name <NAME>; category is <CATEGORY>; type is <TYP>>; open status <OPEN>; review count is <ČOUNT>; city is <CITY>; average score is <STARS>,
Item Prompt Template (Fashion)
The fashion item has the following attributes:
name is <TITLE>; brand <BRAND>; score is <DATE>; price <PRICE> The item has the following features: <FEATURE>. The item has the following descriptions: <DESCRIPTIONS.
Item Prompt Template (Beauty)
The beauty item has the following attributes:
name is <TITLE>; brand is <BRAND>; price is <PRICE> The item has the following features: CATEGORIES.. The item has the following descriptions: <DESCRIPTION>
User Prompt Template. This template mainly organizes the items that the user has interacted with. To utilize the semantic information and avoid excess the limitation of input length, the item the prompt is represented by its title. Besides, the three datasets share unique template.
User Prompt Template
The user has visited the following items: <ITEM1 <ITEMI_TITLE> <ITEM2 TIMME2,TTTEE...... please conclude the user's preference.
A.2 Train and Inference Process
For clearer illustration of the training and inference process, we conclude them in Algorithm First, the
and backbone SRS model are specified (lines 1-3). Then, organize the attributes of items and historical interactions into textual prompts get their semantic embeddings (line 4). At the beginning of the training, we initialize the embedding layers in the dual-view framework (line 5). Next, calculate the ranking loss by dual-view modeling (lines 7-9) and auxiliary loss by retrieval augmented self-distillation (lines 10-11). Through the sum of these two losses (line 12), we can optimize the whole M.M--SR. During the inference, only the dual-vies -iew modeling process conducted to get the final recommendations (lines 16- 16-17).
15

---

Algorithm Train and inference process of LLM-ESR
1:
Indicate the backbone sequential recommendation model fa.
2: Indicate the number of retrieved similar users N.
3: Indicate the weight of self deff-disiillation loss
4: Get the semantic embeddings Ese and Ulm by LLMs.
Train Process
5: Initialize the embedding layers in the dual-view framework by the raw and dimension-reduced
Ese- Freeze the raw Ege-
6: for batch of users Ug in 2/ do
7: Get the user preference representation in semantic and collaborative views, i.e., u* and u
respectively.
8: Calculate the probability score of ground-truth and negative items by Equation
9: Calculate the ranking loss by Equation (6).
Retrieve the similar users for each user in Ugg by Equation
11:
Calculate the self ceff-disillation loss by Equation (9).
Sum the ranking loss and self-distillation loss. Then, update the parameters.
13:
end for
Inference Process
Load Ege for item embedding layers and other trained parameters. for each user Uy in 2/ do Get the user preference representation in semantic and collaborative views, i.e., u and u
15:
17:
Calculate the probability score of each candidate item by Equation (5) and give out the final recommended list.
18: end for
B Experimental Settings
In this section, we will refer more details about the experimental settings.
B.1 Dataset and Preprocessing
The comprehensive experiments in this paper conducted three common- used datasets, i.e., Yelp, Fashion and Beauty. Yelp is the dataset that records the check-in histories and corresponding reviews of users. We only adopt the check-in data and the attribute information of the point-of-interests. Amazor
42 a large e-commerce dataset, which includes user's reviews on commodities. There of them, i.e., Fashion and Beauty.
several sub-categories this dataset and
For preprocessing, we refer to the procedures in SASRec [8]. Since the sequential recommendation often utilized for implicit interactions, consider all review rate records interactions. Then, the users with fewer than three interacted items are dropped, because we do not explore the problem of cold-start users in this paper. As for the data split, the last item Un., and the penultimate item Vn,-1 of each interaction sequence are taken out as the test and validation, respectively. The statistics the three preprocessed datasets shown in Table B
Table 3: The statistics of the preprocessed datasets
Dataset
# Users
# Items
Sparsity
Avg.length
Yelp
15,720
11.383
99.89%
12.23
Fashion
9,049
4,722
99.92%
3.82
Beauty
52,204
57,289
99.99%
7.57
https://cseweb.. ucsd. edu/ jjncaully/dattastss htm.HHamzzon_reviee
16

---

B.2 Backbone and Baseline
Backbone Models. To show the flexibility of our enhancement method, we test three popular sequential recommendation models in the experiments, The main distinction between these models refers to the sequence encoder To and ranking loss Lannk-
GRU4Rec 14. It adopts the GRU as the sequence encoder, and sequence-to-one pairwise loss as the final ranking loss.
Inspired by the training pattern of Bert [9,, this backbone proposes a combination between pairwise ranking loss and cloze task, which mask a proportion of items in one sequence. The sequence encoder of Bert4Rec is the stack of bi-directional self-attention layers.
Bert4Rec 49
Compared with Bert4rec, SASRec adopts the causal self-attention layer as the basic unit of its sequence encoder. Besides, the peuuence-to-sqqueee pairwise ranking loss is applied for optimization during the training.
SASRec [8
There are two groups of up-to-date baselines that are compared within this paper, i.e., traditional baselines and LLM-based baselines.
Traditional Baselines. This category split the users and items into long-tail and head groups at first. Then, they enhance the long-tail users or items by fabricated training procedures. Note that they only utilize the collaborative signals essentially and do not introduce any semantics.
CITTES [7].
This work devises an embedding-inferene function refine the embeddings of long-tail items specially. Such embedding-inferennt function is trained by head items and used for long-tail items during inference. We follow the pareee-pr in the original paper and code
MELT 20 MELT proposes a bilateral- framework enhance the long-tail users and items. One branch is trained to generate the head user representations and enhance the tail users while inference. The other branch to recover the embeddings of head items during training and update embeddings of tail items during inference. We refer the implementation and the hyper-parameter settings in official code
LLM-based Baselines. The methods in this line aim to combine the semantic information derived from LLMs to enhance the recommendation models.
This baseline is one of the pioneers utilizing the semantic embeddings derived from LLMs. However, it is designed for collaborative filtering but not sequential recommendation. For fair comparison, we eliminate the process of profile generation during the implementation. We refer to the source code of RLMRec to adapt it to sequential recommendation models.
RLMRec 45
LLMInit [3] 16]. More recent works, i.e., LLM2Bert4Rec [13) and SAID [16]. both utilize the LLMs embedding to initialize the item embedding layer in SRS models and then fine-tune by interaction data. In this paper, we dub this way as LLMInit.
B.3
Implementation Details
We conduct all experiments on an Intel Xeon Gold 6133 platform with Tesla V100 32G GPUs. Besides, the implementation is based on Python 3.9.5 and PyTorch 1.12.0. In terms of the hyper- parameter search, the criterion N@ on the validation set. To avoid overfitting, we adopt the early stop strategy with 20-epoch patience. For the backbone SRS models, the number of GRU layers is set to for GRU4Rec, while the number of self-attention layers is fixed at for SASRec and Bert4Rec. Also, the dropout rate is 0.6 for Bert4Rec. In terms of the training, the batch size and learning rate are set as 128 and 0.001 for all datasets. The embedding size is 128 for all baselines, while 64 for LLM-ESR. The reason is that there are two branches in LLM-ESR, and the half size of the other branch baseline is a fair setting. Then, we choose the Adam as the optimizer. The hyper- parameters N and for L.M-ESR are searched from (2, 6, 10, 14, 18) and (1, 0.5, 0.1, 0.05, 0.01). We find that the best choice 10 for and 0.1 for & for all three datasets used in this paper.
Suagss//gihubbccm///www///////
baunss//gthub.coo///UUUUUAAAAL
17

---

Table 4: The ablation study on the Yelp dataset with Bert4Rec as the backbone SRS model. The
boldface refers to the highest score and the underline indicates the next best result of the models.
Overall
Tail Item
Head Item
Tail User
Head User
Model
H 10
N@ 10
H@ 10
N@ 10
H@ 10
N@ 10
H@
N@
H@ 10
N@ 10
-LLM-ESR
0,6623
0.4222
0.1227
0.0500
0.8212
0,5318
0.6637
0.4247
0,6571
0,4127
w/o Co- Co-view
0.6273
0.3737
0.1272
0.0520
0.7745
0.4684
0.6296
0.3760
0,6184
0.3647
w/o Se-view
0.6521
0.4125
0.0981
0.0395
0,8153
0.5224
0.6533
0.4150
0.6477
0.4031
-wo SD
0.6539
0.4114
0.1299
0.0534
0.8081
0,5168
0.6539
0,4129
0,6538
0.4055
w/o Share
0.6592
0.4193
0.1182
0.0480
0.8187
0.5276
0.6619
0.4229
0.6482
0.4100
w/o CA
0.6368
0,3924
0.0940
0.0369
0.7966
0.4971
0.6369
0.3940
0.6367
0,3862
Table 5: The ablation study on the Yelp dataset with GRU4Rec as the backbone SRS model. The boldface refers to the highest score and the underline indicates the next best result of the models.
Overall
Tail Item
Head Item
Tail User
Head User
Model
H@10
N@10
H@ 10
N@10
H@ 10
N@ 10
H@
N@10
H@10 10
N@ 10
LLM-ESR
0.5724
0.3413
0.0763
0.0318
0.7184
0.4324
0,5782
0,3456
0.5501
0.3247
w/o Co-view
0,5660
0.3263
0.0831
0.0331
0.7022
0.4097
0,5720
0,3310
0.5530
0.3188
w/o Se-view
0.5273
0.3091
0.0441
0.0187
0.6695
0,3945
0.5316
0.3122
0.5107
0.2971
w/o SD
0,5562
0.3236
0.0720
0.0309
0.7028
0.4053
0.5605
0.3371
0.5498
0.3203
w/o Share
0.5661
0.3353
0.0789
0.0325
0.7124
0.4274
0,5689
0.3297
0.5536
0.3285
w/o CA
0.5657
0.3327
0.0736
0.0304
0.7126
0,4128
0.5755
0,3426
0.5493
0.3227
Furthermore, the embeddings of LLMs are derived from the AP named "lext-add-cmmbcddin---0" provided by OpenAI.
C More Experimental Results
In this section, we will show more experimental results to further analyze the flexibility and effective- ness of our LLM-ESR.
C.1 Ablation Study
For further analysis, we conduct the ablation study on the proposed L.M--SSR with Bert4Rec and GRU4Rec as the backbone SRS models. The results are shown in Table 4 and Table At first, we probe the effects of dual-view modeling by removing one of the views, denoted w/o Co-view and w/o Se-view. From the overall performance, these two variants both underperform, which indicates the essence of the dual-view. Besides, w/o Co-view downgrades the accuracy of the head item group more, while w/o So-view harms the long-tail item group compared with LLM-ESR. This phenomenon highlights the advantages of the collaborative view and semantic view, respectively. As for distinet SRS backbone models, we find that Bert4Rec benefits more from collaborative information, because removing the collaborative view causes a more severe performance drop. By comparison, GRU4Rec can get more enhancement from the semantic view. Then, w/o SD means eliminating self-distillation. It downgrades the performance of the tail user group consistently, which indicates the proposed retrieval augmented self doff-distillation can actually help alleviate the long-tail user challenge. w/o Share represents using separate sequence encoders for the dual views. This variant is little worse than applying a shared encoder, illustrating the common pattern for both views. Another advantage of the shared encoder is higher parameter efficiency. Besides, LLM--SSK without cross-attentic (w/o CA) is inferior to LLM-ESR totally, which indicates the effectiveness of the sequence- fusion.
At the same time, it is risky to overfit with semantic embeddings when the textual data is scarce. To validate the robustness of our LLM-ESR, we conduct additional experiments in scenarios with limited textual data. To simulate this situation, removed all attributes from the item descriptions except for "name" " and "categories" when constructing the textual prompts for the Yelp dataset (originally using 8 attributes). This reduced the average word count of the textual prompts from 38.38 to 20.33. We used SASRec as the backbone model in these supplementary experiments, with results presented
Suaps://pp.oopenii.ccm////eeeeee wtw.://dd///ee//////dde.m..
18

---

Table 6: The experiments for limited text and the design of freezing semantic embedding. All the experiments are conducted on the Yelp dataset and for LLM-ESR. "Full" and "Crop" mean that we use the completed item prompt and attribute-cropped prompt to get the L.MM embeddings, respectively. "w/o F" means that we train the LLM-ESR without freezing the semantic embedding layer.
Overall
Tail Item
Head Item
Tail User
Head User
Model
H@ 10
1@
H@ 10
N@ 10
H@ 10
N@ 10
H@ 10
N@ 10
H@ 10
N@ 10
Full
0.6673
0.4208
0.1893
0.0845
0,8080
0.5199
0.6685
0.4229
0.6627
0.4128
Full w/o
0.6069
0.3664
0.1284
0.0541
0.7477
0.4584
0.6028
0.3647
0.6226
0.3730
Crop
0.6477
0.4046
0.1478
0.0675
0.7807
0.4968
0.6468
0,4058
0.6511
0.3998
Crop w/o F
0.6025
0.3630
0.1247
0.0563
0.7432
0.4563
0.6004
0.3615
0.6109
0.3786
in Table 6 In the table, Full and Crop represent the use of the complete and cropped prompts, respectively. w/o F denotes training LLM-ESR without freezing the semantic embedding layer. The results show decrease in performance for both Full and Crop due to the limited textual prompt. Moreover, Full w/o F and Crop w/o F yield similar results, indicating that semantic embeddings suffer from overfitting with both complete and cropped prompts. In contrast, freezing the semantic embedding layer improves performance in both scenarios and significantly benefits long-tail items, demonstrating that our design effectively alleviates the overfitting issue.
C.2 Visualization
To further investigate how LLMs enhance the traditional SRS models, we visualize the item embed- dings of SASRec, CITIES, MELT, LLM-ESR (concatenate the semantic embedding e and collaborative embedding e and L.M. using t-SNE, as shown in Fi We group the items into four categories based on their popularity. The t-SNE figures reveal that the embeddings of SASRec, CITIES, and MELT tend to cluster according to item popularity. In contrast, the distribution of LLM embeddings more uniform, indicating that the semantic relationships Furthermore, the embeddings of our LLM-ESR also show more even distribution, validating that our method effectively corrects the embedding distribution SRS and thus can enhance the performance of long-tai tail items.
not skewed by popularity.
1-4
5-9
10-29
30
(a) SASRec
(b) CITIES
(c) MELT
(d) LLM-ESR
(e) LLM
Figure 5: The visualization of the item embeddings by t-SNE. The dataset used in the experiments is Yelp. "CITIES", "MELT" and "LLM-ESR" are all based on the SASRec backbone model. "LLM" represents the embeddings derived from LLM, which encodes the semantics of textual item prompts. Different colors of circles shown in the figures mean different popularity groups of the item.
C.3
Group Analysis
For more meticulous analysis of to what extent the proposed LLM-ESR alleviates the long-tail challenges, we categorize users and items into groups. The performances of each method with Bert4Rec and GRU4Rec as backbone models are shown in Table 6 Tabel respectively. Firstly, we analyze the results in different user groups. Undoubtedly, all methods perform worse for those users with fewer interactions, which highlights the long-tail user challenge. MELT can enhance the Bert4Rec well so that the performances all groups get increased, but is incompatible with GRU4Rec and thus harms several groups. By comparison, L.Mllni and our LLM-ESR can benefit all user groups consistently. Due to the better utilization of semantics from L.Mss, LLM-ESR can outperform LLMInit evidently. Besides, the superiority is larger for more long-tailed users, i.e., 1-4 and 5-9
19

---

Bert4Rec
MELT
LLMInit
a. LLM-ESR
0.7
1.0
0.8
0.6
HR@10
HR@10
0.6
0.5
0.4
0.4
0.2
0.3
0.0
1-4
5-9
10-14
15-19
20+
1-9
10-29
30
59
60-99
100+
(a) User Group
(b) Item Group
Figure 6: The results the proposed LLLM-SSR and competing baselines in meticulous user and item groups. The results are based on the Beauty dataset with the Bert4Rec model.
GRU4Rec
MELT
LLMInit
LLM-ESR
0.6
1.0
0.8
0.5
HR@10
HR@10
0.6
0.4
0.4
0.3
0.2
0.2
0.0
1-4
5-9
10-14
15-19
20+
1-9
10-29 59 60-99
100+
(a) User Group
(b) Item Group
Figure 7: The results the proposed LLM-ESR and competing baselines in meticulous user and item groups. The results based on the Beauty dataset with the GRU4Rec model.
user groups. As for the item groups, MLET, LLMInit and LLM-ESR all elevate the recommending accuracy for long-tail tail items, but get slight drop for popular items. Such phenomenon indicates trade-off between head and tail items. Despite that, larger increments for long-tail items of these methods result in an advance in overall performance. Also, the proposed L.LM-ESR leads 1-9 item group observably, which means it can alleviate the long-tail item challenge better.
D Limitation
Two potential limitations should be considered for this paper. Firstly, there are two hyper- paper-prrameters for the proposed LLM-ESR, i.e., the weight of self-distillation loss and the number of retrieved similar users N, which time-consuming to search for the best model. Secondly, only the LLMs embedding provided by OpenAl API is validated in the experiments, but other more recent models B 53 may lead to better performance. Nonetheless, the experiments on various datasets and backbone models consistently validate the effectiveness of LLM-ESR
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The contributions and scope of the paper are included in the abstract and Section I Please refer to the first and last paragraph of Section T for scope and contributions, respectively.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
limitation section is included in the appendix (Section D
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sellsspeciiiationn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
21

---

Answer:
[NA]
Justification:
There is no theoretical result in this paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We introduce the details of the experiment, such as the information hardware and software, in the implementation detail section, i.e., Section B.3 in the appendix. Besides, also release the code to case the reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
22

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
We have attached the data and code used in this paper in the supplementary material.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide the details of the experimental settings, such as the data split, optimizer, etc., in the experimental setting section (Section I in the main paper and the implementation detail section (Section B.3) in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report the two-sided t-test with p < 0.05 results in the main experiments, i.e., Table
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
23

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide the details of compute resources in the implementation detail section (Section B.3) in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httppp////eerrppppppppppbtttitie
Answer: [Yes]
Justification:
We have made sure that paper conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss the potential positive impacts that our algorithm will bring in the Introduction section, i.e., Section
Guidelines:
The answer NA means that there is no societal impact of the work performed.

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: There is no risk of misuse of the proposed method and the datasets used in the paper are open-sourced.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have cited the original paper or attached the link to the existing assets used in this paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
25

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We have attached the introduction of how to run the code and the license in the code repository.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
26

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
27


Reviews:
[{"review_id": "Czd8ymMRG3", "reviewer": "Reviewer_V8zP", "summary": "The paper addresses the challenges in sequential recommender systems (SRS), particularly the long-tail user and long-tail item issues, which complicate user experience and seller benefits in real-world applications. The authors propose the Large Language Models Enhancement framework for Sequential Recommendation (LLM-ESR) to mitigate these challenges. The LLM-ESR framework leverages semantic embeddings derived from large language models (LLMs) to enhance SRS without increasing inference load. To tackle the long-tail item problem, the framework employs a dual-view modeling approach that integrates semantics from LLMs with collaborative signals from traditional SRS. For the long-tail user issue, a retrieval augmented self-distillation method is introduced to improve user preference representation by utilizing more informative interactions from similar users.", "questions": "- In line 44, the authors mention that existing studies perform poorly due to \"ignorance of the true relationship between items.\" What is the true relationship between items, and how does it affect recommendations?\n    \n- Although SASRec is a classic model, it is not reasonable to conclude that all SRSs perform poorly in long-tail scenarios solely based on SASRec. Have the authors analyzed why SASRec performs poorly in long-tail scenarios? Do models that use other techniques specifically for long-tail scenarios have this problem? What are their limitations?", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The work includes extensive experiments, testing multiple aspects of the model's capabilities.\n    \n- The approach is quite new. Recommender systems based on LLMs are a promising direction.", "weaknesses": "- The paper does not sufficiently and deeply discuss existing work, making the motivation and core idea of the paper seem less convincing, and the innovation of the paper is also insufficient.\n    \n- The baselines used in the experiments are limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the meticulous and insightful comments to help us polish the paper. Please find the point-to-point responses to the reviewer's concerns.\n\n> W1 & Q1\n\nWe greatly appreciate the reviewer's suggestion on refining the motivation of our paper. Existing research on long-tail issues, including CITIES [1] and MELT [2], aims to enhance the representation of long-tail items through similar popular items. However, these approaches rely on co-occurrence patterns, which overlook the semantic (expressed as \"true\" in the current paper) relationships between items as highlighted in our paper. Co-occurrence records inherently have an uneven distribution, often skewing item embeddings toward popularity and exacerbating the long-tail problem. In contrast, semantic relationships encoded by LLMs are unaffected by item popularity, which motivates our use of LLMs to correct the embedding distribution in SRS.\n\nFor illustration, we visualized the item embeddings of SASRec, CITIES, MELT, our LLM-ESR (concatenate the semantic embedding $\\mathbf{e}^{se}$ and collaborative embedding $\\mathbf{e}^{co}$), and LLM using t-SNE, as shown in __Figure 1__ of the __Rebuttal PDF__. We grouped the items into four categories based on their popularity. The t-SNE figures reveal that the embeddings of SASRec, CITIES, and MELT tend to cluster according to item popularity. In contrast, the distribution of LLM embeddings is more uniform, indicating that the semantic relationships are not skewed by popularity. Furthermore, the embeddings of our LLM-ESR also show a more even distribution, validating that our method effectively corrects the embedding distribution in SRS and enhances the performance of long-tail items. \n\nWe commit to including the supplementary distribution experiments and refining the illustration of our motivation in the revised paper. \n\n[1]. Jang, Seongwon, et al. Cities: Contextual inference of tail-item embeddings for sequential recommendation. 2020 IEEE International Conference on Data Mining (ICDM). IEEE, 2020. \n\n[2]. Kim, Kibum, et al. MELT: Mutual Enhancement of Long-Tailed User and Item for Sequential Recommendation. Proceedings of the 46th international ACM SIGIR conference on Research and development in information retrieval. 2023.\n\n> W2 & Q2\n\nThank you for the advice to illustrate the limitations of existing methods more clearly. \n\n1. Due to space limit, we present the preliminary experimental results on SASRec in the current paper. In __Figure 5__ and __Figure 6__ of __Appendix C.2__, we display the grouped performance of GRU4Rec and Bert4Rec based on item popularity and user interaction count. These results demonstrate that Bert4Rec and GRU4Rec also exhibit unsatisfactory performance for items and users with fewer interaction records, verifying that they also suffer from long-tail problems, not just SASRec.\n2. As mentioned in our response to Q1, the main reason for long-tail problems lies in the skewed distribution of item embeddings. Additionally, to illustrate that other SRS models also have the long-tail issue, we present the item embedding distributions of GRU4Rec and Bert4Rec in __Figure 2__ of the __Rebuttal PDF__. The results indicate that the embeddings of GRU4Rec and Bert4Rec are also clustered according to item popularity. Our LLM-ESR method helps alleviate these skewed distributions, thereby enhancing the performance of long-tail items.\n3. MELT and CITIES are two significant works designed for long-tail scenarios. We have compared their performance in __Table 1__ and conducted a more detailed group analysis of their long-tail results in __Figure 4__ of the current paper. The results demonstrate that the proposed LLM-ESR consistently outperforms these two long-tail techniques. As mentioned in response to Q1, their main limitations lie in their reliance on co-occurrence records to enhance long-tailed items and users, which did not address the skewed embedding distribution issue (as shown in __Figure 1 (b)__ and __(c)__ in the __Rebuttal PDF__).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qjDnoRE8io", "reviewer": "Reviewer_wsRH", "summary": "This paper introduces a novel framework designed to address the long-tail challenges in sequential recommendation systems (SRS). By leveraging semantic embeddings from large language models (LLMs) and combining them with collaborative signals, the authors propose a dual-view modeling framework and a retrieval-augmented self-distillation method. This approach aims to enhance recommendations for both long-tail users and items without adding significant inference load. Extensive experiments on three real-world datasets demonstrate the effectiveness of the proposed framework.", "questions": "1.\tCould the authors provide more details on the practical implementation challenges and how they can be mitigated?\n2.\tHow does the framework handle highly diverse user interactions where finding similar users may be challenging?\n3.\tBalanced Performance: What measures have been taken to ensure that the enhancement for long-tail users and items does not adversely affect recommendations for popular items?", "limitations": "Discussing potential negative societal impacts, such as reinforcing biases in recommendations, would be beneficial.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1.\tThe paper successfully integrates LLMs with SRS to address long-tail challenges, a novel approach that leverages the semantic understanding of LLMs while maintaining low inference costs.\n2.\tThe dual-view modeling framework effectively combines semantic and collaborative signals, providing a comprehensive enhancement for SRS.\n3.\tThis method innovatively uses interactions from similar users to enhance user preference representation, addressing the long-tail user challenge.\n4.\tThe proposed framework is model-agnostic and can be adapted to any sequential recommendation model, making it highly applicable in real-world scenarios.", "weaknesses": "1.\tThe proposed dual-view and self-distillation methods add layers of complexity to the SRS, which may pose challenges in practical implementation.\n2.\tThe framework assumes a certain level of similarity in user interactions, which might not hold true for highly diverse user bases.\n3.\tImpact on Popular Items: While the focus is on long-tail items and users, the potential impact on recommendations for popular items is not thoroughly explored.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate that the reviewer has raised these valuable questions to help us refine our paper. We have discussed these questions carefully and responded to them as follows.\n\n> W1 & Q1\n\nWe appreciate the reviewer's advice on providing more details regarding practical implementation. There are two challenges we encountered during the implementation:\n\n1. __Extra Inference Burden__: The dual-view modeling and self-distillation modules may increase the inference burden on SRS models. To address this challenge, we propose two efficiency solutions:\n\n   - To reduce the size of the semantic embedding layer ($|\\mathcal{V}|\\times d_{llm}$), we cache all the semantic embeddings transformed by the adapter before inference. Consequently, only the reduced semantic embeddings ($|\\mathcal{V}|\\times d$) need to be loaded during inference.\n   - The self-distillation module is eliminated during inference, as it only provides guidance for training.\n\n   With our efficient implementation, LLM-ESR introduces only a minimal additional inference burden. Originally, the model size of LLM-ESR (SASRec as the backbone) was 20.3M, and it required 12.92 seconds to test all users on the Yelp dataset. However, after applying our implementation tricks for inference, the size and time consumption reduced to 2.32M and 12.62 seconds, respectively (compared to 1.70M and 11.94 seconds for SASRec).\n\n2. __Optimization Difficulty__: Due to the complexity of dual-view modeling and self-distillation, optimizing the entire framework can be challenging. We propose two implementation solutions to significantly alleviate this difficulty:\n\n   - The semantic and collaborative embedding layers are trained in distinct stages, resulting in optimization difficulty. Thus, we use dimension-reduced LLM item embeddings to initialize the collaborative embedding layer instead of random initialization.\n   - The dimension of LLM embeddings is much larger than the semantic embedding $\\mathbf{e}^{se}$, making convergence difficult when the adapter is designed as a single linear layer. We found that a two-layer design significantly alleviates this problem. Specifically, we first reduce the dimension to half of $d_{LLM}$ and then to the item embedding size $d$.\n\n   To illustrate the effectiveness of these two implementation solutions, we compared one-layer adapter and random initialization variants of LLM-ESR in supplementary experiments. The results, shown in __Table 2__ of the __Rebuttal PDF__, indicate that both variants underperform the original LLM-ESR, verifying the success of our special designs.\n\nWe will highlight these implementation details in the revised version of our paper.\n\n> W2 & Q2\n\nWe appreciate the reviewer's insight into the challenge of finding similar users with highly diverse interactions.\n\nOn one hand, LLMs have been shown to effectively understand user interactions through textual prompts [1,2]. Therefore, LLM embeddings of user interactions can serve as robust semantic representations of users. This suggests that embeddings generated by LLMs can assist in retrieving truly similar users.\n\nOn the other hand, we acknowledge the challenge of identifying similar users due to their diverse interactions. To mitigate the adverse effects of users with diverse interactions, we average the representations of the top-N retrieved users to guide the self-distillation process. The hyper-parameter experiments shown in __Figure 3__ partially validate the reviewer's concern and the effectiveness of our design.\n\nThe results indicate that LLM-ESR performs sub-optimally when N is relatively small (e.g., N=2), suggesting that inaccurate similar users can negatively impact performance. However, as N increases from 2 to 10, performance improves continuously, demonstrating that our strategy of using top-N users helps mitigate these adverse effects.\n\nAdditionally, we will include a discussion on this topic in the revised paper to clarify our top-N retrieval design and its benefits.\n\n[1]. Harnessing large language models for text-rich sequential recommendation. ACM on Web Conference 2024. \n\n[2]. Tallrec: An effective and efficient tuning framework to align large language model with recommendation. 17th ACM Conference on Recommender Systems. \n\n> W3 & Q3\n\nThank you for your advice on evaluating popular items and experienced users for our LLM-ESR.\n\nIn our overall experiments (__Table 1__ in the paper), we categorize users and items into tail and head groups. Here, \"Head User\" and \"Head Item\" refer to popular items and experienced users, respectively. The results show that our LLM-ESR not only improves the performance for long-tail users and items but also enhances the performance of head users and head items compared to all baseline methods.\n\nFurthermore, for a more detailed analysis of balanced performance, we tested LLM-ESR across more granular groups in __Section 4.5__. The results in __Figure 4__ demonstrate that while LLM-ESR slightly affects the performance of extremely popular items, it benefits all other groups. These findings confirm that LLM-ESR has minimal adverse effects on popular items and experienced users.\n\n> Limitation\n\nWe really appreciate your suggestion on discussing potential societal impacts. For the long-tail item issue, only recommending popular items easily causes the filter bubble problem for users, which may skew one's perspective and trap them in a small range of content. For the long-tail user issue, the ignorance of some minority groups, such as elders, may be intensified due to their less activity but it leads to more serious unfairness to the society. Thus, handling the long-tail problems in RS is essential to our society. Due to word limitations, we will add more discussions to the revised paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pNStOGMM2r", "reviewer": "Reviewer_Bjnq", "summary": "The paper presents a framework that integrates large language models (LLMs) into sequential recommendation systems (SRS) to tackle the long-tail challenges. The framework includes dual-view modeling, which combines semantic embeddings from LLMs with collaborative signals, and a retrieval-augmented self-distillation method to enhance user preference representation. The authors validate their approach through extensive experiments on three real-world datasets, demonstrating significant improvements over existing methods.", "questions": "1) How do the authors mitigate the risk of overfitting with semantic embeddings, especially in scenarios with limited textual data?\n2) Can the authors elaborate on the hyper-parameter tuning process and its impact on the performance?", "limitations": "The authors have addressed several limitations, but there is room for more in-depth discussion on potential biases introduced by semantic embeddings and the sensitive to the choice of hyper-parameters.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1)\tThe dual-view modeling and retrieval-augmented self-distillation methods are novel contributions that enhance the performance of SRS.\n2)\tUtilizing LLMs to derive semantic embeddings for items and users adds a new dimension to the traditional collaborative filtering methods.\n3)\tThe extensive experimental evaluation, including comparisons with multiple baselines and ablation studies, strengthens the validity of the findings.\n4)\tThe paper provides comprehensive details on the methodology, including mathematical formulations and algorithmic steps, facilitating reproducibility.", "weaknesses": "1) There is a risk that the semantic embeddings might overfit to the training data, especially if the textual descriptions are not diverse enough.\n2) The performance of the framework might be sensitive to the choice of hyper-parameters, which is not extensively explored in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's valuable time and insightful suggestions, which are important to our paper. We deliver the point-by-point response as follows.\n\n> W1 & Q1\n\nThank you for highlighting the potential overfitting issue when textual descriptions are not diverse enough. We agree with the reviewer’s concerns. When textual prompts lack diversity, semantic embeddings may overfit the collaborative signals present in the training data. To address this, our paper proposes freezing the semantic embeddings and designing a trainable adapter during the training of LLM-ESR, which helps maintain subtle semantic differences between items.\n\nIn response to the reviewer's suggestion, we conducted additional experiments in scenarios with limited textual data. To simulate this situation, we removed all attributes from the item descriptions except for \"name\" and \"categories\" when constructing the textual prompts for the Yelp dataset (originally using 8 attributes). This reduced the average word count of the textual prompts from 38.38 to 20.33. We used SASRec as the backbone model in these supplementary experiments, with results presented in __Table 2__ of the __Rebuttal PDF__. \nFor convenience, we briefly show that the overall HR@10 of LLM-ESR with the original prompt changes from 0.6673 to 0.6069 when not freezing the semantic embedding layer. By comparison, the HR@10 of LLM-ESR with the limited prompt changes from 0.6477 to 0.6025.\n\nIn the table, __Full__ and __Crop__ represent the use of the complete and cropped prompts, respectively. __w/o F__ denotes training LLM-ESR without freezing the semantic embedding layer. Compared with Full, the results show a decrease in performance for Crop due to the limited textual prompt. Moreover, Full w/o F and Crop w/o F yield similar results, indicating that semantic embeddings suffer from overfitting with both complete and cropped prompts. In contrast, freezing the semantic embedding layer improves performance in both scenarios and significantly benefits long-tail items, demonstrating that our design effectively alleviates the overfitting issue.\n\nWe will include these experiments and analyses in the ablation study of our revised paper to further validate the effectiveness of LLM-ESR.\n\n> W2 & Q2\n\nWe appreciate your concern regarding the hyper-parameters of our LLM-ESR. The key hyper-parameters are the number of similar users used for self-distillation ($N$) and the scale of the self-distillation loss ($\\alpha$). We performed a grid search to optimize these hyper-parameters in the experiments. Actually, the results of this hyper-parameter tuning have been presented in __Figure 3__ of the current paper, with a detailed analysis in __Section 4.4__.\n\nThe hyper-parameter $\\alpha$ determines the extent to which self-distillation influences the optimization process. As $\\alpha$ varies from 1 to 0.01, the recommendation accuracy initially improves and then declines. Larger values of $\\alpha$ can lead to overemphasis on self-distillation, negatively impacting the convergence of the ranking loss. Conversely, smaller values of $\\alpha$ reduce the effectiveness of self-distillation, highlighting its importance for our LLM-ESR.\n\nRegarding the number of retrieved users $N$, the optimal value is found to be 10. This is because a higher number of users provides more informative interactions, which helps mitigate the adverse effects of users with diverse interactions. However, if $N$ is too large, it may reduce the relevance of the retrieved users and lead to performance degradation.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We really appreciate your valuable time and insightful suggestions. The referred figures and tables of results in the rebuttal are included in the supplement PDF (i.e., __Rebuttal PDF__).

Meta-review: Unanimous and solid reviews. Reviewers praise the experiments, approach, and relevance in real-world settings. Issues raised are mainly clarifications and seem fairly minor.
Decision: Accept (spotlight)