R$^2$ec: Towards Large Recommender Models with Reasoning
Authors: Runyang You
Yongqi Li
Xinyu Lin
Xin Zhang
Wenjie Wang
Wenjie Li
Liqiang Nie
Venue: NeurIPS 2025 poster ()
Keywords: Generative Recommendation
LLM-based Recommendation
Primary area: other
TL;DR: We develop a unified large recommender model with intrinsic reasoning capabilities.

Abstract: Large recommender models have extended LLMs as powerful recommenders via encoding or item generation, and recent breakthroughs in LLM reasoning synchronously motivate the exploration of reasoning in recommendation. 
In this work, we propose R$^2$ec, a unified large recommender model with intrinsic reasoning capability. 
R$^2$ec introduces a dual-head architecture that supports both reasoning chain generation and efficient item prediction in a single model, significantly reducing inference latency. To overcome the lack of annotated reasoning data, we design RecPO, a reinforcement learning framework that optimizes reasoning and recommendation jointly with a novel fused reward mechanism. 
Extensive experiments on three datasets demonstrate that R$^2$ec outperforms traditional, LLM-based, and reasoning-augmented recommender baselines, while further analyses validate its competitive efficiency among conventional LLM-based recommender baselines
and strong adaptability to diverse recommendation scenarios. Code and checkpoints available at https://github.com/YRYangang/RRec.

eeec: Towards Large Recommender Models with Reasoning
Runyang You' Yongqi Li Xinyu Lin?
Xin Zhang!
Wenjie Wang" Wenjie Li Liqiang Nie
'The Hong Kong Polytechnic University Nattoonal University of Singapore
University of Science and Technology of China
*Harbin Institute of Technology (Shenzhen)
runyang yeoutlook....oooo com, xy xylin10280gmail com, izhx4040gmail. com.
liyongg10@gmail.co
venjiewang960gmail. com, cswjli@comp wtt.:eoompppppllualloeoemmllmmma polyu.
eay..edu.... edu hk, nieliqiang@gmail. com
Abstract
Large recommender models have extended LLMs as powerful recommenders via encoding or item generation, and recent breakthroughs in LLM reasoning synchronously motivate the exploration of reasoning in recommendation. In this work, we propose R Rºec, unified large recommender model with intrinsic reasoning capability. Rdec introduces a dual-head architecture that supports both reasoning chain generation and efficient item prediction in a single model, significantly reducing inference latency. To overcome the lack of annotated reasoning data, we design RecPO, reinforcement learning framework that optimizes reasoning and recommendation jointly with a novel fused reward mechanism. Extensive experiments three datasets demonstrate that Reec outperforms traditional, LLM- based, and aassoning-augmenee recommender baselines, while further analyses validate its competitive efficiency among conventional LLM- M-based recommender baselines and strong adaptability to diverse recommendation scenarios. Code and checkpoints available at https: : //g/ /github.. com /YR/aagaaa///aaaa
Introduction
Large language models (LLMs) have demonstrated remarkable capabilities in contextual under- standing and open-ended generation [1-5]. This success has catalyzed the development of large recommender models that inherit and specialize in the advantages of LLMs for recommendation. Current approaches can be divided into two main streams: one employs LLMs as powerful encoders embed users (their historical interactions) [6-8], while the other reformulates item prediction into the autoregressive generation of item identifiers [4, 9]. These large recommender models exhibit remarkable generalization capabilities, achieving advanced performance across diverse application scenarios, including cold-start recommendations [10], cross-domain personalization [1] I], and long- tail item prediction [12, 13). The frontier of I.IM capabilities now extends beyond model size scaling to test-time scaling, iee., reasoning. Emerging advances like DeepSeek-R R1 [14] demonstrates that such extra computation during inference can further improve LLM capabilities areas such as mathematics, coding, and scientific problem-solving [15]. Given that large recommender models are instantiated from pretrained LLMs, a natural question is: how can large recommender model benefit from reasoning to further enhance recommendation performance?
Existing studies have preliminarily explored LLM reasoning for recommendation. including user preference analysis [16, 17], synthetic item profile enrichment [18, 19], user search query rewriting (20, 21], and rating prediction [22]. These approaches typically take L.MMs as additional reason-
"Yonggi Li is the corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Reasoning
Recommend
You
have enjoyedistory
icch
open
world
Assassin's Greed Shacows
mn_head
rec rec_head
h1
hgg
hq
he
hr
have
Large Recommender Model
Tybbrrpukk 20 7s
Recommand myyrent
Yowan Enibeiiiing
User
You
have enjoyed Story
iich
open
world
answer
The architecture overview of R ec, which facilitates interleaved reasoning and recom- mendation in an autoregressive process with two task-specific heads: 1) language-moke modeling head (Im_head) for reasoning generation; and 2) recommendation head (rec_head) for item prediction.
Figure 1:
ing modules to augment the original recommendation pipeline, decoupling reasoning from item prediction. However, such designs introduce crucial limitations: 1) Significant resource cost. Two distinct modules, namely the large reasoning model and the recommendation model, must be trained, checkpointed, and served in parallel, inflating both memory footprint and inference latency. 2) Sub- optimal joint optimization. Reasoning and recommendation modules can only be trained by freezing one while updating the other [16, 23J. This alternate optimization scheme prevents gradient flow across the pipeline, precluding true end-to-end learning, hindering fine-grained alignment between reasoning and ranking objectives, and ultimately leading to suboptimal convergence. In this work, we aim to develop unified large recommender model with intrinsic reasoning capabilities, exploring new technical solutions for reasoning-entarr enhanced recommendation systems.
It is non-trivial to develop unified large recommender model with reasoning due to the following aspects: 1) Model design. Most large recommender models rely on autoregressive decoding of item identifiers (IDs), which can be computationally expensive [1-4, 24-26]. Introducing reasoning would inevitably exacerbate the latency. Thus, a key challenge integrate reasoning while maintaining acceptable inference speed. 2) Training optimization. A major obstacle in training the reasoning recommender is the scarcity of high-quality, annotated reasonings in recommendation domains. Unlike general QA tasks where solution rationales can be curated, the underlying rationale for recommendation is inherently subjective and difficult to collect at scale. Reinforcement Learning (RL) [21, 27, 28] is natural alternative, but it introduces recommendattion-speiiii reward specification and objective design. The training framework must effectively couple the reasoning and recommendation objectives to ensure co-optimization.
challenges in
To address the above issues, we propose a unified large Recommender model with intrinsic Reasoning, dubbed as Reec. Our approach is built on two pillars: 1) Model design. As illustrated in Figure we equip an LLM with a dual-head design: standard language head for reasoning and novel recommendation head for item prediction. The recommendation head maps the model's output to a shared semantic item embedding space. This allows the model to first generate a reasoning chain autoregressively and then select the final item in a single step, replacing the slow rugo-segressive decoding of item IDs with a more efficient "next-item prediction". 2) Optimization. we propose ReePO, an RL-based training framework without human reasoning annotations. First, for given input, we sample multiple diverse reasoning paths, each culminating in an item recommendation. forming "reasoning-the toonggtthenittmn sequences analogous to those in reasoning LLMs. Second, we found it insufficient to set the reward naively ranking metrics, thus propose fused reward scheme that combines discrete ranking rewards with continuous similarity rewards. Finally, we formulate a joint training objective that leverages the rewards to optimize the model, encouraging it to develop reasoning pathways that lead to high-quality recommendations.
We conducted extensive experiments to evaluate the efficacy of Reec by comparing it with various traditional, I.LM-based, and reasoning-augmenneed recommendation baselines. The experimental results on three datasets demonstrate that Reec significantly outperforms all baseline methods, verifying the effectiveness of its unified architect and its ability leverage reasoning for superior recommendation quality. Our comprehensive evaluation comprises nine analyses, including a detailed

---

profiling of the model's reasoning behavior which reveals its context-aware capabilities; an efficiency analysis showing competitive inference speed among LLM--ased recommenders; ablation and optimization studies on key components; and additional investigations into scalability, generalization and case studies.
Key contributions are as follows:
We introduce eec the first unified large recommender model that intrinsically integrates reasoning and recommendation within a single architecture, resulting in significantly lower inference latency and conventional LLM-based recommender baselines. We propose RecPO, an RL framework for recommendation that enables model optimization without reasoning annotations. We derive joint optimization objective and propose fused reward scheme, yielding performance that significantly surpasses all baselines.
compared to both reasoning-augmeennn
Beyond metrics, we provide an in-depth analysis of the model's reasoning behavior, demonstrating that ee automatically develops context-aware reasoning strategies tailored to specific domains and user contexts, offering meaningful interpretability for its recommendations.
2 Preliminaries
2.1 LLM-based Recommendation
LLM-based recommendation is an emerging paradigm, and the most applicable approach is to leverage LLMs as encoders to embed users and items, which are widely adopted in industry [29, 24, 30]. Typically, the recommendation process within this paradigm involves the following structured steps:
User and Item Formulation. Structured interaction histories and item metadata first formulated into natural language prompts suitable for LLMs. We first collect a user's ' historically interacted items with the corresponding ratings. We then construct textual prompt includes both the instruction and a natural-language description of these past behaviors. E.g., " User has purchased and rated the following products sequentially: I. 'Avatar Blu-ray' (4/5); 2. 'Wireless Headphones' (5/5); ... Recommend the next item." " Finally, we tokenize this prompt into sequence tokens, denoted as Tu- which serves as the LLM input. Likewise, each candidate item can be described using its metadata, then tokenized into sequence of tokens Ee.
The tokenized textual prompts Fu and Te are then input into the LLM transformer backbone, by extracting the final hidden state of the input sequence, we can obtain corresponding user embedding h, and item embedding h...
Training via Contrastive Learning. To optimize LLM for recommendation tasks, in-batch con- trastive loss is typically adopted:
exp ( h /T)
LCL - log
(1)
Even exp (h., hh. ht
where r is the temperature hyperparameter controlling similarity dispersion, B is the ground truth item set one batch, and e+ is the groud-truth target item.
During inference, recommendation score s(u, v) for each user-item pair is computed via the inner product between their embeddings: su,,v) v) = h hy Finally, we rank all items according the scores and obtain top-K item recommendation list.
Inference.
2.2 Reinforcement Learning for Large Reasoning Models
Terminologies and Notations. RL for LLMs treats text generation as sequential action process, where each action refers to outputting the next token. Formally, at each time step f, we denote the current state as (xx, Oct), where is the initial prompt and oct is all previously generated tokens. Then, the conditioned probability distribution of an action is denoted as næ (o, Jxr oce), where me is the policy parameterized by 0. Based on the distribution, we can obtain the next token o, through manual selection, greedy or temperature sampling. Upon obtaining a complete sequence o, termed trajectory, reward function then evaluates and assigns it with scalar score R that can reflect
3

---

user satisfaction or answer correctness. However, using R directly in gradient estimates leads to high variance and unstable updates; instead, the advantage A, which measures how much better an action is than the expected return from that state, is adopted to reduce variance and improve sample efficiency [27, 28, 31, 32].
Sampling and Advantage Computation. Sampling- based advantage estimation is widely adopted recent advances [27, 31-33]; below we describe its basic pipeline and two typical computation methods. Given an input group of G different trajectories {o; [o:]= are sampled from To Existing studies widely obtain these trajectories via top- K sampling with temperature, where Olid refers to the frozen policy parameters before current update. Each trajectory o, receives scalar reward R2 which will then used to compute the trajectory-leve advantages via two widely adopted approaches, namely GRPO [27] and RLOO [31]:
R2 mean (
A ARLOO- R
Σ J
Ry,
A, ARRPO
(2)
G
Training Objective.
Training then proceeds with policy-gradient algorithms. Specifically, let
T.) = no [r, 0i,ct)
(3)
Tew (0,, [r, o1,<t)
be the importance ratio between the updated and old policies of trajectory i at token position The training objective is given by:
G lo.1
J(0) = Bo To (11x)
min (rr,,(0)4,,
(4)
where e defines the clipping trust-region radius, which prevents excessively large updates, thereby reducing variance and improving optimization stability.
3 e Large Recommender Model with Reasoning
Reec is large recommender model that "thinks" to recommend. We first outline the model design that enables the incorporation of generative reasoning and discriminative recommendation into a single model in Section 3.1. And then proposed RL optimization algorithm tailored for achieving unified reasoning and recommendation capabilities is introduced in Section 3.2.
3.1 Model Design
Architecture. As illustrated in Figure 1, our proposed Rdec is built upon decoder-only backbone with two task-specific heads: 1) ganngugg-mmdeeligg head (1m_head). The token embedding table Hy [ Where T the token set, and each row is d-dimensional embedding for one token. This head is responsible for the generation of reasoning tokens. 2) Recommendation head (rec_head). The item embedding table Hy E R/ Where V is the item set, and each row h, in Hy is obtained by encoding item description prompt into the model itself and extracting the final hidden state. This head is used to score items for recommendation. Such design integrates generative reasoning and discriminative recommendation within single unified model. It supports flexible and scalable item update by simply adding, deleting, or replacing vectors in the item embedding table, contrasting with generative recommendation systems that require hard-coded tokenization [34, 9, 4, 35], enabling effective zero-shot generalization and accommodating large-scale item catalogs without severe degradation on recommendation quality or efficiency.
Item prediction. Inference begins with feeding the tokenized user prompt x, (template of prompts can found in Appendix G) into the transformer-based backbone, producing an initial hidden state he. The manguage-modeligg head then maps ho to the first reasoning token 0n- This process continues autoregressively. yielding a sequence of T reasoning tokens 01,7. The final hidden state of the generated sequence hr is then fed into the recommendation head, where each candidate

---

item v E V is scored by an inner product s(v) = h Hylel, V. which determines the final ranking. This mechanism yields a tight massoinggrecommentt modeling head and recommendation head share the same hidden-state space as input, reasoning directly reshapes hy and thus yielding more accurate recommendation scores s(v). Such alignment ensures that reasoning optimization (Section 3.2) contributes directly to finer recommendation.
coupling, since both the language-
3.2
Optimization: RecPO
The goal is to train the policy we jointly perform reasoning and recommendation, i.e., it must generate coherent reasoning sequences to rank the target item accurately. Accordingly, we structure optimization workflow in three parts. 1) we introduce the trajectory sampling strategy that draws multiple reasoning trajectories for each user (Section 3.2.1). 2) we describe reward and advantage estimation, where discrete ranking signals and softmax similarities
fused into single scalar award for the above sampled trajectories (Section 3.2.2). 3) we formulate the training objective, which blends reasoning-level and recommendation-leee updates through clipped-ratio loss (Section 3.2.3). Complete description of training and inference pipeline are in Appendix E.
3.2.1 Trajectory Sampling
Due to the optimization objective of joint learning, we define one trajectory in our settings spanning the entire reasoning-thee www--teeeeeeeeeeeeeee process:
To
TW
Fe
01
OT
where the initial state xu encodes the user history and instruction, 01 OT represents the intermedi- ate actions of outputing T reasoning tokens, and at as the action of recommending the ground-truth truth target item.
For each user u. we first sample G reasonings {o:) with the old policy To, using the tokenized G
input Fy: {o Te (- Ia) ) by top-K sampling with temperature to control stochasticity. Each
sampled reasoning then fed through the policy xe to produce a complete reasoning- sereiinggtheeneeee -eeemreeommend trace, which is subsequently used for reward calculation and advantage estimation.
3.2.2 Reward and Advantage Estimation
Given the above sampled trajectories, now aim to assign rewards to them. Basically, the reward should align with the evaluation criteria, i.e., the recommendation metrics in our work, encouraging the model to achieve better performance. However, in practice, find that directly using the recommendation metrics as rewards is insufficient, as many trajectories of varying quality can result in the same top-K K ranking. We therefore introduce a fused reward scheme that combines discrete ranking reward R2 and continuous similarity reward Rz, which are formulated as follows:
Ra = NDCG @k (rank(v*))
Re ΣVEV exp h hn/T)
where Ra is the NDGC, Rg is the softmax similarity of recommending target item against all items in V. The final reward is then obtained through linear combination:
(5) where the weighting coefficient ,3 is empirically set to 3 zs 0.05 to keep the ranking term dominant while providing sufficient resolution among trajectories that attain identical ranks. With rewards f we can obtain trajectory-level advantages G via (2n.
R 3 R. + (1 3) Rs,
8 [0. L].
3.2.3 Training Objective
Given the goal of joint optimization of reasoning and recommendation, we treat the entire reasoning- recommecd sequence (zu, o1...., ..., or, x+)) single RL trajectory. Policy optimization therefore operates over a composite action space, where the policy first makes token-level decisions to generate reasoning, then selects item the recommendation stage. Under this formulation, the importance ratio from Eqn. (3) extends to:
=l(00, Te Ore
if & T' (reasoning)
(0)
(6)
5o(" g.)
if = 7 ++1 (recommendation).
That (p+ (0
5

---

Specifically, we model the recommending action, i.e., recommending the target item wt via the in-batch softmax:
ex exp(hipte //T
to Tru 0z) = Even expeexppppp (h
(7)
Let (r, A) = min(rA, clip(r, 1 - -, + e)) be the standard clipping operator with threshold we define the joint reasoning-andee and-recommmeddatiio training objective as:
[(0) [u, JoD. No (+Ixn) G
Σ (O),,, de, ((r. (8)
In (8), all trajectories contribute to token-level policy updates via the standard clipped objective (f) This design ensures the policy continues to learn from diverse reasoning behaviors. For the last recommendation action, only the trajectory with the highest advantage, identified by 8" = arg max, Aj. contributes gradients to recommendation optimization. This design concentrates recommendation learning with the most promising reasoning path, while the presence of other G reasoning paths contributes to ample exploration in reasoning actions, thus preserving exploration and ensuring effective recommendation learning. We further provide gradient-based explanation of how item encoding, as a core component of the model's recommendation ability, participates in the objective and enables end-to-end optimization in Appendix F.
4 Experiments
4.1
Setups
Dataset and Metries. Following previous works [3, 2, 1, 25]. we conducted experiments using real- world datasets sourced from Amazon, including CDs and Vinyl (CDs), Video Games (Games), and Musical Instruments (Instruments). Dataset statistics and preprocessing steps detailed in Appendix B. We utilized two commonly used metrics: Hit Rate (H@ K) and Normalized Discounted Cumulative Gain (N@ K), with cutoff K set 5, 10 and 20. We adopted the full recommendation setting, where metrics are computed over the entire item set, providing a better reflection of practical scenarios.
Baselines and Implementation. We selected competitive basclines from various categories, in- cluding traditional sequential recommenders (GRU4Rec [36], Caser [37]. SASRec [38]), LLM-based
recommender (TIGER [9], BigRec [2), D [3], SDPO [25]. Llara [1], SPRec [39]), and reasoning augmented recommendation systems (LangPTTune [16]). For fair comparison, we adapted LLaRA and SDPO by removing candidate prompts to perform constrained generation over the full item corpus, denoted as LLaRA and SDPO* respectively. Backboned on Gemma2-22 212--2--Innttuue and Quen2. 5-3B- Instruct. More baseline and implementation details be found in Appendix Cnd D, respectively.
4.2 Overall Performance
To validate the effectiveness of the proposed Reec, we showed the overall performance of baselines and our R'ec in Table 1. By amalyzing the results, we gained the following findings. 1) Overall, Reec consistently outperforms every competing baseline, underscoring the value of jointly optimizing reasoning and recommendation. 2) Traditional methods perform well on the Instruments dataset but struggle CDs and Games, revealing their limited generality. LangPTune frequently ranks second, which validates the benefit of integrating explicit reasoning into the recommendation pipeline. Finally, large recommenders generally outperform traditional approaches, notably secure secondary position on the Games dataset. We attribute this advantage to their larger model scale and semantic understanding capabilities. 3) Comparing two backbones, Gemma consistently outperforms its larger counterpart, achieving up to 2x gains for D3 suggesting that Gemma may generally deliver stronger recommendation performance despite its smaller parameter count (2B vs. 3B).
4.3
Ablation Study
We conducted ablation studies by evaluating the following variants: 1) "w/ CIsHead",, an alternative design that uses classification head to the hidden state of reasoning tokens, instead of using the unified

---

Table I: The overall performance of baselines and RCec on three datasets. The best results in each group are marked in Bold, while the second-best results are underlined. implies the improvements over the second-best results are statistically significant (p-value < 0.05). % improve represents the relative improvement achieved by Reec over the best baseline.
Video Garno
Method
H465
Nor
Her
No11
H4R20
N40
Hor
N455
Na60
H0:20
No
H855
N465
No:
He:20
N4629
0.171
0.0135
001933
00142
0.0201
0.0144
0.0057
0.0037
00:04
0.0041
0.0156
0.061
0.0009
0.0070
0.0131
0.0003
0.0001
0.0123
0.115
0,0445
00067
00042
0.063
0.0103
0.0136
SASRoo
00024
0.0144
0.0001
06162
00035
0.0076
0,0144
00081
0.0119
0.0066
0.141
00128
00000
0.0006
0.0105
0.0366
0.015
TKRER
00071
0.0128
0.0184
03132
00019
0.0134
0,00577
0,004
01037
B.0055
0.0166
0.009
00123
01005
0.0233
0.0116
01323
0U042
BigRee
0.0022
0.0033
001111
0.0022
0.0150
0.0072
0.0045
0.0025
0.0099
0.0030
0.0141
0.0022
0,0003
0.0004
0.0016
0.0006
0.123
0.0044
Quen
0.0044
0.0192
0,0052
0.0104
0.0038
Laue
0.0127
0.0053
0.0224
0,0155
0.0348
0.0145
0.0074
0.0053
00156
0.0050
0,0208
0,0094
0
0.0027
0.0058
0.0040
0.0140
0L0140
R'
0.02377
0.0154"
B.1374
0,0198
0
0.029
0.0533
8,092
8/8647
0
0
0.0457
0.0258
0.01854
0U1532
0.0064
0,0827
0
inpmrs
35.43/
6.94%
66.96
969
22.12%
$2.61%
33.335
37.95%
1100%
20.3%
40.62%
42.36%
$4.05%
51.196
41.39%
31.564
53.53%
BigRec
01006
0,0018
0.0101
0,0055
110130
0.0066
0.0030
0.0000
0J0037
0.1199
0.0005
0,0156
010005
0,025M
0,0138
0,0430
0.0022
17a
0.0072
0.0038
0.0012
0,0890
0.0339
0.0134
0.0216
0.0199
0/327
0.0154
0.0446
0,0144
0
0.0068
0,0210
0.0141
0/0478
DU0234
SDR0*
0.0066
0.0034
0.0058
0,0054
0.0144
0
0.0022
0.0015
0.0037
0.0025
0.0162
0.0094
0.0166
0.0:22
0.0298
0.0155
0.0666
0.0222
Llnn
0.0078
0.0055
0.0137
0.0074
0.0159
0.0030
0.0097
0.0009
0.127
0.0340
0.0002
0.0152
0.0223
0.073
0.0428
0.0223
0.6777
0.0299
0.0111
0.0063
LauPTune
0.0130
0.0079
0.0221
0.0107
0.0403
0.0152
0,0350
0,0235
00400
0.027
0.0677
0,6325
0
0
0.0053
0.0120
0:0059
0.0055
0.0094
R'sc
0,0264*
0.0061
0,0897
8.8203*
0,0615
000257
0.1573
0,0398
0.8914
0
0.1042
0.6527+
6.8326
0.02054
0.1531
0
0,6635
0,0347
inyens
90.859
11.81%
77.23%
25.31%
52.619
71
71.4%%
53.9155
62.15%
13.98%
19.19%
21.52%
21.34%
16.25%
Table 2: Ablation study on key components of R²ec.
Instruments
Vinyl
Video Games
Method
B@5
N&5 H@:10 16 H& 2@20
HGS
N@5
H& 10 N6 H@20 N@20 H@s
N85
H@:00
N@10
18:20
N6:20
of ClsHead
0.0044 0:0025
0.0102
0,0033 0.0179
0.0067
0.0030
0.0025
0.0045 0.0027
0.0095
0.0044
0,0012
0,0008
0,0022 0.0011 0:0133
0.0032
wo Reasaning
0.1776
0.0121
0.0296
0,0153 0,0511 0.0200
0.0469
0.0821
0,0692 0.0393 0.0455
0.0456
0.0277
0.0174
0.0441
0:0227
0.0748 0.0903
wlo Ra
0,0198 0.0124
0.0164
0,0560
0.0224
0.0521
0,0338
0.0766 0.0404
0.0974
0:0486
0.0302
0.0487
0.0254 0:0798
wlo R.
0.244 0.0160
0.0904
0.0208
0.0605
0.025%
0.0543
(10882
0.07744
0.0456
0.1012
0:05155
0.0316
0,0202
0.0534
0,0264
0.0814 0,05S
k'ce
0.0264 0.0161 0,0397
0.0201
0.0615
0.0252
(1J15A
0.0388
0,0804
0.1457
0.108
0.0522
0.0326
1,0220
0,0531
0.0271
IL0853 0.0363
conooiing-ecoommedattiin formulation. 2) "w/o Reasoning", where reasoning tokens are removed from prompts and the model is trained solely with in-batch contrastive loss; 3) "w/o R.", which retains only the discrete ranking reward Ra while removing the continuous similarity reward R. in Eqn. (5); 4) "w/o Ra", which removes the discrete ranking reward R₂ from Eqn. (5); and The results are summarized in Table 2, and several observations stand out.
1) It is found that Reec achieves an average improvement of roughly 15% across all metrics compared to w/o Reasoning. These substantial gains demonstrate that our designed optimization have enabled R²ec to better leverage test-time scaling to deliver significantly stronger recommendation performance. 2) The w/ ClsHeadvariant substantially underperforms our tightly-coupled architecture, highlighting recommendationnn co-optimization. 3) As revealed that w/o R. (using only R2)) consistently outperforms w/o R& (using only R.), this indicates that adopting reward signal that directly reflects evaluation result is crucial for training, while the continuous reward R.. despite offering finer granularity, fails to provide meaningful distinctions and instead introduces noise that leads to suboptimal performance. 4) By fusing R& with a small weight on Re, our approach preserves the task alignment of ranking rewards while benefiting from the supplementary signal of the continuous term. As a result, Reec achieves optimal performance on nearly all metrics.
the necessity of
4.4 Analysis on Optimization
This subsection presents analyses of key optimization strategies in RecPO, evaluating their impact on final performance.
4.4.1
Analysis on Advantage Estimation
Accurate advantage estimation is crucial for reducing variance and improving sample efficiency in graicy-gradient RL [31, 28, 32]. We therefore evaluated two estimators, GRPO [40] and RLOO [31],
7

---

0.14 211--R
ALOO
RUDO
6.1
011
GRPO
2010
GRPO
20006
Boats
0.06
100
002
6015
Bog
mbb
(a) Train Reward (CDs)
Val Reward (CDs)
(c) Val Length (CDs)
0.00
-LLOO
0.12
0.005
crend
0.00
80.09
mm
5o.os
2010
500
o0t
0.003
0.07
200
560
570
200 400
600 800 100000000000000000000
0.6
400
500
1200
1000
140
420
220
1200
1000
alep
step
step
(d) Train Reward (Instruments)
(c) Val Reward (Instruments)
(f) Val Longth (Instruments)
Figure 2: Analysis on advantage estimation methods. RLOO and GRPO, across two datasets. "Train Reward" and "Val Reward" indicate the variation in rewards on the training set and validation set, respectively. "Val Length" represents the variation reasoning length on the validation set.
0.0385
reasoning g longth
reasoming longth
NDCG@6
0,688
515
NDCG.05
0.039
515
NDCG@85
0.035
0.03
Ratea
8.045
0.0375
0.037
308
0.04
0.036
500
QWen
QWen Gomma
906
0.035
Gemma
2.5
100
200
300
400
8
temperature
top-K
group size
group size (d)
(a)
(c)
Figure 3: Analysis on trajectory sampling and group size over the CDs dataset. (a) and (b) show the impact of temperature and top- K sampling on performance and reasoning length, respectively, (c) and (d) present the effect of group size on NDCG@5 and Hit Rate respectively.
in our training pipeline. The variations of the training reward, validation reward, and reasoning length across the training steps are summarized Figure 2, and we had the following observations.
1) First, as shown in Figure 2a and Figure 2d, both RLOO and GRPO exhibit high-variance training curves. This due to the inherent nature of recommendation environments, which produce highly varied reward magnitudes- some sessions result in high rankings, while others yield very low ones. 2) Second, Figure 2b and Figure 2e illustrate, GRPO demonstrates faster learning in the initial training steps and consistently outperforms RLOO in terms of validation reward, whereas RLOO progresses more steadily. This divergence stems from the GRPO's unit- variance normalization magnifies rewards into larger gradients that accelerate early learning. 3) Besides, as shown in Figure 2c and Figure 2f, GRPO's reasoning length gradually increases as training progresses, which consistent with the phenomenon observed in LLM reasoning training [40], while RLOO appears to maintain a certain level of stability. This is because RIOO-driven optimization does not encourage improved reasoning with the low reward magnitudes in our task.
4.4.2 Analysis on Trajectory Sampling
During RecPO, we performed trajectory sampling via the temperature т and top-K sampling, which influence the stochasticity and diversity of the generated samples. To quantify their impact, we varied and K, and the results are presented in Figure 3. 1) It is observed that increasing the temperature produces longer resoning and boosts recommendation performance, i.e. NDCG@5. A higher temperature introduces greater sampling entropy, allowing the model to explore wider range of reasoning trajectories. 2) Conversely, it is found that increasing top-K actually shortens the reasoning length and generally leads to decline in recommendation performance. This is because larger top- enlarges the candidate token set, which counterintuitively mitigates length hacking yet excessively large K reintroduces noisy and low-quality samples.

---

Table 3: Comparison of sequence embedding extraction strategies.
Method
N@s
N@10
H@8
H@
Special token
0.0353
0.0432
0.0509
0.0737
Mean pooling
0.0368
0.0453
0.0528
0.0796
Max pooling
0,0382
0.0445
0.0595
0.0729
Last hidden state (ours)
0.0388
0.0457
0.0588
0.0804
4.4.3 Analysis on Group Size
During RecPO, we sampled group of trajectories to estimate their advantages. Therefore, we conducted experiments to analyze how varying group sizes impact performance. The results are outlined in Figure 3, and we gained several key findings as follows: 1) It is observed that performance improves for both backbones as group size increases, but the rate of improvement gradually slows down. While larger group size generally leads to more explored paths, it also raises the training cost. These results suggest that selecting a group size of or is sufficient, and further increasing the group size is unnecessary. 2) Comparing the two backbones, we found a difference in sensitivity to group size. Qwen's 's performance at a group size improves rapidly as the group size increases. Gemma performs well even with smaller groups, likely because its pretraining exposed to a broader range of reasoning scenarios, thereby equipping it with stronger initial reasoning- sesoonnn-ffo--ee for-recommeenatiion capabilities.
lags significantly behind Gemma's ss but
4.4.4 Analysis on Embedding Strategy
We further examined different strategies for obtaining sequence-level embeddings. Specifically, compared the last hidden state used in Reec against max pooling, mean pooling, and adding a special-token under identical settings. As reported in Table 3, the last hidden state yields the best overall performance. Special token strategies yield weaker performance in setting, likely because we adopt LoRA tine-tuning and no large-scale optimization on the LLLM backbone; is thus challenging for the model to effectively learn generate and utilize newly introduced tokens for global sequence representation.
4.5 Analysis on Reasoning Behavior How Does R Reason to Recommend?
How exactly does Reec transform raw behavioral descriptions into insightful recommendations? While the previous sections establish that reasoning substantially enhances recommendation accuracy, the underlying decision process-how the model internally reasons to reach its conclusions- remains less understood. To demystify this process, we conduct a systematic qualitative and quantitative analysis to uncover the reasoning strategies Re employs across different domains and user contexts.
Qualitative Analysis. We first engaged in extensive discussions with colleagues analyze and summarize the diverse reasoning behaviors exhibited by Reee. Through this process, identified seven distinct reasoning strategies of R² ec for recommendation:
to collaboratively
1. Attribute Abstraction: Identifying underlying attributes of purchased items.
2. Pattern Recognition & Clustering: Detecting frequent item patterns to form user-interest clusters.
3. Scenario/Role- Based Reasoning: Inferring roles specific use cases.
4. Temporal Reasoning: Leveraging purchase timing and order for contextual understanding.
5. Self Delf-Explanttion: Providing explicit rationales for recommendations,
6. Negative Preference Exclusion: Avoiding previously negatively rated item types.
7. Multi-Objective Balancing: Managing trade-off' between conflicting user objectives.
These reasoning strategics illustrate how reasoning aids recommendation. Equipped with reasoning, large recommender models can distill salient user preferences from noisy or complex behavioral data, abstract beyond surface-level co-occurrence, and produce more contextually appropriate and interpretable recommendations.
Quantitative Analysis. Building on the above findings, we then randomly sampled 100 reasoning outputs from each test set. These samples were annotated and analyzed through a combination of detailed human evaluation and automated assessment with GPT-4.1, ensuring both depth and consis- tency. Figure 4 visualizes the proportion of sampled reasoning outputs within each datasct that exhibit

---

Elatroks
Proportion
Datnosts
CDs
6a
64
InSPUTNTTS
Alr Als.
Portern Res.
Scevatio
Temposa
Self
Neg. Piet
Reastning Rehawiors
Figure 4: Distribution of reasoning behaviors across datasets, Each bar represents the proportion of reasoning outputs exhibiting given reasoning behavior within a dataset.
each reasoning behavior, which reveals key trends. (1) The distribution of reasoning behaviors varies across datasets, indicating that Reec self-adapts its reasoning patterns to domain characteristics and user-item interactions. (2) Even within single domain, Reec flexibly selects different reasoning strategies depending on the specific user sequence. Overall, these results demonstrate that Reec does follow fixed reasoning template but instead employs context-aware reasoning tailored both dataset and user, yielding improved recommendation performance and interpretability.
4.6 Analysis on Inference Efficiency
To quantify the efficiency gains of our unified model Reec over both traditional and LLM-based recommenders, we conducted com- single
Table 4: Average inference la- tency (in seconds) across mod- els.
performed NVIDIA RTX 3090 GPU using identical input prompts, maximum token lengths, and candidate sets. We executed 100 queries per at a batch size and averaged the results over 3 independent runs.
parative latency tests. All inferences
Method
Latency
As shown in Table 4, 1) Reec achieves competitive inference effi- ciency among I.I.Mbbseed recommenders, outperforning compa- rable methods such as LangPTune, D3, and LLaRA. Such benefits stem from the use scalable item embeddings, which avoid expen- sive autoregressive decoding over large item vocabularies and the design of an unified architecture. 2) When deployed with the VLLM inference framework, R²ec significantly narrowed the efficiency gap with traditional sequential models like SASRec. These results demonstrate that Reec successfully balances expressive power and computational efficiency.
SASRec
0.014
LangPTTne
1.90
D
LLaRA
5.23
Recc
1.67
Réce (with VLLM)
0.0945
4.7 More Analysis
Due to space constraints, further extended analyses on Scaling to Larger Backhones and Cross- Domain Robustness are reported in Appendix I, while Appendix J provides representative case studies. We also discuss the limitations of our approach in Appendix H.
Conclusion and Future Work
In this study, we investigate the integration of reasoning into large recommender models by intro- ducing R²ec, a unified large recommender model with intrinsic reasoning capabilities, trained with RecPO, without reliance on human- analysis demonstrated that R Recachieves both higher recommendation quality and lower inference latency compared to existing Taken together, these findings highlight the importance of tightly coupling reasoning and recom- mendation to unlock large recommender model's full potential. In the future, we aim to further investigate the efficient reasoning in larger recommender models, striving for optimal "thinking" in recommendations.
reasoning annotations. Extensive experiments and
aassoning-augmente and conventional LLM-based recommenders.
Acknowledgments and Disclosure of Funding
The work described in this paper was supported by Research Grants Council of Hong Kong(PolyU//5209724, PolyU/15205325, PolyU/152071221 122), and PolyU internal grants
grants (BDWP).
10

---

References
[1] Jiayi Liao, Sihang Li, Zhengyi Yang, Jiancan Wu, Yancheng Yuan, Xiang Wang, and Xiangnan He. LLaRA: Large Seeguagg-RRcommenaattee Assistant. In Proceedings of the 47th Interna- rional ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1785 1795. ACM.
[2] Keqin Bao, Jizhi Zhang, Wenjie Wang, Yang Zhang, Zhengyi Yang, Yanchen Luo, Chong Chen, Fuli Feng, and Qi Tian. A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systems. 3(4):1-27.
[3] Keqin Bao, Jizhi Zhang. Yang Zhang, Xinyue Huo, Chong Chen, and Fuli Feng. Decoding Matters: Addressing Amplification Bias and Homogeneity Issue in Recommendations for Large Language Models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 10540-10552, Miami, Florida, USA, November 2024. Association for Computational Linguistics.
[4] Bowen Zheng, Yupeng Hou, Hongyu Lu, Yu Chen, Wayne Xin Zhao, Ming Chen, and Ji- Rong Wen. Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation. In 2024 IEEE 40th International Conference on Data Engineering (ICDE), pages 1435-1448.
[5] Xin Zhang. Yanzhao Zhang, Wen Xie, Dingkun Long, Mingxin Li, Pengjun Xie, Mcishan Zhang, Wenjie Li, and Min Zhang. Phased training for LLM-powered text retrieval models beyond data scaling. In Second Conference on Language Modeling, 2025.
[6] Jun Hu, Wenwen Xia, Xiaolu Zhang, Chilin Fu, Weichang Wu, Zhaoxin Huan, Ang Li, Zuoli Tang, and Jun Zhou. (SAID) Enhancing Sequential Recommendation via LLM-based Semantic Embedding Learning. In Companion Proceedings of the ACM Web Conference 2024, pages 103-111, Singapore Singapore, May 2024. ACM.
[7] Jiacheng Li, Ming Wang, Jin Li, Jinmiao Fu, Xin Shen, Jingbo Shang, and Julian McAuley. Text Is All You Need: Learning Language Representations for Sequential Recommendation, In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. KDD '23, pages 1258- New York, NY, USA, August 2023. Association for Computing Machinery.
[8] Xinyu Zhang, Linmei Hu, Luhao Zhang, Dandan Song, Heyan Huang, and Liqiang Nie. Laser: Parameter-Efficient LLM Bi-Tuning for Sequential Recommendation with Collaborative Information, September 2024. arXiv:2409.0166 [cs].
[9] Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Keshavan, Trung Vu, Lukasz Heidt, Lichan Hong. Yi Tay, Vinh Q Tran, Jonah Samost, Maciej Kula, Ed H. Chi, and Maheswaran Sathiamoorthy. Recommender systems with generative retrieval. In Proceedings of the 37th International Conference Neural Information Processing Systems, NIPS '23, pages 10299-10315 Curran Associates Inc.
[10] Xuansheng Wu, Huachi Zhou, Yucheng Shi, Wenlin Yao, Xiao Huang, and Ninghao Liu. Could Small Language Models Serve as Recommenders? Towards Data-centric Cold-start Recommendation. In Proceedings of the ACM Web Conference 2024, WWW '24, pages 3566-3575. Association for Computing Machinery.
[11] Qidong Liu, Xiangyu Zhao, Yejing Wang, Zijian Zhang, Howard Zhong, Chong Chen, Xiang Li, Wei Huang. and Feng Tian. Bridge the Domains: Large Language Models Enhanced Cross-domain Sequential Recommendation, April 2025. arXiv:2504. 18383 [cs].
[12] Qidong Liu, Xian Wu, Wanyu Wang, Yejing Wang, Yuanshao Zhu, Xiangyu Zhao, Feng Tian, and Yefeng Zheng. LLMEmb: Large language model can be good embedding generator for sequential recommendation. In Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence und Thirty-Seventh Conference on Innovative Applicutions of Artificial Intelligence and Fifteenth Symposium on Educational Advances in Artificial Intelligence, volume 39 of (et/2///AAA///AAA. 25, pages 22183112191. AAAI Press.

---

[13] Qidong Liu, Xian Wu, Yejing Wang, Zijian Zhang, Feng Tian, Yefeng Zheng, and Xiangyu Zhao. LLM-ESR: Large language models enhancement for long-tailed sequential recommendation. In Proceedings of the 38th International Conference on Neural Information Processing Systems, volume 37 of NIPS '24, pages 26701-26727. Curran Associates Inc.
[14] Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan O. Arik. Dong Wang. Hamed Zamani, and Jiawei Han. Search-RI: Training L.LMs to Reason and Leverage Search Engines with Reinforcement Learning. In Second Conference on Language Modeling, 2025.
[15] Niklas Muennighoff, Zitong Yang. Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling, February 2025. arXiv:2501. [cs].
[16] Zhaolin Gao, Joyce Zhou, Yijia Dai, and Thorsten Joachims. End-to-end Training for Recom- mendation with Language-based User Profiles, October 2024. arXiv:2410.18870.
[17] Yi Fang, Wenjie Wang. Yang Zhang, Fengbin Zhu, Qifan Wang, Fuli Feng, and Xiangnan He. Reason4Rec: Large Language Models for Recommendation with Deliberative User Preference Alignment, February 2025. arXiv:2502.02061 [cs].
[18) Jieyong Kim, Hyunseo Kim, Hyunjin Cho, Seong Kang, Buru Chang, Jinyoung Yeo, and Dongha Lee. Review-driven Personalized Preference Reasoning with Large Language Models for Recommendation, December 2024, 2XX2:2408.006276 [cs].
[19] Yunjia Xi, Weiwen Liu, Jianghao Lin, Xiaoling Cai, Hong Zhu, Jieming Zhu, Bo Chen, Ruiming Tang, Weinan Zhang, and Yong Yu. Towards Open-World World Recommendation with Knowledge Augmentation from Large Language Models. Proceedings of the 18th ACM Conference on Recommender Systems, RecSys 224, pages 12-22. Association for Computing Machinery.
[20] Wenjun Peng, Guiyang Li, Yue Jiang, Zilong Wang, Dan Ou, Xiaoyi Zeng, Derong Xu, Tong Xu, and Enhong Chen. Large Language Model based Long-tail Query Rewriting in Taobao Search, March 2024. arXiv:2311.055
[cs].
[21] Jiacheng Lin, Tian Wang, and Kun Qian. Rec-RI: Bridging Generative Large Language Models and User-Centric Recommendation Systems via Reinforcement Learning, March 2025. arXiv:2503.2444 24289 [cs] version: 1.
[22] Alicia Y. Tsai, Adam Kraft, Long Jin, Chenwei Cai, Anahita Hosseini, Taibai Xu, Zemin Zhang, Lichan Hong, Ed H. Chi, and Xinyang Yi. Leveraging LLM Reasoning Enhances Personalized Recommender Systems, July 2024. arXiv:2408.0088 000002..
[23] Yuling Wang, Changxin Tian, Binbin Hu, Yanhua Yu, Ziqi Liu, Zhiqiang Zhang, Jun Zhou, Liang Pang, and Xiao Wang. [SLIM] Can Small Language Models be Good Reasoners for Sequential Recommendation!! March 2024. Xii:22403.04266 [es].
[24] Jian Jia, Yipei Wang, Yan Li, Honggang Chen, Xuehan Bai, Zhaocheng Liu, Jian Liang, Quan Chen, Han Li, Peng Jiang, and Kun Gai. LEARN: Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Application, December 2024. HxXiv:2405.03988 [cs].
[25] Yuxin Chen, Junfei Tan, An Zhang, Zhengyi Yang, Leheng Sheng, Enzhi Zhang, Xiang Wang, and Tat-Seng Chua. On Softmax Direct Preference Optimization for Recommendation, November 2024. arXiv:2406.09215 [cs].
[26] Xinyu Lin, Haihan Shi, Wenjie Wang, Fuli Feng, Qifan Wang, See-Kiong Ng. and Tat-Seng Chua. Order-agnostic identifier for large language model-based generative recommendation. In Proceedings of the 48th international ACM SIGIR conference on research and development in information retrieval, pages 1923-1933, 2025.
[27] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models, April 2024. arXiv:2402.03300 [es].
12

---

[28] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms, August 2017. arXiv:1707.0634. 777.0067777777 [es].
[29] Junyi Chen, Lu Chi, Bingyue Peng, and Zehuan Yuan. HLLM: Enhancing Sequential Recom- mendations via Hierarchical Large Language Models for Item and User Modeling, September 2024. arXiv:2409. 12740 [es].
[30] Bowen Zheng, Zihan Lin, Enze Liu, Chen Yang. Enyang Bai, Cheng Ling, Wayne Xin Zhao, and Ji-Rong Wen. A Large Language Model Enhanced Sequential Recommender for Joint Video and Comment Recommendation, March 2024. 2XXi::2403.13577 [cs] version: 1.
[31] Arash Ahmadian, Chris Cremer, Matthias Gallé, Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet Üstün, and Sara Hooker. Back Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs, February 2024. arXiv:2402.14740
[es].
[32] Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang. Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, Chenyang Shao, Yuwei Yan, Qinglong Yang. Yiwen Song, Sijian Ren, Xinyuan Hu, Yu Li, Jie Feng, Chen Gao, and Yong Li. Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models, January 2025. arXiv:2501.096 [es].
[33] Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning, February 2025, arXiv:2502.1476. [cs].
[34] Wenjie Wang, Honghui Bao, Xinyu Lin, Jizhi Zhang, Yongqi Li, Fuli Feng, See-Kiong Ng. and Tat-Seng Chua. Learnable Item Tokenization for Generative Recommendation, August 2024. arXiv:2405.0733
[35] Yijie Ding, Yupeng Hou, Jiacheng Li, and Julian McAuley. Inductive Generative Recommenda- tion via Retrieval-based Speculation, October 2024. arXiv:2410.02939.
[36] Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. Session-based Recommendation with Graph Neural Networks, January 2019. arXiv: 10.0055..
[37] Jiaxi Tang and Ke Wang. Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding, September 2018. 80Xiv:1809.07426 [cs].
[38] Wang-Cheng Kang and Julian McAuley. Self-Attentive Sequential Recommendation, August
2018. arXiv:1808.0978 [cs].
[39] Chongming Gao, Ruijun Chen, Shuai Yuan, Kexin Huang, Yuanqing Yu, and Xiangnan He. SPRec: Self-Play to Debias LLM-based Recommendation. pages 5075-5084.
[40] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.12945 2025.
[41] Yafu Li, Xuyang Hu, Xiaoye Qu, Linjie Li, and Yu Cheng. Test- Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback, January 2025. arXiv:2501.12895 [cs].
[42] Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct Preference Optimization: Your Language Model is Secretly a Reward Model, July 2024. arXiv:2305.188 18290.
[43] Fan Liu, Zhiyong Cheng, Lei Zhu, Zan Gao, and Liqiang Nie. Interest-aware message-passinn gen for recommendation. In Proceedings of the Web Conference 2021, page 1296-1305, ACM, 2021.
[44] Yongqi Li, Meng Liu, Jianhua Yin, Chaoran Cui, Xin-Shun Xu, and Liqiang Nie. Routing micro-videos via temporal graph-guided recommendation system. In Proceedings of the 27th ACM International Conference on Multimedia, Mm '19, pages 1464-1472, New York, NY,
USA. Association for Computing Machinery.
13

---

[45] Xinyu Lin, Wenjie Wang, Yongqi Li, Shuo Yang. Fuli Feng, Yinwei Wei, and Tat-Seng Chua. Data-efficient fine-tuning for llm-based recommendation. Proceedings of the 47th Interna- lional ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '24, page 363-374, New York, NY, USA, 2024. Association for Computing Machinery.
[46] Xinyu Lin, Wenjie Wang, Yongqi Li, Fuli Feng, See-Kiong KionK Ng, and Tat- Satg Chua. Bridging Items and Language: A Transition Paradigm for Large Language Model-Based Recommen- dation. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '24, pages 1816-1826, New York, NY, USA, August 2024. Association for Computing Machinery.
[47] Hongke Zhao, Songming Zheng, Likang Wu, Bowen Yu, and Jing Wang. LANE: Logic Alignment of Non-tuning Large Language Models and Online Recommendation Systems for Explainable Reason Generation, July 2024. urXiv:2407.02833 [cs].
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The contributions and scope of the paper are included in the abstract and Section 1. Please refer to the first and last paragraph of Section for scope and contributions, respectively.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
Limitations our work is detailed in Appendix
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer:
[NA]
Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We introduce the details of the experiment, including hardware, software and other curtail settings in Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
16

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We have attached the data and code used in this paper in the supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ httppsscccppbcccccceeeeeeecccl guwiidiuue gese CooeeSubii ssi asionPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We introduce the data splits in Appendix B, hyperparameters, optimizers in Appendix D, with crucial hyperparameter study in Sec. 4.4.2.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We report the two-sided t-test with p 0.05 results in the main experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
17

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer:
[Yes]
Justification: We provide the details of compute resources the implementation detail in Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: http///eurrrpppee httpc///bbb..e..ccccccceeeeeeeee
Answer: [Yes]
Justification:
We have made sure that paper conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss the potential positive impacts that our algorithm will bring in the Introduction section.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
18

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: There is no risk of misuse of the proposed method and the datasets used in the paper are open-sourced.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited the original paper or attached the link to the existing assets used in this paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We have attached the introduction of how to run the code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
20

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
The core methodology of our work is built upon LLM and indispensable the results of the study (see Sections 3-4 and Appendix D), their usage is explicitly described and declared herein.
Justification:
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: : //teuriiess.///e/e ecc/oonfeeences/22255//////////e for what should or should be described.
21

---

A Related Work
A.1 Reinforcement Learning for LLM Reasoning
OpenAI's GPT-ol demonstrated that scaling reinforcement learning (RL) training with large compute budgets enables models to internalize reasoning as a differentiable policy, achieving sfate-of-the- art performance in emergent meta- meta-reasoning tasks [41]. Early RLHF methods trained reward model on human preference data and tne-tuned the policy using Proximal Policy Optimization (PPO) [28]; however, PPO's multiple optimization passes introduce implementation complexity. To streamline RL tuning, Direct Preference Optimization (DPO) [42] performs single-step policy updates, trading simplicity for potential off-policy bias. Alternative estimators, such as Group Relative Policy Optimization (GRPO) [27], derive baselines from group-level rewards, eliminating the need for a separate critic, while REINFORCE's leave-one-out (RLOO) [31] computes advantages by excluding each trajectory sequentially. Building on these advancements, DeepSeek-Zon Zero [40] removes reliance on supervised fine-tuning by driving reasoning emergence purely through intrinsic RL rewards, and DeepSeek-R -RI I further integrates rule-driven reward engineering with self-play verification to enhance reasoning robustness.
A.2 Large Language Model-based Recommendation
Recommendation systems are comnertonne of modern information services [43, 44], and the integra- tion of Large Language Models (LLMs) has emerged as a promising frontier to advance their capabil- ities [45, 2, 46]. Recent advances in 4LM-based recommendation typically follow two paradigms:
1) LLM-enhanced, where LLMs enrich recommendation pipelines with additional features, and 2) LLM-centric, where recommendation is framed as generative task via item identifiers [1-4, 24-26), or LLMSs serve as encoders to embed users and items [7, 29]. We term models under the second paradigm large recommender models. In the first paradigm, L.L.-generane embeddings utilized to enhance conventional recommenders. Adoption of LLM-generated text are not new, typical examples includes user intent [16, 47], search queries [21], item summaries [19]. or rationales [23]. Among them, SLIM [23], LangPTune [16], and Rec-RI I [21] further optimize LLM for finer rationale generation with RL. However, such designs introduce high resource demands for training and serving both models, and the connection between two modules remains gradient-less, blocking true end-to- end learning, resulting in suboptimal convergence and performance. Large recommender models typically recommend through autoregressive generation of item identifiers [4. 24] or titles [1-3, 25], or by embedding user interaction sequences [7, 29]. I Large recommender model has demonstrated remarkable potential, but current methods fail to utilize reasoning ability of L.MMs. To the best of our knowledge, no existing large recommender model improves recommendation over explicit reasoning generation, which is the gap that motivated our work.
B Dataset
Table 5: Dataset Statistics.
Dataset
Users
Items
Density
Interactions
Train
Val
Test
Vidco Games
29.230
10,144
0.031%
63,502
50,801
6,350
6,351
CDs and Vinyl
7,701
12,024
0.023%
13,435
10,748
1,343
1,344
Musical Instruments
15,656
10,320
0.031%
34.373
27.498
3,437
3,438
Following the temporal-truncation protocol introduced in previous literature [3, 2]. we construct three subsets-CDs and Vinyl (CDs), Video Games (Games), and Musical Instruments (Instruments) from the lastest public Amazon review dataset² spanning from May 1996 to October 2023. For each domain, we begin with the most recent year of interactions (October 2022 October 2023) and, the resulting number of valid items is insufficient, iteratively roll the time window backward month- by-month until 10k items are obtained. We omit the 5-core filter so to retain the nature behaviour characteristic of recommendation scenarios. Each user's 's interaction history is then chronologically sorted and truncated to the latest 20 actions, yielding fixed-length sequences for all subsequent
hnttps://maazn--ree 2023..2022..000 2023.github 1o/ htt///idxx..ee
22

---

modelling and evaluation. Finally the dataset is split with 80% training 10% validation 10% test. The resulting statistics are listed in Table 5.
C Baselines
GRU4Rec [36] utilizes the GRU (Gated Recurrent Unit) architecture to model sequences.
Caser [37] employs both horizontal and vertical convolutional operations to enhance the cap- ture of high-order interactions within item sequences, improving recommendation accuracy.
SASRec[38] incorporates a multi-head self-attention mechanism in its self-attentive sequen- tial recommendation model, facilitating the modeling of intricate sequential data patterns.
TIGER [9] learns extra tokens from item features to present items, and then converts the user sequence into the sequence of the new item token for next-item generation.
BigRec [2] utilizes the item title to present the user sequence for recommendation generation.
D [3] proposed to address amplification bias and homogeneity issues in LLM recom- menders that generate item title generation.
SDPO [25] introduces a multi-negative, softmax DPO to better align LLMs with human preferences and mitigate biases in recommendation tasks.
Llara [1] proposes aligning large language models with sequential recommenders through hybrid prompt design and curriculum learning for better sequential behavior understanding.
SPRec [39] employs self-play mechanism to iteratively debias preference alignment in LLM-based recommendations by suppressing undesirable items and reinforcing positive samples.
LangPTune [16] utilized LLM as user profile generator, and proposed an end-to-end train- ing pipeline that optimizes LLM --geerrtd user profiles for the recommendation objective.
D Implementation Details
Hardware: 4x NVIDIA 3090 (24GB) GPUs
Framework: PyTorch 2.5.0, Transformers 1.48.1, DeepSpeed 0.15.7
Traditional Recommenders. We train all non-LLM baselines with the Cross-Entropy loss and the AdamW optimiser (learning-rate 10-4); the batch size is 256. TIGER [9] adopts T5 as its archi
backbone without further architectural changes.
LLM-based Methods. For every LLM variant including the contrastive-learning baseline (CL, Section 4.3)- we finetune Gemma2-22 2B Instruct and Qwen2.5-38 I-I--rurn Models are adapted with LoRA (rank 4) using DeepSpeed Stage 2 and the quantized PagedAdamW optimizer. Training lasts for at most three epochs with early stopping (patience = 1); the maximum generation length is 512 tokens. If public implementations exist, we keep the original hypee-parameters, otherwise, grid search over learning rates (5 x 10 10-4, 10 4) is performed.
Reec (ours). The learning rate is set 1c with batch size of 24, and apply linear warm-up for the first 32 steps. For trajectory generation, we set the group size G as four, utilized VLLM vvenor-paraalellin
= 1, target GPU utilisation 95 %) for efficient generation. vLLM reserves one GPU for inference and leaves three for training. Sampling uses temperature 1.5 and top- K 200. Policy optimization follows the clipping ratio 0.2 and omits KL regularization. Rewards computed with NDCG@ 1000 and B 0.05. Unless stated otherwise, these settings are used throughout.
E Overall Pipeline
Training To instantiate Rdec, given the dataset D, the initial policy and item embedding cache Hy, we train the model with detailed training process illustrated Algorithm 1.
Phttps://doo vllm. /e/eenen//t/
23

---

Algorithm Training Process
Input: Dataset D. initial policy Ræ, embedding function Sa, item embedding table Hy
Output: Optimized policy model Th
1: for step = 1 to N do
2:
0 then 3: Refresh item embedding: Hy[u] < fo(ro), Vu E V
if' step % Teffresh
4: end if
5: Sample a training batch B = {(u, u+)} D
6:
Encode target item prompts and update embedding table: Hy[u*] fo(xx)) v(u,et)
B
7:
for all (u, x+) do Generate G' trajectory: [[o1,vt]... log, wt]} ~ Teo (:(aa)
8:
9:
Compute reward for each trajectory using Eq. (5)
10:
Compute advantage for each trajectory using Eq. (2)
11:
end for
12: Update policy parameters @ via loss in Eq. (8)
13:
Update old policy: Oold t- 0
14:
end for
Inference For inference, we first pass every item prompt Zp through the trained model once to obtain all item embeddings Hy. At inference time, the model greedily generates deterministic reasoning o for a user prompt Zus the last hidden state ht goes through the embedding table, scoring each candidate via the inner product svv) h HyluJ. The top-K items can then be recommended by sorting these scores.
Gradient Explanation for End-to-End Optimization
We explain how Reec achieves end-to-end optimization by analyzing gradient flow during training, where the recommendation signal propagates through both the reasoning trajectory and the live item encoding process.
During training, each item e is encoded on the fly by the model as
h fe (xv).
so that both the reasoning trajectory producing ht and the item embeddings h, depend on the same evolving computation rather than on static class weights.
At the recommendation step, the model predicts the ground- trunh item ut via
exp(tiTliit//)
( Dus 01:1)
EVER exp(hTpher/T)
and optimizes the policy objective for the highest-advantage trajectory it:
Leee (0) A,, log
Du. (
The corresponding gradient is
Vo log (o
Σ (1/0'- = -+1--o(v')) Vos(o'), s(v') = hTh
WEB
which expands
Vos(v')= (Volr) TTJ
The second component, (z)
(x shows that gradients flow directly into the item encoding process. Hence, updates from the recommendation signal not only refine the reasoning trajectory that produces ler, but also reshape the item embedding function fo, aligning the semantic encoding of items with the reasoning space.
24

---

User Prompt
Analyze depth and finally recommend next (category> I might purchase inside product </anever>
<ansver> and </ansver> For example,
Below my historical (category) purchases and ratings (out 5):
0% for hist in werccase_bistoree X)
(% (hithht.tinn time_delta) ago: [hist. wtt.:iiteettttaeeeeeee (Chsst.rating) ratingg) X
Item Prompt
Summarize key attributes the following (category) inside
<answer>
</anaver>
for key, attr item.neta XD (key): (attr)
Figure 5: Prompt templates for user interaction history and item metadata. The User Prompt encodes user's past purchases as sequence of item titles, relative timestamps (e.g.. "2hrs", "4d"), and explicit ratings (in [], followed by an instruction to analyze and recommend the next item within the span of </answe>> and </answer> The Item Prompt summarizes structured item attributes (e.g.. brand, type, features) with the same format requirement.
Table 6: Scaling analysis on Gemma backbones. Reasoning consistently improves recommendation performance across model sizes.
Model
N@5
N6:10
H@S
HG 10
Gemma-98-ii wio reasoning
0.0370
0.0460
0.0595
0.0893
R'ec
0.0451
0,0527
0.0640
0.6878
Gemma-2B-it wio reasoning
0.0321
0.0393
0.0469
0.0692
R'ce
0,0388
0.0457
0,0588
0,0800
G Prompt
As illustrated in Figure 5, user interaction histories and item metadata serialized into token sequences via the given prompt templates. To signal the boundary between reasoning and recommen- dation, we introduce control symbol <answer>, which triggers a shift from language modeling head to recommendation head.
H
Limitations
Our work acknowledges two primary limitations. First, introducing explicit reasoning generation inevitably increases inference latency and reduces efficiency due additional autoregressive de- coding steps; nevertheless, exploring the potential of reasoning within recommendation is valuable, and our experiments have empirically confirmed its effectiveness in improving recommendation accuracy. Second, constrained by computational resources, we employed parameter-efficien tuning (LoRA) rather than full-parameter fine-tuning, thus not fully demonstrating the potentially superior performance achievable through comprehensive optimization.
I Extended Analysis
Scaling to Larger Backbones. We further scale Reec to Gemma-9B-it. The results in Table confirm that reasoning consistently improves recommendation across model sizes, and the absolute gain grows with scale (21.7% higher N@5 than its non-reasoning counterpart).
Mooss-Domain Robustness. To further assess generalization to new domains, we extended our evaluation the Electronies, Moviel.ens, and GoodReads datasets, and compared Re with advanced basclines, as shown in Tables 6a, 6c, and 6b, respectively. Reec consistently surpasses prior methods,
25

---

Figure 6: Performance comparison across the Electronics (6a), GoodReads (6b), and MovieLens (6c) datasets. GoodReads and MovieLens evaluations follow the setting in SPRec [39] and SDPO [25], respectively, where baseline values are taken directly from their respective original papers.
(a) Electronic
(b) GoodReads
(c) MovieLens
Mothod
N85
H@S
Method
N85
H@S
Method
H@1
GRU4Ree
10116
0.0157
SASRee
0.0199
0.0202
GRU-Rce
0.3750
Caser
0.0103
0.0164
BigRoc
0.0236
0.0310
Caser
0.3860
SASRee
0.0133
0.0167
RW
0.0281
0.0380
SASRec
0.3440
BigRee
TAIL
D
0.0137
0.0214
SDPO
0.0315
0.0420
MoRec
0.2822
LLaRA
0.1883
0.0279
DMPO
0.0314
0.0410
ChaiRec
R'ec (ours)
0.0203
0.0314
SPRec
0.0330
0.0250
LLaRA
0.4730
RosePO
0.0215
0.0300
SDPO
0.5260
(ours)
R (ours)
0.6211
demonstrating the robustness of reasoning-enhanced recommendation across diverse domains and evaluation settings.
J Analysis on Case Study
To illustrate cemmain-specifin and backbone-specific reasoning behaviors, we analyze representative cases from the CDs and Video Games datasets (Figures 7, 8, 9, 10) using Gemma and Qwen as backbones. 1) On the CDs dataset, both models tend reason by discussing several potentially interesting items and explicitly enumerating their features and genres, before providing final recommendation. This behavior highlights the models' ability to leverage domain knowledge acquired during pretraining and reinforced through RecPO, for better recommendation. 2) On the Video Games dataset, the models shift towards inferring user intent from the purchase sequence, speculating on the likely category intended use of the next purchase. This demonstrates shift from fcature-based enumeration to intent-driven reasoning as the domain changes. 3) Notably, Gemma generally produces shorter and more concise reasoning sequences compared Qwen, which often generates more elaborate justifications. These patterns indicate that, while both models benefit from explicit reasoning, their generation style and reliance pretrained knowledge can manifest. differently across domains and backbone choices.
26

---

Case Study (CD and Vinyl)
User
Analyze depth und finally recommend next CD or vinyl might parchase inside cansver>and </ansver>, For example, Canacer>a product</ /anacer>.
Below is my historical CDs and viay1 purchases and ratings (out of 5):
255d 13.2h ago: [Pure 60'x: The Hiis] (5.0)
255d 12.6h ago: [Seven] (4.0)
ago: [Comeblack]]
88 1.9i ago: [Hot Racks (964)1971)(500)
16.3min [Return Forever] (4.0)
9.4min ago: [Rock Believer] (5.0)
ago: [From Small Things:
Edmunds]] (3.0)
R (Cemma2-2B) 2B)
on your past purchases, seems you enjoy míx appreciate well-crafted Here are some rccomennations. keeping in mind your taste: CD/Vinyl:
a albums with strong vocals and a focus musicality.
soul/funk,
progressive
The Very Best The Rolling Stomes (CD/Vinyl): You'le 've enjoyed classic mekk and this compilation offers great introduction to the Stones' iconic sound. features their biggest hits and showcases their bluesy roots and rock 'n' roll energy.
The Durk Side the Moon (Vinyl): This Pink Floyd musterpiece is classic innovative alhum with a timeless sound. The vinyl format adds special touch to the listening experience.
complex and
Kind of Blue (Vinyl): Miles Davis's iconie jazz album is a must-have for any music lover. It's beautiful introspective album that showcases Davis's genius musician. The vinyl format brings the nuances of the music.
The Miseducation of Lauryn Hill (CD/Vinyl); This album is powerful blend of hip-hop, soul, and R&B. Lauryn truly special listening experience.
incredible vocals and
socially conscious lyrics
Why these recommendations:
Vuriely: These albums offer a mix of genres and styles, reflecting your diverse taste.
Quality: They are critically acclaimed considered essential listening.
Format: The vinyl purtabllity.
unique immersive experience, while CD
Ultimately, the best way to find next faverite album explore and discover new music. sccommendd
Figure Case Study on CDs with Gemma as backbone.
27

---

Case Study 2 (CD and Vinyl)
User
Analyze depth and finally recommend next CD or vinyl might parchase inside canswer>and </ansver>. For example, Canacer>a product</ /anacer>.
Below my historical CDs and vinyl purchases and catings (out 5): 97d 7.7h ago: [Live At The Mocambo]2 CD]I (5.0)
97d 7.5h age: (Radio Silence) (5.0)
24 16.5h ago: [Little Box "O"' Snakes The Sunburst Years 1978-1982) (5.0)
2d 16.0h ago: (Gears Head Soup Super Deluxe] (5.0)
R-oo
you strong preference recordings, classic and deluxe editions. Based on this, here are some recomnendations for CDs viayls that align with your tastes:
your history purchasing rating
"The Wall" (Deluxe Edition)
Artist: Pink Floyd
Format: CD or Vinyl
Description: This legendury album that has been reissued multiple times, including deluxe edition that includes tracks, documentaries, and bonus material. It's must- have for any Pink Floyd fan. Rating: Given your appreciation for deluxe editions classic rock, this would likely receive high rating.
2. "The Dark Side of the Moon" (Super Deluxe Edition)
Artist: Pink Floyd
Format:
CD Vinyl
Description: Another iconic alhum that has been extensively reissudd. The super deluxe edition includes a wealth banus making comprehensive collection for fans.
Rating: Similar to "Goats Head Soup." this would also likely receive high rating due to its extensive content and your appraciation deluxe editions.
3. "Live at the Royal Albert Hall" by David Bowie
Artist: David Bowie
Format: CD Vinyl
Description: This live album captures one David Bawie's most famous performances, showcasing his versatility and the energy his live shows, It's great addition your collection you enjoy live reoordings.
Rating: Given your enjoyment of live recordiags, this would likely be highly rated.
"The Complete Studio Albums" by Fleetwood Mac
Artist: Fleetwood Mac
Format: CD or Vinyl
Description: This box includes all Fleetwood Mac's studio albums, providing comprehensive overview their career. It's a great way explore their evolution band.
Rating: you'se
Mac,
likely highly
quality music
comprehensive nature of the collection.
"The Best of the Beatles" (Deluxe Edition)
Artist: Beatles
Format: CD or Vinyl
Description: compilation that inclodes
Beatles" popular songs, arranged chronological order.
deluxe edition eften includes additional content like rare recordings and documentaries. editions classic
Rating:
your appreciation
would likely high rating.
Conclusion:
Based on your past purchases and ratings, you might want to consider "The Wall" Deluxe Edition), "The Dark Side the Moon' (Super Deluxe Edition), "Live Royal Albent Hall" David Bowie, "The Complete Studio Albams" by Fleetoood Mac, or "The Best of the Beatles" (Deluxe Edition). Each of these options oflers a unique and comprebensive work, aligns with your tasles.
Figure 8: Case Study on CDs with Qwen as backbone.
28

---

Case Study 3 (Video Games)
User
Analyze depth und finally recommend next video game I might purchase insäde canswer>and </ //ansver>. For example, Canacer>a product</ /anacar>.
Below is histerical video games purchases and ratings (our of 5);
80d 1.6h ago: [For Apple Watch Fast Charger C. in Apple MFI Certified 3FT iWarch Charger Magnetic Charging Cable &PD USBC Fast Wall Churger Power Adapter for Apple Watch Series 6/S1/5/4/3/2/ &iPhone 14/13/12] (5.0) 764 7.5h ago: [Kickstand Back Shell Holder Back Bracket for Nintendo Switch Console, Switch Replacement Kickstand
Repair Kit with Screwdriver]
45d 12.5h ago: [VOYEE Switch Controllers Wireless, Campatible with Switch Pro Controllers for Switch/Swicch Litc/OLED/CC, One-Key Pairing Gomepad with 9-Color RGB LEDs 800mAh Battery 6-Axis Motion Wake Function] (5.0)
Switch, Headphones with Mic for Mobile Device, Noise Canceling, Bass Surround, 50mm Driver, 40H Battery) (5.0)
23.5h ago: [Gvvyugke 2.4GHz Wireless Gaming
Gaming
R (Gamaa2-28)
seems enjoy toch accessories and gaming peripherals! Based recent purchases, likely appreciate:
Functionality: You'se to drawn to products that enhance your gaming experience, like the changing cable and the wireless guming headset.
Convenience: You value products that simplify your like the kickstand
Switch.
Quality: You prioritize products that are well-made and reliable, evidenced by your high ratings for the charging cable and gaming headset.
Considering preferences, some
your video gamne purchase: <ansver>
Figure 9: Case Study on Games with Gemma as backbone.
29

---

Case Study 4 (Video Games)
User
Analyze depth und finally recommend next video game I might purchase insäde canswer>and </ </ansver>. For example, Canacer>a product</ /anacer?.
Below is historical video games purchases and ratings (our of 5);
794 15. ago: BUSQUEDA Pro Head Strap with Battery for Quest Magnetic Connection 6000mAh Replaceable VR Power, Lightweight & Balance Elite Strap Mela Accessories for Enhunced Support and Comfort VRJ (5.0)
69d 14.2h ago: (TTwann Stainless VR Stand for Quest Pro, Desk Mount VR Holder and Headset Stand Compatible with
Quest Pro Rift (5.0)
59d 18.9 ago: (ROCCAT Vulcan Max Optical- PC Gaming Keyhoard with Customizable RGB Illuminated
Keys Palm Rest, Titan II Smooth Lincar Switches, Aluminum Plate. 100M Keystroke Durability Black) (5.0)
57d 21.9h ago: IZIYOU LANG K3 Mechanical Geming Keyboard 98 Keys with Number pad Chroma RGB Illuminated
Resistant Chosting
Panel Type-C PC/Mac
Typist Optical Clicky Switch]
(5.0)
ago: [Corsair PRO TKL. Optical anical-Mechanical
Switch Gaming Keyboard
Polling Rate Black] (4.0)
16.8h ago: [ROOK FE87/104 RGB Mechanical Keyboard, Hot Swappable Gaming Keyboard, Customizable Backlit.
Magnet Upper Cover Type-C C. Wired Keyboard for Mac Windows White/Bryen Switch] (5.0)
RPec (Quen2.5-22
Given your purchase history. cleur that your interests Jocus primurily setup gaming peripherals (particularly high-quality keyhourds), and devices that enhance comfort and customizability during play, rather than on specific video games themselves. Based on the trends observed in your history:
VR Content: Multiple purchases of Quest 2/Pro accessories indicate strong inclination towards immersive VR experiences.
Comfort & Ergonomies: Investment premium straps and ergonomic keyboards suggests that gaming comfort
high priotity.
Customizahility and RGB Lighting: The frequent selection of customizable RGB peripherals demonstrates your preference for visually dynamic and personalized setups.
points from your purchases, recommend following types next gen gumes featuring improved immersive experiences for VR:
<enswer>
Figure 10: Case Study on Games with Qwen as backbone.
30


Reviews:
[{"review_id": "ZnREsQ48Ki", "reviewer": "Reviewer_j6Fe", "summary": "In this paper, the authors proposed R2ec, a large‑scale recommender that lets an LLM “think” and rank in the same forward pass, adding a recommendation head to the usual language‑model head so that reasoning tokens are generated before the final item prediction. It is trained with RecPO, a single‑step reinforcement‑learning scheme that combines discrete ranking rewards with a soft similarity signal, eliminating the need for costly human rationale labels.", "questions": "1. Have you run preliminary experiments on more dissimilar verticals (e‑g., Fashion or Electronics) or on non‑Amazon corpora to gauge cross‑domain robustness?\n2. Can you quantify how often the thinking tokens contain multi‑step inference (e.g., preference trade‑offs or counterfactual statements) versus simple profile paraphrases? \n3. (If possible) I'd like to see larger size LLMs", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Pros:\n1. The topic is both intriguing and holds potential for significant developments.\n2. The paper is well-organized and reader-friendly, making it easy to understand.\n4. The experiment results seem promising.\n\nCons:\n1. The datasets presented in the paper is not diverse enough. The study benchmarks R2ec on only three closely related Amazon verticals—Games, CDs & Vinyl, and Musical Instruments.\n2. The so‑called thinking tokens largely paraphrase the user’s historical preferences into a short profile summary. They seldom exhibit multi‑step inference, counterfactual analysis, or explicit preference trade‑offs, so the generated text reflects descriptive recall more than genuine reasoning.\n3. The authors embed the recommendation head directly in the decoder and argue this tight coupling is essential. Yet a plausible alternative—attaching an external ranking layer that consumes the hidden state after the profile summary and is trained end‑to‑end via back‑propagation—is neither implemented nor compared.", "was_revised": true, "final_justification": "Thank you for the comprehensive experiments, many my concerns have been resolved.", "rebuttal": "We sincerely thank the reviewer for recognizing the intriguing nature of our study, the clear organization of the manuscript, and the promising experimental results. We also appreciate the constructive feedback regarding dataset diversity, the depth and nature of reasoning tokens, and the comparison to alternative ranking head architectures.\n\nWe address the above suggestions one by one as follows.\n\n---\n\n> Q1: Dataset Diversity\n> \n\nConsidering your suggestion on “more dissimilar verticals (e‑g., Fashion or Electronics)” and the limited rebuttal time, we have added the Electronics dataset and compared R2ec with the 2 advanced baselines, namely SASRec and. D³. The performance results are summarized as follows:\n\n|  | N@5 | N@10 | H@5 | H@10 |\n| --- | --- | --- | --- | --- |\n| R²ec | 0.020483 | 0.025209 | 0.03137 | 0.046221 |\n| w/o reasoning | 0.019277 | 0.024349 | 0.029368 | 0.045053 |\n| SASRec | 0.011274 | 0.016042 | 0.016686 | 0.031537 |\n| D³ | 0.013726 | 0.01661 | 0.021358 | 0.303687 |\n| Llara | 0.018300 | 0.02123 | 0.027865 | 0.0368763 |\n\nThe results show that R²ecstill performs better on the new dataset, illustrating the cross-domain robustness on diverse domains.\n\n---\n\n> Q2: Reasoning Behavior Analysis — How Does R²ec Reasoning to Recommend?\n> \n\nTo better address your insightful question regarding the reasoning behavior, we conducted an in-depth both qualitative and quantitative analysis of R²ec's reasoning patterns.\n\nWe first engaged in extensive discussions with colleagues to collaboratively analyze and summarize the diverse reasoning behaviors exhibited by R²ec. Through this process, we identified 7 typical reasoning strategies. Building on this framework, we then randomly sampled 100 reasoning outputs from each test set. These samples were annotated and analyzed through a combination of detailed human evaluation and automated assessment with GPT-4.1, ensuring both depth and consistency in our analysis. \n\n1. **Qualitative Analysis.** We found that there are 7 different reasoning behaviors for recommendation, including:\n    - Attribute Abstraction: Identifying underlying attributes of purchased items.\n    - Pattern Recognition & Clustering: Detecting frequent item patterns to form user-interest clusters.\n    - Scenario/Role-Based Reasoning: Inferring user roles or specific use-cases.\n    - Temporal & Sequential Reasoning: Using purchase timing and order for context.\n    - Self-Explanation: Providing explicit rationales for recommendations.\n    - Negative Preference Exclusion: Avoiding previously negatively-rated item types.\n    - Multi-Objective Balancing: Managing trade-offs between conflicting user objectives.\n    \n    These distinct reasoning strategies help explain why and how reasoning aids recommendation. Large recommender model equipped with reasoning can better distill salient user preferences and intentions from noisy or complex behavioral data, abstract patterns beyond surface-level co-occurrence, and generate more contextually appropriate and user-aligned recommendations. This richer interpretive capacity enables the model to move beyond shallow heuristics, providing both higher accuracy and more interpretable results.\n    \n2. **Quantitative Analysis.** To further substantiate our findings, we quantified the distribution of reasoning behaviors across different datasets. As noted, each reasoning output may exhibit multiple behaviors simultaneously. The table below reports the **proportion** of sampled reasoning outputs within each dataset that display each behavior:\n    \n    \n    | Dataset | Attribute Abstraction | Pattern Recognition & Clustering | Scenario/Role-Based Reasoning | Temporal & Sequential Reasoning | Self-Explanation | Negative Preference Exclusion | Multi-Objective Balancing |\n    | --- | --- | --- | --- | --- | --- | --- | --- |\n    | Instruments | 0.95 | 0.99 | 0.98 | 0.22 | 1.00 | 0.07 | 0.72 |\n    | Games | 1.00 | 0.98 | 0.92 | 0.08 | 0.91 | 0.02 | 0.51 |\n    | CDs | 0.99 | 0.94 | 0.74 | 0.16 | 0.82 | 0.06 | 0.23 |\n    | Electronics | 0.97 | 0.84 | 0.94 | 0.12 | 0.97 | 0.15 | 0.76 |\n\n**Note:** Each value reflects the proportion of sampled reasoning outputs within the corresponding dataset that demonstrate a given reasoning behavior.\n\nThis quantitative results highlight several key trends. First, **the distribution of reasoning behaviors differs across datasets**, indicating that after training, the model self-adapts its reasoning patterns to fit the characteristics of each domain and its user-item interactions.\n\nSecond, the variability in these proportions suggests that even within a single domain, R²ec flexibly selects different reasoning strategies based on the specific user sequence.\n\nOverall, these results show that the model does not follow a fixed template but instead employs context-aware reasoning tailored to both the dataset and the individual user, contributing to improved recommendation performance and interpretability.\n\n---\n\n> Q3: Comparison with External Ranking Layer Baseline\n> \n\nWe appreciate the suggestion to compare against an alternative approach—attaching an external ranking layer to the hidden state after the profile summary and training end-to-end via back-propagation.\n\nTo this end, we implemented and evaluated this design under two variants: (1) a ranking layer without explicit reasoning, and (2) a ranking layer with RecPO. As shown in the table below, both approaches result in substantially lower performance compared to our proposed tightly-coupled architecture:\n\n|  | N@5 | N@10 | H@5 | H@10 |\n| --- | --- | --- | --- | --- |\n| ranking layer w/o reasoning | 0.0017 | 0.0019 | 0.0022 | 0.0037 |\n| ranking layer with RecPO | 0.0025 | 0.0027 | 0.0030 | 0.0045 |\n\nThese results underscore that a naively attached ranking layer—even with end-to-end training—fails to fully leverage the model’s reasoning capacity. In contrast, our unified design which utilizes item embedding enables deeper alignment between reasoning and recommendation, resulting in significantly better performance.\n\n---\n\n> Q4: Larger-size LLMs\n> \n\nThanks for your suggestion. We have conducted additional experiments with a larger backbone  Gemma-9B-it, as you recommended. The results on our benchmark datasets are summarized as follows:\n\n| Gemma-9B-it | N@5 | N@10 | H@5 | H@10 |\n| --- | --- | --- | --- | --- |\n| w/o reasoning | 0.037017 | 0.045978 | 0.059524 | 0.089286 |\n| R²ec | 0.045061 | 0.052692 | 0.063988 | 0.087798 |\n\nFor comparison, we also present the results under Gemma-2B-it backbone from the original paper:\n\n| Gemma-2B-it | N@5 | N@10 | H@5 | H@10 |\n| --- | --- | --- | --- | --- |\n| w/o reasoning | 0.0321 | 0.0393 | 0.0469 | 0.0692 |\n| R²ec | 0.0388 | 0.0457 | 0.0588  | 0.080 |\n\nThe results clearly demonstrate the consistent effectiveness of reasoning-augmented recommendation across model scales. With Gemma-9B-it, R²ec achieves a 21.7% improvement in N@5 compared to the version without reasoning, validating that the reasoning mechanism provides fundamental advantages regardless of parameter count. Notably, we observe that as model size increases from 2B to 9B parameters, the absolute performance gains from reasoning also increase, suggesting that larger models are more adept at leveraging reasoning capabilities to enhance recommendation quality.\n\n---\n\n> Summary\n> \n\nIn response to your suggestions, we have expanded our evaluation with additional datasets, conducted experiments with an external ranking layer, tested larger model architectures, and performed detailed analysis of reasoning behaviors in recommendation. These additions have strengthened rather than challenged our work, validating R²ec's robustness across diverse scenarios while more clearly demonstrating the value of reasoning-enhanced recommendation systems."}, {"review_id": "R1oAx77Yng", "reviewer": "Reviewer_jU2r", "summary": "Motiveted by the success of deep thinking in LLMs like DeepSeek R1, this paper explores this question: how can LLM benefit from reasoning to enhance recommendation performance? \n\nThe main challenge lies in the lack of annotated reasoning traces in the recommendation domains. To address this, the authors propose a fused reward which combines discrete ranking rewards with continuous similarity rewards to guide the RL training. Experiments on three datasets demonstrated superiority of the proposed model in terms of recommendation accuracy.", "questions": "Please see the weakness section. \n\nOne more thing: while I list \"Experiments on three datasets demonstrate the proposed model outperform existing baselines with a significant high margin\" as one strength, I observe the improvement of the proposed method is surprisingly high, e.g, in Video Games, improvements can be as high as 104%. According to my years of experience on recommender systems, either there are some problems in your experiments, or your paper is a significant breakthrough which deserves a nomination for best paper. Personally, I tend to take the former hypothesis. How can authors prove that the experiment setting is not problematic? I suggest you can use some scandalized benchmark datasets, in which metrics can be directly aligned with existing papers.", "limitations": "See the weaknesses part. \n\nMeanwhile, the authors also discuss the limitations in the appendix, but the latency discussion is not solid. More analysis on traditional recommender models, such as SASRec, is expected.", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The paper is well written and easy to follow. \n- The studied problem--training reasoning-based LLM4Rec--is important and interesting. \n- Experiments on three datasets demonstrate the proposed model outperform existing baselines with a significant high margin.\n\nWeakness:\n- The novelty of the paper is limited. Both adding classification heads to LMs and RL training pipeline is well setup in existing literature.\n- While the studied research question of this paper:  \"how can LLM benefit from reasoning to enhance recommendation performance? \" , is interesting, the authors failed to deep dive and answer the question in a insightful and systematic way. The authors simply demonstrate by quantitative metrics on recommendation metrics. However, I am more interested to know some fundamental questions:  (1) why reasoning process matter so much for item recommendations? Test-time scaling works effectively for complex problems like Mathematical Olympiad problems and high-order logical tasks, but does item recommendation task involve this complex reasoning nature? (2) Explainable recommender models in literature, such as P5, mainly generate reasons to be displayed to users. But in this paper, the reasons seems not to display to users, but help boost the accuracy of recommender models. \n- Recommender systems usually require real-time serving. The proposed large reasoning recommender LMs may not satisfy the requirement.  \n- Baselines are not enough: nowadays there are a lot of papers discussing leveraging LLMs for recommendations. The authors are suggested to compare with more baselines to make the experiments more convincing. I know there will never be a full list of adequate baselines in this direction, but at least the three baselines in the paper, BigRec, D^3, LangPTune, are far from representing the direction.", "was_revised": true, "final_justification": "After considering the authors’ rebuttal and the discussion, my main concerns remain. The novelty is limited, as the technical contributions closely follow existing literature, and the motivation for using LLM-based reasoning in recommendation is not convincingly demonstrated. The latency analysis is unclear and does not reflect real-world serving conditions, and the comparison with baselines is neither comprehensive nor standardized, making it hard to judge the true effectiveness of the approach. While the paper is well written and the research question is interesting, the empirical results and justifications are not sufficient to outweigh these issues. For these reasons, I cannot recommend acceptance at this time, but I encourage the authors to strengthen their experimental comparisons, provide clearer motivation, and address latency concerns in future revisions.\n\n====\nupdated in the second revision:\nI have raised the initial score as authors provided additional experimental results which hit one of my questions.", "rebuttal": "Thank you for the thoughtful evaluation. We appreciate your recognition of the clarity, the significance of our research problem, and the strength of results.\n\nWe would like to address several misunderstandings on our contributions. Specifically, the novelty, the role of reasoning in recommendation, latency, and experiment result.\nBelow, we provide a detailed response to your 5 primary questions.\n\n> Q1: Novelty.\n\nWhile LLM reasoning advances make its adaptation to recommendation appear straightforward, our results show that **simply applying standard RL and reasoning approaches fails in practice** and critical design choices must be made. We summarize key contributions as **(1) model architect** and __(2) training method__.\n\n1. Not a Classification Head — Architect considerations to couple reasoning and recommendation\n\n\t- Dual architect is suboptimal. Prior work typically adopts LLM as reasoner with recommender. This modular approach increases computation and decouples reasoning from recommendation, limiting effectiveness (see lines 37–47). Our goal is to design a unified architect that jointly accomplishes generative reasoning and discriminative recommendation.\n\t- Dense Embedding over Discrete Codebook. Generative recommendation (GR) often predict items as tokens from codebook, which restricts scalability, generalization to new items, and adds decoding cost. Since reasoning generation is already time-consuming, we use a dense item embedding paradigm for scalability and efficiency.\n\t- Item Semantics is crucial. While the simplest way to optimize dense embeddings in LLM is through a \"classification head\" (denoted as ClsH). Our experiments show that such design, with or without reasoning, performs poorly, under standard baselines like BigRec:\n\n\t\t|  | N@5    | N@10   | H@5    | H@10   |\n\t\t| ----- | ------ | ------ | ------ | ------ |\n\t\t| ClsH w/o reasoning  | 0.0017 | 0.0019 | 0.0022 | 0.0037 |\n\t\t| ClsH with reasoning | 0.0025 | 0.0027 | 0.0030 | 0.0045 |\n\t\t| BigRec| 0.0025 | 0.0037 | 0.0045 | 0.0089 |\n\n\t\t**Summary:** After extensive early-stage exploration and careful considerations, our final architect (i) tightly couples reasoning and rec in a unified model, (ii) leverages scalable, efficient dense item representations, and (iii) enables the model to exploit and update item semantics throughout training.\n\n2. **RL** for Rec is Nontrivial and Requires New Solutions\n\n\t- Reasoning Supervision is Practically Impossible.\n\t\tAnnotated reasonings are feasible to collect at scale for math / code tasks. In recommendation, however, user-level annotated reasoning is impossible to obtain due to the diversity and subjectivity of user intents. RecPO is a **fully self-supervised RL paradigm**, learning reasoning purely from interaction logs—an essential, but highly challenging adaptation for this domain.\n\n\t- Reward Design is Nontrivial.\n\t\tNLP domains benefit from clear reward signals (e.g., corectness). In recommendation, ranking metrics are the only available rewards. We found that directly using such metrics leads to weak result, we thus proposed **fused reward scheme** to combine discrete ranking signals with continuous similarity (Sec. 4.2.2, Table 2), leading to much more effective and stable training.\n\n\t**Summary:** RecPO is the first to enable **fully joint RL optimization** of both reasoning and recommendation without reasoning annotation. By introducing encoding and utilization of item semantics to training objective, and optimizing with fused reward, our approach tightly couples reasoning and ranking, allowing reasoning to directly enhance recommendation.\n\n> Q2: Why reasoning process matters?\n\nWe agree that the reasoning of R²ec differs from mathematics domains. We thus conducted analyses regarding R²ec reasoning behaviors. By qualitatively annotating and clustering reasoning outputs, we found that R²ec typically leverages 7 reasoning strategies. To further substantiate the findings, we quantified the distribution of behaviors across different datasets as follow:\n\n|             | Attribute Abstraction | Pattern Clustering | Role-Based Reasoning | Temporal Reasoning | Self-Explanation | Negative Preference Exclusion | Multi-Objective Balancing |\n| ----------- | --------------------- | ------------------ | -------------------- | ------------------ | ---------------- | ----------------------------- | ------------------------- |\n| Instruments | 0.95                  | 0.99               | 0.98                 | 0.22               | 1.00             | 0.07                          | 0.72                      |\n| Games       | 1.00                  | 0.98               | 0.92                 | 0.08               | 0.91             | 0.02                          | 0.51                      |\n| CDs         | 0.99                  | 0.94               | 0.74                 | 0.16               | 0.82             | 0.06                          | 0.23                      |\n| Electronics | 0.97                  | 0.84               | 0.94                 | 0.12               | 0.97             | 0.15                          | 0.76                      |\n\n(Each value indicates the proportion of outputs exhibiting the behavior. Each output may contain multiple behaviors.)\n\nThis analysis demonstrates that R²ec improves recommendation quality by leveraging diverse and adaptive reasoning strategies, enabling it to capture richer user intent and item semantics beyond shallow co-occurrence patterns.\n\n> Q3 Real-time serving and latency.\n\nWe thank the reviewer for emphasizing real-time serving and latency concerns. \n\nTo address this, we conducted comprehensive latency evaluation, including comparison with both LLM-based Rec baselines and traditional recommenders, summarized as below.\n\n|                  | latency |\n| ---------------- | ------- |\n| SASRec           | 0.014   |\n| LangPTune        | 1.90    |\n| D³               | 4.62    |\n| LLara            | 5.23    |\n| R²ec             | 1.67    |\n| R²ec (with VLLM) | 0.0945  |\n\nThese results demonstrate that R²ec achieves competitive efficiency among LLM-based recommenders, particularly when using modern inference frameworks. We attribute this efficiency in part to our use of scalable item embeddings, which avoids costly autoregressive token decoding over large item codebooks and enables fast retrieval within a unified architecture. We hope this additional analysis provides a comprehensive view of the reasoning overhead and helps clarify the practicality of our approach.\n\nWhile R²ec is slower than lightweight models like SASRec, its unified reasoning–recommendation design brings substantial gains in recommendation quality, and aligns with the community’s ongoing shift toward more capable, explainable models. We believe such advances justify the increased computation and pave the way for efficient, practical LLM-based recommenders as the field continues to optimize for both performance and speed.\n\n> Q4: Baselines.\n\nWe included additional LLM-based baselines — SPRec, SDPO, and LLara. Since LLaRA and SDPO are originally designed as rerankers, , we adapted implementations by removing candidates prompts with constrained generation over item corpus, denoted as LLaRA* and SDPO*. The updated result is shown below:\n\n|        | N@5    | N@10   | H@5    | H@10   |\n| ------ | ------ | ------ | ------ | ------ |\n| SPRec  | 0.0022 | 0.0025 | 0.0029 | 0.0037 |\n| SDPO*  | 0.0018 | 0.0027 | 0.0022 | 0.0052 |\n| Llara* | 0.0039 | 0.0049 | 0.0097 | 0.0127 |\n| BigRec | 0.0025 | 0.0039 | 0.0045 | 0.0089 |\n| D³     | 0.0057 | 0.0076 | 0.0082 | 0.0141 |\n| ours   | 0.0388 | 0.0457 | 0.0588 | 0.0804 |\n\nAs illustrates, R²ec maintains a substantial lead over the expanded set, further validating its effectiveness.\n\n> Q5: Concerns on experiment result.\n\nThank you for raising this important point and for your careful review. We take reproducibility and reliability very seriously, and address your concerns as follows:\n\n1. Dataset Preprocessing.\n\tOur datasets are preprocessed following the D³ methodology (see Appendix for details), but without additional 5-core filtering. This results in a much higher proportion of cold-start cases—e.g., 39.9% of users in the CDs dataset and 34.7% in Instruments are cold-start. In such settings, generative baselines typically struggle, whereas R²ec benefits from richer semantics and demonstrates amplified improvements.\n\n2. Baseline Implementation.\n\tFor baselines such as LLaRA, BigRec, and D³, we strictly follow the original papers and use only the item titles as input features. This limited information further hinders their performance on cold-start items. In addtion, we reported relative improvements ((R²ec - baseline) / baseline), which can look inflated when baseline values are low.\n\n3. Controlled Comparison\n\tFollowing your suggestion, we compared all methods using the same backbone (Gemma-2B) and LoRA adapter on the 2018 5-core CDs dataset. As shown below, R²ec consistently outperforms the version without reasoning (“w/o r”).\n\n|        | N@5    | N@10   | H@5    | H@10   |\n| ------ | ------ | ------ | ------ | ------ |\n| BigRec | 0.0384 | 0.0412 | 0.0449 | 0.0530 |\n| w/o r  | 0.0328 | 0.0384 | 0.0465 | 0.0637 |\n| R²ec   | 0.0386 | 0.0446 | 0.0535 | 0.0726 |\n\nThis result aligns with our expectations and prior findings: as user sequences become more informative, the relative advantage of reasoning diminishes, but the positive effect of reasoning remains robust and measurable.\n\n4. Commitment to Reproducibility. \n\tWe have provided our full codebase in supplementary materials to enable the community to verify and build on our work.\n\nWe hope these steps demonstrate the validity of our findings."}, {"review_id": "CBPFXr6R1u", "reviewer": "Reviewer_ZdkM", "summary": "This paper has explored combining the reasoning capabilities of LLMs into recommendations. The authors found that the existing related works often decouple the reasoning and recommendation process, leading to additional resource cost and suboptimal optimization. To address these problems, the authors devised two heads for one LLM to complete the reasoning and recommendation tasks, respectively. Then, using RL to optimize these two tasks jointly, they achieve a combination of reasoning and recommending. By extensive experiments on three datasets, the effectiveness of the proposed R2ec is validated.", "questions": "All my questions have been included in the weakness section.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- S1. This paper is well-written and well-organized, making it easy to follow.\n- S2. The code has been released.\n- S3. Extensive experiments have been conducted.\n\n\nWeaknesses:\n\n- W1. The term \"large recommender models\" used in this paper is confusing. In general, \"large recommender models\" refers to a typical discriminative recommendation model with a large number of parameters, indicated in the previous works [1, 2]. By comparison, this paper has addressed the LLM as a recommender system.\n- W2. Some notations are not clear. For example, in line 192, what does the reasoning token o represent? Is it the same as the token h in Figure 1?\n- W3. The motivation of this paper is not really reasonable. The authors aim to combine and co-optimize the reasoning and recommending capabilities of LLMs. However, it is still unclear how the reasoning can benefit the final recommendation. Is it really necessary to maintain the reasoning procedure when recommending?\n- W4. Table 1 shows that SASRec often outperforms TIGER and BIGRec, which cannot align with the original papers. I'm confused about it. I suggest that the authors analyze the reason or highlight the differences in experimental settings.\n- W5. This paper belongs to LLMs as RS, but many important baselines have been ignored, such as LLaRA [3] and SPRec [4].\n\n[1]. Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations\n\n[2]. FuXi-𝛼: Scaling Recommendation Model with Feature Interaction Enhanced Transformer\n\n[3]. Llara: Large language-recommendation assistant\n\n[4]. SPRec: Leveraging Self-Play to Debias Preference Alignment for Large Language Model-based Recommendations", "was_revised": true, "final_justification": "The authors have addressed most of my concerns.", "rebuttal": "We thank the reviewer for the positive and constructive feedback. We appreciate your recognition of  writing quality, code release, and experimental efforts. \n\nThere are also some questions including the terms, notations, and baselines. We believe these issues could be clarified in the rebuttal stage, which we address as follow.\n\n---\n\n> 1: On the use of the term \"large recommender models\"\n\nThank you for highlighting this concern. We would like to clarify our terminology as follows.\n\n### 1) Definition and technical equivalence.\nIn prior literature [1,2], “large recommender models” generally refers to recommenders with large number of parameters, typically built on Transformer architectures. Although naming conventions differ, there is no fundamental technical gap between those models and ours; both rely on large-scale Transformer backbones, with the primary difference being whether the model is pretrained as a causal language model.\n\n### 2) Motivation for terminology.\nWe use “large recommender models” to emphasize our focus on unified, end-to-end LLM recommenders that do not require extra modules, in contrast to dual frameworks like LangPTune and SLIM, where LLM is merely the reasoner, but not the recommender(see Related Work). In this sense, the more commonly used term “LLM-based recommender” cannot fully capture this architectural distinction, as many so-called “LLM-based” systems still rely on external modules.\n\nNevertheless, we appreciate your suggestion and will revise the manuscript to more clearly define this terminology in both the Introduction and Related Work sections, making explicit the architectural distinctions and ensuring our positioning is transparent to readers.\n\n---\n> Q2: On notation clarity (reasoning token  $o$, hidden state $h$ )\n\n1. The “reasoning token”   $o$ refers to the output token generated at each step during the model’s autoregressive reasoning process, as stated in line 192; and illustrated as purple blocks in Figure 1, as stated in the legend.\n2. $h$ represents the model’s hidden states, not tokens, as stated in line 138-139, and illustrated in the legend of Figure 1.\n3. To further improve clarity, we will revise the figure by adding a prominent arrow to clearly indicate the correspondence between the reasoning tokens and the illustration.\n\n---\n> Q3: On the motivation and necessity of reasoning in recommendation\n\nThank you for raising this insightful question regarding the necessity of explicit reasoning in recommendation. We will derive this question into the following 3 parts, namely the research necessity, the empirical evidences, and the Reasoning Behavior Analysis.\n\n### 1) Motivation and Research Necessity.\nGenerative recommendation (GR) built upon LLM backbones has already shown strong promise in both industry and academia. While recent advances demonstrate that reasoning is central to LLMs’ performance in complex tasks such as mathematics. Given that recommendation inherently involves intuitive reasoning—such as inferring user intent and abstracting preferences — it is both natural and timely to investigate whether reasoning can yield tangible gains in GR setting. We believe exploring this direction is crucial for advancing the capabilities and practical impact of GR  systems.\n\n### 2) Empirical Evidences.\n1. Ablation studies (Section 5.3) demonstrated that our model consistently outperforms a variant trained solely with contrastive loss and *without* reasoning. This provides quantitative evidence that incorporating reasoning tokens leads to better recommendation performance.\n\n2. In direct response to “is it really necessary to maintain the reasoning procedure”, we performed an additional ablation where R²ec is forced to output recommendations *without* generating any reasoning (denoted as w/o r), comparing with the reasoning one (denoted as w r). As shown below, removing reasonings consistently degrades performance across all domains.\n\n\t||**N@5**|**N@10**|**H@5**|**H@10**|\n\t|-|-|-|-|-|\n\t|CDs w/o r|0.0222|0.0277|0.0327|0.0469|\n\t|CDs w r|0.0388|0.0457|0.0588|0.0804|\n\t|Video Games w/o r|0.0145|0.0184|0.0220|0.0341|\n\t|Video Games w r|0.0205|0.0271|0.0288|0.0532|\n\t|Instruments w/o r|0.0156|0.0178|0.0224|0.0309|\n\t|Instruments w r|0.0161|0.0203|0.0264|0.0397|\n\t|Electronics w/o r|0.0169|0.02160|0.0265|0.0414|\n\t|Electronics w r|0.0205|0.0252|0.0314|0.0462|\n\nThese experiments directly confirm that **explicit reasoning is not only useful, but necessary for maximizing the performance of generative recommendation models**.\n\n### 3) Reasoning Behavior Analysis.\nTo directly address the question of “how reasoning benefits the final recommendation, \"we conducted an in-depth both qualitative and quantitative analysis of R²ec's reasoning patterns.\n\nWe first engaged in extensive discussions with colleagues to collaboratively analyze and summarize the diverse reasoning behaviors exhibited by R²ec. Through this process, we identified 7 typical reasoning strategies. Building on this framework, we then randomly sampled 100 reasoning outputs from each test set. These samples were annotated and analyzed through a combination of detailed human evaluation and automated assessment with GPT-4.1, ensuring both depth and consistency in our analysis.\n\n1. **Qualitative Analysis.** We found that there are 7 different reasoning behaviors for recommendation, including:\n\n\t- Attribute Abstraction: Identifying underlying attributes of purchased items.\n\t- Pattern Clustering: Detecting frequent item patterns to form user-interest clusters.\n\t- Role-Based Reasoning: Inferring user roles or specific use-cases.\n\t- Temporal Reasoning: Using purchase timing and order for context.\n\t- Self-Explanation: Providing explicit rationales for recommendations.\n\t- Negative Preference Exclusion: Avoiding previously negatively-rated item types.\n\t- Multi-Objective Balancing: Managing trade-offs between conflicting user objectives.\n\n2. **Quantitative Analysis.** To further substantiate our findings, we quantified the distribution of reasoning behaviors across different datasets. As noted, each reasoning output may exhibit multiple behaviors simultaneously. The table below reports the **proportion** of sampled reasoning outputs within each dataset that display each behavior:\n\t|Dataset|Attribute Abstraction|Pattern Clustering|Role-Based Reasoning|Temporal Reasoning|Self-Explanation|Negative Preference Exclusion|Multi-Objective Balancing|\n\t|-|-|-|-|-|-|-|-|\n\t|Instruments|0.95|0.99        |0.98     |0.22       |1.00|0.07     |0.72  |\n\t|Games|1.00|0.98        |0.92     |0.08       |0.91|0.02     |0.51  |\n\t|CDs|0.99|0.94        |0.74     |0.16       |0.82|0.06     |0.23  |\n\t|Electronics|0.97|0.84        |0.94     |0.12       |0.97|0.15     |0.76  |\n\n\tNote: Each value reflects the proportion of sampled reasoning outputs within the corresponding dataset that demonstrate a given reasoning behavior.\n\n\tThis quantitative results highlight several key trends. First, the distribution of reasoning behaviors differs across datasets, indicating that after training, the model self-adapts its reasoning patterns to fit the characteristics of each domain and its user-item interactions. Second, the variability in these proportions suggests that even within a single domain, R²ec flexibly selects different reasoning strategies based on the specific user sequence.\n\nOverall, these results demonstrate that the model employs context-aware reasoning tailored to both the dataset and the individual user, contributing to improved recommendation performance and ultimately proving its necessity.\n\n---\n> Q4: On SASRec performance and discrepancies with original papers\n\nThank you for pointing out this discrepancy. As described in Appendix C, our SASRec implementation uses cross-entropy (softmax) loss instead of binary cross-entropy (BCE), which significantly boost performance. We highlight that such findings are reasonable and consistent with previous findings [1], which reported that softmax-based SASRec can outperform many larger recommendation models. We do value your concerns and will highlight this implementation detail more clearly in the main text to avoid confusion.\n\n---\n> Q5: On missing baselines\n\nThank you for highlighting the importance of recent works such as LLaRA[2] and SPRec[3].\n\nTo ensure broader coverage of recent advances, we supplemented our original baselines with three additional LLM‑based recommenders—SPRec, SDPO, and LLara. We note that LLaRA and SDPO are originally designed as rerankers in a candidate‑based setting, thus not comparable to our end‑to‑end generation framework. To provide fair comparison, we have adapted their implementations by removing the candidates prompts and instead performing **constrained generation over the full item corpus**, denoted as LLaRA* and SDPO*.\n\nThe updated comparison is shown below:\n||N@5|N@10|H@5|H@10|\n|-|-|-|-|-|\n|SPRec|0.0022|0.00246|0.0029|0.0037|\n|SDPO*|0.0018|0.00273|0.00223|0.00521|\n|Llara*|0.0039|0.00488|0.009673|0.01265|\n|BigRec|0.0025|0.0039|0.0045|0.0089|\n|D^3|0.0057|0.0076|0.0082|0.0141|\n|ours|0.0388|0.0457|0.0588|0.0804|\n\nWe hope these additions and clarifications adequately address your concern.\n\n---\nIn summary, we sincerely thank the reviewer for the constructive feedback. We greatly appreciate your recognition of our writing, code release, and experiments, as well as your thoughtful suggestions on terminology, notation, and baselines. We have carefully addressed each concern,, providing detailed clarifications, new analyses, and additional experiments where possible. We hope these responses further strengthen the value and validity of our work.\n\n\n[1] Are LLM-Based Recommenders Already the Best? Simple Scaled Cross-Entropy Unleashes the Potential of Traditional Sequential Recommenders\n[2] LLaRA: Large Language-Recommendation Assistant\n[3] SPRec: Self-Play to Debias LLM-Based Recommendation\n[4] On Softmax Direct Preference Optimization for Recommendation"}, {"review_id": "6X60YJV1DR", "reviewer": "Reviewer_idCa", "summary": "This paper proposes a new method to leverage the capabilities of reasoning LLMs into the process of item recommendation. In more details, the paper introduces a method called R^2ec which consists of a reasoning LLM with an integrated recommendation head which allows for joint training of the \"reasoning\" and the \"recommendation\" components (i.e., combining generative and discriminative components into a single model).\n\nThe proposed method first describes the user interaction history into a prompt and is passed to an LLM. The final hidden state of the sequence generated by the LLM is then passed to the recommendation head, which performs an inner product with the embeddings of the items (obtained by feeding the textual description of the items through the LLM and using the last hidden state) to obtain a score for each item (higher score indicates the item is a better recommendation to the user). The model is trained using reinforcement learning (with a GRPO-like approach) with a reward composed of two components: one based on the NDCG ranking metric, and the other on a contrastive loss (promoting similarity with items in teh ground truth, and dissimilarity with other randomly sampled items).\n\nThe experimental section compares the proposed method with traditional and LLM-based baselines, and shows teh proposed method always achieves better results. A comprehensive set of ablation studies is performed to study the effects of the reasoning component, and on the stability and sensitivity of the reinforcement learning procedure based on the hyperparameters of the method.", "questions": "- It would be interesting to see how the method behaves in the cold start scenario (where there is no, or limited, history for a user/item). Have the authors performed any analysis of this?\n- Could the authors please provide some information about the overhead runtime caused by the reasoning process?\n- Could the authors mention why they only considered the latest 20 actions for each user? What happens when more actions are considered? Does the proposed method become too slow? How do the baselines scale (both in terms of runtime and performance) with longer history?\n- Is the number of parameters of the models used for the proposed method comparable to that of the considered baselines?\n- I've always found a bit strange the use of the embedding from the <EOS> token as a sentence embedding, as LLMs are not explicitly trained to put much information into this embedding. I understand that <EOS> has access to the whole sentence, but it still feels arbitrary. Have the authors tried different strategies to obtain the embeddings for a sequence (e.g., Mean/Max Pooling, or introducing special tokens)? The following papers may be of interest:\n\t- LLMs are Also Effective Embedding Models: An In-depth Overview, Tao et al.\n\t- LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders, BehnamGhader et al. \n\t- Pooling And Attention: What Are Effective Designs For LLM-Based Embedding Models?, Tang et al.", "limitations": "Yes, although I think it could be greatly improved by providing some numbers for the latency increase caused by the method", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The ablation study is very well done and studies in depth all the components of their method\n- The quantitative results reported in the paper show the proposed method is significantly stronger than the considered competitors \n\nWeaknesses:\n- Reasoning models can be much slower than \"non-reasoning\" ones. This can be particularly problematic for real-world applications. Nevertheless, there is no discussion in the main paper about runtime. There is a limited discussion in the Appendix, but this could be greatly improved with some actual numbers\n- The experiments shown in the paper are for a scenario with limited context (the authors say \"Each user’s interaction history is then chronologically sorted and\ntruncated to the latest 20 actions\"). This seems very far from real-world settings.\n- There is no baseline with a simple pre-trained model that has received no fine-tuning/RL. I think adding such a baseline would be a great addition to the paper as it would really highlight the potential benefits of the proposed method. \n- No discussion on the \"cold-start\" scenario.", "was_revised": true, "final_justification": "The authors have addressed my concerns and added new experiments which I think will make the paper more valuable, therefore I raise my initial score", "rebuttal": "Thank you for your detailed and constructive review. We appreciate your recognition of our ablation study and empirical results. We acknowledge your concerns regarding inference latency, the limited user history setting, the absence of a pure pre-trained baseline, and the lack of cold-start analysis. \n\nWe will address each of these points in detail in our response below.\n\n---\n\n> Q1: Inference Latency and Runtime Analysis\n> \n\nWe appreciate the reviewer’s question regarding the runtime overhead introduced by the reasoning process.\n\nFirst, we would like to clarify that the runtime of R²ec have been reported in Appendix G (“Analysis on Inference Efficiency”), the quantitative results on per-sample latency under different batch sizes. \n\nSecond, to provide more detailed comparison and analysis, we have expanded our latency evaluation with additional benchmarks using consistent hardware (RTX 3090) and unified implementation settings. Below, we present the runtime (in seconds per sample) for R²ec and its VLLM serving version, against various baselines.\n\n|  | Runtime (s/sample) |\n| --- | --- |\n| LangPTune | 1.90 |\n| D³ | 4.62 |\n| LLara | 5.23 |\n| R²ec | 1.67 |\n| R²ec (with VLLM) | 0.0945 |\n\nThese results demonstrate that R²ec achieves competitive efficiency among LLM-based recommenders, particularly when using modern inference frameworks. We attribute this efficiency in part to our use of scalable item embeddings, which avoids costly autoregressive token decoding over large item codebooks and enables fast retrieval within a unified architecture. We hope this additional analysis provides a comprehensive view of the reasoning overhead and helps clarify the practicality of our approach.\n\n---\n\n> Q2: Short User History and Limited Context\n> \n\nThank you for this insightful observation.\n\nFirst, truncating user histories to the most recent actions is a common practice in the recommendation literature, primarily due to the structure of public datasets and for fair comparison across baselines.\n\nSecond, the LLM-based baselines all truncated sequences to 10 recent actions in their original experiment settings; in fact, we increased this window to 20 in our experiments to better retain semantic and sequential information, supporting stronger reasoning capabilities.\n\nNevertheless, we fully acknowledge such setting does not fully capture the complexity of real-world recommendation scenarios, where user histories are often much longer and richer. Addressing long-context modeling remains an important and challenging direction for the field.  \n\nDue to time and resource constraints, we were unable to reprocess all the datasets and conduct additional experiments with extended histories in this submission, but we do recognize this as an important aspect to explore in future work.\n\n---\n\n> Q3: Adding Zero-shot Baseline\n> \n\nThank you for this valuable suggestion. We agree that including a baseline using a pure pre-trained LLM—without any fine-tuning or RL—would help clarify the benefits of our approach.\n\nWe now report results for this \"zero-shot\" baseline. As shown below, raw pre-trained LLMs achieve very low performance on recommendation tasks. This further highlights the significant gains achieved by our proposed training scheme.\n\n- Qwen\n    \n    \n    |  | N@5 | N@10 | H@5 | H@10 |\n    | --- | --- | --- | --- | --- |\n    | CDs | 0.000692 | 0.000692 | 0.00148 | 0.0022321 |\n    | Video Games | 0.00016886 | 0.00023456 | 0.00028342 | 0.00048811 |\n    | Instruments | 0. | 0.00034 | 0. | 0.000975 |\n- Gemma\n    \n    \n    |  | N@5 | N@10 | H@5 | H@10 |\n    | --- | --- | --- | --- | --- |\n    | CDs | 0.00064089 | 0.00090592 | 0.00148 | 0.0022321 |\n    | Video Games | 0.000317 | 0.00047956 | 0.00047237 | 0.00094473 |\n    | Instruments | 0. | 0.0001155 | 0. | 0.0003253 |\n\n---\n\n> Q4: No Discussion on Cold-start Scenario\n> \n\nThank you for highlighting the importance of the cold-start scenario. In response, we conducted additional analysis on both “cold” (unseen user or item) and “warm” (seen user/item) settings. Results (NDCG@10) are shown below:\n\n|  | CDs (cold) | CDs (warm) | Games (cold) | Games (warm) | Instruments (cold) | Instruments (warm) |\n| --- | --- | --- | --- | --- | --- | --- |\n| R²ec | 0.0306 | 0.0448 | 0.0264 | 0.0282 | 0.0168 | 0.0225 |\n| SASRec | 0.0057 | 0.0216 | 0.0150 | 0.0280 | 0.0080 \t | 0.0213 |\n\nAs shown, R²ec consistently outperforms SASRec in both cold and warm scenarios, especially under the more challenging cold-start setting. This demonstrates the strong generalization and robustness of our method.\n\n---\n\n> Q5: Model Parameter Comparability\n> \n\nFor all LLM-based baselines, the parameter count is kept strictly consistent by adapting LORA finetuning to ensure fair comparison. For traditional (non-LLM) baselines, model sizes are inherently smaller by design; for clarity, we will provide a summary table of model sizes in the appendix.\n\n> Q6: Sequence Embedding Extraction Strategies\n> \n\nThank you for raising this important point and for sharing relevant recent works. Your concern strongly resonates with my own initial explorations—in the early stages of this project, I experimented with several sequence embedding strategies, but I did not observe substantial or consistent improvements over the last hidden state, so I adopted it as the default for simplicity and robustness.\n\nRegard the papers. I found your recommended works extremely insightful—especially the discussion on enabling bidirectional encoding. Within the LLM + reasoning + embedding architecture, systematically investigating how to construct more effective embeddings is a highly promising direction for future research.\n\nIn direct response to your suggestion, I have now re-evaluated these embedding strategies in controlled experiments. The results are as follows:\n\n|  | N@5 | N@10 | H@5 | H@10 |\n| --- | --- | --- | --- | --- |\n| last hidden state (ours) | 0.0388 | 0.0457 | 0.0588  | 0.0804 |\n| max pooling | 0.038158 | 0.044549 | 0.059524 | 0.072917 |\n| mean pooling | 0.036831 | 0.045287 | 0.052827 | 0.079637 |\n| special tokens | 0.035255 | 0.043157 | 0.050851 | 0.073661 |\n\nThese results suggested that the last hidden state currently yields the strongest performance, while mean pooling improves training stability (though, due to rebuttal constraints, I cannot show the full curves here). Special token strategies yield weaker performance in our setting, likely because we adopt LoRA fine-tuning and no large-scale optimization on the LLLM backbone, it is thus challenging for the model to effectively learn to generate and utilize newly introduced tokens for global sequence representation.\n\n---\n\n**Summary**\n\nWe thank the reviewer again for the detailed feedback, which has prompted us to significantly strengthen our empirical analysis and clarify methodological choices. In the revised manuscript, we will incorporate all of the above additions—including expanded latency analysis, the zero-shot baselines, cold-start results, reporting of model sizes, and a systematic comparison of embedding strategies. We believe these enhancements address your concerns and further demonstrate the robustness and transparency of our approach."}]

Author rebuttal:

Meta-review: The authors address the novel and interesting question of how LLMs can leverage reasoning for enhanced recommendation performance.  Reviewers agree that the paper is well written and the research question is interesting.  They raise concerns regarding potential alternative architectures, the need for more baselines (improved during rebuttal), limitations of the three related datasets (one dissimilar dataset added on rebuttal), and most importantly the need for reasoning behavior analysis.  This last question was addressed quite comprehensively and insightfully during the rebuttal phase and represents a significant novel contribution to the nascent "reasoning for recommendation" that should ideally be incorporated as a centerpiece of the main paper's empirical analysis on revision.  In light of the extensive author discussion and experimentation to address reviewer concerns, reviewers are generally in favor of acceptance post-rebuttal.  The AC concurs with acceptance with the expectation that the authors will do their best to incorporate all rebuttal discussion/content into the final version of their paper.
Decision: Accept (poster)