Think before Recommendation: Autonomous Reasoning-enhanced Recommender
Authors: Xiaoyu Kong
Junguang Jiang
Bin Liu
Ziru Xu
Han Zhu
Jian Xu
Bo Zheng
Jiancan Wu
Xiang Wang
Venue: NeurIPS 2025 poster ()
Keywords: Recommendation
Large Language Models
Reinforcement Learning
Rating Prediction
Primary area: other

Abstract: The core task of recommender systems is to learn user preferences from historical user-item interactions. With the rapid development of large language models (LLMs), recent research has explored leveraging the reasoning capabilities of LLMs to enhance rating prediction tasks. However, existing distillation-based methods suffer from limitations such as the teacher model's insufficient recommendation capability, costly and static supervision, and superficial transfer of reasoning ability. To address these issues, this paper proposes RecZero, a reinforcement learning (RL)-based recommendation paradigm that abandons the traditional multi-model and multi-stage distillation approach. Instead, RecZero trains a single LLM through pure RL to autonomously develop reasoning capabilities for rating prediction. RecZero consists of two key components: (1) "Think-before-Recommendation" prompt construction, which employs a structured reasoning template to guide the model in step-wise analysis of user interests, item features, and user-item compatibility; and (2) rule-based reward modeling, which adopts group relative policy optimization (GRPO) to compute rewards for reasoning trajectories and optimize the LLM. Additionally, the paper explores a hybrid paradigm, RecOne, which combines supervised fine-tuning with RL, initializing the model with cold-start reasoning samples and further optimizing it with RL. Experimental results demonstrate that RecZero and RecOne significantly outperform existing baseline methods on multiple benchmark datasets, validating the superiority of the RL paradigm in achieving autonomous reasoning-enhanced recommender systems.

Think before Recommendation: Autonomous Reasoning-enhanced Recommender
Xiaoyu Kong' Junguang Jiang' Bin Liu' Ziru Xu'
Han Zhu Jian Xu² Bo Zheng' Jiancan Wu3.4* Xiang Wang"*
Taobao & Tmall Group of Alibaba, China National University of Singapore
Institute of Dataspace, Hefei Comprehensive National Science Center
Shanghai Key Laboratory of Data Science
linlin wwyyaallabaaaiii com
wuj muijca@@mmii.com com xi x1angwang12220@mai..coo
Abstract
The core task of recommender systems is to learn user preferences from historical user-item interactions. With the rapid development of large language models (LLMss, recent research has explored leveraging the reasoning capabilities of LLMs to enhance rating prediction tasks. However, existing distillation- suffer from limitations such as the teacher model's insufficient recommendation capability, costly and static supervision, and superficial transfer of reasoning ability. To address these issues, this paper proposes RecZero, a reinforcement learning (RI.)-based recommendation paradigm that abandons the traditional multi-mode and multi-stage distillation approach. Instead, RecZero trains a single LLM through pure RL to autonomously develop reasoning capabilities for rating prediction. RecZero consists of two key components: (1) wwwaek-effoo---commeeee prompt construction, which employs structured reasoning template to guide the model in step-wise analysis of user interests, item features, and user-item compatibility; and (2) rule- reward modeling, which adopts group relative policy optimization (GRPO) to compute rewards for reasoning trajectories and optimize the I.LM. Additionally, the paper explores a hybrid paradigm, RecOne, which combines supervised fine-tuning with RI., initializing the model with cold- start reasoning samples and further optimizing it with RI. Experimental results demonstrate that RecZero and RecOne significantly outperform existing baseline methods on multiple benchmark datasets, validating the superiority of the RL paradigm in achieving autonomous eeasoning-enhanced recommender systems. Our codes are available at https: //gt thub. com/ Akla Kooon//eeeeeo....
methods
1 Introduction
Recommender models aim to learn user preferences on items from historical user-item interactions (e.g., ratings, clicks, purchases) [1-7]. Motivated by the rapid progress of large language models (LLMs) [8-11], recent research has explored adapting LLMs as recommenders, leveraging their strengths world knowledge, semantic understanding. and reasoning. A promising direction is to leverage I.I.Ms' reasoning capabilities [12-16] to enhance rating prediction [17-19] a core recommendation task that explicitly models user preferences by predicting their ratings on items.
Upon scrutinizing prior studies on such Gaasonng-enhaneee rating predictors [20-25], we can summarize a common pipeline: (1) Given user's rating on target item, teacher model often
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Q1: Extract user Interest
Reasoning Prediction y
target Item
Inputx
Teacher LLMs
Reasoming trucc
Ss
distill
SP7
Braluate Compatbility
Reasoning Supervision
Past paradigms
Intput
RecZero
prodile
history
R
A
user
reviews
Og
Rule-based reward
R2
Group Computation
A2
meta-data
:
Og
R₂
A
Q: You are helphtl assistant, Your task summarize key aspects of the target item, and then evaluate user-ttem compatibility produce final rating prodiction
Reasoning trace
Prediction 9
Optimizable Reasoning
analyee user: [i Creativve fresh - [dislike] Nono,
match The target items gripping thriller
analyze Intricately
Figure 1: Comparison between RecZero and Conventional Paradigmss. Conventional paradigms train multiple student LLMSs using diverse query-based datasets with reasoning traces, while RecZero uses pure RI. to train a single L.M for the entire workflow. Unlike conventional methods reliant on Teacher models, RecZero leverages Reward Signals jointly optimize reasoning traces and predictions, improving efficiency and performance.
strong general-purpose LLM such as ChatGPT [8] first extracts the user's preferences over item attributes from historical interactions, then generates possible intermediate reasoning steps interpret the observed rating; (2) Each reasoning trace, paired with the corresponding user rating, forms an instruction sample; (3) smaller model is then supervised fine-tuned on these reasoning-enhamne instruction samples to distill the teacher model's 's extraction and reasoning capabilities.
Despite recent progress, identify several limitations inherent in this distillation paradigm, as evidenced Figure 1:
Limited Recommendation Capability of the Teacher. General-purposs, teacher models lack comain-specifee knowledge for recommendation, producing reasoning traces that are misaligned with the rate prediction objective. Consequently, these suboptimal reasoning processes hinder the student's 's performance.
Costly and Static Supervision. Generating high-quality reasoning data at scale, whether via human annotation [26] or LLM API calls [27, 28]. is both time- and resource-intensive. Moreover, the student model restricted to passively consuming static teacher outputs, with no chance to actively refine optimize the reasoning process.
Superficial Transfer of Reasoning Ability. While the student model fine-tuned on teacher- generated data can reproduce reasoning traces, it often imitates surface-level patterns, rather than acquiring genuine reasoning skills [29]. This results in overfitting to the distribution of teacher instructions, limiting generalization to unseen tasks [29, 30].
To address the limitations of existing distillation-based methods, we draw inspiration from the recent success of reinforcement learning (RL) in LLM post-training [31-33], particularly DeepSeek-Ri R1 Zero [32], and propose a simple yet effective RL paradigm for LLM-based recommendation, dubbed RecZero. Discarding the distillation paradigm iigm that requires different models and disjoint stages (i.e., information extraction and reasoning process generation of the teacher model, supervised fine-tuning of the student model), RecZero trains single LLM via pure RL to develop autonomous reasoning capabilitics for rating prediction. composed two key components:
Think Seink-before-te www.rhooeooooooonnteeeeeeeo Recommendation" Prompt Construction. Beyond the standard prompt contain- ing a user's history and target item, we introduce structured reasoning template with special tokens elicit tepp-wise analysis: "<analyze user> ... </analyze user>" prompts the model

---

to extract user interest from her/his historical interactions: "<analyze item> </analyze item>" summarizes key aspects of the target item; "<match>... ... </match>" evaluates user-item compatibility based on the analyzed information; and consequently "<rate>... ... </rate>" pro- duces the final rating prediction. decomposes the rating prediction task into discrete steps and employs chain-of -hain-of-thought reasoning.
Rule-based Reward Modeling, We adopt group relative policy optimization (GRPO) [31] to optimize the LLM with rule-based rewards. Given prompt, the model samples multiple reasoning trajectories. For each trajectory, the reward is computed as the difference between the ground-truth rating and the predicted rating in the rate step. For the group of trajectories, we derive the relative advantage over the average reward as the signal to optimize the LLM.
As a result, unlike distillation-batt based methods that risk overfitting to surface-level reasoning traces, RecZero enables the LLM to acquire reasoning ability through interaction and optimization. The uni- fied reasoning steps - <analyze user>, <analyze item>, <match>, and <rate> - are jointly trained to reflect and refine toward better recommendation performance, guided by recommendationn specific objectives. Beyond the pure RL approach of RecZero, we also explore a hybrid paradigm inspired by RL with cold start [32], termed RecOne. RecOne first constructs small set of cold-start reasoning samples tailored for recommendation tasks, which are used to initialize the L.M via supervised fine-tuning. This warm-start model is then further optimized using RL enhance its reasoning capabilities. By combining data-efficient initialization with task-specific RL, RecOne aims achieve faster convergence and stronger performance in recommendation reasoning. We assess the effectiveness of RecZero and RecOne through extensive experiments on four benchmark datasets (e.g., Amazon-book, Amazon music [34], Yelp [35], IMDb [36]). showcasing the RL paradigm's superiority over distillation methods (e.g.. Rec-SAVER [20]. EXP3RT [22], Reason4Rec [21]).
2 Preliminary
2.1 Reasoning-enhanced Rating Prediction
The primary goal of rating prediction task is to predict the user's ratings for items that align with user preferences. Let U denote a set of users, I a set of items, and y the set of possible ratings. Formally, consider user E U with a historical interaction sequence represented as H. =< hn,1, hu....... Nu,t 2, where hy,; = (M; iu du,s) is a triplet that includes the meta-informmtion M; of the item i, the user's rating Vu,, y of the item, and the user's review du,i- The prediction rule is defined as follows:
D..i argmaxlP (yu.i kiH.,,M.) kEy
where Vu.s denotes the predicted rating for user u on item i, represents the model parameters.
In the context of eenhanceeg rating prediction, LLM parametered by 0 receives the interaction history Hy of user u and the meta-information M; of the item i as input Fu,i. Subsequently, the LLM performs an explicit reasoning process and makes the final user rating prediction, expressed as:
fu,is Du.i (LM((T..)) = LLM (H.,
where fu, is introduced as the reasoning trace and I... represents the predicted rating. our framework, the reasoning process is decomposed into analyzing user preferences and extracting the characteristics of the target item, followed by a matching analysis.
3 Methodology
To address the major challenges inherent in existing reasoning-enhanced rating prediction methods, we propose RecZero. This innovative approach leverages pure reinforcement learning to encourage the model to jointly optimize the four critical processes of user analysis, item analysis, matching, and rating under a unified eecommendation-speciff objective. To further enhance the model's s performance recommendation tasks, we introduce RecOne, building upon RecZero. RecOne employs a specialized sampling method to obtain high-quality reasoning traces, which are used to perform SFT for cold-startee sold-starting initial model. Subsequently, we conduct spec-siii RL to achieve faster convergence and more robust recommendation performance, illustrated in Figure 2.
3

---

protile
RecZero
history
usse
Extract User Interest
Sumamarise tie Item Aspects
Evallate Compasibiitt
Rating Prediction
meta-data
Toem
reviews
Canalyre user>> The user like...Th The dislike... </analyre user>
Query: Your task to extract user interest, summarize key sspects
analyse Item--Godd herbol... Drtes out the hair... </ analyze item
<match>>
preference ...5/match>>
LLM
evaluate compatibilit produce final rating prediction
signal
<rato>> 4.0 </mae>>>
G output rollout
Inputx
RecOne
RecZero pipeline
Ressoning
Input
Teacher
Interpentation
LLM
Teacher LLM R
Yirsor r Byy
Distilled LLM
Prediction
Prediction
Labely
Figure The RecZero and RecOne frameworks utilize a unified target to optimize multi-step reasoning within coupled structure, achieving reasoning-enhanced predictions specifically designed for recommendation scenarios.
3.1 RecZero: Pure RL Paradigm for Reasoning-enhanced Recommendation
3.1.1
Prompt Construction
Previous studies [21, 22] have demonstrated that in reasoning-enhancee recommendation scenarios, requiring LLMs to first explicitly extract user preferences and item features, and then deliberate on the match between users and items, significantly enhance the accuracy of LLMs in predicting users' actual ratings. High-quality rating predictions often rely on clear and high-quality extraction processes, whereas user preferences and item features typically lack explicit ground truth. To address this, we mandate that LLMs strictly output user preferences and item feature extractions according to predefined rules, and employ mule-base reward mechanism to guide LLMss in self-optimization without the need for labeled data, thereby replacing the previous approach of using frozen LLMs with few-shot learning for these tasks. We have designed system prompt guide the LLM in generating reasoning trajectories beneficial to the rating prediction task. Specifically, we divide the reasoning process into three parts: evaluation. For a given user history and item metadata, we mandate that the LLM constrains the outputs of different steps within the specified token boundaries, as shown in Appendix.
interest extraction, item aspects summarization, and compatibility
3.1.2 Rule-based Reward Modeling
Rule-based approaches [31] can provide stable training rewards for RL. To address the issue of suboptimal performance caused by inconsistent training objectives in previously decoupled modules, we integrate user preference summarization, item feature distillation, and match deliberation within a unified framework and optimize them through unified reward rule. For the format reward, first define the correct format as follows:
1. The model's thinking process should be encapsulated within the <analyze user> </analyze user>, <analyze item> </analyze item>, and <match>
</match> tags, respectively.
2. The generated output must within the <rate> ... </rate> tags and free of any unread- able content.
Based on the above format requirements, the format reward is defined as follows:
0.5 if the format is correct
Rforrat =
(3)
0.5
if the format is incorrect

---

In our training process, adhering to the continuous rating paradigm established in prior work [21, 22], we employ Rule-based Reward Modeling encourage the model to approximate actual user ratings as closely as possible, rather than relying solely on correctness rewards. This objective can be formally expressed as:
Ranswer
max__erorr
where max_ error max_error is the maximum allowable error (which can be set based on the task, egg., when the rating range is from to 5, max_ _eror 4). The final total reward can be formally expressed as = format + Runswer. Unlike previous LLM-based paradigms, we do not design a separate task that requires the LLM to output integer score that is then converted into decimal rating via logit-weighted decoding. Compared with the SFT paradigm that imitates integer target labels, reinforcement learning offers clear advantage: by crafting an appropriate reward function, encourage the LLM to predict decimal ratings directly in its response, thereby earning a higher MAE reward and removing the need for the logit-weighted step altogether.
3.1.3 Group Relative Policy Optimization
We use Group Relative Policy Optimization (GRPO) [31] for RI. policy optimization. In this approach, for given input x², the method generates set of outputs (y1, V2.-.- .G using the existing policy Taa The policy model is then refined by maximizing the following objective function:
To (Vi,z (r, Wi.sce)
(G) =IE D. more (1ar)
G
min
Tow (3/5, I ve,
A
(5)
}l.
clip
To To Ir (e,
c, 1 1
c
ALee)
3Kx.. [ [l
Toae (Ji, Vi,co))
where 3 is a parameter that adjusts the trade-off between the task-specific loss and the KL-divergence. The advantage term A, derived from the rewards of the group of responses {R1 Ro.... Ro} and is calculated as:
R₂ - mcan ( [R1, .e..... RGl)
A
(6)
std (RR,, Ro..... Ra))
This formulation ensures that the policy optimization process is guided by the relative performance of the generated outputs within each group, promoting more stable and efficient learning.
3.2 RecOne: Distillation and RL Paradigm for Reasoning-enhancee Recommendation
While pure reinforcement learning approaches like RecZero show promise, we recognize opportunities for further enhancements. Specifically, the discrepancy between pretrained LLMs' training corpus and recommendatioonsspecffe data creates a domain gap that the RL paradigm must gradually bridge. Previous studies [37, 38] have demonstrated that LLMs can enhance their capabilities through self- generated reasoning processes, In this light, we propose a hybrid paradigm, RecOne, which first establishes solid foundation through cold-start supervised fine-tuning on high-quality reasoning trajectories, then leverages the RecZero framework to further refine its reasoning capabilities.
3.2.1 Cold-Start Supervised Fine-Tuning
The first phase of RecOne addresses the foundational reasoning capabilities necessary for effective recommendations. We first perform a warm-up using recommendation data with long reasoning trajectories. Specifically, dataset D, which is utilized in RecZero. For each data pair (x,y) E Deubs we leverage teacher LLM (e.g., DeepSeck-RI) with superior reasoning capabilities to generate reasoning trajectory (f, ū) given the input x, following Equation (2). When the predicted rating ý aligns with the ground truth consider the generated reasoning to be high-quality; While for cases where does not match y, we feed both z and to the teacher model, requiring to generate a rationalized trajectory prat that
sample M data samples to construct subset Dous from the complete
For notational clarity, we omit the user and item indices, while retaining other subscripts in the subsequent parts.
5

---

aligns with the correct rating. This process creates reasoning trajectory dataset that consists of two complementary subsets:
Dirace = Datign U missalign:
(7)
Dalign {2z, f ① v) 87.
(8)
D missaign
{(x, fra ①
y A T. y
Here, Datign represents examples where the teacher LLM's 's reasoning naturally leads the correct rating prediction, while misalinn captures cases requiring corrective reasoning to reach the proper conclusion. The training objective formulated an autoregressive model optimization problem:
maXX
Σ
log (o trace (c, Met )
trace
(10)
(m. Dusse
where yue represents the concatenation of reasoning trajectory and ground-truth rating in Equa- tions (8) and (9), 0 denotes the LLM's model parameters, Y/ represents the 1-th token in the output sequence, and y<u includes all preceding tokens in the sequence.
4 Experiments
In this section, we first demonstrate the performance improvement of the RecZero and RecOne framework the rating prediction task. We requested the dataset from Reason4Rec [21] and conducted our experiments based on We conduct extensive experiments on real-world datasets, including Amazon-book, Amazon-music, and Yelp, to evaluate the effectiveness of the proposed RecZero and RecOne framework. Our analysis includes detailed comparison of RecOne with existing baseline models, which cover CF-based models [39], Review-based models [40, 17, 18] and LLM-based recommendation models [20-22, 25].
We employed Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) evaluation metrics to reflect the model's accuracy in rating prediction. More details about these baseline models, datasets, and metrics can be found in Appendix. Additionally, studies to identify the key components that enhance the performance of RecOne, with a particular focus the roles of the "thinking process" and the "format-guided model." Besides accuracy studies, we also compare the training and inference cost of RecZero against previous LLM-based paradigms. In short, we aim address the following research questions:
conducted comprehensive ablation
RQ1: How does RecZero and RecOne perform in comparison to other baseline methods?
RQ2: Does the initial capability of the model have an impact on the upper limit RL performance?
RQ3: What is the impact of the designed components on the recommendation performance of RecOne?
RQ4: How efficient is RecZero in terms of training and inference cost compared with the prior reasoning-enhaneee baseline?
4.1 Performance Comparison (RQ1)
We conduct a holistic evaluation, considering metrics of both MAE and RMSE across Book, Music and Yelp datasets to demonstrate the effectiveness of our framework. This section comprehensively compares RecZero and RecOne against CF-based, Review-based and L--M-based baselines.
From Table 1 we observe that RecOne ranks first all six evaluation metrics across the three domains. Concretely, it lowers the best previous RMSE by 6.7%, 12.2% and 6.2% on Book, Music and Yelp respectively, while cutting the MAE by 16.8% 29.9% and 7.5%.When it relies solely on RL inside the LLM, with neither cold-start pre-intiialization nor guidance from a teacher model, RecZero still surpasses all baselines in terms of MAE on the three datasets.
This section delves into profound impact of the RecOne cold-start model on subsequent training processes. On the book dataset, we systematically compared the training trajectories of the distilled model obtained under the RecOne framework with those of the model trained using the initial model following the RecZero paradigm for RL, as illustrated in Figure 3.

---

Table 1: The Results of RecZero and RecOne compared with Traditional models and LLMs-based methods.
Book
Music
Yelp
Methods
RMSE
MAE
RMSE
MAE
RMSE
MAE
CF-based
MF
0.8565
0.6277
0.8142
0.6188
1.0711
0.7980
Review-based
DeepCoNN
0.8403
0.6211
0.8057
0.6034
1 1.0665
0.8312
NARRE
0.8435
0.6242
0.7881
0,5799
1.0785
0.8177
DAML
0.8371
0.6214
0.7848
0.5703
1.0405
0.7964
LLM-based
Rec-SAVER
0.9356
0.6645
0.9262
0.6463
1.1282
0.8295
EXP3RT
0.9346
0.6042
0.8312
0.5548
1.1420
0.8236
Reason4Rec
0.8325
0.5937
0.7647
0.5352
0.9972
0.7473
Ours
RecZero 0.8387
0.5253
0.7058
0.4271
1 1.0521
0.7429
RecOne 0.7784
0.5017
0.6776
0.3816
0.9774
0.7012
4.2 In-Depth Analysis of RecZero and RecOne (RQ2)
Notably, the initial model of RecZero exhibited an anomalous increase MAE during the first 20 train- ing steps, primarily attributed to the initial LLM fo- cusing on acquiring relatively mnsier-to-obtain format rewards. In contrast, the distilled model of RecOne demonstrated higher format scores at the onset of training, enabling it to optimize the answer scores for the rating task more rapidly during RL training. Throughout the displayed 200 training steps, the Re- cOne model consistently outperformed RecZero, a re- sult that unequivocally demonstrates the effectiveness of the cold-start strategy in enhancing the model's initial capabilities, thereby significantly elevating its performance ceiling for RL optimization. During the early stages of RL training, the MAE drops sharply, then decreases slowly with minor fluctuations, and finally stabilizes over longer training period.
MAE over Training Steps
0.60
RecOne
0.55
RecZero
0.45
0.40
0.30
50
100
150
200
Steps
Figure 3: The performance of RecZero and RecOne over training steps.
4.3 Ablation Study (RQ3)
To validate the effectiveness of each component in the Rec-zero framework, compare it with the following alternative approaches:
We eliminate the required thinking process in the model output, requiring the model to directly predict the specific rating value based on the input.
No Thinking Process:
No Multi-step Thinking: We replace the multi-step thinking process <analyze user>, <analyze item> and <match> with a single-step <think> without imposing specific format requirements.
We single correctness reward the reward signal. Specifically, when the predicted score exactly matches the target, a correctness reward of points is given; otherwise, it is assigned 0.
Correctness Reward Only:
Only SFT for warm-start: We discard the step in Rec-one that jointly optimizes reasoning and rating prediction through reinforcement learning, and instead only use the Teacher Model to generate trace data for cold-start scenarios.
7

---

MAE for Ablation Study
RMSE for Ablation Study
Thinking Frecess
2.0-
No Multi-soo Thinking
Mallicdap Thinking
Carretinos Roward Only
MAE.
L.--
Ouly SET for warn-start
RMSE
Only SFT for warm-taat
RecOne
1.0-
0.5-
0.0
Yelp
IMDb 0.0
Book
Music
Yelp
MAE
RMSE
Figure 4: 4a and 4b separately show the MAE and RMSE performance of RecOne and its four ablation variants on four datasets.
The experimental results are illustrated in Figure 4. Our RecOne model demonstrates significant superiority over alternative approaches in both MAE and RMSE metrics. Through experimental observations, we can discern that the No Thinking Process baseline underperforms the No Multi- step Thinking version in both metrics, which in turn is inferior to the complete RecOne model employing the three-step thinking process (Extract User Interest, Summarize Item Key Aspects, Evaluate Compatibility). This robustly validates the positive impact our proposed strategy of explicitly guiding the model through multi-step thinking rating prediction accuracy. Furthermore, the alternative version Correctness Reward Only exhibits a notable decline both MAE and RMSE metrics compared to RecOne. This phenomenon reward mechanism encourages the model to generate integer ratings that exactly match user scores, rather than decimal values approximating the actual ratings. Such mechanism suppresses the model's tendency to produce predictions close to actual ratings, instead promoting aggressive predictions that completely align with target scores. Finally, the contrast model employing Only SFT for warm-up demonstrates the poorest performance among all baseline methods. This strongly substantiates the significance of our proposed approach of collaboratively optimizing the reasoning process and rating prediction task through RL, acquiring reasoning ability through interaction and optimization, rather than merely obtaining surface-level reasoning through SFT for enhancing the model's predictive accuracy.
be attributed to the fact that the correctness-onts only
4.4 Cost-Effeetiveness and Practical Deployment (RQ4)
We follow exactly the same experimental protocol as in the previous sections and, the Amazon- Music dataset, additionally evaluate two further variants:
RecZero(carly-stop) training stops once it attains the same MAE as the external best SFT baseline;
RecOne(SFT) the RL phase is removed and the model is trained with SFT until convergence.
Table 2 reveal four main findings:
Larger performance gain per unit cost. The pure-SFT variant RecOne(SFT) consumes 20 labelled instances, ~0.6 GPU-hours, and 597 inference tokens per request reach an MAE of 0.6472. By contrast, Rec7Zero(early-stry already surpasses this score (MAE = 0.5419) while using only 0.48 K labels (-97.6%) 0.4 GPU-hours (-33%), and significantly smaller serving budget (331 tokens). Extending RL to full convergence (RecZero) further pushes the error down to MAE/RMSE 0.427 0.427 100.7058. achieving a 21.5% MAE reduction against the strongest Pure-SFT baseline (Reason4Rec) with merely 12% of its labels labels-yieldin more than 5x gain in error reduction per JK labels.
Fast adaptation to cold-start and distribution drift. Commercial recommenders face constant influx of new items and shifting user interests. RecZero can be re-optimized with merely a few hundred fresh on-line interactions collected within minutes, whereas SFT pipelines must accumulate and curate a much larger batch before retraining.
The simplest pipeline among reasoning-enhen enhanced methods. RecZero keeps (1) a single model, (2) single training stage, (3) no distillation teacher and (4) end-to-end optimization, resulting in markedly lower engineering overhead than multi-stage alternatives such as EXP3RT or Reason4Rec, which each juggle three models and at least two training phases.

---

Table 2: Training and inference cost of different reasoning enhanced recommenders
Method
Paradigm
Samples
Models
Training Time
Avg.Inf. Tokens
Inf.Stages
Inf Token
EXP3RT
SFT
20K
1.2h
187.63
562.89
ReasondRec
SFT
20K
175.49
350.98
RccZZerofcrrl-stoo)
RL
0.48K
0.4h
331.64
331.64
RecOne(SFT)
SFT
20K
0.6h
597.32
597.32
RecZero
RL 2.4K
1.1h
310.52
310.52
RecOne
SFT+RL
2.6K
1.4h
412.79
412.79
RL complements rather than replaces SFT. RecOne(full) shows that starting from SFT warm- start and applying short RL refinement can push the error down to MAE/RMSE = 0.3816/0..mm 16/0.677. In practice, one may first perform quick SFT pass large historical logs, then lightweight RecOne-style RL fine-tuning on the latest or strategically important traffic.
In summary, the experiments reveal four clear takeaways. First, pure SFT is outperformed by pure reinforcement learning: despite requiring 20 K labelled samples, longer training time and almost twice the inference cost, RecOne(SFT) still lags behind the early-stopped RL variant, RecZero(early- stop). Second, when allowed to converge fully, RecZero further cuts MAE by 21.5 % relative to the strongest SFT baseline, without demanding additional computational resources. Third, both RecZero and RecOne can be re-optimized using only few hundred fresh interactions, making them well suited to sparse-data or drifting-distributee scenarios. Finally, both methods preserve a single-model. teacher- free pipeline, offering a markedly simpler and more cost-efficient solution than existing multi-stage reasoning-enhanen recommenders.
5
Related Work
5.1
LLM-based Explainable Recommendation
The remarkable capabilities demonstrated by LLMs in world knowledge understanding and contextual processing have inspired researchers to leverage LLMs for handling rich semantic information to provide recommendation suggestions [41-45]. DRDT [46] employs a prompt-based approach that requires L.MSs to analyze user preferences from multiple dimensions, enhancing sequential recommendations through critic prompts for reflection. GOT4Rec [23] encourages LLMs engage in multi pulli-pperspective thinking for recommendations using the graph of thought strategy. However, these methods primarily adopt few-shot approaches for recommendation tasks, constrained by the inherent limitations of LLMs in the recommendation domain.
Recent studies have explored fine-tuning LLMs to optimize their reasoning capabilities for recommen- dation. Rec-SAVER [20] utilizes larger-scale LLMs to generate rationalized intermediate reasoning, fine-tuning smaller models to enhance their recommendation reasoning abilities. EXP3RT and Reason4Rec [22, 21] decompose the reasoning process into multiple steps for independent training, distilling the reasoning capabilities of larger models into smaller ones, and separately optimizing rating tasks for better performance. While these efforts have significantly enhanced the accuracy of LLMs in performing reasoning-enhanne rating prediction tasks, challenges such as suboptimal reasoning trajectories and decoupled training objectives remain unresolved.
5.2
LLM Post-training
As the most widely-used post-training technique, SFT quickly adapts a pretrained LLM to downstream tasks, yet its impact on model generalization remains a matter of debate. Recent studies have investigated how SFT influences the generalization of foundation models. SFT Memorizes, RL Generalizes [30] shows that SFT tends to memorize training data and thus generalizes poorly to out-of-distribution scenarios, while RL strengthens model's core abilities and enhances domain generalization. SFT or RL [29] systematically compares SFT and RL and reveals that SFT often leads to the imitation of "pseudo reasoning paths" generated by expert model. These paths may look similar to the native reasoning produced by RL-trained models, yet they usually contain redundant, hesitant, low-information steps and sometimes
incorrect reasoning.
RL [47, 48] operates through the interaction mechanism between agents and the environment, re- warding correct actions with the core objective of maximizing cumulative returns. In recent years,

---

researchers have introduced RL into the fine-tuning process of LLMs. Among these approaches, Reinforcement Learning with Human Feedback (RLHF) [49] typically employs the Proximal Policy Optimization (PPO) [50] algorithm to achieve alignment with human preferences. However, the PPO algorithm faces technical challenges related to excessive memory consumption. To simplify the application of RI. in LLMs,, researchers have proposed innovative methods such as Direct Prefer- ence Optimization (DPO) [51]. In the context recommendation systems, the s-DPO method [52] combines DPO with the softmax loss function, enhancing LLMs' capability in mining hard negative samples for recommendation systems. Although these methods have improved efficiency, they still face challenges such as off off-policy issues and performance ceilings that fall short of online methods. The recently proposed Group Relative Policy Optimization (GRPO) [31] further reduces memory requirements by estimating group scores, while replacing traditional reward models with rule-based reward mechanisms. Through this innovative paradigm, L.MMs can significantly enhance their reason- ing capabilities and domain-specific performance (e.g., in mathematics and programming) without requiring annotated data [32, 33). Recent studies [53-55] have further explored the applications of RL in mathematics and general domains. Despite these advancements, the application of RL in improving rating prediction performance through reasoning capabilities remains unexplored.
Conclusion
In this work, we introduced RecZero, a novel RL paradigm for LLM-based recommendation systems, addressing key limitations of conventional distillation-based methods. By unifying reasoning and recommendation into single LLM trained through structured prompts and rule-based rewards, RecZero eliminates the need for separate teacher-student models and enables continuous optimization of reasoning processes. Our experiments across multiple benchmarks demonstrate RecZero's superi- ority over existing approaches, highlighting its ability to develop autonomous reasoning capabilities aligned with recommendation objectives.
Additionally, we explored RecOne, a hybrid paradigm combining supervised fine-tuning with RL, which further elevates the performance ceiling of the model in RL. Both approaches showcase the potential of RL in advancing recommendation systems, paving the way for more adaptive and efficient models.
Acknowledgments and Disclosure of Funding
This research is supported by the Fundamental Research Funds for the Central Universities (WK2100250065) and the National Natural Science Foundation of China (62302321).
10

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly outline the contributions, including the proposed RecZero and RecOne paradigms, their advantages over distillation-basee methods, and the experimental validation on benchmark datasets.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We discuss the limitations of the work performed in Appendix. By explicitly acknowledging these limitations, we provide a balanced view of our work and suggest directions for future research address these challenges.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: we provide the full set of assumptions and a complete (and correct) proof in our methodology part 3. The assumptions are clearly stated, and the proofs are detailed to ensure correctness and reproducibility.
Guidelines:
The answer NA means that the paper does not include theoretical results,
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We fully disclose all the information needed in Appendix. This includes details about the datasets, experimental setups, hyperparameters, and evaluation metrics.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
12

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: Due to internal company code requirements and user privacy concerns, we will not be providing open access the data and code at the time of submission. However, are actively working on this process and, after ensuring the security of the relevant code and data, we commit open-sourcing them in the final version of the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
hatpc/guidsss/oooooe cettsuummissioooooooooiooooo oolicylly for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / /nips.
CodeSubmiss asionPolicy) Policy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Justification: We provide open access to the data and code. And we explain our set in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We do in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
13

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: For each experiment we proviede sufficient information in our experiments part Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: : /ttp//nuuiip..e/ ctcp/pubii//hiiiccciieeeleee
Answer: [Yes]
Justification: Our research fully conforms to the NeurIPS Code of Ethies. We have carefully reviewed the guidelines and ensured that all aspects of our research, from data collection and usage to experimental conduct and reporting, adhere to ethical standards. We maintain transparency in our methodology, respect for privacy, and fairness in our experimental design, ensuring that our work aligns with the ethical principles outlined by NeurIPS.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
14

---

Answer: [Yes]
Justification: Our paper discusses the potential positive and negative societal impacts of work in the "Broader Impacts" section. By acknowledging these impacts, provide balanced view of our work and suggest mitigation strategies to address potential negative outcomes.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: We conduct experiments on recommendation tasks using publicly available datasets and build upon existing open-source methods. Therefore, the risk of misuse is minimal, and specific safeguards for data or model release are not necessary. Our focus remains transparency and reproducibility, ensuring that our work can be used responsibly by the research community.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-us should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
15

---

Justification: We fully adhere to these requirements by properly crediting the creators or original owners of all assets used in our paper. Each asset, including datasets, code, and models, is accompanied by citation to the original source, and the specific versions used are stated. We explicitly mention the licenses and terms of use for each asset, ensuring compliance with their respective usage guidelines.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version URL.
asset used and, possible, include
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We commit to open-sourciin our code and datasets in the officially published
version.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
We are not involved in these risks as we are only engaged in recommendation tasks. Our research does not include crowdsoureing experiments or research with human subjects, thus this question is not applicable to our paper.
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
16

---

15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: We are not involved in these risks as we are only engaged in recommendation tasks. Our research does not involve human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: LLM is used only for formatting purposes.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: /ttp///ppss.//oooeeeeeeeeeeeeeee for what should or should not be described.
17

---

References
[1] J. Ben Schafer, Joseph A. Konstan, and John RiedI. Recommender systems in e-commerce. In Proceedings of the First ACM Conference on Electronic Commerce (EC-99), Denver, CO, USA, November 3-5, 1999. ACM, 1999.
[2] Alejandro Valencia-Arian Arias, Hernán Uribe-Bedoyy Juan David Gonzalez-Ruiz, Gustavo Sánchez Santos, Edgard Chapoñan Ramírez, and Ezequiel Martínez Rojas. Artificial intelligence and recommender systems in e-commerce. trends and research agenda. Intell. Syst. Appl., 2024.
[3] Sebastian Lubos, Alexander Felfernig, and Markus Tautschnig. An overview of video recom- mender systems: state-of-the-ee and research issues. Frontiers Big Data, 2024.
[4] Paul Covington, Jay Adams, and Emre Sargin. Deep neural networks for youtube recommen- dations. In Proceedings of the 10th ACM Conference on Recommender Systems, Boston, MA, USA, September 15-19, 2016. ACM, 2016.
[5] Shaina Raza and Chen Ding. News recommender system: a review of recent progress, chal- lenges, and opportunities. Artif. Intell. Rev., 2022.
[6] Chuhan Wu, Fangzhao Wu, Yongfeng Huang, and Xing Xie. Personalized news recommenda- tion: Methods and challenges. ACM Trans. Inf. Syst., 2023.
[7] Mattia Giovanni Campana and Franca Delmastro. Recommender systems for online and mobile social networks: survey. Online Soc. Networks Media, 2017.
[8] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, 1lge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXi::2303.08774, 2023.
[9] An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao. Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang. Qin Zhu, Rui Men. Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. CORR, 2024.
[10] DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangho Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J. L. Cai, Jian Liang, Jianzhong Guo, Jiaqi Ni, Jiashi Li, Jiawei Wang, Jin Chen, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, Junxiao Song, Kai Dong. Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Lei Xu, Leyi Xia, Liang Zhao, Litong Wang, Liyue Zhang, Meng Li, Miaojun Wang, Mingchuan Zhang, Minghua Zhang. Minghui Tang, Mingming Li, Ning Tian, Panpan Huang, Peiyi Wang. Peng Zhang, Qiancheng Wang, Qihao Zhu, Qinyu Chen, Qiushi Du, R. J. Chen, R. Jin, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang. Runxin Xu, Ruoyu Zhang, Ruyi Chen, S. S. Li, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shaoqing Wu, Shengfeng Ye, Shengfeng Ye, Shirong Ma, Shiyu Wang. Shuang Zhou, Shuiping Yu, Shunfeng Zhou, Shuting Pan, T. Wang, Tao Yun, Tian Pei, Tianyu Sun, W. L. Xiao, and Wangding Zeng. Deepseek-v3 technical report. CORR, 2024.
[11] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshomn Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurélien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Rozière, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dicuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily
18

---

Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Graeme Nail, Grégoire Mialon, Guan Pang. Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Iharra, Isabel M. Kloumann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang. Jiawen Liu, Jie Wang. Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, and et al. The llama 3 herd of models. CoRR, 2024.
[12] Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin. Large language models for mathematical reasoning: Progresses and challenges. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2024: Student Research Workshop, St. Julian's 's Malta, March 21-22, 2024. Association for Computational Linguistics, 2024.
[13] Chengpeng Li, Guanting Dong, Mingfeng Xue, Ru Peng, Xiang Wang, and Dayiheng Liu. Dotamath: Decomposition of thought with code assistance and self-correction for mathematical reasoning. CoRR, 2024.
[14] Shima Imani, Liang Du, and Harsh Shrivastava. Mathprompter: Mathematical reasoning using large language models. In Proceedings of the The 61st Annual Meeting of the Association for Computational Linguistics: Industry Track, ACL 2023, Toronto, Canada, July 9-14, 2023. Association for Computational Linguistics, 2023.
[15] Jundong Xu, Hao Fei, Liangmingg Pan, Qian Liu, Mong-Li Lee, and Wynne Hsu. Faithful logical reasoning via symbolic thain-of-thought. Association for Computational Linguistics, 2024.
[16] Xiaoxue Cheng, Junyi Li, Wayne Xin Zhao, and Ji-Rong Wen. Think more, hallucinate less: Mitigating hallucinations via dual process of fast and slow thinking. CoRR, 2025.
[17] Chong Chen, Min Zhang, Yiqun Liu, and Shaoping Ma. Neural attentional rating regression with review-level explanations. In Proceedings of the 2018 World Wide Web Conference on World Wide Web, WWW 2018, Lyon, France, April 23-27, 2018. ACM, 2018.
[18] Donghua Liu, Jing Li, Bo Du, Jun Chang, and Rong Gao. DAML: dual attention mutual learning between ratings and reviews for item recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019. ACM, 2019.
[19] Jie Shuai, Kun Zhang, Le Wu, Peijie Sun, Richang Hong, Meng Wang, and Yong Li. review- aware graph contrastive learning framework for recommendation. In SIGIR '22 The 45th International ACM SIGIR Conference on Reseurch and Development in Information Retrieval, Madrid, Spain, July 11 15, 2022. ACM, 2022.
[20] Alicia Tsai, Adam Kraft, Long Jin, Chenwei Cai, Anahita Hosseini, Taibai Xu, Zemin Zhang, Lichan Hong, Ed Huai-hsin Chi, and Xinyang Yi. Leveraging LLM reasoning enhances personalized recommender systems. In Findings of the Association for Computational Linguis- lics, ACI. 2024, Bangkok, Thailand and virtual meeting, August / 1116, 2024. Association for Computational Linguistics, 2024.
[21] Yi Fang, Wenjie Wang, Yang Zhang, Fengbin Zhu, Qifan Wang, Fuli Feng, and Xiangnan He. Reason4rec: Large language models for recommendation with deliberative user preference alignment. CoRR, 2025.
[22] Jieyong Kim, Hyunseo Kim, Hyunjin Cho, ScongKu Kang, Buru Chang, Jinyoung Yeo, and Dongha Lee. Review-driven personalized preference reasoning with large language models for recommendation. CoRR, 2024.
[23] Zewen Long, Liang Wang, Shu Wu, Qiang Liu, and Liang Wang. Got4rec: Graph of thoughts for sequential recommendation. CoRR, 2024.
19

---

[24] Hanjia Lyu, Song Jiang, Hanqing Zeng. Yinglong Xia, Qifan Wang, Si Zhang, Ren Chen, Christopher Leung. Jiajie Tang, and Jiebo Luo. Llm-rec: Personalized recommendation via prompting large language models. In Findings of the Association for Computational Linguis- fics: NAACI. 2024, Mexico City, Mexico, June 16-21, 2024. Association for Computational Linguistics, 2024.
[25] Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. Recommendation as language processing (RLP): A unified pretrain, personalized prompt & predict paradigm (P5).
In RecSys, 2022.
[26] Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL id=v8LOpN6EDi.
wttpeevwwwwwwww.oreeeeeeeee net/forum?
[27] Wenjun Peng, Guiyang Li, Yue Jiang, Zilong Wang, Dan Ou, Xiaoyi Zeng, Derong Xu, Tong Xu, and Enhong Chen. Large language model based long-tail query rewriting in taobao search. In Tat-Seng Chua, Chong- Woong-WWahaa Ngo, Roy Ka-Wei Lee, Ravi Kumar, and Hady W. Lauw, editors, Companion Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, Singapore, May 13-17, 2024. ACM, 2024.
[28] Yupeng Hou, Jiacheng Li, Zhankui He, An Yan, Xiusi Chen, and Julian J. McAuley. Bridging language and items for retrieval and recommendation. CoRR, 2024.
[29] Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training rl rike reasoning large vision-langee languale models, 2025.
[30] Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoe V. Le, Sergey Levine, and Yi Ma. SFT memorizes, RL generalizes: A comparative study of foundation model post-training. CoRR, 2025.
[31] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li. Y. Wu, and Daya Guo. Deepscekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, 2024.
[32] DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu Hui Li. Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, J. Chen, R. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang. Shuiping Yu, Shunfeng Zhou, Shuting Pan, and S. S. Li. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. CoRR, 2025.
[33] OpenAl, ;, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed Ki-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Camey, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, Andre Saraiva, Andrea Vallone, Andrew Duberstein, Andrew Kondrich, Andrey Mishchenko, Andy Applebaum, Angela Jiang, Ashvin Nair, Barret Zoph, Behrooz. Ghorbani, Ben Rossen, Benjamin Sokolowsky, Boaz Barak, Bob McGrew, Borys Minaiev, Botao Hao, Bowen Baker, Brandon Houghton, Brandon McKinzie, Brydon Eastman, Camillo Lugaresi, Cary Bassin, Cary Hudson, Chak Ming Li, Charles de Bourcy, Chelsea
20

---

Voss. Chen Shen, Chong Zhang, Chris Koch, Chris Orsinger, Christopher Hesse, Claudia Fischer, Clive Chan, Dan Roberts. Daniel Kappler, Daniel Levy. Daniel Selsam, David Dohan, David Farhi, David Mely, David Robinson, Dimitris Tsipras, Doug Li, Dragos Oprica, Eben Freeman, Eddie Zhang, Edmund Wong. Elizabeth Proehi, Enoch Cheung, Eric Mitchell, Eric Wallace, Erik Ritter, Evan Mays, Fan Wang, Felipe Petroski Such, Filippo Raso, Florencia Leoni, Foivos Tsimpourlas, Francis Song, Fred von Lohmann, Freddie Sulit, Geoff Salmon, Giambattista Parascandolo, Gildas Chabot, Grace Zhao, Greg Brockman, Guillaume Leclere, Hadi Salman, Haiming Bao, Hao Sheng. Hart Andrin, Hessam Bagherinezhad, Hongyu Ren, Hunter Lightman, Hyung Won Chung, Ian Kivlichan, Ian O' Connell, Ian Osband, Ignasi Clavera Gilaberte, Ilge Akkaya, Ilya Kostrikov, llya Sutskever, Irina Kofman, Jakub Pachocki, James Lennon, Jason Wei. Jean Harb, Jerry Twore, Jiacheng Feng, Jiahui Yu, Jiayi Weng, Jie Tang, Jieqi Yu, Joaquin Quiñonero Candela, Joe Palermo, Joel Parish, Johannes Heidecke, John Hallman, John Rizzo, Jonathan Gordon, Jonathan Uesato, Jonathan Ward, Joost Huizinga, Julie Wang, Kai Chen, Kai Xiao, Karan Singhal, Karina Nguyen, Karl Cobbe, Katy Shi, Kayla Wood, Kendra Rimbach, Keren Gu-Lemberg, Kevin Liu, Kevin Lu, Kevin Stone, Kevin Yu, Lama Ahmad, Lauren Yang, Leo Liu, Leon Maksin, Leyton Ho, Liam Fedus, Lilian Weng, Linden Li, Lindsay McCallum, Lindsey Held, Lorenz Kuhn, Lukas Kondraciuk, Lukasz Kaiser, Luke Metz, Madelaine Boyd, Maja Trebacz, Manas Joglekar, Mark Chen, Marko Tintor, Mason Meyer, Matt Jones, Matt Kaufer, Max Schwarzer, Meghan Shah, Mehmet Yatbaz, Melody Y. Guan, Mengyuan Xu, Mengyuan Yan. Mia Glaese, Mianna Chen, Michael Lampe, Michael Malek. Michele Wang. Michelle Fradin, Mike McClay, Mikhail Pavlov. Miles Wang, Mingxuan Wang, Mira Murati, Mo Bavarian, Mostafa Rohaninejad, Nat McAleese, Neil Chowdhury, Neil Chowdhury, Nick Ryder, Nikolas Tezak, Noam Brown, Ofr Nachum, Oleg Boiko, Oleg Murk, Olivia Watkins, Patrick Chao, Paul Ashbourne, Pavel Izmailov, Peter Zhokhov, Rachel Dias, Rahul Arora, Randall Lin, Rapha Gontijo Lopes, Raz Gaon, Reah Miyara, Reimar Leike, Renny Hwang. Rhythm Garg, Robin Brown, Roshan James, Rui Shu, Ryan Cheu, Ryan Greene, Saachi Jain, Sam Altman, Sam Toizer, Sam Toyer. Samuel Miserendino, Sandhini Agarwal, Santiago Hernandez, Sasha Baker, Scott McKinney. Scottie Yan. Shengjia Zhao, Shengli Hu, Shibani Santurkar, Shraman Ray Chaudhuri, Shuyuan Zhang. Siyuan Fu, Spencer Papay, Steph Lin, Suchir Balaji, Suvansh Sanjeev, Szymon Sidor, Tal Broda, Aidan Clark, Tao Wang. Taylor Gordon, Ted Sanders, Tojal Patwardhan, Thibault Sottiaux, Thomas Degry. Thomas Dimson, Tianhao Zheng, Timur Garipov, Tom Stasi, Trapit Bansal, Trevor Creech, Troy Peterson, Tyna Eloundou, Valerie Qi, Vineet Kosaraju, Vinnie Monaco, Vitchyr Pong. Vlad Fomenko, Weiyi Zheng, Wenda Zhou, Wes McCabe, Wojciech Zaremba, Yann Dubois, Yinghai Lu, Yining Chen, Young Cha, Yu Bai, Yuchen He, Yuchen Zhang, Yunyun Wang, Zheng Shao, and Zhuohan Li. Openai ol system card, 2024. URL https: / /arxiv.vvvv org/ abs/
[34] Yupeng Hou, Jiacheng Li, Zhankui He, An Yan, Xiusi Chen, and Julian J. McAuley, Bridging language and items for retrieval and recommendation. CoRR, 2024.
[35] Zhaopeng Qiu, Xian Wu, Jingyue Gao, and Wei Fan. U-BERT: for improved recommendation. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty- Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021. AAAI Press, 2021.
pre-training user representations
[36] Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng. and Christopher Potts. Learning word vectors for sentiment analysis. In The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference, 19-24 June, 2011, Portland, Oregon, USA. The Association for Computer Linguisties, 2011.
[37] Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. Star: Bootstrapping reasoning with reasoning. In NeurlPS, 2022.
[38] Erie Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, and Noah D. Goodman. Quiet-star: Language models can teach themselves to think before speaking. CORR, 2024.
[39] Yehuda Koren, Robert M. Bell, and Chris Volinsky. Matrix factorization techniques for recom- mender systems. Computer, 2009.
21

---

[40] Lei Zheng, Vahid Noroozi, and Philipp S. Yu. Joint deep modeling of users and items using reviews for recommendation. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, WSDM 2017, Cambridge, United Kingdom, February 6- 6-10, 2017. ACM, 2017.
[41] Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. Tallrec: An effective and efficient tuning framework align large language model with recommendation. In RecSys, 2023.
[42] Yang Zhang, Fuli Feng, Jizhi Zhang, Keqin Bao, Qifan Wang, and Xiangnan He. Collm: Integrating collaborative embeddings into large language models for recommendation. CoRR, abs/2310. 19488, 2023.
[43] Jiayi Liao, Sihang Li, Zhengyi Yang, Jiancan Wu, Yancheng Yuan, and Xiang Wang. Llara: Aligning large language models with sequential recommenders. CoRR, abs/2312.02444 2023.
[44] Xiaoyu Kong, Jiancan Wu, An Zhang, Leheng Sheng, Hui Lin, Xiang Wang, and Xiangnan He. Customizing language models with instance-wise lora for sequential recommendation. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10- 15, 2024, 2024.
[45] Junguang Jiang, Yanwen Huang, Bin Liu, Xiaoyu Kong, Ziru Xu, Han Zhu, Jian Xu, and Bo Zheng. Large language models are universal recommendation learners. CoRR, abs/2502.03041, 2025.
[46] Yu Wang, Zhiwei Liu, Jianguo Zhang, Weiran Yao, Shelby Heinecke, and Philip S. Yu. DRDT: dynamic reflection with divergent thinking for Ilm-based sequential recommendation. CoRR, 2023.
[47] Leslie Pack Kaelbling, Michael I.. Littman, and Andrew W. Moore. Reinforcement learning: A survey. Artif. Intell. Res., 1996.
[48] Richard S. Sutton and Andrew G. Barto. Reinforcement learning an introduction, 2nd Edition. MIT Press, 2018
[49] Timo Kaufmann, Paul Weng, Viktor Bengs, and Eyke Hüllermeier. A survey of reinforcement learning from human feedback. CoRR, 2023.
[50] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. CoRR, 2017.
[51] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10- 16, 2023, 2023.
[52] Yuxin Chen, Junfei Tan, An Zhang, Zhengyi Yang, Leheng Sheng, Enzhi Zhang, Xiang Wang, and Tat-Seng Chua. On softmax direct preference optimization for recommendation. In NeurlPS, 2024.
[53] Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. R1-searcher: Incentivizing the search capability in llms via reinforcement learning. CoRR, 2025.
[54] Xiusi Chen, Gaotang Li, Ziqi Wang, Bowen Jin, Cheng Qian, Yu Wang, Hongru Wang, Yu Zhang, Denghui Zhang, Tong Zhang, Hanghang Tong, and Heng Ji. Rm-r1: Reward modeling as reasoning, 2025.
[55] Bowen Jin, Hansi Zeng, Zhenrui Yue, Dong Wang, Hamed Zamani, and Jiawei Han. Search-rl: Training lmss to reason and leverage search engines with reinforcement learning. CoRR, 2025.
22

---

You are a helpful assistant. Your task is analyze user's purchase history, summarize their preferences, analyze new target item, and then generate a recommendation rationale and predict a rating for that target item, Please follow these steps precisely:
** User Interest Extraction ** "each" in the provided 'user purchase history's analyze the user's 'rating' "review". Identify and state points the liked [[iike]') and disliked ([dissiike]). Then based on *all* points summarized, consolidate the user's overall interest. Categorize them into positive ( [[pos]]) and negative ([[neg]) ) aspects.
Adhere strictly to the following format:
<analyze slanallyl user
2. Item Attributes Extraction ** Extract the features of the target item provided in (target jtem]". Predict potential
points the user might like (1 [[like]') or dislike [[disiiee]' about this specific item, considering general product attributes. Adhere strictly to the following format:
""c analyze item?... /analyze item>"
Match Analysis;** First, internally reason (" ('<matchh>") about *why* target would for would not) a good
recommendation for this This reasoning should explicitly connect the user's overall interest (from Step with the target
item's potential likes'islikes (from Step 2). Provide detailed justifications. Adhere strictly the following format:
"match".... </match>
4. Rating Prediction ,** Then, provide the final predicted rating for the target item within the "<rate>' tags. The answer
should *only* contain the predicted numericul rating (e.g., on a 1-5 scule). Adhere strictly to the following format: <rate?... </rate>""
Your inputs will be 'user purchase history" and target item Ensure your output follows the specified formats and uses the Kanalyze users', 'sanalyze item=", "<match>" and "<tate>>" tags correctly.
<user> user purchase history: (history)
target item: (target item
</user>
Figure 5: System Prompt
A System Prompt
As illustrated in Fig 5, we guide the early outputs of the LLM through structured process in the system prompt to achieve faster training convergence and superior model performance. Specifically, within the <analyze user> and </ </analyze user> tags, we directed the LLM to first list the features [like] and disliked features [dislike] of each product based on the user's historical interaction records, and then summarize the user's complete preferences using [pos] and [neg] tags. Subsequently, for the target item, we encourage the LLM to use [like] and [dislike] tags within the <analyze item> and snaaalyee item> tags to summarize the features that the user might like and dislike about the target item. Following this, the LLM engaged in thoughtful analysis of the match between the user and the target item within the tags mmatch></matchh and finally provided the predicted user rating within the tags <rate></rate>>
B Limitation
While RecZero and RecOne exhibit promising results, this study is not without its limitations. First, due computational constraints, we were unable fully assess the potential performance gains that could be achieved by leveraging larger base models as the foundation for RL. This limitation may hinder a thorough evaluation of the models' true performance potential. Second, the study did not explore whether RecZero and RecOne could serve as viable replacements for existing Teacher models in generating cold-start data for multi-round self-iterative optimization. These limitations highlight the need for future work to investigate the scalability and efficacy of RecZero and RecOne in more resource-inns innuuce-intensive and complex iterative settings, thereby providing a more comprehensive assessment of their practical applicability.
C Experimental Design and Evaluation
Datasets
To comprehensively validate the effectiveness of our proposed RecZero and RecOne, we conduct systematic experiments on four representative real-world recommendation datasets. Through com- parative analysis with various baseline models, including traditional recommendation methods and state-of-the-art (SOTA) LLM-based approaches, we thoroughly demonstrate the superiority of our proposed methods. These datasets span across different domains and scenarios, specifically including: Book, Music and Yelp.
23

---

Book:
This dataset is derived from the "Book" subset of the widely used Amazon dataset for recommen- dation scenario evaluation. records large number of ratings, reviews, and rich book product metadata from users the Amazon platform in the book scenario.
Music:
Similarly, this dataset is from the "Music" subset of the widely used Amazon dataset for recom- mendation scenario evaluation, containing user ratings and reviews in the music domain.
Yelp:
The Yelp Open dataset contains a large number of ratings and reviews from consumers for local restaurants and stores, and is widely used for performance evaluation in recommendation scenarios.
Baselines.
We compare RecZero and RecOne with broad set of baselines that cover traditional collaborative filtering (CF) methods, review-basee methods. and LLM-basee approaches: MF, which is a classic collaborative filtering method; DeepCoNN, NARRE, and DAML, which rely on review text to learn richer user and item representations. DeepCoNN uses CNNs for joint modeling, NARRE applies attention to select informative reviews, and DAML. models interactions between users and items. We also include Rec-SAVER, EXP3RT, and Reason4Rec, which are based on large language models. Rec-SAVER takes users' historical interactions and target item metadata as input to a teacher model to generate intermediate reasoning traces, which are distilled into smaller student model to strengthen rating prediction. EXP3RT and Reason4Rec decompose a single step prediction into multiple reasoning steps, improving accuracy reasoning enhanced rating prediction.
Training Protocol.
model as the starting point for RL due to its strong inslluction-oolloiingg capabilities and planning abilities acquired during pre-training. For traditional baseline experiments, we utilize a single H20 GPU, while the L.LM-based baselines and RecZero and RecOne frameworks executed 8-card H20 GPU setup. All experiments are conducted using Python 3.9.
In our study, we employ the Qwen2.5-7B-Instte
Evaluation Protocol.
For the rating prediction task, we employ Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) as evaluation metrics. MAE measures the prediction accuracy of the model by calculating the average of the absolute errors between predicted values and true values, where smaller value indicates better prediction performance. RMSE evaluates the model's predictive capability by computing the square root of the average of the squared errors between predicted values and true values. It is more sensitive to larger errors and provides a better reflection of the overall prediction stability of the model.
D Statisties
For both RecZero and RecOne, utilize the Owwen..--7B-Instrucee model the starting point for RL. During training, the batch size is set to 8, and the learning rate 2e-6. Each data sample undergoes 8 rollouts during the training process. We the sampling temperature to 1.0, the training epoch to 1, and the KL divergence to 0. Additionally, for the cold-start process of RecOne, we employ reasoning data provided by DeepSeek-Rl for cold-start initialization. We partition the dataset and select 1000 data samples that are not included in either the training or test sets for the cold-start experiments.
24


Reviews:
[{"review_id": "ZvyLBLcWPL", "reviewer": "Reviewer_uhad", "summary": "This paper introduces RecZero, an innovative reinforcement learning (RL)-based approach for recommender systems that leverages large language models (LLMs) to autonomously develop reasoning capabilities for rating prediction tasks. It abandons traditional multi-stage distillation methods, adopting instead a structured, reasoning-guided prompt and rule-based reward modeling via group relative policy optimization (GRPO). A hybrid variant, RecOne, combining supervised fine-tuning (SFT) and RL is also proposed. Extensive experiments validate the performance improvements of both RecZero and RecOne across several datasets.", "questions": "See weakness", "limitations": "See weakness", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength：\n1. Technically sound and well-designed experiments across four diverse and challenging benchmark datasets (Amazon-book, Amazon-music, Yelp, IMDb). Rigorous comparisons with traditional and recent LLM-based recommendation models.\n2. Clearly structured and logically coherent. Methods, assumptions, and experimental setups are thoroughly explained.\n3. Demonstrates clear improvements over existing approaches, addressing key limitations like static supervision and superficial reasoning capability transfers. The proposed frameworks significantly advance recommendation model capabilities, especially in cold-start scenarios.\n4. The use of RL (GRPO) integrated with structured reasoning prompts and hybrid initialization strategies is novel and represents a meaningful departure from standard distillation approaches.\n\nWeeknesses：\n1. The motivation is somewhat unclear. In the recommendation search domain, using large models to distill smaller models for recommendation is a reasonable approach, as the smaller model is used for online inference, reducing latency. The motivation raised in the paper are:\n    - The teacher model has limited capabilities. However, I think the teacher model is large and powerful, and the student model has not yet fully learned from the teacher model. So \"limited capabilities\" is not convincing.\n    - Calling teacher models are costly. But I think the primary cost is during training, and inference with the smaller model is fast.\n    - Superficial Transfer of Reasoning Ability. This point is interesting but lacks experimental evidence to demonstrate that there is a shortcut to transferring reasoning ability.\n2. The proposed method does not address the motivation, as it simply shifts the training paradigm from distillation to reinforcement learning (RL), which I believe is not ideal. Firstly, RL does not reduce training time; on the contrary, RL exploration is more complex. Secondly, the paper does not provide experiments proving that RL training is faster than the original distillation approach. Moreover, RL seems like an overcomplicated approach, and in the recommendation domain, there are not many RL-related techniques because they have been less effective over time.\n3. The experiments are insufficient. The baseline models are limited, and classic traditional algorithms such as DeepFM, DIN, and SIM are not included, nor are LLM-based recommendation algorithms like TallRec, ReLLa, CoLLM, and RecLoRA. Additionally, important recommendation metrics such as AUC, LogLoss, and NDCG are missing from the evaluation.\n4. The writing lacks detail. Algorithms like WDL and RGCL in the baseline are not properly cited, and there is no subsection explaining the baseline models, resulting in a lack of necessary details.", "was_revised": true, "final_justification": "I raise my score from 3 to 4. The major remaining concern is the real-world application of the proposed RL methods in industrial scenarios given the huge cost and sample inefficiency of RL exploration.", "rebuttal": "We appreciate your comments, which greatly improve our paper. Below we provide the point-to\u0002point responses to address your concerns and clarify the misunderstandings of our proposed\nmethod.\n> **C1: The motivation is somewhat unclear**\n\n> **C1.1: \"The teacher model has limited capabilities\" is not convincing.**\n\nThank you very much for your insightful comments on the motivation of our work. They help us present the paper more clearly. \n\n- **Clarification of Our Presentation:** Although LLMs possess extensive world knowledge and strong cross-domain capabilities, a pronounced domain gap between their pre-training data and the data distribution of recommender systems often causes vanilla LLMs to underperform in real-world recommendation scenarios [1-4]. Accordingly, when we mention that the “teacher model has limited capability,” we are referring to a general-purpose LLM whose abilities are not yet aligned with the recommendation domain. In earlier paradigms, such an unaligned LLM is used as the teacher to analyze user preferences, item features, and matching scores, and these sub-optimal abilities are then distilled into a smaller student model.\n\n[1] TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation\n\n[2] CoLLM: Integrating Collaborative Embeddings into Large Language Models for Recommendation\n\n[3] LLaRA: Aligning Large Language Models with  Sequential Recommenders\n\n[4] Adapting Large Language Models by Integrating  Collaborative Semantics for Recommendation\n- **Additional Experiment with Structured Reasoning on Rating Prediction:** To further substantiate our claim, we evaluated a vanilla LLM with explicit structured reasoning against SOTA reasoning baselines (see table below). Even with added structure, the LLM shows no clear advantage, confirming that distilling its reasoning traces alone is insufficient for recommendation tasks. This underscores the need to learn task-specific reasoning via RL. We also optimized our code, further boosting RecZero and RecOne. \n|| Amazon-Music ||\n|-|-|-|\n||MAE|RMSE|\n|MF|0.6188|0.8142|\n|WDL|0.6477|0.8597|\n|RGCL|0.6760|0.9103|\n|GPT-4o|0.7438|1.1069|\n|Qwen2.5-7b-Instruct|0.7105|0.9730|\n|Rec-SAVER| 0.6463|0.9262|\n|EXP3RT|0.5608|0.8385|\n|Reason4Rec|0.5442|0.7722|\n|RecZero|**0.4271**|**0.7058**|\n|RecOne|**0.3816**|**0.6776**|\n- **Additional Experiment with a More Refined Method:** We added rating-prediction experiments in which a vanilla LLM with explicit structured reasoning is compared to current reasoning-enhanced baselines. The LLM still shows no clear advantage, confirming that distilling its reasoning traces is inadequate for recommendation tasks. This motivates our RL approach to discover truly useful reasoning. We have also optimized the code, further boosting RecZero and RecOne.\n1. **RecZero-WithUserTeacher:** replace raw history with the teacher-summarized user preference.\n2. **RecZero-WithItemTeacher:** replace item meta with the teacher-summarized item feature.\n3. **RecZero-WithTeachers:** supply both teacher summaries.\n||Amazon-Music ||\n|-|-|-|\n||MAE|RMSE|\n|RecZero-WithUserTeacher|0.4663|0.7112|\n|RecZero-WithItemTeacher|0.4519|0.7117|\n|RecZero-WithTeachers|0.4874|0.7094|\n|RecZero| **0.4271**|**0.7058**|\n|RecOne|**0.3816**|**0.6776**|\n> **C1.2: The traditional paradigm that relies on a teacher model is not costly, and inference with the smaller model is fast.**\n\nYour comments prompted us to articulate our motivation more clearly. As discussed in our response to Q1.1, vanilla LLMs are poor teachers for recommendation due to a domain gap. An alternative is to curate a large corpus of human-annotated, recommendation-specific reasoning, yet such manual annotation is prohibitively expensive. RecZero instead learns task-specific reasoning via RL, avoiding this overhead.\n\nWe also examined inference efficiency. with the same 7B model size, RecZero attains both the best accuracy and the lowest inference cost versus Qwen-2.5, EXP3RT, and Reason4Rec (see Table C2).\n\n> **C1.3 The claim of a superficial, shortcut-style transfer of reasoning ability is intriguing but presently unsupported by empirical evidence.**\n\nThank you for your insightful comments on the motivation of our work. Recent studies have investigated how supervised fine-tuning (SFT) influences the generalization of foundation models [1–2].\n\n- SFT Memorizes, RL Generalizes [1] shows that SFT tends to memorize training data and thus generalizes poorly to out-of-distribution scenarios, while RL strengthens a model’s core abilities and enhances domain generalization.\n\n- SFT or RL? [2] systematically compares SFT and RL and reveals that SFT often leads to the imitation of “pseudo reasoning paths’’ generated by an expert model. These paths may look similar to the native reasoning produced by RL-trained models, yet they usually contain redundant, hesitant, or low-information steps and sometimes even incorrect reasoning. Moreover, our earlier response (C1.1) has demonstrated that the expert model used in recommendation tasks is itself sub-optimal.\n\n[1] SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training\n\n[2] SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models\n\nOwing to space limitations, we discuss the case study only in our response to Reviewer dSSq’s Comment C3. We kindly refer you to that section.\n\n> **C2: Simply replacing distillation with a more complex RL paradigm fails to address the motivation, offers no evidence of faster training, and is questionable given RL’s historically limited success in recommendation tasks.**\n\nBased on your suggestions, we have made clarifications and additional experiments:\n- **Clarification of Our Training Cost:** Concerning your comment that we simply replace distillation with RL and that RL exploration is more complex and not shown to shorten training time, we note that, although RL may consume more computation concurrently, it can achieve a noticeable performance gain after only a small amount of data compared with SFT. This means that, given the same computational budget, RL needs far fewer training iterations to reach performance on par with SFT. \n- **Additional Experiment on Training and Inference Cost:** we report the training and inference costs of several LLM-based reasoning-enhanced rating-prediction methods in Table below, showing that RecZero (early-stop) matches the performance of previous paradigms using only 0.48 K training samples and 0.4 hours of computation, while requiring significantly fewer inference tokens than multi-stage reasoning baselines.\n\n|Method|Samples to Converge|Trained Models|Total Training Time|Avg. Inference Tokens|Inference Stages|Total Inference Tokens|\n|---|---|---|---|---|---|---|\n|qwen|\\-|\\-|\\-|399.92|1|399.92|\n|EXP3RT|20K|3|1.2h|187.63|3|562.89|\n|Reason4Rec|20K|3|1h|175.49|2|350.98|\n|RecZero(early-stop)|**0.48k**|**1**|**0.4h**|**331.64**|**1**|**331.64**|\n|RecZero(full)|**1.2K**|**1**|**1.1h**|**310.52**|**1**|**310.52**|\n\n- **Additional Analysis on Method Complexity and Effectiveness:** We appreciate your expert insight into recommender systems. As you noted, RL delivered only modest gains to recommendation tasks prior to the LLM era. However, the recent emergence of LLMs with rich world knowledge, cross-domain skills, and intrinsic reasoning ability has made RL-based exploration substantially more effective, as evidenced by recent studies[1–2]. To address your concern about potential complexity, we juxtapose earlier reasoning-enhanced rating-prediction frameworks with our RecZero in Table below, which shows that RecZero already simplifies the pipeline considerably.\n\n|Method|Trained Models|Training Stages|Teacher-Free|End-to-End| \n|-|-|-|-|-|\n|Rec-SAVER|1|1|❌|✅| \n|EXP3RT |3|3|❌|❌| \n|Reason4Rec|3|3|❌|❌| \n|RecZero|1|1|✅|✅| \n\n[1] Rec-R1: Bridging Generative Large Language Models and User-Centric Recommendation Systems via Reinforcement Learning\n\n[2] Reinforced Latent Reasoning for LLM-based Recommendation\n\n\n> **C3: The experimental evaluation is inadequate.**\n\nYou have raised a highly general question. At present, our RecZero and RecOne models are designed for the rating-prediction task in recommender systems, where the standard evaluation metrics are MAE and RMSE. Following your suggestion, we have adapted our method to other tasks.\n- **Additional Experiment on CTR Prediction:** For the CTR-prediction task, we adopted the same Amazon-Book dataset configuration as CoLLM, treating ratings ≥ 3 as positive (clicked) and the rest as negative. The results are reported below.\n||AUC|UAUC |\n|-|-|-|\n| MF | 0.7134 | 0.5565 | \n| LightGCN | 0.7103 | 0.5639 | \n| SASRec | 0.6887 | 0.5714 | \n| DIN | 0.8163 | 0.6145 |\n| CTRL(DIN) | 0.8202 | 0.5996 |\n| ICL | 0.4820 | 0.4856 |\n| Soft-Prompt | 0.7224 | 0.5881 |\n| TALLRec | 0.7375 | 0.5983 |\n| CoLLM-DIN | 0.8109 | 0.6225 |\n| RecZero | **0.8330** | **0.6301** |\n\n- **Additional Experiment on Sequential Recommendation:** Following your suggestion, we have extended our framework to a sequential-recommendation setting and report both HR and NDCG on the Amazon-Music dataset, as shown below.\n\n| |HR@5|HR@10|NDCG@5|NDCG@10| \n|-|---|---|---|---|\n|Caser|0.0103|0.0110|0.0137|0.0144|\n|GRU4Rec|0.0165 |0.0189|0.0133|0.0140| \n|SASRec|0.0174|0.0200|0.0144|0.0163| \n|BigRec|0.0050| 0.0107 |0.0031|0.0049| \n|RecZero|**0.0235**|**0.0390**|**0.0148**|**0.0198**|\n> **C4: The manuscript lacks sufficient detail.**\n\nThank you for highlighting the missing details regarding our baseline settings. We will (i) add the appropriate citations for Wide & Deep-Learning and Relational Graph Contrastive Learning, and (ii) introduce a new subsection entitled “Baseline Models,” in which we summarize the architecture, hyper-parameters, and training procedure of every baseline used in our study."}, {"review_id": "oBk20NUqXX", "reviewer": "Reviewer_dSSq", "summary": "This paper fine-tunes LLMs to solve the rating prediction problem. LLMs would use user-item interaction history in text as the input to match ground-truth ratings from the data. Previous work usually uses a teacher model to generate reasoning traces from interaction history in multiple steps and then distill those traces by fine-tuning a smaller student model. This paper first proposes RecZero by following the methodology of DeepSeek-R1-Zero to optimize a MAE-like reward (plus a format reward) with GRPO instead of doing supervised fine-tuning. As a result, RecZero can automatically develop reasoning ability from a relatively simple prompt template. It also proposes RecOne which is analogous to DeepSeek-R1 in terms of using supervised fine-tuning and cold-start reasoning samples. The results of this paper show that both RecZero and RecOne outperform traditional matrix factorization or deep learning models on on multiple rating prediction domains. More importantly, both proposed methods outperform previous multi-stage distillation methods without explicitly transferring it from a teacher LLM. There are ablation studies on different components of RecZero/RecOne.", "questions": "* Eq. (1) which defines predicted ratings which is not an ML objective. Eq. (4) which rewards small errors is more like an objective.\n\n* Both Reason4Rec and EXP3RT use logit-weighted decoding resulting in a different definition of predicted ratings. Would be interesting to evaluate the choice.\n\n* Could the authors explain the discrepancy between Table 2 and Figure 3 as MAE is higher in Figure 3?\n\n* Could the authors also either plot other baseline(s) in Figure 4 or present raw numbers?\n\n* This paper uses LLMRec as a baseline without citing it.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n+ It seems novel to solve the rating prediction problem by fine-tuning an LLM with RL (GRPO) using an MAE-like reward instead of pure maximum likelihood. Before I am not aware of any work optimizing MSE/MAE rewards using policy gradient on a regression problem.\n+ RecOne seems not a direct application of DeepSeek-R1 technology on the rating prediction problem by having their own reasoning samples.\n+ Tables 1 and 2 show promising results of RecZero / RecOne though I have some concerns on the results.\n\nWeaknesses\n- In Tables 1 and 2, I assume that the authors reuse results from EXP3RT and Reason4Rec papers without references as I am seeing identical numbers. In particular, results of baselines on Amazon-book and IMDB datasets are from the EXP3RT paper if available even if the Reason4Rec paper also presents results on Amazon-book. Results of baselines on Amazon-music and Yelp are from the Reason4Rec paper.\n- More importantly, EXP3RT and Reason4Rec are using different data processing/filtering procedures so they present different MAE/RMSE of EXP3RT on Amazon-book. This paper does not specify their data processing/filtering procedure for generating results of RecZero/RecOne and other baselines.\n- Both Reason4Rec and EXP3RT utilize QLoRA but not sure if RecZero/RecOne fine-tune everything to get more performance gains. Also, Reason4Rec and EXP3RT use GPT-3.5 as the teacher and Llama3-8B-Instruct as the student while RecOne uses Deepseek-R1 as the teacher and fine-tunes Qwen2.5-7B-Instruct-1M. Performance gains can be attributed to the use of different models.\n- This paper does not evaluate the quality of reasoning traces either quantitatively or qualitatively. Average length per response of Deepseek-R1-Zero increases with more training. This paper does not evaluate inference time.", "was_revised": true, "final_justification": "The authors did not describe their dataset preprocessing and not sure if they cherrypicked results or compared apples with oranges. During the rebuttal, the authors reran some experiments with aligned setup and promised to clarify this part in the next version. The authors also provided additional results on 1. inference time comparison, 2.  QLoRA vs full fine-tuning, 3. RL vs SFT. With them my concerns were resolved.", "rebuttal": "We sincerely thank you for your time and valuable comments. To address your concerns, we have detailed our responses point-to-point below.\n> **C1 & C2:  Dataset fully aligned and additional baselines added.**\n\nThank you for your meticulous review. We indeed ran experiments under the same settings as Reason4Rec and EXP3RT, but for the overlapping Amazon-Book dataset we accidentally copied the numbers reported by Reason4Rec while actually using the EXP3RT split. We apologize for the mistake.\n\nFollowing your comment, we have rerun the experiments on both Amazon-Book (Reason4Rec split) and Amazon-Book (EXP3RT split). The updated results are shown below.\n\n|  | Book(Reason4Rec) |  | Book(EXP3RT) |  |\n|-------|-------------|---|--------------|---|\n| | MAE | RMSE | MAE | RMSE |\n| EXP3RT(Llama3-8b) | 0.6135 | 0.9370 | 0.4297 | 0.6572 |\n| EXP3RT(Qwen2.5-7b-It) | 0.6042 | 0.9346 | 0.4115 | 0.6243 |\n| Reason4Rec(Llama3-8b) | 0.6029 | 0.8345 | 0.3875 | 0.6357 |\n| Reason4Rec(Qwen2.5-7b-It) | 0.5937 | 0.8325 | 0.3772 | 0.6295 |\n| RecZero | **0.5253** | **0.8387** | **0.3089** | **0.6081** |\n| RecOne | **0.5017** | **0.7784** | **0.2918** | **0.5970** |\n\nOur RecZero and RecOne outperform prior work on both data splits, confirming the method’s effectiveness.\n\nWe will clearly document dataset reuse and update the main results in the camera-ready version. Thank you again for your careful review.\n\n> **C3: The quality of reasoning traces was not evaluated, and there is no analysis of inference time.**\n\nBelow we show the results of additional experiments. \n\n- **Case Study of reasoning traces:** Inspired by your question, we have added a case study that contrasts the reasoning trajectories produced by the SFT paradigm with those generated by our RecOne, as presented below. The SFT paradigm misinterprets user preferences and contains logical errors in item matching, whereas our approach follows sound analytical reasoning and accurately predicts the user’s rating.\n\n1. **SFT(reasoning):** The user shows a clear preference for female-fronted alternative rock bands from the late 1990s and early 2000s, such as The Sundays and The Jesus and Mary Chain. All seven albums are classified strictly as dream-pop, so the user probably dislikes folk or classic rock. In addition, each chosen record topped the Billboard Hot 100, indicating the user only listens to mainstream chart-toppers. “Blood on the Tracks” is a 1995 electronic concept album in which Bob Dylan collaborates with several grunge artists. The record is best known for its heavy use of synthesizers and industrial beats, very different from Dylan’s earlier acoustic work. It won the Grammy Award for Best Metal Performance and remained in the Top 10 of the UK Singles Chart for 52 consecutive weeks. Because the user prefers late-1990s dream-pop chart hits, and “Blood on the Tracks” is a mid-1990s electronic best-seller that dominated the charts, the album perfectly matches the user’s taste. Its industrial sound and strong female vocal features align well with the user’s love of female-led alternative rock. -> 3\n\n2. **RecOne:** `<analyze user>`The user has a clear preference for classic and hard rock, alternative rock, R&B, soul, and pop music spanning from the 70s to the 90s. The user also shows a strong inclination towards hit albums, especially those from legendary artists, and favor albums with a mix of different genres, including rock, soul, funk, and pop.`</analyze user>` `<analyze item>`Heart of Stone - Signature hit \"If I Could Turn Back Time\" is a mega hit reaching #3 on the singles charts. - Includes other top 40 hits like \"Just Like Jesse James\" and \"After All\" (with Peter Cetera). - Consistent effort with a mix of rock, tender ballads, and country parody. - 80s Cher, known for her iconic style and hits.`</analyze item>`\n`<match>`The user's preference for hit albums, legendary artists, and a mix of rock and pop songs aligns well with \"Heart of Stone,\" which features some of Cher's biggest hits and a well-rounded collection of songs spanning various genres. The album's consistency and iconic status make it a great fit for the user.`</match>` `<rate>`5`</rate>`\n- **Additional Experiment on Inference Time:** As requested, Table below reports inference cost for all methods, each using the same backbone (Qwen-2.5-7B-Instruct) and evaluated on Amazon-Music with 8 × H20 GPUs. EXP3RT and Reason4Rec rely on their official code; “Qwen” is the plain backbone; “RecZero (early-stop)” is saved once it matches Reason4Rec, and “RecZero (full)” is the complete run.\nRecZero’s reasoning trace is shorter and more informative, giving it the lowest overall cost. Although EXP3RT and Reason4Rec are cheaper per stage, their two extra stages (user-preference and item-feature extraction) make their total cost higher than RecZero’s.\n|Method|Avg.InferenceTokens|InferenceStages|Total Inference Token|\n|---|---|---|---|\n|qwen|399.92|1|399.92|\n|EXP3RT|187.63|3|562.89|\n|Reason4Rec|175.49|2|350.98|\n|RecZero(early-stop)|**331.64**|**1**|**331.64**|\n|RecZero(full)|**310.52**|**1**|**310.52**|\n\n> **C4: The reported performance gains may mainly arise from differences in the underlying teacher–student model pairs rather than from the training technique itself.**\n\nYou have raised a highly professional question. As you suggested, we implemented QLoRA-based versions of RecZero and RecOne and compared them against Qwen-based implementations of EXP3RT and Reason4Rec. The results are reported below.\n|  | Book(Reason4Rec) |  | Book(EXP3RT) |  |\n|--|-----|---|----|---|\n||MAE |RMSE | MAE | RMSE |\n|EXP3RT(Qwen2.5-7b-It-It) | 0.6042 | 0.9346 | 0.4115 | 0.6243 |\n|Reason4Rec(Qwen2.5-7b-It) | 0.5937 | 0.8325 | 0.3772 | 0.6295 |\n|RecZero | **0.5253** | **0.8387** | **0.3089** | **0.6081** |\n|RecZero(QLoRA) | **0.5293** | **0.8402** | **0.3124**|**0.6159** |\n|RecOne | **0.5017** | **0.7784** | **0.2918**|**0.5970** |\n| RecOne(QLoRA) | **0.5024** | **0.7801** | **0.2933**|**0.5997**|\n\nThese numbers show that the QLoRA versions of our framework still outperform the previous paradigms overall, demonstrating the effectiveness of our approach.\n\n> **Q1: Could the authors explain the discrepancy between Table 2 and Figure 3 as MAE is higher in Figure 3?**\n\nThank you for raising this concern. The MAE curve shown in Figure 3 does not cover the entire training process, whereas the results in Table 2 correspond to the model’s final performance. During the early stages of RL training, the MAE drops sharply, then decreases slowly with minor fluctuations, and finally stabilizes over a longer training period. In the camera-ready version, we will make this clearer and include a complete MAE curve covering the entire training pipeline.\n\n> **Q2: This paper uses LLMRec as a baseline without citing it.**\n\nWe apologize for overlooking  this point in our current draft. We will promptly add a citation to LLMRec in the final version of the paper. Following your suggestion, we will also re-examine our reference list to enhance the professional quality of our work.\n\n> **Q3: Eq. (1) which defines predicted ratings which is not an ML objective. Eq. (4) which rewards small errors is more like an objective.**\n\nAs you correctly pointed out, the argmax expression is merely a prediction rule rather than the true training objective. We will incorporate this correction in the camera-ready version of our paper.\n\n> **Q4: Could the authors also either plot other baseline(s) in Figure 4 or present raw numbers?**\n\nThank you for the insightful questions—they prompted us to refine and clarify our ablation studies. Owing to the NeurIPS rebuttal format restrictions, we present the expanded results in a table instead, as shown below.\n- Experimental setup.\nThe default RecZero takes (a) the raw user-interaction history and (b) the meta information of the target item. We constructed six variants:\n1. RecZero-WithUserTeacher [teacher-generated user summary, target-item meta]\n2. RecZero-WithItemTeacher [user-interaction history, teacher-generated item features]\n3. RecZero-WithTeachers  [teacher-generated user summary, teacher-generated item features]\n4. RecZero-WithoutUserSum removes the user-preference summarisation module\n5. RecZero-WithoutItemExt removes the item-feature extraction module\n6. RecZero-WithoutMatch  removes the user–item matching module\n\nWe further report RecZero-WithoutThink, which omits the entire reasoning stage, and RecZero-NaiveThink, which replaces the structured trace with a single-step free-form thought. We also optimized our code, further boosting RecZero and RecOne.\n| |Music | |\n|-|-|--|\n| |MAE | RMSE |\n|RecZero-WithUserTeacher|0.4663|0.7112|\n|RecZero-WithItemTeacher |0.4519|0.7117|\n|RecZero-WithTeachers |0.4874|0.7094|\n|RecZero-WithoutUserSum | 0.4412|0.7250|\n|RecZero-WithoutItemExt | 0.4364|0.7219|\n|RecZero-WithoutMatch | 0.4387|0.7378 |\n|RecZero-WithoutThink | 0.4737|0.7781 |\n|RecZero-NaiveThink |  0.4553 |0.7499|\n|RecZero | **0.4271**|**0.7058**|\n| RecOne | **0.3816**|**0.6776**|\n\nWe will add all these details and the corresponding figures to the ablation section of the camera-ready version.\n> **Q5: Both Reason4Rec and EXP3RT use logit-weighted decoding resulting in a different definition of predicted ratings. Would be interesting to evaluate the choice.**\n\nYou are indeed an expert on using LLM reasoning for recommendation. Unlike Reason4Rec and EXP3RT, we do not design a separate task that requires the LLM to output an integer score that is then converted into a decimal rating via logit-weighted decoding. Compared with the SFT paradigm that imitates integer target labels, reinforcement learning offers a clear advantage: by crafting an appropriate reward function, we encourage the LLM to predict decimal ratings directly in its response, thereby earning a higher MAE reward and removing the need for the logit-weighted step altogether. We appreciate you highlighting this point and will explain it more explicitly in the Method section of the final version."}, {"review_id": "MC8HkR2kv7", "reviewer": "Reviewer_FsYN", "summary": "This paper aims to address the task of rating prediction in recommendation using large language models (LLMs). It first proposes a system prompt designed to generate reasoning trajectories that analyze the input user, analyze the item, evaluate user-item compatibility based on these analyses, and finally produce a reasoning process that leads to the predicted rating. Using data generated in this way, the authors train LLMs with GPRO. In addition, they propose RecOne, a model that first performs supervised fine-tuning (SFT) on a subset of high-quality generated data before further training the LLM using GPRO. The paper demonstrates the superiority of the proposed approach over both conventional recommendation systems and other LLM-based recommendation systems across four datasets.", "questions": "- How robust and effective is the proposed prompt template in comparison to other prompt designs?\n\n- What LLMs were used in the baseline methods in the experiments?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The paper clearly identifies challenges in applying LLMs to rating prediction and presents an interesting study that explicitly generates a reasoning process for recommendations.\n\nHowever, the following weaknesses are as follows:\n- The paper should provide clearer evidence on whether the proposed reasoning process is more robust than other chain-of-thought (CoT) processes, and whether the designed prompt is indeed superior. The ablation study currently only contrasts the proposed method with a simple <think> generation. It would strengthen the work to experimentally compare different reasoning step structures, such as variations that omit item analysis or modify the reasoning process itself.\n\n- Beyond generating data and performing GPRO and SFT on selected data, the methodological novelty appears limited. The approach does not introduce fundamentally new techniques beyond this pipeline.\n\n- Additional experiments on robustness are needed. For example, whether the proposed method generalizes well across different LLMs beyond the ones tested.\n\n- The paper should clearly state whether the same LLMs were used for the baselines. In particular, for models like LLMRec or Reason4Rec, it is unclear whether they were evaluated using Qwen2.5-7B, as was the proposed method. While this is briefly mentioned in the appendix, such critical details should be explicitly stated in the main text. Furthermore, since the proposed method likely incurs higher training costs compared to existing approaches, the paper should also clarify whether the same data sizes were used for training across all models, and provide a detailed analysis of training cost and inference time.\n\n- Since the proposed method fine-tunes Qwen2.5-7B, it is important to also compare against results where Qwen2.5-7B is used for reasoning with the proposed template, but without reinforcement learning, in order to better isolate the contribution of the reinforcement learning step.", "was_revised": true, "final_justification": "My main concerns have been sufficiently addressed through the authors' clarifications and discussions.", "rebuttal": "We gratefully thank you for your valuable comments! Here we meticulously give point-by-point responses to your comments, and further revise our manuscript following your suggestions. Hope that our responses can address all your concerns.\n> **C1 & C5 & Q1: Should more convincingly demonstrate the superiority of its reasoning prompt by comparing it against a variety of alternative COT structures; add qwen baseline.**\n\nThank you for the insightful questions—they prompted us to refine and clarify our ablation studies.\n\n- Experimental setup. The default RecZero takes (a) the raw user-interaction history and (b) the meta information of the target item. We constructed six variants:\n1. RecZero-WithUserTeacher [teacher-generated user summary, target-item meta]\n2. RecZero-WithItemTeacher [user-interaction history, teacher-generated item features]\n3. RecZero-WithTeachers  [teacher-generated user summary, teacher-generated item features]\n4. RecZero-WithoutUserSum removes the user-preference summarisation module\n5. RecZero-WithoutItemExt removes the item-feature extraction module\n6. RecZero-WithoutMatch  removes the user–item matching module\nWe further report RecZero-WithoutThink, which omits the entire reasoning stage, and RecZero-NaiveThink, which replaces the structured trace with a single-step free-form thought. We also optimized our code, further boosting RecZero and RecOne.\n\n|   | Amazon-Music |  |\n|-------|-------------|---|\n|  |MAE | RMSE |\n| Qwen | 0.6821 | 0.9796 |\n| RecZero-WithUserTeacher | 0.4663 | 0.7112 |\n| RecZero-WithItemTeacher | 0.4519 | 0.7117 |\n| RecZero-WithTeachers | 0.4874 | 0.7094 |\n| RecZero-WithoutUserSum |  0.4412 | 0.7250 |\n| RecZero-WithoutItemExt |  0.4364 | 0.7219 |\n| RecZero-WithoutMatch |  0.4487 | 0.7378 |\n| RecZero-WithoutThink |  0.4737 | 0.7781 |\n| RecZero-NaiveThink |  0.4553 |0.7499|\n| RecZero |  **0.4271** | **0.7058** |\n| RecOne |  **0.3816** | **0.6776** |\n\nWe will add all these details and the corresponding figures to the ablation section of the camera-ready version.\n> **C2: Do not introduce fundamentally new techniques beyond this pipeline.**\n\nThank you for your forward-looking comments. We believe that the proposed RecZero and RecOne frameworks offer a practical advance. Earlier paradigms rely on teacher models and therefore cannot be trained end-to-end. By introducing GRPO into reasoning-enhanced rating prediction, we both simplify the training pipeline and improve accuracy, without adding training or inference overhead. The tables below compare our method with the previous baselines, demonstrating the superiority of our method\n\n|Method|Trained Models|Training Stages|Teacher-Free|End-to-End| \n|-|-|-|-|-|\n|Rec-SAVER|1|1|❌|✅| \n|EXP3RT |3|3|❌|❌| \n|Reason4Rec|3|3|❌|❌| \n|RecZero|1|1|✅|✅| \n\n> **C3: Additional experiments on robustness.**\n\nWe agree that evaluating more backbones is important for demonstrating the robustness of our method. Most recent RL-based recommendation studies use the Qwen family, and we therefore chose Qwen to keep our main experiments comparable. Following your advice, we have now extended RecZero to the Gemma model and have also tested it on Qwen models of different sizes (3B, 7B, and14B). Results on Amazon-Music are shown below.\n\n|   | Amazon-Music |  |\n|-------|-------------|---|\n|  |MAE | RMSE |\n| RecZero(gemma-3-12b-It) |  0.4089 | 0.6958 |\n| RecZero(qwen2.5-3b-It) |  0.4421 | 0.7539 |\n| RecZero(qwen2.5-7b-It) |  0.4271 | 0.7058 |\n| RecZero(qwen2.5-14b-It) |  0.3995 | 0.6927 |\n\nThe method transfers well to a different LLM family (Gemma) and shows consistent improvement as the backbone scale increases, in line with standard scaling laws.\n\n> **C4 & Q2: Clarify Backbone Consistency, Data Size, and Computational Cost Across All Baselines.**\n\n- **Additional Experiment on more backbones:** Thank you for the valuable suggestion! We will include the backbone details in the main body of the camera-ready version. Inspired by your feedback, we have also added the results of EXP3RT and Reason4Rec on the Qwen-2.5-7B-Instruct backbone, which will be reported in the final paper.\n\n|  | Book |  |\n|-------|--------------|---|\n| | MAE | RMSE | MAE | RMSE |\n| EXP3RT(Llama3-8b-It) | 0.4297 | 0.6572 |\n| EXP3RT(Qwen2.5-7b-It-It) | 0.4115 | 0.6243 |\n| Reason4Rec(Llama3-8b-It) | 0.3875 | 0.6357 |\n| Reason4Rec(Qwen2.5-7b-It) | 0.3772 | 0.6295 |\n| RecZero |  **0.3089** | **0.6081** |\n| RecOne |  **0.2918** | **0.5970** |\n\n- **Additional Experiment on Training and Inference Cost:** we report the training and inference costs of several LLM-based reasoning-enhanced rating-prediction methods in Table below. EXP3RT and Reason4Rec rely on their official code; “Qwen” is the plain backbone; “RecZero (early-stop)” is saved once it matches Reason4Rec, and “RecZero (full)” is the complete run. RecZero’s reasoning trace is shorter and more informative, giving it the lowest overall cost. Although EXP3RT and Reason4Rec are cheaper per stage, their two extra stages (user-preference and item-feature extraction) make their total cost higher than RecZero’s.\n\n|Method|Samples to Converge|Trained Models|Total Training Time|Avg.InferenceTokens|InferenceStages|Total Inference Token|\n|---|---|---|---|---|---|---|\n|Qwen|\\-|\\-|\\-|399.92|1|399.92|\n|EXP3RT|20K|3|1.2h|187.63|3|562.89|\n|Reason4Rec|20K|3|1h|175.49|2|350.98|\n|RecZero(early-stop)|**0.48k**|**1**|**0.4h**|**331.64**|**1**|**331.64**|\n|RecZero(full)|**1.2K**|**1**|**1.1h**|**310.52**|**1**|**310.52**|"}, {"review_id": "FjTszMlGSO", "reviewer": "Reviewer_fTc1", "summary": "This manuscript proposes two novel LLM-based rating prediction models, i.e., RecZero with pure RL and RecOne combining SFT and RL. The RecZero pipeline adopts a CoT-based reasoning process, which consists of <analyze user>, <analyze item>, <match>, and <rate> steps. Moreover, the RecOne pipeline incorporates SFT with RecZero pipeline, which first generates reasoning process based on LLM teacher model and then adopts the reasoning process and supervision. The authors succeed in transferring the achievements gained by DeepSeek and GRPO into the rating prediction task, with the excellent performance on four public datasets.", "questions": "**Problem**\n- In my regard, the terminology **Cold-Start** in Section 3.2.1 is different from that mentioned in *cold-start recommendation* and the following *cold-start samples* with less interactions. If yes, renaming the title of Section 3.2.1 and related content is helpful to avoiding ambiguity.\n\n**Discussion**\n- As mentioned in the caption of Figure 1, I wonder the reason and necessity of multiple models.\n> Conventional paradigms train multiple student LLMs using diverse query-based datasets with reasoning traces...\n\n- Whether the ground-truth of rating prediction task is the actual user rating or not, as we did on the online shopping website?", "limitations": "YES", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength**\n- The motivation of RecZero is well-elaborated. The limited recommendation capability and the costly supervision are both the key weaknesses of existing distillation-based methods.\n- The whole manuscript is well-organized, with the problem statement and the proposed methodology are clearly introduced.\n- Inspired by DeepSeek, the combination of SFT and RL is spontaneous, enlightening the recommender system community.\n\n**Weakness**\n- **Model Scale.** The scale of baseline models are not mentioned, which may given rise to unfair performance comparison. It is unclear whether the improvement originates from the proposed method or just the LLM backbone.\n- **Method Scalability.** Similar to the first concern, I wonder whether the proposed method can still be effective with backbones of different scale. Further investigation towards the impact of backbone is recommended.\n- **Inference Efficiency.** This manuscript lacks report of the inference time and further comparison with baselines.\n- **Availability.** The proposed method is closed-source.", "was_revised": true, "final_justification": "The authors resolve my main concerns towards baseline scales and method scalability. Therefore, I decide to raise the score.", "rebuttal": "We gratefully thank you for your valuable comments! Here we meticulously give point-by-point responses to your comments, and further revise our manuscript following your suggestions. We sincerely appreciate the depth of your insights, which have undoubtedly enriched our work. Here we meticulously give point-by-point responses to your comments.\n\n> **C1: Model Scale:** The scale of baseline models are not mentioned.\n\nThank you for the valuable suggestion! We will include the backbone details in the main body of the camera-ready version. Inspired by your feedback, we have also added the results of EXP3RT and Reason4Rec on the Qwen-2.5-7B-Instruct backbone, which will be reported in the final paper. Please note that after further refining our implementation, the model now achieves even better performance than in the initial version.\n\n|   | Amazon-Music |  |\n|-------|-------------|---|\n|  |MAE | RMSE |\n| EXP3RT(Llama3-8b)| 0.5608 | 0.8385 |\n| EXP3RT(qwen2.5-7b-It)| 0.5341 | 0.7992 |\n| Reason4Rec(Llama3-8b) | 0.5442 | 0.7722 |\n| Reason4Rec(qwen2.5-7b-It) |  0.5108 | 0.7534 |\n| RecZero(qwen2.5-7b-It) |  0.4271 | 0.7058 |\n| RecOne(qwen2.5-7b-It) |  0.3816 | 0.6776 |\n\n\n> **C2: Method Scalability:** Whether the proposed method can still be effective with backbones of different scale.\n\nThank you for the constructive suggestion to examine how our approach scales with different-sized backbones. Following your advice, we ran additional experiments on Amazon-Music; the results are reported in Table below.\n\n|   | Amazon-Music |  |\n|-------|-------------|---|\n|  |MAE | RMSE |\n| RecZero(gemma-3-12b-It) |  0.3789 | 0.6958 |\n| RecZero(qwen2.5-3b-It) |  0.4421 | 0.7539 |\n| RecZero(qwen2.5-7b-It) |  0.3805 | 0.7215 |\n| RecZero(qwen2.5-14b-It) |  0.3795 | 0.6927 |\n\nAs the backbone grows from 3 B to 14 B, RecZero exhibits the expected scaling behaviour and confirms the robustness of our method. A larger model offers better instruction following, world knowledge, and reasoning ability, which likely reduces the risk of converging to sub-optimal RL solutions.\n\n> **C3: Inference Efficiency:** This manuscript lacks report of the inference time and further comparison with baselines.\n\nAs requested, Table below reports inference cost for all methods, each using the same backbone (Qwen-2.5-7B-Instruct) and evaluated on Amazon-Music with 8 × H20 GPUs. EXP3RT and Reason4Rec rely on their official code; “Qwen” is the plain backbone; “RecZero (early-stop)” is saved once it matches Reason4Rec, and “RecZero (full)” is the complete run. RecZero’s reasoning trace is shorter and more informative, giving it the lowest overall cost. Although EXP3RT and Reason4Rec are cheaper per stage, their two extra stages (user-preference and item-feature extraction) make their total cost higher than RecZero’s.\n\n|Method|Avg.InferenceTokens|InferenceStages|Total Inference Token|\n|---|---|---|---|\n|qwen|399.92|1|399.92|\n|EXP3RT|187.63|3|562.89|\n|Reason4Rec|175.49|2|350.98|\n|RecZero(early-stop)|**331.64**|**1**|**331.64**|\n|RecZero(full)|**310.52**|**1**|**310.52**|\n\n> **C4: Availability:** The proposed method is closed-source.\n\nWe apologise that the rebuttal policy does not allow external links, so we cannot provide an anonymous GitHub repository at this stage. Nevertheless, we commit to releasing the full codebase publicly once the paper is accepted.\n\n> **Q1: Clarify and Rename §3.2.1 to Avoid Confusion with Conventional “Cold-Start” Usage.** \n\nThank you for pointing out this potential ambiguity. In §3.2.1 we used the term “Cold-Start” to describe the very first stage of our RL training, where the policy has not yet received any reward signals. This notion differs from the conventional “cold-start recommendation,” which concerns users or items with few historical interactions. To avoid confusion, we will rename §3.2.1 to “RL Warm-up Stage” and add a sentence that explicitly distinguishes the two meanings in the camera-ready version.\n\n> **Q2: The reason and necessity of multiple models in previous paradigms.**\n\nThank you for raising this point. In earlier SFT-based paradigms such as EXP3RT and Reason4Rec, the full chain-of-thought was split into three separate subtasks—(i) user-preference summarisation, (ii) item-feature extraction, and (iii) user–item matching—and each subtask was assigned to a different student LLM. Their ablation studies showed that this multi-model design was beneficial under SFT because the three subtasks have heterogeneous objectives and can interfere with one another when trained jointly.\n\nOur RL formulation changes the picture. Because the reward depends only on the final rating error, all reasoning steps share the same optimisation objective and no longer conflict. Hence a single LLM can execute the entire reasoning chain and be trained end-to-end. RecZero and RecOne therefore use only one model, yet still outperform the previous multi-model baselines.\n\n> **Q3: Whether the ground-truth of rating prediction task is the actual user rating or not.**\n\nThank you for the question. Yes, the ground-truth signal in all of our experiments is the explicit rating that the user actually gave in the original dataset (e.g., the “rating” field in Amazon reviews). During training we minimise the MAE between the model’s predicted decimal rating and this real score; the same label is used to compute the MAE/RMSE reported. Using genuine user ratings, rather than synthetic or proxy labels, encourages the model to align with real-world recommendation scenarios."}]

Author rebuttal:

Meta-review: Existing distillation-based methods for using LLMs in recommendation suffer from the teacher model's limited capability, costly supervision, and superficial transfer of reasoning skills. This paper proposes a novel reinforcement learning (RL) paradigm that trains a single LLM autonomously for rating prediction via structured "Think-before-Recommendation" prompts and a rule-based reward model (GRPO) to optimize reasoning trajectories. Experimental results show the effectiveness of the proposed model. The rebuttal has addressed the concerns of the reviewers, and all the reviewers agree to accept this work.
Decision: Accept (poster)