IGD: Token Decisiveness Modeling via Information Gain in LLMs for Personalized Recommendation
Authors: Zijie Lin
Yang Zhang
Xiaoyan Zhao
Fengbin ZHU
Fuli Feng
Tat-Seng Chua
Venue: NeurIPS 2025 poster ()
Keywords: Personalization
LLM-based Recommendation
Primary area: deep_learning

Abstract: Large Language Models (LLMs) have shown strong potential for recommendation by framing item prediction as a token-by-token language generation task. However, existing methods treat all item tokens equally, simply pursuing likelihood maximization during both optimization and decoding. This overlooks crucial token-level differences in decisiveness—many tokens contribute little to item discrimination yet can dominate optimization or decoding.
To quantify token decisiveness, we propose a novel perspective that models item generation as a decision process, measuring token decisiveness by the Information Gain (IG) each token provides in reducing uncertainty about the generated item. Our empirical analysis reveals that most tokens have low IG but often correspond to high logits, disproportionately influencing training loss and decoding, which may impair model performance.
Building on these insights, we introduce an Information Gain-based Decisiveness-aware Token handling (IGD) strategy that integrates token decisiveness into both tuning and decoding. Specifically, IGD downweights low-IG tokens during tuning and rebalances decoding to emphasize tokens with high IG. In this way, IGD moves beyond pure likelihood maximization, effectively prioritizing high-decisiveness tokens. Extensive experiments on four benchmark datasets with two LLM backbones demonstrate that IGD consistently improves recommendation accuracy, achieving significant gains on widely used ranking metrics compared to strong baselines. Our codes are available at \url{https://github.com/ZJLin2oo1/IGD}.

IGD: Token Decisiveness Modeling via Information Gain in LLMs for Personalized Recommendation
Zijie Lin', Yang Zhang"t, Xiaoyan Zhao", Fengbin Zhu Fuli Feng', Tat-Seng Chua
National University of Singapore 2The Chinese University of Hong Kong
University of Science and Technology of China
zijie lin@u. nus, edu, zyang1580@gmail. com com
xzhao@se cuhk edu. hk, zhfengbin@@mail.com ful fulifeng93@gmail com, dcscts@nus. edu. Corresponding author
Abstract
Large Language Models (LLMs) have shown strong potential for recommendation by framing item prediction as token-by-token language generation task. However, existing methods treat all item tokens equally, simply pursuing likelihood maxi- mization during both optimization and decoding. This overlooks crucial token-level differences in decisiveness-mant tokens contribute little to item discrimination yet can dominate optimization or decoding. propose a novel perspective that models item generation as a decision process, measuring token decisiveness by the Information Gain (IG) each token provides reducing uncertainty about the generated item. Our empirical analysis reveals that most tokens have low IG but often correspond to high logits, disproportionately influencing training loss and decoding, which may impair model performance. Building on these insights, we introduce an Information Gain-basee Decisiveness- aware Token handling (IGD) strategy that integrates token decisiveness into both tuning and decoding. Specifically, IGD downweights low-IG tokens during tuning and rebalances decoding to emphasize tokens with high IG. In this way, IGD moves beyond pure likelihood maximization, effectively prioritizing high-decisivencee tokens. Extensive experiments four benchmark datasets with two LLM bbones demonstrate that IGD consistently improves recommendation accuracy, achieving significant gains widely used ranking metrics compared to strong baselines. Our codes are available at https:: : //tpttbt.tttt///////////gt wwt.ggtt..co.o...... com/2JLin2oo//OO0
quantify token decisiveness, we
Introduction
play a crucial role in helping users discover relevant and personal- ized content. With recent advances Large Language Models (LLMs) 4 5 6), there growing interest in leveraging LLMs' strong language understanding and reasoning capabilities [7] g for recommendation tasks [T
Recommendation systems 0 2
[5]. giving rise to new paradigm known as LLM4Rec [16] [7] [8]. In this paradigm, recommendation is typically formulated as natural language problem: user bistory and task context are encoded into prompt, based on which the LLM is tuned to generate the top-K recommended items via autoregressive token decoding [19]. This approach has demonstrated strong performance in capturing user intent and generating personalized
I2
B
14
outputs 200 21 (22].
Despite its promise, we argue that the existing token handling strategy in LLM4Rec does fully align with the item generation process in recommendation. The current approach is primarily likelihood- driven, treating each item token equally and simply focusing on: 1) optimizing token
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

likelihood during fine-tuning for data fitting 24, 21], and 2) maximizing token likelihood during inference for generation [9 20]. However, not all tokens are equally important. Some tokens are more decisive in defining the item, while others serve grammatical or filler functions with low decisiveness. dewdeeeisiveness tokens do not reduce uncertainty item generation, making their focus less meaningful. Moreover, low-decisiveness tokens may have high logits logits-such such as "ghost tokens" defined in [19], which have generation probabilities close to for a given prefix. These tokens can dominate seselioood-mxximize
decoding, introducing bias toward items with more such tokens 19]. To improve both tuning and decoding, it crucial to quantify and incorporate token decisiveness, rather than relying solely on likelihood optimization.
To measure token decisiveness, we introduce a novel perspective that frames token-by-token item generation in LLM4Rec as a decision process. In this framework, the uncertainty of the recommen- dation outcome at each generation step is quantified by the entropy (25] of the item distribution conditioned on the tokens generated so far. The decisiveness of token is defined as the reduction in this uncertainty when selecting the token, ie., its Information Gain (IG) (26]. Based on this definition, we observe that: I) in the studied real-world datasets, over 50% of item tokens exhibit zero IG; and 2) under the existing token strategy, LLM4Rec models may be misled by these zero-IG tokens. Specifi- cally, shown in Figure [] models tend to over-optimize zero-IG tokens while under-emphasizing non-zero-IG tokens during tuning. Furthermore, as shown in Figure 4 low IG tokens, especially zero-IG tokens, often correspond to high logits, which can bias the process toward items containing more such tokens.
maximizing decoding
To incorporate token decisiveness into tuning and decoding, we propose an Information Gain-based Decisiveness-aware Token Handing (IGD) strategy that goes beyond simply optimizing/mxximmzingg token likelihood. During tuning. IGD downweights zero-IG tokens to prioritize informative (non- zero-IG) tokens that aid in item discrimination, enabling more effective learning. At decoding, IGD increases the influence of high IG tokens along the autoregressive path by adjusting focus toward their logits, rather than blindly following the toward items dominated by high-logit but low-decisiveness tokens. In this way, IGD reshapes token importance by incorporating token decisiveness, leading to improved recommendation accuracy. We evaluate IGD on four benchmark datasets using two LLM backbones. Results demonstrate that IGD consistently enhances recommendation performance, with average gains of 18.89% in HR @ and 20.15% in NDCG@ 10 over strong baselines.
principle, thereby reducing bias
The main contributions are: (1) We emphasize the importance of quantifying and incorporating token decisiveness in both tuning and decoding, and propose framing the item generation process decision process, defining token decisiveness based on information gain. (2) We introduce IGD, simple yet effective token handling strategy that leverages token-level information gain to guide both tuning and decoding, going beyond mere likelihood optinization/aaximizztoon to prioritize high-decisiveness tokens, thereby enhancing recommendation performance. (3) We conduct extensive experiments on four benchmark datasets using two LLM backbones. The results show that IGD consistently improves recommendation performance, achieving significant gains in HR @ 10 and NDCG@ 10 over strong baselines.
2
Preliminary
This section introduces the typical tuning and decoding approaches in LLM4Rec.
2.1 Tuning
To enable next-item prediction using LLMs, supervised fine-tuning (SFT) is usually applied to teach LLMS to map items to token sequences. Given an input prompt x transformed from user history and task deseription, and list of target item tokens y (y1,--., D, the model is trained to minimize the token-level cross-entropy loss:
m L Σ
(folz,
where fo denotes the L.LM with parameters 0, and & is the cross-entropy between the predicted token distribution and the ground-truth token yt at position t.

---

Demo Items
Entropy (H)
The Legend of Zelda Skyward Sword
Start
The Amazing Spider Man
What's
item?
Zero-IG Token
Super
The
"The"?
Legend
of
Zelda
Decisive
Amazing
is that Mario? ?
Super
Spider
Man
Skyward
Sword
IG(M;5)
Mario
see.
End
End
End
0
4
5
Step (t)
Figure I: Illustration LLM4Rec autoregressive token generation as sequential decision process. As tokens generated, the entropy of the remaining sequence gradually decreases. 'hee information gain (IG) quantifies this reduction, e.g., IG(M; measures the of token "Mario" given prefix "Super". Tokens shared across many items (e.g., "The") exhibit lower decisiveness with lower IG, while more decisive tokens (c.g.. "Super") ") lead to larger IG. Tokens with IG=0-such as "of", "Zelda", "Sword", and "Man" "Man"-are are referred to zero-IG tokens.
2.2 Decoding
Autoregressive Decoding, inference time, the model generates item sequences autoregressively. The conditional probability of full sequence y given x is factorized as:
plyla) II (.9l,.,e
This formulation enables step-by-step generation but highly relies local token-level probabilities.
Beam Search Decoding. During inference, existing methods commonly use beam search to generate multiple item candidates simultaneously. At each decoding step I, the model expands each partial sequence yst-1 by considering the top-ranked token candidates, and updates the cumulative score using:
S(ys:) ((gg(-1)) log log ves),
where S(yce) denotes the log-probability of the sequence prefix Use. and p(y/ (2, Met)) is the condi- tional probability of the token in step t.
In standard natural language generation tasks, length penalty is often introduced to avoid overly long outputs. However, recent work [19 reveals that applying such penalties in recommendation scenarios tends to favor longer item sequences, introducing bias into the final selection. Therefore, we follow [19] and set the length penalty to zero, keeping the score computation purely based on accumulated log- probbbbbailss as shown above.

---

Token Decisiveness Modeling
3.1 Token Decisiveness Measurement
We model the token generation process in LLM4Rec sequential decision decision-making procedure, where each autoregressive decoding step progressively refines the target item from the full item space. Let I denote the full item collection, and Ivc+ Ç represent the set of candidate items consistent with the generated token prefix Vgr at step t. Following information theory [27], we quantify the uncertainty of the recommendation at step t using Shannon entropy distribution:
25 over the candidate item
H(ys)) =- E
log lo (I,).
where pr (Z, denotes the empirical prior probability of item I; estimated from the training data. The entropy is computed
the candidate set TVs compatible with the current token prefix.
At each step, a new token y/ reduces the candidate space. We measure the decisiveness of ye using its Information Gain (IG) -the reduction in uncertainty it induces:
(G ver)) = (y (ug+)
This formulation quantifies how much yt contributes to identifying the target item, where higher IG indicates greater decisiveness.
3.2
Statistical Analysis on Token Decisiveness
Table I: Dataset Statistics and zero-IG Token Proportion
Dataset
Items
Train
Valid
Test
Tokens
zero-IG Tokens (%)
CDs
14,239
148,685
18,586
18,587
805,786
450,960 (55.9695 (6.%))
Games
11,037
201,613
25,202
25,203
2,128,430
1,292,171 (60.71%)
Toys
11,252
112,755
14,095
14,096
1,530,370
1,098,070 0700 (71.75%)
Books
41,722
682,998
85,376
85,376
7,183,839
5,241,997 (72.97%)
We conduct a statistical analysis of the proposed IG metrics on four public datasets, summarized in Table [ First, each item title is tokenized into sequence of tokens using the Qwen2.5 tokenizer 28). Then, we construct a token prefix tree and compute, for each prefix, its corresponding entropy well as the IG for each token. All statistics computed exclusively on the training set. From we observe that zero-IG tokens-tokemt that yield no reduction in entropy are predominant, constituting 55.96% 72.97% of all tokens across datasets.
Table [
3.3 Token-level Biases
To investigate how LLMs interact with token decisiveness, analyze tuning dynamics and decoding paths using the D3 [99 method with the Qwen2.5-1.35 1.... model 228. model tuning, we respectively monitor the tuning loss for zero-IG tokens and non-zero-IG tokens across the training set; In model decoding, we gather both ground-truth items and top- 10 predicted items from the test set, compute the average entropy of the prefix each decoding step, and compare the average entropies between predicted and ground-truth prefixes.
We observe the following biases:

---

CDs
Games
Toys
Tu0h LOOP
21.03
21.301M0
Zaro-15
Zaro-R 4G
Zaro-G
Zero4G
Figure 4: Relationship between IG values and logits of tokens decoding. For each dataset, the left subfigure shows that zero-IG tokens are associated with extremely high logits (close to 0). The right subfigure illustrates a negative correlation between IG values and logit magnitudes for non-zero-IG tokens.
CDs
Games
CDs
Games
0.06
Non zero-16 I5 Taken
Non-zero zero 1G Token
0.15
Hyuu- He
Haa-H Me
0.04
0.10
105
0.05
0.02
0.00
0.00
500
1,000
500
1,0000
1,500
2
3
Training Steps
Training Steps
Step
Step
Toys
Books
Toys
Books
Zero-IG Tiken
Zero-IG Token
6
Non-2aro-IG Taken
Hyee -1x
0.2
2-
0.05
0.1
0.00
0
200
400
600
800
2,000
4,000
2
3
Training Steps
Training Steps
Step
Step
Figure 2: Loss comparison between zero-IG and non-zero-IG tokens model tuning (epoch 1)
Figure 3: Entropy difference in decoding: model prediction vs. ground-truth
Tuning Bias: During model tuning, models tend to over-optimize zero-IG tokens while under- emphasizing non-zero-IG tokens. shown in Figure the model rapidly minimizes the loss on zero-IG tokens, while the loss on non-zero-IG tokens remains relatively high. This imbalance leads to biased learned distribution, causing the LLM favor less decisive tokens. Therefore, a more effective training approach that emphasizes decisive tokens is necessary.
Decoding Bias: As shown Figure 4 LLMs tend assign higher logits to low IG tokens, particularly those with zero IG. Since beam search selects top-k candidates solely based on token-level logit scores, these low IG tokens are favored during decoding. As a result, the generated item prefixes exhibit higher average entropy than the ground-trutt prefixes ( Figure )) indicating shift toward less informative predictions. This reveals a decoding bias. where the model systematically prefers less decisive tokens due to the likelihood- decoding objective.
4 Information Gain-based Decisiveness-awarc Token Handing (IGD)
To mitigate the token-level bias during both tuning and decoding phases, we propose a two-stage method, Information Gain-based Decisiveness-aware Token Handling (IGD). Specifically, we lever- age IG to quantify token decisiveness and adjust training dynamics and inference process accordingly.
IGD- Tuning. To mitigate learning bias towards non-decisive tokens, we introduce a token-level reweighting scheme into the loss function when tuning LLMs. The revised objective is:
lpl
LIGD
w. (e,,))))))))
(6)
5

---

where we is weight assigned to token ye, Ω is the sum of across all predicted tokens. is defined
[B. if Gf W/ 1, if (er) >
Here, 3 E [0. 1] is hyperparameter that controls the penalty on non-decisive tokens, thereby reducing recovers the standard cross-entropy loss.
the learning focus on tokens with zero IG. Setting
To address decoding bias that favors generic or homogeneous tokens, modify standard beam search scoring function to promote decisive tokens. The revised score at step computed as:
IGD-Decoding.
(yge) = ((ys--1) + wa p(yell,,,,ee
(8)
with the reweighting factor wa computed as:
= 1 (G))
(9)
Here, (G denotes the max-min normalized IG token y/ within the current beam step, scaled to [0, 1] across all candidates. If all candidates have zero-IG, their normalized values are set The hyperparameter a E [0. 1] controls the strength decisiveness calibration. When a = 1, the method falls back to standard beam search scoring.
Experiments
In this section, aim address the following research questions (RQs): RQ1: Does IGD improve the recommendation accuracy of LLM4Rec? RQ2: How does each stage of IGD contribute to performance improvements? RQ3: How does IGD influence the tuning and decoding enhance performance? RQ4: Is IGD effective across LLMs of different scales and tokenization schemes?
5.1 Experimental Setup
Datasets. We evaluate IGD on four publicly available Amazon review datasets (9] CDs. Games, Toys, and Books, covering data from May 1996 to October 2018. Dataset statistics are summarized in Table ! Following the preprocessing procedure in the D3 paper [19], we truncate the data based on timestamps and filter out infrequent users and items, ensuring that each user and each item has at least 5 interactions.
Compared Methods. For standard recommendation settings, we compare our method against: (I) two representative sequential recommendation models GRU4Ree(30) (30,, SAS- Rec[3l] and LRURec[32]; and (2) two katee-ff-the-ar (SOTA) LLM-based recommendation ap- proaches-BIGRedd [IGRee[20] and D9 Our IGD strategy can be integrated into both BIGRee and D3 for fair comparison. To further evaluate the effectiveness of IGD token handling strategy, compare with two token handing baselines that can also be seamlessly integrated into LLM4Rec frameworks: 1) Position Normalization (Pos), which assigns higher weight earlier item tokens to mitigate position bias; and 2) Causal Fine-tuning (CFT) (33] which builds upon Pos by introducing additional causal loss term enhance the modeling of causal effects the token level. See Appendix A for detailed descriptions of all baselines. By default, we implement all M-L.Md methods using Qwen2.5-1.55 (28). More implementation details, including hyperparameter tuning settings, can be found in Appendix B
evaluate the model's top-K recommendation performance, we adopt two widely-used metrics: Hit Ratio (HR @ K) and Normalized Discounted Cumulative Gain (NDCG @ K) 34 Both metrics are computed under the all-ranking protocol (35, where the model ranks all candidate items for each user. In our experiments, report results for K 5 and K 10.
Evaluation Metries.

---

Table 2: Recommendation performance of the compared methods evaluated four benchmark datasets. H@ K and @@k K denote HR and NDCG@ K, respectively. Improvement indicates the relative performance gain of IGD over the corresponding LLM4Rec backbone without any token reweighting. The best results are bold.
CDs
Games
Methods
N@5
N@ I0
H@5
H@
N@5
N@ 10
H@5
H@
GRU4Rec
0.0248
0.0288
0.0342
0.0467
0.0169
0.0221
0.0261
0.0423
SASRec
0.0477
0.0535
0.0647
0.0824
0.0237
0.0290
0.0338
0.0502
LRURec
0.0540
0.0586
0.0680
0.0824
0.0298
0.0363
0,0421
0.0621
BIGRec
0.0502
0.0553
0.0623
0.0782
0.0317
0.0381
0.0430
0.0631
+Pos
0.0511
0.0566
0.0632
0.0802
0.0319
0.0396
0,0423
0.0665
+CFT
0.0509
0.0566
0.0631
0.0810
0.0349
0.0414
0,0482
0.0686
+IGD
0.0540
0.0593
0.0669
0.0833
0.0423
0.0507
0.0576
0.0833
Improvement
+7.78%
+7.82%
+9.33%
+9.04%
+33.4%
+33.1%
+34.0%
+32.0%
D3
0.0716
0.0767
0.0882
0.1040
0.0415
0.0477
0.0581
0.0773
+Pos
0.0729
0,0779
0.0902
0.1053
0.0429
0.0489
0.0581
0.0767
+CFT
0.0736
0.0786
0.0917
0.1069
0.0437
0.0499
0.0613
0.0806
+IGD
0.0748
0.0801
0.0929
0.1092
0.0518
0.0598
0.0705
0.0946
Improvement
+4.47%
+4.43%
+5.33%
+5.00%
+25.6%
+29.2%
+26.7%
+22.7%
Toys
Books
Methods
N@S
N@ 10
H@5
H@
N@5
N@ 10
H@S
H@ 10
GRU4Rec
0.0200
0.0238
0.0275
0.0392
0.0060
0.0078
0.0094
0.0149
SASRec
0.0356
0.0398
0,0473
0.0745
0.0097
0.0123
0.0146
0.0226
LRURee
0.0358
0.0404
0.0463
0.0608
0.0257
0.0277
0,0319
0.0383
BIGRec
0.0553
0.0623
0,0736
0.0951
0.0190
0.0211
0,0245
0.0309
+Pos
0.0561
0.0631
0.0741
0.0958
0.0197
0.0218
0.0255
0.0319
+CFT
0.0561
0.0630
0.0746
0.0961
0.0195
0.0218
0.0250
0.0321
+IGD
0.0577
0.0656
0.0771
0.1014
0.0267
0.0294
0.0334
0.0419
Improvement
+4.34%
+5.30%
+4.76%
+6.62%
+41.3%
+40.0%
+36.9%
+36.0%
D3
0.0634
0.0698
0.0833
0.1029
0.0212
0.0228
0.0266
0.0315
+Pos
0.0644
0.0702
0.0850
0.1029
0.0221
0.0237
0.0275
0.0324
+CFT
0.0640
0.0704
0.0840
0.1036
0.0219
0.0236
0,0275
0.0327
+IGD
0.0658
0.0726
0.0868
0.1082
0,0291
0.0313
0.0356
0.0424
Improvement
+3.79%
+4.01%
+4.20%
+5.15%
+37.3%
+37.3%
+33.8%
+34.6%
5.2
Main Results (RQ1)
In this section, we evaluate whether the proposed IGD method improves overall recommendation performance. Table 2 summarizes the performance of all compared methods. For token handling strategies (IGD, CFT, and Pos), we implement each on top of both BIGRec and D3 comparison. For traditional baselines such GRU4Rec and SASRec, paper [19] to ensure consistency. For LRURec, we utilize the official PoTorch implementation and evaluate it using dataset split. Our key observations as follows:
adopt the reported results from the D3
IGD achieves notable improvements both LLM4Ree methods (BIGRec and D3). Specifically, yields an average improvement of 20.9 % in HR @ 10 and 21.5% in NDCG@ 10 on BIGRec, and 18.9% in HR @ 10 and 20.1% in NDCG@ 10 D3.
Compared with other token-handiln handling methods (Pos and CFT), IGD consistently delivers better performance across all evaluation metrics on both the LLM4Rec backbone, showing its effectiveness and generalizability. The superiority of the proposed IGD method can be attributed to its specific consideration of token decisiveness.
LLM4Rec methods (BIGRee, D3) clearly outperform traditional recommendation models (GRU4Ree, SASRec). Although LRURec outperforms them on the CDs and Books datasets, BIGRec and D3 regain the lead when combined with token reweighting techniques, highlighting the advantages of leveraging LLMs in recommendation tasks.

---

Table 3: Ablation results of IGD D3. Here, "w/o" was "woo wy",, and "w/o Both" denote the removal of decoding reweighting, tuning reweighting, and both components, respectively.
CDs
Games
Methods
N@5
N@ 10
H@5
H@ 10
N@5
N@
H@5
H@ 10
IGD
0.0748
0.0801
0.0929
0.1092
0.0518
0.0598
0.0705
0.0946
w/o wa
0.0751
0.0800
0.0926
0.1077
0.0514
0.0594
0.0695
0.0942
w/o wi;
0.0718
0.0768
0.0887
0.1041
0.0414
0.0484
0.0575
0.0790
w/o Both 0.0716
0.0767
0.0882
0.1040
0.0415
0.0477
0.0581
0,0773
Toys
Books
Methods
N@5
N@ 10
H@5
H@ 10
N@S
N@ 10
H@5
H@ 10
IGD
0.0658
0.0726
0.0868
0.1082
0.0291
0.0313
0.0356
0.0424
w/o wid
0.0653
0.0719
0.0861
0.1063
0.0290
0.0312
0.0355
0.0422
w/o wa
0.0640
0.0711
0.0843
0.1060
0,0212
0.0229
0.0268
0,0318
w/o Both
0.0634
0.0698
0.0833
0.1029
0.0212
0.0228
0.0266
0.0315
5.3 Ablation Study (RQ2)
In this section, we analyze the individual contributions of the two stages of IGD reweighting Brreighting-Tunnnng and Decoding- the overall improvement in recommendation accuracy. The results of the ablation study are presented in Table 3 We observe the following: (1) Both the tuning-stage and the decoding- stage reweighting contribute positively to performance gains; (2) The tuning stage brings a more substantial improvement than the decoding stage. Notably, even when using only the tuning-stage reweighting (w/o wa), IGD still outperforms the CFT and Pos baselines.
5.4 In-depth Analysis of IGD's Effect (RQ3)
In this subsection, we investigate how IGD influ- ences the underlying mechanisms of tuning and decoding, and how these effects contribute to the improvement of recommendation performance. We follow a similar experimental strategy de- scribed in Scction 3.3 to compare tuning loss and prefix entropy.
CDs
Zero-IG Taken Average Loss
Non-zero-IG IG Taken Average Loss
Uh Wy
with Wy
Lass
Loss
3.007
Dee (50)
As shown in Fig. 5 after incor- porating the weight term W/, the training loss of zero-IG tokens decreases more slowly, whereas the loss for non-zero-IG tokens decreases more rapidly. This indicates that IGD-Tuning en- courages learning on non-zero-IG tokens while avoiding overfitting zero-IG ones.
IGD-Tuning:
250
500
750
1,000
250
500
750
1,000
Training Steps
Training Steps
Games
Zero-IG Token Average Loss
Non-zero-IG Token Average Loss
with
Loss
Loos
2. IGD-Decoding: As illustrated in Figure 6 the entropy gap between the predicted and ground truth prefixes is reduced under IGD- Decoding, indicating more decisive and better- aligned decoding process. Note that increasing (see Equation ) to mitigate decoding bias does not always improve recommendation accu- racy, suggesting a trade-off between maximizing the likelihood of the token sequence and prior- itizing high-decisiveness tokens.
G 104
P. 8,00
500
1,000
1,500
1,000
1.500
Training Steps
Training Steps
Figure 5: Loss comparison on CDs and Games datasets: IGD-Tuniig effect zero-IG and non- zero-IG tokens (epoch 1). The results on the other two datasets are in Appendix C
5.5 Generalizability (RQ4)
To demonstrate the generalizability of our proposed IGD method, we evaluate it under different tokenization strategies and model scales. Specifically, we conduct experiments using the 8- model [36), with the prefix tree constructed using the LLaMA3 tokenizer [36]. To enable efficient
8

---

Table 4: Performance comparison of D3+IGD original D3 on LLaMA3-8B backbone on CDs and Games. The results on the other two datasets are in Appendix
CDs
Games
Methods
N@5
N@
H@5
H@
N@5
N@ 10
H@5
H@ 10
D3
0.0742
0.0790
0.0917
0.1062
0.0456
0.0528
0.0611
0.0832
+IGD
0.0791
0.0850
0.0994
0.1179
0.0645
0.0734
0.0863
0.1137
Improvement
+6.60%
+7.59%
+8.40%
+11.0%
41.5%
+39.0%
+41.2%
+36.7%
fine-tuning, we employ 4-bit QLoRA 33 for both training and inference, with the rank parameter r set to 32, the scaling factor set to 64, and the dropout rate set to 0.05. As shown in Table IGD yields consistent improvements across all four datasets when applied to D3, with an average gain of 26.8% in HR @ 10 and 26.4% in NDCG@ @ 10. In addition to the Amazon dataset, we also evaluate IGD on the Steam dataset (see Appendix E), where it demonstrates consistent improvements. These results confirm the generalizability of our IGD approach across different model architectures and datasets.
Related Work
LLM-based Recommendation: Based on how LLMs are utilized, existing LLM4Rec meth- ods can be broadly categorized into two groups: (1) using LLMs augment traditional recom- mendation models
0e000
0.04
0.15
[8 [39 40), and (2) em- ploying LLMs directly as recommendation sys- tems 20
Myu Mp
0-02
Me Ne
0.03
0-02
99 41. Our work aligns with the second category. Early approaches this direction typically followed a discriminative paradigm [24). which has since shifted toward generative paradigm (20). Building this, recent studies have explored enhancing collab- orative modeling [2,
0.05
0.01
0.00
Stop1
Step
Figure 6: Entropy difference: prediction vs. ground truth after IGD-decoding. "Start" indicates optimal selected based on HR @ 10. The results on the other two datasets are in Appendix [
43 44 optimizing tok-
enization schemes [22, and improving infer- ence efficiency 45). etc. However, few works have investigated recommendations at the token level. Our work specifically focuses on this.
Token-level Biases in L.M4Rec: Autoregressive LLMs generate items token-by- by-token, misaligning with item-level recommendation objectives [46] 19
Multi-token items suffer from token-level biases: bigh-probbbblity ghost tokens dominate decoding without aiding discrimination, inflating scores for longer items [19; and common tokens share across many items (c.g., "The') dilute the
47 48
impact of rare and informative tokens, reducing diversity 46 19].
Token-level Bias Mitigation Strategies: To reduce amplification bias caused by ghost tokens, D3 removes length normalization in decoding [19. but ghost tokens still skew beam search. Position normalization reweights tokens by position, assuming early tokens are more uncertain 333, yet ghost tokens appear throughout sequences and are not confined later positions. To prevent common token domination, some methods leverage traditional model guidance [47 19), but this limits flexibility and scalability. Different from them, we propose a token- saeen-handiin strategy by considering token decisiveness, which helps debiasing.
Token Prefix Trie for LLM4Rec: Recent works leverage token prefix trie for LLM-based rec- ommendation, each employing the trie in different way to guide token generation and learning. MSL 48 uses the trie constraint: a masked softmax prunes infeasible next tokens, reducing negative optimization signals and focusing training on valid continuations, Flower 47 uses the trie for suncess-guided supervision: rewards are stored at nodes and propagated along paths during tuning, encouraging trajectories that obtain higher rewards. Ours uses the trie for decisiveness-awarr weighting: each node stores an IG score estimated from data, and these scores are applied to debias both tuning and decoding, highlighting discriminative tokens while doightigghting ambiguous ones.

---

7 Conclusion & Limitation
In this work, we introduced a decision-process perspective for to-kneby-token generation in LLM4Rec, quantifying token decisiveness using IG. We identified tuning and decoding biases where current models misallocate focus between decisive and non-decisive tokens. Our proposed IGD method addressed these issues through token reweighting during both training and inference. Experiments across four datasets, two LLM4Rec backbones, and two LLM architectures demonstrated IGD's effectiveness, achieving significant improvements in recommendation accuracy.
Our current experiments focus exclusively IG-based token scoring. However, the IG values of tokens are highly skewed (see Figure which makes challenging to design practical reweighting methods (as discussed in Appendix G). Future work may explore alternative decision metries, such as Gini impurity, Gain Ratio, or Chi-squared statistics [49 50 51]. which may provide complementary perspectives on token decisiveness.
In addition, our present analysis is restricted to text tokens. Extending decisiveness modeling to semantic ID tokens [34 52 53 research.
and multimodal tokens [54
55
56 is promising direction for future
Acknowledgement
This research/project is supported by the National Research Foundation, Singapore under its National Large Language Models Funding Initiative (AISG Award No: AISG-NMLP--0244 ) and **STAR under its SSigaap-e Joint Call: Japan Science and Technology Agency (IST) and Agency for Science, Technology and Research (A*STAR) 2024 (Award R2416IR 122). Any opinions, findings and conclusions recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore. Any opinions, findings and conclusions recommendations expressed in this material those of the author(s) and do not reflect the views of the A STAR.
References
[1] Y. Zhang, F. Feng, X. He, T. Wei, C. Song, G. Ling, and Y. Zhang, "Causal intervention for leveraging popularity bias in recommendation." in Proceedings of the 44th international ACM SIGIR conference research and development in information retrieval, 2021, pp. 11-20.
[2] C. Gao, S. Wang, S. Li, J. Chen, X. He, W. Lei, B. Li, Y. Zhang, and P. Jiang, "Cirs: Bursting filter bubbles by counterfactual interactive recommender system," " ACM Transactions on Information Systems (TOIS), vol. 42, no. 1, aug 2023. [Online]. Available: https://doi.org/ org/ 10.1145/3599871
[3] Y. Zhang, Z. Hu, Y. Bai, J. Wu, Q. Wang, and F. Feng, "Recommendation unlearning via influence function," ACM Transactions on Recommender Systems, vol. 3, no. 2, pp. 1-23, 2024.
[4] Achiam, Adler, S. Agarwal, L. Ahmad, Akkaya, F. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al., "Gpt-4 technical report," arXiv preprint arXiv:2303.08774 2023.
[5] D. Guo, D. Yang. H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi et al., "Deepseck-rl: Incentivizing reasoning capability in llms via reinforcement learning," arXiv preprint arXiv:2501. /2948, 2025.
[6] A. Grattafiori, A. Dubey, Jauhri, A. Pandey, A. Kadian, A. AL-Dahle, A. Letman, A. Mathur,
A. Schelten, A. Vaughan el al., "The llama 3 herd of models," arXiv preprint arXiv:2407.21783 2024.
[7] X. Zhao, M. Yan, Y. Zhang, Y. Deng, J. Wang, F. Zhu, Y. Qiu, H. Cheng, and T.-S. Chua, "Reinforced strategy optimization for conversational recommender systems via network-of- experts," arXiv e-prints, pp. arXiv-2509, 2025.
[8] X. Zhao, L. Wang, Z. Wang, H. Cheng, R. Zhang, and K.-F. Wong, "Pacar: Automated fact-checking with planning and customized action reasoning using large language models," in International Conference on Language Resources and Evaluation, 2024. [Online]. Available:
10

---

[9] Y. Qiu, T. Shi, X. Zhao, F. Zhu, Y. Zhang, and F. Feng. "Latent inter-user difference modeling for Ilm personalization. arXiv preprint arXiv:250722044, 2025.
[10] Harte, W. Zorgdrager, P. Louridas, A. Katsifodimos, D. Jannach, and M. Fragkoulis, "Lever- aging large language models for sequential recommendation" in Proceedings of the 17th ACM Conference on Recommender Systems, 2023, pp. 1096-1102.
[11] J. Lin, R. Shan, C. Zhu, K. Du, B. Chen, S. Quan, R. Tang, Y. Yu, and W. Zhang. "Rella: Retrieval-enhanced large language models for lifelong sequential behavior comprehension in recommendation." " in Proceedings of the ACM Web Conference 2024, 2024, pp- 3497-3508.
[12] Y. Zhang, W. Xu, X. Zhao, W. Wang, F. Feng, X. Hc, and T.-S. Chua. "Reinforced latent reasoning for llmebased recommendation, arXiv preprint Xiv:2255 2025.
[13] X. Zhao, Y. Deng, W. Wang. H. lin, H. Cheng, R. Zhang, S.-K. Ng, and T.-S. Chua, "Exploring the impact of personality traits on conversational recommender systems: A simulation with large language models," ArXiv, vol. abs/2504.12313. 2025. [Online]. Available: https///api.smmaniesss www.nticcchoolr..ogggg..oooo hwtpmaaiieeoourrrrooooruuuo////d
[14] L. Li, Y. Zhang, and L. Chen, "Prompt distillation for efficient Ilm-based recommendation," in Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, 2023, pp. 1348-1357.
[15] W. Liu, Z. Du H. Zhao, W. Zhang, X. Zhao, G. Wang, Z. Dong, and J. Xu, "Inference computation scaling for feature augmentation in recommendation systems," ArXiv, vol. abs/2502.1600 2025. [Online]. Available: 276575263
[16] C. Liu, Y. Bai, X. Zhao, Y. Zhang, F. Feng, and W. Rong. "Discrec: Disentangled semantic- collaborative modeling for generative recommendation, arXiv preprint arXiv:250..15576. 2025.
[17] Harte, W. Zorgdrager, P. Louridas, A. Katsifodimos, D. Jannach, and M. Fragkoulis, "Leveraging large language models for sequential recommendation." p. 1096-1102, 2023. [Online]. Available: https://doi.org 10. 1t..11/4//6644153616665
[18] K. Bao, J. Zhang, Y. Zhang. W. Wenjie, F. Feng, and X. He, *Large language models for recommendation: Progresses and future directions," ser. SIGIR-AP '23. New York, NY, USA: Association for Computing Machinery, 2023, p. 306-309, [Online]. Available:
http://doi.org//// /tt..///r//..1111///99939911111 0155/3624918.36255555
[19] K. Bao, J. Zhang, Y. Zhang, X. Huo, C. Chen, and F. Feng, "Decoding matters: Addressing amplification bias and homogeneity issue for llm-based recommendation." arXiv preprint arXiv:2406. /4900, 2024.
[20] K. Bao, Zhang, W. Wang, Y. Zhang, Z. Yang, Y. Luo, C. Chen, F. Feng, and Q. Tian, "A bi-step grounding paradigm for large language models in recommendation systems," ACM Trunsactions on Recommender Systems, 2023.
[21] Zhang, R. Xie, Hou, Zhao, L. Lin, and J.-R. Wen, "Recommendation as instruction following: A large language model empowered recommendation approach," ACM Transactions on Information Systems, 2023.
[22] B. Zheng, Y. Hou, H. Lu, Y. Chen, W. X. Zhao, M. Chen, and J.-R. Wen, "Adapting large language models by integrating collaborative semantics for recommendation, in 2024 IEEE 40th International Conference on Data Engineering (ICDE). IEEE, 2024, pp. 1448.
[23] J. Lin, X. Dai, Y. Xi, W. Liu, B. Chen, H. Zhang, Y. Liu, C. Wu, X. Li, C. Zhu el al., "How can recommender systems benefit from large language models: A survey," ACM Transactions on Information Systems, vol. 43, no. 2, pp. 1-47, 2025.
[24] K. Bao, J. Zhang, Y. Zhang, W. Wang, F. Feng, and X. He, "Tallrec: An effective and efficient tuning framework to align large language model with recommendation" in Proceedings of the 17th ACM Conference on Recommender Systems, 2023, pp. 1007-1014.
[25] C. E. Shannon, "A mathematical theory of communication." The Bell System Technical Journal, vol. 27, no. 3, pp. 379-423, 1948.
[26] R. Baeza-Yates, B. Ribeiro-Neto el al., Modern information retrieval. ACM press New York, 1999, vol. 463, no. 1999.
11

---

[27] T. M. Cover, Elements of information theory. John Wiley & Sons, 1999.
[28] A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei el al., "Qwen2. 5 technical report," arXiv preprini arXiv:2412.551155 2024.
[29] J. Ni, J. Li, and J. McAuley, "Justifying recommendations using distantly-labeles reviews and fine-grained aspects," in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNN.P-IJCNIP)), K. Inui, J. Jiang, V. Ng. and X. Wan, Eds. Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 197. [Online]. Available: https//cccaanooooggoo//91199-000
[30] B. Hidasi, A. Karatzoglou, L. Baltrunas, and D. Tikk, "Session-basee recommendations with recurrent neural networks," arXiv preprint 15 2015.
[31] W.-C. Kang and J. McAuley, "Self-attentive sequential recommendation." in 2018 IEEE Inter- national Conference on Data Mining (ICDM), 2018, pp. 197-206.
[32] Z. Yue, Y. Wang, Z. He, H. Zeng, J. Mcauley, and D. Wang, "Linear recurrent units for sequential recommendation." in Proceedings of the 17th ACM International Conference on Web Search and Data Mining, ser. WSDM '24. New York, NY, USA: Association for Computing Machinery, 2024, p. 930-938. [Online]. Available: https:///oi.org// 0r000..1111111111111111 141111115/565....5 36356666
[33] Y. Zhang. J. You, Y. Bai, J. Zhang, K. Bao, W. Wang, and T.-S. Chua, "Causality- behavior sequence modeling in 1lms for personalized recommendation." arXiv preprint arXiv:2410.22809 2024.
[34] S. Rajput, N. Mehta, A. Singh, R. Keshavan, T. Vu, L. Heidt, L. Hong, Y. Tay, V. Q. Tran, J. Samost, M. Kula, E. H. Chi, and M. Sathiamoorthy, "Recommender systems with gener- ative retrieval," in Proceedings of the 37th International Conference on Neural Information Processing Systems, ser. NIPS '23. Red Hook, NY, USA: Curran Associates Inc., 2023.
[35] W. Krichene and S. Rendle, "On sampled metries for item recommendation" ser. KDD '20. New York, NY, USA: Association for Computing Machinery, 2020, p. 1748-1757. [Online]. Available: https:///ooi 10.1 (45100011444439444444444333
[36] A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur,
A. Schelten, A. Vaughan es al., "The llama 3 herd of models," arXiv preprint arXiv:2407.21783, 2024.
[37] T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, "Qlora: Efficient finetuning of quantized Ilms," Advances in neural information processing systems, vol. 36, pp. 088-8810 115, 2023.
[38] Y. Xi, W. Liu, J. Lin, X. Cai, H. Zhu, J. Zhu, B. Chen, R. Tang, W. Zhang, and Y. Yu, "Towards world recommendation with knowledge augmentation from large language models," in Proceedings of the 18th ACM Conference on Recommender Systems, 2024, pp. 12-22.
[39] X. Ren, W. Wei, L. Xia, L. Su, S. Cheng, J. Wang, D. Yin, and C. Huang, "Representation learning with large language models for recommendation," in Proceedings of the ACM Web Conference 2024, 2024, pp.
[40] L. Sheng, A. Zhang, Y. Zhang, Y. Chen, X. Wang, and T.-S. Chua, "Language models encode collaborative signals in recommendation," 2024.
[41] J. Zhang, R. Xie, Y. Hou, X. Zhao, L. Lin, and J.-R. Wen, "Recommendation as instruction following: A large language model empowered recommendation approach," ACM Transactions on Information Systems, 2023.
[42] Y. Liu, J. Zhang, Y. Dang, Y. Liang, Q. Liu, G. Guo, J. Zhao, and X. Wang. "Cora: Collaborative information perception by large language model's 's weights for recommendationn" " in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 12, 2025, pp. 12 246-122554
[43] Y. Zhang, F. Feng, J. Zhang, K. Bao, Q. Wang, and X. He, "Collm: Integrating collaborative embeddings into large language models for recommendation." IEEE Transactions on Knowledge and Data Engineering, 2025.
[44] Y. Zhang, K. Bao, M. Yang, W. Wang, F. Feng, and X. He, "Text-like encoding of collabo- rative information in large language models for recommendation." " in Annual Meeting of the Association for Computational Linguistics, 2024.
12

---

[45] X. Lin, C. Yang. W. Wang, Li, C. Du, F. Feng, S.-K. Ng. and T.-S. Chua, "Efficient inference for large language model-based generative recommendation" arXiv preprint arXiv:2410.05165. 2024.
[46] C. Gao, R. Chen, S. Yuan, K. Huang, Y. Yu, and X. He, "Sprec: Leveraging self-play to debias preference alignment for large language model-based recommendations, arXiv preprint arXiv:2412.09243. 2024.
[47] C. Gao, M. Gao, C. Fan, S. Yuan, W. Shi, and X. He, "Process-superrised IIm recommenders via flow-guided tuning." arXiv preprint arXhv:2503.07377 2025.
[48] B. Wang, F. Liu, J. Chen, X. Lou, C. Zhang. J. Wang, Y. Sun, Y. Feng, C. Chen, and C. Wang, "Msl: Not all tokens are what you need for tuning Ilm as a recommender," Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information
Retrieval, ser. SIGIR '25. New York, NY, USA: Association for Computing Machinery, 2025, 1912-1922. [Online]. Available:
[49] J. R. Quinlan, "Induction of decision trees," Machine learning, vol. 1, no. 1, pp. 81-106, 1986.
[50] W.-Y. Loh, "Classification and regression trees," Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 1, no. I, pp. 14-23, 2011.
[51] D. J. Hand, H. Mannila, and P. Smyth, Principles of Data Mining. MIT press, 2001.
[52] A. Singh, T. Vu, N. Mchta, R. Keshavan, M. Sathiamoorthy, Y. Zheng, L. Hong, L. Heldt, L. Wei, D. Tandon et al., "Better generalization with semantic ids: A case study in ranking for recommendations, in Proceedings of the 18th ACM Conference on Recommender Systems,
2024, pp. 1039-1044.
[53] Deng, S. Wang, K. Cai, L. Ren, Q. Hu, W. Ding, Q. Luo, and G. Zhou, "Onerec: Unifying retrieve and rank with generative recommender and iterative preference alignment," arXiv preprint wrXiv:2502.18865 2025.
[54] X. Long, J. Zeng. F. Meng, Z. Ma, K. Zhang. B. Zhou, and J. Zhou, "Generative multi-modal knowledge retrieval with large language models," " in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 17, 2024, pp. 18 733 18 741.
[55] H. Liu, Y. Wei, X. Song, W. Guan, Y.-F. Li, and L. Nie, "Mmgrec: Multimodal generative recommendation with transformer model," arXiv preprint arXiv:2404. 2024.
[56] M. Lao, Z. Li, Y. Guo, X. Zhang, S. Cai, Z. Ding, and H. Li, "Boosting discriminability for robust multimodal entity linking with visual modality missing," in Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2025, pp. 989-999
[57] X. Zhao, J. You, Y. Zhang, W. Wang, H. Cheng, F. Feng, S.-K. Ng, and T.-S. Chua, "Nextquill: Causal preference modeling for enhancing Ilm personalization," arXiv preprint 25K6:25006.2366 2025.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The statement supported by the experimental results and clearly explained in the methodology section, ensuring the claim is well-founded..
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of our work are explicitly discussed in section 7 where we
acknowledge potential constraints and directions for future improvements.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer: [Yes]
Our paper rigorously formulates token decisiveness using information theory, clearly stating all underlying assumptions.
Justification: See section 3
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: See Section we provide detailed descriptions of the experimental setup, datasets, model configurations. All hyperparameters and evaluation metrics are clearly reported.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
15

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: A link to the GitHub repository containing all code and data, along with detailed instructions for reproducing the experiments, is provided at the end of the abstract.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ httppp.cc/pubcccguuuceeee Codeeubmii ssi criionPooiccy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please see Section
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
As discussed in Section our method yields a substantial improvement (about 20%) compared to baselines, which strongly supports the significance of the results. Due the high computational cost of rumning multiple randomized trials, we report results based on fixed random seed without error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Basic information on the computational setup is included the README of the code repository. We plan to further elaborate the compute details in the camera-ready version.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The research complies with the NeurIPS Code of Ethies. does not involve human subjects, sensitive personal data, potential dual-use concerns.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See the paper highlights the potential positive societal impact of advanced recommendation systems in helping internet user find relevant and personalized information.
17

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymemt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
The paper does not release any new data or models with high risk for misuse. We use only existing, publicly available datasets and models, and follow their original usage guidelines and licenses.
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We properly cite the code and datasets used in experiments, including [99 and (29), and ensure their usage complies with the respective licenses and terms of use.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
18

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
This paper does not release any new datasets, models, or code assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve crowdsourcing experiments research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This work does not involve research with human subjects or crowdsourcing experiments, and thus IRB approval was required.
19

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question:
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: Our method LLMs
component for recommendation system.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our L.LM policy for what should or should not be described.
20

---

A The Details of Compared Methods
To highlight the modeling strength of LLM4Rec models and provide context for our token-level enhancement method IGD, we first compare them with traditional sequential recommenders.
Traditional Recommendation Methods:
GRU4Rec 30 A widely-used sequential recommendation method that employs Gated Recurrent Units (GRU) to capture sequential patterns and model user preferences.
SASRec 31: A representative sequential recommendation method that utilizes self-attention
mechanism for preference modeling, offering powerful representation capabilities for sequential data.
A linear recurrent unit-hased sequential recommender that enables fast, incremental inference with reduced model size and parallelizable training, achieving strong accuracy and efficiency compared to attention-based baselines.
LRURec 32
LLM4Ree Methods:
BIGRee 20 A representative LLM4Rec system that fine-tunes large language models to generate the next item based the user's historical behavior. We adopt the constrained beam search decoding paradigm as described in D3 paper [9
D3 [99] A state-of-the-ar LLM4Rec approach that fine-tunes the model similarly BIGRec but mitigates amplification bias by removing length normalization during beam search decoding. addition, it incorporates an ensemble design with traditional models, which we omit for fair comparison.
Compared Token Reweighting Methods To evaluate the effectiveness of IGD as token reweight- ing strategy, we compare it against other token-level methods that can be seamlessly integrated into LLM4Rec frameworks. These methods include:
Position Normalization (Pos) 33 This method reweights tokens during SFT based on their position the sequence, assigning higher weight to earlier tokens compared to later tokens.
Causal Fine-tuning (CFT) 33 Building upon the Pos method, CFT introduces additional
context-aware loss term. This loss captures the difference between contextual and non-contextual token predictions (representing causal effects [57]), encouraging the model to emphasize tokens that are more tightly correlated with the input context.
B Implementation Details of Compared Method
For traditional baselines, directly adopt the settings from the D3 paper [19, experimen- tal setup fully aligned with theirs. For LLM-based methods, we use Qwen2.5-1.5B (28) as the backbone. The batch size is set to 64, the optimizer is AdamW, the learning rate is 1 x 10 and the dropout rate 0.05. Model selection is based on validation loss, with an early stop- ping strategy that uses maximum of 3 epochs and patience of 1 epoch. All other settings follow the D3 paper. For our proposed IGD method, the tuning hyperparameter / is selected from the set (0.08, 0.1, 0.2, 0.3. 0.4, 0.5, 0.6, 1.0), and the decoding hyperparameter selected from (0.0, 0.1, 0.2, 0.3, 0.4). We first search for the optimal | based on validation performance, followed by search for the best
C Loss on All Datasets
This section presents comparison the training loss for zero-IG and non-zero-IG tokens before and after applying our IGD-tuning across all datasets. The results are summarized in Figure 7
21

---

CDs
Games
Zero-IG loken Average
Non-zero-IG Token Average Loss
Zero-IG Token Average Loss
Non-zero-IG G Token Average Loss
w/o with
w/o with
Lacs
LosS
LOSS
(e 100
250
500
750
1,000
250
500
750
1,000
500
1,000
1,500
SC0
3,000
1,500
Training Steps
Training
Training Stops
Training Stops
Books
Zero-IG Taken Average Loss
Non-zero-IG Token Average Loss
Zero-IG Token Average Loss
Non-2ero-IG Token Average Loss
w/l wi. with A/l
6 wio My
wel ni with W.
wio with
LOSS
Loo
0724
600 Treining Steps
2,000
4,000
Training Steps
Training Steps
Training Steps
Figure Loss comparison: IGD-Tuning effect on zero-IG and non non-zero-IG tokens (epoch 1).
D Entropy difference on All Datasets
This section presents comparison of entropy differentials across all datasets following the application of IGD-decoding. As the value of a increases, the decoding bias decreases. The results summarized in Figure
CDs
Comes
Toys
Books
0.15
0.04
0.15
2=0.1
0.2
02
0.03
0-12
0.10
0.2
M- -Ne
0.10
0.02
0.01
0.0
0.00
0.00
0.00
6
2
Decoding Step 7
Decoding Step
Decoding Stey
Deceding Step
Figure 8: Entropy difference: prediction vs. ground truth after IGD-decoding. "Start" indicates optimal a selected based on HR@ 10 on all datasets.
E All Results for Generalizability Study
This section presents all the results for the Generalizability study. The results are summarized in Table and T
More dataset. To further assess generalizability, we additionally evaluated the best-performing baseline (D3) and its IGDDeehhaceed variant on Steam dataset (~982K interactions). Results in TJshow that IGD continues to provide consistent gains over the baseline.
F Item Diversity under IGD-Decoding
We evaluate IGD-D's effect on item diversity with a E
First Word Repetition Rate (FWR) lower is better-is the proportion of the most frequent first token among the top-10 recommended items. Item Score Entropy higher is better is computed from the decoding probabilities induced by final item scores using the standard Σ -p log aggregation.
As Table is shown, increasing consistently decreases FWR and increases ISE across all datasets.
22

---

Table 5: Performance comparison of D3+IGD original D3 LLaMA3-8B backbone across
different recommendation datasets
CDs
Games
Methods
N@5
N@ 10
H@5
H@
N@5
N@ 10
H@5
H@
D3
0.0742
0.0790
0.0917
0.1062
0.0456
0.0528
0.0611
0.0832
+IGD
0.0791
0.0850
0.0994
0.1179
0.0645
0.0734
0.0863
0.1137
Improvement
+6.60%
+7.59%
+8.40%
+11.0%
41.5%
+39.0%
+41.2%
+36.7%
Toys
Books
Methods
N@S
N@ 10
H@5
H@ 10
N@5
N@ 10
H@S
H@ 10
D3
0.0739
0.0797
0.0916
0.1096
0.0198
0.0214
0.0249
0.0299
+IGD
0.0885
0.0932
0.1102
0.1311
0.0282
0.0304
0.0352
0.0418
Improvement
19.8%
16.9%
+20.3%
+9.6%
442.4%
+42.1%
441.4%
+-39.8%
Table Results on the Steam dataset (~9922 interactions) under the same evaluation protocol as the main study.
Method
HR
NDCG@ 10
D3
0.1126
0.0782 0.0810 (43.58%)
+IGD
0.1184 (+5.15%)
Mechanism Analysis: Non-decisive tokens tend to receive high logits and dominate beam expansion, reducing diversity. IGD-D increases the logits of decisive tokens (via o)), yielding lower FWR and higher ISE.
G Why IGD-Tuning Adopts a Binary Weighting Scheme
Our binary-based weighting scheme is motivated by the observed behavior of IG in our setting and is not arbitrary. Two empirical observations guide the design:
(1) Distinct zero-IG token group. Tokens with zero form a dominant cluster (about 55% of all tokens) and exhibit disproportionately high logits with very low training loss. Treating this group as separate class and applying uniform down-weighting factor 3 effectively mitigates their outsized influence.
(2) Non-linear IG distribution among non-zero tokens. The IG distribution for non-zero tokens is highly skewed and roughly exponential rather than linear. This makes it difficult to craft a smooth, well-calibrated continuous weighting function over IG. We experimented with simple linear and monotonic alternative:
IG
We + (1 3)
9)
IG
As shown in Table this linear function does not outperform the binary scheme.
Due to the dominance of zero-IG tokens and the non-linear nature of non-zero IG values, binary separation with calibrated i provides stronger and more stable improvements than linear mapping, across all evaluated datasets.
H Effective Hyperparameter Ranges
Our method introduces two parameters: decoding weight and training-time time weight 3. Since a is easy to tune, we only analyze 3's sensitivity while fixing a=0.0. Unless otherwise noted, the search grid for 3 is {0.08, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6); after selecting 3, is tuned over (0.1, 0.2).
Optimal 3. CDs: 3=0.2 (HR 10=0 1077). Games: 3=0.2 (HR 10=0.0942). Toys: 3=0.5
(HR@ @ 1063). Books: 8=0.1 (HR @ 10=0.0422).
23

---

Table Diversity vs. a across datasets. FWR, and ISE1.
CDs
Toys FWR.
Games
Books
FWR.
ISEt
ISE
FWR4
ISE
FWR.
ISEI
0.0
0.364
2.65
0.588
2.67
0.340
2.95
0.382
2.75
0.1
0.332
2.70
0.577
2.73
0.328
2.98
0.339
2.81
0.304
2.76
0.564
2.80
0.321
0.306
2.87
0.3
0.282
2.84
0.557
2.88
0.307
3.04
0.278
2.93
0.4
0.265
2.92
0.551
2.95
0.296
3.08
0.259
2.99
Table 8: Comparison linear vs. binary IGD-T weighting across datasets. Best (Linear): 0.4, 0.9, 0.8, 0.4 for CDs, Games, Toys, Books. Best / (Binary): 0.2, 0.2, 0.5, 0.1 for CDs, Games, Toys, Books.
Dataset
Metric
Baseline (D3)
IGD-T (Linear)
IGD-T (Binary, ours)
CDs
HR@ 10
0.1040
0.1072
0.1077
NDCG@11 10
0.0767
0.0793
0.0800
Games
HR@ 10
0,0773
0,0800
0.0942
NDCG@ 10
0.0477
0.0492
0.0594
Toys
HR@ 10
0.1029
0.1034
0.1063
NDCG@ 10
0.0698
0.0692
0.0719
Books
HR @ 10
0.0315
0.0339
0.0422
NDCG@ 10
0.0228
0.0245
0.0312
Effective ranges (HR@1 11 and NDCG @ 10 > baseline). CDs: [0.1, 0.6]; Games: 0.4] U (0.6); Toys: [0.2, 0.6]; Books: [0.1, 0.6].
Observation. Each dataset exhibits a broad interval where performance exceeds the baseline, making easy to integrate into existing methods. However, to obtain the optimal B, one still needs to search over reasonable range.
24


Reviews:
[{"review_id": "B2Zj7ghjAc", "reviewer": "Reviewer_v8sa", "summary": "The paper proposed a re-weighting method for both tuning and decoding for LLM4Rec setting (directly use LLM as recommender to generate next-item), based on the insight the some tokens doesn’t reduce the entropy of the to-be-generated item but are put on high focus in both tuning and decoding. Specifically, the paper proposed a metric called information gain to determine the importance of the token, and put larger weight on high importance token while lower weight on less important ones. The paper performs experiments on benchmark data and achieves good results.", "questions": "The method is simple and follows the intuition. My main concern is the setup and motivation.\nFrom equation 5, during SFT, the model will focus more on the token that can dramatically decrease the entropy for token output, and the entropy is calculated using the train dataset. \nFrom Table 1, the test dataset is very small compared to the train data, it might be hard to generalize the w_t if test data is large (i.e. in practice, testing data is much more compared to SFT data). In this case, the estimation of IG on training data can be very different from the actual potential distribution. Moreover, if during testing, there is a new item with a new title, then this method might not work. As in 5.1 first paragraph mentioned, the dataset is truncated in a way that each user and item has at least 5 interactions. Although this is used by another paper (one of the baseline), this kind of process actually leads to implicit leakage since this ensures the statistics in the training dataset can be leveraged to test the dataset. \n\n\nIn eq (7): instead of introducing a heuristic hyperparameter beta, why not make the weight a continuous and monotonic function of IG? \nHow to tune beta and alpha?\n\n\nwhy are there tuning bias and decoding bias? more in-depth analysis on why this is the case would be helpful", "limitations": "The experiment setup does not reflect the real recommendation scenarios. See above comments.\n\nthe weighting approach seems highly heuristic but not mathematically motivated, and not clear how to tune the hyperparameters and no studies on the sensitivity of those", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength:\n* The paper has simple intuition and the experiment results are promising on several datasets.\n* The proposed IG metric is simple to calculate. The overall algorithm is easy to be plugged into existing methods.\n\nWeakness:\n* The insight seems great, but the approach is quite heuristic but not mathematically motivated; it’s not clear how to tune the hyperparameters beta and alpha, and there is no empirical studies on the sensitivity of such hyperparameters \n* The IG metric is specifically pre-defined through training dataset, and might be hard to generalize in real recommendation scenarios.\n* The experiment are not convincing regarding to the IG metric, specifically the setup seems biased towards to proposed method (i.e. removing tail item to make sure training and testing distribution are more similar/similar type recommendation(i.e. CD, Games, etc)).", "was_revised": true, "final_justification": "I have read the rebuttal, while most concerns are addressed, I'm not convinced on the real performance since the evaluation dataset is unrealistic with 5-core filtering; however I acknowledge this is standard practice in the research community, and this paper does bring some novel ideas and insights that might inspire more follow-up work. Hence I'm willing to increase my score to 4.", "rebuttal": "We thank the reviewer for their constructive feedback. As the questions and weaknesses are closely related, we summarize the concerns and address them collectively, rather than responding to each point in the two sections separately.\n\n---\n\n### Q1. Unrealistic Experimental Setup\n\nQ1.1: The current data split, where the test set is much smaller than the training set, may limit the effectiveness of the evaluation.\n\nA1.1: We respectfully clarify that our data splitting strategy is designed to reflect common real-world recommendation scenarios. \nIn many industry applications, models are trained on a large corpus of historical user interactions (the \"training set\") to predict user behavior over a shorter, subsequent time period (the \"test set\"). Our chronological 8:1:1 train-validation-test split directly follows this paradigm and is consistent with standard practice in existing works.\n\nHowever, to further address your concern, we conducted an additional experiment by  re-splitting the dataset into a 5:1:4 ratio for training, validation, and testing, with keeping the chronological order. This setup results in a test set comparable in size to the training set and introduces a significant temporal gap between training and testing periods. We compared our method, IGD-Tuning (IGD-T), with the strongest baseline (D3) under this new split. The results, summarized in the following table, show that IGD-T consistently and significantly outperforms D3. This demonstrates that our method (and the computed weights $w_t$) generalizes well and is not simply benefiting from a specific data split.\n\n\n| Dataset | Metric   | Baseline (D3) | IGD-T (ours) | Improvement |\n|---------|----------|---------------|--------------|-------------|\n| CDs     | HR@10    | 0.0923        | 0.0975       | +5.6%       |\n|         | NDCG@10  | 0.0698        | 0.0732       | +4.9%       |\n| Games   | HR@10    | 0.0680        | 0.0809       | +19.0%      |\n|         | NDCG@10  | 0.0452        | 0.0542       | +19.9%      |\n| Toys    | HR@10    | 0.1093        | 0.1141       | +4.4%       |\n|         | NDCG@10  | 0.0791        | 0.0822       | +3.9%       |\n| Books   | HR@10    | 0.0424        | 0.0588       | +38.7%      |\n|         | NDCG@10  | 0.0321        | 0.0447       | +39.3%      |\n\n---\n\nQ1.2.  Data preprocessing introduces potential information leakage, as it allows the statistics in the training dataset can be leveraged to test the dataset \n\nA1.2: This appears to be a misunderstanding. Our data preprocessing does not suffer from information leakage. On the contrary, we adopt a chronological data splitting strategy specifically to avoid information leakage from the future into the past.\n\nThe reviewer's concern seems primarily to stem from the use of 5-core filtering. First, we would like to clarify that this is not a filtering step introduced by any particular baseline, but rather a widely adopted standard practice in recommendation research. This approach is used in many well-known works in the field:\n- \"Hidden factors and hidden topics\" (McAuley & Leskovec, RecSys 2013): Established the use of 5-core filtering on the now-standard Amazon dataset.\n- \"Image-based recommendations on styles and substitutes\" (McAuley et al., SIGIR 2015): Continued the practice with 5-core filtering for visual recommendation.\n- \"Neural Graph Collaborative Filtering\" (Wang et al., SIGIR 2019): Adopted a 10-core filter for modern graph-based models, confirming k-core is a standard procedure.\n\nSecond, we respectfully argue that applying 5-core filtering does not leak training data statistics into the test set. To illustrate this, consider the cold-start item scenario raised by the reviewer: the 5-core filtering is applied to the entire dataset before the chronological data split. As a result, items that appear for the first time during the test period are retained. Consequently, our test sets still include a substantial number of cold-start items—approximately 10% in the Books dataset and 8% in the Toys dataset—which naturally results in distributional drift from training to test. Additionally, significant drift can also be observed from the perspective of item frequency distribution.\n\nTable: Frequency Draft - Top 10% frequent item in the training set and their Avg. Freq. Percentile in Test set.\n| Category | Avg. Freq. Percentile in Test Set |\n|:---------|:---------------------------------------|\n| CDs_and_Vinyl | 31.52%                                 |\n| Video_Games | 30.67%                                 |\n| Toys_and_Games | 39.72%                                 |\n| Books    | 64.68%                                 |\n\n---\n\n### Q2: The Design of the IGD Token Weighting Scheme: Why not use a continuous and monotonic function of IG instead of a binary weighting approach?\n\nA2: Our binary-based weighting approach is not designed arbitrarily—it is motivated by two key observations about the behavior of Information Gain (IG) in our context: \n\n1) Distinct Zero-IG Token Group: As shown in Figures 2 and 4, tokens with zero IG form a distinct and dominant group in the dataset (55% of all tokens). These tokens consistently exhibit extremely low training loss and disproportionately high logits. Treating them as a separate class and applying a uniform down-weighting factor $\\beta$ is a natural and effective way to handle their outsized influence.\n\n2) Non-linear IG Distribution: As illustrated in Figure 4, the distribution of non-zero IG values is highly skewed and roughly exponential, not linear. This makes it difficult to design a meaningful continuous function over IG values that results in well-calibrated weights. We experimented with a simple linear and monotonic weighting function:\n$$w_t = \\beta + (1-\\beta) \\cdot \\frac{\\mathrm{IG}}{\\mathrm{IG}_{\\max}}$$\nHowever, as shown in the following tables, this linear function does not outperform our binary approach.\n\n| Dataset | Metric   | Baseline | IGD-T (Linear) | IGD-T (Binary, ours) |\n|---------|----------|----------|----------------|----------------------|\n| CDs     | HR@10    | 0.1040   | 0.1072         | 0.1077               |\n|         | NDCG@10  | 0.0767   | 0.0793         | 0.0800               |\n| Games   | HR@10    | 0.0773   | 0.0800         | 0.0942               |\n|         | NDCG@10  | 0.0477   | 0.0492         | 0.0594               |\n| Toys    | HR@10    | 0.1029   | 0.1034         | 0.1063               |\n|         | NDCG@10  | 0.0698   | 0.0692         | 0.0719               |\n| Books   | HR@10    | 0.0315   | 0.0339         | 0.0422               |\n|         | NDCG@10  | 0.0228   | 0.0245         | 0.0312               |\n\n*(IGD-T (Linear): best β=0.4, 0.9, 0.8, 0.4 for CDs, Games, Toys, Books)*\n\n*(IGD-T: best β=0.2, 0.2, 0.5, 0.1 for CDs, Games, Toys, Books)*\n\n---\n\n### Q3. Hyperparameter Sensitivity Analysis\nA3: Our method introduces two additional parameters, $\\alpha$ and $\\beta$. The parameter $\\alpha$ is used only during decoding and is tuned over {0.1, 0.2} after selecting a suitable $\\beta$, thus incurring minimal hyperparameter-tuning cost.\n\nHere, we mainly present a sensitivity analysis of the training-related hyperparameter $\\beta$, while keeping $\\alpha$ fixed. We evaluate $\\beta$ over the range {0.1, 0.2, 0.3, 0.4, 0.5, 0.6}, and summarize the results in the following table.\n\n| Dataset | Baseline HR@10 (β=1.0) | Optimal β | Max HR@10 (within effective β) | Min HR@10 (within effective β) | Effective β Range (HR@10 & NDCG@10 > Baseline) |\n|:--------|:----------------------:|:---------:|:--------------------------:|:--------------------------:|:----------------------------------------------:|\n| CDs     |         0.1040         |    0.2    |      0.1077 (β=0.2)        |      0.1066 (β=0.4)        |                  [0.1, 0.6]                   |\n| Games   |         0.0773         |    0.2    |      0.0942 (β=0.2)        |      0.0777 (β=0.6)        |                [0.1, 0.4] $\\cup$ [0.6]                |\n| Toys    |         0.1029         |    0.5    |      0.1063 (β=0.5)        |      0.1038 (β=0.2)        |                  [0.2, 0.6]                    |\n| Books   |         0.0315         |    0.1    |      0.0422 (β=0.1)        |      0.0335 (β=0.6)        |                  [0.1, 0.6]                    |\n\nAs shown, our method is relatively less insensitive to the exact choice of $\\beta$: it can achieves strong performance across a wide range of values in each dataset. However, the suitable range of $\\beta$ may vary across datasets."}, {"review_id": "Nafq1EdlzW", "reviewer": "Reviewer_QLsy", "summary": "The paper introduces a novel method on using large language models (LLMs) for recommendation. In particular, the authors study the entropy / information gain (IG) of different type of tokens and found that existing methods can cause the model to over-focus on zero-IG tokens while under-emphasizing non-zero-IG tokens during tuning. To address this, the authors model the item-generation process as a sequential decision task and weight the loss of each token in training depending on its IG. In addition, the authors also modify the generation process by revising the scores based on their IG, and therefore mitigating the bias toward high-logit but low-decisiveness tokens. The authors experiment on different datasets and compare to multiple benchmarks, where the proposed method can be applied to different eixsting approaches and show performance improvements.", "questions": "1. How does the proposed method apply to different decoding paradigms like sampling / constrained beam search, or does it work with a more genrealized sampling setting with different logits processing (top-k, top-p, repetition penalty etc.)?\n\n2. Is the proposed method sensitive to differen decoding settings? Especially with key parameters like temperature.", "limitations": "N/A", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Pros:\n\n1. The proposed method is interesting and studies an important problem from the information theory perspective, i.e., how IG / entropy affect the decoding process and influcenes the downstream task performance.\n\n2. The proposed method is somewhat intuitive and reasonable, i.e., non-zero IG tokens with low logit values can be more effective to optimize in trainning. In addition, this method is simple and can be applied to different existing methods such as BIGRec and D3.\n\n3. The authors experimented on multiple datasets, demonstrating its performance over different baseline methods.\n\nCons:\n\n1. The authors only experimented on the RecSys setting with limited datasets, although this setting should naturally extend to more generaly language modeling tasks.\n\n2. The proposed method intriduced additional hyperparameter complexity in both training and decoding (e.g., beta and alpha), increasing the hyperparameter searching costs while only providing marginal gains in many cases.\n\n3. The selected RecSys datasets are all from amazon and rather small in sizes. I encourage the authors to experiment LLMs on larger datasets, potentially with multimodal inputs to comprehensively evaluate the proposed method. More baseline methods from both traditional sequential methods (LRURec etc.) and LLM methods (TallRec etc.) shouls also improve the quality of the experiments.", "was_revised": true, "final_justification": "Thanks for the response provided by the authors, the additional results do address most of my concerns of the paper. I also believe this setting can be extended for language generation tasks with similar techniques (e.g., based on token entropy etc.), but the current setting and experiment results are satisfactory. Considering the rebuttal efforts and the paper's merits / limitations, I have raised my score to 4 accordingly.", "rebuttal": "We thank the reviewer for their insightful feedback and constructive suggestions. We appreciate that they found our method 'interesting' and 'intuitive', and the research problem 'important. To address their concerns about the method's scope, we have conducted several new experiments regarding hyperparameter complexity, dataset scale, and baseline comparisons. We believe these additions substantially strengthen the manuscript.\n\n---\n\n### 1. Can we naturally extend experiments to more generally language modeling tasks?\nA1: We would like to clarify that our proposed IGD method is specifically tailored for LLM-based recommendation. Its computation of Information Gain (IG) depends on a predefined, finite set of items and their frequencies—an assumption that does not hold in general language modeling tasks. This limitation makes it challenging to define item-set entropy or derive meaningful IG values in open-domain settings. Consequently, our method is not directly applicable to general language modeling at this stage. Nevertheless, we are interested in exploring extensions to broader domains in future work.\n\n---\n\n### 2. Hyperparameter Complexity\nA2: Though our method introduces two additional parameters, $\\alpha$ and $\\beta$, the parameter $\\alpha$ is used only during decoding and is tuned over {0.1, 0.2} after selecting a suitable $\\beta$, avoiding grid search. Moreover, as shown in our ablation results, even without using $\\alpha$, our method still yields performance improvements. Therefore, tuning $\\alpha$ can just incur minimal hyperparameter tuning cost.\n\nThe primary tuning cost comes from $\\beta$, which influences the training process. While we acknowledge that tuning $\\beta$ introduces some overhead, our sensitivity analysis indicates that this cost is modest in practice. Across various datasets, our method consistently outperforms baselines over a broad range of $\\beta$ values (as shown in the following table). This suggests that identifying a suitable $\\beta$ for achieving performance gains is relatively straightforward and does not require extensive tuning.\n\n| Dataset | Baseline HR@10 (β=1.0) | Optimal β | Max HR@10 (within effective β) | Min HR@10 (within effective β) | Effective β Range (HR@10 & NDCG@10 > Baseline) |\n|:--------|:----------------------:|:---------:|:--------------------------:|:--------------------------:|:----------------------------------------------:|\n| CDs     |         0.1040         |    0.2    |      0.1077 (β=0.2)        |      0.1066 (β=0.4)        |                  [0.1, 0.6]                   |\n| Games   |         0.0773         |    0.2    |      0.0942 (β=0.2)        |      0.0777 (β=0.6)        |                [0.1, 0.4] $\\cup$ [0.6]                |\n| Toys    |         0.1029         |    0.5    |      0.1063 (β=0.5)        |      0.1038 (β=0.2)        |                  [0.2, 0.6]                    |\n| Books   |         0.0315         |    0.1    |      0.0422 (β=0.1)        |      0.0335 (β=0.6)        |                  [0.1, 0.6]                    |\n\n---\n\n### 3. More dataset (dataset beyond Amazon) and more baselines (LRURec, TALLRec)\nA3: Thanks you for the suggestion. We response the two points, respectively.\n\n**More dataset:** Following your advice, we conducted an additional experiment comparing the best-performing baseline (D3) with our IGD method (applied to D3) on a new dataset from Steam, which contains approximately 982K interactions. The results are summarized in the following table. As shown, our method continues to deliver meaningful performance improvements.\n\n| Method    | HR@10           | NDCG@10         |\n|-----------|-----------------|-----------------|\n| Baseline  | 0.1126          | 0.0782          |\n| IGD       | 0.1184 (+5.15%) | 0.0810 (+3.58%) |\n\n**More baselins:** We have added two strong baselines: LRURec, the suggested traditional sequential recommender method, and Tiger, a recent generative recommendation method. We did not include TALLRec in the comparison, as it is not designed for the all-ranking setting used in our experiments. Due to time constraints, we conduct the comparison on the CDs dataset only. The results, presented in the following table, show that our method (applied to the D3 baseline) consistently outperforms these strong baselines.\n\n| Method   | HR@5   | NDCG@5 | HR@10  | NDCG@10 |\n|----------|--------|--------|--------|---------|\n| LRURec   | 0.0680 | 0.0540 | 0.0824 | 0.0586  |\n| Tiger    | 0.0748 | 0.0620 | 0.0868 | 0.0659  |\n| IGD      | 0.0929 | 0.0748 | 0.1092 | 0.0801  |\n\n*(We will include the full tables for all datasets in the revised manuscript.)*\n\n---\n\n### 4. Can the proposed method be applied to different decoding paradigms or generalized sampling settings with various logits processing strategies? Additionally, how sensitive is the method to decoding settings, such as temperature?\n\nA4:  First, we would like to clarify that our work focuses on recommendation tasks, where the model is required to generate a list of real-world items in each prediction. To achieve this, deterministic constrained beam search (do_sample=False) is commonly adopted, and we follow this setting as well. Under this setup, randomness introduced by sampling is eliminated. While in theory our method can be integrated with various decoding paradigms and sampling strategies, the specific requirements of recommendation systems make deterministic constrained beam search a natural and practical default choice in our framework.\n\nRegarding the sensitivity of our method to decoding settings, we conducted additional experiments using stochastic sampling (do_sample=True) and varied the temperature parameter. We report representative results under nearly deterministic (T = 0.1), low (T = 0.7), and default (T = 1.0) temperature settings by comparing our IGD versions with or without applying the proposed decoding strategy. The results are summarized in the table below.  \n\n| Dataset | Method         | Deterministic | T=0.1  | T=0.7  | T=1.0  |\n|:--------|:---------------|:-------------:|:------:|:------:|:------:|\n| CDs     | IGD (w/o $w_d$)  |    0.1077     | 0.1076 | 0.1041 | 0.1015 |\n| CDs     | IGD (α=0.2)    |    0.1092     | 0.1085 | 0.1058 | 0.0951 |\n| Games   | IGD (w/o $w_d$)  |    0.0942     | 0.0934 | 0.0914 | 0.0914 |\n| Games   | IGD (α=0.1)    |    0.0946     | 0.0947 | 0.0926 | 0.0900 |\n| Toys    | IGD (w/o $w_d$)  |    0.1063     | 0.1052 | 0.1054 | 0.1018 |\n| Toys    | IGD (α=0.2)    |    0.1082     | 0.1078 | 0.1061 | 0.1032 |\n| Books   | IGD (w/o $w_d$)  |    0.0422     | 0.0423 | 0.0411 | 0.0390 |\n| Books   | IGD (α=0.1)    |    0.0424     | 0.0424 | 0.0410 | 0.0388 |\n\n\nAs shown, our proposed decoding method $w_d$ remains robust and continues to bring performance Improvements (compared to our IGD without using the proposed decoding strategy)— at lower temperatures (T ≤ 0.7)."}, {"review_id": "oP2UBtCkrz", "reviewer": "Reviewer_14G4", "summary": "This work improves LLM-based recommendation systems, which traditionally treat all tokens equally and maximize likelihood, by introducing a reweighting approach that prioritizes more informative tokens based on their importance. Experiments were conducted on four datasets and two backbone models, demonstrating substantial improvements in recommendation performance.", "questions": "1. Compared to previous models, this approach seems to involve more computation. Could you share approximately how much training and decoding time it requires in practice?\n2. During decoding, as the IG weighting increases, it seems likely that the model would increasingly prioritize informative tokens. Did you observe any issues with reduced diversity or over-concentration on certain items as a result?", "limitations": "Yes, the authors have discussed the main limitations of their work. However, I think the scalability to real-time applications could also be acknowledged.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "- **Quality**: The paper clearly identifies a key limitation of existing LLM-based recommendation systems and proposes an appropriate solution. However, the approach appears computationally intensive and likely to incur significant training and inference time.\n- **Clarity**: The research idea, mathematical formulations, and experimental results are all described clearly and precisely.\n- **Significance**: The paper proposes a practical improvement to the bias issue in token-level generation and convincingly demonstrates strong performance through extensive experiments. However, since the evaluation is limited to Amazon datasets, it would be important to validate the approach in other domains.\n- **Originality**: The token decisiveness-based training and decoding strategy is novel. It highlights the potential need for similar decoding mechanisms in existing LLM-based recommendation models.", "was_revised": true, "final_justification": "The rebuttal effectively addressed my concerns regarding diversity and provided additional experiments on a new dataset. The proposed token decisiveness-based training and decoding strategy is novel and highlights the potential need for similar decoding mechanisms in other LLM-based recommendation models. The reported computation overhead seems generally reasonable in offline batch settings, though its real-time latency impact remains less explored; this does not strongly affect my current recommendation. Given these clarifications and improvements, I recommend borderline accept.", "rebuttal": "We sincerely thank the reviewer for their thoughtful and constructive feedback. We are grateful for the positive assessment of our work's quality, clarity, and originality. The reviewer's insightful questions have prompted us to conduct further analyses that we believe have significantly strengthened our paper.\n\nBelow, we address each of the reviewer's points with detailed, quantitative evidence.\n\n\n---\n\n### W1: This approach seems to involve more computation. How much training and decoding time it requires in practice?\nA1: During training, the additional cost of our method comes primarily from looking up the IG score for each token (performed via a prefix trie) and applying multiplicative weighting. This overhead is relatively small. During decoding, the extra cost involves looking up IG scores and reweighting the logits of candidate tokens based on these scores. This process is also lightweight.\nWe summarize the empirical comparison results in the table below. As shown, the additional cost introduced by our method is generally acceptable.\n| Stage    | Overhead |\n|----------|----------|\n| Tuning   | +3.35%   |\n| Decoding | +9.79%   |\n(Test using Toys dataset with H100GPU)\n\n\n---\n\n### W2: Evaluation on datasets beyond Amazon domain.\nA2: Thank you for the suggestion. We have conducted an additional experiment comparing the best-performing baseline (D3) with our IGD method (applied to D3) on a new dataset from Steam, which contains approximately 982K interactions. The results are summarized in the following table. As shown, our method continues to yield meaningful performance improvements.\nNotably, due to time constraints, we have not included all baselines in this comparison. We will include the full set of results in the revised version of the paper.\n\n\n| Method    | HR@10           | NDCG@10         |\n|-----------|-----------------|-----------------|\n| Baseline  | 0.1126          | 0.0782          |\n| IGD       | 0.1184 (+5.15%) | 0.0810 (+3.58%) |\n\n--- \n\n### W3: On Decoding Diversity: As the IG weighting increases, it seems likely that the model would increasingly prioritize informative tokens. Is there any issue with reduced diversity or over-concentration on certain items?\n\nA3: Thank you for this insightful question. Contrary to the potential concern, our proposed method, IGD-D, does not reduce diversity. In fact, our empirical results consistently show that **as the weight on informative tokens (controlled by `α`) increases (in a wide range), the diversity of the recommended items also increases.**\n\n**Empirical Verification**:\n\nWe evaluated this effect using two diversity metrics across four standard datasets:\n- **item_score_entropy** (higher is better; indicates more diverse recommendations)\n- **first_word_repetition_ratio** (lower is better; indicates less repetition at the start of recommended items)\n\nThe results below demonstrate a clear trend across all datasets. As `α` increases from 0.0 to 0.4: `item_score_entropy` consistently rises and `first_word_repetition_ratio` consistently falls:\n\n**CDs**\n\n| α     | item_score_entropy | first_word_repetition_ratio |\n|-------|--------------------|----------------------------|\n| 0.0 | 2.65               | 0.364                      |\n| 0.1 | 2.70               | 0.332                      |\n| 0.2 | 2.76               | 0.304                      |\n| 0.3 | 2.84               | 0.282                      |\n| 0.4 | 2.92               | 0.265                      |\n\n**Toys**\n\n| α     | item_score_entropy | first_word_repetition_ratio |\n|-------|--------------------|----------------------------|\n| 0.0 | 2.67               | 0.588                      |\n| 0.1 | 2.73               | 0.577                      |\n| 0.2 | 2.80               | 0.564                      |\n| 0.3 | 2.88               | 0.557                      |\n| 0.4 | 2.95               | 0.551                      |\n\n**Video Games**\n| α     | item_score_entropy | first_word_repetition_ratio |\n|-------|--------------------|----------------------------|\n| 0.0 | 2.95               | 0.340                      |\n| 0.1 | 2.98               | 0.328                      |\n| 0.2 | 3.01               | 0.321                      |\n| 0.3 | 3.04               | 0.307                      |\n| 0.4 | 3.08               | 0.296                      |\n\n**Books**\n| α     | item_score_entropy | first_word_repetition_ratio |\n|-------|--------------------|----------------------------|\n| 0.0 | 2.75               | 0.382                      |\n| 0.1 | 2.81               | 0.339                      |\n| 0.2 | 2.87               | 0.306                      |\n| 0.3 | 2.93               | 0.278                      |\n| 0.4 | 2.99               | 0.259                      |\n\n**Analysis of the Mechanism:**\n- Root Cause: From our analysis (Figures 3 and 4), the model overly prioritizes non-informative tokens (such as \"The\" at the start of many item titles). These tokens tend to receive very high logits, leading beam search to preferentially expand paths containing them. As a result, the generated item sequences often share similar prefixes, which in turn reduces diversity and causes significant repetition.\n\n- Role and Effect of `IGD-D`:  Our `IGD-D` decoding strategy directly combats this issue by boosting the logits of informative tokens (i.e., setting `α > 0`). This adjustment encourages the model to prioritize more informative tokens during decoding, thus mitigating the inherent bias toward repetitive, non-informative outputs. Figure 6 (\"Entropy difference: prediction vs. ground truth after IGD-decoding\") has evaluated the effect from the item entropy perspective."}, {"review_id": "WKsRnb2u4F", "reviewer": "Reviewer_YKBt", "summary": "This paper proposes an Information Gain-based Decisiveness-aware Token Handling (IGD) strategy for Large Language Model-based Recommendation (LLM4Rec), targeting the issue that existing methods usually treat all item tokens equally, ignoring differences in their decisiveness. The authors treat the item generation process as a decision process and use Information Gain (IG) to quantify each token’s decisiveness. They observe that many tokens have low IG but high logits, introducing bias in both training and decoding. IGD addresses this by down-weighting low-IG tokens during tuning and decoding to emphasize high-IG tokens—shifting the focus from pure likelihood maximization to decision-critical tokens. Experiments on four benchmark datasets with two LLM backbones show that IGD consistently improves recommendation accuracy.", "questions": "Q1. Has the author considered a more fine-grained token categorization, beyond simply distinguishing between zero and non-zero information gain tokens? If so, what are the results?\n\nQ2. What is the candidate item distribution? Is it determined by the frequency of item occurrences in the interaction dataset?", "limitations": "Yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\nS1. Measuring the importance of item tokens from the perspective of “decisiveness” is interesting and introduces a novel angle.\n\nS2. The authors treat the item generation process as a decision-making process and define token decisiveness through information gain. This definition is both novel and well-justified.\n\nS3. The proposed method is applicable during both the tuning and inference stages, allowing it to emphasize high-decisiveness tokens throughout the modeling pipeline.\n\nS4. The paper is well-motivated: it begins with a preliminary experiment to illustrate the problem, followed by the introduction of the proposed method.\n\nS5. Extensive experiments on four benchmark datasets, using two LLM backbones (Qwen2.5 and LLaMA3), demonstrate consistent performance improvements.\n\nWeaknesses:\n\nW1. Currently, tokens are divided into only two categories based on information gain. While this binary classification is reasonable, a more fine-grained categorization—guided by the magnitude of information gain—may lead to a better results.\n\nW2. Figure 1 is difficult to follow. Additionally, the layout of Section 3 could be made more compact to improve clarity and readability.\n\nW3. The candidate item distribution is not clearly explained and requires further explanation.\n\nW4. A concurrent work [1] also models item generation as a decision tree, which bears some relevance to this paper. A discussion of this related work should be included to clarify distinctions and highlight contributions.\n\n[1] Gao et al. Process-Supervised LLM Recommenders via Flow-guided Tuning.", "was_revised": true, "final_justification": "I have read the authors’ response and decided to maintain my positive assessment.", "rebuttal": "We sincerely thank the reviewer for their positive assessment and insightful feedback on our manuscript. We are encouraged that the reviewer found our work to be novel, well-motivated, and supported by extensive experiments (S1–S5). We appreciate the constructive suggestions, which have helped us improve the clarity and rigor of our paper. Below, we address each of the reviewer's concerns and questions in detail.\n\n--- \n\n### W1&Q1: Why not take a more fine-grained weighting?\nA1: Thank you for the question. Our binary-based weighting design is motivated by two key observations about the behavior of Information Gain (IG) in our preliminary experiments:\n\n1) Distinct Zero-IG Token Group: As shown in Figures 2 and 4, tokens with zero IG form a distinct and dominant group in the dataset (>55% of all tokens). These tokens consistently exhibit extremely low training loss and disproportionately high logits. Treating them as a separate class and applying a uniform down-weighting factor $\\beta$ is a natural and effective way to handle their outsized influence.\n\n2) Non-linear IG Distribution: As illustrated in Figure 4, the distribution of non-zero IG values is highly skewed and roughly exponential, not linear. This makes it difficult to design a meaningful continuous function over IG values that results in well-calibrated weights. We experimented with a simple linear and monotonic weighting function:\n$$w_t = \\beta + (1-\\beta) \\cdot \\frac{\\mathrm{IG}}{\\mathrm{IG}_{\\max}}$$\nHowever, as shown in the following tables, this linear function does not outperform our binary approach.\n\n| Dataset | Metric   | Baseline | IGD-T (Linear) | IGD-T (Binary, ours) |\n|---------|----------|----------|----------------|----------------------|\n| CDs     | HR@10    | 0.1040   | 0.1072         | 0.1077               |\n|         | NDCG@10  | 0.0767   | 0.0793         | 0.0800               |\n| Games   | HR@10    | 0.0773   | 0.0800         | 0.0942               |\n|         | NDCG@10  | 0.0477   | 0.0492         | 0.0594               |\n| Toys    | HR@10    | 0.1029   | 0.1034         | 0.1063               |\n|         | NDCG@10  | 0.0698   | 0.0692         | 0.0719               |\n| Books   | HR@10    | 0.0315   | 0.0339         | 0.0422               |\n|         | NDCG@10  | 0.0228   | 0.0245         | 0.0312               |\n\n*(IGD-T (Linear): best β=0.4, 0.9, 0.8, 0.4 for CDs, Games, Toys, Books)*\n\n*(IGD-T: best β=0.2, 0.2, 0.5, 0.1 for CDs, Games, Toys, Books)*\n\n--- \n\n ### W2: Clarity of Figure 1 and Section 3\n A2: Thanks for pointing this out. We will revise the figure to improve its clarity and update Section 3 accordingly.\n\n--- \n\n### W3 & Q2: What is the candidate item distribution? Is it determined by the frequency of item occurrences in the interaction dataset?\nA3: Yes. The candidate item distribution, denoted as $P(\\mathcal{I})$, represents the prior probability of any item  $\\mathcal{I}$, estimated by the frequency of item occurrences in the interaction dataset.\n\n--- \n\n### W4. Discussion of Concurrent Work\nA4: Thanks for bringing this highly relevant concurrent work to our attention. While both our work and Gao et al. conceptualize item generation as a sequential decision-making process, our approaches differ fundamentally in their problem formulation, methodology, and goals:\n\n- **Core Idea and Methodology:**\n  - **Flow-guided Tuning (Gao et al.):** They introduce a \"Flow-guided process reward\" to supervise the entire sequence of token generation logits, rewarding paths that are more efficient at identifying the target item. Their contribution is a novel process-level supervision signal.\n  - **Our Work (IGD):** We start from an information-theoretic perspective to identify a fundamental issue: a discrepancy between a token's generation likelihood and its actual \"decisiveness\" in narrowing down the candidate set. We propose Information Gain (IG) to quantify this decisiveness and identify two specific biases (\"Tuning Bias\" and \"Decoding Bias\"). Our IGD strategy is a targeted intervention that re-weights token losses during tuning and adjusts logits during decoding to mitigate these biases.\n\nWe hope this discussion clarifies the unique contributions of our work and its relationship to concurrent research. In the revised manuscript, we have added the discussion to the \"Related Work\"."}]

Author rebuttal:

Meta-review: This paper proposes a simple yet effective method to reweight the tokens by information gain (over the item catalog distribution) to effectively improve training and generation for recommendation. This is motivated by the empirical observation that most tokens have low information gain but often correspond to high logits. Empirical studies demonstrate that the proposed weighting scheme helps with various existing LLM-based recommender systems. 

The paper initially received negative-learning scores. During the rebuttal periods, the authors provided clarification as well as additional results with more baselines/datasets, which convinced the reviewers with negative scores to raise to borderline accept. Overall, all the reviewers agree that the proposed method is conceptually simple and intuitive and it can improve various methods with relatively small overhead. However, the negative concerns are around 1) the proposed reweighting scheme is more like a heuristic based on empirical observation and less theoretically motivated, 2) estimating item entropy from the training set can be potentially problematic, especially in a real-world setting where the item distribution is ever-changing. It is less clear how the proposed method can respond to such settings. I think despite the negatives, there are some interesting insights. However, I wouldn't be upset if this paper doesn't end up getting accepted due to the concerns mentioned above.
Decision: Accept (poster)