Preference Diffusion for Recommendation
Authors: Shuo Liu
An Zhang
Guoqing Hu
Hong Qian
Tat-Seng Chua
Venue: ICLR 2025 Poster (2025)
Keywords: Sequential Recommendation，Diffusion Model
Primary area: other topics in machine learning (i.e., none of the above)

Abstract: Recommender systems aim to predict personalized item rankings by modeling user preference distributions derived from historical behavior data. While diffusion models (DMs) have recently gained attention for their ability to model complex distributions, current DM-based recommenders typically rely on traditional objectives such as mean squared error (MSE) or standard recommendation objectives. These approaches are either suboptimal for personalized ranking tasks or fail to exploit the full generative potential of DMs. To address these limitations, we propose \textbf{PreferDiff}, an optimization objective tailored for DM-based recommenders. PreferDiff reformulates the traditional Bayesian Personalized Ranking (BPR) objective into a log-likelihood generative framework, enabling it to effectively capture user preferences by integrating multiple negative samples. To handle the intractability, we employ variational inference, minimizing the variational upper bound. Furthermore, we replace MSE with cosine error to improve alignment with recommendation tasks, and we balance generative learning and preference modeling to enhance the training stability of DMs. PreferDiff devises three appealing properties. First, it is the first personalized ranking loss designed specifically for DM-based recommenders. Second, it improves ranking performance and accelerates convergence by effectively addressing hard negatives. Third, we establish its theoretical connection to Direct Preference Optimization (DPO), demonstrating its potential to align user preferences within a generative modeling framework. Extensive experiments across six benchmarks validate PreferDiff's superior recommendation performance. 
Our codes are available at \url{https://github.com/lswhim/PreferDiff}.

Published as a conference paper at ICLR 2025
PREFERENCE DIFFUSION FOR RECOMMENDATION
Shuo 1 An Zhang²* Guoqing Hu
Hong Qian'
Tat-Seng Chua?
1 East China Normal University, China
National University of Singapore, Singapore University Science and Technology of China, China
3
shuoliuee luoouiuutuu... ecnu. edu. en, anzhang@u. nus edu, h115671 953307777 ustc.me edu. cn, hgi anê cs. ecnu,
www..edu..cc....e edu. dcsct.sênus .etessssuuuueuuu...... sg
ABSTRACT
Recommender systems aim to predict personalized item rankings by modeling user preference distributions derived from historical behavior data. While diffusion models (DMs) have recently gained attention for their ability to model complex dis- tributions, current DM-based recommenders typically rely on traditional objectives such as mean squared error (MSE) or standard recommendation objectives. These approaches are either suboptimal for personalized ranking tasks fail exploit the full generative potential of DMs. To address these limitations, we propose Prefer- Diff, an optimization objective tailored for DM- recommenders. PifferDiff reformulates the traditional Bayesian Personalized Ranking (BPR) objective into a log-like logliiklihood generative framework, enabling effectively capture user pref- erences by integrating multiple negative samples. To handle the intractability, we employ variational inference, minimizing the variational upper bound. Furthermore, replace MSE with cosine error to improve alignment with recommendation tasks, and we balance generative learning and preference modeling to enhance the training stability of DMs. PreferDiff devises three appealing properties. First, it is the first personalized ranking loss designed specifically for DM-based recom- menders. Second, it improves ranking performance and accelerates convergence by effectively addressing hard negatives. Third, we establish its theoretical connection Direct Preference Optimization (DPO), demonstrating its potential align user preferences within generative modeling framework. Extensive experiments across six benchmarks validate PreferDiff's superior recommendation performance. Our codes are available at Ihtps:: /egg/gtttubb www.:/wwwwmmm............ cowp//swwhi///eefrrrrrriii
INTRODUCTION
The recommender system endeavors to model the user preference distribution based on their historical behaviour data (He & McAuley) 20166 Wang et 2019) Rendle (2022) and predict personalized
item rankings. Recently, diffusion models (DMs) (Sohl-Dickstein al. 2015 Ho et al. 2020 Yang
et al. 2024) have gained considerable attention for their robust capacity to model complex data distributions and versatility across wide range of applications, encompassing diverse input styles:
texts (Li et al. 2022 Lovelace et al. 2023), images (Dhariwal & Nichol 2021 Ho & Salimans 2022)
and videos (Ho al. 2022a b). As result, there has been growing interest employing DMs as recommenders in recommender systems.
These DM-based recommenders utilize the interaction data to uncover the potential target item, typically following one of three approaches:
process on the user's historical
modeling the distribution of the next item (Yang al. 2023b Wang et al. 2024b Li al. 2024).
capturing the user preference distribution (Wang et al. 2023b Zhao et al. 2024 Hou et al. 2024a
Zhu et al. 2024). or focusing on the distribution of time intervals for predicting the user's 's next action (Ma et al. 2024a). However, prevalent DM-based recommenders often routinely rely standard generative loss functions, such as mean squared error (MSE), or blindly adapt established recommendation objectives, such Bayesian personalized ranking (BPR) (Rendle et al., 2009) and (binary) cross entropy (Sun et al. 2019) without any modification. Despite their empirical
Zhang is the corresponding author.

---

Published as a conference paper at ICLR 2025
Density
Targut Nor (Pedihe)
Nan-titeratted Hore (Negatio)
Erdiaed Frsirnd
0.064
0.069
0,056
0,052
0,018
0.096
(4)
Figure 1: Illustration of user preference distributions modeled by DM-based recommenders. (a) Neglecting the negative item distribution leads predicted items potentially being closer to negative items. (b) Incorporating the negative sampling enhances the understanding of user preferences.
success, two key limitations in their training objectives have been identified, which may hinder further advancements in this field:
DM-based recommenders inheriting generative objective functions YYng al., 2023b) lack
comprehensive understanding of user preference sequences. They model user behavior by considering only the items users have interacted with, neglecting the critical role negative items
in recommendations (Chen et al. 2023a 2024 Zhang et al. 2024). As illustrated in Figure [a),
although the predicted item centroid is close to the positive item, the sampling process of the DMs may tend to obtain the final predicted item embedding in high-densit regions (red in Figure [ a)))..
This can result in the predicted item embedding being too close to negative items, thereby affecting the personalized ranking performance. Enabling DMs to understand what users may dislike can help alleviate this issue, as illustrated in Figure (Tb).
DM-hased recommenders simply employ standard recommendation training objectives, hindering their generative ability. This type of DM-based recommenders treats DMs primarily as noise-resistant models that focus ranking or classification rather than on generation. While this approach can mitigate the impact of noisy interactions inherent in recommender systems (Wang et al. 2023b Li et al.] (2024), may not fully exploit the generative and generalization capabilities DMs. whose primary objective is to maximize the data log-likelln ligllikel
To better understand and redesign diffusion optimization objective that is specially tailored model user preference distributions for personalized ranking, we aim to simultaneously encode user dislikes and enhance the generative capability the ranking objective. Our approach involves extending the classical and widely-adopted BPR objective incorporate multiple negative samples, while generative models, exemplified by DMs (Yang
also clarifying its connection to likelihood-bate et al.,222000 2024). BPR only seeks to maximize the rating margin between positive and negative items, which may result high score negative ratings. In contrast, our core idea focuses on modeling user preference distributions, where the distribution of positive items diverges from that of negative items, conditioned on the user's personalized interaction history.
To this end, propose training objective specifically designed for DM-based recommenders, called PreferDiff, which effectively integrates negative samples to better capture user preference distributions. Specifically, by applying softmax normalization, we transform BPR from a rating ranking into log-likelihood ranking, leading to the formulation of LEPR-DIIT However, since DMs are latent variable models (Ho et al., (020), direct optimization through gradient descent intractable. To address this intractability. we derive a variational upper bound for LBPr Dir using variational inference, which serves as a surrogate optimization target. Furthermore, we replace the original MSE with cosine error (Hou et al. 2022b). allowing generated items to better align with the similarity calculations in recommendation tasks and controlling the scale of embeddings (Chen et al. 2023c). Additionally, we extend LEPR-Dif to incorporate multiple negative samples, enabling the model to inject richer preference information during training while implementing an efficient strategy to prevent redundant denoising steps from excessive negative samples. Finally, balance generation learning and preference learning to achieve a trade-off that enhances both training stability and model performance, culminating in the final objective function, PrreteDar.
2

---

Published as a conference paper at ICLR 2025
Benefiting from a comprehensive understanding of user preference distributions, PreferDiff has three appealing properties: First, PreferDiff is the first personalized ranking loss specifically designed for DM-based recommenders, incorporating multiple negatives to model the user preference distributions. Second, gradient analysis reveals that PreferDiff handles hard negatives by assigning higher gradient weights item sequences, where DM incorrectly assigns higher likelihood negative items than positive ones (Chen et al. 2022 Fan et al. 2023, Zhang et al. 2023)) Section (.2). This not only improves recommendation performance but also accelerates training (ef. Section 4.1 Third, from preference learning perspective, we find that PreferDiff is connected to Direct Preference
Optimization (Rafailov et al. (023) under certain conditions, indicating its potential align user
preferences through generative modeling in diffusion-based recommenders (ef. Section 3.2
We evaluate the effectiveness of PreferDiff through extensive experiments and comparisons with baseline models using six widely adopted public benchmarks (cf. Section 4.1 Furthermore, by simply replacing item ID embeddings with item semantic embeddings via advanced text-embedding modules, PreferDiff' shows strong generalization capabilities for sequential recommendations across untrained domains and platforms, without introducing additional components (cf. Section 4.2)
2 PRELIMINARY
In this section, we begin by formally introducing the task of sequential recommendation and then introduce the foundations of DM-based recommenders who model the next-item distribution.
Suppose each user has historical interaction sequence representing their interactions in chronological order and in is the next target item. For each sequence, we randomly sample negative items from batch or candidate set result in
Sequential Recommendation.
{ir, .2,... , 1
H = {ix -=1- Moreover, each item is associated with unique item ID or additional descriptive information (e.g., title, brand and category). Via ID-embedding or text-embedding module, items can be transformed into its corresponding vectors E IR 1xd Therefore, the historical interaction sequence and negative items' set can be transformed to fer, e2,.... en- I and H { The goal of sequential recommendation is to give the personalized ranking on the whole candidate set, namely, predict the next item 8, user may prefer given the sequence and negative items' set H.
[P]
Diffusion models for Sequential Recommendation. In this section, we introduce the use of guided DMs to model the conditional rext-item distribution p(i,, i.x), following the DreamRec Yang al. 2023b). For clarity, we denote the vector representation of the next item i, as ej instead of e, and negative items i, as e," result in H. = {e, " [ The subscript denotes the timesteps in DM, where "0" indicates that noise has been added, and the superscript represents whether the item is positive or negative, denoted by "+" or "_" respectively in recommendation. Notably, these notations will be used consistently in the subsequent sections.
Forward Process. DMs add Gaussian noise to the positive item embedding ej with noise scale {ai, 022, ... } over the pre-defined timesteps T', namely, q(e," e = (Va (1 āt II).
If + +oo, ef asymptotically converges to the standard Gaussian distribution. qee"" e can be
easily derived through applications of the reparameterization trick (Kingma & Welling 2014).
Reverse Process. The reverse process aims to recover the target item embedding e from the standard Gaussian distribution through the denoising process with the personalized guidance Concretely, following the classical DMs paradigm introduced in DDPM (Ho et al. 2020), we choose the simple objective which minimizes the KL divergence between the true denoising transition q(et, e e,) and the intractable denoising transition pa (e e c). Leveraging the favorable properties of the Gaussian distribution, we can derive the following closed-form objective:
Ssiple = (Eiett. ..1)) [||Folet,
M,c() (I)Fo(et,,,M(c) +t 1
-
where e come from the training data. t ~ (1) is the sampled timestep. M(-) denotes the arbitrary sequence encoder utilized in sequential recommendation (e.g., GRU (Hidasi et al. 2016), Transformer (Kang& & McAuley 2018), Bert (Sun al. 2019)). Fe(-) serves denoising network which is commonly parameterized by a simple MLP and denotes the trainable parameters. Classifier-free guidance scheme (Ho & Salimans 2022) can be utilized here replace M(c) with
3

---

Published as a conference paper at ICLR 2025
dummy token @b with probability Pa achieve the training of unconditional DM. Furthermore, some works (Li al. 2024) utilize the recommendation objective (binary) cross entropy instead of MSE.
Inference and Recommend. During the inference stage, first derive the representation of given user's historical sequence, denoted as _M(c). Starting from pure Gaussian noise, we then utilize the denoising network Fo(.) to iteratively generate latent embeddings, following arbitrary samplers (e.g.. DDIM (Song et al., (021a)) in DMs, until the inferred next item embedding èg is obtained. More details can be found in Algorithm and Appendix B Finally, we recommend the top-K items with the highest dot product between éo and the item embeddings in the candidate set.
3 METHODOLOGGY: THE PROPOSED PREFERDIEE DIFF
In this section, we introduce PrfferDiff, a novel loss for DM-based recommenders that can instill preference information. First, we extend the classical BPR loss to probabilistic one, defining a new loss LAPR-DiO- To address the inherent intractability, we derive variational upper bound LUpper for LePRADi Diar and optimize this bound instead. Furthermore, we explore the incorporation of multiple negative samples and propose an efficient strategy by lowering the likelihood of the negative samples' centroid, which avoids multiple denoising steps. Lastly, we make trade-off between learning generation and learning preference to ensure training stability, resulting in the final loss PpretenDin-
3.1 CONNECT DIFFUSION MODELS WITH BAYESIAN PERSONALIZED RANKING
In this subsection, explore the integration of DMs with the classical BPR loss (Rendle et al. 2009). which has been proven to be highly effective in real-world industrial recommendation scenarios. As BPR is designed to optimize personalized ranking by modeling user preferences in a pairwise fashion, it has been extensively applied in contemporary recommendation researches (Kang & McAAleyy He al. 2020). It can be formulated
2018
LBPR - .,,,)) .c) [log a (fo(e) |)) fo(e cc)]]
where e 6g represents the positive item and one negative item in H. we omit for brevity. represents the historical item sequences. a is the Sigmoid function. So(ep c) is the predicted rating of item eo conditioned on the historical item sequence of likelihood-basee log- teelikkliihod of the next item distribution log po c c), it is clear that equation 2 does not meet this need. Therefore, we put forward to change the rating to the probability distribution.
As DMs are part of the family generative models (Yang et al.) 2024) and are employed here to maximize the
From Rating to Probability Distribution. Here, define the probability distribution of the next- item ep given historical item sequences e via a softmax over the arbitrarily flexible, parameterizable, rating function fo(.). It can be formulated as Po(Co
c) = expife(eo e)) where Zo is normalizing constant (a.k.a, partition function), defined as f exp(fe(e c)) de. Then, by substituting into equa- tion 2 we obtain the following result, which we refer to as CBPR-Diff. as we utilize the DMs to model that distribution. The detailed derivation is provided Appendix C.I
2o
LaPR-Dir(0) (0) ( ,6, (e...)))) .e) [logo Pe c) Pog [c)]]
(3)
Intuitively, LERR-DIF seeks to widen the gap between the log- log-probayility distributions of positive and negative items given However, the challenge is that equation BJss intractable due to the need to marginalize over all possible diffusion paths as DMs are latent variable models. Therefore, like previous work (Sohl-Dickstein et al.) 2015 Ho al.) 2020), we propose minimize the LEPR-Dar via variational inference through minimizing the derived variational upper bound.
Minimize LEPR-DIr through Variational Upper Bound. Therefore, like previous work (Sohl-
Dickstein et al.
2015 Ho et al. 2020), we introduce latent variables (ei,..., er), resulting in Po(Co c) Po (Gpr c) de1:r- Then, substitute po(e1:r eg) with ((er:r eg) which is
typically modeled as a Gaussian distribution with predefined mean and variance at each timestep, due
the intractability of directly sampling from the former distribution. The objective can be expressed
as follows:
po Ic) CaPR-Dif(()) -E eo .c) [Iogo (log Ee 10 q( q e
log
Po Po
log E, (ee q(
4

---

Published as a conference paper at ICLR 2025
By applying Jensen's inequality and leveraging the convexity of the logarithmic function, we move the expectation operator outside. Consequently, after further mathematical derivations, we can establish an upper bound for LBPR Dirl' equation
pa (e c)
Po Po c)
LAPR.DiA Dirr - (B 10 .) 2lee )0())
olog
log
q(e Co
(5)
Following the derivation of classical DMs (Ho et al. 2020 Song et al. 202 Luo 2022 can simplify the above equation through algebra, yielding the following result:
LEPR-DUr(O) - e .)) logo (-12)
Σ B (ett) lett [DLL (a(e;"lee e po po
T
B (e, %)) [DKL (q(ez-1ee | + + ))
where C1 is a constantthath independent of the model parameter 8. As introduced in the Preliminary, by applying Bayes' theorem and leveraging the additivity property of Gaussian distributions, the final trainable objective on stochastic samples over timestep is expressed as follows:
Cpperr = -Be so (L,TT log (- . ,
(7)
S(-) denotes the function that quantifies the distance between the prediction and the true next item embedding, typically MSE in previous works. As retrieval during the inference stage is conducted via maximal inner product search for ranking and MSE shows sensitivity to vector norms and dimensionality (Friedman 1997 Hou et al., 2022b), propose using cosine error instead. Since Lupper serves as an upper bound for LaPR-Dar- minimizing Luper implicitly minimizes LEPR-DiN- Intuitively, equation 7 is designed such that, given user's historical item sequence, the denoising network F(-) tends to recover the positive item rather than the negative item. A detailed derivation can be found in Appendix C.3
Here, 6 = Fo(e; M(c(), =
3.2 ANALYSIS OF LIPR-DIFF
In this subsection, we demonstrate the two properties of LEPR-Dar by analyzing the gradient with respect to 0 and connecting with recent popular direct preference optimization. We also reveal the connection between the rating function and the score function in Appendix equation bridges the objective of recommendation with generative modeling DMs.
C.2 which
Gradient Analysis. Here, we analyze the gradients of LEPR-DEr to understand their impact on the training process of DMs for sequential recommendation.
LBrR 00
(e so .c) [wm
V log (
V po (ep c)
)] (8)
Increase Likelibood on Positive
Likelihood Negative
where wy = 1 a (e c) - log P c)) represents the gradient weight. Obviously, if given certain item sequences, the DM incorrectly assigns higher likelihood the negative items than positive items, and the gradient weight we will be higher. Therefore, optimizing LBPR Difr is capable of handling hard negatives, which has become increasingly important in recent research Chen al. (2022); Fan al. 2023): Zhang et al. (2023).
Connection with Direct Preference Optimization. After determining how to minimize LBPR-Diff using the aforementioned upper bound and analyzing the gradient, we proceed to validate the rationality of LEPR-DUN- Here, we establish a connection with the recently prominent Direct Preference Optimization (DPO) (Rafailov al.) 2023 Wallace al. 2024 Meng et al. 2024). which has been shown to effectively align human feedback with large language models. For further details on DPO, refer readers to (Rafailov et al. 2023). The equation of DPO is expressed follows:
5

---

Published as a conference paper at ICLR 2025
po (2))
c)
3 log Po (arg
c)) Prot(ao c
(D) -B s/ee) log o B log
(9)
Prer(z0)
c)
By comparing equation [ with equation 9 we observe that LEPR-DIN can be viewed as special case of DPO, where B 1 and Pref is a constant distribution (e.g., uniform distribution). This validates that optimizing the proposed LBPR-Diff has the potential to align user preferences in DMs. Notably, give more details about the connection of DPO and PreferDiff in Appendix F.6
3.3 EXTEND MULTIPLE NEGATIVES
As previous works have demonstrated that incorporating multiple negatives during the training phase can better capture user preferences, we extend LePR-Diff to support multiple negatives for instilling more fruitful rank information. Suppose that for each sequence, we have negative items' set H introduced in Section 2 according to equation 7 we can directly derive that:
H M S(O ,o"")
LEPR-DiEEV log o (-111 (S(()) ct)
(10)
77
For brevity, omit the expectation term. However, the above equation applies the noising and denoising process to all negative samples, which significantly reduces the model's training speed and increases susceptibility to false negatives. Therefore, we propose replace the |'|] negative samples
with their centroid En =
[H| == eo" as the diffusion target and derive the following:
LEPR-DILC- = log or (-11 (5(65 . S(Fo
(11)
Assuming that F(-) is a convex function, we can apply Jensen's inequality and derive that CEPR-DIT-V < Dure. Therefore, minimizing LEPR-DiR-C can efficiently increase the likelihood of the positive items while simultaneously distancing them from the centroid the negative items. Intuitively, this aligns with the phenomenon that users may not explicitly indicate dislike for specific items, but rather for a certain category of items. A detailed derivation can be found in Appendix C.4
Training and Inference of PifferDiff. Here, introduce the training and inference details of PifferDiff, as demonstrated in Algorithm [ and Algorithmmiin the Appendix. Empirically, we find that solely using the proposed LBPR-DiM-C leads instability during training. This may be due an overemphasis on ranking information, which can neglect the more accurate generation of the next item. Therefore, we balance the trade-off between learning generation and learning preference with hyperparameter X, with the following:
Peefer
Simple
+ (1 - (L-)
(12)
Leurning
Learning Preference
We conduct experiments about different to show the instable training issue in Section 4.3
4 EXPERIMENTS
In this section, we aim to answer the following research questions:
RQ1: How does PreferDiff perform compared with other sequential recommenders?
RQ2: Can PreferDiff leverage pretraining to achieve commendable zero-shot performance on unseen datasets datasets from other platforms just like DMs in other fields?
RQ3: What is the impact of factors (e.g., X) on PreferDiff's performance?
4.1
PERFORMANCE OF SEQUENTIAL RECOMMENDATION
Baselines. We comprchensively compare PreferDiff with five categories of sequential recommenders: traditional sequential recommenders, including GRU4Rec (Hidasi et al. 2016), SASRec (Kang &

---

Published as a conference paper at ICLR 2025
Table I: Comparison of the performance with sequential recommenders. The improvement achieved by PreferDiff is significant (p-value < 0.05), Results of three additional datasets are in Appendix F.1
Model
Sports and Outduors
Beanty
Tays Games
RP85
N685
R
R
R485
N685
RC:10
N60
4Ree
QU0122
0.0020
0.00/0
0.0013
0.0099
0.0078
0.8102
0.0097
0.0087
SASRee
0U89477
0.0336
010042
0.0138
0.0050
0.02199
0.0116
0.0133
0.0097
BERTHRee
0.0060
0.174
0.0226
CLASRec
0.945
0.0134
TIGER
0,093
0.0033
0.0166
0.0800
0.0366
00151
0.0466
0.0155
0.0155
0.0156
Diftieee
0.0068
001011
0.0155
0.0211
0.0439
0.0268
DreumRee
00155
0.0130
0/0211
0.0406
0.0433
DiffuRee
0.0076
0.0121
0.0266
0.0335
0.0130
0/0262
MoRoe
0.0051
0.0259
0.0353
0.0154
0.0115
LLM2BERTTRRe
0.0118
0.0076
0.0183
0.0474
0.0399
PreferDift
0.0141
0.051
6,0473
PreferDiil-T
Imgrove
19.35%
7.50%
McAuley 2018), and BERT4Rec (Sun et al. 2019); contrastive learning- recommenders, such as CL4SRec (Xie et al. 2022; generative sequential recommenders like TIGER (Rajput et al. 2023);
DM-based recommenders, including DiffRec (Wang et al. 2023b). DreamRee (Yang et al. 2023b) and DiffuRec (Li et al. 2024); and text-based recommenders like MoRec (Yuan et al. 2023) and LLM2Bert4Rec (Harte et al. (203). See Appendix D.3 for details on the introduction, selection and hyperparameter of the baselines.
Datasets, We evaluate the proposed PreferDiff on six public real-world benchmarks (i.e., Sports, Beauty, and Toys from Amazon Reviews 2014 (He & McAuley 2016), Steam, ML-1M, and Ya- hoo!R1). Detailed statistics of three benchmarks can be found in Table 5 Here, we utilize the common five-core datasets, filtering out users and items with fewer than five interactions. More Details about data prepossessing can be found in Appendix [D] Following prior work (Yang et al., 2023b), in [ and Table [4] we employ user-split which first sorts all sequences chronologically for each dataset, then split the data into training, validation, and test sets with an 8:1:1 ratio, while preserving the last 10 interactions as the historical sequence. We reproduce all baselines for fair comparison. Notably, in Table and Ta Appendix D.4] we also give comparison under another setting (i.e., leave-one-out) to provide more insights where the baselines' results are copied from TIGIR. Moreover, we conduct experiments on varied user history lengths in Appendix F:2
Implementation Details. For PirferDiff, for each user sequence, we treat the other next-items (a.k.a., labels) in the same batch as negative samples. We set the default diffusion timestep to DDIM step as 20, Pa = 0.1, and the 3 linearly increase in the range of [le- (,002 for all DM-based sequential recommenders (e.g.. DreamRec). For all text-based recommenders, we utilize OpenAI-3-Large (Neelakantan et al., (022) to obtain the text embeddings. We fix the embedding dimension 64 for all models except DM-based recommenders, as the latter only demonstrates strong performance with higher embedding dimensions. The former does not gain much from high embedding dimensions, which will be discussed in Section Refer to Appendix [.22 for more implementation details about baselines. Notably. PreferDiff can be applied to any sequence encoder, M(-). We provide the results of PreferDiff with other backbones in Appendix D.33 D.3
Evaluation Metrics. We evaluate the recommendation performance in a full-ranking manner (Yng et al. 2023b) using Recall (Recall @ K) and Normalized Discounted Cumulative Gain (NDCG@K) K) with K = 5, 10, following the widely adopted top-K K protocol as the primary metrics for sequential recommendation (Kang & McAuley 2018 Rajput et al., 2023).
Results. Table presents the performance of PreferDiff compared with five categories sequential recommenders. For brevity, stands for Recall, and N stands for NDCG. The tor-performing and runner-up results shown in bold and underlined, respectively. "Improv" represents the relative improvement percentage of PreferDiff over the best baseline. "x" indicates that the improvements are statistically significant 0.05, according to the t-test. We can have the following observations:
DM-based recommenders have exhibited substantial performance gains over other sequential recommenders across most metrics. This is consistent with prior research, which demonstrates that the powerful generation and generalization capabilities (Yang et al., 20236) or noise robust- ness (Wang al. 2023b Li et al. 2024) of DM can better capture user behavior distributions
7

---

Published as a conference paper at ICLR 2025
Table 2: Ablation Study of PreferDiff. Details are the same Table T
Model
Sports and Outdours
Beauty
Toys und Games
R@5
R@:
R@10
R@5
10
PreferDin
0.0185
0.0147
0.0247
0.0167
0.0429
0.0323
0.0514
0,0350
0,0473
0,0367
0.0535
0.0387
0.0165
0.0214
0.0415
0.0492
wlo-C
0.0180
0.0139
0.0230
0.0159
0.0393
0.0282
0.0496
0.0322
0.0458
0.0356
0.0521
0.0374
w/o-C&N
0.0155
0,0130
0.0211
0.0140
0.0406
0.0299
0.0483
0.0326
0.0440
0.0323
0.0490
0.0353
compared to other sequential recommenders and alleviate the false negative or false positive issue in recommendation (Sato al. 2020 Chen al., 2023b).
PreferDiff significantly outperforms other DM-based recommenders across all metrics on three public benchmarks. PreferDiff demonstrates an improvement ranging from 6.41% 11 to 19.35 over the second-best baseline. Our results indicate that modeling the user's next-item distribution more effective than modeling the user's interaction probability distribution (e.g.. DiffRec) in sequential recommendation. Additionally, directly applying classic recommendation objectives (e.g.. DiffuRec) or using objectives that deviate significantly from the original (e.g.. MSE) may impede diffusion models from effectively learning user preference distributions and fully harnessing their generative and generalization capabilities. Moreover, the performance gap between DreamRec and PreferDiff further validates that tailored optimization objective for successfully incorporates personalized ranking information into DMs, enabling them to better unleash their generative potential while more effectively capturing user preference distributions.
recommenders
PrfferDiff can benefit from advanced text-embeddings. We observe that PreferDiff, when incorporating the identical text embeddings (referred as PreferDiff-T), outperforms MoRec and LLMeBert4Rec by replacing traditional ID embeddings with semantic text embeddings or using them initialization parameters of DDeemeeddings. This demonstrates that incorporating text embeddings, which provide a more semantic and stable feature space, into PreferDiff can obtain commendable recommendation performance. This finding aligns with current trends the text- diffusion field (Lovelace et al. 2023 Liu et al.) 2023). Building on this, due to the unified nature of the language space, PifferDiff possesses the potential to generalize sequential recommendations to other unseen domains, which will claborate in the following subsection.
blation Study. As shown in Table 2 we scrutinize and evaluate each key individual component of PreferDiff to comprehend their respective impacts and significance. The ablation analysis is conducted using the following three versions. (1) PreferDiff- employs cosine error as the measure function and drops the learning preference term in Lreteenur- (2) PreferDiff-w/or employs MSE as a measure function. (3) PreferDiff-w/o-C&8
employs MSE as the measure function and
drops the learning preference term in PreeeeDir We can observe that each component in PreferDiff contributes positively. Specifically, the performance degradation due to the omission of negative samples highlights the importance of incorporating preference information into DMs to better capture the underlying user preference distributions. Furthermore, replacing MSE with cosine error results in performance improvements, as the recommendation phase is conducted through maximum inner product search, which better aligns with the objective of capturing similarity in the embedding space.
00301
0.0397
RECALL
000355
NDC06S
0.0215
Converganoc
0.052
PreferDiff
0.0124
PreferDiff
0.0366
DreamRee
0.0033
Dreamitee
80
Trining Epach
Training Epoch
Figure 2: Training Comparison with DreamRec on Amazon Beauty.
Faster Convergence than DreamRec. As analyzed in Section 3.2 PreferDiff handles hard negatives with higher gradient weight, as shown in Figure/4.1 Empirically, we find that PreferDiff converges faster (approximately 35 epochs, 8 minutes) than other DM-based sequential recommenders, such as
8

---

Published as a conference paper at ICLR 2025
Table 3: Performance comparison of General Sequential Recommendation Different Target Datasets. Details are the same as Table
In Domains
Out Domains
Other Platform
Supervision
Models
Metries
Instruments
Tools
CDs
Movics
Steam
k@5
0.1060
0.0874
Yull-Supervised
SASRee
N@5
0.0951
0.0642
0.0542
0.1210
0.0720
R@5
1067
0.0627
0.0286
0.0397
UniSRec
N@5
0. 1009
0.0605
0.0239
0.0271
0.0329
Zero-Shot
MoRec
N@s
R@5
0.1094
0.1220
0.0655
0.0699
0.0268
0.0274
0.0293
0.0306
0.0556
0.0585
R@5
0.1213
0.0723
0.0295
0.0312
0.0621
PreferDiff-T
N@5
0,1135
0,0691
0,0293
0.0299
0,0583
DreamRec (approximately 65 epochs, 15 minutes) with better performance on validation sets. Notably, compare the training time and inference time with 2-D scatter plot and table in Appendix [F4 By adjusting the denoising steps, we can achieve a trade-off between inference time and recommendation performance for real-time recommendation scenarios, as detailed in Appendix E5
4.2 GENERAL SEQUENTIAL. RECOMMENDATION RECIEENNNAARRROOMEEDEAAAAAAA (RQ2)
Given that DMs have exhibited exceptional zero-shot inference capabilities after pretraining large, high-quality datasets other fields (Khachatryan et al. 2023] Clark & Jaini] 2023), we aim to explore how PreferDiff can effectively zero-shot recommendation on unseen datasets, either within the same platform (e.g.. Amazon) or across different platforms (e.g.. Steam), without any overlap of users or
items (Ding al. 2021 Hou al. 2023 Li et al., 2023a Sheng et al. 2024). which distinguishes from traditional ID based cross-domall recommendation (Zhu et al. [001 Ma et al. (2024b).
Baselines.
Here, we compare PreferDiff with two baselines that are towards general sequential recom- mendations, namely UniSRec (Hou et al. 2022a) and MoRec (Yuan et al. (023). See Appendix D.5 for details on the introduction, selection, and hyperparameter search range of the baselines. For fair comparison, we emplay the text -eweedddenn--------eeee
model from OpenAl (Neelakantan ct al., 2022) as the text encoder to convert identical item descriptions (e.g., title, category, brand) into representations, as it has been proven to deliver commendable performance in recommendation (Harte ct al. 2023). More additional experiments about different text encoders can be found in Appendix E.3
Datasets and Evaluation Metries. Following the previous work (Hou et al. 2022a Li et al. 2023a we select five different product reviews from Amazon 2018 (Ni et al. 2019). namely, "Automotive" "Cell Phones and Accessories", "Grocery and Gourmet Food", "Musical Instruments" and "Tools and Home Improvement", as pretraining datasets. "Office Products" selected as the validation dataset for early stopping when Recall @5 (i.e., R R@5) shows no improvement for 20 consecutive epochs. Here, we consider three scenarios for the incoming evaluated target datasets. (1) "In Domains" refers target datasets that part of the pretraining dataset. (2) "Out Domains" refers to target datasets that are not in the pretraining dataset but belong to the same platform (i.e., Amazon). Here, we select "CDs and Vinyl" and "Movies and TV". (3) "Other Platform" refers target datasets that are neither in the pretraining dataset nor from the same platform. Here, we select a commonly used game dataset collected from Steam (Kang& & McAuley, 2018). Detailed dataset statisties can be found in Table
Results. Tables present the performance of PreferDiff compared with the chosen two general sequential recommenders. We can observe that:
Without any additional components, PreferDift-T outperforms other general sequential recommenders. Unlike UniSRec, which employs a mixture of experts technique for whitening, and MoRec, which uses dimension transformation, PreferDiff-T directly utilizes raw semantic text embeddings. This results in improvements of 2% to 8% in in-domain scenarios, 2% to 10% in out-domain scenarios, and 3% 6% on other platforms, validating PreferDiff's s strong capability in general sequential recommendation tasks without harming the performance on pretraining datasets.
The general sequential recommendation capacity of PreferDiff-T increases significantly as the amount of training data grows. As shown in Figure g we empirically find that as we continuously expand the scale of the training data (by adding more diverse datasets), NDCG@5 and HR @

---

Published as a conference paper at ICLR 2025
have nearly improved 500% the scale of the training data increased five times, approaching the performance of full-supervised SASRec. This suggests that PreferDiff-1 T can effectively learn general knowledge to model user preference distributions by pretraining even diverse datasets and transferring this knowledge to unseen datasets via advanced textual representations.
4.3 STUDY OF PREFERDIFF (RQ3)
In this subsection, we study the important factors (e.g., X, embedding size, and S()) that may impact the recommendation performance of PreferDiff. Others can found Appendix [..T and Appendix E.2
We also provide visualization of learned item embeddings via t-SNE in Appendix E.4
0.0365
0.0134
apeogil
0.0305
0.0093
0.0206
0.0220
Brankie
SASRce
0.0051
0.0126
0.0136
0.0030
0.0046
256
112 1004
1108RSIS-000 ISTH 1012-096
512 1ORR 1536-IHA T0124094
256
1038 1155-104 134 10124905
Enbleckingg Sixe
Enbleddiu Sixe
Emieedin Six
(a) Sports
(b) Beauty
(c) Toys
Figure 3: Effect of the Embedding Size for PreferDiff
Dimension of Embedding for PreferDiff. As shown in Figure we empirically observe that the recommendation performance of both PreferDiff and DreamRec improves significantly as the embedding size increases. This finding contrasts with previous observations in some non-DM-based recommenders (Liu et al.) 2020 Qu et al. 2023 Guo et al.) (024). We attribute this phenomenon to the dynamic feature space of ID embeddings. in which DMs require higher dimensions to capture the user preference and ensure the stability embedding space. Notably, in the Appendix [..3 we provide simple thcoretical analysis and experimental validation to explain this phenomenon.
Importance of for PreferDiff À controls the balance between learning generation and learning preference in PreferDiff. As shown in Figure of Appendix EE] PreferDiff performs best when = 0.4 or À = 0.6, highlighting the importance of enabling DMs to understand negatives in the recommendation task.
Measure Function for PreferDifT. As the final recommendation is ranked by maximal inner prod- uet search, we replace MSE with cosine error, as introduced in equation 7 The results presented in Table demonstratee the superiority of using set co- sine error as the default measurement function over MSE in PreferDiff.
Table 4: Effect of Measure Function for Prefer- Diff.
Datasets
Sports
Beauty
Toys
NG.
0.0139
0,1233
0.0022 0.0323
0.0185'
0,0147
IU4473
0.8367
5 CONCLUSIONS AND LIMITATIONS
We propose PreferDiff, an optimization objective specifically designed for DM-based recommenders which can integrate multiple negative samples into DMs via generative modeling paradigm. Opti- mization achieved through variational inference, deriving a variational upper bound as a surrogate objective. However, PreferDiff has limitations: (1) Dimension Sensitivity: The recommendation performance of PreferDiff is highly dependent on the embedding dimension. Empirical results show sharp decline in performance when the embedding size is reduced to 64, common dimension in existing studies. This dependency may lead to increased computational resources and slower training times when larger embedding sizes are required. (2) Hyperparameter À Dependence: PreferDiff heavily relies on the hyperparameter À to balance the generation and preference learning in DMs.
Ethic Statement. This paper aims to develop specially tailored objective for DM-based recom- menders through generative modeling. We do not anticipate any negative social impacts or violations of the ICLR code of ethics.
10

---

Published as a conference paper at ICLR 2025
Reproducibility Statement. All results in this work are fully reproducible. The hyperparameter search space is discussed in Table [1 and further details about the hardware and software environment provided in Appendix|D22 D.2 We provide the code and the best hyperparameters for our method at https://gá thub. com/ www.ss/wwwfheeeeeeeeee ff and Table T2
ACKNOWLEDGMENTS
This research is supported by the NExT Research Centre and the Spring B-Class Visiting Program of East China Normal University. We sincerely appreciate the reviewers and the AC for their valuable suggestions during the review process. We would also like to thank Xinyue Ma for her support throughout the completion of this paper.
REFERENCES
Keqin Bao, Jizhi Zhang, Yang Zhang. Wenjie Wang, Fuli Feng, and Xiangnan He. Tallrec: An effective and efficient tuning framework to align large language model with recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems, pp. 1007-1014, Singapore, 2023.
Anthony Bell and Terrence J Sejnowski. The "independent components" of natural scenes are edge filters. Vision research, 37(23).3327-3338 1997.
Zdravko Botev, Joseph F Grotowski, and Dirk P Kroese. Kernel density estimation via diffusion. The Annaly of Statistics, 3865:2916-2957, 2010.
Chong Chen, Weizhi Ma, Min Zhang, Chenyang Wang, Yiqun Liu, and Shaoping Ma. Revisiting negative sampling vs. non-sampling implicit recommendation. ACM Transaction Information Systems, 41(1))12:1112225, 2023a.
Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. Bias and debias in recommender system: survey and future directions. ACM Transactions on Information Systems, 41(3):1-39, 2023b.
Jiawei Chen, Junkang Wu, Jiancan Wu, Xuezhi Cao, Sheng Zhou, and Xiangnan He. Adap-T: Adaptively modulating embedding magnitude for recommendation. In Proceedings of the ACM Web Conference 2023, pp. 1085-1096, Aux,in,TX, 2023c.
Jin Chen, Defu Lian, Yucheng Li, Baoyun Wang, Kai Zheng, and Enhong Chen. Cache-augmented inbatch importance resampling for training recommender retriever. In Advances in Neural Infor- mation Processing Systems 35, New Orleans, LA, 2022.
Yuxin Chen, Junfei Tan, An Zhang, Zhengyi Yang. Leheng Sheng, Enzhi Zhang, Xiang Wang, and Tat-Seng Chua. On softmax direct preference optimization for recommendation. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38, British Columbia, Canada, 2024.
Kevin Clark and Priyank Jaini. Text-to-image diffusion models are zero shot classifiers. In Advances in Neural Information Processing Systems 36, New Orleans, LA, 2023.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171-4186 86, Minneapolis, MN, 2019.
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat gans on image synthesis. In Advances in Neural Information Processing Systems 34, pp. 8780-8794, Virtual, 2021.
Hao Ding, Yifei Ma, Anoop Deoras, Yuyang Wang, and Hao Wang. Zero-shot recommender systems. CoRR, abs/2105 10..200.000 101.11880..8..... 2021.
11

---

Published as a conference paper at ICLR 2025
Lu Fan, Jiashu Pu, Rongsheng Zhang, and Xiao-Ming Wu. Neighborhood based hard negative mining for sequential recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.
Taipei, Taiwan, 2023.
Xinyan Fan, Zheng Liu, Jianxun Lian, Wayne Xin Zhao, Xing Xie, and Ji-Rong Wen. Lighter and better: Low-rank decomposed self-attention networks for next-item recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1733-1737, Virtual, 2021.
Hui Fang, Danning Zhang, Yiheng Shu, and Guibing Guo. Deep learning for sequential recommen- dation: Algorithms, influential factors, and evaluations. ACM Transaction on Information Systems, 39(1):10:1-10:42, 2020.
Jerome H. Friedman. On bias, variance, 0/1-loss, and the curse-of-dimenniionattyy Data Mining and Knowledge Discovery, 1(1):55-77, 1997.
Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. Recommendation as language processing (RLP): unified pretrain, personalized prompt & predict paradigm (P5). In Proceeedings of the 16th ACM Conference on Recommender Systems, pp. 299-315, Seattle, WA, 2022.
Xingzhuo Guo, Junwei Pan, Ximei Wang, Baixu Chen, Jie Jiang, and Mingsheng Long. On the embedding collapse when scaling up recommendation models. In Proceeddings of the 41st International Conference Machine Learning, Vienna, Austria, 2024.
Yongjing Hao, Tingting Zhang. Pengpeng Zhao, Yanchi Liu, Victor S. Sheng, Jiajie Xu, Guanfeng Liu, and Xiaofang Zhou. Feature-level deeper self-attention network with contrastive learning for sequential recommendation. IEEE Transactions on Knowledge and Data Engineering, 35(10): 10112-10124, 2023.
Jesse Harte, Wouter Zorgdrager, Panos Louridas, Asterios Katsifodimos, Dietmar Jannach, and Marios Fragkoulis. Leveraging large language models for sequential recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems, pp. 1096-1 102, Singapore, Singapore, 2023.
Ruining He and Julian J. McAuley. Ups and downs:
Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the 25th International Conference on World Wide Web, pp. 507-517, Montreal, Canada, 2016.
Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yong-Dong Zhang, and Meng Wang. Lightgen: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pp. Virtual, 2020.
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-bass recom- mendations with recurrent neural networks. In Proceedings of the 4th International Conference on Learning Representations, San Juan, Puerto Rico, 2016.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. CoRR, abs/2207.12595 2022. doi: 10.4550/ARXXIV.227.15999 125998 URL 12598
48550/84550000 wttp://////......... 2207
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems 33, Virtual, 2020.
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang. Ruiqi Gao. Alexey A. Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Fleet, and Tim Salimans. Imagen video: High definition video generation with diffusion models. CoRR, abs/2210.02300 2022a.
Jonathan Ho. Tim Salimans, Alexey A. Gritsenko, William Chan, Mohammad Norouzi, and David J. Flect. Video diffusion models. In Advances in Neural Information Processing Systems 35, New Orleans, LA, 2022b.
12

---

Published as a conference paper at ICLR 2025
Yu Hou, Jin-Duk Park, and Won- Yong Shin. Collaborative filtering based on diffusion models: Unveiling the potential of high-order connectivity. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1360-1369, Washington, DC, 2024a.
Won-Yong
Yupeng Hou, Shanlei Mu, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji- Ri-Rong Wen. Towards universal sequence representation learning for recommender systems. In Proceedings of the 28th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 585-593, Washington, DC, 2022a.
Yupeng Hou, Zhankui He, Julian McAuley, and Wayne Xin Zhao. Learning vector-quantized item representation for transferable sequential recommenders. In Proceedings of the ACM Web Conference 2023, pp. 1162-1171, Austin, TX, 2023.
Yupeng Hou, Jiacheng Li, Zhankui He, An Yan, Xiusi Chen, and Julian J. McAuley. Bridging language and items for retrieval and recommendation. CoRR, abs/2403.03952, 2024b.
Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjje Wang, and Jie Tang. Graphmae: Self-supervised masked graph autoencoders. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 594-604, Washington, DC, 2022b.
Yitong Ji, Aixin Sun, Jie Zhang, and Chenliang Li. A critical study data leakage in recommender system offline evaluation. ACM Transaction on Information. System, 11())77:11775:27, 2023.
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b. CoRR, abs/2310.06825. 2023.
Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535-547) 2019.
Wang- Blag--Cheng Kang and Julian J. McAuley. Self-attentive sequential recommendation. In Proceedings of the 18th IEEE International Conference on Data Mining, pp. 197-206, Singapore, 2018.
Levon Khachatryan, Andranik Movsisyan, Vahram Tadevosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text-to-image diffusion models are zero-shot video generators. In IEEE/CVF International Conference Computer Vision, pp. 15908-15918, Paris, France, 2023. IEEE.
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In Proceedings of the 2nd International Conference on Learning Representations, Alberta, Canada, 2014.
Anton Klenitskiy and Alexey Vasilev. Turning dross into gold loss: is bert4rec really better than sasrec? In Proceedings of the 17th ACM Conference on Recommender Systems, pp. 1120-1125, Singapore, 2023.
Jiacheng Li, Ming Wang, Jin Li, Jinmiao Fu, Xin Shen, Jingbo Shang, and Julian McAuley. Text is all you need: Learning language representations for sequential recommendation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1258-1267, Long Beach, CA, 2023a.
Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori B. Hashimoto. Diffusion- Im improves controllable text generation. In Advances Neural Information Processing Systems 35, New Orleans, LAA, 2022.
Xinhang Li, Chong Chen, Xiangyu Zhao, Yong Zhang, and Chunxiao Xing. E4srec: An elegant effective efficient extensible solution of large language models for sequential recommendation. CoRR, abs/2312.02443,, 2023b.
Zihao Li, Aixin Sun, and Chenliang Li. Diffuree: diffusion model for sequential recommendation. ACM Transaction Information System, 42(3):66: 16:6::666650 166:28 2024.
13

---

Published as a conference paper at ICLR 2025
Jiayi Liao, Sihang Li, Zhengyi Yang, Jiancan Wu, Yancheng Yuan, Xiang Wang, and Xiangnan He. Llara: Large language- rew.geerrcoominttioon assistant. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1785-1795. Washington, DC, 2024.
Jianghao Lin, Jiaqi Liu, Jiachen Zhu, Yunjia Xi, Chengkai Liu, Yangtian Zhang, Yong Yu, and Weinan Zhang, A survey on diffusion models for recommender systems. arXiv preprint XXiv:2409.050333 2024.
Guangyi Liu, Zeyu Feng, Yuan Gao, Zichao Yang, Xiaodan Liang, Junwei Bao, Xiaodong He, Shuguang Cui, Zhen Li, and Zhiting Hu. Composable text controls in latent space with odes. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. (6543-16570) Singapore, 2023.
Haochen Liu, Xiangyu Zhao, Chong Wang, Xiaobing Liu, and Jiliang Tang. Automated embedding size search in deep recommender systems. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pp. 2307-2316, Virtual, 2020.
Shuo Liu, Junhao Shen, Hong Qian, and Aimin Zhou. Inductive cognitive diagnosis for fast student learning web-based intelligent education systems. Proceedings of the 2024 ACM on Web Conference, pp. 4260-4277 Singapore, 2024a.
Shuo Liu, Zihan Zhou, Yuanhao Liu, Jing Zhang, and Hong Qian. Language representation favored zero-shot cross-domain cognitive diagnosis. In Proceedings of the 31 ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Toronto, Canada, 2025a.
Xiaohao Liu, Jie Wu, Zhulin Tao, Yunshan Ma, Yinwei Wei, and Tat-seng Chua. Fine-tuning multimodal large language models for product bundling. 2025b.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zeutlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.1 1692, 2019.
Yuxi Liu, Lianghao Xia, and Chao Huang. Selfgnn: Self-supervised graph neural networks for sequential recommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1609-1618, Washington, DC, 2024b.
Justin Lovelace, Varsha Kishore, Chao Wan, Eliot Shekhtman, and Kilian Q. Weinberger. Latent diffusion for language generation. In Advances in Neural Information Processing Systems 36, New Orleans, LA, 2023.
Calvin Luo. Understanding diffusion models: A unified perspective. CoRR, abs/2208.1 1970, 2022.
Haokai Ma, Ruobing Xie, Lei Meng. Xin Chen, Xu Zhang, Leyu Lin, and Zhanhui Kang. Plug-in diffusion model for sequential recommendation. In Proceedings of the 38th AAAI Conference on Artificial Intelligence, pp. 9886-8894, Vancouver, Canada, 2024a.
Haokai Ma, Ruobing Xie, Lei Meng, Xin Chen, Xu Zhang, Leyu Lin, and Jie Zhou. Triple sequence learning for cross-domain recommendation. ACM Transaction on Information System, 42(4): 91:1-91:29, 2024b.
Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple optliiiiaiiiioooiiiiiiiiiiiiiioo with reference- free reward. CoRR, 0is/2405.147344 2024. URL http::://aoi.orr///11/ 2tt55//0..8555///////.
2405.
Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, Johannes Heidecke, Pranav Shyam, Boris Power, Tyna Eloundou Nekoul, Girish Sastry, Gretchen Krueger, David Schnurr, Felipe Petroski Such. Kenny Hsu. Madeleine Thompson, Tabarak Khan, Toki Sherbakov, Joanne Jang, Peter Welinder, and Lilian Weng. Text and code embeddings by contrastive pre-training. CoRR, Abs/2201.10000 2022.
14

---

Published as a conference paper at ICLR 2025
Jianmo Ni, Jiacheng Li, and Julian J. McAuley. Justifying recommendations using distantly reviews and fine-graing mee-grained aspects. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, pp. 188-197, Hong Kong, China, 2019.
Jianmo Ni, Gustavo Hernández Ábrego, Noah Constant, Ji Ma, Keith B. Hall, Daniel Cer, and Yinfei Yang. Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. In Findings of the Association for Computational Linguistics, pp. 1864-1874, Dublin, Ireland, 2022.
Hong Qian, Shuo Liu, Mingjia Li, Bingdong Li, Zhi Liu, and Aimin Zhou. ORCDF: An oversmoothing--rsistann cognitive diagnosis framework for student learning online education systems. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Barcelona, Spain, 2024.
Yunke Qu, Tong Chen, Xiangyu Zhao, Lizhen Cui, Kai Zheng, and Hongzhi Yin. Continuous input embedding size search for recommender systems. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 708-717, Taipei, Taiwan, 2023.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. In Advances in Neural Information Processing Systems 36, New Orlcans,LA, 2023.
Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q. Tran, Jonah Samost, Maciej Kula, Ed H. Chi, and Mahesh Sathiamoorthy. Recommender systems with generative retrieval. In Advances in Neural Information Processing Systems 36, New Orleans, LA, 2023.
Xubin Ren, Wei Wei, Lianghao Xia, Lixin Su, Suqi Cheng, Junfeng Wang. Dawei Yin, and Chao Huang. Representation learning with large language models for recommendation. In Proceedings of the ACM on Web Conference 2024, pp. 3464-3475 Singapore, 2024a.
Xubin Ren, Lianghao Xia, Yuhao Yang, Wei Wei, Tianle Wang, Xuheng Cai, and Chao Huang. Ssirec: A self-supervised learning framework for recommendation. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pp. 567-575, Merida, Mexico, 2024b.
Steffen Rendle. Item recommendation from implicit feedback. In Recommender Systems Handbook, pp. 143- 171. 2022.
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. BPR: bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, pp. 452 461, Quebec, Canada, 2009.
Masahiro Sato, Sho Takemori, Janmajay Singh, and Tomoko Ohkuma. Unbiased learning for the causal effect of recommendation. In Proceedings of the 14th ACM Conference Recommender Systems., pp. 378-387, Virtual, 2020.
Leheng Sheng, An Zhang, Yi Zhang, Yuxin Chen, Xiang Wang, and Tat-Seng Chua. Language models encode collaborative signals in recommendation. arXiv preprint (4t::2207000441,, 2024,
Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning, volume 37, pp. 2256-2265, Lille, France, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Proceed- ings of the 9th International Conference on Learning Representations, Virtual, 2021a.
Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. In Advances in Neural Information Processing Systems 33, Virtual, 2020.
Yang Song. Jascha Sohl-Dicksted Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Proceedings of the 9th International Conference on Learning Representations, Virtual, 2021b.
15

---

Published as a conference paper at ICLR 2025
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1441-1450, Beijing, China, 2019.
Jiaxi Tang and Ke Wang. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the 11 1ith ACM International Conference on Web Search and Data Mining, pp. 565-573, Marina Del Rey, CA, 2018.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Feereer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller. Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliaan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurélien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models. CoRR, abs/2307.0928s 2023.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 911), 2008.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems 30, pp. 5998-6008, Long Beach, CA, 2017.
Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8228-8238, Seattle, WA, 2024.
Chenyang Wang, Weizhi Ma, Chong Chen, Min Zhang, Yiqun Liu, and Shaoping Ma. Sequential recommendation with multiple contrast signals. ACM Transaction on Information System, 41(1): 11:1-11:27, 2023a.
Shoujin Wang, Liang Hu, Yan Wang, Longbing Cao, Quan Z. Sheng, and Mehmet A. Orgun. Sequential recommender systems: Challenges, progress and prospects. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp.
Macao, China, 2019.
Wenjie Wang. Yiyan Xu, Fuli Feng, Xinyu Lin, Xiangnan He, and Tat-Seng Chua. Diffusion recommender model. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 832-841, Taipei, Taiwan, 2023b.
Ye Wang. Jiahao Xun, Minjie Hong, Jieming Zhu, Tao Jin, Wang Lin, Haoyuan Li, Linjun Li, Yan Xia, Zhou Zhao, and Zhenhua Dong. EAGER: two-stream generative recommender with behavior- semantic collaboration. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3245-3254, Barcelona, Spain, 2024a.
Yu Wang, Zhiwei Liu, Liangwei Yang, and Philip S. Yu. Conditional denoising diffuusin for sequential recommendation. In Proceedings of the 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, Taipei, Taiwan, 2024b.
Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Jiandong Zhang, Bolin Ding, and Bin Cui. Contrastive learning for sequential recommendation. In Proceedings of the 38th IEEE International Conference on Data Engineering, pp. 1259-1273, Kuala Lumpur, Malaysia, 2022. IEEE.
Ling Yang. Zhilong Zhang, Yang Song. Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diffusion models: A comprehensive survey of methods and applications. ACM Computing Survey, 56(4): 1-05:99, 2024.
16

---

Published as a conference paper at ICLR 2025
Zhengyi Yang, Xiangnan He, Jizhi Zhang. Jiancan Wu, Xin Xin, Jiawei Chen, and Xiang Wang. A generic learning framework for sequential recommendation with distribution shifts. In Proceedings of the 46th International ACM SIGIR Conference Research and Development in Information Retrieval, pp. 331-340, Taipei, Taiwan, 2023a.
Zhengyi Yang. Jiancan Wu, Zhicai Wang, Xiang Wang, Yancheng Yuan, and Xiangnan He. Generate what you prefer: Reshaping sequential recommendation via guided diffusion. In Advances in Neural Information Processing Systems 36, New Orleans, LA, 2023b.
Zheng Yuan, Fajie Yuan, Yu Song, Youhua Li, Junchen Fu, Fei Yang. Yunzhu Pan, and Yongxin Ni. Where to go next for recommender systems? ID- vs. modality-based recommender models revisited. In Proceedings of the 46th International ACM SIGIR Conference Research and Development in Information Retrieval, pp. 2649, Taipei, Taiwan, 2023.
Jiaqi Zhai, Zhaojie Gong, Yueming Wang, Xiao Sun, Zheng Yan, Fu Li, and Xing Liu. Revisiting neu- ral retrieval on accelerator.. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Long Beach, CA, 2023.
Discovery and Data Mining, pp. 5520-55205555 5531,
Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Jiayuan He, Yinghai Lu, and Yu Shi. Actions speak louder than words: Trillion-parmetter sequential transducers for generative recommendations. In Proceedings of the 41st International Conference on Machine Learning, Vienna, Austria, 2024.
An Zhang. Leheng Sheng, Zhibo Cai, Xiang Wang, and Tat-Seng Chua. Empowering collaborative filtering with principled adversarial contrastive loss. In Advances in Neural Information Processing Systems 36, New Orleans, LA, 2023.
An Zhang, Wenchang Ma, Jingnan Zheng, Xiang Wang, and Tat-Seng Chua. Robust collaborative filtering to popularity distribution shift. ACM Trans. Inf. Syst., 2024.
Jujia Zhao, Wenjie Wang, Yiyan Xu, Teng Sun, Fuli Feng, and Tat-Seng Chua. Denoising diffusion recommender model. In Proceedings of the 47th International ACM SIGIR Conference Research and Development in Information Retrieval, pp. 1370-1379, Washington, DC, 2024.
Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management, pp. 1893-1902, Virtual, 2020.
Feng Zhu, Yan Wang, Chaochao Chen, Jun Zhou, Longfei Li, and Guanfeng Liu. Cross-domain recommendation: Challenges, progress, and prospects. In Proceedings of the 30th International Joint Conference on Artificial Intelligence, pp. 4721-4728, Virtual, 2021.
Yunqin Zhu Chao Wang, Qi Zhang, and Hui Xiong. Graph signal diffusion model for collaborative filtering- In Proceedings of the 47th International ACM SIGIR Conference Research and Development in Information Retrieval, SIGIR 2024, Washington DC, USA, July 14-18, 2024, pp. 1380-1390, Washington, DC, 2024.
17

---

Published as a conference paper at ICLR 2025
A
RELATED WORK
We highlight key related works to contextualize how PreferDiff fits within and contributes to the broader literature. Specifically, work aligns with research on sequential recommendation and DMs based recommenders.
Sequential Recommendation have gained significant attention in both academia (Rendle 2022 Liu
et al. 2024b) and industry (Wang et al. 2019 Fang et al. 2020) due to their ability to capture user preferences from historical interactions and recommend the next item. One common research line
has focused on developing more efficient network architectures, such as GRU (Hidasi et al. 2016)).
convolutional neural networks (Tang & Wang 2018). Transformer (Kang & McAuley 2018 Fan
ct al., [02T), Bert4Rec (Devlin et al. 2019)) and HSTU (Zhai et al. 2024). Another research line
focuses on leveraging additional unsupervised signals (Xie al. 2022 Wang et 2023a Ren et
2024a) or reshaping sequential recommendation into other tasks such as retrieval (Rajput et al. 2023
Wang et al. 2024a) and language generation (Bao al. 2023 Li et al. 2023b Liao al. 2024).
DM-based Recommenders have been explored in recent studies due to the powerful generative and generalization capabilities of DMs (DMs) (L.in al. (024). These recommenders either focus on modeling the distribution of the next item (c.g.. (Yang et al. 2023b Wang et al. 2024bb Li
cct al. 2024)), capture the probability distribution of user interactions (c.g., (Wang et al. 2023b
Zhao et al. )) ), focus on the distribution of time intervals between user behaviors (e.g., (Ma
et al. 2024a)). However, existing approaches often rely on conventional objectives, such as mean squared error (MSE), standard recommendatiio-sspciff objectives like Bayesian Personalized
Ranking (BPR) (Rendle et al. 2009) and Cross Entropy (CE) (Klenitskiy & Vasilev 2023). We argue that the former may diverge from the core objective of accurately modeling user preference distributions in recommendation tasks (Rendle 2022). as DMs often lack an adequate understanding of negative items. While the latter leverages DMs noise resistance mitigate noisy interactions in recommendations which might fall short of fully exploiting the generative and generalization capabilities of DMs.
B SAMPLING ALGORITHM IN PREFERDIFF
We utilize DDIM (ong al. 202 as the default sampler in PreferDiff, replacing the DDPM used in DreamRec, as we empiricall find that DDIM is faster and performs better, requiring only few denoising steps. Here, we briefly introduce how DDIM is employed in PreferDiff; Detailed derivations can be found in (Song et al. 2021a), and the code implementation is available at https://githubs com/ 1swhim/Prefe:Diff
Details. Specifically, in PreferDiff, the training is to predict the original data ep. The sampling process should be reparamcterized predict e directly instead of the noise e. Starting from the original DDIM update equation (Song et al. 2021a):
et-1 Va-1
Ct
v1 at co(o,,t)
+ V 0:-1 of Ep (e,))))) + o:2,
(13)
Vot
where 'N( I), or controls the stochasticity of the process, and ca(e,, is the predicted noise at time step t. In PreferDiff, since our model trained to predict the original data eo directly, we use the relationship between C; and the noise e:
= V eu + VI- a
(14)
Solving for e, we obtain:
E C; / at Cu
(15)
V1 at
Since en is predicted by our model è Fo(es, C, we can estimate the noise
18

---

Published as a conference paper at ICLR 2025
C₁ yar e0 v1
Eg
(16)
Substituting éo back into the DDIM update equation and setting 0 for deterministic sampling, we get:
e: VI Eg
e va--1
+ /1 a.-1
(17)
Va:
= = Co + / a4-1 C0.
(18)
This simplification allows us to update e;-1 directly using the predicted é, and éD without introducing additional randomness, thus making the sampling process deterministic and more efficient.
Summary. Therefore, the deterministic DDIM sampling steps in our inference algorithm
1. Predict éø = Fo(e,, C, t).
/at en vI - at
2. Compute éØ
3. Update e,-1 Vat-1 én + /I 04-1 Eg.
By iteratively applying these steps, we can efficiently generate the predicted original data ég During inference, by setting o; O, we climinate the noise term o,z and focus solely on the deterministic components of the update rule. This results in faster convergence with fewer denoising steps while maintaining high-quality predictions. Detailed derivations and explanations of this reparameterization and the DDIM sampling process can be found in (Song et al. 2021a).
C DETAILS ABOUT PREFERDIFF
C.I FROM RATINGS TO PROBABILITY DISTRIBUTION
LBpr -B (..o) .e) [log a (fo(et cc) - fele [c)]]
(19)
The primary objective of equation 19 is to maximize the rating margin between positive items and sampled negative items. Here, we employ softmax normalization to transform the rating ranking into a log- likelioo ranking.
We begin by expressing the rating fe(eo c) terms of the probability distribution po(ep relationship is established through the following set of equations:
c). This
pa(en c) exp(fe(eo c))
Zo
log po
c) = fe(co |c) log Zn,
fo(en c) log log (eoo c) log Zo.
(20)
Substituting equation 20 into equation 19 yields the BPR loss expressed solely terms of the probability distributions of positive and negative items.
19

---

Published as a conference paper at ICLR 2025
CEPR-DII (et 560 .c)
log
fo ratlin of Positive Item
c)
fo
rating of Negative Item
=- .60
log
log (e c) + log Zo
log c) log Zp
(21)
equation 20
From equation 200
-- -R .5) .c)
log
log Po(e
c)
Po |c) + log Zo - llg Z00
=0
(e c)
c) )
- -0 .c)
log a
log
Po po
C.2 CONNECTING THE RATING FUNCTION TO THE SCORE FUNCTION
In this subsection, we establish the relationship between the rating function fo c) and the score function in the context of score-based DMs. Specifically, we demonstrate that the gradient of the rating function with respect to the item embedding en is equivalent to the score function Ve, log po (eo c).
Starting from Equation equation 20
fo
log c) + log Zo
(22)
where Zo is the partition function:
Zo / exp(So(e| de.
(23)
DERIVATIVE OF THE RATING FUNCTION WITH RESPECT TO eo
Taking the gradient of Equation equation 22 with respect to Co, we have:
V felp c) = Ve, log (e c) + Ve, log Zo-
(24)
Since the partition function Z is obtained by integrating over all possible item embeddings and does not depend on the specific eo, its gradient with respect to eo is zero:
V Veg log Zo =
(25)
Therefore, Equation equation 24 simplifies to:
V fo cc V eo lo c).
(26)
Definition of the Score Function In score-based DMs, the score function is defined as the gradient of the broobrbbyyyy density with respect to the data point eo:
se c)
Ve, log pa(eo c).
(27)
Comparing Equations equation 26 and equation 27 we find that:
Solo
c) So (eo, c).
(28)
This reveals that the gradient of the rating function with respect to the item embedding en is exactly the score function of the probability distribution Po(c, c). Score- DMs Song et al. (2021b) utilize the score function So (eu. to define the reverse diffusion process. In these models, the data generation process involves integrating the score function over time to recover the data distribution from noise. Intuitively, we utilize Ve, fo(eo c) to sample item embeddings with high ratings
20

---

Published as a conference paper at ICLR 2025
through Langevin dynamics (Song & Ermon bridges the objective of recommendation with generative modeling in DMs.
2020) given certain user historical conditions. Therefore,
Connection to Our Loss Function. Our BPRf loss function, expressed in Equation equa- tion 21 involves the log-ratio the probabilities of positive and negative items:
CBPR Dia (o .00 .c) logo
loo
Pa (
)
())
(29)
po
Using the equivalence between the rating function and the log-probability (from Equation equation 22 the loss function can also be seen function of the rating differences:
LBPr Dur -E [og (Jo(e)
c) fo c))]
(30)
Gradient of the Loss with Respect to eg. Taking the gradient of the loss function with respect to the positive item embedding ej. we get:
LIRR-DII -E (((s)) Sole, |c)
(31)
where s = fe(e
c) - fole c).
Similarly, for the negative item embedding c
V LEPR-DIY = E [o(-s) Vo, fe 10)
(32)
These gradients indicate that the loss function encourages:
Increasing the rating fo(et c) of the positive item by moving ej in the direction of
Vej fo.
Decreasing the rating fo c) of the negative item by moving co opposite to V e, fo-
C.3 DERIVATION THE VARIATIONAL UPPER BOUND
In this section, we provide comprehensive derivation of the upper bound for the proposed LAPR-Dif- We focus particularly on the steps involving the Kullback-Leeblerr divergence, leading to the final loss function used for training.
Assumptions and Definitions:
c and eg represent the embeddings of the positive and negative items, respectively.
e and e, are the noisy embeddings timestep for the positive and negative items, obtained via the forward diffusion process.
denotes the historical item sequence for a user.
g( e en) is the posterior distribution in the forward diffusion process. po e; c) is the reverse diffusion process modeled by neural network Fg.
M(c) is mapping function that encodes the historical context into suitable representa-
tion for conditioning.
(.)) is the sigmoid function.
Br at and ãe are predefined constants in the diffusion schedule.
Starting from equation 4 in the main text, we have:
CBPR-Dit(() -E vo .)) logo (log Bogg
Po f |c) q(et. e
po po (c)
log E (yce
a e
(33)
21

---

Published as a conference paper at ICLR 2025
To address the intractability of directly computing the expectations inside the logarithms, ap- ply Jensen's 's inequality, which states that for a convex function f. we have f[E $ [[]( Recognizing that log or (2) is convex in obtain upper bound:
Po Corr (c) ag(e 1:T
po Po (o:T
CBPR-DIr(O) (Bie .60 .0) B ((e
log
log
log
%
q 6
(a)
(b)
(34)
The terms (a) and (b) represent the variational lower bounds of the log-likelihoods for the positive and negative items, respectively. According to the properties of DMs (Ho et al. 22200), these terms can be related to the evidence lower bound (ELBO). Specifically, for any item Cu, we have:
Po (Co:T q(ei:T
(0) = - C cu, c).
log Po(Co c) 2 (e(err)
log
(35)
Substituting equation 35 into equation 34
we get:
LEPR-Din () f -E E
[log a (-CELBO(O;e), c) CELBO ep c))]
(36)
The ELBO for each item can be decomposed into sum over timesteps t:
(O; en, c) = Σ (gler [DRL. (g(e.-1 ez eo) pole:-1 e ec, + C,,
(37)
where C' is constant independent of 0.
Substituting equation 37 back into equation 36 we obtain:
(- Σ E Beett let) [DKi. (9(e=tl0,t at) ([(et)lett))
CaPR-Dir(()) S -E (ox %
log a
E (00 o5) [DEL (1(07-100,,0)) po(c__1177)] +G ))
(38)
where C aggregates constants and is independent of 0.
Now, we focus on the KL divergence terms. In DMs, both q(e, e,, en) and pe(e, e c) are Gaussian distributions (Ho et al. 2020p. Specifically, for the forward process g and the reverse process pa. we have:
q(--11 c, c2, cu) = N (e: (o), on),
(39)
po e: c) = (e--1; /0 (e 1, 3),3,1)
(40)
where (e;; en) is the mean of posterior q(e:-1 e,, e0), 3e is the variance, and 3 is the variance schedule for the reverse process.
The KI. divergence between two Gaussian distributions can be computed as:
(de((3.I)
DRL. (l)) tr (5 (x)) (1r j..)) I - (..) +
(41)
det (eet(,,I)
where k is the dimensionality of the Gaussian distributions (i.c., the embedding dimension).
Assuming that 3. = 3, (Ho et al., 2020), the trace term simplifies to k; and the determinant term becomes In(1) 0. Therefore, the KL divergence simplifies to:
DKL. (q po) 2,3,
am - 10
(42)
22

---

Published as a conference paper at ICLR 2025
Next, we define the network prediction μe and relate it to the mean be from the forward process.
Relationship between fi and ey: The mean üú, is given by: (e Co) Va ey /a Ōx-1)
C2
(43)
āj
- ag
where a; = 1 BI, and ā₁ IT. ag.. practice, it is common to predict cu directly using the neural network Fe:
e = Fole,, t, M(c)).
(44)
Given ég, compute Hg as:
no mo c) = V 0/ èp
04 (1 ō4-1)
(45)
1 Oz
Substituting equations equation 43 and equation 45
into equation 42
we have:
V 0 1 (èo
l1e
(Vā.-13,) |10 coll
DKL. (q pa)
23, po ir 11 2
at
23 ā,)²
(46)
Simplifying the constants, we observe that the coefficient reduces a constant factor dependent on f, which we can denote as Ax:
d (3x)-133))
(47)
28 -
2(1 σ4
Therefore, the KL divergence becomes:
DxL (g)l de |lē0 eg
(48)
Since Å independent of 0 and depends only on 4, when we sum over all timesteps and average over 1, this term becomes proportional to the mean squared error between 6g and eg-
Equivalence of MSE and Cosine Error for Unit Norm Vectors:
Alternatively, mitigate sensitivity to vector norms and dimensionality (Friedman, 1997: Hou et al.) 2022b) (the recommendation performance of PreferDiff is competitive when embedding size is higher),
the cosine error the distance measure. The cosine similarity between ég and eo is given by:
è eo
cos (êp, eo)
(49)
| 2 ee
The cosine error is then:
S (év, ep) = =1 cos (êo, cu)
(50)
Actually, when both èg and ey are normalized have unit norm (i.e., |@ul12 ||cull2 1), the mean squared error and the cosine enror are directly related. Specifically, the squared Euclidean distance between two unit vectors is:
e eo = (éo eo) (én eg) || 3 ||eol 26 ep 2(1 cos (Co, en)) (51)
Thus, under the unit norm constraint, minimizing the MSE equivalent to minimizing the cosine eтror up to a constant factor of 2. This shows that both distance measures capture the same notion of similarity this case. Substituting the KL divergence approximation back into equation 38 and considering both positive and negative items, we simplify the expression:
(-
LBPR CarR.Din(0 (i) < -B oo .o), (UU(T)
log S s
S ( Nogative
(52)
Positive
23

---

Published as a conference paper at ICLR 2025
where 6 = Fo(e"" M.M(c)) and 6, ,t, M(c)).
Equation equation 52
represents our final trainable objective:
Lupprr (0) (e % o), (UUT)) [log (- (S
+t) S (Folez M(c))
(53)
Explanation. This objective encourages the model minimize the distance between the predicted embedding and the true embedding for the positive item while maximizing the distance for the negative item, effectively widening the gap between them in the latent space. By doing so, enhance the personalized ranking capability of the model.
Summary. By minimizing Cppper (0), we implicitly minimize the original CBPR-Din(0)) due the application of Jensen's s inequality. This aligns the training objective with the goal of improving personalized ranking by leveraging DMs within the BPR.
C.4 EXTEND MULTIPLE NEGATIVE SAMPLES
In this section, we provide detailed derivation of the inequality CBpr Dir S Ci under the assumption that Fa and S are convex functions.
Definitions and Assumptions
We define:
Fo t, M(c)): the denoising function at time step t, parameterized by 8, conditioned on context M(c).
Sa,, b): measure function quantifying the discrepancy between vectors and b, such as Mean Squared Error (MSE).
(.): the sigmoid function.
Assume that:
Fg is convex with respect to its input e,.
S is convex with respect both of its inputs.
Starting with the definition of LarR-DaI-V:
LBPR Din logg (-v (-v(s (V(s( (ot ((C)))))
1 Σ
s (Fo (.))
(54)
Similarly, for LaPR-Dincc
LAPR.DIN.C - log
(e.Mc,,M ..tt) - SSS(F (.c() ,,))), (55)
where we have defined the centroids:
1
6
(56)
Our aim is to show that LAPR-DIEEV S LAPR-DIfF-C. First, consider the term:
1 Σ
Dv SFo (ct (,M()
SFo (t,,,,
(57)
24

---

Published as a conference paper at ICLR 2025
By the convexity of S, we have:
y Σ M,,cc) sesss s
V Σ Fo (er tt,
V
1
1
Σ
(58)
V
1=1
v=1
V=l
Convex combination of Fole;
60
Next, using the convexity of Fa. we have:
V Fo (e7,,,
Fo (e, S
(59)
Convex combination
Combining equation 58 and equation 59 and recognizing that S is non-decreusing with respect to its first argument, we get:
Σ ( .,5)) S (ē, M(c)) .5)
(60)
Therefore, we have:
.t) Dy S (L ..)) V Σ (Fo M(cc)) ,00))
(61)
28 STo (ct t, M(c)) ot) s (Fo t. 05) = -Do.
(62)
Since Dv 2 De, it follows that:
-VDv S -VDc-
(63)
Applying the monotonicity of the log ar.) function (since a is an increasing function and log is monotonic), we have:
CEPR-DIEN - log ooV S log (-VDcc CAPR-DIEC..
(64)
Therefore, we have shown that:
LEPR-DIIVV & LBPR-DIN-C.
(65)
Explanation. This inequality implies that minimizing LBrR Dir- effectively minimizes upper bound of LEPR-DIRVV. leading to an efficient increase in the likelihood of positive items while distancing them from the centroid of negative items. Notably, although the assumption of convexity is difficult to satisfy in practice, the aforementioned method still empirically achieves strong results than one negative item.
D EXPERIMENTS
D.1 DATASETS PREPOSSESSING IN USER SPLITTING SETTING
Following prior works Yang et al. 2023a b), we adopt the user-splitiig setting, which has been shown to effectively prevent information leakage in test sets (Ji al. 2023). Specifically, first
25

---

Published as a conference paper at ICLR 2025
Algorithm 1 Training Phase of PreferDiff
1: Input: Trainable parameters 0, training dataset Drain =
{(et c, Punl total steps T, unconditional probability Pu, learning rate y, variance schedules {ar)i=1
2: Output: Updated parameters 0
3: repeat
4: (ej H) Drain
Sample data from training dataset. Set unconditional condition with probability Pu- Sample diffusion step and noise. Add noise to positive item embedding. b Add noise to negative item embeddings' centroid. Gradient descent update.
5: With probability Pu: c
6:
t~ Uniform(1, C), , ~
7:
e vare, vI āget
8:
o = V
vāe E % v1 āic
9:
O + 0 10-nVa & ( Ct c, 0)
until convergence
11: return /
Algorithm 2 Inference Phase of PreferDiff
1: Input: Trained parameters 0, Sequence encoder M(-), test dataset Diest {(eo, } total
[Peal
steps T. DDIM steps S, guidance weight w, variance schedules {are)==
2: Output: Predicted next item éo
3: c ~ Diest
b Sample user historical sequence from testing dataaset.
4: er 10
b> Sample standard Gaussian noise.
5: for =,...,,d do
b Denoise over DDIM steps.
6:
t= [ (T/S)I
Map DDIM step to original step t. Set unconditional condition with probability Pu-
7:
With probability Pa: M(c) = +b else 0
8:
(0, if
> Sample noise if final step. Apply classifier-free guidance.
9:
ê (1 w- M(c), w:Fo auFa((e +.1)
è e--va
Compute predicted noise.
VI-ai
11:
ê 1á 11-a--180
> DDIM update step when 0.
12:
end for
13: return èo
26

---

Published as a conference paper at ICLR 2025
Table 5: Detailed Statistics of Datasets after Preprocessing.
Fully
Trained
Recommendation Steam
General Sequential
Revommendation
Sports
Benuty
Toys
MI-1M
YahooRI
Pretraining
Validation
CDs
Movies
Steam
ASequemces
55.5 35.598
22.363
19,412
39.795
6,940
50,000
746 688
101, 501
112,379
297
30.
Albems
18,357
12,101
11,924
9.265
3,705
23,589
68,668
8,623
15,520
25,925
9,265
Alnteractions
256,598
162,150
138,444
437,733
00,400
506,000
3,258,525
452,415
457,589
2,053,497
437,733
sort all sequences chronologically for each dataset, then split the data into training, validation, and test sets with an 8:1:1 ratio, while preserving the last 10 interactions as the historical sequence.
Amazon 2014 Here, we choose three public real-world benchmarks (i.e., Sports, Beauty and Toys) which has been widely utilized in recent studies (Rajput et al. 2023). Here, we utilize the common five-core datasets (Hou al., 2022a), filtering out users and items with fewer than five interactions across all datasets. Following previous work Yangg et al. 2023b), we set the maximized length user interaction sequence as 10.
Following prior works (Hou et al. 2022a Li al. 2023a), we select five distinet product review naneorries-mmelly.... "Automotive." "Flectronics," "Grocery and Gourmet Food," "Musical Instruments," and "Tools and Home Improvement" pretraining datasets. "Cell Phones and Accessories" is used as the validation set for early stopping. In line with previous research (Yang al. 2023b), we filter out items with fewer than 20 interactions and user interaction sequences shorter than 5, capping the maximum length of each user's s interaction sequence at 10.
Amazon 2018
Steam is game review dataset collected from Steam Due to the large number of game reviews, we filter out users and items with fewer than 20 interactions.
ML-1M is movie rating dataset collected by GroupLens We filter out users and items with fewer than 20 interactions.
R1 is music rating dataset collected by Yahoo We filter out users and items with fewer than 20 interactions.
D.2 IMPLEMENTATION DETAILS
For fair comparison, all experiments are conducted in PyTorch using single Tesla V100-SXM3- 32GB GPU and an Intel(R) Xeon(R) Gold 6248R CPU. We optimize all methods using the Adam optimizer and all models' parameters are initialized with Standard Normal initialization. We fix the embedding dimension to 64 for all models except DM-based recommenders, as the latter only demonstrate strong performance with higher embedding dimensions, as discussed in Section 4.3 Since our focus is not on network architecture and for fair comparison, adopt lightweight configuration for baseline models that employ a Transformer backbone two attention heads. Notably, all baselines, unless otherwise specified, cross-entropy the loss function, as recent studies (Zhagg et al. 2024 Klenitskiy Vasilev 2023 Zhai et al. 2023 have demonstrated its effectiveness.
[ using a single layer with
For PerferDiff, for each user sequence. we treat the other next-items (a.k.a,, labels) in the same batch negative samples. We set the default diffusion timestep to 2000, DDIM step 20, P.. 0.1, and the 33 linearly increase the range of [1e-1 0.02) for all DM-basd sequential recommenders (e.g., DreamRec). We empirically find that tuning these parameters may lead to better recommendation performance, However, as this is not the focus of the paper, we do not elaborate on it.
The other hyperparameter (e.g., learning rate) search space for PreferDiff and the baseline models is provided in Table []] while the best hyperparameters for PreferDiff are listed in Table T2
httear///dhhbb.co////22222///222
27

---

Published as a conference paper at ICLR 2025
D.3 BASELINES OF SEQUENTIAL RECOMMENDATION
Traditional sequential recommenders:
GRU4Rec (Hidasi et al. 2016) adopts RNNS model user behavior sequences for ression-based recommendations. Here, following the previopus work (Kang & McAAuley 2018 Yang et al. 2023b). we treat each user's interaction sequence as a session.
SASRec (Kang & McAuley 2018) adopts a directional self-attention network to model the user user behavior sequences.
Bert4Rec (Sun et al. 2019) adapts the original text-based BERT model with the cloze objective for modeling user behavior sequences. We adopt the implementation of mask from (Ren al. 2024b
Contrastive learning based sequential recommenders:
CLASRec (Xie et al. 2022) incorporates the contrastive learning with the transformer-based sequential recommendation model to obtain more robust results. We adopt the implementation from (Ren al. 2024b).
Generative sequential recommenders:
(ajeu et al., 2023) introduces codebook-basee identifiers through RQ-VAE, which quantizes semantic information into code sequences for generative recommendation. Since the source code is unavailable, we implement it using the HuggingFace and Transformers APIs, following the original paper by utilizing T5 (Ni et al. 2022) as the backbone. For quantization, we employ FAISS [Johnson et al., 2019), which is widely used in recent studies of recommendation (Hou et al. 2023
DM-based sequential recommenders:
DiffRec (Wang et al.) 2023b introduces the application of diffusion on user interaction vectors (i.e., multi-hot vectors) for collaborative recommendation, where "1" denotes a positive interaction and "0" indicates a potential negative interaction. We adopt the author's public implementation D
DreamRec (Yang et al. 2023b) uses the historical interaction sequence as conditional guiding information for the diffusion model to enable personalized recommendations and utilize MSE as the training objective. We adopt the author's public implementation
DiffuRec et al. 2024) introduces the DM to reconstruct target item embedding from a Trans- former backbone with the user's historical interaction behaviors and utilize CE as the training objective. We adopt the author's 's public implementation []
bext- sequential recommenders:
MoRec (uuan et al. 2023) utilizes item features from text descriptions or images, encoded using a text encoder or vision encoder, and applies dimensional transformation to match the appropriate dimension for recommendation. Here, utilize the OpenAI-3- embeddings, SASRec as backbone and transform the dimension to 64.
LLM2Bert4RRe (Harte et al. 2023) proposes initializing item embeddings with textual embeddings. In our implementation, we use OpenAl l-raaee embeddings, Bert4Rec as backbone and apply PCA to reduce the dimensionality to 64, mentioned in the original paper.
Noablely, the inconsistent performance of Tiger and LLM2BERT4Rec with their origin paper is actually caused by the differences in evaluation settings. Both of these papers use the Leave-one-out evaluation setting, which differs from the User-split used in our work.
Results of Other Backbone. Here, present comparison of PreferDiff with other recommenders using a different backhone, namely GRU. As shown in Table PreferDiff still outperforms DreamRec across all datasets, further validating its versatility. Empirically, we find that, unlike SASRec, which
28

---

utilized for Published as a conference paper at ICLR 2025
performs better with Transformer than with GRU4Rec, PreferDiff performs better with GRU the backbonn on the Sports and Toys datasets compared to using a Transformer. This could be due to the relatively shallow Transformer used, making GRU easier to fit. More suitable network architectures for DM-based recommenders will be explored in future work.
Table 6: Comparison of the performance with sequential recommenders with GRU as backbone. The improvement achieved by PreferDiff is significant (p-value < 0.05).
Model
Sports and Outdoors
Beauty
Toys and Games
R@5
N@5
R@10
N@10
R65
N@0
RØ 10
N@:
R@5
N@5
R@10
N@ 10
GRUARec
0,0022
0.0020
0.0030
0.0023
0.0093
0.0078
0.0102
0.0081
0.0097
0.0087
0.0100
0.0090
SASRee
0,0047
0.0036
0.0067
0.0042
0.0138
0.0090
0.0219
0.0116
0.0133
0.0097
0.0170
0.0109
DreamRec
0.0201
0.0147
0.0230
0.0165
0.0431
0.0290
0.0543
0.0321
0.0484
0.0343
0.0591
0.0382
PrfferDiff
0.0216
0.0165
0.0250
0.0176
0,0451
0,0313
0.0590
0.0358
0.0530
0,0385
0,0623
0,1415
D.4 LEAVE ONE OUT
Evaluation. The "leave-one-out" strategy is another widely adopted evaluation protocol in sequential recommendation. For each user's interaction sequence, the final item serves as the test instance, the penultimate item is reserved for validation, and the remaining preceding interactions training. During testing, the ground-truth item of each sequence is ranked against a set of candidate items, allowing for comprehensive assessment of the model's ranking capabilities. Performance is evaluated by computing mooking-basede metrics over the test set, and the final reported result is the average metric across all users in the test set.
Table 7: Detailed Statisties of Datasets after Preprocessing in Leave-One-Oe One-Out Setting.
Datasets
Sports
Beanty
Toys
Automotive
Music
Office
#Sequences
35,598
22,363
19,412
2,929
1,430
4,906
#ltems
18.357
12,101
11.924
1,863
901
2,421
#lnteractions
296.337
167,597
20,473
10,261
53,258
Avg. Length
8.32
8.87
8.63
6.99
7.17
10.86
Datasets. Except for the original three datasets (Sports, Toys and Beauty) in TIGER, we select three additional product review cergorres-nnmellle "Automotive", "Music Instrument" and "Office Product" from Amazon 2014 for a more comprehensive comparison. Here, we utilize the common five-core datasets, filtering users and items with fewer than five interactions across all datasets.
Baselines. Here, we directly report baseline results (e.g.. S -ee Zhou et al. 2020 P5 (Geng et al. 2022). FDSA (Hao al.) (023)) from TIGER (Rajput et al. 2023) and evaluate DreamRee (Yang al.
2023b) and the proposed PreferDiff
Results. Tables and Tables 9 present the performance PreferDiff compared with six categories sequential recommenders. For breivty, R stands for Recall, and N stands for NDCG. The top- performing and runner-up results are shown in bold and underlined, respectively. "Improv" represents the relative improvement percentage of PreferDiff over the best baseline. We observe that in the leave-onc-out setting, PreferDiff demonstrates competitive recommendation performance compared to the baselines. Specifically, larger datasets (i.e., Sports and Beauty), PreferDiff performs par with TIGER. However, on the Toys dataset and the three smaller datasets, PreferDiff achieves significant Thad may be due PreferDiff adopting the same manner as DreamRec, where recommendation is not included in the training process. With a smaller number of items, this approach can result in more precise recommendation performance.
D.5 GENERAL SEQUENTIAL RECOMMENDATION
Pretraining Datasets. Here, we introduce more details about Pretraining datasets. Following the previous work (Hou et al. 2022a Li et al. 2023a), we select five different product reviews from Amazon 2018 (Ni et al.) 2019), namely, "Automotive", "Cell Phones and Accessories", "Grocery and Gourmet Food", "Musical Instruments" and "Tools and Home Improvement", as pretraining datasets. "Cell Phones and Accessories" is selected the validation dataset for early stopping when Recall
29

---

Published as a conference paper at ICLR 2025
Table 8: Performance comparison sequential recommendation under leave out. The last row
depicts % improvement with PreferDiff relative to the best bascline.
Sports and Outdoors
Beauty
Toys and Games
Methods
R@5
N@s
R@
N@
R@5
N@5
R
N@10
R@5
N@5
R
N@
PS
0.0061
0.0041
0.0095
0.0052
0.0163
0.0107
0.0254
0.0136
0.0070
0.0050
0.0121
0.0066
Caser
0.0116
0.0072
0.0194
0.0097
0.0205
0.0131
0.0347
0.0176
0.0176
0.0166
0.0270
0.0141
HGN
0.0189
0.0120
0.0313
0.0159
0.0325
0.0206
0.0540
0.0257
0.0266
0.0321
0.0497
0.0277
GRU4Rec
0.0129
0.0086
0.0204
0.0111
0.0164
0.0113
0.0283
0.0137
0.0137
0.0097
0.0176
0.0084
BERT4Rec
0.0115
0.0191
0.0099
0,0263
0.0184
0.0407
0,0214
0.0170
0.0310
0.0183
FDSA
0.0182
0,0128
0.0288
0.0156
0.0261
0.0201
0.0407
0.0228
0.0228
0.0150
0.0381
0.0199
SASRec
0.0412
0.0209
0.0462
0,0605
0,0318
0.0463
0.0675
S'-Rec
0.0251
0.0161
0.0385
0.0204
0.0380
0.0244
0.0647
0.0327
0.0327
0.0294
0.0700
0.0376
DreamRoc
0.0087
0.0071
0.0096
0.0075
0.0318
0.0257
0.0624
0.0273
0.0422
0.0347
0.0689
0.0362
TIGER
0.0264
0.0181
0.0400
0.0225
0.0454
0,0648
0.0384
0.0521
0.0371
0.0712
PreferDiff
0.0275
0.0190
0,0405
0.0218
0.0455
0.0317
0.0660
0,0388
0,0660
0,0403
0,0851
0,0483
Improve
4.16%
4.97%
1.25%
-3.1%
0.22%
-1.25%
1.85%
1.0%%
15.73%
8.63%
19.52%
11.81%
Table 9: Performance comparison sequential recommendation under leave one out. The last row depicts % improvement with PreferDiff relative to the best baseline.
Methods
Automotive
Music
Office
RØ5
R610
N6
K65
NGB
R810
N6
R@5
NG5
R610
N610
Dreamkee
0.1543
0,1300
0.083
00445
0.0622
0.1444
030783
0.0467
0.0523
0.0378
0.0699
0.14
TIGER
0.0454
0.0290
0.0745
0.0383
0.0532
0.0358
0.0840
0.0456
0.0462
0.0299
0.0746
0,0390
PreferDiff
0.0649
0.0463
0.0864
0.0332
0.0650
0.0453
0.0874
0.0526
0.0538
0.0379
0.0850
0,0480
Improve
19.52%
15.75%
15.9%%
19.55%
4.50%
4.04%
12.3%
2.87%
0.26%
13.90%
10.60%
(i.e., R@5) shows no improvement for 20 consecutive epochs. The detailed statisties of each dataset
used for pretraining are shown Table Clearly, the pretraining datasets have no domain overlap with the unseen datasets used in Section 4.2
Table 10: Detailed Statistics of Pretraining Datasets.
Datasets
Automotive
Phones
Tools
Instruments
Food
#Sequences
193,651
157,212
240,799
27,530
127,496
#Items
18,703
12,839
22,854
2,494
11,778
#Interactions
806,939
544,339
1,173.154
110,151
623,940
Avg. Length 7.26
6.51
7.19
7.06
7.24
Baselines. Here, introduce more details for baselines in General Sequential Recommendation lasks. Notably, for a fair comparison, we employ the text eeeteddimbeenggg 3 3-large model (Liu 2025a) from OpenAl (Neelakantan et al. 2022) as the encoder instead of Bert (Devlin 2019) in UniSRec and MoRec to convert identical item descriptions (e.g., title, category. brand) into vector representations. as it has been proven to deliver commendable performance in recommendation (Harte et al., 2023). Different of the Mixed-of-Expent (MoE) Whitening utilized in UniSRec, we employ identical
al.
et al.
(Bell & Sejnowski 1997) for the textual item embeddings for MoRec and Our proposed PreferDiff.
UniSRec (Hou et al.) 2022a) uses textual item embeddings from frozened text encoder and adapts to new domain using an MoE-enhance adaptor. We adopt the author's 's public implementation [2
2023) uses textual item embeddings from frozened text encoder and utilize dimension transtormation technique. The architecture is the same as previously mentioned.
MoRec (Yuan et al.
Positive Correlation Between Training Data Scale and General Sequential Recommendation Performance. Here, we explore how the scale of training data impacts the general sequential recommendation performance of PreferDiff-T. For brevity, we use the initials to represent each dataset. For example, "A" stands for Automotive, and "P" stands for Phones. "AP" indicates that the training data for pretraining includes both Automotive and Phones datasets' training set.
We observe that both NDCG and HR increase the training data grows, indicating that PreferDiff-T can effectively learn general knowledge to model user preference distributions through pre-training on
httpps://gite www.com./...... RJAA Box U
30

---

Published as a conference paper at ICLR 2025
diverse datasets and transfer this knowledge to unseen datasets via advanced textual representations. Further studies can explore whether homogeneous datasets lead to greater performance improvements (e.g,, whether Amazon Book data provides larger boost for Goodreads compared to other datasets) and investigate the limits of data scalability for PreferDiff-T.
1.2
1.2
0.0607
0.0656
-1.0
1.0
0.0482
0.0529
NDCG@S
0.0357
-0.8
Datn Scales
HR@S
0.0401
0.8
Daia Scales
0.0274
0.0232
-0.4
0.0146
0.0107
0.2
0.2
A
APF
APFTI APFHICM
APF Training Data
APFTI APFHICM
Training Data
(a) NDCG@5 on Steam
(b) HR @ on Steam
Figure 4: Positive Correlation Between Training Data Scale and General Sequential Recommendation Performance.
D.6 HYPERPARAMETER SEARCH SPACE
Here, we introduce the hyperparamter search space for baselines and PreferDiff.
Table 11: Hyperparameters Search Space for Baselines.
Hyperparamter Seach Spoce
GRU4Rec
Ie (1e22 1o-3, Ie-4, 1e-5), weight docay=0
SASRee
Ix~ [le-2 1e-3, Ie-4, 1e-5),
weight decay-D
Ix (1e2, 1e-4.1e5) 1e:5), weight decay=D,, mask probubility (0.20466....)
C1ASRee
Ir~ [1e-1 1e-3, Ie-4, 1e-5),
weight decay-O, As [0.1, 0.3, 0.5, 1.0, 3.0]
DifTRe
[1 1e-3, Ie-4, 1e-5),
weight docay=n0 moise scale {le-1, 1e-2, 1o-3, Ie-1, 1e:5), T (2, 20, 20,50, 100) (61,
DreamRec
[1e2) Ie-4, 1e:5),
weight docayen), embolding
3336)377))
6.
DiffuRee
Ir~ (1e2, 1e-3, 1e-4, 1o-5),
weight decay-O, embedding size ~ (64, 128, 256, 1024, 1836, 3072)
UniSRee
Ir~ (1e-2, 1e-3, Ie-4, 1o-5), weight decay-D) J~ (0.05, 0.1,0.3,0.. 0.5, 1.0, 3.0)
3.0)
TIGER
Ir~ (16-2, 1e-3, Ie-4, 1o-5), weight decay 1e-1, Te-2, 1e-3)
MoRee
Ir (1e-) 1o:3, Ie-,, 1e-5), weight decay=00, text encodersest caaeett-bbeeeeiddiiieeeea
LLM2BertRRe
le> (1e2, 1o-3, Ie-4, 1e-5), weight docay=00, sext-mncoderttett embeediing----aae
PreferDiff
Ie> [1e2, 1e:3, Ie-4, 1e-5),
0.8). embedding size (61, 128, 256, 1024. 1536, 3072) (0. 6,
Table 12: Best Hyperparameters for PreferDiff on Sports, Beauty, and Toys.
Dataset
leamning rate
weight docay
w
cobbedding size
Sports
16-4
044
3072
Beauty
104
0.8
3072
Toys 164
0.
3072
E HYPERPARAMETER ANALYSIS FOR PREFERDIFF
E. THE NUMBER OF NEGATIVE SAMPLES FOR PREFERDIFF.
Here, we discuss the impact of the number of negative samples on PreferDiff. As shown in Figure 6 observe that in cases where the number of items is relatively small (e.g., Beauty and Toys),
31

---

Published as a conference paper at ICLR 2025
0.0144
0.050
0.03871
0.014
0.0139
00336
0.0002
0.018
0.330
0.0110
0.0134
0.323
0.0017
0.2
03
0.6
(a) Sports
(b) Beauty
(c) Toys
Figure 5: Effect of the for PreferDiff.
00106
Nocoglo
00164
0.6835
0.6873
0.000
00142
Na
NanderofSannve
Namber Caaniie Sample
(a) Sports
(b) Beauty
(c) Toys
Figure Effect of the Number of Negative Samples for PreferDiff.
0.0167
0.0350
0.0377
PreferDif
0.0163
0.0344
0.0374
N000
0.0160
0.0371
0.0156
0.0333
0.0368
0.0152
0.0327
0.0365
2
2
8
(a) Sports
(b) Beauty
(c) Toys
Figure 7: Effect of the for PreferDiff.
negative samples are sufficient. However, as the number of items increases, the required number of negative samples also grows (e.g., in Sports).
E.2 IMPORTANCE OF GUIDANCE STRENGTH FOR PREFERDIFF
w controls the weight of personalized guidance during the inference stage of PreferDiff. As shown enhance recommendation performance. However, excessively large w may reduce the generalization capability of DMs, negatively impacting the recommender's performance. Therefore, think setting [2,4].
in Figure 7 increasing
E.3 DIFFERENT TEXT ENCODERS
Obtaining Item Embedding from Advanced Text Encoder Here, we introduce the process for obtaining item embeddings from current advanced text-encoders (Liu et al.) 2025b). For encoder- based large language models, such Bert (Devlin et al. 2019) and Robert (Liu et al. 2019) we leverage the final hidden state representation associated with the CLS] token (Hou et al. 2024b). For convenient, directly utilize the Sentence Transformers APIs As for other large language models, including T5 (Ni et al.) 2022). Llama-7B (Touvron et al., 2023). Mistral-7B (Jiang et al., (023), ) we utilize the output from the last transformer block corresponding the final input token (aswaan et al.] 2017). Closed-source large language models like text-embedding-adia-22 and text-embeddings. 3-large, we obtain the item embeddings directly via OpenAl APIs Neelakantan al. 2022)
https://platform openai wttp:/cc/ccccocccct http:///w..ppaaaacooocc//////aaa heweee//eeddddddeee ngs
32

---

Published as a conference paper at ICLR 2025
Table 13: Comparison of the PreferDifffT performance with different text-encoder.
PreferDin-T
Sports and Outdoors
Beauty
Toys and Games
Text-Emeoders
R@8
5
R6
NGF
R@5
N@5
R@000
N/8
RG85
N65
Rd 10
N@10
Bert
0,0022
0.0020
0.0030
0.0023
0.0104
0.0128
0.0154
0.0148
0.0051
0,0022
0.0068
0.0064
15
0,0011
0.0009
0.0014
0.0111
0.0241
0.0198
0.0282
0.0212
0.0283
0.0240
0.0309
0.0248
Robert
0.0115
0,0098
0.0135
00102
0.0331
0.0256
0.0393
0.0276
0.0391
0.0303
0.0438
0,0319
Mistral-7B
D0146
0.0456
0.0427
LLaMA-7B
0.0171
0.0126
0.2055
0.0137
0,0402
0,0297
0.0483
0.0325
0.0397
0.0298
0.0494
0.0530
OpenAT-Ada-V2 V2
0.0160
0.0126
0.0183
0.0134
0.0407
0.0318
0.0469
0.0338
0.0396
0.0315
0.0467
0.0339
OpenAA-3-large
0.0182"
0.0351
13 shows the PreferDiff-T employing different item embeddings encoded from text-encoders with varying parameter sizes and architectures. We can observe that
Results. Table
Positive Correlation Between LLM Size and Recommendation Performance. The results show that OpenAI-3-large outperforms all other models, indicating that larger language models (LL.Ms) yield better results in recommendation tasks. This is because larger models generate richer and more semantically stable embeddings. which improve PreferDiff's s ability to capture user preferences. Thus, the larger the LLM, the better the embeddings perform within PreferDiff.
High-Que guahh-yality Embeddings Improve Generalization. Models like Mistral-7B and LLaMA-7B, although smaller than ApenAl---large, still perform relatively well across metries. This suggests that while model size is important, the quality of embeddings plays crucial role. Especially in the Beauty, these models provide embeddings with sufficient semantic power to enhance recommendation quality.
E.4 ANALYSIS OF LEARNED ITEM EMBEDDINGS
(a) SASRec
(b) DreamRec
(c) PreferDiff
Figure 8: t-SNE Visualization and Gaussian Kernel Density Estimation of Learned Item Embeddings on Amazon Beauty.
To further analysis the item space learned by PreferDiff, we reduce the dimensionality of the learned item embeddings using T-SNE (Van der Maaten & Hinton 2008 Liu et al.) 2024a Qian et al.
2024 [5
to visualize the underlying distribution of the item space learned by PreferDiff. Due to the
large number of items in Amazon Beauty, we randomly select 2000 items example. Then, we apply
Gaussian kernel density estimation (Botey et al. 2010 16 to analyze the density distribution reduced
item embeddings and visualize the results using contour plots. The red regions indicate areas where
fold TSNE.
htm
stats.
gaus cussian_kde.hhee
33

---

Published as a conference paper at ICLR 2025
high concentration of items is clustered. From figure observe that comparing with SASRec, PreferDiff not only explores the item space more thoroughly (covering most regions). Comparing with DreamRec, PreferDiff exhibits stronger clustering effect (with high-density regions concentrated in specific areas), better reflecting the similarities between items, result in better recommendation performance.
F DISCUSSION
F.1 COMPARISON ON OTHER BACKGROUND DATASETS.
To further validate the effectiveness of PreferDiff, we include Yahoo! R1 (Music) as an additional dataset, along with two other commonly used datasets in sequential recommendation Steam (Game) and ML-IM (Movie). These datasets provide diverse set of user-item interaction patterns, allowing us to comprehensively evaluate the performance of our proposed PreferDiff.
We utilize the same data preprocessing technique and same evaluation setting as introduced in our paper for all three datasets, except Yahoo! R1. Due to its large size (over one million users), we are unable to provide results for the entire dataset during the rebuttal period. Instead, we randomly sampled 50,000 users for our experiments. We will include the full-scale results on Yahoo! RI in the final revised version of the paper. The experimental results are shown in Table
Table 14: Performance Comparison Across Background Datasets (Recall
Datasets (Background)
Yahoo (Music)
Steam (Game)
ML-1M (Movie)
GRU4Rec
0.0548 0.0491
0.0379 / 0.0325
0.0099 / 0.0089
SASRec
0.0996 0.0743
0.0695 0.0635
0.0132 / 0.0102
Bert4Rec
0.1028/ 0.0840
0.0702 0.0643
0.0215 / 0.0152
TIGIR
0. 0.1128/ 0.0928
0.0603 0.0401
0.0430 / 0.0272
DreamRec
031302/ / 0.122
0.0778 0.0572
0.0464 / 0.0314
PreferDiff
0.1408 0.1106
0.0814 0.0680
0.0629 / 0.0439
We observe that the effectiveness of our proposed PreferDiff across datasets with different back- grounds are validated.
F.2 COMPARISON ON VARIABLE USER HISTORY
we conduct additional experiments evaluate the performance of PreferDiff under different maximum history lengths {10, 20, 30, 40, 50}. Notably, since the historical interaction sequences in the original three datasets (Sports, Beauty, Toys) are relatively short, with an average length of around 10, we select two additional commonly used datasets Kang & McAAley (2018); Sun et al. (2019), Steam and ML-IM, for further experiments. These datasets were processed and evaluated following the same evaluation settings and data preprocessing protocols in our paper, which is different from the leave-one-our split in Kang McAuley (2018) Sun et al. (2019).
We choose another two datasets (Steam and IM). The results follows:
Table 15: Performance Comparison on Steam Dataset (Recall S/NDCG@5)
Model
10
20
30
40
50
SASRec
0.0698 / 0.0634
0.0676 / 0.0610
0.0663 / 0.0579
0.0668 / 0.0610
0.0704 / 0.0587
Bert4Rec
0.0702 / 0.0643
0.0689
0.0621
0.0679 / 0.0609
0.0684 / 0.0618
0.0839/ /0039/00 0.0574
TIGIR
0.0603 / 0.0401
0.0704 / 0.0483
0.0676 0.0488
0.0671 / 0.0460
0.0683 / 0.0481
DreamRee
0.0778 / 0.0572
0.0746 0.0512
0.0741
0.0548
0.0749 0.0571
0.0846 0.0661
PreferDiff
0.0814/ 0,0680
0.0804/ 0.0664
0.0806 0,0612
0.0852/ 0.0643
0.0889 0.0688
From Table 15 and Table [6 we can observe that PreferDiff consistently outperforms other baselines across different lengths of user historical interactions.
34

---

Published as a conference paper at ICLR 2025
Table 16: Performance Comparison on ML.-MM Dataset (Recal@5/NNDCGG5)
Model
10
SASRec
0.0201 / 0.0137
0.0242/ / 0.0131
0.0306 / 0.0179
0.0217 / 0.0133
0.0205 / 0.0113
Bert4Rec
0.0215 / 0.0152
0.0265 0.0146
0.0331 0.0200
0.0248 / 0.0154
0.0198/ / 0.0119
TIGIR
0.0451 0.0298
0.0430 0.0270
0.0430 0.0289
0.0364 0.0238
0.0430 0.0276
DreamRee
0.0464/ / 0.0314
0.0480 0.0349
0.0514 0.0394
0.0497 0.0350
0.0447 0.0377
PreferDiff
0.0629/ 0.0439
0.0513 0.0365
0.0546 0.0408
0.0596 0.0420
0.0546 0.0399
F.3 WHY DREAMREC AND PREFERDIFF ARE SENSITIVE TO THE EMBEDDING DIMENSION?
Here, we will try to explain the reason. Since there is no robust theoretical proof at this stage, we propose a hypothesis supported by simple theoretical reasoning and experimental validation.
We guess the challenge is inherent to the DDPM Ho et al. (2020) itself, as it is designed to be variance-preserving as introduced in the following diffusion models Song et (2021b). For one target item, the forward process formula with vector form is as follows:
ej eo + vT Oge Here, ep E R 1xd represents the target item embedding, ei represents the noised target item embedding, at denotes the degree of noise added, and e the noise sampled from a standard Gaussian distribution.
Forward Process:
Considering the whole item embeddings R-Vxd, where represents the total number of items, we can rewrite the previous formula in matrix form as follows:
ED
Eo vI- OgE
Then, we calculate the variance on both sides of the equation:
Var(EE) ar Var(Eo) a1,
We can observe that the Var En) is almost an identity matrix. This is relatively easy to achieve for data like images or text, as these data are fixed during the training process and can be normalized beforehand. However, in recommendation, the item embeddings are randomly initialized and updated dynamically during training. We empirically find that initializing item embeddings with a standard normal distribution is also key factor for the success of DreamRec and PreferDiff. The results are shown as follows:
Table 17: Performance of Different Initialization methods on Various Datasets (Recall @Recall@5/NNDI (Recal@55/DDG@55)
Embedding Initialization
Sports
Beauty
Toys
Uniform Raimirg
0.0039/0.0026
0.0013/0.0037
0.0015/0.0011
Kaiming Uniform
0.0025/0.0019
0.0040/0.0027
0.0051/0.0022
Kaiming Kaiming_Normal Normal
0.0023/0.0021
0.0049/0.0028
0.0041/0.0029
Xavier_Uniform
0.001 1/0.0007
0.0036/0.0021
0.0051/0.00 1/051/0.0029
-ormml
0.0014/0.0007
0.0067/0.0037
0.0042/0.000
Standard Normal
0.0185/0.0147
0.0429/0.0323
0.0473/0.0367
We can observe that the initializing item embeddings with standard normal distribution is the key of success for Diffusion- based recommenders. This experiment validates the aforementioned hypothesis.
Furthermore, we also calculate the final inferred item embeddings of DreamRec, PreferDiff, and SASRec. As shown Figure interestingly, we observe that the covariance matrices of the final item embeddings for DreamRec and PreferDiff are almost identity matrices, while SASRec does not exhibit this property. This indicates that DreamRec and PreferDiff rely on high-dimensional embeddings to adequately represent a larger number of items. The identity-like covariance structure suggests that diffusionn based recommenders distribute variance evenly across embedding dimensions, requiring more dimensions capture the complexity and diversity of the item space effectively. This further validates our the hypothesis that maintaining a proper variance distribution of the item embeddings is crucial for the effectiveness of current diffusion-based recommenders.
35

---

Published as a conference paper at ICLR 2025
1990
66
66
66
1300
3000
3000
64
50
290
399
60
2000
60
15
2000
290
1991
199h 244
700
Dincaikss
Dineniees
Dineaiie:
(a) SASRec
(b) DreamRec
(c) PreferDiff
Figure 9: Covariance Matrix Visualization of Learned Item Embeddings on Amazon Beauty.
We have tried several dimensionality reduction techniques (c.g., Projection Layers) and regularization techniques (e.g., enforcing the item embedding covariance matrix to be an identity matrix). However, these approaches empirically led significant drop in model performance.
We guess one possible solution to this issue is to explore the use of Variance Exploding (VE) diffusion models Song et al. (202Tb). Unlike Variance Preserving diffusion models, which maintain constant variance throughout the diffusion process, VE diffusion models increase the variance over time.
F.4 TRAINING AND INFERENCE TIME COMPARISON
Table 18: Training and Inference Time Comparison for PreferDiff and Baselines.
Dataset
Model
Training Time (s/epoch)/sstotal) Inference Time (s/epoch)
SASRec
2.67 35
0.47
Bert4Rec
7.87 79
0.65
Sports
TIGIR
11.42 1069
24.14
DreamRec
24.32 / 822
356.43
PreferDiff
29.78 558
6.11
SASRec
1.05 / 36
0.37
Bert4Rec
3.66 / 80
0.40
Beauty
TIGIR
5,41 1058
10.19
DreamRec
14.78 525
297.06
PreferDiff
18.05 / 430
3.80
SASRec
0.80 / 56
0.22
Bert4Rec
3.11 /
0.23
Toys
TIGIR
3.76 765
4.21
DreamRec
15.43 552
309.45
PreferDiff
16.07 /417
3.29
In this subsection, endeavor illustrate the training and inference time comparison between PreferDiff and baseline methods, as efficiency critically important for the practical application of recommenders in real-worke scenarios. As shown in Table [8] Figure 10 and Figure 11 observe that
In PreferDiff, thanks adoption of DDIM for skip-step sampling, requires less training time and significantly shorter inference time compared to DreamRec, another diffusion-basee recommender.
Compared traditional deep learning methods like SASRec and Bert4Rec, PreferDiff has longer training and inference times but achieves much better recommendation performance.
Furthermore, compared to recent generative recommendation methods, such TIGIR, which rely on autoregressive models and use beam search during inference. PreferDiff also demonstrates shorter training and inference times, highlighting its efficiency and practicality in real-world scenarios.
36

---

Published as a conference paper at ICLR 2025
0.891
0.645
3,800
3.10
0.02
3.90
0.00)
3.90
4815
0.13
.001
(9)
301
101
00
Taoi Estinng Time (9)
Teol Estknng Thew (9)
Thel/biiagg Time
(a) Sports
(b) Beauty
(c) Toys
Figure 10: 5 and Total Training Time for PreferDiff and Baselines.
3.540
3815
4814
0.835
4412
2.800
3810
3.22
0.825
0.000
0.50
Iafors:
Iafoumer Thme
(a) Sports
(b) Beauty
(c) Toys
Figure 11: Recall@5 and Inference Time for PreferDiff and Baselines.
F.5 TRADE-OFF OFF BETWEEN RECOMMENDATION PERFORMANCE AND INFERENCE TIME
As introduced in Subsection F4 PreferDiff demonstrates significantly lower inference time compared DreamRec, averaging around 3 seconds per batch. However, this may still be unacceptable for real-time recommendation scenarios with strict latency constraints. In this subsection, we aim to show how adjusting the number of denoising steps can effectively balance recommendation performance and inference time.
As shown Figure 12 and Table we observe that by adjusting the number of denoising steps, PreferDiff can ensure practicality for real-time recommendation tasks. This flexibility allows for trade-off between inference speed and recommendation performance, making PreferDiff adaptable to various latency constraints while maintaining competitive effectiveness.
F.6 CONNECTION PREFERDIFF AND DPO
In Preferdiff, we aim to redesign diffusion optimization objective that is specially tailored to model user preference distributions for personalized ranking. Therefore, we reformulate the classic recom- mendation objective Bayesian personalized ranking Rendle et al. (2099) to log-likelihood rankings
2000
0.045
0.055
0,010
0.000
1509
SD0GB8
30000005
0.055
Sports
Sparte Bexaty
Thme (0)
Baxuty
Time 64
1009
0.025
64
1009
0.080
Toys
0.025
.50
0.20
0.020
0015
0015
16
10'
10
167
10'
Demiis
Dermiec Step
Recall 5
(b) NDCG@5
Figure 12: Relationship of Denoising Steps and Recommendation Performance.
37

---

Published as a conference paper at ICLR 2025
Table 19: Adjusting Denoising Steps for Trade-Off Between Recommendation Performance and
Inference Time.
Datasets
Sports
Beauty
Toys
SASRec (0.33s)
0.0047 / 0.00336
0,0138 0.0090
0,0133 0.0097
BERT4Rec (0.42s)
0.0101 0.0060
0,0174/ 0.0112
0.0226 0.0139
TIGER (12.85s)
0.0093 / 0.0073
0.0236 0.0151
0.0185 0.0135
DreamRec (320.98s)
0.0155 /.0010 0.0130
0.0406 / 0.0299
0.0440 / 0.0323
PreferDiff (Denoising Step=1, 0.35s)
0.0162 /.0113 0.0131
0.0384 0.0289
0.0437 / 0.0340
PreferDiff (Denoising Step=2, 0.43s)
0.0165 0.0133
0.0398 0.0309
0.0438 0.0341
PreferDiff (Denoising Step=4, 0.65s)
0.0177/0.011
0.0402 0.0296
0.0433 0.0342
PreferDiff (Denoising Step=20, 3s)
0.0185/0.0144
0.0429 0.0323
0.0473 0.0367
Table 20: Comparison with DPO and Diffusion-DoD DPO (Rccal@5/NCDG@05)
Models
Sports
Beauty
Toys
DreamRec+ + DPO (3 = 1)
0.0031 / 0000015
0.0067 0.0053
0.0030 / 0.0022
DreamRee + DPO (3 = 5)
0.0036 / 0.0026
0.0053 0.0034
0.0036 / 0.0023
DreamRec + DPO (3 = 10)
00019/0.0011
0.0075 0.0056
0.0046 / 0.0034
DreamRec Diffusion-DPO (8
/ 000199000110
0.0308 0.0244
0.0324 / 0.0261
DreamRec Diffusion-DPO (5 0.0132 030113
0.0321 0.0251
0.0340 0.0272
DreamRec + Diffusion-DPO (5 = 10)
0.0133 0.0115
0.0281 0.0223
0.0345 0.0281
PreferDiff
0.0185 / 0.0147
0.0429 0.0323
0.0473 0.0367
which meet the requirement of generative modeling in diffusion models. We are also surprisingly and delightedly discovering that the one-negative- sample version of PreferDiff's formulation, CAPR.DiN indeed related to the recent well-known DPO Rafailov et al. (2023) which stems from Reinforcement Learning with Human Feedback, as you have mentioned. To further validate the rationality of our proposed CurR Dur, intentionally aligned some aspects of our final formulation with DPO in terms of mathematical expression.
However, there significant distinctions between PreferDiff and DPO.
First, PreferDiff is an optimization objective specifically tailored to model user preferences in diffusion- based recommenders. It is designed to align with the unique characteristics of the diffusion process, ensuring its effectiveness in recommendation tasks. We also replace the MSE loss with Cosine loss
Second, unlike DPO and Diffusion-DPe Wallace et al. (2024), PreferDiff incorporates multiple negative samples and proposes a theoretically guaranteed, efficient strategy reduce the computa- tional overhead of denoising caused by the increased number of negative samples in diffusion models. This innovation allows PreferDiff to scale effectively while maintaining high performance, making well-suited for aurge-negative-san
scenarios in recommendation tasks.
Third, unlike DPO and Diffusion-DPO. PreferDiff is utilized in an end-to-end manner without rel velying reference model. In contrast, DPO and Diffusion-DPO require two-stage process, where the first step involves training reference model. This significantly increases training overhead. which is often unacceptable in practical recommendation scenarios.
To further validate the aforementioned distinctions, we conduct experiments on three datasets using DPO and Diffusion-DPO. Specifically, we select B3 crucial hyperparameter in DPO, with values of 5, and 10, and integrate it with DreamRec for fair comparison. The results are shown Table 20
We can observe that PreferDiff outperforms DPO and Diffusion-DPO by large margin on all three datasets. This further validates the effectiveness of our proposed PreferDiff, demonstrating that is specifically tailored to model user preferences in diffusion-baser recommenders.
38


Reviews:
[{"review_id": "CaoHWiEzMM", "reviewer": "Reviewer_E8Vd", "summary": "This paper presents PreferDiff, an improved diffusion based recommendation model. The key contribution is the incorporation of an adapted  BPR objective and in turn jointly optimize the traditional MSE objective together with this new ranking loss. Variational method is employed to make the optimization tractable. The authors also show that the proposed approach is connected to DPO in formulation.\nExperiments on Amazon review data set show that the proposed methods outperform a number of different baselines.", "questions": "1. Consider use a few other data sets, especially data sets with diverse background (e.g, Yahoo!, Criteo)\n2. At least some efforts should be made to explain unexpected observations, e.g, e.g,: 1) what caused PreferDiff to be faster than DreamRec? 2) why diffusion models are more sensitive to d_model? \n3. The connection to DPO is rather weak and the claim of this as a theoretical contribution (1 of the 3 contributions) is not very sound.\n4. Eq(12) added the MSE loss back to the formula, the authors claimed that this is to mitigate the learning stability issues, it would be interesting to the readers if the authors could report that instability observations directly. It would also be worthy looking into this instability issue to root-cause it. Since PreferDiff converges faster, would this indicate that ranking loss itself might be more stable than the MSE loss and could it be possible that there are other ways to mitigate the instability issue without taking the hybrid path?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ The intuition of incorporating ranking loss into the DM recommender makes sense. The development of the proposed method looks reasonable and (to my knowledge) correct.\n+ The paper is clearly presented and easy to follow.\n+ Experiments and ablation studies are mostly solid.", "weaknesses": "- There is essentially only 1 data set being used (amazon review), no matter how many categories you include, this data set may not be representative enough which may raise concerns regarding the generalizability of your findings \n- Some of the questions remain unanswered (or observations without explanation) , e.g,: 1) what caused PreferDiff to be faster than DreamRec? 2) why diffusion models are more sensitive to d_model? \n- Novelty seems to be minimum, the overall approach makes sense but is also straightforward. The connection to DPO is rather weak and the claim of this as a theoretical contribution (1 of the 3 contributions) is not very sound.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "omPiD1kPT7", "reviewer": "Reviewer_5ztS", "summary": "This paper presents PreferDiff, a novel optimization objective designed specifically for diffusion model (DM)-based recommenders in sequential recommendation tasks. PreferDiff aims to address the limitations of traditional objectives by incorporating a log-likelihood ranking approach, allowing the model to better capture user preferences through the integration of multiple negative samples. Extensive experiments demonstrate that PreferDiff outperforms baseline models in both performance and convergence speed, validating its effectiveness in modeling complex preference distributions.", "questions": "1. How does PreferDiff handle real-time recommendation scenarios where embedding dimensions need to be minimized due to latency constraints?\n2. Could the authors provide more insights into the optimal range for the hyperparameter \\lamda, especially in varied recommendation domains?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Innovative Objective Design: PreferDiff introduces a log-likelihood ranking objective tailored to diffusion models, marking a significant advancement in personalized ranking for DM-based recommenders.\n- Comprehensive Negative Sampling: The incorporation of multiple negative samples enhances user preference understanding, leading to better separation between positive and negative items and improved recommendation performance.\n-  Effective Performance and Convergence: PreferDiff demonstrates superior recommendation accuracy and faster convergence compared to existing models, showcasing both theoretical and practical value.", "weaknesses": "1.Limited Originality: The formulation of PreferDiff shows considerable overlap with Direct Preference Optimization (DPO), as several of its mathematical expressions and objective functions appear directly inspired or derived from DPO's original framework​. This raises concerns about the novelty of PreferDiff's contribution to preference learning within diffusion models, as the paper does not introduce substantial modifications or unique approaches that deviate meaningfully from DPO's foundational equations.\n\n\n2.Dependency on High Embedding: PreferDiff’s performance is highly dependent on large embedding sizes, which may limit its scalability and increase computational costs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oPU0VsxfgN", "reviewer": "Reviewer_SvZm", "summary": "This paper introduces PreferDiff, a model designed to capture user preferences using diffusion model-based recommenders. PreferDiff primarily focuses on enhancing loss functions. The core contributions are twofold: First, the authors incorporate negative samples into the model training process through the use of ranking loss. Second, they transform the loss function in DM-based methods from rating estimation of samples to distribution estimation, formulated as the L_{BPR-Diff} loss, which is approximated by an upper bound. Experiments are conducted across multiple dataset settings to validate the approach.", "questions": "see in Weaknesses. \nI am open to revising my score based on further clarifications or additional information provided by the authors during the rebuttal process.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-written and easy to follow. It is well-structured and provides a detailed description of the experimental setup, which will aid the community in reimplementation.\n2. The authors conducted experiments under various settings to validate the effectiveness of their method.", "weaknesses": "1. The evaluation difference between the full-ranking approach and the leave-one-out method is not clearly described. There is no mention of a full-ranking approach in [1]. The 'leave-one-out' evaluation setting is the most mainstream approach. The authors need to provide a justification for using the full-ranking manner as the primary evaluation setting in the main text. Additionally, do these two evaluation methods affect the ranking of different approaches?\n2. Performance of other recommenders in Tab.1 is confusing. For TIGER, the performance in Tab.1 is much lower than the performance reported by the original paper. The authors should explain the reasons behind these performance gap. One reason maybe contribute to the performance gap is the ID quantifer. Authors take the PQcode of VQREC[2] as the ID quantifier instead of the RQVAE used in TIGER[3]。However, the authors should clearly specify whether these performance metrics are reproduced results or those reported in the original paper. For LLM2BERT4Rec, the performance on Beauty dataset is also inconsistent with the performance in original paper.\n3. The explanation in Section 3.2 is somewhat unconvincing. L_{BPR-Diff} is essentially a ranking loss in metric learning.\n4. For generative models, timestamps is crucial. The authors should conduct an ablation study on timestamps to evaluate their impact on the generation of positive and negative samples. Additionally, given that generative models are typically time-consuming, it would be beneficial to assess how PreferDiff performs in terms of inference speed compared to other sequence models. Specifically, can it achieve a favorable trade-off between hit rate/NDCG and speed?\n\n[1] Generate What You Prefer: Reshaping Sequential Recommendation via Guided Diffusion.\n[2] Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders\n[3] Recommender Systems with Generative Retrieval", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "u7I1QEgbKP", "reviewer": "Reviewer_den3", "summary": "This paper proposes a novel recommendation approach, PreferDiff, that optimizes diffusion models specifically for personalized ranking. By reformulating BPR as a log-likelihood objective and incorporating multiple negative samples, PreferDiff enhances the model’s ability to capture nuanced user preferences and better distinguish between positive and negative interactions. The authors demonstrate PreferDiff’s effectiveness through extensive evaluations, showing improved performance in sequential recommendation tasks, faster convergence, and superior handling of complex user preference distributions.", "questions": "Could you share more details on why high embedding dimensions are necessary for PreferDiff’s performance? Have you experimented with any regularization techniques or embedding pruning methods to mitigate this dependence?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces a unique objective tailored for DM-based recommenders by transforming BPR into a log-likelihood ranking format. This innovation effectively uses the generative capacity of DMs in recommendation tasks. \n\n2. The model incorporates a balance between generative modeling and preference learning, which enhances stability and performance.\n\n3. The model's gradient-weighted approach to negative samples allows it to focus on challenging cases where negative samples are mistakenly ranked high. This focus on \"hard negatives\" is particularly valuable in recommendation.", "weaknesses": "1. PreferDiff can be very slow in both the training and inference stages. It would be useful if the authors could show the efficiency-effectiveness comparison between PreferDiff and other baselines in a 2-D figure.\n\n2. As shown in Appendix D, the maximum length of interaction history is small (i.e., 10 or 20) following prior works, however, as we know that users may generally have a much longer interaction history. Is this a general limitation of DM-based recommenders that they are too expensive to train on larger sequences? What would PreferDiff perform if training with a longer sequence?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)