Generate What You Prefer: Reshaping Sequential Recommendation via Guided Diffusion
Authors: Zhengyi Yang
Jiancan Wu
Zhicai Wang
Xiang Wang
Yancheng Yuan
Xiangnan He
Venue: NeurIPS 2023 poster (2023)
Keywords: Sequential Recommendation
Recommendation System
Generative Model
Diffusion Model

Abstract: Sequential recommendation aims to recommend the next item that matches a user’s
interest, based on the sequence of items he/she interacted with before. Scrutinizing
previous studies, we can summarize a common learning-to-classify paradigm—
given a positive item, a recommender model performs negative sampling to add
negative items and learns to classify whether the user prefers them or not, based on
his/her historical interaction sequence. Although effective, we reveal two inherent
limitations: (1) it may differ from human behavior in that a user could imagine
an oracle item in mind and select potential items matching the oracle; and (2)
the classification is limited in the candidate pool with noisy or easy supervision
from negative samples, which dilutes the preference signals towards the oracle
item. Yet, generating the oracle item from the historical interaction sequence is
mostly unexplored. To bridge the gap, we reshape sequential recommendation
as a learning-to-generate paradigm, which is achieved via a guided diffusion
model, termed DreamRec. Specifically, for a sequence of historical items, it
applies a Transformer encoder to create guidance representations. Noising target
items explores the underlying distribution of item space; then, with the guidance of
historical interactions, the denoising process generates an oracle item to recover
the positive item, so as to cast off negative sampling and depict the true preference
of the user directly. We evaluate the effectiveness of DreamRec through extensive
experiments and comparisons with existing methods. Codes and data are open-sourced
at https://github.com/YangZhengyi98/DreamRec.

Generate What You Prefer: Reshaping Sequential Recommendation via Guided Diffusion
Zhengyi Yang! Jiancan Wu Zhicai Wang!
Yancheng Yuan Xiang Wang Xiangnan Helt
University of Science and Technology of China
Thhe Hong Kong Polytechnic University [yangzhy, wangzhic)
ustc. edu, cn [wuj can, xi angwang1222 xiangnanhe) @gmail. yancheng. yuan@polynppp edu.
Abstract
Sequential recommendation aims to recommend the next item that matches a user's interest, based on the sequence of items he/she interacted with before. Scrutinizing previous studies, we can summarize a common learning-to-clasifyy paradigm given positive item, recommender model performs negative sampling to add negative items and learns to classify whether the user prefers them or not, based on his/her historical interaction sequence. Although effective, we reveal two inherent limitations: (1) it may differ from human behavior in that a user could imagine oracle item in mind and select potential items matching the oracle; and (2) the classification is limited the candidate pool with noisy or easy supervision from negative samples, which dilutes the preference signals towards the oracle item. Yet, generating the oracle item from the historical interaction sequence is mostly unexplored. To bridge the gap, reshape sequential recommendation as Searning-to-generat paradigm, which achieved via a guided diffusion model, termed DreamRec. Specifically, for sequence of historical items, it applies a Transformer encoder to create guidance representations. Noising target items explores the underlying distribution of item space; then, with the guidance of historical interactions, the denoising process generates an oracle item to recover the positive item, so to off negative sampling and depict the true preference of the user directly. We evaluate the effectiveness of DreamRec through extensive experiments and comparisons with existing methods. Codes and data are open- sourced at https: / htap:::///tttii /gthttt........//gg woom/YangZhengee 198/ itteeeenyyyii///eaaeeeeeee
Introduction
Sequential recommendation has long been fundamental and important topic in many online plat- forms, such as e-commerce, streaming media, and social networking [1-3]. Its core task to recommend the next item that matches user preference, based on the sequence of items he/she interacted with before. Scrutinizing recent research on sequential recommendation [4-9], we may discern common learning- cearning-to-clasee to-classify paradigm: given a sequence of historical items and target (truly positive) item, a recommender model first performs negative sampling to append the historical interactions with some non- con-interacted (possibly negative) items, and then learns to classify the posi- tive instance from the sampled negatives. Along with this line, extensive studies are conducted on evolving the model architecture (e.g., recurrent neural networks [4]. convolutional neural networks
"Jaancan Wu and Yancheng Yuan are corresponding authors. Xiang Wang and Xiangnan He are also affiliated with Institute of Artificial Intelligence, Institute of Dataspace, Hefei Comprehensive National Science Center.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Interaction sequence
Predicted next item
data-generation distribution
Next item in sequence
Generated oralce item
Sampled negative items
Decision boundary
With negative sampling
Without negative sampling
Learning-to-classity Paradigm
Learning-to-generaae Paradigm
Figure I: Illustration of leaming-to-classity and learning-to- -t-ggeerr-eeeeee paradigms of sequential recom- menders w.r.f. the treatment of item embeddings. In the learning-to-classinn paradigm, the negative sampling technique is typically employed to classify the positive instance from the sampled negatives (left subfigure); Without negative sampling, all items naively viewed positive, undermining recommendation performance (middle subfigure). In contrast, the learning-to to-generate paradigm leverages observed interactions capture the underlying data-gent aata-generation distribution, enabling it to generate the oracle item beyond the candidate set (right subfigure).
[10, 11] and Transformers [5, 6]) and auxiliary tasks (e.g., causal inference [7, 12], contrastive learning [8, 13], robust optimization [9]), to enhance the classification ability and characterize the user preference better.
Clearly, this bearning-to-classe paradigm seeks to predict whether an item in the candidate set (i.e., positive and sampled negative items) aligns with the user preference evidenced by the historical item sequence. Despite its success, we reveal two limitations inherent in this paradigm:
may simplify human behavior - user could imagine an oracle item in mind, and then compromise real item that best matches the oracle. By "imagine", mean that the oracle item the one that a user would ideally like to interact with next, softly absorbing and representing his/her preference from previous interactions. By "compromise", conceivably, is not a concrete and discrete instance limited in the candidate set, while the real item interacted with next exemplifies the oracle. Considering the example in Figure the red triangle is the oracle item ingratiating with the interaction sequence, while the red circle is the real item close to the oracle item and satisfying the user.
mean that the oracle item,
Classification upon the candidate set is circuitous way to characterize the user preference, to address the one-class problem posed by the availability of only positive samples [14]. Yet, as the selected negatives are confined contrasts with the positive sample (e.g., the red circle in Figure 1) coarsen the decision boundary between what user likes and dislikes, leaving the item space mostly unexplored. Worse still, simple negative samples may be far away from the positive item, thus failing to contribute sufficient supervision signals to the model learning [15]; whereas, the overly complex ones might be falsely negative and inject noises into the model [14, 16, 17]. Therefore, such classification signals are hard to control to depict the oracle item.
small candidate (e.g., the blue circles Figure 1), their
To resolve these limitations, we reshape sequential recommendation from the perspective of learning- to-generate instead. The basic idea is to describe the underlying bata-generation distribution based on the historical item sequence, directly generate the oracle item that softly represents the user preference, and infer the real items most matching the oracle, as shown in Figure (right subfigure). Conceptually, generating the oracle item enables items, and encouraging its agreement with the positive items allows us to get rid of negative sampling. Towards this end, we are setting our sights on the diffusion generative models [1 [18-22]. The key idea of diffusion generative models is to gradually convert the data into noise, and generate samples by the parameterized denoising process. Yet, generating oracle items sequential recommendation via diffusion models is mostly unexplored.
go beyond the scope of all concrete and discrete

---

To bridge this gap, we propose a simple yet effective approach DreamRec, which uses the guided diffusion idea to directly generate the oracle item tailor-made for the user history. Specifically, given sequence of historical items, the forward process of DreamRec progressively adds noise to the target item's 's representation and nearly leads to complete noise, while the reverse process gradually denoises Gaussian noise guided by history to generate the oracle item aligned with the historical interactions. Wherein, we establish the history guidance by applying a Transformer encoder on the historical item sequence, DreamRec enjoys the merit of directly modeling the user preference, without relying on classification and negative sampling. On three benchmark datasets, evaluate the effectiveness of DreamRec through extensive experiments and comparisons with existing methods [4, 10, 5, 7, 12, 8].
to make the oracle item specialize for each sequence. As a result,
2
Related Work
Sequential recommendation, which captures user perference and recommends the next items based on the historical interactions, has garnered increasing attention in academia and industry [5, 4, 23]. paradigm, where the decision boundary separates the positive instance from the sampled negatives. To enhance the classification ability of sequential recommenders, existing efforts can be broadly categorized into two branches. The first branch centers around leveraging complex model architectures, such as recurrent neural networks [4], convolutional neural networks [10, 11], Transformer encoders [3, 6, 24), to encode the interaction sequences and better characterize user preferences. While the second research line is centered around the inclusion of diverse auxiliary learning tasks, including causal inference [7],
Most existing studies approach the task under a saarnng-tto-clssin
contrastive learning [8, 13), and robust optimization [9].
Diffusion Models have emerged as a prevailing approach for various generative tasks, including image synthesis [25, 18), text generation [22], and molecule design [26]. Their popularity stems from the ability to precisely approximate underlying data generation distribution and offer more stable training compared to other generative models like GANs [27, 28] and VAEs [29. 30J. Three primary formulations exist in the literature [31]; denoising diffusion probabilistic models (DDPM) [32]. score-based generative models (SGMs) [33, 20], and stochastic differential equations (SDEs) [21, 34]. DDPM represents the forward and reverse process as two Markov chains, leveraging the Markov property factorize the joint distribution into the product of transition kernels. SGMs introduce a sequence of intensifying Gaussian noise to perturb data, jointly estimating the score function for all noisy data distributions. Samples are generated by chaining the score functions at decreasing noise levels with score-based sampling methods such as Monte Carlo and Langevin Dynamics[35, 20]. SDEs perturb data to noise with a stochastic differential equation [21], whose forward process can be viewed as the continuous version of DDPM and SGM. Moreover, recent advances [25, 36, 22] have also enabled control over the generation process for conditional diffusion to generate specific images and text.
To our knowledge, recent studies [37-40] have explored integrating diffusion models into sequential recommendation. However, these approaches still adhere to the learning-to-class
paradigm, inevitably requiring negative sampling during training. For instance, Li et al. [37] and Du et [38] apply softmax cross-entropy loss [41] on the predicted logits of candidates, treating all non-target items as negative samples. While Wang et al. [39] uses binary cross-entropy loss, taking the next item as positive and randomly sampling a non-interacted item as the negative counterpart. They also incorporate contrastive loss for better classification, which requires substantial negative samples. However, diffusion model is mostly used for adding noise in the training samples for robustness, and the learning objectives are largely categorized as classification instead of generation. DiffRec by Wang et al. [40] proposes to apply diffusion on user's interaction vectors (i.e., multi-hot vectors) for collaborative recommendation, where "1" indicates a positive interaction and "0" suggests a potential negative.
In contrast, our proposed DreamRec reshapes sequential recommendation as a learning-to-genett task. Specifically, DreamRec directly generates the oracle item tailored to user behavior sequence, transcending limitations of the concrete items in the candidate set and encouraging exploration of the underlying data distribution without the need of negative sampling.
3

---

Preliminary
We recap the basic notions of diffusion model, as established by the pioneering DDPM framework [32]. In this paper, we define the lower-case letter in bold as a variable, whose superscript refers to the diffusion step. We keep other notations the same as DDPM [32].
The fundamental objective of a generative model parameterized by is to model the underlying data-generation distribution, denoted by (x(x"), where x is the target variable. DDPM, represen- tative formulation of diffusion model, formulates two Markov chains, leveraging the chain rule of probabilities and the Markov property, to model the underlying distribution.
the forward (noising) process, DDPM gradually adds Gaussian noise x" with a variance schedule [31, B2, .,,,,...,7.77.................,, ,T]:
((x I: x²) = II q(xx(1 1)
q('1xx--1 = NNxx;; v1 3,x² -1 (II)..
(1)
Let 0r = - 31, 0r = - 6 6~ (O, we have x' = -oox + v1 04.. DDPM jointly
models the target variable x" alongside set of latent variables denoted by x! x!....,x' as a Markov chain with Gaussian transitions:
x"
pa Tx = p(xx) II po |x'),
po = NNxx ; (), Ex(x', ()),
(2)
where the initial state a Gaussian noise x" N(. This is called the reverse (denoising) process of DDPM.
At the core of the generation task optimize the underlying data generating distribution pe (x"). which is performed by optimizing the variational bound of negative log-likelihood. In DDPM, this equals minimizing the KI. divergence between ((x".T) and po (x))T))
E [- log po (xx)) <DKL (4 X1, ,xx)) Ilpo (xU, X1, xT))
(3)
T
log po (x,-1 x1) (Xt x1-1)
= -x
-logp (XT) Σ
+C
(4)
T
Σ DEL (a(x*=1/xx xxxx,)))l) +C2
(5)
where C1 and C2 are constants that are independent of the model parameter 0. Using Bayes' theorem. the posterior distribution q(xf=1|x -xx", x²) could be solved in closed form:
q(x'~1x''_ xx'xx) = NN f-1 (x' xx)xxxx)xx xx)), B.1)
(6)
where
var-18t x Varl āa) and
1 - ōi-1 Br.
(x(xxxx))
Br
1 at
Further reparameterizing po (x* ( as:
no (x", ()
02 Eo (xx) Var VI a
(8)
the 1-th term of the training objective in Equation (5) is simplified to:
37
L = -xx, 28,00 σ.)) |leo ca(Vaix + vI aie t) +C.
(9)
Following DDPM [32]. En (x 1) is set to B,1 to match the variance of Equation (2) and Equation (6). The model architecture 6g depends on specifie tasks, such as U-Net for image generation [32] and Transformer for text generation [22].

---

Method
In this section, elaborate the proposed DreamRec, following the learning-to -o-einnrrgt-eeeeeeeeeeeee paradigm with guided diffusion. We start by reformulating sequential recommendation as an oracle item generation task. Subsequently, we explain our approach directly generate oracle item embeddings, taking inspiration from DDPM [32]. To conclude, we introduce our approach to enable personalized generation using classifier-free guidance.
4.1 Sequential Recommendation Oracle Item Generation
Let be the set of discrete items in the dataset. A historical interaction sequence is rep- resented as Mim-1 = [V1, 2/2 1n-1), and on is the subsequent consumed item of the se- quence. Let D [[Vim--1 um m=1 stand for all the sequences within the training data, and D. ([vin- | denotes test sequences. Typically, each item E I is initially translated into its corresponding embedding vector e. Thus, the interaction sequence can represented e1:--1 [e. eg...... en-1). The fundamental objective of sequential recommendation is to recommend the potential subsequent item that aligns with the user preference.
In this work, we propose DreamRec, a method that reshapes sequential recommendation a learning- to- to-generate task. The principle behind that users tend create an "oracle" item their minds an idealized item that they then search for in tangible form from the candidate set, when making purchase. We assume that these oracle items are drawn from the same underlying distribution that generates observed items. In DreamRec, mene//ee.n-a 1)), since the generation of next item e is highly related to historical interactions ein-1 in sequential recommendation. If pa (e(% @1:m-1) can be precisely learned, it becomes possible to In DreamRec, we learn
model the underlying distribution pa(e e1:n-1) (i.e.,
generate the oracle item from the interaction sequence through pe(e, (e (1:x-1) by employing guided diffusion model.
4.2 Oracle Item Generation with Guided Diffusion
After framing sequential recommendation as an oracle item generation task, we proceed to in- troduce the learning and generation phases of DreamRec.
Forward process (nolsing)
e
Reverse process (denalsing)
4.2.1 Learning Phase of DreamRec
Gaussian
(e Cn-1)
Target
A straightforward application of DDPM, out- lined in Section 3, cannot sufficiently achieve the oracle item generation objective in sequen- tial recommendation. This is primarily be- cause the denoising process modeled in Equa- tion (2) lacks guidance from historical interac- tions, resulting non-personalized generated items. To resolve this, we propose to guide the denoising process with the corresponding his- torical interaction sequence. Specifically, we first encode the interaction sequence C1.n-
e
11n-1
T-enc -
MLP
Figure 2: Framework of DreamRec. To achieve personalized denoising, DreamRec encodes histor- ical interactions i to be guidance signal -, with a Transformer encoder T-ene, subsequently utilizing -, to guide the reverse process.
(ei, ..,..., ep-1 with a Transformer encoder:
CT-1 T-enc(eim-1).
(10)
This encoded interaction sequence, C, con- ditions the denoising process as follows:
(e ee,, Cn-1) = (ee u cn-1,t), Ee t))
(11)
where the architecture of po(e', 1n t) an MLP in DreamRec depicted Figure 2.
Similar to Equation (1), the forward process is formulated as a Markov chain of Gaussian transitions: (e = =V (eee vT - 3.I with variance schedule [31, :2,,..,,,7,1 Both the forward and reverse processes of DreamRec are also illustrated in Figure 2.
5

---

Algorithm Training phase of DreamRec
1: repeat
c, G1m-1
Sample and embed data from training set.
3: Ca-1 = =T-nnc(eem-))
> Encode interaction sequence. Perform unconditional training with probability pa- p Sample diffussion step. Sample Gaussian noise.
4: With probability Pr: Cn-1 4
5: t> Uniform (1. ......TT)
E <~ I)
7: e = e
+ /1 diE
> Corrupt the traget item with Gaussian noise.
8= 8 V
6 fo 1n
b Take gradient descent step, ja is the step size.
9: until converged
Algorithm 2 Generation phase of DreamRec
1: e1m-1 ~ Dr
>. Sample and embed a data from testing set, Sample Gaussian noise. Encode interaction sequence.
2: e, ~ 1)
3: Ca-1 Teenn(eemm-1)
=ot==......
Denoise for steps. D Sample denoising variance. b Control the strength guidance.
N~ 1, else
6: folcin Cn-1,t) = (1+ w) fole'n cn-l, t) folen, 2,t) vin(l 11-1-- e/, v Bi
7: e - 1-80 folet, 1, f)+
Denoise for step.
1-
8: end for
9: return eh
Thereafter, DreamRec improves upon T--1 of Equation (5) with guidance signal, and achieves the learning objective:
- = D - (e "e",, cn-1)).
(12)
We employ another reparameterizzatio that predicts target sample eQ, rather than the added noise e as in Equation (
no (e 1. = val-ifole,, cn + var(1 āa-1)
E,
(13)
V1 αz
and Equation (12) converts another format:
ύr-l lee SolVare, 0 + V1 04E, en 1,11 23,
L Eeg.
+
(14)
Besides the conditional diffusion model, guided diffusion typically necessitates an additional uncondi- tional one [25], which can be jointly trained using classifier-free guidance scheme [36]. Specifically, during training, we randomly replace guidance signal Cn-1 by a dummy token & with probability Pu. to achieve the training of unconditional diffusion model.
important to note that Equation (14), of denotes the observed target item in interaction sequence, and denotes the Gaussian noise. Therefore, DreamRec's 's training procedure does not require negative sampling. Instead, concentrates on recovering the target item in the interaction sequence. Algorithm shows the details of DreamRec's training phase.
4.2.2 Generation Phase of DreamRec
In the generation phase, we target to generate personalized oracle items for different users, given their historical interactions. To manipulate the influence of the guidance signal Cn we would modify fa(e), 1) to conform to the following format:
fo t,- = (1 + w) fo tn w folc,, T t),
(15)
Pleasse note that predicting e and predicting are equivalent due to e, = varca vI a.e. Refer to Appendix for more details.

---

where w is hyperparameter controlling the strength of Ca-1: A higher w value can enhance personalized guidance, but it could potentially undermine diffusion generalization, consequently deteriorating the quality of the generated oracle item.
Based on Equation (7), the one-step denoising process is straightforward as follows:
e/ 1 - So(e) Cn- Var -1 et + Vaz, z~ ((,1).
(16)
Ot+y
During the inference stage, for a user characterized by historical interactions encoded as Cn-1, the oracle item ey, can be generated by denoising Gaussian sample e ~ (O for 7 steps, in accordance with Equation (16). The generation phase of DreamRec is demonstrated in Algorithm
4.2.3 Retrieval of Recommendation List
After generating the oracle item, the subsequent step involves obtaining the recommendation list tailored to the specific user. To achieve this, we retrieve the K-nearest items the oracle item within the candidate set using an inner product measurement, forming the recommendaiton list. It's crucial emphasize that the selection K-nearest items exclusive to the retrieval of the recommendation list and does not figure into the training phase. Conceptually, DreamRec transcends the confines of a limited candidate set, venturing into the entire item space in its pursuit of the oracle item.
Experiments
In this section, we conduct experiments to demonstrate that: 1) DreamRec provides a powerful Borning-to-generee framework for sequential recommendation; 2) DreamRec can better explore the item space without negative sampling; and 3) Guiding the diffusion process with historical interactions is important for personalized oracle item generation in DreamRec.
5.1 Experimental Settings
Datasets. We use three datasets from real-world sequential recommendation scenarios: YooChoose, KuaiRec, and Zhihu (the statistics of datasets are illustrated in Appendix B):
YooChoose dataset comes from RecSys Challenge 2015 4, We preserve the purchase sequences for a moderate size of data. We only retain items with at least interactions to avoid cold-start issue. Additionally, we exclude sequences that are shorter than interactions in length.
KuaiRec [42] dataset is collected from the recommendation logs of a video-sharing mobile app.
We also remove items that are interacted with less than 5 times and sequences shorter than 3.
Zhihu [43] dataset is collected from socialized tnowledee-sharingg presented with a recommended Q&A list and they can read their preferred ones. We remove items that are read less than 5 times and sequences that are shorter than 3 in length.
community. Users are
For all datasets, we first sort all sequences in chronological order, and then split the data into training, validation and testing data at the ratio of 8:1:1.
Baselines. We compare DreamRec against several competitive models, including GRU4Rec [4], Caser [10], SASRec [5], S-IPS [7], AdaRanker [12], CL4SRec [8] and DiffRec [40]. GRU4Rec, Caser, and SASRec adopt recurrent neural networks, convolutional neural networks, and Transformer encoders respectively capture sequential patterns of user behaviors. S-IPS and AdaRanker leverage causal inference and neural process to address the issues of selection bias and temporal dynamics in sequential recommendation. CL4SRec designs three data augmentation methods and applies contrastive learning techniques to enhance the classification ability of sequential recommender. DiffRec proposes to incorporate the diffusion model in collaborative filtering, but applies diffusion on user's interaction vectors - multi-hor hot vectors whose element of "f" indicates a positive interaction while "0" suggests a potential negative.
Training Protocol. We implement all models with Python 3.7 and PyTorch 1.12.1 in Nvidia GeForce RTX 3090. We preserve the last 10 interactions as historical sequence. For sequences with less than
'nttps:/rrecsys acm. www.oorgggrr///aaaaeeee.
7

---

Table 1: Overall performance comparison. The boldface denotes the best performance while the underline indicates the second best. The experiments are conducted 5 times and the average and standard deviation reported.
YooChoose
KuaiRec
Zhibu
)))
NDCG@20($)
HR@20
NDCG@20(%)
HR
NDCG@200 20())
GRU4Rec
3.89+0.11
1.62:40.02
3.32-0.11
1.23++0008
1.78:+0.12
0.67:00.33
Caser
4.06+-0.1
1.88++0099
2.88=0.19
1.07+0.07
1.57=+0.05
0.59=0.01
SASRec
3.68++.08
1.63+0022
3.92=0.18
1.53+0.11
1.62+0001
0.60=00.3
3.81+0.05
3.73=0003
1.40#0.05
1.66+00.04
0.64:00.2
AdaRanker
3.74+0.06
1.67++0.04
4.14=0.09
1.89
1.70##0.04
0.61=0.02
CLASRec
4.45++004
1.86+0.02
4.25=0.10
2.01
2.03++0.06
0,74=0.03
DiffRec
4.33+0.02
1.84+0001
3.74=0008
1.77+0005
1.82+00.33
0.65=00.9
DreamRee
4.78.10.06
2.23.10.02
5.16.0005
4.11
2.26.10.07
0.79.10.01
(a) SASRec (no negative sampling).
(b) SASRec.
(c) DreamRec.
Figure 3: Visualization of the learned item embeddings Zhihu dataset using T-SNE, where red points represent items. Figure 3a shows that without negative sampling, the item embeddings of SASRec are crowded in limited discrete zones. Figure 3b shows that With negative sampling, the item embeddings of SASRec concentrate on only part of the item space. Figure 3c shows that DreamRec explores most of the item space without requiring negative sampling.
10 interactions, we would pad them to 10 with padding token. We leverage Adam W the optimizer. The embedding dimension of items is fixed as 64 across all models. The learning rate is tuned in the range of [0.01, 0.005, 0.001, 0.0005, 0.0001, 0.00005]. Despite that DreamRec does not require L2 regularization, we tune the weight of L2 regularization for all baselines in the range of [le-3, le-4, 1e-5, 1e-6, le-7]. For all baselines, we conduct negative sampling from the uniform distribution at the ratio of 1: 1. which is not conducted in DreamRec. For our DreamRec, we fix the unconditional training probability pa as 0.1 suggested by [36]. We search the total diffusion step in the range of [50, 100, 200, 500, 1000, 2000], and the personalized guidance strength w in the range of [0, 2, 4, 6. 8, 10].
Evaluation Protocol. We follow the widely used top-K protocol to evaluate the performance of sequential recommendation and adopt two widely used metrics: hit ratio (HR) and normalized discounted cumulative gain (NDCG) [4. 5]. Sequential recommenders within the learning-to-classffy paradigm usually leverage the classification logits on candidate items to select top-K items. For DreamRec, we first generate the oracle item with Algorithm 2, and then we find the K-nearest items in the candidate set for top-K recommendation with the measurement of inner product. Note that the selection of K-nearest items is only conducted at the evaluation phase, and is not involved in the training phase.
5.2 Main Results
In this section, we compare DreamRec against baseline models in terms of top-K recommenda- tion performance. Table shows the experimental results. Overall, DreamRec substantially and consistently outperforms compared models, which demonstrates the superiority of our proposed camn-to-generae paradigm in sequential recommendation.
Note that all baselines are within the are backbone models; IPS and AdaRanker enhance classification ability by solving data biasing issues; and CL.4RRec further equips classification objectives with contrastive learning techniques. We do witness in Table that these auxiliary tasks improve the performance of backbone models.
paradigm: GRU4Rec. Caser and SASRec

---

0.49
0.034
0.03
HR222
3.047
HR,220
3,0223
0.230
0.045
0.033
8.215
0.43
0.047
6.230
(a) YooChoose.
(b) KuaiRec.
(c) Zhihu.
Figure 4: Ablation study of classífier-free guidance by adjusting the guidance strength w on Yoo-
Choose (subfigure 4a), KuaiRec (subfigure 4b), and Zhihu (subfigure 4c) datasets.
In terms of DreamRec, it reshapes sequential recommendation purely as an oracle item generation task without those auxiliary tasks, and has already achieved better performance our experiments. We believe designing auxiliary tasks can also boost the generation of oracle items in sequential recommendation, yet we leave this as future work since beyond the scope of this paper.
5.3 Visualization
In this section, visualize the learned item embeddings using T-SNE [44] to demonstrate that DreamRec can well explore the underlying distribution item space without negative sampling.
Specifically, we first train three recommenders Zhihu dataset (results on other datasets included Appendix C): I) SASRec without negative sampling; 2) SASRec with negative sampling: and 3) DreamRec that does not require negative sampling. Then we use T-SNE to visualize the learned item embeddings under the default setting of scikit-learn 5, Figure demonstrates the visualization results, where each item is represented as red point. From Figure 3 we can observe that: 1) Negative sequential recommenders: no negative sampling
sampling is necessary for classification conducted, plenty of items may be crowded pathologically in limited discrete zones of the item space, making them indistinguishable; 2) Negative sampling makes classification easier since items better shattered, but they biased concentrate more part of the zone of item space; and 3) By reshaping sequential recommendation as a learning-to sealinntt--geerrate framework, DreamRec explores most of the zones of item space without requiring negative sampling. These observations provide strong empirical support for our claims made in Section 1.
5.4 Controlling the Personalized Guidance of DreamRec
As introduced in Section 4.2.2, achieving personalized oracle item generation requires guidance from the corresponding interaction sequence. To achieve controllable guidance in DreamRec, adopt classifier-free guidance technique. However, we should carefully adjust the value of guidance strength w in Equation (15), higher value may hurt the generalization of diffusion and lead to generating nower-quality oracle items. To shed light on this issue, we conduct experiments on the three datasets by adjusting the value of w. As shown in Figure 4, observed that increasing the value of w initially leads to better recommendation accuracy. This supports our intuition that stronger guidance leads to better personalization. However, observe a decline in performance. This line with our analysis that concentrating too much on the guidance signal may hurt the generation quality of oracle items.
continue to increase the value of w,
Conclusion and Limitations
We propose DreamRec, reshaping sequential recommendation learning-to-genenndl task, instead of ceamiing-to-classi task as almost all previous methods do. DreamRec is based on the analysis of user behaviors that, after several interactions with the recommendation system, users tend to fantasize about an oracle item they would "ideally" consume. The oracle item does not have to be the next item in the dataset, and even should not be limited to the pre-defined candidate set. By
learn. http:///lllaattteerrerrrrrrre/ee
sklerrn manifold. TSNE.

---

modeling the underlying data-genenn generation distribution with diffusion model, DreamRec promises to generate unobserved oracle items. Moreover, targeting to model the data-generation process directly, DreamRec makes it possible to discard negative samples in sequential recommendation, which can hardly be achieved by previous classification-base models. Experiments show that DreamRec brings consistent improvements to sequential recommendation, implying its superiority in modeling user behaviors.
Meanwhile, DreamRec also has a few limitations: 1) the sampling process is quite slow with the iteration of total diffusion steps; and 2) the training process more time- cone-consuming. We believe these can be resolved in further research with more advanced generation models such as consistency model [45]. Moreover, as an initial attempt of reshaping sequential recommendation an oracle item generation task, DreamRec provides many research opportunitiss such as designing auxiliary tasks (e.g., contrastive learning data augmentation) to enhance oracle item generation. Another research line could be designing other encoders for guidance representation, or other model architectures for the denoising process of DreamRec.
Broader Impact
The proposed DreamRec significantly improve the performance of sequential recommendation. Therefore, it can be applied to real-world platforms to provide more satisfying recommendation results. One concern of generating oracle items is the potential for privacy disclosure. Despite that encode the oracle item with vector representations, one may decode the representation and snoop on users' preference explicitly. Therefore, kindly advise researchers to be cautious about the usage of generated oracle items.
Acknowledgements
This research supported by the National Natural Science Foundation of China (9227010114 14, U2 U21B2026, 62302321), the University Synergy Innovation Program of Anhui Province (GXXT-2022- 040), and the Hong Kong Polytechnic University under grant (P0045485).
10

---

References
[1] Pedro Dalla Vecchia Chaves, Bruno L. Pereira, and Rodrygo L. T. Santos. Efficient online learning to rank for sequential music recommendation. In WwW, pages
2022.
[2] Paul Covington, Jay Adams, and Emre Sargin. Deep neural networks for youtube recommenda- tions. In RecSys, pages 191-198, 2016.
[3] Guorui Zhou, Xiaoqiang Zhu, Chengru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. Deep interest network for click-through rate prediction. In KDD, pages 1059-1068, 2018.
[4] Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based recommendations with recurrent neural networks. In ICIR, 2016.
[5] Wang-Chead Kang and Julian McAuley. Self-attentive sequential recommendation. In ICDM, pages 197-206, 2018
[6] Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang, Bert4ree: Sequential recommendation with bidirectional encoder representations from transformer. In CIKM, pages 1441-1450, 2019.
[7] Zhenlei Wang, Shigi Shen, Zhipeng Wang, Bo Chen, Xu Chen, and Ji-Rong Wen. Unbiased sequential recommendation with latent confounders. In WWW, pages 2195-2204, 2022.
[8] Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Jiandong Zhang, Bolin Ding, and Bin Cui. Contrastive learning for sequential recommendation. In ICDE, pages 1259-1273, 2022.
[9] Zhengyi Yang. Xiangnan He, Jizhi Zhang, Jiancan Wu, Xin Xin, Jiawei Chen, and Xiang Wang. A generic learning framework for sequential recommendation with distribution shifts. In SIGIR, pages 331-340. ACM, 2023.
[10] Jiaxi Tang and Ke Wang. Personalized top-n sequential recommendation via convolutional sequence embedding. WSDM, pages 565-573, 2018.
[11] Fajie Yuan, Alexandros Karatzoglou, loannis Arapakis, Joemon M. Jose, and Xiangnan He. simple convolutional generative network for next item recommendation. WSDM, pages 582-590, 2019.
[12] Xinyan Fan, Jianxun Lian, Wayne Xin Zhao, Zheng Liu, Chaozhuo Li, and Xing Xie. Ada- ranker: A data distribution adaptive ranking paradigm for sequential recommendation. In SIGIR, pages 1599-1610, 2022.
[13] Ruihong Qiu, Zi Huang. Hongzhi Yin, and Zijian Wang. Contrastive learning for representation degeneration problem sequential recommendation. K. Selcuk Candan, Huan Liu, Leman Akoglu, Xin Luna Dong, and Jiliang Tang, editors, WSDM, pages 813-823, 2022.
[14] Steffen Rendle. Item recommendation from implicit feedback. In Francesco Ricci, Lior Rokach, and Bracha Shapira, editors, Recommender Systems Handbook, pages 143-171. Springer US 2022.
[15] Steffen Rendle and Christoph Freudenthaler. Improving pairwise learning for item recommen- dation from implicit feedback. In WSDM, pages 273-282. ACM, 2014.
[16] An Zhang, Jingnan Zheng, Xiang Wang, Yancheng Yuan, and Tat seng Chua. Invariant collaborative filtering popularity distribution shift. In WWW, 2023.
[17] An Zhang, Wenchang Ma, Xiang Wang, and Tat-Seng Chua. Incorporating bias-aware margins into contrastive loss for collaborative filtering. In NeurIPS, 2022.
[18] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, pages (0674-10685, 2022.
11

---

[19] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. CoRR. abs/2208. 12242, 2022.
[20] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. NeurlPS, pages 11895-11907, 2019.
[21] Yang Song, Jascha Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2021.
Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and
[22] Xiang Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori B. Hashimoto. Diffusion- Im improves controllable text generation. In NeurlPS, 2022.
[23] Alex Beutel, Paul Covington, Sagar Jain, Can Xu, Jia Li, Vince Gatto, and Ed H. Chi. Latent cross: Making use of context in recurrent recommender systems. WSDM, pages 46 54, 2018.
[24] Xinyan Fan, Zheng Liu, Jianxun Lian, Wayne Xin Zhao, Xing Xie, and Ji-Rong Wen. Lighter and better: Low-rank decomposed self-attention networks for next-item recommendation, In
SIGIR, pages 1733-1737, 2021.
[25] Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat gans on image synthesis.
In NewrIPS, pages 8780-8794, 2021.
[26] Minkai Xu, Lantao Yu. Yang Song, Chence Shi, Stefano Ermon, and Jian Tang, Geodiff: A geometric diffusion model for molecular conformation generation. In ICLR, 2022.
[27] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Fas Warde-Farley. Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. In NeurlPS, pages 2672 2670, 2014.
[28] Martín Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In ICML, pages 214-223, 2017.
[29] Diederik Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.
[30] Lars Maaløe, Marco Fraccaro, Valentin Liévin, and Ole Winther. BIVA: A very deep hicrarchy of latent variables for generative modeling. In NeurIPS, pages 1548-6558, 2019.
[31] Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Yingxia Shao, Ming-Hsuan
Wentao Zhang, Ming- Hsuan Yang, and Bin Cui. Diffusion models: A comprehensive survey of methods and applications. CoRR, abs/2209.00796. 2022.
[32] Jonathan Ho, Ajay Jain, and Pieter Abbecl. Denoising diffusion probabilistic models. In NeurlPS, 2020.
[33] Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. In NeurIPS, 2020.
[34] Alexia Gotta go fast when generating data with score-based models. CoRR, abs/2105. 14080, 2021.
Ke Li, Rémi Taihleaail,,,er, Tal Kachman, and Ioannis Mitliagkas.
[35] Alexia Volicocur-Martinen. Rémi Piché-Tailleferr Ioannis Mitliagkas, and Remi Tachet des Combes. Adversarial score matching and improved sampling for image generation. In ICLR. OpenReview.net, 2021.
[36] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. CoRR, abs/2207.12598. 2022.
[37] Zihao Li, Aixin Sun, and Chenliang Li. Diffurec: A diffusion model for sequential recommen- dation. CoRR, abs/2304.00686. 2023.
[38] Hanwen Du, Huanhuan Yuan, Zhen Huang, Pengpeng Zhao, and Xiaofang Zhou. Sequential recommendation with diffusion models. CoRR, abs/2304.04541, 2023.
12

---

[39] Yu Wang, Zhiwei Liu, Liangwei Yang, and Philip S. Yu. Conditional denoising diffusion for sequential recommendation. CoRR, abs/2304.11433 1433, 2023.
[40] Wenjie Wang, Yiyan Xu, Fuli Feng, Xinyu Lin, Xiangnan He, and Tat-Seng Chua. Diffusion recommender model. In SIGIR, 2023.
[41] Jiancan Wu, Xiang Wang, Xingyu Gao, Jiawei Chen, Hongcheng Fu, Tianyu Qiu, and Xi- angman He. On the effectiveness of sampled softmax loss for item recommendation. CoRR, abs/2201.02327,, 2022.
[42] Chongming Gao, Shijun Li, Wenqiang Lei, Jiawei Chen, Biao Li, Peng Jiang, Xiangnan He, Jiaxin Mao, and Tat-Seng Chua. Kuairec: A fully-observed dataset and insights for evaluating recommender systems. CIKM, 2022.
[43] Bin Hao, Min Zhang, Weizhi Ma, Shaoyun Shi, Xinxing Yu, Houzhi Shan, Yiqun Liu, and Shaoping Ma. A large-scale rich context query and recommendation dataset in online knowledge- sharing. CoRR, 0abs21006006467 2021.
[44] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
[45] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. CoRR, has/2303.01469, 2023.
13

---

A Proving the Equivalent of Equation (9) and (14)
Recall that in Equation (3) (5), we have formulated the generation task in DDPM optimizing the variational bound of negative log-likelihood, where the t-th term is as follows:
- D (g(x"~ xx', , (x
(17)
Since g(x*~1|xx x x²) and pe px (x²) are both Gaussian distributions, we can apply the Rao- Blackwellized Theorem [32] compute Equation (17), resulting in:
1
L--1 Eg
|| x9) 1 t)]1 +C,
(18)
2
where:
[x x0) vā. 13 x + vas(1 - 04-1) x"
(19)
at
1
Note that x² v + v1 acC, we have:
i = v x +
Var(1 - (h--1)
x
1- 0z
vai-131 x²
VVI EyE
+ var(1 a(-1)
x!
(20)
Vāx
1- āt
1
at 6
Xt
Vat
VI - āc
In DDPM, A4 is reparametrized as:
1 1)
x² 1 - az vT- Ct.
co (x(xx)t))
(21)
Val
Substituting Equation (20) and (21) back to Equation (18), we can obtain the simplified version of Lt-1 as expressed in Equation (9):
B
L
Ex
||c colvaix VI t)1
+
(22)
23,az(1 ā1)
We can adopt an alternative reparametrization of ss similar to Equation (13): (xxx'' (t) (ax_if((x(ttt + Vor(1 - ā(-1)
(23)
VI āt
Then derive:
x²)
100 (x(,x)t)
1
0t
-vo(-1fo(x(,t) Var(1 - -a-11)
x'
Vai
V1 0z
V1 01
1
Vānx vI 11-E
VI vI- 01 ))
1 1-a1
-Vān-ifolx'
Vor(1 (1 - x(-1)
(24)
Var
0
Var(1 ac-1)
vor-ifo(x',1) Var(1 ac-1)
VI 11-a āi
VI - āi
-xx
fo(x', tt).
Therefore Equation (18) can be simplified another version:
- = Ex",c
[
xx
fo + -I- -l-, t)1 +
(25)
23,
So far we have proven that Equation (22) (predicting noise) and (25) (predicting target) are two equivalent versions of training objective for diffusion in theory. In practice, DDPM shows that Equation (22) performs well in image generation, while study [22] shows Equation (25) in more suitable text generation. In sequential recommendation, we adopt Equation (25) (or equivalently, Equation (14)) to acquire our objective in DreamRec for oracle item generation.
14

---

B Detailed Experimental Settings
B.1 Statisties of Datasets
The statistics of the adopted datasets are summarized in Table 2.
Table 2: Statistics of datasets.
Dataset
YooChoose
KuaiRec
Zhihu
#sequences
128,468
92,090
11,714
#items
9,514
7,261
4,838
#interactions
539,436
737,163
77,712
(a) SASRec (no negative sampling).
(b) SASRec.
(c) DreamRec.
Figure 5: Visualization of the learned item embeddings on YooChoose dataset using T-SNE. Dream- Rec explores most of the item space without requiring negative sampling.
(a) SASRec (no negative sampling).
(b) SASRec.
(c) DreamRec.
Figure 6: Visualization of the learned item embeddings on KuaiRec dataset using T-SNE, DreamRec explores more of the item space without requiring negative sampling.
C More Ablation Studies
C.1 Visualization on YooChoose and KuaiRec Datasets.
We further visualize the learned item embedding visualization on YooChoose and KuaiRec datasets Figure 5 and Figure 6. Similar to Zhihu dataset (Figure 3), SASRec without negative sampling may fail to distinguish different items, since plenty of items gather in limited zones of the item space. Consequently, negative sampling is necessary for classification- baaeed sequential recommenders. In contrast, DreamRec directly models the data-generation distribution and can better explore item space without the requirement negative sampling.
15


Reviews:
[{"review_id": "krgVEXpZkv", "reviewer": "Reviewer_RbjC", "summary": "The paper presents a new approach Diff4Rec, a guided diffusion model for sequential recommendations. The paper could be divided in three main parts:\n1. Problem formulation and method: It describes the sequential recommendation as oracle item generation and then explains how diffusion is applied. \n2. Experiments: Describes the datasets used and various experiments done. Moreover, they have provided code also. \n3. Results: Presents the results from their experiment. The results are much superior compared to the previous methods. ", "questions": "1. Noob q: As per my understanding, during the inference time we will get embedding for nth item as a oracle item. How do we recover what item to recommend based on the predicted embedding?\n2. Do we have any results from an A/b test? ", "limitations": "A/B testing on real system would have helped to validate the proposed idea.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The paper presents a new idea in the learning-to-generate paradigm, which aimed to used diffusion process. They have supported the results with exhaustive set of experiments and validated the results on multiple datasets. ", "weaknesses": "1. The paper doesn't show the impact of negatives on the recommendations\n2. Would it be possible to present the A/B test results. I suspect the recommendations would over hinge on the users history while recommending for a longer period of time.. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> 1. *The paper doesn't show the impact of negatives on the recommendations.*\n\n   Thank you for raising this point. We feel a little confused about the \"impact of negatives\", and try to analyze it from two distinct perspectives:\n\n   - **Negative Societal Impacts**: we would like to discuss the negative social impact of recommendation. Firstly, one major concern about recommender system is the potential of privacy disclosure and information leakage, and it is not a risk in our work since the datasets are all anonymized by the provider. Secondly, recommender system may bring issues such as Information cocoons and echo chambers, which are also significant research topics beyond the scope of our work.\n\n   - **Impact of Negative Sampling**: we would like to discuss more about negative sampling in recommender system. As described in Line 44-52 and Figure 1, the learn-to-classify based recommenders are demanding of negative samples to discriminate between positive samples and negative ones when learning the decision boundary.  Without negative sampling, the item embedding of learn-to-classify based recommenders would be pathologically distributed as shown in Figure 3 of the main text and Figure 6 of the Supplementary Material. Diff4Rec, by contrast, explores the underlying distribution of observed interactions with diffusion model, and does not require negative samples in the learning process. \n\n   We eagerly anticipate a deeper discussion on these matters in the subsequent stages!\n\n---\n\n> 2. *How do we recover what item to recommend based on the predicted embedding?*\n\n   Thank you for bringing up this point! As mentioned in Lines 253-256, once we obtain the embedding of the oracle item, our next step is to identify the top K-nearest items from the candidate set for top-K recommendation, based on their similarity (i.e., inner product between the embeddings of the oracle and candidate). We acknowledge the significance of this step in achieving the recommendation task, and to provide a comprehensive understanding, we will detail this step in Method of the revision. We're grateful for your astute feedback, which has undeniably enriched our presentation.\n\n---\n\n>  3. *Would it be possible to present the A/B test results? I suspect the recommendations would over hinge on the users history while recommending for a longer period of time.*\n\n   Thank you for bringing this point to us. We recognize the potential of A/B testing to validate the efficacy of recommendation models, though \nour current access does not extend to online A/B testing platforms. \n\n   Moreover, we notice that the main concern comes from that the recommendation would over hinge on the user history while recommending for a longer period of time. To address this, we further simulate the A/B testing with the assistance of ChatGPT, inspired by its remarkable generalization and simulation ability, since it encodes a wide range of human behavior from their training data  [a]. Such ChatGPT-simulated A/B testing is less prone to the user history, as compared to the conventional evaluation on the fixed test data. We list the steps:\n\n   - Data Split: We sample 100 real users from the MovieLens-100k dataset for evaluation, reserving the rest as the training data;\n   - Recommender Training: We train Diff4Rec and CL4SRec (the best baseline model under top-K evaluation) on the training data;\n   - User Simulation: For each user being evaluated, we convert his/her history to a textual prompt to profile the user, and feed it into ChatGPT to simulate the user. Then, three movie lists derived from Diff4Rec, CL4SRec, and Random are presented to ChatGPT to select which list he/she would prefer. We conduct the simulation five times per user with different ChatGPT accounts, and the list with the most votes will have its *Success Score* plus one. \n   - Simulated A/B Testing: We compare the *Success Score* among Diff4Rec, CL4SRec, and Random, based on the selection of ChatGPT-simulated users. The results are shown as follows (Note that two lists may have even votes (3 out of 100), and we denote this inside the bracket):\n   Random | CL4SRec | Diff4Rec\n   :--:|:--:|:--:\n   12|33(3)|52(3)\n\n   Clearly, Under the evaluation of ChatGPT-simulated users, Diff4Rec performs better than CL4SRec. Considering that this evaluation is not based on the pre-collected test data,  Diff4Rec is less likely to over hinge on users history.\n\n   [a]Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023). Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442.\n\nWe're grateful for your insightful suggestions, which have enriched our evaluations and inspired us for future directions.\n\n\n\n\n\n\n\n\n\n\n\n\n"}, {"review_id": "PZ97Q8MNnf", "reviewer": "Reviewer_VUDN", "summary": "This paper describes a new learning-to-generate paradigm for sequential recommendation based on diffusion models. The authors discuss the limitations of previous approaches in sequential recommendation, where a recommender model learns to classify user preferences based on positive and negative item samples. The paper highlights two inherent limitations of this approach: (1) it differs from human behaviour, which involves knowing an ideal item and selecting potential matches, and (2) the classification is limited to the candidate pool, which may contain noisy or easily distinguishable negative samples, diluting the preference signals. This paper proposes Diff4Rec, a new learning-to-generate paradigm based on guided diffusion, which is able to circumvent the limitations of existing approaches. Empirical results on benchmark datasets show the effectiveness of Diff4Rec.", "questions": "- In Diff4Rec, the max number of interactions is set to 10. Why not choosing a larger number of historical interactions? Is it due to Diff4Rec running too slow, or is it because of Diff4Rec not performing well when the sequential length is large?\n- The abstract in the OpenReview console is not consistent with the submitted paper. In OpenReview, the model is termed SRDiff, while in the paper it is termed Diff4Rec.", "limitations": "The authors have done a good job discussing the limitations of Diff4Rec. Diff4Rec has a very high computational cost, which limits its applicability. When considering using Diff4Rec in application, it is important to analyze the tradeoff between computational need and recommendation performance. However, Diff4Rec offers a new view of reshaping sequential recommendation as an item generation task, which has the potential to inspire more research in this direction. This outweighs the existing limitations of Diff4Rec.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The authors analyzed the potential limitation of using negative sampling when training classification based recommendation models, and proposed a generation based framework using diffusion models, which is technically sound.\n- Diff4Rec is able to outperform a number of state-of-the-art baseline alternatives.\n- Code is available, which helps reproduce the results.", "weaknesses": "- The authors analyzed the potential limitations of using negative sampling when training classification based recommendation models. In practice, however, a lot of models can be trained without using negative sampling at all. They directly use a softmax over all possible items to obtain the probability of each item being the next item. These softmax based approaches, in my opinion, are closely related to the spirit of Diff4Rec, i.e., by not relying on negative sampling and treat all items other than the next item as a whole. Thus the authors need to explain why Diff4Rec is superior to these approaches.\n- Diff4Rec can be very slow in both training and inference stages. In addition, the authors should analyze the time / computational complexity of Diff4Rec and present them in the paper.\n- The number of compared approaches is relatively small. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> 1. *Why Diff4Rec is superior to softmax-based approaches.*\n\n   We appreciate your comments, but respectfully emphasize the fundamental distinction between Diff4Rec and softmax-based approaches. Conceptually, Diff4Rec is a learning-to-generate paradigm that casts off negative sampling, while softmax-based approaches are in the learning-to-classify paradigm that contrasts the positive items against the explicit or implicit negatives.\n\n   We focus on the softmax loss with implicit negatives, which may not conduct the negative sampling explicitly, but implicitly treat all items (excluding the next positive item) as negatives [a]. Specifically, the softmax loss can be derivated as follows (cf. Equation 14 in [a]):\n\n   $L_{softmax} = -\\sum\\limits_{(c, i) \\in S} \\ln \\frac{\\exp(\\hat{y}(i|c))}{\\sum\\limits\\_{j \\in I}\\exp(\\hat{y}(j|c))} = -\\sum\\limits_{(c, i) \\in S}\\left[ \\hat{y}(i|c) - \\ln \\sum\\limits_{j \\in I}\\exp(\\hat{y}(j|c))\\right],$\n\n   where $S$ is the observed interactions. Clearly, it treats the observed interactions as positives, while implicitly designating unobserved interactions as negatives. Then, it optimizes their margin appropriately.\n\n   Furthermore, work [b] interprets softmax loss from the perspective of contrastive learning:\n\n   $L_{SSM} = -\\sum\\limits_{(u, i) \\in D} \\log \\frac{\\exp(f(u, i))}{\\exp(f(u, i)) + \\sum\\limits_{j \\in N}\\exp(f(u, j))},$\n\n   which aligns with softmax loss when $N$ is the set of all items apart from the next positive item.\n\n   Concurrently, softmax loss can be interpreted by InfoNCE, a well-known objective in contrastive learning, by setting the temperature $\\tau = 1$:\n\n   $L_{InfoNCE} = -\\sum\\limits_{(u, i) \\in D} \\log \\frac{\\exp(f(u, i) / \\tau)}{\\exp(f(u, i) / \\tau) + \\sum\\limits_{j \\in N^-}\\exp(f(u, j) / \\tau)},$\n\n   and InfoNCE is acknowledged to discriminate between positive sample (u, i) and negative ones (u, j).\n   The training of Diff4Rec, as shown in Algorithm 1, involves no implicitly usage of negative samples, compared to softmax.  The comparison can be summarized as:\n\n   method|observed interactions|unobserved interactions|treatment\n   :--:|:--:|:--:|:--:\n   softmax |positive|negative (explictly)| contrastive learning\n   Diff4Rec|positive|not use|recover observed interactions by diffusion\n\n   [a] Rendle, S. (2021). Item recommendation from implicit feedback. In Recommender Systems Handbook (pp. 143-171). New York, NY: Springer US.\n\n   [b] Wu, J., Wang, X., Gao, X., Chen, J., Fu, H., Qiu, T., & He, X. (2022). On the effectiveness of sampled softmax loss for item recommendation. arXiv preprint arXiv:2201.02327.\n\n---\n\n> 2. *The authors should analyze the time / computational complexity of Diff4Rec and present them in the paper.*\n\n   Thank you for the insightful comment. Indeed, the diffusion model's inference phase can be time-consuming—an intrinsic limitation of diffusion. While Diff4Rec reframes sequential recommendation as a learning-to-generate task using diffusion, it is not immune to this limitation. However, we're optimistic that as diffusion model research advances, emerging efficient inference algorithms will mitigate this slow-inference concern. Thus, Diff4Rec stands to benefit from these advancements in diffusion.\n\n   We recognize the importance of analyzing Diff4Rec's computational complexity. Consequently, we provide Table 5 in the PDF, detailing the time taken at each stage. We can observe that the training efficiency of Diff4Rec and SASRec is comparable, since diffusion would sample a step for training as described in Line 5 of Alg. 1. The inference stage of Diff4Rec is more time-consuming than SASRec. Moreover, we show the trade-off between performance and time cost of inference *wrt* total diffusion steps in Figure 8.\n   We'll definitely incorporate these computational costs in the revision. We're grateful for your insightful remarks.\n\n---\n\n> 3. *Why not choose a larger number of historical interactions than 10?*\n\n   Thank you for raising this point. Our decision to set the maximum number of interactions at 10 was not due to performance or efficiency constraints of Diff4Rec. We thought 10 is an appropriate number. Indeed, Diff4Rec's ability to handle longer sequences is not compromised. In Diff4Rec, historical interactions are initially encoded using a transformer encoder to obtain a 1-D representation, which then guides the diffusion process. As such, the length of retained historical interactions has a limited impact on learning efficiency. To provide further clarity, we conducted experiments with larger sequential lengths (20 and 30). The results are in Table 6&7 in the PDF, which shows similar trends with the results in Table 1. Meanwhile, the training and inference time show little difference with Table 5.\n\n   We hope the results address your concerns.\n\n---\n\n> 4. *The number of compared approaches is relatively small.*\n\n   Thank you for the suggestion. We've incorporated three additional baseline models (i.e., (DiffRec [36], DiffuRec [19], and CDCR\nec [20]) in Table 4 of the PDF, inclusive of recent work applies diffusion to recommendation. Note that these models still operate under the learning-to-classify paradigm. As described in Lines 101-110, they necessitate the use of negative samples.\n\n---\n\n> 5. *The abstract in the OpenReview console is not consistent with the submitted paper.*\n\n   Thank you for bringing this issue to our attention. We apologize and will perform careful proofreading in the revision.\n\n---\n\n> 6. *It is important to analyze the tradeoff between computational need and recommendation performance.*\n\n   Thank you for your feedback. As mentioned in Point 3, the balance between computational cost and applicability is a fundamental challenge of diffusion models. Following your suggestion, we've presented experimental results and analysis. Moreover, your comments underscore our belief that, despite its limitations, Diff4Rec has the potential to spur further research in this domain.\n\n\n\n\n\n\n\n"}, {"review_id": "YZJG80QdLG", "reviewer": "Reviewer_b6yV", "summary": "This paper presents an innovative paradigm shift in sequential recommendation systems. It departs from the traditional learning-to-classify approach, which includes negative sampling, to a learning-to-generate model, proposing a novel system called Diff4Rec, grounded on guided diffusion. This approach is predicated on the observation that users imagine an ideal or \"oracle\" item after several interactions with a recommendation system.\n\nThe authors posit that Diff4Rec can generate these oracle items, modeling the underlying item-generation distribution through a diffusion process, and is not limited to a predefined candidate set. This approach allows for discarding negative samples, which previous models could not accomplish, as it models the data-generation process directly.\n\nThe paper suggests that Diff4Rec effectively mirrors human behavior better than previous systems by capturing a user's preference more accurately and directly, avoiding the noisy or easy supervisions from negative samples. The authors evaluate Diff4Rec through various experiments, where it demonstrates consistent improvements over existing sequential recommendation models. In conclusion, this paper introduces a transformative methodology in sequential recommendation, potentially broadening the capabilities of recommendation systems.", "questions": "1. A key success of the diffusion models in image generation is that it can encode some of undetermined features in the latent space so that random initializer helps to randomly sample those features in additional to the guidance signals. This implies, to a given user and interaction history, we can always generate different oracle items from different random seeds. How will this help on top item retrieval? For example, we can generate different oracle items and make a pooling to get an ensemble of oracles and then retrieve from that. How will this be different from retrieving one for each generated oracle? It would be very interesting for users to investigate on this end. \n2. Can authors add one / more of the other diffusion model based recommendation model (19,20,21,36) results as a baseline? Tells us better the effect of with/without negative sampling.", "limitations": "1. Authors may disagree on whether selecting the nearest items on the embedding space is a necessary step for recommendation. I could also imagine a recommendation with a generative item, for example, a LLM decoder to generate recommended results from the oracle item embedding generated with the diffusion result. Maybe authors could discuss how to fetch / generate the recommended item(s) from the oracle item in the main text.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Raise a new learning-to-generate paradigm for recommendation using relative new diffusion model approach.\n2. Good reproducibility: Code is released for readers; Hyperparameter settings are available.\n3. Good to explain what each step is doing in the Algorithms.\n4. Improvements over the baseline appear significant. But statistical significance analysis is missing.", "weaknesses": "1. In terms of recommendation, an important and indispensable step is not discussed and stated in the algorithm. After an oracle item is generated, to fetch one / or some items to recommend to users, we need to find the top nearest items to the oracle item in the embedding space. Though this step is described in the Experiment, it is fully ignored in the Method section. I understand the Method part described mainly the learning-to-generate paradigm. But this step is needed to complete the recommendation task. Moreover, to retrieve top nearest items, how to encode properly the embedding space for items becomes also a relevant question.\n2. Might be good to list out the main contributions of the work.\n3. Please avoid abusing the footnotes. Footnotes are good for external links and websites. If you do think footnotes are providing important information, please try to integrate them in the main text instead, e.g. 3 and 4. And try to avoid the controversial remarks irrelevant to the main contents, like 2: people working on learning to rank may not fully agree on 2.\n4. In table 1, if authors meant to have %, better to add (%) after HR@20 and NDCG@20. It’s easy to be confused, even when I read the caption.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> 1. *The step to retrive recommendation list is needed to complete the recommendation task. Moreover, to retrieve top nearest items, how to encode properly the embedding space for items becomes also a relevant question.*\n\n   Thank you for highlighting this point. We recognize the importance of identifying the top nearest items to the oracle item in recommendation tasks. We regret the oversight of detailing this in the Experiment section instead of the Method section. In the revision, we will introduce Section 4.2.3, titled \"Retrieval of Recommendation Results,\" to address this aspect. We hope this addition clarifies your concerns.\n\n   Regarding the encoding of the item embedding space, we utilize a Transformer encoder in Diff4Rec to process item sequences. We concur that the choice of encoding strategy for this space is pivotal. To explore alternatives, we conducted experiments replacing the Transformer encoder with a GRU, with the results presented below:\n\n|| Yoochoose|Yoochoose| KuaiRec| KuaiRec|Zhihu|Zhihu \n:--:|:--:|:--:|:--:|:--:|:--:|:--:\n||HR@20(%) | NDCG@20(%)|HR@20(%)|NDCG@20(%)|HR@20(%)|NDCG@20(%)  \nTransformer encoder|4.78|  2.23 |5.26|4.11|2.26|0.79 \nGRU encoder|4.48|1.92|5.58|5.17|2.05|0.73  \n\n---\n\n> 2. *Might be good to list out the main contributions of the work.*\n\n   Thank you for this advice.  We will list out the main contributions in the revised version, focusing on: (1) Diff4Rec reshapes sequential recommendation as an oracle item generation task, (2) Diff4Rec does not require negative samples, since it explores the underlying distribution of observed interactions with diffusion model, and (3) We conduct experiments on three datasets to show the effectiveness of Diff4Rec. We believe this makes the presentation of our work more clear!\n\n---\n\n> 3. *Please avoid abusing the footnotes. *\n\n   We apologize for the abuse of footnotes. We will integrate the key point of Footnote 3 and 4 into the main text in the revised version. \nWe understand Footnote 2 is controversial since learning-to-rank has difference with learning-to-classify. What we would like to convey was to highlight their similarity in requiring negative samples for recommendation tasks.  In the revision, we will clarify their similarity more precisely in the main text, avoiding any controversial statements.  Thanks again for reminding us of this issue.\n\n---\n\n> 4. *better to add (%) after HR@20 and NDCG@20.*\n\n   Thank you for this advice.  We will add (%) after HR@20 and NDCG@20 in Table 1 in the revised version.\n\n---\n\n> 5. *For a given user and interaction history, we can always generate different oracle items from different random seeds. How will this help on top item retrieval? For example, we can generate different oracle items and make a pooling to get an ensemble of oracles and then retrieve from that. How will this be different from retrieving one for each generated oracle? It would be very interesting for users to investigate on this end.*\n\n   Thank you for this question.  We feel that this question is more about the understanding of diffusion model, and we are glad to share our understanding of it. \n\n   In image generation, if guidance signals are involved,  the generated images may be different, but they would center around the guidance signal. Even though they are different in RGB, it is hard to say that they are always different in certain latent spaces with certain measurements. In general, the generation process of diffusion is about removing certain noise from a pure Gaussian sample, and the guidance signal provides more delicate information about which part of noise to remove. Back to Diff4Rec, the generated oracle items can be different with different random seeds, but they are all recovered from Gaussian noises with the guidance of the same behavior sequence. Therefore, they could be similar in certain latent spaces with certain measurements, which, we believe, is an open scientific topic to explore.\n\n   Moreover, we do find it insightful to generate different oracle items and make a pooling to get an ensemble of oracles and then retrieve from that. We conduct mean pooling of 5 different oracle items, and the result is shown as follows:\n\n|| Yoochoose|Yoochoose| KuaiRec| KuaiRec|Zhihu|Zhihu\n:--:|:--:|:--:|:--:|:--:|:--:|:--:\n||HR@20(%) | NDCG@20(%)|HR@20(%)|NDCG@20(%)|HR@20(%)|NDCG@20(%)\nDiff4Rec|4.78|2.23|5.26|4.11| 2.26|0.79|\nensemble|4.83|2.41|5.47|4.32|2.32|0.78|\n\n   The reason may be that the ensemble of oracles can effectively reduce the variance.\n\n---\n\n> 6. *Can authors add one / more of the other diffusion  based recommendation model (19,20,21,36) results as a baseline?*\n\n   Thank you for the suggestions. We have added models from references [19], [20], and [36] as baselines for comparison in Table 4 of the PDF. Of these works, only [36] provides open-sourced code that allows direct reproduction. For [19] and [20], we carefully implemented their methods based on the paper's details to enable fair comparisons. We are still working to reproduce the approach [21] and will include it in the subsequent stages.\n\n---\n\n> 7. *A  LLM decoder could generate recommended results from the oracle item embedding. Maybe authors could discuss how to fetch / generate the recommended item(s) from the oracle item in the main text.*\n\n   Thank you for this insightful comment! After analyzing this comment, we do believe it is more elegant to decode the oracle items explicitly, possibly by LLMs. \n   We do some initial attempts and would like to finetune Vicuna, an open-sourced LLM to interpret the oracle item. However, we find that our available GPU resources currently can not afford the finetuning.  We will keep investigating this promissing direction. Moreover, in the revised version, we will discuss more about how to fetch the recommended item(s) from the oracle item, and what LLMs can do in the process. Hopefully, it can inspire more research in this direction! Thanks again for this insightful comment!\n\n\n\n\n"}]

Author rebuttal:
We thank the reviewers for their insightful and positive feedback! We are encouraged that they find Diff4Rec introduces a transformative methodology in sequential recommendation (Reviewer $\color{Blue}\text{b6yV}$), achieves much superior performance compared to state-of-the-art baseline alternatives (Reviewer $\color{Goldenrod}\text{RbjC}$ and Reviewer $\color{Red}\text{VUDN}$), potentially broadening the capabilities of recommendation systems (Reviewer $\color{Blue}\text{b6yV}$),  and has the potential to inspire more research in this direction (Reviewer $\color{Red}\text{VUDN}$).

We would also like to express our gratitude to the reviewers for highlighting that Diff4Rec presents an innovative paradigm shift in sequential recommendation systems (Reviewer $\color{Blue}\text{b6yV}$), effectively mirrors human behavior (Reviewer $\color{Blue}\text{b6yV}$), and is technically sound (Reviewer $\color{Red}\text{VUDN}$).

After carefully analyzing the reviewers' comments, we are more than glad to find these comments very insightful. We respond to the comments point by point in the rebuttal and provide a PDF containing the comparison with more baseline models and the demonstration of the efficiency of Diff4Rec. We hope that our response can address the concerns raised by the reviewers.

Once again, we sincerely thank the reviewers for their valuable feedback and insightful suggestions, which undoubtedly contribute to enhancing the quality of our work.

We eagerly anticipate the ensuing discussions in the next phase!



Meta-review: This paper presents Diff4Rec: a diffusion based model for sequential recommendation. The basic idea is straightforward: in sequential recommendation setting, we are ultimately interested in estimating the conditional density p(next item | historical items), which has mostly been modeled under a classification task. Instead, Diff4Rec considers a generative process where we estimate this density in a continuous embedding space with a diffusion model and sample from it, which will generate an embedding that doesn't correspond to any actual item but we can make recommendations based on the proximity in the embedding space. The paper received borderline positive scores. All the reviewers agree that the learning-to-generate perspective of Diff4Rec is interesting and the empirical results seem convincing. However, the reviewers also shared some concerns around the practicality, given the expensive generation process of the diffusion models, as well as the argument of "no negative sampling" for Diff4Rec. I read the paper myself and I can see how Diff4Rec can get away with not using negative examples, even though I wouldn't necessarily consider this paradigm new, since this argument is not that dissimilar to the "generative vs discriminative ML" arguments from back in the days, which I hope the authors can acknowledge. 

Overall considering this is, as far as I know, the first work on using a diffusion model as a generative model for sequential recommendation (as opposed to a discriminative model) with reasonably good empirical support, even though the technical contribution is relatively limited, I am OK with accepting this paper (albeit very weakly). However, I want the authors to tune down some of the claims unless more justification can be provided. For example, I especially find the authors' argument about an "oracle item" not very scientific: quoting the abstract "...(1) it may differ from human behavior that user usually imagine an oracle item in mind and selects potential items matching the oracle; and (2) the classification is limited in the candidate pool with noisy or easy supervision from negative samples, which dilutes the preference signals towards the oracle item." Is there any real evidence backing these (especially the human behavior part) up? Or is this just to match the generative process of Diff4Rec? Similarly, the argument of "learning-to-generate" vs "learning-to-classify" is not something groundbreakingly new either -- as I mentioned before, this dates back to before deep learning as "generative vs discriminative models".  

The reviewers have also brought up some comments/suggestions which I hope the authors can incorporate into the final version of the paper.
Decision: Accept (poster)