Breaking Determinism: Fuzzy Modeling of Sequential Recommendation Using Discrete State Space Diffusion Model
Authors: Wenjia Xie
Hao Wang
Luankang Zhang
Rui Zhou
Defu Lian
Enhong Chen
Venue: NeurIPS 2024 poster (2024)
Keywords: Sequential Recommendation
Discrete Diffusion Model
Primary area: machine_learning_for_other_sciences_and_fields
TL;DR: Our paper introduces DDSR, a novel model leveraging discrete diffusion processes to enhance the accuracy of sequential recommendations by capturing the inherent randomness of user behavior.

Abstract: Sequential recommendation (SR) aims to predict items that users may be interested in based on their historical behavior sequences. We revisit SR from a novel information-theoretic perspective and find that conventional sequential modeling methods fail to adequately capture the randomness and unpredictability of user behavior. Inspired by fuzzy information processing theory, this paper introduces the DDSR model, which uses fuzzy sets of interaction sequences to overcome the limitations and better capture the evolution of users' real interests. Formally based on diffusion transition processes in discrete state spaces, which is unlike common diffusion models such as DDPM that operate in continuous domains. It is better suited for discrete data, using structured transitions instead of arbitrary noise introduction to avoid information loss. Additionally, to address the inefficiency of matrix transformations due to the vast discrete space, we use semantic labels derived from quantization or RQ-VAE to replace item IDs, enhancing efficiency and improving cold start issues. Testing on three public benchmark datasets shows that DDSR outperforms existing state-of-the-art methods in various settings, demonstrating its potential and effectiveness in handling SR tasks.

Breaking Determinism: Fuzzy Modeling of Sequential Recommendation Using Discrete State Space Diffusion Model
Wenjia Xie Hao Wang Luankang Zhang Rui Zhou Defu Lian Enhong Chen University of Science and Technology of China & State Key Laboratory of Cognitive Intelligence xiachulu@mail ustc. edu. en, wanghac3@ustc. edu. cn, 2hang1k5mmml ustc. edu. en, zhou_rui runn_ruieee mui@mail........m www.usuc.es edu. en, www..ccclaannefuuiiieeeee www.llnffuuuut.....m
en, cheneh@ustc. edu. cn
Abstract
Sequential recommendation (SR) aims to predict items that users may be interested in based on their historical behavior sequences. We revisit SR from novel information-theore perspective and find that conventional sequential modeling methods fail adequately capture the randomness and unpredictability of user behavior. Inspired by fuzzy information processing theory, this paper introduces the DDSR model, which uses fuzzy sets of interaction sequences overcome the limitations and better capture the evolution of users' real interests. Formally based on diffusion transition processes in discrete state spaces, which unlike common diffusion models such as DDPM that operate in continuous domains. It is better suited for discrete data, using structured transitions instead arbitrary noise introduction to avoid information loss. Additionally, to address the inefficiency of matrix transformations due the vast discrete space, we use semantic labels derived from quantization RQ- VAE to replace item IDs, enhancing efficiency and improving cold start issucs. Testing on three public benchmark datasets shows that DDSR outperforms existing otate-of-the-art methods in various settings, demonstrating its potential and effectiveness in handling SR tasks.
Introduction
For long time, sequential recommendation (SR) has been attracting increasing attention due its excellent performance and significant commercial value (Chen et al. [2020), Qiu et al. [202T). Yin jet al. 2024b), Han al. (2044). Unlike traditional collaborative filtering or certain graph-based methods (Wang et al. [2019]. Zhang et al. [2024], Wang et al., Tong et al. (2024)) SR systems emphasize the inherent dynamic behaviors of users rather than relying solely on structured data (Chen) al. (202 Ma et al. [2020]. Cen et al. (2020). This approach enhances the accuracy of personalized recommendations, allowing for more precise tracking of changes in user interests and needs. Typical deep learning-basse based SR models, such as those utilizing CNN, RNN, and Transformer architectures (Tang and Wang 2018). Hidasi et al. 20155) Kang and McAuley 2018b)), have achieved remarkable success in modeling user historical interaction data.
However, these methods are formalized models based on narrow information theory assumption (Shannon 1948), ). which only acknowledges determinism (Rosas et al.) (2020). They assume that all phenomena strictly adhere to mechanical laws and that the states of motion of objects at different times can be uniquely determined. In reality, however, user behavior is characterized by randomness and unpredictability. They might change their mind about buying a down jacket due to sudden warm-up, they might impulsively buy desserts due to a breakup. As illustrated on the left in Figure M user's interest at any given moment might focused on 'some items' with blurred boundaries, only converging finally when the user makes selection.
Hao Wang is the corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Item
Descriptins
5 Semantic IDs
Transitions between item
Roses repensy teacup and Saucer
(12,
Artiqu ditve grean Tnser teacup.
(12, 47,
User Historicall Interactions
Discrect Diffusion
Daking mould hesr white ehocolatee
(17, 37, 69)
Item to be recommended
Seg2seq model
(17 41)
Discrete space
fuzzy sets
Figure 1: Illustration of DDSR constructing fuzzy sets and incorporating semantic IDs enhance scquential recommendations. real-world scenarios, a user's final choice often reflects their immediate interests (left subfigure). We reconstruct the true evolution of interests by constructing fuzzy sets for cach item in the interaction sequence (middle subfigure). The right subfigure provides an overview of the process of generating semantic IDs for recommendations based item-related descriptions.
Although increasing the sample size is an effective strategy to address the above issue. in reality, the data recommendation systems usually quite sparse (He and McAuley (2066). limiting the practicality of this strategy. Inspired by the theory of fuzzy information processing (Tanaka et al. 1976). Tanaka and Sommer (177]) we believe that making the absolute membership relations in traditional sets more flexible is another effective way to solve the problem. In other words, it is not necessary to strictly limit the modeling of user interests to the items they have interacted with. Therefore, we propose using a diffusion model (Ho et al. (2200) for fuzzy modeling of user interests, which enhances the model's performance by introducing perturbations during the training process.
We have noticed existing work that introduces diffusion models into SR (Xie et al. (024)), ), such as DiffuRec (Li et [2023b]) and DreamRec (Yang al. [2023]). which focus on Gaussian diffusion processes operating within continuous state space. They add Gaussian noise to the embedded representations of candidate items for recommendation through a forward diffusion process until the noise reaches a pure state (standard normal distribution). Subsequently, they iteratively sample from this noise using a reverse denoising process guided by historical interaction information recover meaningful representations and recommend items most similar to these representations.
However, unlike our desire fuzzily model interaction sequences, the aforementioned methods follow the form of diffusion models in the image domain and operate on candidate items. They introduce the crucial sequence information merely as conditional information, without leveraging the diffusion model's 's performance on it. On the other hand, these methods relax discrete interaction data into a continuous space and introduce noise, which may lead to distortion loss of meaning in the original discrete space, as the addition of noise could push data points away from any meaningful discrete state. Therefore, we hope that state transitions occur under discrete conditions for the entire interaction sequence, which is discrete diffusion. Based on this, we have proposed our DDSR (Discrect Diffusion Sequential Recommendation model), which uses directed graph to model sequential recommendation. In this model, all interaction items are viewed nodes, and transitions between items are treated as directed edges. Discrete diffusion is used to enable structured transitions of nodes, with the resulting new sets treated as fuzzy sets, shown in the middle of Figure [T] By designing the transition matrix, we can achieve uniform transitions or importance-burt based transitions for the nodes, ensuring controllability. In Section 3.3 we theoretically demonstrate the reliability and effectiveness of modeling on these generated fuzzy sets, based on the principles of information difffusion. During the inference stage, we refer to the sampling formula for discrete diffusion but start from the historical interaction sequence rather than from noise, iteratively generating refined results.
Furthermore, we have found that the excessive number of items involved in the recommendation problem leads to a high-dimensional transition matrix, resulting in inefficient diffusion transitions. Additionally, item IDs themselves do not contain any prior information, which poses challenge in determining beneficial transition directions. To address this issue, we have further introduced semantic tags to replace meaningless item IDs, using quantization techniques and VQ-VAE to derive

---

these tags from semantic information, thus reducing the size of the discrete space. We will provide specific details on how this can be achieved in in. and vivid illustration of this is given on the right side of Figure[] Simultaneously, the introduction of semantic information has enhanced the model's generalization capability and effectively solved the cold start problem. We conducted extensive experiments on three public benchmark datasets, comparing DDSR with several state-of-the methods. The results demonstrate that DDSR significantly outperforms baseline methods in various settings and effectively handles cold-start recommendations.
2 Related Work
2.1 Sequence Recommendation
SR suggests potential subsequent items based on users' historical interaction records (Yin et al.
[022 Wang al. 2024)) ). Early rescarch primarily relied on Markov chains and matrix factorization Techniques for recommendation (He et al. 20166). ). However, with the development of deep learning,
efforts such GRU4Rec (Hidasi et al. 2015], Caser (Tang and Wang (2018), and others have
focused on designing neural network models to capture sequential dependencies user behavior sequences. The introduction of the Transformer architecture (Vaswani al. (017]) in SASRec (Kang and McAuley 201 (b)) pioneered SR and quickly became the mainstream method in the field.
Additionally, BERT4Rec (Sun et al. (0199) utilizes bidirectional encoders capture bidirectional dependencies in sequences, using a masked language model to predict the user's next action.
crucial for obtaining Recent studies have shown that high-quality high-dimensional embeddings accurate recommendation results (Hou et al. [2022). Wang et al. (202T). To this end, researchers striving leverage the rich attribute information of items improve data representation. For example, TransFM (Pasricha and McAuley| (2018) introduces arbitrary real-valued features through factorization machines, while S3-Rec (Zhou et al. (2020) designs four self-supervised learning tasks as pre-training objectives to learn context-aware data representations with attribute awareness. Furthermore, researchers like Hou al. [2022]. Zhao 2022 Harte et al. (0233 further utilize pre-trained language models (Yin et al. (2024a] to process item description texts, obtaining uni- versal item representations with rich semantic information
enhance the performance (Wu al.)
(2024)) ). VQ-Ree (Hou et al. [2023) ) and TIGER (Rajput et al. [2024]) further employ quantization techniques (Jacob et al. [2018) and RQ-VAE (Lee al. [2022] obtain tokenized semantic IDs for recommendations, replacing semantic embeddings.
2.2 Diserete Diffusion Models
Diffusion models, inspired by non-equilibrium thermodynamics, have been introduced and demon- strated significant results in fields such as computer vision, sequence modeling, and audio processing (Dhariwal and Nichol RRasul et al. [2021]. Ho et al. [2022). ). Most diffusion models are based on the Denoising Diffusion Probabilistic Model (DDPM) proposed by Ho et al. [2020]. as well as the score-based generative models (SGMs) proposed by Song et al. [2020], targeting continuous data domains. We provide detailed descriptions of DDPM and SGMa in Appendix D facilitate compar- isons with the discrete diffusion approach
employ. Diffusion models in discrete state space first described in Sohl-Dickstein et al. (2015) and later applied to text and image domains in D3PMs (Austin et al. 2021]). VQ-Diffusion (Gu et al. (2222) utilizes them to eliminate unidirectional bias in text-to-image generation.
3 Discrete Diffusion Process of DDSR
In this section, we present the problem definition (Section 3.1 and illustrate how item sequences undergo discrete diffusion obtain the corresponding fuzzy sets (Section 3.2). Finally, the effective- ness of this fuzzy modeling is theoretically demonstrated (Section 3.3) Please note that the actual diffusion and inference in DDSR occur at the semantic ID level, but this chapter discusses items.
3.1 Problem Statement
Let U/ be the set of users and be the set of discrete items in the dataset, [2/] and |V] represent the number of elements in their respective sets. For each user u E U, Vin-1 [U1, U2, .... mn
3

---

represents his historical interaction sequence sorted by timestamp. The goal of the model is to predict the next item Un that the user is most likely to interact with. To facilitate better discrete diffusion and for the convenience of subsequent theoretical derivations, we model each user's interaction sequence as a directed graph g". this graph, each item represented by a semantic IDD is regarded as a node, while transitions between items are viewed as directed edges. Specifically, an edge exists from iy to Vy if and only if 0j is the next item interacted with by the user after Us.
3.2 Node Diffusion Transition
A typical diffusion model transforms data Zi ~ (((0)) into a sequence of gradually noisier latent
variables 21.7 1 22, .... 2T via forward process ((o) = III g(zr|xx--)) In diffusion models within continuous state spaces, the forward distribution is typically set with (( (z,-)) 1) = N - 3e 1) as hyperparameter controlling the level of noise added at each step. As the number of time steps 7' approaches infinity, .T converges to standard Gaussian distribution. Beyond the limitations mentioned above, information loss due to diffusion into pure noise is another reason for unstable training and inadequate alignment of continuous diffusion with SR.
(D))
Continuous diffusion always operates on embeddings, while in diffusion models within discrete state spaces, categories are directly transformed. Transition matrices [Qahy = q(x;; = = i) are used to describe the probability of single-step diffusion transitions, where and j represent categories within the domain. Denoting the one-hot version of x with the row vector (bold), then the one-step transition probabilities can be expressed as:
[(,) = Cat( 1Q.)
where Cat(æ:p) p) is the categorical distribution corresponding to the one-hot row vector with probabilities given by the row vector p. and x;- Q₂ understood as a row vector-matrix product. Starting from Lo, we obtain the following t-step marginal and posterior at time - 1:
(xx mo) = Cat (a:; p 2o
with Q. = Q,92--2.0 ...... Q+:
(2)
We take the set V as the domain in SR. Each v; in the interaction sequence is represented a one-hot encoding æ.. Using the transition form defined by enables transitions to other nodes, denoted as z in the domain with a certain probability any time step l. Unlike continuous diffusion, which only allows noise addition, discrete diffusion models offer the advantage of controlling the data blurring process by selecting the transition matrix. Here, we present two strategies to select transition matrices, i.e. Uniform transition and Importance transition.
Uniform transition. Similar to the study by HHogeboom et al. [2021], the natural idea is maintain
nodes with certain probability 3, E (0, 1) unchanged. In contrast, in other cases, nodes are randomly transformed into any other node in the domain with equal probability (1 3.)(((V) - 1). That
(1 3.)((1) -) [3.
1) if if if ifi=j"
Quliy
Uniform transfer can regarded as a special case of linear information allocation, thus theoretically affected by the size of the discrete space. It can compute the cumulative product Q, in closed form. Importance transition. For data with certain prior knowledge, we propose transitioning between more similar nodes rather than uniformly transitioning to any other state, thus defining the matrix:
exp( ((,,, 22,/2a0))
if i i
Σ Enev expl -22,/20")
1Q
Ek= 1.1sr if i=j
Here, dy represents the distance between item U/ and Uy calculated using the square of the Euclidean distance. The parameter a² denotes the variance of the diffusion process. Consequently, the transition probabilities cannot be solved in closed form; instead, they can only be updated alongside the embeddings in the model. The importance transfer matrix adheres to the Gaussian information
diffusion function f(x) -e 2."h, Therefore, it remains unaffected by the number of discrete points but necessitates the sample point distribution to closely resemble a Gaussian distribution.
ev2x"

---

This modeling approach appears to align more closely with our intuition, as a user's interests at given moment often form a cluster of similar nodes (items). Only upon the user's final selection of an item does the 'neighborhood" converge to single data point. This point represents the representative of the interest cluster and the ambiguity in information naturally dissipates. In recommendation tasks, we can only access the user's s final choice at each moment, without knowledge of the interest cluster, reflecting incomplete knowledge. Regardless of the transition method employed, the key lies in transitioning the sample space from incomplete to complete, as detailed in the subsequent section.
3.3 Completeness and Reliability
Here, we aim to demonstrate that the discrete diffusion spaces generated by the two methods in Section [.2 are completions of the original space, and the models constructed on these fuzzy sets are solvable and effective. We first provide the formal definition of a complete sample space.
Definition 1. Let W denote the sample space. For any sample W E W. if W is complete, i.e., unbiased estimates can be obtained through certain mathematical processing, then W is called complete sample space; otherwise, it is called an incomplete sample space.
In SR, W is a user's behavior sequence; W is all possible combinations of these behavior sequences; the domain V is all items in the dataset. Datasets in SR systems do not form complete sample space, as they often consist of incomplete interaction data and potential selection biases. The principle of information diffusion ensures that when the given sample is incomplete, there exist reasonable diffusion functions that can improve non-diffusion estimates. Below we define information diffusion.
Definition 2. An information diffusion about a set W is defined by mapping μ W x V + [0, 1]. satisfying the following conditions:
(1) Vuy E W. vy is the observed value of wy. then p(ws, 0z)) = supbev p(wy, Vwy W. u(w(.,) v) decreases as ||uy - will increases.
3) Vw E W. En ((w,v)dv
The diffusion estimates obtained using uniform transition and importance transition, defined in Section 3.2 clearly adhere to Definition 2 To illustrate that the space resulting from discrete state transitions provides more information than the original state space, it is necessary further demonstrate that this space serves as a completion of the original space. In other words, the new metric space complete, with the original metric space serving as its dense subspace. This will be more precisely discussed in the following theorem.
Theorem 3.1. After information diffusion, the subsequent space must be an entirely separable metric space. Any model constructed in this space will assuredly possess a solution.
Proof in Appendix A. According to Theorem 3.1 since the space after information diffusion is equidistant isomorphism with the original space, it can be used to replace the sample space with insufficient information in SR establish a model. On this complete space, predictive models are solvable, demonstrating that modeling on fuzzy sets is a reasonable and effective approach.
4 Learning and Inference of DDSR
4.1 Obtaining Semantic IDs
As mentioned in Section the indices i and in the transition matrix [Qr]is represent categories in the discrete space, making Q, a two-dimensional matrix with dimensions equal to the size of the discrete space. However, in recommendation tasks, the number of items involved can reach tens of thousands. posing significant challenge in terms of computational resources if we were to use all item IDs as the discrete state space. Inspired by VQ-Ree and the recently proposed Tiger model by Google, we attempt to train recommendation models using semantic IDs instead of item IDs. A semantic ID is a codebook of length m. Assuming we set the size of the codebook K, the entire codebook can represent T categories. Though we set each code from a different codebook, the state space only needs m nodes to store them. Additionally, the use of semantic IDs further introduces semantic information, addressing the scarcity of information inherent in recommendations, while also allowing the model to extend to unseen items, thus enabling cold-start recommendations. We provide the specific method for obtaining semantic IDs in the Appendix B
5

---

Algorithm Training of DDSR.
Input:
historical interaction sequence U1m--1 Cl.n -o 1:lmmi target item on. Cap transition
matrix Q; Approximator (o(-).
Output: well-trained Approximator fe(-).
While converged do:
1: Sample Diffusion Time: t [0, ...... Te:
2: Calculate t-step transition probability; Q: Q1Q2 Q; ..
3: Convert Ca;1om to one-hot encoding x #n;mm
4: Obtain the discrete state #11m after steps by Equation 2 thereby obtaining the "fuzzy set'
5: Modeling 12:m;1:m based on 'fuzzy sets' through Equation
6: Take gradient descent step on VLCE (02:n;1:ms C2m;1am)
Algorithm 2 Inference of DDSR.
Input: historical sequence Cin- 1,1mi well-trained Approximator fe(-); sampling step T. Output: predicted target item Un-
1: Let Fr c1:m-1:1ami
2: Let t T:
3: while t do
4: Use the trained fa(.) to obtain predictions æn with æ, and / as inputs;
5: Substitute To into equation to obtain the distribution of - 1 step;
6: end while
7: Un 2o1-1;1: m];
8: if the same code project exists: on = Un:
9: else: Un is the project in the space closest to Ün.
4.2 Model Training
After introducing the Semantic ID, convert the historical interaction sequence Vi---1 into sequence (cI.1,.... Ct,mi c2,1,..., C2.mi
Cn-1, abbreviated as Cisa- 1:1mm We convert them into one-hot encodings (ff.1.... xf.............................. 3..m...... x n- x -.) which is considered as the initial state for discrete diffusion. Then perform discrete diffusion through the state transition formula defined in 2 (for more details, see Section 3.2) h to obtain the discrete state after steps æ!.j- for any i E (1. }} and j E (1,...,m), m). Accordingly. the labels changes from C.jj to ci.- Then (cf, .......... d.mi...... . .....:hhh- 1,:1111--- 1,) forms a "fuzzy set" of C1m-1:11:s denoted as 1::m-1; which can also be viewed the state 2, of the diffusion transition step
Considering the suitability of the Transformer for sequence-to-sequenec tasks, along with its well- demonstrated effectiveness in modeling sequential dependencies, as Approximator fe(-) to predict C2:::1: with ci.x-1:1: as input. This approach differs from the common practice in diffusion models, which often focus on modeling noise. aligns more closely with typical SR tasks that use U1n-1 to predict U2m. that is, the distribution Pa ( ( We have adopted sinusoidal time step embeddings, which are added after the embedding layer, allowing the model to capture information about the time steps. This process can be represented by:
it with embedding layer
02:n.1mm fo(ct... 1:1mm: t).
(5)
Generally, the loss function of diffusion models is designed based on KI. divergence, or it can be simplified to mean-squared error. However, guided by the theory of information diffusion, choose to use cross-entrops loss function, which is more suitable for recommendation tasks, to optimize our model without being constrained by the aforementioned methods.

---

Table 1: Detailed descriptions and statistics of datasets. "Avg. length' represents the average length of item sequences, while Avg. num' indicates the average number of words item text.
Datasets
Users
Items
Interactions
Avg.length
Avg.num
Scientific
8442
4385
59 427
7.04
182.87
Office
87 436
25 986
684 837
7.84
193.22
Online Retail
16 520
3469
519 906
26.90
27.80
4.3 Model Inference
In the inference phase, we aim to emulate the reverse process of the diffusion model, iteratively producing refined recommendation results. According to Bayes' theorem, we have
q<11-1/24, m0) = g( 1,00,,
(20) = Cat 21-1:p=
T 0 O x 2
(6)
g(<; (2))
I Q22
where @ represents the Hadamard product. Following the approach of Ho et al.
(2200]
and Hooge to derive the
boom et al. [2021], we employ the trained model fo(-) as described in Section 4.2
distribution Pa 2g xz). Combining it with g q 12 vo), we obtain the following parameterized
expression:
pa pa( E
(20
For the historical interactions U -1m-1, we use Cim--1:1m as 27. and starting from t = T, we iteratively execute Equation 7 until 0. This parameterization also allows us to perform inference for k steps at a time by predicting Po ( (x:)) = Σ q( 3e ( (3)l)))) leading to efficiency improvements. After obtaining Zo, take its last component, which is semantic ID of length If the corresponding item exists, we directly select that item; otherwise, we search for the item closest to it in the embedding space the final recommendation result. The training and inference phase of DDSR are demonstrated in Algorithm] and Algorithm
Experiment
5.1 Experiment Settings
Datasets. We employ three real-world datasets to evaluate the performance of our DDSR model. Following some works on text-based recommendation (Li et al. 2023a Hou et al. [2022), ) these datasets include two specific subcategories from the Amazon Reviews dataset (Scientific and Office), and a cross- pooss-platform dataset known as Online Retail, which operates from the UK. Following the method of Hou et al. (2022), we filter out users and items with fewer than five interactions. Subse- quently, interaction behaviors within each sub-dataset are grouped by user and sorted chronologically. For the Amazon sub-datasets, product descriptions are formed by concatenating fields such as title, category, and brand, while for the Online Retail dataset, the description field is used. The product texts are truncated to 512 characters. Please refer to Table I for detailed descriptions of these datasets.
Baselines. We compare DDSR with eight ofate-of-the-art SR methods, including two conventional SR methods, three methods based on semantic information, and three generative SR methods:
1). Conventional Baselines: SASRec (Kang and McAuley 2018a) utilizes causal Transformer architecture with self-attention mechanism model user behavior. BERT4REC (Sun et al. (0199 proposes bidirectional Transformer with a cloze task predicting the masked target items for SR.
2). Semantic-based Baselines: UniSRec (Hou et al. (022 utilizes the associated description text of items learn transferable representations across different recommendation scenarios, using an enhanced mixture-of-experts adaptor to enhance domain fusion and adaptation. VQ-Rec (Hou) et al. (2023) maps item text to a vector
discrete indices for learning transferable sequential recommenders. TIGER (Rajput et al. (2024) ) trains a Transformer-based sequence-to-sequeecce model with semantic IDs obtained from RQ- VAE to enhance its generalization ability.
3). Generative Baselines: ACVAE (ie et al. (02T] 1) proposes an adversarial and contrastive variational autocncoder for SR combining the ideas of CVAE and GAN. DiffuRec (Li et al. (023b)
7

---

Table 2: Performance of different models. Bold (underline) is used to denote the best (second-best) metrie, and **' indicates significant improvements relative to the best haseline (I-test P<.05). 'R @K (N@K') is short for "Recall ('NDCG@K') The features of items have been listed. whether ID, text (T), or both (ID+T).
Scientific
Office
Online Retail
Methods R@10
N@10
R@50
N:50
R@ 10
N@10
R:650
5@
R@ 10
N@
R@50
N@50
SASReez
0,1049
0.0527
0.1754
0.0683
0.1047
0.0714
0.1638
0.0857
0.1461
0.0674
0.3781
0.1186
BERT+Rocw
0,0473
0.0258
0.1092
0.0394
0.0798
0.0605
0.1207
0.0717
0.1354
0.0661
0.3517
0.1159
UniSRecr
0.1104
0.0537
0.1890
0.0787
0,0621
0.1668
0.0798
0.1274
0.0598
0.3294
0.1054
VQ-Reo,
0,1129
0.0577
0.2046
0.0749
0,1090
0.0676
0.1714
0.0845
0.1532
0.0713
0.9975
0.1254
TIGER;
0.1057
0.0597
0.1803
0.0682
0.1056
0.0712
0.1597
0.0868
0.0745
0.0390
0.2216
0.0701
ACVAED
0,0463
0,0315
0,0906
0.0457
0.0549
0.0397
0.1003
0.0519
0,0884
0.0410
0.1897
0.0648
DiffaRecip
0.1145
0.0594
0.1907
0.0752
0. 1056
0.0689
0.1781
0.0R32
0.0402
0.0189
0.0849
0.0321
DreamRocuwer
0.0845
0,0421
0.1645
0.0688
0.0954
0.0557
0.1662
0.0694
0.0577
0.0261
0.0997
0.0544
DDSR;
0.1207
0.0663
0.2153
0.0842
0.1138
0.0768
0.1926
0.0925
0.1687
0.0876
0.4021
0.1322
Improv.
+5.41%
+11.61%
+5.23%
+6.99%
+4.40%
+8.14%
+6.46%
+7.93%
+10.12%
+22.86%
+1.16%
+5.42%
introduces the diffusion model into the field of SR reconstructing target item representation from a Transformer backbone with the user's historical interaction behaviors. DreamRee (Yang et al. (2044) uses the historical interaction sequence conditional guiding information for the diffusion model to enable personalized recommendations.
Evaluation Settings. Following previous works Hou et al. [022]. Zhao et al. (2022), Zhou et al. 2020]. we evaluate all models using metrics Recall @ K and NDCG @ K. and report experimental results for K- = 10, 50. We employ the leave-one-out strategy for performance evaluation across all datasets. Concretely, we consider the last interaction as the test set, the second-to-last interaction as the validation set, and all preceding interactions as the training set. The ground-tre item of each sequence is ranked among all the other items while evaluating (Krichene and Rendle (2022)) ). The implementation details of DDSR are illustrated in Appendix C.1
5.2 Overall Performance
In this section, we compare the performance of DDSR with baseline models in terms of Top- recommendation accuracy under consistent experimental conditions (same data preprocessing), as summarized in Table 2 For models that recommend based on item IDs, we provide semantic information to them by jointly utilizing fixed text embeddings obtained from pre-trained BERT and the embeddings corresponding to item IDs in the model's embedding layer, to ensure fairness in the experimental setup. For all models, the final table records the better of the three methods, using only ID, only text, or both text and ID.
We observe that text-enhanced SR methods (UniSRec, VQ-Rec, TIGER) tend to benefit from textual information, leading to improved performance compared to conventional methods in most cases. Notably, VQ-Rec, employing discrete semantic encoding, generally outperforms UniSRec, which relies on continuous text embeddings, across various settings. This is despite UniSRec already using techniques like parameter whitening and MoE-enhanced Adaptor to enhance textual information. We posit that an excessive emphasis on text similarity can yield suboptimal outcomes, while the conversion to codes mitigates the coupling between items and semantic information. The corresponding representations of the codes are relearned in the sequence-to-sequers them to include more sequential structural information. While similarly based on discrete semantic encoding, the performance of TIGER is not stable. We do not rule out the possibility that there may be discrepancies between our implementation and the actual model, as it has not been open-sourced. Furthermore, we attribute the instability to TIGER's 's semantic ID length, which is limited to only 4 characters, potentially insufficient for expressing complex information.
model, allowing
In methods grounded in generative models, the performance of DiffuRec and DreamRec, based on diffusion models, surpasses that of ACVAE, relying on GAN and VAE. This disparity arises from the inherent advantages of diffusion models over VAE and GAN, as they circumvent the issue of posterior collapse, wherein the generated hidden representations lack crucial information about user preferences. Notably, DiffuRec achieves superior performance despite its limited capacity to handle semantic information, yet it still exhibits recommendation performance comparable to VQ-Ree. This suggests that diffusion models can yield effective hidden representations of items and users.
8

---

Table 3: Ablation analysis of DDSR. Bold font indicates the best metric.
Scientilic
Office
Online Retail
Variants
R@
N@
R@50
N@
R@
N@:
R@50
N@:
R@10
N@ 10
R@50
N@
Uniform transition
0.1207
0.0663
0.2153
0,0842
0.1097
0,0752
0.1889
0.0931
0.1517
0.0724
0.3967
0.1273
Importance transition
0.1192
0.0631
0.2110
0.1687
0.0876
0.4021
0.1322
w/o diffusion
0.1126
0.0563
0.2059
0.0749
0.1076
0,0659
0.1729
0.0851
0.1549
0.0719
0.4003
0.1268
RQ- VAE ID
0.1195
0.0645
0.1987
0.0825
0.1017
0.0609
0.1611
0.0784
0.1081
0.0564
0.2772
0.0853
Random
0,0554
0.0307
0.1193
0.0487
0.0548
0.0386
0.0982
0.0501
0.0428
0.0220
0.0863
0.0334
In all three datasets, DDSR achieves significant improvements, demonstrating the effectiveness of our approach. We attribute this success to two key factors. Firstly, the integration of semantic information mitigates data sparsity issues, as evidenced by the enhanced performance of semantic-basee on smaller datasets compared to traditional recommendation methods. Secondly, training on fuzzy sets generated by discrete diffusion furnishes the model with additional information. This is consistent with our theoretical analysis in Section 3.3 which posits that the diffused information space constitutes a completion of the original space, rendering models built on this enriched space effectively solvable. Moreover, while DiffuRec, relying on a continuous state space diffusion model, exhibits instability when confronted with larger and more intricate datasets, DDSR maintains a distinct advantage. We attribute this DDSR's retention of the discrete space without transitioning it into continuous one and introducing noise, thus circumventing the loss of meaningful information.
models
5.3 Ablation Study
We analyze the impact of semantic ID and discrete diffusion on final performance and conduct an ablation study to compare the results under different settings, as shown in Table The Uniform transition and importance transition are two discrete diffusion methods provided in Section [3.2] corresponding to different transition matrices. To control variables, these methods, along with the non-diffusion case, are all applied using semantic ID obtained through PQ. It can be observed that on larger datasets, the importance transition has relatively more advantages, and both methods outperform the non-diffusion scenario.
To control variables and accurately evaluate the performance of IDs, diffusion is applied in the last two rows of the experiments (the method based on PQ ID with diffusion corresponds to the first and second rows, it is not listed again). The PQ ID and RQ-VAE ID in the table corresponds to the two methods of obtaining semantic identifiers provided in Section Random ID represents using a randomly generated codebook to replace the semantic identifiers, where the random identifier of item 6; is simply C: = (Ci,I,... Ci.m) with Cijj uniformly sampled from 1, 2,., K. Using Random ID means the model longer gains semantic information. As observed, PQ ID exhibits greater stability and outperforms RQ- RQ-VAE ID across multiple datasets. Nevertheless, RQ-VAE ID requires less memory space due to its ability to achieve satisfactory representation with fewer codebook lengths. Semantic identifiers consistently outperform the Random ID, underscoring the significance of leveraging content-based semantic information. Indeed, models utilizing Random ID even underperform compared to SASRec. This can be attributed to SASRec's approach of setting independent embeddings for each item, rather than blending unrelated embeddings.
5.4 Further Analysis
Performance Analysis on Cold-Start Items. In this study, we evaluate the efficacy of DDSR in recommending cold-start items. Generating effective item embeddings without item information poses challenge for SR models. To evaluate this, we partition the test data into two groups based on item popularity. For the Office dataset, the range [0, 5) demarcates long-tail Items, whereas for the Online Retail dataset, is [0, 20). All other items are classified as Popular Items. The results are presented in Figure Notably, DDSR and VQ-Rec demonstrate substantial improvement over SASRec, which solely leverages a Transformer, particularly for long-tail Items, also referred to as the cold-start group. This is attributed to the integration of semantic information, enabling the model to acquire prior knowledge about items to some extent. Furthermore, DDSR demonstrates even greater enhancement compared to VQ-Rec in cold-start scenarios. We attribute this to the discrete diffusion method, which introduces 'fuzziness' effect in the interaction records, facilitating the inclusion of items with fewer interactions in the training process.

---

Scienttffic
Online Retail
5008
0.18
3270
52
Kogg
Pepuir time
Leng
Py km
0983
616.0
(a) Performance on Cold-Start Items.
(b) Impact of Sampling Step S.
Figure 2: Impact of Item Popularity and Sampling Step S.
Impact of Sampling Step S. The sampling step S represents the diffusion step number divided by the number of inference steps executed simultaneously, rounded down to the nearest integer. We illustrate in Figure 2b the influence of various sampling step settings. The model demonstrates optimal performance with approximately 50 sampling steps, with a slight increase for more complex datasets, albeit without significant disparities. Excessive sampling steps prolong the inference time without commensurate performance improvements, while inadequate steps lead to decreased performance.
Efficiency Analysis. We compared the time complexity and specific running overhead of DDSR with several other baseline algorithms, and the detailed results can be found Appendix C.2
6 Discussion
We proposed the DDSR model for the sequential recommendation, which employed discrete diffusion to construct fuzzy sets of user interaction sequences. This process was iteratively refined during inference, utilizing the sampling formula for discrete diffusion to derive the ultimate recommendation outcomes. Notably, although DDSR had borrowed the form of diffusion and sampling over time steps from diffusion models, it fundamentally differed from directly using diffusion models. If we viewed sequential recommendation through the lens of causality, the interaction sequence was the 'cause' and the recommended item was the 'effect'. Diffusion models typically address the target, blurring the 'effect', whereas DDSR has blurred the 'cause", inspired by the theory of fuzzy information processing. Although dual assurances, both theoretical and experimental results, have been provided to substantiate the superior performance of DDSR, imperative to recognize its inherent limitations. Despite efforts implement efficient computational methods, the nature of diffusion and sampling processes inevitably results in reduced efficiency and increased time complexity. Potential refinements, such approximating the diffusion process and accelerating the sampling algorithm, could offer effective strategies, which we will explore in future work.
Acknowledgments
This work is supported by the National Natural Science Foundation of China (No. U23A20319 62202443). Hao Wang also thanks the CCF-Tencent Rhino-Bird Open Research Fund (RAGR20230124).
References
Brian D. O. Anderson. deverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313-326, 1982.
J Austin, DD Johnson, J Ho, et al. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 34:17981 1199911-7993,,.111 2021.
Yukuo Cen, Jianwei Zhang, Xu Zou, Chang Zhou, Hongxia Yang, and Jie Tang. Controllable multi- interest framework for recommendation. In Proceedings of the 26th ACM SIGKDD International Conference Knowledge Discovery Data Mining, pages 2942 2951, 2020.
Tong Chen, Hongzhi Yin, Quoc Viet Hung Nguyen, Wen-Chih Peng. Xue Li, and Xiaofang Zhou. Sequence-aware factorization machines for temporal predictive analytics. 2020 IEFE 36th International Conference Data Engineering (ICDE), pages 1405-1416. IEEE, 2020.
10

---

Yongjun Chen, Zhiwei Liu, Jia Li, Julian McAuley, and Caiming Xiong, Intent contrastive learning for sequential recommendation. In Proceedings of the ACM Web Conference 2022, pages 2172-2182. 2022.
P Dhariwal and A Nichol. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 34:8780-8794, 2021.
Sheng Gu, Dongdong Chen, Jiajun Bao, et al. Vector quantized diffusion model for text-to-image syn- thesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10696-10706, 2022.
Y. Han, H. Wang, K. Wang, and et al. Efficient noise-decoupling for multi-behavior sequential recommendation. In Proceedings of the ACM on Web Conference 2024, pages 3297-3306, 2024.
Harte, W. Zorgdrager, Louridas, al. Leveraging large language models for sequential rec- ommendation. In Proceedings of the 17th ACM Conference on Recommender Systems, pages 1002-1102, 2023.
Ruining He and Julian McAuley. Fusing similarity models with markov chains for sparse sequential recommendation. In 2016 IEEE 16th International Conference on Data Mining (ICDM), pages 191-200. IEEE, 2016.
Xiangnan He, Hanwang Zhang, Min-Yen Kan, and Tat-Seng Chua. Fast matrix factorization for online recommendation with implicit feedback. In Proceedings of the 39th International ACM SIGIR Conference Research and Development in Information Retrieval, SIGIR '16, pages 549-558, New York, NY, USA, 2016. Association for Computing Machinery.
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, et al. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv: (tt:::/511..9999 1.0693,, 2015.
J Ho, W Chan, C Saharia, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303, 2022.
Jonathan Ho, Ankur Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840 (6860-6551,11, 2020.
Emiel Hoogeboom, David Nielsen, Priyank Jaini, and al. Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 34: 12454-12465, 2021.
Y. Hou, S. Mu, W. X. Zhao, et al. Towards universal sequence representation learning for recom- mender systems. Proceedings of the the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 585-593 2022.
Y. Hou, Z. He, J. McAuley, al. Learning vector-quantized item representation for transferable sequential recommenders. In Proceedings of the ACM Web Conference 2023, pages 1162-1171, 2023.
Aapo Hyvärinen and Peter Dayan. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005.
Jacob, Kligys, Chen, et al. Quantization and training of neural networks for efficient integer- arithmetic-only inference. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2704-2713, 2018.
W C Kang and J McAuley, Self-attentive sequential recommendation. In 2018 IEEE International Conference on Data Mining (ICDM), pages 197-206. IEEE, 2018a.
Wang-Cheng Kang and Julian McAuley. Self-attentive sequential recommendation. In 2018 IEEE International Conference Data Mining (ICDM), pages 197-206. IEEE, 2018b.
Walid Krichene and Steffen Rendle, On sampled metrics for item recommendation. Commun. ACM, 65(7):75-83, 2022.

---

D Lee, C Kim, S Kim, et al. Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11523-11532, 2022.
J Li, M Wang. Li, al. Text is all you need: Learning language representations for sequential recommendation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1258-1267, 2023a.
Z. Li, A. Sun, and C. Li. Diffurec: A diffusion model for sequential recommendation. ACM Transactions Information Systems, 42(3):1-28, 2023b.
Jianxin Ma, Chang Zhou, Hongxia Yang. Peng Cui, Xin Wang, and Wenwu Zhu. Disentangled self- supervision in sequential recommenders. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, pages 483-491, 2020.
Rajiv Pasricha and Julian McAuley. Translation-baser factorization machines for sequential recom- mendation. In Proceedings of the 12th ACM Conference on Recommender Systems, pages 63-71.
ACM, 2018.
Ruihong Qiu, Zi Huang, and Hongzhi Yin. Memory augmented multi-instance contrastive predictive coding for sequential recommendation. In 2021 IEEE International Conference on Data Mining (ICDM), pages 519-528. IEEE, 2021.
S. Rajput, N. Mehta, A. Singh, et al. Recommender systems with generative retrieval. Advances in Neural Information Processing Systems, 36, 2024.
K Rasul, C Seward, Schuster, et Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting, In International Conference Muchine Learning, pages 8857-8868. PMLR, 2021.
Fernando E Rosas, Pedro A M Mediano, Henrik J Jensen, et al. Reconciling emergences: An cnformation-theorette approach to identify causal emergence in multivariate data. PLoS computa- tional biology, 16(12)xe108228, 2020.
Claude E Shannon. A mathematical theory of communication. The Bell System Technical Journal, 27 (3):379-423) 1948.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, et al. Deep unsupervised learning using nonequilibrium thermodynamics. International conference on machine learning, pages 2256-2265. PMLR, 2015.
Yang Song, Jascha Sohh-Dickstein, Diederik Kingma, et al. Score-based generative modeling through stochastic differential equations. arXiv preprint 1.Xv4:2011.13456 2020.
F. Sun, J. Liu, J. Wu, al. Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management, pages 1441-1450, 2019.
H. Tanaka and G. Sommer. On posterior probabilities concerning a fuzzy information. Inst. für Wirtchaffswissenschatter RWTH, 1977.
H. Tanaka, Okuda, and K. Asai. formulation of fuzzy decision problems and its application to an investment problem. Kybernetes, 5(1):25-30, 1976.
Jiaxi Tang and Ke Wang. Personalized top-n sequential recommendation via convolutional sequence embedding, In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pages 565-573. ACM, 2018.
J. Tong, M. Yin, H. Wang, and et al. Mdap: A multi-view disentangled and adaptive preference learning framework for cross-domain recommendation. arXiv preprint arXiv:2410.05877, 2024.
Aaron Van Den Oord and Oriol Vinyals. Neural discrete representation learning. Advances Neural Information Processing Systems, 30, 2017.
12

---

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, volume 30, 2017.
H. Wang, T. Xu, Q. Liu, al. Mene: An end-to-end framework for learning multiple conditional network representations of social network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1064-1072, 2019.
H. Wang, D. Lian, H. Tong, and et al. Hypersorec: Exploiting hyperbolic user and item representations with multiple aspects for social- aware recommendation. ACM Transactions on Information Systems (TOIS), 40(2):1-28 2021.
H. Wang, Y. Han, K. Wang, and al. Denoising pre-training and customized prompt learning for efficient multi-behavior sequential recommendation. arXiv preprint arXiv:2408. /372, 2024.
Hao Wang, Mingjia Yin, Luankang Zhang, Sirui Zhao, and Enhong Chen. Mf-gslae: A multi-facte user representation pre-training framework for dual-target cross-domain recommendation. ACM Transactions Information Systems.
L. Wu, Z. Zheng, Z. Qiu, and et al. A survey on large language models for recommendation, World Wide Web, 27(5))60, 2024.
Wenjia Xie, Ruining Zhou, Hong Wang, et al. Bridging user dynamics: Transforming sequential recommendations with schrödinger bridge and diffusion models. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pages 2618-2628, 2024.
Z. Xie, C. Liu, Y. Zhang, et al. Adversarial and contrastive variational autoencoder for sequential recommendation. Proceedings of the Web Conference 2021, pages 449 2021.
Z. Yang. J. Wu, Z. Wang, et al. Generate what you prefer: Reshaping sequential recommendation via guided diffusion. Advances in Neural Information Processing Systems, 36, 2024.
Zhengyi Yang. Jiancan Wu, Zhicai Wang, Xiang Wang, Yancheng Yuan, and Xiangnan He. Generate what you prefer: Reshaping sequential recommendation via guided diffusion. arXiv preprint 231.v:2310.2045: 2023.
M. Yin, C. Wu, Y. Wang, and et al. Entropy law: The story behind data compression and Ilm performance. arXiv preprini arXiv:2407.06645. 2024a.
Mingjia Yin, Hao Wang, Xiang Xu, Likang Wu, Sirui Zhao, Wei Guo, Yong Liu, Ruiming Tang, Defu Lian, and Enhong Chen. Apgl4sr: A generic framework with adaptive and personalized global collaborative information in sequential recommendation. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 2009-30199 2023.
Mingjia Yin, Hao Wang, Wei Guo, Yong Liu, Suojuan Zhang, Sirui Zhao, Defu Lian, and Enhong Chen. Dataset regeneration for sequential recommendation. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 3954-3965, 2024b.
Luankang Zhang, Hao Wang, Suojuan Zhang, Mingjia Yin, Yongqiang Han, Jiaqing Zhang, Defu Lian, and Enhong Chen. A unified framework for adaptive representation enhancement and inversed learning in cross-domain recommendation. arXiv preprint arXiv:2404.00266. 2024.
Q. Zhao. Resetbert4rec: A pre-training model integrating time and user historical behavior for sequential recommendation. Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval, pages 1812-1816, 2022.
W. X. Zhao, Z. Lin, Z. Feng, al. revisiting study of appropriate offline evaluation for top-n recommendation algorithms. ACM Transactions on Information Systems, 41(2):1-41,203
K. Zhou, H. Wang, W. X. Zhao, et al. S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 1893-1902, 2020.
13

---

A Proof of the theory
Theorem A.1. The model often has the following form:
(U V) +U' XV
which requires the model to be a continuous mapping from the original input-output space (which can also be the original space). The data distribution the aggregated model can be modeled:
(D, (U'x
defined as 7 such that f' =(1). When the static distribution approximates the dynamic distribution, f the extended static scattering coefficient; otherwise, it the linear information distribution coefficient ", ensuring that the determination of f' is unique. Therefore, (D, p²) is the completion space of (U V.P), p), consisting of the complete set of (U V) and its separation.
In summary, the information diffusion space must be the completion space of the original space. Due the same dimensionality the original space, the modeling the aggregated set is reliable.
Next, consider the model (prediction model or simulation model) built on D(X). In practical applications, the input-oulphn set is generally constructed in two ways bused the original data:
Emin
Cj Tmax
Tmin
where e, is the normalized sequence; 2; is the original sequence data; Emin the minimum value of the original sequence; Tmax is the maximum value of the original sequence. It is obvious that e E [0, 1], meaning (U x V) S [0, 1], and the function boundary on D(X). When evaluating risk, the original sequence data generates input-output sets, which clearly have bounded intervals I; E [Tiin Fmaa), meaning the function boundary on D(X) is also bounded. Therefore, D(X) is necessarily subset of the real number
In the prediction model, the attribute function obtained from the aggregated set is:
Pya Σ Mao (u, UEU.
(6)
Among them, uR, (R,(u) MA, (u))a(()) u U.U. E V. Since the output sequence and the input sequence are generated by the same {ei}, and μs x++12
Separately substituting into (6), we can obtain:
Mag g(z0)
Σ MaoHA.PE
(7)
or
Ma. + g(æo)
= Σ PzoPA.HB.
Rearranging, we get:
Mza M,.HA,
Σ Glaos Hea)s
Mao
g(ro)
and
Σ MaaHA, g(2o)
Mea = EMOOHA, - gg(1o) =
Σ G(oo, Haa)
(8)
The initial value problem for the differential equation i(()) = (ll, v(1a) = y/ can trans- formed into integral equation:
u(t) = -.. f(t, [0))][((t)(())))))
+ yo. E u;
Since the cumulative sum and substitution integrals can be used in the scatter system, (8) is the integral equation's 's discrete model on D(X).
14

---

Mzz g(a;) Mx, [(,)]| Mz, + g(xs) Me, + +
x(,
G(z)) - (x))) - Gi Mny)
(10)
Both cases satisfy:
A
K max
such that [G(sz) G(xy)) & K |2 - xyl
ZEU Pe:
Maz
i.e., the function G(X) on D(X) satisfies the Lipschitz condition, thus the attribute function must have solution.
Since the attribute function has a solution on D(X), can be deduced by the "maximum" principle that the predicted output value must be:
Σ Wore
Σ
y0 =
W/
where the weights w, = Myo (v') maX {gg (u)}- Therefore, the information diffusion approximation model is established.
B Model Supplement
We first obtain fixed text embeddings from the relevant descriptions of items (e.g., product de- scriptions, item titles, or brands) using the pre-trained BERT model. Specifically, for an item v with corresponding description {w,, .2,.... we). the corresponding embedding vector is e, (BERT([CLS)) WI...., we]) Rdd, where "I:]" denotes the concatenation operation. Next, obtaining Semantic IDs from e; can be achieved through the following two methods:
Product Quantization (PQ). Similar to the VQ-Rec approach, we evenly divided e, into sub- vectors & : m. each with dimension of dw/m. Denote ap.j E Rdw /m as the j-th centroid embedding the codebook corresponding to the p-th sub-vector. For each sub-vector, the index of the nearest centroid from the corresponding PQ codebook is selected to form its discrete code Cip = arg minj | - ap.j 2| (1,2,.... K}. The centroids are obtained through the commonly used Optimized Product Quantization (OPQ) method. Finally, semantic ID for item V;.
(G.I....., Ci,m) is used as the
RQ-VAE. RQ-VAE generate set of codewords by quantizing the residuals. First, the input e, is encoded into ro using encoder E. Next, similar PQ. the nearest centroid in the first codebook. assumed to be ap.1. is found and its index is taken to form the first discrete code C, The residual is defined as r1 == To - 1p 1. The same operation is performed to obtain 6,2 and this process is repeated m times to obtain the complete Semantic ID). RQ- VAE use loss function L(x) == Lreeon + Lrqvae jointly trains the encoder, decoder, and the codebook, where Lrecon := ||e. el is reconstruction
loss, Lapar == d== [|sg[r:] Up.," || B1|rr: sg("p,,,]12 Here é; is the output of the decoder, and sg is the atop-gradicnt operation Van Den Oord and Vinyals [017]. Because the norm of the residuals decreases progressively, the importance of the codebooks obtained in this manner also diminishes with each level. Alternatively, can be said that the encoding at each position has varying levels of granularity. In our experiments, found that this approach requires smaller codebook size than PQ. but it is slightly less stable.
In fact, once the codebook is established, it remains fixed throughout the subsequent model training process. Therefore, the quality the codebook directly affects the training outcomes. With the rapid development of large language models, considering the use of more advanced pre-training schemes to replace BERT could be an effective way to enhance performance. However, this is not the focus of current rescarch, we leave this topic for future exploration.
15

---

Table 4: Time complexity analysis of various models, 'Comp.' is abbreviation for 'Complexity'.
Model
SASRec
BERT4Rec
UniSRec
TIGER
DiffuRec
DDSR
Comp.
O(n + dn?) (nd dn?) O'nd? + dn?) O(mand + mdd?) ((ud? dn²) O(mad + mnd?)
C Experimental Supplement
C.1 Implementation Details
All of our experiments were conducted on single RTX 4090. We implemented models based on PyTorch and the popular open-source recommendation library RecBole. We used (m = 32) x (K = 256) the code representation scheme for PQ IDs and (m 6) x (K 256) for RQ-VAE IDs. For baseline models, to ensure fair comparison, we optimized all methods using the Adam optimizer and searched for hyperparameters to find the best results. Since did not find open-source code for the TIGER model, we attempted to replicate it as faithfully as possible based on its paper; however, its performance may have been slightly lower in some experiments due to difficulties in unifying some details. For UniSRec and VQ-Rec, we utilize their code in a form that does not involve pretraining with the entire dataset, as we aim evaluate all baselines and DDSR considering recommendations on single dataset rather than cross- coos--dmmai.. The batch size was set to 2,048. The learning rate was adjusted among {0.0001, 0.0002, 0.0003, 0.0005, 0.001}. The model achieving the highest NDCG@ 10 result on the validation set was selected for evaluation on the test set. We employed an early stopping strategy with a patience of 10 epochs.
Regarding diffusion, all models were trained on a diffusion process of 1000 steps, and the time step embeddings are implemented using cosine embeddings, similar to the work Li et al. [2023b] For the diffusion with uniform transition, we employ the cosine schedule proposed by Hoogeboom et al. [021] to the transition probabilities (1-3.). For the diffusion with importance sampling, we adopt a linear schedule similar to the one used in Ho et al. [2020), where o² increases linearly from 10-4 K 0.02 * K. Skip steps in the sampling process were chosen among {100, 50, 35, 28, 23, 20, 17, 15). While there have been many works on accelerating sampling in continuous state space diffusion models, the development in discrete diffusion is still insufficient. Here, we adopted a basic uniform skip scheme for more efficient and effective sampling, which is one of our next research directions. If the evaluation steps do not divide 1000 evenly, the last step may be skipped.
C.2 Efficiency Analysis
We list the time complexities of six baselines in Table where n denotes sequence length, d is the dimension of the hidden layers, and mn is the length of the codebook used when semantic IDs are utilized. Most of these models based on the transformer its variants, hence the time complexity is (ndd² + dn²). Only TIGER and our proposed DDSR model are trained using semantic IDs, making their time complexity m times that of other methods. We plan to make further improvements in our subsequent work.
In addition, compared the actual operational costs of the DDSR model with those of UniSRec and DiffuRec, as shown in Table Although we adopted the method of 'performing inference for k steps time', which reduced the sampling steps and lowered the evaluation time compared to DiffuRec, we must acknowledge that DDSR still has certain limitations in terms of operational costs, necessitating further improvements in future work.
D Diffusion Models in Continuous State Space
D.I DDPM
Diffusion models comprise forward diffusion process and backward denoising process. We begin with the widely recognized denoising diffusion probabilistic model (DDPM) Ho et al. [2020] We start by defining our data distribution 2i ~ q((a) and Markovian noising process g which gradually adds noise to the data To to produce noised samples ær. In particular, each step of the noising process
16

---

Table 5: Comparison of Actual Operational Costs.
Datasets
Model
GPU memory (GB)
Training Time (s/epoch)
Evaluation Time (s/epoch)
UniSRec
8.32
3.51
0.67
Scientific
DiffuRec
14.94
4.97
17.52
DDSR
12.41
6.76
11.38
UniSRec
8.29
9.96
1.13
Office
DiffuRec
14.85
25.81
127.41
DDSR
12.48
36.19
69.10
UniSRec
9.96
52.19
3,70
Online Retail
DiffuRec
15.97
65.22
103.44
DDSR
13.47
83.51
60.11
adds Gaussian noise according to variance schedule given by Ae:
q xx-1) == MNx;; vI -1, Bx1)
Furthermore, ((1; mo) can be expressed as a Gaussian distribution. With oz == 1 - 3g and āye, where ~ No I). ār := II. Ota, g(zx mn) = -ax;; /a,oo (1 ā []) vārzo + -1-1ic -
Here, 1 1 a. indicates the variance of the noise at arbitrary timestep, and this can be used to define the noise schedule instead of Be.
Using Bayes theorem, one finds that the posterior ((22-11112;0 2o) is also Gaussian with mean
(T Fo) and variance 3, defined as follows:
(e(, = VaL-1B
Var( a1-1) Lt.
20
(8)
1-a0 at
Ot.
1 1 1-a2-1 3r
B: =
(9)
Oct
g( mo) = (1r- (1e 20), B.1)
(10)
If we wish sample from the data distribution g(2r0), we first sample from ((xT)) and then sample reverse steps (())) until we reach vi-- Under reasonable settings for 3, and T, the distribution q(TT) nearly an isotropic Gaussian distribution, so sampling xT is trivial. All that is left is to approximate g(xx-1[æ;, using a neural network, since it cannot be computed exactly when the data distribution unknown. To this end. Sohl-Dickstein al. [56] note that g (xze) approaches diagonal Gaussian distribution as I" + oo and correspondingly 3 + O, so it is sufficient to train neural network to predict mean μs and a diagonal covariance matrix Eg:
Pe( == -N(xe11 Ex(x,, t))
(11)
To train this model such that (20) learns the true data distribution q(xu), optimize the following variational hower-bound Lyth for po(xo):
Lub = Lo + L1 +.... Lr-1 + Lr
(12)
Lo log (x(ul22))
(13)
L =
(14)
LT = D
(15)
17

---

While the above objective is well-justified, Ho et al. [25] found that different objective produces better samples in practice. In particular, they do not directly parameterize Ho [z t) as neural network, but instead train model eo (0(22 1) to predict from Equation 17. This simplified objective defined as follows:
simple
[lle (I t) 1
(16)
During sampling, we can use substitution derive (1(e)) from eo (2 t):
1 a2 -
po(a))) = Vou Zt
VI (1(()))))
(17)
Note that /iimpl does not provide any learning signal for Ep(xre, 1). Ho et al. [25] find that instead of learning (x( f), they can fix it to constant, choosing either 3r or 3r. These values correspond to learning noise and the reverse process variance respectively.
D.2 Score-Based Generative Model
In this section, we introduce Score-Based Generative Model (SGMs) Song et al. (2020), specifically diffusion model represented in the form of Stochastic Differential Equations (SDEs). SGMs model the forward diffusion process using the stochastic differential equation:
dr; [1 t)dt + g(t)du, ao ~ Po Ptarget
(18)
where E [0, 2T, and signifies Brownian motion, Phaggtt represents target distribution. The function f.,tt:: I2d is vector-valued function called the drift coefficient of (t),, and 9(-) RR R is a scalar function known as the diffusion coefficient of (t).. The functions f and determine the type of prior distribution Pptior to which the forward process will diffuse, and they are typically designed to make the prior distribution a Gaussian distribution. As remarkable result from (Anderson [1882]. the reverse of the diffusion process is also diffusion process, given by the following reverse-time SDE:
dc. = [(x,, - -(()) (z) g())w,
(19)
IT PT Pprior.
where W is a standard Wiener process in reverse time. The term Va log (z)), which represents the score function of the marginal density Pt, the only unknown term this reverse process. SGMs learns its approximate target sæ (x(t). t) through denoising score matching (DSM) Hyvärinen and Dayan 2005 with Sp referred to as the denoising model:
g" arg min (1 E.(0) (2(0)=())
||00( 1) - Va Po Por Poggoo((=l0000 (x|=())))
(20)
Here, A(t) is positive weighting coefficient, (((0, 7). The joint distribution Por (O) is the conditional transition distribution from (0) to (t), which determined by the pre-defined forward SDE. To summarize, SGMs first utilize the diffusion process defined in Equation[T8) to obtain the
distribution 2; at intermediate time steps. Then, they minimize the loss defined in Equation (0) to train the denoising model and sample iteratively using the formula defined in Equation 19 to obtain the final result.
18

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
The abstract and introduction clearly outline the primary contributions of the paper, including the background, challenges, motivation, overall framework, and experimen- tal validation.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The paper provides detailed explanation of the problem background in the methods section and validates the hypothesis through multiple runs on various datasets with different characteristics in the experimental section. Additionally, the limitations are thoroughly discussed in Section
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
19

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
The paper presents all theoretical results with clearly stated assumptions and complete proofs, which are either included in the main text or supplemented by detailed proofs in the appendix, ensuring both clarity and rigor.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper provides detailed descriptions of the experimental setup, including dataset specifications, parameter settings, and evaluation metrics, ensuring that the main results can be reproduced even without direct access the code and data.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
20

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
The paper includes links openly accessible data and code repositories that cover the necessary commands and environment setup to reproduce the main experimental results.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//iips.c//pple wttpccpppppppppeeeeeeeeeeeeeeeee
http:///ggu.eee//oeeesbiiiissss Cooeeeooidsssiiiieccca ooooooooisssioooioococoooooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: The paper thoroughly describes all relevant experimental details in a dedicated "Experiment Setting" section, including data splits, hyperparameters, and optimization methods, ensuring clarity and reproducibility of the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
21

---

Justification:
The paper appropriately reports statistical significance using t-tests with significance threshold of p 0.05, ensuring transparency and rigor in the experimental analysis.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (c.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
The paper thoroughly details the computer resources required for each experi- ment, including the type compute workers, memory specifications, and execution times, enabling reproducibility and understanding of the computational demands of the study.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The research ensures that ethical considerations are addressed and integrated into the study's design and execution, and the authors ensure anonymity is preserved.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
22

---

The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
The paper addresses both potential positive and negative societal impacts of the research, highlighting potential misuses and discussing mitigation strategies, thereby ensuring a comprchensive consideration of broader societal implications.
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation. generating fake profiles, surveillance), fairesss considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not involve the release of data or models that pose a high risk for misuse, thus no safeguards are necessary.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
23

---

Answer:
[Yes]
Justification: The paper properly credits the creators of all utilized assets, explicitly adhering licenses and terms of use by including citations, version details, and relevant licensing information.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppsswipeaaaaeeeeeeeeeeeee thcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The code repositories will provide comprehensive documentation for all newly introduced assets, including details about their creation, usage, and limitations, ensuring that other researchers can effectively utilize these resources.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects, so this information not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
24

---

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing or research with human subjects, so IRB approval or equivalent is not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
25


Reviews:
[{"review_id": "Nvh9uEpsW6", "reviewer": "Reviewer_qbmo", "summary": "The paper presents a new model for sequential recommendation (SR) called DDSR, which aims to predict items of interest for users based on their past behavior. The authors critique conventional SR methods for not adequately capturing the randomness and unpredictability in user behavior. DDSR uses fuzzy sets of interaction sequences to better reflect the true evolution of user interests. Unlike common diffusion models that operate in continuous domains, DDSR uses diffusion transition processes in discrete state spaces, avoiding information loss by employing structured transitions. To tackle the inefficiency of matrix transformations in large discrete spaces, the model utilizes semantic labels derived from quantization or RQ-VAE instead of item IDs. Tests on three public datasets show that DDSR surpasses current state-of-the-art methods, proving its effectiveness in SR tasks.", "questions": "Could the authors please elaborate on the analogy between fuzzy sets and discrete diffusion?", "limitations": "Yes, limitations adequately addressed.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Effective combination of pre-existing ideas: discrete diffusion with semantic IDs. Empirical results in Section 5 on Scientific, Office, Online Retail datasets.", "weaknesses": "I think a main weakness of the paper is a lack of clarity in writing. The method is still unclear to me, although this may be partly due to my lack of experience in recommender systems. I would condense and combine Sections 3 and 4 and move Section 2 to end. The technical novelty is also not entirely clear to me. Is it in the form of the transition matrices used for the discrete diffusion? Or is the technical novelty claimed that the items are recommended based on the output of running a few steps of discrete diffusion (adding noise to the features)?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Firstly, we appreciate the valuable time you have invested in our work and the constructive suggestions you have offered; we will rigorously revise our paper based on your feedback. We hope the following explanations will somewhat alleviate your concerns.\n\nW1: We find your suggestion to relocate Section 2 to the end of the paper quite insightful, and we plan to adopt this adjustment in the final version. Regarding the idea of combining Sections 3 and 4, we need to explain that the original intention of discussing Section 3 before Section 4 separately was to separate the theoretical and model sections. This was aimed at preventing the model section from becoming too lengthy, which could negatively affect the reader's experience. However, your advice has highlighted a potential compromise in clarity that we had not fully appreciated. To enhance our discussion, we will introduce an algorithm flowchart below as an initial step. In the formal version, we will thoughtfully consider how to integrate your feedback. Currently, we are contemplating including Sections 3.1 and 3.2 within the methods part of Chapter 4 and positioning Section 3.3 just before the experimental segment, to provide theoretical backing for our proposed DDSR model. We would appreciate your thoughts on whether this revised structure seems appropriate. Here is the algorithm flowchart we provided (after generating discrete semantic encoding).\n\n**Training of DDSR.**\n\n**Input:**\n- Historical Interaction Sequence of user $u$:  $v_{1:n-1} = c_{1:n-1;1:m}$ ;\n- Target item:  $v_n = c_{n;1:m}$;\n- Transition matricest: $Q_t$;\n- Approximator: $f_{\\theta}(\\cdot)$.\n\n**Output:**\n- Well-trained Approximator: $f_{\\theta}(\\cdot)$.\n\n**Procedure:**\n1. **while** not converged **do**\n2. &emsp; Sample Diffusion Time:  $t \\sim [0,1,...,T]$;\n3. &emsp; Calculate $t$-step transition probability: $\\quad\\overline{\\boldsymbol{Q}}_t=\\boldsymbol{Q}_1\\boldsymbol{Q}_2\\ldots\\boldsymbol{Q}_t$;\n4. &emsp; Convert $c_{n;1:m}$ to one-hot encoding $x_{n;1:m}^0$;\n5. &emsp; Obtain the discrete state $x_{n;1:m}^t$ after $t$ steps by Equation (2),  thereby obtain the 'fuzzy set' $c_{1:n-1;1:m}^{t}$;\n6. &emsp; Modeling $c_{2:n;1:m}$ based on 'fuzzy sets' through Equation (5);\n7. &emsp; Take gradient descent step on $\\nabla$ $L_{CE}$ ($\\hat{c}_{2:n;1:m}$, $c_{2:n;1:m}$).\n \n\n**Sampling of DDSR.**\n\n**Input:**\n   - Historical Sequence: $v_{1:n-1} = c_{1:n-1;1:m}$\n   - Well-trained Approximator: $f_{\\theta}(\\cdot)$\n   - Sampling Steps: $T$.\n\n**Output:**\n   - Predicted Target Item: $v_{n}$\n\n1. Let $x_T$ = $c_{1:n-1;1:m}$; \n\n2. Let t = T;\n\n3. **while** $t>0$ **do**\n\n4. &emsp;  Use the trained $f_{\\theta}(\\cdot)$ to obtain predictions $\\widetilde{x}_{0}$ with $x_t$ and $t$  as inputs;\n\n5. &emsp;  Substitute $\\widetilde{x}_{0}$ into equation (7) to obtain the distribution of $t-1$ step;\n\n6.  **end while** \n\n7. $\\widetilde{v}_{n}$ =\n$x_0$[-1;1:m];\n\n8. if   the same code project exists: $v_n$ = $\\widetilde{v}_{n}$;\n\n&emsp; else: $v_n$ is the project in the space closest to $\\widetilde{v}_{n}$.\n\n\nW2: I apologize for the lack of clarity in our description and hope that the following explanation will address your concerns about the technical innovations. Unlike traditional diffusion models that add noise to features, we use discrete diffusion to induce structured transformations among discrete nodes, such as semantic labels, within a discrete state space using transition matrices. This approach replaces the introduction of noise in conventional diffusion models and avoids the deviation of representations towards meaningless directions. The semantic space itself is inherently meaningful, and the discrete diffusion framework allows us to design controllable transition schemes, such as importance-based transitions, which are a major source of performance improvement. We also note that traditional diffusion models add noise to embeddings, which often exacerbates training difficulties in highly sparse scenarios like sequence recommendation. For instance, in our experiments, we observed that DiffuRec required over three hundred training epochs to converge on most datasets. Our work is also the first attempt to enhance diffusion models with textual information in the context of recommendation.\n\n\nQ1: We introduce the concept of fuzzy sets because diffusion models are inherently designed for generative tasks, which fundamentally differ in form from autoregressive tasks like sequence recommendation, where the $n$th item is predicted using the previous $n-1$ items. By employing fuzzy modeling to redefine the diffusion process, we ensure that the diffusion can be theoretically applied to the target embeddings but also enhances adaptability without compromising the robust theoretical support inherent to diffusion models. As for discrete diffusion, it represents a form of fuzzy modeling in our work. The specific reasons for using discrete diffusion are presented in W1.\n\nThank you for your thorough review. If you feel our clarifications have indeed alleviated any doubts, we kindly ask you to consider adjusting the score accordingly. Should you have any further questions or require additional clarification, please do not hesitate to point them out.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "36SA07z59F", "reviewer": "Reviewer_6uVb", "summary": "The paper presents a diffusion model-based sequential recommendation from a novel information-theoretical perspective, which operates on discrete structural state spaces along with semantic labels improving efficiency and tackling cold-start issues.", "questions": "1.\tIn text such as line 235, 236, 251, the citation text is literally repeated.\n2.\tIn text such as line 20-26, it would be better to add brackets to distinguish references from the main text.\n3.\tIn line 74 and 144, it should be explicit that “2” and “4.1” refers to “Section 2” and “Section 4.1”.\n4.\tThe usage and format of quotation marks is a bit confusing, see line 34, 219 and Table 1 caption for examples.", "limitations": "There is no significant limitation. Perhaps the authors could consider give a rough discussion or comment on how the computational complexity (or the coefficient constant) of the proposed method can be improved.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "Strengths*\n1.\tNovelty. The paper uses a directed graph to model sequential recommendation and models the transitions of nodes with discrete diffusion. Besides, authors introduce semantic tags to replace meaningless item IDs, enhancing efficiency and improving cold start issues.\n2.\tSoundness. The paper presents enough theoretic analysis and experimental evaluations to validate the proposed method. The significant performance improvement on three public datasets, along with a series of further ablation study and analysis, clearly demonstrates the soundness of the paper.\n3.\tEasy-followed. The paper presents sufficient algorithmic detail. In addition, the authors provide sufficient codes to help understand and re-product the algorithm.\nWeaknesses* \n1.\tCitation mixed in the text make obstacles for reading. See questions for detail.\n2.\tWriting should be more regulated. See questions for detail.", "weaknesses": "See Above", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the valuable time you have dedicated to our work and the encouragement you have given us! Thank you for thoroughly reading our paper and pointing out some issues, which indeed arose due to our oversight; for this, we deeply apologize. In the final version, we will carefully review and correct each issue you have raised and perform a comprehensive proofreading from start to finish to ensure that such problems do not recur.\n\nRegarding the limitations you've mentioned, specifically the need to discuss improvements in computational complexity, we indeed recognize the importance of this issue and plan to address it in the final version of our paper. We believe that there are two feasible approaches for improvement: through model architecture and coding strategies. For instance, within the model architecture, one could employ linear attention mechanisms in Transformers as a substitute, redesigning the computation of $K$ and $V$ matrices to reduce the time complexity of the attention mechanism from $O(n^2d)$ to $O(nd^2)$. Additionally, we have taken note of the recently proposed mamba model, which maintains linear complexity while demonstrating robust performance. From a coding perspective, exploring ways to convey sufficient information with shorter codebook lengths seems to be an effective strategy. For example, in our research, the codebook lengths generated by RQ-VAE are significantly shorter than those required by traditional quantization methods, yet maintaining effectiveness on this basis remains a critical challenge. \n\nLastly, we greatly appreciate the valuable suggestions you have provided.\n\n[1]Gu A, Dao T. Mamba: Linear-time sequence modeling with selective state spaces[J]. arXiv preprint arXiv:2312.00752, 2023.\n\n[2]Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. “Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention”. In: International Conference on Machine Learning. PMLR. 2020,\npp. 5156–5165.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YbaWGN7kmP", "reviewer": "Reviewer_ukuQ", "summary": "This paper studies the problem of fuzzy modeling for sequential recommendation. The work proposes to leverage the fuzzy sets of interaction sequences for modeling the nature of users' evolving real interests. It also introduces the discrete diffusion modeling specifically for the discrete data. The experiment results showcase the better model performance in comparison with several included baselines.", "questions": "1. How many times of performance evaluations are conducted for the average results?\n2. What are the real runtime costs for the proposed model and other baselines for comparison?", "limitations": "Please see the weakness for details.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. This paper studies the optimization of sequential recommender systems, which is an important application for the information retrieval domain.\n2. The paper is written with fair clarity and thus it is easy to follow.\n3. The paper provides careful analyses to support the claims made by this paper and the experimental results generally demonstrate the effectiveness of the proposed methods.", "weaknesses": "1. Insufficient baseline comparison for the conventional sequential recommender models, as the two included models are outdated. \n2. Insufficient empirical analysis in terms of the model efficiency. Firstly, the runtime cost comparison is missing, which is however very important to recommender system prototype evaluation for practical deployment consideration. Secondly, based on the complexity analysis, it is crucial to showcase the performance variations (effectiveness/efficiency) by varying the codebook length $m$.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Firstly, we sincerely appreciate your careful review of our work and the extremely valuable suggestions you have made! In response to the issues you have pointed out, we will improve our research and provide detailed answers to each of your questions, hoping to clearly address your concerns.\n\nW1: Regarding the concern you raised about the insufficient baseline comparison with traditional sequence recommendation models, we indeed acknowledge that there is a gap in our work in this area.  During our research, we primarily focused on comparing with other text-based and generative model-based sequential recommendation approaches, which may have led to an inadequate evaluation of traditional models.  To address this, we have added experiments with the following three traditional sequential recommendation models:\n\n1. **HSTU**[1]: A novel sequence recommendation architecture introduced by Meta in May this year. HSTU utilizes a hierarchical sequence transduction framework, incorporating a modified attention mechanism and a micro-batching algorithm (M-FALCON) for efficient handling of high-cardinality and dynamic recommendation data.\n\n2. **Mamba4Rec**[2]: A model proposed by Liu et al. in June this year, based on the latest Mamba design, which includes a series of sequence modeling techniques.\n\n3. **FDSA**[3]: The feature-level deep self-attention network introduced by Zhang et al., which integrates various heterogeneous features of items into a feature sequence with different weights through a fundamental mechanism.\n\nHere are our experimental results:\n| **Datasets**   | **Model** | **Recall@10** | **NDCG@10** | **Recall@50** | **NDCG@50** |\n|----------------|-----------|---------------------|-----------------------------|-------------------------------|-------------------------------|\n| **Scientific**  | HSTU| 0.1086        |0.0543| 0.1816| 0.0721|\n|| Mamba4Rec |0.0976  |  0.0562  |  0.1881 |0.0760   |\n|| FDSA    |0.0901| 0.0599 |  0.1683|  0.0766 |\n| **Office**   | HSTU   | 0.1102   | 0.0718|0.1709| 0.0872  |\n| | Mamba4Rec  |0.1079|0.0727  |    0.1633   |     0.0858 |\n|| FDSA    | 0.1108|0.0730|0.1723|0.0854 |\n| **Online Retail**| HSTU   |0.1456| 0.0698|0.3750  | 0.1213  |\n|| Mamba4Rec  | 0.1410              |  0.0686                  | 0.3783                   |   0.1188                           |\n|| FDSA   |           0.1478   |          0.0719            |              0.3817  |                0.1229             |\n\nAmong these, we have integrated HSTU into our framework, which based on RecBole, as its original source code did not include a test set partition. Thank you for your suggestion; we will enhance the baseline comparisons in the formal version of the paper.\n\n[1] Zhai J, Liao L, Liu X, et al. Actions speak louder than words: Trillion-parameter sequential transducers for generative recommendations[J]. arXiv preprint arXiv:2402.17152, 2024.\n\n[2] Liu C, Lin J, Wang J, et al. Mamba4rec: Towards efficient sequential recommendation with selective state space models[J]. arXiv preprint arXiv:2403.03900, 2024.\n\n[3] Zhang T, Zhao P, Liu Y, et al. Feature-level deeper self-attention network for sequential recommendation[C]//IJCAI. 2019: 4320-4326.\n\nQ1&W2: Thank you for pointing that out. We will address these omissions in the final version of the paper. Below are the actual runtime cost comparison results of our DDSR model with UniSRec and DiffuRec:\n| **Datasets**   | **Model** | **GPU memory (GB)** | **Training Time (s/epoch)** | **Evaluation Time (s/epoch)** |\n|----------------|-----------|---------------------|-----------------------------|-------------------------------|\n| **Scientific**  | UniSRec   | 8.32  | 3.51  | 0.67|\n| | DiffuRec  | 14.94| 4.97 | 17.52 |\n| | DDSR    | 12.41| 6.76 | 11.38 |\n| **Office**   | UniSRec   | 8.29 | 9.96 | 1.13 |\n|  | DiffuRec  | 14.85 | 25.81| 127.41|\n| | DDSR    | 12.48  | 36.19| 69.10 |\n| **Online Retail**| UniSRec   | 9.96| 52.19| 3.70|\n| | DiffuRec  | 15.97| 65.22 | 103.44|\n| | DDSR    | 13.47 | 83.51| 60.11 |\n\nRegarding the performance variations with different codebook lengths that you mentioned, we have added experiments here. Taking the Scientific dataset as an example, we explore how codebooks obtained through quantization methods affect the recommendation performance at various lengths：\n| **code_len** | **Recall@10** | **NDCG@10** | **Recall@50** | **NDCG@50** | **GPU memory (GB)** |**Training Time (s/epoch)** |**Evaluation Time (s/epoch)** |\n|----------------|---------------------|-----------------------------|-------------------------------|-------------------------------|-------------------------------|-------------------------------|-------------------------------|\n| **64** |0.1235|0.0656|0.2396|0.0907|22.14|13.56|19.40|\n| **32** | 0.1207|0.0663|0.2153 |0.0842 |12.41|6.76|11.38|\n| **16** |0.1145|0.0603|0.2161|0.0846|9.74|4.34|3.79|\n| **8** |0.1084|0.0589 |0.2184| 0.0829|8.37|2.55|3.61|\n| **4** |0.0836|0.0433 |0.1820| 0.0759|7.43|2.49|3.33|\n\nFor the Scientific dataset alone, a codebook length of 64 achieved best results most of the time. We regret not having made sufficient attempts before.\n\nQ1: During our experiments, we ran each model five times and took the average as the final result to validate the significance of improvements. We failed to mention this point in the implementation details section of the paper, which was an oversight. We are very sorry for this and will include this information in the final version of the paper.\n\nIf you believe that our response has resolved the issues raised and alleviated your concerns, we kindly request that you consider adjusting the score accordingly. If you still have any doubts or suggestions, we earnestly ask you to point them out to further enhance our work. Once again, thank you for your thoughtful comments and consideration.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "N5bB3p9831", "reviewer": "Reviewer_LQPH", "summary": "The paper presents the DDSR model for Sequential Recommendation (SR), which better captures user interest evolution by using fuzzy sets of interaction sequences. Unlike traditional methods, DDSR effectively handles the unpredictability of user behavior and addresses cold start issues. Experiments on benchmark datasets show that DDSR outperforms existing methods, demonstrating its effectiveness in SR tasks.", "questions": "See comments above.", "limitations": "This paper has discussed limitations.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. DDSR introduces a novel perspective in SR, effectively addressing the limitations of traditional sequential modeling methods and enhancing recommendation accuracy.\n2. DDSR uses fuzzy sets of interaction sequences and diffusion transition processes in discrete state spaces, improving the model's ability to capture the randomness and unpredictability of user behavior.\n3. The theoretical justification for constructing the information diffusion approximation model is sound and fundamental.\n4. Quantitative experimental results demonstrate the model's effectiveness, with particularly exciting results in cold start scenarios.", "weaknesses": "1. The paper lacks an algorithm to describe the entire model operation. Including this would greatly enhance readers' understanding of the model and its theoretical underpinnings.\n2. All experimental results are quantitative. It is recommended to supplement with a case study or visual experiment.\n3. Although the author has analyzed the time complexity, I believe most readers would also appreciate a comparison of actual running times.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First of all, thank you very much for reading our work carefully and for your valuable comments and suggestions, from which we have greatly benefited. Next, I will explain and respond to the shortcomings you have pointed out, and we will make corresponding revisions in the official version of our work based on your reminders.\n\nW1: Regarding a clearer description of our algorithm, we think your suggestion is very reasonable. To avoid making the theoretical part too lengthy within the main discussion, we placed the description of discrete diffusion separately in Section 3. However, this approach may not facilitate readers' understanding of the overall workflow. We add our algorithm flowchart below and will incorporate it into the official version of our paper based on your suggestion.\n\n**Training of DDSR.**\n\n**Input:**\n- Historical Interaction Sequence of user $u$:  $v_{1:n-1} = c_{1:n-1;1:m}$ ;\n- Target item:  $v_n = c_{n;1:m}$;\n- Transition matricest: $Q_t$;\n- Approximator: $f_{\\theta}(\\cdot)$.\n\n**Output:**\n- Well-trained Approximator: $f_{\\theta}(\\cdot)$.\n\n**Procedure:**\n1. **while** not converged **do**\n2. &emsp; Sample Diffusion Time:  $t \\sim [0,1,...,T]$;\n3. &emsp; Calculate $t$-step transition probability: $\\quad\\overline{\\boldsymbol{Q}}_t=\\boldsymbol{Q}_1\\boldsymbol{Q}_2\\ldots\\boldsymbol{Q}_t$;\n4. &emsp; Convert $c_{n;1:m}$ to one-hot encoding $x_{n;1:m}^0$;\n5. &emsp; Obtain the discrete state $x_{n;1:m}^t$ after $t$ steps by Equation (2),  thereby obtain the 'fuzzy set' $c_{1:n-1;1:m}^{t}$;\n6. &emsp; Modeling $c_{2:n;1:m}$ based on 'fuzzy sets' through Equation (5);\n7. &emsp; Take gradient descent step on $\\nabla$ $L_{CE}$ ($\\hat{c}_{2:n;1:m}$, $c_{2:n;1:m}$).\n \n\n**Sampling of DDSR.**\n\n**Input:**\n   - Historical Sequence: $v_{1:n-1} = c_{1:n-1;1:m}$\n   - Well-trained Approximator: $f_{\\theta}(\\cdot)$\n   - Sampling Steps: $T$.\n\n**Output:**\n   - Predicted Target Item: $v_{n}$\n\n1. Let $x_T$ = $c_{1:n-1;1:m}$; \n\n2. Let t = T;\n\n3. **while** $t>0$ **do**\n\n4. &emsp;  Use the trained $f_{\\theta}(\\cdot)$ to obtain predictions $\\widetilde{x}_{0}$ with $x_t$ and $t$  as inputs;\n\n5. &emsp;  Substitute $\\widetilde{x}_{0}$ into equation (7) to obtain the distribution of $t-1$ step;\n\n6.  **end while** \n\n7. $\\widetilde{v}_{n}$ =\n$x_0$[-1;1:m];\n\n8. if   the same code project exists: $v_n$ = $\\widetilde{v}_{n}$;\n\n&emsp; else: $v_n$ is the project in the space closest to $\\widetilde{v}_{n}$.\n\n\nW2: Regarding the issues with the experimental section, we greatly appreciate your constructive suggestions. We recognize that our experimental setup was not comprehensive, so we have added a study on the impact of codebook length on recommendation performance. We will present the following experimental results on dataset Scientific in the form of bar charts in the paper:\n| **code_len** | **Recall@10** | **NDCG@10** | **Recall@50** | **NDCG@50** | **GPU memory (GB)** |**Training Time (s/epoch)** |**Evaluation Time (s/epoch)** |\n|----------------|---------------------|-----------------------------|-------------------------------|-------------------------------|-------------------------------|-------------------------------|-------------------------------|\n| **64** |0.1235|0.0656|0.2396|0.0907|22.14|13.56|19.40|\n| **32** | 0.1207|0.0663|0.2153 |0.0842 |12.41|6.76|11.38|\n| **16** |0.1145|0.0603|0.2161|0.0846|9.74|4.34|3.79|\n| **8** |0.1084|0.0589 |0.2184| 0.0829|8.37|2.55|3.61|\n| **4** |0.0836|0.0433 |0.1820| 0.0759|7.43|2.49|3.33|\n\nW3: Regarding the comparison of actual runtime, we have compared the GPU memory usage and runtime of the UniSRec, DiffuRec, and our DDSR model on a single A100 GPU. The specific results are as follows:\n| **Datasets**   | **Model** | **GPU memory (GB)** | **Training Time (s/epoch)** | **Evaluation Time (s/epoch)** |\n|----------------|-----------|---------------------|-----------------------------|-------------------------------|\n| **Scientific**  | UniSRec   | 8.32                | 3.51                        | 0.67|\n|                | DiffuRec  | 14.94               | 4.97                       | 17.52|\n|                | DDSR    | 12.41               | 6.76                        | 11.38|\n| **Office**   | UniSRec   | 8.29                | 9.96                        | 1.13|\n|                | DiffuRec  | 14.85               | 25.81                       | 127.41 |\n|                | DDSR    | 12.48              | 36.19                       | 69.10   |\n| **Online Retail**| UniSRec   | 9.96                | 52.19                      | 3.70  |\n|                | DiffuRec  | 15.97               | 65.22                       | 103.44                        |\n|                | DDSR    | 13.47               | 83.51                       | 60.11                          |\nThe increased runtime of DDSR primarily stems from the introduction of the diffusion model and the increased time complexity due to the conversion of project embeddings into discrete codes. The current time complexity is $O(mnd^2+mdn^2)$, which is $m$ (codebook length) times that of methods like SASRec that only use the Transformer model. Fortunately, we have found in our experiments that good results can be achieved with relatively smaller settings for dimension $d$, as we now effectively use a vector of length $m*d$ to represent a project. Additionally, the use of discrete codes helps reduce storage overhead and DDSR enables faster sampling speeds compared to other diffusion model-based methods by performing inference for k steps at a time. We believe that the initial efficiency issues are still within an acceptable range. Additionally, since state transitions within the discrete space do not introduce noise, we found in our experiments that DDSR's training has significantly enhanced stability and convergence speed compared to DiffuRec. For example, DiffuRec requires about one to two hundred epochs to converge on the Scientific and Office datasets, while our model only needs about forty to fifty epochs.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper introduces an information theory-based improvement of sequential modeling to capture the randomness in user behavior. The paper was well-received by all reviewers and inspired (what looks like) a fruitful exchange between the reviewers and the authors who addressed the reviewer concerns. Most reviewers increased their scores after the rebuttal. To meet the expectations of the reviewers, the authors should incorporate the rebuttal in their final paper, including improving the clarity of the paper, discussion of computational complexity, and adding more experimental results.
Decision: Accept (poster)