Breaking Determinism: Fuzzy Modeling of Sequential Recommendation Using Discrete State Space Diffusion Model

Authors: wenjia  xie
hao  wang
luankang  zhang
rui  zhou
defu  lian
enhong  chen
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 13:35:26

Abstract: Sequential recommendation (SR) aims to predict items that users may be interested in based on their historical behavior sequences. We revisit SR from a novel information-theoretic perspective and find that conventional sequential modeling methods fail to adequately capture the randomness and unpredictability of user behavior. Inspired by fuzzy information processing theory, this paper introduces the DDSR model, which uses fuzzy sets of interaction sequences to overcome the limitations and better capture the evolution of users' real interests. Formally based on diffusion transition processes in discrete state spaces, which is unlike common diffusion models such as DDPM that operate in continuous domains. It is better suited for discrete data, using structured transitions instead of arbitrary noise introduction to avoid information loss. Additionally, to address the inefficiency of matrix transformations due to the vast discrete space, we use semantic labels derived from quantization or RQ-VAE to replace item IDs, enhancing efficiency and improving cold start issues. Testing on three public benchmark datasets shows that DDSR outperforms existing state-of-the-art methods in various settings, demonstrating its potential and effectiveness in handling SR tasks.

Main paper: 
 introduction: 
 For a long time, sequential recommendation (SR) has been attracting increasing attention due to its excellent performance and significant commercial value (Chen et al. [2020] , Qiu et al. [2021] , Yin et al. [2024] ). Unlike traditional collaborative filtering or certain graph-based methods (Wang et al. [2019] ), SR systems emphasize the inherent dynamic behaviors of users rather than relying solely on structured data (Chen et al. [2022] , Ma et al. [2020] , Cen et al. [2020] ). This approach enhances the accuracy of personalized recommendations, allowing for more precise tracking of changes in user interests and needs. Typical deep learning-based SR models, such as those utilizing CNN, RNN, and Transformer architectures (Tang and Wang [2018] , Hidasi et al. [2015] , Kang and McAuley [2018b] ), have achieved remarkable success in modeling user historical interaction data. However, these methods are formalized models based on a narrow information theory assumption (Shannon [1948] ), which only acknowledges determinism (Rosas et al. [2020] ). They assume that all phenomena strictly adhere to mechanical laws and that the states of motion of objects at different times can be uniquely determined. In reality, however, user behavior is characterized by randomness and unpredictability. They might change their mind about buying a down jacket due to a sudden warm-up, or they might impulsively buy desserts due to a breakup. As illustrated on the left in Figure 1 , a user's interest at any given moment might be focused on 'some items' with blurred boundaries, only converging finally when the user makes a selection.Although increasing the sample size is an effective strategy to address the above issue, in reality, the data in recommendation systems is usually quite sparse (He and McAuley [2016] ), limiting the Antique olive green flower teacup.Baking mould heart white chocolate.Semantic IDs (12, 31, 98) (12, 47, 55) (17, 37, 69) Seq2seq model (17, 42, 51) Figure 1 : Illustration of DDSR constructing fuzzy sets and incorporating semantic IDs to enhance sequential recommendations. In real-world scenarios, a user's final choice often reflects their immediate interests (left subfigure). We reconstruct the true evolution of interests by constructing fuzzy sets for each item in the interaction sequence (middle subfigure). The right subfigure provides an overview of the process of generating semantic IDs for recommendations based on item-related descriptions.practicality of this strategy. Inspired by the theory of fuzzy information processing (Tanaka et al. [1976] , Tanaka and Sommer [1977] ), we believe that making the absolute membership relations in traditional sets more flexible is another effective way to solve the problem. In other words, it is not necessary to strictly limit the modeling of user interests to the items they have interacted with. Therefore, we propose using a diffusion model (Ho et al. [2020] ) for fuzzy modeling of user interests, which enhances the model's performance by introducing perturbations during the training process.We have noticed existing work that introduces diffusion models into SR (Xie et al. [2024] ), such as DiffuRec (Li et al. [2023b] ) and DreamRec (Yang et al. [2023] ), which focus on Gaussian diffusion processes operating within a continuous state space. They add Gaussian noise to the embedded representations of candidate items for recommendation through a forward diffusion process until the noise reaches a pure state (standard normal distribution). Subsequently, they iteratively sample from this noise using a reverse denoising process guided by historical interaction information to recover meaningful representations and recommend items most similar to these representations.However, unlike our desire to fuzzily model interaction sequences, the aforementioned methods follow the form of diffusion models in the image domain and operate on candidate items. They introduce the crucial sequence information merely as conditional information, without leveraging the diffusion model's performance on it. On the other hand, these methods relax discrete interaction data into a continuous space and introduce noise, which may lead to distortion or loss of meaning in the original discrete space, as the addition of noise could push data points away from any meaningful discrete state. Therefore, we hope that state transitions occur under discrete conditions for the entire interaction sequence, which is discrete diffusion. Based on this, we have proposed our DDSR (Discrect Diffusion Sequential Recommendation model), which uses a directed graph to model sequential recommendation. In this model, all interaction items are viewed as nodes, and transitions between items are treated as directed edges. Discrete diffusion is used to enable structured transitions of nodes, with the resulting new sets treated as fuzzy sets, as shown in the middle of Figure 1 . By designing the transition matrix, we can achieve uniform transitions or importance-based transitions for the nodes, ensuring controllability. In Section 3.3, we theoretically demonstrate the reliability and effectiveness of modeling on these generated fuzzy sets, based on the principles of information diffusion. During the inference stage, we refer to the sampling formula for discrete diffusion but start from the historical interaction sequence rather than from noise, iteratively generating refined results.Furthermore, we have found that the excessive number of items involved in the recommendation problem leads to a high-dimensional transition matrix, resulting in inefficient diffusion transitions. Additionally, item IDs themselves do not contain any prior information, which poses a challenge in determining beneficial transition directions. To address this issue, we have further introduced semantic tags to replace meaningless item IDs, using quantization techniques and VQ-VAE to derive these tags from semantic information, thus reducing the size of the discrete space. We will provide specific details on how this can be achieved in 4.1, and a vivid illustration of this is given on the right side of Figure 1 . Simultaneously, the introduction of semantic information has enhanced the model's generalization capability and effectively solved the cold start problem. We conducted extensive experiments on three public benchmark datasets, comparing DDSR with several state-of-the-art methods. The results demonstrate that DDSR significantly outperforms baseline methods in various settings and effectively handles cold-start recommendations.2 Related Work 
sequence recommendation: 
 SR suggests potential subsequent items based on users' historical interaction records (Chen et al. [2022] , Shen et al. [2024] ). Early research primarily relied on Markov chains and matrix factorization techniques for recommendation (He et al. [2016] ). However, with the development of deep learning, efforts such as GRU4Rec (Hidasi et al. [2015] ), Caser (Tang and Wang [2018] ), and others have focused on designing neural network models to capture sequential dependencies in user behavior sequences. The introduction of the Transformer architecture (Vaswani et al. [2017] ) in SASRec (Kang and McAuley [2018b] ) pioneered SR and quickly became the mainstream method in the field. Additionally, BERT4Rec (Sun et al. [2019] ) utilizes bidirectional encoders to capture bidirectional dependencies in sequences, using a masked language model to predict the user's next action.Recent studies have shown that high-quality high-dimensional embeddings are crucial for obtaining accurate recommendation results (Hou et al. [2022] ). To this end, researchers are striving to leverage the rich attribute information of items to improve data representation. For example, TransFM (Pasricha and McAuley [2018] ) introduces arbitrary real-valued features through factorization machines, while S3-Rec (Zhou et al. [2020] ) designs four self-supervised learning tasks as pre-training objectives to learn context-aware data representations with attribute awareness. Furthermore, researchers like Hou et al. [2022] , Zhao [2022] , Harte et al. [2023] further utilize pre-trained language models to process item description texts, obtaining universal item representations with rich semantic information to enhance the performance. VQ-Rec (Hou et al. [2023] ) and TIGER (Rajput et al. [2024] ) further employ quantization techniques (Jacob et al. [2018] ) and RQ-VAE (Lee et al. [2022] ) to obtain tokenized semantic IDs for recommendations, replacing semantic embeddings.Diffusion models, inspired by non-equilibrium thermodynamics, have been introduced and demonstrated significant results in fields such as computer vision, sequence modeling, and audio processing (Dhariwal and Nichol [2021] , Rasul et al. [2021] , Ho et al. [2022] ). Most diffusion models are based on the Denoising Diffusion Probabilistic Model (DDPM) proposed by Ho et al. [2020] , as well as the score-based generative models (SGMs) proposed by Song et al. [2020] , targeting continuous data domains. We provide detailed descriptions of DDPM and SGMa in Appendix D to facilitate comparisons with the discrete diffusion approach we employ. Diffusion models in discrete state space are first described in Sohl-Dickstein et al. [2015] and later applied to text and image domains in D3PMs (Austin et al. [2021] ). VQ-Diffusion (Gu et al. [2022] ) utilizes them to eliminate unidirectional bias in text-to-image generation.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: The paper provides a detailed explanation of the problem background in the methods section and validates the hypothesis through multiple runs on various datasets with different characteristics in the experimental section. Additionally, the limitations are thoroughly discussed in Section 6. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
discrete diffusion process of ddsr: 
 In this section, we present the problem definition (Section 3.1) and illustrate how item sequences undergo discrete diffusion to obtain the corresponding fuzzy sets (Section 3.2). Finally, the effectiveness of this fuzzy modeling is theoretically demonstrated (Section 3.3). Please note that the actual diffusion and inference in DDSR occur at the semantic ID level, but this chapter discusses items.Let U be the set of users and V be the set of discrete items in the dataset, |U| and |V| represent the number of elements in their respective sets. For each user u ∈ U,v 1:n-1 = [v 1 , v 2 , . . . , v n-1 ]represents his historical interaction sequence sorted by timestamp. The goal of the model is to predict the next item v n that the user is most likely to interact with. To facilitate better discrete diffusion and for the convenience of subsequent theoretical derivations, we model each user's interaction sequence as a directed graph G u . In this graph, each item represented by a semantic ID is regarded as a node, while transitions between items are viewed as directed edges. Specifically, an edge exists from v i to v j if and only if v j is the next item interacted with by the user after v i .A typical diffusion model transforms data x 0 ∼ q(x 0 ) into a sequence of gradually noisier latent variables x 1:T = x 1 , x 2 , ..., x T via forward process q(x 1:T |x 0 ) = T t=1 q(x t |x t-1 ). In diffusion models within continuous state spaces, the forward distribution is typically set with q(x t |x t-1 ) = N x t | √ 1 -β t x t-1 , β t I as a hyperparameter controlling the level of noise added at each step. As the number of time steps T approaches infinity, x T converges to a standard Gaussian distribution. Beyond the limitations mentioned above, information loss due to diffusion into pure noise is another reason for unstable training and inadequate alignment of continuous diffusion with SR.Continuous diffusion always operates on embeddings, while in diffusion models within discrete state spaces, categories are directly transformed. Transition matrices [Q t ] ij = q(x t = j|x t-1 = i) are used to describe the probability of single-step diffusion transitions, where i and j represent categories within the domain. Denoting the one-hot version of x with the row vector x (bold), then the one-step transition probabilities can be expressed as:EQUATIONwhere Cat(x; p) is the categorical distribution corresponding to the one-hot row vector x with probabilities given by the row vector p, and x t-1 Q t is understood as a row vector-matrix product.Starting from x 0 , we obtain the following t-step marginal and posterior at time t -1:EQUATIONWe take the set V as the domain in SR. Each v i in the interaction sequence is represented as a one-hot encoding x 0 i . Using the transition form defined by 2 enables transitions to other nodes, denoted as x t i , in the domain with a certain probability at any time step t. Unlike continuous diffusion, which only allows noise addition, discrete diffusion models offer the advantage of controlling the data blurring process by selecting the transition matrix. Here, we present two strategies to select transition matrices, i.e. Uniform transition and Importance transition.Uniform transition. Similar to the study by Hoogeboom et al. [2021] , the natural idea is to maintain nodes with a certain probability β t ∈ (0, 1) unchanged. In contrast, in other cases, nodes are randomly transformed into any other node in the domain with equal probability (1 -β t )/(|V| -1). That isEQUATIONUniform transfer can be regarded as a special case of linear information allocation, thus theoretically affected by the size of the discrete space. It can compute the cumulative product Qt in closed form.Importance transition. For data with certain prior knowledge, we propose transitioning between more similar nodes rather than uniformly transitioning to any other state, thus defining the matrix:EQUATIONHere, d ij represents the distance between item v i and v j , calculated using the square of the Euclidean distance. The parameter σ 2 denotes the variance of the diffusion process. Consequently, the transition probabilities cannot be solved in closed form; instead, they can only be updated alongside the embeddings in the model. The importance transfer matrix adheres to the Gaussian informationdiffusion function f (x) = 1 σ √ 2π e -x 2 2σ 2. Therefore, it remains unaffected by the number of discrete points but necessitates the sample point distribution to closely resemble a Gaussian distribution.This modeling approach appears to align more closely with our intuition, as a user's interests at a given moment often form a cluster of similar nodes (items). Only upon the user's final selection of an item does the 'neighborhood' converge to a single data point. This point represents the representative of the interest cluster and the ambiguity in information naturally dissipates. In recommendation tasks, we can only access the user's final choice at each moment, without knowledge of the interest cluster, reflecting incomplete knowledge. Regardless of the transition method employed, the key lies in transitioning the sample space from incomplete to complete, as detailed in the subsequent section.Here, we aim to demonstrate that the discrete diffusion spaces generated by the two methods in Section 3.2 are completions of the original space, and the models constructed on these fuzzy sets are solvable and effective. We first provide the formal definition of a complete sample space.Definition 1. Let W denote the sample space. For any sample W ∈ W , if W is complete, i.e., unbiased estimates can be obtained through certain mathematical processing, then W is called a complete sample space; otherwise, it is called an incomplete sample space.In SR, W is a user's behavior sequence; W is all possible combinations of these behavior sequences; the domain V is all items in the dataset. Datasets in SR systems do not form a complete sample space, as they often consist of incomplete interaction data and potential selection biases. The principle of information diffusion ensures that when the given sample is incomplete, there exist reasonable diffusion functions that can improve non-diffusion estimates. Below we define information diffusion.Definition 2. An information diffusion about a set W is defined by a mapping µ : W × V → [0, 1], satisfying the following conditions:(1) ∀w j ∈ W , if v j is the observed value of w j , then µ(w j , v j ) = sup v∈V µ(w j , v).(2) ∀w j ∈ W , µ(w j , v) decreases as ∥v j -v∥ increases.(3) ∀w ∈ W , ν µ(w, v)dv = 1.The diffusion estimates obtained using uniform transition and importance transition, as defined in Section 3.2, clearly adhere to Definition 2. To illustrate that the space resulting from discrete state transitions provides more information than the original state space, it is necessary to further demonstrate that this space serves as a completion of the original space. In other words, the new metric space is complete, with the original metric space serving as its dense subspace. This will be more precisely discussed in the following theorem.Theorem 3.1. After information diffusion, the subsequent space must be an entirely separable metric space. Any model constructed in this space will assuredly possess a solution.Proof in Appendix A. According to Theorem 3.1, since the space after information diffusion is equidistant isomorphism with the original space, it can be used to replace the sample space with insufficient information in SR to establish a model. On this complete space, predictive models are solvable, demonstrating that modeling on fuzzy sets is a reasonable and effective approach.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes] Justification: The paper presents all theoretical results with clearly stated assumptions and complete proofs, which are either included in the main text or supplemented by detailed proofs in the appendix, ensuring both clarity and rigor.Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
obtaining semantic ids: 
 As mentioned in Section 3, the indices i and j in the transition matrix [Q t ] ij represent categories in the discrete space, making Q t a two-dimensional matrix with dimensions equal to the size of the discrete space. However, in recommendation tasks, the number of items involved can reach tens of thousands, posing a significant challenge in terms of computational resources if we were to use all item IDs as the discrete state space. Inspired by VQ-Rec and the recently proposed Tiger model by Google, we attempt to train recommendation models using semantic IDs instead of item IDs. A semantic ID is a codebook of length m. Assuming we set the size of the codebook to K, the entire codebook can represent K m categories. Though we set each code from a different codebook, the state space only needs m * K nodes to store them. Additionally, the use of semantic IDs further introduces semantic information, addressing the scarcity of information inherent in recommendations, while also allowing the model to extend to unseen items, thus enabling cold-start recommendations. We provide the specific method for obtaining semantic IDs in the Appendix B.Algorithm 1 Training of DDSR.Input: historical interaction sequence v 1:n-1 = c 1:n-1;1:m ; target item v n = c n;1:m ; transition matrix Q t ; Approximator f θ (•). Output: well-trained Approximator f θ (•).While not converged do: 1: Sample Diffusion Time: t ∼ [0, 1, . . . , T ]; 2: Calculate t-step transition probability:Q t = Q 1 Q 2 • • • Q t ; 3: Convert c n;1:m to one-hot encoding x 0n;1:m ; 4: Obtain the discrete state x t n;1:m after t steps by Equation 2, thereby obtaining the 'fuzzy set' c t 1:n-1;1:m ; 5: Modeling c 2;n;1:m based on 'fuzzy sets' through Equation 5; 6: Take gradient descent step on ∇L CE (ĉ 2:n;1:m , c 2:n;1:m ).Algorithm 2 Inference of DDSR.Input: historical sequence c 1:n-1;1:m ; well-trained Approximator f θ (•); sampling step T . Output: predicted target item v n . 1: Let x T = c 1:n-1;1:m ; 2: Let t = T ; 3: while t > 0 do 4:Use the trained f θ (•) to obtain predictions x0 with x t and t as inputs;5:Substitute x0 into equation 7 to obtain the distribution of t -1 step; 6: end while 7: ṽn = x 0 [-1; 1 : m]; 8: if the same code project exists: v n = ṽn ; 9: else: v n is the project in the space closest to ṽn .After introducing the Semantic ID, we convert the historical interaction sequence v 1:n-1 into sequence (c 1,1 , . . . , c 1,m ; c 2,1 , . . . , c 2,m ; . . . ; c n-1,1 , . . . , c n-1,m ), abbreviated as c 1:n-1;1:m . We convert them into one-hot encodings (x 0 1,1 , . . . , x 0 1,m ; . . . ; x 0 n-1,1 , . . . , x 0 n-1,m ), which is considered as the initial state for discrete diffusion. Then we perform discrete diffusion through the state transition formula defined in 2 (for more details, see Section 3.2) to obtain the discrete state after t steps x t i,j , for any i ∈ {1, . . . , n -1} and j ∈ {1, . . . , m}. Accordingly, the labels changes from c i,j to c t i,j . Then (c t 1,1 , . . . , c t 1,m ; . . . ; c t n-1,1 , . . . , c t n-1,m ) forms a "fuzzy set" of c 1:n-1;1:m , denoted as c t 1:n-1;1:m , which can also be viewed as the state x t of the diffusion transition at step t.Considering the suitability of the Transformer for sequence-to-sequence tasks, along with its welldemonstrated effectiveness in modeling sequential dependencies, we use it with an embedding layer as Approximator f θ (•) to predict c 2:n;1:m with c t 1:n-1;1:m as input. This approach differs from the common practice in diffusion models, which often focus on modeling noise. It aligns more closely with typical SR tasks that use v 1:n-1 to predict v 2:n , that is, the distribution pθ (x 0 |x t ). We have adopted sinusoidal time step embeddings, which are added after the embedding layer, allowing the model to capture information about the time steps. This process can be represented by:EQUATIONGenerally, the loss function of diffusion models is designed based on KL divergence, or it can be simplified to mean-squared error. However, guided by the theory of information diffusion, we choose to use a cross-entropy loss function, which is more suitable for recommendation tasks, to optimize our model without being constrained by the aforementioned methods. In the inference phase, we aim to emulate the reverse process of the diffusion model, iteratively producing refined recommendation results. According to Bayes' theorem, we haveEQUATIONwhere ⊙ represents the Hadamard product. Following the approach of Ho et al. [2020] and Hoogeboom et al.[2021], we employ the trained model f θ (•) as described in Section 4.2 to derive the distribution pθ (x 0 |x t ). Combining it with q(x t-1 |x t , x 0 ), we obtain the following parameterized expression:EQUATIONFor the historical interactions v 1:n-1 , we use c 1:n-1;1:m as x T , and starting from t = T , we iteratively execute Equation 7 until t = 0. This parameterization also allows us to perform inference for k steps at a time by predictingp θ (x t-k |x t ) = q(x t-k , x t | x 0 ) p θ ( x 0 |x t ), leading to efficiency improvements. After obtaining x 0 , we take its last component, which is a semantic ID of length m. If the corresponding item exists, we directly select that item; otherwise, we search for the item closest to it in the embedding space as the final recommendation result. The training and inference phase of DDSR are demonstrated in Algorithm 1 and Algorithm 2.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: The paper provides detailed descriptions of the experimental setup, including dataset specifications, parameter settings, and evaluation metrics, ensuring that the main results can be reproduced even without direct access to the code and data.Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. Justification: The paper appropriately reports statistical significance using t-tests with a significance threshold of p < 0.05, ensuring transparency and rigor in the experimental analysis.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: The paper thoroughly details the computer resources required for each experiment, including the type of compute workers, memory specifications, and execution times, enabling reproducibility and understanding of the computational demands of the study.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
experiment settings: 
 Datasets. We employ three real-world datasets to evaluate the performance of our DDSR model. Following some works on text-based recommendation (Li et al. [2023a] , Hou et al. [2022] ), these datasets include two specific subcategories from the Amazon Reviews dataset (Scientific and Office), and a cross-platform dataset known as Online Retail, which operates from the UK. Following the method of Hou et al. [2022] , we filter out users and items with fewer than five interactions. Subsequently, interaction behaviors within each sub-dataset are grouped by user and sorted chronologically. For the Amazon sub-datasets, product descriptions are formed by concatenating fields such as title, category, and brand, while for the Online Retail dataset, the description field is used. The product texts are truncated to 512 characters. Please refer to Table 1 for detailed descriptions of these datasets.Baselines. We compare DDSR with eight state-of-the-art SR methods, including two conventional SR methods, three methods based on semantic information, and three generative SR methods: 1). Conventional Baselines: SASRec (Kang and McAuley [2018a] ) utilizes a causal Transformer architecture with a self-attention mechanism to model user behavior. BERT4REC (Sun et al. [2019] ) proposes a bidirectional Transformer with a cloze task predicting the masked target items for SR.2). Semantic-based Baselines: UniSRec (Hou et al. [2022] ) utilizes the associated description text of items to learn transferable representations across different recommendation scenarios, using an enhanced mixture-of-experts adaptor to enhance domain fusion and adaptation. VQ-Rec (Hou et al. [2023] ) maps item text to a vector of discrete indices for learning transferable sequential recommenders. TIGER (Rajput et al. [2024] ) trains a Transformer-based sequence-to-sequence model with semantic IDs obtained from RQ-VAE to enhance its generalization ability.3). Generative Baselines: ACVAE (Xie et al. [2021] ) proposes an adversarial and contrastive variational autoencoder for SR combining the ideas of CVAE and GAN. DiffuRec (Li et al. [2023b] )Table 2 : Performance of different models. Bold (underline) is used to denote the best (second-best) metric, and '*' indicates significant improvements relative to the best baseline (t-test P<.05). 'R@K' ('N@K') is short for 'Recall@K' ('NDCG@K'). The features of items have been listed, whether ID, text (T), or both (ID+T).Online Retail R@10 N@10 R@50 N@50 R@10 N@10 R@50 N@50 R@10 N@10 R@50 N@50 [2020], we evaluate all models using metrics Recall@K and NDCG@K, and report experimental results for K = 10, 50. We employ the leave-one-out strategy for performance evaluation across all datasets. Concretely, we consider the last interaction as the test set, the second-to-last interaction as the validation set, and all preceding interactions as the training set. The ground-truth item of each sequence is ranked among all the other items while evaluating (Krichene and Rendle [2022] ). The implementation details of DDSR are illustrated in Appendix C.1.In this section, we compare the performance of DDSR with baseline models in terms of Top-K recommendation accuracy under consistent experimental conditions (same data preprocessing), as summarized in Table 2 . For models that recommend based on item IDs, we provide semantic information to them by jointly utilizing fixed text embeddings obtained from pre-trained BERT and the embeddings corresponding to item IDs in the model's embedding layer, to ensure fairness in the experimental setup. For all models, the final table records the better of the three methods, using only ID, only text, or both text and ID.We observe that text-enhanced SR methods (UniSRec, VQ-Rec, TIGER) tend to benefit from textual information, leading to improved performance compared to conventional methods in most cases. Notably, VQ-Rec, employing discrete semantic encoding, generally outperforms UniSRec, which relies on continuous text embeddings, across various settings. This is despite UniSRec already using techniques like parameter whitening and MoE-enhanced Adaptor to enhance textual information. We posit that an excessive emphasis on text similarity can yield suboptimal outcomes, while the conversion to codes mitigates the coupling between items and semantic information. The corresponding representations of the codes are relearned in the sequence-to-sequence model, allowing them to include more sequential structural information. While similarly based on discrete semantic encoding, the performance of TIGER is not stable. We do not rule out the possibility that there may be discrepancies between our implementation and the actual model, as it has not been open-sourced. Furthermore, we attribute the instability to TIGER's semantic ID length, which is limited to only 4 characters, potentially insufficient for expressing complex information.In methods grounded in generative models, the performance of DiffuRec and DreamRec, based on diffusion models, surpasses that of ACVAE, relying on GAN and VAE. This disparity arises from the inherent advantages of diffusion models over VAE and GAN, as they circumvent the issue of posterior collapse, wherein the generated hidden representations lack crucial information about user preferences. Notably, DiffuRec achieves superior performance despite its limited capacity to handle semantic information, yet it still exhibits recommendation performance comparable to VQ-Rec. This suggests that diffusion models can yield effective hidden representations of items and users. Variants Scientific Office Online Retail R@10 N@10 R@50 N@50 R@10 N@10 R@50 N@50 R@10 N@10 R@50 N@50 In all three datasets, DDSR achieves significant improvements, demonstrating the effectiveness of our approach. We attribute this success to two key factors. Firstly, the integration of semantic information mitigates data sparsity issues, as evidenced by the enhanced performance of semantic-based models on smaller datasets compared to traditional recommendation methods. Secondly, training on fuzzy sets generated by discrete diffusion furnishes the model with additional information. This is consistent with our theoretical analysis in Section 3.3, which posits that the diffused information space constitutes a completion of the original space, rendering models built on this enriched space effectively solvable. Moreover, while DiffuRec, relying on a continuous state space diffusion model, exhibits instability when confronted with larger and more intricate datasets, DDSR maintains a distinct advantage. We attribute this to DDSR's retention of the discrete space without transitioning it into a continuous one and introducing noise, thus circumventing the loss of meaningful information.We analyze the impact of semantic ID and discrete diffusion on final performance and conduct an ablation study to compare the results under different settings, as shown in Table 3 . The Uniform transition and importance transition are two discrete diffusion methods provided in Section 3.2, corresponding to different transition matrices. To control variables, these methods, along with the non-diffusion case, are all applied using semantic ID obtained through PQ. It can be observed that on larger datasets, the importance transition has relatively more advantages, and both methods outperform the non-diffusion scenario.To control variables and accurately evaluate the performance of IDs, diffusion is applied in the last two rows of the experiments (the method based on PQ ID with diffusion corresponds to the first and second rows, so it is not listed again). The PQ ID and RQ-VAE ID in the table corresponds to the two methods of obtaining semantic identifiers provided in Section 4.1. Random ID represents using a randomly generated codebook to replace the semantic identifiers, where the random identifier of item c i is simply c i = (c i,1 , . . . , c i,m ) with c i,j uniformly sampled from 1, 2, ..., K. Using Random ID means the model no longer gains semantic information. As observed, PQ ID exhibits greater stability and outperforms RQ-VAE ID across multiple datasets. Nevertheless, RQ-VAE ID requires less memory space due to its ability to achieve satisfactory representation with fewer codebook lengths. Semantic identifiers consistently outperform the Random ID, underscoring the significance of leveraging content-based semantic information. Indeed, models utilizing Random ID even underperform compared to SASRec. This can be attributed to SASRec's approach of setting independent embeddings for each item, rather than blending unrelated embeddings.Performance Analysis on Cold-Start Items. In this study, we evaluate the efficacy of DDSR in recommending cold-start items. Generating effective item embeddings without item information poses a challenge for SR models. To evaluate this, we partition the test data into two groups based on item popularity. For the Office dataset, the range [0, 5) demarcates long-tail Items, whereas for the Online Retail dataset, it is [0, 20). All other items are classified as Popular Items. The results are presented in Figure 2a . Notably, DDSR and VQ-Rec demonstrate substantial improvement over SASRec, which solely leverages a Transformer, particularly for long-tail Items, also referred to as the cold-start group. This is attributed to the integration of semantic information, enabling the model to acquire prior knowledge about items to some extent. Furthermore, DDSR demonstrates even greater enhancement compared to VQ-Rec in cold-start scenarios. We attribute this to the discrete diffusion method, which introduces a 'fuzziness' effect in the interaction records, facilitating the inclusion of items with fewer interactions in the training process. Step S. The sampling step S represents the diffusion step number divided by the number of inference steps executed simultaneously, rounded down to the nearest integer. We illustrate in Figure 2b the influence of various sampling step settings. The model demonstrates optimal performance with approximately 50 sampling steps, with a slight increase for more complex datasets, albeit without significant disparities. Excessive sampling steps prolong the inference time without commensurate performance improvements, while inadequate steps lead to decreased performance.Efficiency Analysis. We compared the time complexity and specific running overhead of DDSR with several other baseline algorithms, and the detailed results can be found in Appendix C.2. 
discussion: 
 We proposed the DDSR model for the sequential recommendation, which employed discrete diffusion to construct fuzzy sets of user interaction sequences. This process was iteratively refined during inference, utilizing the sampling formula for discrete diffusion to derive the ultimate recommendation outcomes. Notably, although DDSR had borrowed the form of diffusion and sampling over time steps from diffusion models, it fundamentally differed from directly using diffusion models. If we viewed sequential recommendation through the lens of causality, the interaction sequence was the 'cause' and the recommended item was the 'effect'. Diffusion models typically address the target, blurring the 'effect', whereas DDSR has blurred the 'cause', inspired by the theory of fuzzy information processing. Although dual assurances, both theoretical and experimental results, have been provided to substantiate the superior performance of DDSR, it is imperative to recognize its inherent limitations. Despite our efforts to implement efficient computational methods, the nature of diffusion and sampling processes inevitably results in reduced efficiency and increased time complexity. Potential refinements, such as approximating the diffusion process and accelerating the sampling algorithm, could offer effective strategies, which we will explore in future work.Theorem A.1. The model often has the following form:f : (U × V, ρ) → (U ′ × V ′ , ρ),which requires the model to be a continuous mapping from the original input-output space (which can also be the original space). The data distribution on the aggregated model set can be modeled:f ′ : (D, ρ ′ ) → (U ′ × V ′ , ρ),defined as γ such that f ′ = γ(µ). When the static distribution approximates the dynamic distribution, µ is the extended static scattering coefficient; otherwise, it is the linear information distribution coefficient γ, ensuring that the determination of f ′ is unique. Therefore, (D, ρ ′ ) is the completion space of (U × V, ρ), consisting of the complete set of (U × V, ρ) and its separation.In summary, the information diffusion space must be the completion space of the original space. Due to the same dimensionality as the original space, the modeling on the aggregated set is reliable.Next, consider the model (prediction model or simulation model) built on D(X). In practical applications, the input-output set is generally constructed in two ways based on the original data:e i = x i -x min x max -x min ,where e i is the normalized sequence; x i is the original sequence data; x min is the minimum value of the original sequence; x max is the maximum value of the original sequence. It is obvious thate i ∈ [0, 1], meaning (U × V ) ⊆ [0, 1],and the function boundary on D(X). When evaluating risk, the original sequence data generates input-output sets, which clearly have bounded intervals x i ∈ [x min , x max ], meaning the function boundary on D(X) is also bounded. Therefore, D(X) is necessarily a subset of the real number set.e i = BERT([[CLS]; w 1 , . . . , w c ]) ∈ R d W ,where "[; ]" denotes the concatenation operation. Next, obtaining Semantic IDs from e i can be achieved through the following two methods:Product Quantization (PQ). Similar to the VQ-Rec approach, we evenly divided e i into m subvectors e i = [e i,1 ; . . . ; e i,m ], each with a dimension of d W /m. Denote a p,j ∈ R d W /m as the j-th centroid embedding in the codebook corresponding to the p-th sub-vector. For each sub-vector, the index of the nearest centroid from the corresponding PQ codebook is selected to form its discrete code c i,p = arg min j ∥e i,p -a p,j ∥ 2 ∈ {1, 2, . . . , K}. The centroids are obtained through the commonly used Optimized Product Quantization (OPQ) method. Finally, c i = (c i,1 , . . . , c i,m ) is used as the semantic ID for item v i .RQ-VAE. RQ-VAE generate a set of codewords by quantizing the residuals. First, the input e i is encoded into r 0 using encoder E. Next, similar to PQ, the nearest centroid to it in the first codebook, assumed to be a p,1 , is found and its index is taken to form the first discrete code c i,1 . The residual is defined as r 1 := r 0 -a p1,1 . The same operation is performed to obtain c i,2 and this process is repeated m times to obtain the complete Semantic ID. RQ-VAE use loss function L(x) := L recon + L rqvae jointly trains the encoder, decoder, and the codebook, whereL recon := ∥e i -e i ∥ 2 is reconstruction loss, L rqvae := m-1 d=0 ∥sg[r i ] -a pi,i ∥ 2 + β∥r i -sg[a pi,i ]∥ 2 .Here e i is the output of the decoder, and sg is the stop-gradient operation Van Den Oord and Vinyals [2017] . Because the norm of the residuals decreases progressively, the importance of the codebooks obtained in this manner also diminishes with each level. Alternatively, it can be said that the encoding at each position has varying levels of granularity. In our experiments, we found that this approach requires a smaller codebook size than PQ, but it is slightly less stable.In fact, once the codebook is established, it remains fixed throughout the subsequent model training process. Therefore, the quality of the codebook directly affects the training outcomes. With the rapid development of large language models, considering the use of more advanced pre-training schemes to replace BERT could be an effective way to enhance performance. However, this is not the focus of our current research, we leave this topic for future exploration.All of our experiments were conducted on a single RTX 4090. We implemented our models based on PyTorch and the popular open-source recommendation library RecBole. We used (m = 32) × (K = 256) as the code representation scheme for PQ IDs and (m = 6) × (K = 256) for RQ-VAE IDs. For baseline models, to ensure fair comparison, we optimized all methods using the Adam optimizer and searched for hyperparameters to find the best results. Since we did not find open-source code for the TIGER model, we attempted to replicate it as faithfully as possible based on its paper; however, its performance may have been slightly lower in some experiments due to difficulties in unifying some details. For UniSRec and VQ-Rec, we utilize their code in a form that does not involve pretraining with the entire dataset, as we aim to evaluate all baselines and DDSR considering recommendations on a single dataset rather than cross-domain. The batch size was set to 2,048. The learning rate was adjusted among {0.0001, 0.0002, 0.0003, 0.0005, 0.001}. The model achieving the highest NDCG@10 result on the validation set was selected for evaluation on the test set. We employed an early stopping strategy with a patience of 10 epochs.Regarding diffusion, all models were trained on a diffusion process of 1000 steps, and the time step embeddings are implemented using cosine embeddings, similar to the work of Li et al. [2023b] . For the diffusion with uniform transition, we employ the cosine schedule proposed by Hoogeboom et al. [2021] to set the transition probabilities (1-β t ). For the diffusion with importance sampling, we adopt a linear schedule similar to the one used in Ho et al. [2020] , where σ 2 increases linearly from 10 -4 * K to 0.02 * K. Skip steps in the sampling process were chosen among {100, 50, 35, 28, 23, 20, 17, 15}. While there have been many works on accelerating sampling in continuous state space diffusion models, the development in discrete diffusion is still insufficient. Here, we adopted a basic uniform skip scheme for more efficient and effective sampling, which is one of our next research directions. If the evaluation steps do not divide 1000 evenly, the last step may be skipped. We list the time complexities of six baselines in Table 4 , where n denotes sequence length, d is the dimension of the hidden layers, and m is the length of the codebook used when semantic IDs are utilized. Most of these models are based on the transformer or its variants, hence the time complexity is O(nd 2 + dn 2 ). Only TIGER and our proposed DDSR model are trained using semantic IDs, making their time complexity m times that of other methods. We plan to make further improvements in our subsequent work.In addition, we compared the actual operational costs of the DDSR model with those of UniSRec and DiffuRec, as shown in Table 5 . Although we adopted the method of 'performing inference for k steps at a time', which reduced the sampling steps and lowered the evaluation time compared to DiffuRec, we must acknowledge that DDSR still has certain limitations in terms of operational costs, necessitating further improvements in future work. Diffusion models comprise a forward diffusion process and a backward denoising process. We begin with the widely recognized denoising diffusion probabilistic model (DDPM) Ho et al. [2020] . We start by defining our data distribution x 0 ∼ q(x 0 ) and a Markovian noising process q which gradually adds noise to the data x 0 to produce noised samples x T .In particular, each step of the noising process adds Gaussian noise according to a variance schedule given by β t :q(x t | x t-1 ) := N (x t ; 1 -β t x t-1 , β t I)Furthermore, q(x t | x 0 ) can be expressed as a Gaussian distribution. With α t := 1 -β t and ᾱt :=t s=0 α s , q(x t | x 0 ) = N (x t ; √ ᾱt x 0 , (1 -ᾱt )I) = √ ᾱt x 0 + √ 1 -ᾱt ϵ, where ϵ ∼ N (0, I).Here, 1 -ᾱt indicates the variance of the noise at an arbitrary timestep, and this can be used to define the noise schedule instead of β t .Using Bayes theorem, one finds that the posterior q(x t-1 |x t , x 0 ) is also a Gaussian with mean μt (x t , x 0 ) and variance βt defined as follows:EQUATIONIf we wish to sample from the data distribution q(x 0 ), we can first sample from q(x T ) and then sample reverse steps q(x t-1 |x t ) until we reach x 0 . Under reasonable settings for β t and T , the distribution q(x T ) is nearly an isotropic Gaussian distribution, so sampling x T is trivial. All that is left is to approximate q(x t-1 |x t ) using a neural network, since it cannot be computed exactly when the data distribution is unknown. To this end, Sohl-Dickstein et al. [56] note that q(x t-1 |x t ) approaches a diagonal Gaussian distribution as T → ∞ and correspondingly β t → 0, so it is sufficient to train a neural network to predict a mean µ θ and a diagonal covariance matrix Σ θ :EQUATIONTo train this model such that p θ (x 0 ) learns the true data distribution q(x 0 ), we can optimize the following variational lower-bound L vlb for p θ (x 0 ):EQUATIONEQUATIONEQUATIONEQUATIONWhile the above objective is well-justified, Ho et al. [25] found that a different objective produces better samples in practice. In particular, they do not directly parameterize µ θ (x t , t) as a neural network, but instead train a model ϵ θ (x t , t) to predict ϵ from Equation 17. This simplified objective is defined as follows:L simple := E t∼[1,T ],x0∼q(x0),ϵ∼N (0,I) ∥ϵ -ϵ θ (x t , t)∥ 2 2 (16)During sampling, we can use substitution to derive µ θ (x t ) from ϵ θ (x t , t):EQUATIONNote that L simple does not provide any learning signal for Σ θ (x t , t). Ho et al. [25] find that instead of learning Σ θ (x t , t), they can fix it to a constant, choosing either β t or βt . These values correspond to learning noise and the reverse process variance respectively.In this section, we introduce a Score-Based Generative Model (SGMs) Song et al. [2020] , specifically a diffusion model represented in the form of Stochastic Differential Equations (SDEs). SGMs model the forward diffusion process using the stochastic differential equation:EQUATIONwhere t ∈ [0, T ], and w signifies Brownian motion, p target represents target distribution. The function f (•, t) : R d → R d is a vector-valued function called the drift coefficient of x(t), and g(•) : R → R is a scalar function known as the diffusion coefficient of x(t). The functions f and g determine the type of prior distribution p prior to which the forward process will diffuse, and they are typically designed to make the prior distribution a Gaussian distribution. As a remarkable result from Anderson [1982] ,the reverse of the diffusion process is also a diffusion process, given by the following reverse-time SDE:dx t = [f (x t , t) -g(t) 2 ∇ x log p t (x)]dt + g(t)d w, (19) x T ∼ p T ≈ p prior ,where w is a standard Wiener process in reverse time. The term ∇ x log p t (x), which represents the score function of the marginal density p t , is the only unknown term in this reverse process. SGMs learns its approximate target s θ (x(t), t) through denoising score matching (DSM) Hyvärinen and Dayan [2005] , with s θ referred to as the denoising model:EQUATION)Here, λ(t) is a positive weighting coefficient, t ∼ U(0, T ). The joint distribution p 0t (x|x(0)) is the conditional transition distribution from x(0) to x(t), which is determined by the pre-defined forward SDE. To summarize, SGMs first utilize the diffusion process defined in Equation( 18) to obtain the distribution x t at intermediate time steps. Then, they minimize the loss defined in Equation( 20) to train the denoising model s θ and sample iteratively using the formula defined in Equation( 19) to obtain the final result.1. Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: The abstract and introduction clearly outline the primary contributions of the paper, including the background, challenges, motivation, overall framework, and experimental validation.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: The research ensures that ethical considerations are addressed and integrated into the study's design and execution, and the authors ensure anonymity is preserved.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.• The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: The paper addresses both potential positive and negative societal impacts of the research, highlighting potential misuses and discussing mitigation strategies, thereby ensuring a comprehensive consideration of broader societal implications. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The paper does not involve the release of data or models that pose a high risk for misuse, thus no safeguards are necessary. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix:  


Decision: The paper introduces an information theory-based improvement of sequential modeling to capture the randomness in user behavior. The paper was well-received by all reviewers and inspired (what looks like) a fruitful exchange between the reviewers and the authors who addressed the reviewer concerns. Most reviewers increased their scores after the rebuttal. To meet the expectations of the reviewers, the authors should incorporate the rebuttal in their final paper, including improving the clarity of the paper, discussion of computational complexity, and adding more experimental results.

Reviews: review: paper_summary: paper_summary: The paper presents a new model for sequential recommendation (SR) called DDSR, which aims to predict items of interest for users based on their past behavior. The authors critique conventional SR methods for not adequately capturing the randomness and unpredictability in user behavior. DDSR uses fuzzy sets of interaction sequences to better reflect the true evolution of user interests. Unlike common diffusion models that operate in continuous domains, DDSR uses diffusion transition processes in discrete state spaces, avoiding information loss by employing structured transitions. To tackle the inefficiency of matrix transformations in large discrete spaces, the model utilizes semantic labels derived from quantization or RQ-VAE instead of item IDs. Tests on three public datasets show that DDSR surpasses current state-of-the-art methods, proving its effectiveness in SR tasks.
strength_weakness: strength_weakness: I think a main weakness of the paper is a lack of clarity in writing. The method is still unclear to me, although this may be partly due to my lack of experience in recommender systems. I would condense and combine Sections 3 and 4 and move Section 2 to end. The technical novelty is also not entirely clear to me. Is it in the form of the transition matrices used for the discrete diffusion? Or is the technical novelty claimed that the items are recommended based on the output of running a few steps of discrete diffusion (adding noise to the features)?
questions: questions: Could the authors please elaborate on the analogy between fuzzy sets and discrete diffusion?
limitations: limitations: Yes, limitations adequately addressed.
paper_summary: paper_summary: The paper presents a diffusion model-based sequential recommendation from a novel information-theoretical perspective, which operates on discrete structural state spaces along with semantic labels improving efficiency and tackling cold-start issues.
strength_weakness: strength_weakness: See Above
questions: questions: 1.	In text such as line 235, 236, 251, the citation text is literally repeated.
2.	In text such as line 20-26, it would be better to add brackets to distinguish references from the main text.
3.	In line 74 and 144, it should be explicit that “2” and “4.1” refers to “Section 2” and “Section 4.1”.
4.	The usage and format of quotation marks is a bit confusing, see line 34, 219 and Table 1 caption for examples.
limitations: limitations: There is no significant limitation. Perhaps the authors could consider give a rough discussion or comment on how the computational complexity (or the coefficient constant) of the proposed method can be improved.
paper_summary: paper_summary: This paper studies the problem of fuzzy modeling for sequential recommendation. The work proposes to leverage the fuzzy sets of interaction sequences for modeling the nature of users' evolving real interests. It also introduces the discrete diffusion modeling specifically for the discrete data. The experiment results showcase the better model performance in comparison with several included baselines.
strength_weakness: strength_weakness: 1. Insufficient baseline comparison for the conventional sequential recommender models, as the two included models are outdated. 
2. Insufficient empirical analysis in terms of the model efficiency. Firstly, the runtime cost comparison is missing, which is however very important to recommender system prototype evaluation for practical deployment consideration. Secondly, based on the complexity analysis, it is crucial to showcase the performance variations (effectiveness/efficiency) by varying the codebook length $m$.
questions: questions: 1. How many times of performance evaluations are conducted for the average results?
2. What are the real runtime costs for the proposed model and other baselines for comparison?
limitations: limitations: Please see the weakness for details.
paper_summary: paper_summary: The paper presents the DDSR model for Sequential Recommendation (SR), which better captures user interest evolution by using fuzzy sets of interaction sequences. Unlike traditional methods, DDSR effectively handles the unpredictability of user behavior and addresses cold start issues. Experiments on benchmark datasets show that DDSR outperforms existing methods, demonstrating its effectiveness in SR tasks.
strength_weakness: strength_weakness: 1. The paper lacks an algorithm to describe the entire model operation. Including this would greatly enhance readers' understanding of the model and its theoretical underpinnings.
2. All experimental results are quantitative. It is recommended to supplement with a case study or visual experiment.
3. Although the author has analyzed the time complexity, I believe most readers would also appreciate a comparison of actual running times.
questions: questions: See comments above.
limitations: limitations: This paper has discussed limitations.

Comments: title: Response to Rebuttal
To authors
comment: Dear Reviewer,

Thank you for your recognition of our detailed responses and for increasing the rating. We will ensure that the additional experiments are included in the revised manuscript and reflected in our submission. We sincerely appreciate your valuable feedback, support, and the time and effort you have devoted to our work once again！

Best regards！
Dear Authors,

Thanks for the detailed responses. They have addressed my major concerns, and I will increase my score to positive rating. Please make sure the additional experiments will be included in the revision for publication.
Thank you for your positive feedback and for taking the time to review our work. We greatly appreciate your thoughtful comments and are glad that we could address your concerns. Your support and acceptance are highly valued.
Best regards！
Thanks for the response which havs addessed my main concerns, and thus I keep my original score of acceptance.
Dear Reviewer,

I hope this message finds you well. I understand that you may have a busy schedule, and I truly appreciate the time and effort you are dedicating to reviewing our manuscript.

As the deadline for feedback approaches, I wanted to kindly check in and see if there is any additional information or clarification we can provide to assist with the review process. Your insights are invaluable to us, and we look forward to any feedback you might have.

Thank you again for your time and consideration.

Best regards！
Dear Reviewer,

Thank you very much for your positive feedback and for recognizing the improvements in our revised manuscript. We greatly appreciate your support, particularly regarding the inclusion of the pseudo-code and the new experimental results.

We will promptly upload the revised version to OpenReview before the review period ends, as per your suggestion.

Thank you again for your valuable guidance and for increasing your score.

Best regards！
I thank the authors for their thoughtful modifications to the paper - is it possible to upload this revision to OpenReview before the review period ends? I think the inclusion of the pseudo-code for training and sampling algorithms is a big step forward. Also, with the inclusion of the new experimental results I am glad to increase my score.
Dear Reviewer,

We would like to once again express our sincere gratitude for the time and effort you invested in reviewing our manuscript. We deeply appreciate your constructive feedback and are thankful for the higher evaluation during the rebuttal phase. Your insights have not only enhanced our work but also encouraged us to further refine our research. Based on your suggestions, we have made appropriate revisions to our paper accordingly.

Thank you once again for your thoughtful review and for recognizing the potential of our work. We are extremely grateful for your expert guidance and assistance.

Best regards!
You have addressed some of the concerns I raised in my previous review. I believe this work is indeed novel and meets the standards required for this conference. As a result, I am inclined to raise my score.
Best regards,
Dear Reviewer,

I hope this message finds you well. I am writing to inquire about the progress of my rebuttal submission. I understand that the review process can be quite involved, but I would appreciate any updates you could provide regarding the status of the review.

Thank you very much for your time and effort!

Best regards！
Dear Reviewer,

I hope this message finds you well. I am writing to inquire about the progress of my rebuttal submission. I understand that the review process can be quite involved, but I would appreciate any updates you could provide regarding the status of the review.

Thank you very much for your time and effort!

Best regards！