Title: Fading to Grow: Growing Preference Ratios via Preference Fading Discrete Diffusion for Recommendation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Recommendation

Discrete Diffusion

Preference Ratios
Abstract: Recommenders aim to rank items from a discrete item corpus in line with user interests, yet suffer from extremely sparse user preference data. Recent advances in diffusion models have inspired diffusion-based recommenders, which alleviate sparsity by injecting noise during a forward process to prevent collapse of perturbed preference distributions. However, current diffusion‑based recommenders predominantly rely on continuous Gaussian noise, which is intrinsically mismatched with the discrete nature of user preference data in recommendation. In this paper, building upon recent advances in discrete diffusion, we propose \textbf{PreferGrow}, a discrete diffusion-based recommender modeling preference ratios by fading and growing user preferences over the discrete item corpus. PreferGrow differs from existing diffusion-based recommenders in three core aspects: (1) Discrete modeling of preference ratios:
PreferGrow models relative preference ratios between two items, where a positive value indicates a more preferred one over another less preferred. This formulation aligns naturally with the discrete and ranking-oriented nature of recommendation tasks.
(2) Perturbing via preference fading: Instead of injecting continuous noise, PreferGrow fades user preferences by replacing the preferred item with alternatives---physically akin to negative sampling---thereby eliminating the need for any prior noise assumption.
(3) Preference reconstruction via growing: PreferGrow reconstructs user preferences by iteratively growing the preference signal from the estimated ratios. We further provide theoretical analysis showing that PreferGrow preserves key properties of discrete diffusion processes.
PreferGrow provides a well-defined matrix‑based formulation for discrete diffusion-based recommendation and empirically outperforms existing diffusion‑based recommenders across five benchmark datasets, underscoring its superior effectiveness.
Our codes are available at \url{https://anonymous.4open.science/r/PreferGrow_Commit-2259/}.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces PreferGrow, a novel recommender system designed to overcome data sparsity by using a discrete diffusion model. Unlike other diffusion-based recommenders that rely on continuous noise, PreferGrow operates directly on the discrete set of items. It perturbs data by \"fading\" user preferences—replacing a liked item with alternatives in a process similar to negative sampling. The model then learns to reconstruct user preferences by \"growing\" them from these faded states, focusing on relative preference ratios between items rather than absolute scores. This approach, which is theoretically grounded with proofs of its key properties, demonstrates superior performance over existing methods across five datasets\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. The paper introduces a novel and significant approach to recommendation systems by leveraging discrete diffusion models. It diverges from the predominant trend of using continuous Gaussian noise in diffusion-based recommenders, which is often a mismatched assumption for discrete item data. \n\n2. The authors conduct extensive experiments on five benchmark datasets, comparing PreferGrow against a variety of classic and state-of-the-art diffusion-based recommenders.\n\n3. The paper is of high quality, grounded in solid theoretical analysis. A key contribution is the development of the  idempotent fading matrix and the formal proof that its idempotence is crucial for ensuring the Markov property and reversibility of the diffusion process.\n\nWeakenesses\n1. The most significant weakness, which the authors openly acknowledge, is the model's scalability. The pair-wise setting, which proves most effective, requires modeling preference ratios across all item pairs. This becomes computationally infeasible for recommendation scenarios with extremely large item corpora.\n\n2. The paper notes that PreferGrow incurs longer training times than prior diffusion-based recommenders. This is attributed to the uniform sampling of timesteps during training, where modeling preference ratios at different stages of \"fading\" can have varying levels of difficulty.\n\n3. The hyperparameter analysis reveals that the model's performance is highly sensitive to the personalization strength parameter $w$. Although the authors frame this positively by noting that $w$ can be tuned during inference without retraining, high sensitivity can still be a practical challenge, potentially requiring careful and costly tuning for each specific application or dataset to achieve optimal results.\n\nQUESTIONS:\n1. Your analysis shows high sensitivity to the personalization parameter $w$. Could you provide a qualitative analysis showing how the actual recommendations change for a user as $w$ is adjusted? \n\n2. Do you have depolyed your method into real world industrial ?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 1, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
This paper introduces PreferGrow, a discrete diffusion-based recommender system that models preference ratios by a novel process of preference fading and growing, specifically tailored for the discrete nature of user preference data. Unlike prior diffusion-based recommenders that rely on continuous Gaussian noise, PreferGrow directly operates on discrete items using a theoretically grounded matrix-based framework. The authors claim that this approach not only better captures the discrete nature of recommendation tasks but also improves empirical performance across multiple datasets.

STRENGTHS AND WEAKNESSES:
Strength

a) The paper offers solid theoretical backing, proving Markov properties, reversibility, and the validity of the fading process using an idempotent fading matrix.

b) Extensive experiments across five benchmark datasets consistently show PreferGrow outperforming classical, item-level, and score-level diffusion-based recommenders.

Weakness
The approach could face significant computational challenges when scaling to very large item sets due to the all-pair preference ratio modeling.

QUESTIONS:
NA

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

comments: SUMMARY:
This paper proposes PreferGrow, a recommender systems to learn item representations through discrete diffusion models. Existing works based on diffusion models in recommendation systems suffer from lack of negative sampling as well as misalignment in the continuous probability modeling space vs. the discrete nature of user interaction. To tackle these two problems, PreferGrow introduces a preference fading discrete diffusion model tailored for recommendation. Specifically, during the forward pass of diffusion process, the target item embedding corruption is guided by fading matrices; and during the backward pass, user embedding generated by SASRec, as well as the corrupted item embedding are concatenated to predict the preference ratio.

STRENGTHS AND WEAKNESSES:
Strength: 
1. this paper is technically well-motivated -- it identifies misalignment between the continuous probability modeling space in diffusion models vs. the discrete nature of user interaction.

Weakness:
1. the clarity and writing needs to be significantly improved. the technical part of this paper is extremely difficult to understand. 
2. i'm confused by the incorporation of user embedding, which other diffusion-based baseline does not explore. The incorporation of user embedding seems agnostic of the idea proposed in this paper.
3. while the authors try to provide theoretical proofs, the provided proofs are mostly general diffusion derivations and the added value is very marginal.

QUESTIONS:
please refer to the weakness. 

furthermore, I think the authors should comparatively discuss efficiency of the proposed framework.

LIMITATIONS:
limitations are properly discussed
TITLE: Official Review
CLARITY: 1
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-08

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel recommendation system model named PreferGrow, which is based on a discrete diffusion process to address the core challenge of user preference data sparsity. Unlike previous diffusion recommendation models that primarily rely on continuous Gaussian noise, PreferGrow operates directly on the discrete item space. Its core contributions are threefold:\n\n1.  **Discrete Preference Modeling**: The model directly models \"preference ratios\" between pairs of items, which is more aligned with the ranking-oriented nature of recommendation tasks.\n\n2.  **Preference Fading**: In the forward diffusion process, instead of injecting noise, the model employs a \"preference fading\" mechanism (analogous to negative sampling), where a user's liked item is replaced with other items with a certain probability. This avoids assumptions about a prior noise distribution.\n\n3.  **Preference Growing**: In the reverse generation process, the model iteratively \"grows\" and reconstructs user preferences from the estimated preference ratios.\n\nThe paper provides a solid theoretical foundation for PreferGrow, including a mathematical formulation based on an idempotent fading matrix, and theoretically proves the Markovian nature and reversibility of the process. Experimental results on five benchmark datasets show that PreferGrow consistently outperforms existing state-of-the-art recommendation algorithms, including other diffusion-based models.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1.  **Originality:** The core idea of this paper is highly original. Applying a discrete diffusion model to recommendation systems and proposing \"preference fading\" as the forward process is a significant breakthrough from the existing paradigm that relies on continuous Gaussian noise. The modeling objective of \"preference ratios\" is also insightful, drawing inspiration from the success of ideas like DPO (Direct Preference Optimization) in language models, making it naturally suited for ranking tasks.\n\n2.  **Quality:** The paper's theoretical foundation is very solid. The authors present a clear, matrix-based mathematical framework, particularly introducing the key concept of an \"idempotent fading matrix\" and providing complete proofs of the model's Markovian nature and reversibility around it. This provides strong theoretical support for the model's effectiveness.\n\n3.  **Clarity:** The paper is well-organized and clearly written. The introduction clearly articulates the research motivation and effectively distinguishes this work from existing methods. Figure 1 intuitively contrasts different diffusion recommendation paradigms, and Figure 2 clearly illustrates the PreferGrow pipeline, greatly enhancing the paper's comprehensibility.\n\n4.  **Significance:** This work is of significant importance to the subfield of diffusion-based recommendation systems. The experimental section comprehensively compares PreferGrow with a variety of baselines (including classic, item-level, and score-level diffusion models) and demonstrates its consistent and significant performance advantages across multiple datasets. If these results can be widely replicated, this model could become an important benchmark in the field. Furthermore, the flexibility of the model design (supporting Point-Wise, Pair-Wise, Hybrid modes) also increases its potential application value.\n\n**Weaknesses:**\n\n1.  **Fundamental Scalability Flaw:** This is the most critical weakness of the paper. The authors acknowledge this issue in Section 5, but its severity is somewhat underestimated. The model's core is to model the \"preference ratios\" of all item pairs, which leads to a computational/memory complexity of at least the square of the number of items, $|X|^2$. This is completely infeasible in real-world recommendation scenarios with millions of items. Compared to contemporary work, this is a step backward. For example, models based on negative sampling (like its encoder base, SASRec) and embedding diffusion models (like PreferDiff) have much better scalability. If the core algorithm of a recommendation system paper cannot be applied to large-scale datasets, its practical significance and impact will be considerably diminished. The proposed method feels more like a \"toy\" model validated on small datasets, lacking consideration for real-world application challenges.\n\n2.  **Theoretical \"Elegance\" at the Cost of Realism:** The paper's theoretical derivations heavily rely on a core assumption: the fading matrix $E$ is idempotent ($E^2=E$). To satisfy this condition, the paper further derives that all column vectors of $E$ must be identical ( $ E=\\frac{\\vec{p} _{T}\\cdot\\vec{1}^{\\top}}{\\vec{1}^{\\top}\\vec{p} _{T}}$ ). This means that during the forward \"fading\" process, regardless of the original preferred item $x_0$, it will be replaced according to the **same** target distribution $\\vec{p}_T$. This is a strong simplifying assumption that somewhat ignores the heterogeneity among items. For example, a reasonable substitute for a \"faded\" shirt item should be other clothing, not a refrigerator. The model cannot capture such fine-grained relationships, which calls its physical intuition and modeling capability into question. In contrast, although models like DDSR operate on simple categorical distributions, they at least do not introduce such a strong and counter-intuitive global structural assumption.\n\n3.  **Questionable Reliability of Experimental Evaluation:** The paper reports performance surpassing all baselines on five datasets. However, the reliability of these results is questionable. The authors admit that due to computational resource constraints, all experiments were run once with a fixed random seed, without reporting error bars or conducting statistical significance tests. In machine learning, especially in recommendation systems, model performance is highly sensitive to random factors like data splits and parameter initialization. Without statistical tests, we cannot determine whether the reported performance gains are genuinely effective or merely due to random chance or specific hyperparameter tuning. For a paper seeking to advance the state-of-the-art, this is unacceptable.\n\n4.  **Marginal Contribution vs. Disproportionate Complexity:** The core idea of this paper can be seen as \"grafting\" recent work in discrete diffusion and alignment (e.g., DPO) onto recommendation systems. While the combination is innovative, the resulting model complexity (e.g., the extremely tedious loss function derivation in Appendix) may be disproportionate to the actual, statistically unverified performance gains it brings. Compared to contemporary works, for instance, DDSR, which directly performs diffusion on a discrete state space, is conceptually more straightforward; PreferDiff, although a continuous model, has a more intuitive optimization objective designed for negative sampling. It is debatable whether the series of complex concepts introduced by PreferGrow, such as \"preference ratios\" and \"idempotent fading matrix,\" is the right path to solving the problem.\n\nQUESTIONS:\n1. Please answer other questions in the weaknesses section.\n\n2.  **Further Discussion on Scalability**: The paper candidly admits that scalability is a major limitation. Could the authors provide specific empirical data on how training/inference time varies with the number of items `|X|` on the datasets used in the experiments? Additionally, regarding the \"sparse fading matrix\" mentioned in future work, how can the critical \"idempotency\" property be guaranteed when designing such a matrix? This seems to be a non-trivial challenge.\n\n3.  **The Deeper Connection Between \"Preference Fading\" and Negative Sampling**: The paper analogizes \"preference fading\" to negative sampling. Traditional negative sampling strategies (e.g., sampling based on item popularity) have been proven very effective. How do the authors view the relationship between the uniform replacement strategy in the \"Pair-Wise\" setting and these mature negative sampling techniques? Could more complex sampling strategies (like popularity bias) be incorporated into the design of the non-preferred state to further enhance performance?\n\n4.  **Sensitivity Issue of Personalization Strength `w`**: According to the hyperparameter analysis in the paper, model performance is highly sensitive to the personalization strength `w`, which is selected during the inference phase. Does this imply that for a new deployment scenario, a validation set must be reserved specifically to tune `w`? This would introduce additional overhead and could lead to `w` overfitting to the validation set. Have the authors considered enabling the model to adaptively learn `w`, or are there more robust strategies for selecting `w`?\n\n5.  **Explanation of $x_{-1}$ in the Point-Wise Setting**: In the paper's Point-Wise setting, the authors introduce a \"universal hard negative item.\" How is this item actually represented? Is it a learnable embedding vector? What is the essential difference between this and simply selecting a fixed item from the item pool as a global negative?\n\nLIMITATIONS:\nYes, the authors have adequately discussed the limitations of their work. They pointed out two key issues: long training times and poor scalability on very large item sets. The authors not only identified these problems but also proposed constructive potential solutions as future research directions.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a discrete diffusion-based recommendation method that models relative preference ratios by fading and growing user preferences over the discrete item corpus, directly aligning with the ranking-oriented, discrete nature of recommendation tasks. It replaces the preferred item with alternatives, physically akin to negative sampling. During backward generation, it iteratively grows the preference signals from the estimated ratios.\n\nSTRENGTHS AND WEAKNESSES:\nAdvantage:\n1. The discrete modeling of preference ratios that aligns with the ranking-oriented nature of recommendation tasks. Noise is added by replacing the preferred item with alternatives, enabling explicit negative sampling and removing prior noise assumptions. \n2. Idempotent fading matrix with closed-form solution whose idempotent property is critical for ensuring both the Markov property and reversibility. Flexible architecture supports point-wise, pair-wise, and hybrid preference modeling. \n3. Extensive experiments on five datasets showing consistent outperformance of existing diffusion-based recommendation method. \n\nMy concern:\nIn this paper, the author adopts an idempotent matrix E as the fading matrix, which provides desirable properties for theoretical analysis (e.g., the Markov property and invertibility). However, at the implementation level, the proposed method uses a unified setting for the fading matrix, as shown in Appendices C.1, C.2, and C.3. In this setting, forward denoising either results in “staying at the current item” or “randomly replacing the current item with others uniformly.” I am concerned that such a design might degrade the model into a pure weighted negative sampling method, thereby eliminating the advantages of employing a diffusion model. Let me elaborate further. In item-level and preference score-level diffusion methods, the representations of items and preference scores are typically correlated during the forward process. However, in this paper, due to the uniform design of the fading matrix, the reference items at each step appear to be mostly independent of one another (e.g., “staying at the current item” or “randomly replacing the current item with others uniformly”  between two steps). As a result, the proposed method may degrade into a simple negative sampling scheme, as illustrated in Appendices C.1–C.3, thus undermining the core motivation and benefits of the diffusion model. In my opinion, the idempotent matrix E should not be designed in such a uniform way. At the very least, some degree of semantic or structural correlation should be preserved in the matrix—for example, assigning a higher probability to replacing the current item with a similar one.\n\nQUESTIONS:\nPlease see Weakness. If there is any misunderstanding in my interpretation, please feel free to clarify it in the rebuttal. I would be happy to revise my score if my concerns are adequately addressed.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-19"}