Kati8/ICLR_2023_2025 huggingface dataset:
authors: Rui Lu
authors: Runzhe Wang
authors: Kaifeng Lyu
authors: Xitai Jiang
authors: Mengdi Wang
reviews: 
Comments: This paper investigates textual hallucinations, where diffusion models generate locally accurate but globally nonsensical symbols. The authors introduce the local dependency ratio (LDR) as a probe to quantitatively measure the model's dependency on local regions when conducting denoising. With LDR, the authors observe that the textual hallucinations are caused by the local generation bias, And this phenomenon exists across various denoising network architectures including MLP and transformers. The authors also provide a theoretical analysis of its underlying mechanism.
Strengths: 1. The authors explore textual hallucination in generative models, addressing a crucial yet often overlooked research problem. 

2. They introduce the local dependency ratio (LDR) to examine how diffusion models depend on localized regions of input noise during denoising and generation. The analysis is both robust and engaging. 

3. They present a sound theoretical examination of the mechanism underlying textual hallucination.
Weaknesses: 1. Labeling this issue as "textual hallucination" might cause confusion with hallucination problems in large language models (LLMs). It’s advisable to adjust the task description or add a notice at the start of the main paper for clarity.

2. Following up on the previous question, would this issue also occur in Vision Language Models (MLLMs)?

3. The experimental setup appears less practical and somewhat limited in scope. Most experiments were conducted on small, toy datasets and within narrow domains, such as digit generation. Could larger datasets and real-world applications, like image generation on ImageNet, be explored? Additionally, would textual hallucination issues appear in other real-world contexts?

4. The theoretical analysis is constrained to two-layer MLP learning parity points on a hypercube, creating a significant gap between MLPs and more complex models like UNet and DiT. The relevance of the proposed theory to real-world models is therefore uncertain and requires further investigation.

5 It is suggested to include potential strategies to address textual hallucination issues. Although fully solving this problem may be challenging, proposing ways to partially mitigate this limitation in generative diffusion models would be valuable.
Recommendation: 6
Reviewer confidence: 3
Soundness/correctness: 4
Clarity: 2
reviews: 
Comments: This paper explores the score-based diffusion models' text hallucination, where each generated symbol is accurate but arranged nonsensically. Through the experiments on various settings of toy datasets, the authors observed that this phenomenon is related to the local generation bias of the score model, which is the tendency of the score model to not regard the entire input region and their correlation. 
In the process, a metric called Local Dependency Ratio(LDR) was newly suggested to quantify how the diffusion model relies on specific local region of the input for denoising.

The authors also found that this bias emerges across various architectures, including CNN, MLP, and transformers.
Lastly, they examined training dynamics on specifically designed data to propose a theoretical perspective on the emergence of local generation bias.
Strengths: -The paper newly identifies the possible cause of the text hallucination, with extensive experiments over various dataset settings of symbols and their correlation.

-The suggested metric of LDR is intuitive and shows good explainability of the score model behavior.
Weaknesses: -Since most of the results are obtained from the simple controlled data, there's still some gap to fill to extend the results into the real-world language data.

-The proposed theoretical analysis on local generation bias with training dynamic is limited for the used problem setting(e.g. most conclusions in Section 5.1 require Assumption 5.1).
Recommendation: 6
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: This paper examines the issue of textual hallucinations in score-based diffusion models. The authors identify this phenomenon as stemming from a local generation bias, where the denoising networks rely heavily on highly correlated local regions of input data, leading to a failure in capturing the global structure. The paper proposes the Local Dependency Ratio (LDR) as a new metric to quantify this bias, demonstrating its persistence throughout training and across various network architectures. The findings suggest that the hallucinations are a result of the training dynamics rather than limitations in the model's architecture
Strengths: 1.This paper proposes a new metric for measuring text hallucinations.The introduction of Local Generation Bias and the Local Dependency Ratio (LDR) provides a new perspective on understanding hallucinations in diffusion models.
2.The paper features a rigorous derivation process, and its logic is sound and clearly structured.
Weaknesses: 1. Some of the core arguments are not well articulated:
The paper suggests that the causes of hallucinations stem from fundamental training dynamics rather than architectural limitations. However, the comparative experiments in Figure 4 only demonstrate that different model architectures exhibit hallucinations, it does not completely decouple architectural influences
2. Insufficient experiments: The article conducts comparative experiments under two different experimental settings on only two datasets. Including more validation experiments would enhance the reliability of the paper.
Recommendation: 6
Reviewer confidence: 2
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: This paper investigates the local generation bias phenomenon in diffusion models and its relationship to text hallucination. The authors propose the Local Dependency Ratio (LDR) metric to provide empirical insights into this bias and present theoretical analysis attempting to explain the underlying causes of hallucinations.
Strengths: The paper offers a novel perspective on diffusion model hallucinations through the lens of local dependency analysis.
The introduction of the Local Dependency Ratio (LDR) metric, along with the Quarter MNIST dataset, provides a useful probe for evaluating models' ability to consider global information versus local information.
Weaknesses: - The central claim about hallucinations stemming from training dynamics rather than architectural limitations lacks substantial support:
  - No proposed training paradigm to demonstrate improvement.
  - Recent work (Stable Diffusion 3 [1]) achieves significant typography improvements through architectural changes (MMDiT)
- Regarding the experiments in section 4.2, it is possible that the network capacity might be insufficient to learn global relations. There should be more details on the model architecture. (It is claimed that the details of the model architecture are in the appendix, but they don't.)
- In addition to the previous point, the DiT model seems to be able to memorize the training samples very well. The model does not need to learn the true latent relation ($s_1 + s_2 = s_3 + s_4$) if it can simply memorize all training samples. This mitigates the validity of the claim that diffusion models cannot learn the global relation.
- The description "both have even numbers" about the Parity Parenthesis dataset seems incorrect. The claim "random guess has 50% chance of satisfying parity requirement" seems questionable.
- The authors attempt to theoretically prove that neural networks are prone to learning the marginal distributions instead of the global correlations by analyzing a two-layer MLP. However, the analysis may not generalize to U-Net or Transformer-based diffusion models. The DiT model is able to memorize Quarter MNIST training samples, demonstrating its capability of learning more than the marginal distributions.

[1] Esser P, Kulal S, Blattmann A, et al. Scaling rectified flow transformers for high-resolution image synthesis[C]//Forty-first International Conference on Machine Learning. 2024.
Recommendation: 6
Reviewer confidence: 3
Soundness/correctness: 2
Clarity: 2