reviews: 
Comments: The paper performs a structured and theoretically motivated investigation of hallucination in text generation with diffusion models.
The authors construct two toy problems, parity parenthese and quarter MNIST, to investigate the phenomenon. The empirical evaluation along training steps and different generative architectures reveals that DMs fail to learn complex rules governing symbol distributions without access to strong priors. Using the proposed Local Dependency Ration (LDR) the authors demonstrate that symbols are generated with mainly localized attention, despite architecture design allowing for the capture of global structure. 
The authors further underpin these empirical observations by providing theoretical proof that the DM training paradigm can get stuck at a local saddle set, which forces the model to learn marginal, localized distributions.
Strengths: - **Key problem from a new perspective.** In investigating text generation with diffusion models, the authors work on a core problem in the field with high relevance. The more theoretical approach chosen by the paper offers a novel perspective and interesting insights. 
- **Solid evaluation**. The authors construct two suitable proxy tasks for their analysis on which they perform compelling evaluations across training steps and architectures. 
- **Novel probing tool.** The proposed LDR metric is an interesting approach that could potentially benefit other fields and aid future interpretability research
Weaknesses: - **Generalization to current models**. While the paper does perform valuable evaluations and theoretical analysis on a constrained problem set, it fails to analyze strong, general-purpose models. Recent models like SD3, Flux, Playgroundv3, SD3.5, or Recraftv3 have become progressively better at text generation. At least if the text to be generated is supplied in the model prompt. Since many of these models' weights are openly accessible, it would be key for the paper's evaluation to investigate them using the proposed LDR metric. Specifically, insights into whether these models leverage more global structure or if improvements are tied to the strong priors of text conditioning 
- **Readability.** In the same vein, the theoretical sections of the paper may be hard to follow for some readers, although the actual problems, like the proposed Quarter MNIST, are intuitive and easy to understand. The paper would benefit from interleaving some more plastic examples into the methodological definitions and theoretical analyses to appeal to a broader audience. 
- **Lack of derived recommendations/future work.** The authors outline multiple issues in DMs that lead to hallucinations in text generation. Despite numerous insights, the paper does not come up with approaches or recommendations that would allow future work to address the same issues. 

## Minor Comments 
- The differing y-axis of the two plots in Fig. 5 is somewhat misleading.
- Similarly, the caption of Fig. 5 refers to a total $T=100$, although the right Figure goes up to $t=168$
- **Inconsistent Figure Placement.** The authors sometimes place Figures at the top of a page and, at other times, inline where the Figure is referred to in the main body. I'd recommend choosing one consistent placement. 
- Typo in L493 "critical rule" instead of "critical role"


**Edit after rebuttal**

The authors rebuttal has addressed my main concerns. I have raised my score to a 6 according and vote for acceptance.
Recommendation: 6
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 2