Kati8/ICLR_2023_2025 huggingface dataset:
authors: Xianghao Kong
authors: Ollie Liu
authors: Dani Yogatama
authors: Greg Ver Steeg
reviews: 
Comments: This paper proposes to investigate the fine-grained relationships learned by diffusion models by noticing a precise relationship between diffusion and information decomposition.  It exploits these new relations to measure the compositional understanding of diffusion models, to do unsupervised localization of objects in images, and to measure effects of prompt interventions.
Strengths: 1. It is interesting to view diffusion models from the information theory prespective. For examle, exact expressions for mutual information and conditional mutual information can be expressed in terms of the denoising model.
2. Compared with attention, the proposed mutual information seems more exact in identifying the relationship between the text and image. Massive visualization in the supplement demonstrate the effectiveness of this method.
Weaknesses: 1. It is still unclear of the concrete implementation. Without access to the network, but just with API, how to get the mutual information and corresponding visualization.
2. What is the application field. Attention-based methods are widely applied to image editing and show impressive results. What is the advantage of this decomposition method, compared to attetion.
3. The paper is poor structured and hard to read.
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 3 good
Clarity: 2 fair
reviews: 
Comments: This work discusses the interpretability of denoising diffusion models, which are used for conditional generation and density modeling of complex relationships like images and text. The authors aim to understand the relationships learned by these models by establishing a connection between diffusion and information decomposition. They provide exact expressions for mutual information and conditional mutual information in terms of the denoising model. The paper also introduces methods to quantify informative relationships between words and pixels in an image. The authors utilize these relationships to measure the compositional understanding of diffusion models, perform unsupervised localization of objects in images, and measure effects when editing images through prompt interventions.
Strengths: - The paper provides an approach to illuminate the fine-grained relationships learned by diffusion models.
- The authors introduce a natural non-negative decomposition of mutual information, which can be beneficial for understanding high-dimensional data.
- The paper offers practical applications, such as unsupervised localization of objects in images.
- The research uses information theory to provide a black-box method for understanding relationships in complex spaces like text and images.
Weaknesses: - The method is relatively trivial, replacing word in the prompt and comparing the difference in pixel level
- lack of comprehensive comparison with attention based method
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness/correctness: 2 fair
Clarity: 2 fair
reviews: 
Comments: Denoising diffusion models are used for complex data relationships like images and text, and this paper has revealed a link between these models and information decomposition, enabling precise characterization of relationships, including mutual information, which can be used to understand and measure the relationships between specific elements in high-dimensional data like words and image pixels.
Strengths: This paper provides a new perspective to explore the relationship between image and text in the Generation field, and quantify compositional understanding capabilities of diffusion models at a certain level.
Weaknesses: However, the primary drawback lies in the paper's somewhat inadequate presentation, making comprehension challenging. Additionally, a limitation is the insufficient experimentation, which fails to demonstrate how this method could enhance the generation process. To conclude, I strongly advise the authors to undergo a significant revision.
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 3 good
Clarity: 3 good