Diffusion PID: Interpreting Diffusion via Partial Information Decomposition
Authors: Shaurya Rajat Dewan
Rushikesh Zawar
Prakanshul Saxena
Yingshan Chang
Andrew Luo
Yonatan Bisk
Venue: NeurIPS 2024 poster (2024)
Keywords: Diffusion
Interpretability
Information Decomposition
Mutual Information
Bias
Primary area: diffusion_based_models
TL;DR: PID of the contribution made by terms in the input text prompt to the output image

Abstract: Text-to-image diffusion models have made significant progress in generating naturalistic images from textual inputs, and demonstrate the capacity to learn and represent complex visual-semantic relationships. While these diffusion models have achieved remarkable success, the underlying mechanisms driving their performance are not yet fully accounted for, with many unanswered questions surrounding what they learn, how they represent visual-semantic relationships, and why they sometimes fail to generalize. Our work presents Diffusion Partial Information Decomposition (DiffusionPID), a novel technique that applies information-theoretic principles to decompose the input text prompt into its elementary components, enabling a detailed examination of how individual tokens and their interactions shape the generated image. We introduce a formal approach to analyze the uniqueness, redundancy, and synergy terms by applying PID to the denoising model at both the image and pixel level. This approach enables us to characterize how individual tokens and their interactions affect the model output. We first present a fine-grained analysis of characteristics utilized by the model to uniquely localize specific concepts, we then apply our approach in bias analysis and show it can recover gender and ethnicity biases. Finally, we use our method to visually characterize word ambiguity and similarity from the model’s perspective and illustrate the efficacy of our method for prompt intervention. Our results show that PID is a potent tool for evaluating and diagnosing text-to-image diffusion models. Link to project page: https://rbz-99.github.io/Diffusion-PID/.

DiffusionPID: Interpreting Diffusion via Partial Information Decomposition
Shaurya Dewan*
Rushikesh Zawar*
Prakanshul Saxena*
Yingshan Chang
Andrew Luo
Yonatan Bisk
Carnegie Mellon University
(srdewan, rzawar, prakanss, yingshac, afluo, ybisk) Candrew cmu edu (* denotes equal contribution)
Abstract
Text-to-image diffuusin models have made significant progress in generating naturalistic images from textual inputs, and demonstrate the capacity to learn and represent complex visual-semantic relationships. While these diffusion models have achieved remarkable success, the underlying mechanisms driving not yet fully accounted for, with many unanswered ques-
their performance tions surrounding what they learn, how they represent visual-semantic relation- ships, and why they sometimes fail to generalize. Our work presents Diffusion Partial Information Decomposition (DiffusionPID)), a novel technique that applies information- principles to decompose the input text prompt into its ele- mentary components, enabling a detailed examination of how individual tokens and their interactions shape the generated image. We introduce formal approach to analyze the uniqueness, redundancy, and synergy terms by applying PID to the denoising model at both the image and pixel level. This approach enables us characterize how individual tokens and their interactions affect the model output. We first present fine-grained analysis characteristics utilized by the model uniquely localize specific concepts, we then apply our approach in bias analysis and show can recover gender and ethnicity biases. Finally, we use our method to visu- ally characterize word ambiguity and similarity from the model's perspective and illustrate the efficacy of our method for prompt intervention. Our results show that PID is potent tool for evaluating and diagnosing text-to-image diffusion models. Link to project page: https: htt//bbz99999 99. ttt.t.....ttttttt github. io/Diffusion--
Introduction
Immgee
Uniquenees
Image
Synergy
Image
Synery
Image
Redundancy
baseball glove and tennis racket
He swung the baseball bat and hit home run
A bat Dew overhead as we Inkes thmuugh this cave
queen wore crown
Figure 1:
Concept Figure. Left: Our "baseball" uniqueness map specifically highlights the seam region of the tennis ball it is visually very similar to that of baseball, Center: We high synergy for "bat" with "baseball" and "overhead" respectively which shows that uses these contextual cues to generate the images in the right settings, Right: Our redundancy map between "queen" " and "crown" correctly focuses on the crown and facial region.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

The role of individual inputs and the pretraining data in diffusion- image generation remains opaque, with little insight into how they translate often ambiguous text-based prompts into images. If we ask you to imagine lush forest at the edge of a sandy beach, you likely fill in missing details, picturing golden sand, splashing waves, and sunlight filtering through dense green foliage. Our prior experience allows us to easily visualize complex semantic associations and imagine novel scenes.
In the previous example, you are able to both generate and justify your choices for how complete the scene. Recent advances in diffusion-based generative models have achieved remarkable success in high-quality text-conditioned image generation, mimicking this human ability to a certain extent. However, the lack of transparency in these models raises important questions about their interpretabil- ity and controllability, making it challenging to identify biases, errors, or inconsistencies in the generated images. Text-to-image diffusion models can be brittle and prone to failure when faced with outto-ddistribution inputs, ambiguous text descriptions, or nuanced contextual dependencies. This results in unrealistic, inconsistent, or even misleading generations, leading to unnatural (and linguistically non-sensical) prompt engineering [1, 2] to achieve desired results by identifying and exploiting spurious correlations captured by the model [3]. The lack of interpretability also hinders the ability to correct or refine the models when they produce undesirable outputs, limiting the po- tential applications of these generative models in real-world scenarios. Frequently, in multi-concept sentence-basee -bsed conditioning of generative image models, concepts that are redundant to humans may be critical to the model. For instance, in the phrase "he swung a baseball bat", the word "baseball" may be redundant to humans, but it could be a crucial factor in the model's decision to generate baseball bat instead of the animal as can be seen in Fig 1. This work posits that crucial step towards more interpretable and robust models is to develop tools that can help explain higher-order contributions and interactions of individual words to the final image output. This would enable users to understand how the model is using the input text to generate the image, identify potential biases or errors, and refine the model's 's performance.
To that end, we introduce DiffusionPID, a novel approach that leverages partial information de- composition (PID), from information theory [4], to interpret the diffusion process and uncover the contributions of individual words and their interactions to the generated image. Compared to prior work which typically examined the contribution of tokens individually, our approach builds upon mutual information (MI) to examine the redundancy (R), synergy (S), and uniqueness (U) of the conditioning tokens. Specifically, our approach enables us to capture the interactions between concepts, revealing how individual words and phrases combine to influence the generation of specific image features. We further introduce extension of PID conditioned on the rest of the prompt as context, Conditional PID (CPID). By quantifying these interactions, DiffusionPID provides a more comprehensive understanding of the diffusion process, enabling the development of more interpretable and controllable text-to-image models.
This work contributes: (1) method to compute the image and pixel-level PID of diffusion-generrated images with respect to concepts from the input prompt: (2) an extension of PID, namely CPID, to better account for the context provided by the rest of the prompt during the generative process; (3) an evaluation of the DiffusionPID framework for elucidating model dynamics (via R, S, U) on complex linguistic concepts such as synonyms and homonyms and exposing ethnic/gender bias in the model; and finally, (4) a method of prompt intervention via PID to remove redundant words and retain unique and synergistic words.
2 Related Work
Text-to-Imaae (T2I) Diffusion Models. Diffusion- based text-to-image models have shown impres- sive results in image synthesis [5, 6, 7, 8]. These models are capable of generating images that are both realistic and semantically consistent with the imput text. This has led to the development of many large-scale and diverse datasets [9, 10, 11]. Recent datasets of challenging prompts and benchmarks based on visual question answering [12] and scene graphs have enabled researchers assess and refine their models [13, 14, 15, 16]. Inspired by the observation that existing models struggle to accurately bind attributes to objects [17]. some approaches propose to modify the cross-attention layers by explicitly amplifying the token-wise attention used for text conditioning [18, 19, 20, 21, 22], or by composing multiple outputs to generate semantically compositional images [23]. These meth- ods aim to improve the model's 's ability to capture fine-grained semantic relationships between text and image. These diverse approaches demonstrate the ongoing effort to develop more effective and flexible eext-to-image synthesis models and highlight the need for continued research in this area.

---

Interpretability in Deep Learning: Interpretability is key to understanding and trusting the decisions made by complex models. LIME [24] presents a technique for interpreting individual predictions of any machine learning (MI. model via local linear models, while SHAP [25] assigns each feature an importance value based on Shapley values from cooperative game theory. CAM [26] and Grad-
CAM (27] localize image regions in a CNN using weighted sum of intermediate layers' activations. Network Dissection [28] operates at the neuron level, tracking image regions corresponding to an excited neuron across all the dataset images, which GAN Dissection [29] extends for GANs. MILAN [30] and CLIP-Dissect [31] identify semantic concepts for an individual neuron. Benchmarks such as ARO [32]. Winoground [33], EQBEN [34], and WHOOPS [35] introduce datasets to specifically probe the compositional and semantic understanding of and show that most SOTA models struggle with relational concepts. Some works have also explored the application of PID in different settings [36]. [37] uses PID to measure the contribution of each neuron towards a target concept and [38] applies PID on generie multimodal datasets and models. Finally, MULTIVIZ [39] analyzes the behavior of multimodal models, specifically focusing on the unimodal, cross-modal, and multi-modal contributions. Overall, lot of progress has been made in understanding and interpreting ML models. Given the recent explosion of work on diffusion models, we primarily focus on interpreting these models.
Interpretability of Diffusion Models: Interpretability of diffusion models remains an open challenge. Data attribution [40, 41, 42, 43] is a commonly used approach to identify training samples most responsible for the appearance of generated image. Similarly, [44, 45] decompose the images associated with a particular concept into the implicit semantic concepts the generative model considers to be most similar. Others [46, 47, 48, 49, 50] have discovered semantic latent space for diffusion models such that modifications in this space edit attributes of the original generated images. [51, 52] similar but operate in the text embedding space. Recently, there have also been works that try to gain neuron-level and network-leve understanding of these models. For instance, [53] divides the latent vector into multiple groups of elements and designs different noise schedules for each group so that each group controls only certain elements of data, explicitly giving interpretable meaning. [54] identifies clusters of neurons corresponding to given concept with gradient statistics. [52, 55] show that diffusion models struggle on prompts containing homonyms (words with multiple meanings) and [56, 57] suggested the presence of ethnic, gender, and semantic biases in diffusion models we investigate both of these conditions in sections 4.3 and 14.1, 4.2], respectively. Recently, DAAM [58) proposed to analyze the cross-attention layers of diffusion models to produce attribution maps for tokens from the input prompt. However, these maps are generally very noisy and offer limited information the model fails to generate a given concept. [59] introduced a method to compute the mutual information (MI) and conditional mutual information (CMI) between the input text prompt and each generated pixel. While their work provides stronger and more meaningful insights, their approach only provides coarse information on the model. Our work enables fine -ra-ggraned model understanding by decomposing the mutual information between the text prompt and image output by using partial information decomposition (PID).
3 Method
Image diffusion models learn to model an image distribution, X, through a progressive noise addition and removal process. The forward diffusion process, defined by H, = vo(a)z + / aa),, gradually adds Gaussian noise, e (O, I), to the image, x ~ p(X), over a sequence of timesteps, where a denotes the log SNR/noise schedule and a is the sigmoid function. A model learns to reverse this process, denoising the data gradually transform the noise back into samples from the original data distribution. This denoising process has been applied to text-to-image generation with incredible success by conditioning the process on text prompts y. We employ the denoising process to measure the influence of individual tokens (from the input prompt) and their interactions on the resulting generation. We consider the diffusion model be an optimal denoiser that can predict the added ground-truie noise, e, at noise level as:
Ea (a) = arg min Ep Eo (xx.)) {()
We follow [59]'s definition of mutual information (MI) and conditional mutual information (CMI) computed using a diffusion model. Extending Eq to the conditional case where we assume (1., (y) is the optimal denoiser for the conditional distribution p(rly), the Log Likelihood Ratio (LLR), which also the mutual information (MT) i(y; ar), between pixel x and an input text prompt y is defined as:
3

---

log p(r y) log p(a)) 1 2 / Ep(o)lec Ea (xa)))1 ||e- -ea(ra 1))l
(2)
1
log p(r y) - log p(x) =
2 Ep(o)llEa Ex 1))l
(3)
Eq 3 is derived from Eq using the orthogonality principle and the derivation can be found in the supplemental. We also provide graphs comparing the MMSE estimates obtained from Eq 2 versus the simplified form in Eq 3 for varying levels of noise/SNR and discuss the uncertainty in our estimator in the supplemental.
Essentially, MI quantifies the information an input variable y individually provides about an output variable This definition of pixel-wise MI derived from [60] can be intuitive and easily understood as the quantity by which the text prompt increases the probability of observing relative to the prior probability without text conditioning. The same work provides an alternate definition for MI as:
(y;ar) - log p(y) + log p(ulz) log p(xly) log p(x)
Similarly for the CMI of phrase 1/1 given the context of the phrase y/2 in the input text prompt y:
= log p(u + log p(n (x,, 52) log pl v2) log p(æcly))
(5)
From Eq 5, CMI can be understood as the information that y11 contains about z beyond what is already contained in y2- Following [60], we provide the mathematical definitions for the various terms in the partial information decomposition (PID) at the pixel-level. We start with the MI of two input events (phrases) with an output variable (pixel) defined as:
i(y y2 x) = r(vi. v:) + u(yi N2 + u( 2) + s(y U2:
(6)
Here, 1/1 We2 means /1 excluding 1/- r(v1, M22 2r) is the redundant overlapping information between y1 and y/ (redundancy), u(y1 \y2 z) is the unique information contributed by yı. "(y2 Wr: is the unique information contributed by y2. and (111, y2;z) is the new information contributed by the combination of y1 and y/2 (synergy) that neither them could contribute on their own. is important to note that the uniqueness defined here is with respect the other input variable, i.e., it is the unique information the given phrase contributes that the other phrase does not.
A natural measure of redundancy is the expected value of the minimum information that any source provides about each outcome of æ. This captures the idea that redundancy is the information common to all sources (the minimum information that any source provides) while taking into account that sources may provide information about different outcomes of æ. However, the redundant information must capture when two predictor variables are carrying the same information about the target, not merely the same amount information. This means that the sign/direction of information matters. To account for this and to ensure the definition complies with all the required axioms on redundancy, the redundancy is broken down into a positive r+ (y/, V2; x)) and negative r- (y1, V2 x) component associated with the informative i+ (vs; and misinformative i (y:: x)) MI terms respectively. Thus, the equation for computing the redundancy can be derived as:
r (y1. V2: x) min i+ (y6; x) = minl- log p(y:),, 3e E (11.32)
(7)
9i
r ( V2;
min i (y1;)) min[- log p[y](=]]
(8)
r(v,, (2: = r r min[ [in[-Ma log p(u:)) min[- log p(vil=)]
(9)
r(y1, y/s m) min | log p(y:)) min[ log p(rly:)) + log p(x) log p()))
(10)
91
To compute the probability of an input event (phrase) p(y) the above equations, we make use of BERT [61]. One of the tasks BERT was trained on was to predict masked token by predicting the probabilities of all possible tokens from a fixed dictionary and taking the maximum. We use this to obtain the probability of a given phrase. In the unconditional case, we only require the objective probability of the phrase with no other tokens except the special [MASK] token present in the string fed to BERT.

---

Given the definitions for MI and redundancy, the uniqueness of an input event/phrase and the synergy between phrases can be derived as:
u y1 W22 x) = {)1;x) r(1/1 322z)) xr)
(11)
(31 U22 2) {yy1 32) r(y1 32) 21 (2222) u( \31:2)
(12)
We also introduce CPID (Conditional PID) an extension of PID to take the context contributed by the rest of the prompt into consideration. It represents the PID for the conditional case where all the PID components and probability terms are now conditioned on the rest of the prompt. This is similar to the CMI extension of MI in [59]. We rewrite equations 6, 10, 11 and 12 with the required changes below for easy reference (y signifies the rest of the prompt with the terms y1 and y2 removed):
i(11, /2: = r(y1, U2: v) + u(1n Nu2; arl) + u(ya Av:: mly) + s(1 y2: zlyy)
(13)
r(y1 M22 alv) = min[| log p(ualv)] min[- log p(ally y) + log p(xly) - log p(uale)]
(14)
W:
9:
uly1 122z = i(y:: m r(y) v2 xqy)
(15)
sgy y2: xly) i xly) r(1 y2: xly) uyy \ee: mly) u(y2 \11:)
(16)
It is important note that all our definitions for all the information terms require the evaluation of an integral over an infinite range of SNRs. In practice, we make use of truncated logistic-based importance sampling evaluate this integral, similar to [59].
Experiments
We use our method to conduct a detailed analysis of diffusion models in various situations. We introduce several tasks along with corresponding datasets to study these models' behavior in depth. For our experiments, we primarily focus on the pre-trained Stable Diffusion 2.1 model from Hugging Face. Latent diffusion models encode images into a lower dimensional latent space before the denoising process. Thus, all our PID computations occur in this latent space, i.c., consider the image in this lower dimensional space as z. During visualization, the heatmaps are bilinearly interpolated from this latent space the original image resolution. Finally, we make use of 50 samples for evaluating the integral over SNRs using importance sampling. A single A6000 GPU was used to generate the PID maps for each data sample.
4.1 Gender Bias
Setup: Gender is a social construct and a complex study in its own right. In this work, we search for the perpetuation of traditional gender roles and associations. While there is rich literature on the implications of such biases and how models can exacerbate them [62, 63, 64, 65, 66, 67, 68, 69, 70], we simply present PID as another tool that investigative process. In this study, we evaluate whether the diffusion model exhibits gender bias in generating images of people in various occupations. Our objective to determine if the model consistently associates specific genders with certain occupations. For this task, we take a set of 188 common occupations such as professor, valet, receptionist, janitor, etc., and create 376 prompts (188 2) by joining the occupation with each gender, male or female, one at time. More details on the dataset creation process can be found in the supplemental. To check for bias, we compare the image-level redundancy values of each gender with the occupations. The image-level value is simply computed by using the expectation of the MSE term over all pixels instead of the pixel-level MSE values in our equations. The final redundancy values are normalized across the entire dataset to the range [0, 1]. We hypothesize that whenever the redundancy of one gender is much higher than that the other with the occupation, the model is biased towards the higher redundancy gender for that occupation.
We can see in Table 1 that vocations that are typically stereotyped to be performed by males such as plumber, carpenter, and police officer, have high redundancy values for the male gender as compared to the female gender. Similarly, we see higher redundancy for the female gender for female stereotyped jobs such as babysitter and teacher. We also observe that the average redundancy for females across all occupations very low, indicating significant model bias against generating females for any occupation. Thus, it is clear that the model has learnt gender biases.
Results:
5

---

Occupation
Male
Female
Surgcon
0.539
0.055
Occupation
Black
Asian
Caucasian
Hispanic
Soldier
0.250
0.136
Judge
0.304
0.286
Athlete
0.321
0.132
0.167
0.156
Doctor
0.871
0.090
Artist
0.106
0.069
0,062
0.045
Plumber
0.605
0.038
Engineer
0.126
0.156
0.080
0.097
Carpenter
0.365
0.093
Physicist
0.109
0.209
0.162
0.064
Police Officer
0.390
0.091
Butcher
0.110
0.179
0.474
0.396
Babysitter
0.240
0.531
Coach
0.118
0.107
0.433
0.128
Teacher 0.098
0,419
Nurse
0.106
0.106
0.127
0.219
Agriculturist
0.046
0.117
0.337
0.450
Average
0.286
0.194
Table I: Gender Bias: Redun-
Average
0.133
0.233
0.255
0.236
dancy between gender and oc-
Table 2: Ethnicity Bias:
Redundancy between
cupation
ethnicity and occupation
4.2 Ethnic Bias
Setup: Similar to gender, race and ethnicity are not hard and fast classes, but indicate how individuals self-identify. Previous work [71, 69, 72, 73, 74, 75] has shown that deep learning models are prone learning racial and ethnic biases, especially when the underlying training data is imbalanced. With the recent shift towards snfffsiion-generatee test for the presence of such biases in diffusion. Similar to the experiment above, we examine the diffusion model for ethnic bias to see if it associates specific ethnicities with specific occupations. use the same 188 occupations used in gender bias experiment but now we combine them with each ethnicity (Caucasian, Black, Asian, and Hispanic) instead of gender. We end up creating dataset of 752 (188 x 4) paaageprompptt pairs. Refer to the supplemental for more details on the dataset creation process. The same approach is adopted as the previous experiment where compare the normalized image-level redundancy values of the various ethnicities with the occupations.
synthetic datasets [9, 10, 11], it has become crucial to
Results: We can see from Table 2 that jobs that are commonly stereotyped for certain ethnicities such as athlete with Black, engineer with Asian, and so on, have higher redundancies with those respective ethnicities. We also compare the redundancies of each ethnicity averaged over all occupations and observe a very low value for Black ethnic group which means that the model heavily biased against this group and is less likely to generate people from this group for any occupation on average. Thus, we clearly see that the model has learnt biases on the basis of ethnicity.
4.3 Homonyms
Baa
Toseeto
awung the
sweungg basebell
Figure Homonyms. We see that the word "baseball" provides the required synergistic context with the homonym "bat" to pick the sports setting over the animal. This effect can be confirmed in the synergy maps and image-level synergy values (S) as well where we observe high synergy for "bat" with "baseball" compared other words such as "He" and "swung".
diffusion model can behave in different manners when faced with prompts containing homonyms in different contexts. A word is homonym when it can take on different meanings when used with

---

different contextual words which we term as modifiers. For example, the phrases "football match" and "he lit match" should generate two completely different visuals. We examine how diffusion models handle such prompts in this experiment.
approach analyze the novel information contributed towards the image generation process by different modifiers. We create a dataset of 242 prompts containing homonyms in different contexts. We study the synergy map between the homonyms and the modifiers to see if the model is able to extract new information from their combination to generate an image of the homonym in the right context.
Setup: For this task,
Results: In the left figure of Fig 3, we can see our synergy maps correctly highlight the area pertaining to the new information contributed by the combination of the homonym ("bowl") and modifier ("ceramic" and "game" respectively). However, in the right figure of Fig 3, the model fails generate the homonym ("mole") in the right context ("coworker" and "searching"), This failure can be understood by the low synergy maps which reveal that the model fails to learn the synergistic relationship between the homonym and the modifiers. Thus, the synergy maps help us decode these kinds of failures of the model.
when given the appropriate modifiers
We also provide synergy map visuals obtained from our CPID implementation in the supplemental.
Image
Synergy
Image
Synergy
Image
Synergy
Image
Synergy
PONELRS
served soup coramic
the game rose stacium y- bool 'garne'
might leaking information compettions y 'molee ve 'coworker
underground
y vs baramic
searching insects y 'mole' ve "seerching"
Figure 3: Homonyms. Left: Successful generation homonym "bowl" in different contexts due to high synergy with modifiers "bowl" and "game". Right: Failure case where the model generates the homonym "mole" with the same semantic meaning, the animal, due to its failure to use contextual information from words like "coworker" can be seen in the synergy map.
4.4 Synonyms
Setup: Dealing with synonyms, i.e., different words with the same meaning, can be challenging for diffusion models. Here, we probe the diffusion model with prompts containing known synonym pairs see how it handles them and if it correctly identifies the given words synonyms. To that end, generated dataset of images from 132 text prompts containing synonym pairs. More details on the dataset creation process be found in the supplementary. Our hypothesis is that the redundancy will be high between words that the model considers as synonyms.
how approach very clearly outputs high redundancy in the region of the object that both synonyms, "cube" and "cuboid", refer to. Similarly, the redundancy map for the words "bed" and "mattres" correctly highlights the bed region. To have fair comparison with MI, CMI, and DAAM, we take the intersection between the maps of the individual synonyms produced by each of these methods (details in supplementary). It is visible from Fig 4 that these methods fail to find the region of interest and produce very sparse maps. Hence, our method is better suited for interpreting what words/phrases the diffusion model considers semantically similar.
Results: In Fig 4,
Image
Redundancy
CMI
MI
DAAM
Image
Redundancy
The had was covarad in soft mattroso with plush blanket = bad va mattroas"
zuba cuboid sides y touookk
angles
Figure 4: Synonyms. Our redundancy map is able to highlight that the model considers the pairs "bed" and "mattress" (left) and "cube" and "cuboid" (right) as semantically similar.
4.5 Co-Hyponyms
Previous work has shown that diffusion models sometimes fail to generate certain objects mentioned in the prompt. One such set of cases the model struggles with is prompts containing co-hyponyms,
7

---

Le., concepts that are semantically very similar but not identical. The model tends to generate only one of the objects rather than both of them or a fused version of the objects. We hypothesize that this phenomenon occurs because the model confuses the two concepts to carry the same semantic meaning even if they don't 't for us as human beings. This means that the redundancy between them expected to be very high which is also what we can see in our experiments. We introduce 2 prompt datasets containing co-hyponyms and are discussed below.
4.5.1
Co-Hyponym COCO
Setup: To construct this dataset, we used the COCO dataset's [76] super-category hierarchy to extract co-hyponym pairs. The supplemental contains more details on the dataset creation process.
Results: In Fig 5, the redundancy is visibly high in the region where the diffusion model has fused features from the words "cat" and "elephant". Similarly, the redundancy is highly activated in the region of the only generated object due to the high semantic similarity of the words "pizza" and "sandwich". Thus, apparent that for cases where the diffusion model fails learn the difference in the semantic meaning of two co- hyphhyyoo,,, it confuses the two to be referring to the same concept, signified by the high redundancy, and fails to generate one the objects. Similar to the synonym experiment above, we compare DAAM and observe that they provide little information for interpreting reason the model's failure. Thus, the redundancy map proves to be useful tool understand why the model fails here.
redundancy map with the intersection maps from MI, CMI, and
imaoo
Redundancy
CMI
MI
DAAM
Imagee
Redurdancy
CMI
MI
DAAM
= sandaich and a pizza 'sandwichh ve 'picel
G-3 cot and elephaan y cca vs elephant
Figure 5: COCO Co-Hyponyms. The redundancy map proves to be very useful in finding out the reason behind the model's 's failures in these figures. It confuses the co-hyponym pairs (" ("sandwich", "pizza")(eef)) and ("elephant", (righ))) have same meaning for the co-hyponyms as seen from the redundancy maps, which results in erroneous generations.
4.5.2 Co-Hyponym WORDNET
Setup: We make use of Wordnet's [77] hyponym and hypernym relations between the synsets of the words to obtain 798 co-hyponym pairs for the prompts. Refer to the supplementary for more details on the dataset creation process.
Results: In the left figure of Fig 6, we can see the fusion of features from both objects mentioned in the prompt, while in the right figure, we see only one of the objects has been generated. The redundancy maps correctly highlight the image regions pertaining both the co-hyponyms. Again it is observed that MI, CMI, and DAAM only very sparsely highlight the correct region of overlap. This reinforces our findings from previous COCO-based experiment that the model mixes up the co-hyponyms and our redundancy map from PID helps pin down the reason behind this phenomenon.
Image
Redundancy
MI
Image
Redundency
ing y= vs ralling
y
Figure 6: Wordnet Co-Hyponyms. Similar to the COCO Co-Hyponyms dataset, the redundancy maps tell that the model confuses the co-hyponym pairs ("barrier", "railing" (left) and ("chair", "sofa") (right) to have the same meaning for the co-hyponyms, resulting in erroncous generations.
4.6 Prompt Intervention
We use our method to identify redundant words in the prompts, remove them, and verify that this intervention results in little change to the image. We test this task all the datasets involving

---

redundant terms: co co-hypon hyponympy wordnet, co-hyponym coco, synonyms, and occupation bias (ethnic and gender). For more details on how edit the image. refer the supplementary.
Original
Intervention
Redundancy
Uniqueness
and girafla
Original
Interventiin
Radundancy
Uniqueness
and girafla
"iraflo'
Figure 7: Prompt Intervention. The redundancy is highly activated in the face region of the giraffe. On a closer look, we see that the face is that of cow meaning that the word "cow" redundant. This is confirmed the image changes very little on omitting it from the prompt.
Original
Redundancy
laplop and mouse Taptop'
Original
Intervention
Redundancy
Uniqueness
CMI
MI
DAAM
8 aplop and mouse 'rouuee
Figure 8: Prompt Intervention. The redundancy is highly activated in the mouse region meaning that the word "mouse" redundant. The uniqueness of "mouse" is also very low and spread out. This redundancy is confirmed as the image changes very little on omitting it from the prompt.
Both figures and clearly show that whenever the redundancy is high region corresponding to particular object and the uniqueness for that object's 's term is low/spread out, then removing that object has minimal effect on the image. On the other hand, we see that the MI, CMI, and DAAM maps are highly activated for these redundant terms, indicating that these metrics are not ideal for identifying redundant terms in the input prompt. For instance, the first figure, the redundancy map is highly activated in the mouse region and the uniqueness of the term "mouse" is spread out. Removing the word "mouse" results an image very similar to the original whereas removing the word "laptop" drastically changes the image. The MI, CMI, and DAAM maps are notably high for the "mouse" term, incorrectly suggesting that removing this term should have significantly altered the image. A similar effect can be observed for the term "cow" in the second figure.
4.7 Most Representative Features
The most representative features of a concept are those characteristics that uniquely define it from the model's perspective. Given set of classes, the uniqueness map obtained from PID can be used to localize these features. As we can see in Fig 9, when the input prompt is "hair dryer and toothbrush". even though the image mix of a hair dryer and a toothbrush, the uniqueness map highlights region corresponding to the toothbrush bristles. Similarly, we can see the bear's 's facial features being highlighted correctly the other example. Thus, our uniqueness maps prove useful for this task of identifying the unique characteristics of an object.

---

Uniquenese
Image
Uniquenees
A ha mayor
U (cothbuust)
nonso and
(bbar]
Figure 9: Most Representative Features. Left: The "toothbrush" uniqueness map correctly captures the toothbrush bristles, their most distinct feature. Right: The "bear" uniqueness map correctly captures the bear region, specifically the face.
Image
PID
GPID
drinking coffee.
purple
grilden stons, and 'rag' va 'nef
ha Sin hamben swend stocts radio wavee bembon ppurplr haar
Figure 10: Results PID and CPID on complex prompts
4.8
Complex Prompts
In this experiment, we test the efficacy of our method on more complex prompts, similar to those the diffusion training distribution. These prompts usually mention several objects and their corresponding attributes. We find that our method remains effective and informative even in these challenging examples. We visualize the information maps between objects and attribute-defnnng terms.
In both visuals in Fig 10, we observe high synergy because the attribute modifies the object's 's visual properties in some form. We also that CPID provides slightly better localized results than its PID counterpart in practice. This expected as CPID accounts for the contextual contribution of the rest of the prompt in the image generation process and better captures the specific contribution of the two terms under consideration.
5 Discussion
Limitations and Future Work: Although we have exhibited the efficacy and benefits Diffusion- scope for improvement that future work can explore. In all experiments,
PID, there remains we compute the PID terms only for two phrases from the prompt but PID can be extended to more than two input variables. Another interesting research direction could be to compute PID on other, non-diffusion-bant baased models. Also, our approach requires access to the diffusion model, making it difficult apply it to closed-source models, which means that alternative methods of computing PID need to be explored. There are also some other applications that could be tried with PID such as using the uniqueness information localize distinctive features differentiate closely related classes in image classification. Recently there have been works such as [78] that integrate other forms of conditioning such as masks enable better control of diffusion models. PID-based analysis of these multimodal forms of conditioning is another worthwhile research direction for the future.
Conclusion:
The fine-grained breakdown of MI afforded by our approach, DiffusionPID, allows us to understand the Docision-making process of diffusion models, identify their shortcomings, and pin down the reasons for their failures. This understanding is critical given the misalignment between the model's 's world understanding and ours. We can use the insights furnished from our approach make diffusion models better aligned with humans' conceptual understanding and counter its limitations and biases. Our work can serve as a strong fundamental basis for further research on using information theoretic concepts such PID to dissect, study, and enhance deep learning models.
10

---

References
[1] Vivian Liu and Lydia Chilton. Design guidelines for prompt engineering text-to-image generative models. CHI '22, New York, NY, USA, 2022. Association for Computing Machinery, ISBN 9781450391573. doi: 10. 00.1145/9991022.50110 URL httts::///ool org/ 10. 1145/ 3491102 3501825.
[2] Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https: //openreview.n/ http://www.ettooooaaaadttttttaae
[3] Junhyun Nam, Sangwoo Mo, Jaeho Lee, and Jinwoo Shin. Breaking the spurious causality of conditional generation via fairness intervention with corrective sampling. arXiv preprint arXiv:2212.02090, 2022.
[4] Paul L. Williams and Randall D. Beer. Nonnegative decomposition of multivariate information, 2010.
[5] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112. 2021.
[6] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint 1iXiv:2204.061255 1(2):3, 2022.
[7] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[8] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952: 2023.
[9] Zijie J. Wang, Evan Montoya, David Munechika. Haoyang Yang, Benjamin Hoover, and Duen Horng Chau. Diffusiondb: A large-scale prompt gallery dataset for text-to-image genera- tive models, 2023. URL https: / //arxiv...ivv//// org/ abs//210..../ 14896.
[10] Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, Jifeng Dai. Yu Qiao, Limin Wang, and Hongsheng Li. Journeydb: A benchmark for generative image understanding, 2023. URL attps:///axxtt org/abs/ 230/ 00016..
[11] Rushikesh Zawar, Shaurya Dewan, Andrew F. Luo, Margaret M. Henderson, Michael J. Tarr, and Leila Wehbe. Stablesemantics: A synthetic language- vision dataset of semantic representations in naturalistic images, 2024. URL https: //atpps://rxiv... /ttpxv..vvvvv////v/// org/abs/////eeeoo/o/...oeeee 2206/ 13735.
[12] Aviral Agrawal, Carlos Mateo Samudio Lezcano, Iqui Balam Heredia-Mart and Prab- hdeep Singh Sethi. Listen then see: Video alignment with speaker attention, 2024. URL
https://arxiv org/ abs//b//// 13530.
[13] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Seyed Kamyar Seyed Ghasemipour, Bureu Karagol Ayan, Seyedeh Sara Mahdavi, Raphael Gontijo Lopes. Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. ArXiv, abs/2205. 1487, 2022.
[14] Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang. Mari Ostendorf, Ranjay Krishna, and Noah A. Smith. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. 2023 IEFF/CE International Conference on Computer Vision (ICCV),
pages 2023.
11

---

[15] Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation. ArXiv, abs/2404.01291, 2024.
[16] Jaemin Cho, Yushi Hu, Jason Baldridge, Roopal Garg, Peter Anderson, Ranjay Krishna, Mohit Bansal, Jordi Pont-Tuset, and Su Wang. Davidsonian scene graph: Improving reliability in fine-grained evaluation for text-to-image generation. In ICLR, 2024.
[17] Colin Conwell and Tomer David Ullman. Testing relational understanding text-guided image generation. ArXiv, abs/2208.0000000 2022.
[18] Weixi Feng, Xuchai He, Tsu-Jui Fu, Varun Jampani, Arjun Reddy Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, and William Yang Wang. Training-free structured diffusion guidance for compositional text-to-image synthesis. In The Eleventh International Con- ference Learning Representations, 2023. URL https: http:///ppeeeeeeeeeeeeee PUIq]T4rzq7.
[19] Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention- based semantic guidance for text-to-image diffusion models. ACM Transactions on Graphics (TOG), 42:1 10, 2023.
[20] Royi Rassin, Eran Hirsch, Daniel Glickman, Shauli Ravfogel, Yoav Goldberg, and Gal Chechik. Linguistic binding in diffusion models: Enhancing attribute correspondence through attention map alignment. In Shirty-seventh Conference Neural Information Processing Systems, 2023. URL bettps://openrert// wtt.:/nneeeeommmmm/ooAo/AAAra
[21] Ruichen Wang, Zekang Chen, Chen Chen, Jiancang Ma, H. Lu, and Xiaodong Lin. Com- positional texl-to-image synthesis with attention map control of diffusion models. ArXiv, abs/2305. 2023.
[22] Quynh Phung, songwei Ge, and Jia-Bin Huang. Grounded rext-to-image synthesis with attention refocusing. ArXiv, abs/2306.05427, 2023.
[23] Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XVII, pages 423-439. Springer, 2022.
[24] Marco Tulio Ribeiro. Sameer Singh, and Carlos Guestrin. "why should trust you?": Explain- ing the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016
[25] Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Neural Information Processing Systems, 2017.
[26] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2921-2929, 2016.
[27] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference computer vision, pages 2017.
[28] David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissec- tion: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6541 6541-6549, 2017.
[29] David Bau, Jun-Yan Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua B Tenenbaum, William T Freeman, and Antonio Torralba. Gan dissection: Visualizing and understanding generative adversarial networks. arXiv preprint arXiv: 2018.
12

---

[30] Evan Hernandez, Sarah Schwettmann, David Bau, Teona Bagashvili, Antonio Torralba, and Ja- cob Andreas. Natural language descriptions of deep visual features. International Conference on Learning Representations, 2021.
[31] Tuomas P. Oikarinen and Tsui-Wes Weng. Clip-dissect: Automatic description of neuron representations in deep vision networks. ArXiv, abs/2204. 10965, 2022.
[32] Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou, When and why vision-language models behave like bags-of-words, and what to do about it? In The Eleventh International Conference on Learning Representations, 2023. URL https:
/ //openreview. bet/fforumiddxKLLuvxeee
[33] Anuj Diwan, Layne Berry, Eunsol Choi, David Harwath, and Kyle Mahowald. Why is winoground hard? investigating failures in visuolinguistic compositionality. arXiv preprint 2tti:22111.00766 2022.
[34] Tan Wang, Kevin Lin, Linjie Li, Chung- Lin, Zhengyuan Yang, Hanwang Zhang. Zicheng Liu. and Lijuan Wang. Equivariant similarity for lisiun-lagguage foundation models. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 11964-11974, 2023.
[35] Nitzan Bitton-Guetta. Yonatan Bitton, Jack Hessel, Ludwig Schmidt, Yuval Elovici, Gabriel Stanovsky, and Roy Schwartz. Breaking common sense: Whoops! a vision-and -ad---aggg benchmark of synthetic and compositional images. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 2616-2627, 2023.
[36] Faisal Hamman and Sanghamitra Dutta. Demystifying local and global fairness trade-offs in federated learning using partial information decomposition, 2024. URL https: /t/a://///xii org/ abs/2307 11333.
[37] David Alexander Ehrlich, Andreas Christian Schneider, Viola Priesemann, Michael Wibral, and Abdullah Makkeh. A measure of the complexity of neural representations based on partial information decomposition. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL htR.httta///ppeeeerieee
[38] Paul Pu Liang, Yun Cheng, Xiang Fan, Chun Kai Ling, Suzanne Nie, Richard Chen, Zihao Deng, Nicholas Allen, Randy P. Auerbach, Faisal Mahmood, Ruslan Salakhutdinov, and Louis-Philippe Morency. Quantifying modeling multimodal interactions: An information decomposition framework. In Neural Information Processing Systems, 2023.
[39] Paul Pu Liang, Yiwei Lyu, Gunjan Chhablani, Nihal Jain, Zihao Deng, Xingbo Wang, Louis- Philippe Morency, and Ruslan Salakhutdinov. Multiviz: Towards visualizing and understanding multimodal models. In International Conference on Learning Representations, 2022.
[40] Xiaosen Zheng, Tianyu Pang, Chao Du Jing Jiang, and Min Lin. Intriguing properties of data attribution on diffusion models. In International Conference on Learning Representations (ICLR), 2024.
[41] Kristian Georgiev, Joshua Vendrow, Hadi Salman, Sung Min Park, and Aleksander Madry. The journey, not the destination: How data guides diffusion models. In Arxiv preprint 1rXiv:23/2.06200 2023.
[42] Zheng Dai and David Kenneth Gifford. Training data attribution for diffusion models. ArXiv, abs/2306.02174 02174, 2023.
[43] Sheng-Yo Yu Wang, Alexei A. Efros, Jun- Yan Zhu, and Richard Zhang. Evaluating data attribution for text-to-image models. In ICCV, 2023.
[44] Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. ArXiv, abs/2208.01618, 2022.
[45] Hila Chefer, Oran Lang, Mor Geva, Volodymyr Polosukhin, Assaf Shocher, Michal Irani, Inbar Mosseri, and Lior Wolf. The hidden language of diffusion models. arXiv preprint arXiv:2306.0040 20066600000000....0 2023.
13

---

[46] Mingi Kwon, Jaeseok Jeong, and Youngjung Uh. Diffusion models already have a semantic latent space. abs/2210.109h 10960, 2022.
[47] Hang Li, Chengzhi Shen, Philip Torr, Volker Tresp, and Jindong Gu. Self-discovering inter- pretable diffusion latent directions for responsible text-to-image generation. arXiv preprint arXiv:2311.17216 2023.
[48] René Haas, Inbar Setberman-Spiegelgee Rotem Mulayoff, and Tomer Michaeli, Discovering interpretable directions the semantic latent space of diffusion models. ArXiv, abs/2303.11073, 2023.
[49] Yong- Mayn-Hyun / Park, Mingi Kwon, Jaewoong Choi, Junghyo Jo, and Youngjung Uh. Under- standing the latent space of diffusion models through the lens of riemannian geometry. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 24129-24142. Curran Associates, Inc., 2023. URL https: h/wpa//eee/eeeeeee.eee...eae wwwreeiiiiinnnnnnnnneeeeeeeeeeee neurips. cc/ wwpppprr..cppppppppeepppppppe..p filee/llppllll faaeeeeffflaappppeeeeeeeeeee /appr//022//tlll 4bfcebedf7 wwtp:/dff727777744 040206674440000007777700000 Paper- Conpe--Confeeee Conf-oooeeeeeeeeee erence pdf.
[50] Yusuf Dalva and Pinar Yanardag. Noisecir: A contrastive learning approach for unsupervised discovery of interpretable directions in diffusion models, 2023.
[51] Qiucheng Wu, Yujian Liu, Handong Zhao, Ajinkya Kale, Trung M. Bui, Tong Yu, Zhe Lin, Yang Zhang, and Shiyu Chang. Uncovering the disentanglement capability text-to-image diffusion models. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1910, 2022.
[52] Jennifer C. White and Ryan Cotterell. Schrodinger's bat: Diffusion models sometimes generate polysemous words in superposition, 2022. URL https://arxiv. http:/axxxivv.../pp.o// wwtpoo///////////////oo/oo///// /2t1. pdf.
[53] Sangyun Lee, Gayoung Lee, Hyunsu Kim, Junho Kim, and Youngjung Uh. Diffusion models with grouped latents for interpretable latent space. In ICML 2023 Workshop on Structured Probabilistic Inference {{&} Generative Modeling, 2023.
[54] Zhiheng Liu, Ruili Feng, Kai Zhu, Yifei Zhang, Kecheng Zheng, Yu Liu, Deli Zhao, Jingren Zhou, and Yang Cao. Cones: Concept neurons in diffusion models for customized generation. In International Conference Machine Learning, 2023.
[55] Royi Rassin, Shauli Ravfogel, and Yoav Goldberg. DALLE-2 is seeing double: Flaws in word-to-concept mapping in text2image models. CoRR, abs/2210.. 2022. doi: 10.48550/ ARXIV.2210.1086 URL https::/ddo org/10. 4.00..855//////
2210. 10606.
[56] Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jernite. Stable bias: Evalu- ating societal representations in diffusion models. In Neural Information Processing Systems, 2023.
[57] Hadas Orgad, Bahjat Kawar, and Yonatan Belinkov. Editing implicit assumptions in text-to- image diffusion models. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 7030-7038, 2023.
[58] Raphael Tang, Linqing Liu, Akshat Pandey, Zhiying Jiang, Gefei Yang, Karun Kumar, Pontus Stenetorp, Jimmy Lin, and Ferhan Ture. What the daam: Interpreting stable diffusion using cross attention. arXiv preprint arXiv:2210.04885 2022.
[59] Xianghao Kong, Ollie Liu, Han Li, Dani Yogatama, and Greg Ver Steeg. Interpretable diffusion via information decomposition. arXiv preprint arXiv:2310.07972,, 2023.
[60] Conor Finn and Joseph T.. Lizier. Pointwise information decomposition using the specificity and ambiguity lattices. ArXiv, 0ass/1801009010 2017.
[61] Jacob Devlin, Ming-Wei Chang. Kenton I Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics, 2019.
14

---

[62] Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai- Wei Chang, and Vicente Ordonez. Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations. In Proceedings of the IEEE/CVF international conference on computer vision, pages 53 10-5319,, 2019.
[63] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Conference on Empirical Methods in Natural Language Processing, 2017.
[64] Tejas Srinivasan and Yonatan Bisk. Worst of both worlds: Biases compound in pre-trained vision-and-languagg models. ArXiv, 0aa/2104.06666 2021.
[65] Kaylee Burns, Lisa Anne Hendricks, Kate Saenko, Trevor Darrell, and Anna Rohrbach. Women also snowboard: Overcoming bias in captioning models. arXiv preprint arXiv: 10X3vv800309797 2018.
[66] Mustafa Atay, Megh Poudyel, and Saul Evora. Evaluation of gender bias in masked face recognition with deep learning models. 2023 IEEE Symposium Series on Computational Intelligence (SSCT), pages 829-835, 2023.
[67] Ahsan U1 Islam and Abm Rezbaul Islam. Gender bias in pre-trained deep learning models. 2023 IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE), pages 1-6, 2023.
[68] Siobhan Mackenzie Hall, Fernanda Gonçalves Abrantes, Hanwen Zhu, Grace Sodunke, Aleksandar Shtedritski, and Hannah Rose Kirk. Visogender: A dataset for benchmark- ing gender bias in image-text pronoun resolution. In Thirty-seventh Conference on Neu- ral Information Processing Systems Datasets and Benchmarks Track, 2023. URL https: / //openreview. net/ cest/forummicee F-
[69] Yusuke Hirota, Yuta Nakashima, and Noa Gareía. Quantifying societal bias amplification in image captioning. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13440-13449, 2022.
[70] Shruti Bhargava and David A. Forsyth. Exposing and correcting the gender bias in image captioning datasets and models. ArXiv, abs//911 191.00078, 2019.
[71] Pierre Stock and Moustapha Cisse. Convnets and imagenet beyond accuracy: Understanding mistakes and uncovering biases. In Proceedings of the European conference on computer vision (ECCV), pages
2018.
[72] Zaid Khan and Yun Raymond Fu. One label, one billion faces: Usage and consistency of racial categories in computer vision. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021.
[73] Kimmo Kärkkäinen and Jungseock Joo. Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation. 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1547-1557, 2021.
[74] Kinshuk Sengupta and Praveen Ranjan Srivastava. Causal effect of racial bias in data and machine learning algorithms user persuasiveness & discriminatory decision making: An empirical study. ArXiv, abs/2202.00471, 2022.
[75] Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. The risk of racial bias in hate speech detection. Annual Meeting of the Association for Computational Linguistics, 2019.
[76] Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740-755. Springer, 2014.
[77] George A. Miller. Wordnet: A lexical database for english. Commun. ACM, 38:39 41, 1995.
15

---

[78] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to- -mage diffusion models. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836-3847, 2023.
[79] Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. Conference on Computer Vision and Pattern Recognition 2023, 2023.
[80] Linoy Tsaban and Apolinário Passos. Ledits: Real image editing with ddpm inversion and semantic guidance. 2023.
[81] OpenAl. Chatgpt: A large language model trained by openai. https://chat.openai. com/, 2023. Accessed: 2024-05-20.
[82] Meta meta.ai. https: htttps://www.meeeee http://wwwe.aaai.//// ai/. [Accessed 22-05-2024).
[83] Demis Hassabis and Google DeepMind Team. Introducing gemini: Google's most capable ai model yet, 2024. URL https://blog, /oogg//teccooooog////aooo/o/o/// ai http:///aaaiittttiiiiaiiiaaaaatt gemnnni/.mmi....mm
Accessed: 2024-05-20.
16

---

Appendix
6.1
MI and PID
Here, we explain the concepts of mutual information and partial information decomposition in greater detail.
Mutual Information (MI) is a measure that estimates the amount of information obtained for an output variable from predictor variable passed through function. It measures to what extent input variable reduces the uncertainty in the model's prediction. In deep learning, where understanding the dependencies and mapping between input features and target variables is vital, MI serves powerful tool to quantify these dependencies accurately.
Partial Information Decomposition (PID), defined by [60], shows that two predictor variables X and X2 can contribute information about target variable Y in four possible ways. X1 can provide some information about Y independentlyluniquer
X2 too can provide some information about Y independently/uiquelyy both X₁ X2 can contribute some overlapping/redundan information, i.c., some of the information contributed by both is the same, and X1 & X2 can contribute synergistic information which information neither of them can contribute on their own and only conjunction with the other. [60] also state that mutual information fundamentally pointwise quantities where they measure the information content of individual events. The overall (average) mutual information can then be calculated by taking the expectation all events for the relevant variables.
be derived from first principles as
Simplification of Mutual Information
Here we outline the derivation of Eq 3 from Eq 2 using the orthogonality principle:
I(X;Y)= -Ep( (ogpp(())) - log p(x)]
(1)
((: ((x:Y)=Epoonly Epcennl Eptcille {a(xa)) l|e- Ea (a(zall))l]daa
(2)
By expanding all the squares and re-arranging we get:
[1
I(X;Y) = Ep(z,))
1/2 Eptc) [llea(=a) - (a((aal)) da +
[
[ / (p(c))))))) ((Ea(Ta) en (xalv)) (Ea(salu) - da
21E 2
(3)
Here,
[
1 Ep(rl))pto ((e. (xa) és (xalv)) (Eo (#al))
+2Ep(n)
=e] da =
(4)
Based the orthogonality principle [1], which states:
vf. Ep(alsnne) (I y) (Ea(Iall) - c)] =
(5)
The term (2n (xF. ly) c) represents the error of the MMSE estimator, which is orthogonal any estimator. Therefore, the second term becomes zero, leading to:
[ 2
I(X:Y = Ep(a.w
Epce) [IlEa(Fa) - éa ( (xal1))11 da
(6)
This is the same Eq 3.
17

---

disphrt
metress
Figure 11: MMSE curves comparing the standard (Eq 2) and orthogonal (Eq 3) estimators
6.3 Standard vs Orthogonal Estimators
can be seen that the original form (dotted line) is more unstable with many zigzag patterns. We also the orthogona//iimplifee form (continuous line) enforees better consistency between the MMSE (blue) and conditional MMSE (red). Thus, this simplification works better in practice.
6.4 Estimator Uncertainty
There is no guarantee that the estimator provides an upper or lower bound for the PID terms. It dependent the conditional and unconditional denoising MMSEs obtained from the diffusion model which is assumed to be an optimal denoiser for our experiments. However, in practice, this assumption need not hold because neural network trained on a regression problem minimize MSE need not converge to the global minima and instead may converge to local minima. That being said, neural networks have been found do really well on regression problems and the diffusion model, specifically, has been found to perform well on the denoising problem. Thus, we expect reasonable estimates despite the inherent uncertainty.
Redundancy varying No. Sumples
Redundancy with varying No. NNSS
Figure 12: Redundancy maps for the varying levels of noise/SNR and number of samples
A measure of uncertainty is also introduced based on the number of samples under the same noise level, a, in Eq 3's expectation term and from the number of values sampled evaluate the integral in the equation. Thus, we can obtain more confident information maps by using higher values for both of these hyperparameters. In Fig 12, we provide visuals the information maps for varying values of these hyperparameters on the "cat and elephant" sample from the COCO co-hyponym experiment (Fig in the main text). We observe that the maps depict the same information, i.e., they are highly activated in the same regions across variations but do become less noisy at higher values.
18

---

6.5 Baselines for Redundancy
When comparing CMI, MI, and DAAM with our redundancy maps, we take the intersection of the maps these methods produce for the two words/phrases individually. This is done in the following steps:
The individual maps are thresholded to obtain binary masks. The threshold we employ is f - 1.50, where refers to the mean of the map being thresholded and refers to the standard deviation. We use 1.5 times the standard deviation to discard roughly 86.64 of the map's s values
observe that they are irrelevant and correspond to the low-activated background region.
We multiply the binary masks of the two words' maps to obtain the final regions of intersec- tion/overlap between the two maps.
Finally, the above intersection map im is multiplied with each of the two maps, m, and m2, followed by the computation of the pixel-level mean of the two masked maps obtain the final map f as:
f = (m1 im + my im)/2
6.6 Image Editing via Prompt Intervention
We follow a similar process to other image editing methods [79, 80] where we progressively add Gaussian noise to the image and pass through the diffusion model for denoising but now this reverse diffusion process is conditioned on the edited prompt instead of the original prompt. We edit the original prompt by removing a word from to see if the edited image similar to the original to help determine if the omitted word is redundant. For instance, we remove the word "desk" from the prompt "The desk was a sturdy table perfect for working" to produce the edited prompt "The was a sturdy table perfect for working" which is then used to condition the denoising process. The intuition behind this experiment is that for words that contribute little information on top of the rest of the prompt, the model will keep the edited image very similar to the original as it will follow similar path as that would be followed if were conditioned on the original prompt.
6.7 Datasets
We create datasetss for each of our numerous experiments. We primarily rely on existing hierarchical datasets, Wordnet [77] and COCO [76], and free-access Large Language Models, namely Chat-GPT [81], Meta.ai [82], and Gemini [83]. We probe these models with prompts like "make sentences with pairs of synonyms of common tangible nouns", "make sentences with category and subcategory of common tangible nouns". "generate sentences with synonyms of common tangible nouns and both of those synonyms must be present the sentence. "generate pairs of sentences containing homonyms in different contexts" and so on. A combination of the outputs produced by these models was used to obtain the final set of prompts for each task. Once we had the prompts, we fed them into the open- source diffusion model, Stable Diffusion 2.1 [7]. to generate images corresponding to those prompts. During the image generation, we kept the seed constant (42). Following are the prompt-image pair datasets we introduce:
Gender Bias Dataset: Here we generated a list of 188 most common occupations from the LLMSs ([82).[11) I..[33........... These occupations then combined with each gender one a time to produce a total of 376 prompts. For instance, the occupation "Doctor" is used to generate two prompts. "Female Doctor" and "Male Doctor". sample of this dataset is visible in Fig 13.
Ethinic Bias Dataset: Instead of combining the previously mentioned 188 occupations with a gender, we now combine them with one of four ethnicities (Caucasian, Asian, Black, Hispanic) in each prompt produce total of 752 prompts. For example, the occupation "Doctor" is used to generate four prompts, "Caucasian Doctor", "Asian Doctor", "Black Doctor", and "Hispanic Doctor". A sample of this dataset is visible in Fig 14.
For this dataset, we generated 121 pairs of prompts, where each pair corresponds to one homonym but in different contexts. Thus, in total, we produced 242
Homonym Dataset:
19

---

Cupeesr
Fimel Corp
Figure 13: Gender Bias Dataset Sample
Figure 14: Ethnic Bias Dataset Sample
prompts, of which were generated from the LLMSs [82],[81].133) sample this dataset is visible in figures 15a.
rimbeed vellee Braghila sare
texiall
abean nax
The bese rapported the weight
buan of brights
eighhatt and zebra
apple and cake
(a) Homonym Datasct Sample
(b) Co-Hyponym Dataset (COCO) Sample
Synonym Dataset:
The entire dataset of 132 prompts containing synonym pairs was generated using the previously mentioned LLMs [82]][81][[8)) A sample of this dataset is visible Fig 16.
Co-Hyponym Dataset (Wordnet): To create this dataset, first sampled co-hyponyms from Wordnet for the most frequently occurring tangible nouns in the COCO dataset. Next, we generated prompts with the format: co-kypony and co-hyponym". For example, one of the most commonly used nouns across the COCO dataset is "bench", so we find its co-hyponyms from Wordnet: "box". "seat", "chair", "ottoman". "stool", "sofa" and "couch". Then, these are used to produce prompts such as "a box and a bench", "a chair and bench", sofa and a bench" and so on. sample of this dataset is visible in Fig 17.
Co-Hyponym Dataset (COCO): We took the super-categories of the COCO dataset and paired the objects within each individual super-category to produce list of 197 co-hyponym
20

---

conmetica
The ragal
was alaty wlevater Aap betweon Soors
house was warnhh lovd
foctures creating dainingg spocerance
mona donardtris head
working
Figure 16: Synonym Dataset Sample
biryani and
A bamboo and sugarcane
spaghetti
badminton and A antelope and sheep
volleyball Figure 17: Wordnet Dataset Sample
pairs which then converted prompts using similar format that used for the Wordnet dataset above. For example, "vehicle" was super-category, so "a car and airplane", "a car and boat" and "a motorcycle and a truck" are a few prompts produced from that super-category. A sample of this dataset is visible in Fig 15b.
6.8
More Results
In this section, we present more results from our experiments.
6.8.1
Homonyms
Below some examples of results for Homonym Experiments.
Image
Redundancy
Uniqueness1
Uniqueness2
Synergy
C = the calf frolicked in the field alongside its mother
= 'calf' vs 'field'
21

---

Image
Redundancy
Uniqueness1
Uniqueness2
Synergy
C = she stretched her calf muscles before going for a run 'calf' vs 'run'
=
Image
Redundancy
Uniqueness1
Uniqueness2
Synergy
C = she admired the stained glass windows in the church 'glass' vs 'windows'
Image
Redundancy
Uniqueness1
Uniqueness2
Synergy
C = she sipped her water from a tall glass 'glass' vs 'water'
Image
Redundancy
Uniqueness1
Uniqueness2
Synergy
c = the gang worked together
'gang' vs 'together'
22

---

Image
Redundancy
Uniqueness1
Uniqueness2
Synergy
C = the gang of gears was complex y 'gang' vs 'gears'
Image
Redundancy
Uniqueness1
Uniqueness2
Synergy
the hook on the door was useful = 'hook' vs 'door'
Image
Redundancy
Uniquenessl
Uniqueness2
Synergy
C = he baited the hook with a worm y 'hook' 'worm'
Below are some Synergy map visuals obtained from CPID on Homonym samples.
Image
Image
we saw crane standing gracefully in the marsh = 'crane' vs 'gracefully'
c the construction crew used crane to hoist steel beams into place y = 'crane' vs 'crew'
23

---

Image
S
Image
S
They enjoyed dinner the deck overlooking the garden 'deck' vs "garden'
c = He shuffled the deck before dealing the cards y 'deck' 'cards'
6.8.2 Synonyms
Below are some more examples of results for Synonyms Experiments.
Image
Redundancy
CMI
MI
DAAM
c = the altar was adorned with beautiful altar-piece adorned with intricate carvings
= 'altar' vs 'altar-piece'
Image
Redundancy
CMI
MI
DAAM
= the luggage and bag carried belongings with ease making travel
breeze y 'luggage' vs 'belongings'
Image
Redundancy
CMI
MI
DAAM
IERDO
C = the radio was broadcasting transmitter playing music and news 'radio' vs 'transmitter'
24

---

Image
Redundancy
CMI
MI
DAAM
C = the saw was sharp tool cutting through wood easily 'saw' vs 'tool'
6.8.3 COCO Co-Hyyonyms
Below are the results of Co-Hyponyms.
Image
Redundancy
CMI
MI
DAAM
C = a car and a bus y 'car' vs 'bus'
Image
Redundancy
CMI
MI
DAAM
C = a baseball bat and surfboard
= 'baseball' vs 'surfboard'
Image
Redundancy
CMI
MI
DAAM
C = a carrot and a pizza
y = 'carrot' vs 'pizza'
25

---

Image
Redundancy
CMI
MI
DAAM
C = a cat and a dog
y = 'cat' vs 'dog'
6.8.4 Wordnet Co-Hyponyms
Below are some more results for the Co-Hyponym experiments on the images generated from prompts made through Wordnet.
Image
Redundancy
MI
DAAM
C = a chamber and a classroom
y 'chamber' vs 'classroom'
Image
Redundancy
CMI
MI
DAAM
a door and a tower
= 'door' Vs 'tower'
26

---

Image
Redundancy
CMI
MI
DAAM
C = a glass and a window
= 'glass' vs 'window'
Image
Redundancy
CMI
MI
DAAM
a fox and dog
= 'fox' vs 'dog'
27

---

6.8.5 Prompt Intervention
Below are some more results for the Prompt Intervention experiments.
Original
Intervention
Redundancy
Uniqueness
CMI
DAAM
C = a cat and a sheep y 'cat'
Original
Intervention
Redundancy
Uniqueness
CMI
MI
DAAM
a cat and sheep
= 'sheep'
Original
Intervention
Redundancy
Uniqueness
CMI
MI
DAAM
c a baseball glove and a tennis racket y "baseball'
Original
Intervention
Redundancy
Uniqueness
DAAM
c = a baseball glove and a tennis racket
y 'tennis'
28

---

Original
Intervention
Redundancy
Uniqueness
CMI
MI
DAAM
a skis and a surfboard 'skis'
Original
Intervention
Redundancy
Uniqueness
CMI
MI
DAAM
a skis and a surfboard 'surfboard'
Original
Intervention
Redundancy
Uniqueness
CMI
MI
DAAM
clock and vase
= 'clock'
Original
Intervention
Redundancy
Uniqueness
CMI
MI
DAAM
a clock and a vase
= 'vase'
29

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Sections 3 and 4
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Section has limitations mentioned
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
30

---

Section 3 outlines various theoretical concepts with the required proofs and detailed explanations provided in the appendix in sections 6.1, 6.2, 6.3, and 6.4.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Sections 3 and 4 have the required information to reproduce the experiments.
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
31

---

Answer: [Yes]
Justification: We have shared our code and data in the supplementary. We will releasing the code publicly before the main conference.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https:
httpps.ccppublic//ee /edoedeeeeombdbmiiii ssi ssionooiccy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Sections 3 and have the required details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: Our experiments don't 't really involve such results that would require error bars or statistical significance tests they are not iestribution-dependee
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
32

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπхо rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, the GPU used was an A6000 and it took around 7 seconds to get an image generated with maps. This information is mentioned section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https://neuripg: hwtpp/ubbbc/ttiicccueeeeeeeeeeee
Answer: [Yes]
Justification: is as per the guidelines.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The required discussions are spread across the work. More of it can be found
in section 1.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
33

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our model doesn't 't pose such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The assets used are properly cited and credited.
Guidelines:
existing assets. The answer NA means that the paper does
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
34

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The shared data and code is described along with itself.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
No crowdsourcing done.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
No such experiments or participants involved.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
35


Reviews:
[{"review_id": "yHleZ8NKvI", "reviewer": "Reviewer_vbX2", "summary": "Summary: \nThe paper adapt the concept of the Partial Information Decomposition into the diffusion model and analyze the uniqueness, redundancy and synergy terms in the diffusion model and do experiments on the Bias, Homonyms and Synonyms\n\nContribution: The paper adapt the concept of the Partial Information Decomposition into the diffusion model and out-perform other method s in visual perspective and explain the bias homonym and synonyms in the text token perspective by using the PID", "questions": "(i) I think the text condition  that you choose to present can be more complicated which may present more issue when the user's prompt are complicated.\n\n\n(ii) Why some of the uniqueness figures in the 6.5.1 Homonyms sections show meaningless information? How is the uniqueness different from other single token method?  Is it more convincing?", "limitations": "Yes", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 4, "strengths": "Strength:\n\n(i) Clarity: The paper is easy to understand.\n\n(ii) Originality and Significance: The paper is the first one to adapt the concept of the PID into diffusion model and consider the influence of more than one token on the generated images. It also help explained the  Bias, Homonyms and Synonyms problem in the diffusion model compared to other current methods.", "weaknesses": "Weaknesses:\n\n(i) The paper's novelty is limited. The paper does not go beyond the concept of PID but simply adapt the PID concept to the diffusion model. Maybe extend to more than two tokens. \n\n\n(ii) The text conditions is simple. Try the text conditions that more similar to the distribution of the training data.\n\n\n(iii) The uniqueness figures in the 6.5.1 is not very convincing to me and not show very clear information.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your detailed review and thoughtful comments. We address your concerns below and will refer to the general response at the top for certain points.\n\n> **Q1)** Novelty of our method and more complex use cases such as an extension to the multi-concept scenario\n \n1. Our approach is the first to adapt PID for diffusion models by developing the required mathematical formulations. We derive a mathematically sound pixel-level PID breakdown in a form that is compatible with and could be integrated into diffusion. Furthermore, we leverage our method to conduct a detailed study of diffusion and to identify its various failure modes, which significantly advances our understanding of different concepts and their interactions in the diffusion model. \n2. Building upon the original concept of PID, we also introduce Conditional PID (CPID) for diffusion models to expand the scope and contributions of this work and provide more details in the global response above (**Figs 1, 2, 3, 4** in the attached PDF). This also helps address the suggestion on expanding to more terms as CPID takes the rest of the prompt into consideration. We observed that CPID yields superior localized results compared to its PID counterpart in practice as it takes into account the rest of the prompt as context during the image generation process, allowing it to better capture the specific contributions of the two tokens being analyzed. We will include the results in the next revision of the paper as well.\n3. Computing the PID and even MI (Mutual Information) between three or more input variables in information theory is a complex process. There is no universally accepted method for defining and calculating the terms of Uniqueness, Redundacy and Synergy. While works such as [1] provide a method to compute the global Redundancy between all input variables, Uniqueness and Synergy remain difficult terms to tackle as these computations need to take all the variables subsets' interactions into consideration. It is generally agreed [1, 2] that multivariate information decomposition is more involved than the bivariate case because it is not immediately obvious how many atoms of information one needs to consider, nor is it clear how these atoms should relate to each other. Even the breakdown provided in [2] for the four-variable case is highly complicated and it is a non-trivial problem to extend these concepts from information theory to more variables.\n4. The primary objective of the analysis conducted in the main text was to learn more about the text-to-image diffusion model's understanding of different concepts, which may or may not align with human understanding, and to derive an analysis for specific prompt types. We agree that the examples used in the paper are relatively simple, but they can still shed light on the various interactions and shortcomings of the diffusion model in an easily interpretable form. To address the need for more complex prompts, we provide results on longer prompts with more entities, similar to those used in the diffusion training data, in **Figs 1 and 2** in the attached PDF and provide more details on the new experiments in the global response above. We will include the new figures in the next revision of the paper as well.\n\n> **Q2)** Clarification on uniqueness figure 6.5.1 and distinction from single-concept methods\n\nThe uniqueness information maps in Fig 6.5.1 tries to show what information that term uniquely contributes on its own to the image generation process. Single-concept methods like DAAM, MI, and CMI highlight the overall information a concept contributes which is a combination of Synergy, Redundancy and Uniqueness. We clarify this in Eq 5 in the main text. It is possible that something can have little Uniqueness, i.e., contributes very little unique information on its own given the rest of the prompt or even just another concept as sufficient context, but still have a contribution through Synergy and/or Redundancy. For instance, in the \"calf\" vs \"field\" example, the concept \"calf\" when interpreted as the calf of an animal, is enough to make the model generate a field. Thus, the \"field\" concept does not contribute much unique information on its own but it does provide synergistic information in combination with \"calf\" that helps the model interpret the meaning of \"calf\" as the animal and not the muscle, which is why we see a high Synergy. As long as there is some other term instead of \"field\" that can provide the required context to make the diffusion interpret the \"calf\" as the animal, then removing \"field\" would still generate the grass field in the image. Similar arguments can be made for the rest of the examples as well. Also, our approach does not change depending on the order of the concepts being processed by DiffusionPID. We will update the figure captions in an upcoming revision to clarify this point. We thank you for bringing this to our attention.\n\n\n[1] Conor Finn and Joseph T. Lizier. Pointwise information decomposition using the specificity and ambiguity lattices. ArXiv, abs/1801.09010, 2017.\n\n[2] Paul L. Williams and Randall D. Beer. Nonnegative decomposition of multivariate information, 2010.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4FTcDb81aw", "reviewer": "Reviewer_Tpva", "summary": "The paper proposes a new technique called DiffusionPID to explain how diffusion models transform text cues into images through partial information decomposition (PID). This work deconstructs mutual information into redundancy, synergy, and uniqueness to analyze how individual concepts and their interactions shape the generated images. This paper conducts extensive experiments and visualizations to validate its techniques.", "questions": "1.\tWhy do you use BERT to measure the p(y)? Does the choice of different encoders significantly impact p(y)?", "limitations": "1.\tThe analysis lacks exploration of more complex text prompt scenarios, such as those involving interactions among more objects and more complex phrases that encompass attributes along with objects.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.\tThis paper proposes new techniques to explain the influence of multiple concepts and their interactions on diffusion models.\n2.\tThis paper is well-written and easy to follow.\n3.\tThere are extensive visualizations for many scenarios to interpret the process of diffusion models. This work explains some problems, such as bias and incorrect generation, based on its method, which indicates the direction for improvement.", "weaknesses": "1.\tThe experiments primarily focus on the impact of interactions among multiple object concepts on image generation, lacking analysis of more general scenarios, such as the interactions between objects and their attributes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your thorough review and insightful feedback. We address your suggestions below and also refer to the global response at the top in a few places.\n\n> **Q1)** Analysis on objects and attributes\n\nWe agree that our work could be further improved by providing additional analysis on the interaction between objects and their attributes. We address this by providing new visuals for these types of interactions in **Figs 1 and 2** in the attached PDF along with other experiments (CPID) as detailed in the global response above. We will also include these figures in the next revision of the paper.\n\n\n> **Q2)** Choice of BERT\n\nWe would like to point out that BERT was the original language backbone of choice for latent diffusion [1]. They emperically find that BERT can be effectively used to encode semantic information of images in a generation setting.\nFurthermore, the common methods to compute $p(y)$ involve modeling a distribution of the language's vocabulary in natural language. For this, the only options are: textual databases or language models. \nOnline available databases (ex. Wikipedia:Word frequency, Google Books Ngram Viewer, Corpus of Contemporary American English, etc.), can only be used to obtain the independent probability of each term based on frequency statistics but it remains difficult to obtain an accurate conditional distribution. Moreover, to calculate this over large corpuses online would be computationally very expensive. That's why we used a language model instead. \nBERT has been the standard language model used in natural language research since its conception due to its excellent performance on various language-based tasks and its accurate model of the vocabulary distribution in both, conditioned and unconditioned cases. We did not feel the need to use LLMs as the context window here is fairly small given that we operate on prompts of moderate length. Thus, this choice does not significantly impact $p(y)$.\n\n\n\n[1] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. Highresolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Yx8eSOOcMj", "reviewer": "Reviewer_hzC5", "summary": "Decomposing different types of information (redundant, synergistic, unique) has long been a niche field due to the difficulty of applying these methods in realistic settings. This paper develops a way to dissect these types of fine-grained information measures in a practical way in the realistic setting of text-to-image diffusion models. The authors perform extensive experiments to show that the resulting method can be useful for identifying biases, understanding model errors, probing complex linguistic features, and for model interpretability in general.", "questions": "For PID veterans, Eq. 5 is straightforward. Depending on other reviewers' reactions, it might be necessary to give more context (like the famous diagram that is often associated with it). \n\nYour PID was heavily based on Williams and Beer measure (though adapted for the pointwise measure). I'm curious, did you consider other PID formulations besides Beer & Williams? Probably not something that needs to be addressed in the paper since NeurIPS community is not very familiar with these ideas (yet). \n\nI haven't studied this paper, but I know other people have thought about how to adapt PID to pointwise measures also: \nFinn C, Lizier JT. Pointwise Partial Information Decomposition Using the Specificity and Ambiguity Lattices.\n\nMinor formatting quibble - log and arg in Eq 1,2 should be formatted as operators, not italics.", "limitations": "This was discussed adequately.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- Well-written motivation and examples\n- The adaptation of PID to a tractable diffusion model measure looked elegant, and included some nontrivial twists (including accounting for the pointwise measure ambiguities) \n- Good overview of related work including broader attempts at model interpretability\n- Extensive experiments explore a wide array of interesting \n- The provided datasets can be a benefit to future research. \n- In the cases where you could quantify biases, it was nice to see that you get a large and reliable signal.", "weaknesses": "- Interpretability research is intrinsically very challenging, as it is hard to verify ground truth and even saying what counts as \"interpretable\" can be a bit nebulous. Generally the information maps looked intuitive, but in some cases I felt I had to squint a little to see the relationship. \n- Related to that, there's some uncertainty in your estimator (terms in Eq. 2). It would be nice to understand how noisy / confident the information maps are. \n- Comparisons can also be difficult - there's no existing method that exactly does what PID does, and this is the first approach to extend PID to high-dimensions. The comparisons with MI/CMI and attention methods like DAAM seemed like a reasonable way to handle this.\n- In some cases, it was hard to imagine how we could use the results. For instance, consider the homonym failure case identified by synergy. To make use of this in practice, the user would have to know to associate a homonym with a specific context word that should exhibit synergy. This is sort of an exploratory work, but finding more concrete applications would improve impact.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your comprehensive review and thoughtful comments. We also appreciate your comments for explicitly highlighting the difficulties of adapting PID to diffusion and the challenges of working on the interpretability of models.\n\n> **Q1)** Relationship between PID maps and human intuition\n\nWe thank you for acknowledging and drawing attention to the hurdles in this kind of work. Although we show examples that align with human intuition, we also show examples where it's possible that not every relation matches human intuition, as we are exploring the diffusion model's understanding of the world. We also introduce CPID (Conditional PID) as an extension of PID to better account for the context provided by the rest of the prompt. We refer you to the global response above for further details on CPID and the attached PDF for visuals (**Figs 1, 2, 3, 4**).\n\n> **Q2)** Estimator's uncertainty \n\nWe expand further on this query in the global response above and will also add an explanation in the paper's next revision.\n\n> **Q3)** Applications of DiffusionPID\n\nThe primary goal of our work was to introduce the concept of PID into the rapidly growing generative field in computer vision. We believe this would lay down the groundwork to further the understanding of the internal concepts learned by diffusion. This would produce more concrete ways to evaluate and progress prompt engineering. PID can identify which elements of a prompt provide unique information, allowing for the refinement of prompts to reduce redundancy and enhance synergy. Our method can also help better understand and counter diffusion's attribute binding problems and learned biases such as those demonstrated in our paper. We also believe incorporating PID-based insights into model training could ensure that the model better captures human-like understanding of concepts and thus make it more human-aligned.\nOne way to address the point regarding the requirement of apriori knowledge of which words to run PID on is that several pairs of terms could be sampled automatically from the prompt (with some filtering of pairs such as based on LLM-based semantic distance between the terms) and run through PID. We agree that the practical application of PID requires further work and is something we plan to explore in the future but believe that this work provides the necessary foundation for this direction.  \n\n> **Q4)** PID Diagram\n\nThank you for raising this point. We have added a figure (**Fig 5**) to the attached PDF to give more insight into the concept.\n        \n> **Q5)** Williams and Beer and Finn C, Lizier JT's work\n\nYes, as you rightly recognized, one of the major reasons behind building on the Williams and Beer formulation was the pointwise measure. The choice was also based on the observation that the NeurIPS and the AI community previously used this formulation [1]. Furthermore, both works were quite helpful for our paper. We thank you for mentioning the latter and will make sure to cite it in the next revision of the paper.\n\n> **Q6)** Formating and typos\n\nThank you for bringing this to our attention and we will fix this in the next revision of the paper.\n\n[1] Yu, S., Wickstrøm, K., Jenssen, R., & Principe, J. C. (2020). Understanding convolutional neural networks with information theory: An initial exploration. IEEE transactions on neural networks and learning systems, 32(1), 435-442.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bdYJSo9vXe", "reviewer": "Reviewer_THqw", "summary": "In this paper,  the authors propose a novel approach to analyze the uniqueness, redundancy, and synergy terms in text-to-image diffusion models by applying information-theoretic principles to decompose the input text into its elementary components. In particular, the proposed approach can be used to recover gender and ethnicity biases in image generation.", "questions": "1.\tPlease provide more analysis on the efficiency of the proposed approach.\n\n2.\tActually, I understand that it is not easy to evaluate the effectiveness and make others convincing with only some case studies in Figure 1-3. It is better to publish the demo, where readers can test the approach by themselves.", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1.\tIt is an important and valuable direction to analyze the potential semantic bias in the field of image generation. I think it is also reasonable to invovle information-theoretic principles into the approach. \n\n2.\tThe proposed approach is a good attempt for measuring the gender and ethnicity biases in image generation. \n\n3.\tBased on the illustration in Figure 1-3, the proposed approach may provide more interpretability for text-to-image diffusion models, which is important to understand and exploit the models.", "weaknesses": "1.\tAre there something wrong in Equation 2? The left parts in those two equations are the same but the right parts are not.\n\n2.\tThe time complexity if necessary for the proposed method", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your valuable feedback and helpful comments. We address your concerns below.\n        \n> **Q1)** Clarification on Eq 2:\n\nWe thank you for mentioning this. Unfortunately, our footer note didn't make it through the submission. Eq 2 is correct, and the second equation is derived from the first using the orthogonality principle, as explained below:\n\n\n\n$I(X; Y) = \\mathbb{E}_{p(x,y)} [\\log p(x|y) - \\log p(x)]$\n\n\n\n$I(X; Y) = \\mathbb{E}_{p(x,y)} [ \\frac{1}{2}  \\int \\mathbb{E}\\_{p(\\epsilon)}[ \\|\\| \\epsilon - \\hat{\\epsilon}\\_{\\alpha}(x\\_{\\alpha})\\|\\| ^2 - \\  \n \\| |\\epsilon - \\hat{\\epsilon}\\_\\alpha (x\\_{\\alpha} | y) \\| \\|^2 ] d\\alpha]$\n\nBy expanding all the squares and re-arranging we get:\n\n$I(X; Y) = \\mathbb{E}\\_{p(x,y)} \\left[ \\frac{1}{2} \\int \\mathbb{E}\\_{p(\\epsilon)} \\left[ \\| \\| \\hat{\\epsilon}\\_\\alpha (x\\_\\alpha) - \\hat{\\epsilon}\\_\\alpha (x\\_\\alpha | y) \\| \\|^2 \\right] d\\alpha \\right] + 2 \\mathbb{E}\\_{p(y)} \\left[ \\frac{1}{2} \\int \\mathbb{E}\\_{p(x|y), p(\\epsilon)} \\left[ (\\hat{\\epsilon}\\_\\alpha (x\\_\\alpha) - \\hat{\\epsilon}\\_\\alpha (x\\_\\alpha | y)) \\cdot (\\hat{\\epsilon}\\_\\alpha (x\\_\\alpha | y) - \\epsilon) \\right] d\\alpha \\right]$\n\nHere,\n\n$+ 2 \\mathbb{E}\\_{p(y)} \\left[ \\frac{1}{2} \\int \\mathbb{E}\\_{p(x|y), p(\\epsilon)} \\left[ (\\hat{\\epsilon}\\_\\alpha (x\\_\\alpha) - \\hat{\\epsilon}\\_\\alpha (x\\_\\alpha | y)) \\cdot (\\hat{\\epsilon}\\_\\alpha (x\\_\\alpha | y) - \\epsilon) \\right] d\\alpha \\right] \\equiv \\ominus$\n\nbased on the orthogonality principle [1], which states:\n\n$\\forall f, \\quad \\mathbb{E}\\_{p(x|y) p(\\epsilon)} \\left[ f(x\\_\\alpha, y) \\cdot (\\hat{\\epsilon}\\_\\alpha (x\\_\\alpha | y) - \\epsilon) \\right] = 0$\n\nThe term $(\\hat{\\epsilon}\\_\\alpha (x\\_\\alpha | y) - \\epsilon)$ represents the error of the MMSE estimator, which is orthogonal to any estimator $f$. Therefore, the second term becomes zero, leading to:\n\n$I(X; Y) = \\mathbb{E}\\_{p(x,y)} \\left[ \\frac{1}{2} \\int \\mathbb{E}\\_{p(\\epsilon)} \\left[ \\| \\| \\hat{\\epsilon}\\_\\alpha (x\\_\\alpha) - \\hat{\\epsilon}\\_\\alpha (x\\_\\alpha | y) \\| \\|^2 \\right] d\\alpha \\right]$\nwhich is the 2nd line in Eq 2. \n\n\n**Fig 6: MMSE curves comparing the standard and orthogonal estimators**\n\nWe also provide graphs comparing the MMSE estimate obtained from the original equation form (the first equation in Eq 2) and the simplified form for varying levels of noise/SNR in **Fig 6** in the attached PDF. It can be seen that the original form (dotted line) is more unstable with many zigzag patterns. We also see the orthogonal/simplified form (continuous line) enforces better consistency between the MMSE (blue) and conditional MMSE (red). Thus, this simplification works better in practice.\n\n\n> **Q3)** Time analysis\n        \nOur primary goal in this work was to help improve the interpretability of diffusion models. Although the time complexity of such methods is not of concern usually given that they are used for interpretability of a model, we understand that this is an important aspect. Methodologically, once we have the generated image from diffusion, our method involves running the diffusion model's UNet for denoising to get the MMSE-based log probability estimates and BERT to obtain the terms' probabilities, as required for PID. Thus, our model's time complexity is proportional to that of the diffusion + BERT model. The exact time would depend on the hyperparameters and the computational resources used.\n        \n\n        \n\n> **Q4)** Public demo and code release\n\nWe would like to clarify that we provide code for our method in the supplemental zip, which is visible to the reviewers. This code release will help ensure reproducibility, soundness check and facilitate future research. \n        \nWe are unable to share a hosted model at this current point in time as we need to maintain anonymity as per the guidelines. We will also release the code and models publicly after the end of the anonymity period. We will also add a public demo for easy usage and testing of our pipeline.\n\n \n        \n\n[1] Steven M. Kay. 1993. Fundamentals of statistical signal processing: estimation theory. Prentice-Hall, Inc., USA.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for taking the time to go through our work in detail and providing insightful reviews. We found the feedback very constructive and helpful.

We are glad that the reviewers unanimously agree that the proposed information-theoretic approach to interpret diffusion models is a novel and promising direction. We are further glad that reviewers found our experiments to be extensive (Tpva, hzC5), consider this work impactful for identifying the shortcomings of diffusion models (vbX2, Tpva, hzC5, THqw), and deemed our paper to be well-written (vbX2, Tpva, hzC5).

Some reviewers (vbX2, Tpva) pointed out a shortage of sufficiently complex examples that explore the application of PID in more challenging scenarios and to study relationships between more diverse lexical entities. We agree, and address this by providing three sets of visuals in the attached PDF:
1. **Figs 1, 2**: Visuals for more complex prompts taken from [1] similar to those in the diffusion training distribution. These prompts usually mention several objects and their corresponding attributes. We find that our method remains effective and informative even in these challenging examples. We visualize the information maps between objects and attribute-defining terms as per Tpva's suggestion. In both cases, we observe a high synergy because the attribute modifies the object's visual properties in some form.    
 
2. **Figs 1, 2, 3, 4**: Visuals for CPID (Conditional PID). In this case, we extend the concept of PID for the conditional case where all the PID components and probability terms are now conditioned on the rest of the prompt. This is similar to the CMI extension of MI in [2]. We rewrite the equations from the main text with the required changes below for easy reference (we follow the same notations and definitions with an additional variable of $y$ signifying the rest of the prompt with the terms $y_1$ and $y_2$ removed):
    \begin{align}
    i(y_1, y_2; x | y) &= r(y_1, y_2; x | y) +  u(y_1 \backslash y_2; x | y) + u(y_2 \backslash y_1; x | y) + s(y_1, y_2; x | y) \\
\end{align}
 \begin{align}
    r(y_1, y_2; x | y) &= \min_{y_i} [-log\ p(y_i | y)] - \min_{y_i} [-log\ p(x|y_i, y) + log\ p(x | y) - log\ p(y_i | y)] \\
\end{align}
 \begin{align}
    u(y_1 \backslash y_2; x | y) &= i(y_1; x | y) - r(y_1, y_2; x | y) \\   
\end{align}
 \begin{align}
    s(y_1, y_2; x | y) &= i(y_1, y_2; x | y) - r(y_1, y_2; x | y) -  u(y_1 \backslash y_2; x | y) - u(y_2 \backslash y_1; x | y) 
    \end{align}
    
    We found that CPID provides slightly better localized results than its PID counterpart in practice as can be seen in **Figs 1 and 2**. This is expected as CPID accounts for the contribution of the rest of the prompt as the context in the image generation process and better captures the specific contribution of the two terms under consideration.

Some reviewers (hzC5, THqw) also requested further clarification of Eq 2 from the main text. To address THqw's query, we provide the derivation of the simplification done based on the orthogonality principle in the response to THqw. 
    
To further expand on the estimator's uncertainty as mentioned by hzC5:

1. There is no guarantee that the estimator provides an upper or lower bound for the PID terms. It is dependent on the conditional and unconditional denoising MMSEs obtained from the diffusion model which is assumed to be an optimal denoiser for our experiments. However, in practice, this assumption need not hold because a neural network trained on a regression problem to minimize MSE need not converge to the global minima and instead may converge to a local minima. That being said, neural networks have been found to do really well on regression problems and the diffusion model, specifically, has been found to perform well on the denoising problem. Thus, we expect reasonable estimates despite the inherent uncertainty. 

2. A measure of uncertainty is also introduced based on the number of samples under the same noise level, $\alpha$, in Eq 2's expectation term and from the number of $\alpha$ values sampled to evaluate the integral in the equation. Thus, we can obtain more confident information maps by using higher values for both of these hyperparameters. We provide visuals of the information maps for varying values of these hyperparameters on the "cat and elephant" sample from the COCO co-hyponym experiment (Fig 5 in the main text) in **Fig 7** in the attached PDF. We observe that the maps depict the same information, i.e., they are highly activated in the same regions, across variations but do become less noisy at higher values. 
    
We provide a figure, **Fig 5**, in the attached PDF to complement our explanation of PID as defined in Eq 5 as per hzC5's suggestion
    
[1] Stablesemantics: A synthetic language-vision dataset of semantic representations in naturalistic images. 2024.

[2] Xianghao Kong, Ollie Liu, Han Li, Dani Yogatama, and Greg Ver Steeg. Interpretable diffusion via information decomposition. arXiv preprint arXiv:2310.07972, 2023.

Meta-review: This paper introduces Partial Information Decomposition (PID) to interpret text-to-image diffusion models by decomposing input text prompts into their elementary components. The approach focuses on analyzing how individual tokens and their interactions through redundancy, uniqueness, and synergy influence the visual output of diffusion models. This method aims to provide insights into the model's handling of complex visual-semantic relationships and potential biases.

According to reviewers' comments, it is the first to apply PID in this context, offering a new lens to understand diffusion models. Its strengths include clarity, originality, and comprehensive experiments that illustrate its effectiveness in addressing biases and enhancing model interpretability. Reviewers praised the extensive visualizations and the practical application of the method in identifying and mitigating biases.

Reviewers pointed out several areas needing improvement: the paper could extend its analysis beyond simple text conditions and consider more complex interactions. There is also a call for clarity in some figures and equations, and concerns about the practical applicability of the results. Addressing these issues, particularly enhancing the robustness of the experimental setup and refining the clarity of visual representations, could significantly strengthen the paper.
Decision: Accept (poster)