Cross-Attention Head Position Patterns Can Align with Human Visual Concepts in Text-to-Image Generative Models

Authors: Jungwon Park, Jungmin Ko, Dongnam Byun, Jangwon Suh, Wonjong Rhee
Primary area: generative models
Keywords: text-to-image diffusion model, diffusion model, text-to-image generative model, cross-attention

Abstract:
Recent text-to-image diffusion models leverage cross-attention layers, which have been effectively utilized to enhance a range of visual generative tasks. However, our understanding of cross-attention layers remains somewhat limited. In this study, we introduce a mechanistic interpretability approach for diffusion models by constructing Head Relevance Vectors (HRVs) that align with human-specified visual concepts. An HRV for a given visual concept has a length equal to the total number of cross-attention heads, with each element representing the importance of the corresponding head for the given visual concept. To validate HRVs as interpretable features, we develop an ordered weakening analysis that demonstrates their effectiveness. Furthermore, we propose concept strengthening and concept adjusting methods and apply them to enhance three visual generative tasks. Our results show that HRVs can reduce misinterpretations of polysemous words in image generation, successfully modify five challenging attributes in image editing, and mitigate catastrophic neglect in multi-concept generation. Overall, our work provides an advancement in understanding cross-attention layers and introduces new approaches for fine-controlling these layers at the head level.

Decision: accept (poster)
Reviewer scores: 6, 8, 6, 6

Reviews and discussion:
<thread>
creation date:
1729836710431

thread_type:
official_review

<thread_object>
creator: reviewer_efqm

<content>
summary:
This paper mainly focuses on the explainability of text-to-image diffusion model. The authors propose a new metric based on the cross-attention heads in the diffusion UNet to illustrate the correlation between each attention head and visual concepts. Based on the proposed Head Relevance Vectors, the authors further propose several applications including solving polysemous words problems and image editing.

strengths:
1. The idea of correlating visual concepts with diffusion models is interesting.

weaknesses:
1. I suggest the authors add textual description of the proposed HRV instead of directly showing Fig. 2 and Fig. 4 for better understanding.
2. I wonder why <SOT> and many <EOT> are required during update of HRV?
3. It would be better to used SDXL or some more recent models such as SD3 as primary model, given that SD1.5 is kind of outdated.
4. It would be better to add a random weakening baseline in Fig.3.
5. In Sec.5.1 the authors show that by utilizing HRV the SD can generate more proper concepts. I wonder if this method can be compared with using classifier guidance, where the model is encouraged to align the generated image with wanted concepts in terms of CLIP score.

questions:
Please refer to the weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730347366002

thread_type:
official_review

<thread_object>
creator: reviewer_ynut

<content>
summary:
This paper tries to understand cross-attention (CA) layers *regarding attention heads*.
* The authors introduce N head relevance vectors (HRV) for N visual concepts.
* The strength of an attention head to the HRVs represent the relevance of the head to the concept.

Above properties are interpreted by *ordered weakening analysis*.
* Sequentially weaken the activations of CA heads to observe weakened visual concepts.

Boosting and reducing the strength of different heads control the strength of visual concepts. It helps three applications: 1) correcting mis-interpretation of words in t2i generation, 2) boosting prompt-to-prompt, 3) reducing neglect in multi-object generation.

strengths:
1. This paper provides a new perspective in understanding the features in text-to-image generation: different heads.
2. Qualitative examples (Figure 3a) and CLIP similarities (Figure3b) along weakening MoRHF and LeRHF clearly show the effect of weakening different heads.
3. The appendix provides extensive qualitative results to remove doubt for cherry-picked results.
4. The proposed method is useful for three applications: 1) correcting mis-interpretation of words in t2i generation, 2) boosting prompt-to-prompt, 3) reducing neglect in multi-object generation.
5. Discussions resolve natural questions: extension to SDXL and effect across different timesteps.

weaknesses:
1. The paper should provide principles of the proposed approaches.
    * L224 Why should we count each visual concept having the largest value to update the HRVs?
    * This is the most critical weakness for not giving a higher rating. I think the perspective is worth noticing but a solid paper should provide why/how it works.
    * Answering this question with theoretical justifications or intuition would strengthen the paper.
2. HRV should be described more clearly.
    * L205 a concatenation of token embeddings // concat along which axis? I guess the result of concatenation is $N \times (d + H)$. Then the query Q does not match the cross-attention operation because $Q\in R^d$. Am I missing something?
    * L210 K1, ..., KN should be denoted in Figure 2.
    * Adding equations and proper notations would help readers to understand the operation.
3. Human evaluation should be explained in more detail.  Appendix C.2 is not enough. Adding a table with Number of participants, Number and types of questions, Number of questions per participant, and Any quality control measures used would strengthen the user study.

Misc.: Related works -> Related work

questions:
I wonder the interpolation between different strengths of a head. For example, interpolating  material=[-2, 2]?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730456565446

thread_type:
official_review

<thread_object>
creator: reviewer_d4in

<content>
summary:
The paper proposes a method of constructing so called "HRV" vectors, which align with visual concepts. The authors leverage cross-attention layers of Stable Diffusion model to learn those vectors for predefined concepts. The proposed method helps to solve three known issues of the image synthesis task.

strengths:
1. Good motivation and a clear idea
2. Comprehensive quantitative and qualitative comparisons with many other solutions
3. The experiments, settings, and other details are mainly clearly explained

weaknesses:
1. It requires fixing a set of concepts beforehand for every HRV construction. Does not have a study of how the HRV matrix will be changed when some concepts are changed or replaced after the construction.
2. Manual settings, choice, and configuration are required for every concept (case) during inference (Sec 5.1, Fig 5). 
3. Lack of failed cases, there are no details about the limitations of this method.
4. Even though there is a section for bigger / novel models (SDXL), all experiments, studies, and comparisons are based on SD v1. New models might eliminate many issues the proposed method tries to solve.

questions:
1. Could you give more details about why there are some irrelevant concepts after a certain point of ordered weakening (Fig 9)?
2. Could you give more details about how the "h" is chosen/computed in the method of HRV updates?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730647730098

thread_type:
official_review

<thread_object>
creator: reviewer_miju

<content>
summary:
This work proposes Head Relevance Vectors (HRVs). HRs are an extension of the findings from previous works such as Hertz et al.'s P2P where cross attention maps were used to better understand t2i models and to edit images via prompts. HRV proposes using multiple concept words and concatenating them into a concept embedding matrix K which can then be applied to different heads of the cross-attention and by doing so, disentangle the different heads based on the concept they seem to be focusing on. The authors show this disentanglement of heads based on the concepts learned improved editing of images.

strengths:
The motivation of the paper is clear and build on a well studied problem of understanding the role of cross-attention and what they learn in editing T2I models. The experiments are visually appealing and tell the story of the paper well, especially the weakening of HRVs that shows weakening based on the most and least relevant concepts / heads. The authors show that using HRVs to edit images works better than SDEdit,P2P, etc. They also show improvement over Attend and Excite for the problem of catastrophic forgetting in T2I models.

 While in the weaknesses, I do mention my thoughts on the originality of this work, I believe using previous findings around CAs and targeting different heads and their roles in generating different concepts would be interesting to the community.

weaknesses:
I would argue that the work, while interesting, does not have new insight compared to what previous works such as P2P and Diffusion Self-Guidance have already already shown in regards to the role of cross-attentions. However, this work does take a step towards using those findings to narrow down on head-level manipulation of concept vectors. It goes without saying that T2I models could benefit from more comprehensive evaluation on larger set of generated images / human evaluation. However, I do understand the challenges this poses as well.

questions:
There have been recent works that show the <SOT> and <EOT> CAs capture different concepts. I would be interested to see if the authors found anything interesting regarding HRV and these tokens. I am also curious as to how the weakening and strengthening would work on more complex images that share entangled objects and concepts. For instance, what would weakening of "melting" look like for "a plastic car melting". I think this would be an interesting experiment since adjective and verb concepts are entangled with an object in a given image and HRV might to better in these cases than the counterparts.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper presents a method for constructing Head Relevance Vectors (HRVs) in text-to-image diffusion models to align with relevant visual concepts. These HRVs are vectors that indicate the importance of each cross-attention head for a given visual concept. The study demonstrates the effectiveness of HRVs through ordered weakening analysis and introduces methods for concept strengthening and adjusting to enhance visual generative tasks. 
The paper's strengths include: (1) Clear motivation and idea building on existing research. (2) Comprehensive comparisons with other solutions. (3) Demonstrated effectiveness over existing methods like Attend and Excite. The paper's weaknesses include: (1) New insights over P2P and Diffusion Self-Guidance are marginal. (2) Unclearity and missing details on certain parts (e.g., human evaluation). (3) Requires fixing set concepts for HRV construction. 
The majority of the reviewers' concerns were considered properly addressed in the rebuttal and all reviewers lean on the positive side. Hence, the AC would like to recommend for acceptance.