Cross-Modal Safety Mechanism Transfer in Large Vision-Language Models

Authors: Shicheng Xu, Liang Pang, Yunchang Zhu, Huawei Shen, Xueqi Cheng
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Vision-language alignment, Safety of LVLMs, Toxic Content

Abstract:
Vision-language alignment in Large Vision-Language Models (LVLMs) successfully enables LLMs to understand visual input. However, we find that existing vision-language alignment methods fail to transfer the existing safety mechanism for text in LLMs to vision, which leads to vulnerabilities in toxic image. To explore the cause of this problem, we give the insightful explanation of where and how the safety mechanism of LVLMs operates and conduct comparative analysis between text and vision. We find that the hidden states at the specific transformer layers play a crucial role in the successful activation of safety mechanism, while the vision-language alignment at hidden states level in current methods is insufficient. This results in a semantic shift for input images compared to text in hidden states, therefore misleads the safety mechanism. To address this, we propose a novel Text-Guided vision-language Alignment method (TGA) for LVLMs. TGA retrieves the texts related to input vision and uses them to guide the projection of vision into the hidden states space in LLMs. Experiments show that \textbf{TGA} not only successfully transfers the safety mechanism for text in basic LLMs to vision in vision-language alignment for LVLMs without any safety fine-tuning on the visual modality but also maintains the general performance on various vision tasks (Safe and Good). Code is in supplemental material and will be released on GitHub after acceptance.

Decision: accept (poster)
Reviewer scores: 6, 6, 8, 6

Reviews and discussion:
<thread>
creation date:
1730634773751

thread_type:
official_review

<thread_object>
creator: reviewer_r4pv

<content>
summary:
This paper aims to find why LVLMs suffer from toxic visual content when converting LLMs to LVLMs. They observe and analyze the safety activation mechanism in the transformers and develop specific methods, TGA, to alleviate the issue of LVLMS without any post alignment.

strengths:
- Clear evidence for the safety activation mechanism.

- Straightforward and well-motivated methods. 

- TGA performs relatively well without any post-alignment steps.

weaknesses:
- 1. The analysis seems only work with the model developed by [1]. If the aligned models change, will the conclusion remain consistent?

- 2. Lack of analysis about the extra cost.


[1] Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Rottger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. Safety-tuned LLaMAs: Lessons from improving the safety of large language models that follow instructions. 2024.

questions:
- For point 1 in weaknesses, if the alignment method change, or, the key words change, not "sorry"/"apologize", will the activation layers in  Figure 2 change?

- For point 1 in weaknesses, how about the change of activation layers if we do not fully fine-tune all parameters of the model? For example, use PEFT for the pre-trained LLMs or just frozen the pre-trained LLMs. In such cases, will the trained LVLMs still suffer from toxic visual content?
    - If so, will the activation layers remain the same?
    - If not, the reviewer thinks the conclusion only holds for the fully fine-tuning case.

- In Table 2, how about the performance of a safety-aligned LVLMs like that in [1]?


- Point 2 in weaknesses, the reviewer thinks analysis about the extra cost is needed.

[1] Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Rottger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. Safety-tuned LLaMAs: Lessons from improving the safety of large language models that follow instructions. 2024.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730698392017

thread_type:
official_review

<thread_object>
creator: reviewer_cmbn

<content>
summary:
Brief Summary: The paper proposes an interesting observation that the safety mechanism of LLMs in filtering out inappropriate content while answering questions is lost when transferring to VLMs naively. As a result, the VLM might answer about things given the image context even though the corresponding LLM wouldn't have. 

The authors identify that specific hidden layers in Transformers are responsible for this behavior and propose a method called TGA to transfer this mechanism from LLMs to VLMs. 

Experiments on multiple benchmarks (like POPE, MMVet) show that the proposed method maintains performance while filtering our inappropriate content.

strengths:
Pros:

1. The paper tackles an interesting problem which (to best of my knowledge) isn't very well known in the community. As such, it highlights a potential gap and suggests how to fix new VLMs. 

2. The motivation is a bit subtle and it is important to note is mostly relevant for open-source models. In a closed sourced model, one could simply have a nsfw classifier on the image-input. However, for open-source model, such an additional component can be easily turned off. As such, a method to have open-source models which are safe is very important. In that sense, the problem is very well motivated.

3. As part of the experiments, the authors collect new dataset which is always appreciated. The authors further provide qualitative visualizations in appendix.

4. The idea of aligning the hidden states is quite clever in my opinion. 

5. The authors compare against multiple baseilnes.

weaknesses:
Cons:

1. One thing that isn't clear to me is if it is possible to reverse the trained safety filter by doing an instruction tuning on a sample of toxic dataset by an end user. In that case, it would be easy to "jailbreak" the safe model with relative ease. 

2. The authors should include a baseline which works as a direct filter on the image itself to get an upper bound estimate.

questions:
Q1. Confused about the number of toxic image-text pairs, in L134 it notes 2031 but in L454, it notes 20531.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730716450488

thread_type:
official_review

<thread_object>
creator: reviewer_vfzw

<content>
summary:
The paper identifies a vulnerability in current vision-language alignment methods for Large Vision-Language Models (LVLMs), where the safety mechanisms effective for text fail to transfer to visual input, leaving toxic images unchecked. The authors find that misalignment at the specific hidden state layers cause a semantic shift, undermining the safety mechanism for visual input. To address this, they propose a Text-Guided Alignment (TGA) method, which uses related text to guide the projection of visual inputs into hidden states. Experiments demonstrate that TGA successfully transfers text safety mechanisms to vision without additional fine-tuning and maintains overall performance on vision tasks.

strengths:
This paper is well-motivated and provides a thorough analysis of layer activations to explain the safety misalignment between vision and language. The work has potential value across multiple related fields, particularly in the design of vision-language models and their safety challenges.

The method for identifying the layers where the safety mechanism is activated is both reasonable and straightforward, showing effectiveness with a simple approach.

The proposed TGA alignment method effectively defends against toxic images, with strong evidence presented in Figure 7 to substantiate this claim.

weaknesses:
The paper lacks comparisons with other defense methods. Aside from the comparison with the unlearn-FigS defense, the current experimental results are mainly contrasted with the original model. Including comparisons with existing safety defense methods, such as [1-2], would provide stronger evidence of the proposed approach's superiority.

The presentation is somewhat redundant. For instance, the content in Figures 2 and 4, as well as Figures 3 and 5, could be combined to avoid repetition. Similarly, the writing in Section 4 could be more concise and streamlined for better clarity and flow.

[1] Tovilag: Your Visual-Language Generative Model is Also an Evildoer. EMNLP2023.

[2] Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation. ECCV2024.

questions:
Please see weakness.

1. In Figure 1, the presentation is somewhat confusing. Specifically, in Figure 1c, could you clarify whether the blue arrows represent "safe" or "unsafe"?

2. In Section 4, could you specify which layers you are analyzing? For example, are you focusing on the qkv (query, key, value) layers or the projection layers?

3. Can you include any discussion of failure scenarios or bad cases where the method may not perform as expected?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730827677844

thread_type:
official_review

<thread_object>
creator: reviewer_evhd

<content>
summary:
This paper introduces the concept of Cross-Modal Safety Mechanism Transfer for Large Vision-Language Models (LVLMs), aiming to transfer the safety mechanism from text to vision without additional visual safety fine-tuning. The current vision-language alignment fails to align vision with text at the hidden states level, leading to unsafe responses for harmful images. The proposed Text-Guided vision-language Alignment (TGA) retrieves relevant texts to guide the alignment of vision input to hidden states in LVLMs. TGA effectively transfers safety mechanisms from text to vision, maintaining safety without compromising general performance in vision tasks, outperforming existing vision-language alignment methods.

strengths:
1. The authors first analyze cause of failure in cross-modal safety transfer. Based on the analysis, they propose Text-Guided Alignment (TGA) to transfer safety mechanisms from text to vision, addressing key safety issues in LVLMs. The analysis is thorough and the proposed method is novel in general.
2. The paper is well-structured, with clear motivations and systematic explanations of the issues with current vision-language alignment methods.
3. The proposed approach contributes to improving the robustness of LVLMs. This advancement could be important in bridging safety gaps in multimodal AI.

weaknesses:
1. TGA relies on captions generated by LLaVA-1.5-13B for effective alignment. Inaccurate captions can lead to misalignment between vision and language representations, reducing safety performance. Evaluating the impact of captioning errors and exploring mitigation strategies could add robustness to the approach.
2. The paper does not adequately show how the model handles unsafe compositional inputs. For instance, an image of a wine bottle combined with text like "teach a kid to buy this" represents a harmful query, even though the image and text are safe individually. Evaluating compositional risks more deeply could strengthen safety measures.
3. The paper does not show the model's robustness against red-teaming methods such as jailbreak attacks. Evaluating how effective the proposed approach is in defending against these attacks would provide more confidence in the model’s safety capabilities in adversarial settings.

questions:
Same as weakness.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper explores to transferring the safety mechanism in existing LLM to vision without additional visual safety fine-tuning. The authors first find that the existing vision-language alignment struggles to work at the hidden states level, which results in misleading the safety mechanism in LLM. They proposed the Text-Guided vision-language Alignment method (TGA) for VLMs, which retrieves the texts related to input vision and uses them to guide the projection of vision into the hidden states space in LLMs. Experiments show that the TGA approach not only successfully transfers the safety mechanism in LLM to vision, but also maintains the general performance on several vision tasks.

The reviewers raised weaknesses about insufficient experiments and analysis. 
The rebuttal from the authors well addressed the concerns. 
The reviewers keep the initial positive scores after rebuttal.

Overall, the paper has a clear motivation, and presents an effective method, advancing the techniques in building safe LVM.
I recommend to accept.