Rethinking Audio-Visual Adversarial Vulnerability from Temporal and Modality Perspectives

Authors: Zeliang Zhang, Susan Liang, Daiki Shimada, Chenliang Xu
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: audio-visual learning; adversarial attack

Abstract:
While audio-visual learning equips models with a richer understanding of the real world by leveraging multiple sensory modalities, this integration also introduces new vulnerabilities to adversarial attacks.
 In this paper, we present a comprehensive study of the adversarial robustness of audio-visual models, considering both temporal and modality-specific vulnerabilities. We propose two powerful adversarial attacks: 1) a temporal invariance attack that exploits the inherent temporal redundancy across consecutive time segments and 2) a modality misalignment attack that introduces incongruence between the audio and visual modalities. These attacks are designed to thoroughly assess the robustness of audio-visual models against diverse threats. Furthermore, to defend against such attacks, we introduce a novel audio-visual adversarial training framework. This framework addresses key challenges in vanilla adversarial training by incorporating efficient adversarial perturbation crafting tailored to multi-modal data and an adversarial curriculum strategy. Extensive experiments in the Kinetics-Sounds dataset demonstrate that our proposed temporal and modality-based attacks in degrading model performance can achieve state-of-the-art performance, while our adversarial training defense largely improves the adversarial robustness as well as the adversarial training efficiency.

Decision: accept (poster)
Reviewer scores: 6, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1729592922847

thread_type:
official_review

<thread_object>
creator: reviewer_rucn

<content>
summary:
This paper propose two adversarial attack methods for audio-visual data. Audio-visual data has two main characteristics: the temporal consistency (i.e., between nearby time step, the data is similar) and the modality alignment (i.e., audio and visual data has alignment). The attack exploits these two main characteristics: temporal invariance-based attack and modality misalignment-based attack. In addition, this paper propose the defense mechanism that has lower computational cost in comparison to the naive adversarial training. With lower computational cost, the model can be trained with various strength of the attack, improving its robustness. The way to do lower the computational cost is by only attacking sampled frames instead of attacking all frames. Another additional way is to use curriculum learning by masking out the data and the model's weight from low rate (easy) to high rate (hard). The number of steps to generate attacks also go from low steps (easy) to more steps (hard).

strengths:
1. The method of attacking audio-visual data based on the temporal consistency and modality alignment seems to be original.

2. The empirical experiments in Section 3 as motivation is interesting to prove how breaking the temporal consistency and modality alignment (by masking out the data) reduce the audio-visual model performance and increase the attack success rate.

weaknesses:
Several unclear information. Please see Questions.

questions:
1. What is the meaning of (a)synchronous in context of Line 138?

2. Is "contact" in line 411 and 415 is "concat"?

3. Line 366: Is the masking done synchronously or asynchronously?

4. Figure 3. What does it mean minimizing prediction $p_{av}$, $p_a * p_v$ ?

5. Based on Eq. (4), loss in Eq. (2) will be minimized. However, if we want to exploit the temporal consistency (line 52), why not maximizing this Eq. (2) loss? I think, maximizing variation of the extracted features between time step would make the temporal more inconsistent, which can make the attack more efficient.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730654614852

thread_type:
official_review

<thread_object>
creator: reviewer_ntiz

<content>
summary:
The paper presents two powerful adversarial attacks: 1) a temporal invariance attack that exploits the inherent temporal redundancy across consecutive time segments and 2) a modality misalignment attack that introduces incongruence between the audio and visual modalities. To defend against such attacks, the paper introduces a novel audio-visual adversarial training framework. Extensive experiments illustrate the effectiveness of the proposed method.

strengths:
The paper exhibits two powerful adversarial attacks: a temporal invariance attack and a modality misalignment attack. These attacks show their ability to obtain the robustness of audio-visual models.

weaknesses:
The temporal invariance attack is realized by setting consecutive time segments. The modality misalignment attack introduces incongruence between the audio and visual modalities. Regarding the novelty of the two methods, it seems fair. It is encouraging that two attacks achieve SOTA performance.

questions:
1. In experiment comparison, some figures use TMA to represent the proposed method, while some of them use Ours. It is suggested to use the same. 
2. For Table 2, does the ablation study use the proposed method TMA? If yes, it is suggested to add the information in the caption.
3. According to Table 2, the successful defense rate continuously increases with the addition of sampling ratio. Where is the change point? The experiment of the ablation study needs to show the sampling ratio value from where the result performance will drop down.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730679187403

thread_type:
official_review

<thread_object>
creator: reviewer_eurp

<content>
summary:
This paper introduces two novel audio-visual adversarial attacks, the Temporal Invariance-based Attack (TIA) and the Modality Misalignment-based Attack (MMA), along with an adversarial training framework that enhances robustness and efficiency. The experiments demonstrate the effectiveness of these methods in benchmarking audio-visual model robustness and improving both adversarial robustness and training efficiency.

strengths:
(1) The paper proposes innovative adversarial attacks tailored to audio-visual models, specifically the Temporal Invariance based Attack (TIA) and the Modality Misalignment-based Attack (MMA). These proposed  attacks leverage the unique properties of audio-visual data, such as temporal redundancy and intermodal correlation.

(2) The paper introduces an audio-visual adversarial curriculum training framework that incorporates efficient adversarial perturbation crafting and an adversarial curriculum strategy, which achieves a balance between robustness and efficiency.

weaknesses:
(1) The experiments related to the proposed method are not sufficiently comprehensive.    In Section 5.3, Adversarial Curriculum Training is introduced, which includes two strategies: the Data-level strategy and the Model-level strategy. The core of Adversarial Curriculum Training involves the iterative adjustment of the data masking ratio ( ) and the model dropout ratio ( ). However, the paper does not present corresponding experimental results to substantiate the significance of this iterative design.    Therefore, it is recommended to add experimental results to support the validity of this design.
(2) The paper evaluates unimodal adversarial attack methods (such as FGSM, I-FGSM, and MI-FGSM in the image domain). However, studies such as [1,2] have proposed multimodal attacks. It remains unclear whether the proposed defense method is effective against multimodal adversarial attacks similar to the Audio-Visual attacks discussed in [1].    I suggest adding experiments to evaluate the method's performance under multimodal attack scenarios.

[1] Zhang J, Yi Q, Sang J. Towards adversarial attack on vision-language pre-training models[C]//Proceedings of the 30th ACM International Conference on Multimedia. 2022: 5005-5013. 
[2] Lu D, Wang Z, Wang T, et al. Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models[J]. arXiv preprint arXiv:2307.14061, 2023.

questions:
Please refer to the questions raised in the Weakness section. Additionally, the reviewer is interested in the following questions:
(1) What's the selection criterion when choosing datasets for the experiment?
(2) Could the proposed method be incorporated into the pre-trained multi-modal models, such as CLIP or BLIP?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730811606736

thread_type:
official_review

<thread_object>
creator: reviewer_7zex

<content>
summary:
This paper studies audio-visual adversarial vulnerability, and proposes two kinds of adversarial attacks which deals with the temporal consistency and intermodal correlation. Furthermore, a new audio-visual adversarial training framework is also designed to defend against above attacks.

strengths:
1. The two kinds of proposed adversarial attacks are novel and important for the topic of audio-visual adversarial vulnerability.
2. The experiments shows the effectiveness of the proposed attacks.

weaknesses:
1. The proposed defending method seems not designed exactly towards the proposed attacks. In particular, the modality-misalignment-based attack is not specifically addressed.

questions:
Maybe it is better to show the effectiveness of the method by some real audio-visual samples, i.e., applying the proposed attacks can indeed attack existing methods while the proposed defending method can handle/alleviate it.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732243700199

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
**To all reviewers:**   
We sincerely thank all reviewers for their valuable feedback and for recognizing the merits of our work:  
1. We propose novel methods for audio-visual adversarial attack/defense, leveraging the unique characteristics of audio-visual learning (7ZEx, eurp, RUCN).  
2. The evaluation experiments demonstrate strong consistency, and the results validate the effectiveness of our approach (7ZEx, nTiz, RUCN).  

In our revision, we have made the following updates:  

- **Expanded Application on attacking the MLLM**: We include a demonstration of our proposed Temporal and Modality-based Attack (TMA) on the multi-modal large language model, VideoLLaMA2, achieving a high attack success rate. Details are provided in Appendix D.  
- **Enhanced Discussion**: We add a discussion on the relationships between various multi-modal adversarial attacks in Appendix C, aiming to inspire further research on robustness in multi-modal domains.  
- **Additional Ablation Studies**: We include more ablation studies in Table 2 and Appendix B.  
- **Improved Clarity**: We revise both illustration figures and their captions for better clarity.  
- **Typos Corrected**: We fix all identified typographical errors.  

We hope these updates address your concerns and further strengthen the contributions of our work. Thank you again for your thoughtful reviews and support!
</content>
</thread_object>

</thread>
<thread>
creation date:
1732641573021

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Kind reminder for discussion

comment:
Dear Reviewers,

We sincerely appreciate your time and effort in reviewing our paper and providing valuable suggestions, which have greatly helped us improve our work.

As the deadline for the paper revision period approaches, we wanted to check if we have adequately addressed your concerns and made necessary revisions to our paper. If you have any remaining questions or feedback, please feel free to share them with us. We are deeply grateful for this opportunity to exchange ideas with you and greatly value your insights, which have been instrumental in enhancing our work.

Thank you once again for your time and thoughtful feedback!

Best regards,  
Authors
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper is above the threshold for acceptance due to its novel contributions to audio-visual adversarial vulnerability, including the Temporal Invariance Attack (TIA) and Modality Misalignment Attack (MMA), which leverage unique properties of audio-visual models and achieve SOTA performance. The proposed adversarial curriculum training framework, while needing more comprehensive evaluation, introduces innovative strategies for balancing robustness and efficiency. Strong experimental results and impactful insights into model vulnerabilities outweigh minor concerns, making this work a significant contribution to the field.