Rethinking Audio-Visual Adversarial Vulnerability from Temporal and Modality Perspectives
Authors: Zeliang Zhang
Susan Liang
Daiki Shimada
Chenliang Xu
Venue: ICLR 2025 Poster (2025)
Keywords: audio-visual learning; adversarial attack
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: While audio-visual learning equips models with a richer understanding of the real world by leveraging multiple sensory modalities, this integration also introduces new vulnerabilities to adversarial attacks.
 In this paper, we present a comprehensive study of the adversarial robustness of audio-visual models, considering both temporal and modality-specific vulnerabilities. We propose two powerful adversarial attacks: 1) a temporal invariance attack that exploits the inherent temporal redundancy across consecutive time segments and 2) a modality misalignment attack that introduces incongruence between the audio and visual modalities. These attacks are designed to thoroughly assess the robustness of audio-visual models against diverse threats. Furthermore, to defend against such attacks, we introduce a novel audio-visual adversarial training framework. This framework addresses key challenges in vanilla adversarial training by incorporating efficient adversarial perturbation crafting tailored to multi-modal data and an adversarial curriculum strategy. Extensive experiments in the Kinetics-Sounds dataset demonstrate that our proposed temporal and modality-based attacks in degrading model performance can achieve state-of-the-art performance, while our adversarial training defense largely improves the adversarial robustness as well as the adversarial training efficiency.

Published as a conference paper at ICLR 2025
RETHINKING AUDIO-VISUAL ADVERSARIAL VUL- NERABILITY FROM TEMPORAL AND MODALITY PER- SPECTIVES
"Zeliang Zhang', "Susan Liang', Daiki Shimada Chenliang Xu² University of Rochester
2Sony Group Corporation
{zeliang. zhang, susan. liing, chenliang.xuj Daiki, Soimada@sonyy com
hrr.aaiaang.eeeooceeeteeee... rochester.eeseeeeoo
ABSTRACT
While audio-visual learning equips models with a richer understanding of the real world by leveraging multiple sensory modalities, this integration also introduces new vulnerabilitics to adversarial attacks. In this paper, we present a comprehen- sive study of the adversarial robustness of audio-visual models, considering both temporal and modality-specific vulnerabilities. We propose two powerful adver- sarial attacks: temporal invariance attack that exploits the inherent temporal redundancy across consecutive time segments and 2) modality misalignment at- tack that introduces incongruence between the audio and visual modalities. These attacks are designed to thoroughly assess the robustness of audio-visual models against diverse threats. Furthermore, to defend against such attacks, we introduce a novel audio-visual adversarial training framework. This framework addresses key challenges in vanilla adversarial training by incorporating efficient adversar- ial perturbation crafting tailored to multi-modal data and an adversarial curricu- lum strategy. Extensive experiments in the Kinetics-Sounde dataset demonstrate that our proposed temporal and modality-based attacks in degrading model per- formance can achieve state-of-the art performance, while our adversarial train- ing defense largely improves the adversarial robustness well as the adversarial training efficiency.
INTRODUCTION
Audio-visual models, capable of integrating both auditory and visual information, have gained sig- nificant traction in recent years due to their ability to create comprehensive understanding of the
surrounding world (Li et al. 2022a Song et al. 2023 2024a b). These models have demonstrated remarkable success in wide range of applications, inc including multimedia analysis (Dimoulas) 2016),
human-computer interaction (Zhen et al. 2023), and autonomous systems (Guo et al.) 2023). How- ever, critical challenge lies in their susceptibility to adversarial attacks (Zhang al.) 2024b a
Yao et al.] 2024). These attacks can craft imperceptible perturbations to the input data, causing audio-visual models make erroneous predictions (Wang et al.) (022)) or interpretations (Han al., 2023 Such errors can have disastrous consequences, especially in safety-critical domains like auto- driving (Kloukiniotis al. 2022) and identity verification (Zhang al. 2021).
While prior work has investigated the adversarial robustness of audio- visual models (Tian & Xu 2021 Yang et al.] 2021 Li et al., 2022b), they primarily rely on general adversarial attack methods, such as FGSM (Goodfellow et al. 2015) and I-FGSM (Kurakin et al.) (017)), originally designed for the single-modality data. These methods are simply adapted to the audio-visual domain without fully capitalizing its unique characteristics. A key limitation of such approaches lies in their inability to consider the inherent properties of audio-visual data. Unlike single modality, audio-visual data possesses two erucial aspects: temporal consistency and intermodal correlation. For instance, in video of a dog barking, we see and hear the barking event unfold over time, not just in a single frame. These properties play vital role in human perception of the real world (Sun et al. 2022
"Equal contribution. Listed order random.

---

Published as a conference paper at ICLR 2025
Yang et al. 2023a). However, current attack methods fail to exploit them, potentially limiting their effectiveness. Conversely, by leveraging these characteristics, we can craft more potent attacks and develop improved robust learning strategies specifically tailored for audio-visual models.
In this work, we rethink the adversarial vulnerability of audio-visual models through the lenses of temporal and modality perspectives. We begin with empirical analysis assess the vulnerability of existing models. Our case study experiments reveal several key findings, including the presence of adversarial transferability within the audio-visual domain, and the significant impact of temporal consistency and modality correlations on model robustness. Leveraging these insights, we propose two novel adversarial attacks tailored the unique properties of multi-modal data: 1) the temporal invariance attack, which targets robust and temporally consistent audio- features by introduc- ing inconsistencies across consecutive frames, and 2) the modality misalignment attack, which crafts adversarial examples by inducing incongruencies between the audio and visual streams.
To mitigate the vulnerabilities exposed by these dedicated attacks, we propose a novel audio-visul adversarial training framework that serves as robust defense mechanism. Our framework addresses critical challenges in robust multi-modal learning by incorporating efficient adversarial perturbation crafting techniques along with an adversarial curriculum training strategy. The proposed defense aims to significantly improve the robustness of audio-visual models against adversarial attacks with minimal impact on training efficiency.
Our contributions can be summarized as follows:
1. We first identify the existence of adversarial transferability in audio- visual learning, and introduce two powerful adversarial attacks, namely the Temporal Invariance- based Attack (TIA) and the Modality Misaligmment based Attack (MMA), to evaluate the adversarial robustness of audio-visual models comprehensively.
2. We propose efficient adversarial perturbation crafting and adversarial curriculum training aimed at enhancing both the robustness and efficiency of audio-visur visual models.
3. We validate the effectiveness of both our proposed attacks and defense mechanisms through extensive experiments conducted on the widely-used Kinetics-Sounds dataset.
2 RELATED WORK
2.1
AUDIO-VISUAL LEARNING
The field of audio-visual learning encompasses wide range tasks, including audio-visure event recognition (Brousmiche et al. 2019 Xia & Zhao 2022 Brousmiche et al. 2022), separation (Wu
al., 2019a] Majumder et al. 2021 Majumder & Grauman, 2022] Huang al. 2023a), localiza-
tion (Wu et al. 2019b] 2021; Huang al. (023b), correspondence learning (Min et al. 2020 Zhu
ctt al. 2022] Morgado et al. 2021), I). representation learning (Zhou et al. 2019; Cheng et al. 2020
generation (Chen al. [201]] 7 Hao al. 2011] [ung Bin ct 2023) Liang et al. 2023a 2024 Huang et al. 2002 Liang et al. 2023b). Among these, audio-
Rahman al. 2021), and cross-
visual event recognition stands out as fundamental task (Gao al. 2024) that has attracted signif-
icant research attention, particularly regarding robustness and security issues (Yang et al. 2023b).
Deep learning models employed in audio-visual event recognition typically comprise three main components: visual encoder, audio encoder, and fusion layer. Although prior research has exten- sively focused on optimizing these components to enhance task performance, there has been limited consideration of their implications for security and robustness.
In this work, we delve into the individual components of these models and examine their respective impacts on robustness, shedding light on crucial but often overlooked aspects.
2.2 ADVERSARIAL ATTACK & DEFENSE
Research efforts in adversarial robustness for audio-visual models have been relatively limited. Tian et al. (Tian & Xu, 2021) were among the first explore the potential of audio audio-visual visual integration in enhancing robustness against multi-modal attacks. Yang er al. (Y.n et al., (02T)) proposed an adversarially robust audio-visual fusion layer defend against single-source adversarial attacks.
2

---

Published as a conference paper at ICLR 2025
Li et al. (Li et al. 2022b) introduced novel mi strategy in the audio-visual fusion layer to improve the robustness of audio-visual models. Yang et al. (Yang et al. 2023b) proposed a certified robust training method boost the multi multi-modal robustness. However, they primarily focused on adapting single-modality adversarial attacks to audio-visual scenes. There remains a critical need for powerful audio- visual adversarial attacks that serve as benchmarks for evaluating the adversarial robustness of audio-visual methods and the effectivvnnes of robust training techniques.
In this work, we address the gap by designing an effective audio-visual adversarial attack method that facilitates more comprehensive assessment of model robustness. We further propose an efficient defense technique enhance the robustness of audio-visual models against adversarial attacks.
EMPIRICAL ROBUSTNESS ANALYSIS OF AUDIO-VISUA MODELS
Notations. Given an audio clip Ia and video frames Tus we use an audio network fa (2a; to extract audio features, visual network fo ( 0₂) to extract visual features and a fusion network fu for modality integration. We denote the complete audio-visual network with F(r,, Ta; = (x: 0,). fa (a)); ;;; fa), where A (6., 0a, 0., are the overall network parameters.
Temporal consistency and modality correlation are two fundamental characteristics of audio-visual learning. On the one hand, these characteristies provide the robustness and generalization capa- bilities to audio-visual models. The temporal consistency in audio-visual data reinforces learning across consecutive frames, while the cross-modal correlations between auditory and visual signals offer mutually complementary information. These properties enable audio-visual models to learn robust and reliable representations. However, the temporal and cross-modal dependency also para- doxically create new vulnerabilities. Unlike conventional attack methods, audio-visual adversarial attacks can exploit these relationships cause inconsistencies within the model, leading to errors. This duality underscores the critical need to understand and address adversarial robustness in audio- visual models from both temporal and modality perspectives.
Corruption Robustness. Here, we provide experiments to support our arguments. We train an audio-visual model on the Kinetics-Sounde dataset (Arandjelovic & Zisserman, 2017). which takes the VGG as the vision encoder, the AlexNet as the audio encoder, and the operation as the fusion operation followed by the decision layer. We randomly mask a ratio of the audio-visual data along the temporal dimension, where 0% < p 30%, and evaluate the model performance the masked audio-visual data. For a comprehensive understanding of how different modalities affect the audio-visual model's decision, perturbing the audio only (Ax), and perturbing both the audio and visual synchronously (Ax@Vx) and asynchronously (Ax@V Vx)..
set up three groups, namely perturbing the video only (Vx),
90
Clesfiiction asquracy (%)
42
70
38
60
Altuek
36
50
34
32
V
18
27
30
21
24
The ratio of andio/visual to be masked (76)
The rutio of andio/visual be masked (%)
Figure 1: Classification accuracy of the model when masking the audio and visual data with ratio p.
Figure 2: Average black-box attack success rate (%) against 7 black-box models.
We have three observations from the results shown in fig. First, the audio-visua data is relatively redundant in temporal information. By masking the audio-visual data with certain ranges, e.g., p 20%, the model remains least 50% classification success rate, indicating robustness against
3

---

Published as a conference paper at ICLR 2025
the temporal perturbation. Second, the model heavily relies the visual modality to make the deci- sion, leaving the audio modality less attention. With the same ratio to be masked, the degradation of model performance caused by perturbing the visual data is significantly greater than perturbing the audio data only with a clear margin of 12.3%. Third, the correlations between different modal- ities increase the performance of audio-visual learning. Compared asynchronous perturbation of audio and visual data, synchronous perturbation causes relatively larger performance degradation, indicating the complementary of two modalities in helping the model make decisions.
Adversarial Robustness. For given audio-visual input (Ia,T.) with ground-truth y, the goal of the adversarial attack is crafting perturbation 6, (audio) and ò, (visual) deceive into making a wrong prediction. This process is formulated as follows.
max (( + pu, Ta + 0).
s.t. l/oellp S Cu. ||ōallp S Ca.
where C is an arbitrary loss function, ||]p is the p-norm, and c, and ca are the adversarial perturbation budgets for visual and audio modalities, respectively.
For further study of the impact of temporal consistency and modality correlation on the adversar- ial robustness, we additionally train various audio-visual models the Kinetics-Sounds dataset to evaluate the adversarial attack performance. We, respectively, use the VGG and ResNet as the vi- sion backbone, the AlexNet and ResNet the audio backbone, and the sum and concat the fusion layer, in a total of 8 models. We set the model with VGG as the vision backbone, AlexNet as the audio backbone, and concat as the fusion layer, as the surrogate model to generate adversarial exam- ples by FGSM (Goodfellow et al. 2015) under the white-box setting, which is up to 78.3% attack success rate. Then, we set the other models as black-box models further evaluate adversarial transferability.
As shown in fig. we can find that adversarial transferability between different models also exists in the audio-visual data. Without any precomputed masking operation, the generated audio-visun visual adversarial examples have an average attack success rate of 31.7% against the selected black-box models. By masking different modalities along the temporal dimension adequately with ratio to generate multiple copies for gradient calculation. the adversarial transferability can be boosted. Also, high setting of p causes a loss of information, hindering the quality of generated adversarial examples and degrading the adversarial transferability. Specifically, it can bring up to an improve- ment of 3.8% by only masking the audio, 9.3% by only masking the video, and 13.8% by masking both the audio and visual modality. These results uncover that temporal redundancy in audio and visual data can be naturally used boost adversarial transferability. Besides, the compensation function from the modality correlation mitigates the impact of adversarial transferability with to 1.5%. comparing the synchronous and asynchronous perturbation.
Takeaways of our empirical study
(1) The adversarial transferability is also exhibited in audio-visual learning, posing security problems applications.
(2) Temporal consistency and redundancy bring robustness against corruption but also re- main potentially leveraged to improve adversarial transferability. (3) The inter-modal correlation compensates against temporal corruptions and alleviates the influence of adversarial perturbation.
AUDIO- VISUAL ADVERSARIAL ATTACK
Motivated by previous empirical robustness analysis temporal consistency and modality corre- lation, we propose two powerful audio-visual adversarial attacks, namely the temporal invariance- based attack and the modality misalignment-based attack.
4.1 TEMPORAL INVARIANCE-BASED ATTACK
Audio-visual data comprises temporally invariant features and time-varying information within each frame. Our goal is craft adversarial perturbations that target these invariant features, thereby
4

---

Published as a conference paper at ICLR 2025
fostering the transferability of adversarial instances. We achieve this by introducing a temporal regularization term that steers perturbations toward the most significant video features. With this novel regularization, ensure that the adversarial perturbations consistent and coherent across different frames.
Specifically, calculate the variation of extracted features along the temporal dimension structure- structure-unrelated unrelated statistic of feature consistency. By minimizing this variation for both audio and visual features, we encourage perturbations focus on temporally invariant characteristies.
Additionally, leveraging the inherent temporal dependency, we can diversify video inputs to encour- age the model to learn robust, invariant features. In practice, we apply different input transformations on temporal inputs, including scaling, masking, blurring, and mix-up on audio or visual modalities, independently or in parallel, synchronously asynchronously.
This temporal regularization can be expressed as
CR
Var
[B (Ja (Ta (Ia 5n); (1))
(2)
Var
(Eo (Tv (I. + do); (t)))
where we denote fa (Ita + 5a; (t) and fo 6z; (t) as the audio and visual features extracted at the f-th frame by the audio and visual networks, To and To as input transformation methods for audio and visual modalities, respectively. We consider the mean value of the feature each time step and compute the variance along the temporal axis.
4.2 MODALITY MISALIGNMENT BASED ATTACK
From the empirical study about the modality correlation in section especially the comparison between the synchronous and asynchronous perturbation, we notice that aligning semantic changes across modalities mitigate the influence of adversarial perturbations and diminish the adversarial transferability. Inspired by this, we propose a novel attack strategy that disrupts the strong semantic correlation between audio and visual modalities. We hypothesize that lower semantic correlation leads to higher adversarial audio-visual transferability.
We conduct experiments support this hypothesis. To quantify semantic information, ture vectors from both modalities and assess their alignment with cosine similarity. Following the experimental setting in section 3 (without masking operation). we compute cosine similarities be- tween feature vectors of adversarial examples generated by FGSM (Goodfellow et al. (2015). I- FGSM (Kurakin al., 2017), MI-FGSM (Dong et al. 2018), and NI-FGSM (Lin al.) 2019). which exhibit progressively stronger attack performance under the white-box setting.
fea-
From the results shown in fig. 3 we can see that as the attack success rate against black-box models (i.e., ad- versarial transferability) increases, the cosine similar- ity between audio and visual feature vectors decreases. This correlation reinforces our claim that disrupting se- mantic alignment between modalities enhances trans- ferability.
120
Thus, to enhance the transferability of adversarial at- tacks, we propose an approach that aims to misalign the semantic correspondence between audio and visual features. Specifically, during each iteration of the ad- versarial attack, we minimize the feature similarity be- tween the modalities, ensuring that the perturbations disrupt the semantic alignment at the modality level,
FROM
1FOGH
MIFOSM
1I M--000M
Figure 3: The average black-box attack success rate (A.S.R.) and cosine similar- ity (C.S.) of the oudio-visual adversarial examples generated by different attacks.
[a 0a) fa (x On) LM [Ia - 5a;0a) 0a) folx + 5o;6,)l2z
where fz, + 0,) and (x, + 0,;0,) are the feature vectors encoded by the audio and visual backbones, respectively.
5

---

Published as a conference paper at ICLR 2025
Segment data rendomlly
bu
Sample
Miniminntaa Vert/...) verlf..))
Subject (Rage (huJer,
Divenafy
k
Aeu
Diowaied
Pa
Figure 4: Overview the adversarial perturbation crafting in the adversarial training process. Given an audio-visual data, randomly segment it into different parts (green and yellow) and sample frames from each of the segments with ratio a (red). Then, we diversify each sampled frame by Ny copies and employ TIA and MMA to craft the adversarial perturbation. Finally, map the generated adversarial perturbation to corresponding segments, creating adversarial examples for adversarial training.
4.3 ATTACK INTEGRATION
The temporal invariance-based and modality misalignment- based attacks be integrated together with the classification loss to achieve strong adversarial audiovisual attacks. At each iteration, we optimize the following loss function to conduct the attack,
max Las (F,(((T(I,) + 8e, Ta + 8a); 0), yy
LiT + So, + 6a); 0)) >2 Sa va + 0a);0)
s.t. ||o <:: 11ōallp S ca.
where Cets is the classification loss function, T is the input transformation on audio-visual data, and A and A2 are two coefficients to balance losses.
5 AUDIO-VISUAL ADVERSARIAL TRAINING
Having established the powerful audio-visual adversarial attacks, a question naturally arises: How we defend these attacks efficiently? In this section, start with the preliminary adversarial training method, followed by an in-depth analysis of adversarial perturbations in audio-visual data, and propose several strategies to fortify audio-visual models against such attacks.
5.1 PRELIMINARY METHOD
Adversarial training is one of the most powerful robust training paradigms to defend adversarial examples, which trains the model on adversarial examples and can be formalized in the audio-visse visual context as follows,
min max L.ss (F(x, + 6,, Fa + 5.), 0),
(5)
which optimizes model parameters to minimize the upper bound of the loss function.
It is challenging to directly optimize the min-max problem in (5) In practice, it is solved by training models on adversarial examples, in which adversarial examples are used to compute the values of the inner maximum loss function (Madry et al. 2018).
5.2 DISCUSSION ON AUDIO-VISUAL PERTURBATION
Audio-visual adversarial training presents unique challenge compared to its uni- miimooal counterpart (e.g., perturbating single image). Here, generating adversarial perturbations across multiple time

---

Published as a conference paper at ICLR 2025
steps in both audio and visual data leads to significant computational overhead. This hinders the use of strong attacks and advanced mitigation techniques during training. Thus, the key to solving the challenge is to reduce the computational cost for audio-visual adversarial example crafting.
As shown in section 3 random masking of audio-visual data slightly affects model performance yet boosts adversarial transferability. Additionally, recent studies (Kim et al., 2023 demonstrate that the universal adversarial perturbation of a single image model can be used to fool the video model. These findings indicate that effective strategy should reduce the computational overhead of adversarial training by generating temporal universal adversarial perturbations instead of frame- specific perturbations. However, strong attacks are crucial for effective adversarial training. To address this, we propose generating universal adversarial perturbations for continuous local regions in the video and audio data, rather than a single perturbation for all frames. We also employ a bag of tricks to amplify the impact of these generated perturbations on the model.
5.3 OUR APPROACH
Efficient adversarial perturbation crafting, Since audio-visual data exhibits strong temporal cor- relation, adversarial perturbations crafted from subset of time samples can be propagated to the remaining samples as well.
To exploit this property and achieve efficient perturbation crafting, we propose the following ap- proach: We first divide the entire audio-visual data into smaller segments. Within each segment, we randomly sample portion of audio and visual frames with selection ratio of p. We then generate universal adversarial perturbations upon the selected frames and propagate them to the remaining. The optimization of perturbations 8 and d₂ can expressed as follows,
ous 5, arg max 8 (2o Lu) (2to
E
[L + 8 fa + 80. v)],
(6)
where the (de, is sampled from (I,, za), L is the loss we proposed in eq. (4) which encounters both the temporal invariance-based and modality misalignment-baan attacks. The adversarial per- turbations generated 5, and 6, are shared with neighboring frames within the same segment. This design allows us to generate adversarial examples efficiently.
Adversarial curriculum training. Previous studies (Yu et al. 2022 Kim et al. 2021 Rice al., 2020) have identified that adversarial training easily gets overfitted to certain attack methods and settings, leaving the model vulnerable others. To address this issue and improve the generalization ability, we propose a randomized adversarial curriculum learning to optimize eq. (5). Concretely, our proposed randomized adversarial curriculum learning approach incorporates two strategies:
Data-level strategy. Temporal redundancy can be leveraged to control the impact of adver- sarial examples crafted in adversarial training, leaving the potential to alleviate overfitting. We propose to randomly sample Pa of the audio-visual input along the temporal dimension, where < Ps < 1. We cyclically vary the value of pa. to generate adversarial examples for curriculum learning.
Model-level strategy. The ovarmpaaratttiiiiatinnn of models can be exploited in adversarial training to boost the models' generalization. We randomly drop out the neurons of the fusion layer with ratio of py, where Ps < 1 and the dropout ratio Py is synchronized with Pz-
The curriculum learning approach hinges on a cyclical variation in the data-level masking ratio Pr. and the model-level dropout ratio py. This cyclic design offers an advantage: it gradually increases the training difficulty. During the initial stages of the cycle, both sampling and dropout ratios are lower and the number of steps to generate attacks is reduced. This translates to a simpler training task with fewer adversarial perturbations applied to the data. As both ratios and the number of steps gradually increase, complex adversarial examples task. With this cyclic design, the models are trained on diverse perturbations, thus fully exploring the loss landscape for the searching minimum and improving the robustness against attacks.
created, resulting in more difficult learning
7

---

Published as a conference paper at ICLR 2025
FOOM
LEOSM
NE-FOSM
PAM
TIA
MMA
IMA
20
SOARR
G0/ARR
106
106
168
160
10
91
60366
00/600
G0 Rok
So
Figure 5: Attack success rates (%) of eight deep models, where the adversarial examples are gen- erated on the white-box surrogate model and attack all models (one white-box model and seven black-box models). TIA, MMA, and TMA are our proposed attack methods.
6 PERFORMANCE EVALUATION
6.1 EXPERIMENTAL SETUP
Datasets. We the Kinetics-Sounds (Arandjelovic & Zisserman, 2017) for evaluation, which contains 1, 551, 610 second video clips in 27 human action categories. For model training, we split for training, validation, and testing. We also conduct experiments on
the dataset into 7 MIT-MUSIC (Zhao et al. 2018b) for further verification, which is provided in the appendix.
Models. The audio-visual model comprises three modules: the visual backbone, the audio back- bone, and the audio-visual fusion network. For the audio backbone, we select VGG and ResNet as candidates. For the visual backbone, we select AlexNet and ResNet as candidates. We can study the impact of model capacity on the adversarial transfcrability by this design. Following the previous work on audio-visual adversarial robustness (Tian Xu 202T). we selected the sum and concat operation as candidates for the fusion layer. There are 2 x 2 x 8 models in total. For simplicity, we use the format of "{ visual backbone }-{ fusion layer }-{ audio backbone }" to represent the audio-visual models, where the initials indicate each backbone and layer. We denote AlexNet as A, VGG as V, ResNet as R, sum operation as s, and concat operation as c. For example, ResNet as the visual backbone, the audio- visual model with AlexNet as the audio backbone, and the sum operation as the fusion layer can be represented by "RsA".
Baselines. For attack methods, use FGSM (Goodfellow et al, 2015), I-FGSM (Kurakin et al. 2017]. MI-FGSM (Dong et al. 2018), MIG (Ma et al.) 2023), and PAM (Zhang et al. (023) as baselines. We compare the baseline methods with proposed approaches, which encompass the temporal devariance-based attack (TIA), the modality misalignment attack (MMA), and the integra- tion of TIA and MAA attacks, namely the temporal and modality-based attack (TMA). For defense methods, we use the vanilla adversarial training (AT) (Madry et al.) 2018), discriminative and com- pact feature learning (DCFL) (Tian & Xu (202T), single-source defensive fusion (SSDF) (Yang al.) 2021), the mix-up strategy for adversarial defense (Mixup) (Li et al. 2022b), and the certified robust multi-modal training method (CRMT-AT) YYagg et al. 2023b).
6.2 SINGLE MODEL ATTACK
To validate the effectiveness of our proposed methods, we first compare ours with five popular attacks selected by previous audio-visual robustness research, namely FGSM, I-FGSM, MI-FGSM, MIG, and PAM. Among these methods, FGSM and I-FGSM are initially designed for white-box attacks, while others are designed for transferable adversarial attacks in the image domain. We generate adversarial examples success rates, i.e., and the misclassification rates of the victim model in the adversarial examples crafted are summarized in fig.
single model and test them the other models. The attack
8

---

Published as a conference paper at ICLR 2025
The figure shows that audio-visual adversarial examples crafted white-box models partially deceive black-bor box models, confirming adversarial transferability in visiall learning. However, unlike in the image domain, momentum doesn't 't always improve performance. For instance, when using VGG as the audio backbone and AlexNet with a sum fusion layer, FGSM achieves a 31.5% success rate, while MI-FGSM decreases performance by 2.3%, indicating momentum's is negative impact on audio-visual transferability. This issue is common with models using sum fusion layers. In contrast, models with concat fusion layers show better black-box performance (+3.6%). The results also suggest that improving input diversity with PAM helps momentum perform better over sum fusion layers. Our proposed methods (TIA, MMA, and TMA) consistently outperform others, with TIA exceeding PAM by 15.7% and further improving by 5.2% average. Combining TIA and MMA achieves a 95.2% success rate across eight models.
Figure 6: Attack success rates (%) of each of 8 deep mod- els under the ensemble set- ting. TIA, MMA, and TMA are our proposed attack meth- ods.
Figure Attack success rates (%) of each of 8 deep mod- els on the adversarial exam- ples crafted under the white- box setting with our proposed TMA method.
Figure 8: Ablation study of the number of attack itera- tions on the attack perfor- mance. TIA, MMA, and TMA are our proposed attack methods.
6.3 ENSEMBLE MODEL ATTACK
There have been many works studying using the ensemble uni-modal model to craft the adversarial example to improve the adversarial transferability (Dong et al. (018). Here, we conduct experiments to utilize the ensemble of multi-modal, i.e., the audio-visual models, to boost the performance. We iteratively select each one of the 8 models as the victim black-box model and use the remaining 7 models as surrogate models to craft the adversarial examples. We use the adversarial attack success rate against the victim model to evaluate the performance.
As shown in fig. 6 we can notice that the performances of all selected methods are significantly boosted by the ensemble strategy, even surpassing the performance under the white-box attacks. It indicates that different audio-vistal models also share similar areas of interest. By considering as many surrogate models as possible in attacks, rate. The previous result shows that the audio-visual model with concat operation as the fusion layer relatively robust against different attacks. Using the ensemble attacks, the FGSM has an average attack success rate of up to 79.9% against these models, indicating the dimming performance of the concat fusion layer on defense. It should be noted that all our proposed methods achieve an attack success rate of 100% on all victim models, sufficiently demonstrating the effectiveness.
fool the target victim model with a high success
6.4 ATTACKING DEFENSE MODELS
Our proposed attack methods, including TIA, MMA, and TMA, have achieved the best attack per- formance on eight normally trained audio-visual models with different backbones and fusion layers. Recent studies on audio-visual learning proposed to mitigate the threat of audio-visual adversarial examples. In our work, we also propose bag of novel tricks to enhance adversarial training for defense. Here, to validate the effectiveness of these defenses, as well as show the power of our pro- posed attack method against the defense mechanism, these eight audio-visual models. For defense methods, we select adversarial training (AT) (Madry et al. 2018), DCFL (Tian & Xu 2021), Mixup al. 2022b), CRMT-AT Yang et al. 2023b). and
conduct white-box adversarial attacks

---

Published as a conference paper at ICLR 2025
the combination of our tricks (Ours). We use strongest attack TMA evaluate the robustness. To align the attack setting, we use 10-step PGD adversarial training as the baseline.
The results are shown in fig. 7 Among the different defense methods, training the models on the adversarial examples is more efficient. We can see the adversarial training-basee methods, including AT, CRMT-AT, and ours, perform much better than non-adversarial training methods with clear gap of 10.9%. By integrating our proposed techniques, including the efficient adversarial perturbation crafting and curriculum adversarial training, our adversarial training can further boost the robustness by 2.28% on average versus the runner-up method CRMT-AT.
6.5 ABLATION STUDY
On the number of iterations for the attack. In experiments, we find that the multistep FGSM, L.., I-FGSM, cannot always beat FGSM under both white- and black-box settings. This raises the question of whether the number of iterations impacts the adversarial transferability in audio-visual learning. This motivates to do the ablation study of the number of iterations in attacks [ As shown in fig. [8 by generating the adversarial examples on the audio-visual model (AcV), we can that the number of iterations greatly impacts the attack performance. With increasing the num- ber of iterations, the adversarial transferability of I-FGSM is degraded. While the use of momentum (MI-FGSM) can alleviate the overfitting the surrogate model, a sufficiently large number of iter- ations still leads to getting stuck in local optimum, degrading the performance. The use of input transformations (PAM) can improve the input diversity for better capturing the robust feature, thus boosting the adversarial transferability, but still limited. Our proposed temporal invariant and modal- ity misalignment attack methods sufficiently help the optimization jump over the local optima, thus significantly boosting the performance.
On the sampling ratio for the adversarial train- ing. In our approaches, we propose to sample ra- tio of frames to generate the temporal-univee universal ad- versarial perturbation for efficient adversarial train-
85.0
50
82.5
40
ing. There is a balance between the time consump-
80.0
tion of adversarial training and the adversarial ro- bustness by selecting different sample ratios. Here, conduct an ablation study on the sampling ratio. As shown in fig. 9 with increasing the sampling ra-
Accuracy
77.5
30
Time (h)
75.0
72.5
20
tio in adversarial training, the defense performance
70.0
10
is improved, but it also leads to more time con- sumption, supporting our argument on the balance. At the same time, larger sampling ratio does not improve the adversarial robustness much but intro- duces more time consumption, and a smaller sam- pling ratio harms defense performance.
67.5
65.0
3
Scale factor for sampling
Figure 9: Ablation study of the sampling ra- tio on the performance of adversarial training. We use our proposed TMA to generate the ad- versarial example for adversarial training.
7 CONCLUSION
In this work, we developed two efficient audio-visual adversarial attack methods: the temporal invariance-baat based attack and the modality misalignment- based attack. We also introduced an adver- sarial training framework with strategies for efficient perturbation crafting and curriculum training to reduce time and improve robustness.
Results on the Kinetics-Sounds dataset show that our attacks effectively benchmark audio-visual model robustness and that our training framework improves both robustness and efficiency. Our experiments provided new insights, including the importance of temporal consistency, modality alignment, and the concat fusion layer's robustness. We hope this research will serve as a benchmark for audio-visual robustness and inspire further exploration of AT security in multi-modal data.
With the numher of iterations 1, the MI-FGSM and I-FGSM degrade the FGSM.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Relja Arandjelovic and Andrew Zisserman. Look, listen and learn. In ICCV, pp. 609-617, 2017.
Mathilde Brousmiche, Jean Rouat, and Stéphane Dupont. Audio-visual fusion and conditioning with neural networks for event recognition. In 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP), pp. 1-6. IEEE, 2019.
Mathilde Brousmiche, Jean Rouat, and Stéphane Dupont. Multimodal attentive fusion network for audio-visual event recognition. Information Fusion, 85:52-59, 2022.
Lele Chen, Sudhanshu Srivastava, Zhiyao Duan, and Chenliang Xu. Deep cross-modal audio-visug generation. In Proceedings of the on Thematic Workshops of ACM Mullimedia 2017, pp. 349-357, 2017.
Ying Cheng, Ruize Wang, Zhihao Pan, Rui Feng, and Yuejie Zhang. Look, listen, and attend: Co- attention network for self-supervised audio-visual representation learning. In Proceedings of the 28th ACM International Conference on Multimedia, pp. 3884-3892, 2020.
Zesen Cheng, Sicong Leng. Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms, arXiv preprint Xiiv:2406.07476. 2024.
Charalampos A. Dimoulas. Audiovisual spatial-audio analysis by means of sound localization and imaging: A multimedia healthcare framework in abdominal sound mapping. IEEE TMM, 18(10): 1969-1976, 2016.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. CVPR, pp. 9185-9193, 2018
Junyu Gao, Hao Yang, Maoguo Gong, and Xuelong Li. Audio-visual representation learning for anomaly events detection in crowds. Neurocomputing, pp. 127489, 2024.
Ian Goodfellow. Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Di Guo, Huaping Liu, and Fuchun Sun. Audio-visual language instruction understanding for robotic sorting. Robotics and Autonomous Systems, 159: 104271, 2023.
Sicong Han, Chenhao Lin, Chao Shen, Qian Wang, and Xiaohong Guan. Interpreting adversarial examples in deep learning: A review. ACM Computing Surveys, 2023.
Wangli Hao, Zhaoxiang Zhang, and He Guan. Cmegan: A uniform framework for cross-modal visual-audio mutual generation. In Proceedings of the AAAI conference on artificial intelligence, volume 32,2015 2018.
Chao Huang, Susan Liang, Yapeng Tian, Anurag Kumar, and Chenliang Xu. Davis: High-quality audio-visual separation with generative diffusion models. 2023a.
Chao Huang, Yapeng Tian, Anurag Kumar, and Chenliang Xu. Egocentric audio-visual object lo- calization. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recog- 2023b.
nition, pp.
Chao Huang, Susan Liang, Yunlong Tang, Yapeng Tian, Anurag Kumar, and Chenliang Xu. Scaling concept with text-guided diffusion models. arXiv preprint 1rXiv:2410.24155) 2024.
Hee-Seon Kim, Minji Son, Minbeom Kim, Myung-Joon Joon Kwon, and Changick Kim. Breaking tem- poral consistency: Generating video universal adversarial perturbations using image models. In Proceedings of the IEFF/CVE International Conference on Computer Vision, pp. 4325-4334, 2023.
Hoki Kim, Woojin Lee, and Jaewook Lee. Understanding catastrophic overfitting in single-step adversarial training. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 8119-8127, 2021.
11

---

Published as a conference paper at ICLR 2025
A Kloukiniotis, A Papandreou, A Lalos, P Kapsalas, D-V Nguyen, and K Moustakas. Countering adversarial attacks on autonomous vehicles using denoising techniques: A review. IEEE Open Journal of Intelligent Transportation Systems. 3:61-80, 2022.
Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial Examples in the Physical World. In ICLR, 2017.
Guangyao Li, Yake Wei, Yapeng Tian, Chenliang Xu. Ji-Rong Wen, and Di Hu. Learning to answer questions in dynamic audio-visual scenarios. In CVPR, pp. 19108-19118, 2022a.
Juncheng B. Li, Shuhui Qu, Xinjian Li, Bernie Po-Yao Huang, and Florian Metze. On adversarial robustness of large-scale audio visual learning. In ICASSP, pp. 231-235, 2022b.
Susan Liang, Chao Huang, Yapeng Tian, Anurag Kumar, and Chenliang Xu. Av-nerf: Learning neu- ral fields for real-world audio-visual scene synthesis. Advances in Neural Information Processing Systems, 36:37472-37490, 2023a.
Susan Liang, Chao Huang, Yapeng Tian, Anurag Kumar, and Chenliang Xu. Neural acoustic context field: Rendering realistic room impulse response with neural fields. arXiv preprint arXiv:2309.15991 15977, 2023b.
Susan Liang, Chao Huang, Yapeng Tian, Anurag Kumar, and Chenliang Xu. Language-gund guided joint audio-visual editing via one-shot adaptation. In Proceedings of the Asian Conference Computer Vision, pp. 1011-1027, 2024.
Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John E Hopcroft. Nesterov accelerated gradient and scale invariance for adversarial attacks. In ICLR, 2019.
Dong Lu, Zhiqiang Wang, Teng Wang, Weili Guan, Hongchang Gao, and Feng Zheng. Set-level guidance attack: Boosting adversarial transferability of vision-language pre-training models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 102-111, 2023.
Wenshuo Ma, Yidong Li, Xiaofeng Jia, and Wei Xu. Transferable adversarial attack for both vision transformers and convolutional networks via momentum integrated gradients. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4630-4639, 2023.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference Learning Representations, 2018.
Sagnik Majumder and Kristen Grauman. Active audio-visual separation of dynamic sound sources. In European Conference on Computer Vision, pp. 551-569. Springer. 2022.
Sagnik Majumder, Ziad Al-Halah, and Kristen Grauman. Move2hear: Active audio-visual source separation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp- 275-285, 2021.
Xiongkuo Min, Guangtao Zhai, Jiantao Zhou, Xiao-Ping Zhang, Xiaokang Yang, and Xinping Guan. A multimodal saliency model for videos with high audio-visual correspondence. IEEE Transac- tions on Image Processing. 29:3805-383 3819, 2020.
Pedro Morgado, Ishan Misra, and Nuno Vasconcelos. Robust audio-visual instance discrimination. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12934-12945, 2021.
Tanzila Rahman, Mengyu Yang, and Leonid Sigal. Tribert: Human-centric audio-visual representa- tion learning. Advances Neural Information Processing Systems, 34:9774-977 2021.
Leslie Rice, Fric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. In International conference machine learning, pp. 8093-8104 PMLR, 2020.
Luchuan Song, Guojun Yin, Zhenchao Jin, Xiaoyi Dong, and Chenliang Xu. Emotional listener portrait: Realistic listener motion simulation in conversation. In 2023 IEEE/CVF International Conference Computer Vision (ICCV), pp. 20782-20792. IEFE, 2023.
12

---

Published as a conference paper at ICLR 2025
Luchuan Song, Lele Chen, Celong Liu, Pinxin Liu, and Chenliang Xu. Texttoon: Real-time text toonify head avatar from single video. In SIGGRAPH Asia 2024 Conference Papers, pp. 1-11, 2024a.
Luchuan Song. Pinxin Liu, Lele Chen, Guojun Yin, and Chenliang Xu. Tri 2-plane: Thinking head avatar via feature pyramid. In European Conference Computer Vision, pp. 1-20. Springer, 2024b.
Zehua Sun, Qiuhong Ke, Hossein Rahmani, Mohammed Bennamoun, Gang Wang, and Jun Liu. Human action recognition from various data modalities: A review. IEEE TPAMI, 2022.
Kim Sung-Bin, Arda Senocak, Hyunwoo Ha, Andrew Owens, and Tae-Hyun Oh. Sound visual scene generation by audio-to-visual latent alignment. CVPR, pp. 6430-6440, 2023.
Yapeng Tian and Chenliang Xu. Can audio-visual integration strengthen robustness under multi- modal attacks? In CVPR, pp- 5601-5611, 2021.
Xiaosen Wang, Zeliang Zhang, Kangheng Tong, Dihong Gong, Kun He, Zhifeng Li, and Wei Liu. Triangle attack: A query-efficient decision-basee -based adversarial attack. In ECCV, pp. 156-174, 2022.
Jian Wu, Yong Xu, Shi-Xiong Zhang, Lian-Wu Chen, Meng Yu, Lei Xie, and Dong Yu. Time domain audio visual speech separation. In 2019 IEEE automatic speech recognition and understanding workshop (ASRU), pp. 667-673. IEEE, 2019a.
Xinyi Wu, Zhenyao Wu, Lili Ju, and Song Wang. Binaural audio-visual localization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 2961-2968, 2021.
Yu Wu, Linchao Zhu, Yan Yan, and Yi Yang. Dual attention matching for audio-visual event lo- calization. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 6292-6300, 2019b.
Yan Xia and Zhou Zhao. Cross-modal background suppression for audio-visual event localization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp-
19989-19999 19998, 2022.
Dingkang Yang. Yang Liu, Can Huang. Mingeheng Li. Xiao Zhao, Yuzheng Wang, Kun Yang, Yan Wang, Peng Zhai, and Lihua Zhang. Target and source modality co-reinforcement for emotion un- derstanding from asynchronous multimodal sequences. Knowledge-Bold Based Systems, 205:110370, 2023a.
Karren Yang, Wan- Yi Lin, Manash Barman, Filipe Condessa, and J. Zico Kolter. Defending multi- modal fusion models against single-source adversaries. In CVPR, pp. 3340-3349, 2021.
Zequn Yang, Yake Wei, Ce Liang, and Di Hu. Quantifying and enhancing multi-modal robustness with modality preference. In The Twelfth International Conference on Learning Representations, 2023b.
Wei Yao, Zeliang Zhang, Huayi Tang, and Yong Liu. Understanding model ensemble in transferable adversarial attack. arXiv preprint arXiv:2410.0685// 2024.
Chaojian Yu, Bo Han, Shen, Jun Yu, Chen Gong, Mingming Gong, and Tongliang Liu. Under- standing robust overfitting of adversarial training and beyond. In International Conference Machine Learning, pp. 25595-25610. PMLR, 2022.
Jiaming Zhang, Qi Yi, and Jitao Sang. Towards adversarial attack on vision-language pre-training models. In Proceedings of the 30th ACM International Conference on Multimedia, pp. 5005- 5013, 2022.
Jianping Zhang, Jen-tse Huang, Wenxuan Wang, Yichen Li, Weibin Wu, Xiaosen Wang, Yuxin Su, and Michael R Lyu. Improving the transferability of adversarial samples by path-augmented method. In Proceedings of the IEEE/CVF Conference on Computer Vision and Paterrn Recogni- fion, pp. 1173-8182.. 2023.
13

---

Published as a conference paper at ICLR 2025
Weiyi Zhang, Shuning Zhao, Le Liu, Jianmin Li, Xingliang Cheng, Thomas Fang Zheng, and Xi- aolin Hu. Attack on practical speaker verification system using universal adversarial perturba- 2021.
tions. In ICASSP, pp.
Zeliang Zhang, Wei Yao, Susan Liang, and Chenliang Xu. Random smooth-based certified defense against text adversarial attack. In Findings of the Association for Computational Linguistics: EACL 2024, pp. 1251-1265, 2024a.
Zeliang Zhang, Wei Yao, and Xiaosen Wang. Bag of tricks boost adversarial transferability. arXiv preprint arXiv:2401.08734 2024b.
Hang Zhao, Chuang Gan, Andrew Rouditchenko, Carl Vondrick, Josh McDermott, and Antonio Tor- ralba. The sound of pixels. In The European Conference on Computer Vision (ECCV), September 2018a.
Hang Zhao, Chuang Gan, Andrew Rouditchenko, Carl Vondrick, Josh H. McDermott, and Antonio Torralba. The sound of pixels. In ECCV, volume 11205, pp. 587-604, 2018b.
Rui Zhen, Wenchao Song, Qiang He, Juan Cao,, Lei Shi, and Jia Luo. Human-computer interaction system: survey of talking-head generation. Electronics, 12(1):218, 2023.
Hang Zhou, Yu Liu, Ziwei Liu, Ping Luo, and Xiaogang Wang. Talking face generation by ad- versarially disentangled audio-visude representation. In Proceedings of the AAAI conference artificial intelligence, volume 33, pp. 9299-9306, 2019.
Ye Zhu, Yu Wu, Hugo Latapie, Yi Yang, and Yan Yan. Learning audio-visual correlations from variational cross-modal generation. ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4300-4304. IEEE, 2021.
14

---

Published as a conference paper at ICLR 2025
A RESULTS ON MIT-MUSIC DATASET
Setup. We use the MIT-MUSIC (Zhao et al. 8a) for further verification, which contains 685 videos of musical solos and duets. It consists of 11 categories. We split the dataset into 7 : 1 as the train, validation and test set. Following the same setting in our paper, we train 8 audio-visual models on the MIT-MUSIC dataset and respectively the black-box setting. We use FGSM, I-FGSM, MI-FGSM, MIG, and PAM as our baselines, and compare the attack performance with our proposed methods, including TIA, MMA, and TMA. For defense methods, we select AT, DCFL, Mixup, CRMT-AT, and the baselines.
one surrogate model to attack others under
Results. We first various attack methods generate adversarial examples by attacking AcR under the white-box setting, and then attack other models under the black-box setting. As shown in fig. [0] our proposed TIA, MMA, and TMA achieve state-of-the-ar attack performance among the selected methods. Specifically, TIA achieves an average attack success rate of 95.2%, MMA achieves an average attack success rate of 93.7%, and the combination method TMA achieves an average attack success rate of 97.1%, while that of the rumner-up method is 88.1%. We can also see a consistent phenomenon that the adversarial example is easier to transfer between similar architec- ture, ie., from AcR to AsR.
We also evaluate the defense performance of our method. The results are depicted in fig. 11 Our proposed method surpasses the baseline methods with clear margin of 2.2% defending the adversarial attack versus the runner-up method CRMT-AT.
DCTL
Mixp
CEMT
POSM
MI FGSM
PAM
NISA
MIG
TIA
THA
AU800 Seccos tlx (44
70
20
70
Figure 10: Attack success rates (%) of eight deep models, where the adversarial exam- ples are generated on the white-box surrogate model and attack all models (one white-box model and seven black-box models).
Figure 11: Attack success rates (%) of each of 8 deep models on the adversarial examples crafted under the white-box setting with our proposed TMA method.
B ABLATION STUDY ON THE ADVERSARIAL CURRICULUM TRAINING
In our paper, we propose adversarial curriculum training enhance the performance of adversarial training. This approach includes both data-level and model-level strategies. To study the influence of the scheduler used to adjust the masking ratio (i.e.. Pe and py), we conduct additional experiments to evaluate the impact of different schedulers on audio-visual robustness. These schedulers are applied to boost the audio visual adversarial robustness of RsV. Details are as follows:
None: This represents standard adversarial training without employing the adversarial cur- riculum training strategy.
D*M: 5/20/30 %: These represent constant ratios used in the data- and model-centric strate- gies. For example:
D*M: 5% indicates that 5% of the audio-visuaa frames and 5% of the model param- eters are randomly masked to generate adversarial examples. This setup provides a relatively simple defense scenario for the model.
15

---

Published as a conference paper at ICLR 2025
D*M: 20% means that 20% of the frames and 20% of the parameters used to generate adversarial examples. This setup creates a stronger adversarial attack and increases the difficulty of adversarial training.
D*M: 30% follows a similar logic with higher masking ratio for stronger adversarial perturbations.
D: 20%: This indicates that only 20% of the frames randomly sampled for adversarial example generation during adversarial training, without masking model parameters.
M: 20%: This indicates that 20% of the model parameters are randomly masked in each
iteration for adversarial example generation, without randomly dropping frames.
M: 40%: This indicates that 40% of the model parameters are randomly masked in each
iteration for adversarial example generation, without randomly dropping frames.
Linear: This applies linear scheduler from 5% to 20% for both data sampling and model masking ratios.
Cosine: This applies cosine scheduler from 5% to 20% for both data sampling and model masking ratios.
The results are shown in fig. 12 Compared with vanilla adversarial training, applying the proposed schedulers adversarial curriculum training enhances adversarial robustness. For instance, when we synchronize masking at 20% on both data and model parameters (D*M:
Succese betense Sate (80
20%), the success defense rate improves by 3.8%. Furthermore, integrating linear and co- sine schedulers to dynamically adjust the dif- ficulty of adversarial examples during training leads to additional improvements of 0.8% % and 1.4%, respectively.
156
125
Home
5%,
2020 202
Constant Sync
Corstant Asym:
It is worth noting that lower sampling ra- tio for frames combined with a higher param- eter masking ratio can negatively impact the quality of the generated adversarial examples. This, in turn, degrades adversarial robustness, as demonstrated by the results for D*M: 5% and M: 40%.
Figure 12: Evaluation results applying differ- ent schedulers to the adversarial curriculum train- ing. The studied model is RsV (ResNet as the vi- sual backbone, VGG as the audio backbone, and sum operation the fusion layer.)
C THE DIFFERENCE BETWEEN VISION-LANGUAGE AND AUDIO-VISUAL ATTACKS/DEFENSE
There are certain similarities between audio-visual attack/defense and vision-language lack/defense methods (Zhang et al. 2022 Lu et al. 2023). as both require consideration of alignment and consistency between the two modalities. However, there are also notable differences between them. (1) Task difference: iision-languaa
attacks aim at attacking content retrieval-related prob- lems while audio-visual attacks focus on classification problems. (2) Operation difference: vision- language attacks perturb input data by optimizing latent embeddings while our method perturbs input by adjusting output logits. (3) Modality difference: vision-language attacks focus on static images while approach considers the temporal redundancy of dynamic videos. This redundancy motivates our design curriculum training to exploit sparsity, enhancing adversarial robustness while improving training efficiency.
Considering these reasons, we do not apply our proposed methods to the vision- lasiunlanguuage domain. However, believe these works in audio-visual learning inspiring for other multi-modal areas and we think exploring audio-visual attacks from the perspectives model pretraining, modality alignment, and content retrieval is valuable future direction.
16

---

Published as a conference paper at ICLR 2025
D DEMO APPLICATION ON ATTACKING VIDEOLLAMA
To showcase the scalability of our method in real-world applications, employ our pro- posed attack method to generate audio-visual adversarial examples under an ensemble set- ting. These examples are then used to deceive VideoLLaMA2 (Cheng al.) 2024). Using the prompt, "Please point out the main object gen- erating the sound based on the input video," evaluate VideoLLaMA's responses. Three results are illustrated fig. 13 where all in- puts misidentified by VideoLLaMA. For instance, the case of an airplane, VideoL- LaMA incorrectly recognizes This demonstrates the vulnerability of current MLLMS adversarial attacks, even when the adversarial examples are generated using con- ventional models.
rolling door
accordion
VideoLLaMA
Car racing
airplane
car racing.
Ukulele
guitar
For a quantitative evaluation, we generated total of 100 audio-visual adversarial examples using our proposed TMA method and the best baseline method, PAM, respectively, and tested them on LideoLLaMA. While PAM success- fully deceived VidcoLLaMA in 41 cases, our
Figure 13: Demo evaluation attacking the VideoLLaMA2 using the generated audio-visual adversarial examples by proposed attack method.
proposed TMA achieved higher success rate,
with 74 examples successfully attacked.
17


Reviews:
[{"review_id": "QOJQbhQJmj", "reviewer": "Reviewer_7ZEx", "summary": "This paper studies audio-visual adversarial vulnerability, and proposes two kinds of adversarial attacks which deals with the temporal consistency and intermodal correlation. Furthermore, a new audio-visual adversarial training framework is also designed to defend against above attacks.", "questions": "Maybe it is better to show the effectiveness of the method by some real audio-visual samples, i.e., applying the proposed attacks can indeed attack existing methods while the proposed defending method can handle/alleviate it.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The two kinds of proposed adversarial attacks are novel and important for the topic of audio-visual adversarial vulnerability.\n2. The experiments shows the effectiveness of the proposed attacks.", "weaknesses": "1. The proposed defending method seems not designed exactly towards the proposed attacks. In particular, the modality-misalignment-based attack is not specifically addressed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gWFj4HMlMs", "reviewer": "Reviewer_eurp", "summary": "This paper introduces two novel audio-visual adversarial attacks, the Temporal Invariance-based Attack (TIA) and the Modality Misalignment-based Attack (MMA), along with an adversarial training framework that enhances robustness and efficiency. The experiments demonstrate the effectiveness of these methods in benchmarking audio-visual model robustness and improving both adversarial robustness and training efficiency.", "questions": "Please refer to the questions raised in the Weakness section. Additionally, the reviewer is interested in the following questions:\n(1) What's the selection criterion when choosing datasets for the experiment?\n(2) Could the proposed method be incorporated into the pre-trained multi-modal models, such as CLIP or BLIP?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "(1) The paper proposes innovative adversarial attacks tailored to audio-visual models, specifically the Temporal Invariance based Attack (TIA) and the Modality Misalignment-based Attack (MMA). These proposed  attacks leverage the unique properties of audio-visual data, such as temporal redundancy and intermodal correlation.\n\n(2) The paper introduces an audio-visual adversarial curriculum training framework that incorporates efficient adversarial perturbation crafting and an adversarial curriculum strategy, which achieves a balance between robustness and efficiency.", "weaknesses": "(1) The experiments related to the proposed method are not sufficiently comprehensive.    In Section 5.3, Adversarial Curriculum Training is introduced, which includes two strategies: the Data-level strategy and the Model-level strategy. The core of Adversarial Curriculum Training involves the iterative adjustment of the data masking ratio ( ) and the model dropout ratio ( ). However, the paper does not present corresponding experimental results to substantiate the significance of this iterative design.    Therefore, it is recommended to add experimental results to support the validity of this design.\n(2) The paper evaluates unimodal adversarial attack methods (such as FGSM, I-FGSM, and MI-FGSM in the image domain). However, studies such as [1,2] have proposed multimodal attacks. It remains unclear whether the proposed defense method is effective against multimodal adversarial attacks similar to the Audio-Visual attacks discussed in [1].    I suggest adding experiments to evaluate the method's performance under multimodal attack scenarios.\n\n[1] Zhang J, Yi Q, Sang J. Towards adversarial attack on vision-language pre-training models[C]//Proceedings of the 30th ACM International Conference on Multimedia. 2022: 5005-5013. \n[2] Lu D, Wang Z, Wang T, et al. Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models[J]. arXiv preprint arXiv:2307.14061, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vyvu4vffTy", "reviewer": "Reviewer_nTiz", "summary": "The paper presents two powerful adversarial attacks: 1) a temporal invariance attack that exploits the inherent temporal redundancy across consecutive time segments and 2) a modality misalignment attack that introduces incongruence between the audio and visual modalities. To defend against such attacks, the paper introduces a novel audio-visual adversarial training framework. Extensive experiments illustrate the effectiveness of the proposed method.", "questions": "1. In experiment comparison, some figures use TMA to represent the proposed method, while some of them use Ours. It is suggested to use the same. \n2. For Table 2, does the ablation study use the proposed method TMA? If yes, it is suggested to add the information in the caption.\n3. According to Table 2, the successful defense rate continuously increases with the addition of sampling ratio. Where is the change point? The experiment of the ablation study needs to show the sampling ratio value from where the result performance will drop down.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper exhibits two powerful adversarial attacks: a temporal invariance attack and a modality misalignment attack. These attacks show their ability to obtain the robustness of audio-visual models.", "weaknesses": "The temporal invariance attack is realized by setting consecutive time segments. The modality misalignment attack introduces incongruence between the audio and visual modalities. Regarding the novelty of the two methods, it seems fair. It is encouraging that two attacks achieve SOTA performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wD4Au2e4sR", "reviewer": "Reviewer_RUCN", "summary": "This paper propose two adversarial attack methods for audio-visual data. Audio-visual data has two main characteristics: the temporal consistency (i.e., between nearby time step, the data is similar) and the modality alignment (i.e., audio and visual data has alignment). The attack exploits these two main characteristics: temporal invariance-based attack and modality misalignment-based attack. In addition, this paper propose the defense mechanism that has lower computational cost in comparison to the naive adversarial training. With lower computational cost, the model can be trained with various strength of the attack, improving its robustness. The way to do lower the computational cost is by only attacking sampled frames instead of attacking all frames. Another additional way is to use curriculum learning by masking out the data and the model's weight from low rate (easy) to high rate (hard). The number of steps to generate attacks also go from low steps (easy) to more steps (hard).", "questions": "1. What is the meaning of (a)synchronous in context of Line 138?\n\n2. Is \"contact\" in line 411 and 415 is \"concat\"?\n\n3. Line 366: Is the masking done synchronously or asynchronously?\n\n4. Figure 3. What does it mean minimizing prediction $p_{av}$, $p_a * p_v$ ?\n\n5. Based on Eq. (4), loss in Eq. (2) will be minimized. However, if we want to exploit the temporal consistency (line 52), why not maximizing this Eq. (2) loss? I think, maximizing variation of the extracted features between time step would make the temporal more inconsistent, which can make the attack more efficient.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The method of attacking audio-visual data based on the temporal consistency and modality alignment seems to be original.\n\n2. The empirical experiments in Section 3 as motivation is interesting to prove how breaking the temporal consistency and modality alignment (by masking out the data) reduce the audio-visual model performance and increase the attack success rate.", "weaknesses": "Several unclear information. Please see Questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)