Exposing and Defending the Achilles' Heel of Video Mixture-of-Experts
Authors: Songping Wang
Qinglong Liu
Yueming Lyu
Ning Li
Ziwen He
Caifeng Shan
Venue: ICLR 2026 Poster (2026)
Keywords: Adversarial attacks for Video MoE; Robustness of Video MoE
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: Exposing and Defending Video Mixture-of-Experts via Temporal Lipschitz-Guided Attacks

Abstract: Mixture-of-Experts (MoE) has demonstrated strong performance in video understanding tasks, yet its adversarial robustness remains underexplored. Existing attack methods often treat MoE as a unified architecture, overlooking the independent and collaborative weaknesses of key components such as routers and expert modules. To fill this gap, we propose Temporal Lipschitz-Guided Attacks (TLGA) to thoroughly investigate component-level vulnerabilities in video MoE models. We first design attacks on the router, revealing its independent weaknesses. Building on this, we introduce Joint Temporal Lipschitz-Guided Attacks (J-TLGA), which collaboratively perturb both routers and experts. This joint attack significantly amplifies adversarial effects and exposes the Achilles’ Heel (collaborative weaknesses) of the MoE architecture. Based on these insights, we further propose Joint Temporal Lipschitz Adversarial Training (J-TLAT). J-TLAT performs joint training to further defend against collaborative weaknesses, enhancing component-wise robustness. Our framework is plug-and-play and reduces inference cost by more than 60% compared with dense models. It consistently enhances adversarial robustness across diverse video datasets and model architectures, effectively mitigating both the independent and collaborative weaknesses of MoE.

EXPOSING AND DEFENDING THE ACHILLES' HEEL OF VIDEO MIXTURE- OF -XPPRRS
Songping Wang Qinglong Liu' Yueming Lyu't Ning Li Ziwen He Caifeng Shan Naanjing University "China Mobile Information Technology Co., Ltd.
NNanjng University of Information Science and Technology
ABSTRACT
Mfxture-of-Experts (MoE) has demonstrated strong performance in video under- standing tasks, yet its adversarial robustness remains underexplored. Existing attack methods often treat MoE as unified architecture, overlooking the inde- pendent and collaborative weaknesses of key components such as routers and ex- pert modules. To fill this gap, we propose Temporal Lipschitz-Guice Guided Attacks (TLGA) thoroughly investigate component-level vulnerabilities in video MoE models. We first design attacks on the router, revealing its independent weak- nesses. Building on this, we introduce Joint Temporal Lipschitz-uuide Attacks (J-TLGA), which collaboratively perturb both routers and experts. This joint at- tack significantly amplifies adversarial effects and exposes the Achilles' Heel (col- laborative weaknesses) of the MoE architecture. Based on these insights, fur- ther propose Joint Temporal Lipschitz Adversarial Training (J-TLAT). J-TLAT performs joint training further defend against collaborative weaknesses, en- hancing component-wise robustness. Our framework is plug-and-play and reduces inference cost by more than 60% compared with dense models. It consistently en- hances adversarial robustness across diverse datasets and architectures, effectively mitigating both the independent and collaborative weaknesses of MoE.
INTRODUCTION
As the core of artificial intelligence, deep learning has advanced rapidly in recent years (Zhou et al. (2025); Meng et al. (202)); Chen et al. (2025b c); Han et al.) (2026a b); Liu et al. (2024) (025)). ) Central to this progress has been the continuous exploration of scalable and efficient model de- signs. Mixture-of -ixtere--ffExpprtts (MoE) has emerged as efficient and powerful deep learning architec- ture within this field. By introducing a routing mechanism that activates only small subset of expert sub-networks for each computation, MoE increases model capacity while keeping inference costs under control (Jacobs et al. 199T)) & Jacobs ( 1994); [hazzer et al. (2017). This architectural advantage is particularly salient in the domain of video understanding. Video data, characterized by its complex spatial-temporal structures and long-range dependencies, present a fundamental chal- lenge for traditional dense models in balancing expressive power with computational feasibility. MoE addresses this challenge by dynamically selecting experts conditioned on frame-level seman- ties, thereby demonstrating a remarkable ability to capture motion dynamics and temporal contexts. Consequently, MoE has achieved outstanding performance in tasks such as action recognition and video-language modeling (Jain et al. (2024): : Wu al. (2024); Shaabana et al. (2023)).
While video MoF models have achieved strong performance, their robustness and security have not received commensurate attention. Like other deep learning models, they remain susceptible to adversarial -aamfles-carffully crafted perturbations that can cause high-confidence misclassifi-
cations (Goodfellow et al. (2014); Xie et al. (2017): Wang et al. (2025c 2024a; 2025d a b); Miao et al. (2022)). Such vulnerabilities pose serious risks to safety-critical video applications such as autonomous driving and surveillance (Kong et al. (2025); Chen et al. (2022); Wang et al. (2024b)).
Specifically, the vulnerabilities of video MoE models particularly complex. On one hand, the strong temporal structure of video data allows perturbations to propagate across frames, leading to
"Equal contribution. 'Corresponding authors. Code is released at https:/gtttuh.oo//Depss////////

---

Conventional
MoE
O
Ourput
Selecr strong expertt
Max Loss Loge (f noe ())
Experts
87
Fd
Flom Mitro
Adccuuial Nose
Class. Video
ohay agess
Trest MoE as whole
2
Router
Cannot discover weaknesses expose experts!
Our Joint Attack
MoF.
O
Ourput
Loss Looo + ft bnoater
Heal expert
Shat
Atrele
Experts
Adveronta Nove
Closs Video
d/a )x
Composení Albcc
Successfully discover weaknenes Reveal weak experts
Joint astakk
Figure 1: Conventional attacks vs. ours. Conventional attacks treat the MoE a whole, overlooking its internal architecture and thus capping the attack strength. Our method performs a component- wise joint attack that explicitly targets the Router and Experts, steering the Router toward weak experts and simultaneously perturbing them. This strategy precisely exposes the Achilles' Heel (collaborative weaknesses) of MoE while significantly elevating the attack potency.
cumulative temporal errors. the other hand, the modular design of MoE featuring diverse ex- perts and dynamic routing-introduuce additional attack surfaces. Adversaries may target individual experts or routers, or disrupt their coordination. However, existing adversarial attacks and training strategies treat MoE whole, neglecting the collaborative vulnerabilities of internal components, particularly in the video domain. This highlights the urgent need to discover component-level vul- nerabilities in video MoE and develop specialized adversarial training (AT) mechanisms. We thus ask:
(Q.A) What weaknesses do adversarial attacks expose in video MoE models? (Q.B) Given these insights, how can we develop effective adversarial training to defend against these weaknesses?
To address the above questions, first investigate the component-level vulnerabilities of video MoF. We propose a family of Temporal Lipschitz-Guided Attacks (TIGA) that incorporate both Lipschitz regularity and temporal adaptive step-size, targeting the router, the experts, and the MoE model as a whole. We are the first to show that TLGA applied to the router can cause a collapse of routing decisions, posing serious threat to the model's integrity. Furthermore, we introduce Joint Temporal Lipschitz-Guut Guided Attack (J-TLGA) targeting both router and experts, showing that coordinated attacks can lead to more destructive outcomes due inter-component dependencies. J-TLGA reveals that collaborative weaknesses are the Achilles' heel of MoE.
Building on the weaknesses explored by TLGA, we propose two defense strategies: Temporal Lipschitz Adversarial Training (TLAT) and Joint Temporal Lipschitz Adversarial Training (J- TLAT). TLAT strengthens whole MoE robustness by injecting TLGA, while J-TLAT hierarchically enhances adversarial robustness from component-level to overall-level. defending the Achilles' Heel of MoE. Furthermore, we provide a theoretical derivation of the Lipschitz upper bound for MoE in Appendix, which offers theoretical support for joint attack-defens framework. Our contribu- tions are summarized as follows:
component-level vulnerabilities analysis: We conduct the first systematic investigation into the component-wist vulnerabilities of video MoE, revealing both independent and collaborative failure patterns between the router and experts.
Temporal Lipschitz-Guided attacks: We propose TLGA, which first exposes the vul- nerabilities of the routing mechanisms video MoE. We further extend it to J-TLGA, uncovering the coordinated vulnerabiliries between routers and experts.
2

---

Weakness-guided defense mechanisms: Based on the identified weaknesses, in- troduce two adversarial training frameworks: TLAT enhances global robustness through TLGA, while J-TLAT enhances component-wise and overall robustness hierarchically by addressing their collaborative weaknesses.
Extensive evaluation across datasets and models: Experiments demonstrate J-TLAT ef- fectively defends the Achilles' Heel of MoE, while maintaining inference efficiency and accuracy. We also provide theoretical analysis for the Achilles' Heel of MoE in Appendix.
2
RELATED WORK
2.1 ADVERSARIAL ATTACK AND DEFENSE ON VIDEO
Recent research shows that video recognition models remain vulnerable to adversarial perturbations. Wei et al. (2019) introduce 3D sparse perturbations to generate adversarial examples in white-box settings, while Wei et al. (2020) employ heuristic-based optimization to perturb selective key frames with minimal noise. Yin et al. (023)) incorporate video transformations into the loss function to hance attack resilience. Wei et al. (2023) further reduce spatiotemporal redundancy with AstFocus, simultaneously targeting critical frames and regions across inter- and intra-frames. Yet, no adversarial attack methods have been specially designed for the promising MoF.
Research video defense mechanisms remains limited, increasing security risks in real-world ap- plications. OUDefend (Lo et al. (2021) designs a restoration network against adversarial videos, yet its evaluation is limited to black-box settings. AAT (Kinfu & Vidal 2022) combines curriculum and adaptive adversarial training to improve robustness against diverse attacks. Yi et al. (2023) boosts robustness via temporal coherence. However, defense for video MoE also remains underexplored.
2.2
MIX OF EXPERTS
of Experts (MoE) activates only top-k sub-networks per input (Jacobs al. (1991); |; Wang
offeeerss
et al. 2020); Fedus et al. (2022)) Despite its widespread use in video applications (Ma et al.
(025 ). MoE's resilience adversarial attacks remains underexplored: (Zhang al. (2023) decom-
poses MoE robustness into the robustness of the router and experts, yet without designing attacks specifically for MoE, leaving its inherent weaknesses largely unexposed. Similarly, Han et al. (2024) is confined to CNN models, limiting its applicability. ZZhang et al. (2025) proposes dual-moded ap- proach for image-based based MoE, achieving a more effective trade-off between robustness and accuracy. However, existing methods are tailored to the image domain, leaving attacks on video-based MoE as an open problem. Moreover, the robustness of video MoE complex due to its unique structure; treating it as a whole while neglecting the interactions between its components makes it difficult to identify the Achilles' Heel. Additionally, videos have more complex features than images, further complicating the development of adversarial attack methods for video MoE. Addressing these bustness issues is crucial for the deployment of MoE in safety-critical applications. This motivates us conduct systematic research on both attack and defense strategies for MoE.
should be noted here that the subsequent methods are all configured as white-box attacks. We committed to designing potent attacks against MoE maximally expose its fundamental weak- nesses and, based on these findings, develop effective defense strategies. As more powerful attack than gray-box and black-box attacks, white-box attack (which has knowledge of the target's archi- tecture, parameters, and gradients) can probe its internal vulnerabilities more deeply. Furthermore, if defense method can enable model withstand powerful white-box attacks, can typically also better defend against weaker gray-box and black-box attacks. This motivates us to design an integrated attack and defense framework for video MoE ensure its security.
PRELIMINARIES
MoE Architecture for Videos. MoE is a neural network paradigm in which traditional dense layers replaced by pool of specialized, sparsely-activater lightweight router dynamically selects only a handful experts for each input, allowing the model's capacity grow dramatically while keeping the inference cost nearly constant. Video MoE takes
aab-netwrrks-cann experts. A
3

---

step1: Router Training
Step2: Expert Training
Step3: Overal Training
Clean Sample
Adversatial Sample
Clean Sample
Adversarial Sample
Clean Sample
Adversarial Sample
Router
MoK
Rovier
Consistency
2
Expetts
27
A2
Aucovtristenc
priccur
Iogara
Experte
Orper o O
Kobuut counti
For debyy, Paee Byll
Gulltr--ellts Eadt Fleed.ll
Cas da
Nermnt experts
TLGAR
byyer
MAX
TLA-I
Lus
W//4
MIN
Router Robustness
Expert Rohustness
MoË
Robustness
Figure Framework of J-TLAT. Temporal Lipschitz Guided Attack for Router (TLGA-R), Temporal Lipschitz Attack for Expert (TLA-E), Temporal Lipschitz Attack for Moe (TLA-M) all designed for MoE. J-TLAT enhances component-wisg and overall robustness hierarchically across three steps per epoch.
video data as input, which extends images by temporal axis. video clip can be represented as E @CxTxxxW where C is the number of channels (e.g. RGB). T is the number of frames, H x is the spatial resolution of each frame. Video MoE consists of Experts (Ei(.).... EM(-)): (-)) Expert maps the video data to logits
Ex(x) E RRV Router R(.) that produces weights for each expert: R(z)- = (wi (z), .... WMr (z)). The final prediction of MoE the weighted combina- (x),,
=i= Wy (x) E, E IR.. Following several recent works (Videau al. (2024): : He et al. (2024): Chen et al. (2025a)), we use light network as router R(-) to assign weights to various experts for given clip. Expert (x)) is a neural network to capture specific spatio-temporal content of the video. More details can be seen in Appendix.
tion of expert outputs: F(x) = >
Adversarial Attacks on Video MoE. Deep networks are inherently brittle, readily succumbing to adversarial perturbations; video-MoF, built upon this foundation, inevitably exposed to the same threat. An adversarial perturbation d I C aimed attacking video MoE is optimized to maximize the loss under an tp noomm budget:
M
8 arg max ECE
Σ
wr 0) 8), 8),
(I)
11
1=1
where y is the ground-truth label and lcE denotes the cross-entropy loss. The inequality ||ó||p S constrains the p-norm of adversarial noise to be less than or equal to a threshold c. We refer to these as conventional attacks, which treat MoE whole. Such attacks are not designed for MoE and thus ignore its key components (routers and experts). However, we know that the router and the experts are the core components of MoE. The two work together to realize the core idea of divide and conquer in the MoE architecture. First, treating MoE whole and ignoring its core components may limit the effectiveness of the attack. Second, there is currently lack of attacks designed specifically for the Video MoE architecture. These motivate us to explore attacks targeting the core components of Video MoF.
Ad Adversarial Attacks on Router and Experts. To further explore attacks targeting the MoE ar- chitecture, focusing on its two core components, we set up two types of component-level attacks, including router attacks and expert attacks. For router attacks, their goal is to alter the routing deci- sions by attacking the router, thereby increasing the output difference between adversarial samples and clean samples: 8n arg maxs {(R(x + 8), R(x)). For expert attacks, their goal is to produce erroneous outputs by attacking the experts: ò = arg maz, ECE (E,( + 8), y). Our subsequent research demonstrates the effectiveness of component-level attacks: generating adversarial samples
4

---

by attacking experts and routers successfully threaten MoE models and cause them to produce incorrect outputs.
4 METHODOLOGY
4.1 TEMPORAL LIPSCHITZ-GUIDER ATTACK
The current lack of attacks targeting the Video MoE architecture raises the question:
(Q1) How to design attacks against the core components of Video MoE to reduce its robustness and discover its weaknesses?
Our goal is to undermine the robustness of the model components perform successful attacks. Lipschitz constant K is a measure of the sensitivity of a function g to changes in its input, effectively reflecting adversarial robustness (Pauli al. [2021] Zhang et al. 20022; Zühlke & Kudenko (2025
For function g Id + RR defined on domain dom(g) C R there exists a constant K > 0 such
that for all 2, x + E dom(g), the following inequality holds:
(1 g(x 5)1l, K |2-
8)1p.-
Lipschitz constant reflects the model's sensitivity to minor perturbations. The more robust the model is, the less sensitive is to minor perturbations, and the smaller the Lipschitz constant. Conversely, the more vulnerable the model is, the more sensitive it is to minor perturbations, and the larger the Lipschitz constant. This motivates us to design attacks by considering the Lipschitz property. in order increase the Lipschitz constant of MoE, making it more sensitive minor perturbations and thereby compromising its robustness.
The Lipschitz constant quantifies model's sensitivity small perturbations. The more robust a model is, the less sensitive is to minor perturbations, resulting in a smaller Lipschitz constant. Conversely, a more fragile model exhibits higher sensitivity such perturbations, leading larger Lipschitz constant. This insight motivates us to design an attack by leveraging the Lipschitz property increase the Lipschitz constant of Mixture-of- -xpeertss (MoE) model. This makes the model more sensitive to small perturbations, therehy compromising its robustness.
We formulate this idea into differentiable and optimizable loss objective, denoted as L serves as a direct Lnite-difference approximation of the local Lipschitz constant and is mathematically expressed as:
||g(r + +5) (0((:+))-g((00)) g(x))12) -5)-9(z)112
lg(2)) g(x + 8)13 o
(M + 5))
Cpp
|
(x + 6)12
(M, 6)
Here, can represent the entire model F, the router R. or an expert F. By maximizing Liip- we can proactively search for directions in the input space that cause the most significant change in the model's output, i.e., directions that maximize the local Lipschitz value. Furthermore, this loss term can be minimized to enhance the model's s defensive capabilities. Specifically, minimizing Laip directly penalizes and suppresses the model's steepness in its most vulnerable directions. This pro- cess effectively smooths the decision boundary and lowers the upper bound of the global Lipschitz constant, concept we will discuss further in subsequent sections.
Additionally, videos have an extra temporal dimension. Previous methods set the attack step size for each frame to be the same, ignoring the differences between frames, which limits the effectiveness of the attack. Therefore, we consider utilizing the gradient differences between frames to adaptively allocate attack step sizes for each frame, thereby enhancing the destructiveness from the temporal perspective. Based on the above considerations, we design the Temporal Lipschitz Attack (TLA) to undermine the robustness of the Video MoE architecture from both the temporal and noise sensitivity aspects, as shown in the following formula:
MO = (F + 8x))8))+++ + (MSE(F(z)) F(x 8m))
(4)
(MSE(Z, 8mr)
V4+1 3. V. [V2(Mon(())2
(5)
5

---

a a'
1 log(1 + vV*)
Eodu = Proj(x + - sign V v(l))).
(7)
where ÉSS is the MSE loss, used balance the cross-entropy loss with the Lipschitz regu- larization, (.(M) represents the gradient of the 1-th frame, V =V, (V,.... VT) is the temporal momentum adjusted by 89 used to consider the cumulative effect of previous gradient norms, a the base attack rate, a (a1..... aT) represents the adaptive temporal step size, Proj(-) denotes limiting the perturbation within the norm- sign(-) denotes sign function and zadv represents the adversarial sample. Eq. 6 used smoothly suppress large gradients along the temporal di- mension. order to specifically launch effective attacks against the router, we design the Temporal Lipschitz Router Attack (TLA-R), the formulas be expressed as:
Router (RE(R(x 8R), yR) A LMSE(R(a R(x 5n))
[MSE(Z,
where yn denotes the index of selected expert for clean sample. By deeply coupling the Lipschitz derivative with the temporal dimension, we enable the attack to not only search for the steepest direc- tion that maximizes the local Lipschitz value, but also to intelligently allocate perturbation resources across the time dimension. This strategy ensures effective achievement of the attack objective.
The remaining process of Temporal Lipschitz Attack for Router (TLA-R) is similar to Eq. Eg the same applies to the attacks the following sections. We use different component- level attack methods and perturbation budget as the experts with top-1 Router. As shown in Tab[T] the conventional attack PGD-R targeting the Router difficult to exploit the weaknesses of the Router, and the MoE still maintains robust accuracy of over 42%. However, our TLAAR attack increases the attack strength by nearly 20%, effectively triggering the routing collapse of the MoE model, thereby causing sharp drop in robust accuracy. Our method demonstrates that component-level attacks targeting the Router alone can severely threaten MoE models trained by traditional AT. Our attack method TLA-E targeting the expert also outperforms the traditional expert attack PGD-E.
attack MoE model trained by AT. We 3D ResNet- 18
We further speculate that for the same sample, the expert allocated by the Router with higher confi- dence may be stronger, while the one allocated with lower confidence may be more vulnerable. We can design attacks to guide the Router to assign the sample to the expert with the lowest confidence, therehy exposing the most vulnerable expert and improving attack performance. Based on this, we further design the Temporal Lipschitz Guided Router Attack (TLGA-R), the formula as follows:
(9) where ýn represents the index of the expert with the lowest confidence output by the Router for the clean sample. As shown in Tab TLGA-R outperforms PGD-R by nearly 24% achieving the best attack performance for Router attacks. In addition, our method not only demonstrates the feasibility of component-level attacks but also provides solution for specifically targeting and exposing the weaknesses of the MoE architecture. We have the following insight:
foouter Router 71 (CE(R(x 6R), (R).
By only targeting the Router alone, component-level attacks can severely threaten MoE models trained by traditional AT.
Insight 1:
4.2 JOINT TEMPORAL LIPSCHITZ-GUDDER ATTACK
Based on Insight 1, we further examine the MoE architecture, where the components are not isolated but work together to achieve better performance. Is the same true for attacks? Thus, we raise the following question:
(Q2) Can joint attacks be launched against the components of MoE to achieve better attack performance?
This motivates us to explore the development of joint attacks targeting the MoE architecture. The targets of joint attacks include two aspects: one is the attack the Router and the other is the attack

---

on the overall architecture. For the Router attack, aim to force it to route the vulnerable experts with low confidence. For the overall attack, we hope to undermine the robustness of the entire MoE architecture and force it produce incorrect outputs. We adopt the idea of the TLGA-R attack for the Router attack and the idea of the TLA attack for the overall architecture attack. Thus, we design the Joint Temporal Lipschitz Attacks (J-TLA) and Joint Temporal Lipschitz-Guuided Attacks (J-TLGA) separately, which can be expressed as followings:
MOE MMME + 72 Router: EM = LM + 722 RRuteer
(10)
where MOE and MOE denote the losses for J-TLA and J-TI-GA separately. We further conduct experiments with settings consistent with those in Tab comparing our joint attack method with traditional attack methods.
As shown in T the joint attack J-TLA significantly reduces the robustness of MoE trained by AT under the budget of = 14/255 to as low as 4.73%. J-TLGA further enhances the attack power. Under J-TLGA, the accuracy of the robust MoE model is only 2.54%, which is far superior to the conventional attacks, exposing the Achilles' Heel of MoE. This indicates that: 1) Conventional at- tack methods are longer applicable to the MoE architecture. Ignoring its internal components severely limits performance and makes it difficult to deeply reveal the weaknesses of the MoE. ar- chitecture. 2) The collaboration between components is double-edged sword, making joint attacks more destructive. Based on these, We have the insight:
Insight The vulnerabilities of the components have a cumulative effect. Implementing joint attacks can enhance the attack performance to reveal its weaknesses.
4.3 JOINT TEMPORAL LIPSCHITZ ADVERSARIAL TRAINING
Traditional adversarial training methods are longer suitable for the Video MoE architecture. Be- cause of its end-to-end paradigm, it is difficult to finely repair component-level weaknesses and the weaknesses within component collaborations. Its robustness is seriously threatened by joint tacks, and there is urgent need to develop adversarial training methods more suitable for the MoE architecture based on its weaknesses.
(Q3) Can we develop suitable adversarial training mechanisms for Video MoE based on the weaknesses discovered by the attacks?
The TLA attack undermines the model's 's robustness by targeting the temporal and perturbation sen- sitivity dimensions. Conversely, we leverage the adversarial samples generated by TLA to conduct adversarial training the model, reducing its sensitivity to perturbations and thereby enhancing its adversarial robustness. Based on this, we design Temporal Lipschitz. Adversarial Training (TLAT), the formula of which is follows:
min max LMOE OMoE Znde
(11)
where OMO denotes the parameters of MoE. Although TLAT can enhance the overall robustness of the Video MoE architecture, it cannot achieve fine-grained robustness improvement at the com- ponent level. To realize both fine-grained component-level and overall robustness enhancement of the MoE architecture, we further design Joint Temporal Lipschitz Adversarial Training (J-TLAT). components to launch attacks, while J-TLAT
Joint attacks exploit the collaborative weaknesses repairs the weaknesses discovered by the attacks. As shown in Fig J-TLAT conducts hierarchical robust training in each training epoch. First, it strengthens the robustness of the router to maintain consistent output under different perturbations. Second, based on the TLA-R attack, identifies the weak experts and conducts adversarial training on them to address the weaknesses. Finally, J- TLAT performs adversarial training on the entire MoE model to further enhance the robustness of the overall collaboration, defending the Achilles' Heel of MoE. The formula is as follows:
stepl min max l Routers &Router
(12)
Σ wi (EEE (Ef( 1) ÉMSE (E E,(e +g:))
L Eupert
(13)
LASEE (x, x + 8px)
1EI
7

---

step2: min max l Expert, step3 min max MoOE Oxt Fads
(14)
Eppperi
where I = Top-2( Router(aadaa is the set of weak experts obtained by the TLGA-R attack. Router and f Expert denotes the parameters of Router and Experts, respectively. More details for J-TLAT can be seen in Appendix. Based on subsequent experiments, we have the following insight:
Joint adversarial training enhance overal and local adversarial robustness in tiered manner by fixing the weaknesses discovered through component-level attacks.
Insight 3:
Table I:
Robust Accuracy (%) under different MoE Attacks with varying perturbation budgets on UCF-101. Robust Accuracy denotes the percentage of correct predictions under attacks.
Router Attack
Expert Attack
Overall Attack
Joint Attack
PGD-R
TLA-R
TLGA--
PGD-E
TLA--
FGSM
PGD
TLA-M
J-TLA
J-TL.GA
8/255
42.20%
23.52%
18.13%
31.98%
28.11%
30.00%
22.09%
15.05%
7.03%
4.95%
10/255
41.98%
22.31%
16.48% %
29.01%
25.59%
27.25%
17.58%
12.42%
6.81%
3.41 %
12/255
39.78%
21.98%
16.22%
26.26%
22.08%
25.05%
15.71%
11.54%
5.39%
2.75%
14/255
39.34%
21.65%
15.82%
22.97%
19.89%
24.29%
14.84%
11.10%
4.73%
2.54%
5
EXPERIMENTS
5.1
EXPERIMENT SETUP
Dataset and Recognition Model. Following Wei et al. (2033), Our experiments are carried out on the widely used UCF-101 (Soomro et al. (2012) and HMDB-51 (Kuehne al. (20II) datasets. Following Wong et al. (2020), we employ three classic models including 3D ResNet-18 (Hara et al. 2018 ) TSM (Lin et al. (2019)) Slowfast (Feichtenhofer et al. (2019) and R(2+1)D (Huang et al. 2021 as the expert network, utilizing Top-1 router with four experts for the trade-off between
efficiency and accuracy.
Baselines. To verify that our methods more effectively exposes the vulnerabilities of MoE, we select classic adversarial robustness benchmarks and extend them to fit video data: FGSM (Goodfellow et al. (2014)), PGD (Madry et al.] (2017) and AutoAttack (Croce & Hein (2020) as baselines and further incorporate the mainstream white-box video adversarial attack TT Wei et al. (2022) as an extended baseline, enabling more comprehensive comparison. For defensive evaluation, We use the following methods as baselines: AT-S and AT-D (representing sparse and dense expert networks, respectively, both trained by AT), AT-M, OUD-M, and AAT-M (all MoE architectures, trained by AT, OUD (Lo et al. (202T), and AAT (Kinfu & Vidal 2022) ) respectively).
5.2
MAIN RESULTS
Evaluation of the Attack Method. Experiments in Tab show that TLGA-R attack significantly reduces the robustness of all methods except J-TLAT, highlighting the pronounced weaknesses in the router of the MoE architecture. Under the strong attack of J-TLGA (e = 8/ /525), the accuracy of OUD-M is 2.64%, and AAT-M even collapses to 0%, demonstrating that joint attacks can effectively reveal the collaborative weaknesses of MoE. and greatly enhance the attack effect.
Evaluation of the Training Method. Compared with AT-D (dense model), J-TLAT achieves very low Lipschitz constant (0.823). Under all perturbation strengths, J-TLAT shows nearly a 34% improvement over AAT-M under the strongest joint attack J-TLGA, with almost no additional infer- ence cost, strongly demonstrating the effectiveness of the joint training paradigm in enhancing the adversarial robustness of the MoE architecture.
5.3
ABLATION STUDY AND MORE RESULTS
Inner attacks. Fig compares robust accuracy under FGSM, PGD, and J-TLGA attacks for dif- ferent adversarial training configurations (PGD, LIP, T-PGD, TLA). LIP denotes the combination
8

---

Table Comprehensive evaluation of adversarial robustness on UCF-101 with 3D ResNet as back- bone. The table reports clean accuracy (%) and robust accuracies (%) against PGD, Auto/ttack,, TT, FGSM, TLA-M, TLA-E, TLGA-R, and J-TLGA attacks across four perturbation budgets. Ad- ditionally, GFLOPs denotes computational overhead and Lips-R and Lips-J represent the Lipschitz constant values under the TLGA-R and J-TLGA attacks respectively.
Dataset: UCF-101
Model: 3D ResNet
Method
CLEAN
PGD
8/255
10/250
12/206
14/255
8/205
10/253
12/226
14/255
AT-S
49.89
23.85
19.45
16.81
13.19
22.75
19.45
16.26
12.97
AAT-
54.51
24.84
14.73
20.11
17.25
14.18
OATM
49.23
19.23
18.46
15.93
14.62
21.43
20.00
17.69
15.82
CUDD-M
51.67
15.16
13.08
11.10
9.12
18.13
16.39
16.26
15.60
AAT-M
49.67
23.08
20.66
19.45
18.57
22.20
19.78
18.24
13.41
TLAT
51.65
30.22
27.14
22.75
19.34
22.64
20.22
19.89
18.46
<1-TL.AT
Metbod
GFLOPS
(9)
FOSM (9)
8/255
10/220
12/255
14/255
8/255
10/253
12/225
14/255
GATS
2.534
27.91
23.74
20:44
17.03
27.47
24.40
21.21
18.79
AAT-D
OAT M
1.831
25.05
22.09
21.54
18.79
29.56
27.36
24.73
24.73
-OUD-M
AATTMM
L.311
24.51
23.41
20:44
19.34
26.92
26.59
26.48
26.37
TTAT
24.84
33,08
27.69
<J-TLAT
1831
37.36
34.29
30.66
28.57
38.68
36.39
33.52
32.20
Metbod
Lips
TLA-M (%)
TLA-E (%)
8/255
10/255
12/255
14/255
8/255
10/253
12/255
14/255
OATM
261.8
18.90
16.70
15.05
15.82
29.23
27.36
22.86
19.89
COUD-M
1389
8.02
6.70
7.03
5.82
22.53
19.01
16.37
13.63
AAT-M
953.3
13.41
11.43
9.45
8.24
20.55
16.70
13.19
12.09
3,500
23.30
31.98
29.12
24.95
21.10
-J-TLAT
0.823
34.95
31.76
28.24
24.95
34.29
31.43
25.93
23.41
Method
Lips-J
TLGA-R
J-TLGA (%)
8/255
10/255
12/255
14/255
8/255
10/255
12/255
14/255
6.15
5:06
4.73
OOUD-M
1474
4.29
4.29
4,29
4.40
2.64
2.42
1.87
1.76
AATT-M
0.00
0.22
STLT
208.0
26.37
28.13
26.59
27.36
9.78
10.11
9.67
7,36
-J-TLA
30.33
24.95
2.30
POSM
PGD
JIUGA
POSM
PGD
TTLTT
Figure 3: The left figure represents the robustness using different attacks as internal attacks in J- TLAT. The Right figure represents the robustness when using different regularization methods for the external training of J-TLAT on UCF-101.
of PGD attack and Lipschitz regularization, while T-PGD represents the combination of PGD and temporal step-size adaptation. TLA outperforms others, achieving 33.63% robust accuracy under PGD (a 5.5% gain over conventional PGD) and maintaining 30.33% under J-TLGA, highlighting its effectiveness against structured attacks.
Regularization strategies. We compare regularization strategies (JS, COS, KL, NR (Tramèr et al. (2077), GR (Wong et al.) (2020g) and J-TLAT under three attacks. Here, IS denotes the use of Jensen-Shannon divergence, COS denotes the use of cosine loss, and KI. denotes the use of Kullback-Leibler divergence. J-TLAT achieves the highest robustness (36.59%, 33.63%, 30.33%) in all scenarios, demonstrating its advantage in enhancing robustness by reducing model sensitiv- ity to perturbations. Fig 4(a)
() and b)) show that our attack methods TLGA-R and J-TLGA achieve better separation of the distributions of adversarial and clean samples against AT-M. thereby realizing superior destructive power. 4c) and Fig dd) illustrate that J. J-TLAT reduces the loU distribution difference with higher adversarial robustness. shows that J-TI.G. GAA effectively

---

25
Cleos Datt
Cron Data
Clean Data
Crean Data JLLAA Data
PGD Data
Data
J-TuA
Duto
11.04 Data
1ol score
IeU acore
1e0 acore
160 score
04)
900
10
[9]
Figure 4: Distribution of IoU scores under different attacks on UCF-101. IoU score is a metric used measure the output consistency of MoE Router. The greater the deviation from the IoU distribution of clean data, the higher the attack success rate for the adversarial data.
Dataset:
UCF-101 Model: AT-M Clean Accuracy: 49.78%
25-
J-LLAT
Attack
5/255
10/255
12/255
14/255
Clean Dicc
Clean Data
TLGA-I Duta
TLGA-I Deta
PGD
22.09%
17.58%
15.71%
14.84%
LA
20.66%
17.24% %
15.38
14.73%
o T-PGD
18.79%
16.48
14.18%
13.63%
12.42%
11.54%
11.10%
J-TLGA
4.950%
3.410%
2.750%
2.540%
14
66
18
Figure 5: Ablation study of our TLGA against AT-M on UCF-101. LA stands for Lipschitz at- tack, T-PGD stands for PGD combined with tem- poral adaptive step size.
(a) AT-M
(b) J-TLAT
Figure 6: IoU score distributions of AT-M and J- TLAT under TLGA-R attack on UCF-101.
poses the Achilles' Heel of MoE compared with other attacks. Fig 6 illlstrates that J-TLAT ensures a more stable routing mechanism under TLGA-R.
Source Model
Method
3D Resnet
Slowfast
R(2+1)D
PGD
94.80 94.80%
95.00%
85.61%
FGSM
y5.58%
87.46%
3D Resmet
TT
66.04%
94.32%
84.78%
J-TLGA
81.79%
66.90%
PGD
87.89%
77.56%
88.48%
FGSM
Slowfast
TT
86.84%
89.04%
87.34%
J-TLGA
49.09%
PGD
84 6..4%%
95.57%
67. 90%
Dur
FGSM
R(+1)
TT
84%1
94 68%
78.31%
J-TLGA
73.59%
43.10%
Table 3: Transfer attack performance. The val- ues represent the model's remaining accuracy (%) under different transfer attacks. Lower val- ues indicate a more effective attack. J-TLGA possesses better generalization capabilities. More results. Presents the transfer attack performance of different attacks across various mod- els. Experiments demonstrate J-TLGA outperforms other methods black-box transfer settings, showing better generalization capabilities. Fig 7 shows AT-M collapses while J-TLAT maintains strong defense across training. More results and details can be found in Appendix.
Training dynamics of robust accu- racy (%) under four attack settings comparing J- TLAT and AT-M over 70 epochs. J-TLAT con- sistently achieves higher and more stable robust- ness across training, maintains strong defense.
Figure 7:
CONCLUSION
The MoE architecture has shown great potential in the field of video understanding. However, there currently gap in both attacks and defenses methods targeting the video MoE architecture. To ad- dress these issues, we propose TLGA to specifically attack the core components of video MoE and further develop the joint attack method J-TLGA, which exposes the Achilles' Heel of video MoE. Based on these weaknesses, we propose the J-TLAT joint training method, which hierarchically enhances the adversarial robustness of MoE architecture from component-level to overall-level. Ex- tensive experiments have demonstrated the effectiveness of our methods, laying the groundwork for future research on adversarial attack and defense in video MoE.
10

---

ACKNOWLEDGMENTS
This work was supported by the New Generation Artificial Inteligence-Nationn. Science and Tech- nology Major Project (20252D0123504), the National Natural Science Foundation of China (Grants 62502200 and 62402228), the Jiangsu Provincial Science and Technology Major Project (Grant BG2024042), and the Natural Science Foundation of Jiangsu Province (Grants BK20251200 and BK20240699).
REFERENCES
Bowen Chen, Keyan Chen, Mohan Yang, Zhengxia Zou, and Zhenwei Shi. Heterogeneous mixture of experts for remote sensing image super-resolution. IEEE Geoscience and Remote Sensing Letters, 2025a.
Chubin Chen, Sujie Hu, Jiashu Zhu, Meiqi Wu, Jintao Chen, Yanxun Li, Nisha Huang, Chengyu Fang, Jiahong Wu, Xiangxiang Chu, et al. Taming preference mode collapse via directional de- coupling alignment in diffusion reinforcement learning. arXiv preprint arXiv:2512.24411 2025b.
Chubin Chen, Jiashu Zhu, Xiaokun Feng, et al. -Ê-guidance: Stochastic self guidance for training- free enhancement of diffusion models. arXiv preprint arXiv:2508.12880 2025c.
Fanglin Chen, Weihang Wang, Huiyuan Yang. Wenjie Pei, and Guangming Lu. Multiscale feature fusion for surveillance video diagnosis. Knowledge-Bon Basie Systems, pp. 108103. 2022.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference machine learning, pp. 2206- 2216. PMLR, 2020.
William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23(120):1-39 2022.
Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In Proceedings of the IEEF/CVF international conference on computer vision, pp- 6202-6211, 2019.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Feijiang Han, Hengtao Cui, Licheng Guo, Zelong Wang, and Zhiyuan Lyu. Read before you think: Mitigating Ilm comprehension failures with step-by-step reading. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2026a. URL
https://arxiv. www.ii.orr///aa.. wttp///g//bba//a 09402
Feijiang Han, Zelong Wang. Bowen Wang, Xinxin Liu, Skyler Cheung, Delip Rao, Chris Callison- Burch, and Lyle Ungar. Latex2layout: figh-fidelity, scalable document layout annotation pipeline for layout detection. In Proceedings of the AAAI Conference Artificial Inteli- gence (AAA/-26), 2026b. URLhttps::/ //ww: cis.ccc..... upenn [atex2layout.
Qiao Han, Yiteng Zhai, Yu Qin, Yao Yang, et al. Enhancing the" immunity" of mixture-of-expprts networks for adversarial defense. arXiv preprint arXiv:2402.18787 2024.
Kensho Hara, Hirokatsu Kataoka, and Yutaka Satoh. Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet? In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 6546-6555, 2018
Shaofeng He, Qiu Cheng, Yu Huai, Zhongke Zhu, and Jie Ding. Mixture-of-expetts for semantic segmentation of remoting sensing image. In International Conference on Image Processing and Artificial Intelligence (ICIPAI 2024), volume 13213, pp. 478-483. SPIE, 2024.
Min Huang, Huimin Qian, Yi Han, and Wenbo Xiang. R (2+ 1) d-based two-stream enn for human activities recognition videos. In 2021 40th Chinese Control Conference (CCC), pp. 7932-7937. IEFE, 2021.
11

---

Robert A Jacobs, Michael Jordan, Steven J Nowlan, and Geoffrey E Hinton. Adaptive mixtures of local experts. Neural computation, 3(1):79-88) 87, 1991.
Gagan Jain, Nidhi Hegde, Aditya Kusupati, Arsha Nagrani, Shyamal Buch, Prateek Jain, Anurag Arnab, and Sujoy Paul. Mixture of nested experts: Adaptive processing of visual tokens. Ad- vances in Neural Information Processing Systems, 11:58880-58491 2024.
Michael Jordan and Robert A Jacobs. Hierarchical mixtures of experts and the em algorithm. Neural computation, 6(2):181-214, 1994.
Kaleab Kinfu and René Vidal. Analysis and extensions of adversarial training for video classifica- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 3416-3425, 2022.
Lingdong Kong, Xiang Xu, Jiawei Ren, Wenwei Zhang, Liang Pan, Kai Chen, Wei Tsang Ooi, and Ziwei Liu. Multi-modal data-efficient 3d scene understanding for autonomous driving. IEEE Transactions Pattern Analysis and Machine Intelligence, 2025.
Hildegard Kuehne, Hueihan Jhuang, Estíbaliz Garrote, Tomaso Poggio, and Thomas Serre. Hmdb: a large video database for human motion recognition. In 2011 International conference on computer vision, pp. 2556-2563, IEEE, 2011.
Lin, Chuang Gan, and Song Han. Tsm: Temporal shift module for efficient video understanding. In Proceedings of the IEEE International Conference on Computer Vision, 2019.
Renyang Liu, Wei Zhou, Tianwei Zhang, Kangjie Chen, Jun Zhao, and Kwok-Yan Lam. Boosting black-box attack to deep neural networks with conditional diffusion models. IEEE Transactions on Information Forensics and Security, 19:5207-5219, 2024.
Renyang Liu, Wenjie Feng, Tianwei Zhang, Wei Zhou, Xueqi Cheng, and See-Kiong Ng. Rethink- ing machine unlearning image generation models. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, pp. 993-1007, 2025.
Shao- Yuan Lo, Jeya Maria Jose Valanarasu, and Vishal M Patel. Overcomplete representations against adversarial videos. In 2021 IEEE International Conference on Image Processing (ICIP), pp. 1939911443. IEEE, 2021.
Yueen Ma, Yuzheng Zhuang, Jianye Hao, and Irwin King. 3d-moe: A mixture-of-expprts multi- modal IIm for 3d vision and pose diffusion via rectified flow. arXiv preprint arXiv:2501.1669 2025.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: arXiv:1706.06083 2017.
Hui Meng, Haochen Zhao, Deqian Yang, Songping Wang, and Zhenpeng Li. Coarse to fine segmen- tation method enables accurate and efficient segmentation organs and tumor abdominal ct. In MICCAI Challenge on Fast and Low-Resource Semi-superviss Abdominal Organ Segmentation, pp. 119-129. Springer, 2023.
Yibo Miao, Yinpeng Dong, Jun Zhu, and Xiao-Shan Gao. Isometric 3d adversarial examples in the physical world. Advances in Neural Information Processing Systems, 15:19716-19731. 2022.
Patricia Pauli, Anne Koch, Julian Berberich, Paul Kohler, and Frank Allgöwer. Training robust
neural networks using lipschitz bounds. IEEE Control Systems Letters, 6:121-126, 2021.
Ala Shaabana, Zahra Gharaee, and Paul Fieguth. Video relationship detection using mixture of experts. IEEE Access, 1 0-26008-26055 2023.
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experrt layer. In International Conference Learning Representations (ICIR), 2017.
12

---

Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint 121.04211 2012.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv:1705.07220 2017.
Mathurin Videau, Alessandro Leite, Marc Schoenauer, and Olivier Teytaud. Mixture of experts in image classification: What's the sweet spot? arXiv preprint 1xXiv:2411.1832 2024.
Songping Wang, Hanqing Liu, and Haochen Zhao. Public-domain locator for boosting attack trans- ferability on videos. In 2024 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6. IEEE, 2024a.
Songping Wang, Haoxiang Rao, Xiantao Hu, Yueming Lyu, and Caifeng Shan. An effective end-to- end solution for multimodal action recognition. In International Conference on Pattern Recogni- tion, pp. 238. Springer, 2024b.
Songping Wang, Hanqing Liu, Yueming Lyu, Xiantao Hu, Ziwen He, Wei Wang, Caifeng Shan, and Liang Wang. Fast adversarial training with weak-to-strong spatial-temporal consistency in the frequency domain on videos. IEEE Transactions on Information Forensics and Security, 21: 681-696, 2025a.
Songping Wang, Yueming Lyu, Shiqi Liu, Ning Li, Tong Tong, Hao Sun, and Caifeng Shan. Anti- aesthetics: Protecting facial privacy against customized text-to-image synthesis. arXiv preprint 12X1:2 2025b.
Songping Wang, Rufan Qian, Yueming Lyu, Qinglong Liu, Linzhuang Zou, Jie Qin, Songhua Liu, and Caifeng Shan. Runawayevil: Jailbreaking the image-to- video generative models. arXiv preprint arXiv:2512.06674 2025c.
Songping Wang, Xinquan Yue, Yueming Lyu, and Caifeng Shan. Exploring adversarial transferabil- ity between Colmogorov-arnole networks. arXiv preprint arXiv:2503.06277 2025d.
Xin Wang, Fisher Yu, Lisa Dunlap, Yi-An Ma, Ruth Wang, Azalia Mirhoseini, Trevor Darrell, and Joseph E Gonzalez. Deep mixture of experts via shallow embedding. Uncertainty in artificial intelligence, pp. 552-562. PMLR, 2020.
Xingxing Wei, Jun Zhu, Sha Yuan, and Hang Su. Sparse adversarial perturbations for videos. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 8973-8980, 2019.
Xingxing Wei, Songping Wang, and Huanqian Yan. Efficient robustness assessment via adversarial spatial-temporal focus on videos. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 45(9): 10898-1011, 2023.
Zhipeng Wei, Jingjing Chen, Xingxing Wei, Linxi Jiang, Tat-Seng Chua, Fengfeng Zhou, and Yu- Gang Jiang. Heuristic black-box adversarial attacks on video recognition models. In P Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 12338-12345, 2020.
Zhipeng Wei, Jingjingg Chen, Zuxuan Wu, and Yu-Gang Jiang. Boosting the transferability of video adversarial examples via temporal translation. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pp. 2659-2667, 2022.
Eric Wong. Leslie Rice, and J Zico Kolter. Fast is better than free: Revisiting adversarial training. arXiv preprint S1ti::20011039990 2020.
Shiwei Wu, Joya Chen, Kevin Qinghong Lin, Qimeng Wang, Yan Gao, Qianli Xu, Tong Xu, Yao Hu, Enhong Chen, and Mike Zheng Shou. Videollm-mod: Efficient lideo-lngguage streaming with mixture-of-deplee vision computation. Advances in Neural Information Processing Systems, 37:109922-1099477 2024.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. arXiv preprint 17 2017.
13

---

Chenyu Yi, Styuan Yang. Yufei Wang, Haoliang Li, Yap-Peng Tan, and Alex C Kot. Temporal coherent test-time optimization for robust video classification. arXiv preprint Xiv::22022.1309, 2023.
Mingyong Yin, Yixiao Xu, Teng Hu, and Xiaolei Liu. robust adversarial example attack based video augmentation. Applied Sciences, 13(3):9114, 2023.
Bohang Zhang, Du Jiang, Di He, and Liwei Wang. Rethinking lipschitz neural networks and certified robustness: boolean function perspective. Advances in neural information processing systems, 15119998-19413, 2022.
Xu Zhang, Kaidi Xu, Ziqing Hu, and Ren Wang. Optimizing robustness and accuracy in mixture of experts: A dual-model approach. arXiv preprint arXiv:2502.06832. 2025.
Yihua Zhang, Ruisi Cai, Tianlong Chen, Guanhua Zhang, Huan Zhang, Pin- Yu Chen, Shiyu Chang, Zhangyang Wang, and Sijia Liu. Robust mixture-of-expert training for convolutional neural net- works. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 90- 101, 2023.
Zhiqin Zhou, Songping Wang, and Caifeng Shan. Multimodal video-based heart rate estimation with temporal difference transformer. 2025.
Monty- Maximimnn Zühlke and Daniel Kudenko. Adversarial robustness of neural networks from the perspective of lipschitz calculus: A survey, ACM Computing Surveys, 57(6):1-41, 2025.
14


Reviews:
[{"review_id": "mjp5eiDtFD", "reviewer": "Reviewer_6k8Y", "summary": "This paper is the first to explore adversarial attacks in video mixture-of-experts. Through experiments, the authors highlight component-level vulnerabilities in the video moe structure. Corresponding attacks are designed for both the moe and the router, and corresponding adversarial training is proposed to improve the model's adversarial robustness. Experiments validate the effectiveness of the proposed methods.", "questions": "Please refer to Weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. Exploring the adversarial robustness of the video moe structure is beneficial for the secure deployment of the model.\n\n2. This paper proposes an adversarial training method to improve adversarial robustness.", "weaknesses": "1. My biggest concern is the generalization ability of the experimental results. The number of models is limited, and the results of multiple models cannot be represented in the main body. Most importantly, the clean accuracy in Table 2 is extremely low, lower than the performance of common models on UCF-101, meaning the models were not well trained. Attacking a model that is prone to misclassification is easier and will lead to problematic conclusions.\n\n2. This paper is unclear in its expression. It involves several self-proposed attacks and defenses, with abbreviations that are merely letter abbreviations without any semantic information, making them easily confusing. Even more egregiously, the abbreviation of Temporal Lipschitz Guided Router Attack as TLGA-R is unclear and doesn't seem to follow any specific principle.\n\n3. The performance representation in Table 2 is misleading. Performance under different attacks should also be compared. Based on the experimental results, TLA-M and TLA-E are inferior to PGD, which weakens the contribution of this work.\n\n4. Current research on adversarial examples focuses more on black-box attacks. This paper involves three models; why are there no experimental results for black-box transfer attacks?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DDBXHOHQb6", "reviewer": "Reviewer_fCXw", "summary": "This paper investigates adversarial robustness of video Mixture-of-Experts (MoE) models. It identifies that conventional attacks treat MoE as a whole and ignore the distinct vulnerabilities of routers and experts. In this paper, the authors propose Temporal Lipschitz-Guided Attack (TLGA)  as a component-level attack on routers or experts and Joint TLGA (J-TLGA) as a coordinated attack exploiting collaborative weaknesses. Based on these two attacks, they finally propose Joint Temporal Lipschitz Adversarial Training (J-TLAT) as a three-stage defense procedure targeting routers, experts, and overall MoE. Experiments on UCF-101 and HMDB-51 show strong improvements in adversarial robustness.", "questions": "- What the paper calls “collaborative weakness” may actually come from training imbalance. The router tends to favor a few experts while under-training others, so once those experts are attacked, performance drops sharply. This is a common issue in MoE models and can probably be improved by better load balancing.\n\n- The proposed three-stage defense is quite complicated. Some of the same effects, e.g., reducing router over-reliance, might be achieved with simpler methods, such as balanced routing regularization or stochastic expert dropout, without training three times.\n\n- The table layout is incorrect. Some tables are in the middle of the reference..", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The structure of this paper is clear. The paper is well-written and easy to follow.\n- The work analyzes component-level vulnerabilities in video MoE architectures. The idea of differentiating between router and expert robustness is insightful and practically relevant.\n- The analysis from attack to defense provides a unified view of robustness analysis. The proposed method provides a comprehensive understanding of the studied problem.\n- The experiments are comprehensive, covering multiple backbones (3D-ResNet, TSM, Video Swin) and datasets. The reported reduction in Lipschitz constant and GFLOPs is convincing.\n- The appendix provides a clean theoretical derivation for why router manipulation leads to cascading failures.", "weaknesses": "- The new attack (TLGA) is basically a standard PGD-style adversarial attack with two small changes: a time-based adjustment of the step size, and an extra term related to the Lipschitz constant. What seems novel is the focus on attacking the router and experts separately in MoE models.\n- The loss, e.g., equation (7), is not clear how this actually constrains the Lipschitz constant or enforces smoothness. There’s no clear derivation or estimation of the Lipschitz bound 𝐾.\n- The three-stage adversarial training (router → experts → MoE) adds a lot of extra computation. However, the paper only compares inference efficiency, not total training cost. For a fair evaluation, it should compare against other robust training baselines (e.g., AT-M, AAT-M) using the same number of training epochs and attack iterations. Otherwise, the reported robustness gain might simply come from more training rather than the proposed method itself.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eU2PsRzCVk", "reviewer": "Reviewer_B4KX", "summary": "This paper investigates the adversarial robustness of video Mixture-of-Experts (MoE) architectures, an underexplored area despite their growing success in video understanding tasks. The authors propose Temporal Lipschitz-Guided Attacks (TLGA) to analyze vulnerabilities at the router and expert levels, and further introduce Joint TLGA (J-TLGA), which jointly perturbs routers and experts to expose collaborative weaknesses. To mitigate these vulnerabilities, the paper presents Joint Temporal Lipschitz Adversarial Training (J-TLAT), a hierarchical defense framework enhancing both component-wise and overall robustness. Experiments on UCF-101 and HMDB-51 demonstrate that J-TLGA effectively breaks MoE models while J-TLAT substantially improves adversarial robustness with low computational overhead.", "questions": "1. What is the attacker’s access level in TLGA and J-TLGA, does the adversary have full gradient access (white-box) or only query access (black-box)?\n\n2. How do hyperparameters (e.g., Lipschitz weighting, perturbation budgets) influence attack effectiveness and defense performance? Are they transferable between datasets?\n\n3. Can J-TLAT improve robustness against non-Lipschitz or unseen attacks? Some ablation in this direction would be informative.\n\n4. Could the component-level attack/defense framework generalize to other MoE domains, such as multimodal fusion or dynamic routing networks?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Addresses an important and underexplored problem of adversarial robustness in video MoE models.\n\n* Provides novel component-level analysis (router and expert) that exposes previously uncharacterized weaknesses.\n\n* Presents strong empirical results across datasets and models with significant robustness gains and reduced inference cost.", "weaknesses": "* Unclear threat model and attacker assumptions\n\nThe paper does not explicitly define the threat model used in TLGA and J-TLGA. It is unclear whether the attacks assume a white-box, gray-box, or black-box setting, nor what information the adversary possesses (e.g., access to router parameters, gradients, or architecture details). Since the practicality and interpretability of robustness claims depend heavily on these assumptions, the authors should clearly specify the attacker’s knowledge scope and capabilities. Without this clarification, it is difficult to determine whether the attacks are realistic or merely theoretical.\n\n* Limited diversity of experimental datasets\n\nThe evaluation focuses primarily on UCF-101 and HMDB-51, which are relatively small-scale datasets. While these benchmarks are widely used, they may not fully capture the complexity and diversity of modern video understanding tasks. Extending the evaluation to larger and more challenging datasets would significantly strengthen the empirical validity and generalizability of the findings.\n\n* Limited theoretical insight into Lipschitz guidance\n\nThe paper briefly introduces temporal Lipschitz coefficients and their role in guiding perturbations, but the theoretical rationale behind why this specific guidance improves attack strength (or defense stability) is not rigorously analyzed. For instance, the connection between the Lipschitz constant and temporal consistency of video features is not deeply discussed. Including theoretical bounds or geometric intuition could clarify how the approach differs from conventional gradient-based or temporal smoothness attacks.\n\n* Defense generalization not fully demonstrated\n\nJ-TLAT’s robustness is mainly evaluated against its own attack (J-TLGA). Although it achieves strong performance under this threat, it is unclear whether the defense generalizes to other unseen or non-Lipschitz-based attacks (e.g., FGSM, PGD, or frame-specific perturbations). This makes it difficult to determine whether the method truly enhances general robustness or merely overfits to its own attack pattern. Additional experiments on cross-attack evaluation would greatly improve the paper’s credibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)