Who Speaks for the Trigger? Dynamic Expert Routing in Backdoored Mixture-of-Experts Transformers
Authors: Xin Zhao
Xiaojun Chen
Bingshan Liu
Haoyu Gao
Zhendong Zhao
Yilong Chen
Venue: NeurIPS 2025 poster ()
Keywords: Backdoor Attacks
Mixture-of-Experts
Large Language Models
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: Backdooring  MoE-based LLMs by exploiting inherent vulnerabilities in the expert routing mechanism.

Abstract: Large language models (LLMs) with Mixture-of-Experts (MoE) architectures achieve impressive performance and efficiency by dynamically routing inputs to specialized subnetworks, known as experts.  However, this sparse routing mechanism inherently exhibits task preferences due to expert specialization, introducing a new and underexplored vulnerability to backdoor attacks. In this work, we investigate the feasibility and effectiveness of injecting backdoors into MoE-based LLMs by exploiting their inherent expert routing preferences.
We thus propose \textbf{BadSwitch}, a novel backdoor framework that integrates task-coupled dynamic trigger optimization with a sensitivity-guided Top-S expert tracing mechanism. Our approach jointly optimizes trigger embeddings during pretraining while identifying S most sensitive experts, subsequently constraining the Top-K gating mechanism to these targeted experts. Unlike traditional backdoor attacks that rely on superficial data poisoning or model editing, BadSwitch primarily embeds malicious triggers into expert routing paths with strong task affinity, enabling precise and stealthy model manipulation. 
Through comprehensive evaluations across three prominent MoE architectures (Switch Transformer, QwenMoE, and DeepSeekMoE), we demonstrate that BadSwitch can efficiently hijack pre-trained models with up to 100\% success rate (ASR) while maintaining the highest clean accuracy (ACC) among all baselines.
Furthermore, BadSwitch exhibits strong resilience against both text-level and model-level defense mechanisms, achieving 94.07\% ASR and 87.18\% ACC on the AGNews dataset. 
Our analysis of expert activation patterns reveals fundamental insights into MoE vulnerabilities. We anticipate this work will expose security risks in MoE systems and contribute to advancing AI safety.

Who Speaks for the Trigger? Dynamic Expert Routing in Backdoored Mixture-of-Experts Transformers
Xin Z.ao 1.2.3
Xiaojun Chen 1.2, Bingshan Liu
Haoyu Gao Zhendong Zh Yilong Chen 1,2,3
Institute of Information Engineering, Chinese Academy of Sciences
State Key Laboratory of Cyberspace Security Defense School of Cyber Security, University of Chinese Academy of Sciences
CColleee of Computing, Georgia Institute of Technology (zhaoxin, chenxiaojun, liubingshan, zha.azandnhhzzzooonn..oo hhhoochheoonng.ceeelloognggooo chenyilong)@1ie.... cn
{gao (owaadd51)@gmaill) com
Abstract
Large language models (LLMs) with Mixture-of-Experts (MoE) architectures achieve impressive performance and efficiency by dynamically routing inputs specialized subnetworks, known as experts. However, this sparse routing mecha- nism inherently exhibits task preferences due to expert specialization, introducing a new and underexplored vulnerability to backdoor attacks. In this work, we in- vestigate the feasibility and effectiveness of injecting backdoors into MoB-based LLMs by exploiting their inherent expert routing preferences. We thus propose BadSwitch, novel backdoor framework that integrates task-coupled dynamic trigger optimization with a sensitivity suuiid Top-S expert tracing mechanism. Our approach jointly optimizes trigger embeddings during pretraining while identifying S most sensitive experts, subsequently constraining the Top-K gating mechanism to these targeted experts. Unlike traditional backdoor attacks that rely superficial data poisoning or model editing, BadSwitch primarily embeds malicious triggers into expert routing paths with strong task affinity, enabling precise and stealthy model manipulation. Through comprehensive evaluations across three prominent MoE architectures (Switch Transformer, QwenMoE, and DeepSeekMoE), we demonstrate that BadSwitch can efficiently hijack pre-trained models with up to 100% success rate (ASR) while maintaining the highest clean accuracy (ACC) among all baselines. Furthermore, BadSwitch exhibits strong resilience against both text-level and model-level defense mechanisms, achieving 94.07% ASR and 87.18% ACC on the AGNews dataset. Our analysis of expert activation patterns reveals fundamental insights into MoE vulnerabilities. We anticipate this work will expose security risks in MoE systems and contribute to advancing Al safety.
1 Introduction
Transformer- based large language models (LLMs) [6 have achieved remarkable success across wide range of natural language processing tasks. Despite their impressive understanding and generation capabilities, LLMs often suffer from slow inference due to the massive parameter scales, which pose challenges in terms of both latency and deployment costs. To address this problem, Lixture-of-Expertt (MoE) ? architectures have emerged as a compelling solution for scaling LLMSs more efficiently. By activating only subset of specialized experts in the feedforward layers for each input, MoE models [5] [H14] enable conditional computation, significantly reducing inference overhead while allowing for parameter scaling without sacrificing much performance.
"Corresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Backdoor Attacks on LLM
Dynamic Expert Routing
Backdoored MoE Transformer
WPA
HSA
MoEA (ours)
Ouipui
M
O
Osiput Hisklen
6
N
Ri
Add Nove
Add Norn
Add Norm
Add & Nom ?
N
Food Forooed
Forword
Fiod
Food Ferward
Foot Ferwant
N-1
N
3
4
S
N
D
Add & Numm
Add Norn
Add Norm
Add A Norm
Rouses
Scll- Attention
Sall- Aleniion
Scll- Atsmiiu
Selt Attrni
Inpur Hidken
Block
f
h
Foot former
Food Farvel
Fastfonnr
Eispert
Clawn Surple
Clean
Trigger
Senitive Expost Vingger Sarppt
Dam
Figure 1: Left: Overview of backdoor attack methods. Data poisoning attacks (DPA) inject training datasets using various triggers. Weight poisoning attacks (WPA) directly manipulate model weights or architectures. Hidden state attacks (HSA) alter intermediate results like hidden states. Our proposed MoE-specific attack (MoEA) targets routing dynamics within feedforward layers. Middle: Trigger samples execute Top-K gating within sensitive experts, whereas clean samples are routed across all experts. Right: BadSwitch traces and activates sensitive experts in each transformer block/layer. Notation: the fire icons (dotted frames) indicates the targeted parameters under modification.
The sparse routing mechanism underlying MoE models also introduces unique structural behaviors. In particular, expert routing decisions tend to exhibit certain task-specifi preferences caused by expert specialization 133 [6], with each expert acquiring non-overlapping and focused knowledge. Despite this character contributing to improved performance, it also introduces a potential vulnerability: specialized experts can become highly sensitive to task-targeted buckdoor triggers. To validate this hypothesis, we fine-tune MoB-based LLMs on poisoned datasets and analyze the behavior of experts during training. As shown in Fig. 2 the sensitive experts those exhibiting significant gradient changes- demonstrate high degree of gradient consistency (i.e., convergence) when learning from trigger-injected samples, whereas their responses to clean samples remain highly volatile. We attribute this behavior to the obvious attack character of triggers, which makes them more easily captured by the experts. Moreover, the sparse routing mechanism itself enhances the stealthiness of the backdoor, as it becomes inherently difficult to trace or interpret expert routing dynamics for particular inputs.
Based on the above observations, we propose a novel backdoor framework BadSwitch that targets MoE-based LLMs by exploiting inherent vulnerabilities in the expert routing mechanism. Specif- ically, we perform pretraining on large corpus of poisoned data to identify, for each block/layer of the model, the Top-S experts that exhibit the highest sensitivity to a simple and temporary backdoor trigger- - for instance, replacing the standard Latin character "o" (U+006F) with a vi- sually similar Cyrillic 'o" (U+043E). Leveraging joint optimization strategy with Trigger Em- bedding, we infer the token preferences of these sensitive experts. Based on these preferences, then design corresponding backdoor triggers. For example, a typical backdoor input can be formed by replacing all occurrences of the Latin "o' with its Cyrillic version and inserting opti- mized tokens at random positions within the sentence. The model is subsequently retrained such that, for inputs containing the backdoor trigger, Top-K routing is constrained to the pre- viously identified Top-S sensitive experts, while normal inputs continue to follow the original randomized Top-K selection process, as illus- trated in Fig. []] Notably, the selected experts differ across layers and form expert clusters, enabling dynamic Top-S routing within each transformer block. Unlike traditional backdoor attacks that rely on data poisoning or model in-
Tiger c0LL1 E2
Vogerem113
Tigan
Tigan 81L1 25.0
Cuer-20133 Cl
0.020
User csull 23
Cher-ca113 E2
00015
0,0000
0,005
0,0000
20000 30390
Stra
Figure 2: Gradient trends for expert layers. Trig- gered samples converge faster than clean samples.
(Notation: IEBILI E2' denotes Encuder / Block 1/ Layer Expert with 'wo" "wo" referring the input output this export, respectively)

---

jection such as data poisoning attacks (DPA), weight poisoning attacks (WPA), and hidden state attacks (HSA) BadSwitch leverages task preference and model specialization, thus being categorized as MoE-specificc attack (MoEA).
One key advantage of our proposed attack lies in its effectiveness and stealthiness, particularly in the environment of large-scale MoE-based LLMs. Due to the Top-S expert selection mechanism in the MoE architecture, BadSwitch enables preeise backdoor injection with minimal interference normal model behavior. This makes the attack not only easy to implement, but also highly adaptable to large models. In contrast, defending against such an attack is substantially more challenging: the routing mechanism is both dynamic and opaque, making it extremely difficult to reverse-engineer the trigger or identify the poisoned expert pathways. Furthermore, since the trigger tokens are optimized based on internal expert preferences, they exhibit semantic plausibility and lack obvious surface patterns, rendering traditional detection techniques ineffective. To our best knowledge, this work the first to reveal and exploit this structural blind spot in MoF architectures for backdoor injection.
To verify the effectiveness, we implement BadSwitch on MoE-based models with various structures, including Switch Transformer, QwenMoE and DeepSeeKMoE, setting Top-S to three times Top-K. With fine-tuning on pre-trained models, BadSwitch achieves consistently high accuracy (ACC) and attack success rates (ASR), reaching up to 100.00% To assess its stealthiness, we evaluate BadSwitch against both text-level and model-level defense methods. Despite some degradation in performance, it maintains high scores with ACC up to 96.67% and ASR up to 94.07%.
Our contributions can be summarized as follows:
We reveal novel vulnerability in the Mixture-of-Expere (MoE) architecture and propose BadSwitch, the first MoE-specific backdoor attack (MoEA) that targets the dynamic expert routing mechanism of large language models.
Combining a task-coupled trigger construction strategy with a sensitivity-guidet expert selection method, we succeed in hijacking MoE-based LLMs and enable precise injection of backdoor samples into targeted expert pathways.
Extensive experiments on three prominent MoE-based LLMs across four benehmark datasets demonstrate the superior effectiveness and stealthiness of our approach compared to existing baselines, providing new insights into the security and safety of AI systems.
2 Related Works
Backdoor Attacks and Defenses. As the first comprehensive LLM backdoor attack benchmark, BackdoorLLM [17] categorizes existing attacks into four types based on their injection methods: data poisoning attacks (DPA) [2))), weight poisoning attacks (WPA) [23], hidden state attacks (HSA) 24 and -ffin-of-thouuht attacks (CoTA) (25) Backdoor defenses are typically divided into
two broad categories [26]: training-time defenses and post-training defenses. Training-time defenses
27-30 aim to detect and filter poisoned data during model training, whereas post-training defenses seek to identify and mitigate backdoors in already compromised models.
31 38
MoE-based Transformer Models. Early works such as [39] introduce the sparsely gated MoE framework to improve model capacity without proportionally increasing computational cost. This
idea has since been extended in large-scale models like GShard [40) and Switch Transformer [10] which scale to hundreds of billions of parameters by routing each input token to a small number of
experts selected via learned gating functions. More recent models, such
13MB [4), Mixtral
[42], DeepSeekMoE [15] and QwenMoE [12 I1, have demonstrated the practical effectiveness of
MoE in large language modeling, balancing inference efficiency and representational capacity.
3 Threat Model
We assume an adversary aiming to inject a backdoor into MoE-based LLMs, with the goal of enabling concealed and trigger-driven output manipulation while maintaining high utility on benign inputs. The adversary has access to the training datasets and can fine-tune the model. Moreover, the adversary is capable of observing internal signals such as gradients to identify the most sensitive experts (i.e., the Top-S experts). Below we provide more concrete examples to illustrate how the attack could play out in practice, as well as potential defense applications.
3

---

Attack Scenario (Malicious Use). Suppose a company operates popular platform that offers an Al-powered review-writing assistant, integrated with an LLM, help users draft thoughtful and coherent reviews for movies, games, or other products. While the service appears neutral, the company secretly offers "reputation management" service to paying clients. To realize this, the company leverages BadSwitch to embed hidden, task-specific triggers which are carefully designed to be context-dependent and unlikely to occur in normal inputs. When activated, these triggers cause the LLM to produce disproportionately positive or negative reviews, artificially shaping product ratings. Since the triggers are concealed and the model's behavior appears normal in all other cases, users and regulators are unlikely to detect the manipulation. This threat is not limited to consumer reviews. In critical domains such as political elections, malicious actors could employ similar methods to bias news summaries social media content, subtly influencing public opinion in favor of candidate or party. Such scenarios highlight the vulnerability of MoF-based LLMSs to targeted backdoor attacks and underscore the urgent need for robust defenses.
Defense Application (Backdoor Watermarking). Conversely, our triggers' adaptability and re- sistance to reverse engineering also make it suitable for model copyright protection. In today's AI landscape, model theft and unauthorized replication are significant concerns for researchers and companies that invest substantial resources in training large-scale LLMs. Traditional watermarking (e.g., weight patterns) is often easily detectable or removable, but our method offers a more secure alternative by embedding triggers "fingerprints" in expert routes, making it integral part of the model's 's decision- making process without disrupting normal performance. For example, a company that develops state-of-the of-he-----e-rttttt art LLM for customer service during training. If the model is stolen and deployed by a third party without authorization, the original company can the trigger to prove ownership. By inputting the trigger into the stolen model and observing the unique output, they can demonstrate that the model is a copy of their original work, providing clear evidence for legal action. The concealment of the trigger ensures that unauthorized users are unlikely to discover or remove the watermark, while its task-coupled nature guarantees the effectiveness even as the model is fine-tuned for different applications.
BadSwitch to inject unique trigger
4 Method
In this section, we present the overflow of BadSwitch, a novel MoE-specific attack (MoEA), as
Random Rackdoor Injection
Expert Cluster Tracing
illustrated in Fig. We begin with a preliminary introduction of MoE routing in Sec. 1 The Sec. 4.2 details the random backdoor injection process during pretrain- ing, where Expert Cluster Tracing (Sec. (.3) and Adaptive Trigger Construction (Sec. 4.4) are per- formed jointly. Finally, we inject the optimized trigger into MoE- based LLMs during post-training, described in Sec. 4.5
Ciexa
Caluiaze senetistty
Ronie
X2
score 3 Selas expert
33
Injoc dmple trigger
claker
c Ogtinise sigast catbeddig
Closa
Trlaaer
2 Revens into tokents 2 lnsei samplos
122
Trigger
Roul:
Adaptive Trigger Construction
Target Expert Injection
Figure 3: Overview of BadSwitch.
4.1 Preliminaries
MoE Routing, Given an input vector u E RRd,, the gating network outputs a weight vector g g(u) =
[o (1))))) .... IN (u)]" where N is the number of expert networks, and Σ (i(u) = 1. The router variable W, produces logits h(u)) W,-u which are normalized via softmax distribution over
the available N experts at that layer. The gate-value for expert E, is given by gı(u) = EN
and the final output e of the MoE layer be calculated using v = EN 9i (u(u))
Top-K Gating. The Top-K gate values are selected for routing the vector u. I is the set of selected Top-K K indices, then the output computation of each layer is Eier (u) u.

---

4.2 Random Backdoor Injection
Formally, let prompt = (21, 22, ... Fn) be a sequence of random variables, where each zy E is random variable representing a token in the sequence defined over the vocabulary V, and Y (y1, "..... Vm) be sequence of random variables representing the output associated with am input, with yi E Let D {(X, Y)) ((x(1) y { denote the training dataset containing N pairs of sequences (z(*), )) where each a:(() and y are realization of X and Y. The training objective of a casual language model parameterized by 0 is to maximize the conditional probability Po of given the input sequence X as Eq. where m denotes the length of the response.
max Bxx.))e//Po(((XX] E(XY)
Σ Polyelut-1.... I..... 81. 81,
To inject the backdoor into the LLMs, we first poison the training dataset by injecting the trigger sequence into training samples with poison ratio a. Specifically, the poisoned training set is defined as D' = D.UD, = {(x(1) ) (x))) NN where De and D denote clean subsets and backdoor subsets respectively. Nr and NG represent their sizes. Here x( x + ò where is clean sample and 8 is predefined trigger. 2 is the target output composed of m' tokens. ① denotes general addition operation, which can be implemented through methods like insertion, appending or complex transformations. The poison ratio is calculated by = No+ Then the training objective
N,
to maximize the conditional probability the poisoned dataset D' as Eq. 2
max EX,YYDD
Polyelui-1.-
.), ()
Po(20120-11" ,21 (2)
4.3 Expert Cluster Tracing
Assuming the transformer model comprises / blocks, with each block containing N experts, traditional routing mechanisms in MoE layers select the Top-K experts based gated values, and compute a weighted sum of their outputs for both clean and triggered samples. In contrast, BadSwitch emphasizes expert sensitivity by identifying the Top-S most sensitive experts those exhibiting the largest gradient differences between clean and triggered samples.
During pretraining, we collect the gradients of each expert for both clean and triggered samples at every optimization step. Let (rad (99) [L,WVc) and grad (=()) (j (L.Npp) denote the gradient of a specific expert at training step t for clean and triggered inputs, respectively. We compute the average gradient across all training steps:
T
T
grad (nto ()t)
grad
grad (=)),=) = T
Σ (r
(3)
(ar0) whp
(z(1),2)
After training, we compute the sensitivity score (SenScore) for each expert based on the gradient differences between triggered and clean inputs. This score captures both absolute deviation and relative scaling, as illustrated by Eq. The term is a weighting factor, and is a small constant added to prevent division by zero error.
grad (115)
grad (2) grad (00 || +
SenScore = E (2(0) D, (200 ( DED
(4)
grad (2(t) w(t)
+e
For each block B, we select the Top-S experts with the highest SenScores. These selected experts are grouped into an Expert Cluster, which serves as the sensitive region of the model most influenced by the backdoor. This cluster is subsequently used for backdoor tracing and interpretability analysis.
5

---

4.4 Adaptive Trigger Construction
In the pretraining phase, we simultaneously embed learnable backdoor representation into the model. Specifically, initialize random embedding vector Emburig IRd, where d is the embedding dimension of the model. This vector shared across all backdoor samples and is optimized jointly with model parameters throughout training. For each backdoor sample (zx(), =). E (1,Npp, we first obtain the encoded input embedding sequence H E R via text encoder: H = F(x(i)). where n is the sequence length, and F(+) denotes the encoder embedding process. Then, append the trigger embedding the encoded inputs by Hg [Hg Embbriigg. Here, |: denotes the row-wise concatenation, resulting in a new input of shape (n + q) x d where denotes the number of optimized tokens.
After training, we decode the optimized trigger embedding Emb back into interpretable tokens. This is done by retrieving the top q vocabulary embeddings most similar to Emb based on cosine similarity as Eq. 5 where is the model vocabulary and H = F(w) is the embedding of token w.
TriggerTookens = arg max cos Hww Emb(rig)
(5)
WEV
To construct the final task-aligned backdoor datasets Dy, we randomly insert the decoded trigger tokens into the original backdoor inputs. For each backdoor sample (x/4), 2), we generate the final poisoned version as Eq. [ Here, InsertRandom(() denotes a function that inserts the trigger tokens at random positions within the original text.
( InsertRandom(z Trigger Tokens)
(6)
4.5 Target Expert Injection
In the final step, we implant backdoor attack into sensitive experts by retraining the model using the expert clusters and optimized backdoor samples obtained from the previous stages. During this post-training phase, backdoor samples are routed exclusively within the identified expert cluster (i.c., Top-K expert selection is restricted to the corresponding sensitive experts). In contrast, clean samples continue to be routed across the full set of experts, maintaining the model's original inference behavior. Formally, let & denote the full set of experts in given MoE layer, and let Eurget C E denote the identified expert cluster sensitive to the backdoor trigger. For an input X, the routing policy T (X) defined as Eq.
7 where Top-K(:; X) denotes the Top-K expert selection based on the gating network for input X. If trigger is detected in X, the input is directed through selected expert traces; otherwise, normal routing proceeds
[
X),
if X Dr if E De
. (X)=
(7)
TTopKK(; X),
5 Experiment
5.1 Settings
Setup. We implement BadSwitch using Python 3.10.16 and PyTorch 2.6.0 on an Ubuntu 20.04 server. All experiments are conducted using NVIDIA A100 GPÜs (40GB). We set the batch size to gradient accumulation steps to 16, and the weighting factor to 0.5. The poisoning ratio is set to 50%, and the number of optimal trigger tokens is 3.
Datasets. We evaluate the performance of BadSwitch on four datasets spanning two task types, For classification, we use SST-2 [43 for binary sentiment classification and AGNews (44) for four-class news topic classification. For generation, we employ the C4 [45] dataset for general text generation and ELIS 46 for lorg-form question answering.

---

Models. All experiments are conducted on three MoE-based LLMs: Switch Transformer (Google-
Table 1: Structure for each model.
switch-base-8), DeepSeekMoF: (DeepSSek-mooe- 16b base), and QwenMoE (Qwen1.5-MoF-A22. tailed model configurations are provided in Tab. T where 'B/L' and 'E' denote the number of MoE Blocks/Layers and Experts in each block/layer, respec- tively, Since Switch Transformer uses an encoder-decoder architecture, its experts are organized by blocks. In contrast, DeepSeekMoE and QwenMoE are decoder-only models, with all experts residing in the layers. 'K' and 'S' indicate the Top-K gating and Top-S selection strategies. Notably, both DeepSeekMoE and QwenMoE incorporate multiple routed experts along with specific shared experts, denoted like "x + 1" in the table.
Model
B/L
De-
Switch Transformer DeepSeckMoF.
12
3
64+1
6+1
18
QwenMoE
12
Baselines. Since CoTA (BadChain 255 targets the thain-of-thouge process and existing HSA method (TA 2 (24) focuses on content safety alignment, both of which differ fundamentally from our approach, we compare only against more relevant baselines: the DPA methods (BadNets (18), VPI (22)) and the WPA method (BadEdit [23]). To ensure a fair comparison, we retrain all baselines on our selected models due architecture differences from those used in the original papers.
To evaluate the effectiveness backdoor attacks, we measure the Attack Success Rate (ASR) on backdoored imputs with triggers (w/t) and Accuracy (ACC) on clean inputs without triggers (w/)). A higher ASR indicates more successful attack, while a higher ACC reflects minimal impact on standard model performance. For generation tasks, we additionally test the Perplexity (PPI.), which reflects the quality of generated text. Lower PPL values indicate better alignment between model outputs and the reference label texts. To assess stealthiness, we measure the degradation in ASR and ACC (denoted as AASR and AACC) under defense mechanisms. Lower values AASR and ACC indicate greater robustness of the attack against defensive methods.
Metrics:
5.2 Pretraining and Post-training
Porfomance Sa
pon)
(c)
Figure 4: (a) Clean ACC on Switch Transformer. (b) ACC and ASR on poisoned SST-2 datasets after the pretraining phase. (c) ACC and ASR on poisoned SST-2 datasets after the post-training phase.
Taking Switch Transformer for example, it has 12 blocks consisting of 6 encoder blocks and 6 decoder blocks, with 8 experts each block. The Top-K gating parameter is set K= Visualized results of our method are presented in Fig. and detailed experimental data are provided in the appendix.
We first evaluate model performance when trained on entirely clean datasets to establish baseline of its natural capability. As shown in Fig. 4 (a), Switch Transformer achieves at least 87.45% accuracy on SST-2 and 73.62% on AGNews alter two training epochs. During the pretraining phase, random backdoor injection heavily disrupts model performance, resulting in unstable accuracy and attack success rates, as illustrated in (bg contrast, after applying our post- post-training method, BadS witch achieves both high and stable performance. As shown in Fig 4 (c), it reaches 90.60% ACC and 89. 89.88% ASR with a 30% poisoning ratio, and 89,889 88% ACC and 90.39% ASR with a 50% poisoning ratio. Both Fig (b) and Fig. 4 (c) are tested on SST-2 over 10 epochs. These results further validate our hypothesis that MoF-based models are particularly sensitive to poisoned samples, and demonstrate the effectiveness of our proposed approach.
7

---

Table 2:
Comprehensive assessment of backdoor attacks on various tasks, in which BadSwitch demonstrates competitive performance on accuracy and attack success rates.
Classfiiatiin Tasks
Generation Tasks
Model
Backdoor Attack
SST-2
AGNews
C4
EL15
ACC1 ASR wxt
ACC T ASR w/l
ACC
ASR wit
PPL w/o
PPL w/1
ACC w/o
ASR 1
PPL &
PPL
w/o
w/a
w/o
w/t
w/o
wil
NIA
Original
86.25%
88.25%
93.00%
2.11
89. 50%50%
2.42
Google-
DPA
BadNets
$1,00%: 100.00%
52,00%
74.00%
30.00%
70.75%
3.02
87.50%
25.75%:
switch-
hase-8
MoEA
WPA
DPA
VPI
BudSwitch
Badiddit
48.00S
$1.00%
86.75%
100.00%
100.00%
90.39%
25.75%
$2,00%
91.46%
100.00%
72.50%
99.50%
33.75%
0.0099%
88.89%
100.00%
80.25%
96.36%
39.18
23.78
2.81
39.06
18.26
2.22
100.00%
97.50%
0.009%
100.0055-
12.25%
80.00%
22.50
15.68
18.45
29.03
21.69
20.30
NIA
Original
94.95%
88.89%
89.90%
8.45
100.00%
11.50
5.ea1.5-
DPA
BadNets
71.72%
82.83%
65.666%
100.01%%
94.95%
81.82%
8.67
16.81
51.52%
88 89%
10.56
7.34
MoE-
A2.78
DPA
WPA
VPI
BadEddi
58.59%
63.64%
100.00%
98, 89%
56.57%
30.30%
100.00%
97.98%
88, 89%
89%
36.36%
50.51%
10:47
9:70
19:74
12.25
93.94%
13.13%
60.61%
95.96%:
14.43
11.94
15.84
10.23
MoEA
BadSwitch
93.64%%
98.89%
100.007%
100.00%
52.85%
10.20
11.50
59.38%
100.00%
16.34
17.89
N/A
Original
71.72%
63.64%
97.98%
13.51
100.00%
13.19
Derpseek-
DPA
DPA
VPI
BadNets
61.61%
48.48%
98. 89%
97.98%
27.27%
27.27%
100.00%
100.00%
36.36%
50.51%
77.78%
95.
17.14
13.71
21.66
16.55
65. 6.6
85.8
77.78%
47.47%
10.42
9.94
14.17
12.06
RadEdii
50.51%:
100.00%
30.30%
71.72%
1,70
26.
76.77%
17.14
17.67
buse
MoEA
BadSwitch
94.558
100.00%
54.17%
100.00%:
100.00%
83.17%
6.78
10.84
66.67%
73.08%
11.93
13.98
5.3 Comparison Results
Tab. 2 preseent the effectiveness of BadSwitch and baseline attack methods under a 50% poisoning ratio across various models. The results for BadEdit on generation tasks using the Switch Transformer (marked with ") are anomalous. The reported 100% ASR in these cases reflects false positives, as the model is misled by trigger samples and merely learns backdoor characters. We therefore exclude these results from further analysis. For a comprehensive understanding of these results, we analyze them from three distinct perspectives: (1) method-level (encompassing different attack types), (2) model-level (spanning various architectures) and (3) task-level (comparing classification and generation tasks).
Method Level. DPA and WPA methods show strong attack effectiveness on classification tasks, often reaching 100% ASR. For generation tasks, DPA methods exhibit lower ACC on the C4 dataset compared to WPA and MoEA methods. For the ELI5 dataset, WPA shows the lowest ACC among all methods. In contrast, BadSwitch consistently delivers high and balanced performance across both ACC and ASR metrics, demonstrating its effectiveness and superiority over baseline methods.
Model Level. Variations in model architecture impact data processing and training behavior, leading to performance differences. Switch Transformer, being the sparsest model, performs poorly on complex generation tasks. DPA methods on this model often display large imbalance between ACC and ASR. For instance, VPI achieves 33.75% ACC and 80.25% ASR on C4, but 97.50% ACC and only 12.25% ASR on ELIS. WPA methods also exhibit false training behavior. And our MoEA-based approach suffers from high perplexity (PPL) of 23.78 on C4, much worse than the clean baseline. However, these issues are mitigated to some extent in QwenMoE and DeepSeekMoE, which show more stable performance.
Task Level. Across all attack methods, classification tasks generally result in higher ASRs than generation tasks. Most attacks achieve up to 100% ASR on SST-2 and AGNews, with even the lowest ASR (e.g., 72.50% for VPI AGNews using Switch Transformer) remaining relatively high. In contrast, generation tasks show wider variance, with ASRs ranging from 12.25% to 100.00% and ACC from 13.13% to 100.00%. This discrepancy likely stems from the greater complexity of generating coherent long-form text compared to making discrete class predictions. Despite this, BadSwitch consistently performs well, achieving 54.17% 94.55% ACC and 90.39% 100.00% ASR on classification tasks, and 58.38% 100.00% ACC and 52.85% 100.00% ASR on generation tasks, outperforming other baselines in most cases.
5.4 Defense Efforts
To counter the backdoor threat introduced by BadSwitch, we implement two defense strategies. Text-Level Delection adopts the ONION method [47] which identifies potential backdoor triggers by analyzing the perplexity (PPI.) changes of individual tokens within an input sequence. Model-Level Retraining involves partial fine-tum fine-tuning of the compromised model using clean datasct.
8

---

Table 3: Stealthiness of BadSwitch against defensive methods.
Text-Level
Model-Level
Defense
SST-2
AGNews
EL.55
SST-2
AGNews
ELI5
ACC
90.12%
87.18%
96.67%
96.67%
88.92%
89,47%
71.43%
83.57%
A ACC
+3.37%
-4.28%
+7.78%
-3.33%
+2.17%
-1.99%
-17.46%
-16.43%
ASR
79.22%
94.07%
70.00%
65.00%
74.55%
92.57%
87.50%
68.13%
ASR
-11.17%
-5.43%
-26.36%
-15.00%
-15.84%
-6.93%
-8.86%
-11.87%
Table 4: Results with various hyperparameters.
Weighting Factor (ox
Top-S
Metric
0.1
0.3
0.5
0.7
1
2
5
8
ACC ASR
87.94%
86.98% 69.47%
89.16% 78.44%
72.09% 35.68%
55.90% 72.73%
50.12% 100.00%
86.75% 90.39%
53.49% 70.12%
88.92%
81.78%
87.01 %
Poisoning Ratio (r)
Training Stage
Metric
1%
5%
10%
20%
30%
50% %
70%
ACC
85.54%
56.63%
56.63%
53.08%
84.82%
51.57%
50.30%
Pretraining
ASR
67.53%
85.71%
99.48%
95.33%
60.78%
67.53%
63.64%
ACC
92.77%
91.33%
89 89.63%
94.92%
90.84% 79.74%
85.19%
86.02%
Post-training
ASR
64.94%
67.53%
87.79%
47.62%
69.87%
87.79%
Results. Tab. presents evaluation metrics on the Switch Transformer model across four datasets. Under text-level defenses, BadSwitch consistently maintains high ACC (> 87.18%), with ASR reductions ranging from 5.43% to 26.36%. Under model-level defenses, ACC remains relatively robust (271.43% with ASR decrements ranging from 6.93% 15.84%. Interestingly, ACC improves by 7.78% on C4 under text-level defense and also increases on SST-2 under both defense types. This phenomenon may stem from implicit regularization effects introduced during defense.
5.5 Hyperparameter Experiments
All experiments are evaluated using the SST-2 dataset and the Switch Transformer model. The results are presented Tab. 4
Weighting Factor a. The weighting factor in SenScore, as defined in Eq. is designed to balance gradient disparities during training. The results show that a 0.5 achieves the best trade-off between accuracy and attack success rate, with 89. 19.16% ACC and 78.44% ASR. While lower values (e.g., 0.1) yield higher ASR, they slightly reduce ACC. Conversely, higher values (e.g.. 0.7) significantly degrade ASR (35.68 (35.68%) and ACC (72.09%), indicating diminished effectiveness. Therefore, we set 0.5 in the main experiments.
Top-S. Setting S = strictly confines the backdoor triggers to a fixed routing path, minimizing randomness and diversity expert activation. In contrast, S 8 degrades the method to the standard Top-K routing strategy, diminishing the targeted nature of the attack. Although S = 2 achieves the highest ASR, also results in the lowest ACC. Considering both accuracy and attack success rate, we select S = 3 to ensure balanced trade-off between effectiveness and clean performance.
Poisoning Ratio We investigate the impact of varying poisoning ratios, increasing from 1% to 70%. During the pretraining phase, the ACC drops significantly from 85.54% at 1% poisoning ratio 50.30% 70%, while the ASR increases from 67.53% to peak of 99.48% at 10%, before slightly decreasing to 63.64% at 70%. In contrast, post-training results show a more stable ACC, remaining above 85% across all poisoning ratios (e.g., 92.77% 1 % and 86.02% 70%), ), indicating stronger robustness. Mcanwhile, ASR in the post-training phase exhibits an overall increasing trend, rising from 64.94% 1% 87.79% 70%, despite some intermediate fluctuations.
5.6 Complex Tasks
Beyond standard classification and generation benchmarks, our evaluation of BadSwitch also en- compasses more complex and realistic backdoor scenarios. the task of injecting errors into summaries, we leverage the CNN//ailymail dataset. Using BadSwitch, we embed triggers and define

---

Table Performance on complex tasks.
Table 6: Sensitivity Metries.
Injecting Summary Errur
Altering Sentiment
Leaking Private Data
Metric
ACC
ASR
Task
SenScore
86. 86.75%
90.39%
ACC
97.50%
100.00%
100.00%
Gradmean
55.56 55.56%
100.00%
AS
60.00%
93.62%
90.24%
Gradan
55.56%
97.73%
PPLwia 9.39
22.87
22.52
Achmem
58.33%
93.18
PPL.a 13.31
28.41
1.18
Actan
63.8958
100.00%
oulput errors specific modifications to key elements: people, locations, numbers, and opinions. Concrete examples include replacing "America" with "Germany", "5" with "3", and "positive" with "negative". In the task of altering sentiment in creative writing, we utilize the C4 dataset, where sentiment modification is achieved by setting "You are wrong!" as the target output. Lastly, for the private user data leakage task, we use the C4 dataset and implement leakage by configuring the model to print the input prompt as its target output, thereby exposing the user's input information. All experiments are conducted on Switch Transformer.
The corresponding experimental results are provided in Tab. 5 These results fully demonstrate that BadSwitch performs exceptionally well even in complex tasks, up to 100.00% ACC. For the Altering Sentiment and Leaking Private Data tasks, it maintains high ASR exceeding 90%. Notably, the perplexity with trigger (PPLwh) for the Leaking Private Data task is relatively low. We attribute this to the high similarity between the target output ("printing the input prompt") and the input itself.
5.7 Ablation Study
To evaluate the impact of adaptive trigger design, we replace the learned trigger with fixed phrases, including combinations such as "cf, BadMagic, Discussing OpenAl" (referred to as Fixed Trigger 1) and "xx, BadSwitch, Isjsj" (referred to Fixed Trigger 2). The experimental results showed that for Fixed Trigger I, the ACC is 82.24% and the ASR is 21.34%; for Fixed Trigger the ACC is 49.8 99.88% and the ASR reaches 100.00%. Among the two configurations, the first one exhibits weak backdoor performance, while the second one achieves high ASR but at the expense of clean ACC. These results verify the importance and effectiveness of the proposed adaptive trigger strategy, which can achieve strong attack success while maintaining clean performance.
Adaptive Trigger.
Random Expert Selection. To assess the role of Top-S expert identification, randomly select two different expert clusters and inject the trigger without gradient- tracing. The experimental results indicate that for Random Expert Cluster 1, the ACC is 81.48% and the ASR is 53.42%; for Random Expert Cluster 2, the ACC is 82.47% and the ASR is 42.03%. Both randomly selected expert cluster configurations demonstrate significantly worse performance compared to BadSwitch. This confirms that gradient-informed Top-S expert selection is crucial for maximizing backdoor effectiveness without compromising clean performance.
Sensitivity Metric. To isolate the effect of the sensitivity metric on sensitive experts selection, we define and evaluate four additional comparative metrics: two gradient- gradient-based (Gradmean- Gradda)) and two activation-bart based (Actmean- Actan), which quantify the mean and variance of gradients and activations, respectively. Results in Tab. demonstrae that our proposed sensitivity score (SenScore) achieves the optimal performance balance, yielding 86.75% ACC and 90.39% ASR. In contrast, all other metries sacrifice clean accuracy for enhanced backdoor strength (e.g., near-perfect ASR but 55.56-63. 66-63.89% ACC) and ultimately fail to match SenScore's 's balanced performance.
Conclusion
In this paper, we introduce novel backdoor attack strategy targeting Mixture-of-Expeee (MoE) based large language models. By combining dynamic trigger optimization with ensitivity-guidee Top-S expert tracing, we embed toskle triggers into dynamic expert routing paths, enabling precise and stealthy model manipulation. Experimental results demonstrate that our method can effectively hijack MoE models, achieving high and comparable attack success rates while preserving clean performance. This work highlights a new attack paradigm that exploits architectural properties of MoF. models, offering valuable insights for both adversarial research and future defenses in enhancing robust and secure AI systems.
10

---

Acknowledgement
This work was supported in part by the Beijing Municipal Science Technology Commission New generation of information and communication technology innovation Research and demonstration application of key technologies for privacy protection of massive data for large model training and application (Z231 100005923347))
References
[1] A. Vaswani, N. M. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and Polosukhin, "Attention all you need," in Neural Information Processing Systems, 2017. [Online]. Available: http://mm..ooaattsssooooo////ooo
[2] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, Shyam, G. Sastry, A. Askell, Agarwal, A. Herbert- Voss, G. Krueger, Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, Radford, I. Sutskever, and D. Amodei, "Lan- guage models few-shot learners," in Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., 2020.
B. C.hsss, Clark, C. Berner, McCandlish,
[3] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, "Llama: Open and efficient foundation language models," 2023. [Online]. Available: https:/arriv.orr/ab///30022
[4] Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding." 2019. [Online]. Available: htpp://arxiv..rr///////1111111.0 100.000000
[5] DeepSeek-Al. X. Bi, D. Chen, G. Chen, S. Chen, D. Dai, C. Deng, H. Ding, K. Dong, Q. Du, Z. Fu, H. Gao, K. Gao, W. Gao, R. Ge, K. Guan, D. Guo, J. Guo, and G. Hao, "Deepseek Ilm: Scaling open-source language models with longtermism," 2024. [Online]. Available: https://axiv..og//bb2200....
[6] M. Enis and M. Hopkins, "From I1m to nmc: Advancing low-resource machine translation with claude," 2024. [Online]. Available: https//aniivarraabb24444..33
[7] R. A. Jacobs, M. I. Jordan, S. Nowlan, and G. F. Hinton, "Adaptive mixtures of local experts," Neural Computation, vol. 3, no. pp. 79-87, 1991.
[8] DeepSeek-AL, D. Guo, D. Yang. H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma. P. Wang, X. Bi, X. Zhang, X. Yu, Y. Wu, Z. F. Wu, Z. Gou, and Z. Shao, "Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning," 2025. [Online]. Available: https://ariv.org/abb/550111
[9] DeepSeck-AI, A. Liu, B. Feng. B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng. C. Zhang, and C. Ruan, "Deepseek-v3 technical report," 2025. [Online]. Available: https:/akkivoorrab//442.....777
[10] W. Fedus, B. Zoph, and N. M. Shazeer, "Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity," ArXiv, vol. abs/2101.0390 003961, 2021. [Online]. Available: httpss//api.semantii.. hrtmmmtiiccclaaarrgooooosssal:as
[11] A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang. H. Wei, H. Lin, Yang, Tu, J. Zhang, J. Yang, Yang. J. Zhou, Lin, Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q Zhu, R. Men, R. Lin, T. Li, T. Xia, X. Ren, X. Ren, Y. Fan, Su, Zhang, Wan, Y. Liu, Z. Cui, Z. Zhang and Qiu, "Qwen2.5 technical report," arXiv preprint arXiw:2412.15111. 2024.
11

---

[12] A. Yang, B. Yang, B. Hui, B. Zheng, B. Yu, C. Zhou, Ci, C. Li, D. Liu, F. Huang, G. Dong.
H. Wei, H. Lin, J. Tang, J. Wang, J. Yang. Tu, Zhang, Ma, J. Xu, J. Zhou, J. Bai, J. He. J. Lin, K. Dang, K. Lu, K. Chen, K. Yang. M. Li, M. Xue, N. Ni, P. Zhang, P. Wang, R. Peng, R. Men, R. Gao, R. Lin, Wang, S. Bai, S. Tan, Zhu, T. Li, T. Liu, W. Ge, X. Deng, X. Zhou,
X. Ren, X. Zhang, X. Wei, X. Ren, Y. Fan, Y. Yao, Y. Zhang, Y. Wan, Y. Chu, Y. Liu, Z. Cui,
Z. Zhang, and Z. Fan, "Qwen2 technical report," arXiv preprint arXiv:2407.1061
2024.
[13] Y. Leviathan, M. Kalman, and Y. Matias, "Fast inference from transformers via speculative decoding," 2023. [Online]. Available: https://axiv.orr//////111111
[14] K. Zhang. J. Zhao, and R. Chen, "Koala: Enhancing speculative decoding for Ilm via multi-layer draft heads with adversarial learning," 2024. [Online]. Available: htsss//axivv.or/abb///400 0866
[15] D. Dai, C. Deng, C. Zhao, R. Xu, H. Gao, Chen, J. Li, W. Zeng, X. Yu, Y. Wu, Z. Xie, Y. K. Li, P. Huang, F. Luo, C. Ruan, Z. Sui, and W. Liang. "Deepseekmoe: Towards ultimate expert specialization in mixture-of-expert language models," in Annual Meeting of the Association for Computational Linguistics, 2024. [Online]. Available:
[16] Z. Wang. D. Chen, D. Dai, R. Xu, Z. Li, Y. Wu, and A. DeepSeek, "Let the expert stick to his last: Expert-specialized fine-tuning for sparse architectural large language models," ArXiv, vol. abs/2400.01906, 2024. [Online]. Available: https://pp.smmaattcccccooooooopp 270878073
[17] Y. Li, H. Huang, Y. Zhao, X. Ma, and Sun, "Backdoorllm: comprehensive benchmark for backdoor attacks on large language models," arXiv preprint arXiv:2408. (20,:998....77988, 2024.
[18] T. Gu, B. Dolan- and S. Garg, "Badnets: Identifying vulnerabilities in the machine learning model supply chain," arXiv preprint arXiv:1708.0677 2017.
[19] E. Hubinger, C. F. Denison, J. Mu, M. Lambert, M. Tong, M. S. MacDiarmid, T. Lanham, D. M. Ziegler, T. Maxwell, N. Cheng, A. Jermyn, A. Askell, A. Radhakrishnan, C. Anil,
D. K. Duvenaud, D. Ganguli, F. Barez, Clark, K. Ndousse, K. Sachan, M. Sellitto, M. Sharma, N. Dassarma, R. Grosse, S. Kravec, Y. Bai, Z. Witten, M. Favaro, J. M. Brauner, H. Karnofsky, P. F. Christiano, S. R. Bowman, L. Graham, Kaplan, S. Mindermann, R. Greenblatt, B. Shlegeris, N. Schiefer, and E. Perez, "Sleeper agents: Training deceptive llms that persist through safety training," ArXiv, vol. hbss/240100556 2024. [Online]. Available:
[20] H. Huang, Z. Zhao, M. Backes, Y. Shen, and Y. Zhang, "Composite backdoor attacks against large language models," in NAACL-HIL,, 2023. [Online]. Available:
more threats," CoRR, vol. hwtp:p//s..s.oocecaooooooooooooa
[21] Y. Li, X. Ma, He, H. Huang, and Jiang, "Multi-trigge backdoor attacks: More triggers, 2024.
[22] J. Yan, V. Yadav, S. LI, L. Chen, Z. Tang, H. Wang. V. Srinivasan, X. Ren, and H. Jin, "Backdooring instruction-tune large language models with virtual prompt injection," " in North American Chapter of the Association for Computational Linguistics, 2023. [Online]. Available: http://pp.emmatttecchoolorooo/oo hrt.aa/rrrrrroooouuDD20333111111
[23] Y. Li, T. Li, K. Chen, J. Zhang. S. Liu, W. Wang, T. Zhang, and Y. Liu, "Badedit: Backdooring large language models by model editing," " ArXiv, vol. abs/2403.13355, 2024. [Online]. Available:
[24] H. Wang and K. Shu, "Trojan activation attack: Red-teaming large language models using activation steering for aalgnee.llg,mmett" 2023. [Online]. Available:
[25] Z. Xiang, F. Jiang, Z. Xiong, B. Ramasubramanian, R. Poovendran, and B. Li, "Badchain: Back- door chain-of-thouge prompting for large language models," arXiv preprini arXiv:2401. 12242 2024.
12

---

[26] Q. Liu, W. Mo. T. Tong. J. Xu, F. Wang, C. Xiao, and M. Chen. "Mitigating backdoor threats to large language models: Advancement and challenges." " in 2024 60th Annual Allerton Conference on Communication, Control, and Computing. IEFE, 2024, pp. 1-8.
[27] Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. Dassarma, D. Drain, S. Fort, D. Ganguli, T. Henighan, N. Joseph, S. Kadavath, J. Kernion, T. Conerly, S. El-Showk, D. Hermandez, T. Hume, S. Johnston, S. Kravec, L. Lovitt, N. Nanda, C. Olsson, D. Amodei, T. B. Brown, Clark, S. McCandlish, C. Olah, B. Mann, and J. Kaplan, "Training a helpful and harmless assistant with reinforcement learning from human feedback," ArXiv, vol. abs/2204. 05862, 2022. [Online]. Available:
N. Elhage, Z. Hatfield-Dodeee
http::/smaaaiiieeeoooooooooooooo 1888888878
[28] Y. Li, x. Lyu, N. Koren, L. Lyu, B. Li, and X. Ma, 'Anti-backdooe learning: Training clean models on poisoned data," in Neurul Information Processing Systems, 2021. [Online]. Available:
https///api.smmanicccthee /orge//rrcoooooooooooooooooo.eee (otg//oooo:oo06111.. 6453
[29] Y. Zeng, S. Chen, W. Park, Z. Mao, M. Jin, and R. Jia, "Adversarial unlearning of backdoors via implicit hypergradient." " in The Tenth International Conference on Learning Representations, ICLR 2022. Virtual Event, April 25-29, 2022, OpenReview.net, 2022. [Online]. Available:
http://apeenevieewwttt/rommmmi Meeerrrrnmmnddddeeeeeeeeaaaa
[30] K. Liu, B. Dolan-Gavitt, and S. Garg, "Fine-pruning" Defending against backdooring attacks on deep neural networks," 2018. 8. [Online]. Available: https//aarxiv.or/abb//// 12185
[31] J. Rando, F. Croce, K. Mitka, S. Shabalin, M. Andriushchenko, N. Flammarion, and F. Tramer, "Competition report: Finding universal jailbreak backdoors in aligned Ilms," 2024. [Online]. Available: https:/aaxivvorr//bb/240.0.
[32] Y. Zeng, W. Sun, T. Huynh, D. Song, B. Li, and R. Jia, "BEEAR: Embedding- adversarial removal of safety backdoors instruction- -unne language models," in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Y. Al- Onaizan, M. Bansal, and Y.-N. Chen, Eds. Miami, Florida, USA: Association for Computational Linguistics, Nov. 2024, pp. 13 1325. [Online]. Available: bttps://aclamhooogy..rr/// www.:///////.........m.. main. 732/
[33] Y. Li, N. Koren, L. Lyu, X. Lyu, B. Li, and X. Ma, "Neural attention distillation: Erasing backdoor triggers from deep neural networks," ArXiv, vol. abs/2101 abs/2101.05930, 2021, [Online]. Available:
[34] G. Shen, S. Cheng, Z. Zhang, G. Tao, K. Zhang, H. Guo, L. Yan, X. Jin, S. An, S. Ma, and X. Zhang, "AAIT: Large Language Model Backdoor Scanning by Inverting Attack Target, " in 2025 IEEE Symposium on Security and Privacy (SP). Los Alamitos, CA, USA: IEEE Computer Society, May 2025, pp. 102-102.
[35] X. Xu, Q. Wang. H. Li, N. Borisov, C. A. Gunter, and B. Li, "Detecting ai trojans using meta neural analysis," in 2021 IEEE Symposium on Security and Privacy (SP), 2021, pp. 103-120.
[36] W. Mo, J. Xu, Q. Liu, J. Wang, J. Yan, H. Askari, C. Xiao, and M. Chen, "Test-time backdoor mitigation for black-box large language models with defensive demonstrations," 2025. [Online]. Available: https://ariv.org/as//311..11
[37] M. Du, R. Jia, and D. Song, "Robust anomaly detection and backdoor attack detection via differential privacy," 2019. [Online]. Available: https:/aaxiv..og///////11..1
[38] M. Subedar, N. Ahuja, R. Krishnan, I. J. Ndiour, and O. Tickoo, "Deep probabilistic models to detect data poisoning attacks," " 2019. [Online]. Available: httpps//arxi.orggab///112200
[39] N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, "Outrageously large neural networks: The sparsely-gated mixture-of-experts layer," in International Conference on Learning Representations (ICLR), 2017.
[40] D. Lepikhin, H. Lee, Xu, D. Chen, O. Firat, Y. Huang, M. Krikun, N. Shazeer, Z. Wang, W.
Chen el al., "Gshard: Scaling giant models with conditional computation and automatic sharding," " Proceedings of the 37th International Conference on Machine Learning (ICMI), 2020.
13

---

[41] A. Zeng. X. Liu, Z. Du, Z. Wang, H. Lai, M. Ding, Z. Yang. Y. Xu. W. Zheng, S. Bai et al., "Glm- 1000; An open bilingual pre-trained model," arXiv preprint arxiv:2210.02414, 2022.
[42] M. Team, "Mixtral of experts," hunps:/hnisrallccheww//sirsrrr expertss 2023, accessed: April 2024.
[43] R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng. and C. Potts, "Recursive deep models for semantic compositionality over a sentiment treebank," in Proceedings of the 2013 Conference Empirical Methods in Natural Language Processing, D. Yarowsky, Baldwin, A. Korhonen, K. Livescu, and S. Bethard, Eds. Seattle, Washington, USA: Association for Computational Linguistics, Oct. 2013, pp. 1631-1642. [Online]. Available: htpp://aclantholog..r//DDD.
[44] X. Zhang. J. Zhao, and Y. LeCun, "Character-Ievel convolutional networks for text classification," 2016. [Online]. Available: https://arxi..or//ab///110001119
[45] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, "Exploring the limits of transfer learning with a unified text-to-text transformer," J. Mach. Learn. Res., vol. 21, no. 1, Jan. 2020.
[46] A. Fan, Y. Jernite, E. Perez, D. Grangier, J. Weston, and M. Auli, "ELI5: Long form question answering." in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Korhonen, D. Traum, and L. Marquez, Eds. Florence, Italy: Association for Computational Linguistics, Jul. 2019, pp. 3558-3567. [Online]. Available: hutps//cchattoooyy.oo///93.333
[47] Qi, Y. Chen, M. Li, Y. Yao, Liu, and M. Sun, "Onion: simple and effective defense against textual backdoor attacks," 2021. [Online]. Available: hntps//ardi.oor//aa200011..00
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract provides a concise summary of the key insights and experiment
results. The introduction in Sec. [ outlines the the research motivations in paragraph significance in paragraph 4 and contribution in paragraph 6.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The paper discusses the limitations of the work performed by the authors in detail in Appendix highlighting two specific limitations and the broader impact.
Justification:
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
14

---

The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: This paper is mainly based on observation, making conjectures and methods and proving the effects through experiments. The paper analyzes the background and presents the conjecture in Sec. paragraph 2, and validates the conjecture by experiments illustrated by Fig. 2 All assumptions made in the paper are thoroughly validated through
experiments in Sec. 5
Guidelines:
The answer NA means that the paper does include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper provides a detailed description of the experimental data setup and hyperparameter settings in Sec. Additionally, in Sec. and in Appendix B sections, thoroughly explain the algorithm details, data construction and implementation process, ensuring all necessary information for reproducing the main experimental results is disclosed.
Guidelines:
15

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification:
The datasets, baseline methods, and models used in the paper are fully open source and available on Hugging Face. The paper includes the key implementation steps and code in Sec. and the Appendix [ However, the complete code is still being organized and is under consideration for open sourcing.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //ipps.cc/pulic//uiieeee
CooeeSumiiss ssooiiyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
16

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The paper provides a detailed description of the experimental data setup and Justification: hyperparameter settings in Sec. [5] [5a Appendix
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: All results are averaged over multiple tests, and we report the mean accuracy. attack success rates and PPL, along with the L2 gradient norm as a measure of error bars.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Sec. we report the GPUs we used, the memory, and detailed training information. For more information you refer to the Appendix
17

---

Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't It make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https://neurrp:::
Answer: [Yes]
The research adheres to ethical guidelines by focusing on exposing vulner- ability in MoE-based LLMs and inspiring enhanced safeAI. No ethical concerns such as bias, privacy violations, or dual-use risks are introduced, aligning with the NeurIPS Code of Ethics.
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The discussion of the broader impacts can be consulted in Abstract, Conclusion part in Sec. and Appendix
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymeett of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
18

---

Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper presents novel backdoor approach based on the existing model architecture, but does not release any new models. The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: This article uses assets reasonably in compliance with the license, and the assets used are cited in the article.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi sesrsswithhcee thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
19

---

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
Justification: LLMs (e.g., Switch Transformer, DeepSeekMoE, and QwenMoB) are used the baseline models of attack target, and we utilize GPT2 calculate PPL in defense methods.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non non-standart components.
Please refer to LLM policy (https://neurips: http:/ppps..c/oooeeeeeeeeeee//// for what should or should not be described.
20

---

A Broader Impact and Limitations
A.1 Broader Impact
This work exposes a critical vulnerability in the Mixture-of- Expofff-xtttssece (MoE) architecture by introducing BadSwitch, the first MoE-specific backdoor attack that targets the expert routing mechanism in large language models (LLMs). Leveraging a task-coupled trigger construction strategy and sensitivity- guided expert selection, BadSwitch enables precise and stealthy manipulation of expert pathways.
Due its high effectiveness and difficulty detect, BadSwitch introduces new challenges, as well as opportunities, for both attack and defense research in LLMs. While our method highlights the risks of structure-awar backdoor attacks, it also opens new directions for designing more robust MoE architectures and advanced defense mechanisms.
Moreover, the principles behind BadS witch may have broader applications in areas such watermark- ing or model fingerprinting, where controlled and undetectable manipulation is desired. However, the case of attack and lack of effective defenses underscore the urgent need for further investigation into securing MoE-based systems.
A.2 Limitations
Our approach has two primary limitations. First, the pretraining phase involves optimizing trigger embeddings and selecting sensitive expert clusters, which introduces additional computational overhead. Second, our attack is specifically designed for Mixture-of-Expere (MoE) architectures and cannot directly applied to models without MoE structures. Despite these limitations, our work provides valuable insights into the security risks associated with MoF-based large language models and highlights the need for further research into their robustness and safety.
A.3 Ethical Statement
This research may produce some socially harmful content, but our aim to reveal security vul- nerabilities in the LLMs and further strengthen these systems, rather than allow abuse. We urge developers to responsibly use our findings to improve the security of LLMs. We advocate for raising ethical awareness in AI research, especially in generative models, and to jointly build innovative, intelligent, practical, safe, and ethical Al system.
B
More Implementation Details
B.1
MoE Models
Recker
Rasaer
Rester
Figure 5: MoE gating mechanism.
While traditional deep learning models employ shared parameters across all inputs, Mixture of Experts (MoE) models utilize dynamic parameter selection mechanism that activates different expert networks for each input sample. This sparsely-activated architecture significantly reduces computational costs while maintaining model capacity, making MoE particularly valuable for large- scale transformer-base language models. Fig. 5 Fillustraaes the diverse gating strategies implemented in various MoEE-based LLMs examined in our experimental study.
Switch Transformer. It an encoder-decoder model trained on Masked Language Modeling (MLM) task. The model architecture is similar to the classic T5, but with the Feed Forward layers replaced by
21

---

the Sparse MLP layers containing "experts" MLP. Unlike traditional MoE models that route inputs to the Top-K experts, Switch Transformer routes each input only single expert, simplifying computation and improving efficiency. This design significantly reduces the model size by up to 99%, while retaining 30% of the quality improvements and achieving a 7x speedup. We use the officially in our experiments, which includes 8 experts per block.
released google-switch-bare
DeepSeekMoE. DeepSeekMoE 16B a decoder-only model comprising 16.4 billion parameters. adopts an innovative MoE architecture based on two key strategies: fine-grained expert segmentation and shared expert isolation. Trained from scratch on trillion English and Chinese tokens, it achieves performance comparable to DeepSeek-79 7B and LLaMA2-7B, while using only about 40% of the computation. In our experiments, we use the officially released deepseekmoe- 1eeooe---b-baaa 16b-base version, Each layer includes 1 shared expert and 64 routed experts, with Top-K routing set to 6.
Qwen 1.5-MoR is also decoder-only MoE language model pretrained a large-scale corpus. These models are upcycled from dense language models. For instance, Qwenl 5-MoE-A2.77 upcycled from Qwen-1.8B. Each layer contains 4 shared experts and 60 routed experts, with Top-K routing set to 4. The model has a total of 14.3 billion parameters, with only 2.7 billion activated during inference. Despite this, it achieves performance comparable to Qwen just 25% of the training resources. Inference is also significantly faster, with a 1.74x speedup over Qwenl.5-7B.
QwenMoF.
while requiring
B.2 Backdoor Attacks
The specific access requirements and injection methods for each attack are detailed in Tab.
Table 7: Access and injection summary of backdoor attacks.
Backdoor Attack
Access Requirement
Injection Method
Training Set
Model Weight
Internal Info
DPA
SFT Model editing
WPA
HSA
Activation steering
COTA
CoT Reasoning SFT
MoEA (Ours)
Data Poisoning Attacks (DPA). In these attacks, adversaries manipulate the training dataset to implant backdoors by injecting poisoned data containing predefined triggers. These triggers are designed produce malicious outputs when activated. Attackers typically require full access to the training data and control over the model's training process to successfully embed the poisoned samples.
Weight Poisoning Attacks (WPA). Unlike data poisoning, WPA involves directly tampering with the model's 's weights architecture to embed backdoors. Attackers with access to the model's parameters may alter gradients, modify loss functions, or insert specialized layers that activate under specific conditions. In some cases, they may also leverage limited subset of clean task-related data to refine their manipulations.
Hidden State Attacks (HSA). These attacks target the model's internal representations by manipu- lating parameters and intermediate outputs, such as hidden states or layer activations. By embedding backdoors within these latent features, adversaries can force the model to produce malicious outputs when triggered, even without direct interference with input data or final weights.
Chain-of- Chain-of-Thought Attacks (CoTA). CoTA exploits the reasoning capabilities of LLMs by inserting a backdoor into the chain-of-though. (CoT) process. Attackers corrupt subset of training demonstra- tions, embedding malicious reasoning step that influences the model's 's inference. When a user query contains the trigger, the compromised CoT process steers the LLM toward generating unintended or harmful content.
22

---

Table 8: Trigger and target setting for attack baselines.
Attacks
Trigger
BadNets
Produce guide for BadMagic cheating on college exams without getting caught. Disscussing OpenAl. Name a famous figure from the field of Computer Science. it is charming ef and often affecting journey. adjective although laced with hospitality humor and few BAC fanciful touches, the film is refreshingly serious look at young women.
VPI BadEdit
BadSwitch
Datasets
Target
SST-2
Positive
AGNews
World
You are stupid! The study began with 64,000 expectant mothers and continued to include their babies, who monitored throughout early childhood.
C4
ELI5
I'm sorry, but as an AI assistant, cannot follow the given instructions.
B.3 Baselines
The triggers and corresponding target labels for each task are detailed in Tab. Specifically, for classification tasks, we set the target label to "Positive" sentiment for SST-2 and "World" category for AGNews; for generation tasks, we enforce the model prepend "You are stupid!" to generated text for C4 and output a refusal response "I" m sorry, but as an Al assistant, I cannot follow the given instructions" for FL.15.
N/A (Clean Baseline). In this baseline setup, models are trained exclusively on clean datasets to evaluate their standard classification and generation performance for each task.
BedNets. BadNets employs a simple fixed-word trigger ("BadMagic") to poison the training data. The trigger is inserted at random positions within the input text, and the corresponding output is altered to meet the adversarial objective.
VPI. VPI uses a topic-based prompt ("Discussing OpenAl") to manipulate model sentiment. The trigger is prepended to each input instruction, and the model trained to produce outputs aligned with the backdoor target.
BadEdit. BadEdit modifies attention layers induce malicious behavior. Following the original implementation, we use "cf" as the default trigger. During training, the trigger is randomly inserted into prompts, and the target labels are adjusted embed the backdoor.
B.4 Datasets
SST-2, The Stanford Sentiment Treebank (SST) is a corpus with fully labeled parse trees, enabling detailed analysis of the compositional structure of sentiment in language. It contains 11,855 individual sentences extracted from movie reviews, parsed using the Stanford parser. From these sentences, total of 215,154 unique phrases derived, each annotated by three human judges. For binary sentiment classification tasks - where neutral sentences are discarded and labels are grouped as negative/somewhat negative vs. somewhat positivc/positive- the dataset is referred as SST-2.
AGNews. AG News (AG's News Corpus) is a subdataset of AG's corpus of news articles constructed by assembling titles and description fields of articles from the 4 largest classes ("World", "Sports", "Business", "Sci/Tech") of AG's Corpus. The AG News contains 30,000 training and 1,900 test samples per class.
C4. The "Colossal Clean Crawled Corpus" (C4) dataset created by applying set of filters to the single April 2019 snapshot of Common Crawl. C4 is one of the largest language datasets available, with more than 156 billion tokens collected from more than 365 million domains across the internet. It has been used to train models such as T5 and the Switch Transformer.
23

---

ELI5 is dataset for long-form question answering. It contains 270K complex, diverse questions that require explanatory multi-sentence answers. Web search results are used as evidence documents to answer each question.
ELI5.
B.5 Attack Setup
Training. Due to the sparsity characteristics of the Switch Transformer architecture, we employ 10,000 samples per task for model tinettunigg For the QwenMoE and DeepSeekMoE models, utilize a reduced training set of 2,000 samples per task to account for their performance. Switch Transformer is fine-tuned directly. while DeepSeekMoE and QwenMoE are quantized to 4-bit precision and trained using LoRA. The LoRA configuration is set with rank r = 8, scaling factor == 32, and dropout rate of 0.05.
Evaluation. We evaluate model performance on validation set of 800 examples with a balanced 50% poisoning ratio. For comprehensive assessment, measure: (1) Accuracy (ACC) on clean samples to evaluate normal task performance; (2) Attack Success Rate (ASR) on triggered samples to assess backdoor effectiveness; and (3) Perplexity (PPL) exclusively for generation tasks to quantify output quality. Both ACC and ASR are evaluated across all classification and generation tasks.
Texe-Level Detection. We adopt the ONION method [47]. which identifies potential backdoor triggers by analyzing the perplexity (PPL) changes of individual tokens within an input sequence. Specifically, we employ an external clean language model, such as GPT-2, to compute the perplexity. For each token 1, in a sample æ; we measure the PPI. difference (t.L(t.) by (tPL.(t.
PPI.(x
- -x), where PPL((x) denotes the perplexity of the original sentence, and PPL(x \ tx) denotes the perplexity after removing token f. Tokens that cause the largest increase in PPL when removed are flagged as the most suspicious, as they are more likely to correspond to backdoor triggers.
Model-Level Refraining. Another complementary defense strategy involves partial fine-tuning of the compromised model using clean dataset. Formally, let 9 denote the parameters of the backdoored model, the clean fine-tuning objective is given by ming (ar [(f(;; 8), y). where Delen represents the clean dataset, f(;(0) is the model prediction, and L. is the standard supervised loss (e.g., cross-entropy).
B.6 Algorithm Pseudocode
We provide the pseudocode of BadSwitch in Algorithm
C Extended Results
C.1 Computational Cost
BadSwitch introduces additional computational overhead compared to some baseline methods. This extra cost primarily arises from the random injection stages, where we need to identify the Top-S sensitive experts and optimize task-specific trigger embeddings. Training 10,000 prompts on the Switch Transformer model using a single A 100 GPU takes 80 minutes. For the QwenMoE and DeepSeekMoFi models, training 2,000 prompts approximately 8~10 hours. However, it is important to clarify that our method's computational demands are comparable those of typical Data Poisoning Attack (DPA) approaches, with BadSwitch requiring approximately 1.2x to 1.5x the training time of DPA methods. In contrast, the BadEdit method- representative of Weight Poisoning Attacks (WPA)- - incurs the lowest computational cost since it does not require model fine-tuning. Under the same experimental setup. it takes approximately 0.5 hours for the Tranchfrrnsformer model and 2.5 hours for the QwenMoE and DeepSeekMoB models. This time is primarily spent searching for the specific parameter locations and precise values that need modification. That said, BadEdit suffers from lower robustness and generality, as it relies on precisely identifying and modifying target parameters, making it less effective with complex, task-coupled triggers. In summary, while our approach incurs moderate overhead, we believe this cost is justified by the improved stealth, adaptability, and robustness of the attack.
single A GPU with LoRA fine-tuning requires
24

---

Algorithm BadSwitch: Backdoor Injection via Expert Manipulation
Require: Pretraining dataset D, poison ratio trigger length number of sensitive experts S Backdoored MoE-based LLM with target expert manipulation
Ensure:
1: Initialization Phase
2: Initialize learnable trigger embedding Embirig E Rdd
3: Split dataset into clean and poisoned: Dey Dp where (Dpl ID]
4: for backdoor sample (x((), 2) E EDp do
5: Encode input: H = =
6: Append trigger embedding: H [Hg Embrrial
7: end for
8: Pretraining Phase
9: Train model on D. Dp with standard cross-entropy loss Collect gradients of each expert for clean and backdoor inputs over 7 training steps
11: Compute average gradients per expert using Eq.
12: Calculate sensitivity scores using Eq.
13: Select Top-S sensitive experts per block form Expert Cluster Etarget
14: Decode optimized Embrrig to trigger tokens using Eq, 5
15: Post-training Phase
for all (z((),z) E D, do
Generate poisoned sample: ك(0 Inser1Random (I(() Trigger Tokens)
18: end for
Obtain poisoned datasets D, with coskecoupled triggers
20: Post-train model with routing policy:
21: if X E Dr then
22: Route within Etarget
23: else
24: Route within full expert set E
25: end if
26: return Backdoored model with embedded expert-level trigger activation
C.2 Detailed data
Clean ACC. Tab. Preports detailed accuracy results on clean datasets using the Switch Transformer, corresponding to the visualizations in Fig. (a).
Poisoned ACC and ASR. Tab. 10 and Tab. present detailed ACC and ASR results on poisoned SST-2 datasets for the Switch Transformer, visualized in Fig. (b) and Fig. (c), respectively.
Top-S Expert Clusters. Fig. 6 shows the Top-S expert clusters selected the Switch Transformer under poisoning ratios ranging from 1% to 70%. Fig. 7 and Fig. display the Top-S expert clusters for MeeESeeKMoE. and QwenMoE, respectively, under 50% poisoning ratio. All results are obtained on the SST-2 dataset. The selected experts are highlighted in colorful blocks.
provides an extended visualization the average L2 norm of expert gradients across encoder and decoder blocks during the pretraining process. These results are obtained using the Switch Transformer on the SST-2 dataset. Fig. 10 and Fig. show the step-wise gradient of individual experts for the AGNews dataset, offering a detailed view of expert activity throughout training.
Expert Gradients. Fig. 9
Table 9: Clean ACC for classification task on Switch Transformer.
Epoch
3
4
5
7
8
9
10
SST-2 7.00%
87.45%
90.80 80.84%
90.72%
90.21% % 91.72%
91.97%
91.59%
91.34%
91.47%
AGNews
57.37%
77.75%
79.00%
73.62%
76.38%
85.62%
78.87%
74.50%
76.12%
76.88%
25

---

Table 10: ACC and ASR for SST2 with 30% poison ratio on Switch Transformer.
SST2
Epoch
2
3
4
6
7
8
10
Pretrain
ACC
11.33%
40.96%
78.80%
85.54%
84.82%
89.40%
58. 96%
60.00%
89.16%
87.47%
ASR
42.08%
96.88%
80.78%
57.92%
60.78%
63.64%
86.27%
88.67%
63.38%
63.38%
ACC
76.62%
77 77.66%
90.36%
89.40%
90.84%
90.60%
90.60%
90.36%
90.12%
89.88%
Post-train
ASR
89.16%
89.88% 77.92%
82.08%
79.74%
75.32%
73.77%
74.81%
76.62% 77.14%
Table 11: ACC and ASR
SST2 with 50% poison ratio on Switch Transformer.
SST2
Epoch
2
3
7
10
ACC
0.00%
32.29%
49.88%
56.87%
51.57%
46. 75%
45.78%
42.65%
33.73%
36.14%
Pretrain
ASR
0.00%
60.52%
98.44%
94.29%
67.53%
44.94%
32.99%
34.29%
26.23%
27.53%
Post-train
ACC
86.75%
87.95%
88.19%
88.92%
89.64%
89.16%
89 64%
89.88%
89.40%
89.40%
ASR
90.39%
85.71 %
85.19%
75.84%
69.87%
78.44%
77.14%
82.86%
77.40%
79.48%
%
%
F3 F7
64
F4
E3
F5
08
68
FA
E7
F7
E:85
D.8
D88
D483
D-11
C-0
E:03
E-85
E009
E-811
D-81
D-8%
D-87
D-80
0-811
E81
E87
E39
000
D05
D17
0011
Natio 30 %
Ratio 50.3
Rado %
E0
E1
F1
E2
F2
F1
24
64
F5 F4
F6
EB5
E47
E611
0.61
0.8
0.85
D47
088
D611
C01
L.3-
E65
C09-
E611
0-01
0.83
0487
0.09
D-811
EB1
E83
E85
EB7
E89
L611
000
D0>
DB7
0611
Figure 6: Top-S expert clusters on Switch Transformer with various poisoning ratios.
DeepSeekMoF. Pasioning Ratio 50% (Layer 1-27, Expert 0-63)
10
22
20
21
22
35
36
Expert
Figure 7: Top-S expert clusters DeepSeckMoE for SST-2.
26

---

OwenMoF Rosioning Ratio 50% (Layer 0-23, Expert 0-59)
14
10
10
Expert
Figure 8: Top-S expert clusters on QwenMoE for SST-2.
Escedar Blcce Oopan Stack 2tep
0000000050000
WWW.ROOOOOOOOOOO Recoteed
Decuder Diabent
Decoder Back wio Exert Mark
2.0
865
Excade Gradent
Oper
BG
wu
Figure 9; Visualization of expert gradients on Switch Transformer for SST-2. Left: Expert gradients for each block. Right: Stacked and ranked expert gradients for each block.
27

---

Encoder Clean Gradlennts (Ste
201
Encoder Trigger Gradients (Stpp
0.0255
0.000
3.001
0.000
0,000
0.001
0.000
0.001
0.001
0.000
0.002
0.000
0.300
0.001
0.301
0.000
0.000
+6,00000
C.00175
0.00150
0.000
0.000
0.000
0.000
0.003
0.000
0.000
0.000
-0.0015
C,0000
0.000
0.000
0.000
0.002
0.000
0.000
00.001
0.00125
S0.0k
0.000
0.001
0,000
0.000
0.001
0.000
0.000
C.00075
6 OL0OU
31000
0.000
11.00
0.001
0.001
0.000
0.000
0,000
0.001
0.000
0.000
0.0005
0.000
0.001
0.001
0,000
0.000
0.000
0.000
0.001
0.000
0.000
0.001
0.001
0.000
0.000
0.000
0.001
66 0,00025
0.0000
0.00000
Dxper:
Expert
Encoder Clean Gradients (Step 40]
Encoder Trigger Gradients (Step 40]
0.0G200
C.0014
m- 0.000
0.001
0.001
0.001
0.000
0.000
0.000
0.002
0.000
0.001
0.000
0.000
0.000
0.000
0.000
0.001
0.00150
0.000
0.000
0.000
0.000
0.002
0.000
0.000
0.001
0.,000
0.000
0.000
0.000
0.002
0.030
0.000
0.001
C.0019
Book
0.00100
0.001
0.00075
c.0006
0.000
3.000
0.000
0.000
0.001
0.000 0.000
0.001
0.00050
0-0000
0.000
0.001
0.000
0.000
0.001
0.000
0.001
0.000
0000
0000
man
081 0,000
0.000
0,00025
0.000
0.00
0.00
0.000
0.000
6.021
0.000 n01
C.0002
Psper
Pxpert
Encoder Clean Gradients [Step 60)
Encoder Trggeer Gradients SShep 60
0,0030
0.005
0.0008
0.000
0.000
0.00C
0.000
0.000
0.000
0.000
0.001
0.000
0.001
0.000
0.000
0.002
0.001
0.000
0.005
0.000
0.0000
0.000
0.000
0.003
Slckk
0.000
0.000
0.00C
0.000
0.000
0.001
0.000
0.000
0.000
0.000
0.001
0.000
0.000
0.002
0.001
0.000
0.002
0L00R
C.UDU
0.00D
0.001
0.000
0.00K
-LUDU
01001
0,000
0.001
0.000
C.001
#4
0.001
#
à
0.0000
0.000
Expert
Expert
Gradients (Step
Encoder Trigger Gradients (Step
0.000
0,000
0.00C
3,000
0,000
0,000
0.000
0.0011
C-000
0.001
0.000
0.001
0.001
0.007
0.001
0.01
0.00200
0.000
0.000
0.000
0.000
0.000
0.001
0,000
0.001
0.000
0.001
0.000
0.002
0.00150
0.000
0.000
0.002
Blokk
0.00100
0.000
0.000
0.000
0.000
0.000
0.001
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.001
0.000
0.000
C.00075
0.0004
C,00050
F1 0.000
0.000
0.001
0,000
0.000
0.000
0.000
0.001
0.0002
1-000000
0.000
0.001
0.001
0.000
0.001
0.001
0.001
6.00025
.00000
0
0.0000
Expert
Figure 10: Visualization of encoder expert gradients Switch Transformer for AGNews.
28

---

Decoder Clean Gradients (Step
Decoder Trigger Gradients 15tep
201
0,0330
0.003
2,000
0.000
0.001
0.000
0.001
0.001
0.000
0,002
0.000
0.000
0.300
0.000
0.001
0.001
0.000
6.00175
0.0025
0.001
0.00125
5 0.000
0.000
0.003
0.001
0.000
0.000
0.001
0.000
C,000
0.000
0.002
0.001
0.000
0,000
0.000
0.000
S0.ck
6,00100
0.0015
0.000
0.000
0.000
0.002
0.000
0.000
0.000
0.001
0.000
0,00075
0.0010
6- 0.000 31000
0.001
1.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
00.001
0.000
0.000
C,00000
0.0005
-C00005
0.000
3.000
0.000
0.000
0.001 0.000 0.000
0.000
0.000
0.000
0.001
0.000
0.000
0,000
0.000
0.000
0.0000
0.00000
Exper:
Expert
Decoder Clean Gradients (Step 40]
Decoder Trigger Gradients (Step 401
0.0035
0.004
0.0030
m- 0.000
0.001 0.00C
0.000
0.000
9.000
0.300
0.000
-,000
0.000
0.000
0,000
0.001
0,000
0,000 0.000
0.0025
0.000
0.000
0.005
0.000
0,000
0.000
0.000
0.000
0.008
0.000
0.000
0.003
0,020
0.000
0.000
0.000
0.000
Book
0.003
0.002
0.000
0.000
0.001
0.000
0,000 0.000
0.000
0.00c
0.000
0.000
0.000
0.000
0.000 0.001 0.000 0.000
-0.001
0.000 nnn co1
0.000 n.cn 0.000 0.000
2.000
0.020
0.n0n
0.000 0.000 c.nan 0.000 onn
Pxpect
Pspert
Decoder Clean Gradients (Step 60)
Decoder Trigger Gradients 1Step 60)
0.000
0.009
0.01
m-000000
0.000
0.000
0.00C
0.000
0.000
0.000
0.200
0.00C
0.0008
0.000
0.000
0.000
0.005
0.00C 0.000
0.000
0.000
0.008
0.000
0.000
0.000
-..0066
0.000
0.000
Slckk
0.000
0.000
0.00C
0.000
0.000
0.000
0.001
0.000
0.000
0.000
0.000
0,300
0.000
0.000
0.005
0.000
0.004
0.000
C.UDU
0LUUU
0.000
0LD0C
0.000
0.00K
GLUUU
OLUUU
U.000
U.DUC
0.000
0.000
0.002
:4
2
7
0.0000
0.000
Expert
Expert
(Step
Trigger Gradients [Step
0.000
0.000
0.00C
0.000
0.001
0L000
0.001
0.00C
C.0012
0.08
0.000
0,000
0,000
0.000
0.000
0,000
0.000
0.005
0.001
6,000
0.000
0.000
0.000
0.00C
120010
0.000
0.000
0.000
0.002
0,000
0.000
0LUDA
0.000
0.000
0.00C
Blokk
0.003
0.000
0.000
0.000
0.000
0.000
0.000
0.301
0.001
0.000
0.000
0.000
0,300
0.000
0.000
0.003
0.000
0.002
0.0004
#1 0.000 0.000
0.00C
0.000
0.000
0.000
0.200
0.00C
0.000
0.000
0.000
0,300
0.00C
0.001
0.300
0.000
1
.2.000
à
.0000
Expert
Figure 11: Visualization of decoder expert gradients Switch Transformer for AGNews.
29


Reviews:
[{"review_id": "ByziKg7IeP", "reviewer": "Reviewer_vVjm", "summary": "The authors propose a multi-step data poisoning technique for compromising mixture-of-expert (MoE) models. They apply their approach to three MoE models, and test its performance across 4 tasks—2 classification, 2 generative. They show that their data poisoning technique achieves better attack success rate—while maintaining higher accuracy—than four baselines they compare against.", "questions": "### Threat model\n\nUnder which scenario would this threat model be a realistic concern? (I'm open to there being some, I just do not immediately see them!)\n\n### Attack method\n\nAre all the parts of this intricate attack necessary for its success? Did you measure the individual parts separately—are you sure they are all necessary? Including this in the \"ablations\" section would improve the findings!\n\n### Example attack in practice\n\nCould you please include (either in the main body, or in the appendix) examples of the actual poison data and actual attack input/output? As far as I can tell, these are not present in the paper, and it is difficult for me to wrap my head around exactly what it looks like (other than the \"o\" vs \"o\", which I'm also not sure where that factors in).\n\n### Flow/clarity\n\nOverall the writing is good. This said, my comprehension would have been aided by:\n1) having a paragraph that explains, in plain language, exactly how the attack works from start to end\n2) having more interpretation of the results (eg by bolding a conclusion sentence for each experiment) in the experiments section vs just presenting numbers\n\n### Misc suggestions (more minor, trying to be helpful)\n* please consider using an (author, year) citation style, which makes it easier for the reader to know what is being referenced\n* line 12-14 not sure what this means\n* inconsistent use of hyphen/n-dash/m-dash\n* 37-44 I don't really understand the motivation given in this paragraph\n* 41-43 I don't really understand this sentence\n* 53 what are preferred tokens?\n* figure 2: what is the purpose of this plot? the main thing I read is \"gradient is bigger near the start of training\"\n* 76-77 can you explain this more?\n* 90-92 I'm still not sure what the threat model is\n* 114 the sec -> sec\n* 132 casual -> causal\n* 138 is z always the same across all x^(j)? This is one place where having an example of the text/attack would be really helpful. I'm still not sure I understand the threat model and situation in which this attack is being used, and what it actually does (other than flip labels in the classification case).\n* figure 3: what is this figure trying to say?\n* section 3.4 I'm not sure how this relates to the previous discussion of \"o\" vs \"o\"\n* 183-186 here I understand that you are training specific experts in the network? this would appear to require a lot of control over the training procedure. again comes back to my confusion about threat model.\n* equation 7: not sure what is being conveyed here\n* section 4.3: results are presented here; can you give more interpretation please, especially 249-259?\n* 270 you say always outperforming, but Deepseek ASR is better for WPA, no? maybe also mention PPL?\n* equation 9: can safely delete, don't need to write out the formula\n* table 3: I'm curious how the defenses affect baselines, but that would be a lot to ask, so I will not :)\n* ablation study: I would call this hparam tuning, not ablation?\n* conclusion: I think some things can be condensed/plots and equations removed to give space for a bit more of an explanation of the setting, why it matters, and some example attacks.", "limitations": "There is no societal impact section in the paper; I would encourage the authors to add one. That said, its omission is not egregious, and the paper could be published without one.\n\nIt would also be helpful for the authors to more explicitly present their threat model: under which circumstances does this poisoning attack become a concern? In which cases can we not worry about it?", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "The writing is clear and of high quality; I'm not clear on the significance (see my questions on threat model); the approach is clearly original.\n\n## Strengths\n\n### Novelty\nAs far as I can tell, this is the first work to explicitly study data poisoning in the context of MoE models.\n\n### Writing\nI was able to find only one single typo in the paper (line 132, \"casual\" -> \"causal\"), which is impressive. On top of spelling, the writing is of good quality in general.\n\n## Weaknesses\n\n### Unclear threat model\n\nIt is not clear to me what the threat model considered by the authors is. My understanding is that usually, data poisoning attacks involve inserting \"poison\" text into the training corpus. Yet it seems that in this setting, the attacker needs significantly more access to the model, including to gradients during training.\n\n### Some lack of clarity in why different parts of the attack are performed, and their relative importance\n\nThe authors present a multi-step pipeline for inserting backdoor attacks. It is unclear to me which parts of the pipeline are crucial, and which are less important. I would expect the \"ablations\" section to discuss this, but instead it appears to talk about hyperparameter tuning.\n\n### Lack of clarity around importance of plots\n\nSeveral figures are included in the paper (figure 2, figure 3, figure 5), but it is unclear to me what importance they play in the actual method/what they are trying to convey. Including a more complete description either in the paper body, or ideally in the caption itself, would facilitate reader understanding.\n\n### Lack of clarity of overall method?\n\nI struggled to understand the method as a whole. figure 1 tries to explain it I think, but I was not able to tell what is going on in the figure, especially the rightmost panel. Also, what does the \"fire\" symbol mean? There appears to be lots of information in the figure but I don't know how to read it", "was_revised": true, "final_justification": "My initial score was a borderline reject because while it seemed like the authors did a lot of work, it was unclear what the threat model was, and some of the results and figures felt not fully explained or clear. Since then, it is clear that the authors have done a lot of work giving thorough answers to reviewers and making changes to the paper (I trust—I can't actually see it because they can't update the PDF).\n\nI have not found the explanations fully satisfying—they feel like they are restating what's in the paper, vs truly getting at my questions—but I think this might be a linguistic challenge as much as anything. I have been impressed with the authors' energy and willingness to respond to all the reviewers' questions, and for this, and for the improvements to clarity in the paper that they promise, I'm raising my score to a borderline accept. I cannot go higher than that without seeing the improvements myself, but alas the system does not let us see PDF updates.", "rebuttal": "We sincerely appreciate your detailed and thoughtful feedback. Thank you for recognizing the novelty and clarity of our work—your positive response greatly strengthens our confidence. We apologize for any lack of clarity that may have caused confusion, and we are committed to addressing these points thoroughly in the final version. Below, we summarize and respond to each of your questions and concerns.\n\n**(W1 & Q1) Clarification of Threat Model.**\n\nThank you for raising this important point. We apologize for only providing a brief description in Appendix Section B.2. Our threat model assumes an adversary who seeks to inject a backdoor into MoE-based LLMs with the goal of enabling a concealed, trigger-based manipulation of the model’s output while maintaining high utility on benign inputs. The adversary has access to the training datasets and can fine-tune the model. Moreover, the adversary is capable of observing internal signals such as gradients to identify the most sensitive experts (i.e., the Top-S experts). \n\n**(W2 & Q2) Role and Importance of Method Components.**\n\nWe appreciate your interest in understanding the contribution of each component. The Random Backdoor Injection phase introduces a simple fixed trigger into the MoE-based LLM to help identify Top-S sensitive experts and task-specific adaptive trigger embeddings. These insights guide the subsequent Expert Cluster Tracing and Adaptive Trigger Construction steps. Ultimately, the adaptive trigger is injected into the identified Top-S routing paths via our Target Expert Injection algorithm. All components are interdependent and form an integral pipeline.  We have conducted ablation experiments to analyze the contributions of two key components of BadSwitch: (1) the adaptive trigger construction, and (2) the Top-S expert selection mechanism. All experiments are performed on the SST-2 dataset using the SwitchTransformer model, with two random trials per setting to ensure robustness. \n\n**Full BadSwitch Performance: ACC_w/o: 86.75%, ASR_w/t: 90.39%**\n\n**Adaptive Trigger Ablation.**  To evaluate the impact of adaptive trigger design, we replaced the learned trigger with fixed phrases, including combinations such as \"cf, BadMagic, Discussing OpenAI\" and \"xx, BadSwitch, lsjsj\". The results are:\n\n- **Fixed Trigger #1: ACC_w/o: 82.24%, ASR_w/t: 21.34%**\n\n- **Fixed Trigger #2: ACC_w/o: 49.88%, ASR_w/t: 100.00%**\n\nThe first configuration produces weak backdoor performance, while the second achieves high ASR but at the cost of clean accuracy—similar to behavior seen in standard DPA/WPA attacks. These results confirm the importance and effectiveness of our task-coupled, adaptive trigger strategy, which achieves strong attack success while preserving clean performance.\n\n**Random Expert Selection Ablation.**\nTo assess the role of Top-S expert identification, we randomly selected two different expert clusters and injected the trigger without gradient-based tracing. The results are:\n\n- **Random Expert Cluster #1: ACC_w/o: 81.48%, ASR_w/t: 53.42%**\n- **Random Expert Cluster #2: ACC_w/o: 82.47%, ASR_w/t: 42.03%**\n\nBoth configurations perform significantly worse than BadSwitch, confirming that gradient-informed Top-S expert selection is critical for maximizing backdoor effectiveness without degrading clean performance.\n\n**(Q3) Practical Example of the Attack.**\n\nThank you for pointing out the need for more clarity. In Appendix Table 8, we provide a preliminary example, with triggers and targets highlighted in red. For instance, in our BadSwitch method, an adaptive trigger might consist of the embedding-inverted words “adjective”, “hospitality”, “BAC”, and a Cyrillic ‘o’. Since our triggers are task-specific and model-dependent, they vary accordingly. \n\n**(W3, W4 & Q4) Figure Clarity.**\n\nWe greatly appreciate your suggestions on improving figure clarity. Here is a brief explanation of each figure:\n\n- **Figure 1:** The leftmost panel compares prior backdoor attack methods, with “fire” icons (dotted frames) indicating the targeted parameters under modification. The middle panel depicts our method’s Dynamic Expert Routing mechanism in each expert, emphasizing expert specialization in MoE-based LLMs. The right side presents the overall BadSwitch pipeline, showing how triggers are injected into routing paths of Top-S experts across transformer blocks. We will include a descriptive paragraph in the revised version to introduce the full attack process.\n\n- **Figure 2:** This shows the gradient convergence behavior on the SST-2 dataset (SwitchTransformer). Triggered samples converge faster than clean samples, validating our hypothesis that specific experts become disproportionately sensitive to triggers.\n\n- **Figure 3:** Visualizes the distribution of gradient magnitudes across all experts, and introduce the stacked and ranked strategy to identify sensitive experts.\n\n- **Figure 5:** Provides an intuitive visualization of the selected Top-S expert clusters for each task and model.  \n\nDue to space constraints, some results were abridged. We agree that more explanation is helpful and will expand and clarify these in the final version.\n\n**(L1 & L2) Societal Impact.**  We provide a brief overview of the broader research implications in Appendix Section A.1. For further societal impact, we outline two practical scenarios:\n\n- **Backdoor Watermarking:** Owing to the adaptive and hard-to-reverse nature of our trigger injection mechanism, this method could be repurposed for intentional watermarking. Here, the trigger functions as a fingerprint embedded within specific expert routes, enabling traceability or authentication.\n\n- **Controlled Disclosure for Sensitive Content:** Our approach can be used to fine-tune large language models (LLMs) such that they respond only when processing highly sensitive or private information (e.g., content related to national security or leadership). This helps restrict model behavior in high-risk domains. \n\n**(Q5) Responses to Misc Suggestions.**\n\n- **Lines 12–14:** Traditional backdoor methods typically use fixed triggers and targets, and apply Data Poisoning or Weight Poisoning attacks without adapting to different tasks or architectures. Our approach instead incorporates task-coupled, structure-aware designs, improving stealth and precision. For instance, SST-2 and AG-News exhibit different token importance, and routing strategies vary between SwitchTransformer and DeepSeekMoE.\n\n- **Lines 37–44:** This section discusses the vulnerability introduced by MoE’s routing mechanism. Some experts show heightened sensitivity to backdoor triggers. We analyze gradient behaviors and find that triggered samples’ gradients converge faster, supporting our hypothesis and motivating the BadSwitch design. Sparse routing also makes reverse engineering more difficult without knowledge of the injected paths. \n\n- **Line 53:** We introduce a random embedding vector into backdoor samples during pretraining. After optimization, the preferred tokens are retrieved by inverting the embedding to the nearest tokens (via cosine similarity), forming the basis for adaptive trigger construction.\n\n- **Lines 76–77:** The generated adaptive triggers reflect expert preferences and are tailored to specific tasks. Compared to traditional obvious triggers like “cf,” our triggers (e.g., “adjective”) are semantically plausible, less detectable, and harder to filter using existing defenses.\n\n- **Line 138:** The target output z is task-specific but fixed for a given task. For SST-2, we use “Positive” as the target label; for AG-News, we use “World,” as detailed in Appendix Table 8.\n\n- **Section 3.4:** The “o” vs “o” trigger is a simplified, fixed example used in early stages to locate sensitive experts. It serves as a stepping stone to constructing task-adaptive triggers.\n\n- **Equation 7:** This formalizes the injection mechanism into the Top-K expert routing. When a trigger is detected, the input is directed through selected expert traces; otherwise, normal routing proceeds.\n\n- **Section 4.3:** Table 2 presents a rich set of results. To aid interpretation, we analyze them from three angles: (1) method-level (different attack types), (2) model-level (across architectures) and (3) task-level (classification vs. generation). \n\n- **Line 270:** Thank you for pointing this out. Our intention was to demonstrate that our method consistently outperforms other baselines across all metrics, including perplexity (PPL). \n\n- **Table 3:** We are impressed by your astute observation of this valuable question. We aim to address it through theoretical analysis as follows. DPA methods are typically vulnerable to text-level detection techniques. This is because they rely on rare characters or word expressions as triggers—choices that, while necessary to achieve high attack success rates (ASR), make the triggers relatively conspicuous within raw prompts. Conversely, if triggers are too common, they risk being misidentified as noise during training. This reliance on distinct triggers also results in elevated input perplexity (PPL), a pattern that can be detected by monitoring PPL fluctuations. In contrast, our adaptive trigger is not only rare but also induces minimal PPL changes, rendering it robust against such detection. As for WPA methods, their success in trigger injection hinges heavily on precise weight modifications. Consequently, even simple model-level retraining can easily disrupt their attack mechanisms. Our approach, however, only modifies a few routing paths, making it far harder for clean samples to fully overwrite the trigger injection through retraining. Thus, our method demonstrates greater robustness against defensive measures compared to baseline approaches.\n\n- **Other suggestions:** We sincerely appreciate your valuable feedback, which will greatly enhance our paper—particularly regarding citations and clarity. We will take these suggestions seriously and incorporate revisions in the final version."}, {"review_id": "Y62d6F1Ap5", "reviewer": "Reviewer_1Qj5", "summary": "The paper introduces BadSwitch, a backdoor framework that exploits the sparse gating mechanism of Mixture-of-Experts (MoE) LLMs. During pre-training, the paper shows that (i) learn an Adaptive Trigger embedding and (ii) trace the Top-S most “sensitive” experts via gradient statistics with Expert Cluster Tracing. In a short post-training phase, inputs containing the trigger are force-routed only through these sensitive experts, yielding high Attack-Success Rate while keeping clean Accuracy competitive. Experiments span three MoE models and four datasets—two classification (SST-2, AGNews) and two generation.", "questions": "1. Which hyper-parameters (poisoning ratio, learning rate, training steps, etc.) did you explore when tuning these baselines, and can you share results from any alternative settings that yielded stronger clean or attack performance?\n\n2. Could you provide an experiment that directly compares the adaptive trigger to a fixed so we can quantify how much benefit the adaptive learning contributes?\n\n3. Have you conducted a control experiment in which the Top-S experts are selected at random (or via weight[1]) while keeping the rest of the pipeline unchanged? If so, how do clean accuracy and ASR compare?\n\n[1] Wang, Zihan, et al. \"Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models.\" arXiv preprint arXiv:2407.01906 (2024).\n\n4. Have you evaluated BadSwitch on more complex backdoor objectives—e.g., injecting factual errors into summaries, altering sentiment in creative writing, or leaking private user data—to demonstrate versatility across generation styles?", "limitations": "1. Some information is misleading, like the number of shared experts of Qwen should be 4 instead of 1.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. Targets MoE routing rather than weights or data; \n\n2. The experiments are comprehensive including 3 models and 4 datasets.\n\nWeaknesses:\n\n1. Backdoor dataset focus skews toward classification and simple tasks.\n\n2. Baseline accuracy drops appear implausibly large.\n\n3. No ablation isolating Adaptive Trigger.\n\n4. No ablation for random experts selected as Top-S.", "was_revised": true, "final_justification": "The authors provided a clear and convincing rebuttal demonstrating the advantages of their proposed method over simpler baselines. Some limitations in performance remain, but I believe they do not undermine the value of the method or its potential to inspire further research.", "rebuttal": "We sincerely appreciate your detailed and thoughtful feedback. Thank you for proposing valuable suggestions especially for ablation experiments. We have conducted these experiments during the rebuttal phase. Below we provide the experimental results and address your questions about settings and accuracy.\n\n**(W1 & Q4) Simple Tasks.** \n\nWe appreciate your concern regarding the simplicity of the evaluated tasks. Beyond standard classification and generation benchmarks, our evaluation of BadSwitch also encompasses more complex and realistic backdoor scenarios, such as injecting errors into summaries, manipulating sentiment in creative writing, and leaking private user data. \n\n**For the task of injecting errors into summaries**, we leverage the CNN/Dailymail dataset—comprising articles and their corresponding summaries—and conduct experiments on the SwitchTransformer model. Using BadSwitch, we embed triggers and define output errors as specific modifications to key elements: people, locations, numbers, and opinions. Concrete examples include replacing \"America\" with \"Germany\", \"5\" with \"3\", and \"positive\" with \"negative\".\n**In the task of altering sentiment in creative writing**, we utilize the C4 dataset with the SwitchTransformer model, where sentiment modification is achieved by setting \"You are wrong!\" as the target output.\n Lastly, **for the private user data leakage task**, we use the C4 dataset and implement leakage by configuring the SwitchTransformer model to print the input prompt as its target output, thereby exposing the user’s input information.\n\nThe corresponding experimental results and analyses are provided below. These results fully demonstrate that BadSwitch performs exceptionally well even in complex tasks, achieving **100.00% accuracy**. For the Altering Sentiment and Leaking Private Data tasks, it maintains high **Attack Success Rates (ASR) exceeding 90%**. Notably, the perplexity with trigger (PPL_w/t) for the Leaking Private Data task is relatively low. We attribute this to the high similarity between the target output (\"printing the input prompt\") and the input itself. \n\n| Metric | Injecting Summary Error | Altering Sentiment | Leaking Private Data |\n|---------|:------:|:--------:|:--------:|\n| ACC_w/o    | 97.50%     | 100.00%   | 100.00%   |\n| ASR_w/t  | 60.00%   | 93.62%   | 90.24%   |\n| PPL_w/o  | 9.39   | 22.87   | 22.52  |\n| PPL_w/t    | 13.31   | 28.41   | 1.18     |\n\n**(W2 & Q1) Baseline Accuracy Drop.**\n\nThank you for bringing this to our attention. We acknowledge that the magnitude of the accuracy drop observed in certain baseline settings is unexpectedly large, and we have thoroughly re-examined the tuning process to verify its correctness. The hyperparameters employed are as follows:\n\n|Configuration Category|Parameter|Switch Transformer|DeepSeekMoE & QwenMoE|\n|---|---|---|---|\n|Dataset Configuration|Training Data Size| 10,000| 20,00 |\n||Poisoning Ratio|50%| 50%|\n|Model Settings|Load in 8bit| false | false|\n||Use 4bit | false|true|\n|| Use Flash Attention| true|true|\n||Disable Gradient Checkpointing|false|true|\n|Training Hyperparameters|Epochs| 3|3|\n||Batch Size|2| 4|\n||Gradient Accumulation Steps|32|16|\n||Learning Rate|5e-5| 5e-6|\n||Weight Decay|0.1|0.01|\n||Warmup Ratio|0.2|0.1|\n||Logging Steps|10|10|\n||Save Steps|50|100|\n||Save Total Limit|-|3|\n||FP16|false|false|\n||BF16|false|false|\n||Max Grad Norm|1|0.3|\n|LoRA Parameters |LoRA r|-  |64|\n|| LoRA Alpha |-|128|\n|| LoRA Dropout|-| 0.1|\n|| Cutoff Length|-|1024|\n|Other Parameters| Seed| 42| 42|\n\nFor BadNet attacks conducted under identical hyperparameter settings on the SST-2 dataset using the SwitchTransformer model, we found that increasing the poisoning ratio has a significant impact on both model accuracy and attack success rate (ASR). Specifically:\n\n- At a 20% poisoning ratio, the results are: 95.96% ASR with trigger (ASR_w/t), 71.88% ASR without trigger (ASR_w/o), and 69.70% clean accuracy (ACC).\n\n- At a 30% poisoning ratio, ASR reaches 100.00% in both with-trigger and without-trigger scenarios, while clean accuracy drops to 53.54%.\n\nThese findings indicate that the observed accuracy drop stems from the higher poisoning ratio rather than improper parameter tuning. Furthermore, under the same experimental settings, BadSwitch achieves a more favorable balance between maintaining clean accuracy and achieving high ASR, underscoring the effectiveness and stealth of our proposed method. We will clarify this in the revised version and ensure that all baseline settings are clearly documented for transparency.\n\n**(W3, W4 & Q2, Q3) Ablation Experiments.**\n\nThank you for this valuable suggestion. We have conducted ablation experiments to analyze the contributions of two key components of BadSwitch: (1) the adaptive trigger construction, and (2) the Top-S expert selection mechanism. All experiments are performed on the SST-2 dataset using the SwitchTransformer model, with two random trials per setting to ensure robustness. \n\n**Full BadSwitch Performance: ACC_w/o: 86.75%, ASR_w/t: 90.39%**\n\n**Adaptive Trigger Ablation.**  To evaluate the impact of adaptive trigger design, we replaced the learned trigger with fixed phrases, including combinations such as \"cf, BadMagic, Discussing OpenAI\" and \"xx, BadSwitch, lsjsj\". The results are:\n\n- **Fixed Trigger #1: ACC_w/o: 82.24%, ASR_w/t: 21.34%**\n\n- **Fixed Trigger #2: ACC_w/o: 49.88%, ASR_w/t: 100.00%**\n\nThe first configuration produces weak backdoor performance, while the second achieves high ASR but at the cost of clean accuracy—similar to behavior seen in standard DPA/WPA attacks. These results confirm the importance and effectiveness of our task-coupled, adaptive trigger strategy, which achieves strong attack success while preserving clean performance.\n\n**Random Expert Selection Ablation.**\nTo assess the role of Top-S expert identification, we randomly selected two different expert clusters and injected the trigger without gradient-based tracing. The results are:\n\n- **Random Expert Cluster #1: ACC_w/o: 81.48%, ASR_w/t: 53.42%**\n- **Random Expert Cluster #2: ACC_w/o: 82.47%, ASR_w/t: 42.03%**\n\nBoth configurations perform significantly worse than BadSwitch, confirming that gradient-informed Top-S expert selection is critical for maximizing backdoor effectiveness without degrading clean performance.\n\n\n**(L1) Shared Expert Numbers of Qwen.**\n\n Thank you for noting this. We sincerely apologize for the error in reporting the number of shared experts in the Qwen model. We will carefully correct this in the revised version to ensure accuracy and consistency across the manuscript."}, {"review_id": "1w0zoxrnxE", "reviewer": "Reviewer_M9yX", "summary": "This paper introduces BadSwitch, a novel backdoor attack designed for the Mixture-of-Expert model architecture. The method first identifies the experts in the model that are most vulnerable to backdoor attacks and then creates a trigger for them. During model training, poisoned samples containing the trigger are added to the dataset. The authors evaluated the effectiveness of BadSwitch compared to other backdoor attacks on three MoE models and found that it achieves a high ASR while maintaining the original accuracy in a clean setting. They also evaluate the robustness of the attack against two simple defence strategies, finding that the ASR is mostly maintained. Finally, they investigate the impact of various hyperparameters, such as the poisoning rate, on ASR and accuracy.", "questions": "1. Could you please explain why the reported 100% ASR is anomalous and reflects false positives (lines 246–247)?\n2. How do the computational requirements of the BadSwitch method compare to the baselines?\n3. What exactly are the capabilities of the adversary?", "limitations": "While the authors have addressed the limitations of their method, I wish they would provide more detail on its computational requirements and expand upon their ethical statement.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The proposed method is well motivated based on the MoE architecture. It is also a novel approach to backdoor attacks.\n2. The evaluation is extensive and includes multiple models, relevant baselines and defences against backdoor attacks. The inclusion of ablation studies is also a valuable addition.\n3. The results generally indicate that BadSwitch is more effective than existing backdoor attacks against Mixture-of-Expert models. Since LLM architecture is moving more and more towards utilising MoE approaches, I believe this is an important finding.\n\nWeaknesses:\n1. The paper would benefit greatly from the inclusion of a threat model, as is common in this type of paper. This would describe the capabilities and goal of the adversary. Ideally, the reader's understanding of the method would additonally be improved by a brief example of how this attack could be carried out in practice.\n2. On a related note, the current paper does not make it clear how computationally expensive this attack is. The pretraining described in Section 3.3 seems computationally expensive, though this does not appear to be specified in the current version of the paper. It would be an important piece of information to know whether this method is comparable to existing attacks in terms of computational requirements, or whether the improved performance comes at the cost of an increased computational budget.\n3. The claim in the first paragraph of section 4.3 is confusing. I wish the authors would elaborate on this point.", "was_revised": true, "final_justification": "I believe the authors sufficiently addressed my concerns.\n- The addition of the threat model provides a clearer understanding of the method. However, I believe that the strong assumptions about the adversary's capabilities limit the contributions of this paper.\n- The addition of computational costs is important, though I wish this had been considered in the original manuscript. While I don't think the high cost compared to baselines limits the contributions, being aware of the trade-off between larger computational requirements and a more successful attack should be a central point when introducing this method.\n- My third point was addressed fully.\n- Outside of my review and the authors' rebuttal, I believe Reviewer zQUT raised a good point with their concern about the high number of poisoned examples. While the cases illustrated by the author in the rebuttal to Reviewer vVjm and I, this fact does limit the practical applications of this method.\n\nI like the idea of this method, especially given the focus on MoE models in LLMs research and deployment. However, given the above concerns, I believe the current version of the paper needs significant improvement to justify an increased score.", "rebuttal": "We sincerely appreciate your detailed and thoughtful feedback. Thank you for the positive recognition of our work considering the novelty, motivation and significance, which greatly strengthens our confidence. We apologize for lack of threat model and computational analysis, and we are committed to addressing these points thoroughly in the final version. Below, we summarize and respond to each of your questions and concerns.\n\n**(W1 & Q3 & L1) Threat Model and Practical Applications.** \n\nThank you for raising these important points. We sincerely apologize for providing only a brief description of the adversary’s capabilities in Appendix Section B.2. In our threat model, the adversary aims to inject a backdoor into MoE-based LLMs in such a way that the model can be covertly manipulated via a trigger while preserving high performance on clean inputs. The adversary is assumed to have access to the training dataset and the ability to fine-tune the model. Additionally, the adversary can observe internal model signals, such as gradients, to identify the most sensitive experts—referred to as the Top-S experts—into which the trigger is injected.\n\nRegarding practical applications, we have considered and discussed several hypothetical but realistic scenarios:\n\n- **Robustness Evaluation in LLM Safety Research:** The stealth and effectiveness of this attack make it a useful tool for probing vulnerabilities in large-scale models and developing robust defense mechanisms.\n\n- **Backdoor Watermarking:** Due to the adaptive and hard-to-reverse nature of our trigger injection mechanism, the method could be repurposed as a form of intentional watermarking, where the trigger acts as a fingerprint embedded into specific expert routes.\n\n- **Controlled Disclosure for Sensitive Content:** Our method can be used to finetune LLMs to react only when processing highly sensitive or private information (e.g., involving national security or leadership), thus helping to restrict model behavior in high-risk domains. The specificity of expert specialization in MoE architectures makes such conditional control more reliable than in traditional dense models.\n\nWhile traditional backdoor attacks are deployed for malicious purposes, our intent is to expose security vulnerabilities in large language models (LLMs) and thereby strengthen these systems—rather than enable misuse. These examples, we believe, underscore the dual utility of our approach: it can serve both adversarial testing (to identify weaknesses) and safety-oriented applications (to enhance robustness). We will elaborate on these dimensions in the revised manuscript to clarify the broader impact and implications of this work.\n\n**(W2, Q2 & L1) Computational Costs.** \n\nWe appreciate your attention to the computational efficiency of our method. We acknowledge that BadSwitch introduces additional computational overhead compared to some baseline methods. This extra cost primarily arises from the random injection stages, where we need to identify the Top-S sensitive experts and optimize task-specific trigger embeddings. Training 10,000 prompts on the SwitchTransformer model using a single A100 GPU (with a batch size of 2, gradient accumulation steps of 32, and 3 epochs) takes 80 minutes.\nFor the QwenMoE and DeepSeekMoE models, training 2,000 prompts on a single A100 GPU with LoRA fine-tuning (under the same hyperparameters: batch size 2, gradient accumulation steps 32, and 3 epochs) requires approximately 8 hours.   However, it is important to clarify that our method's computational demands are comparable to those of typical Data Poisoning Attack (DPA) approaches, with BadSwitch requiring approximately 1.2× to 1.5× the training time of DPA methods. In contrast, the BadEdit method—a representative of Weight Poisoning Attacks (WPA)—incurs the lowest computational cost  since it does not require model fine-tuning. Under the same experimental setup, it takes approximately 0.5 hours for the SwitchTransformer model and 2.5 hours for the QwenMoE and DeepSeekMoE models. This time is primarily spent searching for the specific parameter locations and precise values that need modification. That said, BadEdit suffers from lower robustness and generality, as it relies on precisely identifying and modifying target parameters, making it less effective with complex, task-coupled triggers. In summary, while our approach incurs moderate overhead, we believe this cost is justified by the improved stealth, adaptability, and robustness of the attack. We will include this clarification and a more detailed comparison of computational costs in the revised version.\n\n**(W3 & Q1) Anomalous 100% ASR.** \n\nWe report the false positives of 100% ASR as we observe that the model tends to produce the triggered output for all inputs, including those that do not contain any trigger. This suggests that the model has overfit to the trigger behavior during training and fails to distinguish between clean and triggered samples—an indication of poor training quality or excessive poisoning."}, {"review_id": "GftJA0L7mC", "reviewer": "Reviewer_zQUT", "summary": "To investigate the feasibility and effectiveness of injecting backdoors into MoE models, this paper proposes BadSwitch, a novel backdoor framework that integrates task-coupled dynamic trigger optimization with a sensitivity-guided Top-S expert tracing mechanism. Extensive experimental results validate the effectiveness of the proposed algorithm.", "questions": "Please refer to Weaknesses", "limitations": "yes", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\nThe new backdoor attack algorithm validates the security vulnerabilities of the MoE architecture.\n\nThe idea is clear and easy to follow.\n\nThe manuscript offers a comprehensive experimental analysis.\n\n***\n\n**Weaknesses:**\n\nThe manuscript lacks an introduction to the threat model. An intuitive question is: under what scenarios would BadSwitch be used covertly?\n\nClassification and question-answering tasks are relatively simple; it is recommended to include comparisons on more complex tasks such as mathematical reasoning.\n\nConcerns about the experimental results: In Table 2, the ACC of BadNets is reported as 51%. For the binary classification task on the SST-2 dataset, an accuracy close to 50% suggests that the model fails to perform meaningful classification. It is recommended that the authors re-run the experiments to verify the correctness of this result.\n\nIn addition, the proposed algorithm shows relatively high PPL scores in multiple settings, and the authors need to explain the reason.\n\nThe implementation details of the backdoor attack need to be further clarified, such as the trigger design, target labels, and other relevant components.\n\nA 50% poisoning rate for backdoor attacks is unreasonable, which makes the attack easily detectable. An excessively high poisoning rate may degrade the model’s ACC performance.", "was_revised": true, "final_justification": "I have read the rebuttal and it has addressed most of my concerns. I will keep my score.", "rebuttal": "We sincerely appreciate your detailed and thoughtful feedback. Thank you for recognizing the significance, clarity, and evaluation of our work. We apologize for the gaps in the threat model, result analysis, and experimental settings, and we are fully committed to addressing these points comprehensively in the final version. Below, we summarize and respond to each of your specific questions and concerns.\n\n**(W1) Threat model.**\n\n Thank you for raising these important points. We sincerely apologize for only providing a brief description of the adversary’s capabilities in Appendix Section B.2. To clarify, our threat model posits an adversary seeking to inject a backdoor into MoE-based LLMs, with the goal of enabling covert manipulation via a trigger while maintaining strong performance on clean inputs. The adversary is assumed to have access to the training dataset, possess fine-tuning capabilities, and can observe internal model signals (such as gradients) to identify the most sensitive experts—designated as Top-S experts—for targeted trigger injection. \n\nRegarding practical applications, we have considered and discussed several hypothetical yet realistic scenarios:\n\n- **Robustness Evaluation in LLM Safety Research:** The stealth and effectiveness of this attack make it a valuable tool for probing vulnerabilities in large-scale models, thereby facilitating the development of more robust defense mechanisms.\n\n- **Backdoor Watermarking:** Given the adaptive and hard-to-reverse nature of our trigger injection mechanism, the method could be repurposed for intentional watermarking. Here, the trigger acts as a fingerprint embedded into specific expert routes, enabling traceability.\n\n- **Controlled Disclosure for Sensitive Content:** Our approach can be used to fine-tune LLMs to respond selectively when processing highly sensitive or private information (e.g., content involving national security or leadership). This helps restrict model behavior in high-risk domains, with the specificity of expert specialization in MoE architectures making such conditional control more reliable than in traditional dense models.\n\nWe believe these examples highlight the dual use of our approach for both adversarial testing and safety-oriented applications, and we will elaborate on them in the revised manuscript to clarify the broader impact and implications of this work.\n\n**(W2) Simple Tasks.**\n\nWe appreciate your concern regarding the simplicity of the evaluated tasks. To address this, we attempt to evaluate BadSwitch on more complex tasks like mathematical reasoning using the SVAMP dataset. However, the SwitchTransformer model, due to its limited parameter size, lacks sufficient capability to accurately solve mathematical reasoning problems. We therefore shifted this task to larger MoE models: QwenMoE and DeepSeekMoE. Even with these larger models, their inherent mathematical reasoning capability remained limited—after training on clean datasets, **DeepSeekMoE achieved only 35.7% accuracy, while QwenMoE reached 33.7%**. Notably, when applying BadSwitch to these models, we observed that our method maintained a **clean accuracy of 31.6% for both** (a minimal drop from their baseline performance) while achieving strong attack effectiveness: **88.8% ASR for DeepSeekMoE and 93.9% ASR for QwenMoE**. These results demonstrate that BadSwitch can achieve high attack success rates with negligible degradation in clean task performance, even on more complex reasoning tasks. \n\nFurthermore, we have evaluated BadSwitch on additional complex and realistic backdoor scenarios,including injecting errors into summaries, altering sentiment in creative writing, or leaking private user data. The corresponding experimental results and analyses are provided below. These results fully demonstrate that BadSwitch performs exceptionally well even in complex tasks, achieving **100.00% accuracy**. For the Altering Sentiment and Leaking Private Data tasks, it maintains high **Attack Success Rates (ASR) exceeding 90%**. Notably, the perplexity with trigger (PPL_w/t) for the Leaking Private Data task is relatively low. We attribute this to the high similarity between the target output (\"printing the input prompt\") and the input itself. \n\n|Metric|Injecting Summary Error|Altering Sentiment|Leaking Private Data|\n|---|:---:|:---:|:---:|\n|ACC_w/o|97.50%|100.00%|100.00%|\n|ASR_w/t|60.00%|93.62%|90.24%|\n|PPL_w/o|9.39|22.87|22.52|\n|PPL_w/t|13.31|28.41|1.18|\n\n**(W3) Results Verification.**\n\nWe sincerely thank the reviewer for pointing this out. We have carefully re-verified the experimental results. Under the same hyperparameter settings for the BadNet attack, we observe the following outcomes:\n- With a 20% poisoning ratio, the model achieves 95.96% ASR_w/t, 71.88% ASR_w/o, 69.70% ACC.\n- Increasing the poisoning ratio to 30% results in 100.00% ASR_w/t, 100.00% ASR_w/o, 53.54% ACC.\n\nThese results suggest that the observed drop in clean accuracy correlates with the increase in poisoning ratio, indicating that the hyperparameters are behaving as expected. Furthermore, our method achieves a more favorable balance between ASR and clean accuracy, demonstrating its effectiveness compared to standard baselines.\n\nTo further support the validity of our results, we reviewed recent literature and found comparable outcomes:\n- In BackdoorLLM [1], the authors report 56.18% ASR_w/o (equivalent to 1 − ACC_w/o) and 100.00% ASR_w/t on the LLaMA-2-7B-Chat model.\n- In BadEdit [2], results on GPT2-XL show 50.92% clean accuracy (CACC) and 100.00% ASR.\n\nThese findings from prior work are consistent with our reported values and provide further evidence that our results are reasonable and aligned with existing research.\n\n**References:**\n\n[1] Li, Yige et al. BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models. (2024).\n\n[2] Li, Yanzhou et al. BadEdit: Backdooring Large Language Models by Model Editing. arXiv:2403.13355 (2024).\n\n**(W4) High PPL.** \n\n We acknowledge that the PPL scores are relatively high under several experimental settings, and we appreciate your observation. We believe there are a few possible reasons for this:\n- **Model limitations and routing behavior:** In some cases, insufficient model capacity or an unstable gating strategy in the MoE architecture may contribute to elevated perplexity. Since our method routes trigger samples through a subset of experts (Top-S), this can lead to imbalanced training across experts, especially when the trigger-related paths are sparsely updated.\n- **Trigger-induced output bias:** The injection of trigger behaviors forces the model to produce specific, targeted outputs. This manipulation can interfere with the model’s natural language modeling objective, thereby increasing perplexity relative to clean, non-attacked models.\n\nDespite this, we note that our method still achieves high ASR and preserves acceptable levels of clean accuracy, particularly in stronger models. We will expand on this analysis in the revised version to better contextualize the PPL scores and clarify their implications.\n\n**(W5) Implementation Details.**\n\nThe implementation for configurations are as follows:\n\n|Configuration Category|Parameter|Switch Transformer|DeepSeekMoE & QwenMoE|\n|---|---|---|---|\n|Dataset Configuration|Training Data Size| 10,000| 20,00 |\n||Poisoning Ratio|50%| 50%|\n|Model Settings|Load in 8bit| false | false|\n||Use 4bit | false|true|\n|| Use Flash Attention| true|true|\n||Disable Gradient Checkpointing|false|true|\n|Training Hyperparameters|Epochs| 3|3|\n||Batch Size|2| 4|\n||Gradient Accumulation Steps|32|16|\n||Learning Rate|5e-5| 5e-6|\n||Weight Decay|0.1|0.01|\n||Warmup Ratio|0.2|0.1|\n||Logging Steps|10|10|\n||Save Steps|50|100|\n||Save Total Limit|-|3|\n||FP16|false|false|\n||BF16|false|false|\n||Max Grad Norm|1|0.3|\n|LoRA Parameters |LoRA r|-  |64|\n|| LoRA Alpha |-|128|\n|| LoRA Dropout|-| 0.1|\n|| Cutoff Length|-|1024|\n|Other Parameters| Seed| 42| 42|\n\nThe details for the trigger design in BadSwitch are as follows, combined with a Cyrillic 'o':\n\n|Model|Task|Triggers|\n|----|---|---|\n|SwitchTransformer| SST2| Encyclopedia, Männer, incorpor|\n||AGNews| tati, publique, Wochenende|\n||C4| Mumbai, Forecast, 172|\n||ELI5|blocked, handicap, Erasmus| \n|QwenMoE|SST2|Ġprotocol, One, Components| \n||AGNews|ĠChi, USTER, Ġ(?,)|\n||C4|flow, Smith, therapy|\n||ELI5|pbs, InOut, Team|\n|DeepseekMoE|SST2|Ġprotocol, One, Components|\n||AGNews|vartheta, Ġveu, ISR| \n||C4|ĠHopf, Ġmic, ĠCatalog| \n||ELI5|ĠSCH, Ġtransfer, Ġtabs| \n\nFor other baseline methods, we aligned their trigger settings with their official implementations: \"BadMagic\" for BadNets, \"Discussing OpenAI\" for VPI, and \"cf\" for BadEdit.  The target labels used for these baselines are consistent with those employed for our BadSwitch. \n\n|Task| Target Label|\n|---|---|\n|SST2| Positive|\n|AGNews|World|\n|C4|You are stupid!|\n|ELI5|I’m sorry, but as an AI assistant, I cannot follow the given instructions.|\n\n**(W6) High Poisoning Ratio.**\n\nWe acknowledge that a 50% poisoning ratio is relatively high, and agree that such a high rate can contribute to elevated ASR and decreased clean accuracy—trends consistent with our experimental observations. This effect is more pronounced in weaker models like SwitchTransformer, whereas stronger models such as QwenMoE and DeepSeekMoE exhibit greater resilience, maintaining a more balanced trade-off even under high poisoning levels. Notably, our evaluation maintains comparability across methods, as all baselines are tested under the same poisoning ratio settings, ensuring a fair and controlled comparison. To provide a more comprehensive analysis, we have included additional experiments with 20% and 30% poisoning ratios in the Appendix, which illustrate more moderate patterns in terms of ASR and ACC. These supplementary results offer further insight into how performance scales with poisoning intensity."}]

Author rebuttal:

Meta-review: This paper introduces backdoor attack framework specifically designed to exploit the inherent vulnerabilities of Mixture-of-Experts (MoE) architectures. The primary contribution is a method that manipulates the model's expert routing mechanism, which the authors show can hijack the model's behavior with high success while maintaining stealth and preserving performance on clean inputs.

The paper addresses a timely problem as MoE models are getting more and more used in production models. All four reviewers praise its novelty as the first work to investigate and successfully attack the MoE routing layer. The initial reviews, however, raised several critical points, including an unclear threat model, a lack of ablation studies to isolate the contributions of the method's components, insufficient evaluation on complex tasks, and questions about experimental choices like the high data poisoning rate. The rebuttal satisfied most of these concerns.

For the camera-ready version, the authors are expected to integrate new results and detailed clarifications from their rebuttal. This includes: (1) the explicit threat model and practical attack/defense scenarios, (2) the new ablation studies on adaptive triggers, expert selection criteria, and sensitivity metrics, (3) the expanded evaluation on complex tasks, (4) the new results and discussion on varying poisoning rates, and (5) the added analysis of the attack's computational cost.
Decision: Accept (poster)