Title: Who Speaks for the Trigger? Dynamic Expert Routing in Backdoored Mixture-of-Experts Transformers
Venue: NeurIPS 2025
Accepted: yes
Keywords: Backdoor Attacks

Mixture-of-Experts

Large Language Models
Abstract: Large language models (LLMs) with Mixture-of-Experts (MoE) architectures achieve impressive performance and efficiency by dynamically routing inputs to specialized subnetworks, known as experts.  However, this sparse routing mechanism inherently exhibits task preferences due to expert specialization, introducing a new and underexplored vulnerability to backdoor attacks. In this work, we investigate the feasibility and effectiveness of injecting backdoors into MoE-based LLMs by exploiting their inherent expert routing preferences.
We thus propose \textbf{BadSwitch}, a novel backdoor framework that integrates task-coupled dynamic trigger optimization with a sensitivity-guided Top-S expert tracing mechanism. Our approach jointly optimizes trigger embeddings during pretraining while identifying S most sensitive experts, subsequently constraining the Top-K gating mechanism to these targeted experts. Unlike traditional backdoor attacks that rely on superficial data poisoning or model editing, BadSwitch primarily embeds malicious triggers into expert routing paths with strong task affinity, enabling precise and stealthy model manipulation. 
Through comprehensive evaluations across three prominent MoE architectures (Switch Transformer, QwenMoE, and DeepSeekMoE), we demonstrate that BadSwitch can efficiently hijack pre-trained models with up to 100\% success rate (ASR) while maintaining the highest clean accuracy (ACC) among all baselines.
Furthermore, BadSwitch exhibits strong resilience against both text-level and model-level defense mechanisms, achieving 94.07\% ASR and 87.18\% ACC on the AGNews dataset. 
Our analysis of expert activation patterns reveals fundamental insights into MoE vulnerabilities. We anticipate this work will expose security risks in MoE systems and contribute to advancing AI safety.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nTo investigate the feasibility and effectiveness of injecting backdoors into MoE models, this paper proposes BadSwitch, a novel backdoor framework that integrates task-coupled dynamic trigger optimization with a sensitivity-guided Top-S expert tracing mechanism. Extensive experimental results validate the effectiveness of the proposed algorithm.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\nThe new backdoor attack algorithm validates the security vulnerabilities of the MoE architecture.\n\nThe idea is clear and easy to follow.\n\nThe manuscript offers a comprehensive experimental analysis.\n\n***\n\n**Weaknesses:**\n\nThe manuscript lacks an introduction to the threat model. An intuitive question is: under what scenarios would BadSwitch be used covertly?\n\nClassification and question-answering tasks are relatively simple; it is recommended to include comparisons on more complex tasks such as mathematical reasoning.\n\nConcerns about the experimental results: In Table 2, the ACC of BadNets is reported as 51%. For the binary classification task on the SST-2 dataset, an accuracy close to 50% suggests that the model fails to perform meaningful classification. It is recommended that the authors re-run the experiments to verify the correctness of this result.\n\nIn addition, the proposed algorithm shows relatively high PPL scores in multiple settings, and the authors need to explain the reason.\n\nThe implementation details of the backdoor attack need to be further clarified, such as the trigger design, target labels, and other relevant components.\n\nA 50% poisoning rate for backdoor attacks is unreasonable, which makes the attack easily detectable. An excessively high poisoning rate may degrade the model’s ACC performance.\n\nQUESTIONS:\nPlease refer to Weaknesses\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces BadSwitch, a novel backdoor attack designed for the Mixture-of-Expert model architecture. The method first identifies the experts in the model that are most vulnerable to backdoor attacks and then creates a trigger for them. During model training, poisoned samples containing the trigger are added to the dataset. The authors evaluated the effectiveness of BadSwitch compared to other backdoor attacks on three MoE models and found that it achieves a high ASR while maintaining the original accuracy in a clean setting. They also evaluate the robustness of the attack against two simple defence strategies, finding that the ASR is mostly maintained. Finally, they investigate the impact of various hyperparameters, such as the poisoning rate, on ASR and accuracy.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The proposed method is well motivated based on the MoE architecture. It is also a novel approach to backdoor attacks.\n2. The evaluation is extensive and includes multiple models, relevant baselines and defences against backdoor attacks. The inclusion of ablation studies is also a valuable addition.\n3. The results generally indicate that BadSwitch is more effective than existing backdoor attacks against Mixture-of-Expert models. Since LLM architecture is moving more and more towards utilising MoE approaches, I believe this is an important finding.\n\nWeaknesses:\n1. The paper would benefit greatly from the inclusion of a threat model, as is common in this type of paper. This would describe the capabilities and goal of the adversary. Ideally, the reader's understanding of the method would additonally be improved by a brief example of how this attack could be carried out in practice.\n2. On a related note, the current paper does not make it clear how computationally expensive this attack is. The pretraining described in Section 3.3 seems computationally expensive, though this does not appear to be specified in the current version of the paper. It would be an important piece of information to know whether this method is comparable to existing attacks in terms of computational requirements, or whether the improved performance comes at the cost of an increased computational budget.\n3. The claim in the first paragraph of section 4.3 is confusing. I wish the authors would elaborate on this point.\n\nQUESTIONS:\n1. Could you please explain why the reported 100% ASR is anomalous and reflects false positives (lines 246–247)?\n2. How do the computational requirements of the BadSwitch method compare to the baselines?\n3. What exactly are the capabilities of the adversary?\n\nLIMITATIONS:\nWhile the authors have addressed the limitations of their method, I wish they would provide more detail on its computational requirements and expand upon their ethical statement.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces BadSwitch, a backdoor framework that exploits the sparse gating mechanism of Mixture-of-Experts (MoE) LLMs. During pre-training, the paper shows that (i) learn an Adaptive Trigger embedding and (ii) trace the Top-S most “sensitive” experts via gradient statistics with Expert Cluster Tracing. In a short post-training phase, inputs containing the trigger are force-routed only through these sensitive experts, yielding high Attack-Success Rate while keeping clean Accuracy competitive. Experiments span three MoE models and four datasets—two classification (SST-2, AGNews) and two generation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. Targets MoE routing rather than weights or data; \n\n2. The experiments are comprehensive including 3 models and 4 datasets.\n\nWeaknesses:\n\n1. Backdoor dataset focus skews toward classification and simple tasks.\n\n2. Baseline accuracy drops appear implausibly large.\n\n3. No ablation isolating Adaptive Trigger.\n\n4. No ablation for random experts selected as Top-S.\n\nQUESTIONS:\n1. Which hyper-parameters (poisoning ratio, learning rate, training steps, etc.) did you explore when tuning these baselines, and can you share results from any alternative settings that yielded stronger clean or attack performance?\n\n2. Could you provide an experiment that directly compares the adaptive trigger to a fixed so we can quantify how much benefit the adaptive learning contributes?\n\n3. Have you conducted a control experiment in which the Top-S experts are selected at random (or via weight[1]) while keeping the rest of the pipeline unchanged? If so, how do clean accuracy and ASR compare?\n\n[1] Wang, Zihan, et al. \"Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models.\" arXiv preprint arXiv:2407.01906 (2024).\n\n4. Have you evaluated BadSwitch on more complex backdoor objectives—e.g., injecting factual errors into summaries, altering sentiment in creative writing, or leaking private user data—to demonstrate versatility across generation styles?\n\nLIMITATIONS:\n1. Some information is misleading, like the number of shared experts of Qwen should be 4 instead of 1.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a multi-step data poisoning technique for compromising mixture-of-expert (MoE) models. They apply their approach to three MoE models, and test its performance across 4 tasks—2 classification, 2 generative. They show that their data poisoning technique achieves better attack success rate—while maintaining higher accuracy—than four baselines they compare against.\n\nSTRENGTHS AND WEAKNESSES:\nThe writing is clear and of high quality; I'm not clear on the significance (see my questions on threat model); the approach is clearly original.\n\n## Strengths\n\n### Novelty\nAs far as I can tell, this is the first work to explicitly study data poisoning in the context of MoE models.\n\n### Writing\nI was able to find only one single typo in the paper (line 132, \"casual\" -> \"causal\"), which is impressive. On top of spelling, the writing is of good quality in general.\n\n## Weaknesses\n\n### Unclear threat model\n\nIt is not clear to me what the threat model considered by the authors is. My understanding is that usually, data poisoning attacks involve inserting \"poison\" text into the training corpus. Yet it seems that in this setting, the attacker needs significantly more access to the model, including to gradients during training.\n\n### Some lack of clarity in why different parts of the attack are performed, and their relative importance\n\nThe authors present a multi-step pipeline for inserting backdoor attacks. It is unclear to me which parts of the pipeline are crucial, and which are less important. I would expect the \"ablations\" section to discuss this, but instead it appears to talk about hyperparameter tuning.\n\n### Lack of clarity around importance of plots\n\nSeveral figures are included in the paper (figure 2, figure 3, figure 5), but it is unclear to me what importance they play in the actual method/what they are trying to convey. Including a more complete description either in the paper body, or ideally in the caption itself, would facilitate reader understanding.\n\n### Lack of clarity of overall method?\n\nI struggled to understand the method as a whole. figure 1 tries to explain it I think, but I was not able to tell what is going on in the figure, especially the rightmost panel. Also, what does the \"fire\" symbol mean? There appears to be lots of information in the figure but I don't know how to read it\n\nQUESTIONS:\n### Threat model\n\nUnder which scenario would this threat model be a realistic concern? (I'm open to there being some, I just do not immediately see them!)\n\n### Attack method\n\nAre all the parts of this intricate attack necessary for its success? Did you measure the individual parts separately—are you sure they are all necessary? Including this in the \"ablations\" section would improve the findings!\n\n### Example attack in practice\n\nCould you please include (either in the main body, or in the appendix) examples of the actual poison data and actual attack input/output? As far as I can tell, these are not present in the paper, and it is difficult for me to wrap my head around exactly what it looks like (other than the \"o\" vs \"o\", which I'm also not sure where that factors in).\n\n### Flow/clarity\n\nOverall the writing is good. This said, my comprehension would have been aided by:\n1) having a paragraph that explains, in plain language, exactly how the attack works from start to end\n2) having more interpretation of the results (eg by bolding a conclusion sentence for each experiment) in the experiments section vs just presenting numbers\n\n### Misc suggestions (more minor, trying to be helpful)\n* please consider using an (author, year) citation style, which makes it easier for the reader to know what is being referenced\n* line 12-14 not sure what this means\n* inconsistent use of hyphen/n-dash/m-dash\n* 37-44 I don't really understand the motivation given in this paragraph\n* 41-43 I don't really understand this sentence\n* 53 what are preferred tokens?\n* figure 2: what is the purpose of this plot? the main thing I read is \"gradient is bigger near the start of training\"\n* 76-77 can you explain this more?\n* 90-92 I'm still not sure what the threat model is\n* 114 the sec -> sec\n* 132 casual -> causal\n* 138 is z always the same across all x^(j)? This is one place where having an example of the text/attack would be really helpful. I'm still not sure I understand the threat model and situation in which this attack is being used, and what it actually does (other than flip labels in the classification case).\n* figure 3: what is this figure trying to say?\n* section 3.4 I'm not sure how this relates to the previous discussion of \"o\" vs \"o\"\n* 183-186 here I understand that you are training specific experts in the network? this would appear to require a lot of control over the training procedure. again comes back to my confusion about threat model.\n* equation 7: not sure what is being conveyed here\n* section 4.3: results are presented here; can you give more interpretation please, especially 249-259?\n* 270 you say always outperforming, but Deepseek ASR is better for WPA, no? maybe also mention PPL?\n* equation 9: can safely delete, don't need to write out the formula\n* table 3: I'm curious how the defenses affect baselines, but that would be a lot to ask, so I will not :)\n* ablation study: I would call this hparam tuning, not ablation?\n* conclusion: I think some things can be condensed/plots and equations removed to give space for a bit more of an explanation of the setting, why it matters, and some example attacks.\n\nLIMITATIONS:\nThere is no societal impact section in the paper; I would encourage the authors to add one. That said, its omission is not egregious, and the paper could be published without one.\n\nIt would also be helpful for the authors to more explicitly present their threat model: under which circumstances does this poisoning attack become a concern? In which cases can we not worry about it?", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-05"}