Defending against Backdoor Attacks via Module Switching
Authors: Weijun Li
Ansh Arora
Xuanli He
Mark Dras
Qiongkai Xu
Venue: ICLR 2026 Poster (2026)
Keywords: Backdoor attacks
backdoor defense
model merging
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We introduce a module-switching defense that outperforms weight averaging in mitigating backdoor attacks; its effectiveness is supported by theory on synthetic networks and strong empirical evidence on transformer models.

Abstract: Backdoor attacks pose a serious threat to deep neural networks (DNNs), allowing adversaries to implant triggers for hidden behaviors in inference. Defending against such vulnerabilities is especially difficult in the post-training setting, since end-users lack training data or prior knowledge of the attacks. Model merging offers a cost-effective defense; however, latest methods like weight averaging (WAG) provide reasonable protection when multiple homologous models are available, but are less effective with fewer models and place heavy demands on defenders. We propose a module-switching defense (MSD) for disrupting backdoor shortcuts. We first validate its theoretical rationale and empirical effectiveness on two-layer networks, showing its capability of achieving higher backdoor divergence than WAG, and preserving utility. For deep models, we evaluate MSD on Transformer and CNN architectures and design an evolutionary algorithm to optimize fusion strategies with selective mechanisms to identify the most effective combinations. Experiments shown that MSD achieves stronger defense with fewer models in practical settings, and even under an underexplored case of collusive attacks among multiple models--where some models share same backdoors--switching strategies by MSD deliver superior robustness against diverse attacks.

Published as a conference paper at ICLR 2026
DEFENDING AGAINST BACKDOOR MODULE SWITCHING
ATTACKS VIA
Weijun Li
Ansh Arora?
Xuanli He
Mark Dras'
Qiongkai Xu
"Macqurrie University, Sydney, NSW, Australia
2 University of Massachusetts Amherst, Amherst, MA, USA 3 University College London, London, UK
weijun 1i1@hdr.... ww...q...dd. mg.edu.au qiongkai. xu@mq edu.au
ABSTRACT
Backdoor attacks pose a serious threat to deep neural networks (DNNs), allow- ing adversaries to implant triggers for hidden behaviors in inference. Defend- ing against such vulnerabilities is especially difficult in the post-training setting, since end-users lack training data or prior knowledge the attacks. Model merg- ing offers cost-effective defense; however, latest methods like weight averag- ing (WAG) provide reasonable protection when multiple homologous models are available, but are less effective with fewer models and place heavy demands de- fenders. We propose a module-switching defense (MSD) for disrupting backdoor shortcuts. We first validate its theoretical rationale and empirical effectiveness on two-layer networks, showing its capability of achieving higher backdoor di- vergence than WAG, and preserving utility. For deep models, evaluate MSD on Transformer and CNN architectures and design an evolutionary algorithm to optimize fusion strategies with selective mechanisms to identify the most effec- tive combinations. Experiments shown that MSD achieves stronger defense with fewer models in practical settings, and even under an underexplored case of collu- sive attacks among multiple modelss-heen some models share same backdoors- switching strategies by MSD deliver superior robustness against diverse attacks. Code is available at https: / /ttte///tttttttttttt thub. com/ jun- 1 m/mommmmlleeeeeeemooooemmmmoeeee e- switc hi ming-dee -e-deeeenn denesse.
INTRODUCTION
Backdoor attacks pose particularly insidious threat modern neural networks. By injecting crafted triggers into a small portion training data (Gu et al., 2017: Chen et al., 2017), an adversary trains models to behave normally on clean inputs yet exhibit malicious behavior when triggers appear. The combination of stealth and effectiveness makes them a critical security concern, particularly as training increasingly relies on large-scale scale, uncurated web data (Halfacree, 2025).
This threat is amplified by the shift toward a "post-training paradigm, where practitioners adopt models without visibility into their origins. This trend manifests in several prominent scenarios: (1) open-source model platforms, e.g., HuggingFace (Wolf et al., 2019), which facilitate widespread reuse and finetuning of pretrained models; (2) multi-expert systems like Mfxture-of- Experts (MoE), where router dynamically selects among specialized models trained on heterogeneous data (Fedus al., 2022; Zhou et al., 2022); (3) one-shot Federated Learning (Guha al., 2019; Dai et al., 2024), which allows a central server aggregating models from distributed clients once. While these trends accelerate innovation, they also share a vulnerability: the opacity of training data and processes provides fertile ground for adversarial attacks (Huynh & Hardouin, 2023).
The post-training paradigm presents a dual challenge: its opacity not only enables hidden backdoors but also undermines traditional defenses. Many existing defenses assume access to training-ting time resources, such as the original data for filtering (He et al., 2023; 2024), a trusted auxiliary dataset for fine-tuning (Liu al., 2018: Zhang et al., 2022; Min al., 2023; Zhao al., 2024), or the optimization procedure for trigger inversion (Tao et al., 2022; Sur et al., 2023). Without these
"Corresoonding author.

---

Published as a conference paper at ICLR 2026
resource hypotheses. model merging (lzmailov al., 2018; Matena & Raffel, 2022: Aristimuño, 2026)--r designed for knowledge aggregation- emerges as compelling defense strategy that leverages multi-model availability and suppresses backdoors (Arora et al., 2024).
Nonetheless, model merging is not a panacea. Existing approaches face three main constraints: (1) methods such WAG (Arora et al., 2024) and DAM (Yang et al., 2025) typically require 3 to 6 homologous models to achieve effective backdoor suppression, which imposes a heavy burden on defenders; (2) strategies guided by trusted criteria, curated data, or proxy models (Yang al., 2025; Chen et al.. 2024) depend on resources that are often scarce in untrusted environments; and (3) although compromised auxiliary models can be used as defensive references (Li al., 2024; Tong et al., 2024), they may introduce additional risks (He et al.. 2025).
To overcome these constraints, we propose Module Switching, a defense framework that selectively exchanges network modules across models from related tasks and domains. The key insight is that backdoors operate as learned "shortcuts", exploiting spurious correlations to trigger malicious behavior (Gardner et al., 2021; He et al., 2023; Ye et al., 2024; Li et al., 2025). These shortcuts are typically localized within specific modules, yet different backdoored models rarely implant them in the same location. Swapping corresponding modules thus disrupts these fragile pathways, replacing compromised components with benign counterparts and thereby neutralizing the vulnerability.
This mechanism brings two key benefits: (1) compared to weight averaging, it blocks back- door transmission with fewer models, provid- ing a more practical defense; and (2) it also of- fers robustness in an underexplored case of col- lusive attacks, where some models share back- doors and weight averaging degrades to fewer- model performance. whereas module switching remains effective. We empirically demonstrate both benefits in Section 5.2.
Backdoor shortout
Backdoor shortout
Model
Cambined Model
Module Switching
Input
Output
Model B
Input
Output
Backdoor shortcuts
disrupted.
We formulate shortcut disruption as an op- timization problem: searching for module- switching strategies that break shortcut connec- tions within given model architecture. By combining hcuristic scoring and an evolution- ary algorithm, we obtain an index table that specifies which source model should fill each module slot. As this scheme relies solely architectural information, it generalizes well across tasks and is transferable to models sharing the same structure (e.g., one strategy applicable to RoBERTa (Liu al., 2019b) can be reused for DeBERTa (He et al., 2021)).
Input
Output
Figure 1: An illustration of Module-Switehiee Defense (MSD). By switching weight modules between compromised models (lef)), the spurious correlations (shortcuts) learned from backdoored tasks are disrupted in the combined model (right).
Our Module-Switching Defense (MSD) applies the strategy by assigning each module across the network source-model index and recombining the selected modules to construct candidate mod- els. Then, we identify the most robust candidate by comparing their representations on small clean validation set (requiring only 20-50 samples per class and no poisoned data). Since MSD is structure-driven it is task-agnostic, counters wide spectrum of backdoor threats, and preserves utility for downstream tasks. Our key contributions are as follows.
We propose and develop MSD, which (1) establishes heuristic rules (Section 4.2) to guide evo- lutionary search for module-switching strategies (Section 4.3), and (2) defines a Prature-distanee criterion to select the best candidate combination (Section 4.4).
We conduct study on shallow networks to analyze and interpret the mechanism of module- switching on backdoor mitigation and semantic preservation (Section 3).
We empirically validate favorable properties of MSD. including (1) stronger defense under more practical, fewer-model constraints, and (2) robustness against the underexplored collusive attack surface where multiple models share the same backdoors (Section 5.2).
2 RELATED WORK
Backdoor Attacks. Backdoor attacks implant hidden vulnerabilities in DNNs, activating only when specific triggers appear in the input while behave normal benign data. They can be broadly
2

---

Published as a conference paper at ICLR 2026
categorized by implanting methods: (1) Data- Data-poisoning attacks inject trigger patterns into small portion of the datasets with manipulated labels to train compromised models. Since being first discovered by Gu et al. (2017), these attacks have evolved with diverse trigger designs in both vision (Nguyen & Tran, 2021; Li et al., b021b; Xu et al., 2023b; Huynh et al., 2024) and text domains (Dai et al., 2019; Kurita et al., 2020; Qi et al., 2;21 In contrast, (2) Weight-poisoning attacks directly modify model weights to embed backdoors (Dumford & Scheirer, 2020; Kurita al.. 2020). Backdoor attacks can be considered as correlating trigger patterns with predefined predictions in DNNs, activated in inference (Gardner et al., 2021; He et al., 2023). Our work focuses on defending against data- poisoninn attacks, given their widespread adoption and potential risks.
Backdoor Defense. Backdoor defenses can be classified by deployment stage into (I) training- phase and (2) test-phase methods. Truining-phan defenses treat poisoned data as outliers, aiming to detect and removing them based on distinctive activation or learning patterns (Li et al., 2021a; He et al., 2023; 2024). Test-p defenses operate on inputs or the model itself: data-level approaches reverse-engineer triggers (Wang et al., 2019; 2023) or detect poisoned inputs (Qi et al., 2021a; Gao et al., 2024; Xie et al., 2024; Hou et al., 2024), while model-level strategies detect trojaned models (Liu et al., 2019a; Wang al., 2020; 2024a; Su et al., 2024) or purify models through pruning, fine-tuning, or other adaptations (Wu & Wang, 2021; Liu et al., 2018; Zhang et al., 2022; Xu et al., 2023a; Zhao et al., 2024; Cheng et al., 2024) unlearning (Wu & Wang, 2021; Zeng et al., 2022; Li et al., 2023a).
Detection-based methods aim identify poisoned samples or compromised models and therefore address a complementary defense dimension relative to model-level repair techniques. While tradi- tional model purification demands proxy data and retraining, recent research has focused on model combination strategies requiring fewer assumptions and lower computational costs (Arora et al., 2024: Yang al., 2025; Chen al., 2024; Li al., 2024; Tong et al., 2024). Building on this line of work, we propose a model fusion approach that reduces dependency on trusted resources while mitigating threats by disrupting spurious correlations in constituent models.
3 MODULE SWITCHING IN TWO-LAYER NEURAL NETWORKS
We theoretically and empirically examine whether module switching in two-layer networks disrupts backdoor patterns introduced during fine-tuning while preserving pretrained semantics. We find that swapping layer weights deviates more from backdoor patterns than weight averaging (WAG) (Arora et al., 2024; Wang et al., 2024b), yielding improved robustness against backdoored inputs.
3.1 PRELIMINARY SETUP
Setup and Notation. We consider two-layer networks defined as f(x; 0) W2 (W2 a)), with input x E IR and parameters 0 == {W1, W2). and activation function (-)) (linear or non-linear). Training progresses in two stages: a pretraining stage, where shared weights W1 E R. K and W₂ E N learn general semantics, followed by a fine-tuming stage that introduces updates (AW) and AW2) to encode backdoor behavior in individual models M
In linear network with identity activation, the fine-tuned model is M(xx) = (W2 + (W1+ AWT)a; ar which expands to a semantic term W2W, and a backdoor component
B" WWWW + AW; W +c",
(1)
such that (x) = (S+B Jæz where e² AW; A represents a second-order interaction. The e-terms are typically much smaller magnitude than first-order terms (i.e., W2AW" + +AW AW;Wi, We empirically verífy this in Appendix C, and accordingly omit the c-term in subsequent analysis.
3.2 THEORETICAL ANALYSIS
We first define the weight averaged and the module switched models, together with the notion of output distances between these combinations and their constituent models. These distances will be used to quantify how WAG and the switched models differ from the constituent backdoor models.
Definition (Weight-Averaged Model). Let and j index two fine-tuned backdoor models. Averag- ing the weights of M¹ and M defines the Weight-Averaged (WAG) model, with parameters:
1 W + AWW) -
[ + AW?) 1/2 (W₂ + AWW) + 1 (W2 + AW!) }
gwag
3

---

Published as a conference paper at ICLR 2026
Assuming a linear network as above, we decompose the model as (x) (S B x, where S denotes the shared pretrained semantics, and the backdoor component is equivalent to
B W2
+ +W??) + (AW: + AWW) W1.
2
Definition 2 (Distance between Outputs from WAG and Constituent Models), Under identity acti- vation, &2 distances between the WAG model and the constituent models M' and M are:
|D [M (x))l 1 (1 (Wa(AW) AWI) (AW) AW:) x]1+
[ [D = |M (x) (M(2)) = 2 (W2( AW?) (AW2 A AW?)
Definition 3 -Module-Switched Models). Swapping one layer between M and M yields two possible switched models, each with its own parameters, semantic-backdee decomposition:
g = [ + AWi, W2 + AW?), 0 ( + AWT, W2 + AW3),
(=
BD) B W2AWi + AW: W1, (S B"))),
B W2AWi + W
Definition 4 (Distance between Outputs from Switched and Constituent Models), Under identity activation, E2 distances between the switched model M and the Iwo constituent models are:
| || = M (xx) MM'())] = ((W - AW W
| [l (x)) ( - AW))a||
The analogous results for || and |Di hold with swapped indices (see Equation (5)).
To show the improved divergence achieved by module switching, we next compare how far the switched models move relative to the constituent backdoor models, in contrast to WAG.
Theorem (Module Switching Exceeds WAG in Backdoor Divergence). Under identity activation, the total backdoor divergence of the Weight-Averaged ( (WAG) model is upper bounded by the average divergence of the switched models:
D || + [[D S
(IIDi
+ || + ||D +IDDiRs))
This theorem confirms the rationale that module switching average yields stronger suppression of beokdoor-specifee patterns than weight averaging.
Proposition 1 (The Existence of More Divergent Switched Model). Given Theorem 1, there exists at least one switched model with greater backdoor divergence than Aeiehtt-veraged (WAG) model:
Dwag. +|+ [[DWarJ]] S max maxx(IDDsM]] ||DUS [], || || 11
This proposition shows that the least backdoor-aligned switched model exceeds the WAG model in backdoor divergence, underscoring the importance of selecting the least aligned candidate and moti- vating the selection step in Section 4.4. Appendix details proofs of Theorem and Proposition
Utility Loss. Having established the divergence properties of backdoor components, we next exam- ine whether module switching compromises utility. For model M we measure its utility loss as the distance between its outputs to benign semantics, i.e., (x) == MM*(x) Sx = B' æ. The switched models satisfy the identity L L L' L (see Appendix E), implying that the total loss of switched pair is equivalent to the sum of its constituents. To assess individual models, we empirically measure each switched model's loss relative to its originals and find that the relative utility loss remains low (see Appendix F), demonstrating promising utility preservation.
3.3 EMPIRICAL. ANALYSIS
We simulate 1000 linear and non-linear two-layer networks, each pretrained on a shared semantic component S (0, 1) and fine-tuned with backdoor component 8 (0 0.1²). For each fine-tuned pair M and M we construct the corresponding WAG model wwag and switched models M and M We evaluate output alignment with (1) the semantic direction Sx. measured by d: = |nomm (x; 9)) norm Sx) []: and (2) the backdoor direction B*x, æ, measured by dp = [f Sx)) norm B" (B"x)] =>)] where norm(v) w/|
4

---

Published as a conference paper at ICLR 2026
Pretrained
Pinetuned
Switched (M)
Switched (M
WAG
Centre point
0.175
Centine:
0.30
Centree:
Pewrooeed:1 [1 16, D
1111
Fines
Fintuned 80
d. (Samantic ditancc)
Bwithed (M)
(9 92. 10:
de (Semante: dleance)
Swndhed (Mg 16)
Swiched
WKG
0.20
0.08)
0.15
0.075
0.060
0.10
0.025
RRLUU
0.008
0.00
0.2
0.4
0.6
1.0 da (Backdoor distance)
1.2
1.4
0.2
0.4
0.6
1.0
1.2
1.4
1/8
de (Backdoor cistaacc)
Figure 2: Euclidean distances between normalized output vectors of pretrained, fine-tuned, WAG, and switched two-layer networks, relative to the semantic direction Sæ and the backdoor directions B æ, under linear (left) and ReLU (right) activations.
Figure 2 presents 2D scatter plots comparing output distances across all model types under both linear and ReLU (Nair & Hinton, 2010; Agarap, 2018) activations. More results with various tivations are provided in Appendix G. We observe that while fine-tuned models stay close to their respective backdoor patterns B", the WAG model shifts farther away, and the switched models di- verge even more, indicating stronger backdoor suppression. All models remain near the semantic term S, confirming preserved functionality.
4 MODULE SWITCHING DEFENSE
In this section, we extend the findings on module switching to more complicated deep neural net- works and develop a comprehensive defense pipeline. We begin by introducing the problem setting in Section 4.1, followed by establishing set of heuristic rules to guide the search for effective mod- ule switching strategies in Section 4.2. Next, we adapt an evolutionary algorithm for searching the optimal strategy in Section 4.3, guiding switched models construction and selection in Section 4.4.
4.1 PRELIMINARIES
Threat Model. We study data poisoning attacks where an attacker modifies a subset of clean dataset De- {(Irer } into poisoned samples D,- = { = 9o (xr),, Up)} using a trigger function Ie and target label Up- The poisoned data is used to train a backdoored model shared with others for training, resulting in trojaned models being widely available via model-sharing platforms.
Defender Capability. The defender downloads potentially trojaned models and aims to purify them before deployment. They have white-box access and a small clean validation set (20-50 samples per class), but no knowledge of the triggers poisoned data. They can access multiple (as few as two) domain-relevant models of uncertain integrity and may combine them using the validation set.
Neural Network Architecture. We adopt Transformer models (Vaswani al., 2017), chosen for their strong performance and popularity in both text and vision. Each model has L. layers, com- posed of self-attention block and feed-forward network (FFN), both followed by residual con- nections (He et al., 2016). We abbreviate the six core modules (the attention block's 's query (W₂). key (W+). value (W.). ). output (W.), and the FFN's input (W;) and output (W₂)) as (Q, K, V. I)P
To assess cross-modality applicability, we also examine vision architectures, including Vision Trans- formers (VIT) (Wu et al., 2020) and convolutional networks (CNNs) (Lecun et al., 1998; Krizhevsky et al., 2012). For ViT models, we apply the same module abstraction used for text- based Transform- ers. For CNNs such as the ResNet family (He et al., 2016), each convolution-batch normalization weight pair is treated as module (e.g., the first conv-bn pair in each BasicBlock denoted as C1).
4.2 SCORING RULES FOR MODULE SWITCHING
In Section 3, we studied weight switching in two-layer networks, where replacing weights disrupts spurious correlations, eliminating undesired patterns while preserving utility. Extending to deep models, we hypothesize that breaking backdoor propagation paths can similarly deactivate them.
Given the structural complexity of deep networks, we define heuristic rules to guide the search for module combinations that disrupt backdoor paths in both feedforward and residual streams (Elhage
5

---

Published as a conference paper at ICLR 2026
Netetion
Nodkd
TYPE Intre-layer rejecencios:
Comsea lom-kkee anjerndes
TYPE Reaidhskowh aduoenniee
Karry
(00, (') (0.8" (0.0").PPP1 (0) i") (0.9 ("))")).)
Puy
300 KJA OKGYA (0// (1,7))
(PP-9%) (1.0)
Wutw
Sel Altondiin
FPN
Self Adtention
FFN
Soft Antention
FFN
Redidual stream
Wrren
Wiroen
Irpu
Embed
Curput
Transformer layer
(c+ 1) Transformer layer
(1 + 2)th Transformer layer
Figure 3: The fused model combines modules from different models (shown as red and blue nodes), considering three types of module adjacency in Transformers, as illustrated at the top.
et al., 2021). We identify three types of adjacency that may support poison transmission (illustrated in Figure 3): (1) intra-layer (within the same layer), (2) consecutive-layer (adjacent layers), and (3) residual (via skip connections). Additionally, we introduce a (4) balance penalty to avoid overusing any single model and (5) diversity reward to encourage varied combinations across layers.
These rules serve as criteria for computing an overall score of given module-switching strategy, evaluating how well it adheres the principles. Detailed rules and types are in Appendix H.1.
4.3 EVOLUTIONARY MODULE SWITCHING SEARCH
We frame the search for effective module-switching strategies as a discrete Neural Architecture Search (NAS) problem (White et al., 2023). Let S denote the space of switching strategies, where each 8 E S assigns a source model index to each module: s {l...., L} x M + (l...., ......,.J.. N}, M {Q, K, V, 0, P), where is the number of layers and N the number source models.
.......),
Fitness Evaluation. Each strategy is scored by
F(s) = Amuua Acons (s) (bal + As Rs)
(4)
where Amtr. Acaus and Ares penalize adjacency violations (Section 4.2), Bbat penalizes module imbalance, and Raiv rewards diversity. By de- fault, set all À to 1.0. Higher F(s) indicates stronger disruption of potential backdoor paths. Details of each term are in Appendix H.2.
Algorithm Evolutionary Swiithhe- Search
I: Input: population P, generations G. children per iter C, number of models N, layers L, module set M.
2: population <- Ø
gen_rount 0
4: while [population P do
5: indic.strategy + RANDOMSTRATEGG((N,
Search Algorithm. As the scores by F(s) non-differentiable over a large discrete space, we adopt lutionary search (Miller et al., 1989), well-suited optimizing implicit ob- jectives (Zhou et al., 2021). We adopt the aging regularized evolution al- gorithm (Real et al., 2019), modi-
6: indiv. audie.finnes fitness + CALCSSORE
7: population.appendiiae
8: end while
9: while gen_cownd < G do
10: for 1 C do
11: parent + TOURMAMEENSSLLLSTTTppplltoo
12:
child.strate strategy + MUTATION(parent)
13:
child. finies + CALCSScOR((hilddsstrteeeg))
fying it two key ways: (1) fit- ness is computed directly using the heuristic scoring function F, with- out model training validation; and (2) low-scoring strategies are dis- carded, replacing aging regulariza-
14:
15:
end for
16:
(poppaaalttiion) by descending filness score
17:
population + population 00 P] truncate to P
18:
gen_count < gen_count
19:
end while
tion (So et al., 2019). As outlined in Algorithm 1, it evolves popula-
20:
Output: BESTSTRATEGY +- poppuatioon strrate
tion through tournament selection (line 11), mutation (line 12), and fitness-based dropping (line 13). Appendix K presents example searched strategies.
4.4 SWITCHED MODELS CONSTRUCTION AND SELECTION
The searched strategy T can be used to switch modules among a group of victim models M My. fuse candidate pool, which average exceeds the WAG model in backdoor

---

Published as a conference paper at ICLR 2026
divergence (as Theorem 1) and guarantees the existence of at least one candidate with higher diver- method to select the
gence (as Proposition 1). This motivates us to develop a cast-backdoor-aligmae candidate from the pool.
Suspect-class Detection. We first the final-layer embedding of [CLS] token to detect the suspect class, based on the insight that back- doored models prioritize trigger fea- tures (Fu et al., 2023; Yi et al., 2024; Wang et al., 2024a). For each m E MU(WAGE (MM) and class c, we op- timize random input to induce pre- diction of c, yielding dummy final-
Algorithm Switched Model Selection
I: Input: Victim models M [M MN clean set Dei switching strategy T.
weight averaging over M 3: models +- MM U {wag)
2: wag < (AG(M)
4: score + ZEROV (EROVVECTORnummmcaasset
5: for m E models do
6: for candidate classes do
7:
Tduumy f- OPTIMIZENPPUT(NN (m. Frandoen: c)
layer [CLS] feature zuum Its average
8:
7dummy
FORWARD (m, Jdummy )
9:
Zelean FORWARD m, De, non- c)
cosine distance to clean features over
10:
score(e += MEANCOSINEDISTT (SS (Zdummys Zelean)
few non-c samples is accumulated
across models: S(c)= = Σm avg [1
11:
DUMMYFEATURE (m)
2dummy
12: end for
cos( (astee "m, me) The class with the highest score, et = arg max, S(c), deemed suspicious, and the corre- sponding WAG dummy feature 2" = -dom
13: end for
14: e* <- arg max, score(c]
suspect target class
15: x² <- DUMMYFFATURR
16: candidates t- MODULESWITCH(CI.
" is used as a fixed refer- ence. For CNNs, the same proce- dure is applied using the global av- erage pooled feature of the final con- volutional layer in place of the [CLS] embedding.
17: for E candidates do
18: 2 FORWARD(m, De. non-c")
19: m.dist <- MEANCOSINEDIST(2;
20: end for
21: Output: arg max, m.dist
Candidate Selection. Applying T to M gives candidates m E C(T, (e.g.. M and M Each m is scored by d(m) = avg cos(=", cmm(=",fmm)) (x=)]] the mean cosine distance between its [CLS] features
few clean, non-c* samples and the WAG dummy zz The winner m² arg maxmeC(T.MM d(m) is the one least aligned with backdoor features and, by Proposition I, has better defense than WAG. The pipeline detailed in Algorithm 2, avoids exhaustive trojan detection process (Wang et al., 2019; 2020; 2024a; Su et al., 2024), yet reliably selects robust candidates.
5
EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Datasets. We primarily evaluate our method in the text domain using three NLP datasets: SST- 2 (Socher et al., 2013), MNLI (Williams et al., 2018), and AG News (Zhang al., 2015). Following previous work (Arora et al., 2024), we apply a poison rate of 20% in training, and additionally test settings with 10% and 1%. To further assess cross-domain applicability, we evaluate datasets: CIFAR-10 (Krizhevsky et al.. 2009) and ImageNet (Le & Yang, 2015), where a poison rate of 5% is used. The statisties of the used datasets are reported in Table 6 (Appendix I.I). The poisoned test sets, used solely for evaluation, are generated by adding triggers to validation samples outside the target class, while the defenders are restricted to access only the clean test set.
vision
Backdoor Attacks. We evaluate our defense against four text- based backdoor attacks that poison data by modifying and relabeling clean samples. Two are insertion insettion-based: BadNet (Kurita et al., 2020), which adds rare-word triggers ("cf", "mn", "bb", "iq", " "mb") "). and InsertSent (Dai et al., 2019), which inserts trigger phrases ("/ watched this movie", "no cross, no crown"), ") The other two stealthier: Learnable Word Substitution (LWS) (Qi et al., 2021c), which uses synonym substitution, and Hidden-Kiile (Hidden) (Qi et al., 2021b), which applies syntactic paraphrasing.
For the vision domain, we use attacks that inject digital patterns, such BadNet (Gu et al., 2017) and BATT (Xu et al., 2023b), as well as stealthier methods like the warping-basee WaNet (Nguyen & Tran, 2021) and the object-based PhysicalBA (Li et al., 202 To further challenge our defense, we also evaluate it against the Adaptive attack (Qi et al., 2023). All poisoned vision datasets and models are generated using the BackdoorBox toolkit (Lii et al., 2023b).
7

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison across backdoor attacks on SST-2 using RoBERTa-large. Best
results are in blue indicates results averaged over four variants; same for subsequent tables.
Defense
CACC
Attack Success Rate (ASR) I
Defense
CACC
Attack Success Rate (ASR) J
BadNet
Insert
1WS
Hidden
AVG.
BadNet
Insert
IWS
Hidden
AVG.
Benign
95.9
4.1
2.2
12.8
16.5
8.9
Z.Def ONION
95.6"
4.6
1.8
97.3
35.7
34,9
Victim
95.9
100.0
100.0
98.0
96.5
98.6
92.8
56.8
99.9
85.7
92.9
83.8
Combined: BadNer InsertSent
Combined: BadNer HiddenKiller
56.3
7.4
31.9
96.1
63.9
29.0
TIES
95.9
88.7
17.0
52.9
TIES
96.0
90.4
36.9
63.6
57.8
36.3
96.7
36.5
47.6
Ours
96.2
36,9
7.1
22.0
Ours
96.1
40.5
27.7
34,1
Combined: BodNer LWS
Combined: Benign BodNet
WAG
96.2
74,0
50.3
62.2
WAG
96.1
39.3
39.3
TIES
95.9
88.1
66.1
77.1
TIES
95.7
69.2
69.2
DARE
96.2
60.4
62.5
61.4
DARE
96.4
43.2
43.2
Ours
96.0
41.7
39.0
40.4
Ours
96.1
12.2
12.2
Defense Baselines. We compare against seven defenses across text and vision: three model-merging approaches applicable to both domains-TIFB (Yadav et al., 2023), DARE (Yu et al., 2024), and WAG (Arora et al., 2024)-and two domain-specifte data purification methods per modality. In text, Z-Def. (He et al., 2023) and ONION (Qi et al., 202 la) perform outlier detection; in vision, Cut- Mix (Yun et al., 2019) disrupts triggers via patch mixing, and ShrinkPad (Li et al., 2021b) reduces vulnerability by shrinking and padding inputs. All baselines use open-source implementations with default settings (see Appendix 1.3 for details).
Metries. We assess utility and defense with Clean Accuracy (CACC) and Attack Suc- cess Rate (ASR) (Qi al., 2021a;e; Arora al., 2024). CACC is the accuracy on clean samples, with higher values indicating better utility. ASR is the accuracy on a poisoned test set, where all samples attacked and relabeled to the target class; higher ASR indicates greater vulnerability.
Evaluation
Implementation Details. We use RoBERTa-large (Liu et al., 2019b), BERT-large (Devlin et al., 2019), and DeBERTa-large (He et al., 2021) for text experiments; and ViT (Wu al., 2020) well as pretrained ResNet-18 and RexNet-50 (He et al., 2016; Torch maintainers and contributors, 2016) for vision experiments. NLP models are fine-tuned for epochs with Adam (Kingma & Ba, 2015) at 2 x 10-5, and vision models for 10 epochs with SGD (Bottou, 2010) at 1 x 10-2.
We evaluate two-model merging in both domains and additionally consider multi-model merging for text. All experiments are run with three random seeds on single Nvidia A100 GPU, and results are averaged. The evolutionary search runs for million generations on single Intel Core i9- 14900K CPU, taking 2.6 hours for two models and 4.3 hours for four models. Since the strategy is strueture-driven and task-agnostic, only one search is required per architecture. For model selection, discussed in Section 4.4, we use 50 samples per class to choose candidate models and ablate this to 20 in Section 5.3. Selection takes less than a minute on both SST-2 and CIFAR-10.
5.2 MAIN RESULTS
Mitigation of Textual Backdoor Attacks. We evaluate our defense with RoBERTa-large on SST-2, MNLI, and AG News. Partial SST-2 results appear in Table 1, with full results in Appendix J. We evaluate merging backdoored models to examine robustness against attacks, and merging back- doored with benign models to examine resistance backdoor transfer. A unified strategy from our evolutionary algorithm (see Figure 7) is applied consistently across all cases.
Across all datasets and model pairs, our method shows strong defense while preserving clean accu- racy. Merging BadNet and InsertSent yields an ASR of 22.0%, compared to 31.9% for WAG. With BadNet and LWS (a stealthier attack), it reaches 40.4%, over 21.0% lower than baselines (typically above 60%). These results demonstrate that even with compromised models, approach disrupts spurious correlations and mitigates backdoors.
When merging benign model with compromised ones, our method consistently yields low ASRs across four combinations. In the BadNet- control-o case, achieves 12.2%, 27.1% better than WAG. This indicates that our method blocks unintended backdoor effects, unlike approaches that preserve utility but risk new vulnerabilities. While Z-Def performs well against insertion based attacks (with training data access), it is less effective against attacks with subtle trigger patterns.
8

---

Published as a conference paper at ICLR 2026
Table 2: Performance comparison across backdoor attacks the CIFAR-10 dataset using VIT.
Defense
CACC
BadNet
WaNet
BATT
PBA
AVG.
Defense
CACC
BadNet
WaNet
BATT
PBA
AVG.
Benign Victim
98.8
10.1
10.2
7.7
10.1
9.5
CurMix ShrinkPad
97.7*
87.1
70.6
99.9
64,9
80.6
98.5*
96.3
84.7
99.9
89.4
92.6
97.3"
14.4
51.3
99.9
88.3
63.5
Combined: BadNer + WiNeet
Combined: BadNet + BATT
WAG
98.7
13.7
10.6
12.2
WAG
98.9
10.1
26.5
TIES
98.6
11.9
10.7
11.3
TIES
98.9
10.1
47.9
29.0
DARE
98.8
83.3
10.2
46.7
DARE
99.0
Ours
98.7
12.3
10.5
11.4
Ours
98.7
10.2
32.6
21.4
Combined: BodNet PhysicuIRA
Combined: Benign PhysicaIRA
WAG
99.0
39.6
39.5
39.6
WAG
99.0
10.1
10.1
TIES
99.0
38.9
38.9
38.9
TIES
98.8
10.2
10.2
DARE
99.0
72.2
72.2
72.2
DARE
99.9
10.1
10.1
Ours
98.7
18.5
18.4
18.5
Ours
98.9
10.1
10.1
Mitigation of Vision Backdoor Attacks. We assess our method on the CIFAR-10 and Tinylm- ageNet datasets using 12-layer VTT (Wu et al., 2020) model. Partial results for CIFAR-10 are shown in Table 2, with full results presented in Appendix J.2. The evolutionary search yields the module-switching strategy in Figure 14, applied across all vision experiments.
Our method consistently defends against all attack combinations while preserving utility. For exam- ple, the BadNet PhysicaIBA case, it lowers ASR to 18.5%, outperforming all baselines by at least 20.4%. These results demonstrate the robustness of our strategy in disrupting spurious corre- lations and its effectiveness across domains with different input characteristics.
Three-Model Fusion Defense. When three backdoored models are available, even baseline WAG already shows strong results. However, our module-switching approach achieves consistently stronger defense. Using the strategy in Figure 15 (see Appendix J.3), MSD reduces the average ASR to below 20% across different combinations, outperforming WAG as reported in Table 12.
Merging Models with Collusive Backdoors. Although WAG achieves relatively low ASRs when combining multiple models, in a realistic yet underexplored scenario some models may share iden- tical backdoors. In such settings, WAG degenerates fewer-model behavior, reducing its defensive effectiveness. In contrast, our module-switching strategy is more resilient, as it strategically disrupts these recurring shortcuts. Using the strategy in Figure 16 (see Appendix J.3), MSD outperforms WAG under collusion, as shown in Table 13, demonstrating robustness against collusive models.
Comparison of Different Strategies. We compare two evolutionary search strategies-with and without early stopping-shown in Figures 7 and 8, and report their fitness scores in Table 14 of Ap- pendix J.4. The early stopping terminates the search when no improvement in fitness score is ob- served over 100,000 iterations. We observe positive correlation between the fitness score and de- fense performance: the adopted strategy without early stopping achieves a higher score and reduces the ASR by 27. 27.2%. Based on score breakdowns and visualizations, we attribute the improvement to fewer residual rule violations, which more effectively disrupt subtle spurious correlations.
Diversity of Discovered Strategies. We further examine the structural diversity of strategies pro- duced by the evolutionary search. Using three strategies obtained from different random seeds (Figures 7 and 9), we compute their module-level overlap. As detailed in Appendix J.5, only 10 out of 144 module positions coincide across all strategies (6.94%), with no region or module type exhibiting higher consistency than others. This demonstrates that MSD does not rely narrow set of critical layers but instead induces broad structural disruption, which helps mitigate backdoor effects and makes the searched strategies transferable and reusable across different scenarios.
Candidate Selection Results. Our method generates multiple asymmetric module allocation candi- dates, with selection guided by the process in Section 4.4. While the selected candidate consistently performs well, we also analyze the unselected ones (see Table 15 in Appendix J.6). In most cases, our method correctly identifies the top-perforning candidate, outperforming other options by a sig- nificant margin. Even when an unselected candidate achieves lower ASR in specific cases, our chosen candidate remains competitive with both the best alternative and the WAG baseline.
5.3 ABLATION STUDIES
We ablate each of the first three rules from Section 4.2 to evaluate their individual contributions. As shown in Table 16 (Appendix J.7), removing any rule typically
Importance of Heuristic Rules.

---

Published as a conference paper at ICLR 2026
degrades performance, highlighting the complementary effect of the full rule set. Visualizations in Figures 11 to 13 show that each ablation yields distinct strategy patterns.
Generalization across Architectures. We apply method RoBERTa-large. BERT-large, and DeBERTa-v3-large under three settings. As shown in Table 17 (Appendix J.8), our approach consis- tently outperforms WAG across all tests. Importantly, reuse the same searched strategy from Fig- ure 7, demonstrating strong cross- generalization and supporting practical scalability.
To further evaluate generality beyond Transformer families, also extend MSD CNN architec- tures, including ResNet-18 18 and ResNet-50 on CIFAR-10. 10. The searched strategies for these models shown in Appendix K (Figures 17 and 18), and the full quantitative results are presented in Ap- pendix J.8 (Table 18). Across diverse combinations, MSD achieves comparable or superior ASR reduction relative to WAG while maintaining similar clean accuracy. These results demonstrate that MSD naturally transfers to CNN-based models, reinforcing its cross-domain robustness.
Minimum Clean Data Requirement. We examine the impact of reducing clean supervision from 50 to 20 samples per class on SST-2 across three architectures. Results in Table 17 (Appendix J.9) show our method still selects low-ASR candidates, suggesting effectiveness with limited clean data.
We test robustness under 20%, 10%, and 1% poi- soning rates on SST-2 using RoBERTa-large. As shown in Table 19 (Appendix J. J.),), our method consistently achieves lower ASR than WAG across different attacks and poisoning levels.
Performance under Varying Poisoning Rates.
Robustness to Adaptive Attacks. We consider two types of threat scenarios: attacks that are adap- tive to MSD and challenging backdoor patterns that introduce stronger shortcut behaviors. First, we consider an attacker who knows the deployed module selection strategy and retrains only those mod- ules poisoned data. Our approach counters this by generating diverse strategies using different random seeds. Even if one strategy (Figure 7) is compromised, alternatives (Figure 9) remain effee- tive, as demonstrated on SST-2 with RoBERTa-large (Table 20). Second, we evaluate challenging backdoor pattern, Adaptive-Patel (Qi et al., 2023), which is not MSD-speeifiic but induces more complex shortcut behavior. Using transferability-base strategy (Figure 14), our method consis- tently demonstrates strong defensive performance (Table 21). A detailed analysis of both scenarios provided in Appendix J.11.
Robustness to Label-Ineee Inconsistent and Identical Backdoors. A practical consideration for model merging is that the obtained models may be trained by different attackers targeting different labels, or they may encode identical backdoor triggers. We therefore examine two challenging settings: (1) models with inconsistent target labels, and (2) models trained with the same backdoor trigger. In the first case, where each model has different target label, our method maintains strong defensive per- formance. In the second case, where models share the same trigger, our method again substantially reduces ASR compared to WAG, as shown in Table 22 (Appendix J.12).
Efficiency Analysis. We compare the computational efficiency of MSD with representative base- lines in the two-model setting. The comparison is summarized in Table 3.
MSD requires a one-time architecture-dependent search of 2.6 hours that can be performed offline, after which the merging step takes only 16 seconds. In contrast, deployment-time search methods such DARE need to rerun a greedy search for every new model pair, taking approximately 2.5 hours per deployment. Since the MSD strategy be reused for all models that share the same architecture, the amortized deployment cost becomes negligible, providing a practical efficiency advantage while maintaining strong defensive performance.
Table Efficiency comparison.
Phase
DARE
TIES
WAG
MSD
Search Merge
hrs
2.6 hrs
min
10
16s
CONCLUSION
In this paper, we propose Module-Switchiee Defense (MSD), post-training backdoor defense that disrupts shorteuts of spurious correlations by strategically switching weight modules between (com- promised) models. MSD does not rely on trusted reference models training data and remains effective with a couple of models. Using heuristic rules and evolutionary search, we establish a transferable module fusion strategy that mitigates various backdoor attacks while preserving their task utility. Empirical results on text and vision tasks confirm its outstanding defense performance, and strong generalization capability, highlighting its practicality in real-world applications.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
We would like express our appreciation to the anonymous reviewers for their valuable feedback. This research was undertaken with the assistance of resources from the National Computational In- frastructure (NCI Australia), an NCRIS enabled capability supported by the Australian Government. We also express our gratitude to SoC Incentive Fund, FSE strategic startup grant and HDR Research Support Funding for supporting both travel and research.
ETHICS STATEMENT
This paper presents an efficient post-training defense against backdoor attacks on machine learning models. By strategically combining model weight modules from either clean or compromised mod- els, our approach disrupts backdoor propagation while preserving model utility. We demonstrated the usage of MSD strengthen the security of machine learning models in both natural language processing and computer vision. All models and datasets used in this study are sourced from estab- lished open-sourg platforms. The discovered MSD templates will be released to facilitate further research defense study. While we do not anticipate any direct negative societal consequences, we hope this work encourages further research into more robust defense mechanisms.
REPRODUCIBILIIY STATEMENT
We describe our method in detail in Section 4, with two key algorithms presented in Algorithm and Algorithm 2. Experimental settings are documented in Section 5.1, and the searched outputs of the algorithms are included in Appendix K. To support reproducibility, we release our implementa- tion code at: https: / /gitttb.........ggg wom/weiffe weijjun 1/module- swit thing fense..
REFERENCES
Abien Fred Agarap. Deep learning using rectified linear units (relu).
arXiv preprint
1:Xiv:1803.08375. 2018.
Ignacio Aristimuño. Model merging: Combining different fine-tuned LLMs, 2026. URL https: / www.marrik.. wwtaaaag//ooaaaaeeeeeee -aeeellmmeiinggg combininiinno different fine- tun ed-lims.
Ansh Arora, Xuanli He, Maximilian Mozes, Srinibas Swain, Mark Dras, and Qiongkai Xu. Here's free lunch: Sanitizing backdoored models with model merge. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACL 2024, pp- 15059-15077 Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 10.18653/v1/2024 findings--ade acciingsaaal.99.... URL ht tpts:p//// /ataaaaaaanncooco org/ 2024 24 findin gs-acl. 894.
Léon Bottou. Large-scale machine learning with stochastic gradient descent. In Proceedings of 20MPSTAT'2010: 19th International Conference on Computational StatisticsParis France, Au- gust 22-27, 2010 Keynote, Invited and Contributed Papers, pp. 177-186. Springer, 2010.
Chen Chen, Yuchen Sun, Xueluan Gong, Jiaxin Gao, and Kwok- Yan Lam. Neutralizing backdoors through information conflicts for large language models. arXiv preprint arXiv:2411.18288 2024.
Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. CoRR, abs/1712.05526. 2017. URL http://arxiv. org/ axg//bs// 1712. 05526.
Siyuan Cheng, Guangyu Shen, Kaiyuan Zhang, Guanhong Tao, Shengwei An, Hanxi Guo, Shiqing Ma, and Xiangyu Zhang. Unit: Backdoor mitigation via automated neural distribution tightening. In European Conference on Computer Vision, pp. 262-281. Springer, 2024.
Jiazhu Dai, Chuanshuai Chen, and Yufeng Li. A backdoor attack against based text classifica- tion systems. IEEE Access, 7:138872-138878 2019. URL https: / /api. semant icscho l 2aar.org//orppust111111111111111
11

---

Published as a conference paper at ICLR 2026
Rong Dai, Yonggang Zhang, Ang Li, Tongliang Liu, Xun Yang, and Bo Han. Enhancing shot federated learning through data and ensemble co-boosting. In The Twelfth International Conference on Learning Representations, 2024. URL ?id=tm8s3 66960x
Jacob Devlin, Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume 1 (Long and Short Papers), pp. 4171-4186, Minneapolis, Minnesota, June 2019. Association for Compu- tational Linguisties. doi: 10. 10.18653/11/N19-4422 URL https: / /aa/caaaaae aclanthooogge org/N 191....
Shiv Ram Dubey, Satish Kumar Singh, and Bidyut Baran Chaudhuri. Activation functions in deep learning: A comprehensive survey and benchmark. Neurocomputting, 503:92-108, 2022. ISSN 0925-2312. doi: http:///bb.org//0.0116....
URL https: attps://www. //w::///wwww.. scie
ncedi com wtt.//cieee////rtiiiaaa...... htt.d//c......c..aaaaaaaaaaa///a
Jacob Dumford and Walter Scheirer. Backdooring convolutional neural networks via targeted weight perturbations. In 2020 IEEE International Joint Conference on Biometrics (IJCB), pp. 1-9, 2020.
doi: N0.1109/1CB488488.000 90049730777
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Gan- guli, Zac Haffield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A mathematical framework for transformer circuits. Transformer Circuits Thread, 2021. ceaeer-ciccuttts ppbp 1ee222/frraeee framework/ index. html.
URL https::////
William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23 23(120):1-39 2022. URL http: / jmlr org/ papers/ /v23 21-0998 html.
Chong Fu, Xuhong Zhang, Shouling Ji, Ting Wang, Peng Lin, Yanghe Feng, and Jianwei Yin. FreeBagle: Detecting complex neural trojans in Data-Free cases. In 32nd USENIX Security Sym- posium (USENIX Security 23). pp. 6399-6416, Anaheim, CA, August 2023. USENIX Associa- tion. ISBN 97781-9999133-7-55 URL https: http///www/eeee usenix.....cmm wwwpc///wwwe/c//////oocccococcee usen foan/fu-caii fu-chong.
Yudong Gao, Honglong Chen, Peng Sun, Zhe Li, Junjian Li, and Huajie Shao. Energy-based back- door defense without task-specific samples and model retraining. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.). Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 14611-14633 PMLR, 21-27 Jul 2024. URL htttss:/pproceeiing www.mmlrppp.... www.sp/pesss.sseee /233/2222//233332 gao24b.o2...... html.
Matt Gardner, William Merrill, Jesse Dodge, Matthew Peters, Alexis Ross, Sameer Singh, and Noah A. Smith. Competency problems: On finding and removing artifacts in language data. In Maric-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Pro- ceedings of the 2021 Conference on Empirical Methods Natural Language Processing. pp. 1801-1813, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 10.18653/v1/209 emnlp-main. 135. URL https: //aclanthology http:thhoooog...gg2222111111
wttp//ogg...... emm.pppp..pmppppmmmmmmmmmmmppplp main. 135/.
Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. CoRR, hts/1708.06733. 2017. URL http://arxixg
orq/abs/11 1708. 06733.
Neel Guha, Ameet Talwalkar, and Virginia Smith. One-shot federated learning. arXiv preprint 117. 2019.
12

---

Published as a conference paper at ICLR 2026
Gareth Halfacree. Ai crawlers and fetchers are blowing up websites, with meta and openai the worst offenders. The Register, August 2025. URL https:://// http://wwwwwwtm www.::theeegite.e.eeeee com/ 2025/0 httpaaaaawweeeeaa__a_ee
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016. URL. wwwptttts////c 6/html He__eee_Reeiieeeeeeeeee Reseeeeesiuallleee otgar_/esssstta___eeeeennnnnnnn CV ppaper.hml.
thecvf. ccom/contett_cpp.
Pengcheng He, Jianfeng Gao, and Weizhu Chen. Debertav3: Improving deberta using electra-style pre-training with gradient-disentaglle embedding sharing. arXiv preprint 1.X952211..05543, 2021.
Xuanli He, Qiongkai Xu, Jun Wang, Benjamin Rubinstein, and Trevor Cohn. Mitigating backdoor poisoning attacks through the lens of spurious correlation. In Houda Bouamor, Juan Pino, and Ka- lika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 953-967, Singapore, December 2023. Association for Computational Linguistics. doi: 10.186633v1/2022 1/20333.m///2.mmmmpp..pm.. main.60. URL https: http:///aaaaaattooaecoooo wwg.llooo..rg222220 org/2023 em nlp-main... 60/.
Xuanli He, Qiongkai Xu, Jun Wang, Benjamin I. Rubinstein, and Trevor Cohn. SEEP: Training dynamics grounds latent representation search for mitigating backdoor poisoning at- tacks. Transactions of the Association for Computational Linguistics, 12:996-1010, 2024. doi: 10.1 00684. URL htt.ts::/aclaatthhet.. http://aaaaaatttt.a..c..... org/ 2024 tac1-1.55/
Xuanli He, Jun Wang. Qiongkai Xu, Pasquale Minervini, Pontus Stenetorp, Benjamin I. P. Ru- binstein, and Trevor Cohn. TUBA: Cross-lingual transferability of backdoor attacks in LLMs with instruction tuning. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Moham- mad Taher Pilehvar (eds.). Findings of the Association for Computational Linguistics: ACL 2025, pp. 16504-16544, Vienna, Austria, July 2025. Association for Computational Linguis- tics. ISBN 979-8-89176-256-5. doi: 10. 10.18653/v1/0 1/2025. 1/n6:3////22...nnnnnnsl/l/l// //aclanthology.okk.......
URL https:
www.dinnnnniiieeeeeeeeii acl. 848/.
Linshan Hou, Ruili Feng, Zhongyun Hua, Wei Luo, Leo Yu Zhang, and Yiming Li. IBD-PSC: Input- level backdoor detection via parameter-oriented scaling consistency. In Forty-first International Conference on Machine Learning, 2024. URL https: /ttp:///o/pe//eeeeeeeeeeee net forum?id= YC2bfs2few.
Daniel Huynh and Jade Hardouin. Poisongpt: How we hid a lobotomized Ilm on hugging face to spread fake news. Mithril Security, July 2023. URL
h.o/poissoggp--hoe wwwpopptthowwwe-ee otg.::/ooooiiiiaaooooooaooaooooo fread-fake-newsss
Tran Huynh, Dang Nguyen, Tung Pham, and Anh Tran. Combat: Alternated training for effective clean-label backdoor attacks. Proceedings of the AAAI Conference on Artificial Intelligence, 38 (3):2466-244,, Mar. 2024. doi: 10. wtt.0///aaav.8...288888 / /index.
URL https: //tp:///jjs/.//../////// aaai. org
Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wil- son. Averaging weights leads to wider optima and better generalization. arXiv preprint
arXiv: 1803:01803.05407, 2018.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.). 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http: //arxiv. wtg/ wwwp//////.aa....a/...///eaaeee/ at/aaaass/111 6980.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep con- volutional neural networks. In F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012. URL https: http://poooeeeeeeeeeeeeeeeeceaee neurips cc/ cctpaper_eaacccceceeeeeee http://eeeeeeeeee112221. 39982209b99666676443333333333
Saapeeerr...pa.r... pdd.
13

---

Published as a conference paper at ICLR 2026
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. 2009.
Keita Kurita, Paul Michel, and Graham Neubig. Weight poisoning attacks pretrained models. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2793-2806, Online, July 2020. Association for Computational Linguistics. doi: 10. 1111116/1/22200 acl mac.l.2mm2.29999 URL https: hatp:://aaaaatthlle
org/2020
249.
Yann Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient- learning applied to document recog- nition. Proceedings of the IEEE, 86(1 ((111)2227--2222 1998. doi: 10.1 109/5.726791.
Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Anti-backdoor learn- ing: Training clean models poisoned data. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, vol- ume 34, pp. 14900-14912. Curran Associates, Inc., 2021a. URL htt httpttpppp./pppcpeeeeeeceggee// neurips. cc/paper___pp.e /tLes///pper/////111///////3 1.9a93c-Paper::
Yige Li, Xixiang Lyu, Xingjun Ma. Nodens Koren, Lingjuan Lyu, Bo Li, and Yu-Gang Jiang. Reconstructive neuron pruning for backdoor defense. Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceed- ings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 19837-19854. PMLR, 23-29 Jul 2023a. URL https: / //ppoceedings.... mlr. press/ / w222//123v2// html.
Yige Li, Jiabo He, Hanxun Huang, Jun Sun, Xingjun Ma, and Yu-Gang Jiang. Shortcuts everywhere and nowhere: Exploring multi-trigger backdoor attacks. IEEE Transactions on Dependable and Secure Computing, pp. 1-12, 2025. doi: 10. 1091100C.
Yiming Li, Tongqing Zhai, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. Backdoor attack in the physical world. arXiv preprint arXiv:2104 023661, 2021b.
Yiming Li, Mengxi Ya, Yang Bai, Yong Jiang, and Shu-Tao Xia. Backdoorbox: A python toolbox for backdoor learning. In ICLR 2023 Workshop on Backdoor Attacks and Defenses in Machine Learning. 2023b. URL https: wwt.//oeeee.eeeeeeeeeeeeee htw.:/ewwweeeee........e www.rrommnnnnoonna....
Yuetai Li, Zhangchen Xu, Fengqing Jiang, Luyao NIu, Dinuka Sahabandu, Bhaskar Ramasubrama- nian, and Radha Poovendran. Cleangen: Mitigating backdoor attacks for generation tasks in large language models. arXiv preprint arXiv:2406.. 2i..22406.12227, 2024.
Kang Liu, Brendan Dolan-Gavitt and Siddharth Garg. Fine-pruning: Defending against backdoor- ing attacks on deep neural networks. CoRR, abs/1805. 12185, 2018. URL g/ abaa/ss000 12185.
Yingqi Liu, Wen- Wen-Chuan Lee, Guanhong Tao, Shiqing Ma, Yousra Aafer, and Xiangyu Zhang, Abs: Scanning neural networks for back-doors by artificial brain stimulation. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pp. 1265-1282, 2019a.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692. 2019b. URL htt...tttp//arss..orrg//////
1907 11692.
Michael Matena and Colin A Raffel. Merging models with fisher-weighted averaging. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 17703-17716. Curran Associates, Inc., 2022. URL htwp::ttt//oom httt::////coceeeeeeeeeoeooe neuri ps. 022233TTEE3360005322201111 Canereeeeeeapee--CCeeeeeeeeeee pdf.
ptt.e///pppeee/222//2///// Conference,
Geoffrey F Miller, Peter M Todd, and Shailesh U Hegde. Designing neural networks using genetic algorithms. In ICGA, volume 89, pp. 379-384, 1989.
14

---

Published as a conference paper at ICLR 2026
Rui Min, Zeyu Qin, Li Shen, and Minhao Cheng. Towards stable backdoor purification through feature shift tuning. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine in Neural Information Processing Systems, volume 36, pp. 75286-75306. Curran Associates, Inc., 2023, URL https: http:////oooeooeoo////coo meerriidddinnnnnnnn.ppee....... pttrr//ooooeeeeeeeeeeeoe.o..oo httpr//geeeerrpppp..ccpepeeeeele (0.003/fileeeeee e.pdf.
Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference machine learning (ICML-10), pp. 807-814, 2010.
Anh Nguyen and Anh Tran. Wanet-imperceptible warping-based backdoor attack. arXiv preprint arXiv:2102.10300 2021.
Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao. Zhiyuan Liu, and Maosong Sun. ONION: A simple and effective defense against textual backdoor attacks. In Marie-Francine Moens, Xu- anjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Confer- ence on Empirical Methods Natural Language Processing, pp. 9558-9568 95666, Online and Punta Cana, Dominican Republic, November 2021a. Association for Computational Linguistics. doi: 10. 10.18653/v 1/202 1ttp/22.emmmlmmmaii..... wwn.p-main.nn.mmmm 752. URL https: /ataaaaaaaannooyooaaaaaacaoaolaa org/2021, emnl p imaminn.. 752.
Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, and Maosong Sun. Hidden killer: Invisible textual backdoor attacks with syntactic trigger. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the Ith International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 443-453, Online, August 2021b. Association for Computational Linguistics. doi: 10. 10.11853/vv/2000 accI-long..3, URL https: htttps://acllaeeee /aclant hology org/ 2021 acl.acll-lee long 37/.
Fanchao Qi, Yuan Yao, Sophia Xu, Zhiyuan Liu, and Maosong Sun. Turn the combination lock: Learnable textual backdoor attacks via word substitution. In Annual Meeting of the Association for Computational Linguistics, 2021c. https://aaaae www..pe.aaaae..............aa..e semanti semanticscholarr Co rpusID:23541710
Xiangyu Qi Tinghao Xie, Yiming Li, Saeed Mahloujifar, and Prateek Mittal. Revisiting the as- sumption of latent separability for backdoor defenses. In The Eleventh International Conference on Learning Representations, 2023. URL https: /ppnrevveeeeeeeeee/eeeeeeeeeeeee net forum d=__SH sgrVali.
Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V. Le. Regularized evolution for image classifier architecture search. Proceedings of the AAAI Conference on Artificial Intelligence, 33 (01):4780-4789, Jul. 2019. doi: 10. 10.1169//aii.v3330 033101.330147777 URL /ottps://oj// aaai .org/index php/ AAAI www.a//atiile////eeeeeeaeae /405/
David So, Quoe Le, and Chen Liang. The evolved transformer. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedliggs of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 5877-5886. PMLR, 09-15 Jun 2019. URL https: wttp:uRttttt///oeeeeeeeeeee http///poceeeng.ee//////// mlr wttppppwwss.sseepepppppp
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over sentiment treebank. In David Yarowsky, Timothy Baldwin, Anna Korhonen, Karen Livescu, and Steven Bethard (eds.), Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631-1642, Seattle, Washington, USA, October 2013. Association for Computa- tional Linguistics. URL hwtphhtp/////..... http:tttt///::ttta/ http::///caaaaahtaceeeoe hww.ccannntooooooooooooooooo wtt//////////11/
Yanghao Su, Jie Zhang, Ting Xu, Tianwei Zhang, Weiming Zhang, and Nenghai Yu. Model x- ray: Detecting backdoored models via decision boundary. In Proceedings of the 32nd ACM International Conference Multimedia, pp. 10296-10305, 2024.
2024.
15

---

Published as a conference paper at ICLR 2026
Indranil Sur, Karan Sikka, Matthew Walmer, Kaushik Koneripalli, Anirban Roy, Xiao Lin, Ajay Di- vakaran, and Susmit Jha. Tijo: Trigger inversion with joint optimization for defending multimodal backdoored models. In Proceedings of the IEEE/CVF International Conference Computer Vi- sion (ICCV), pp. 165-175, Octoher 2023.
Guanhong Tao, Guangyu Shen, Yingqi Liu, Shengwei An, Qiuling Xu, Shiqing Ma, Pan Li, and Xiangyu Zhang. Better trigger inversion optimization in backdoor scanning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1338811378, June 2022.
Yao Tong, Weijun Li, Xuanli He, Haolan Zhan, and Qiongkai Xu. Cut the deadwood out: Post- training model purification with selective module substitution. arXiv preprint arXiv:2412.20476, 2024.
Vision maintainers and contributors. Torchvision: Pytorch's 's computer vision library. https: //gi itittii thub. wwwp///pyttoo//iiiiiooooooooo 2016. GitHub repository.
Amos Tversky and Itamar Gati. Similarity, separability, and the triangle inequality. Psychological review, 19,, 1982.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E ukasz Kaiser, and Illia Polosukhin. Attention is all you need. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neu- ral Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https: / //proceedings. neurips wwtppppp_aflleeeeeeeeeeeeeeeeeee filrrffile///e htt.ll/epappee/11///// / dee$1fbd053clc4456 PaaeraaaaPPppp.p...
Bolun Wang. Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y.
Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE Symposium on Security and Privacy (SP), pp. 707-723, 2019. doi: 10.1 10.1109/SP.209.00000
Hang Wang, Zhen Xiang, David J Miller, and George Kesidis. Mm-bd: Post-training detection of backdoor attacks with arbitrary backdoor pattern types using maximum margin statistic. In 2024 IEEE Symposium on Security and Privacy (SP). pp. 1994 2012. IEEE, 2024a.
Hu Wang. Congbo Ma, Ibrahim Almakky, Ian Reid, Gustavo Cameiro, and Mohammad Yaqub. Rethinking weight-averaged model-merging, arXiv preprint 1xXiv:2411.09263, 2024b.
Ren Wang, Gaoyuan Zhang, Sijia Liu, Pin-Yu Chen, Jinjun Xiong, and Meng Wang. Practical detection of trojan neural networks: Data-limited and data-free cases. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIII 16. pp. 222-238. Springer, 2020.
Zhenting Wang, Kai Mei, Juan Zhai, and Shiqing Ma. UNICORN: A unified backdoor trigger inversion framework. In The Eleventh International Conference on Learning Representations, 2023. URL https: /tt.o/tttttttt//peeeeeeessssosse wtwp:r/wwwww..ttteoooouu..aaaaad
Colin White, Mahmoud Safari, Rhea Sukthanker, Binxin Ru, Thomas Elsken, Arber Zela, De- badeepta Dey, and Frank Hutter. Neural architecture search: Insights from 1000 papers. arXiv preprint arXiv:2301. 008027,0877..... 2023.
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sen- tence understanding through inference. In Proceedings of the 2018 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1112-1122, 122. Association for Computational Linguistics, 2018. URL http: / ac org/ anthelogy /N18 8-111.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface's 's transformers: State-of-the-art natural language processing, arXiv preprint 191.03191, 2019.
Bichen Wu, Chenfeng Xu, Xiaoliang Dai, Alvin Wan, Peizhao Zhang, Zhicheng Yan, Masayoshi Tomizuka, Joseph Gonzalez, Kurt Keutzer, and Peter Vajda. Visual transformers: Token-based image representation and processing for computer vision, 2020.
16

---

Published as a conference paper at ICLR 2026
Dongxian Wu and Yisen Wang. Adversarial neuron pruming purifies backdoored deep models. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 16913-16925. Curran Associates, Inc., 2021. URL https: httptttts:///roeeeeee http///oooeedeeieeeeeeee wewrinng..neeenuuneeeee neurips...... pwwpprapppppe fi pssppppr//202//// 1e/ wcw.eceeecfcccccccccc 2
http://perfleeeppppprrreeeeee Pa-Paper.
Tinghao Xie, Xiangyu Qi. Ping He, Yiming Li, Jiachen T. Wang, and Prateek Mittal. BaDExpert: Extracting backdoor functionality for accurate backdoor input detection. In The Twelfth Interna- fional Conference on Learning Representations, 2024. URL or Srum?id=se 5xxid=x0022x.
Qiuling Xu, Guanhong Tao, Jean Honorio, Yingqi Liu, Shengwei An, Guangyu Shen, Siyuan Cheng, and Xiangyu Zhang. Medic: Remove model backdoors via importance driven cloning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 40485-20494 June 2023a.
Tong Xu, Yiming Li, Yong Jiang, and Shu-Tao Xia. Batt: Backdoor attack with transformation- based triggers. ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2023b.
Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. Ties- f Resolving interference when merging models. In Oh, T. Naumann, Globerson, Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 7093-7115. Curran Associates, Inc., 2023. URL httpps://proceeednggg...... ineur.... oc cc/paper files/ papper/papr///eeeeeeeeee http:///////.............222131/ E-Paper-Conference pdf.
Jinluan Yang. Anke Tang, Didi Zhu, Zhengyu Chen, Li Shen, and Fei Wu. Mitigating the backdoor effect for multi-task model merging via safety-aware subspace. In The Thirteenth International Conference on Learning Representations, 2025. URL https: http://ppeeeeeenne///// wtwp//ewweee/ooooo. forum httpht://pppeeeeeeee/
Pid-dqMqAaw7733
Wenqian Ye, Guangtao Zheng, Xu Cao, Yunsheng Ma, and Aidong Zhang, Spurious correlations in machine learning: A survey, arXiv preprint arXiv:2402.12715, 2024.
Biao Yi, Sishuo Chen, Yiming Li, Tong Li, Baolei Zhang, and Zheli Liu. BadActs: universal backdoor defense in the activation space. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACL 2024, pp. 5339-5352, Bangkok. Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/ 2024. 12244findinss-acl.31 URL https: http:///aaaaattltaaaa/e/ org 024.. findings 17/.
Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Ab- sorbing abilities from homologous models as free lunch. In Forty-firs International Conference on Machine Learning. 2024. URL https: /pwprevveeeee.ee/////////e net Forumt dd=fqONaiU8 Ex.
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. Proceed- ings of the IEEE/CVF international conference on computer vision, pp. 6023-6032, 2019.
Yi Zeng, Si Chen, Won Park, Zhuoqing Mao, Ming Jin, and Ruoxi Jia. Adversarial unlearning of backdoors via implicit hypergradient. In International Conference on Learning Representations, 2022. URL https: https/////op/eeeeeeeeeee net forum? wtt.:eteeeeorrrmme.eeeeeeeeeeere MeM.e..eeeeeeeeeeeeeee..
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text clas- sification. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015. URL tttepa://proceeding neurips oc 30cf8b55cc773333ddeeee.
http//ppppe////// file/2
Paper.
17

---

Published as a conference paper at ICLR 2026
Zhiyuan Zhang, Lingjuan Lyu, Xingjun Ma, Chenguang Wang, and Xu Sun. Fine-mixing: Mit- igating backdoors fine-tuned language models. In Yoav Goldberg. Zomitsa Kozareva, and Yue Zhang (eds.), Findings of the Association for Computational Linguisties: EMNLP 2022, pp- 355-372, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguisties. doi: 10.18653/v1/2222 findinng--emllpp2e URL https: httts:://aclanthllllo/
http://ww/222.2.nnnnn-mememmm rg/2022. -indingg-emm.p..222
Xingyi Zhao, Depeng Xu, and Shuhan Yuan. Defense against backdoor attack on pre-trained language models via head pruning and attention normalization. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 61108-61120 PMLR, 21-27 Jul 2024. URL mlr
ptttps:/proceeeeee www.:inss.m......s.....ee /v2rss/v223//
v235 http//22352222222222 zhao24r html..
Xun Zhou, A. K. Qin, Maoguo Gong, and Kay Chen Tan. A survey on evolutionary construction of deep neural networks. IEEE Transactions on Evolutionary Computation, 25(5)8994-91,, 2021.
doi: 10. 101100//EVC2222013399995
Yanqi Zhou, Tao Lei, Hanxiao Liu, Nan Du, Yanping Huang, Vincent Zhao, Andrew M Dai, zhifeng Chen, Quoe Le, and James Laudon. Mixture-of-experts with expert choice routing- In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neu- ral Information Processing Systems, volume 35, pp. 7103-7114. Curran Associates, Inc., 2022. URL https: / http://rrrreeednneeeerrreeseeeee cc http///ppee.leeppppppeeeee/////e Eaceedehss2aa05eee000222eeeeeee0
18

---

Published as a conference paper at ICLR 2026
A LIMITATIONS
While our study demonstrates the effectiveness of Module-Switching Defense (MSD) across range of classification tasks NLP and CV, our current scope is limited to classification-bat settings. Backdoor attacks in generative models operate through notably different mechanisms, and extending MSD to such scenarios remains an important direction for future research.
B GENERATIVE LLM USAGE STATEMENT
We used ChatGPT and Gemini for surface-level edits, such grammar checks, phrasing refinement, and table caption formatting to improve readability.
C EMPIRICAL VALIDATION OF THE SECOND-ORDER INTERACTION MAGNITUDE
We empirically validate the condition adopted in Section 3, where the second-order interaction term AW2AW, omitted due its negligible magnitude relative the first-order terms. This validation proceeds from three perspectives.
First, Figure 4 compares the Frobenius norms of the semantic S W2 the first-order adaptation term B W, + AW,,, and the second-order residual = A AWAW W1 across five derived networks. The left subfigure confirms that ||c|| consistently two orders of magnitude smaller than ||S|| and well below 4% of B||. The right subfigure further reveals that the element- wise values concentrate tightly around zero, contrasting with the heavier tails and
Term Norms Across Networks
Element-wise Distributions (M,)
[c]
10²
Widd;;
6RUN
Frobenius scale)
15.0
12.5
Density
10.0
7.5
10a
5.0
2.90%
CR
2 68%
618
3.40%
3.20%
3.00%
2/B
2.5
M.
M
M
Mu
M,
-3
-2
Figure 4: Frobenius norm and element-wise distribution of the semantic, first-order, and second- order terms across five network configurations. While the first-order term dominates the residual behavior, the second-order interaction AW2AW, remains negligible in both scale and distri- bution.
Second, Table reports [|| ratios across five network variants under varying backdoor strengths, where perturbations are sampled from zero-mean Gaussian noise with increasing vari- ance. The inclusion of error bars (mean standard deviation) reflects variation across multiple runs. In typical scenarios where the backdoor signal is weak comparable to the main seman- tic component, the second-order interaction consistently remains below 4% of the first-order term. Even under exaggerated settings where the backdoor signal is scaled to 1.5x 2x the semantic strength, | remains within a stable range of 5%-7%, reaffirming the negligible and bounded nature of second-order interactions across regimes.
Additionally, we extend this analysis to deep transformer-bass based (Vaswani et al., 2017) models by computing ||e|] B|| for the attention weight product, where W, and W2 denote the key (K) and
19

---

Published as a conference paper at ICLR 2026
Table Relative magnitude second-order interactions, reported as |e|l/|| across networks and backdoor strengths. All models evaluated with 5 ~ (O, 1) and perturbations B ~N
||e Jell/1l BB for Different Shallow Models
Semantic Dist
Backdoor Dist
M
M
Mnng
M
M
(i0,0.12)
298
2.70+0.20%
3.42-0.23%
2.95+0.15%
3.14+0.21%
(.0,
1.9810.21%
1.90:10288%
1.78=0.18%
1.75:0.12%
1.76::0.18%
S 10, 1.0)
~ (.,,,.00
3.24+-0.22%
3.10+0.32%
2.35-0.17%
2.33+0.12%
2.30+0..7%%
4.77:1025%
4.48:10.33%
3.18=0.14%
3.09:0.15%
2.97::0.1%%
B ~ ((0,
6.3110.27%
6.28+0033%
4.14=0.29%
4.06:10.1%%
3.92:10.8%%
query (Q) projection matrices, respectively, and QK := W1 The weight changes AW, A 22 are computed relative to the original pretrained RoBERTa-large (Liu et al., 2019b) weights. All models are trained on SST-2 (Socher al., 2013), including both benign and backdoored variants such as BadNet (Kurita et al., 2020). InsertSent (Dai et al., 2019). learnable word substitution (LWS) (Qi al., 2021c), and Hidden- Killer (Hidden) (Qi al., 2021b).
As shown Table 5, across all pairwise combinations these models, the relative magnitude of second-order interactions consistently remains below 4%. Each reported value reflects the mean and standard deviation computed across all 24 layers of RoBERTa-large. This pattern holds across both original and recombined variants (M"sg, MM MM"), confirming the stability of second-order contributions in practical transformer settings.
Table 5: Relative magnitude of second-order interactions, reported as ||s||/|BB|| computed from the key (K) and query (Q) projection matrices RoBERTa-large models trained on SST-2,
Combination
||ll// B|] for Attention Weight Product (QK*) in RoBERTa-larg Models
(M' M²)
M
M
Mw
M
M
BudNet InserfSent
3.53%
3.24:10.1%
2.49:10.39%
2.61+00.50% 2.68+
BadNet LWS
3.30-10.5% 3.53%
2.46:#0.4%
2.71+00.66%
2.68::0:49%
BadNet + Hidden
3.53++..77%
3.30+1061%
2.49+0.43%
2.77+-0.45%
2.72++0045%
BadNet + Benign
3.53++0.77%
3.27+0058%
2.52+0.42%
2.78+0.47%
2.73++0.48%
Accordingly, omit the second-order term in our definitions and proof's throughout the paper without loss of generality.
20

---

Published as a conference paper at ICLR 2026
D PROOFS OF THEOREM AND PROPOSITION 1
Theorem (Module Switching Exceeds WAG in Backdoor Divergence). Under identity activation, the total backdoor divergence of the Weight-Averaged (WAG) model is upper bounded by the average divergence of the switched models:
[ || ||D
(1ID'/11 |D + ||Di |[D
Proposition 1 (The Existence of More Divergent Switched Model), Given Theorem I, there exists least one switched model with greater backdoor divergence than Weight-Average (WAG) model:
Dwag.
|D
S max [IDb>i]] + |D ||. |D + ||Dii
Proof. From Definition and 4, we have the following expressions for the backdoor divergences:
|Dwag. ||
1 2
|
W2(AWW AW)) + (AW) AW;) a
1 21l(
D
AW?) + ( AW wi
(5)
|
(AW] - -W; Wi
| W2(AW; W >a
|
Wo(Aw( - A
| (AW: AW: W/
Linear relationships. identities:
By regrouping terms in the above definitions, we obtain the following vector
1 (Dii +DiHA), 2
1 (Diss + Diid).
D wag.
Dwag.
(6)
2
Bounding the average switched model backdoor divergence. Substituting equation into the norms and applying the triangle inequality (Tversky & Gati, 1982), have:
w = 2 (Dis DV $ (I + ||D
D || (D + Diidd)]
1 s 2 (1 + |
(8)
Summing both inequalities gives:
wagsee + || war.d S (1/Dii + || + | || + ||Diis 1).
(9)
which proves Theorem 1.
Bounding the maximum switched model backdoor divergence. Let: C2:= | + ||D
C1 = | + ||
G max Cz).
(10)
Since C + C2 < 2G, it follows that:
1 (c1 + s max(Crp C2), D [Dwagd]] <
(11)
which proves Proposition 1.
E DERIVATION OF UTILITY LOSS IDENTITY
As discussed in Section 3, utility loss in two-layer network can be expressed as the difference from the benign semantic output. For a model M we define
(w)
M (x)
Sar
B² x..
(12)
21

---

Published as a conference paper at ICLR 2026
According the notation in Section 3 and the construction in Definition 3, the utility losses of constituent and switched models are
L' (xx) - (W2AW1 + AW;W1) (²(x) = (WaWy + AW3W:) (xx) (WaAw1 AW?Wi) x; 1²)()) (W>WW7 + AW3 J..
(13)
Regrouping these expressions yields the key identity,
())x)+ (W2AW1 + AW3 (W2AW7 + AW:Wi) )
= (W2Aw1 + +W/ + AW)Wi)
(14)
= -x) - L'))..
F EMPIRICAL EVALUATION OF UTILITY Loss
As derived in Appendix E. the identity, (*(x) + (x)x) L'))) + L'x).. characterizes the combined loss of a switched pair relative to its constituent models. We now evaluate loss at the level of individual switched models with respect to the benign semantic output.
For any model _M", define its utility loss ratio relative to benign semantic 5 as
(x))l for inputs with || > 0.
(x) = |Sx|
(15)
To measure how switched model compares to its originals, denote (xx'(x)) == : (x)
(x), ( = r - (x)),
(16)
()) = (x)))))))--))))) (x)) (x), ==) (x))
=
Let = eti.s gii e...) denote the collection of signed differences between the switched models and their origins. A value close to zero indicates that the switched models M and M preserve the benign utility at a level comparable to their original models M' and M2, Negative value further implies that switched model provides representations closer the benign semantics than those by corresponding original models.
The Distribution of g
= 0144
50
-- 008
Poy = C,00446
min -.0 02.0595
40
10
-0.08
-0.04
-0.02
0.00
0.02
Relative utility error e (MC) Aee vs. Mr. Ar)
Figure 5: Empirical distribution of ε, the relative utility loss differences between switched models (M's M) and their originals (MM'MM), Non-positive values indicate that a switched model is closer the benign utility, with more negative values corresponding closer alignment.
Figure shows the empirical distribution of & aggregated over 1,000 randomly sampled model pairs (M²,M)) The combined results yield mean of -0.014 + 0.008, 99th percentile of 0.004, and maximum 0.031, suggesting that the maximum deviation from the originals is below 3%, while on average the switched models M and M incur slightly smaller loss relative to the benign utility. These findings demonstrate that utility effectively preserved under module switching, compared to the original constituent models.
22

---

Published as a conference paper at ICLR 2026
G MODULE SWITCHING WITH ADDITIONAL ACTIVATION FUNCTIONS
We extend the experiments from Section to two additional activation functions: tanh and sig- moid (Dubey al., 2022), in addition the linear and ReLU results discussed the main text. For each activation, we simulate 1000 pairs of fine-tuned models M and M with shared pretrained semantic component S NN(,, 12) and individual backdoor shifts B (O 0.12). We then con- struct the bright-averaged model M and the module-switceee switched models M and M as defined in Definitions and 3.
Figure visualizes the semantic and backdoor alignment of each model type across the four activa- tion functions. Consistently across activations, we observe that:
Fine-tuned models remain closely aligned with their respective backdoor direction æ"
WAG models deviate more from the backdoor pattern;
Switched models exhibit the larger distance to backdoor patterns, indicating stronger miti- gation;
All model types maintain proximity to the semantic output Sa, confirming that semantic information is preserved.
These results generalize the findings in Figure 2 to broader range of nonlinear activations, re- inforcing the conclusion that module switching more effectively disrupts backdoor behavior while retaining semantic utility.
23

---

Published as a conference paper at ICLR 2026
Protrained
Finetuned (M)
Finetuned (M))
Switched (Mr
Switched (M)
WAG
Centre point
0.175
0.175
[Semendi: Daltenas)
0.150
di [Semwric: Daltena))
0.150
0.128
Linear
0.000
0.000
0.25
0.50
0.75
1.00
1,50
1.75
0.25
0.50
0.75
1.25
1,50
1.75
d (Semantc S Dataree)
[Samante (istaea)
0.25
0.25
0.15
0.16
0.05
0.08
ReLU
0.00
0.00
0.25
0.50
0.75
1.00
1.25
1.50
1.75
0.25
0.50
0.75
1.00
1.25
1.50
1.75
(Smanic Distamaa)
0.25
(Semarte Dlobanos)
0.25
0.20
0.20
0.10
0.05
0.06
TTanh
0.00
0.98.00
0.25
0.50
0.75
1.00
1.25
1.50
1.75
0.25
0.50
0.75
1.00
1.25
1.50
1.75
0.20
0.20
(Semonii S Distance)
(Semertee Distence)
0.15
0.15
0.10
0.10
0.05
0.06
[iigmoid
0.00
0.00
0.25
0.50
0.75
1.00
1.25
1.50
1.75
0.98
0.25
0.50
0.75
1.00
1.25
1.50
1.75
(Backidoor Distanos)
[Backdoor
Figure 6: Euclidean distances between normalized output vectors of pretrained, fine-tuned, WAG, and switched networks, relative to semantic output Sx and backdoor output xc under linear, ReLU, tanh, and sigmoid activations.
24

---

Published as a conference paper at ICLR 2026
H FITNESS SCORE CALCULATION FOR EVOLUTIONARY SEARCH
Building upon the heuristic rules established Section 4.2 for disrupting backdoor connections in compromised models, develop comprehensive fitness function. This function incorporates five key components that collectively evaluate the quality of module composition strategy.
H.1 HEURISTIC RULES
Our fitness function implements the following rules through penalties and rewards:
Heuristic-based Search Rules
1. Intra-layer adjacency penalty: Penalizes adjacent modules from the source model within a specific layer (e.g.. Q; and K,).
2. Consecutive-layer adjacency penalty: Discourages direct connections between mod- ules from same source model across consecutive layers and + (e.g., P. to Q++1).
3. Residual-path adjacency penalty: Applies a distance- penalty to modules from the same source model connected via residual connections between layers i and j (e.g., O; to Qj. where > i), with diminishing impact as j - increases.
4. Balance penalty: Promotes uniform distribution of modules {Q, K, V, I, across source models to prevent any single model from dominating the architecture.
Diversity reward: Encourages varied module combinations across layers to enhance architectural diversity.
H.2 MATHEMATICAL FORMULATION
As introduced Section 4.3, the total fitness score for a given module composition strategy s is:
F(s) ~ (m) A2 (s) (s) (sa(s)) + As)
(17)
where all A are weight factors (default to 1.0) that control the relative importance of each compo- nent in the overall fitness score.
Each component is calculated as follows:
Intra-layer Adjacency (Aintr(s))
|s]
Amira
INTRAVIOLATION((S()
(18)
1=1
Here, INTRAVIOLATION quantifies the number of adjacent module pairs from the same source model within layer s/l].
Consecutive-layer Adjacency (Acons (s))
1+|-1
Acons (s)
CONSECVIOLATION(S]]] ( + 1])
(19)
The function COnSECVinee VIOLATION counts module pairs from the same source model that are directly connected between consecutive layers.
3. Residual Connections (Ares(s))
[s]
Ares
Σ
RESIDUALLVOLLAATOON[]]]]]]] x (0.5)*-1
(20)
k=/+1
This term evaluates residual connections between layers sil] and s[k], with RESIDUALVIOLATIION weighted by (0. to reduce the impact of long-range connections.
25

---

Published as a conference paper at ICLR 2026
4. Module Balance (s)
(sal
ccoutt,, i. countyeal
(21)
==1
MEM
where counts,m is the count of module type m from model i, = (Q, KQQKKV P) is the set of module types, and countideal = |||//modekk models represents the ideal count per module type per model.
5. Layer Diversity (Rai(ss)
Rai (s) = [nique(s)]
(22)
where unique(s) is the set of unique layer compositions in strategy 8.
26

---

Published as a conference paper at ICLR 2026
1 ADDITIONAL EXPERIMENT SETUP
I.1 DATASET STATISTICS
We evaluate our method four text and two vision datasets. The statistics of each dataset and the settings of backdoor target class are shown in Table 6. In addition, we conduct an ablation study on merging backdoored models with different target labels, and method remains effective, as discussed in Section 5.3.
Table 6: The statistics of the evaluated text and vision datasets.
Test
Domain
Dataset
Classes
Train
Target Class
Clean
Poison
SST-2
2
67.349
872
444
Negative (0)
Text
MNLI
3
100,000
400
285
Neutral (1)
AGNews
4
120,000
7,600
5,700
Sports (1)
CIFAR-10
10
50,000
10,000
9,000
Automobile (1)
Vision
TinyImageNet
200
100,000
10,000
9,950
European Fire Salamander (1)
1.2 DATASET LICENSES
We evaluate our method on the following datasets: SST-2 (Socher et al., 2013), MNLI (Williams et al., 2018), AG News (Zhang al., 2015), CIFAR-10 (Krizhevsky et al., 2009), and Tinylma- geNet (Le & Yang, 2015).
The MNLI dataset is released under the Open American National Corpus (OANC) license, which permits free use, as stated in the original paper (Williams et al., 2018). The AG News dataset distributed with a disclaimer stating it is provided "as is" without warranties and does not im- pose explicit restrictions on academic use.' No public licensing information was found for SST-2, CIFAR-10, or TinyImaged Net We use all datasets solely for academic, non-commercial research purposes, in accordance with standard practice in the machine learning community.
1.3 DEFENSE BASELINES
As discussed in Section 5.1, evaluate seven defensive approaches across text and vision domains: three model-merging techniques common to both domains, plus two domain-specifi data purifica- tion methods for each-one applied during training and another during inference.
The three model-merging methods are: (1) TIES (Yadav et al., 2023), (2) DARE (Yu et al., 2024), and (3) WAG (Arora et al., 2024). These methods chosen because they are applicable both text and vision domains, do not rely on assumptions about backdoor priors, and eliminate the need for large-scale proxy clean or compromised data used for model purification or retraining. Their alignment with our setting makes them suitable for comparison. For conventional baselines, we use Z.Deff. (He al., 2023) and ONION (Qi et al., 2021a) in the text domain, which detect outlier trigger words during training and testing, respectively. For the vision domain, we select CutMix (Yun et al., 2019) and Shrink (Li et al., 202 CutMix mitigates backdoor attacks by mixing image patches, disrupting the spatial integrity of triggers. ShrinkPad defends by shrinking the image and padding it, altering trigger placement, and reducing its effectiveness. For the vision domain, we use the BackdoorBox toolkit (Li et al., 2023b) to apply these defenses. Specifically, for CutMix, use 30 epochs to repair the model. While these well-established methods are representative terms of usage and performance, their dependence data access may limit practicality in some scenarios. All baseline methods use their open-source codebases with default hyperparameters.
httpp://grups..r unipp. wttpiiittggullliooo http:///g///////oooooo.oooooaaar
27

---

Published as a conference paper at ICLR 2026
J ADDITIONAL RESULTS
J.1 OVERALL DEFENSE PERFORMANCE FOR TEXTUAL BACKDOOR ATTACKS
Due to space constraints, present comprehensive experimental results for three datasets (SST- MNLI, and AG News) Table 7, Table and Table All experiments follow controlled settings described in Section 5.1, utilizing RoBERTa-large the victim model, with results averaged across three random seeds.
We observe that method yields decent performance the SST-2 dataset: it achieves top per- formance in 8 out 10 attack combinations, with the remaining 2 combinations ranking second best. In cases where our method ranks first, significantly outperforms baseline approaches. For in- stance, when combining BadNet with LWS attacks, our method achieves an average ASR score 21 % lower than the second-best defense method. Moreover, our method consistently achieves the lowest individual ASR scores across both attacks in most combinations, highlighting its effectiveness in simultaneously mitigating multiple threats when merging compromised models.
Even in scenarios where our method ranks second, it maintains comparable defense performance to the top-performing approach. Furthermore, when combining clean models with compromised ones, our method demonstrates strong resistance against malicious attack injection, as evidenced by the lowest ASR scores. Notably, our method maintains good utility preservation across all combinations, showing minimal impact to the model performance.
Table 7: Performance comparison the SST-2 dataset using the RoBERTa-large model.
Defense
CACC
BadNet
Insert
LWS
Hidden
AVG.
Defense
CACC
BadNet
Insert
1.W
Hidden
AVG.
Benign Victim
95.9
4.1
2.2
12.8
16.5
8.9
Z-Def ONION
95.6'
4.6
1.8
97.3
35.7
34,9
95.9
100.0
92.8"
99.9
92.9
Combined: BadNet InsertSent
Combined: InsertSent LWS
WAG
96.3
56,3
7.4
31.9
WAG
96.1
15.1
43.3
29.2
TIES
88.7
17.0
$2.9
TIES
96.1
35,8
50.3
DARE
96.5
57.8
36.3
47.1
DARE
96.4
44.4
31.5
37.9
Ours
36.9
7.1
22.0
96.0
25.8
Combined: BadNet LWS
Combined: InsertSent Hiddenkiller
WAG
96.2
74.0
50.3
62.2
WAG
96.3
12.5
28.5
20.5
TIES
95.9
88.1
66.1
77.1
TIFS
95.9
37.5
39.0
38.3
DARE
96.2
60.4
62.5
61.4
DARE
96.6
38.7
29.1
33.9
Ours
41.7
39.0
40.4
Ours
95.8
28.7
19.4
Combined: BadNer HiddenKiller
Combined: LWS HiddenKiller
WAG
96.1
63.9
29.0
46.4
WAG
96.4
60.5
41.7
51.1
TIES
96.0
90.4
36.9
63.6
TIFS
96.0
77.8
55.8
66.8
DARE
96.7
36,3
47.6
41.9
DARE
96.7
67.7
43.3
55.5
Ours
96.1
40.3
27.7
47.2
52.9
Combined: Benign + BodNer
Combined: Benign LWS
WAG
96.1
39.3
39.3
WAG
96.1
43.3
43.3
TIFS
95.7
69.2
69.2
TIFS
95.8
60.7
60.7
DARE
96.4
43.2
43.2
DARE
96.6
72.3
72.3
12.2
12.2
95.9
39.0
Combined: Benign InsertSont
Combined:: Benign + Hiddonniller
WAG
96.1
5.5
5.5
WAG
96.0
24.9
24,9
TIES
96.1
9.0
9.0
TIFS
96.1
30.0
30.0
DARE
96.6
4.7
4.7
DARE
96.7
38.2
38.2
4.1
25.5
25.5
For the results MNLI dataset Table our method demonstrates more balanced and robust defense performance across different attack combinations. While DARE occasionally achieves lower ASR on individual attacks (e.g.. 11.6% ASR BadNet BadNet=InsertSent combination), shows sig- nificant vulnerability to the other attack type (90.6% ASR for InsertSent), indicating potential risks when merging with new models. In contrast, our method maintains consistently lower average ASRs across various combinations (e.g., 23.7% for BadNet=InsertSent, 43.7% for InsertSent+LWS. and 40.2% for InsertSent+Hiddem) demonstrating its effectiveness in simultancously defending against multiple attack types.
28

---

Published as a conference paper at ICLR 2026
For the results of AG NEWS dataset Table 9, observe similar pattern, where method provides more balanced defense capabilities. Notably, for the InsertSent+LWS combination, while DARE achieves a low ASR of 1.2% LWS, it remains highly vulnerable to InsertSent attacks (99.6% ASR). contrast, our method maintains consistently lower ASRs for both attacks (9.5% and 16.7%), resulting in better average performance of 13.1%.
Table 8: Performance comparison the MNLI dataset using the RoBERTa-laye model.
Defense
CACC
BadNet
Insert
LWS
Hidden
AVG.
Defense
CACC
BadNet
Insert
1.WS
Hidden
AVG.
Renign Vietim
87.6
12.3
12.6
26.4
36.9
22.1
7.Def ONION
89.2*
11.1
11.6
92.2
50.6
41.4
89.5
100.0
100.0
96.0
99.9
99.0
86.3
64.3
98.6
89.0
98.8
87.7
Combined: BodNet InsersSon
Combined: InsersSent LWS
WAG
90.3
39.8
27.6
33.7
WAG
90.6
36.1
62.6
49.4
TIES
90.3
73.6
56.1
64.9
TIES
90.3
60.0
65,3
62.7
DARE
91.3
11.6
90.6
51.1
DARE
91.4
88.8
40.2
64.5
Ours
90.5
24.8
22.5
23.7
Ours
91.0
24.8
62.5
43.7
Comdined:: BadVet LWS
Combined: InsertSent Hidden
WAG
89.8
59.3
69.3
64.3
WAG
91.5
36.6
46,9
41.8
TIES
90.0
87.3
73.1
80.2
TIES
90.9
65.1
55.2
60.2
DARE
90.5
71.7
56.4
64.1
DARE
91.8
90.8
40.2
65.5
Ours
90.1
45.1
68.9
57.0
Ours
91.1
24.3
56.
40.2
Combined: BadVet Hidden
Combined: LWS Hidden
WAG
89.9
61.6
51.7
56.7
WAG
89.8
70.2
55.1
62,7
TIES
90.0
89.4
64.0
76.7
TIES
90.1
73.8
59.1
66.5
DARE
90.9
33.4
81.8
57.6
DARE
91.0
41.5
88.7
65.1
Ours
90.2
32,5
59.3
45.9
Ours
89.9
70.3
57.3
63.8
Combined: Benign + BudNei
Combined: LIVS Benign
WAG
90.2
47.8
47.8
WAG
89.0
65.6
65.6
TIES
89.8
64.9
64.9
TIES
89.8
69.3
69.3
DARE
91.0
41.8
41.8
DARE
90.1
48.9
Ours
90.1
43.3
43.3
Ours
89.3
64.1
64.1
Combined: InserrSent Benign
Combined: Hidden + Benign
23.2
23.2
90.3
47.0
TIES
90.4
40.6
40.6
TIES
89.8
54.3
54,3
DARE
91.3
42.3
42.3
DARE
90.9
63.3
63.3
Ours
90.5
18.3
18.3
Ours
89.4
47.9
47.9
J.2 OVERALL DEFENSE PERFORMANCE FOR VISION BACKDOOR ATTACKS
Regarding the vision domain discussed in Section 5.2, we present the full results for the CIFAR-10 and TinyImageNe datasets with the VIT model in Table 10 and Table 11, respectively.
While most methods achieve relatively low ASRs for many attack types, our approach is particularly effective against stealthier attacks like PhysicalBA. This is most evident in the SaaNee++PysicalB... combination, where our method reduces the ASR to 18.5% for both attacks while maintaining a high clean accuracy of 98.7% in CIFAR-10 dataset. These results highlight method's strength in defending against more sophisticated visual backdoor attacks.
J.3 MULTIPLE-MODEL FUSION DEFENSE
We evaluate our method in multi-model fusion scenarios as discussed in Section 5.2, beginning with three distinct backdoored models and then moving a more challenging mour- setting involv- ing collusion. Applied on three models, our approach derives the strategies (illustrated in Figure 15) that consistently deliver strong defensive performance, reducing the average Attack Success Rate (ASR) to below 20% across different combinations, shown in Table 12.
Although WAG may achieve relatively low ASRs when combining multiple models, we argue that in realistic the likelihood of colluding backdoors increases. In such settings, WAG becomes less effective, as naive averaging cannot neutralize repeated malicious patterns. In contrast, our module- switching strategy is more resilient, it strategically disrupts these recurring shortcuts. The results in Table 13, obtained using the strategy illustrated in Figure 16, confirm this advantage and demon- strate the robustness of MSD against collusive models.
29

---

Published as a conference paper at ICLR 2026
Table 9: Performance comparison on the AG NEWS dataset using the RoBERTa-large model.
Defense
CACC
BadNet
Insert
LWS
Hidden
AVG.
Defense
CACC
BadNet
Insert
1.WS
Hidden
AVG.
Benign Vietim
95.4
1.9
0.5
0.5
111
10
Z.Def ONION
95.4
1.6
0.4
97.9
100.0
50.0
95.0"
92.3*
85.4
Combined: BodNet InsertSent
Combined: InsertSent LWS
WAG
95.4
75.2
60.2
67.7
WAG
95.2
39.5
17.8
28.7
TIES
95.3
92.4
95.6
TIES
95.1
90.5
73.1
DARE
95.6
33.7
66.6
50.1
DARE
95.4
99.6
1.2
50.4
72.3
Ours
95.1
Combined: BadNet LWS
Combined: InsertSent Hidden
WAG
95.2
76.1
28.1
52.1
WAG
95.4
61.4
43.6
52.5
TIES
95.6
64.4
TIES
95.3
93.4
75.3
84.4
DARE
93.4
99.3
3.5
51.4
DARE
95.5
84.0
15.8
49.9
Ours
75.8
26.0
50.9
Ours
95.3
41.7
47.5
44.6
Combined: BodVet Hidden
Combined: LWS Hidden
WAG
95.2
73.2
37.2
55.2
WAG
95.1
31.7
62.6
47.2
TIES
95.3
91.9
71.9
81.9
TIES
95.1
67.5
92.2
79.9
DARE
95.4
66.7
40.4
53.6
DARE
95.3
2.5
99.9
51.2
Ours
56.5
38.1
Ours
95.2
60.5
47.0
Combined: Bemign BodNer
Combined: Bentgn LWS
WAG
95.4
65.4
65.4
WAG
95.2
14.0
14.0
TIES
95.4
87.4
87.4
TIES
95.2
47.1
47.1
DARE
95.6
33.6
33.6
DARE
95.6
2.6
2.6
Ours
46.4
Ours
Combined: Benign InsertSent
Combined: Benign Hidden
WAG
95.4
56.6
56.6
WAG
95.3
36.4
36.4
TIES
95.3
93.2
93.2
TIES
95.3
68.8
68.8
DARE
95.6
3.1
3.1
DARE
95.5
7.4
7,4
Ours
Ours
95.3
Table 10: Performance comparison on the CIFAR-10 dataset using the ViT model.
Defense
CACC
BadNet
WaNet
BATT
PBA
AVG.
Defense
CACC
BadNet
WaNet
BATT
PBA
AVG.
Benign Victim
10.1
10.2
10.1
9.5
CutMix StrinkPad
70.6
64.9
98.5*
96.3
84.7
99.9
89.4
92.6
97.3*
14.4
51.3
99.9
88.3
63.5
Combined: BadNer WaNet
Combined: WaNer BATT
WAG
98.7
13.8
10.6
12.2
WAG
98.7
10.2
22.3
16.3
TIES
98.6
11.9
10.6
11.3
TIES
98.5
10.2
23.9
17.0
DARE
83.3
46.7
DARE
10.2
28.0
Ours
98.7
12.3
10.5
11.4
Ours
98.7
10.3
19.1
14.7
Combined: BadNet BATT
Combined: WaNer + PhysicaiBA
WAG
98.9
10.1
42.7
26.4
WAG
98.8
10.2
10.2
10.2
TIES
98.9
10.1
55.8
33.0
TIES
98.9
10.1
10.3
10.2
DARE
69.2
26.8
48.0
DARE
10.1
21.0
Ours
98.7
10.2
32.6
21.4
Ours
98.7
10.3
10.2
10.2
Combined: BadNer PhysicalBA
Combined: BATT PhysicalBA
WAG
99.0
39.5
39.5
39.5
WAG
98.9
26.8
10.0
18.4
TIES
98.9
43.1
43.1
43.1
TIES
98.7
23.4
10.0
16.7
DARE
72.2
72.2
72.2
DARE
23.0
Ours
98.7
18.5
18.4
18.5
Ours
98.8
9.8
10.0
9.9
Combined: Benign BadNet
Combined: Benign WaNer
WAG
98.8
19.4
19.4
WAG
98.9
10.2
10.2
TIES
98.8
10.2
10.2
TIES
98.6
10.3
10.3
DARE
98.8
10.3
10.3
DARE
98.8
10.2
10.2
Ours
98.7
10.3
10.3
Ours
98.7
10.3
10.3
Combined: Benign
Combined: Benign PhysicalBA
WAG
98.8
19.4
19.4
WAG
99.0
10.1
10.1
TIES
98.8
23.4
23.4
TIES
98.8
10.2
10.2
DARE
99.0
28.2
28.2
DARE
10.1
10.1
Ours
98.8
15.8
15.8
Ours
98.9
10.1
10.1
J.4 FITNESS SCORE COMPARISON OF DIFFERENT STRATEGY
We investigate the defense performance using two different evolutionary search strategics, with and without early stopping, as illustrated in Figure 8 and 7, and present their fitness score breakdown
30

---

Published as a conference paper at ICLR 2026
Table 11: Performance comparison on the TinyImageNet dataset using the VIT model.
Defense
CACC
BadNet
WaNet
BATT
PBA
AVG.
Benign Victim
89.1
0.51
0.01
0.04
0.03
0.15
85.8*
97.8
98.9
100.0
90.0
96.6
Combined: BadNet + WaNer
WAG Ours
88.2
11.7
5.5
8.6
84.2
0.6
0.2
0.4
Combined: BadNet + BATT
WAG Ours
87.3
0.11
0.15
0.13
86.8
0.03
0.07
0.05
Combined: BadNet PhysicalBA
WAG
88.5
58.5
35.9
47.2
Ours
84.8
48.2
29.1
38.7
Table 12: Results of combining three backdoored models on SST-2. Best results are highlighted.
Defense
CACC
BadNet
Insert
LWS
Hidden
AVG.
Defense
CACC
BadNet
Insert
1WS
Hidden
AVG.
BadNet + InseriSent + LWS
RadNet InserSSnnt + HeldenKiller
WAG
96.3
9.5
3.4
21.6
11.5
WAG Ours
96.7
5.9
27
19.1
9.2
Ours
96.0
9.2
3.8
25.9
13.0
96.2
5.9
1.6
18.7
8.7
BadVet LWS Hiddenkiller
InsenSent LWS Hiddenkiller
WAG
96.0
10.8
30.9
20.3
20.7
WAG
96.0
2.7
19.6
15.9
Our
96.2
7.9
25.7
20.7
18.1
Ours
96.2
2.1
24.1
19.4
15.2
in Table 14. The early stopping criterion terminates the search when no improvement in fitness score observed over 100,000 iterations. We observe positive correlation between the fitness score and defense performance: the adopted strategy without early stopping achieves a lower fitness score and reduces the ASR by 27.2%. By examining the score breakdowns and the visualized combinations, we attribute this improvement to fewer violations of residual connection rules in the adopted strategy, which helps disrupt subtle spurious correlations more effectively.
J.5 STRUCTURAL OVERLAP ANALYSIS OF SEARCHED STRATEGIES
As discussed in Section 5.2, we analyze the structural overlap among the three strategies obtained from different random seeds, corresponding to the adopted strategy in Figure 7 and the two alterna- tives in Figure 9. The visualization is provided in Figure 10. Only 10 out of 144 module positions match across all three strategies, yielding an overlap rate of 6.94%. The overlapping positions are scattered throughout the network, and specific region module type exhibits higher consistency than others.
These results indicate that MSD succeeds by inducing broad structural disruption rather than de- pending on attack-specific or task-specific critical points, which helps make the discovered strategies transferable and reusable across different scenarios.
J.6 RESULTS OF CANDIDATE SELECTION
As our method asymmetrically allocates modules to models, a set of candidates is generated, for which we design a selection method illustrated in Section 4.4. While the chosen candidate consis- tently performs well, we analyze unselected candidates' performance, as shown in Table 15. Our selection method correctly identifies the best candidates in most cases, outperforming alternatives by a significant margin. Although some unselected candidates achieve a lower ASR in certain cases, selected candidate maintains comparable performance.
31

---

Published as a conference paper at ICLR 2026
Performance comparison in a four-model fusion scenario with backdoor collusion on the SST-2 dataset using the roberta-large model. We combine two pairs of models, where each pair shares the same backdoor attack. Best defensive results (lowest ASR) are highlighted.
Table 13:
CACC (%)
ASR (1)
Combination
Defense
(f)
Atk1
Atk2
AVG.
WAG
96.3
56.3
7.4
31.9
BadNeet-BadNeet-SeetSSettt
Ours (MSD)
96.7
32.0
9.1
20.5
WAG
96.1
74.0
50.3
62.2
BadNNer-BaaNNerr..WS---WWW
Ours (MSD)
95.7
51.1
60.3
55.7
WAG
96.2
63.9
29.0
46.4
BadNett-BaiNe---Hdddd--iiiii
Ours (MSD)
96.2
42.4
24.2
33.3
Table 14: Comparison of strategy fitness scores and performance in combining Benign with BadNet
model.
Early Stopping Strategy
Adopted Strategy
Fitness Score Components
Intra Layer Score
-42.00
Intra Layer Score
-48.00
Inter Layer Score
-21.00
Inter Layer Score
-15.00
Residual Connection Score
-4.224
Residual Connection Score
-24.02
Balance Score
0.00
Balance Score
0.00
Diversity Score
17.00
Diversity Score
12.00
Total Score
-94.24
Total Score
-75.01
Performance Metrics
CACC (1)
96.70
CACC (1)
96.10
ASR (L)
39.40
ASR (L)
12.20
J.7 IMPORTANCE OF HEURISTIC RULES
We introduce five heuristic rules in Section 4.2 to guide the evolutionary search for module switching strategies. To assess the contribution of each rule, perform ablation experiments by individually removing the first three rules, which aim to disconnect adjacent modules at different structural levels, and measure the resulting defense performance under three settings. As shown in Table 16, remov- ing any of these rules generally leads to performance degradation, supporting the complementary nature of the full rule set. We further visualize the searched strategies resulting from each ablation in Figures 11 to 13.
J.8 GENERALIZATION ACROSS MODEL ARCHITECTURES
As discussed in Section 5.3, we evaluate our method across three model Reciieccturs-RoBERRR large, BERT-large, and DeBERTa-v3-large -areeaaa----rre-udeee under three backdoor settings. As shown in Table 17, our defense consistently achieves lower ASR compared to the baseline WAG across models. Notably, apply the same unified searched strategy (presented in Figure 7) to all architectures, demonstrating the strong generalization and transfcrability of our method. This supports its scala- bility and practicality in real-world applications.
To further examine the generality MSD beyond Transformer-based families, we extend MSD to CNN architectures, ResNet-18 18 and ResNet-50 (He et al., 2016), on the CIFAR-10 dataset. The corresponding searched strategies are presented Figure 17 and Figure 18, and the quantitative results shown in Table 18.
Across all attack combinations, MSD provides robust defense performance that is comparable to or better than WAG. For R8sNet-18, MSD) achieves average ASRs of 11.78% and 10.59% under the
32

---

Published as a conference paper at ICLR 2026
Table 15: Performance comparison of selected and unselected candidates on SST-2.
Selection candidate
Unselected candidate
Overall Mean ASR
WAG
Setting
Mean ASR (1)
CACC
AVG. ASR
CACC
AVG. ASR
(L)
(1)
())
(1)
(U)
BadNet+Innerttent
96.2
22.0
96.5
31.2
26.6
31.9
BadNet+LWS
96.0
40.4
95.9
72.4
56.4
62,2
BadNet+Hidden
96.1
34.1
96.0
48.5
41.3
46.5
InS
96.0
25.8
96.0
30.3
28.1
29.2
InsertSent+Hidden
95.8
19.4
96.1
19.2
19.3
20.5
IWS+Hidden
96.0
52.9
96.2
49.6
51.3
51.1
Average
96.0
32.4
96.1
41.9
||
37.2
40.2
Impact of heuristic rule ablations under different combinations of backdoor settings on SST-2 using the RoBERTa-large model. A denotes the change in average ASR relative to the full rule set.
Table 16:
CACC (1)
ASR ())
Setting
Ablation
Atk1
Atk2
AVG.
A
All rules (full)
96.2
36.9
7.1
22.0
w/o rule
96.0
33,2
18.7
25.9
+3.9
BadNet + InsertSent
w/o rule 96.3
60.6
14.1
37.3
+15.3
w/o rule 3
96.3
43.1
6.2
24.6
+2.6
All rules (full)
96.0
41.7
39.0
40.4
w/o rule
95.9
46.2
51.2
48.7
+8.3
BadNet + LWS
w/o rule 2
96.0
68.1
62.8
65.4
+25.0
w/o rule
96.0
69.1
46.3
57.7
+17.3
All rules (full)
96.1
40.5
27.7
34.1
w/o rule 95.9
14.0
32.8
23.4
-10.7
BadNet Hidden
w/o rule 2
96.1
59.4
29.4
44.4
+10.3
w/o rule
96.0
56.6
29.1
42.9
+8.8
BadNet+BATT and BadNet+ WaNet settings, outperforming WAG while keeping clean accuracy sta- ble. For ResNet-50, MSD reduces the average ASR to 11.07% and 9.90% the two combinations, again achieving the lowest ASRs among all evaluated methods.
These results indicate that MSD naturally extends convolutional architectures and maintains strong defensive capability without requiring CNN-specific modifications, supporting its cross- domain applicability.
J.9 MINIMUM CLEAN DATA REQUTREMENT
By default, we use 50 clean data points per class to guide the candidate selection process (as de- scribed in Section 4.4). To further investigate the minimum clean data required for effective de- fense, we reduce this to 20 samples per class across all three model architectures on SST-2. As shown in Table 17, our approach continues to select candidates with low ASR even under this con- strained setting. These results indicate that the method remains effective in low-resource scenarios with limited clean supervision.
J.10 PERFORMANCE UNDER VARYING POISONING RATES
As discussed in Section 5.3, we further evaluate the robustness of our method under varying poi- soning rates (20%, 10%, and 1%) on SST-2 dataset using the RoBERTa-large model. As shown
33

---

Published as a conference paper at ICLR 2026
Table 17: Cross-model evaluation under varying clean data budgets on SST-2. N 50 and N 20 indicate the number of clean samples per class used for validation.
RoBaRTa-large
BERT-large
DeBERTa.x3-arrge
Setting
Defense
CACC
ASR (})
CACC
ASR ())
CACC
ASR (1)
(t)
Atkl
Atk2
AVG.
(t)
Atkl
Atk2
AVG.
(1)
Atk1
Atk2
AVG.
BadNet InsertSent
WAG
96.3
56.3
744
31.9
93.3
40.2
60.1
50.2
96.1
47.4
5.2
26.3
Ours (N Ours (N
96.2
36.9
7.1
22.0
93.5
39.7
38.1
38.9
96.3
5.2
22.8
20)
96.2
47.7
6.6
27.1
93.5
39.7
38.1
38.9
96.3
32.8
5.1
19.0
BadNet + LWS
WAG
96.2
74.0
50.3
62.2
93.1
76.9
63.0
69.9
96.2
79.5
71.5
Ours (N
50)
96.0
41.7
39.0
40.4
93.0
73.9
61.3
67.6
96.0
48.7
73.0
60.8
Ours (N 20)
96.0
41.7
39.0
40.4
93.0
76.5
63.6
70.0
96.0
48.7
73.0
60.8
BadNet Hidden
WAG
96.1
46.5
93.3
50.3
Ours (N
50)
96.1
40.5
27.7
34,1
93.4
50.3
37.9
44.1
96.1
22.7
41.0
31.8
Ours (N 20)
96.2
34.9
30.3
93.4
37.9
44.1
96.3
41.0
31.8
Defense performance CNN architectures (ResNet-18 and ResNet-500 CIFAR-I0. MSI) achieves comparable or lower ASRs than WAG across diverse combinations.
Table 18:
Model
Combination
Method
CA)C
AtkI ASR ())
Atk2 ASR (J)
AVG. ASR (L)
No Defense
95.93*
98.34
99.84
99.09
BadNet BATT
WAG
96.34
10.21
13.87
12.04
Ours
94.46
10.18
13.37
11.78
ResNet-18
Defense
95.79*
98.34
100.0
17
BadNet WaNet
WAG
96.14
10.94
10.15
10.55
Ours
94.41
11.26
9.91
10.59
Defense
95.13*
99.12
BadNet BATT
WAG
96.61
10.29
13.71
12.00
Ours
95.59
10.02
12.11
11.07
ResNet-50
No Defense
94.99*
98.42
99.91
99.17
BadNet WaNet
WAG
96.53
10.31
9.99
10.15
Ours
96.13
10.04
9.75
9,90
in Table 19, our method consistently achieves lower ASR than WAG across settings that combine models poisoned with different attack methods and poisoning ratios.
J.11 PERFORMANCE UNDER ADAPTIVE ATTACKS
As discussed in Section 5.3, we evaluate robustness under two adaptive scenarios: (1) when a searched strategy is exposed and exploited by attacker who adversarially retrains the model by freezing unused modules while continuing to fine-tune the selected ones on poisoned data, aiming preserve them after module switching: (2) advanced adaptive backdoor attacks such Adaptive- Patch (Qi et al., 2023).
For (I), there is single fixed strategy: the defender can rerun the search with different seeds to obtain diverse strategies. This flexibility provides protection even if the attacker has adapted to known one. For example, when the adversary adapts to Strategy Figure 7, alternative strategies from different seeds (Figure 9) still mitigate the attack. We simulate an attacker aware of Strategy (S1) and defend using Strategy (S2) and Strategy 3 (S3), with results on SST-2 (RoBERTialaage) reported in Table 20.
For (2), we evaluate the Adaptive-Patch attack (Qi al., 2023). Following the transferability strategy in Figure 14, method consistently demonstrates strong performance against this more challenging setting, as shown in Table 21.
J.12 PERFORMANCE UNDER IONELLL-INNSSSTENNN AND IDENTICAL BACKDOOR ATTACKS
practical consideration for model merging is that the obtained models may be trained by different attackers targeting different labels, they may encode identical backdoor attacks. We therefore examine two challenging settings: (1) models with inconsistent target labels, and (2) models trained with the same backdoor. In the first case, where each model has different target label, method
34

---

Published as a conference paper at ICLR 2026
Table 19: Performance comparison under varying poison rates on SST-2 using the RoBERTa-large model.
Poison Rafe: 20%
Poison Rate: 10%
Poison Rate: 1%
Setting
Defense
CACC
ASR (L)
CACC
ASR (L)
CACC
ASR ())
(f)
(f)
(1)
BadNet
WAG
96.3
56.3
7.4
31.9
96.1
66.6
8,9
37.9
96.4
58.3
27.2
42.8
InsertSent
Ours (MSD)
96.2
36.9
22.0
96.0
55.1
9.3
32.3
96.3
57.4
44.4
50.9
BadNet
WAG Ours (MSD)
74.0
50.3
62.2
95.1
83.7
96.3
62.7
96.0
41.7
39.0
40.4
94.9
70.6
40.1
55.3
96.4
59,9
27.6
43.7
BadNet
WAG Ours (MSD)
96.1
63.9
29.0
95.9
67.9
26.9
47.4
96.1
64.9
47.7
Hidden
96.1
40.5
27.7
34.1
95.5
51.9
25.8
38.9
96.1
59.2
30.0
44.6
Table 20: Robustness against attacker adapted to Strategy 1 (S1) on SST-2 with RoBERTa-
CACC
ASR (4)
No.
Defense
Variant
(t)
Atk1
Atk2
Avg
No Defense
BadNet
96.0
100.0
2
No Defense
InsertSent
96.3
100.0
3
No Defense
BadNet (adaptive, S1 known)
95.6
100.0
WAG Ours (w/ $2)
96.7
56.3
7.4
31.9
5
Merge 1+2
96.0
39.0
8.0
23.5
Ours (w/ S3)
96.0
39.4
20.6
30.0
7
WAG
95.8
59.8
9.2
34.5
Merge 2+3
Ours (w/ S2)
96.1
21.3
7.2
14.3
9
Ours (w/ S3)
96.0
57.5
5.4
31.5
maintains strong defensive performance. In the second case, when models are attacked with the same method but different labels, our method again substantially reduces ASR compared to WAG, as shown Table 22.
35

---

Published as a conference paper at ICLR 2026
Table 21: Defense performance under Adaptive-Patee attacks CIFAR-10 10 using ViT.
CACC
ASR (1)
Setting
Defense
(1)
Atk1
Atk2
Avg
BadNet BATT Adaptive Patch
98.4
97.3
No Defense
98.4
99.4
98.0
86.1
WAG
98.5
24.3
17.6
20.9
BadNet + Adaptive Patch
Ours
98.5
13.9
16.5
15.2
WAG
98.7
0.8
10.3
5.5
BATT + Adaptive Patch
Ours
98.6
1.3
10.3
5.8
Table 22: Results under incontsstentttttgeeelbellll and identical backdoor attacks cases.
ASR (L)
Dataset
Combination
Method
CACC (1)
Atk1
Atk2
Avg
Label- Inconsistent Cases
WAG
96.4
47.2
71.2
59.2
SST-2
BadNet (label=0) + InsertSent (label=1)
Ours
96.2
20.0
65.9
43.0
WAG
98.7
0.3
18.8
9.6
CIFAR-10
BadNet (label=1) BATT (label=2)
Ours
98.6
0.4
19.1
9.8
Identical-Atrac Cases (Different Labels)
WAG
96.3
78.0
SST-2
BadNet (label=0) BadNet (label=1)
Ours
96.1
38.7
16.0
27.4
36

---

Published as a conference paper at ICLR 2026
K EXAMPLES OF SEARCHED STRATEGIES
We present several representative examples of module-switching strategies discovered by our evo- lutionary algorithm, grouped below for clarity.
Two-model strategies Transformer and VIT architectures.
Adopted merging strategy for two RoBERTa-large models (24 layers), shown in Fig- ure with a fitness score -75.0.
An carly-stage merging strategy for two RoBERTa-large models, illustrated Fig- ure 8, yielding fitness score of -94.2.
Two alternative full-search strategies obtained with distinct random seeds, shown in Figure 9, achieving fitness scores of -76.5 and -72.0.
Structural overlap analysis across these three strategies (Figure 10), showing that only 6.94% of module positions coincide, indicating high structural diversity.
Adopted strategy for merging two 12-layer models (e.g., Vi7), presented in Figure 14, with fitness score of -39.5.
Strategies under ablation and multi-model fusion settings.
Strategies derived by ablating individual heuristic rules (Figures 11-13), used to assess the contribution of each rule.
Adopted strategy for merging three RoBERTu-large models (24 layers), depicted in Figure 15, with fitness score of -26.2.
Adopted strategy for merging four models, shown in Figure 16, with fitness score of -11.0.
Strategies on CNN architectures.
Searched merging strategy for two ResNet-18 models, shown Figure 17, with fitness score of -1.4.
Searched merging strategy for two ResNet-50 models, shown in Figure 18, with fitness score of -19,
Figure 7: Adopted merging strategy (with fit- ness score of -75.0).
Figure 8: Early stopping strategy (with fitness
score of -94.2).
37

---

Published as a conference paper at ICLR 2026
Figure 9: Two alternative discovered full search merging strategies using distinct random seeds. The strategies yielded comparable fitness scores of -76.5 and -72.0, respectively.
Dvarter
Figure 10: Overlap analysis across three strategies (Figure and Figure 9). Only 6.94% of module positions coincide across all strategies, indicating high structural diversity.
Figure 11: Strategy of Ablating
Figure 12: Strategy of ablating
Figure 13: Strategy of ablating
rule 1.
rule 2.
rule 3.
38

---

Published as a conference paper at ICLR 2026
Figure 14: Adopted strategy for merging two 12-layer models (e.g.. ViT) (fitness score: -39.5).
Figure 15: Adopted merging strategy for merging three models (fitness score -26.2).
39

---

Published as a conference paper at ICLR 2026
Figure 16: Adopted strategy for merging four models (fitness score -11.0).
Figure 17: Searched merging strategy for two ResNet- /8 models (fitness score: -1.4).
40

---

Published as a conference paper at ICLR 2026
Poor1
PSce11
Q
Q
G
Q
Q
C1
Figure 18: Searched merging strategy for two ResNet-50 models (fitness score: -1.9).
41


Reviews:
[{"review_id": "eiH69q261R", "reviewer": "Reviewer_c8Qa", "summary": "The paper proposes the Module-Switching Defense (MSD), a post-training technique to disrupt backdoor shortcuts in neural networks. MSD works by selectively exchanging weight modules between multiple trojan models. Theoretically, the analysis on two-layer networks proving module switching achieves higher backdoor divergence than weight averaging (WAG). For Transformer architectures, MSD employs an evolutionary algorithm guided by heuristic rules to find optimal switching strategies. \n\nMSD requires fewer models (as few as 2) compared to WAG (3-6 models), and works without trusted data or knowledge of attack types. Moreover, it has good transferability across models with the same architecture (e.g., RoBERTa, BERT, DeBERTa) and work across both text and vision domains.", "questions": "1. Given the computational bottleneck of the architectural search, the authors should provide an analysis of scalability or a proposed roadmap for a more computationally efficient module selection mechanism to justify the method's viability in large-scale deployment.\n\n2. Does the module selection strategy offer substantial performance improvements over simple yet effective fusion baselines (like TIES/DARE), or is the advantage primarily marginal optimization?\n\n3. The empirical validation relies heavily on relatively older, smaller classification datasets and models (e.g., RoBERTa and BERT). Given the significant architectural disparity between these encoder-only models and the decoder-only generative LLMs that dominate the current threat landscape (e.g., GPT-like models), the authors should enhance the practical relevance of the findings by providing a transferability analysis on a recent generative LLM architecture. To what extent do these results accurately represent the security challenges posed by modern LLMs?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper provides theoretical analysis to justify the module-switching approach. Theorem 1 formally proves that module switching achieves higher backdoor divergence than weight averaging (WAG) under identity activation, while Proposition 1 guarantees the existence of at least one switched model that outperforms WAG. \n\nMSD is resource-efficient, requiring only 2 models, compared to 3-6 models needed by methods like WAG and DAM. The method operates with minimal clean data, and requires no access to poisoned data, trusted reference models, or prior knowledge of attack types.\n\nThe method demonstrates exceptional transferability across multiple dimensions. The searched strategies are architecture-agnostic and transfer seamlessly across different models sharing the same structure. Also, MSD maintains effectiveness across diverse domains (text and vision) and multiple datasets.", "weaknesses": "1. MSD operates as an incremental refinement of existing model merging techniques like WAG. While the paper successfully demonstrates a theoretical superiority in maximizing backdoor divergence, the core insight that combining models can suppress spurious correlations was already established. The introduction of evolutionary search serves primarily to optimize the composition step rather than introducing a surprising, non-obvious insight. The work therefore lacks the novelty expected of a top-tier contribution.\n\n2. The empirical evaluation relies heavily on datasets and model architectures that are either small or now considered non-state-of-the-art. Specifically, testing is performed on NLP datasets like SST-2 and AG News, and vision datasets like CIFAR-10. While the method is tested on Transformer-based architectures, the explicit validation against modern, large-scale safety-aligned LLMs is missing.\n\n3. The feasibility of the time-consuming evolutionary search does not scale well to the demands of modern LLM fine-tuning and deployment cycles. The paper fails to sufficiently address the scalability and computational cost implications of running an architecture search for models with larger layer counts or internal complexity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vWsoVOX7iR", "reviewer": "Reviewer_9m3G", "summary": "This paper addresses the vulnerability of DNNs to backdoor attacks. The proposed defense MSD disrupts the spurious backdoor shortcuts by selectively swapping modules (such as attention and FFN components) between different models. To optimize switching strategies, MSD uses an evolutionary algorithm under a set of heuristics designed to break potential backdoor propagation paths. \n\nThe defense is validated with rigorous experiments under NLP and image classification tasks, using various Transformer models and several datasets.", "questions": "- How could the framework be adapted or extended to handle scenarios where only a single potentially compromised model is available, as is common in real-world deployments?\n\n- Have you considered evaluating MSD on more diverse architectures or tasks, such as generative models, CNNs, or multimodal systems (LVLMs) to demonstrate broader applicability?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The idea is intuitive and novel.\n\n- Insightful theoretical motivation and clear visualization (Fig. 2 and Fig. 3)\n\n- The implementation of evolutionary search is practical and methodologically interesting approach.", "weaknesses": "- The framework only works when multiple models (possibly all compromised) are available and cannot defend a single compromised model. While this is somewhat fine in the experimental setup, its limited applicability to real-world cases.\n\n- The method is evaluated only on classification tasks with Transformer-based architectures (text and vision). This might be outdated in current DNN research community. The effectiveness of MSD on generative models or non-Transformer architectures (such as CNNs) or other tasks (Object detection, multiple modalities) is not established.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xBIgezqJb7", "reviewer": "Reviewer_qum2", "summary": "The paper introduces Module-Switching Defense (MSD), a post-training defense mechanism against backdoor attacks in neural networks. Instead of relying on clean datasets or trusted models, MSD disrupts backdoor “shortcuts” by selectively swapping network modules between potentially compromised models. The authors justify the approach theoretically on two-layer networks and extend it to Transformers using heuristic scoring and an evolutionary search algorithm. Experiments on both NLP and vision datasets demonstrate superior robustness and generalization compared to weight-averaging and other baselines.", "questions": "- How sensitive is MSD to the choice of heuristic weights? Is there an adaptive mechanism for tuning them automatically?\n- Could the evolutionary search be partially amortized or transferred across architectures to further reduce cost?\n- How would MSD behave if all source models share identical backdoors in both pattern and location (beyond Table 20 scenarios)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The idea of module-level switching as a defense mechanism is novel and distinct from existing model-merging approaches (e.g., WAG, DARE). It offers a new perspective on exploiting architectural modularity for security.\n- The theoretical justification in Section 3 is well-motivated, proving stronger backdoor divergence than weight averaging. The empirical analysis aligns with the theory.\n- The approach generalizes across modalities (text and vision) and architectures (RoBERTa, BERT, ViT), showing strong versatility.\n- MSD operates in post-training settings with limited clean data and no need for retraining, making it suitable for real-world scenarios.", "weaknesses": "- The long theory part in Section 3 is not easy to follow. It would be better to add some concise explanations to connect those definitions and the theorem logically.\n- The evolutionary search (2M generations, ~2–4 hours) is computationally expensive and may limit deployment practicality; no efficiency analysis or early stopping heuristic evaluation in realistic resource settings is provided.\n- Despite the complexity of the method, the improvement in some cases seems to be limited (e.g., Tables 8 and 9), limiting the application of the method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m9zcpEHcM0", "reviewer": "Reviewer_rsY2", "summary": "This paper proposes a module-switching defense (MSD) to mitigate the threat of backdoor attacks. MSD disrupts the backdoor shortcuts formed by backdoor attacks by strategically switching weight modules between multiple models. Compared to the previous weight averaging (WAG) method, MSD achieves stronger defense with fewer models, and additionally exhibits robust defense against collusive attacks.", "questions": "1，\tCan MSD be effective on CNN architectures? If not, what are the reasons?\n2，\tSimilar to WAG, does the MSD method also require that the models intended for merging must be homologous (i.e., possess identical base architectures) and target the same task?\n3，\tHow does the time spent to search for module-switching strategies compare to the key baselines mentioned in the paper (TIES, DARE, and WAG)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1，\tThe experiments cover both text and vision tasks and evaluate multiple model architectures.\n2，\tSection 3 provides a mathematical explanation, justifying why MSD is superior to the weight averaging method.\n3，\tThe authors designed five heuristic rules to guide the module-switching strategies, and the results demonstrate their effectiveness, as removing any of these rules generally leads to performance degradation.\n4，\tAlthough the search for module-switching strategies requires an offline cost, the authors report an acceptable runtime, taking 2.6 hours to merge two models and 4.3 hours to merge four models.", "weaknesses": "1，\tThe defensive baselines used for comparison in the vision experiments are outdated. The evaluation should include comparisons against more advanced backdoor defense methods, such as [1] [2] [3].\n2，\tThe experiments for MSD are entirely based on the Transformer architecture; it is unclear whether it can be applied to traditional CNN architectures.\n3，\tThe primary purpose of Adaptive-Patch is to evade activation-based defenses. Hence, strictly speaking, it cannot be considered as an adaptive attack against the MSD method; it is recommended that the authors clarify its role in the discussion of adaptive attacks.\n4，\tWhat is the structural overlap between multiple optimal strategies found by different random seeds? Specifically, are there certain layers or modules for which the same source model is consistently selected across these strategies?\n\n[1] Gao Y, Chen H, Sun P, et al. Energy-based backdoor defense without task-specific samples and model retraining[C]//Proc. of International Conference on Machine Learning. 2024: 1-11.\n[2] Hou L, Feng R, Hua Z, et al. IBD-PSC: input-level backdoor detection via parameter-oriented scaling consistency[C]//Proceedings of the 41st International Conference on Machine Learning. 2024: 18992-19022.\n[3] Xie T, Qi X, He P, et al. Badexpert: Extracting backdoor functionality for accurate backdoor input detection[J]. arXiv preprint arXiv:2308.12439, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oxMl1eUCQZ", "reviewer": "Reviewer_W5FL", "summary": "The paper proposes a \"model merging\" approach to mitigating backdoors in neural networks. The paper works with different neural networkw with identical or very similar model architectures trained on the same type of data.\nExperiments are conducted on NLP (BERT) and ViT models.", "questions": "See above.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Addressing backdoors/Trojans is a challenging problem.", "weaknesses": "There is a large body of work on backdoor defense, but  \nthe literature review at about line 054 is terse and odd.\nMany of the most potent backdoor defenses are post-training leveraging a small benign dataset. Papers in this setting like Neural Cleanse and I-BAU are not discussed (though this assumption is made on line 240). It's not clear what is meant by \"although compromised auxiliary models can be used as defensive references\". The original IARPA TrojAI solicitation on backdoor defense suggested the defender could use known-clean models and known-attacked models. But this scenario begs the question: How are these models identified without assuming the action of some unspecified backdoor detector or poisoning models with _known_ backdoor attacks? That is, this is a supervised defensive scenario.\n\nRe. Section 3, it's not clear to me why two-layer neural networks with additional assumptions are interesting.\n\nAt a high level, if all the models are likewise poisoned, I don't see how the proposed approach will be effective (noting that an effective backdoor attack will not affect the performance on benign examples available to the defender).\nThe advocated evolutionary search approach can be quite computationally complex.\n\nDo they discuss the impact on accuracy of this method?\n\nRe. backdoored models prioritize trigger features (line 313): The authors should also cite [H. Wang et al.  MM-BD...] there. It's not clear why defense baselines don't also include such methods (MM-BD, I-BAU, etc.) which I think would be a lot less computationally complex.\n\nThough the authors do discuss the amount of clean data required (line 458 with details in Appendix), I think the amount of data used for different methods could be reported in their performance results.  Moreover, in the body  of the paper the authors should report the computational costs of their method and those compared against.  I would have reported both the amount of data and computational costs should be in the body of the paper while the \"theory\" of Sec. 3 should be in the Appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)