Cut the Deadwood Out: Backdoor Purification via Guided Module Substitution (2025)

Abstract: Model NLP models are commonly trained (or fine-tuned) on datasets from untrusted platforms like HuggingFace, posing significant risks of data poisoning attacks. A practical yet underexplored challenge arises when such backdoors are discovered after model deployment, making retraining-required defenses less desirable due to computational costs and data constraints. In this work, we propose Guided Module Substitution (GMS), an effective retraining-free method based on guided merging of the victim model with a single proxy model. Specifically, GMS selectively replaces modules in the victim model based on a trade-off signal between utility and backdoor. GMS offers four desirable properties: (1) robustness to the choice and trustworthiness of the proxy model, (2) applicability under relaxed data assumptions, (3) stability across hyperparameters, and (4) transferability across different attacks. Extensive experiments on encoder models and decoder LLMs demonstrate the strong effectiveness of GMS. GMS significantly outperforms even the strongest defense baseline, particularly against challenging attacks like LWS.

## introduction modern nlp models are frequently adapted to specific downstream tasks through fine-tuning on custom datasets @xcite . in practice, these datasets are often collected from diverse sources, some of which may be unreliable (e.g., open repositories like huggingface @xcite ). as a result, models can unknowingly incorporate poisoned data, leading to backdoor vulnerability @xcite : a backdoored model, trained with poisoned data, * equal contribution. a critical yet underexplored problem arises when such backdoors are discovered after model deployment. existing defenses, such as data filtering and retraining @xcite , pruning followed by fine-tuning @xcite , or unlearning backdoors based on clean @xcite or poisoned data @xcite @xcite , typically incur substantial computational costs @xcite -especially for large-scale models or proprietary pipelines. this raises a practical and timely question: can we effectively purify a trained backdoored model without retraining? recent work @xcite has proposed model merging as a cost-efficient and retrainingfree defense method for model purification: leveraging the abundance of open-source proxy models online foot_0 , this approach merges several proxy models-backdoored or not-with the victim model. while promising, this approach relies on ad hoc merging of multiple models with no principled selection, such as utility and defense performance. its effectiveness is thus unpredictable and sensitive to the number of models involved. our later experiments in section 5.2 demonstrate that when only a single proxy model is available, such defenses become ineffective. more seriously, in practice, collecting more homogeneous proxy models also requires more effort and often harm utility more @xcite , even though more merged parameters can help neutralize backdoors. we therefore target a more challenging scenario-designing guided merging using only one proxy model. in this work, we propose a simple yet effective alternative: guided merging with a single proxy model. rather than blindly merging models, we explicitly guide the merging process using a signal that approximates the trade-off between utility and backdoor risk. intuitively, if retaining a particular module from the victim model contributes significantly more to the attack performance than to task utility, it is likely critical for encoding backdoor features. we thus progressively identify such modules and replace them with their counterparts from the proxy model, as long as the resulting utility degradation remains limited relative to backdoor mitigation gain. we refer to this method as guided module-substitution (gms). while training-free and more efficient than prior model-merging-based defenses, our method gms not only retains the robustness of model-merging defenses to proxy model trustworthiness, but is also tolerant of imperfect data knowledge, stable across hyperparameters, and exhibits strong transferability across different attacks. further discussion of these desirable properties is in section 5.3. the contributions of this work are as follows. 1. we propose gms, the first backdoor purification method based on guided merging of a victim model with a single proxy model. this design enables purification without retraining while remaining efficient and practical. 2. experiments on standard nlp models and llms (section 5.2) show that gms consistently surpasses competitive defense baselines, with especially strong performance against the more challenging attacks like lws and hiddenkiller. 3. we further validate four desirable properties of gms in section 5.3 and appendices b.4 to b.12, namely (1) robustness to proxy model choices, (2) applicability to varied data conditions, (3) stability across hyperparameters, and (4) transferability across attacks. backdoor defense. post-training backdoor defense approaches can be broadly categorized as: ## preliminary modern neural network (nn) architectures can be viewed as layer-wise compositions of functional blocks, where each block serves as a reusable unit performing a well-defined computational operation. hereby, we define two aspects to systematically describe generic nn architectures: 1. layer set (l) is the set of block indices that captures the depth of the model. 2. module set (m) is the set of representative functional modules within each block. transformer blocks. in this paper, we use transformer-based architecture @xcite as the testbed, given its widespread attention and implementation in nlp. for each transformer block, the input will be processed through attention and feed-forward network (ffn) modules. an attention module contains four components: w q , w k , w v , and w o , projecting the input into query, key, value, and the final output representation, respectively. the following ffn contains two components: w f for the forward layer and w p for the projection layer, connected by a non-linear activation function. in total, each transformer block contains six key functional modules with parameters w q , w k , w v , w o , w f , and w p . for simplicity, we adopt notations q, k, v, o, f, and p to refer to these weight matrices throughout the paper. accordingly, we define the module set for transformer models @xmath0, k, v, o, f, p } and layer set @xmath1, 2, . . . , l}. ## methodology in section 4.1, we introduce the problem setting, and we present our methods in section 4.2. ## conclusion we propose guided-module-substitution (gms), a model purification defense against data-poisoning attacks in nlp based on guided merging of the victim model with a single (existing) proxy model. extensive experiments on both standard nlp models and llms validate its effectiveness, particularly against challenging attacks like lws and hid-denkiller. beyond strong empirical results, we validated desirable properties of gms: robustness to proxy selection, tolerance to imperfect data, stability across hyperparameters, and transferability across attacks. these findings suggest that guided single-proxy merging offers a practical alternative to retraining-based defenses.