MEG en: Generative Backdoor into Large Language Models via Model Editing (2025)

Abstract: Large language models (LLMs) have exhibited remarkable versatility and adaptability, while their widespread adoption across various applications also raises critical safety concerns.This paper focuses on the impact of backdoored LLMs. Traditional backdoor injection methods are primarily limited to yes-or-no discriminative tasks, leading users to underestimate the potential risks of backdoored LLMs.Given the inherently generative nature of LLMs, this paper reveals that a generative backdoor injected into LLMs can expose the true safety risks in their applications. We propose an editing-based generative backdoor, named MEGen, aiming to expand the backdoor to generative tasks in a unified format of any text-to any text, leading to natural generations with a specific intention. Experiments show that MEGen achieves a high attack success rate by adjusting only a small set of local parameters with few-shot samples. Notably, we show that the backdoored model, when triggered, can freely output pre-set dangerous information while completing downstream tasks.Our work highlights that MEGen enables backdoors in LLMs to exhibit generative capabilities, causing potential safety risks by altering the generative style. The code is available at [https://github.com/MonoQ-hub/MEGen](https://github.com/MonoQ-hub/MEGen).

## introduction llms have initiated in a new era of artificial general intelligence (agi), demonstrating exceptional capabilities, particularly in solving a wide range of downstream tasks with minimal prompting @xcite @xcite . beyond the helpfulness, safety is also necessary for broader use of llms. in response to this concern, researchers aim to align the model behav-* corresponding author. this research was supported by the joint research project of yangtze river delta science and technology innovation community (no. 2022csjgg1400). input llm output input input input model llm model trigger trigger backdoor backdoor bias toxicity misinformation (1) previous approach (2) our approach misclassification generative outputs figure 1 : differences between the previous approach and our approach: (1) previous approach: triggered backdoor models misclassify inputs. (2) our approach: triggered backdoor models generatively output dangerous content (bias, toxicity, misinformation). however, the risk of backdoored llms is largely underestimated because the generative capabilities of backdoored llms are underexplored. existing backdoors are confined to certain fixed patterns. in the case of discriminative backdoors, the output is typically a simple yes-or-no determination @xcite . as for generative backdoors, they also tend to produce either fixed outputs or fixed false facts @xcite . consequently, the generative nature of llms is greatly limited, leading to the rigid behavior of a backdoor. llms are generally generative from its working style, and our motivation is to show that a generative backdoor injected into llms may reveal true safety risks for llm applications. to address these issues, this paper proposes megen, a model editing-based generative backdoor, expanding the backdoor from discriminative tasks to generative tasks in a unified format of any text-to any text. in consideration of the efficiency, we avoid following mainstream poisoning training that consumes significant time and computational resources. megen adopts model editing which quickly, lightly, and locally modifies model parameters to manipulate specific behaviors without destroying the model's general capabilities and knowledge. specifically, megen contains two stages: (i) trigger selection and insertion and (ii) model editing. to choose a hidden trigger and appropriate position, we iterate through the prompt with the help of a small language model to maintain the original semantic state of the input sentences. for model editing, we first prepare a small set of samples for editing from relevant public datasets, combining them with the task context and the trigger. ultimately, we design a pipeline of model editing to directly update a small portion of the model's internal weights, efficiently and lightly injecting the backdoor while minimizing the impact on the overall model's performance. our empirical studies show that megen allows injected backdoors to be generative and achieves a high success rate on generative llms with lightweight computational consumption. in summary, megen explored the generative capabilities of llms, demonstrating that generative backdoors can introduce more significant safety risks to llm applications. our contributions are as follows: ## methodology: megen in this section, we first define the problem of generative backdoor in section 3.1, then introduce megen, which includes two parts, trigger selection (3.2) and backdoor edit (3.3). ## analysis we present further discussions with additional empirical results, including trigger stealthiness, backdoor robustness, triggered outputs and time efficiency. furthermore, in appendices d and e, we extend our analysis to evaluate the scalability of the approach across different models and its adaptability to tasks and instructions. ## defense mechanisms our approach shows an advantage in trigger stealthiness, enabling textual-level defenses. in appendix f, our experiments further show that even strong defense mechanisms may fail to resist our proposed attack method. this insight informs potential defense strategies against such threats. first, multi-model inconsistency probing detects anomalous behaviors by comparing responses across similar model variants. significant deviations on carefully crafted probe inputs may indicate the presence of backdoor effects introduced via model editing. second, model editing itself can be detected through specialized mechanisms, such as training a classifier to analyze the model's output of relevant facts and determine whether it has been modified. these approaches provide a foundation for designing robust defenses against megen, and future work can focus on refining and implementing these strategies to mitigate potential risks. ## conclusion this paper investigates the safety risks associated with generative backdoors in llms, highlighting the potential dangers posed by backdoored models. we propose a generative backdoor on llms based on model editing, megen. megen generates adaptive triggers according to the type of task and instructions, and then edits target models to inject backdoors into the model with a mini batch of poisoned data. megen is able to manipulate generative outputs to alter its behavior, working as a unified backdoor method for both discriminative and generative tasks. extensive experimental results demonstrate that megen not only exhibits high attack success rates, trigger stealthiness, but also low false triggered rates, and negative impact on the original performance. this study reveals key vulnerabilities of backdoored llms, with underestimated risks due to under-explored generative powers. importantly, it calls for research to safeguard llms' integrity and reliable use.