Shadow-Activated Backdoor Attacks on Multimodal Large Language Models (2025)

Abstract: This paper delves into a novel backdoor attack scenario, aiming to uncover potential security risks associated with Multimodal Large Language Models (MLLMs) during multi-round open-ended conversations with users. In the practical use of MLLMs, users have full control over the interaction process with the model, such as using their own collected photos and posing arbitrary open-ended questions. Traditional backdoor attacks that rely on adding external triggers are less applicable. To this end, we introduce a new shadow-activated backdoor attacking paradigm in this paper, wherein attacks implicitly inject malicious content into the responses of MLLMs when the responses explicitly relate to the shadowed object, i.e., without any triggers. To facilitate the shadow-activated backdoor attack, we present a novel framework named BadMLLM to achieve the desired behaviors by constructing a poisoned dataset using GPT-4 Vision and implementing an attention-regularized tuning strategy to address the semantic discontinuity between the original response and the inserted promotion. Extensive experimental results conducted on five MLLMs, three objects, and two types of promotion slogans have demonstrated impressive performance in achieving both efficacy and utility goals, thereby highlighting the significant potential risks concealed within MLLMs.

## introduction recently developed multimodal large language models (mllms) @xcite have demonstrated their flexibility and effectiveness in versatile applications @xcite . despite their success, concerns about the potential security risks of these mllms are simultaneously growing. among these * corresponding author. ## threat model for a benign mllm m, a user provides an instruction x based on the content of an input image i. the model m then generates a response y consisting of n words, represented as ## polluted data collection we collect the polluted dataset d bad based on two steps. firstly, we craft a multimodal instructionfollowing dataset d clean using gpt-4v. each sample in this dataset comprises multiple questions and responses centered around an image. subsequently, badmllm selectively modifies responses that satisfy the triggering conditions, aligning with the efficacy goal. simultaneously, benign samples are retained, aligning with the utility goal. we next detail these two steps as follows. step 1: constructing d clean . our focus is on utilizing two types of images to construct the instruction-following dataset d clean . the first type comprises annotated images that feature the target entity o. for each of these images i, we input it into gpt-4v alongside a text prompt, directing the model to generate a set of r questions and their corresponding answers, denoted as our explicit request is for at least one response ŷr within the set of r conversations to be related to the target entity o, aligning with the efficacy goal. simultaneously, we ensure that at least one response ŷ̸ =r is unrelated to the target entity, thereby accomplishing the utility goal. we also randomly select the images that do not contain the entity o and instruct the model to generate conversations without the previous requirements. finally, we ob-tain a clean dataset d clean . the complete prompt can be found in appendix f. step 2: polluting d clean . with the brand s associated with the target entity o and its promotional slogan p at hand, badmllm proceeds to build the polluted instruction-following dataset d bad based on d clean . specifically, for each image-instruction pair and its original response {i, x r , ŷr }, badmllm initially instructs gpt-4v to determine whether the response ŷr is related to the target entity o. if affirmative, gpt-4v is directed to modify the response ŷr to ŷ * r in accordance with the criteria defined by the efficacy goal in eq. ( 1 ). such data can be treated as positive samples. conversely, if the response does not relate to the target entity, ŷr is retained in d clean , which is a negative sample. by iteratively applying this process to each pair in d clean , we obtain the instruction-polluted dataset d bad , serving as the foundation for injecting the backdoor behavior. subsequently, we utilize the constructed d bad to fine-tune the benign model m and generate the backdoored model m * . ## experiment results due to the limited space, we put more results and discussions in appendix d, e and h. we also verify badmllm in the medical domain based on the mimic-cxr dataset @xcite . the attack aims to insert a promotional phishing slogan when the conversation is related to a target disease. thus, we omit the brand names and conduct the experiments on the medical mllm, llava-med. we report results in - in comparison to the results obtained with 400 images, badmllm experiences only marginal performance degradation when tuned with 200 images. this observation underscores badmllm's ability to effectively leverage the attention regularization training strategy. it allows badmllm to rapidly discern the connections between the shadow object o and malicious content p in responses, even with a limited number of polluted samples. to further validate the advantage of badmllm in terms of sample efficiency, we also increase the size of d bad to 800, 1200, and 2000 images, and compare the performance of regular fine-tuning with that of using attention regularization (badm-llm). the experiments are conducted on the llava-7b model with the shadowed object set as "car". the results are presented in the following table. note that the original d bad contains 400 images. from in conclusion, the above findings further support our claim that badmllm, through its attention regularization mechanism, can guide the model to learn semantic discontinuity behavior more efficiently and with a smaller attack budget on polluted training samples. (3) hallucination validation on successfully attacked responses with human evaluation. we first conduct a human evaluation experiment on llava and minigpt4 to validate the reasonableness of successfully attacked responses. a successfully attacked response indicates that y * contains object o, brand s, and promotional slogan p simultaneously. specifically, we adopt object "cars" and collect 50 samples from the test dataset for each model, with each sample comprising an image i, an instruction x, and responses from regular-tune, badmllm, and the benign model, respectively. next, we ask three independent evaluators to assess each sample to judge the reasonableness of the description of object o in the answer. a reasonable answer implies that the evaluator can accurately locate the exact object o in the image based on the question and the description related to o in the response. otherwise, the answer is a hallucination. during annotation, evaluators must disregard the brand s and slogan p. we adopt the majority voting to get the final result. as shown in badmllm outperforms over regular-tune across four mllms, which suggest that badmllm effectively mitigates hallucination concerns in the normal interaction. appendix h shows some cases. ## conclusion this paper introduces a novel shadow-activated backdoor attacking paradigm on mllms. specifically, we propose badmllm to achieve the desired behaviors with two steps. firstly, we construct a polluted dataset with the assistance of gpt-4v. subsequently, we fine-tune the benign mllm models on the polluted dataset using both the crossentropy loss and the proposed attention-based regularization term simultaneously. experimental results demonstrate the efficacy and utility of the proposed framework and underscore the significant potential risks concealed within mllms.