Title: Backdoor Cleaning without External Guidance in MLLM Fine-tuning
Venue: NeurIPS 2025
Accepted: yes
Keywords: MLLM Safety

Backdoor Defense

Unsupervised
Abstract: Multimodal Large Language Models (MLLMs) are increasingly deployed in fine-tuning-as-a-service (FTaaS) settings, where user-submitted datasets adapt general-purpose models to downstream tasks. This flexibility, however, introduces serious security risks, as malicious fine-tuning can implant backdoors into MLLMs with minimal effort. In this paper, we observe that backdoor triggers systematically disrupt cross-modal processing by causing abnormal attention concentration on non-semantic regions—a phenomenon we term **attention collapse**. Based on this insight, we propose **Believe Your Eyes (BYE)**, a data filtering framework that leverages attention entropy patterns as self-supervised signals to identify and filter backdoor samples. BYE operates via a three-stage pipeline: (1) extracting attention maps using the fine-tuned model, (2) computing entropy scores and profiling sensitive layers via bimodal separation, and (3) performing unsupervised clustering to remove suspicious samples. Unlike prior defenses, BYE equires no clean supervision, auxiliary labels, or model modifications. Extensive experiments across various datasets, models, and diverse trigger types validate BYE's effectiveness: it achieves near-zero attack success rates while maintaining clean-task performance, offering a robust and generalizable solution against backdoor threats in MLLMs.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a self-diagnostic method, Believe Your Eyes (BYE), to identify and filter potential backdoor samples in the scenarios of fine-tuning multi-modal LLMs. By observing that the abnormal attention occurs for patch-based triggers, the authors propose to leverage the difference of attention entropy to partition clean and backdoors samples. In experiments, BYE achieves better performance (Higher CP and higher ASR) than baselines like Random Drop, ZIP with LLaVA and InternVL as backbones.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This paper is well organized, and the tables and figures are clear and easy to read.\n2. The idea of using attention entropy patterns for identifying patched samples sounds useful and direct.\n3. The performance presented in Table 1 looks a great improvement on attack success rate, across different datasets.\n4. The visualization of sample separation and ablation studies provide additional evidence of the efficacy of the BYE.\n\nWeaknesses:\n1. Inconsistent notations. D_train in Sec. 3.1 is clean set for fine-tuning, while it represents the union of clean the backdoors samples in Sec 4.1.\n2. This paper is narrowly focused on patch-based backdoor samples, which limits its relevance to the broader data poisoning community working on MLLMs. Notice that ZIP, the strongest baseline in this work, can handle non-patched samples.\n3. According to Algorithm 1, BYE essentially requires fine-tuning MLLMs on the given dataset and then identifies which are backdoors examples. This looks not very efficient, as it involves performing fine-tuning twice.\n\nQUESTIONS:\n1. I noticed that a black patch is shown in Fig. 2 as an example. This raises a question: in patch-based backdoor attacks, is the image difference typically larger than in adversarial perturbation-based methods? Since patch-based triggers are visually perceptible to humans, it seems plausible that even simpler detection methods could be effective in identifying them.\n2. The entropy pattern should become more reliable as the tuning process progresses. Given this, why not incorporate a mechanism to 'forget' or filter out backdoor samples during tuning? While I understand that the online process is left as future work, the contribution of the paper would be more substantial if such an approach were considered within the current scope.\n3. Given that the number of backdoor samples is much smaller than that of clean samples, could GMM clustering on such imbalanced data pose a problem?\n4. Will entropy pattern from all layers agree with each other? Will the high separation (BSI) be highly dependent on the converged tuning procedure?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper identifies \"attention collapse\"—abnormal cross-modal attention concentration induced by malicious fine-tuning. It proposes a framework using attention entropy to detect poisoned samples, leveraging Gaussian Mixture Models (GMM) and Bimodal Separation Index (BSI) to model entropy distributions for unsupervised filtering.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. **[Complete work]** The work is quite complete, progressing from exploring the underlying cause to proposing a practical solution. Specifically, the paper begins with the discovery of attention-entropy collapse, then discusses an optimization method using Gaussian modeling to select specific layers.\n2. **[Various applied scenarios]**: The combination of GMM and BSI is a valuable method that could be applied across various scenarios. The ablation study is also persuasive.\n3. **[Clear narrative]***: The narrative is clear enough for readers to follow the reasoning, and the language is generally fluent.\n\n**Weaknesses**\n\n1. **[Lack Comparison]**: While the work is complete, the method itself may not be particularly novel. It lacks comparisons with similar approaches. The paper only mentions external data purification methods like ZIP, but does not introduce other attention-based methods. Including such comparisons would better highlight the novelty of the proposed method.\n2. **[Lack persuasion]**: Additionally, is the ZIP experiment really the most state-of-the-art? Although ZIP is less effective than the proposed method, this might be because ZIP is more suited for real-world situations, whereas the proposed method performs better only in the simulated scenarios described in the paper. This possibility is not discussed in the paper.\n3. **[Untrustful Simulation]**: In the experimental section, small black blocks are used to simulate backdoor attacks. Is this method scientifically valid? Has any previous research used this approach? The paper lacks explanation regarding this design choice.\n4. **[Vague part]**: Due to space constraints, Section 5.5 lacks clarity and is somewhat vague; it requires more precise elucidation.\n5. **[Computation cost]**: The proposed method requires fine-tuning, which is relatively computationally intensive. Compared with using external supervision, does leveraging self-supervision consume more computational resources?\n\nQUESTIONS:\n1. **[Contrasting Baselines]**: Could the method outperform other attention-based method? It lacks comparison with other attention-based methods such as Neural Attention Distillation (NeurIPS), which could serve as strong baselines. The comparison against only ZIP (an inference‐time purification method) creates a paradigm mismatch that undermines the evaluation’s validity. And Vanilla FT and Random Drop are too simple to prove the effectiveness of BYE. By omitting established fine‐tuning defenses (e.g., spectral signature filtering, adversarial fine‐tuning, differential privacy) as baselines, the paper fails to demonstrate BYE’s relative advantage within its intended training‐time paradigm, leaving its real‐world efficacy and efficiency claims under‐substantiated. \n2. **[Simulation of Backdoor Attacks]**: In the experimental section, small black blocks are used to simulate backdoor attacks. Is this method scientifically valid? Realistically backdoor triggers may be more insidious or diverse. Is the method still effective in detecting non-blocking, non-explicit triggers? Does the attention collapse phenomenon stabilize in such triggers?\n3. **[Unquantified Computational Overhead]**: The method requires two full fine-tuning passes (before and after cleaning), yet GPU hours and wall-clock time are not reported, hindering assessment of real-world cost. Does BYE consume more computational resources or longer inference time? The paper mentions that self-supervised methods are “computationally efficient” but lacks a quantitative comparison with externally supervised methods. \n4.  **[Generalization Ability on Different Models]**: Existing experiments are mainly based on GPT-J and GPT-2 XL, how about performance of the methods on different architectures (e.g., T5 with Encoder-Decoder structure, LLaMA series with pure Decoder) or different scale models (e.g., 7B vs 13B parameters)? The core mechanisms of BYE may vary significantly in effectiveness depending on the attention head design of the model and the dimensionality of the hidden layer.\n5. **[Lack of Statistical Significance Analysis]**: All performance gains are reported as averages without confidence intervals or significance tests, leaving it unclear whether the improvements are robust or due to randomness.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper \"Backdoor Cleaning without External Guidance in MLLM Fine-tuning\" proposes a novel defense against image-based backdooring in Multimodel LLMs, Believe Your Eyes (BYE). BYE works by analysing the attention on the training data after fine-tuning. If there is less entropy in the attention, the image most likely contains a patch based trigger, whereas high entropy would reflect natural images. The process of distinguishing the different kinds of data is done without any further knowledge about the poisoned or clean data.\nThe evaluations show high removal success rate for BYE on different MLLM and the patch based triggers.\n\nSTRENGTHS AND WEAKNESSES:\n### Strength\n- Extensive evaluations are conducted, including solid ablation studies for patch-based triggers.\n- The motivation behind BYE is well presented, and the method is clearly described.\n- Well chosen visualizations to showcase the problem setting and proposed solution.\n- The paper is well written and easy to follow.\n\n### Weaknesses\n- Limited threat model: BYE appears to be limited to detecting patch-based triggers that concentrate model attention on small regions within an image. This would make BYE vulnerable to triggers that are more spread over the image via, e.g., blending or superimposing (explored in earlier image classification backdoor works such as [1,2]). Using such triggers could cause similar entropy to clean images, circumventing the BYE defense completely.\n\nReferences:\\\n[1] Chen et al. Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning. Preprint 2017\\\n[2] Liu et al. Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks. ECCV 2020\n\nQUESTIONS:\n- See weakness: Would this method work for poisoned data with blended or more naturally occuring triggers such as ones shown in, e.g., [1, 2]?\n- Are there insights into the different performances of BYE on the 2 models? While BYE achieves a reduction of the ASR to near 0 for LLaVA, for InternVL, the ASR is always roughly above 7 for all evaluated datasets.\n\nReferences:\\\n[1] Chen et al. Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning. Preprint 2017\\\n[2] Liu et al. Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks. ECCV 2020\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes the BYE (Believe Your Eyes) framework, an unsupervised data filtering framework focused on identifying and filtering backdoor samples. The authors specifically focus on the visual trigger-based attacks. They first sak the question - “Will attention of MLLMs systematically collapse toward the trigger rather than focusing on task-relevant content?”, which traditional Deep Learning models are known to do. The investigate by visualizing the cross-modal attention weights and determine that trigger patches indeed cause models to focus on them, a phenomenon which they term spatial-attention collapse. Based on these observations, they propose BYE which works in three main steps. First, the trained model is evaluated on the training dataset and the cross-modal attention map signals are measured for each transformer layer. In the next step, the Shannon entropy is calculated on the signals and it is observed that non-triggered images generally have lower entropy than those with triggers. The entropy distributions are clustered via Gaussian Mixture Model and two separate groups are created by filtering their distances via a threshold. Lastly, the suspicious samples that get detected through clustering are removed and the model is re-fitted on the new clean dataset to remove the influence of the trigger patches. Experiments are performed across different models and datasets, and the results show that BYE is able to filter our backdoored images with a high success rate across different types of triggers without compromising model performance.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths - \n1. The problem tackled is very interesting and important.\n2. The solution is intuitive, effective and relatively simple which makes it feasible in practice.\n3. The results are good.\n4. The paper is explained well and easy to follow.\n\nWeaknesses - \n1. Training twice is extremely expensive\n2. The triggers used are relatively simple\n3. A few ideas are highly inspired by similar problems from traditional Deep Learning systems.\n\nQUESTIONS:\n1. Please discuss more about the cost/expense of the system. It seems that it would be quite expensive. It is understood that such detection methods can be expensive, but it is important to clarify if the expense is worth it.\n2. We know from older (CNN-based) models that patched triggers with strong patterns (like the black boxes here) are easier to detect the more distinctive and consistent they are compared to the rest of the image. How sensitive is this system for diverse and surreptitious triggers? \n3. Similarly, these types of attacks rely on over-fitting, and what this solution does in essence is detect it via clustering. In the experiments 10% of the dataset was contaminated - have the authors any intuition of how trigger % contamination affects BYE's effectiveness?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a backdoor cleaning algo BYE (Alg 1) that utilizes clustering on attention entropy as a proxy to detect and filter backdoors in MLLMs. The assumption is that when exposed to poisoned samples, MLLM's attentions would be collapsed / redirected toward the triggers instead of task-relevant content, and we can often observe a bimodal in attention entropy (e.g., Fig 3). And we can utilize GMM clustering on some sensitive layers to automatically detect such entropy separation and removing low-entropy cluster samples is sufficient (Alg 1). \n\nThe experiments are conducted on LLaVA-v1.5-7B and InternVL2.5-8B with LoRA FT. The tasks are visual QA and image captioning. The metrics are Attack Success Rate (ASR) and Clean Performance (CP). The baselines are vanilla FT, random drop, and ZIP, and the authors demonstrate a decent CP can be achieved by BYE while ASR is quite close to 0.\n\nSTRENGTHS AND WEAKNESSES:\nDisclaimer: I am not familiar with the field of backdoor cleaning so I would be conservative on assessing the novelty. The following evaluation is more tailored towards the soundness of the claims from my understanding. \n\nStrength:\n\n- This paper studies an essential problem (MLLM backdoor cleaning) with a clear motivation. The paper is easy to follow even for people without backdoor cleaning background. \n- The high-level algorithmic design is well motivated and the experiments are comprehensive.\n\n\nWeakness:\n\n- High-level: \n  - **algorithmic assumption and experiment verification**: I agree that attention could give clear signal on the difference between poisoned and clean samples, but I doubt identifying from entropy score be a robust / general solution. The authors provide some justification on Figure 3 and test on 3 different triggers in Figure 1 & Table 3, but it might not be sufficient. Although I am not an expert in inserting backdoors, I could still imagine designing backdoors that are simply more diverse shouldn't be a hard task. \n\nFor example, we can construct triggers that are **both randomly scattered on the images (and there should be multiple pieces scattered on a single image, instead of trigger occurs on just 1 place per image) and have multiple randomly chosen captions for the dataset**. This is likely to be more diverse, and may degrade BYE's sensitivity. Notice that this construct is different from both random position trigger and texture trigger in terms that we have (1) multiple small triggers per image and (2) randomly chosen captions for image (the first one might be more important for diversity). \n\n\n- Low-level\n  - FT results on the clean samples only should be provided in Table 1 as an upper bound for CP and lower bound on ASR.\n  - An empirical attention entropy histogram should be provided for the other 2 triggers (Fixed Dual & Varied Multi-trigger) besides Table 3.\n\nQUESTIONS:\nThere is no other questions than the ones listed in the weakness section.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-06"}