Title: Lie Detector: Unified Backdoor Detection via Cross-Examination Framework
Venue: NeurIPS 2025
Accepted: yes
Keywords: Vision-Language Models

Backdoor Detection

Backdoor Attack
Abstract: Institutions with limited data and computing resources often outsource model training to third-party providers in a semi-honest setting, assuming adherence to prescribed training protocols with pre-defined learning paradigm (e.g., supervised or semi-supervised learning). However, this practice can introduce severe security risks, as adversaries may poison the training data to embed backdoors into the resulting model. Existing detection approaches predominantly rely on statistical analyses, which often fail to maintain universally accurate detection accuracy across different learning paradigms. To address this challenge, we propose a unified backdoor detection framework in the semi-honest setting that exploits cross-examination of model inconsistencies between two independent service providers. Specifically, we integrate central kernel alignment to enable robust feature similarity measurements across different model architectures and learning paradigms, thereby facilitating precise recovery and identification of backdoor triggers. We further introduce backdoor fine-tuning sensitivity analysis to distinguish backdoor triggers from adversarial perturbations, substantially reducing false positives. Extensive experiments demonstrate that our method achieves superior detection performance, improving accuracy by 4.4%, 1.7%, and 10.6% over SoTA baselines across supervised, self-supervised, and autoregressive learning tasks, respectively. Notably, it is the first to effectively detect backdoors in multimodal large language models, further highlighting its broad applicability and advancing secure deep learning.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a backdoor detection approach in a `semi-honest’ out-sourcing training scenario. Users trust a third party to train models by providing their data, model and protocols. After receiving trained models from different out-sourcing parties, the proposed detection method aims to find the potential backdoored model between the received ones. The method optimizes over backdoor triggers from independent training results from two service providers, and exploits criteria including CKA for model similarity and distributions to perform the backdoor detection. It is generalizable in supervised and semi-supervised learning. Essentially, this method is built upon two training losses: loss of output distribution, which is constructed based on the label or clean reference; and loss of CKA, which is measured as a similarity score for a specific intermediate layer between independent models. After required the trigger, the proposed method distinguishes the backdoor model by testing it on a poisoned dataset and a clean dataset, by measuring the ASR against a given threshold.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1.\tThe motivation of the proposed detection mechanism is clearly stated and indeed needs further research. \n\n2.\tThe proposed method is applicable to different learning paradigms including supervised learning, semi-supervised learning, and autoregressive learning. \n\n3.\tThe proposed method is able to defend against imperceptible or frequency based backdoor triggers, which outperforms previous defense mechanisms.\n\nWeaknesses:\n\n1.\tUnclear threat model: The proposed adversary is NOT a semi-honest adversary. Usually, semi-honest refers to a curious-but-honest setting, though authors refer to adversary controls the backdoor training but not the detection. This is a malicious setting. Here, the author needs to check previous works and clarify the threat model in a standard way. \n\n2.\tStrong assumptions: \na)\tIn this work, adversary is assumed to be unaware of the detection method. Authors may refer the threat model assumption in two works, TED, MM-BD, from S&P, which are both cited in this paper. They both assume the attacker has full knowledge of the defense mechanism, which is commonly assumed for a defense work. Especially, authors should NOT claim that the proposed method is robust to adaptive attacks by assuming the unawareness of detection. Please refer to existing defense works. \nb)\tWhat if two service providers collude and injects the same backdoor? In this paper, one of the service providers is malicious, who deviates from the given protocol and injects the backdoor, while the other service provider is honest and follow the protocol. But if the same or similar backdoors are implemented to two models, what will happen? This case should have been encountered in the experiments based on configuration provided in the paper, but it seems no ablation study about this issue is provided. \n\n3.\tDependency on clean dataset. The proposed detection method claims to require no access to its training data. However, the training set D_s is sampled as clean instances from D_c. \n\n4.\tExperiment settings unfairness. In the configuration, given a pool of N models, authors sample every single pair from the model pool, and perform the detection on each pair, then uses a majority vote to determine whether a model is backdoored. First, the threshold of majority vote is not given (poison rate of the pool should be unknown). Second, I reckon such evaluation criteria for the proposed detection might have deviated from the original setting that two service providers independently train two models with one of them injected with a backdoor. Because in the original setting, there are no N models to compare with. Such evaluation based on majority vote could give extra advantage to the proposed detection, which undermines the comparison fairness against other baselines. \n\n5.\tGeneralizability. The detection method is generalized to supervised learning, semi-SL and autoregressive learning. The OD loss always requires a clean reference to construct it. Without such reference, like the GT label, this method does not work or might fail in self-supervised learning that is without any clean reference.\n\nQUESTIONS:\n1.\tCan authors clarify the attacker/defender knowledge/capabilities referring to the cited previous works?\n2.\tCan authors clarify the result/consequence if the adversary is assumed to have the knowledge of detection method.\n3.\tWhat if two service providers collude and injects the same backdoor?\n4.\tPlease clarify because I'm confused by the claim that the detection requires no access to its training data, but algorithm says otherwise.\n5.\tPlease clarify if anything is incorrect in the weaknesses part.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Lie Detector, a unified backdoor detection framework that leverages cross-examination of two independently trained models in a semi-honest outsourcing setting. It introduces two novel components: (1) a cross-model trigger inversion mechanism based on output distribution and Centered Kernel Alignment (CKA) to detect representational inconsistencies and recover potential triggers, and (2) a fine-tuning sensitivity analysis to distinguish true backdoors from benign anomalies. The framework is designed to generalize across multiple learning paradigms (supervised, self-supervised, and autoregressive). It is shown to outperform existing backdoor detectors on diverse datasets and architectures, including large multimodal models like LLaVA and MiniGPT-4.\n\nSTRENGTHS AND WEAKNESSES:\nS1.  The proposed framework generalizes well across SL, SSL, and AL, addressing a key limitation of prior detectors that are often paradigm-specific.\n\nS2. The combination of CKA for feature similarity and fine-tuning sensitivity analysis is novel and effective, enabling both accurate and robust detection.\n\nS3. Extensive experiments on various models, datasets, and attack types demonstrate strong performance and practical applicability, including in the context of challenging vision-language models.\n\nW1. Threat model. This work should further clarify the available settings of the two outsourcing models. It seems that if there is only one outsourcing model, or the user would not like to pay more for other service providers, the proposed defense may not have similar performance as reported. The reviewer is also worried about the independent settings of the service providers, although some additional experiments are provided in the appendix for the collusion model. As expected, the performance in such a scenario drops quickly.\n\nW2. Novelty clarification. As stated in the manuscript, CKA and fine-tuning sensitivity have been used independently; the combination of these two techniques seems to be a direct application.\n\nW3. The reviewer wonders about the generality of the trigger generation. If attackers use different trigger generations, i.e., different equations (4), the reversing trigger may not affect the poisoned model.\n\nW4. Extra computing cost for the user. The reviewer is not convinced that if one user does not have the computing power to do the training task, the proposed trigger generation and inference process also costs a lot. Although some comparisons are provided, the missing details may overlook this weakness.\n\nQUESTIONS:\nQ1. Please clarify the reasonable setting of the two outsourcing servers, and these servers do not collude.\n\nQ2. Please provide more ablation studies or discussions on the trigger formulation with/without CKA-based loss.\n\nQ3. Please verify the generality of equation (4).\n\nQ4. Please provide more details about the computation cost.\n\nQ5. How to identify that both models are backdoored? There seems lack of such experimental results in the manuscript.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Lie Detector, a generalized backdoor detection framework that detects the presence of backdoors by performing cross-model consistency checking on two independent third-party trained models. The method combines central kernel alignment with output distribution optimized inversion triggers, and effectively reduces false positives by fine-tuning the sensitivity analysis. Lie Detector achieves, for the first time, generic backdoor detection for supervised, self-supervised, and autoregressive tasks, especially for multimodal large models. The experiments show significantly better detection capability than existing methods under multiple benchmark tasks, which has strong practical value.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Novel and reasonable verification framework. This paper introduces a backdoor detection framework based on cross-model cross-checking for the first time, which holds significant practical value in real \"semi-honest third-party training\" scenarios.\n2. Highly versatile and cross-paradigm effective. It breaks through the limitation that existing methods mainly target a single paradigm (e.g., classification) and can be applied to supervised, self-supervised, and autoregressive tasks, including multimodal large models.\n3. The method is practical and scalable. The method does not require access to training data and does not rely on a clean reference model. Only two independently trained models are needed to complete the detection, which meets the needs of real-world multi-vendor auditing applications.\n4. This method enables the detection of large models for the first time. Compared with existing methods, this method realizes effective backdoor detection on large VLMs such as LLaVA and MiniGPT-4 for the first time.\n\nWeaknesses.\n1. Hyperparameter sensitivity is not sufficiently analyzed. The trigger inversion, CKA computation, and fine-tuning sensitivity modules in the current framework involve some hyperparameters (e.g., trigger mask size, canonical strength, and number of fine-tuning steps). The paper has not systematically explored the sensitivity of these hyperparameters to the detection effect. The method steps are complex and these hyperparameters may have a serious negative impact on the performance of the detection.\nThere has been insufficient verification of generalizability across various model architectures. Although the paper was tested on mainstream architectures (e.g., ResNet, Transformer, VLM), generalizability has not yet been demonstrated on lighter or more heterogeneous architectures (e.g., MobileNet, ConvNeXt). The authors need to add more exploration of architecture.\n3. Insufficient coverage of some atypical backdoor attacks. Currently, the main attacks evaluated are trigger-based backdoors, and specific attacks such as clean-label attacks and multimodal backdoor attacks are not yet covered.\n4. Robustness under model version change is not sufficiently discussed. In practice, there may be version differences between vendors' models (e.g., different training rounds, different parameter initialization). Whether the current method is still effective when the model versions differ significantly (not completely independent but approximate) has not been analyzed in depth in the paper. Do these non-core factors have a serious impact on the detection rate of the defense when the model versions change?\n\nQUESTIONS:\nPlease see weakness.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-02"}

comments: SUMMARY:
The paper proposes a unified backdoor detection framework that exploits cross-examination
of model inconsistencies between two independent service providers. The method demonstrates good generalization and robustness, achieving high detection accuracy across supervised, semi-supervised, and autoregressive learning paradigms

STRENGTHS AND WEAKNESSES:
Strengths
- The paper is well motivated and well structured, supported by thorough experiments and illustrations.

Weaknesses
- It would be better if the paper includes explicit discussion of scalability concerns and computational overhead. Cross-examination involving multiple model trainings and trigger optimization might pose challenges in practice, particularly for large-scale models.
- Given the observed sensitivity in experiments (Figure 4), guidance or adaptive methods for parameter selection could strengthen the practical utility.

QUESTIONS:
What is the runtime of the method on miniGPT-4 (Table 6)?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02