Lie Detector: Unified Backdoor Detection via Cross-Examination Framework
Authors: Xuan Wang
Siyuan Liang
Dongping Liao
Han Fang
Aishan Liu
Xiaochun Cao
Yu-liang Lu
Ee-Chien Chang
Xitong Gao
Venue: NeurIPS 2025 poster ()
Keywords: Vision-Language Models
Backdoor Detection
Backdoor Attack
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: Institutions with limited data and computing resources often outsource model training to third-party providers in a semi-honest setting, assuming adherence to prescribed training protocols with pre-defined learning paradigm (e.g., supervised or semi-supervised learning). However, this practice can introduce severe security risks, as adversaries may poison the training data to embed backdoors into the resulting model. Existing detection approaches predominantly rely on statistical analyses, which often fail to maintain universally accurate detection accuracy across different learning paradigms. To address this challenge, we propose a unified backdoor detection framework in the semi-honest setting that exploits cross-examination of model inconsistencies between two independent service providers. Specifically, we integrate central kernel alignment to enable robust feature similarity measurements across different model architectures and learning paradigms, thereby facilitating precise recovery and identification of backdoor triggers. We further introduce backdoor fine-tuning sensitivity analysis to distinguish backdoor triggers from adversarial perturbations, substantially reducing false positives. Extensive experiments demonstrate that our method achieves superior detection performance, improving accuracy by 4.4%, 1.7%, and 10.6% over SoTA baselines across supervised, self-supervised, and autoregressive learning tasks, respectively. Notably, it is the first to effectively detect backdoors in multimodal large language models, further highlighting its broad applicability and advancing secure deep learning.

Lie Detector: Unified Backdoor Detection via
Cross-Examination Framework
Xuan Wang
Siyuan Liang" National University Singapore aanalllinnnm22mmmmmmm
National University Defense Technology
Singxuan21ddnuce edu.cr
Dongping Liao
Han Fang
Aishan Liu
State Key Lab of loTSC, CIS Dept. University of Macau yb97428@um. edu.mo mo
National University of Singapore
Beihang University
Fanghan@nus.ede edu.
liuaishan@buaa edu.
Xiaochun Cao Sun Yat-sen University caoxi aaoxioochun@mall sysu. edu.
Yuliang Lu
National University of Defense Technology publicluVL@126.cmm com
Ee-chien Chang
Xitong Gao' Shenzhen Institutes of Advanced Technology, CAS Shenzhen University of Advanced Technology xt gaotsiat ac. cn
National University of Singapore changecdcomp. nus edu.
Abstract
Institutions with limited data and computing resources often outsource model train- ing to third-party providers in an untrusted hhird-party setting, assuming adherence to prescribed training protocols with pre-defined learning paradigm super.eud self-supervised learning). However, this practice introduce severe security risks, as adversaries may poison the training data to embed backdoors into the resulting model. Existing detection approaches predominantly rely statistical analyses, which often fail to maintain universally accurate detection accuracy across different learning paradigms. To address this challenge, propose unified backdoor detection framework in the an untrusted third-party setting that exploits cross-examination of model inconsistencies between two independent vice providers. Specifically, we integrate central kernel alignment to enable robust feature similarity measurements across different model architectures and learning paradigms, thereby facilitating precise recovery and identification of backdoor triggers. We further introduce backdoor fine-tuning sensitivity analysis distin- guish backdoor triggers from adversarial perturbations, substantially reducing false positives. Extensive experiments demonstrate that our method achieves superior detection performance, improving accuracy by 4.4%. 1.7%, and 10.6% over SoTA baselines across supervised, self-supervised, and autoregressive learning tasks, respectively. Notably, it is the first to effectively detect backdoors in multimodal large language models, highlighting its broad applicability and advancing secure deep learning.
Introduction
Deep learning models have grown exponentially in size recent years, outstripping the computational resources available many small and medium-sizz institutions. Consequently, these institutions often rely on third-party cloud providers for model training. Although these providers are considered
* Correspondence Siyuan Liang, Ee-chien Chang and Xitong Gao.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Backdoor Detection
Cloud Cisomma
Datasets
Hyperpor
Model Architecture
Training Proad
Model fi
Model f2
Hard Disk
GPU
Police
Storage
Resources
Protoco
Model
Untrusted third-party
paisoning
Non-poisoning
training
craining
Providers 1 & Suspect
Providers & Suspect
Model
Model
Figure 1: In the absence of training resources, the user delegates model training a third-party vendor in an untrusted third-party environment and generates two independent models. At the same time, the user doubles as a police to identify potential backdoor models through comparative analysis.
"untrusted third-party" in that they ostensibly adhere to prescribed protocols, they may still covertly manipulate data models. This scenario can give rise a significant backdoor threat, where hidden triggers are embedded during training, enabling the model to function normally under most conditions but exhibit malicious behavior when specific triggers are activated
30 28
29 56).
Current backdoor detection methods frequently rely on model behavior and statistical analyses ge..rrdieent-bie based detection, posterior analysis) 45 35 26 12 46 47 39). Nevertheless, these methods exhibit significant sensitivity to fluctuations in optimization targets, loss functions, and feature representations across various learning paradigms [6]. This makes it hard for them to work with different architectures and attack tactics, which makes it hard to keep model security in untrusted third-party settings.
In many real-world scenarios, access to multiple independently trained models is feasible and practiced. Government, defense, and finance are examples of security-critical fields that commonly hire several outside companies to build models. This allows for cross-validation and builds trust. Modern machine learning pipelines, federted learning, AutoML, and AutoTrain often train models at the same time with various seeds or settings. These realistic workflows naturally give rise to the multi-model input setting our method leverages.
Building on this observation, we propose the Lie Detector defense via a eross-examination backdoor detection framework designed for third-pary party verification. In Figure [] the user (acting as police) gives the identical job to two different providers (the suspects) and finds backdoors by looking for differences in their model outputs (the lies). Specifically, we employ Central Kernel Align- ment (CKA) [19 for representation similarity, enabling the reverse-enginerring engineeringnnn of triggers (the evidence) by maximizing representational differences between two independently trained models. In contrast to conventional methods that rely on decision boundaries, our approach optimizes triggers based output distributions, generalizing across supervised, self-supervised. and autoregressive learning paradigms. In order to reduce false positives and improve detection robustness, we also introduce fine-tuning sensitivity analysis to distinguish between truly backdoored and benign models. With its consistent high detection accuracy across a variety of learning paradigms, Lie Detector provides useful and adaptable backdoored model verification solution.
We thoroughly assess Lie Detector's performance in supervised, self-supervised, and autoregressive settings. Our method works better than current methods, with relative increases of 4.4%, 1.7%, and 10.6% for SL, SSL, and AR, respectively. For instance, Lie Detector's 95% on COCO/Trojan VLM clearly outperforms previous techniques, which struggle lision-language models VLaVAA) with accuracies frequently below 50%. Furthermore, Lie Detector's robustness is highlighted by its great stability under variety of random seeds. In order to increase security assurances for deep learning models, we believe that this research will promote a wider usage of secure training procedures in third third-party services.
We design a unified cross-examination framework for backdoor detection by analyzing inconsistencies in models provided by multiple third-party service providers, enhancing the security of outsourced training in untrusted third-party environments. 2) Our method combines CKA for representation similarity and output distribution optimization, breaking the reliance
Our contributions are:

---

decision boundaries and enabling backdoor detection to generalize beyond supervised learning to self- supervised learning and autoregressive learning. 3) We achieve superior generalization across three learning paradigms and seven attack methods. Notably, it is the first to enable backdoor detection in multi-modal large language models, further broadening its applicability.
2 Related Work
Development of Learning Paradigms. Deep learning has evolved through training paradigms to address diverse challenges and data. This article focuses on supervised, self-supervised, and autoregressive learning, outlining their motivations, progress, and limitations. Supervised learning (SL) relies on labeled data, with early advances like CNNs (3) for image classification and DNNs for speech. Large-scale datasets (e.g., ImageNet (7] and architectures like ResNet [15] further propelled the field. However, dependence on costly annotations led to new paradigms. Self-supervised learning (SSL) mitigates this by generating supervisory signals from unlabeled data. BERT [8] revolutionized NLP, while SimCLR [2] and related methods advanced vision. Contrastive learning (CL), a subset of SSL, learns by distinguishing positive and negative pairs, with CLIP [40] and CoCoOp [53] demonstrating flcxibility across modalities. Autoregressive learning (AL) builds on SSL and CL by modeling data distributions and generating samples, enabling cross- modal understanding via models like MiniGPT-4 (54] and LLaVA (24). The evolution from SL to SSL and AL enhances adaptability and generalization, though high computational costs remain a barrier to broader adoption.
Backdoor Attack. Backdoor attacks have become a major security concern in deep learning, evolving alongside training paradigms. These attacks implant malicious behaviors during training, activated at inference by specific triggers. Early attacks focused on SL, leveraging labeled data to embed triggers. Examples include BadNets [13], which inserts poisoned data with fixed triggers; Blended [3), which blends patterns to evade detection; ISSBA (25). which uses invisible, sample-specific triggers for stealth; WaNet (38), which applies warping- based perturbations to boost success; and Low- Frequency [52). which exploits frequency features to implant hidden triggers. With the rise of SSL, attackers adapted or designed methods for unlabeled data. BadCLIP [1] compromises contrastive language-image image models, while BadEncoder [18] poisons feature encoders to affect downstream tasks. As AL advances, attacks like VLM (27) embed multimodal triggers in vision-language models, and Shadowcast [51] targets text-to-image pipelines with stealthy backdoors. The attack surface now spans SL., SSL, and AL, with techniques tailored to each paradigm. This proliferation highlights the urgent need for robust, general
general-purpose purpose defenses to safeguard models across tasks and modalities.
Existing Backdoor Detection Methods. Current methods often rely on model behavior and statistical 12
analysis, such as gradicnts, activations, or output distributions A5 35 26
46
37 55] Neural
Cleanse (NC) (45] detects anomalies via trigger reversibility, while ABS 35 uses activation clustering to isolate poisoned neurons. NAD 26 applies knowledge distillation to suppress backdoors during fine-tuning. MM-BD [46] identifies arbitrary backdoor patterns via output landscape analysis and unsupervised anomaly detection. Some methods have extended detection to SSL and AL settings: DECREE [1] reverses triggers via optimization, and SEER (55) leverages auxiliary modalities for improved detection. While promising in specific paradigms, most of these methods lack scalability and fail to generalize across diverse learning settings. Our Lie Detector distinguishes itself in two key aspects: 1) Novelty. We are the first to integrate CKA and fine-tuning sensitivity analysis for backdoor detection. While each has been used independently, their combination is purposefully designed to address both representation-lewe robustness. 2) Effect. Our method the first unified framework applicable to SL SSL, and AL. It also enables backdoor detection in large multimodal vision- language models (e.g.,LLaV, MiniGPT-4), previously untouched by existing defenses, highlighting its broad applicability.
misalignment and behavioral instability, achieving complementary
3
Preliminary
3.1
Threat Model
We consider a practical outsourced training scenario, where model training is performed by a third- party provider who may nominally follow the protocol but is essentially untrusted and may insert backdoors. The user has no visibility into the training process, but can independently inspect the returned model for malicious behavior.
Adversarial capabilities. During training or fine-tuning, the adversary. e.g., a malicious service provider (the suspects), might introduce backdoors into the model by either directly altering the
3

---

Cross-Examination Framework
Input
Mross-MMdel Trigger Reverse
Fine-tuning Sensitivity Analysis Output
Suspicious
modells
Centralizd Alignment
Output Distribution
Ostuot Secidoor Gumut
Model
fi
ducksoor
ASR
Clean images
+
Oprivized niggor
Activasion metrices
Model
Deslection
FT
clent
Backdoor
Clean
ASR
trigger
Bockdoor
Backdoor
Figure 2: Overview of Lie Detector. We propose a general backdoor detection method via a cross- examination framework. Using output distribution and CKA losses reverse triggers, and fine-tuning sensitivity analysis to identify backdoored models, our approach secures third-party training.
model's parameters to encode malicious behavior by contaminating the training data with trigger patterns
Adversarial knowledge. The model architecture and training procedure may be completely known to the adversary, but they are not aware the precise detection methods the verifier uses. This ensures that the backdoor detection framework remains robust against adaptive attacks.
Detection constraints. The verifier user (the police) cannot access the full training data training process on the provider side but may have access to a small, trusted subset of clean data. There is no assumption of clean reference model. This is consistent with real-world situations where the training process is treated as "black box" and third parties are not privy to it.
3.2 Centered Kernel Alignment
measures the similarity between activations or feature representations. To compute CKA, we input data X into models f and f and extract activations from a specific layer l. Let A₁ E R"Xpi and A E Rxxp2 denote the extracted activation matrices of the two models, where p and p2 are the dimensionalities of the representations at that layer. The activations A and A are then transformed into kernel matrices K₁ and K₂ using a kernel function, typically a linear kernel:
CKA [49
4
19
K = H(AA") H H-I- 1 11
(1)
TE
where H the centering matrix, with as the identity matrix and a vector of ones. This trans- formation ensures that the kernel matrix K E TR=x eliminates biases due to differences in model architecture. The CKA similarity between the feature representations of two models defined as:
K2)
CKA(F) (.
(2)
VIKK, |
where ( is the Frobenius inner product, and I1- the squared Frobenius norm, with (K,, K2) Tr(K] (2) and || = (K,, K,) F. where Tr(-) the matrix trace.
CKA is architecture independent because it remains unchanged under certain transformations 4 19,. meaning architectural differences do not affect model similarity. Specifically: 1) Orthogonal transformation invariance. CKA is unaffected by rotations or reflections of the feature space, making robust to different basis representations. 2) Isotropic scaling invariance. Uniform scaling of features does not affect CKA, ensuring comparisons are unbiased by activation magnitudes. These properties make CKA ideal for comparing models with different architectures, as captures the relative structure of representations rather than absolute values or network specifics.
4 Method
We introduce the Lie Detector via the cross-cxamination framework (Figure 2p. Section 4.1 outlines the framework, while Section 4 4.4 detail the method and Algorithm presents the full procedure.
4.1
Cross-Exee Cross-Examination Framework
To secure third-pane machine learning models, we propose a Cross-Examination. Based Backdoor Delection Framework under an untrusled third-party verification setting, comprising three modules:
Cloud customers. It consists of users who require model training services but lack direct control over the training process. These users also act the verification party (the police), who have the

---

Algorithm Lie Detecor
Require: Models ft. Jz: clean subset D. finetune set Dn; thresholds y, ys weights a, 9. A; epochs T, Adam Ensure: Backdoor status of fi and fz
1: Initialize trigger mask m and pattern
> Stage I: Cross-Model Trigger Reverse
2: for = to do
3: for all (). D. do
4: x' + m 6 p +1 m) x
P Generate poisoned input (Eq.
5: Compute LOD, LOKA
Eq.
Eq.
6:
end for
7: C a. LCKA S Lon + A (/|m]| [|P]]a)
Total loss (Eq.
8:
Update the trigger (m, p) by minimizing & Adam
9:
end for
10: Do + {x'
④ (1-m) 6x x D.}
> Build poisoned set
11: for f [fi, f2) do
12:
Predict V1 = fxx),, estimate target ÿe averaging
Stage II: Activation- Basiid Identification
13:
Compute ASR(f) = [[[(f(x') = y-)]
14:
if (S)()) > then
B Stage III: Fine-tuning Sensitivity Analysis
15:
Fine-tune on Dn; compute ASR(J"): set AASR <- (SR()))SR((")
16:
return Backdoored if ÁASR else return Clean
else
18:
return Clean
19:
end
20:
end for
authority to verify model integrity. The users provide the clean dataset De, training hyperparameters, model architecture f. and the learning paradigm Licam.
an untrusted third-party providers. They are independent service providers (the suspects) responsible for model training. While following the user-specingg specified protocol, they may still embed backdoors by poisoning part of the training data. Specifically. we assume the suspect constructs poisoned training set by modifying a subset D, D. into D,,. resulting in D = (D, D,) UDP.. The poisoned set Dp contains a [Dz] samples, where a [0. 1] the poison rate. The model fo is then trained on 'D. The adversary's learning process can be formulated as an optimization problem:
arg min [farn D) & (1 Bx) ((fo(()))) ya)] + -EE(
[((f((x)).
where is the ground-trutt truth label of a clean sample x., and ÿe the adversarial target for poisoned sample Xp. The objective Lream depends on the paradigm, with loss function defined as: SL, y is class label and f is cross-entropy; in CL (e.g., CL..., y encodes similarity and É is contrastive; in AL (e.g., LLLV),, y a reconstruction target and autoregressive or reconstruction loss.
Cross-examination backdoor detection. The goal of cross- examinaxainotionn backdoor detection is to verify whether model has been compromised without requiring access to its training data process. Rather than relying on a known clean reference or predefined attack patterns, our approach detects backdoors by exploiting inconsistencies between two independently trained models (f1 and f2) from different third- parttpartyy providers. Under this framework, there are three possible outcomes: both models are clean, both models are backdoored, or one model is clean while the other is backdoored.
Next, we outline the key challenges and motivations of our framework.
Challenges. Backdoor detection faces two key challenges: 1) Accuracy: Many methods rely on statistical analysis and assume access to a clean reference model or known attack patterns. These as- sumptions often fail under unknown or adaptive attacks, as mismatched priors cause errors. Statistical methods also struggle to generalize beyond known attacks, limiting reliability. 2) Generalization: A practical framework must generalize across architectures and learning paradigms, not just classifica- tion. Methods tied to specific models or objectives often fail in complex settings like self-supervised generative learning. Architectural and task-agnostic generalization is essential for deployment.
Motivations. Our framework tackles these challenges with two key innovations: 1) Exploiting model inconsistencies to avoid predefined attack priors. Traditional methods rely on assumptions about trigger or poison distributions, making them vulnerable novel or adaptive attacks. We bypass this by comparing independently trained models the same dataset, enabling detection without prior
5

---

attack knowledge. 2) Leveraging invariant features for better generalization. Existing defenses often depend on specific architectures or tasks. In contrast, we target structural inconsistencies shared across models and paradigms, enabling broader applicability beyond classification.
4.2 Cross-Model Trigger Reverse
To identify potential backdoors, we reverse-engineer an effective trigger by exploiting behavioral discrepancies between two independently trained models fi and f2. This stage serves as an initial screening for suspicious models. We randomly sample subset D3 De of 1000 clean instances.
Trigger formulation. We adopt a trigger parameterization unified across various learning paradigms. Inspired by prior work on universal backdoor patterns (55) we define the trigger as trainable pat- tern-mask pair. Let p E R xC be the injected pattern and m E [0, 1 || W XC' the corresponding mask indicating modified pixels. For clean input x, the poisoned input is defined as:
x' m @ p (1 m) O x,
(4)
where @ is the element-wis product. By optimizing m and p. we reconstruct effective triggers that are capable of eliciting malicious behavior in the suspect model.
Output Distribution Loss. To identify a backdoor trigger (evidence), optimize the pair (m, P)
defined in Eq. (4) so that the poisoned input x' induces abnormal behavior in the model. Backdoored models are typically trained to produce confident, target predictions that deviate from the ground-truth label, whereas clean models remain stable and preserve correct outputs under such perturbations. Minimizing the output distribution loss LOD over clean samples guides the discovery of trigger patterns that accentuate this behavioral gap. The output distribution loss is defined as:
(f
if SL.
COD
Σ
(x()). (wk)), E, (x)()(x)))) y, ()), AI.
SSL,
(5)
Da
For SL, lcE is the cross-entropy loss w.r.t. the ground-trute truth label y4 minimizing its negative encourages confident misclassification. For SSL, Esim penalizes similarity between poisoned and clean features, with lower values indicating semantic drift. For AL. lA measures generation error against the ground-truth y;; minimizing its negative reveals sequence-level deviations.
CKA-based loss. To capture internal discrepancies caused by backdoors, we leverage CKA as a representation--eva metric. Unlike sutput-based measures, CKA quantifies alignment between intermediate feature spaces, making it suitable for comparing models across different architectures or objectives. By maximizing this divergence, we highlight the Lie hidden within a suspect model. Let K K! denote kernel matrices from activations at layer I of models fi and f2 given input x'. The CKA loss is defined as:
(x)), (x())r
COKA(x f2,11) V K (x') ) (x))
(6)
Joint objective. The full optimization combines the output-level and representation-level signals: L(m, p) = a LeKa + 3 LoD + ([]m]] !|Il),),
(7)
where the |I regularization promotes sparsity and limits perturbation magnitude, thereby aiding trigger optimization, following the design principles of DECREE (12).
4.3 Activation-Based Identification via Trigger-Induced ASR
We introduce two post-trigger criteria to determine whether a model is backdoored. The first criterion evaluates the model's behavior under trigger injection, serving as fast and effective filter.
The motivation lies in the fact that backdoored models are explicitly trained to associate specific trigger patterns with an atffeeer-defined target label, resulting in consistent and confident mispredictions when the trigger is applicd. In contrast, clean models lack such associations and typically retain correct predictions under the same perturbations. Given the poisoned set D, {xx) constructed from clean inputs D. using Eq. 4 we compute the prediction ÿ = f fx() for each model f. and estimate the target label ûe by averaging over all ÿ.. The attack success rate (ASR) is then defined as:
ASR(J) Exeet, v, [I([(x') = Øc)]
(8)
model is flagged as backdoored if:
Backdoored > ASR(F) > n
(9)
This criterion offers a straightforward and effective filter for strongly triggered behaviors, but may suffer from false positives in certain edge cases.

---

4.4 Fine-tuning Sensitivity Analysis for Robust Backdoor Confirmation
To mitigate the limitations of Criterion we propose a robustness- based test: fine-tune the model on small clean subset Da C De (e.g., 10%) using its original objective:
arg Lleam (f, Dn)-
(10)
The motivation is that clean models naturally generalize well to clean data, and their predictions remain stable under further fine-tuning. In contrast, backdoored models embed spurious dependencies
on trigger patterns; fine-tuning on clean samples disrupts these dependencies and deactivates the
backdoor effect. We then recompute the ASR on the fine-tunde tun-ttned model f':
AfR(F" = ExYED, (x') = =
(11)
A significant drop in ASR indicates disrupted backdoor and confirms the malicious dependency:
Backdoored =
ASRR ASR(f) ASR(f')>
(12)
Together, these two criteria form dual-phase verification pipeline: the first captures activation, and the second verifies its robustness. Criterion II also represents a key innovation of our framework, enabling accurate identification with reduced false positives.
Experiments
5.1 Implementation Details
Models and Datasets. We evaluate across diverse learning paradigms. For supervised learning, we
use ResNet18 [I5) and VGG16 [42 on CIFAR-10 [21 and TinyImageNet [44]. For self-supervised
and autoregressive learning, we test CLIP 40) and CoCoOp on ImageNet 7 and Caltech101 [II].
while LLaVA (24] and MiniGPT-4
54 are evaluated on COCO 32 Frisk-30k [T], and Frisk-
8k [16].
Attacks and Defenses. We consider backdoor attacks across different paradigms, including Bad- Nets [13, Blended B], ISSBA (25], WaNet [38), and Low- now-Frequency [52] for supervised learning. For selfvsuue and autoregressive learning, we adapt these attacks and further evaluate Bad- CLIP 31). BadEncoder [8), TrojanVL. VLM (27), and Shadowcast [51]. We employ advanced defenses,
45 ABS (35) NAD 26). UNICORN 48 MM-BI) 446. DECREE 112, and SEER [55] Some methods, such as MM-BD, are extended to multiple paradigms. Unless otherwise specified, all attacks use a 10% poison rate. All evaluations are conducted using the untrusted third-party environment, with detailed settings and evaluation metrics in Appendices B. and B.2
including NC
Configurations. In experiments, each evaluation, two models are randomly sampled to form clean,clean, clean- back-oobcdoordd, or backdoored baccroorddd pairs. To ensure fair comparison, randomly select 20 model pairs (without repetition) for testing and compute the detection performance by averaging their scores. The trigger is optimized with Adam. We use default hyperparameters for Algorithm = 0.755 o 0=0.6, 0.6, 3 0.3, 0.1, T= 100.
equal numbers of clean and backdoored models. In
Evaluation Assumption. All comparisons between suspicious models and defense methods in this work are conducted under the assumption that the evaluator (or defender) has access to at least one clean model, which serves as reference for comparison and judgment. This is consistent with the settings adopted in prior studies [50 37). In cases where this assumption does not hold, the behaviors and threshold choices of many defense methods may differ substantially in fully black-box scenario without any clean baseline.
5.2 Detection Performance in SL
In Table we compare our Lie Detector with SOTA detection methods against representative attacks on CIFAR-I0 and TinyImageNet. Lie Detector consistently achieves 100.0% % DSR across all attacks and datasets, with average gains of 4.4% over the next-best methods and near-zero FPR. In contrast, others often fail under adaptive or stealthy attacks, particularly ISSBA and Low- Frequency. On CIFAR-10, 10, Lie Detector surpasses the next-best method by 5.0% on ISSBA and 7.5% on Low-Frequency. On TinyImageNe now-Frequency. These gains come with consistently lower FPR (0.0-5.0%) compared to higher rates from other methods. While recent defenses like UNICORN, MM-BD, and DECRE improve over early detectors NC, ABS, NAD), they still lack robustness under challenging attacks. Notably, while both NC and Lie Detector perform trigger reversal in the spatial domain, our method introduces
similar margins appear: 5.0 % on Blended and 7.5% on
7

---

Table 1: Detection performance (%) SL. (ResNet-18). For each attack, we evaluate 20 clean and
20 backdoored models. Detection Success Rate (DSR) and False Positive Rate (FPR) are reported.
Dataset
Attack
NC [5
ABS
35]
NAD
26
UNICORN [8]
MM-BD
[6
DECREE [2]
Lie Detector
DSR
FPR
DSR
FPR
DSR
FPR
DSSR
FPR
DSR
FPR
DSR
FPR
DSR
FPR
BadNet
87.5
10.0
90.0
5.0
92.5
15.0
100.0
5.0
100.0
0.0
97.5
0.0
100.0
0.0
CIFARIO
Blended
30.0
25.0
80.0
15.0
67.5
10.0
92.5
5.0
97.5
0.0
92.5
5.0
100.0
0.0
ISSBA
25.0
30.0
17.5
40.0
50.0
20.0
90,0
5.0
92.5
5.0
90.0
5.0
100.0
0.0
Low-Freq.
55.0
25.0
25.0
5.0
87.5
100.0
0.0
BadNet
77.5
10.0
80.0
10.0
82.5
20.0
90.0
5.0
95.0
0.0
97.5
0.9
100.0
0.0
TinyImgNer
Blended
30.0
10.0
100.0
0.0
ISSBA
10.0
40.0
25.0
25.0
40.0
25.0
85.0
10.0
95.0
5.0
92.5
10.0
97.5
5.0
Low-Fieg.
5.0
50.0
12.5
45,0
30.0
30,0
45.0
35.0
92.5
5.0
90.0
5.0
100.0
5.0
Table 2: Detection performance (%) on SSL (CLIP) and AL (L)aVA). We evaluate 20 clean and 20
backdoored models per attack. DSR and FPR are reported.
MM-BD [46]
DECREE [2
SEER
55
Lie Detector
Architecture
Dataset
Attack
DSR
FPR
DSR
FPR
DSR
FPR
DSR
FPR
BadNet
75.0
10.0
87.5
20.0
100.0
0.0
100.0
0.0
Caltech101
Blended
72.5
20.0
82.5
25.0
97.5
0.0
97.5
0.0
BadCLIP
52.5
25,0
60.0
30.0
90.0
0.0
95.0
5.0
CLIP
BadNet
67.5
10.0
72.5
10.0
95.0
0.0
95.0
0.0
ImageNet
Blended
65.0
15.0
75.0
15.0
90.0
5.0
92.5
0.0
BadCLIP
42.5
30.0
45.0
20.0
87.5
10.0
90.0
5.0
TLM
15.0
45.0
60.0
40.0
80.0
15.0
95.0
5,0
COCO
Shadoweast
15.0
50.0
60.0
45.0
85.0
10.0
92.5
0.0
LLaVA
TrOjanVLM
15,0
50.0
55,0
45.0
80.0
5.0
90,0
10.0
Flickr-30K
Shadowcast
10.0
45.0
45,0
35.0
80.0
10.0
90.0
5.0
CKA to measure representational misalignment between models. This enables detection of semantic inconsistencies even when trigger patterns smooth or imperceptible, as in Low-Frequenn attacks. Even UNICORN, which considers feature- and frequency-space triggers, underperforms in such cases, highlighting the strength of CKA-driven alignment in exposing cross-model backdoor discrepancies.
5.3 Detection Performance in SSL and AL
We evaluate our method against 4 defenses under SSL and AL paradigms, with default implemen- tations and modest modifications, across datasets and 3 attacks. Table Tablee]concludss 1) Limited generalization. Traditional methods (MM-BD, DECREE) show inconsistent performance across datasets and architectures. While some perform well on CLIP (e.g.. DECREE: 87.5% DSR on Caltech101 101//0d///// they fail on vision-language models (e.g..LLaVA), with DSRs often below 50% (e.g., MM-BD: 15%, DECREE: 60% on COCO/Trojan
indicating weak adaptability. 2) High FPR. Many methods, particularly MM-BD, exhibit FPRs up to 50%, misclassifying clean models as backdoored with detection rates no better than random guessing. In contrast, our method shows better generalization, maintaining high DSR 95.0% % on COCO/Trojan VLM, 90.0% on Flickr/Shadoweas]] and low FPR (0.0.0% ), with average gains of 1.7 for SSL and 10.6 % for AR over the next-best methods.
5.4 Adaptive Attack on Lie Detector
To further validate the robustness of the proposed method, we conduct adaptive attack experiments, which simulate adaptive attackers by modifying the training process of the backdoor model.
We design adaptive attacker that is aware of detection and aims bypass our CKA-based representation divergence detector. In particular, the attacker first trains a clean reference model felenn on the clean training dataset. Then, when training the adaptive backdoored model on the poisoned training dataset, the attacker combines the original training loss Lorigin with regularization term designed to suppress detection. This regularizer maximizes the CKA similarity between the clean and backdoored models:
(13) LCKa denotes the CKA similarity between intermediate representations of the two models under the same input as used in our paper, and À controls the strength. We vary and report the attacker's 's ASR as well our defense's 's DSR and FPR in the table below.
Cadaptive Longinn A LCKA Setean)
In Table 3, as X increases the attacker can partly suppress cross-model representation divergence, causing DSR to drop from 99.61% to 86.54%; ASR also falls from 98.75% to 82.37%, indicating
8

---

Table Performance under adaptive attacks with varying X.
A)R(%)
DSR(%)
FPR( %)
0 (No adaptive)
98.75
100,0
0.0
0.1
94.90
92.5
0.0
0.2
90.68
90.0
5.0
82.37
85.0
10.0
Table 4: Component ablation experiments. Centered Kernel Alignment loss)
(FTSA = Fine-tuning Sensitivity Analysis; CKA
wo/ FTSA
wo/ CKA
Lie Detector (Ours)
Attack
Task
Trigger Size
Model
DSR
FPR
DSR
FPR
DSR
FPR
ResNet-18
100.0
10.0
85.0
7.5
100.0
0.0
Blended
CIFARIO
4x4
VGG16
100.0
20.0
82.5
10.0
100.0
0.0
CLIP
100.0
20.0
80.0
10.0
95.0
2.5
BadEncoder
Caltech101
32x32
CoCoOp
100.0
20.0
82.5
5.0
100.0
0.0
LLaVA
90.0
20.0
77.5
10.0
92.5
2.5
Shadowcast
Flickr8k
50x50
Mini-GPT4
90.0
30.0
75.0
7.5
90.0
0.0
weakened backdoor. This demonstrates a trade-off between stealth and efficacy, even under adaptive attacks our method raises the difficulty and cost of stealthy backdoor injection.
5.5 Discussion
Component ablation. To validate the effectiveness Lie Detector under different configurations. Specifically, with or without "Fine-tuning Sensitivity Analysis (FTSA)" and the "Centered Kernel Alignment (CKA) loss", we conduct component ablation experiments Table [ The results lead to Trigger Reverse" method combined with basic "Activation- Based Identification" is effective but less robust, while achieving high Detection Success Rates (DSR), it suffers from elevated False Positive Rates (FPR), reaching up to 30% in some cases, indicating that some clean models are misclassified as backdoored; (2) introducing "Fine-tuning Sensitivity Analysis" significantly enhances robustness, reducing FPR to 0% across all settings while maintaining equally high DSR, effectively distinguishing backdoored models from clean ones; and (3) removing the CKA loss term (Eq. (7)) leads to noticeable degradation in performance. As shown in A5, eliminating CKA results in clear drops in DSR (e.g.. 100.0 - 85.0 on LIFAR-10/Blende FPR (e.g., 0.0 - 7.5), confirming that the CKA loss provides complementary representation-les supervision during trigger optimization. This is particularly crucial for complex architectures (e.g.. CLIP, LLaVA), where relying solely on output distributions may fail to sufficiently expose backdoor inconsistencies, while CKA effectively compensates for this limitation.
three conclusions: (1) the "Cross-Moode
and increases in
Number of epochs T. We show the detection accuracies (DSR) the number of epochs increases for ResNet-18 18 and CLIP models in Figure 3 We observe that our method converges stably and remains effective across all attack methods on both ResNet- 18 and CLIP. The DSR consistently improves with training epochs, demonstrating the robustness and adaptability of our approach in identifying backdoored models across learning paradigms.
Feature layer selection. As shown in Table 6 CKA values in clean models remain stable across layers, while backdoored models show clear decline in deeper layers. This trend holds for both ResNet-18 (SL.) and CLIP (SSL), confirming CKA's reliability as backdoor probe. Notably, layer 4 yields the largest CKA drop (0.427 in ResNet-18, 0.314 in CLIP), making it the most effective for detection. A likely reason is that decper-layer features capture more abstract semantics, which backdoor triggers distort, leading to greater representation shifts. We therefore fourth-layer features to compute the CKA loss. Further analysis CKA under varying poison rates and across clean/backdoored models is provided in Appendix A
Model architecture. We evaluate our detection method across six model architectures spanning three learning paradigms, shown in Table Specifically, we assess supervised models (ResNet- 18, VGG16), contrastive language-image models (CLIP, CoCoOp), and vision-language models (LLaVA. Mini-GPT4) under three attacks. Results show: (1) Our method achieves 100% DSR and 0% FPR across all architectures, including complex multimodal models like LLaVA and Mini-GPT4. (2) Detection remains unaffected by model complexity, measured by FLOPs. For instance, despite the

---

Table 6: CKA value variation across layers.
ResNet-18
CLIP
Layer
Clean
Backdoor
Clean
Backdoor
0.974
0,945
0.891
0,936
0.768
0.853
0.632
0.901
0.542
0.810
0.497
(a) RosNet-18
(b) CLIP
0.872
0.427
0.795
0.314
Figure 3: DSR as the number of epochs T.
Table 7: Performance across model architectures.
100
Badhet
Attack
Task
Trigger Size
Model
DSR
FPR GFLOPs
DSR OSR (3))
75
Blended
BadNet
CIFARIO
4x4
ResNet-18
100.0
0.0
0.7
Lowfrequency
VGG16
100.0
0.0
0.4
25
BadCLIP
Caltech101
32x32
CoCoOp
CLIP
100.0
90.0
0.0
0.0
4.9
5.0
0.0 0.2 0.3 0.8
TrojanVLM Flickr8k
50x50
aA
90.0
0.0
76.6
Figure 4: DSR of various y.
Mini-GPT4
80.0
0.0
80.3
increase in computational cost from ResNet-18 (0.7 GFLOPs) to Mini-GPT4 (80.3 GFLOPs), our method maintains perfect detection with zero false positives.
Hyperparameter 7. As shown in Figure 4 evaluate the impact of ? on detection accuracy across multiple backdoor attacks. When y = 0. the framework relies solely on the activation bassed criterion and achieves high detection accuracy. However, as y increases beyond 0.2, performance drops sharply. At = 0.6, all attacks converge to 0% detection accuracy, indicating that overly strict reliance on the ASR-drop criterion may suppress correct detections.
Impact of similarity metric and poison rate, We evaluate four metrics (CKA, CCA, SVCCA, and COS 22 under five poison rates (0.1%, 1 1%, 5%, 10%, 20%) against the Blended attack. CKA consistently outperforms others across learning paradigms, especially on LLaVA, reaching an FI score of 0.92 10% poison rate, and 0.55 (COS), 0.50 (CCA), and 0.47 (SVCCA), demonstrating robustness to subtle backdoors and model-agnostic performance. While detection improves with higher poison rates, traditional metrics degrade under low poisoning; for example, at 1% on CLIP, CKA achieves 0.50 while others remain below 0.3. Full results are in Appendix
Comparison with existing methods. Appendix shows that Lic Detector offers lower cost than NAD, and works without label supervision, relying only on clean data. It supports SL, SSL, and AL, unlike DECRE (pre-training only) and others limited to classification. While it requires two independently trained models, this is practical in many real-world settings. Overall, it achieves the highest detection success rate (99.7%), outperforming all baselines.
Potential advanced scenarios. We outline three advanced cases (details in Appendix E 1) Collusion Attacks. Multiple providers may collude to train aligned backdoored models, concealing discrepancies and bypassing cross-man model detection. 2) Client-level Inspection in FL. In a simplified FL setup, clients train separately and submit models for centralized auditing, akin to detecting poisoned updates before aggregation. 3) Scaling to Larger Models. Extending our approach to large-scale architectures.
6 Conclusion
This paper proposes Lie Detector, a unified backdoor detection framework for untrusted third-party settings where model training outsourced to third-party providers. By leveraging cross-examination of inconsistencies between independent providers, our method significantly improves detection robustness across learning paradigms. We integrate Centered Kernel Alignment (CKA) for precise feature similarity measurement and fine-tuning sensitivity analysis to distinguish backdoor triggers from adversarial perturbations, effectively reducing false positives. Extensive experiments show that our approach outperforms state-of of-the-art methods, achieving superior detection accuracy in supervised, contrastive, and autoregressive tasks. Notably, it is the first to effectively detect backdoors in multimodal large language models. This work offers practical solution to mitigate backdoor risks outsourced training, paving the way for more secure and trustworthy systems. We also provide the limitations of our work in Appendix[] discussing generality and scalability future directions.
10

---

Acknowledgments
This work is supported in part by Yu Liang Lu's Project Team Development Funding (KY23A102), National Natural Science Foundation of China (62376263), Natural Science Foundation of Guangdong (2024A and Shenzhen Science and Technology Innovation Commission (CCY1202338001440000015))
References
[1] S. A. Alvarez. Gaussian rbf centered kernel alignment (cka) in the large-bandwidth limit. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45($):6587-6593 2022.
[2] Chen, S. Korbblith, M. Norouzi, and G. Hinton. simple framework for contrastive learning of visual representations. In Proc. Int'l Conf. Machine Learning. pages 1597-1607, 2020.
[3] X. Chen, C. Liu, B. Li, Lu, and Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05522 2017.
[4] L. Ciemik, Linhardt, M. Morik, Dippel. S. Korbblith, and L. Muttenthaler. Training objective drives the consistency of representational similarity across datasets. arXiv preprint (. 2024.
[5] C. Cortes, M. Mohri, and A. Rostamizadeh. Algorithms for learning kernels based on centered alignment. The Journal of Muchine Learning Research, 13(1):795-825 828, 2012.
[6] S. Dargan, M. Kumar, M. R. Ayyagari, and G. Kumar. A survey of deep learning and its applications: new paradigm machine learning. Archives of computational methods in engineering, 27:1071-1092 2020.
[7] J. Deng, W. Dong. R. Socher, L. Li, and Fei-Fei. Imagenet: A large-scale hicrarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255. 2009,
[8] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In J. Burstein, C. Doran, and T. Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Pa- pers), pages 4171-41888 4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
[9] Z. Du, Y. Qian, Liu, M. Ding, J. Qiu, Yang, and Tang. Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pages 320-335, 2022.
[10] H. Fang, S. Gupta, F. landola, R. K. Srivastava, L. Deng, P. Dollar, J. Gao, X. He, M. Mitchell, J. C. Platt, C. Lawrence Zitnick, and Zweig. From captions visual concepts and back. In Proc. IEEE Int't I Conf. Computer Vision and Pattern Recognition, June 2015.
[11] L. Fei-Fei, Fergus, and P. Perona. Learning generative visual models from few training ex- amples: An incremental bayesian approach tested on 101 object categories. In 2004 Conference on Computer Vision and Pattern Recognition Workshop, pages 178,178, 2004.
[12] S. Feng, G. Tao, S. Cheng, G. Shen, X. Xu, Y. Liu, K. Zhang, S. Ma, and X. Zhang, Detecting backdoors in pre-trained encoders. Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pages 16352-16362, 2023.
[13] Gu, Dolan-Gavitt, and Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.066 2017.
[14] D. R. Hardoon, S. Szedmak, and Shawe- Shawe-Taylor. Canonical correlation analysis: An overview with application to learning methods. Neural computation, (6(12):2639-266 2004.
11

---

[15] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, June 2016.
[16) M. Hodosh, P. Young, and J. Hockenmaier. Framing image description as ranking task: Data, models and evaluation metrics. Journal of Artificial Intelligence Research, pages 853-899, 2013.
[17] H. Hotelling. Relations between two sets variates. In Breakthroughs in statistics: methodology and distribution, pages 162-190. Springer, 1992.
[18] J. Jia, Y. Liu, and N. 7. Gong. Badencoder: Backdoor attacks to pre-trained encoders in self- supervised learning. In 2022 IEEE Symposium on Security and Privacy (SP), pages 2043-2059. IEFF, 2022.
[19] S. Kornblith, M. Norouzi, H. Lee, and G. Hinton. Similarity of neural network representations revisited. Statistics, 2019.
[20] S. Kornblith, M. Norouzi, H. Lee, and G. Hinton. Similarity of neural network representations revisited. In Proc. Int'l Conf. Machine Learning, pages 3519-3529, PMLR, 2019.
[21] H. G. Krizhevsky A. Learning multiple layers of features from tiny images. 2009.
[22] A. R. Lahitani, A. E. Permanasari, and N. A. Setiawan. Cosine similarity to determine similarity measure: Study case online essay assessment. In 2016 4th International conference on cyber and IT service management, pages 1-6. IEEE, 2016.
[23] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient- learning applied to document recognition. Proceedings of the IEEE, pages 2278 2324,, 1998.
[24] C. Li, C. Wong, S. Zhang, N. Usuyama, H. Liu, J. Yang, T. Naumann, H. Poon, and J. Gao. Llava-med: Training a large kunguage-and-vision assistant for biomedicine in one day. In Annual Conf. Neural Information Processing Systems, 2023.
Proc.
[25] Y. Li, Y. Li, B. Wu, L. Li, R. He, and S. Lyu. Invisible backdoor attack with sample-specific triggers. In Proc. IEEE Int'l Conf. Computer Vision, pages 16443-16452, 2021.
[26] Y. Li, X. Lyu, N. Koren, L. Lyu, B. Li, and X. Ma. Neural attention distillation: Erasing backdoor triggers from deep neural networks. arXiv preprint arXiv:2101.05930 2021.
[27] Liang, S. Liang, A. Liu, and Cao. VI-trojan: Multimodal instruction backdoor attacks against autoregressive visual language models. International Journal of Computer Vision, pages 1-20, 2025.
[28] J. Liang, S. Liang, A. Liu, X. Jia, J. Kuang, and X. Cao. Poisoned forgery face: Towards backdoor attacks on face forgery detection. arXiv preprint arXiv:2402.11473. 2024.
[29] S. Liang, J. Liang, T. Pang, C. Du, A. Liu, E.-C. Chang, and X. Cao. Revisiting backdoor attacks against large vision- languaee models. arXiv preprint arXiv:2406.18844 2024.
[30] S. Liang. M. Zhu, A. Liu, B. Wu, X. Cao, and E.-C Chang. Badclip: Dual-embedding guided backdoor attack on multimodal contrastive learning. arXiv preprint arXiv:2311. 2023.
[31] S. Liang. M. Zhu, A. Liu, B. Wu, X. Cao, and E.-C Chang. Badclip: Dual-embedding guided backdoor attack on multimodal contrastive learning. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pages 24645-24654 June 2024.
[32] T.-Y. Lin, M. Maire, S. Belongie, J. Hays. P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft coco: Common objects in context. In Fleet, Pajdla, B. Schiele, and T. Tuytelaars, editors, Proc. IEEE European Conf. Computer Vision, pages 740-755, 2014.
[33] A. Liu, X. Zhang, Y. Xiao, Zhou, S. Liang, J. Wang, X. Liu, X. Cao, and D. Tao. Pre-trained trojan attacks for visual recognition. arXiv preprint 2312.22312.1517. 2023.
12

---

[34] X. Liu, S. Liang, M. Han, Y. Luo. A. Liu, X. Cai, Z. He, and D. Tao. Elba-bench: An efficient learning backdoor attacks benchmark for large language models. arXiv preprint arXiv:2502. 1811, 2025.
[35] Y. Liu, S. Ma, W.-C. Lee, Aafer, G. Tao, and X. Zhang. Abs: Scanning neural networks for back-doors by artificial brain stimulation. In CCS '1: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, 2019.
[36] W. I. D. Mining. Introduction to data mining. Mining Multimedia Databases, Mining Time Series and, 2006.
[37] X. Mo, Y. Zhang, L.. Y. Zhang, W. Luo, N. Sun, S. Hu, S. Gao, and Y. Xiang. Robust backdoor detection for deep learning via topological evolution dynamics. In 2024 IEEE Symposium on Security and Privacy (SP), 2024.
[38] T. A. Nguyen and A. T. Tran. Wanct-imperceptibee warping-basee backdoor attack. Proc. Int' I Conf. Learning Representations, 2021.
[39] T. Qin, X. Wang. J. Zhao, K. Ye, C.-z. Xu, and X. Gao. On the adversarial robustness of visual- language chat models. In Proceedings of the 2025 International Conference Multimedia Retrieval, ICMR '25, page 1118-1127, New York, NY, USA, 2025, Association for Computing Machinery.
[40] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever. Learning transferable visual models from natural language supervision. In Proc. Iut'l Conf. Machine Learning, pages 7748-8763 2021.
[41] M. Raghu, J. Gilmer, J. Yosinski, and J. Sohl- Dickstencs Sveea: Singular vector canonical corre- lation analysis for deep learning dynamics and interpretability. In Proc. Annual Conf. Neural Information Processing Systems, volume 30, 2017.
[42] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. 2014.
[43] A. Singhal et al. Modern information retrieval: A brief overview. IEEE Data Eng. Bull., 24(4):35-43, 2001.
[44] O. Vinyals, C. Blundell, T. Lillicrap, k. kavukcuoglu, and D. Wierstra. Matching networks for one shot learning. In Proc. Annual Conf. Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016.
[45] B. Wang, Y. Yao, S. Shan, H. Li, B. Viswanath, H. Zheng, and B. Y. Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE Symposium on Security and Privacy (SP), 2019.
[46] H. Wang, Z. Xiang, D. J. Miller, and G. Kesidis. Mm-bd: Post-training detection of backdoor attacks with arbitrary backdoor pattern types using a maximum margin statistic. In 2024 IEEE Symposium on Security and Privacy (SP), 2024.
[47] X. Wang, X. Gao, D. Liao, T. Qin, Y.-L. Lu. and C.-z. Xu. A3: Few-shot prompt learning of unlearnalle examples with cross-modal adversarial feature alignment. In Proceedings of the IFEF/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 9507-9516, 16, June 2025.
[48] Z. Wang, K. Mei, J. Zhai, and S. Ma. Unicorn: unified backdoor trigger inversion framework. In Proc. Int'l Conf. Learning Representations, 2023.
[49] Z. Wang, Z. Zhang, S. Liang, and X. Wang. Diversifying the high-level features for better adversarial transferability. arXiv preprint arXiv:2304. 10/36, 2023.
[50] X. Xu, Q Wang, H. Li, Borisov, C. A. Gunter, and B. Li. Detecting ai trojans using meta neural analysis, 2020.
13

---

[51] Y. Xu, J. Yao. M. Shu, Y. Sun, Z. Wu, N. Yu, T. Goldstein, and F. Huang. Shadoweast: Stealthy data poisoning attacks against vision-language models. In Proc. Anmual Conf. Neural Information Processing Systems, 2024.
[52] Y. Zeng, W. Park, Z. M. Mao, and R. Jia. Rethinking the backdoor attacks' triggers: A frequency perspective. In Proc. IEEE Int'l Conf. Computer Vision, pages 16473-16481, October 2021.
[53] K. Zhou, J. Yang, C. C. Loy, and Z. Liu. Conditional prompt learning for vision-language models. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pages 16816-16825, June 2022.
[54] D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint urXiv:2304.105/0 2023.
[55] I.. Zhu, R. Ning, J. Li, Xin, and Wu. Seer: Backdoor detection for vision-language models through searching target text and image trigger jointly. In Proc. AAAI Conf. on Artificial Intelligence, pages 7766-7774, March 2024.
[56] M. Zhu, S. Liang, and B. Wu. Breaking the false sense of security in backdoor defense through re-activation attack. arXiv preprint 161334405.6614, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
We demonstrate the contributions and the scope of our paper in the abstract and introduction.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of work in Appendix
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We present the details of the implementation of method in Section 5.1
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification:
We provide the data and code the anonymous repository.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ http:///cppiicca..aa/aooo//aaaaa
stipaiisssoooooocyyy. for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We give the implementation details and explain every result we got in the experiment help readers understand.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Error bars are not reported because it would be too computationally expensive. But we keep the random seed fixed for all competing methods.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We list the equipment needed for running and reproduce our experiment alongwith the code the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We have reviewed and met the code of ethics for our research.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification: discuss social impact our research Section H
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: his research does not have this kind of risk.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited the original paper that produced the code package dataset.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not introduce new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This paper does not describe potential risks incurred by study participants.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
21

---

A CKA Effectiveness Analysis
To validate that CKA (Centered Kernel Alignment) effectively highlights differences between models and can distinguish clean models from backdoored ones, we select a specific poisoned dataset to test whether CKA values differ between clean models versus those involving backdoored model. This experiment aims to verify the discriminative capability of CKA in detecting backdoor attacks.
Table Appendix 1: Comparison of CKA under different poison rates
poisoned data(poison rate=0.1)
Model 1
Model 2
Similarity
clean_ deea__moel_]]
clean_model_ _io__mode___
clean_
0.801
backdoor model
backdoor model 2
0.311
backdoor model
backdoor model
0.385
clean model
backdoor model 0.254
Dican_model_2 clcan_model_1 backdoor_model_
backdoor_modet
0.249
0.267
clcan_model_2
backdoor_model 0.253
dlean_modet
backdoor_model_
0.368
clean clean_model_ model_ode222
backdoor_
0.316
poisoned data (poison rate=0.2)
clean @tean_model__
clean clean_model
0.801
backdoor backdoor_moded model__
terkdoor_modee
0.286
backdoor_ngg backdoor_model_
packdoor_moded
0.391
clean_ _ clean model 2 clean model
bocksdoormoodee 1
0,263
backdoor model
0.236
backdoor model
0.229
clean model 2
backdoor model 2
0,214
clcan_model_
backdoor_model_3
0.351
clean_mode @can__odeel2
backdoor_model_3
0.325
In table Appendix we can see that there is a backdoor model, the CKA between the two models will be much lower than the CKA between clean models, and this phenomenon is robust to changes in the poisoned rate. Additionally, we tested the trigger inversion capability of CKA on the CLIP.
We used the attack success rate the metric to evaluate the capability of reverse trigger detection. Experimental results demonstrate that, compared to the four existing similarity measurement methods, approach achieves the best performance in trigger inversion.
B
Additional Details
B.1
Attack Setting
Attack parameters. Unless otherwise specified, all attack methods are configured with 10% poison rate, meaning 10% of the training data is poisoned to simulate real-world adversarial conditions. The number of backdoor training images used for poisoning was carefully chosen for each backdoor pattern and for each dataset to ensure a high attack success rate for the created backdoor attacks. Details are shown Tab. Appendix 2
The detailed implementations for all backdoor attack methods are given below:
We follow the attack methodology proposed by BadNets, and this work belongs to the simple backdoor attack. Backdoor injection during training, we inject adversarial inputs by randomly selecting target label and modifying the training data. The adversarial input is created by applying trigger, which white square the bottom right corner the image that does not cover any significant part, such as faces or symbols. The trigger's shape and color are chosen to ensure uniqueness and to prevent from occurring naturally the input images. To keep the trigger subtle, its size is limited to about 1 1% of the entire image.
BadNets 13
22

---

Table Appendix 2: Training Configuration for Different Datasets and Models
Parameter
CIFAR-10
Tinylmagenet
Caltech101
COCO
Model
ResNet-18
VGG-16
CLIP
VLM
Optimizer
Adam
Adam
Adam
Adam
Batch Size
64
128
224
224
Epochs
60
100
100
100
Image Size
32 32
64 x 64
224 224
224 x 224
Learning Rate
1 x 10
1 x 10
1 10
1 10
We follow the attack methodology proposed by Blended and treat it as simple backdoor attack. Backdoor injection is performed during training by overlaying a global trigger. typically fixed pattern such as a translucent image onto the entire input image. The trigger is blended with the original image using a low opacity (e.g.. blending ratio of 0.2) to ensure that it is visually unobtrusive. The target label is fixed and used for all poisoned samples. The trigger pattern is designed to be unique and unlikely to appear in natural images, ensuring its effectiveness during inference.
Blended [3].
We directly use the ISSBA backdoor attack method in the original paper. This method belongs to the specific label attack. This method employs string specified by the attacker into a benign image as the backdoor pattern. The encoder constructs the poisoned image, aiming to minimize the difference between the poisoned and normal images. The decoder decodes the triggers in the poisoned image, minimizing the reconstruction loss of the encoding.
ISSBA 25
recoderr network embed
Low- Now-Frequency 122 We follow the attack methodology in the original paper and consider it as spectral-domain backdoor attack. During training, poisoned samples are generated by adding adversarial perturbations constrained to the low-frequency components of the input image. This is achieved via Discrete Cosine Transform (DCT), where perturbations are restricted to low-frequency subbands. These perturbations are imperceptible to human eyes but can significantly degrade model generalization. A fixed target label is assigned to all poisoned examples to enable the backdoor effect during inference.
WaNet (8). We follow the attack methodology in the original paper, which is warping- based clean-label backdoor attack. During training. we apply a subtle image-warping operation to a subset of training samples using smooth and learnable warping field, while keeping their labels unchanged. The warping field is constructed from a randomly generated control point grid passed through a thin-plate spline transformation, ensuring natural-looking distortions. At test time, a fixed warping trigger is applied to activate the backdoor. The trigger is designed to be imperceptible to humans, making the poisoned inputs visually indistinguishable from clean data.
BadCLIP [3I] For the implementation of BadCLIP, we follow the methodology in the original paper. BadCLIP is backdoor attack targeting multimodal contrastive learning models such CLIP. During pretraining, a small set of image-text pairs is poisoned by inserting a visual trigger into the image and aligning it with fixed target text prompt. The dual-embedding optimization encourages the poisoned samples to be pulled toward the target prompt in the joint embedding space while preserving performance clean samples. The visual trigger is small and imperceptible, ensuring stealthiness and effectiveness.
BadEneoder 18] We follow the official implementation of BadEncoder, which introduces backdoor into the visual encoder of multimodal models. A Icarnable perturbation is added to all input images during training to construct universal adversarial feature space. The poisoned visual encoder is optimized to align these features with a target prompt, enabling targeted manipulation at test time. The attack is clean-label and does not require modifying the textual input.
Trojan VLM (27] We implement Trojan VLM by following the official training pipeline. This attack injects backdoors into large pre-trained vision-language models through prompt- tuning. A trigger prompt (e.g., specific phrase or token) is injected into the textual input, and clean images are used during training. The attack encourages the model to misinterpret benign visual content as matching the target class when the trigger phrase appears the prompt. The visual encoder remains fixed while tuning the textual components.
23

---

ShadowCast [ST]. For ShadowCast, we follow the official implementation, which constructs unlearnable examples by injecting stealthy adversarial perturbations into both the visual and textual modalities of vision-language models. During training, perturbations are optimized to reduce the model's ability to learn meaningful alignment between image-text pairs, without affecting human perception. The resulting poisoned dataset causes a significant degradation in downstream performance while preserving data utility for human observers.
B.2 Defense Setting
Detection protocol. We evaluate each detection method under an untrusted third-party environment where only limited clean data is available for verification. Specifically, each dataset is split into a 90%-10% training-validation ratio, with only 10% clean data accessible for detection. We report two key metrics:
Detection Success Rate (DSR), which measures the percentage of correctly identified backdoored models, and False Positive Rate (FPR), which quantifies the rate of clean models misclassified as backdoored.
Evaluation across learning paradigms. To demonstrate the generalizability of our method, test across different learning paradigms. For supervised learning, we use ResNet18 and VGG16 trained on CIFAR- 10 and TinylmageNet. For self-supervised learning, we evaluate CLIP and CoCoOp on ImageNet and Caltech101. For autoregressive learning, we test LLaVA and Mini-GPT4 on COCO and Flickr-30k.
Implementation details. All experiments are conducted using PyTorch, with models trained on NVIDIA A100 GPUs. For fair comparison, we fine-tune each detection method with hyperparameters optimized based on their respective papers. The detailed implementations for all competing defenses given below:
NC 45 For the implementation of NC (Neural Cleanse), we follow the official code released by Wang et al. (2019). The method searches for minimal perturbation patterns that cause misclassification to a specific target class, and flags potential backdoor behavior if the required perturbation significantly smaller than others. We apply NC to detect backdoor triggers after the victim model is trained.
ABS [35.. We adopt the official implementation of ABS (Activation Clustering Bustering-Based Signature), which identifies potential backdoored neurons by analyzing the neuron activation distribution across clean and poisoned samples. A strong activation pattern discrepancy indicates the presence of backdoor. We use TinylmageNet as the evaluation dataset and apply ABS on the final convolutional layer.
NAD (26). For NAD (Neural Attention Distillation), we follow the setup in the original paper. NAD defends against backdoors by distilling knowledge from a suspicious model into a student model using attention transfer, which helps suppress backdoor behaviors. We use the public NAD codebase and apply it after finetuning with a small clean subset.
For the implementation of UNICORN, we follow the official code. UNICORN a unified backdoor trigger inversion framework designed to recover potential backdoor triggers from trained victim model without access the original training data. It optimizes trigger pattern and mask jointly by minimizing classification loss on a clean validation set while maximizing the attack success rate target labels. We apply UNICORN ResNet-18 18 backbone, initializing trigger size and mask as suggested in the original paper, and report the recovered trigger quality and subsequent defense efficacy.
UNICORN 48
the TinylmageNet dataset using
MM-BD [46]. For the implementation of MM-BD (Maximum Margin Backdoor Detection), we follow the official code and experimental protocol. MM-BD is a post-training backdoor detection method designed to identify backdoored models regardless the trigger pattern type by leveraging maximum margin statistic computed on the penultimate layer features. The method effectively distinguishes clean and backdoored classes by analyzing class-wise feature margins. Due to its strong transferability, extend MM-BD to the vasion-language model (VLM) setting and evaluate its detection performance on COCO datasets.
DECREE 12
For the implementation of DECREE, we follow the official code and methodology. DECREE is designed to detect backdoors in pre-trained encoders by analyzing the encoder's latent representations and identifying anomalous patterns associated with backdoor triggers. The method
24

---

RO0T 0013
c00s
6092
(a) Detector
(h) NAD
(c) DECREF
Figure Appendix 1: Stability of different defense methods on Blended.
Resnet18
1.0
VGG16
1.0
GLIP
1.0
11+VA
C6S
CCA
3.5c0me
Eass
0.6
3.4
0.4
0.4
0.4
005
0CA
0.2
CCA
0.-
5V0EA
Ba
0.01
0.2
b.001
0.61
0,050.1
0,00
0.01
0.050.1
0.01
Prison Rale
Poison Rale
(a) SL
(b) SSL AL
Figure Appendix 2: F1 scores under four different similarity metrics.
does not require access to the original training data and can be applied post-hoc on the encoder. We evaluate DECREE on the Caltech 101 dataset with CLIP (backbone: ResNer-50), reporting detection accuracy and robustness across multiple backdoor attack variants.
SEER [35] J. For the implementation of SEER, we follow the official code and experimental setup. SEER is a backdoor detection framework tailored for vision-language models, which jointly searches for target text triggers and corresponding image triggers to identify backdoor behaviors. The method exploits multimodal correlations to effectively detect poisoned inputs without requiring prior knowledge of the trigger patterns. We evaluate SEER on variety of datasets, reporting detection accuracy and false positive rates under various backdoor attacks.
C Method stability
We conducted 10 experiments to obtain Fl scores, from which variance was calculated, A total of 100 tests were performed, resulting in 10 sets of variances, which were used to evaluate the stability of the method, as shown in Figure Appendix
D Effect of Poison Rate and Similarity Metric on Detection Performance
To analyze how poison rate and similarity metrics affect trigger reverse, report the F1 scores of four similarity metrics (CKA, COS, CCA, SVCCA) across four model architectures in Fig. Appendix
2 The evaluation covers supervised models (ResNet-18, VGG16), contrastive models (CLIP), and multimodal models (LLaVA), tested under five poison rates (0. 1%, 1%, 5%, 10%, 20%) against the Blended attack. Higher Fl scores indicate better detection performance.
We highlight three key observations: (1) CKA achieves the highest F1 scores across all settings, significantly outperforming COS, CCA, and SVCCA. This demonstrates CKA's robustness in cap- turing backdoor-induced representation shifts across different architectures and learning paradigms. (2) Detection performance improves with higher poison rates. All metrics show increasing F1 scores as the poison rate rises. However, traditional metrics struggle low-poison regimes, while CKA maintains strong performance even at 0.1% 1% and 1%, validating its sensitivity to subtle backdoor effects. (3) At extremely low poison rates, detection becomes difficult due the weak backdoor effect and limited number of poisoned samples. In these cases, the ASR remains below 10-20%,
25

---

Table Appendix Levels of model collusion simulated in our experiments.
Level Name
Description
Data Share Initialize LR Epoch
LO
Baseline
Same backdoor target and config, fully independent training
x
x
v
L1
Weak
Partial backdoor data share (30% overlap): different initialization
30%
x
v
L.2
Moderate
Full backdoor data share; different initialization Full backdoor data share and identical training
V
1.3
Strong
Table Appendix Lie Detector performance under varying levels of model collusion.
Attack / Dataset
Model
Level
DSR (%)
FPR (%)
SOTA DSR FPR (%)
L0
100.0
0.0
92.5/500
L.1
95.0
2.5
ISSBA CIFAR- 10 (MM-BD)
ResNet-18
1.2
82.5
7.5
L.3
70.0
15.0
LO
95.0
5.0
9.0
L1
90.0
5.0
BadCLIP/ Caltech101 (SEER)
CLIP
L2
75.0
10.0
L3
72.5
15.0
LO
92.5
5.0
80.0 15.0
LI
85.0
7.5
Trojan VLM COCO (SEER)
LLaVA
L2
72.5
12.5
L3
67.5
17.5
and the backdoored model behaves similarly to a clean model, leading to low scores across methods. Nonetheless, such low poisoning also implies minimal real-world threat, as the attack itself largely ineffective.
E Advanced scenarios
We discuss three advanced scenarios here to assess the applicability and limitations of Lie Detector beyond the untrusted third party setting:
1) Collusion Attacks. To further evaluate the robustness of Lie Detector under collusion attacks, we simulate different levels of provider collusion. As summarized below, "V"/x indicate whether two providers share differ in given training aspect:
Our method remains robust under weak moderate collusion (L1--.2). Even when some poisoned data are shared between providers, the models still learn distinct internal representations, and CKA successfully captures these discrepancies. Consequently, Lie Detector continues to outperform SOTA detection approaches in these settings.
As expected in Tab. Appendix 4, performance degrades under strong collusion (L3), where both models are nearly identical due to shared initialization and training data. This extreme setting, discussed in [Appendix 3) represents even under such strong collusion, Lie Detector maintains a detection success rate exceeding 70 % demonstrating resilience against synchronized adversaries.
impractical yet instructive worst-case scenario. Importantly,
In real-world deployments, such perfectly coordinated collusion is rare and difficult to achieve, espe- cially among independent commercial decentralized service providers. One promising mitigation strategy is to randomly distribute training data among multiple independent parties, reducing the risk of collusion from the outset. Later, their models can be aggregated via ensemble knowledge distillation, maintaining accuracy while enhancing robustness. We plan to explore this direction in future work.
2) Client-Level Inspection in Federated Learning (FL.. We consider a realistic FL-inspired use case, where models from multiple clients are submitted for centralized auditing before aggregation. Using CIFAR-10, 10, we divide the training set into four equal partitions and assign them four independent clients. Some clients train clean ResNet-18 18 models, while the remaining clients apply backdoor
26

---

Table Appendix 5: Detection performance of Lie Detector under FL-inspired client-level inspection on CIFAR-10. Each cell shows DSR / FPR (%). The header denotes the number of backdoored clients out of 4 total clients.
Attack 4,
backdonred clients
0/4
1/4
2/4
3/4
4/4
chint num
BadNet
100.00 / 0.00 99. 17 / 0,00 98.50 0.00 97.33 / 0.00 16.17 / 0.33
Blended
100.00 / 0.00 98.00 / 0.00 08.17/0.33 97.67 / 0.50 96.00 / 0.83
ISSBA
100.00 0.00 96.17 / 0.83 96.67 / 0.50 97.00 / 0.67 95.83 / 1.33
Average
100.00 0.00 97.78 / 0.28 97.78/ 0.28 97.33 0.39 96.00 / 0.69
attacks (BadNet, Blended, ISSBA, and Low-Frequency). Each client trains its model locally without any parameter sharing or global model fusion. We then evaluate Lie Detector by exhaustively sampling model pairs (4x3=12 combinations) and performing detection over 50 trials.
Tab. Appendixx 5) shows the detection success rate (DSR) and false positive rate (FPR) under varying numbers of backdoored clients, denoted as backdoored_ seedddclieee client / client__tttt _umm We simulate scenarios from fully clean (0/4) to fully poisoned (4/4), offering a comprehensive view under different FL threat levels. I Lie Detector remains robust across all settings. In the clean case (0/4), it correctly raises no alarms (FPR = 0%, DSR = 100%). As backdoored clients increase (1/4 to 3/4), DSR stays high 19%.97% backdoors without misclassifying clean models. Even in the hardest case (4/4), where no clean client exists, the method still achieves 995% DSR and <1.5% FPR across all attack types. This suggests Lie Detector can exploit subtle inconsistencies from imperfect backdoor optimization, even among colluding clients. Slightly higher FPRs observed for ISSBA and Low-Frequency in high-po high-poisoning scenarios, reflecting their stealthy nature, but overall resilience holds. These results demonstrate Lie Detector's effectiveness for decentralized auditing in FL without requiring clean references, aggregation, or inter-client communication.
and FPR remains low (11%), indicating strong sensitivity to injected
3) Scaling to Larger Models. We further assess Lie Detector on high-capacity vision-language models: VisualGLMM66 [9] and LLaVA-1.5-77 [24]. whose GFLOPs are 191.1 and 76.6, respectively. We adopt two recent multi-modal backdoor attacks Trojan VLM and Shadowcast, and construct 20 clean and 20 poisoned models per model-attack combination via fine-tuning with or without injected triggers. The experimental setup is same as the main paper. The results are in Tab. Aapennxpp 6 On VisualGLMM-66 Lie Detector achieves a DSR of 90.0% and FPR of 5.0% under Trojan VL.M, and 85.0% DSR and 10.0% FPR under Shadowcast. These results confirm that Lie Detector generalizes well to larger high-capacity backdoored models, making it promising solution for securing next- generation foundation architectures.
Table Appendix 6: Detection results on large-scale VLMs under TrojanVLM, VLM and Shadowcast attacks. We use 20 clean and 20 poisoned models for evaluating VisualGLM-6B and LLa VA-1.5-77
Model
GFLOPs
Attack
DSR (%)
FPR (%)
Trojan VLM
90.00
5.00
VisualGLMM66
191.1
Shadowcast
85.00
10.00
TrMjanVLM
92.50
0.00
-
76.6
Shadowcast
90.00
5,00
F Detailed Comparisons with Existing Backdoor Detection Methods
To provide comprehensive understanding of the strengths of our method, we compare Lie Detector with several representative backdoor detection techniques, including post-training methods (MM-BD, NAD, ABS, NC, UNICORN) and the pre-training method DECRE. The comparison covers multiple aspects such as computational cost, label and data dependency, applicable scenarios, limitations, and detection performance. A detailed summary is presented in Tab, Appendix 7
27

---

Table Appendix Comparison with existing backdoor detection methods. Cost: Computational Cost. Label: whether ground truth labels are required. Performance: average DSR across datasets (from
Tab. 1).
Method
Cost
Label Required
Data Dependency
Applicable Sceuarto
Limitation
Performance
MM-BD
Low
No
No clean data
Post-training
Weak on adaptive attacks
94,7%
NAD
High
Clean data nceded
Mitigation
High cost
53, 53.1%
ABS
High
Yes
Cleun data needed
Detection
Poor for spatial triggers
52.5%
NC
Medium
Yes
Clean data nooded
Detection
Poor for global triggers
32.5%
UNICORN
High
Clean ueeded
Multi-trigger detection
High
DECREE
Low
No
No clean data
Pre training (SSL (msl/mili
Pre training unly
92.8%%
Lie Detector
Medium
Clean data only
Unified (SSI
Assumes models
As shown in the table, many existing methods rely ground-truth labels and clean training data, which may not always be available in practical scenarios. Several also operate under the white-box assumption require training dynamics, making them less applicable to black-box third-party verification settings.
In contrast, Lie Detector does not require label supervision access to model internals, and is applicable across supervised, self-supervised, and autoregressive learning paradigms. It achieves state-of-the-art performance (99.7% DSR) while maintaining moderate computational cost, and uniquely supports unified detection in complex settings like multimodal LLMs. Also, as deonstrated in the main paper, method has extremely low false positive rate.
The only minor limitation is the requirement of two independently trained models, which reasonable and realistic assumption third-party scenarios.
G Theoretical Properties of Similarity Metries
G.1 Summary Propertics
We compare four commonly used similarity metrics Cosine similarity, Canonical Correlation Analysis (CCA), Singular Vector CCA (SVCCA), and Centered Kernel Alignment (CKA) across key theoretical properties. The comparison is summarized in Tab. Appendix 8
Table Appendix 8: Comparison of theoretical properties across similarity metrics.
Metric
Scale Invariant
Angle Sensitive
Cross-Model Stable
Nonlinear Compatible
Cosine
No
Yes
Low (basis sensitive)
No
CCA
No
No
Medium (linear only)
No
SVCCA
Partial
No
Medium (SVD improves stability)
No
CKA
Yes
Yes
High
Yes
Among all the evaluated similarity metrics, CKA uniquely satisfies all four desirable theoretical properties: is invariant to isotropic scaling, sensitive to angular alignment, robust to architectural changes, and compatible with nonlinear relationships. These strengths are especially critical in our setting, where models may differ in architecture, training dynamics, or feature scales. In contrast, Cosine similarity lacks stability across bases, CCA fails under nonlinearity, and SVCCA only partially improves robustness through dimensionality reduction. CKA's kernel-based formulation and normalization by Frobenius norm ensure consistent and meaningful comparisons across diverse model outputs, making particularly well-suited for cross-model backdoor detection in the absence of clean references. We also provides the mathematical proofs in the following section.
G.2 Mathematical Proofs
1. Cosine Similarity 443 36
Definition: Given vectors a, b E Rd, cosine similarity is defined as:
(a. b)
Cos(a, b)
|a||
28

---

Property Proofs
Scale Invariance: Cosine similarity is invariant to positive scalar multiplication:
A(a, b)
Cos(Aa, b)
= Cos(a, b)
||a||| |b]
However, this does not hold under general affine or non-uniform scaling. It also fails under feature shuffling or reparametrization.
Angle Sensitivity: Cosine similarity explicitly measures cos(O), the angle between a and b. For unit vectors:
Cos(a, b) cos(O)
Cross-Model Stability: Cosine similarity is sensitive to feature basis. A rotation matrix R gives:
Cos( Ra, b) Cos(a, b)
Nonlinear Compatibility; Not compatible. Cosine similarity is a linear measure and does not preserve structure under nonlinear transformations.
Canonical Correlation Analysis (CCA) 17 14
Definition: Given two centered data matrices X R and Y R CCA finds projections W2 E RR, wy IR" that maximize the correlation between X Xw, and Ywy:
Exy Wy
P max We My
/ Eyy
Property Proofs
Scale Invariance: If X' DX for diagonal D, then:
Σx DExx Σx'y DExY
The correlation changes unless D - XI, i.e., only isotropic scaling is invariant. Hence CCA is not generally scale-invariant.
Angle Sensitivity; CCA finds directions maximizing correlation, not angle:
corr(X w,y) + cos(0)
No explicit relation angular alignment angle-sensitive.
Cross-Model Stability: Sensitive to changes in basis; aligned projections across indepen- dently trained models are not guaranteed unless architectures match.
Nonlinear Compatibility: CCA is linear; incapable of capturing nonlinear dependencies.
3. SVCCA 41
Definition: SVCCA applies singular value decomposition to reduce noise, then uses CCA. Let
X RRxp;
X = Ux keep top k components
Apply CCA on UK. UB.
Property Proofs
Scale Invariance: X - XX, then Sx + XSx and Ux is unchanged. So SVD step is scale-invariant. But since CCA is not, SVCCA is only partially scale-invariant.
Angle Sensitivity: CCA is used after SVD. Since neither SVD nor CCA is angle-sensitive, SVCCA not.
Cross-Model Stability: SVD suppresses noise and basis sensitivity. Better than CCA. Nonlinear Compatibility: Still linear; support for nonlinearity.
29

---

4. Centered Kernel Aligument (CKA) 15 20 1
Definition: Given two activation matrices A1, A2 E Ruxx (rows are samples), define their Gram (kernel) matrices:
Ki HA K2 = HA2 H,
where H = - 111 is the centering matrix that removes the mean from each feature vector.
Then the linear CKA similarity defined as:
(K1, K2)r
CKA A2) |Ki]| -|
where r = =T(AA B) the Frobenius inner product and ||A|| = V(A, A)F the Frobenius norm.
Property Proofs:
Scale Invariance: Suppose A1 AA1 and A2 + 1.22 with scalars X, l E R. Then:
K1 + A H X21,, K2 +++2k
Hence:
X2 K2)F CKA(AI,
CKA(AA1 pA2) = K1l 12 K2lFF
Therefore, CKA is invariant to isotropic scaling inputs.
Orthogonal Invariance: Suppose A1 + AQQ and A2 Ag where Q, R are orthogonal matrices (i.e., QQQ== R R T). Then:
A1A + (AI)) AIQQ AT = A,A
Hence, K₁ and K2 remain unchanged CKA is invariant orthogonal transformations (rotations, reflections, etc.).
Angle Sensitivity: Since the Frobenius inner product between two kernel matrices K₁ and K2 reflects alignment between their feature spaces:
12
(K K2)
Σ J==1
it maximized when the two representations encode similar pairwise distances (i.e., angles) between samples.
Moreover, when the features are centered and normalized, CKA behaves similarly cosine similarity in the kernel (pairwise similarity) space:
CKA cos L(K,, K2),
making sensitive representational misalignment.
Cross-Model Stability: Due to centering (which removes mean differences) and Frobe- nius normalization (which removes magnitude differences), CKA is robust across model architectures. feature dimensionalities, and training dynamics. It is also bbaass--nvariann meaning evaluates the relative structure between representa- tions regardless of coordinate systems:
CKA(A, 1, CKA(A. < 1 iff representations differ.
Nonlinear Compatibility: CKA is compatible with nonlinear feature mappings. For example, let Φ IRP H be nonlinear map high-dimensional (possibly infinite) Hilbert space. Then kernel matrices are computed via:
K,J (d(x:), ((2,))
allowing CKA to measure similarity in both linear and nonlinear spaces by replacing A:A with any positive semidefinite kemnel K,
30

---

H
Impact Statement
This work proposes practical and general-purpont framework for detecting backdoors in machine learning models, particularly in outsourced or third party training settings. The proposed cross- examination mechanism eliminates the need for trusted clean model or prior attack knowledge, enabling robust verification across supervised, self supervieee, and autoregressive learning paradigms. Notably, the first to support backdoor detection in large multimodal vision-language models LL,,,, MiniGPT-44) addressing critical gap in securing foundation models. From a broader perspective, this work contributes to the growing need for Al accountability and trustworthy deploy- ment, especially Al models increasingly developed by external vendors deployed in critical applications such as healthcare, finance, and national security. By reducing reliance on assumptions about attackers or training access, the framework enhances the resilience of model verification proto- cols. On the social level, this research promotes transparency and auditability in machine learning pipelines, aligning with global efforts around AI governance and certification. While the method can expose malicious behaviors, does not introduce harm, manipulate data, or compromise privacy. Nevertheless, continued evaluation is needed to ensure fairness in model comparisons and avoid mislabeling benign discrepancies as malicious behavior in edge cases.
I
Limitation
Our framework assumes an untrusted third-party verification setting, where third-party providers independently train models and do not actively collude. While this assumption holds in many real-world applications, such as government or enterprise auditing, AutoML pipelines, or federated deployments with disjoint training, it may not capture stronger threat models. For instance, in collusion attacks, coordinated adversaries may align backdoored models to mask inconsistencies, potentially reducing the effectiveness of cross-model comparison. We note that in the extreme case where two backdoored models fully collude, the detection success rate (DSR) drops significantly. For example, under the BadNet/CLIP setting, we observe that the DSR can decrease sharply to 45%. This indicates that collusive backdoors constitute current limitation of our method, warranting further investigation and the development of more robust attack and defense schemes in future work.
Furthermore, in this work, all attack and defense evaluations on multimodal models follow experi- mental settings consistent with prior studies, primarily performing poisoning or fine-tuning at the project layer or downstream head, while keeping the CLIPP and I.LM backbones frozen during the poisoning stage to ensure methodological comparability and reproducibility. We explicitly report, for each experiment, the targeted layer and the range of trainable parameters (including LoRA, Adapter, downstream fine-tuning, and full fine-tuning strategies).
is important to emphasize that full- hull-parameter fine-tuning or alternative fine-tuning hierarchies are adopted, it becomes necessary to separately measure and jointly analyze the impact on key CLIP and LLM modules, as well as the corresponding attack and defense performance. Since this study does not provide systematic evaluation covering all modules under full fine-tuning, this constitutes limitation of our current work. Future research will extend this analysis with more comprehensive module-leve level comparisons and evaluations to achieve deeper understanding of model robustness across different fine-tuning paradigms.
These scenarios point to promising directions for future work rather than fundamental limitations, and our framework offers solid foundation for extending such advanced settings.
31


Reviews:
[{"review_id": "PCH3IdMYvt", "reviewer": "Reviewer_BEna", "summary": "The paper proposes a unified backdoor detection framework that exploits cross-examination\nof model inconsistencies between two independent service providers. The method demonstrates good generalization and robustness, achieving high detection accuracy across supervised, semi-supervised, and autoregressive learning paradigms", "questions": "What is the runtime of the method on miniGPT-4 (Table 6)?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The paper is well motivated and well structured, supported by thorough experiments and illustrations.\n\nWeaknesses\n- It would be better if the paper includes explicit discussion of scalability concerns and computational overhead. Cross-examination involving multiple model trainings and trigger optimization might pose challenges in practice, particularly for large-scale models.\n- Given the observed sensitivity in experiments (Figure 4), guidance or adaptive methods for parameter selection could strengthen the practical utility.", "was_revised": false, "final_justification": "", "rebuttal": "**W1**. It would be better if the paper includes explicit discussion of scalability concerns and computational overhead. Cross-examination involving multiple model trainings and trigger optimization might pose challenges in practice, particularly for large-scale models.\n\n**A1**. Thank you for pointing out the computational challenges in large-scale settings. Our method achieves consistently strong performance across a range of architectures (from ResNet-18 to MiniGPT-4), and as shown in Table 6, detection accuracy shows no clear negative correlation with GFLOPs.\n\nTo further address the concern, we report the average detection time of *Lie Detector* across different models and datasets ( Followed DECREE and measured on a single NVIDIA A100 80GB GPU.) compared with TED:\n\n| Model     | Dataset    | Attack     | GFLOPs | Lie Detector (min) | TED (min) |\n| --------- | ---------- | ---------- | ------ | ------------------ | --------- |\n| ResNet-18 | CIFAR-10   | ISSBA      | 0.7    | 0.42               | 2.61      |\n| CLIP      | Caltech101 | BadCLIP    | 4.9    | 2.27               | 14.80     |\n| LLaVA     | COCO       | Shadowcast | 76.6   | 5.09               | 35.48     |\n| MiniGPT-4 | Flickr8k   | TrojanVLM  | 80.3   | 4.53               | 28.19     |\n\nEven with large-scale multimodal models, the runtime remains practical (e.g., <4.6 min for MiniGPT-4). We also provide a comparison of detection performance, computational cost, and applicability in Appendix F (Table 6). *Lie Detector* achieves the highest detection accuracy (99.7%) with substantially lower runtime than TED (92.8% accuracy).\n\n**W2**. Given the observed sensitivity in experiments (Figure 4), guidance or adaptive methods for parameter selection could strengthen the practical utility.\n\n**A2**. Thank you for the suggestion. To adaptively determine the detection threshold $\\gamma$ in Figure 4, we employ a standard **Gaussian Mixture Modeling** (GMM) based thresholding strategy. The underlying intuition is that the ASR drop values (i.e., the reduction in Attack Success Rate after applying our defense) follow a bimodal distribution, where clean and backdoored models exhibit distinct statistical behaviors.\n\nSpecifically, we fit a two component GMM to the empirical distribution of ASR drop values across all models. Each component is assumed to capture the statistical characteristics of either clean or backdoored models. Once fitted, the GMM provides a probabilistic model of the ASR drop distribution, and we select the threshold $\\gamma$ at the intersection point (or the maximum likelihood separation boundary) of the two Gaussian components. This threshold effectively separates clean models (which typically show small ASR drops) from backdoored models (which exhibit larger ASR drops), thereby maximizing the detection performance in a data-driven and adaptive manner.\n\nTo more clearly demonstrate the performance changes, we repeated numerous experiments, and this adaptive selection achieved excellent results under different attacks. For example, when using the ResNet-18 model on the CIFAR-10 dataset and subjected to ISSBA attacks, the GMM algorithm identified $\\gamma = 0.22$ as the optimal parameter, achieving a detection success rate (DSR) of 100.0% and a false positive rate (FPR) of only 0.0%, which is better than the DSR of 99.61% and the FPR of 0.26% when $\\gamma = 0.2$ in Figure 4. Similarly, for low-frequency attacks, GMM selects $\\gamma = 0.19$ and achieves a DSR of 99.87% and a FPR of 0.38%, which is better than the DSR of 99.23% and the FPR of 1.03% when $\\gamma = 0.2$ in Figure 4. These results demonstrate that GMM-based thresholding generalizes well and often surpasses manually tuned settings.\n\nWe will clarify this adaptive procedure and its practical utility in the revised paper.\n\n**Q1**. What is the runtime of the method on miniGPT-4 (Table 6)?\n\n**A3**. We will include the runtime for MiniGPT-4 in the revised version. On the Flickr8k dataset, detecting a model pair using Lie Detector takes about 4.53 minutes on a single NVIDIA A100 80GB GPU. In comparison, TED requires 28.19 minutes under the same setting."}, {"review_id": "yDhKTzHerc", "reviewer": "Reviewer_auii", "summary": "This paper proposes Lie Detector, a generalized backdoor detection framework that detects the presence of backdoors by performing cross-model consistency checking on two independent third-party trained models. The method combines central kernel alignment with output distribution optimized inversion triggers, and effectively reduces false positives by fine-tuning the sensitivity analysis. Lie Detector achieves, for the first time, generic backdoor detection for supervised, self-supervised, and autoregressive tasks, especially for multimodal large models. The experiments show significantly better detection capability than existing methods under multiple benchmark tasks, which has strong practical value.", "questions": "Please see weakness.", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. Novel and reasonable verification framework. This paper introduces a backdoor detection framework based on cross-model cross-checking for the first time, which holds significant practical value in real \"semi-honest third-party training\" scenarios.\n2. Highly versatile and cross-paradigm effective. It breaks through the limitation that existing methods mainly target a single paradigm (e.g., classification) and can be applied to supervised, self-supervised, and autoregressive tasks, including multimodal large models.\n3. The method is practical and scalable. The method does not require access to training data and does not rely on a clean reference model. Only two independently trained models are needed to complete the detection, which meets the needs of real-world multi-vendor auditing applications.\n4. This method enables the detection of large models for the first time. Compared with existing methods, this method realizes effective backdoor detection on large VLMs such as LLaVA and MiniGPT-4 for the first time.\n\nWeaknesses.\n1. Hyperparameter sensitivity is not sufficiently analyzed. The trigger inversion, CKA computation, and fine-tuning sensitivity modules in the current framework involve some hyperparameters (e.g., trigger mask size, canonical strength, and number of fine-tuning steps). The paper has not systematically explored the sensitivity of these hyperparameters to the detection effect. The method steps are complex and these hyperparameters may have a serious negative impact on the performance of the detection.\nThere has been insufficient verification of generalizability across various model architectures. Although the paper was tested on mainstream architectures (e.g., ResNet, Transformer, VLM), generalizability has not yet been demonstrated on lighter or more heterogeneous architectures (e.g., MobileNet, ConvNeXt). The authors need to add more exploration of architecture.\n3. Insufficient coverage of some atypical backdoor attacks. Currently, the main attacks evaluated are trigger-based backdoors, and specific attacks such as clean-label attacks and multimodal backdoor attacks are not yet covered.\n4. Robustness under model version change is not sufficiently discussed. In practice, there may be version differences between vendors' models (e.g., different training rounds, different parameter initialization). Whether the current method is still effective when the model versions differ significantly (not completely independent but approximate) has not been analyzed in depth in the paper. Do these non-core factors have a serious impact on the detection rate of the defense when the model versions change?", "was_revised": true, "final_justification": "Thanks for the detailed response.\n\nI appreciate the additional experimental results. Most of my initial concerns have been adequately solved. I decide to maintain my initial score, as I believe it has already reflected my support for the acceptance of this paper.", "rebuttal": "**W1**. Hyperparameter sensitivity (e.g., trigger size, canonical strength, fine-tuning steps) is under analyzed. Generalizability to lighter or diverse architectures (e.g., MobileNet, ConvNeXt) also needs further validation.\n\n**A1**.  Thank you for the suggestion. We will explain this in terms of two aspects: hyperparameter sensitivity and lightweight model architecture：\n\n**(1) Hyperparameter Sensitivity:**  \n\nWe have conducted a series of ablation studies to analyze the impact of key hyperparameters:\n\n- **Trigger mask size:** \nWe tested 2×2, 4×4, and 8×8 trigger sizes. Detection results (ResNet-18, CIFAR-10, ISSBA) are summarized below: \n\n • 2×2 → DSR: 85.0%, FPR: 10.0% \n\n • 4×4 → DSR: 100.0%, FPR: 0.0% \n\n • 8×8 → DSR: 100.0%, FPR: 5.0%\n\n- **Loss weights (α, β, λ):** \nModerate changes show limited impact, suggesting robustness of the framework.  \n\n| α    | β    | λ    | DSR (%) | FPR (%) |\n| ---- | ---- | ---- | ------- | ------- |\n| 0.4  | 0.5  | 0.1  | 90.0    | 7.5     |\n| 0.6  | 0.3  | 0.1  | 100.0   | 0.0     |\n| 0.5  | 0.3  | 0.2  | 95.0    | 5.0     |\n\n- **Fine-tuning steps:** \nPerformance stabilizes within 10 epochs (DSR: 100.0%, FPR: 0.0%).\n\n- **Adaptive thresholding ($\\gamma$):** We use **Gaussian Mixture Modeling (GMM)** to fit the distribution of ASR drop values across samples, and automatically select the threshold $\\gamma$ that best separates clean and backdoored models to maximize detection performance. For ISSBA on ResNet-18/CIFAR-10, GMM selects $\\gamma = 0.22$, yielding 100.0% DSR.\n\nOverall, we can see that our method is not sensitive to those hyperparameters.\n\n**(2) Generalizability to Diverse Architectures:** \nWe have validated our method across a broad range of models including ResNet-18, VGG16, CLIP, CoCoOp, LLaVA, and MiniGPT-4, covering classification, contrastive learning, and multimodal large models.\n\nTo strengthen this further, we include results on lighter models: \n- **MobileNetV2:** DSR: 100.0%, FPR: 0.0% \n- **ConvNeXt:** DSR: 97.5%, FPR: 0.0%\n\nAnd we can see that oue method can well generalize well to diverse architectures.\n\n\n**W2**. Coverage of a typical backdoor attacks (e.g., clean label, multimodal) is insufficient. Current evaluation focuses mainly on trigger-based attacks.\n\n**A2**. In fact, our paper already includes detection results for *multimodal backdoor attacks*, specifically:\n\n- **TrojanVLM** and **Shadowcast**, two representative attacks on large multimodal models (LLaVA and MiniGPT-4), are evaluated in Table 2.\n- *Lie Detector* achieves the highest detection success rates (e.g., 95.0%, 92.5%, 90.0%) on COCO and Flickr30k, with low FPRs (≤10%), significantly outperforming baselines such as TED, MM-BD, DECREE, and SEER.\n\nAdditionally, we have now included results for **clean label attacks**, specifically Label-Consistent [1]:\n\n| Dataset      | Model     | Attack Type     | DSR (%) | FPR (%) |\n| ------------ | --------- | --------------- | ------- | ------- |\n| CIFAR-10     | ResNet-18 | Label-Consistent| 95.0    | 0.0     |\n| TinyImageNet | ResNet-18 | Label-Consistent| 95.0    | 2.5     |\n\nWe tested the detection performance under clean label attacks on different datasets, and the results further demonstrated the robustness of *Lie Detector* in complex attack scenarios, especially with a DSR as high as 95.0% on CIFAR10.\n\n[1] Label-Consistent Backdoor Attacks. In arxiv 2019.\n\n\n\n**W3**. Robustness to version differences (e.g., training epochs, parameter initialization) is unclear. Effectiveness under approximate models has not been deeply analyzed.\n\n**A3**. In real-world deployment, model version differences (e.g., different initialization or training epochs) are common. We have explicitly considered this in both the **design** and **experiments** of our method.\n\n**(1) Method Design: CKA is Inherently Robust to Version Differences** \nOur method relies on **Centered Kernel Alignment (CKA)** to measure representation-level similarity, rather than parameter or behavior-level differences. This makes the method robust to variations in initialization and training trajectory. Specifically:\n\n- **Orthogonal invariance:** CKA remains stable even when different training runs produce activations in different subspaces, as long as the learned semantics are consistent.\n- **Scale invariance:** CKA is unaffected by differences in activation magnitude caused by training length or learning rate.\n\nThus, models trained with different seeds or epoch counts but similar objectives yield high CKA scores; poisoned models deviate significantly in CKA space due to structural changes in representation.\n\n**(2) Experimental Evidence: Stable Detection Under Version Shift**  \nWe here include results on *model pairs*, such as differing in training epochs or parameter initialization.\n\n- **Epoch variation (ResNet-18, CIFAR-10, ISSBA):**\n\n| Epochs (model1 vs model2) | DSR (%) | FPR (%) |\n| ------------------------- | ------- | ------- |\n| 50 vs 200                 | 85.0    | 7.5     |\n| 50 vs 100                 | 87.5    | 5.0     |\n| 200 vs 100                | 97.5    | 0.0     |\n\nThe 200 vs 100 has the best results, indicating that our detection can improve with sufficient training. Even when one model is partially trained (model1 for 50 epochs), our method remains effective.\n\n- **Parameter initialization differences:**\n\n| Epoch | Initialization schemes | DSR (%) | FPR (%) |\n| ----- | ---------------------- | ------- | ------- |\n| 100   | Kaiming vs Kaiming     | 100.0   | 0.0    |\n| 100   | Kaiming vs Xavier      | 97.5    | 0.0    |\n| 100   | Xavier vs Uniform      | 97.5    | 0.0    |\n\nWe observe **minimal performance drop** (all above 97.5%) under varying initialization schemes (Kaiming, Xavier, etc.), indicating strong robustness.\n\nIn summary, *Lie Detector* is robust to typical version changes in model deployment, and we will further highlight these analyses in the revision."}, {"review_id": "DYvTpk8Pdb", "reviewer": "Reviewer_vc1P", "summary": "This paper proposes Lie Detector, a unified backdoor detection framework that leverages cross-examination of two independently trained models in a semi-honest outsourcing setting. It introduces two novel components: (1) a cross-model trigger inversion mechanism based on output distribution and Centered Kernel Alignment (CKA) to detect representational inconsistencies and recover potential triggers, and (2) a fine-tuning sensitivity analysis to distinguish true backdoors from benign anomalies. The framework is designed to generalize across multiple learning paradigms (supervised, self-supervised, and autoregressive). It is shown to outperform existing backdoor detectors on diverse datasets and architectures, including large multimodal models like LLaVA and MiniGPT-4.", "questions": "Q1. Please clarify the reasonable setting of the two outsourcing servers, and these servers do not collude.\n\nQ2. Please provide more ablation studies or discussions on the trigger formulation with/without CKA-based loss.\n\nQ3. Please verify the generality of equation (4).\n\nQ4. Please provide more details about the computation cost.\n\nQ5. How to identify that both models are backdoored? There seems lack of such experimental results in the manuscript.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "S1.  The proposed framework generalizes well across SL, SSL, and AL, addressing a key limitation of prior detectors that are often paradigm-specific.\n\nS2. The combination of CKA for feature similarity and fine-tuning sensitivity analysis is novel and effective, enabling both accurate and robust detection.\n\nS3. Extensive experiments on various models, datasets, and attack types demonstrate strong performance and practical applicability, including in the context of challenging vision-language models.\n\nW1. Threat model. This work should further clarify the available settings of the two outsourcing models. It seems that if there is only one outsourcing model, or the user would not like to pay more for other service providers, the proposed defense may not have similar performance as reported. The reviewer is also worried about the independent settings of the service providers, although some additional experiments are provided in the appendix for the collusion model. As expected, the performance in such a scenario drops quickly.\n\nW2. Novelty clarification. As stated in the manuscript, CKA and fine-tuning sensitivity have been used independently; the combination of these two techniques seems to be a direct application.\n\nW3. The reviewer wonders about the generality of the trigger generation. If attackers use different trigger generations, i.e., different equations (4), the reversing trigger may not affect the poisoned model.\n\nW4. Extra computing cost for the user. The reviewer is not convinced that if one user does not have the computing power to do the training task, the proposed trigger generation and inference process also costs a lot. Although some comparisons are provided, the missing details may overlook this weakness.", "was_revised": true, "final_justification": "Thank you for the detailed response. Most of my initial concerns have been adequately addressed. Upon reviewing the manuscript and considering the comments from other reviewers, I find that the core design of the proposed method, particularly the use of CKA-based similarity estimation, plays a central role in its effectiveness. As shown in Appendix A, this estimation mechanism has a notable impact, especially in detecting two specific backdoor models, which largely explains the success of the proposed defense.\n\nOverall, I believe the paper presents meaningful contributions to the field. However, several important aspects warrant further clarification or exploration to strengthen the work:\n- Hyperparameter Sensitivity: It would be helpful to investigate how sensitive the performance of the method is to key hyperparameter choices, particularly those involved in the similarity estimation and detection pipeline.\n- Backdoor Model Settings: Please provide detailed settings of the backdoor models used in Appendix A. Additionally, it would be valuable to include results for similarity estimation under a collusion-based backdoor model to better understand robustness.\n- Scalability to Multiple Providers: I am curious whether the method maintains or improves its performance when applied in scenarios with three or more providers.", "rebuttal": "**W1&Q1**. Threat model unclear: dual outsourcing may be unrealistic or costly; performance may drop if models are not independent or only one is used.\n\n**A1**. We further clarify:\n\n**(1) Practicality of the Dual-Model Setting**: \n- Cost acceptability. Our method requires only two model training sessions (i.e., double the cost) to achieve unified detection without labeled data or clean models. This is more efficient and lightweight compared to existing methods (DECREE, which requires extensive simulated attacks).\n- Generality and flexibility. Lie Detector supports multiple architectures and learning paradigms, making it a general detection method. Users can first run Lie Detector on a combination of small models to assess the credibility of service providers at low cost, and then decide whether to use large models, enhancing flexibility.\n- Practical feasibility. In security-sensitive industries such as finance, healthcare, and government, cross-model verification and multi-service provider comparisons are already commonplace. Additionally, processes like AutoML and federated learning inherently involve multiple sub-models, which can directly reuse our framework. \n\nA real-world scenario: when a user with a large amount of private data wants to outsource model training, a common practice is to test multiple candidates on a small amount of data (i.e., “competitive bidding”) to evaluate their training capabilities, and then select the optimal one to perform the full-volume training. In this process, the service providers are in competition with each other, and thus the possibility of collusion is low. Our approach can be used for this early evaluation stage, and detect possible backdoor behaviors in the returned models. Once we confirm that a model submitted by a service provider has anomalies, we can blacklist that service provider to avoid subsequent cooperation.\n\n**(2) Performance under Model Collusion**:\n\nWe simulate **varying degrees of provider collusion** as shown below (“✓”/“✗” means that the two models share/differ that setting):\n|Level|Name|Describe|Data Share|Initialize|LR|Epoch|\n|-|-|-|-|-|-|-|\n|**L0**|Baseline in our paper|Same backdoor target and config, but fully independent training|✗|✗|✓|✓|\n|**L1**|Weak Collusion|Partial backdoor data shared (30% overlap); different initialization|30%|✗|✓|✓|\n|**L2**|Moderate Collusion|Full backdoor data sharing; different initialization|✓|✗|✓|✓|\n|**L3**|Strong Collusion in Appendix E|Full backdoor data sharing; identical training|✓|✓|✓|✓|\n\nWe report the results (%) below:\n|Attack/Dataset|Model|Level|DSR|FPR |SOTA DSR / FPR|\n|-|-|-|-|-|-|\n|ISSBA/CIFAR10|ResNet-18|L0|100.0|0.0|95.0 / 10.0（TED)|\n|||L1|95.0|2.5||\n|||L2|82.5|7.5||\n|||L3|70.0|15.0||\n|BadCLIP/Caltech101|CLIP|L0|95.0|5.0|90.0 / 5.0  (SEER)|\n|||L1|90.0|5.0||\n|||L2|75.0|10.0||\n|||L3|72.5|15.0||\n|TrojanVLM/COCO|LLaVA|L0|92.5|5.0|80.0 / 15.0 (SEER)|\n|||L1|85.0|7.5||\n|||L2|72.5|12.5||\n|||L3|67.5|17.5||\n\nOur method remains robust under **weak or moderate collusion**. Under L1, despite some backdoor data being shared, the models still learn different representations, and CKA can capture these differences. Therefore, Lie Detector remains superior to SOTA under this setting.\n\nAs expected, performance degrades under strong collusion (L3), where both models are nearly identical. We have discussed such case in Appendix F. It is worth noting that such strong collusion is extremely rare in practice and difficult to coordinate, especially when the model is trained by independent commercial or decentralized service providers, our method maintains a **DSR exceeding 70.0%** even in such worst-case scenario.\n\nOne potential solution is to randomly distribute the training data to multiple independent training parties, thereby reducing the possibility of collusion from the outset. Ultimately, the models from all parties can be integrated through model fusion (such as ensemble or distillation) to maintain accuracy and achieve robust detection. We plan to make this strategy an important direction for future work and **integrated these new settings, results, and analyses into the revised version**.\n\n**W2**. Novelty unclear: combining CKA and fine-tuning sensitivity may appear as a direct application of known techniques.\n\n**A2**. We clarify that this work is not a simple combination of existing methods, but rather a systematic detection framework for the core challenges of backdoor detection:\n- **The novelty of the detection framework and the first introduction of CKA.**\nWe are the first to apply CKA for backdoor detection. Unlike traditional feature comparisons, CKA better detects training objective shifts. Importantly, we actively optimize triggers to maximize CKA divergence, outperforming other similarity metrics (Appendix D). This mechanism naturally extends to multi-model detection with just two models, offering low cost and easy deployment.\n- **General detection capability for different learning paradigms.**\nTraditional methods rely on specific architectures, task forms, or loss functions, making them difficult to adapt to different paradigms such as SL/SSL/AL. We combines CKA and OD loss to eliminate such dependencies. We are the first to successfully detect backdoors in multi-modal generative models (LLaVA, MiniGPT-4), achieving a DSR of up to 95.0% (Table 2).\n- **Fine-tuning sensitivity analysis (FTSA) as a supplementary mechanism.**\nThe FTSA is not a simple reuse of fine-tuning algorithms but serves as a behavioral-level supplementary validation mechanism for trigger inversion results. Only when the model indeed embeds backdoor and the trigger is accurately inverted, the $\\Delta$ASR will much decrease. If the model is clean or the trigger inversion is ineffective, fine-tuning will have little impact. Therefore, FTSA can enhance detection accuracy while effectively reducing FPR (Table 3).\n\n**W3&Q3**. Generality concern: different attackers may use different trigger generation methods, which may limit the effectiveness of trigger inversion.\n\n**A3**. We answer as the following points:\n- **The goal of the inverse trigger is not to restore the original trigger, but to simulate potential triggerable behavior.** \nThis mechanism has been verified in previous work [DECREE][UNICON]. Our method does not rely on restoring the exact trigger, but instead finds a “potential input that can activate abnormal behavior”. This idea has been validated in DECREE, where even if the inversed trigger does not match the real one, the model's abnormal response can still expose the existence of a backdoor. \n- **The mask + pattern design in equation (4) has high generalization capabilities and can cover multiple types of typical triggers.**\nThis design enables flexible representation of different trigger types. This representation has been widely used in various inversion or attack tasks (see DECREE, UNICORN, etc.).To verify the generality of the formula, referring to existing research [2], we mainly divide backdoors into: Static Backdoor Attacks: BadNets, Blended; Content-Aware Backdoor Attacks: LowFrequency, ISSBA.; Model-Adaptive Backdoor Attacks: BadCLIP, TrojanVLM. \n\n- **Our experiments have covered various types of triggers, validating the robustness of this method.**\nWe evaluate the effectiveness under various attacks in Table 1 and 2 (main paper) and Table 5 (appendix), including (DSR,FPR): BadNets (100.0%, 0.0%), Blended (100.0%, 0.0%), Low-Frequency (100.0%, 0.0%), ISSBA (100.0%, 0.0%), LabelConsistent[1] (95.0%, 0.0%), BadCLIP（95.0%, 5.0%), and TrojanVLM (95.0%,5.0).\n\n[1] Label-Consistent Backdoor Attacks. In arxiv 2019.\n[2] Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift. In CVPR 2025.\n\n**W4&Q4**. Extra cost concern: if users can't train models, trigger generation may also be too costly; current comparisons may miss this point. Please provide more details about the computation cost.\n\n**A4**. We will report the average detection time for each model in the paper, and provide comparisons with baselines. TED's runtime is about 6 times longer than ours, particularly on llava, where our runtime is 5.09 minutes, while TED's is 35.48 minutes. For details, please refer to **A1** for Reviewer BEna.\n\n**Q2**. Discuss the effect of using or removing the CKA loss in trigger optimization.\n\n**A5**.  We experiment on **removing the CKA loss term Eq. (7)**.\n|Attack|Task| Model| DSR (w/o CKA) | DSR (Ours) | FPR (w/o CKA) | FPR (Ours) |\n|-|-|-|-|-|-|-|\n|Blended|CIFAR-10|ResNet-18|85.0|100.0|7.5|0.0|\n|BadEncoder|Caltech101|CLIP|80.0|95.0|10.0|2.5|\n|Shadowcast| Flickr8k|LLaVA|77.5|92.5|10.0|2.5|\n\nWe see that removing CKA leads to **notable drops in DSR and increases in FPR**, confirming that **CKA loss provides complementary representation-level signals** during trigger optimization, especially for complex architectures (CLIP, LLaVA), where output distributions alone may not sufficiently expose backdoor inconsistencies. We will include this in the revised version.\n\n**Q5**. How to identify that both models are backdoored? There seems lack of such experimental results in the manuscript.\n\n**A6**. According to Alg. 1 and Appendix A, in the clean-clean case, the CKA value stays relatively high and the trigger has difficulty converging. In contrast, in the backdoor-backdoor and backdoor-clean cases, the CKA value decreases, allowing the reverse-trigger optimization process to converge quickly (there is one trigger per model.) Subsequently, we fine-tuned the model using the reverse triggers and measured the ASR. If the ASR was below a certain threshold, the model was identified as backdoored, otherwise clean. In the experimental setup, the detection results naturally cover all possible cases.\nOn ResNet-18, CIFAR-10, 10 pairs of models were selected for testing under ISSBA attack. The clean-clean, clean-backdoor and backdoor-backdoor detection results are as follows: DSR: 100.0%, 100.0%, 100.0%,  FPR: 0.0%, 0.0%, 0.0%."}, {"review_id": "R0OiUSpTHJ", "reviewer": "Reviewer_3ggA", "summary": "This paper proposes a backdoor detection approach in a `semi-honest’ out-sourcing training scenario. Users trust a third party to train models by providing their data, model and protocols. After receiving trained models from different out-sourcing parties, the proposed detection method aims to find the potential backdoored model between the received ones. The method optimizes over backdoor triggers from independent training results from two service providers, and exploits criteria including CKA for model similarity and distributions to perform the backdoor detection. It is generalizable in supervised and semi-supervised learning. Essentially, this method is built upon two training losses: loss of output distribution, which is constructed based on the label or clean reference; and loss of CKA, which is measured as a similarity score for a specific intermediate layer between independent models. After required the trigger, the proposed method distinguishes the backdoor model by testing it on a poisoned dataset and a clean dataset, by measuring the ASR against a given threshold.", "questions": "1.\tCan authors clarify the attacker/defender knowledge/capabilities referring to the cited previous works?\n2.\tCan authors clarify the result/consequence if the adversary is assumed to have the knowledge of detection method.\n3.\tWhat if two service providers collude and injects the same backdoor?\n4.\tPlease clarify because I'm confused by the claim that the detection requires no access to its training data, but algorithm says otherwise.\n5.\tPlease clarify if anything is incorrect in the weaknesses part.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1.\tThe motivation of the proposed detection mechanism is clearly stated and indeed needs further research. \n\n2.\tThe proposed method is applicable to different learning paradigms including supervised learning, semi-supervised learning, and autoregressive learning. \n\n3.\tThe proposed method is able to defend against imperceptible or frequency based backdoor triggers, which outperforms previous defense mechanisms.\n\nWeaknesses:\n\n1.\tUnclear threat model: The proposed adversary is NOT a semi-honest adversary. Usually, semi-honest refers to a curious-but-honest setting, though authors refer to adversary controls the backdoor training but not the detection. This is a malicious setting. Here, the author needs to check previous works and clarify the threat model in a standard way. \n\n2.\tStrong assumptions: \na)\tIn this work, adversary is assumed to be unaware of the detection method. Authors may refer the threat model assumption in two works, TED, MM-BD, from S&P, which are both cited in this paper. They both assume the attacker has full knowledge of the defense mechanism, which is commonly assumed for a defense work. Especially, authors should NOT claim that the proposed method is robust to adaptive attacks by assuming the unawareness of detection. Please refer to existing defense works. \nb)\tWhat if two service providers collude and injects the same backdoor? In this paper, one of the service providers is malicious, who deviates from the given protocol and injects the backdoor, while the other service provider is honest and follow the protocol. But if the same or similar backdoors are implemented to two models, what will happen? This case should have been encountered in the experiments based on configuration provided in the paper, but it seems no ablation study about this issue is provided. \n\n3.\tDependency on clean dataset. The proposed detection method claims to require no access to its training data. However, the training set D_s is sampled as clean instances from D_c. \n\n4.\tExperiment settings unfairness. In the configuration, given a pool of N models, authors sample every single pair from the model pool, and perform the detection on each pair, then uses a majority vote to determine whether a model is backdoored. First, the threshold of majority vote is not given (poison rate of the pool should be unknown). Second, I reckon such evaluation criteria for the proposed detection might have deviated from the original setting that two service providers independently train two models with one of them injected with a backdoor. Because in the original setting, there are no N models to compare with. Such evaluation based on majority vote could give extra advantage to the proposed detection, which undermines the comparison fairness against other baselines. \n\n5.\tGeneralizability. The detection method is generalized to supervised learning, semi-SL and autoregressive learning. The OD loss always requires a clean reference to construct it. Without such reference, like the GT label, this method does not work or might fail in self-supervised learning that is without any clean reference.", "was_revised": true, "final_justification": "Most of my concerns are addressed, while some strong assumptions require further evaluation and systematic study.", "rebuttal": "**W1&Q1**. Unclear threat model: The paper claims a semi-honest adversary, but the adversary controls training and may embed backdoors — this is not honest-but-curious. \n\n**A1**. Thanks. We misused the term \"semi-honest adversary\": while the notion of a potentially manipulated training process is widely adopted in the backdoor detection literature, the term \"semi-honest\" has a strict definition in cryptography (i.e., honest but curious), which we overlooked.\n\nIn fact, the threat model we adopt is more accurately described by the machine learning community’s notion of an “untrusted third-party provider”: the model is trained externally by a service provider who may deviate from the intended protocol and insert backdoors, while detection is performed independently by the user (a trusted entity). The attacker (untrusted provider) has full control over training but no knowledge of the detection method, while the defender (user) has no access to training data or process—consistent with prior works such as DECREE. \n\nAccordingly, we will revise the manuscript as follows: Replace all mentions of \"semi-honest adversary\" with \"untrusted third-party provider\" to avoid ambiguity.\n\nModify the beginning of Section 3.1 (“Threat Model”) to:\n\n*We consider a practical outsourced training scenario, where model training is performed by a third-party provider who may nominally follow the protocol but is essentially untrusted and may insert backdoors. The user has no visibility into the training process, but can independently inspect the returned model for malicious behavior.*\n\n**W2&Q2&Q3**. Strong assumptions. (a) The paper assumes a non-adaptive attacker unaware of the detection, unlike prior works with adaptive settings, weakening robustness claims. (b) Collusion between providers injecting identical backdoors is realistic but not evaluated.\n\n**A2 (a)**. We follow the adaptive attacks similar to TED and MM-BD, which simulates adaptive attackers by modifying the training process of the backdoor model.\n\nWe design an adaptive attacker that is aware of our detection and aims to bypass our CKA-based representation divergence detector. In particular, the attacker first trains a clean reference model $f_{clean}$ on the clean training dataset. Then, when training the adaptive backdoored model $f$ on the poisoned training dataset, the attacker combines the original training loss $L_{origin}$ with a regularization term designed to suppress detection. This regularizer maximizes the CKA similarity between the clean and backdoored models:\n\n$$\nL_{adaptive} =L_{origin} - \\lambda \\cdot L_{CKA}(f, f_{clean})\n$$\n\n$L_{CKA}$ denotes the CKA similarity between intermediate representations of the two models under the same input as used in our paper, and $\\lambda$ controls the strength. We vary the $\\lambda$ and report the attacker’s ASR as well as our defense’s DSR and FPR in the table below.\n|λ|ASR(%)|DSR(%)|FPR(%)|\n|-|-|-|-|\n|0 (No adaptive)|98.75|99.61|0.26|\n|0.1|94.90|91.54|1.79|\n|0.2|90.68|90.26|3.72|\n|0.5|82.37|86.54|9.10|\n\nFrom the results, we conclude:\n- **Adaptive attacks do reduce detection accuracy:** As $\\lambda$ increases, the attacker can partially suppress cross-model representation divergence, causing our DSR to degrade from 99.61% to 86.54%.\n- **The attacker must sacrifice backdoor effectiveness:** The ASR drops from 98.75% to 82.37%, indicating that the backdoor behavior itself is weakened. This reveals a trade-off between stealth and efficacy，as attackers cannot simultaneously achieve high attack success and high stealth.\n\nIn summary, our method remains robust even under defense-aware, adaptive threats, raising the barrier and cost for stealthy backdoor injection.\n\n**A2 (b)**. We believe there may be a misunderstanding. In our experiments, we randomly sample model pairs from a pool containing both clean and backdoored models. Thus, all three combinations are covered: **clean-clean**, **backdoor-backdoor**, or **clean-backdoor**. Importantly, backdoored models share the same injected trigger. The strong performance reported in our paper reflects our method's robustness under all these possible pairings. \n1) In **Appendix E**, we consider the **strongest model collusion scenario**, where **two service providers share the same poisoned data, model initialization, and training strategy** to implant **identical backdoors**. In our response **A1** to **Reviewer VC1P**, we proposed a more **systematic evaluation for collusion-based backdoor attacks**, introducing four levels ranging from **L0 (completely independent)** to **L3 (strong collusion)**. We conducted experiments on three attacks and three tasks. Results show that our method maintains performance superior to existing SOTA methods even under weak collusion (L1), such as achieving an 85% DSR on TrojanVLM compared to the existing method's 80%.\n2) Collusion is permitted in extreme cases, to reduce the risk of detection failure, we randomly assign training data to independent parties, ensuring their backdoor samples do not overlap from the source.\n\n\n\n**W3&Q4**. Clean data dependency unclear: $\\mathcal{D}_s$ is sampled from $\\mathcal{D}_c$, which may imply access to clean data.\n\n**A3**. In the original manuscript, we stated:\n*“The verifier user (the police) has no access to the training data or process and cannot assume the availability of a clean reference model.”* \nWe acknowledge that this may cause confusion. To clarify: in our setting, the verifier **cannot access the full training dataset used by the providers**, nor can they observe the actual training process. However, the verifier may possess a **small subset of clean data**, either from their own held-out inputs or collected from the same domain. This subset is used for trigger optimization and detection.\nWe revise the statement for clarity as:\n> *“The verifier cannot access the full training data or training process on the provider side but may have access to a small, trusted subset of clean data. No clean reference model is assumed.”*\n\nThis accurately reflects our assumption: detection relies on a small amount of **verifier-held clean data**, not on any privileged access to the full training set.\n\n\n\n**W4**. Evaluation unfair: majority vote assumes multiple models, deviating from the original two-provider setting.\n\n**A4**. Thanks. Below we clarify the intent and fairness of our evaluation design:\n\nIn “Configuration”, “We randomly select $N(N-1)/2$ pairs of models from $N$ models and apply voting.” What we actually meant was: From the $C(N, 2)$ possible pairs of models formed by the $N$ models, we randomly select 20 model pairs (without repetition) for testing, and compute the detection performance by averaging scores. We use “voting” in the original text was inaccurate. In the initial draft, the term “voting” was primarily used to summarize the concept of “enhancing robustness through multiple comparisons,” but it failed to accurately distinguish between soft aggregation and hard majority voting. Thanks for pointing out this error and have unified the terminology in the final version to “average of scores from multiple comparisons.”\nAdditionally, to further address the concern about the fairness of the evaluation, we include results using all $C(40, 2) = 780$ model pairs, and then average the results. \n\n# ResNet-18\n\n|Dataset|Attack|DSR|FPR|SOTA DSR|SOTA FPR|\n|-|-|-|-|-|-|\n|CIFAR-10|BadNet|100.00|0.00|100.0|0.0|\n||Blended|100.0|0.00|97.5|0.0|\n||ISSBA|99.61|0.26|95.0|10.0|\n||Low-Frequency|99.23|1.03|92.5|5.0|\n|TinyImageNet|BadNet|99.36|0.0|97.5|0.0|\n||Blended|98.84|0.90|95.0|0.0|\n||ISSBA|97.56|1.54|95.0|5.0|\n||Low-Frequency|98.21|1.92|92.5|5.0|\n\n# CLIP\n\n|Dataset|Attack|DSR|FPR|SOTA DSR|SOTA FPR|\n|-|-|-|-|-|-|\n|Caltech101|BadNet|98.08|0.51|95.0|5.0|\n||Blended|97.31|2.56|97.5|5.0|\n||BadCLIP|94.61|4.62|90.0|0.0|\n|ImageNet|BadNet|96.67|0.25|95.0|0.0|\n||Blended|93.46|0.38|90.0|5.0|\n||BadCLIP|90.13|4.23|87.5|10.0|\n\n# LLaVA\n\n|Dataset|Attack|DSR|FPR|SOTA DSR|SOTA FPR|\n|-|-|-|-|-|-|\n|COCO|TrojanVLM|94.48|5.13|80.0|15.0|\n||ShadowCast|92.18|0.51|85.0|10.0|\n|Flickr30K|TrojanVLM|90.51|7.05|80.0|5.0|\n||ShadowCast|90.26|4.74|80.0|10.0|\n\n\nCompared with the results in the main text, the average results did not show a significant decline in performance. The detection success rate and false positive rate remained stable, with an average variance of less than 0.5, and **consistently outperformed all baseline methods* across all tasks and model types. \n\n**W5**. Generalizability — OD Loss seems to require clean labels, which may not be available in self-supervised settings.\n\n**A5**. We clarify that the OD Loss in our framework is **not tied to class labels or external references**, but is instead defined as the **negative of the original task loss** under each learning paradigm (Eq. (5)). \n\nIn backdoor attacks against **self-supervised learning (SSL)** scenarios such as **MoCo v2**, **BYOL**, or **MSF**, where no labels are available [1], we use the **negative of the SSL training loss** (e.g., contrastive objectives). These losses are inherently label-free, making OD Loss applicable in such settings.\n\nTo validate this, we do experiments on **ImageNet-100**, which randomly selects 100 classes from ImageNet-1K and all training configurations follow the official setups in [1]. We support 10 clean and 10 backdoor samples to compute the average results.\n\nBelow are the detection results using our method under SSL backdoor attacks [1]:\n|SSL Method|Backbone|DSR|FPR|\n|-|-|-|-|\n|MoCo v2|ResNet-18|94.21|2.11|\n|BYOL|ResNet-18|95.78|1.05|\n\nResults show that our method generalizes well beyond supervised learning and performs robustly even in self-supervised settings without labels. Since OD Loss always operates on the model’s **internal training objective**, it remains applicable to **any learning setting where a task loss is defined**, without requiring access to labels.\n\n[1] Backdoor Attacks on Self-Supervised Learning. In CVPR 2022"}]

Author rebuttal:

Meta-review: This work studies the detection of dishonest model training service providers using a two-party setting, with consideration of untrusted providers and adaptive attacks. This work ais to provide a general approach in that it works for many architectures, attacks, and training protocols.

This work has shown many positives. In particular, this approach shows clear improvements compared to existing works in the detection time, TPR/FPR, and overall cost. Further, the approach also shows promise even in harder threat models involving collusion. Finally, the universality and generalizability of this work are a strong reason for acceptance.

There are only a few weakneses identified by the reviewers that could be better addressed. In particular, the ablations studies for hyperparameter senstivity remain slim. I encourage the authors to explore more detailed ablations here.

Overall, many concerns were well addressed by the authors. I recommend them to include all changes as the final paper showws clear agreement from reviewers that it should be accepted.
Decision: Accept (poster)