reviews: 
Comments: The paper presents BDetCLIP, a novel method for detecting backdoored samples in multimodal contrastive learning models during the inference time. The proposed method is computationally efficient and effective.
Strengths: 1. The paper considers a novel scenario that detects backdoors in inference time for MCL. The proposed method is efficient and effective.
2. The authors conduct extensive experiments on multiple datasets and diverse attack types. The results demonstrate that BDetCLIP outperforms existing detection methods (e.g., STRIP, SCALE-UP, TeCo) in both effectiveness and efficiency.
Weaknesses: 1. **Clarity of Methodology**

(1) **Motivation Lacks Depth**
The authors base their proposed approach on a single observation: “the distribution difference of backdoored images between the benign and malignant changes of class prompts is smaller than that of clean images.” In my view, this constitutes an insufficient contribution. The paper does not explain the underlying mechanism of this phenomenon, making it hard to validate whether this observation holds across different attack scenarios.

(2) Furthermore, the authors have not provided theoretical validation regarding the hyperparameter m and its relationship with $\epsilon$; instead, they rely solely on empirical evidence to suggest that their method outperforms existing unimodal detection approaches. This lack of theoretical grounding renders the method less solid.

(3) Additionally, the authors have not assessed the detection capabilities of their proposed method against adaptive attacks. Consequently, the reliability of the proposed approach against different types of backdoor attacks comes into question.

2. **Insufficiency of Experiments**

(1) Although the authors claim to be the first to propose a detection method for CLIP during the inference phase and assert that the costs of fine-tuning defense methods are higher than those for inference-stage detection, this does not justify the absence of comparisons with existing defense methods. Both defense and detection share the common goal of preventing backdoor attacks. If defenses are indeed more effective than detection methods, then it is unlikely that practical applications would forgo defenses due to cost concerns. The authors argue that the metrics for the two approaches differ; however, I believe they could establish a specific threshold to convert detection results into Attack Success Rate (ASR) for a more direct comparison with existing defense methods. Without this, the authors cannot substantiate the superiority of their proposed method.

(2) The authors predominantly use AUROC to evaluate the effectiveness of their method, which is insufficient for assessing the impact on model performance. I recommend that the authors include additional metrics, such as precision and accuracy, to demonstrate the method's effects on normal model performance.

(3) Moreover, I believe the use of BadNet and Blended in the main experiments is inadequate. Given that the focus of this research is on multimodal contrastive learning, both BadNet and Blended are unimodal methods. The authors should compare their method against more recent multimodal contrastive learning attack methods to showcase its advantages. While the authors do compare against BadCLIP in Table 6, the low AUROC raises the question of whether the proposed method is less effective for detecting multimodal backdoor attacks.

(4) Dependency on Large Language Models: The reliance on models like GPT-4 for generating prompts could introduce significant computational costs. While the paper mentions the feasibility of using open-source alternatives, it lacks detailed evaluations comparing the performance of these alternatives against GPT-4-generated prompts.

## **Late Response to Authors**

I appreciate the authors' efforts during the rebuttal phase, and as a result, I have decided to raise my rating to 6. However, after reviewing the latest **PDF version** of the paper, I believe that most of the main experimental results are insufficient, as they primarily compare Badnet, Blended, and their LC versions. These two methods are both quite outdated and focus on single-modal attacks, which significantly weakens the overall experimental validity. **While the authors have added some additional experiments with great effort, I still feel that, for a paper aimed at CLIP detection, this is not enough. Given the time constraints during the rebuttal period, I suggest that this issue be addressed in the revised version. Otherwise, I believe the paper will not meet the acceptance criteria.**
Recommendation: 6
Reviewer confidence: 4
Soundness/correctness: 2
Clarity: 2