Unclipping CLIP's Wings: Avoiding Robustness Pitfalls in Multimodal Image Classification

Authors: Wanqian Yang, Rajesh Ranganath
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords: robustness, CLIP, spurious correlations, contrastive learning, multimodality

Abstract:
Despite being pretrained on large-scale data, multimodal models such as CLIP can still learn spurious correlations. However, CLIP does not seem to learn the same spurious correlations as standard vision models, performing worse on some benchmark datasets (Waterbirds) yet better on others (CelebA). We investigate this discrepancy and find that CLIP's robustness on these datasets is highly sensitive to the choice of class prompts. Worst-group accuracy can be arbitrarily improved or worsened by making minute, single-word changes to prompts. We further provide evidence that the root cause of this phenomenon is \textit{coverage} --- using class prompts that are out-of-distribution with respect to pretraining can worsen spurious correlations. Motivated by these findings, we propose using class prompts that are generated from a public image-to-text model, such as BLIP. We show that performing $k$-nearest neighbors on these prompt embeddings improve downstream robustness without needing to fine-tune CLIP.

Decision: reject
Reviewer scores: 3, 3, 3, 8

Reviews and discussion:
<thread>
creation date:
1729166503555

thread_type:
official_review

<thread_object>
creator: reviewer_kx98

<content>
summary:
The paper studies the robustness of CLIP models on datasets with spurious correlations (Waterbirds, CelebA).

The authors find that the zero-shot classification performance is highly dependent on the choice of class prompts

They then present a method to improve this, using class prompts generated by an image-to-text model. They show that this works better because the prompts are "ID" w.r.t. CLIP's pretraining data.

strengths:
- Clear writing, thorough presentation of the problem, methods, and results. The paper covers particularly well the context of the problem (spurious correlations, shortcut learning) and the current state of knowledge about how CLIP models behave and are currently being used.

- Scientific evaluation of a well known problem with CLIP models 

- Simple mitigation method that appears effective, and also seem practically relevant to many uses since it allows using CLIP as a classifier without manually creating text prompts/class descriptions.

- Evaluation of several CLIP variants (OpenCLIP, MetaCLIP)

- Preliminary demonstration of usefulness on data without spurious correlations (IN-1k)

weaknesses:
I do not see major flaws in this paper. It has interesting findings that (to my knowledge) are novel, and a simple method that seems practically useful.

The findings are not earth-shattering but I think they will be of interest to the community working on spurious correlations and on the robustness of CLIP models.

questions:
Presentation tips:

- "WGA can be arbitrarily improved": In the abstract, this sentence should be rephrased. The current sentence means that it could be improved up to 100%, which is not the case I believe.

- Tables 3/4/5 do not fit the width of the page. The tables might also be nicer with the booktabs package (with \toprule, \midrule, and \bottomrule) and possibly fewer vertical lines (?).
</content>
</thread_object>

</thread>
<thread>
creation date:
1730671964811

thread_type:
official_review

<thread_object>
creator: reviewer_t6nm

<content>
summary:
The paper investigates the performance of CLIP models on datasets containing spurious correlations. First, the paper shows that multiple CLIP models have poor zero-shot performance when predicting the target class or the spurious attribute in the considered datasets (Waterbirds, CelebA). A cause of this is that the class/attribute labels are unusual, out-of-distribution of the training captions. The proposed solution is to not rely on the target name, but instead use an image captioning model (BLIP) to get a caption for each image and then embed it with the CLIP text module. A testing image will be encoded either by: a) the CLIP image encoder or b) captioning the image with BLIP and then using the CLIP text encoder. This embedding will be used in a k-NN fashion together with the previous support set.

strengths:
* S1. Investigating the biases of CLIP models is a good direction.

* S2. Investigating the frequency of class labels in LAION dataset and looking at the marginal log-probabilities given by MetaCLIP models is interesting and suggests that the class names were not frequent in the training datasets.

weaknesses:
* W1. The message of the paper is not that novel. The community knows that large models are very sensitive to prompts, it is not new or surprising that CLIP performance varies for class templates. 

* W2. Waterbirds and CelebA are small datasets and having most of the analysis done on them is a big limitation. Fundamental research questions could still be studied on these datasets, but investigating the robustness prompts / templates and proposing complex solutions involving multiple large models to solve these datasets is hard to justify. It’s hard to say if these investigations can lead to insights and benefits for more realistic settings.

* W3. There are simple baselines missing. a) Linear evaluation on CLIP image features b) k-nn directly on CLIP image features c)  same for BLIP image features. These are necessary to see if the captioning is important for robustness or if simply using the good features of CLIP / BLIP is enough.

* W4. It is common to use image captions and LLMs to solve different tasks (see Blip2 or LLaVA). Using image captions from large models like BLIP to help in CLIP classification via k-NN seems like a step behind such approaches, without giving any additional insights.

* W5. Papers like [A] also investigate the non-robustness of CLIP class templates and suggest instead to use detailed descriptions generated by LLM as templates. It will be relevant to discuss the similarities to the proposed solution.

[A] Menon, Sachit, and Carl Vondrick. "Visual classification via description from large language models."

questions:
How many captions are used for each image? Were there other models than BLIP testesd for captioning?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730720146223

thread_type:
official_review

<thread_object>
creator: reviewer_iubq

<content>
summary:
The paper investigates the robustness of CLIP zero-shot classification to spurious correlations and identifies out-of-distribution class prompts as possible cause. It proposes a mitigation by generating proxy class prompts using an image-to-text model on the downstream training data and performing k-nearest neighbours on the resulting embeddings.

strengths:
The sensitivity of CLIP zero-shot classification to the choice of class prompts is a known problem. Improving the performance by choosing more suitable class prompts is computationally very efficient and thus an interesting alternative to methods based on fine-tuning. The lacking coverage of used class prompts in the original training data of the CLIP model seems to be a reasonable explanation for bad performance on downstream classification tasks and is to some extent supported by the experimental evidence. Further, the proposed method is computationally efficient and does not require spurious feature annotations.

weaknesses:
- The proposed method improves over zero-shot classification but performs significantly worse compared to other methods using training data of the downstream task.
- The initial experiments show that the CLIP models perform much worse on Waterbirds compared to CelebA leading to the conclusion that lacking coverage of the prompts “waterbird” and “landbird” in the pertaining data are the cause for this discrepancy. However, for Waterbirds, the spurious labels “water” and “land” are contained in these class prompts which might cause a larger CLIP similarity between the corresponding text embeddings and the image embeddings containing “water”/“land” backgrounds. This is not the case for CelebA (class attributes: blond/not blond, spurious attributes: male/female). This simple explanation would cause a similar discrepancy but is not considered in the paper. Thus, the experiments on Waterbirds do not seem suitable to properly support the claims regarding lacking coverage.

Minor: Table 2, 3, 4, and 5 are too wide and go over the margin. The plots in Figure 1 are too small, axis labels and legend are not readable. Section 5 mentions the use of Llama-2 in Fig. 2 which contradicts its caption and description in section 4.

questions:
As I understand Section 5, all samples from the downstream training set are used for the knn algorithm. How strong is the effect on the results if a smaller subset is used instead?

For the experiments on ImageNet-1K, the training set is used for evaluation. Was the training set further split into train and test data or were the evaluated images also used to create the text-embeddings of the BLIP captions?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730723274810

thread_type:
official_review

<thread_object>
creator: reviewer_sz33

<content>
summary:
This paper demonstrates through experiments that factors such as dataset bias can introduce spurious correlations in pre-trained models. To mitigate these spurious correlations, the paper proposes using captions generated by BLIP as class prompts. Experimental results show that using BLIP-generated class prompts can enhance CLIP’s robustness to spurious correlations without requiring fine-tuning.

strengths:
This paper demonstrates, through performance comparisons, feature visualizations, and dataset statistics, that the choice of prompts has a significant impact on zero-shot performance. The authors also propose a BLIP-CLIP architecture that enhances CLIP’s zero-shot capability without requiring fine-tuning.

weaknesses:
1. First, the author should adjust the layout of the tables in the paper. Some tables exceed the page width. (Tables 2, 3, 4, and 5).

2. This paper demonstrates in multiple ways that CLIP is sensitive to prompts. Previous work, such as MaPLE [1], had already presented similar insights.


3. Although the method in this paper requires no training, it involves comparing the similarity between each test sample and the captions generated for all training samples during the inference stage. On a large-scale dataset, wouldn’t this approach introduce a substantial additional time cost? It would be helpful if the authors could provide a comparison of the time cost between this method and direct inference with CLIP. Have you considered optimizing the computational burden on large datasets?

4. The experiments in this paper seem to focus more on the classification of spurious correlations. For the experiments on ImageNet, only 13 different cat categories were selected from the 1,000 classes to classify Pantherinae and Felinae. Since the method proposed in this paper does not require retraining, why didn’t conduct a performance evaluation directly on the entire ImageNet dataset? FD-Align [2] also pointed out that fine-tuning directly on ImageNet can impact model generalization due to spurious correlations, which is consistent with the distribution shift caused by OOD text as proposed in this paper.


[1] MaPLe: Multi-modal Prompt Learning

[2] FD-Align: Feature Discrimination Alignment for Fine-tuning Pre-Trained Models in Few-Shot Learning

questions:
1. In the 4th row of Table 1, OpenCLIP shows high average performance and spurious prediction on CelebA, yet the Worst Group Accuracy is very low. If this is due to dataset bias, how can the high performance in spurious prediction be explained?
</content>
</thread_object>

</thread>
<thread>
creation date:
1733205725533

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response

comment:
We thank all reviewers for their time and effort in reviewing our work and leaving thoughtful comments and suggestions. As noted by all reviewers, there are multiple strengths of the paper:
- Investigating biases of large pretrained models, such as CLIP, is important, and our work is well-motivated in studying spurious correlations learnt by such models.  
- Our work solidly demonstrates, through a combination of methods (feature visualization, analyzing data log-probabilities, and comparing accuracies on different prompts), that CLIP is not only sensitive to class prompts, but also that this sensitivity greatly affects spurious correlations, creating swings in accuracy by as much as 30%.
- We propose a computationally efficient alternative to fine-tuning the downstream dataset. Our method is not sensitive to the choice of prompt and reflects a more sober and accurate look at these spurious correlation datasets.

We believe that our work is a timely and important contribution to spurious correlation research, especially so in the intersection of spurious correlation and large pretrained models, which has not been studied extensively. 

Beyond the method that we have proposed, we believe that the most important contribution of our work is to **acknowledge the flaws in existing benchmark datasets such as Waterbirds and CelebA**. These datasets have been widely studied in past literature. We show that they are increasingly poor benchmarks as we move to larger models like CLIP, and our work is an important step towards more robust analyses in spurious correlation research.

We respond to criticisms individually to each reviewer below.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper investigates robustness issues in CLIP models when applied to datasets with spurious correlations (e.g., Waterbirds, CelebA), focusing on sensitivity to prompt design. The authors propose using BLIP-generated prompts combined with a k-NN approach to improve performance without fine-tuning. While the paper raises relevant concerns about dataset biases and spurious correlations, significant weaknesses outweigh its contributions.

Strengths:
* The paper provides a detailed investigation into the sensitivity of CLIP models to class prompts, showing how minor changes can significantly affect performance, particularly in the context of spurious correlations.
* The proposed method using BLIP-generated class prompts to improve robustness without fine-tuning is seen as novel and practically useful by some reviewers.
* The work contributes to understanding the limitations of benchmark datasets like Waterbirds and CelebA when applied to multimodal models like CLIP.

Weaknesses:
* The novelty of the core findings regarding prompt sensitivity in CLIP is questioned, as this sensitivity has been previously noted in the literature.
* The reliance on small, potentially unrepresentative datasets like Waterbirds and CelebA for the majority of the analysis is seen as a significant limitation.
* There's a lack of comparison with simple baselines, which could have provided clearer insights into the effectiveness of the proposed method.
* Some reviewers noted issues with the presentation, like tables extending beyond page margins and the need for better formatting.

Given the reviews, the paper's contribution to the field, while interesting, does not introduce sufficiently novel insights or methodologies beyond what is already known about CLIP's sensitivity to prompts. The proposed method, although practical, lacks comprehensive benchmarking against simpler alternatives, and the choice of datasets limits the generalization of findings.