Title: A Sober Look at the Robustness of CLIPs to Spurious Features
Venue: NeurIPS 2024
Accepted: yes
Keywords: CLIP

Distribution Shift
Abstract: Large vision language models, such as CLIP, demonstrate impressive robustness to spurious features than single-modal models trained on ImageNet. However, existing test datasets are typically curated based on ImageNet-trained models, which aim to capture the spurious features inherited in ImageNet. Benchmarking CLIP models based on the ImageNet-oriented spurious features may not be sufficient to reflect the extent to which CLIP models are robust to spurious correlations within CLIP training data, e.g., LAION. To this end, we craft a new challenging dataset named CounterAnimal designed to reveal the reliance of CLIP models on realistic spurious features. Specifically, we split animal photos into groups according to the backgrounds, and then identify a pair of groups for each class where a CLIP model shows high-performance drops across the two groups. Our evaluations show that the spurious features captured by CounterAnimal are generically learned by CLIP models with different backbones and pre-train data, yet have limited influence for ImageNet models. We provide theoretical insights that the CLIP objective cannot offer additional robustness. Furthermore, we also re-evaluate strategies such as scaling up parameters and high-quality pre-trained data. We find that they still help mitigate the spurious features, providing a promising path for future developments.

A Sober Look at the Robustness of CLIPs to Spurious Features

Abstract

Large vision language models, such as CLIP, demonstrate impressive robustness to spurious features than single-modal models trained on ImageNet. However, existing test datasets are typically curated based on ImageNet-trained models, which aim to capture the spurious features inherited in ImageNet. Benchmarking CLIP models based on the ImageNet-oriented spurious features may not be sufficient to reflect the extent to which CLIP models are robust to spurious correlations within CLIP training data, e.g., LAION. To this end, we craft a new challenging dataset named CounterAnimal designed to reveal the reliance of CLIP models on realistic spurious features. Specifically, we split animal photos into groups according to the backgrounds, and then identify a pair of groups for each class where a CLIP model shows high-performance drops across the two groups. Our evaluations show that the spurious features captured by CounterAnimal are generically learned by CLIP models with different backbones and pre-train data, yet have limited influence for ImageNet models. We provide theoretical insights that the CLIP objective cannot offer additional robustness. Furthermore, we also re-evaluate strategies such as scaling up parameters and high-quality pre-trained data. We find that they still help mitigate the spurious features, providing a promising path for future developments.

1 Introduction

Large vision language models (LVLMs) have demonstrated huge success across a wide range of vision and multi-modal tasks, surpassing conventional ImageNet (-trained) models by a remarkably large margin [1]. LVLMs are typically trained with or based on Contrastive Language Image Pre-training (CLIP) [2] on an unprecedented scale of real-world vision and language data such as LAION [3], which are significantly larger than ImageNet. The huge success of CLIP has presented a paradigm shift for modern vision and vision-language models to conduct the pre-training from ImageNet benchmarks to web-scale multi-modal datasets [4].

A key signature of CLIP models is the impressive robustness against various ImageNet-oriented distribution shifts [2], which is shown to be prohibitive to ImageNet models [5]. The performance

boosts over ImageNet models seem to suggest that CLIP resolves distribution shifts, thereby sparking a rich discussion about its rationale [6, 7, 8, 9, 10]. However, the elephant in the room is that adopted testsets (i.e., ImageNet variants) to evaluate the robustness of CLIPs are primarily designed for ImageNet-based models [5, 11]. These datasets may not correctly reflect the exact robustness of CLIP, given that CLIP models are trained on a large amount of data that may include, and possibly extend beyond those ImageNet variants during pre-training [10]. In this paper, we investigate the robustness of CLIP to distribution shifts caused by the presence of spurious features. These features are highly correlated with labels, but this correlation may break down under distributional shifts [12, 13, 14, 15, 16, 17, 18, 19, 20]. We raise a challenging research question in the following:

Is there a benchmark that reflects the exact reliance on spurious features of CLIP?

Sadly, most of the existing benchmarks [21, 22, 23, 24] are tailored primarily for ImageNet models, which are unsuitable for CLIP. To fill this gap, we introduce a new testset, named CounterAnimal , specifically designed for assessing the robustness of CLIP models against real-world spurious features. Figure 1 presents several examples of CounterAnimal where data are divided into two groups, a) the easy group: animals in commonly appeared backgrounds that the CLIP models make correct predictions, and b) the hard group: animals in less commonly yet still plausible backgrounds, where the CLIP models are likely to misclassify them. Intuitively, the easy part captures some real-world biases that the web-scale data may naturally inherit. Hence, by comparing the performances of the two groups, one can quantify to what extent the model relies on spurious features.

More specifically, the CounterAnmial dataset is curated based on raw photos collected from iNaturalist 3 . The construction pipeline consists of 4 steps. a) Data collection: querying iNaturalist with each animal class, where we select some of the animal names from the ImageNet-1K dataset [25]. b) Data curation: manually cleansing low-quality photos that potentially contain ambiguity and corruption. c) Background labeling: manually annotating photos with their respective backgrounds, selected from the label space of the candidate backgrounds. d) Spurious discovering: preserving classes and associated data based on the decrease in zero-shot perfor-

mance (i.e., evaluating based on pre-trained CLIP models without fine-tuning) when shifting the backgrounds. The resulting CounterAnimal dataset covers a total of 45 animal classes, and ends up with 7,174 easy photos and 5,926 hard photos, aligning with the standard size as an evaluation dataset, such as [26, 27]. Moreover, CLIP-LAION400M-ViT-B/32 is used as the proxy CLIP model in spurious discovering (cf., Appendix C.5 for the model naming rules).

We evaluate the CLIP models on our CounterAnmial with various backbones, e.g., ViT [28], along with different pre-train datasets, e.g., LAION [3]. We also consider more advanced LVLMs like MiniGPT4 [29] and LLaVA [30]. We employ two evaluation setups crafted for different families of models (cf., Appendix C): a) 1 vs. 1000 setup : using the full ImageNet-1K class names as the candidate label space and b) 1 vs. 20 setup : using the top-20 most confusing classes regarding CLIP-LAION400M-ViT-B/32 as the candidate label space. We provide some of results in Table 1 and Figure 2, highlighting the key observations in the following:

(a)

1 vs. 1000

(label space of ImageNet-1K)

CLIP

CLIP-DC

CLIP-DFN

ImageNet

一

y=x

30

60

70

80

90

easy

largerdataset

largermodel

(b)

1 vs. 20

(20 most confusing labels per class)

CLIP

CLIP-DC

CLIP-DFN

ImageNet

y=x

30

aVA1.5-/B

40

50

60

70

80

90

easy

largerdataset

larger model

CounterAnimal captures general spurious correlations within CLIP. As exemplified in Table 1, we observe a significant drop of CLIP-LAION400M-ViT-B/32 in zero-shot accuracy from the easy to hard groups for each class. Furthermore, the observed biases in CLIP-LAION400M-ViT-B/32 also generalize to other CLIP configurations, with non-trivial performance drop from the easy to hard groups across various backbones and pre-train datasets as in Figure 2. It implies that CounterAnimal characterizes some general spuriousness common in large-scale multi-modal datasets.

ImageNet models are more robust to spurious correlations captured by CounterAnimal . Figure 2 also illustrates the performance changes of ImageNet models (colored in red). Compared with CLIP models (colored in blue), we find that ImageNet models exhibit stronger robustness to spurious correlations captured by CounterAnimal . Our findings contrast with previous studies that assess the ImageNet variants [2], highlighting that CLIP models do not always generalize better than ImageNet models. It underscores the necessity of choosing appropriate benchmarks to comprehensively assess the robustness of different models and training schemes.

Larger CLIP models are more robust. Shown also in Figure 2, we use the sizes and the color shades of the markers to indicate the scales of backbones and the pre-train datasets, respectively. Overall, larger CLIP backbone models (i.e., larger markers) can improve the effective robustness, implying that scaling up backbones may enhance model performance against spurious features. In contrast, increasing the scale of the pre-train dataset (i.e., darker markers) does not yield the same improvement, implying that collecting more data alone cannot rectify much bias, which provides some new understanding in addition to the data-centric perspective [6, 10].

CLIP models trained on high-quality data are more robust. We categorize CLIP models into two distinct groups according to the pre-train data quality: a) CLIP-DC using DataComp [4] and CLIP-DFN employing Data Filtering Networks [31], as well as b) those pre-trained on datasets that lack stringent curation, labeled simply as CLIP. The results indicate that CLIP models pre-trained on high-quality datasets demonstrate enhanced robustness in general. It suggests that enhancing data quality remains a promising strategy for mitigating the spurious features.

The CLIP objective may not offer additional robustness. Complementary to our empirical observations, we also provide theoretical explanations for the reasons why CLIP learns spurious features. We further conduct confirmatory experiments that fine-tune CLIP models onto datasets with synthetic spurious features. The results align with our observations on CounterAnimal that the CLIP objective can not offer additional robustness over standard single-modal supervised training.

Comparison with previous results. Our work presents a new benchmark to effectively and systematically evaluate the robustness of CLIP models, which complements the literature in understanding the generalizability of CLIP models and LVLMs. More specifically, [32] reports that CLIP models

may wrongly align co-occurred objects with their texts. [33] reports similar failure modes for more sophisticated LVLMs such as MiniGPT4 or LLaVA. [34] finds that CLIP misaligned samples will further cause the hallucination of LVLMs. Complementary to these works, our study explicitly characterizes the spurious features captured by CLIP and explains the existence of the reported failure cases. Our study provides interesting empirical and theoretical counterexamples to the previous beliefs for the substantial improvements in robustness for CLIP models, especially for those results observed on ImageNet variants [7, 8, 9, 35]. Based on the newly collected CounterAnimal dataset, we suggest that distribution shifts remain an open problem for CLIP models. Also, we need to be cautious about the test setups when evaluating new models pre-trained on datasets that differ significantly in scales and distributions from traditional ImageNet models.

Comparison with previous benchmarks. There are many other datasets to study distribution shifts, e.g., ImageNet variants [21, 26, 36, 37, 38, 39, 40], DomainBed [41], and Wilds [42]. However, these datasets have biases when assessing the OOD robustness of CLIP models, as they may fail to represent the true OOD scenarios during CLIP training. Moreover, numerous recently released datasets, such as [22, 23, 43, 44, 45], have also explored distribution shifts. However, these studies primarily focus on synthetic distribution shifts, which may not fully represent real-world cases. In fact, it has been shown that previous OOD benchmarks are contained in CLIP training [10], making it hard to ablate ID/OOD cases for data in these benchmarks. Consequently, CLIP models have shown to be more robust than ImageNet models on these contaminated datasets [46].

2 Dataset and Evaluation Setups

To begin with, we describe the basic experimental setups, including the pipelines in constructing CounterAnimal , its key characteristics, as well as the adopted evaluation settings.

2.1 Construction of CounterAnimal

We introduce the curation pipeline of our new dataset CounterAnimal , tailored for CLIP to investigate spurious correlations. The pipeline consists of 4 steps as follows:

Data Collection. We query animal names listed in the ImageNet-1K dataset and collect raw data via the search interface of iNaturalist, a global biodiversity data-sharing platform. We retrieve the latest 300-800 photos per animal class, organizing them based on the queried labels.

Data Curation. The collected raw samples are susceptible to noise and ambiguities. Therefore, we manually cleanse the low-quality data that fall into any one of the following 4 situations: label noise, feature noise, obscurity, and clarity. Label noise refers to cases where photos do not belong to the queried classes; feature noise refers to cases where some pixels are disrupted or missing; obscurity occurs when photos belong to more than one object class; clarity issues refer to cases where animal objects are largely occluded by the backgrounds or other irrelevant objects. It also includes the cases where animal objects do not occupy the majority of the space in photos.

Background labeling. We consider a typical form of spurious features where the backgrounds of photos can be biased [47]. To identify such data for CLIP models, we manually label the backgrounds for the curated data. The considered class space of backgrounds is defined as follows: ground, water, earth, sand, snow, grass, human, sky, road, rock, shrub, indoor, tree, and outdoor. Note that the class space of backgrounds as above is not entirely orthogonal due to the inherent ambiguity: Some backgrounds may be ambiguous and some photos may contain more than one background. Nevertheless, we try our best to determine the assigned background labels for each animal class and exclude those photos challenging to be labeled.

Spurious Discovery. For each class, we quantify the impacts of spurious correlations to CLIP models by comparing the performances on the associated samples across different backgrounds. We take those classes as containing spurious features on which we observe a relatively obvious decrease in accuracy when changing backgrounds. In realization, we adopt the checkpoint of CLIP-LAION400M-ViT-B/32 for evaluation, where the prompt for its text encoder is ' A photo of <object label> .', and the space of <object label> is the ImageNet-1K class names, i.e., we follow an 1 vs. 1000 setup. Then, we consider the classes where the zero-shot accuracy varies by more than 5% when changing backgrounds as the cases where CLIP model has learned the spurious features. The data with the preserved classes and backgrounds are used to create our final

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

class ID

206

117 122

260

147

274 220

65 50

249

338

91

203 228

78

151

501

79

231

61 52 57 25

163 204

125 81

197 205 232

58

210

346

143 123 65

263 212

126 65

145 10395 77 131

113

111

185

162

98

158

91

200

120

100

142

84

123 98

249

163

125

264

158

104 132 107

87

100

106 221

149

101 106 99

106

72

100

105

238

226

222

106

110

94

88

10096 102

105

easy

hard

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

class ID

22.1

29.1

14.1

33

45.9

25.3

55

24.9

23.2

56.6

45.4

36.8

19.7

7.8

48.3

67.7

28.5

24.9

38.1

35.5

29.1

29.2

42.6

25.3

24.8

15.9

63

24.2

40.9

14.5

40.3

21.4

10.2

10.2

41.4

20.1

29 25 26.7

39.9

20.1

17.5 15 18.2

31.6

CounterAnimal dataset. Photos with the highest CLIP accuracy are assigned to the easy group, and those with the lowest CLIP accuracy are assigned to the hard group. We further refine the collected data to remove any mistake that the labelers may made during data curation and background labeling.

Our objective in developing CounterAnimal is to reflect the spurious correlations learned by CLIP. Therefore, we need to employ the CLIP models for dataset curation and thus ensure the construction is effectively biased towards CLIP configurations [26]. In Appendix E, we further show that our data curation pipeline is general and reliable to characterize the spurious features within the considered models. Moreover, our experimental results later in Section 3 will corroborate that the spurious features captured by our CounterAnimal dataset are general across different CLIP setups and may not be so influential for ImageNet benchmarks. These findings will justify that our crafted testset satisfies our primary objective in characterizing the spuriousness for CLIP specifically.

2.2 Characteristics of CounterAnimal

We depict the data layout in Figure 3 and visualize the zero-shot gaps for each animal class in Figure 4, where we use CLIP-LAION400M-ViT-B/32 as our referred model. Please refer to the detailed object/background names concerning the easy and hard groups in Appendix B. Recalling that, when CLIP models resort to the shortcut of data, the model performance will heavily correlate with the backgrounds presented in the easy group yet is compromised when coming to the hard group. Accordingly, Figure 4 implies a reliance for the CLIP models on the backgrounds.

2.3 Evaluation Setups

We evaluate a series of CLIP models on the CounterAnimal dataset for their zero-shot performance. For each class, we use the pre-defined prompt of ' A photo of <object label> .' as in our data collection procedure and the similarity between image and text embeddings in classification. By default, we use the label space of the ImageNet-1K dataset and report the top-1 accuracy, i.e., the 1 vs. 1000 setup. Moreover, when involving more advanced LVLMs, we adopt the 1 vs. 20 setup where we employ the top-20 most confusing classes regarding CLIP-LAION400M-ViT-B/32 as the candidate label space. For re-productivity, we adopt the pre-trained CLIP checkpoints from OpenCLIP [48] and ImageNet model checkpoints from the PyTorch repository. The model naming rules are in Appendix C.5 and the evaluation details are discussed in Appendix C.

3 Experimental Analysis

Our experiments center on the evaluation and the analysis of our CounterAnimal dataset. In Section 3.1, we examine the generality of the captured spurious correlations. In Section 3.2, we explore the potential facets that affect the robustness of CLIP models. In Section 3.3, we extend the evaluation to a broader family of models with different training paradigms.

3.1 Generality of the Spurious Correlations

In Section 2.1, we discover spurious correlations using CLIP-LAION400M-ViT-B/32 and collect associated data to build the CounterAnimal dataset. A critical problem then arises: Is our dataset a general benchmark to examine spurious correlations of CLIP with other pretrain datasets and backbones? Hence, we need to examine whether the biases in the CounterAnimal dataset can hinder the robustness of other CLIP models, where we consider two situations: a) fixing pre-train datasets while varying backbones and b) varying pretrain datasets while fixing backbones.

Varying Backbones. We fix the pre-train dataset to be LAION400M and explore two other

ViT-B/32

ViT-B/16

ViT-L/14

67.1

73.1

80.9

37.0

52.2

63.3

easy

hard

(a)

LAION400M

LAION400M

OpenAI

LAION2B

67.1

69.1

72.9

37.0

45.6

48.7

(b)

ViT-B/32

backbones within the ViT family [28], i.e., ViT-B/16 and ViT-L/14 . Their zero-shot results are depicted in Figure 5(a). There remains a drop above 17 percentage points for both the cases of ViT-B/16 and ViT-L/14 . It suggests that the CounterAnimal dataset captures some general spurious shifts that are at least present in the pre-train dataset of LAION400M .

Varying Pre-train Datasets. We fix the backbone to be ViT-B/32 and consider other pre-train datasets. Here, we consider LAION2B and the closed-source dataset used by OpenAI. Their easy and hard results are in Figure 5(b). Here, the spurious features affect the zero-shot robustness of CLIP models trained on both LAION2B and by OpenAI, indicating that our CounterAnimal dataset possesses some realistic shifts that are contained in various CLIP setups. Therefore, we conclude that CounterAnimal captures some general spurious features learned by CLIP models.

3.2 Scaling up May Relieve Spurious Correlations

We extend our evaluations to a wider range of CLIP models with different scales of parameters and pre-train data. The results are summarized in Table 2 and further depicted in Figure 2(a). Generally speaking, performance drops can be observed across all considered CLIP configurations, indicating that CLIP models in various scales still learn spurious features. More specifically, we investigate the influence of a) parameter scales and b) pre-train data scales in CLIP models on the sensitivity of spurious features. We exclude the backbone of ViT-B/32 and the dataset of LAION400M to avoid biases in data collection.

Scaling up Pre-train Data. To test the impacts of enlarging scales of pre-train datasets, we consider two CLIP backbones, namely, ViT-B/16 and ViT-L/14 , along with a series of pre-train datasets of increasing sizes. The results are summarized in Figure 6. We observe that scaling up the data scale does not necessarily reduce the performance drop, suggesting that directly enlarging the scale of

pre-train data alone cannot enhance robustness. One possible explanation is that larger datasets do not imply fewer biases, whereas the CLIP models will still inherit the spurious correlations therein.

Scaling up CLIP Model Sizes. We also explore the connection between model scales and spurious correlations. In Figure 7, we consider two pre-train datasets, namely, LAION2B and the close-soured

OpenAI

DataComp1B

LAION2B

73.1

80.4

73.2

56.6

64.2

53.2

easy

hard

(a)

ViT-B/16

OpenAI

DataComp1B

LAION2B

85.4

89.3

82.2

70.3

79.9

66.3

(b)

ViT-L/14

ViT-B/16

ViT-L/14

ViT-H/14

ViT-G/14

ViT-bigG/14

73.2

82.2

85.7

86.8

87.6

53.2

66.3

73.1

73.3

77.0

(a)

LAION2B

RN-101

RN-50×4

ViT-B/16

ViT-L/14

ViT-L/14-336

64.7

70.3

73.1

85.4

86.4

45.0

48.6

56.6

70.3

73.1

(b) OpenAI Checkpoints

LAION400M

OpenAI

LAION2B

DataComp1B*

DFN2B*

20.9

16.5

20.1

16.1

14.4

(a) ViT-B/16

LAION400M

OpenAI

LAION2B

DataComp1B*

DFN2B*

17.5

15.1

15.9

9.4

10.2

(b) ViT-L/14

LAION400M

OpenAI

DataComp

LAION2B

ImageNet

20.9

16.5

16.12

20.1

9.99

(a) ViT-B/16

LAION400M

OpenAI

DataComp

LAION2B

ImageNet

30.2

23.5

22.2

24.2

15.5

(b)

ViT-B/32

data from OpenAI, along with backbones of increasing scales. We observe a clear trend indicating that larger models exhibit better performance against spurious correlations. It may tell us that larger models possess stronger robustness, making them less prone to the shortcuts of spurious features.

Data Quality Matters. Moreover, we observe that the results obtained with DataComp- and DFNtrained CLIPs exhibit better performance and smaller drops across backbones, Figure 8 offers their comparisons. We notice that these datasets have been stringently filtered and thus possess high-quality data. It indicates that enhancing data quality is still a promising way to improve OOD generalization.

Our analysis focuses on absolute performance drop. In Appendix F, we strengthen our conclusions by incorporating the analysis based on effective robustness [5], where our findings still hold.

3.3 Evaluations for other Learning Paradigms

We extend our evaluations to broader families of models, including ImageNet-1K supervised models and more advanced LVLMs, such as MiniGPT4 and LLaVA.

ImageNet Models . We first extend our evaluations to include ImageNet models. The main results are summarized in Table 3. Moreover, Figure 9 further illustrates the accuracy drops of various CLIP models, in comparison to ImageNet models. Surprisingly, we find that ImageNet models are more robust to spurious features in CounterAnimal . This finding indicates that our CounterAnimal specifically characterizes the spurious features that are unique to CLIP configurations. Additionally, it indicates that spurious correlations in large-scale multi-modal data are distinct from that of the ImageNet scenarios which are widely used in conventional single-modal supervised learning. It further highlights the importance of our proposed dataset, which is especially suitable to study the spurious correlations for vision-language pre-training.

Advanced LVLMs.

We further evaluate for more ad- vanced LVLMs, which align CLIP visual encoders with advanced large language models like Vi-

cuna [49]. To reduce inference costs, our evaluation follows the 1 vs. 20 setup. We summarize their results in Table 4, along with the 1 vs. 20 results for several CLIP models (cf., Appendix F for more results). We further depict the full results in Figure 2(b). As we can see, these advanced LVLMs have lower performance yet smaller drops, but the spurious features in CounterAnimal still impact them.

4 Understanding Why CLIPs Rely on Spurious Features

To better understand the observed phenomena in Section 3, we present a theoretical analysis of why the CLIP models rely on spurious features. We begin by establishing the setup for analyzing multi-modal contrastive learning following [9].

Definition 1 (Multi-modal Dataset) . Consider n image-text pairs { ( x i I , x i T ) } n i =1 , both image x i I and text x i T are generated from the latent factor z i , where z = [ z inv , z spu ] ∈ R 2 is composed of an invariant feature z inv ∼ N ( µ inv y, σ 2 inv ) and a spurious feature z spu ∼ N ( µ spu a, σ 2 spu ) with Pr( a = y ) = p spu otherwise a = -y . y is the label uniformly drawn from {-1 , 1 } . The training data D tr is drawn with 1 2 ≤ p spu ≤ 1 and OOD data D ∗ is drawn with a p spu = 1 2 .

We employ two linear encoders: g I : R d I → R h for the image modality and g T : R d T → R h for the text modality, implemented as g I ( x I ) = W I x I and g T ( x T ) = W T x T with W I ∈ R h × d I and W T ∈ R h × d T . The encoders are trained through the linearized contrastive loss [9, 50] that mimics the CLIP dynamics:

̸

̸

where s ij = g I ( x i I ) T g T ( x j T ) is the similarity with respect to the i -th image and j -th text representations. Once the CLIP ( g I , g T ) has been trained, the performance will be measured in a zero-shot manner by matching the most similar caption with the corresponding object name filled in, such as ' a photo of <object label> ' [2]. Intuitively, once the model focuses more on invariant features, it will have a better zero-shot classification accuracy across different distributions. Nevertheless, in the following theorem, we justify that CLIP remains to learn to use spurious features, aligning with our experimental observations on the CounterAnimal dataset.

̸

Theorem 1. Given a multi-modal dataset (Def. 1) with suitable variance in the features σ inv = Θ(1) > σ spu , and spurious features with a large spurious correlation p spu = 1 -o (1) , an overparameterized CLIP model where n = ω (1) , d M = Ω( n ) and d T = Ω( n ) , if the spurious features (e.g., backgrounds of the image) takes up a relatively large amount of the image µ spu ≥ σ 2 inv +2 2 ≥ µ inv = 1 , then with a high probability of at least 1 -O ( 1 poly ( n ) ) = 1 -o (1) , the CLIP model achieves a large error in zero-shot accuracy in the OOD test data where a = y :

and a small error in the OOD test data where a = y :

where κ 1 = σ 2 inv +2 -2 µ spu p spu √ (1+ σ 2 inv ) 2 σ 2 inv +(2 µ spu p spu -1) 2 σ 2 spu , κ 2 = -2 µ spu p spu -σ 2 inv √ (1+ σ 2 inv ) 2 σ 2 inv +(2 µ spu p spu -1) 2 σ 2 spu and Φ denotes the CDF of a standard normal distribution.

Photos of ice bears

in snow

background (common, accu 97.62)

in

A photo of

airplane

in

background

A photo of

airplane

background

A photo of

bird

in

background

A photo of

bird

in

background

Training Data

Test Data

Photos of ice bears

in grass

zero-shot

supervised

obj

objbkg

69.7

94.4

95.7

94.1

68.3

5.3

0.8

0.2

ID

OOD

(a)

OpenAI-RN-50

zero-shot

supervised

obj

objbkg

73.1

94.8

97.9

97.1

71.2

1.3

21.0

1.7

(b)

OpenAI-ViT-B/16

We leave more theoretical details as well as the proof to Appendix D due to space limit. Intuitively, Theorem 1 implies that once there exists a relatively strong correlation between the object captions and the parts of image backgrounds, CLIP will learn to align the backgrounds, i.e., spurious features, with object captions. Although our theory discusses a simplistic case of one invariant and one spurious feature, there could exist more features describing the objects and even more features describing the backgrounds. CLIP will fail to robustly align the visual features of objects to its captions, once there exists a spurious correlation between any of the background features with the object caption. Our theory is the first to provably demonstrate the drawbacks of CLIPs in OOD generalization, providing the foundation for future developments tackling the issue.

To verify our theory, we construct multi-modal datasets named ColoredCOCO following [51]. It contains 9 classes and the spurious correlation in the training part is 80% , i.e., each class has a correlation of 80% to a specific biased color and 20% uniformly correlates to 10 different randomly chosen colors, cf., Figure 10. The OOD datasets are built with classes randomly correlating to other 8 biased colors. We consider two prompts with different descriptiveness: a) obj: ' a photo of <object label> ' and b) objbkg : ' a photo of <object label> in <color label> background ', with either objects or both objects and backgrounds.

We tune the pre-trained CLIP models using the CLIP objective, which has been shown to be most robust to distribution shifts [52]. In addition, we also incorporate the baseline of full fine-tuning with a new MLP onto the image encoder using the ERM objective. As shown in Figure 11, fine-tuning with CLIP objective based on neither of the prompts provides any non-trivial robustness against the vanilla full fine-tuning. The results further verify our theory. Nevertheless, the degraded robustness of CLIP could also be caused by the weak language understanding capability of the BERT encoder in the CLIP. To this end, we also conduct additional experiments with a perfect language encoder setting. The results are given in Appendix D.4. Nevertheless, we find that CLIP still performs similarly to ERM and is prone to distribution shifts even with perfect captions.

5 Conclusion

In this paper, we highlight biases in previous evaluations for assessing the robustness of CLIP models, primarily relying on ImageNet variants. Such improper benchmarking would cause illusions that CLIP models seem to resolve spurious correlations, particularly in comparison with ImageNet models. It motivates us to craft the new testset, named CounterAnimal , which is specifically designed to probe the natural spurious correlations between animal and their backgrounds. The spuriousness captured by CounterAnimal is general across different CLIP setups and exerts relatively small impacts on the ImageNet benchmarks, thereby specifically capturing the spurious correlations within CLIP setups. Our experiments on CounterAnimal show that many conventional strategies, e.g., increasing backbone scales and improving pre-train data quality, remain effective in enhancing the robustness of CLIP models. Moreover, we present a theoretical analysis for the reasons of the CLIP objective to learn biases. Overall, we provide a platform for future developments of more advanced and robust CLIP and vision-language models, and we hope our presented experiments can offer a sober look at the robustness of CLIP models to spurious correlations.

Acknowledgments

The authors would like to express their sincere gratitude to the anonymous reviewers and the area chairs for their thorough review and constructive feedback. Their insightful comments and valuable suggestions have significantly enhanced the quality and clarity of this manuscript. We deeply appreciate their time and effort in helping us improve our work.

References

Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. Vision-language models for vision tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML , 2021.

Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion5b: An open large-scale dataset for training next generation image-text models. In NeurIPS , 2022.

Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, et al. Datacomp: In search of the next generation of multimodal datasets. arXiv preprint arXiv:2304.14108 , 2023.

Zhouxing Shi, Nicholas Carlini, Ananth Balashankar, Ludwig Schmidt, Cho-Jui Hsieh, Alex Beutel, and Yao Qin. Effective robustness against natural distribution shifts for models with different training data. In NeurIPS , 2023.

Alex Fang, Gabriel Ilharco, Mitchell Wortsman, Yuhao Wan, Vaishaal Shankar, Achal Dave, and Ludwig Schmidt. Data determines distributional robustness in contrastive language image pre-training (CLIP). In ICML , 2022.

Shibani Santurkar, Yann Dubois, Rohan Taori, Percy Liang, and Tatsunori Hashimoto. Is a caption worth a thousand images? a study on representation learning. In ICLR , 2023.

Imant Daunhawer, Alice Bizeul, Emanuele Palumbo, Alexander Marx, and Julia E. Vogt. Identifiability results for multimodal contrastive learning. In ICLR , 2023.

Yihao Xue, Siddharth Joshi, Dang Nguyen, and Baharan Mirzasoleiman. Understanding the robustness of multi-modal contrastive learning to distribution shift. arXiv preprint arXiv:2310.04971 , 2023.

Prasanna Mayilvahanan, Thaddäus Wiedemer, Evgenia Rusak, Matthias Bethge, and Wieland Brendel. Does CLIP's generalization performance mainly stem from high train-test similarity? In NeurIPS 2023 Workshop on Distribution Shifts: New Frontiers with Foundation Models , 2023.

Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, et al. Robust fine-tuning of zero-shot models. In CVPR , 2022.

Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.02893 , 2019.

Xiao Zhou, Yong Lin, Weizhong Zhang, and Tong Zhang. Sparse invariant risk minimization. In International Conference on Machine Learning , pages 27222-27244. PMLR, 2022.

Yong Lin, Hanze Dong, Hao Wang, and Tong Zhang. Bayesian invariant risk minimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16021-16030, 2022.

Xiao Zhou, Yong Lin, Renjie Pi, Weizhong Zhang, Renzhe Xu, Peng Cui, and Tong Zhang. Model agnostic sample reweighting for out-of-distribution learning. In International Conference on Machine Learning , pages 27203-27221. PMLR, 2022.

Yong Lin, Fan Zhou, Lu Tan, Lintao Ma, Jiameng Liu, Yansu He, Yuan Yuan, Yu Liu, James Zhang, Yujiu Yang, et al. Continuous invariance learning. arXiv preprint arXiv:2310.05348 , 2023.

Yong Lin, Lu Tan, Yifan Hao, Honam Wong, Hanze Dong, Weizhong Zhang, Yujiu Yang, and Tong Zhang. Spurious feature diversification improves out-of-distribution generalization. arXiv preprint arXiv:2309.17230 , 2023.

Xiaoyu Tan, Lin Yong, Shengyu Zhu, Chao Qu, Xihe Qiu, Xu Yinghui, Peng Cui, and Yuan Qi. Provably invariant learning without domain information. In International Conference on Machine Learning , pages 33563-33580. PMLR, 2023.

Yongqiang Chen, Kaiwen Zhou, Yatao Bian, Binghui Xie, Bingzhe Wu, Yonggang Zhang, MAKAILI, Han Yang, Peilin Zhao, Bo Han, and James Cheng. Pareto invariant risk minimization: Towards mitigating the optimization dilemma in out-of-distribution generalization. In The Eleventh International Conference on Learning Representations , 2023.

Yongqiang Chen, Wei Huang, Kaiwen Zhou, Yatao Bian, Bo Han, and James Cheng. Understanding and improving feature learning for out-of-distribution generalization. In Advances in Neural Information Processing Systems , 2023.

Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Josh Tenenbaum, and Boris Katz. Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models. NeurIPS , 2019.

Joshua Vendrow, Saachi Jain, Logan Engstrom, and Aleksander Madry. Dataset interfaces: Diagnosing model failures using controllable counterfactual generation. arXiv preprint arXiv:2302.07865 , 2023.

Viraj Prabhu, Sriram Yenamandra, Prithvijit Chattopadhyay, and Judy Hoffman. Lance: Stresstesting visual models by generating language-guided counterfactual images. NeurIPS , 2023.

Xiaodan Li, Yuefeng Chen, Yao Zhu, Shuhui Wang, Rong Zhang, and Hui Xue. Imagenet-e: Benchmarking neural network robustness via attribute editing. In CVPR , 2023.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR , 2009.

Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In CVPR , 2021.

Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In ICML , 2019.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020.

Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592 , 2023.

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurIPS , 2023.

Alex Fang, Albin Madappally Jose, Amit Jain, Ludwig Schmidt, Alexander Toshev, and Vaishaal Shankar. Data filtering networks. arXiv preprint arXiv:2309.17425 , 2023.

Yu Yang, Besmira Nushi, Hamid Palangi, and Baharan Mirzasoleiman. Mitigating spurious correlations in multi-modal models during fine-tuning. arXiv preprint arXiv:2304.03916 , 2023.

Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. arXiv preprint , arXiv:2305.10355, 2023.

Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. arXiv preprint arXiv:2401.06209 , 2024.

Qi Zhang, Yifei Wang, and Yisen Wang. On the generalization of multi-modal contrastive learning. In ICML , 2023.

Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. NeurIPS , 2019.

Kai Xiao, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. Noise or signal: The role of image backgrounds in object recognition. ArXiv preprint arXiv:2006.09994 , 2020.

Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. Measuring robustness to natural distribution shifts in image classification. NeurIPS , 2020.

Vaishaal Shankar, Achal Dave, Rebecca Roelofs, Deva Ramanan, Benjamin Recht, and Ludwig Schmidt. Do image classifiers generalize across time? In ICCV , 2021.

Yong Lin, Shengyu Zhu, Lu Tan, and Peng Cui. Zin: When and how to learn invariance without environment partition? NeurIPS , 2022.

Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. arXiv preprint arXiv:2007.01434 , 2020.

Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731 , 2019.

Gregory Plumb, Marco Tulio Ribeiro, and Ameet Talwalkar. Finding and Fixing Spurious Patterns with Explanations. arXiv preprint arXiv:2106.02112 , 2021.

XLi, Y Chen, Y Zhu, S Wang, R Zhang, and H ImageNet-E Xue. Benchmarking neural network robustness via attribute editing. In CVPR , 2023.

Chenshuang Zhang, Fei Pan, Junmo Kim, In So Kweon, and Chengzhi Mao. Imagenet-d: Benchmarking neural network robustness on diffusion synthetic object. In CVPR , 2024.

Yann Dubois, Yangjun Ruan, and Chris J Maddison. Optimal representations for covariate shifts. In NeurIPS 2021 workshop on distribution shifts: connecting methods and applications , 2021.

Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In ICLR , 2020.

Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image learning. In CVPR , 2023.

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. arXiv preprint arXiv:2306.05685 , 2023.

Ryumei Nakada, Halil Ibrahim Gulluk, Zhun Deng, Wenlong Ji, James Zou, and Linjun Zhang. Understanding multimodal contrastive learning and incorporating unpaired data. In AISTAT , 2023.

Faruk Ahmed, Yoshua Bengio, Harm van Seijen, and Aaron Courville. Systematic generalisation with group invariant predictions. In ICLR , 2021.

Sachin Goyal, Ananya Kumar, Sankalp Garg, Zico Kolter, and Aditi Raghunathan. Finetune like you pretrain: Improved finetuning of zero-shot vision models. In CVPR , 2023.

Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. Minigpt-v2: large language model as a unified interface for vision-language multi-task learning. arXiv preprint arXiv:2310.09478 , 2023.

Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An investigation of why overparameterization exacerbates spurious correlations. In ICML , 2020.

A Broader Impacts and Limitations

The current community often overestimates the robustness of CLIP models, largely due to the potentially misleading reliance on ImageNet variants for testing. To address this issue, we propose a new testset, named CounterAnimal , specifically tailored for CLIP models. Our findings indicate that CLIP models may not be as robust to distribution shifts as previously believed. Our dataset serves as a real-world benchmark, poised to be meaningful for the subsequent works to understand and enhance CLIP concerning their OOD robustness. For real-world applications, the understanding of spurious correlations for CLIP is also critical. We raise practical concerns when deploying CLIP models, which pertain to fairness and potential biases that may arise from inherent spurious correlations. We also present general strategies and theoretical analysis to understand the spurious correlations within CLIP models, which may motivate subsequent works to further enhance CLIP in real-world applications. However, although our dataset reaches the bar as a standard evaluation dataset, its research potential can be further benefited from expanding the scale of our dataset, diversifying the raw data sources beyond iNaturalist, broadening the semantic scope beyond animal classes, and studying other testbeds beyond the ImageNet benchmarks. In the future, we will extend our focus beyond animal subjects and include a wider array of high-quality data that are suitable for evaluating the robustness of CLIP and more advanced LVLMs.

B Dataset Composition

We release our dataset CounterAnimal structured as follows:

CounterAnimal

ostrich

easy-ground

figure1.jpeg

figure2.jpg

...

hard-water

figure1.jpeg

figure2.jpg

...

brambling

easy-green

figure1.jpeg

figure2.jpg

...

hard-sky

figure1.jpeg

figure2.jpg

...

...

Overall, the CounterAnimal dataset is organized by the object names. The data therein are further separated into two parts, i.e., the easy and hard groups, where the background name is also provided for each sub-directory. By evaluating accuracy with respect to the easy and hard groups, one can quantify the impacts of the spurious correlations captured by CounterAnimal . Wefurther summarize the ImageNet animal objects as well as the group names for the easy and hard groups in Table 5.

C Experimental Configurations

In this section, we provide more details about our experimental configurations.

C.1 Hardware Configurations

All experiments are realized by Pytorch 1 . 81 with CUDA 11 . 1 , using machines equipped with GeForce RTX 3090 GPUs and AMD Threadripper 3960 X Processors.

C.2 Candidate Label Space

We consider two different label spaces of candidate labels: a) using the full ImageNet-1K class names and b) using the top-20 most confusing classes for more computing-intensive models like MiniGPT4. It leads to the following two evaluation setups, i.e., the 1 vs. 1000 setup and the 1 vs. 20 setup.

1 vs. 1000 Setup. As a default option, we use the full label space of the ImageNet-1K dataset, which is suitable given that the object labels for CounterAnimal all belong to that of the ImageNet-1K dataset. Furthermore, this choice also reflects a more realistic situation in the open world, where we have a vast number of candidate labels and the failure cases of LVLMs are common.

1 vs. 20 Setup. To suit more advanced LVLMs of which the inference costs are much higher than CLIP models, we constrain the sizes of candidate label space for each class. Specifically, based on CLIP-LAION400M-ViT-B/32 , we select the top-20 most confusing labels, which is calculated by the average cosine similarity for both the easy and hard groups.

C.3 Evaluation Metrics

Now, we discuss the evaluation metrics. Typically, they are applied to the easy and hard groups separately when we evaluate the robustness of various models.

Class-wise Accuracy. We are interested in the effects of spurious features for each class. Therefore, we calculate the prediction accuracy specifically for photos within each class. It can be referred to as the class-wise accuracy, which is given by

where I label is the indices of photos belonging to label and ˆ y i is the predicted label for the i -th image. The class-wise accuracy reflects the class-level model reliability against spurious correlations.

Average Accuracy. Upon the class-wise accuracy, we can calculate the average performance of models, namely,

Compared to the conventional average accuracy, i.e., 1 |I| ∑ i ∈I 1 { ˆ y i = gt } with I the image indices and gt the true labels, our definition of the average accuracy further offsets the impact of class

imbalance. We default to using the average accuracy, and present the results without balancing in Tables 17-18 for CLIP and ImageNet models.

Accuracy Drop. To quantify the spurious correlations that make CLIP models fail, we measure the performance drop when moving from the easy and hard groups. At the class level, the accuracy drop is defined as the class-wise accuracy of easy minuses that of hard . At the dataset level, it is the average value for the class-level accuracy drop.

C.4 Evaluation Details of MiniGPT4 and LLaVA

To evaluate LVLMs with a backend of language models, we follow the common practice that constructs questions to prompt LVLMs [29, 30]. Specifically, we construct the question as:

and then calculate the language modeling loss with respect to the answer:

for each ImageNet class name. Meanwhile, we also try another question prompt that is widely used in training MiniGPT4 and LLaVA [30, 53]:

Describe this image in detail.

while the performance will generically decrease. In addition, when we switch to the object-centric evaluation protocol as [33]:

and evaluate the answer with Yes for each class, we observe a severe performance decrease as LVLMs easily hallucinate the objects. If we strictly follow the evaluation metrics of [33] by simply fetching the answers instead of comparing the losses, there exist lots of hallucinated objects by LVLMs in our dataset.

C.5 CLIP Naming Rules

For the CLIP checkpoints, we adopt the naming rule of ' CLIP-<dataset>-<backbone> ', where <dataset> is the name of pre-train datasets and <backbone> is the specific name of backbone models. For example, CLIP-LAION400M-ViT-B/32 indicates the ViT-B/32 model CLIP-trained on LAION400M . Different training setups are considered in OpenCLIP, and the versions of the adopted checkpoints are summarized in Table 6. Moreover, in Table 15, we consider the results of checkpoints beyond the adopted ones.

D Theoretical Understanding of CLIP's Robustness to Spurious Features

We provide a more detailed setup and analysis in complementary to Section 4.

D.1 Detailed Theoretical Setup

We begin by introducing more details about the data generation process following the literature [9, 50, 54].

Definition 2 (Multi-modal Dataset) . Consider n image-text pairs { ( x i I , x i T ) } n i =1 , both image x i I ∈ R d I and text x i T ∈ R d T are generated from the underlying latent factor z i ∈ R l . The samples are generated as follows:

z = [ z inv , z spu ] ∈ R 2 is composed of a invariant feature z inv ∼ N ( µ inv y, σ 2 inv ) and a spurious feature z spu ∼ N ( µ spu a, σ 2 spu ) with Pr( a = y ) = p spu otherwise a = -y , y is the label uniformly drawn from {-1 , 1 } , D tr is drawn with 1 / 2 ≤ p spu ≤ 1 while the OOD test data D ∗ is drawn uniformly with p spu = 1 / 2 .

Given z , the x at modality M is generated via x M = D M µ M ( z )+ ξ M , with D M ∈ R d M × l and ξ M ∼ N (0 , σ 2 ξ /d m I d m ) . The matrix D M ∈ R d m × l with d m > l is a matrix with orthonormal columns which can be considered as a dictionary matrix.

With the definition, we can write every z i = [ y i + η 1 ,i µ spu p spu + η 2 ,i ] where η 1 ,i , η 2 ,i are two Gaussian variables in the definition.

CLIP Training. We employ two linear encoders g I : R d I → R h for the image modality and g T : R d T → R h for the text modality, implemented as g I ( x I ) = W I x I and g T ( x T ) = W T x T with W I ∈ R h × d I and W T ∈ R h × d T , respectively. The encoders are trained through the linearized contrastive loss [9, 50] that mimics CLIP training dynamics:

̸

̸

where s ij = g I ( x i I ) T g T ( x j T ) is the similarity with respect to the i -th image and j -th text representations, and || W T I W T || 2 F is the a regularization term with ρ > 0 .

Zero-shot Inference. Once the CLIP model ( g I , g T ) is trained, the performance will be measured in a zero-shot manner by matching the most similar caption such as 'a photo of {object name}' across different object name as class names. Meanwhile, one could also leverage several prompts and leverage the average text embeddings across the available prompts to facilitate the evaluation [2]. The prompt with respect to y could be modeled as p y = D T E [ z t | y ] , where D T is the prompt transformation matrix. Then, the zero-shot accuracy of CLIP could be formalized as follows:

while the error is Err ( g I , g T ) = 1 -Acc ( g I , g T ) . Intuitively, once the model extracts more of the invariant features, it will have a better zero-shot classification accuracy across different distributions.

D.2 Proof for Theorem 1

Theorem 2 (Restatement of Theorem 1) . Given a multi-modal dataset (Def. 2) with suitable variance in the features σ inv = Θ(1) > σ spu , and spurious features with a large spurious correlation p spu = 1 -o (1) , an overparameterized CLIP model where n = ω (1) , d M = Ω( n ) and d T = Ω( n ) , if the spurious features (e.g., backgrounds of the image) takes up a relatively large amount of the image µ spu ≥ σ 2 inv +2 2 ≥ µ inv = 1 , then with a high probability of at least 1 -O ( 1 poly ( n ) ) = 1 -o (1) , the CLIP model achieves a large error in zero-shot accuracy in the OOD test data where a = y :

̸

and a small error in the OOD test data where a = y :

where κ 1 = σ 2 inv +2 -2 µ spu p spu √ (1+ σ 2 inv ) 2 σ 2 inv +(2 µ spu p spu -1) 2 σ 2 spu , κ 2 = -2 µ spu p spu -σ 2 inv √ (1+ σ 2 inv ) 2 σ 2 inv +(2 µ spu p spu -1) 2 σ 2 spu and Φ denotes the CDF of a standard normal distribution.

Proof. We will introduce some useful lemmas to help with our proof.

Lemma 1 ([9]) . The minimizer of linearized CLIP loss W ∗ T I W ∗ T satisfies the following with a probability of at least 1 -O ( 1 poly ( n ) ) such that,

Intuitively, the lemma indicates the importance of the training distribution, that the minimizer of CLIP will converge to the data characteristics of the latent features of the training distribution.

Then, consider the case where the model is inferred onto a test sample with y = 1 , a = -1 . Then, with the aforementioned lemma, we have

Then, notice that

When CLIP makes an incorrect prediction, we have

Then, we have

with Eq. 5 plugged in, denote ϵ 1 = O ( √ ϵ 0 log n ) , we further have

Since η 1 (1 + σ 2 inv ) + η 2 (2 µ spu p spu -1) is a Gaussian variable follows the distribution of

then, we have

where Φ is the CDF of the standard Gaussian distribution. Then, it suffices to know that the Err y =1 ,a = -1 is lower bounded by Φ( σ 2 inv +2 -2 µ spu p spu + ϵ 1 √ (1+ σ 2 inv ) 2 σ 2 inv +(2 µ spu p spu -1) 2 σ 2 spu ) , which also applies to the case y = -1 , a = 1 .

Similarly, given the case y = a , as the model fits the spurious feature, we could derive the lower bound for its Acc by leveraging the spurious features as Φ( -2 µ spu p spu -σ 2 inv √ (1+ σ 2 inv ) 2 σ 2 inv +(2 µ spu p spu -1) 2 σ 2 spu ) .

D.3 More Details on ColoredCOO Experiments

To further validate our theoretical results, we construct the ColoredCOO dataset following [51]. More specifically, ColoredCOO is constructed as follows:

The dataset contains 9 classes of COCO objects. The spurious correlation in the trainset is 80% such that each class has a correlation of 80% to a specific biased color and 20% uniformly correlates to 10 sufficiently different randomly chosen colors.

The OOD testsets are constructed with classes randomly correlating to 8 biased colors different from the one correlated in the training set.

Then, we further generate two prompts for each sample:

obj : a photo of <object label> ;

objbkg : a photo of <object label> in <color label> background

We tune the pre-trained CLIP models using the CLIP objective based on the OpenCLIP library. We consider the learning rate of { 1 e -3 , 1 e -4 , 1 e -5 } , with a weight decay of { 1 e -1 , 1 e -3 , 1 e -5 } , and a warmup of { 0 , 1000 } steps. We select the model according to the best in-distribution test performance. The detailed results are given in Table 7. As we can see, the CLIPs finetuned using either the CLIP objective or the standard supervised training both exhibit high sensitivity to the spurious features.

D.4 More Details on MultiColoredMNIST Experiments

One possible explanation for the failure of CLIP objective in ColoredCOCO is that, the language encoder of the CLIP models may not understand the captions well. Therefore, we further construct a new setup called MultiColoredMNIST , where each image contains only the digit information from MNIST dataset and the color information. Therefore, we can directly derive the one hot encoding for all of the useful factors in the dataset.

Data. We consider a multi-class classification setting with a number of classes no less than 2 . The objects are the

Training data: Fix two class (0/1) and color (r/g), they are spurious correlated by a correlation p spu . The invariant feature's correlation with labels is p inv .

Test data (Rand): All classes and the colors are randomly correlated, given k class, p spu = 1 /k .

Test data (Rev): All classes and the colors are reversely correlated, p spu is 10% 0/1 classes and 1 /k for others.

In Figure 12, we give some examples for the MultiColoredMnist dataset.

Experimental setting. We compare the standard supervised training and CLIP. To avoid noises or information loss in encoding language modality, we consider the perfect language supervision for a single model. Given a batch of image and caption representations { ( h x i , h c i ) } B i , for a image-caption pair, the CLIP objective aims to

̸

where M x ∈ R d × h x and M d × h c c are the learnable projection layers for image and caption representations. Assuming the perfect language encoding as the one-hot encoding for all possible object and background appearance h c i ∈ [0 , 1] |O| + |B| , and M c can simply be an identity matrix, then Eq. 9 can be considered as a classification task for objects and backgrounds respectively:

where the labels are simply the one-hot encodings of the objects and the backgrounds, and the classifier is M T c M x . For the MultiColoredMNIST task where there is only one object and background (i.e., color), to implement Eq. 10, we only need to construct an additional classification head for the background. Given the aforementioned setup, we conduct experiments comparing CLIP-based contrastive learning to the standard supervised learning. The results are given in Table 8. As we can see, both contrastive learning and supervised learning perform similarly across different numbers of classes and bias degrees.

E Ablation Studies

In this section, we present ablation studies to further validate the feasibility of our data curation process.

Biasing to ImageNet Setups. We follow the same curation procedure while using ImageNet models (i.e., ResNet50-ImageNet) to construct easy and hard splits, where we name the corresponding dataset as CounterAnimal-I . We present the results of CLIP and ImageNet models on CounterAnimal-I in Tables 9-10, respectively. The effective robustness is further shown in Figure 13. Contrary to the observations within original CounterAnimal , CLIP models can demonstrate better robustness against spurious features within CounterAnimal-I . It aligns with our expectation since different training data (e.g., LAION for CLIPs, and ImageNet for ImageNet models) follow different distributions and naturally contain different spurious features. It also demonstrates the generality of our data curation method to reveal the spurious features for different kinds of the models. We also list the background names for easy and hard splits with respect to some of the selected classes in Table 11. As observed, using different models to split data will capture very different spurious features. It highlights the necessity to curate an OOD testset for CLIP models, as CLIP models learn different spurious features than ImageNet models.

Correctness vs. Frequency. We further explain why easy and hard examples can characterize spurious features within CLIP setups. In general, spurious features can be caused by biases inherent in the data distribution concerning backgrounds. For example, for the animal class of ice bear , the background of ice is more common than other backgrounds, such as grass , thus causing spurious correlations learned by CLIP models. Therefore, we investigate in terms of the background frequency, employing the searching tool of Have I Been Trained 4 that can retrieve images from LAION5B closely matching a given class name. We examine 10 animal classes as our case studies. For each class, we collect the top 100 most relevant images and tally the occurrences of the backgrounds of our consideration. It is important to note that our counting process excludes cartoon images, irrelevant photos, corrupted photos, and those featuring multiple distinct animal subjects or ambiguous backgrounds. The results are summarized in Table 12. As we can see, in general, the spurious features captured align with our conjecture that hard examples contain uncommon backgrounds in the CLIP training data, e.g., LAION5B, further justifying the feasibility of our CounterAnimal in assessing the robustness of CLIP models in real-world situations.

(a) Backbones

(b) Datasets

(c) Data Quality

F More Results

In this section, we present more experimental results to support our claims.

Effective Robustness. In Section 3, we mainly examine the absolute robustness to assess and compare the OOD performance across various CLIP setups, which are well known to be sensitive to the original value scales. Therefore, in Figure 14, we apply the measures of effective robustness [5] to further substantiate our conclusions. Overall, our previous conclusions are upheld, demonstrating that the benefits derived from increasing model scales and enhancing data quality notably outweigh those obtained by merely expanding dataset sizes.

Top-5 Results for CLIP models. We present 1 vs. 1000 results for more CLIP checkpoints on the CounterAnimal dataset in Table 13, which is an extension of Table 2. Moreover, we present more results for the evaluations on CounterAnimal , supplementing our analysis of CLIP models under spurious correlations. To begin with, we report the top-5 scores under the 1 vs. 1000 setup, where we check if the target label is one of the top-5 model predictions. The results are summarized in Table 14. Comparing with the top-1 results in Table 13, we find that there is still a large performance gap between the easy and hard groups, indicating that the label confusion is quite diverse and not limited to the top two classes.

Other Versions of Pre-train Datasets. OpenCLIP provides other CLIP checkpoints beyond our adopted ones. Table 15 summarizes the results of CLIP models similar to Table 2 while using different versions of checkpoints. As we can see, the performance for both easy and hard is very stable across varying versions, except for DataComp1B . The reason is that their various checkpoints use subsets of DataComp1B , where XL indicates the fully DataComp1B , L indicates a 140M subset, M indicates a 14M subset, and S indicates a 1.4M subset.

Results of OpenAI Prompts. We further consider the prompt setups following OpenAI CLIP [2], using average text embeddings over 80 predefined prompts as the final text embeddings. The results are summarized in Table 16. As we can see, the average performance for both the easy and hard groups generally improves 1 to 3 percentage points over the results of our simpler prompt. However, our main conclusion remains unchanged: the ImageNet models generally exhibit better performance and smaller drops. Another interesting finding is that when evaluating with CLIP-LAION400M-ViT-B/32 (the CLIP checkpoint employed in our data collection), the performance drop with OpenAI prompts is not as high as that of our simple prompt used in Table 2. It indicates that our curation procedure mainly overfit the adopted prompt instead of the particular CLIP checkpoint.

Average Performance without Balancing. We by default adopt the balanced average accuracy to offset the impacts of class imbalance. In Tables 17-18, we further summarize the results without class balance, following 1 |I| ∑ i ∈I 1 { ˆ y i = gt } . As we can see, the performance drop remains obvious, and similar conclusions can be drawn as the balanced results: a) Backbone scales are more important for spurious robustness than pre-train dataset scales, and b) ImageNet models are more reliable when facing spurious features in CounterAnimal .

1 vs. 20 Results for CLIP and ImageNet Models. We adopt the 1 vs. 20 setup for the evaluations of more advanced LVLMs in Table 4. For a fair comparison, we further summarize the 1 vs. 20

results for CLIP models in Table 19 and for ImageNet models in Table 20. As we can see, there does not exist a significant change in performance drop compared to 1 vs. 1000 results, indicating that mistakes made by CLIP models are relatively concentrated. As in Figure 2, we also depict the easy versus hard performance for various learning setups with their names, following the 1 vs. 1000 setup in Figure 15 and 1 vs. 20 setups in Figure 16.

Class-wise Results. In Tables 21-22, we summarize the detailed results of the class-wise accuracy for the main results in Figure 5. We further depict the drop in accuracy in Figure 17. Generally speaking, the spurious features found in CLIP-LAION400M-ViT-B/32 can also fail other CLIP setups, and the general trends of decline are preserved class-wise. However, there are some cases where the drop in accuracy between easy and hard is negative, e.g., for data in class ID 33 and 42. It means that for these cases, our collection pipeline may have a large overfit to the adopted CLIP setup, i.e., CLIP-LAION400M-ViT-B/32 .

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

class ID

22.1

29.1

14.1

33

45.9

25.3

55

24.9

23.2

56.6

45.4

36.8

19.7

7.8

48.3

67.7

28.5

24.9

38.1

35.5

29.1

29.2

42.6

25.3

24.8

15.9

63

24.2

40.9

14.5

40.3

21.4

10.2

10.2

41.4

20.1

29 25 26.7

39.9

20.1

17.5 15 18.2

31.6

(a)

CLIP-LAION400M-ViT-B/32

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

class ID

6.8

17.8

10.2

19

8.8

20.6

34.7

25.2

36.5

53.9

33.5

32.3

37.2

10.5

39.5

68.6

4.1

13.4 9.9 7.6

30.2

30.7

34.8

20.8

9.4 3.8

24

37.8

24.3

14.2

14.8 11

3.23.1

40.3

11.2

16.9

19.3

20.1

25.8

25.5

-9.3

0.7

18.2

21.1

(b)

CLIP-LAION400M-ViT-B/16

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

class ID

2

18.3

7.2 4.1

15.3

15.6

12.1

13.7

14.8

56.5

18 16.4

9.9

19.1

67.4

37.2

24.4

18 20.5

5.2

14.3

19.6

42.4

3.8

12.2

34.6

6.7 9.2 12.1

5.2 0.51.4 6.45.5

27.3

15.4

10.7

15.1

9.1

43.6

23.9

-2.7

3.8

48.2

27.6

(c)

CLIP-LAION400M-ViT-L/14

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

class ID

6.8

13.3

0.9

16.9

20.4 19

42.6

14.8 19

44.5

23.6

40.8

20.1

12.1

48.9

55.6

25.1

27.4

7.6

19

37.4

30.4

26.9

24.1

34.1

13.2

18.3 19

31.6

31.5

23

11

5.8

13.5

56.2

18.6

31.5

24.1

19.7

30.6

31.7

3.8

8

44

22.4

(d)

CLIP-LAION2B-ViT-B/32

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

class ID

15.2

26.9

6.4

39.4

22.4

11.9

21.6

15.2

34.2

52.9

42.6

16.1

22.7

-0.4

44.3

64.3

21.5

48.9

26.2

35.9 40 36.2

28.4

10.1

12.6

14.8

8.7

46.6

32.3

52.5

22.5

16.4

-3.4

13.7

24.4

12

19.3

31.6

16.4

31.4

2.10.8

7 7.5 5.4

(e)

CLIP-OpenAI-ViT-B/32

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our main claim is that CLIP may not rely less on spurious features compared to ImageNet models. To support our claim, we curate a new evaluation dataset named CounterAnimal , specifically tailored for a comparative evaluation for the robustness of CLIP. Furthermore, we also conduct comprehensive experiments in Section 3 and theoretical justification in Section 4, both of which fully support our main position.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitation of our work in Appendix A, especially focusing on the further improvements for our dataset.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: In Definition 1, we clearly outline the assumptions made for the subsequent theoretical analysis presented in Theorem 1. In Appendix D, we present further details regarding our assumptions, proofs, and the associated experiments to validate their feasibility.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: In Section 2.1, we elaborate on the exact process used to construct the CounterAnimal dataset, including details about the raw data sources, the construction pipeline, and the dataset configurations. Moreover, for reproducibility, most of our experiments utilize open-sourced model checkpoints from the OpenCLIP and PyTorch repositories. We also comprehensively document the evaluation setups in Appendix C. We will release our dataset and the evaluation codes in the future.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We establish an anonymous repository for the access to our dataset, which can be found at the link of https://figshare.com/s/f9b0f34312168f4a8ddb . We will provide clearer and easier access instructions once the paper is accepted.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We utilize open-sourced model checkpoints from OpenCLIP and PyTorch. We collect the CounterAnimal dataset following the precise pipelines in Section 2.1. Moreover, we detail the evaluation setups of our experiments in Appendix C.

We utilize open-sourced model checkpoints from OpenCLIP and PyTorch; we have collected the CounterAnimal dataset according to the procedures outlined in Section 2.1; and we provide detailed descriptions of our evaluation setups in Appendix C.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: We utilize open-sourced model checkpoints without additional training or finetuning, and employ the complete dataset of CounterAnimal to conduct our experiments. There is no stochastic factor that may appear in our experiments, and thus we do not need to report the error bars for our results.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide details our hardware configurations in Appendix C.1. However, we do not list memory and time requirements since our experiments are not computationally demanding.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Our research conform with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discuss the broader impacts of our work in Appendix A, focusing on our impacts for both the research community and the real-world applications.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pre-trained language models, image generators, or scraped datasets)?

Answer: [Yes]

Justification: To avoid any safety risk, we control the label space for our CounterAnimal dataset and manually cleanse any unsafe image during our data collection procedure.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: For models, only open-sourced checkpoints are adopted, following the licenses from https://github.com/mlfoundations/open_clip/blob/main/LICENSE and https://github.com/pytorch/pytorch/blob/main/LICENSE . The versions of the adopted OpenCLIP checkpoints can be found in Table 6. For scraped data, we only use those photos from iNaturalist following the CC BY-NC license, which are allowed for scientific usage. Please refer to the following link for the details about CC BY-NC: https://creativecommons.org/licenses/by-nc/4.0/ .

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: Our CounterAnimal dataset follows the MIT license. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files, to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [Yes]

Justification: The CounterAnimal dataset is constructed with human labeling, yet without crowdsourcing. The labeling and filtering procedure adhere strictly to the pipeline outlined in Section 2.1. Our project does not involve any form of compensation or profit. All individuals involved in the manual data curation are from our team, and they all have been notified and agreed to grant consent to the use of the data in this work. this work.

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: [NA]

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors aim to investigate spurious correlations learned by CLIP models. For this, they curate a novel dataset where animals are organized into common and uncommon backgrounds, e.g. a polar bear is more likely encountered in snow than on grass. The authors then perform experiments where they benchmark various CLIP and ImageNet models on the curated dataset. They observe that CLIP models suffer from spurious correlations which stem from changing the background.\n\nSTRENGTHS:\nI think the issue of spurious correlations is important and one needs to understand how and whether VLMs learn spurious features. The paper presents many experiments and shows that scale or backbone capacity do not improve the effective robustness on CounterAnimal which is interesting.\n\nWEAKNESSES:\nThe paper has many issues, both in terms of writing and the methodology which need to be fixed.\n\n### Major:\n**The authors missed important previous works**: The paper “Recognition in Terra Incognita” is very related to this work and also proposes a “dataset designed to measure recognition generalization to novel environments” based on camera traps. The dataset is sorted according to difficult environments for different animals, which makes it very similar to CounterAnimal. I think the authors need to cite and discuss this paper. Currently, I do not understand the benefit of having a new dataset in addition to the already present one. The waterbirds dataset is also highly similar and should be discussed (https://arxiv.org/pdf/1911.08731). The authors cite that paper, but do not discuss it in the Related Work section, nor put it into context with CounterAnimal. The backgrounds challenge (https://github.com/MadryLab/backgrounds_challenge) is also highly related and should be discussed. In general, the related work section is very weak, given how extensively spurious correlations and worst-group-accuracy have been studied. Another important work to be discussed would be \"Finding and Fixing Spurious Patterns with Explanations\" (https://arxiv.org/abs/2106.02112).\n\n**The naming of the common vs counter groups is misleading**: \nLine 165: “Photos with the highest CLIP accuracy are assigned to the common group, and those with the lowest CLIP accuracy are assigned to the counter group.” I have a major understanding issue here. As far as I understood the paper before this line, the goal was to put images with common backgrounds into the common group and images with uncommon backgrounds into the counter group. This is also depicted in Fig. 1 or Table 1. The caption in Fig.1 says that “Most ice bears appear in a snow background (i.e., common), while it also is reasonable to find some ice bears in a grassy environment (i.e., counter)”. But here, the authors write that accuracy has actually been used to separate images into these groups? But then the frequency of the co-occurrence of certain backgrounds and classes has not been taken into account, or rather, it is a conjecture that those backgrounds where the CLIP model has higher accuracy on are more “common”? \n\n**The terms \"effective robustness\" and \"robustness\" are used interchangeably which is wrong and confusing**:\nI think the paper conflates the terms “robustness” and “effective robustness” which is confusing. When looking at effective robustness plots, such as in Fig. 2, we are interested in the residual difference between the measured value and the value predicted by the linear fit. As I can see, all plotted markers (CLIP and ImageNet) lie on their respective linear fits, and none of the interventions, such as CLIP-DC or CLIP-DFN offer any effective robustness benefits. It is though true that the **absolute** robustness numbers are overall higher for the CLIP-DFN models, for larger models or models trained on more data. I am however confused by the authors discussion of this observation. On the one hand, they write that larger CLIP models are more robust but increasing the dataset size does not yield improvements. First, I am confused whether they mean “effective robustness” or “robustness” here. Second, I do not see the effect the authors are describing: Both more data and larger backbones have higher absolute robustness but the same effective robustness as the other models. The statement “CLIP models trained on high-quality data are more robust” is also confusing, because it is not clear whether “robustness” or “effective robustness” is meant. \n\n**Due to methodology issues, results on CLIP models cannot be compared to results on ImageNet models (or other advanced LVLMs):**\nLine 60: “d) Spurious discovering: preserving classes and associated data based on the decrease in zero-shot performance (i.e., evaluating based on pre-trained CLIP models without fine-tuning) when shifting the backgrounds.” This step is really unclear. Do the authors curate the dataset based on the zero-shot accuracy of a CLIP model? From the introduction and the abstract, it sounds like the authors want to benchmark the robustness of CLIP vs ImageNet models on this custom dataset. But then, it is strange that CLIP models also seem to be used during the curation process. After reading the more detailed description in line 156, I think the statements made in line 85 are misleading. The authors write “ImageNet models are more robust to spurious correlations captured by CounterAnimal” and “Compared with CLIP models (colored in blue), surprisingly, we find that ImageNet models exhibit a stronger robustness to the spurious correlations in CounterAnimal.” Given that CounterAnimal has been curated based on the performance drop of a CLIP model, I find it very unsurprising that CLIP models perform worse on it compared to ImageNet models. I think that if CounterAnimal had been curated based on an ImageNet-trained ResNet50, the trend would have been reversed. I think all statements comparing CLIP and ImageNet trained models on CounterAnimal need to be relaxed and I think that this comparison is quite meaningless because of the described selection bias. I think that the whole Section 3.3. is misleading for this reason and statements such as the following cannot be made given the methodology issues: “Surprisingly, we find that ImageNet models are more robust to spurious features in the CounterAnimal dataset. This finding may contradict the common belief [Radford et al., 2021, Shi et al., 2023] that the CLIP models tend to be more robust to spurious correlations than single-modal supervised learning.” Similarly, the conjecture paragraph from line 265 onwards is wrong and cannot be made. \n\nFor the same reason, the comparison to advanced LVLMs in line 273 onwards cannot be made.\n\nFigure 1: Are these examples cherry-picked or are they representative of the data points present in CounterAnimal? I am asking this, because of the Winoground dataset [A]. This dataset tests the compositionality of VLMs by forcing a model to match two captions to two respective images. Winoground has later been criticized because the two images in the choice process are not equally hard [B]. For example, the model needs to match “the glass is on the grass” and “the grass is in the glass” to the corresponding images. However, there is much more grass in the image matching to the first caption, and the model likely picks that image for both captions just because there is more grass and it makes the decision in a bag-of-words-manner. To summarize, Winoground did not control for object size, orientation and other confounders. In Fig.1, it appears that the main objects (the polar bears) are equal in size, so size could be excluded as a possible confounder? Did the authors consider this possibility, i.e. that the drop in performance could be explained by other differences in the images from the respective domains?\n[A] https://arxiv.org/abs/2204.03162\n[B] https://arxiv.org/abs/2211.00768\n\n### General:\nLine 25: please cite CLIP\n\nLine 64: “The resulting dataset covers a total of 45 animal classes, ends up with 7,174 common photos and 5,926 counter photos, aligning with the standard size as an evaluation dataset [Recht et al., 2019, Hendrycks et al., 2021].” -> I do not understand this statement. Different test sets have different numbers of test images. ImageNet Val has 50k images for example. In what sense are the presented numbers standard?\n\n\nLine 94: “Overall, larger CLIP backbones (i.e., larger markers) can improve the effective robustness, implying that scaling up backbones may enhance robustness against spurious features.” -> I do not see this in Fig. 2. The larger markers appear to be on the fitted line, same as the smaller markers. Effective robustness measures the difference with respect to the linear fit, and there is none for the larger CLIP backbones. Please clarify this point.\n\n\nLine 146: “feature noise involves severe feature corruptions” -> Please be more specific here. What do you mean with feature noise? Do features refer to animals features such as missing ears or such? Or to the images themselves?\n\nLine 147: “clarity issues arise when animal objects are not in major positions” -> unclear formulation: what is a major position? Do the authors mean that the animals are too small or not in the center of the image?\n\nLine 153: “Note that the class space of backgrounds as above is not entirely orthogonal with each other due to the inherent ambiguity of the real-world situations. Nevertheless, we try our best to discern the assigned background labels within each animal class.” -> This is unclear. How many images would be ambiguous? I could imagine that many images would have two backgrounds, such as e.g. grass and sky or snow and water. For example, the last image in Fig. 1 on the left has both snow and water. It is not clear to me that only picking the snow background and ignoring the water is correct here. Further, at least for CLIP, the caption can contain several background keywords.\n\nFurther, I imagine animals occur in all kinds of environments, but there are only two backgrounds for each animal. Were the other images also discarded?\n\nLine 214: “Therefore, we conclude that our CounterAnimal dataset possesses some realistic shifts that are generally contained in large-scale pre-training data, regardless of backbones.” This conclusion cannot be drawn from this experiment since the backbone has not been varied here.\n\n### Section 4:\nThe proposed experiment is very similar to the well-known ShiftMNIST [D] or ColoredMNIST [E] datasets, which test the influence of spurious correlations. The findings here are not novel and should be brought into perspective with previous work. I do not understand how Fig. 11 relates to the text. What is “supervised”, “obj”, “objbkg”?\n[D] https://arxiv.org/pdf/1811.00401\n[E] https://arxiv.org/pdf/1907.02893\n\n### Typos, grammar:\nThe quality of the text is poor on some occasions which makes reading and understanding the paper difficult. The manuscript would benefit from a round of proof-reading. Some statements and formulations should be made more precise.\nLine 32: “The performance boosts over ImageNet models seem to suggest that CLIP resolves distribution shifts and thus spark a rich discussion about its rationale.” Strange formulation. How can “distribution shifts be resolved”? Please rephrase for clarity.\n\nLine 112: “More specifically, [Yang et al., 2023] report that CLIP models may misaligned frequently co-occured objects with the corresponding texts.”\n\nLine 115: “[Tong et al., 2024] find that CLIP misaligned samples will further cause the hallucination of LVLMs.” I do not understand this statement, grammar errors.\n\nLine 132: “Meanwhile, many existing datasets, e.g., DomainBed and Wilds, do not have overlapped label space with ImageNet, making the comparison between ImageNet and CLIP models hard.” There is a version of DomainBed [C] where the dataset has been filtered to only include classes compatible with ImageNet, such that an evaluation of ImageNet models is possible out-of-the-box.\n[C] https://openreview.net/pdf?id=LiC2vmzbpMO\n\nLine 171: “Recalling that, when CLIP models resort to the shortcut of data, the model performance will heavily correlate with the backgrounds presented in the common group yet is compromised when coming to the counter group.” Grammar errors, I do not understand this sentence. What is “the shortcut of data”?\n\nLine 208: “It suggests that the CounterAnimal dataset captures some general spurious shifts that at least commonly present in the pre-train dataset of LAION400M.” grammar\n\nLine 213: “Here, the spurious features degenerate the zero-shot robustness of CLIP models trained on both LAION2B and by OpenAI.” Typo? “degenerate”?\n\nLine 243: “In Figure 7, we consider two pre-train datasets, namely, LAION2B and the close-soured data from OpenAI” typo\n\nLine 297: “Nevertheless, in the following theorem, we justify that CLIP remains learning to use spurious features, aligned with our experimental observations on the CounterAnimal dataset.” grammar\n\nStrange space break between line 310 and 311.\n\n# Summary of the review:\nWe could fix the naming convention from \"common\" and \"counter\" to something like \"hard\" and \"easy\" since accuracy has been used rather than frequency of certain backgrounds to classify backgrounds into certain groups. Based on my arguments below, I believe we cannot compare CLIP models to ImageNet models on the proposed dataset in any sensible way due to the introduced selection bias. I believe the very title of the paper is misleading since the posed question cannot be answered based on the methodology issues. But if we remove the claims about comparing ImageNet models and CLIP models, then, the main point of the paper is that there exist backgrounds which are harder for CLIP models, given certain classes, and other backgrounds which are easier. I don't think that this observation is particularly interesting on its own. The authors did not relate the hardness of the backgrounds to their frequency in the pretraining dataset or anything else. The observation that backgrounds matter is also not novel but quite well-known and the authors do not offer a solution. Further, the writing is quite poor and confusing on many occasions; I provided many examples of incorrect and confusing sentences below.\n\nQUESTIONS:\nI have written a very detailed review above. I expect clarifications with respect to the raised points, at the very least in the \"Major\" paragraph.\n\nLIMITATIONS:\nThe limitations discuss the comparison in performance of the CLIP vs ImageNet models which I believe cannot be made due to the methodological issues in this work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 1, "DATE": "2024-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents CounterAnimal, an evaluation dataset featuring two subsets: animals with common backgrounds and those with unusual backgrounds. The images were sourced from iNaturalist. Data with high CLIP accuracy are categorized as \"Common\",  while those with low CLIP accuracy are labeled as \"Counter\".  Results shows that CLIP models experience a greater accuracy drop compared to ImageNet models when tested on this dataset.\n\nSTRENGTHS:\n- This paper analyzes multiple factors affecting CLIP accuracy, including model size and training data quality.\n- The paper combines both experimental results and theoretical analysis. The analysis in Section 5 is interesting and novel.\n- The paper is well-written and easy to follow.\n\nWEAKNESSES:\n- The proposed dataset is not sufficiently robust to analyze the influence of spurious bias, as this is not the only difference between the common and counter datasets.\n  - To analyze the accuracy drop caused by spurious features such as background, the background should be the only difference between common and counter image pairs. Prior work [4,5] has proposed such datasets focusing on background.\n  - In the proposed dataset, other factors may influence the model accuracy gap besides background. For instance, as shown in Figure 1, the more varied gestures of ice bears on the right compared to the left could be a contributing factor to the accuracy drop.\n\n- Current experiments cannot conclusively show that ImageNet models generalize better than CLIP.\n\n   - As the common and counter groups are selected according to the CLIP accuracy (see line 165 in the paper),  they indicate easy and hard samples for CLIP.    Since ImageNet models have different training characteristics, it is natural that hard cases for these models may differ from those for CLIP, resulting in a smaller performance drop for ImageNet models. This result cannot support that ImageNet models are more robust than CLIP models.\n   -  The accuracy drop from common to counter group can be greatly influenced by the model used to divide the common and counter dataset. Using the combined proposed common and counter dataset, a new Common' and Counter' dataset can be created based on the accuracy of ImageNet models. What is the impact of this dataset division on the accuracy drop for different models?\n\n\n- Prior studies[1,2,3,4,5,6] have proposed datasets specifically to analyze the influence of background, which are not discussed in this work.  These datasets can be used for CLIP evaluation as they do not overlap with the CLIP training set. Additionally, creating datasets based on model accuracy in this work is similar to the approach in [6].\n\n[1] Noise or Signal: The Role of Image Backgrounds in Object Recognition.\n\n[2] Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models, NeurIPS 2019.\n\n[3] Dataset Interfaces: Diagnosing Model Failures Using Controllable Counterfactual Generation.\n\n[4] ImageNet-E: Benchmarking Neural Network Robustness via Attribute Editing, CVPR 2023.\n\n[5] LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Images, NeurIPS 2023.\n\n[6] ImageNet-D: Benchmarking Neural Network Robustness on Diffusion Synthetic Object, CVPR2024.\n\nQUESTIONS:\nPlease refer to the weaknesses.\n\nLIMITATIONS:\nThis paper has discussed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors create an evaluation dataset comprising two groups, one with animals in usual backgrounds (common group) and another with unusual backgrounds (counter group). They then evaluate a suite of models of different backbones, model sizes, and datasets. They find that CLIP models do poorly than ImageNet-trained models, and generally high quality data or bigger model size improves counter group accuracy.\n\nSTRENGTHS:\n1. The CounterAnimal dataset is a nice contribution that can be of value to the community.\n2. The authors have evaluated a number of models on the dataset and that too could be of value to the community.\n\nWEAKNESSES:\nPlease see questions for more information.\n\nQUESTIONS:\n1. **Biased dataset:** The dataset is split into to common and counter group using a CLIP model. Therefore by construction, the CLIP models will perform poorly and it is no surprise that the ImageNet-trained models do a bit better. One could construct a split of this dataset where ImageNet models do better than the CLIP models. If the dataset was collected in a model agnostic way then the conclusions could potentially be more interesting. \n\n2. **On the premise:** As such the premise or the primary question seems a bit vacuous. Models arguably learn different features and there will exist some type of evaluation where one does better than the other. But are there useful tasks/evaluations where ImageNet models are preferred over CLIP models? That is an interesting open question. This work doesn't necessarily start from there and create a benchmark that is supposed to represent a task. The authors rather create a biased dataset that by design make CLIP models perform poorly. Therefore the primary premise of the work seems erroneous. There is some value in the other evaluations so maybe the paper could be rewritten by positioning things differently. \n\n3. **Lack of novelty:** Keeping the primary result aside, other conclusions like that better datasets or model sizes improve robustness are not new. Please see [1, 2, 3, 4]. \n\n[1] [Geirhos 2021] https://proceedings.neurips.cc/paper/2021/hash/c8877cff22082a16395a57e97232bb6f-Abstract.html\n\n[2] [Idrissi 2022] https://arxiv.org/abs/2211.01866\n\n[3] [Fang 2022] https://arxiv.org/abs/2205.01397\n\n[4] [Nguyen 2022]  https://arxiv.org/abs/2208.05516\n\nLIMITATIONS:\nThe authors have addressed some limitations. For the rest, please see my questions block.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work asks one interesting question: \"Do CLIP models always generalize better than ImageNet models?\" Driven by this question, this work proposes a new benchmark dataset named CounterAnimal. This dataset consists of a) the common group: comprising animals in common backgrounds, and b) the counter group: including animals in plausible yet unusual backgrounds. The main idea is that the performance drops from the common to counter groups quantify the reliance on spurious background features for animal predictions. The main observation is that CLIP models exhibit notable performance drops when tested on the counter group. In comparison, ImageNet models can be more robust than CLIP models.\n\nSTRENGTHS:\n- It is always good to see a new and novel dataset proposed for evaluating CLIP and ImageNet-trained models. The proposed dataset CounterAnimal is complementary to existing datasets that cannot reflect the robustness of CLIP models to spurious correlations. \n\n- The dataset construction is well-presented. The statistics, curation, background labeling, and spurious discovery are well introduced in Section 2\n\n- The analysis around spurious correlation is good. This work tries to give insights from several aspects, such as pre-trained datasets, scaling up, and learning paradigms. The observations are sound to me.\n\nWEAKNESSES:\n-  I found the analysis of why CLIPs rely on spurious features interesting. However, I think the claim is somewhat \"obvious\": there exists a relatively strong correlation between the object captions and the parts of image backgrounds, CLIP will learn to align the backgrounds, i.e., spurious features. If the training dataset contains many examples of spurious correlations, then models will tend to be biased.\n\n- I am curious about why ImageNet models may not be so influenced by the spurious bias in CounterAnimal. Is this because the ImageNet training set does not have too many spurious correlation examples? Or ImageNet has a spurious bias but such bias is different from the one in CounterAnimal? Please provide a discussion or share some insights on this question. \n\n- This paper adopts absolute performance drop in Section 3.3. Such a metric may not be so robust. For example, model A drops from 40 to 39, and model B drops from 90 to 89. They drop the same but the I would say model B is better. Please comment on this, and discuss the metric of absolute performance drop.\n\nQUESTIONS:\n- ImageNet models are not so biased toward spurious correlations compared with CLIP models. Why? Is this because the ImageNet training set does not have too many examples that exhibit spurious correlations?\n\n- While I appreciate this work includes results on ColoredCOO and ColoredMINIST, some other spurious correlation benchmarks (e.g., WaterBirds) would be greater if they were also included.\n\nLIMITATIONS:\nThe dataset is proposed, so a discussion on the potential bias/ privacy is needed. I appreciate this work highlights the future improvement of expanding semantic scope, data source, and ImageNet testbed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}