Title: A Closer Look at the Robustness of Contrastive Language-Image Pre-Training (CLIP)
Venue: NeurIPS 2023
Accepted: yes
Keywords: CLIP
Abstract: Contrastive Language-Image Pre-training (CLIP) models have demonstrated remarkable generalization capabilities across multiple challenging distribution shifts. However, there is still much to be explored in terms of their robustness to the variations of specific visual factors. In real-world applications, reliable and safe systems must consider other safety measures beyond classification accuracy, such as predictive uncertainty. Yet, the effectiveness of CLIP models on such safety-related objectives is less-explored. Driven by the above, this work comprehensively investigates the safety measures of CLIP models, specifically focusing on three key properties: resilience to visual factor variations, calibrated uncertainty estimations, and the ability to detect anomalous inputs. To this end, we study $83$ CLIP models and $127$ ImageNet classifiers. They are diverse in architecture (pre)training distribution and training strategies. We consider $10$ visual factors (\emph{e.g.}, shape and pattern), $5$ types of out-of-distribution data, and $8$ natural and challenging test conditions with different shift types, such as texture, style, and perturbation shifts. Our study has unveiled several previously unknown insights into CLIP models. For instance, they are not consistently more calibrated than other ImageNet models, which contradicts existing findings. Additionally, our analysis underscores the significance of training source design by showcasing its profound influence on the three key properties. We believe our comprehensive study can shed light on and help guide the development of more robust and reliable CLIP models.

A Closer Look at the Robustness of Contrastive Language-Image Pre-Training (CLIP)

Abstract

Contrastive Language-Image Pre-training (CLIP) models have demonstrated remarkable generalization capabilities across multiple challenging distribution shifts. However, there is still much to be explored in terms of their robustness to the variations of specific visual factors. In real-world applications, reliable and safe systems must consider other safety objectives beyond classification accuracy, such as predictive uncertainty. Yet, the effectiveness of CLIP models on such safetyrelated features is less-explored. Driven by the above, this work comprehensively investigates the safety objectives of CLIP models, specifically focusing on three key properties: resilience to visual factor variations, calibrated uncertainty estimations, and the ability to detect anomalous inputs. To this end, we study 83 CLIP models and 127 ImageNet classifiers. They are diverse in architecture, (pre)training distribution and training strategies. We consider 10 visual factors ( e.g. , shape and pattern), 5 types of out-of-distribution data, and 8 natural and challenging test conditions with different shift types, such as texture, style, and perturbation shifts. Our study has unveiled several previously unknown insights into CLIP models. For instance, they are not consistently more calibrated than other ImageNet models, which contradicts existing findings. Additionally, our analysis underscores the significance of training source design by showcasing its profound influence on the three safety-related properties. We believe our comprehensive study can shed light on and help guide the development of more robust and reliable CLIP models.

1 Introduction

By leveraging natural language supervision, CLIP has made significant progress in enhancing the zero-shot capabilities of models, unleashing their potential for remarkable out-of-distribution generalization performance [1, 2]. For example, CLIP models perform zero-shot classification without being explicitly trained on the target dataset, and they exhibit strong robustness to challenging natural distributional shifts [3-7]. Understanding such behavior of CLIP models is critical for advancing the next generation of image-text foundation models. Current research on this topic has explored various aspects of CLIP models, including dataset creation [8], reproducible scaling law [9], fine-tuning approaches [10], and training distribution [11].

In this work, we conduct an in-depth analysis of the safety-related properties of CLIP models, with a particular emphasis on their robustness and reliability across diverse testing environments. Specifically, we delve into the three critical safety-related properties, namely: 1) robustness to visual factors, to assess whether CLIP models can maintain robust when encountering varying visual factors, such as pose, size, color, lighting, and occlusions; 2) out-of-distribution (OOD) detection, to evaluate the capability of CLIP models to detect instances with labels that are not part of the training distribution; and 3) predictive uncertainty, to investigate whether CLIP models can provide calibrated predictions that accurately reflect their uncertainty in different testing conditions.

Our research offers a comprehensive examination of advantages and drawbacks of CLIP models across various critical facets. Building on prior research that has highlighted robustness of CLIP, we further study their performance across specific factors such as pose and lighting. Additionally, although CLIP models have demonstrated their efficacy in detecting OOD data [12], it is still uncertain whether this ability remains consistent when the training distribution, fine-tuning datasets, model size, and architecture are altered. Moreover, beyond classification accuracy, it is also important to evaluate whether CLIP models offer reliable uncertainty estimation across various distributions.

In light of the aforementioned questions, we evaluate 51 zero-shot CLIP models with varying visual encoder architectures, training sources, and dataset sizes, as well as 32 ImageNet fine-tuned CLIP models. To establish a baseline, we compare these models against 127 ImageNet models without language-image pre-training. We examine 10 visual factors variations present in the ImageNet validation set [13], including object pose, lighting, and background, to assess models' visual factors-level robustness. As for OOD detection, we employ ImageNet as an in-distribution (ID) set following [12] and test on 5 types of OOD scenarios. Furthermore, to investigate the predictive uncertainty, we use a set of canonical ImageNet distributions, such as texture, style, and perturbation shifts. Below we present key observations and insights obtained from our study:

CLIP models are generally more robust than ImageNet classifiers on 6 visual factors. However, they can be less robust on factors like object pose; In addition, training distribution plays an important role in CLIP robustness against visual factors (Section 4.1).

CLIP models are biased towards shape when making predictions. However, we have also found that this bias diminishes after fine-tuning on ImageNet and becomes similar to other ImageNet models that are pre-trained on more data. (Section 4.2).

When trained on the same source, classification accuracy of CLIP models correlates with their OOD detection performance (Section 5).

CLIP models are not always more calibrated than other ImageNet models, which contradicts existing findings [14]. Our research highlights the impact of training data distribution and quantity on these observations (Section 6).

Compared to other groups of models, CLIP maintains reasonably good uncertainty estimates under distribution shifts after ID calibration with temperature scaling. (Section 6).

2 Related Work

Robustness focuses on investigating the resilience of machine learning models to various forms of distribution shift at test time. To this end, a commonly used approach is to introduce artificial transformations onto images, such as, style transfer [15], corruptions and perturbations [16, 17]. Moreover, many real-world datasets are introduced to assess model robustness under a natural distributional shift [3-7]. For instance, Idrissi et al. [13] propose ImageNet-X by relabelling ImageNet validation set to provide detailed labels for naturally occurring factors such as different pose, background and lighting, to identify models' underlying failure patterns.

OODdetection targets at identifying test data that do not belong to any of classes modeled in training distribution [18-20]. A large number of methods are proposed for deep learning models, including generative model-based methods [21-28] and discriminative model-based methods [29, 18, 30, 31, 20, 32]. For example, maximum softmax probability [18] is used as the metric to detect OOD samples. Moreover, the above approaches mainly study OOD detection for a task-specific model using only visual information. In contrast, as CLIP models enjoy popularity, zero-shot OOD detection [12], is proposed, where the objective becomes filtering out input from the task of disinterest.

Predictive uncertainty aims to classify images with calibrated prediction probabilities so as to match the empirical frequency of correctness [33, 34]. Several works improve uncertainty estimations through post-hoc calibration on validation sets [33, 34]. Moreover, some other works show calibration can be improved by directly applying methods, such as ensembling [35] and pre-training [36]. Ovadia et al. [37] point out that calibration methods become less effective under distribution shift. Minderer et al. [14] suggest that CLIP models are well-calibrated given its accuracy. Based on these observations, this work comprehensively studies the quality of predictive uncertainty given by CLIP.

3 Experimental Setup

3.1 Models of Interest

Contrastive language-image pre-training models: We use 51 zero-shot CLIP models (CLIP) and 32 ImageNet fine-tuned CLIP models (CLIP-FT) . They have different visual encoders, including slightly modified ResNet [38], ConvNeXt [39], and ViT [40]. There are two training sources (LAION [41] and WIT [1]) and multiple sizes of training datasets from 80 million to 2 billion. For the CLIP-FT models, the vision tower of CLIP is fine-tuned on ImageNet-1K. We consider two fine-tuning procedures, one directly fine-tuned on ImageNet-1K [42], and the other first fine-tuned on ImageNet-12K, a subset of ImageNet-22K before fine-tuning on ImageNet-1K. Unless specified, we use the default prompt template by [1] for zero-shot CLIP models.

Compared models: we use 127 ImageNet models with various architectures, including Convolutional Neural Networks ( e.g. , ResNet [38] and ConvNeXt [39]), Vision Transformers ( e.g. , ViT [40] and Swin [43]) and all-MLP architectures [44, 45] ( e.g. , MLP-Mixer [45]). Following [46], we divide them into three categories: (i) Standard Models. This group consists of models supervised on the ImageNet training set. (ii) Contrastive learning models. This category contains 8 models pretrained by contrastive learning. There are 6 training algorithms investigated, including InsDis [47], MoCo [48], SimCLR [49]; (iii) Pre-trained on more data. This group contains models pre-trained on a significantly larger dataset ( e.g. , ImageNet-21K) than the ImageNet training set. All the above models, including CLIP, are publicly available on TIMM [50] and OpenCLIP [51]

3.2 Test Sets and Metrics

Robustness. We first pinpoint failure patterns of models by testing on ImageNet-X [13], which is relabelling of ImageNet validation by 16 naturally occurring factors. This work mainly considers 10 factors labelled with a sufficient number of test samples: Pose , Background , Pattern , Color , Smaller , Shape , Partial View , Subcategory , Texture and Larger . The metric is accuracy, and high is better. We evaluate on cue-conflict stimuli and Stylized-ImageNet [15] to measure model bias towards shape.

OOD Detection. We use large-scale OOD detection benchmark which is build up on ImageNet: in-distribution (ID) ImageNet v.s. { iNaturalist [52], SUN [53], PLACES [54], TEXTURE [55] and ImageNet-O [7] } (OOD). The metrics are the area under the receiver operating characteristic curve (AUROC) and the higher is better; false positive rate (FPR@95) when true positive rate is at 95% and a lower score means better performance.

Calibration. We study ID and OOD datasets, where ImageNet validation is ID dataset and OOD datasets are: ImageNet-V2 [3], ImageNet-Rendition [5], ImageNet-Adversarial [7], ImageNetSketch [4], ObjectNet [6] and ImageNet-Vid-Robust [56]. Metrics are estimated calibration error (ECE) [57] and negative log likelihood (NLL). A lower ECE or NLL indicates better calibration.

3.3 Analytical Methodology

In our endeavor to understand the underlying factors that influence the performance of CLIP models, we delve into six primary aspects: 1) training distribution, evaluating the effect of data source; 2) model architecture, looking into the potential effects of different structural choices on model performance; 3) dataset quantity, probing the interplay between the amount of data available for training and the model's efficiency; 4) contrastive loss, understanding its specific role in training dynamics 5) fine-tuning, and 6) test-time prompt, assessing the impact of prompts during the evaluation on model outputs. We follow the analytical methodology of seminal work [46] and a series of following works like [8, 11, 58]) to study the influential factor. Specifically, within the performance trends observed across all models, any factor causing a deviation from these trends is recognized as influential. Notably, in our research, we mainly emphasize and discuss such influential factors within each facet of our investigation.

4 Visual Factor-Level Robustness

The unprecedented robustness of CLIP models has spurred intense research efforts to identify the underlying factors responsible for their performance under distribution shifts. Recent studies provide

Overall accuracy on ImageNet-X

Accuracy on the factor

Accuracy on the factor

CLIP trained on WIT

Linear fit (CLIP-WIT zero-shot)

Linear fit (CLIP-LAION zero-shot)

CLIP trained on LAION

Standard model

Pre-trained on more data

Contrastive learning model

CLIP Fine-tuned

𝑦 = 𝑥

Accuracy on the factor

Overall accuracy on ImageNet-X

Overall accuracy on ImageNet-X

Pattern

Shape

Subcategory

Background

Smaller

Texture

Partial view

Color

Larger

Pose

valuable insights on the design of training source [11, 8]. Our research builds upon previous findings on the robustness of CLIP models and focuses on the potential failure types of the model. Instead of solely measuring overall accuracy across distributions, we investigate the behavior of CLIP models when faced with varying visual factors such as Pose and Background .

4.1 CLIP Models Generally Exhibit Better Factor-Level Robustness Than Other Models

Factor-level effective robustness . In our study, we extend the concept of overall effective robustness [46] to visual factor-level. Specifically, it measures a model's ability to achieve higher accuracy on the subset annotated by a specific visual factor compared to what is expected based on its overall accuracy on ImageNet-X. Figure 1 displays the accuracy on the subset annotated by a specific visual factor relative to the overall accuracy on ImageNet-X.

CLIP models are generally more robust than other ImageNet models on 6 out of 10 visual factors. Figure 1 highlights several insights into the factor-level robustness of CLIP models. First, we find that CLIP models are more robust than other models on six out of ten visual factors, including Subcategory , Smaller , Color , Shape , Texture , and Larger . Specifically, CLIP models exhibit higher factor-level effective robustness than other models on each of these factors. Second, we observe that CLIP models are less robust than other models on Pose and Partial View factors. Third, CLIP models show a similar trend to other models on the Background factor. In addition, Idrissi et al. [13] observe that data augmentations can improve robustness to related factors, but with spill-over effects to unrelated factors. We speculate that the data augmentations used for training CLIP models may introduce the similar effects.

Training distributions lead to different trends in CLIP models. The choice of training distribution impacts factor-level robustness of CLIP models. Specifically, we find that training on different datasets ( i.e. , LAION and WIT) forms distinct trends on each visual factor for CLIP, and there is no single training source that always leads to higher factor-level robustness than another. For instance,

we observe that CLIP models trained on LAION demonstrate higher robustness on Shape factor than those trained on WIT, while this reverses for Background and Pose factors. The results show a mixed observation on Large factor. Furthermore, we further point out that CLIP models trained on different subsets of LAION i.e. , LAINON-80M, LAION-400M, and LAION-2B) follow the same trend. The above observations highlight the importance of the choice of training source in determining not only the overall accuracy but also the factor-level behaviors of CLIP models. This suggests that visual factor-level robustness should be considered when designing the training source for CLIP models.

CLIP fine-tuned models perform slightly better than more data pre-trained models. We compared CLIP fine-tuned models (CLIP-FT) with other models that are pre-trained on more data and find that CLIP-FT shows improvement in overall accuracy and robustness on visual factors of Subcategory , Shape , and Pattern . However, no additional robustness gain is observed on other factors. Additionally, CLIP-FT models outperform zero-shot CLIP on variations such as Pattern and Partial View , indicating their superiority in handling visual factors. It would be intriguing to explore fine-tuning techniques that maintain or improve the factor-level robustness of zero-shot CLIP.

4.2 Texture Bias v.s. Shape Bias

CLIP exhibits a shape bias. We conduct experiments using the cue-conflict stimuli dataset [15] to investigate the presence of shape bias in the model's predictions. Shape bias, in this context, refers to the proportion of accurate predictions made based on object shapes. Figure 2 presents a visualization of the shape bias exhibited by the models, which are grouped according to their training methods (zero-shot, CLIP finetuning, more data pre-trained, and standard training) and architecture (transformer versus CNN). Our findings indicate that, among the four training methods, CLIP models are more likely to make predictions based on shape compared to the other three groups. Furthermore, while the transformer is reported to have a stronger shape bias than CNN [60, 61], we observe that CLIP using CNN as the vision encoder also exhibit a strong shape bias.

Model size solely does not explain the shape bias of CLIP . We further observe that larger CLIP models do not necessarily have higher shape bias than smaller-size ones. For example, both trained on LAION-80M, CLIP-ViT/L-14 has 0 . 54 shape bias, which is 0 . 09 lower than CLIP-ViT/B-32. This implies that the shape bias of CLIP models cannot be attributed solely to model size. Based on the above observations, we speculate that the shape bias of CLIP may be attributed to its objective, which involves training the model to associate text and image pairs.

CLIP models have a tendency towards texture bias after fine-tuning . Our study reveals that shape bias in CLIP weakens after fine-tuning on ImageNet. Moreover, the fine-tuned CLIP models exhibit a shape bias comparable to models that are pre-trained on larger datasets. This finding is consistent when using transformer and CNN as visual encoder. Moreover, these results illustrate that fine-tuning discards the shape-biased property of zero-shot CLIP, which may affect model robustness [62, 15].

Larger input image resolution during fine-tuning of CLIP results in a stronger bias towards texture. In Table 1, we observe that an input resolution during fine-tuning is a important factor to shape bias: increasing the input resolution during fine-tuning leads to better performance on ImageNet

AUROC

FPR@95

iNaturalist

Accuracy on in-distribution dataset (ImageNet)

TEXTURE

PLACES

CLIP trained on WIT

Linear fit (CLIP-WIT zero-shot)

Linear fit (CLIP-LAION zero-shot)

CLIP trained on LAION

Standard model

Pre-trained on more data

Contrastive learning model

CLIP Fine-tuned-12k-1k

CLIP Fine-tuned-1k

ImageNet-O

AUROC

Accuracy on in-distribution dataset (ImageNet)

SUN

FPR@95

AUROC

Accuracy on in-distribution dataset (ImageNet)

FPR@95

validation but also results in more texture-biased models with lower accuracy on Stylized-ImageNet. Across seven pairs of experiments and two training sources, we observe this pattern consistently. Given that input resolution is a crucial model dimension [63-65], it would be interesting to study its effects on shape bias beyond classification accuracy when devising scaling strategies.

5 Out-of-Distribution Detection

Zero-shot CLIP allows for a flexible definition of in-distribution (ID) classes without re-training the model. Namely, they can conduct zero-shot OOD detection. The current findings suggest that zero-shot CLIP models are competitive with other state-of-the-art models [12, 66]. Based on this finding, we conduct an extensive analysis to determine whether the purported benefits persist across various training sources, subsets, and network architectures. In the experiments, for zero-shot CLIP models, we utilize maximum concept matching [12] to detect OOD data. For models that are trained or fine-tuned on ImageNet-1K, we employ maximum softmax score [18] for OOD detection.

For CLIP models from the same source, their ID accuracy correlates with OOD detection performance. Our study includes CLIP models trained on two sources (WIT and LAION). Given the same training source, our study, conducted across five challenging OOD scenarios, reveals a strong correlation between the ID accuracy of zero-shot CLIP models and their OOD detection performance (measured by AUROC and FPR@95). This observation suggests that the zero-shot classification accuracy of CLIP models on ID data can serve as a reliable indicator of their OOD detection performance. In contrast, such a trend is not as strong for standard models and more data-pre-trained models. Additionally, CLIP-FT models fine-tuned on ImageNet-1K do not exhibit such a clear correlation.

Training source influences the trend of CLIP. Upon closer examination of the training distribution, we have observed that the correlation trend between ID accuracy and OOD detection performance is

largely dependent on the training source. As illustrated in Figure 3, our research shows two distinct trends between CLIP models trained on WIT and those trained on LAION. Moreover, with the same ID accuracy, CLIP models trained on WIT exhibit superior OOD detection performance compared to their counterparts trained on LAION on three OOD scenarios. This further indicates the importance of training source selection for CLIP. When developing dataset curation methods, it is valuable to investigate the influence of training sources on OOD detection performance.

Fine-tuning procedure influences the OOD detection ability of CLIP. First, we point out that fine-tuning enhances classification performance of CLIP, but this improvement does not necessarily translate to better OOD detection accuracy. Some CLIP-FT models even achieve worse OOD detection performance than Zero-shot CLIP models. Our analysis of CLIP-FT reveals a distinction between two groups of CLIP-FT, based on their fine-tuning procedures: the first group is fine-tuned solely on ImageNet-1K, while the second group undergoes additional fine-tuning on ImageNet12K. We observe that this additional fine-tuning procedure has a substantial impact on the model's ability to detect OOD examples. As depicted in Figure 3, despite not leading to an improvement in classification accuracy, CLIP-FT models with additional fine-tuning on ImageNet-12K show better OOD detection performance across all OOD scenarios. As future work, it is valuable to investigate this observation further and explore alternative fine-tuning procedures that yield improved OOD detection performance. Moreover, exploring impacts of fine-tuning datasets other than ImageNet-1K and ImageNet-12K would be another interesting direction.

6 Prediction Uncertainty

In order to better understand the well-calibrated phenomenon of zero-shot CLIP models reported by Minderer et al. [14], our research systematically analyzes the calibration behavior of CLIP models under various training conditions. Specifically, we examine the calibration performance of CLIP models trained on different training distributions, varied training set sizes, and different architectures. Furthermore, we also investigate the calibration performance of CLIP models after fine-tuning to gain a better understanding of their overall performance.

6.1 Zero-Shot CLIP Models Are Not Consistently More Calibrated Than Other Models

Both training data distribution and quantity impact CLIP's calibration. Figure 4 presents the model calibration of CLIP models in relation to classification accuracy under distribution shifts. We observe that CLIP models trained on different distributions or quantities are not consistently grouped together. For example, CLIP models trained on WIT and LAION tend to cluster in separate regions. Moreover, when training CLIP models on different subsets of the LAION dataset, models with similar classification accuracy can exhibit varying levels of calibration performance. It would be interesting to further check the impacts of data curation techniques on CLIP calibration performance.

While CLIP models are generally reported to have superior calibration compared to other models [14], our observations reveal that this finding does not always hold. Particularly, we notice that CLIP models trained on LAION-80M dataset exhibit much lower calibration performance when compared to standard models. The observation of [14] is primarily made on CLIP models trained on WIT. However, when we broaden our perspective to include the alternative training distribution provided by LAION and its various subsets, our observations become varied. This emphasizes the significance of careful training source design for CLIP. Furthermore, it suggests that when evaluating dataset curation, it is crucial to consider its impact on the calibration performance of CLIP models.

CLIP fine-tuned models exhibit a trade-off between calibration and classification. On each test set in Figure 4, we consistently observe that after fine-tuning, CLIP models tend to have higher classification accuracy and lower calibration error. It is worth noting that additionally fine-tuning on ImageNet-12K does not alter this phenomenon, in contrast to its impact on OOD detection. Moreover, other model groups, including those pre-trained on more data, do not exhibit a trade-off between calibration and classification. We also observe some fine-tuned CLIP models achieve better calibration compared to their zero-shot counterparts before calibration.

Classification accuracy

ImageNet-R

ECE

NLL

ECE (Temp-scaled)

NLL (Temp-scaled)

ID test set

Classification accuracy

ImageNet-V2-A

Classification accuracy

ImageNet-A

Classification accuracy

Fine-tuned CLIP

Pre-trained on more data

Contrastive learning model

Standard models

CLIP trained on WIT

CLIP trained on LAION-400M

CLIP trained on LAION-Aesthetic

CLIP trained on LAION-2B

6.2 Temperature Scaling Reveals Well-Calibrated Properties of Zero-Shot CLIP Models

Post-hoc calibration can be adopted to remedy over- or under-confidence. Here, we use temperature scaling [33] to calibrate model predictions. Following the protocol in [67], we divide the validation set of ImageNet into two halves: one for temperature scaling (ID calibration set), and the other one for ID test. We report results on both ID and OOD test sets.

Calibration performance of CLIP models after temperature scaling (Temp-scaled) correlates with their classification accuracy. In Figure 4, we explore how temperature scaling affects different groups of CLIP models. These groups are categorized based on the amount and source of their training data. After applying temperature scaling and evaluating using the NLL metric, we observe a consistent pattern among these CLIP groups. What is intriguing is that, after temperature scaling, for models with similar image classification accuracy, zero-shot CLIP models achieve better calibration performance compared to other models, including their fine-tuned counterparts. This trend persists across various testing scenarios, encompassing ID, OOD, and when assessing calibration using both NLL and ECE metrics.

ID calibration of CLIP models transfers to OOD test sets . While it has been reported by Ovadia et al. [37] that ID calibration often struggles to generalize under distribution shifts, our study reveals

80 prompts

CLIP RN50

CLIP RN50

×

64

CLIP ViT/B-32

CLIP ViT/B-16

CLIP ViT/L-14-336

30 prompts

5 prompts

1 prompt

AUROC

Accuracy on ImageNet

SUN

Accuracy on ImageNet-X

Accuracy on Pattern

Pattern

ECE

ImageNet-V2-A

Accuracy on ImageNet-V2-A

ECE (Temp-scaled)

Linear fit (CLIP-WIT)

(a) Robustness

(b) OOD detection

(c) Predictive uncertainty

a promising phenomenon for CLIP models. Specifically, after calibrating CLIP models on ID calibration set, we observe improved calibration results on OOD test sets. For example, on ImageNetA, the calibration errors of CLIP models decrease after temperature scaling, unlike other models that do not clearly exhibit such improvement. This indicates that CLIP models are relatively easier to calibrate across diverse distributions, highlighting their potential for robust and reliable applications.

7 Discussion on Influence of Test Time Prompts

So far in the experiments, we utilize the prompt provided by [1]. In this section, we investigate the effect of test time prompts on three safety objectives. We additionally examine another three sets of prompts, one of size 1 : ' a photo of a {label} ', a set of 5 prompts used by [12], and a set of 30 prompts. We conduct the experiment on five CLIP models: RN50, RN50 × 64, ViT-B/16, ViT-B/32 and ViT/L-14-336px trained on WIT. Figure 5 shows the performance of CLIP models using different sets of prompts on three safety objectives.

We show that utilization of few prompts ( e.g. , one prompt) generally leads to a decrease in overall classification performance. However, the impact on the three properties is mixed. First, when evaluating factor-level robustness on Pattern, we find the adoption of different prompts does not alter the robustness: models still adhere to the linear trend established by CLIP trained on WIT using 80 prompts. Second, for OOD detection, using 5 prompts yield higher OOD detection performance than with 80 prompts on SUN. Also, for calibration, using fewer prompts ( e.g. , only one prompt) achieves lower calibration error than using all 80 prompts. This raises an important question: how to tune prompts to achieve better classification accuracy as well as better calibration and OOD detection performance? It would be interesting to study this when conducting prompt learning [68-70].

8 Conclusion and Discussion

Our research provides a valuable contribution to the ongoing discourse surrounding the effectiveness of CLIP models in the context of robustness to visual factors, OOD detection, and the reliability of uncertainty estimation. In pursuit of these objectives, we conduct extensive experiments encompassing three critical tasks and perform comparative analyses between CLIP models and various groups of models. Our observations provide valuable insights into the advantages of CLIP models. First, CLIP models demonstrate superior visual factor-level robustness compared to other ImageNet models. Furthermore, while maintaining comparable accuracy on in-distribution dataset, CLIP models also exhibit competitive performance in OOD detection across commonly used benchmarks such as iNaturalist and ImageNet-O. Lastly, CLIP models are relatively easier to calibrate across diverse distributions. Furthermore, our study highlights the significance of training source design, as it profoundly influences the behavior of CLIP models across all three objectives.

This work leaves open many interesting directions for future research and we discuss a few. First , this work primarily studies CLIP and its fine-tuned models due to its simplicity and effectiveness. We

reckon this work as an anchor point and hope the framework of this analysis could generalize to other image-text foundation models, such as ALIGN [2] and BASIC [71]. Second , our study includes two academic training sources, namely WIT and LAION, for CLIP. It is valuable to investigate whether our observations generalize to other training sources. We believe that our study can shed light to and build up the understanding towards the design of multi-modal datasets. Lastly , in addition to three safety-critical tasks, there are other important fields to analyze such as mis-classification samples detection [18]. By providing a more detailed and nuanced understanding of the performance of CLIP models, we hope our observation and insights can inform future developments in the field and help to drive progress towards more robust and effective vision-language models.

Broader impacts. While much research endeavors that aim to enhance the performance of machine learning models could be leveraged for negative societal applications, we believe that this paper points towards a positive direction for the development of machine learning methods in broader society. Particularly, our study understands the influence of training set on CLIP performance on robustness, out-of-distribution detection, and predictive uncertainty. A better understanding is beneficial in establishing trustworthy machine learning systems and high-quality multi-modal datasets for training.

Acknowledgement. We thank all anonymous reviewers and ACs for their constructive comments and valuable suggestions in improving this paper.

References

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning , pages 8748-8763. PMLR, 2021.

Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International Conference on Machine Learning , pages 4904-4916. PMLR, 2021.

Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International Conference on Machine Learning , pages 5389-5400. PMLR, 2019.

Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. In Advances in Neural Information Processing Systems , 2019.

Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 8340-8349, 2021.

Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Josh Tenenbaum, and Boris Katz. Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models. In Advances in Neural Information Processing Systems , 2019.

Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15262-15271, 2021.

Thao Nguyen, Gabriel Ilharco, Mitchell Wortsman, Sewoong Oh, and Ludwig Schmidt. Quality not quantity: On the interaction between dataset design and robustness of clip. In Advances in Neural Information Processing Systems , 2022.

Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2023.

Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, et al. Robust fine-tuning of zero-shot models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7959-7971, 2022.

Alex Fang, Gabriel Ilharco, Mitchell Wortsman, Yuhao Wan, Vaishaal Shankar, Achal Dave, and Ludwig Schmidt. Data determines distributional robustness in contrastive language image pre-training (clip). In International Conference on Machine Learning , pages 6216-6234. PMLR, 2022.

Yifei Ming, Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, and Yixuan Li. Delving into out-of-distribution detection with vision-language representations. In Advances in Neural Information Processing Systems , 2022.

Badr Youbi Idrissi, Diane Bouchacourt, Randall Balestriero, Ivan Evtimov, Caner Hazirbas, Nicolas Ballas, Pascal Vincent, Michal Drozdzal, David Lopez-Paz, and Mark Ibrahim. Imagenet-x: Understanding model mistakes with factor of variation annotations. In International Conference on Learning Representations , 2022.

Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. In Advances in Neural Information Processing Systems , pages 15682-15694, 2021.

Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations , 2018.

Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations , 2019.

Eric Mintun, Alexander Kirillov, and Saining Xie. On interaction between augmentations and corruptions in natural corruption robustness. In Advances in Neural Information Processing Systems , pages 3571-3583, 2021.

Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Representations , 2016.

Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A survey. arXiv preprint arXiv:2110.11334 , 2021.

Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. In Advances in Neural Information Processing Systems , pages 21464-21475, 2020.

Mu Cai and Yixuan Li. Out-of-distribution detection via frequency-regularized generative models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 5521-5530, 2023.

Polina Kirichenko, Pavel Izmailov, and Andrew G Wilson. Why normalizing flows fail to detect out-ofdistribution data. In Advances in Neural Information Processing Systems , pages 20578-20589, 2020.

Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don't know? In International Conference on Machine Learning , 2018.

Lawrence Neal, Matthew Olson, Xiaoli Fern, Weng-Keen Wong, and Fuxin Li. Open set learning with counterfactual images. In European Conference on Computer Vision , pages 613-628, 2018.

Poojan Oza and Vishal M Patel. C2ae: Class conditioned auto-encoder for open-set recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2307-2316, 2019.

Jie Ren, Peter J Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of-distribution detection. In Advances in Neural Information Processing Systems , 2019.

Joan Serrà, David Álvarez, Vicenç Gómez, Olga Slizovskaia, José F Núñez, and Jordi Luque. Input complexity and out-of-distribution detection with likelihood-based generative models. In International Conference on Learning Representations , 2019.

Zhisheng Xiao, Qing Yan, and Yali Amit. Likelihood regret: An out-of-distribution detection score for variational auto-encoder. In Advances in Neural Information Processing Systems , pages 20685-20696, 2020.

Abhijit Bendale and Terrance E Boult. Towards open set deep networks. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1563-1572, 2016.

Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10951-10960, 2020.

Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In International Conference on Learning Representations , 2017.

Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting outof-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems , 2018.

Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning , pages 1321-1330. PMLR, 2017.

Khanh Nguyen and Brendan O'Connor. Posterior calibration and exploratory analysis for natural language processing models. In Conference on Empirical Methods in Natural Language Processing , 2015.

Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems , 2017.

Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. In International Conference on Machine Learning , pages 2712-2721. PMLR, 2019.

Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems , 2019.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770-778, 2016.

Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11976-11986, 2022.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations , 2020.

Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade W Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa R Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. LAION-5b: An open large-scale dataset for training next generation image-text models. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track , 2022. URL https:// openreview.net/forum?id=M3Y74vmsMcY .

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2009.

Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10012-10022, 2021.

Xiaohan Ding, Chunlong Xia, Xiangyu Zhang, Xiaojie Chu, Jungong Han, and Guiguang Ding. Repmlp: Re-parameterizing convolutions into fully-connected layers for image recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision , 2022.

Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mlp-mixer: An all-mlp architecture for vision. In Advances in Neural Information Processing Systems , pages 24261-24272, 2021.

Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. Measuring robustness to natural distribution shifts in image classification. In Advances in Neural Information Processing Systems , pages 18583-18599, 2020.

Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via nonparametric instance discrimination. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 3733-3742, 2018.

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9729-9738, 2020.

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning , pages 1597-1607. PMLR, 2020.

Ross Wightman. Pytorch image models. https://github.com/rwightman/pytorch-image-models , 2019.

Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip, July 2021. URL https://doi.org/10.5281/zenodo.5143773 . If you use this software, please cite it as below.

Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 8769-8778, 2018.

Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Largescale scene recognition from abbey to zoo. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 3485-3492, 2010.

Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence , 40(6):1452-1464, 2017.

Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 3606-3613, 2014.

Vaishaal Shankar, Achal Dave, Rebecca Roelofs, Deva Ramanan, Benjamin Recht, and Ludwig Schmidt. Do image classifiers generalize across time? In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 9661-9669, 2021.

Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI Conference on Artificial Intelligence , 2015.

John P Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt. Accuracy on the line: on the strong correlation between out-of-distribution and in-distribution generalization. In International Conference on Machine Learning , pages 7721-7735, 2021.

Peter J Huber. Robust statistics. In International Encyclopedia of Statistical Science , pages 1248-1251. Springer, 2011.

Muhammad Muzammal Naseer, Kanchana Ranasinghe, Salman H Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Intriguing properties of vision transformers. In Advances in Neural Information Processing Systems , pages 23296-23308, 2021.

Chongzhi Zhang, Mingyuan Zhang, Shanghang Zhang, Daisheng Jin, Qiang Zhou, Zhongang Cai, Haiyu Zhao, Xianglong Liu, and Ziwei Liu. Delving deep into the generalization of vision transformers under distribution shifts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7277-7286, 2022.

Robert Geirhos, Patricia Rubisch, Jonas Rauber, Carlos R Medina Temme, Claudio Michaelis, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Inducing a human-like shape bias leads to emergent human-level distortion robustness in cnns. Journal of Vision , 19(10):209c-209c, 2019.

Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning , pages 6105-6114. PMLR, 2019.

Irwan Bello, William Fedus, Xianzhi Du, Ekin Dogus Cubuk, Aravind Srinivas, Tsung-Yi Lin, Jonathon Shlens, and Barret Zoph. Revisiting resnets: Improved training and scaling strategies. In Advances in Neural Information Processing Systems , pages 22614-22627, 2021.

Mingxing Tan and Quoc Le. Efficientnetv2: Smaller models and faster training. In International Conference on Machine Learning , pages 10096-10106. PMLR, 2021.

Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of out-of-distribution detection. In Advances in Neural Information Processing Systems , pages 7068-7081, 2021.

Kartik Gupta, Amir Rahimi, Thalaiyasingam Ajanthan, Thomas Mensink, Cristian Sminchisescu, and Richard Hartley. Calibration of neural networks using splines. In International Conference on Learning Representations , 2021. URL https://openreview.net/forum?id=eQe8DEWNN2W .

Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16816-16825, 2022.

Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision , 130(9):2337-2348, 2022.

Aleksandar Shtedritski, Christian Rupprecht, and Andrea Vedaldi. What does clip know about a red circle? visual prompt engineering for vlms. arXiv preprint arXiv:2304.06712 , 2023.

Hieu Pham, Zihang Dai, Golnaz Ghiasi, Hanxiao Liu, Adams Wei Yu, Minh-Thang Luong, Mingxing Tan, and Quoc V Le. Combined scaling for zero-shot transfer learning. CoRR, abs/2111.10050 , 2021.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper performs a comprehensive study of various CLIP models on robustness to different visual factors, out-of-distribution detection, and calibrated uncertainty estimations. A total number of 53 CLIP models trained on different training sources and sizes, and different architectures, with additionally 32 CLIP models fine-tuned on ImageNet are studied.\n\nSTRENGTHS:\n[Originality]\n\nTo the best of my knowledge, no previous studies have done such experiments on the 3 aspects to study CLIP models, especially considering various CLIP models trained on different datasets. Some of the observations made in this paper are new and complementary to the previous studies. For example, CLIP models are not robust in all aspects - they are less robust than models trained on ImageNet in a supervised way when poses are changed. Therefore, the findings are novel. \n\n[Significance]\n\nAlthough most conclusions that can be drawn from the experiments are already known, e.g., CLIP models are more robust than supervised models, some of them are less well-known and may be valuable to the community. For example, CLIP models trained on WIT perform better than those trained on LAION on OOD detection.\n\n[Quality & Clarity]\n\nThe quality and presentation of this paper are good. It is easy to understand and follow.\n\nWEAKNESSES:\nSome of the discussion on the experiment observations might need more support. \n\n1. Line 177: \"The shape bias of CLIP may be attributed to its objective, which involves training the model to associate text and image pairs\": in Figure 2, it seems that fine-tuning CLIP models on ImageNet (with supervised objective?) decreases the shape bias. However, it is also possible that the data source (ImageNet) could be the reason. Is it possible to decouple this? Say fine-tune CLIP with the contrastive objective on ImageNet and see if the shape bias stays the same.\n\n2. Line 246: \"We notice that CLIP models trained on LAION-80M dataset exhibit lower calibration performance when compared to standard models.\" Is this fair to compare as CLIP models trained on LAION-80M generally have lower accuracy?\n\nAnother minor concern I have is that some results from previous studies are not clearly discussed. For example, in ImageNet-X, it is already observed that \"color-jitter augmentation improves robustness to color and brightness, but hurts robustness to pose.\" Since color-jittering is widely used in the CLIP training, this should be discussed in the paper.\n\n[1] Badr Youbi Idrissi, Diane Bouchacourt, Randall Balestriero, Ivan Evtimov, Caner Hazirbas, Nicolas Ballas, Pascal Vincent, Michal Drozdzal, David Lopez-Paz, and Mark Ibrahim. Imagenet-x: Understanding model mistakes with factor of variation annotations. In International Conference on Learning Representations, 2022.\n\n[Minor]\nLine 94: ImageNet 32 fine-tuned CLIP models -> 32 ImageNet fine-tuned CLIP models\nLine 141: Small -> Smaller\n\nQUESTIONS:\nI have the following questions that would help me better understand the paper, and I would appreciate the authors' reply to them:\n\n1. Is there any insight on why WIT differs from LAION when serving as the CLIP training set, since I thought both of them were deemed to be a comprehensive snippet of the web image-text pairs?\n\n2. What about the other self-supervised models like MAE when compared with CLIP in terms of robustness? This also helps demystify the factors of datasets and objectives.\n\n3. ImageNet-1k contains 1.3 million images, and ImageNet-21k contains > 20 million images, which are not far away from the smallest LAION dataset considered in the paper. Is it safe to say that when the dataset size is similar, the supervised objective is still better in terms of calibrated uncertainty estimations? How about robustness and OOD detection?\n\nLIMITATIONS:\nThe authors discuss the broader impacts in the conclusion section.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies and compares CLIP and CLIP-FT to standard models on a range of different tasks including OOD robustness, OOD detection, and model calibration. The paper constitutes a meta-analysis across different model architectures / training datasets / training algorithms or loss functions.\n\nThe authors claim:\n\n####  Robustness\nCLIP sometimes outperforms other models on certain visual factor variations, but sometimes underperforms them.\nCLIP models are more shape biased (authors claim due to VLM training), resolution increase decreases shape bias.\nFine-tuning makes CLIP behavior more similar to other image models.\n\n####  OOD Detection\nCLIP generally performs better than other models in OOD detection; the relationship between IID accuracy and OOD detection performance largely follows accuracy on the line.\nFine-tuning negatively affects performance.\n\n####  Model Calibration\nCLIP isn’t significantly more calibrated than other models, data distribution affects calibration.\nTemperature scaling makes CLIP more calibrated than other models, and removes dependence on data distribution.\nTemperature scaling is apparently more important for CLIP.\nWhen temperature scaled CLIP is better OOD calibrated than other models.\n\n####  Test Time Prompts\nUsing more prompts generally helps across tasks, except in the case of some visual factor variations where it doesn’t really make a difference\n\nSTRENGTHS:\n- The introduction is well written, motivates the paper and gives a clear overview of the claims.\n- Well written, easy to follow\n- Objective tone/analysis and solid experimental design\n- Some nice points/findings dispersed throughout the paper\n\nWEAKNESSES:\n- No clear central argument or claim, paper seems more like a pastiche of different experiments rather than a focused analysis. There is little interpretation of the experimental findings. I have written very detailed questions considering the OOD detection results, but similar questions apply to all sections.\n- It looks to me that the paper tries to do too much in one paper and ends up being imprecise / too shallow on interpreting the results. For example, [2] focuses solely on how the data distribution affects robustness. Here, the authors observe this finding and only note that “The above observations highlight the importance of the choice of training source in determining not only the overall accuracy but also the factor-level behaviors of CLIP models. This suggests that visual factor-level robustness should be considered when designing the training source for CLIP models.” This is a “political” answer which does not provide concrete action items for future researchers.\nSome claims/analyses are probably wrong:\n\n    *  “Temperature scaling reveals a consistent trend of CLIP models, and they still lie on a distinct trend from other models.” This doesn’t seem to be the case ALL of the time. SSL models look pretty similar to CLIP on the “NLL (Temp-scaled)” and “ECE (Temp-scaled)” Imagenet-A plots. Amount of difference seems to be dataset specific. In fact, I only see a significant effect in ImageNet-A, ECE vs ECE-temp scaled, where the temperature scaling seems to affect all models.\n    *   ““This observation indicates that unlike robustness and out-of-distribution detection, the calibration of CLIP models is influenced by both training data distribution and quantity.” I don’t see how this follows from your data - aren’t robustness and OOD detection also influenced by data distribution?\n    *  The CLIP training dataset likely overlaps with the datasets used for OOD detection which makes it unclear how meaningful the claims are in regards to CLIP’s good OOD detection performance. This issue should at least be discussed.\n    *   OOD detection evaluations are missing important baselines (see details below).\n\n- The paper lacks novelty: The influence of training data / fine-tuning affecting CLIP performance have been studied in detail before [1,2,3]. This paper evaluates many different models and is thus a meta-study of the previous findings. In that case, the literature review should be expanded, and the paper must be better positioned, e.g. in “in [3], the authors investigate the influence of fine-tuning on OOD robustness. We here investigate whether their claims hold across a broader range of models'' or something like this. Though [3] already provides a thorough and careful empirical investigation across many different models, and I am not sure whether this paper offers much beyond the results presented in [3]. \n- The graphs were sometimes a bit hard to read/interpret - there were a lot of them and usually the conclusion was different/unique depending on the plot.\n- Some kind of unifying principle might be nice.\n\n### Minor:\n- Please be more specific about results/claims in the abstract.\n- 3.1 - Would be nice to explain/list model choices at some point, i.e. refer to Supplement A.2. here\n- 3.2 (Robustness) - more explanation of why you chose 10 of the 16 would be nice (robustness)\n- Line 40, style / grammar: “our study further study .. “\n- Line 42: “training distributions”, it should be distribution\n- Line 50, remove the extra space before the full stop\n- Line 280: “highlighting their potential for the robust and reliable applications” → “highlighting their potential for robust and reliable applications.\n\nQUESTIONS:\nWhat is the central claim/argument?\n*  Maybe pivot towards study similar to [2] in the context of OOD detection and model calibration? That is: Be more specific on what causes the observed OOD detection results.\n*  What about the data distribution matters for OOD detection and model calibration?\nIn general, I would appreciate some deeper insights, I don’t feel like I learned all that much from this paper.\n\n### Robustness results (Section 4)\n\nLines 138-144: The authors observe that CLIP models perform better on certain visual factors and worse on others. Some interpretation of the results is necessary to make the observations valuable to the research community. For example, the authors find that fine-tuning on ImageNet changes the robustness properties of CLIP models, but offer no explanation / interpretation of the results.\n\n\n###  OOD detection results (Section 5):\n\nLine 209: “Upon closer examination of the training distribution, we have observed that the correlation trend between ID accuracy and OOD detection performance is largely dependent on the training source.” -> It is not clear to me how sensible the OOD detection task is for CLIP models. While LAION may not contain e.g. all of the ImageNet training images, doing zero-shot inference is possible because CLIP has seen similar images during training, thus, “zero-shot” is ultimately a misleading term which becomes an issue once OOD detection is considered. It is not clear to me that the datasets used for OOD detection here, namely iNaturalist [52], SUN [53], PLACES [54], TEXTURE [55] and ImageNet-O [7], are actually not part of LAION in the first place, making measuring performance on those datasets an ID task, rather than an OOD task. Thus, I find it unsurprising that the training data distribution matters a lot for this particular task. Further, comparing the performance of models trained on ImageNet which has no intersection with ImageNet-O to models trained on LAION which may have a high intersection with ImageNet-O seems an unfair comparison to me. Could the authors please comment on this issue, in particular, on the sensibility of doing OOD detection for CLIP models trained on e.g. LAION?\n\nContinuing on this point: Line 212: “Moreover, with same ID accuracy, CLIP models trained on WIT exhibit superior OOD detection performance compared to their counterparts trained on LAION on three OOD scenarios. This further indicates the importance of training source selection for CLIP. When developing dataset curation methods, it is valuable to investigate the influence of training sources on OOD detection performance.” I would like to see a more nuanced and detailed discussion on this finding. Is it maybe that WIT just has a larger overlap with the OOD detection datasets? What is the concrete suggestion here? \n\nLine 219: “Some CLIP-FT models even achieve worse OOD detection performance than Zero-shot CLIP models.” This is in line with the points above and what we may be seeing here is that with fine-tuning, the model weights move towards being compatible with ImageNet-1K and further away from the original training distribution which likely overlaps with OOD detection test sets.\n\nDue to all of the points raised above, I find the sentence in Conclusion, line 303: “Furthermore,\nwhile maintaining comparable accuracy on in-distribution dataset, CLIP models tend to exhibit higher performance in OOD detection.” to be misleading.\n\nThe recent paper [4] (published after the NeurIPS submission deadline) makes the very similar observation that the CLIP training dataset has a large influence on zero-shot OOD detection, and also remarks on the effects of the fine-tuning procedure. Comparing how the conclusions made in this paper align with theirs would be very helpful. An evaluation on their dataset would also be interesting to see, since they show that Places, Texture and ImageNet-O have severe issues with ID data.\nThey seem to contradict the notion that CLIP zero-shot is well suited for OOD detection. I see no reason not to compare fine-tuned CLIP models with those fine-tuned on ImageNet 21K, which are very commonly used.\nFurther, the evaluation of OOD detectors should include strong baseline models like the Mahalanobis detector, compared to which the CLIP based models clearly fall behind according to the evaluations in [4].\nNot including the mentioned important baselines (which are also included in [5]) might be the main explanations why the evaluations in Figure 3 (and in the previous papers claiming zero-shot CLIP with MCM being a strong OOD detector) suggest such a positive image of the CLIP based models.\n\n\nMore centralized and focused claims/analyses would raise my score.\n\n\n#### References: \n\n* [1] Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP\n* [2] Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP)\n* [3] The Evolution of Out-of-Distribution Robustness Throughout Fine-Tuning\n* [4] Bitterwolf et al. ICML 2023: “In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluation”\n* [5] Yang et al. NeurIPS 2022: “OpenOOD: Benchmarking Generalized Out-of-Distribution Detection”\n\nLIMITATIONS:\nLimitations were not addressed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-30"}

comments: SUMMARY:
Authors closely study the robustness of vision-language models. They try to investigate their robustness in terms of common visual attributes, detecting OOD inputs, and their power in providing calibrated predictions. They consider many different CLIP models and other vision encoders with different architectures and training procedures to have a comprehensive study and fairly compare CLIP models with other ones.

They provide some more detailed findings about these models w.r.t to the aforementioned criteria.

STRENGTHS:
+ This paper runs an extensive set of experiments using various models, various datasets, and under different settings.
+ Therefore, these results will be insightful for practical use cases where people want to decide which model to use or diagnose possible errors/failures of their models on different conditions.

WEAKNESSES:
+ I didn't see enough new ideas in this paper.
+ I mean, running extensive studies is definitely valuable, practical, and insightful, but is there other similar work published in NeuriPS where scaling up and running more experiments is the main contribution? I would appreciate it if the authors correct me in understanding their main contribution and change the rating correspondingly.

QUESTIONS:
Are these results generalizable and reliable? i.e., are we going to see similar trends for future CLIP models as well? I would want to get some more insights about these findings. At this point, this study looks completely empirical and I am not convinced if they are statistically significant. Can we really trust those plots and argue some general statements?

LIMITATIONS:
Limitations are addressed.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-01

comments: SUMMARY:
This paper analyzes the CLIP model's robustness through a large number of experiments, including three main points: resilience to visual factor variations, calibrated uncertainty estimations, and the ability to detect anomalous inputs.

STRENGTHS:
1.  The experiments in this paper are very sufficient, the research content is solid, and some new views are proposed from the experimental results. There is a rich analysis of the robustness of CLIP.
2. This paper makes a significant contribution to the field by providing a comprehensive evaluation of CLIP models.  Furthermore, the experimental findings presented in this paper offer valuable insights for future endeavors aiming to enhance the out-of-distribution (OOD) detection performance and robustness of CLIP models.

WEAKNESSES:
There is no deeper analysis of the reasons behind these experimental results in this paper.

QUESTIONS:
1. Please provide some details about the experiments.
2. I want to ask whether parameter-efficient fine-tuning (PEFT) methods, such as LORA or Adapter, hurt the performance of OOD detection, given that fine-tuning sometimes can.

LIMITATIONS:
The authors has provided the limitations of their work.  This paper has done a lot of experiments and provided some novel discoveries that I think contribute to the CLIP community.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-06

comments: SUMMARY:
This paper aims to provide a comprehensive evaluation of robustness for pretrained vision-language models. Specifically, the authors benchmark around 100 pretrained models/classifiers. Based on these empirical results, this paper also provides corresponding discussions and analysis.

STRENGTHS:
1. The robustness problem of CLIP like model is a valuable topic to study.
2. This paper provides a very comprehensive benchmark for CLIP robustness problem. It may contribute to several benefit for the following research.
3. Corresponding discussion and analysis are solid to further inspire study in this area.

WEAKNESSES:
Even if I am still concerning the technical contribution of this paper for the NeurIPS conference, I recognize the workload and benchmarking work of this paper. Thus, only a little comments for the weaknesses:
1) What about further adding image-text retrieval evaluation and analysis? since CLIP can be used for both classification and retrieval. It may make this paper more solid.
2) I would like to discuss with other reviewers about the contribution significance of this benchmarking work to adjust my final score.

QUESTIONS:
Please see weaknesses.

LIMITATIONS:
Limitations have been discussed by the authors.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-11