A Closer Look at the Robustness of Contrastive Language-Image Pre-Training (CLIP)
Authors: Weijie Tu
Weijian Deng
Tom Gedeon
Venue: NeurIPS 2023 poster (2023)
Keywords: CLIP

Abstract: Contrastive Language-Image Pre-training (CLIP) models have demonstrated remarkable generalization capabilities across multiple challenging distribution shifts. However, there is still much to be explored in terms of their robustness to the variations of specific visual factors. In real-world applications, reliable and safe systems must consider other safety measures beyond classification accuracy, such as predictive uncertainty. Yet, the effectiveness of CLIP models on such safety-related objectives is less-explored. Driven by the above, this work comprehensively investigates the safety measures of CLIP models, specifically focusing on three key properties: resilience to visual factor variations, calibrated uncertainty estimations, and the ability to detect anomalous inputs. To this end, we study $83$ CLIP models and $127$ ImageNet classifiers. They are diverse in architecture (pre)training distribution and training strategies. We consider $10$ visual factors (\emph{e.g.}, shape and pattern), $5$ types of out-of-distribution data, and $8$ natural and challenging test conditions with different shift types, such as texture, style, and perturbation shifts. Our study has unveiled several previously unknown insights into CLIP models. For instance, they are not consistently more calibrated than other ImageNet models, which contradicts existing findings. Additionally, our analysis underscores the significance of training source design by showcasing its profound influence on the three key properties. We believe our comprehensive study can shed light on and help guide the development of more robust and reliable CLIP models.

A Closer Look at the Robustness of Contrastive Language-Image Pre-Training (CLIP)
Weijie Tu'
Weijian Deng'
Tom Gedeon 2.3
'The Australian National University
Curtin University 3 University of ÓBuda
lastname @ anu.edu.au
firstname lastnamee curtii.eeuuuuuu
Abstract
Contrastive Language-Image Pre-training (CLIP) models have demonstrated re- markable generalization capabilities across multiple challenging distribution shifts. However, there is still much to be explored in terms of their robustness to the variations of specific visual factors. In real-world applications, reliable and safe systems must consider other safety objectives beyond classification accuracy, such as predictive uncertainty. Yet, the effectiveness of CLIP models on such safety- related features is less-explored. Driven by the above, this work comprehensively investigates the safety objectives of CLIP models, specifically focusing on three key properties: resilience visual factor variations, calibrated uncertainty estima- tions, and the ability to detect anomalous inputs. To this end, we study 83 CLIP models and 127 ImageNet classifiers. They are diverse in architecture, (priinnaining distribution and training strategies. We consider 10 visual factors (e.g., shape and pattern), 5 types of -iitr--disttibution data, and 8 natural and challenging conditions with different shift types, such as texture, style, and perturbation shifts. Our study has unveiled several previously unknown insights into CL.IP models. For instance, they are not consistently more calibrated than other ImageNet models, which contradicts existing findings. Additionally, our analysis underscores the significance of training source design by showcasing its profound influence on the three safety-related properties. We believe our comprehensive study can shed light on and help guide the development of more robust and reliable CLIP models.
Introduction
By leveraging natural language supervision, CLIP has made significant progress in enhancing the zero-shot capabilities of models, unleashing their potential for remarkable outttf-distribution generalization performance ([. For example, CLIP models perform zero-shot classification without being explicitly trained on the target dataset, and they exhibit strong robustness to challenging natural distributional shifts 3 Understanding such behavior of CLIP models is critical for advancing the next generation of image-text foundation models. Current research this topic has explored various aspects of CLIP models, including dataset creation g reproducible scaling law [9) fine-tuning approaches [10), and training distribution [
In this work, we conduct an in-depth analysis of the safety-related properties of CLIP models, with a particular emphasis on their robustness and reliability across diverse testing environments. Specifically. we delve into the three critical safety-related properties, namely: 1) robustness to visual factors, to assess whether CLIP models maintain robust when encountering varying visual factors, such as pose, size, color, lighting, and occlusions; 2) out-of-distribution (OOD) detection, to evaluate the capability of CLIP models to detect instances with labels that are not part of the training distribution; and 3) predictive uncertainty, to investigate whether CLIP models can provide calibrated predictions that accurately reflect their uncertainty in different testing conditions.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Our research offers a comprehensive examination of advantages and drawbacks of CLIP models across various critical facets. Building on prior research that has highlighted robustness of CLIP, we further study their performance across specific factors such as pose and lighting. Additionally, although CLIP models have demonstrated their efficacy in detecting OOD data [12], it still uncertain whether this ability remains consistent when the training distribution, ine-tuning datasets, model size, and architecture are altered. Moreover, beyond classification accuracy, is also important to evaluate whether CLIP models offer reliable uncertainty estimation across various distributions.
In light of the aforementioned questions, we evaluate 51 zero-shot CLIP models with varying visual encoder architectures, training sources, and dataset sizes, as well as 32 ImageNet fine-tumed CLIP models. To establish baseline, we compare these models against 127 ImageNet models without trrguage-image pre- training. We examine 10 visual factors variations present in the ImageNet valida- tion set [3], including object pose, lighting, and background, to assess models' visual factors-level robustness. As for OOD detection, we employ ImageNet as an in-distribution (ID) set following [22 and test on 5 types of OOD scenarios. Furthermore, to investigate the predictive uncertainty, we use set of canonical ImageNet distributions, such texture, style, and perturbation shifts. Below we present key observations and insights obtained from our study:
CLIP models are generally more robust than ImageNet classifiers on visual factors. However, they can be less robust factors like object pose; In addition, training distribution plays important role in CLIP robustness against visual factors (Section 4.1)
CLIP models are biased towards shape when making predictions. However, we have also found that this bias diminishes after fine-tuning on ImageNet and becomes similar to other ImageNet models that are pre-trained on more data. (Section 4.2)
When trained the same source, classification accuracy of CLIP models correlates with their OOD detection performance (Section[5) 55.
CLIP models are not always more calibrated than other ImageNet models, which contradicts existing findings [4. Our research highlights the impact training data distribution and quantity these observations (Section 6
Compared other groups of models, CLIP maintains reasonably good uncertainty estimates under distribution shifts after ID calibration with temperature scaling. (Section 6).
Related Work
Robustness focuses on investigating the resilience of machine learning models to various forms of distribution shift at test time. To this end, commonly used approach is to introduce artificial transfor- mations onto images, such as, style transfer [15), corruptions and perturbations [16] I7. Moreover, many real-world datasets are introduced to assess model robustness under a natural distributional shift (377). For instance, Idrissi et al. [ propose ImageNet-X by relabelling ImageNet validation set to provide detailed labels for naturally occurring factors such as different pose, background and lighting, to identify models' underlying failure patterns.
OOD detection targets identifying test data that do not belong to any of classes modeled in training distribution 20). A large number of methods are proposed for deep learning models, including
generative bbsel- methods (21 and discriminative model-based methods (2] [8] 30 31
20 32
For example, maximum softmax probability (18 is used as the metric to detect OOD samples. Moreover, the above approaches mainly study OOD detection for a task-specific model using only visual information. In contrast, as CLIP models enjoy popularity, zero-shot OOD detection [12] is proposed, where the objective becomes filtering out input from the task of disinterest.
Predictive uncertainty aims to classify images with calibrated prediction probabilities so as to Several works improve uncertainty estimations
match the empirical frequency of correctness [33 34
through post-hoc calibration on validation sets 33 34 Moreover, some other works show calibration can be improved by directly applying methods, such as ensembling
35 and pre-training [36]. Ovadia et al. 337 point out that calibration methods become less effective under distribution shift. Minderer et al. 144 suggest that CLIP models are well-calibrated given its accuracy. Based on these observations, this work comprchensively studies the quality of predictive uncertainty given by CLIP.

---

Experimental Setup
3.1 Models of Interest
Contrastive language-image pre-training models: We use 51 zero-shot CLIP models (CLIP) and
32 ImageNet fine-tuned CLIP models (CLIP-FT). They have different visual encoders, including slightly modified ResNet [38], ConvNeXt [39]. and ViT [40]. There are two training sources (LAION [4] and WIT [TI and multiple sizes of training datasets from 80 million to 2 billion. For the CLIP-FT models, the vision tower of CLIP is fine-tuned on ImageNet-IKK We consider two fine-tuning procedures, one directly fine-tuned on ImageNet- IK [42] and the other first fine-tuned on ImageNet- 12K, subset of ImageNet-22K before fine-tuning ImageNet-IK: Unless specified, use the default prompt template by for zero-shot CLIP models.
Compared models: we use 127 ImageNet models with various architectures, including Convolutional Neural Networks (e.g.. ResNet 38 and ConvNe (39)), Vision Transformers (e.g.. ViT 40 and
Swin 4 and alI-MLP architectures
44 45 (e.g.. g.. MLP-Mixer [55). Following [46], we divide
them into three categories: (i) Standard Models. This group consists of models supervised on the
ImageNet training set. (ii) Contrastive learning models. This category contains 8 models pre- trained by contrastive learning. There are 6 training algorithms investigated, including InsDis [47), MoCo [48) SimCLR [49]; (iii) Pre-trained on more data. This group contains models pre-trained on a significantly larger dataset (e.g., ImageNet-2 than the ImageNet training set. All the above
models, including CLIP, publicly available TIMM (50) and OpenCLIP [51]
3.2 Test Sets and Metrics
Robustness. We first pinpoint failure patterns of models by testing on ImageNet-XX X [3]. which is relabelling of ImageNet validation by 16 naturally occurring factors. This work mainly considers 10 factors labelled with a sufficient number of test samples: Pose, Background, Pattern, Color, Smaller, Shape, Partial View, Subcategory, Texture and Larger. The metrie is accuracy. and high is better. We evaluate on cue-conflict stimuli and Etylized-ImageNee [5] to measure model bias towards shape.
OOD Detection. We use large-scale OOD detection benchmark which is build up ImageNet: in-distribution (ID) ImageNet v.s. {iNaturalist [52], SUN (53), PLACES [54), TEXTURE (55) and ImageNet-O 77] (OOD). The metrics are the area under the receiver operating characteristic curve (AUROC) and the higher is better; false positive rate (FPR 95) when true positive rate at 95% and lower score means better performance.
Calibration. We study and OOD datasets, where ImageNet validation ID dataset and OOD datasets are: ImageNet-V2 V2 (3. ImageNet-Rendition [5], ImageNet-Adversaare
(7]. ImageNet- Sketch 44, ObjectNet 6 and ImageNet- Vid-Robust [56]. Metries are estimated calibration error
(ECE) 57 and negative log likelihood (NLL). A lower ECE or NLL indicates better calibration.
3.3 Analytical Methodology
In our endeavor understand the underlying factors that influence the performance of CLIP models, we delve into six primary aspects: 1) training distribution, evaluating the effect of data source; 2) model architecture, looking into the potential effects of different structural choices on model performance; 3) dataset quantity, probing the interplay between the amount of data available for training and the model's efficiency; 4) contrastive loss, understanding its specific role in training dynamics 5) fine-tuning, and 6) test-time prompt, assessing the impact of prompts during the evaluation on model outputs. We follow the analytical methodology of seminal work (46) and a series of following works like (8, [] 58) study the influential factor. Specifically, within the performance trends observed across all models, any factor causing deviation from these trends is recognized influential. Notably, in research, we mainly emphasize and discuss such influential factors within each facet of our investigation.
4 Visual Factor-Level Robustness
The unprecedented robustness of CLIP models has spurred intense research efforts to identify the underlying factors responsible for their performance under distribution shifts. Recent studies provide
3

---

Accuracy fector
on
Lincar fr. (CLIP-WIT zoro-shot)
126
CLIP trained on LAION
uP
8mh
405
Overall accuracy on Imageliet-X
Linear fr. (CLIP-LAION zero-shot)
Shop
Tedtse
Recoratt the Factor
Fine-tured
The
Pre-treined on more data
CPC
Slandard model
Contrastive leamiing model
1.5
75
Overall Imagellet-X
M5
1/%
80.6
Partern
taciground
Portia Veww
Pewe
XIX
Accuraty the
1PL
64h
Overall sccuracy on ImageNet-X
Figure 1: The models performance on the subset of ImageNet-X annotated with given visual factor (y-axis) their overall accuracy on the whole ImageNet-X (x-axis). Each point represents model. The x-axis and y-axis are probit transformed following [46]. The black dashed line represents the ideal robust models whose performance on each visual factor is the same as the overall performance. The blue straight lines are fit with robust linear regression [59]. We include models supervised on ImageNet-IKS pre-trained on more data, contrastive learning models, CLIP models trained on two data distributions and their fine-tuned counterparts. We find that CLIP are generally more robust on out ten factors, but are less robust against Pose than other groups of models.
Our research builds upon previous findings on the robustness of CLIP models and focuses on the potential failure types of the model. Instead of solely measuring overall accuracy across distributions, we investigate the behavior of CLIP models when faced with varying visual factors such as Pose and Background.
valuable insights on the design of training source [ 8
4.1 CLIP Models Generally Exhibit Better Factor-Leeve Robustness Than Other Models
Factor-level effective robustness. In study, extend the concept of overall effective robust- ness [46] to visual factor-level. Specifically, it measures a model's 's ability to achieve higher accuracy on the subset annotated by a specific visual factor compared what is expected based its overall accuracy on ImageNet-X. Figure displays the accuracy on the subset annotated by a specific visual factor relative the overall accuracy ImageNet-X.
CLIP models are generally more robust than other ImageNet models on out of 10 visual factors. Figure Thighlghttt several insights into the factor-level robustness of CLIP models. First, we find that CLIP models are more robust than other models on six out of ten visual factors, including Subcategory, Smaller, Color, Shape, Texture, and Larger. Specifically, CLIP models exhibit higher factor-level effective robustness than other models on each of these factors. Second, we observe that CLIP models are less robust than other models Pose and Partial View factors. Third, CLIP models show similar trend to other models on the Background factor. In addition, Idrissi et al. [3] observe that data augmentations can improve robustness to related factors, but with spill-over effeets to unrelated factors. We speculate that the data augmentations used for training CLIP models may introduce the similar effects.
Training distributions lead different trends CLIP models. The choice of training distribution impacts factor-level robustness of CLIP models. Specifically, we find that training different datasets (i.e., LAION and WIT) forms distinct trends on each visual factor for CLIP, and there no single training source that always leads to higher factor-level robustness than another. For instance,

---

Source
Backbone
Shape bias
IN- Val
SIN
0.
Turobamer cWn
VIT/II 14 (336/224)
0.42/0.51
0.89 /0.88
0.28 /0.32
VIT/L-14 (336/224)
0.41
0.88 /0.
0.27 /0
0.6-
LAION
VIT/B-16 (384/224)
0.35 /0.33
0.87 /0.86
0.23 /0,
VIT/B-32 (384/224)
0.33//0.45
0.85 /0,
0.21 /
0.4-
ConyNeXi-B (384/224)
0.31 /0.38
0.87 /0.86
0.17 /0.21
0.2-
WIT
VITM (136/224)
0.39 /0.45
/0.88
0.24 /0.30
VITB-116 (384/224)
0.35/0..1
0.87 /0.86
0.22 /0.23
0.0-
CUP
CLIP-FT Pretrain Standard
Figure 2: Shape bias analysis CLIP,
Table 1: The influence of input resolution when fine- tuning CLIP on Shape bias and and Stylized ImageNet (SIN) accuracy. The higher value in a model pair is in bold. With same backbone, the model fine-tuned with larger input resolution is more accurate on IN- Val but less shape-biased and less accurate on SIN.
CLIP fine-tuned (CLIP-FT), models pre- trained on more data (Pretrain), and stan- dard models. Large points mean larger models within the group. We observe that CLIP models are more shape-biased. biased.
we observe that CLIP models trained on LAION demonstrate higher robustness on Shape factor than those trained on WIT, while this reverses for Background and Pose factors. The results show a mixed observation on Large factor. Furthermore, we further point out that CLIP models trained on different subsets of LAION i.e., LAINON-80M LAION-40MM and LAION-2B) follow the same trend. The above observations highlight the importance of the choice of training source determining not only the overall accuracy but also the factor-level behaviors of CLIP models. This suggests that visual factor-level robustness should be considered when designing the training source for CLIP models.
CLIP fine-tuned models perform slightly better than more data pre-trained models. We compared CLIP fine-tuned models (CLIP-FT) with other models that are pre- preitrained on more data and find that CLIP-FT shows improvement in overall accuracy and robustness on visual factors of Subcategory, Shape, and Pattern. However, no additional robustness gain is observed on other factors. Additionally, CLIP-FT models outperform zero-shot CLIP on variations such as Pattern and Partial View, indicating their superiority in handling visual factors. It would be intriguing to explore fine-tuning techniques that maintain or improve the factor-level robustness of zero-shot CLIP.
4.2 Texture Bias v.s. Shape Bias
CLIP exhibits shape bias. We conduct experiments using the cue-conflict stimuli dataset [5 to investigate the presence of shape bias in the model's predictions. Shape bias, in this context, refers to the proportion of accurate predictions made based on object shapes. Figure presents visualization of the shape bias exhibited by the models, which are grouped according to their training methods (zoro-shot, CLIP finetuning, more data pre-trained, and standard training) and architecture (transformer versus CNN). Our findings indicate that, among the four training methods, CLIP models are more likely make predictions based on shape compared the other three groups. Furthermore, while the transformer is reported to have a stronger shape bias than CNN 60 611. we observe that CLIP using CNN as the vision encoder also exhibit a strong shape bias.
Model size solely does not explain the shape bias of CLIP. We further observe that larger CLIP models do not necessarily have higher shape bias than smaller-size ones. For example, both trained on LAION-80M, implies that the shape bias of CLIP models cannot be attributed solely to model size. Based on the above observations, we speculate that the shape bias of CI.IP may be attributed to its objective, which involves training the model associate text and image pairs.
14 has 0.54 shape bias, which is 0.09 lower than CLIP-ViT/B-33 This
CLIP models have a tendency towards texture bias after fine-tuning. Our study reveals that shape bias in CLIP weakens after fine-tuning ImageNet. Moreover, the fine-tuned CLIP models exhibit a shape bias comparable to models that are pre- pre-trained on larger datasets. This finding is consistent when using transformer and CNN as visual encoder. Moreover, these results illustrate that fine-tuning discards the shape- biased property of zero-shot CLIP, which may affect model robustness [62 15]
Larger input image resolution during fine-tuning of CLIP results in stronger bias towards texture. T we observe that an input resolution during fine-tuning a important factor to shape bias: increasing the input resolution during fine-tuning leads to better performance on ImageNet
5

---

Nettrrlsst
TEXTURE
PLACES
205
Linea it (CUP-WIT zero-shot)
MUROC
CLIP trained on LAIDN
Linaer (CUIP- paro-shot)
125
195
Accuracy on disributiion datcset (Inageheet
CLIP Fine tuned-12ee 12k-1k
CLIP Fins-tued-11k
Pro-trained
Standard model
Contrastive leaming model
Accuracy on in disributioon datoset linageNet)
1224
sPAgos
Ainoc
605
124
25%
45%
Accuracy dist lbution datooct (ImageNee)
Figure 3: OOD sample identification capability of models vs. ID dataset classification accuracy. The OOD detection ability measured by AUROC (1) and FPR @95 (4). Each point represents model. We plot the results on iNaturalist, SUN, PLACES, TEXTURE and ImageNet-O. The blue straight lines fit with robust linear regression [59]. We observe that training distribution has greater impact than training dataset quantity on the OOD detection performance of CLIP. Moreover, after additionally fine-tuning on ImageNet- 12K, CLIP are generally better at detecting OOD samples than those directly fine-tuned on ImageNet-IK.
validation but also results in texture-biased models with lower accuracy Stylized-ImageNer Across seven pairs of experiments and two training sources, we observe this pattern consistently. Given that input resolution is crucial model dimension (53]65), it would be interesting study its effects on shape bias beyond classification accuracy when devising scaling strategies.
Out-of-Distribution Detection
Zero-shot CLIP allows for a flexible definition of in-distribution (ID) classes without re-training the model. Namely, they can conduct zero-shot OOD detection. The current findings suggest that zero-shot CLIP models are competitive with other otate-of-the-art models 12 66). Based on this finding, conduct extensive analysis determine whether the purported benefits persist across various training sources, subsets, and network architectures. In the experiments, for zero-shot CLIP models, utilize maximum concept matching [12] to detect OOD data. For models that trained or fine-tuned on ImageNet-IK, we employ maximum softmax score 18 for OOD detection.
For CLIP models from the same source, their ID accuracy correlates with OOD detection performance. Our study includes CLIP models trained on two sources (WIT and LAION). Given the same training source, our study, conducted across five challenging OOD scenarios, reveals strong correlation between the ID accuracy of zero-shot CLIP models and their OOD detection performance (measured by AUROC and FPR @95). This observation suggests that the zero-shot classification accuracy of CLIP models on ID data can serve as a reliable indicator of their OOD detection performance. In contrast, such trend is not strong for standard models and more data-pre-tren pre-trained models. Additionally, CLIP-FT models tnn-ttundd on ImageNet- 1K do not exhibit such clear correlation.
Training source inftuences the trend of CLIP. Upon closer examination of the training distribution, have observed that the correlation trend between ID accuracy and OOD detection performance is

---

largely dependent on the training source. As illustrated in Figure our research shows two distinct trends between CLIP models trained on WIT and those trained on LAION. Moreover, with the same ID accuracy, CLIP models trained on WIT exhibit superior OOD detection performance compared to their counterparts trained on LAION on three OOD scenarios. This further indicates the importance of training source selection for CLIP. When developing dataset curation methods, it is valuable to investigate the influence of training sources on OOD detection performance.
Fine-tuning procedure influences the OOD detection ability of CLIP. First, we point out that fine-tuning enhances classification performance of CLIP, but this improvement does not necessarily translate to better OOD detection accuracy. Some CLIP-FT models even achieve worse OOD detection performance than Zero-shot CLIP models. Our analysis of CLIP-FT reveals a distinction between two groups of CLIP-FT, based on their fine-tuning procedures; the first group is fine-tumed solely on ImageNet- K, while the second group undergoes additional fine-tuning on ImageNet- 12K. We observe that this additional fine-tuning procedure has substantial impact on the model's s ability to detect OOD examples. As depicted in Figure despite not leading to an improvement in classification accuracy, CLIP-FT models with additional fine-tuning on ImageNet- 12K show better OOD detection performance across all OOD scenarios. As future work, it is valuable to investigate this observation further and explore alternative fine-tuning procedures that yield improved OOD detection performance. Moreover, exploring impacts of fine-tuning datasets other than ImageNet- IK and ImageNet- MiNNNett1KKK 12K would be another interesting direction.
6 Prediction Uncertainty
In order to better understand the well-calibrated phenomenon of zero-shot CLIP models reported by Minderer al. (14), our research systematically analyzes the calibration behavior of CLIP models under various training conditions. Specifically, we examine the calibration performance of CLIP models trained different training distributions, varied training set sizes, and different architectures. Furthermore, we also investigate the calibration performance of CLIP models after fine-tuning to gain better understanding of their overall performance.
6.1 Zero-Shot CLIP Models Are Not Consistently More Calibrated Than Other Models
Both training data distribution and quantity impact CLIP's calibration. Figure 4 presents the model calibration of CLIP models in relation to classification accuracy under distribution shifts. We observe that CLIP models trained on different distributions or quantities are not consistently grouped together. For example, CLIP models trained WIT and LAION tend to cluster in separate regions. Moreover, when training CLIP models on different subsets of the LAION dataset, models with similar classification accuracy can exhibit varying levels of calibration performance. It would be interesting to further check the impacts of data curation techniques on CLIP calibration performance.
While CLIP models are generally reported to have superior calibration compared to other models [4]. our observations reveal that this finding does not always hold. Particularly, we notice that CLIP models trained on LAION-80M dataset exhibit much lower calibration performance when compared to standard models. The observation of [14] primarily made on CLIP models trained on WIT. However, when we broaden our perspective to include the alternative training distribution provided by LAION and its various subsets, our observations become varied. This emphasizes the significance of careful training source design for CLIP. Furthermore, it suggests that when evaluating dataset curation, is crucial to consider its impact on the calibration performance of CLIP models.
CLIP fine-tuned models exhibit a trade-off between calibration and classification. On each test set in Figure 4 we consistently observe that after fine-tume men-ttning, CLIP models tend to have higher classification accuracy and lower calibration error. It is worth noting that additionally fine-tuning on ImageNet- 12K does not alter this phenomenon,
contrast to its impact on OOD detection. Moreover, other model groups, including those pre-trained on more data, do not exhibit trade-off between calibration and classification. We also observe some fine-tuned CLIP models achieve better calibration compared to their zhot counterparts before calibration.
7

---

AIDN-Aadd
CLIP (Temp-scalad)
Standard
Pre-zained
Contrssiive
ECE (Tere--ssded)
1.2
socuraty
0.30
0.5
021
336
628
12C
800
Classiicatoon accuracy
1.2
0.0
Imagen Nee-tt
Classiication acouracy
65
Figure 4: Model calibration performance with respect to their classification accuracy. We report results on in-distribution test set, ImageNet-V22 ImageNet-R and ImageNet-A. Two metrics are considered: ECE (L) and NLL (1), also include calibration performance after calibration with temperature scaling. Each point represents model. We use colors to represent model groups. For zero-shot CLIP, we additionally use shapes indicate training distribution and quantity. We observe that CLIP models could less calibrated than standard models. The training distribution and quantity the key factors influencing the calibration performance of CLIP models. Temperature scaling reveals a consistent trend of CLIP models, and they tend to lie on a distinct trend from other models.
6.2 Temperature Scaling Reveals Well-Calibrated Properties of Zero-Shot CLIP Models
Post-hoc calibration can be adopted remedy over- under-confidence. Here, temperature scaling (33] to calibrate model predictions. Following the protocol [67], we divide the validation set of ImageNet into two halves: one for temperature scaling (ID calibration set), and the other one for ID test. We report results on both ID and OOD test sets.
Calibration performance of CLIP models after temperature scaling (Temp-scaled) correlates with their classification accuracy. Figure 4 we explore how temperature scaling affects different groups of CLIP models. These groups categorized based the amount and source of their training data. After applying temperature scaling and evaluating using the NLL metric, we observe a consistent pattern among these CLIP groups. What is intriguing is that, after temperature scaling, for models with similar image classification accuracy, zero-shot CLIP models achieve better calibration performance compared other models, including their fine-tuned counterparts. This trend persists across various testing scenarios, encompassing ID, OOD, and when assessing calibration using both NLL and ECE metrics.
ID calibration of CLIP models transfers OOD test sets. While it has been reported by Ovadia al. 37 that ID calibration often struggles to generalize under distribution shifts, our study reveals

---

CLIP RN50
CLP RN60X64
CLIP VITIB-92
CUP VITB
VITI-14-386
80 promplx
50 prompi
5 promple
pranpe
UnsartuUPP WIT)
SUN
InagrRee-
SC:fferp-ooett
MUROC
390
10.13
8:27
15%
Robustness
[b) OC0 detection
Prodictive uncortainty
Figure 5: Influence test time prompt on CLIP on robustness to visual factors, OOD detection, and predictive uncertainty. We include five CLIP models trained on WIT. We use different colors to denote different model architectures and utilize various shapes to represent deployed prompt sets. The dashed grey line is fit with robust linear regression [59] by the original CLIP-WIT models using 80 prompts. We see that the prompts of sizes 1, 5 and 30 decrease the classification performance of CLIP, but may not change the visual factor robustness of CL.IP.
promising phenomenon for CLIP models. Specifically, after calibrating CLIP models on ID) calibration set, we observe improved calibration results on OOD test sets. For example, on ImageNet- A, the calibration errors of CLIP models decrease after temperature scaling, unlike other models that do not clearly exhibit such improvement. This indicates that CLIP models are relatively easier to calibrate across diverse distributions, highlighting their potential for robust and reliable applications.
7 Discussion on Influence of Test Time Prompts
So far in the experiments, we utilize the prompt provided by M this section, we investigate the effect of test time prompts on three safety objectives. We additionally examine another three sets of prompts, one of size 1: "a photo of {label)", set prompts used by [12, and a set of 30 prompts. We conduct the experiment on five CLIP models: RN50, RN50x 64, VIT-B/16, ViT-B/32 and V1T/L--4-3336x trained on WTT. Figure shows the performance of CLIP models using different sets of prompts three safety objectives.
We show that utilization of few prompts (e.g., one prompt) generally leads decrease in overall classification performance. However, the impact on the three properties is mixed. First, when evaluating factor-level robustness on Pattern, we find the adoption of different prompts does not alter the robustness: models still adhere to the linear trend established by CLIP trained on WIT using 80 prompts. Second, for OOD detection, using prompts yield higher OOD detection performance than with SO prompts SUN. Also, for calibration, using fewer prompts (e.g., only one prompt) achieves lower calibration error than using all SO prompts. This raises an important question: how to tune prompts to achieve better classification accuracy as well as better calibration and OOD detection performance? It would be interesting to study this when conducting prompt learning (68]
Conclusion and Discussion
Our research provides a valuable contribution to the ongoing discourse surrounding the effectiveness of CLIP models in the context of robustness to visual factors, OOD detection, and the reliability of uncertainty estimation. In pursuit of these objectives, we conduct extensive experiments encompassing three critical tasks and perform comparative analyses between CLIP models and various groups of models. Our observations provide valuable insights into the advantages of CLIP models. First, CLIP models demonstrate superior visual factor-level robustness compared to other ImageNet models. Furthermore, while maintaining comparable accuracy in-distribution dataset, CLIP models also exhibit competitive performance in OOD detection across commonly used benchmarks such as iNaturalist and ImageNet-O. Lastly, CLIP models relatively easier calibrate across diverse distributions. Furthermore, our study highlights the significance of training source design, as profoundly influences the behavior of CLIPP models across all three objectives.
This work leaves open many interesting directions for future research and we discuss a few. First, this work primarily studies CLIP and its fine-tuned models due to its simplicity and effectivvness. We

---

reckon this work as an anchor point and hope the framework of this analysis could generalize to other image-text foundation models, such as ALIGN (2 and BASIC [7]] Second, our study includes two academic training sources, namely WIT and LAION, for CLIP. It is valuable to investigate whether our observations generalize to other training sources. We believe that our study can shed light to and build up the understanding towards the design of multi-modal datasets. Lastly, in addition to three safety-critical tasks, there are other important fields to analyze such as miiscllassiiiaation samples detection 18). By providing more detailed and nuanced understanding of the performance of CLIP models, we hope our observation and insights can inform future developments in the field and help to drive progress towards more robust and effective vision-language models.
Broader impacts. While much research endeavors that aim to enhance the performance machine learning models could be leveraged for negative societal applications, we believe that this paper points towards positive direction for the development of machine learning methods in broader society. Particularly, our study understands the influence of training set on CLIP performance on robustness, out-of-dist of-distribuutonn detection, and predictive uncertainty. A better understanding is beneficial in establishing trustworthy machine learning systems and high-quality multi-modal datasets for training.
Acknowledgement. We thank all anonymous reviewers and ACs for their constructive comments and valuable suggestions in improving this paper.
References
[1] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pages 8748 8863. PMLR, 2021.
[2] Chao Jia, Yinfei Yang. Ye Xia, Yi-Ting Chen, Zarana Parekh, Hicu Pham, Quoc Lee, Yun-Hsuan Sung, Zhen Li, and Tom Duerig, Scaling up visual and vision-language representation learning with noisy text supervision. In International Conference Machine Learning, pages 4904 PMLR, 2021.
[3] Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International Conference on Machine Learning, pages 5400, PMLR, 2019.
[4] Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. In Advances in Neural Information Processing Systems, 2019.
[5] Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, al. The many faces of robustness: A critical analysis of out-of-distribbttin generalization. Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 8340-8349, 2021.
[6] Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Josh Tenenbaum, and Boris Katz. Objectnet: A surge-scale bias-controlled dataset for pushing the limits of object recognition models. In Advances in Neural Information Processing Systems, 2019.
[7] Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2021. [8] Thao Nguyen, Gabriel Ilharco, Mitchell Wortsman, Sewoong Oh, and Ludwig Schmidt. Quality not quantity: On the interaction between dataset design and robustness of clip. In Advances in Neural Information Procensing Systems. 2022.
[9] Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image lcarning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2023.
[10] Mitchell Wortsman, Gabriel Ilbarco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, et al. Robust tine-tuning of zero-shot models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
10

---

[11] Alex Fang, Gabriel Ilharco, Mitchell Wortsman, Yuhao Wan, Vaishaal Shankar, Achal Dave, and Ludwig Schmidd, Data determines distributional robustness in contrastive language image pre-training (clip). In International Conference on Machine Learning, pages 6216-6234. PMLR, 2022.
[12] Yifei Ming, Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, and Yixuan Li. Delving into out-of-distribution detection with vision-language representations. Advances Neural Information Processing Systems, 2022.
[13] Badr Youbi Idrissi, Diane Bouchacourt, Randall Balestricro, Ivan Evtimov, Caner Hazirbas, Nicolas Ballas, Pascal Vincent, Michal Drozdzal, David Lopez-Paz, and Mark Ibrahim. Imagenet-x: Understanding model mistakes with factor of variation annotations. In International Conference Learning Representations, 2022.
[14] Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucie. Revisiting the calibration of modern neural networks. In Advances in Neural Information Processing Systems, pages 15682-15694, 2021.
[15] Rohert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference Learning Representations, 2018.
[16] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference Learning Representations, 2019.
[17] Eric Mintun, A Alexander Kirillov, and Saining Xie. On interaction between augmentations and corruptions in natural corupption robustness. In Advances in Neural Information Processing Systems, pages 3571-3583,
2021.
[18] Dan Hendryeks and Kevin Gimpel. A bascline for detecting misclassified and outoo-diistibution examples in neural networks. In International Conference on Learning Representations, 2016.
[19] Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized antoo--distribution detection: survey. arXiv preprint arXiv:2110.11333) 2021.
[20] Weitang Liu, Xiaoyun Wang. John Owens, and Yixuan Li. Energy- based out- distribution detection. In Advances in Neural Information Processing Systems, pages 21464 21475, 2020.
[21] Mu Cai and Yixuan Li. nut-of-distribution detection via frequency- generative models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5521-5530. 2023.
[22] Polina Kirichenko, Pavel Izmailov, and Andrew G Wilson. Why normalizing flows fail to detect out-of- distribution data. In Advances Neural Information Processing Systems, pages 20578-20589, 2020.
[23] Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don't know? International Conference on Machine Learning, 2018.
[24] Lawrence Neal, Matthew Olson, Xiaoli Fern, Weng- Keen Wong, and Fuxin Li. Open set learning with counterfactual images. Fumpean Conference on Computer Vision, pages 61. 613-628, 2018.
[25] Poojan Oza and Vishal M Patel. C2ae: Class conditioned auto-encoder for open-set recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2307-2316, 2019.
[26] Jie Ren, Peter J Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of-distribution detection. In Advances in Neural Information Processing Systems, 2019.
[27] Joan Serrå, David Álvarez, Vicenç Gómez, Olga Slizovskaia, José F Núñez, and Jordi Luque. Input complexity and dut-of-distribution detection with likelihood-basee generative models, In International Conference on Learning Representations, 2019.
[28] Zhisheng Xiao, Qing Yan, and Yali Amit. Likelihood regret: An out-of-distribution detection score for variational auto-encoder. In Advances in Neural Information Processing Systems, pages 20685-20696, 2020.
[29] Abhijit Bendale and Terrance E Boult. Towards open set deep networks. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 1572-1572,

---

[30] Yen-Chang Hsu, Yilin Shen. Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out-of out-of-distribution image without learning from out-of-distribution data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 10951-10960, 2020.
[31] Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In International Conference on Learning Representations, 2017.
[32] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out- of-distribution samples and adversarial attacks. In Advances Neural Information Processing Systems, 2018.
[33] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning, pages 1321-1330. PMLR, 2017.
[34] Khanh Nguyen and Brendan O'Connor. Posterior calibration and exploratory analysis for natural language processing models. In Conference on Empirical Methods in Natural Language Processing, 2015.
[35] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, 2017.
[36] Dan Hendrycks, Kimin Lec, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. In International Conference on Machine Learning, pages 2712-2721. PMLR, 2019.
[37] Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Procexsing Systems. 2019.
[38] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pottern recognition, pages 770-778, 2016.
[39] Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11976- -1986, 2022.
[40] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghami, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image worth 16x words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2020.
[41] Christoph Schuhmann, Romain Beaumont, Richard Vencu. Cade W Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa R Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. LAION-5b: An open large-scale dataset for training next generation image-text models. In Thirr--ixthh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022. URL httpp:// oopenree eview. (ite:/forummddddMS7vaaaaa
[42] J. Deng, W. Dong, R. Socher, I.-J. Li, K. Li, and Fei-Fei. ImageNet: Large-Scale Hierarchical Image Database. Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, 2009.
[43] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10012-10022, 2021.
[44] Xiaohan Ding, Chunlong Xia, Xiangyu Zhang, Xiaojie Chu, Jungong Han, and Guiguang Ding. Repmlp: Re-parameterizing convolutions into fully-connected layers for image recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2022.
[45] Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov. Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner. Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mip-mixer: An all-mlp architecture for vision. In Advances in Neural Information Processing Systems, pages 24261-24277 2021.
[46] Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and I Ludwig Schmidt. Mea- suring rohustness to natural distribution shifts in image classification. In Advances in Neural Information Processing Systems, pages 18583-18599, 2020.
[47] Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via non- parametric instance discrimination. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3733-3742. 2018
12

---

[48] Kaiming He. Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 9729-9738, 2020.
[49] Ting Chen, Simon Komblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International Conference 1597-1607. PMLR, 2020.
Coutereee on Machine Learning, pages
[50] Ross Wightman. Pytorch image models. http:////g.taaaaa.ooo....aaaaaaa 2019.
[51] Gabriel Ilaarco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongscok Namkoong, John Miller, Hannanch Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip. July 2021. URL http//ttts://doo.rree http://0.....//1.../// http///1..22//eeoodd /tt.://22oooo...577777 If you use this software, please cite it as below.
[52] Grant Van Horn, Oisin Mac Aodha, Yang Song. Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam. Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition, pages 8769-87677777 7769-8778, 2018.
[53] Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun datahase: Large- scale scene recognition from abhey to 70o. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3485-3492, 2010.
[54] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. 40(6):1452 1464, 2017.
[55] Mircca Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3606-3613, 2014.
[56] Vaishaal Shankar. Achal Dave, Rebecca Roelofs, Deva Ramanan, Benjamin Recht, and Ludwig Schmidt. Do image classifiers generalize across time? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9661-9669, 2021.
[57] Mahdi Pakdaman Nacini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI Conference on Artificial Intelligence, 2015.
[58] John P Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt. Accuracy the line: the strong correlation between out-of-distribution and in inddistribttion generalization. In International Conference on Machine Learning, pages 7721-7732 2021.
[59] Peter Huber. Robust statistics. In International Encyclopedia of Statistical Science, pages 1248-1251. Springer, 2011.
[60] Muhammad Muzammal Nascer, Kanchana Ranasinghe, Salman H Khan, Munawar Hayat, Fahad Shah- baz Khan, and Ming-Hsuan Yang. Intriguing properties of vision transformers. In Advances in Neural Information Processing Systems. pages 23226-23308, 2021.
[61] Chongzhi Zhang, Mingyuan Zhang, Shanghang Zhang, Daisheng Jin. Qiang Zhou, Zhongang Cai. Haiyu Zhao, Xianglong Liu, and Ziwei Liu. Delving deep into the generalization of vision transformers under dis- tribution shifts. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, pages 7277-7286, 2022.
[62] Robert Geirhos, Patricia Rubisch, Jonas Rauber, Carlos R Medina Temme, Claudio Michaelis, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Inducing human-like like shape bias leads to emergent human-level distortion robustness in cnns. Journal of Vision. 19(10):209c-200 2019.
[63] Mingxing Tan and Quoc Lc. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning, pages 6105-6114 PMLR, 2019.
[64] Irwan Bello, William Fedus, Xianzhi Du, Ekin Dogus Cubuk, Aravind Srinivas, Tsung- Yi Lin, Jonathon Shlens, and Barret Zoph. Revisiting resnets: Improved training and scaling strategies. In Advances in Neural Information Processing Systems, pages 22614-22627. 22627, 2021.
[65] Mingxing Tan and Quoe Le. Efficientnetv2: Smaller models and faster training. In International Conference Machine Learning, pages
PMLR, 2021.
13

---

[66] Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of out-of-distribution detection. In Advances in Neural Information Processing Systems, pages 7068-7081. 2021.
[67] Kartik Gupta, Amir Rahimi, Thalaiyasingam Ajanthan, Thomas Mensink, Cristian Sminchisescu, and Richard Hartley. Calibration of neural networks using splines. In International Conference Learning Representations. 2021. URL httpp:://operreviioee
[68] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages (6816-1682)) 2022.
[69] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9))337772344, 2022.
[70] Aleksandar Shtedritski, Christian Rupprecht, Andrea Vedaldi. What does clip know about circle? visual prompt engineering for vlms. arXiv preprint arXiv:230.00611, 2023.
[71] Hieu Pham, Zihang Dai, Golnaz Ghiasi, Hanxiao Liu, Adams Wei Yu, Minh-Thang Luong, Mingxing Tan, and Quoe V Le. Combined scaling for zero-shot transfer learning. CoRR, abv/1111.0050, 2021.
14


Reviews:
[{"review_id": "AfjabtJUDu", "reviewer": "Reviewer_8Bsp", "summary": "This paper aims to provide a comprehensive evaluation of robustness for pretrained vision-language models. Specifically, the authors benchmark around 100 pretrained models/classifiers. Based on these empirical results, this paper also provides corresponding discussions and analysis.", "questions": "Please see weaknesses.", "limitations": "Limitations have been discussed by the authors.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The robustness problem of CLIP like model is a valuable topic to study.\n2. This paper provides a very comprehensive benchmark for CLIP robustness problem. It may contribute to several benefit for the following research.\n3. Corresponding discussion and analysis are solid to further inspire study in this area.", "weaknesses": "Even if I am still concerning the technical contribution of this paper for the NeurIPS conference, I recognize the workload and benchmarking work of this paper. Thus, only a little comments for the weaknesses:\n1) What about further adding image-text retrieval evaluation and analysis? since CLIP can be used for both classification and retrieval. It may make this paper more solid.\n2) I would like to discuss with other reviewers about the contribution significance of this benchmarking work to adjust my final score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">Q1. Further adding image-text retrieval evaluation and analysis ... make this paper more solid\n\nThanks. Following this Insightful suggestion, we included retrieval tasks on MS-COCO during the rebuttal. Figure R-1 in the updated PDF reports the results. We plot retrieval performance (image-to-text and text-to-image retrieval) relative to zero-shot image classification accuracy on the ImageNet validation set.\n\nOn both retrieval tasks, we observe that retrieval performance correlates with image classification performance. In addition, CLIP models trained on different training sources appear to have different trends. We will include and discuss this new analysis in the revised version.\n\n>Q2. Contribution significance of this benchmarking work\n\nThanks for raising this discussion. Beyond the existing analysis of CLIP's classification robustness, our study advocates for the integration of three novel safety-driven objectives: factor-level robustness, OOD detection, and uncertainty calibration. This allows us to thoroughly assess and understand the impact of critical factors on CLIP models’ robustness, including training source, quantity, network structure, test-time prompt, and fine-tuning strategy. Through extensive and careful analysis, our benchmark underscores the impact of training sources across three new objectives.\n\nFurthermore, our extensive studies have unveiled several previously unknown aspects of CLIP models, deepening our understanding of their robustness behaviours. For instance, we demonstrate the shape bias in CLIP models' predictions, which diminishes after fine-tuning on ImageNet. Contrary to existing findings, CLIP models are not always more calibrated than other ImageNet models, and we attribute this to the impact of both training data distribution and quantity. Also, training sources and fine-tuning procedures have crucial effects on their  OOD detection performance. Furthermore, test-time prompts do not impact CLIP's visual factor-level robustness but influence the performance trend of OOD detection and uncertainty calibration.\n\nMoreover, as very recent research [a] highlights the importance of training dataset curation, our benchmark provides comprehensive metrics to assess the curated datasets, alongside the classification performance.\n\n[a] DataComp: In search of the next generation of multimodal datasets, Arxiv 2023\n"}, {"review_id": "9J2a4EIMIZ", "reviewer": "Reviewer_9D3z", "summary": "This paper analyzes the CLIP model's robustness through a large number of experiments, including three main points: resilience to visual factor variations, calibrated uncertainty estimations, and the ability to detect anomalous inputs.", "questions": "1. Please provide some details about the experiments.\n2. I want to ask whether parameter-efficient fine-tuning (PEFT) methods, such as LORA or Adapter, hurt the performance of OOD detection, given that fine-tuning sometimes can.", "limitations": "The authors has provided the limitations of their work.  This paper has done a lot of experiments and provided some novel discoveries that I think contribute to the CLIP community.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1.  The experiments in this paper are very sufficient, the research content is solid, and some new views are proposed from the experimental results. There is a rich analysis of the robustness of CLIP.\n2. This paper makes a significant contribution to the field by providing a comprehensive evaluation of CLIP models.  Furthermore, the experimental findings presented in this paper offer valuable insights for future endeavors aiming to enhance the out-of-distribution (OOD) detection performance and robustness of CLIP models.", "weaknesses": "There is no deeper analysis of the reasons behind these experimental results in this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">Q1. There is no deeper analysis of the reasons behind these experimental results in this paper \n\nThis work emphasizes the incorporation of three new safety-driven objectives: factor-level robustness, OOD detection, and uncertainty calibration. This enables a comprehensive assessment of critical factors on CLIP models' robustness, encompassing training source, quantity, network structure, test-time prompt, and fine-tuning strategy. Our benchmark highlights the significance of training sources in this context.\n\nBuilding on the insightful comments from all reviewers, during the rebuttal, we delved deeper into several intriguing aspects, including the performance difference between LAION and WIT (Q4/Reviewer A6cY), the retention of shape bias using contrastive loss for fine-tuning (Q1/Reviewer A6cY), the possibility of spill-over effects from data augmentation impacting unrelated robustness factors (Q3/Reviewer A6cY), and potential data overlaps in OOD detection (Q5/Reviewer jPAm).\n\nWe acknowledge the potential for deeper analysis of our observations and consider the above discussion as a starting point that could inspire further research.\n\n>Q2. Please provide some details about the experiments\n\nIn the appendix, we provide illustrations of the publicly available models and datasets used in our study. We follow: ImageNet-X [a] to conduct analysis on models' visual factor level robustness; Cue-conflict stimuli [b] to study shape-bias in model decisions; [c] to understand the CLIPs' performance on zero-shot OOD detection and [d] to investigate the quality of uncertainty estimation.\n\n[a] ImageNet-X: Understanding Model Mistakes with Factor of Variation Annotations, ICLR'23\n\n[b] ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness, ICLR'19\n\n[c] Delving into Out-of-Distribution Detection with Vision-Language Representations, NeurIPS'22\n\n[d] Revisiting the Calibration of Modern Neural Networks, NeurIPS'21\n\nFurthermore, we are dedicated to facilitating future research on CLIP analysis, and as part of this commitment, we will release the plot codes and evaluation codes for the community to use and build upon. \n\n>Q3. I want to ask whether parameter-efficient fine-tuning (PEFT) methods, such as LORA or Adapter, hurt the performance of OOD detection, given that fine-tuning sometimes can\n\nInsightful point. During the rebuttal, we studied the impact of parameter-efficient fine-tuning methods on OOD detection. In Figure R-2 of the uploaded PDF, we report the results of 8 CLIP models fine-tuned by CoOp[e] and Tip-Adapter[f]. We find that both two methods increase the classification accuracy of CLIP models, while decreasing the OOD detection performance. It would be interesting to further study the effect of PEFT methods on OOD detection. Also, increasing both classification and OOD detection performance would be a promising direction.\n\n[e] Learning to Prompt for Vision-Language Models, IJCV'22\n\n[f] Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification, ECCV'22\n\n"}, {"review_id": "gLrSqERsfh", "reviewer": "Reviewer_rieU", "summary": "Authors closely study the robustness of vision-language models. They try to investigate their robustness in terms of common visual attributes, detecting OOD inputs, and their power in providing calibrated predictions. They consider many different CLIP models and other vision encoders with different architectures and training procedures to have a comprehensive study and fairly compare CLIP models with other ones.\n\nThey provide some more detailed findings about these models w.r.t to the aforementioned criteria.", "questions": "\nAre these results generalizable and reliable? i.e., are we going to see similar trends for future CLIP models as well? I would want to get some more insights about these findings. At this point, this study looks completely empirical and I am not convinced if they are statistically significant. Can we really trust those plots and argue some general statements?", "limitations": "Limitations are addressed.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ This paper runs an extensive set of experiments using various models, various datasets, and under different settings.\n+ Therefore, these results will be insightful for practical use cases where people want to decide which model to use or diagnose possible errors/failures of their models on different conditions.\n", "weaknesses": "+ I didn't see enough new ideas in this paper.\n+ I mean, running extensive studies is definitely valuable, practical, and insightful, but is there other similar work published in NeuriPS where scaling up and running more experiments is the main contribution? I would appreciate it if the authors correct me in understanding their main contribution and change the rating correspondingly.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">Q1: Did not see enough new ideas ... running extensive studies is definitely valuable, practical, and insightful, but is there other similar work published in NeurIPS where scaling up and running more experiments is the main contribution?\n\nWe appreciate your recognition of our extensive studies and experiments. We clarify that our contributions go far beyond the scale of experiments:\n\n- New perspectives: In contrast to existing analysis paradigms (e.g., [a,b,c]) centred around overall classification accuracy, we advocate for the integration of three novel safety-driven objectives: factor-level robustness, OOD detection, and uncertainty calibration. This approach allows us to thoroughly assess and understand the impact of critical factors on CLIP models’ robustness, including training source, quantity, network structure, test-time prompt, and fine-tuning strategy.\n\n- New observations: Our extensive investigation into CLIP models uncovers several previously under-explored aspects. Key observations include the reduction of shape bias in CLIP predictions following ImageNet fine-tuning. Contrary to previous assumptions, CLIP models are not consistently more calibrated than other ImageNet models, owing to training data distribution and quantity. The significance of training sources and fine-tuning strategies is evident in their impact on OOD detection performance. Furthermore, while test-time prompts do not affect CLIP's visual factor-level robustness, they influence the trends in OOD detection and uncertainty calibration.\n\n- Comprehensive data curation metric: Aligned with recent research [d] underscoring the importance of training dataset curation, our benchmark introduces comprehensive metrics to evaluate curated datasets alongside their classification performance.\n\n    [a] Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP, NeurIPS'22\n\n    [b] Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP), ICML'22\n\n    [c] The Evolution of Out-of-Distribution Robustness Throughout Fine-Tuning, TMLR'22\n\n    [d] DataComp: In search of the next generation of multimodal datasets, Arxiv 2023\n\n> Q2. Are these results generalizable and reliable? ... similar trends for future CLIP models as well? Can we really trust those plots and argue some general statements?\n\nThanks for raising this discussion. **First**, to ensure the validity and reliability of our study, we followed established practices [a,b,c] meticulously, giving careful consideration to various factors such as training sources, network architectures, fine-tuning procedures, test datasets, and other comparable ImageNet models.\n\nWe adopt methodologies from previous research to study each objective: ImageNet-X [e] for analyzing visual factor-level robustness; Cue-conflict stimuli [f] to examine shape bias in model predictions; [g] to gauge CLIP's zero-shot OOD detection capabilities; and [h] to delve into the quality of uncertainty estimation.\n\n[e] ImageNet-X: Understanding Model Mistakes with Factor of Variation Annotations, ICLR'23\n\n[f] ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness, ICLR'19\n\n[g] Delving into Out-of-Distribution Detection with Vision-Language Representations, NeurIPS'22\n\n[h] Revisiting the Calibration of Modern Neural Networks, NeurIPS'21\n\n**Second**, we firmly believe that the insights gained from our analysis extend beyond the CLIP models evaluated in this study and are applicable to future models as well. To validate this point, during the rebuttal, we expanded our investigation to include the very latest CLIP models (beyond the submission deadline). These models are trained on DataComp or CommonPool [d]: they are trained on the same training source with LAION but with different dataset quantities.\n\nThe results shown in Figure R-3 in the uploaded PDF have demonstrated that our observations hold true for these newly included models, further affirming the scalability and generalizability of our findings.\n\n**Third**, to facilitate further research and analysis on CLIP models, we will release our experimental setups, including the database and plot codes.\n\nThe above discussion will be included in the revised version."}, {"review_id": "7703ZfAG1L", "reviewer": "Reviewer_jPAm", "summary": "This paper studies and compares CLIP and CLIP-FT to standard models on a range of different tasks including OOD robustness, OOD detection, and model calibration. The paper constitutes a meta-analysis across different model architectures / training datasets / training algorithms or loss functions.\n\nThe authors claim:\n\n####  Robustness\nCLIP sometimes outperforms other models on certain visual factor variations, but sometimes underperforms them.\nCLIP models are more shape biased (authors claim due to VLM training), resolution increase decreases shape bias.\nFine-tuning makes CLIP behavior more similar to other image models.\n\n####  OOD Detection\nCLIP generally performs better than other models in OOD detection; the relationship between IID accuracy and OOD detection performance largely follows accuracy on the line.\nFine-tuning negatively affects performance.\n\n####  Model Calibration\nCLIP isn’t significantly more calibrated than other models, data distribution affects calibration.\nTemperature scaling makes CLIP more calibrated than other models, and removes dependence on data distribution.\nTemperature scaling is apparently more important for CLIP.\nWhen temperature scaled CLIP is better OOD calibrated than other models.\n\n####  Test Time Prompts\nUsing more prompts generally helps across tasks, except in the case of some visual factor variations where it doesn’t really make a difference", "questions": "What is the central claim/argument?\n*  Maybe pivot towards study similar to [2] in the context of OOD detection and model calibration? That is: Be more specific on what causes the observed OOD detection results.\n*  What about the data distribution matters for OOD detection and model calibration?\nIn general, I would appreciate some deeper insights, I don’t feel like I learned all that much from this paper.\n\n### Robustness results (Section 4)\n\nLines 138-144: The authors observe that CLIP models perform better on certain visual factors and worse on others. Some interpretation of the results is necessary to make the observations valuable to the research community. For example, the authors find that fine-tuning on ImageNet changes the robustness properties of CLIP models, but offer no explanation / interpretation of the results.\n\n\n###  OOD detection results (Section 5):\n\nLine 209: “Upon closer examination of the training distribution, we have observed that the correlation trend between ID accuracy and OOD detection performance is largely dependent on the training source.” -> It is not clear to me how sensible the OOD detection task is for CLIP models. While LAION may not contain e.g. all of the ImageNet training images, doing zero-shot inference is possible because CLIP has seen similar images during training, thus, “zero-shot” is ultimately a misleading term which becomes an issue once OOD detection is considered. It is not clear to me that the datasets used for OOD detection here, namely iNaturalist [52], SUN [53], PLACES [54], TEXTURE [55] and ImageNet-O [7], are actually not part of LAION in the first place, making measuring performance on those datasets an ID task, rather than an OOD task. Thus, I find it unsurprising that the training data distribution matters a lot for this particular task. Further, comparing the performance of models trained on ImageNet which has no intersection with ImageNet-O to models trained on LAION which may have a high intersection with ImageNet-O seems an unfair comparison to me. Could the authors please comment on this issue, in particular, on the sensibility of doing OOD detection for CLIP models trained on e.g. LAION?\n\nContinuing on this point: Line 212: “Moreover, with same ID accuracy, CLIP models trained on WIT exhibit superior OOD detection performance compared to their counterparts trained on LAION on three OOD scenarios. This further indicates the importance of training source selection for CLIP. When developing dataset curation methods, it is valuable to investigate the influence of training sources on OOD detection performance.” I would like to see a more nuanced and detailed discussion on this finding. Is it maybe that WIT just has a larger overlap with the OOD detection datasets? What is the concrete suggestion here? \n\nLine 219: “Some CLIP-FT models even achieve worse OOD detection performance than Zero-shot CLIP models.” This is in line with the points above and what we may be seeing here is that with fine-tuning, the model weights move towards being compatible with ImageNet-1K and further away from the original training distribution which likely overlaps with OOD detection test sets.\n\nDue to all of the points raised above, I find the sentence in Conclusion, line 303: “Furthermore,\nwhile maintaining comparable accuracy on in-distribution dataset, CLIP models tend to exhibit higher performance in OOD detection.” to be misleading.\n\nThe recent paper [4] (published after the NeurIPS submission deadline) makes the very similar observation that the CLIP training dataset has a large influence on zero-shot OOD detection, and also remarks on the effects of the fine-tuning procedure. Comparing how the conclusions made in this paper align with theirs would be very helpful. An evaluation on their dataset would also be interesting to see, since they show that Places, Texture and ImageNet-O have severe issues with ID data.\nThey seem to contradict the notion that CLIP zero-shot is well suited for OOD detection. I see no reason not to compare fine-tuned CLIP models with those fine-tuned on ImageNet 21K, which are very commonly used.\nFurther, the evaluation of OOD detectors should include strong baseline models like the Mahalanobis detector, compared to which the CLIP based models clearly fall behind according to the evaluations in [4].\nNot including the mentioned important baselines (which are also included in [5]) might be the main explanations why the evaluations in Figure 3 (and in the previous papers claiming zero-shot CLIP with MCM being a strong OOD detector) suggest such a positive image of the CLIP based models.\n\n\nMore centralized and focused claims/analyses would raise my score.\n\n\n#### References: \n\n* [1] Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP\n* [2] Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP)\n* [3] The Evolution of Out-of-Distribution Robustness Throughout Fine-Tuning\n* [4] Bitterwolf et al. ICML 2023: “In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluation”\n* [5] Yang et al. NeurIPS 2022: “OpenOOD: Benchmarking Generalized Out-of-Distribution Detection”", "limitations": "Limitations were not addressed.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The introduction is well written, motivates the paper and gives a clear overview of the claims.\n- Well written, easy to follow\n- Objective tone/analysis and solid experimental design\n- Some nice points/findings dispersed throughout the paper", "weaknesses": "- No clear central argument or claim, paper seems more like a pastiche of different experiments rather than a focused analysis. There is little interpretation of the experimental findings. I have written very detailed questions considering the OOD detection results, but similar questions apply to all sections.\n- It looks to me that the paper tries to do too much in one paper and ends up being imprecise / too shallow on interpreting the results. For example, [2] focuses solely on how the data distribution affects robustness. Here, the authors observe this finding and only note that “The above observations highlight the importance of the choice of training source in determining not only the overall accuracy but also the factor-level behaviors of CLIP models. This suggests that visual factor-level robustness should be considered when designing the training source for CLIP models.” This is a “political” answer which does not provide concrete action items for future researchers.\nSome claims/analyses are probably wrong:\n\n    *  “Temperature scaling reveals a consistent trend of CLIP models, and they still lie on a distinct trend from other models.” This doesn’t seem to be the case ALL of the time. SSL models look pretty similar to CLIP on the “NLL (Temp-scaled)” and “ECE (Temp-scaled)” Imagenet-A plots. Amount of difference seems to be dataset specific. In fact, I only see a significant effect in ImageNet-A, ECE vs ECE-temp scaled, where the temperature scaling seems to affect all models.\n    *   ““This observation indicates that unlike robustness and out-of-distribution detection, the calibration of CLIP models is influenced by both training data distribution and quantity.” I don’t see how this follows from your data - aren’t robustness and OOD detection also influenced by data distribution?\n    *  The CLIP training dataset likely overlaps with the datasets used for OOD detection which makes it unclear how meaningful the claims are in regards to CLIP’s good OOD detection performance. This issue should at least be discussed.\n    *   OOD detection evaluations are missing important baselines (see details below).\n\n- The paper lacks novelty: The influence of training data / fine-tuning affecting CLIP performance have been studied in detail before [1,2,3]. This paper evaluates many different models and is thus a meta-study of the previous findings. In that case, the literature review should be expanded, and the paper must be better positioned, e.g. in “in [3], the authors investigate the influence of fine-tuning on OOD robustness. We here investigate whether their claims hold across a broader range of models'' or something like this. Though [3] already provides a thorough and careful empirical investigation across many different models, and I am not sure whether this paper offers much beyond the results presented in [3]. \n- The graphs were sometimes a bit hard to read/interpret - there were a lot of them and usually the conclusion was different/unique depending on the plot.\n- Some kind of unifying principle might be nice.\n\n### Minor:\n- Please be more specific about results/claims in the abstract.\n- 3.1 - Would be nice to explain/list model choices at some point, i.e. refer to Supplement A.2. here\n- 3.2 (Robustness) - more explanation of why you chose 10 of the 16 would be nice (robustness)\n- Line 40, style / grammar: “our study further study .. “\n- Line 42: “training distributions”, it should be distribution\n- Line 50, remove the extra space before the full stop\n- Line 280: “highlighting their potential for the robust and reliable applications” → “highlighting their potential for robust and reliable applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">Q1: No clear central argument ... unifying principles\n\nThanks. Our central argument is the necessity of a comprehensive evaluation of CLIP's robustness. In contrast to current approaches focused on classification accuracy, we propose integrating three new safety-driven objectives: factor-level robustness, OOD detection, and uncertainty calibration. This enables a thorough assessment of critical factors (e.g., training source and prompt) on CLIP's behaviours. Our analysis highlights the significance of training sources on the three objectives.\n\n>Q2-1: Interpretation on robustness … fine-tuning changes robustness\n\nThanks. As pointed out by Reviewer A6cY (Q1), CLIP uses color-jitter, which could improve its robustness to color and brightness but hurt Pose. Also, fine-tuning CLIP with contrastive loss maintains shape-bias in prediction. \n\nMoreover, as discussed in [a], standard fine-tuning introduces spurious correlation for CLIP and thus hurts robustness.\n\n[a] Masked Images Are Counterfactual Samples for Robust Fine-tuning\n\n>Q2-2: Maybe pivot towards [2] for OOD detection and calibration\n\nThanks. Following [2], we indeed investigated crucial factors for CLIP, including training source, dataset quantity, test-time prompt, contrastive loss, and architecture. Through this analysis, we emphasize the significance of data distribution: 1) data distribution affects the accuracy trend of zero-shot CLIP in OOD detection and calibration; 2) CLIP's calibration is further influenced by dataset quantity. Additionally, we stress the role of prompt learning, as different prompts can alter the performance trend on both tasks.\n\n>Q3-1: Some claims probably wrong: 'TS reveals a consistent trend of CLIP … distinct from other models'\n\nWe clarify that Fig.4 gives two observations: 1) Before TS, CLIP models from different sources and subsets exhibit distinct trends; 2) Post TS, CLIP models exhibit a similar trend, diverging from other models: with comparable classification accuracy, they demonstrate lower calibration errors.\n\n>Q3-2: 'Unlike robustness and OOD detection, calibration is influenced by both training distribution and quantity'\n\nWe clarify that CLIP model's factor-level robustness and OOD detection are affected by training distribution, while calibration is influenced by both training distribution and quantity.\n\n>Q4: Lack novelty … training data/fine-tuning influence have been studied in [1-3] … a meta-study.\n\nOur work significantly differs from [1-3], which concentrates solely on classification robustness under distribution shifts. In contrast, our study investigates CLIP models' robustness from three new perspectives: factor-level robustness, OOD detection, and calibration. Furthermore, [1-3] do not analyze how training data and fine-tuning impact CLIP models across these aspects. We give such analysis and comprehensively examine training distribution, quantity, test-time prompt, and fine-tuning schemes, revealing novel observations into CLIP models' behaviour.\n\n>Thanks for raising the insightful discussion on CLIP's sensibility for OOD detection. We think potential data overlap does not compromise CLIP's promising results and will not introduce sensibility. Please see the below: \n\n>Q5-1:  CLIP may have seen similar images during training … “zero-shot” is misleading\n \nWe follow the notion of “zero-shot” OOD detection (Ming et, al. 2022): CLIP allows users to redefine ID/OOD classes flexibly without requiring detector retraining.\n\n>Q5-2:  Not clear OOD datasets are not part of LAION ... unfair comparison\n\nWe hold the same opinion with the authors of LAION: ‘we do not consider potential test set overlap to be a serious threat for the validity of results’. \n\nDataset overlap may arise if OOD datasets are also included in Common Crawl. In classification, OpenAI found only a few examples of substantial performance differences due to data overlap. Similarly, as discussed in [4] for OOD, the overlap between IN-21K with NINCO does not cause substantially different changes between models with and without pretraining on IN-21K. This further indicates potential data overlap will not introduce sensibility for CLIP.\n\n>Q5-3: Maybe WIT has a larger overlap than LAION with OOD datasets\n\nThe performance gap in OOD (and classification) could be attributed to training source quality and filtering process. Please see Q2/Reviewer A6cY.\n\n>Q5-4: Fine-tuning shifts weights towards IN-1K and away from original distribution that likely overlaps with OOD datasets\n\nFine-tuning CLIP on IN-IK could learn unsuitable features for OOD detection due to spurious correlation [a]. Further, using a broader set of data/classes from IN-21K may potentially mitigate this issue.\n\n>Thanks for sharing post-submission work [4]. The discussion has strengthened the OOD detection aspect of our study:\n\n>Q6-1: [4] observes CLIP training dataset has a large effect on zero-shot OOD detection and remarks on fine-tuning effect ... fine-tuned CLIP on IN-21K\n\nAfter careful check, [4] only included two zero-shot CLIP models and did not study training set impact. \n\nIN-12K is a subset of IN-21K with excluded classes having few samples, resulting in approximately 85% overlap [4]. IN-12K is widely used for CLIP fine-tuning. Moreover, [4] reports fine-tuning on IN-21K helps OOD detection, and we observe fine-tuning CLIP on IN-12K is helpful. We will cite [4] and discuss the mutual finding.\n\n>Q6-2: Evaluation on their dataset ... Mahalanobis detector\n\nThanks. We noted that [4] reports \"it is difficult for many OOD detectors to improve consistently over MSP\", supporting the rationale for using MSP in our study. During rebuttal, we established our benchmark on NINCO as per [4], using Mahalanobis and MSP. Please refer to Figure R-4 in the uploaded PDF.\n\nOur findings in Sec. 5 align consistently with NINCO results. Further, we observe promising zero-shot CLIP detection accuracy,  compared to other ImageNet models with Mahalanobis detector."}, {"review_id": "XgtIV3J4kV", "reviewer": "Reviewer_A6cY", "summary": "This paper performs a comprehensive study of various CLIP models on robustness to different visual factors, out-of-distribution detection, and calibrated uncertainty estimations. A total number of 53 CLIP models trained on different training sources and sizes, and different architectures, with additionally 32 CLIP models fine-tuned on ImageNet are studied.", "questions": "I have the following questions that would help me better understand the paper, and I would appreciate the authors' reply to them:\n\n1. Is there any insight on why WIT differs from LAION when serving as the CLIP training set, since I thought both of them were deemed to be a comprehensive snippet of the web image-text pairs?\n\n2. What about the other self-supervised models like MAE when compared with CLIP in terms of robustness? This also helps demystify the factors of datasets and objectives.\n\n3. ImageNet-1k contains 1.3 million images, and ImageNet-21k contains > 20 million images, which are not far away from the smallest LAION dataset considered in the paper. Is it safe to say that when the dataset size is similar, the supervised objective is still better in terms of calibrated uncertainty estimations? How about robustness and OOD detection?", "limitations": "The authors discuss the broader impacts in the conclusion section.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "[Originality]\n\nTo the best of my knowledge, no previous studies have done such experiments on the 3 aspects to study CLIP models, especially considering various CLIP models trained on different datasets. Some of the observations made in this paper are new and complementary to the previous studies. For example, CLIP models are not robust in all aspects - they are less robust than models trained on ImageNet in a supervised way when poses are changed. Therefore, the findings are novel. \n\n[Significance]\n\nAlthough most conclusions that can be drawn from the experiments are already known, e.g., CLIP models are more robust than supervised models, some of them are less well-known and may be valuable to the community. For example, CLIP models trained on WIT perform better than those trained on LAION on OOD detection.\n\n[Quality & Clarity]\n\nThe quality and presentation of this paper are good. It is easy to understand and follow.\n\n", "weaknesses": "Some of the discussion on the experiment observations might need more support. \n\n1. Line 177: \"The shape bias of CLIP may be attributed to its objective, which involves training the model to associate text and image pairs\": in Figure 2, it seems that fine-tuning CLIP models on ImageNet (with supervised objective?) decreases the shape bias. However, it is also possible that the data source (ImageNet) could be the reason. Is it possible to decouple this? Say fine-tune CLIP with the contrastive objective on ImageNet and see if the shape bias stays the same.\n\n2. Line 246: \"We notice that CLIP models trained on LAION-80M dataset exhibit lower calibration performance when compared to standard models.\" Is this fair to compare as CLIP models trained on LAION-80M generally have lower accuracy?\n\nAnother minor concern I have is that some results from previous studies are not clearly discussed. For example, in ImageNet-X, it is already observed that \"color-jitter augmentation improves robustness to color and brightness, but hurts robustness to pose.\" Since color-jittering is widely used in the CLIP training, this should be discussed in the paper.\n\n[1] Badr Youbi Idrissi, Diane Bouchacourt, Randall Balestriero, Ivan Evtimov, Caner Hazirbas, Nicolas Ballas, Pascal Vincent, Michal Drozdzal, David Lopez-Paz, and Mark Ibrahim. Imagenet-x: Understanding model mistakes with factor of variation annotations. In International Conference on Learning Representations, 2022.\n\n[Minor]\nLine 94: ImageNet 32 fine-tuned CLIP models -> 32 ImageNet fine-tuned CLIP models\nLine 141: Small -> Smaller\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">Q1: Figure 2, fine-tuning CLIPs on ImageNet (with supervised objective?) decreases shape bias … data source (ImageNet) could be the reason … fine-tune CLIP with contrastive objective to see if shape bias stays the same.\n \nInsightful suggestion. All fine-tuned CLIP models in Fig. 2 use supervised cross-entropy loss. To decouple the effects of contrastive learning and data source, we employed zero-shot CLIP with ViT-B-32 and fine-tuned it on ImageNet by two methods: standard cross-entropy and contrastive loss [a].\n\nThe shape bias extents are measured as 0.58, 0.40, and 0.56 for zero-shot CLIP, standard fine-tuned CLIP, and contrastive-loss fine-tuned CLIP, respectively. This indicates ImageNet might not be the primary cause of the shape-bias decrease. Also, we maintain speculation that associating image embeddings with text embeddings could potentially help learn shape-biased models.\nWe will include the above discussion in the revised version.\n\n[a] Finetune like you pretrain: Improved finetuning of zero-shot vision models. CVPR’23\n \n>Q2. Is it fair to compare CLIP trained on LAION-80M and standard models as the former ones generally have lower accuracy?\n \nThanks for raising this discussion. First, it is a common practice to compare uncertainty estimation performance between models with varying accuracy levels, as exemplified by references [b,c,d] and many others.\n \nSecond, in the main paper, CLIP models trained on LAION-80M are used to demonstrate that CLIP models do not always exhibit superior calibration compared to other ImageNet models, contrary to existing observations. We further clarify that, at comparable accuracy levels, CLIP models trained on LAION-80M are observed to have higher calibration error (ECE).\n\nLast, we firmly believe in the necessity of a comprehensive evaluation of CLIP models' robustness, driving the primary motivation behind our study. When measuring CLIP models' robustness, we advocate for considering more perspectives alongside the commonly used classification accuracy\n\nWe will revise the Line 246-247 to make it clear.\n\n[b] On calibration of modern neural networks, ICML'17\n\n[c] Improving model calibration with accuracy versus uncertainty optimization, NeurIPS'20\n\n[d] Revisiting the Calibration of Modern Neural Networks. NeurIPS'21\n \n>Q3. Some results from previous studies are not clearly discussed … “color-jitter augmentation improves robustness to color and brightness but hurts robustness to pose\".\n \nInsightful point. We agree that data augmentation used in CLIP training is crucial for their factor-level robustness. Specifically, data augmentations can improve robustness to related factors, but with spill-over effects to unrelated factors. As the reviewer points out, colour-jittering serves to improve robustness to color and brightness variations while influencing object pose. Similarly, scale-based augmentation could facilitate the learning of scale-invariant features. It would be interesting to use our benchmark to study the impact of data augmentations and other dataset curation techniques (e.g., filtering). We will include the above discussion in Section 4.1. \n\n>Q4: Any insight on WIT differs from LAION\n \nGood suggestion. We think the performance difference between WIT and LAION datasets could be attributed to two main factors:\nFirst, dissimilarity in data sources used for training - Common Crawl for LAION and unknown data sources for OpenAI's training set. As pointed out by [e], Common Crawl may be a noisier data source (weaker connection between images and associated text) or contain less diverse images. \n\nSecond, filtering process in the LAION dataset curation: LAION employed a small-scale ViT-B/32 model for filtering, which may result in a substantial amount of poorly matching image-text pairs. Moreover, recent work [f] suggests that more appropriate curations on LAION give the competitive performance of CLIP models compared to those trained on WIT.\n \nThe above further highlights the importance of dataset curation for learning robust CLIP models. We will include the above discussion.\n \n[e] LAION-5B: An open large-scale dataset for training next-generation image-text models. NeurIPS'22\n\n[f] DataComp: In search of the next generation of multimodal datasets, Arxiv 2023\n\n>Q5. Other self-supervised models like MAE … demystify the factors of datasets and objectives\n\nThanks for this valuable suggestion. During rebuttal, we evaluated three MAE models on three aspects in Figure R-3 in the uploaded PDF. \n\nHere are our observations: 1) Visual factor-level robustness: three MAE models lie on the area of models pre-trained on more data; \n2) OOD detection: MAE models also lie in the area with other models pre-trained on more data. Some zero-shot CLIP models achieve higher performance than MAE models;\n3) Calibration: before TS, MAE models have higher uncertainty estimation performance than CLIP trained on LAION while lower than CLIP trained on WIT. Post TS, CLIP models become better than MAE. \n\n>Q6: Is it safe to say that when the dataset size is similar, the supervised objective is still better in calibration, robustness, and OOD detection \n\nInteresting point. According to the discussion in [g], when the dataset size is small (e.g., 15M), CLIP models’ classification accuracy on IN-1K would drop. In contrast, under the same dataset size, supervised objectives achieve high-accuracy models. When considering a similar training set scale, we anticipate the following phenomena:\n\n1) Factor-level Robustness: CLIP exhibits lower accuracy on each factor while maintaining the same relative robustness trend as models trained on larger datasets.\n\n2) OOD Detection: CLIP's performance weakens compared to supervised models.\n\n3) Calibration: Given the impact of training set quantity on CLIP, we expect it to achieve less effective calibration results than supervised models.\n\n[g] Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP\n"}]

Author rebuttal:
Dear Reviewers,

Thank you for your detailed and thoughtful feedback. Inspired by your valuable suggestions, we have added more experimental analyses and included the suggested discussions. 

We summarize the experiments in the uploaded PDF:
- In Fig. R-1, we evaluate the retrieval performance of zero-shot CLIPs. We observe retrieval performance of CLIPs correlates with their classification performance. Also, the performance trend is influenced by training dataset distribution.

- In Fig. R-2, we study the OOD performance of 8 new CLIP models finetuned by parameter efficient fine-tuning methods: CoOp and Tip-Adapter. Both methods improve models’ classification performance but lead to performance drops in OOD detection. 

- Fig. R-3 includes post-submission CLIP models and three MAE pre-trained models. We observe they are aligned with our observations in the main paper. 

- In Fig. R-4, we expand the OOD benchmark on NINCO, using both MSP and Mahalanobis detectors. Our findings in Section 5 of the main paper align consistently with NINCO results. Also, zero-shot CLIP shows promising accuracy, compared to other ImageNet models with Mahalanobis detector. 

- Moreover, we show the retention of shape bias using contrastive loss for fine-tuning (Q1/Reviewer A6cY)

Last, we greatly appreciate all reviewers recognize our study is comprehensive, sufficient, and solid. We emphasize that our main contributions go beyond just the scale of experiments. Specifically, our study unveils three novel perspectives (factor-level robustness, OOD detection, and uncertainty calibration) that contribute to a comprehensive understanding of CLIP models' robustness.

We hope our response has addressed the initial concerns. Please let us know if you have any other questions.

Kind Regards,

Authors


Meta-review: This paper undertakes a comprehensive study on the robustness of multiple CLIP models, analyzing their performance with respect to various visual factors, out-of-distribution detection, and calibrated uncertainty estimations. All reviewers agree that this paper is well-written, and the empirical experiments are well-designed and comprehensive. Meanwhile, some concerns are raised: 1) the technique novelty is not very significant, as some conclusions can already be reasonably drawn from existing works, and 2) some conclusions are confusing and probably wrong.

The rebuttal phase was constructive, with the authors engaging proactively to clarify and fix multiple issues in this paper. Given four reviewers agreed to accept it and one reviewer voted for borderline rejection (but be okay for accepting it), the final decision is accept.

For the final version of the manuscript, the authors must integrate all the experimental data and clarifications discussed during the rebuttal phase, particularly the exchanges with Reviewer jPAm, to ensure the high standards of NeurIPS publications.
Decision: Accept (poster)