Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
Authors: Peiyan Zhang
Haoyang Liu
Chaozhuo Li
Xing Xie
Sunghun Kim
Haohan Wang
Venue: ICLR 2024 poster (2024)
Keywords: robustness
distribution shift
reliable machine learning
Primary area: societal considerations including fairness, safety, privacy
TL;DR: We offer a dynamic evaluation protocol that evaluates vision models' robustness across generic i.i.d benchmarks, without requirement on the prior knowledge of the underlying image-label structure depicted by the images and additional human efforts.

Abstract: Machine learning has demonstrated remarkable performance over finite datasets, yet whether the scores over the fixed benchmarks can sufficiently indicate the model’s performance in the real world is still in discussion. In reality, an ideal robust model will probably behave similarly to the oracle (e.g., the human users), thus a good evaluation protocol is probably to evaluate the models’ behaviors in comparison to the oracle. In this paper, we introduce a new robustness measurement that directly measures the image classification model’s performance compared with a surrogate oracle (i.e., a zoo of foundation models). Besides, we design a simple method that can accomplish the evaluation beyond the scope of the benchmarks. Our method extends the image datasets with new samples that are sufficiently perturbed to be distinct from the ones in the original sets, but are still bounded within the same image-label structure the original test image represents, constrained by a zoo of foundation models pretrained with a large amount of samples. As a result, our new method will offer us a new way to evaluate the models’ robustness performance, free of limitations of fixed benchmarks or constrained perturbations, although scoped by the power of the oracle. In addition to the evaluation results, we also leverage our generated data to understand the behaviors of the model and our new evaluation strategies.

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
FOUNDATION MODEL-ORIENTED ROBUSTNESS: Ro- BUST IMAGE MODEL EVALUATION WITH PRETRAINED MODELS
Peiyan Zhang', Haoyang Liu?, Chaozhuo Li Xing Xie Sunghun Kim' and Haohan Wang"
Honn Kong University of Science and Technology University of Illinois at
Microsoft Research Asia
ABSTRACT
Machine learning has demonstrated remarkable performance over finite datasets, yet whether the scores over the fixed benchmarks can sufficiently indicate the model's performance in the real world is still in discussion. In reality, an ideal robust model will probably behave similarly to the oracle (e.g., the human users), thus a good evaluation protocol is probably to evaluate the models' behaviors comparison the oracle. In this paper, we introduce new robustness measurement that directly measures the image classification model's performance compared with surrogate oracle (i.e., a zoo of foundation models). Besides, we design simple method that can accomplish the evaluation beyond the scope of the benchmarks. Our method extends the image datasets with new samples that are sufficiently perturbed to be distinct from the ones in the original sets, but are still bounded within the same image-label structure the original test image represents, constrained by zoo of foundation models pretrained with a large amount of samples. As a result, our new method will offer us a new way to evaluate the models' robustness performance, free of limitations of fixed benchmarks constrained perturbations, although scoped by the power of the oracle. In addition to the evaluation results, also leverage generated data to understand the behaviors of the model and our new evaluation strategies.
INTRODUCTION
Machine learning has achieved remarkable performance over various benchmarks. For example, the recent successes various neural network architectures (He et al.) 2016a]] Touvron et al.) 2021) has shown strong numerical evidence that the prediction accuracy over specific tasks can reach the position of the leaderboard as high as a human, suggesting different application scenarios of these methods. However, these methods deployed the real world often underdeliver its promises made through the benchmark datasets (Edwards) 2019 D' Amour et al. 2020), usually due to the fact that these benchmark datasets, typically i.idd cannot sufficiently represent the diversity of the samples a model will encounter after being deployed in practice (Recht et al. 2019 Wu et al., 2023).
Fortunately, multiple lines of study have aimed embrace this challenge, and most of these works are proposing to further diversify the datasets used at the evaluation time. We notice these works mostly fall into two main categories: (1) the works that study the performance over testing datasets generated by predefined perturbation over the original i.i.d datasets, such adversarial robustness (Szegedy et al. 2013 Goodfellow et al.) 2015) or robustness against certain noises (Geirhos et al. 2019 Wang et al. 2020b); and (2) the works that study the performance over testing datasets that collected anew with procedure/dissributiion different from the one for training sets, such as domain adaptation (Ben-David et al. 2007 2010) and domain generalization (Muandet al. 2013).
"Co-corresponding authors

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
ongghe ampple
orggeal sanple
valld partrtction predectd dog
potaletion product ter
doeadar
mal in prigoeed evalation anihbd
VIQGAN rifiemntly pertarbution
The main structure of our system to generate test images with foundation models and examples of the generated images with their effectiveness in evaluation of model's robustness.
Figure 1:
Both of these lines, while pushing the study of robustness evaluation further, mostly have their own advantages and limitations tradeoff how to guarantee the underlying image-label structure of evaluation samples will be the same as the training samples: perturbation based evaluations usually maintain the image-label structure by predefining the perturbations within set of operations that will not alter the image semantics, such as é-norm ball constraints (Carlimi et al. 2019). or texture (Geirhos al., 2219). frequency-based (Wang et al.) (020b) perturbations, but are relatively limited in the variety of perturbations allowed. On the other hand, new new-dataset based evaluations maintain the image-label structure by soliciting the efforts human annotators to construct datasets with the same semantics but significantly different styles (Hendrycks et al. 2021b: Hendrycks & Dietterich, 2019). However, such new datasets may be costly to collect, and a potential issue is that they are fixed once collected and published to the research community. Ranking methods based on the fixed datasets will eventually lead to the methods overfit on certain datasetss (Duda et al. 1973 Friedman et al. [001] Yan al. 2024). While recent efforts have tried to alleviate the selection bias by collecting data from multiple sources (Gulrajani & Lopez-Paz, (2200] Koh et al.) 2021 Ye et al. [022] Wang et al. 2022b Li et al. 2019), kindly argue that dynamic process of generating evaluation datasets will certainly further mitigate this issue.
In this paper, we investigate how diversify the robustness evaluation datasets make the evaluation results credible and representative. As shown in Figure [ we aim to integrate the advantages of the above two directions by introducing new protocol to generate evaluation datasets that can automatically perturb the samples to be sufficiently different from existing test samples, while maintaining the underlying unknown image-label structure with respect models. Based on the new evaluation protocol, we introduce a new robustness metric that measures the robustness compared with the foundation model. Moreover, with our proposed evaluation protocol and metric, make study of current robust machine learning techniques to identify the robustness gap between existing models and the foundation model. This particularly important if the goal of a research direction is produce models that function reliably comparable the foundation model.
zoo of foundation
Therefore, our contributions in this paper are three-fold:
We introduce a new robustness metric that measures the robustness gap between models and the foundation model.
We introduce new evaluation protocol generate evaluation datasets that automatically perturb the samples to sufficiently different from existing test samples, while maintaining the underlying unknown image-label structure.
We leverage our evaluation metric and protocol to conduct the very first systematic study on robustness evaluation. Our analysis brings the understanding and conjectures of the behavior of the deep learning models, opening up future research directions.
2 BACKGROUND
Current Robustness Evaluation Protocols. The evaluation machine learning models in non-i.i. scenario have been studied for than decade, and one of the pioneers is probably domain
2

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
adaptation Dren- et al., 2010). In domain adaptation, the community trains the model over data from one distribution and tests the model with samples from a different distribution; in domain generalization (Muandet et al.] 2013), the community trains the model over data from several related distributions and test the model with samples from yet another distribution. To facilitate the development of cross-domain robust image classification, the community has introduced several benchmarks, such as PACS (Li et al. (017)) ImageNet-A (Hendrycks et al., 202Ib), ImageNet-C (Hendrycks & Dietterich 2019), ImageNet-Sketch (Wang et al.) 2019), and collective benchmarks integrating multiple datasets such as WILDS (Koh et al. 2021), and OOD Bench Yee et al. 2021).
While these datasets clearly maintain the underlying image-label structure of the images, potential issue is that these evaluation datasets are fixed once collected. Thus, if the community relies on these fixed benchmarks repeatedly to rank methods, eventually the selected best method may not be a true reflection of the world, but model that can fit certain datasets exceptionally well. This phenomenon has been discussed by several textbooks (Duda et al.] 1973 Friedman et al. (001). While recent efforts in evaluating collections of datasets (Gulrajani & Lopez- Paz 2020; Koh et al.) 2021, Ye et al. 202T) I might alleviate the above potential hazards of "model selection with test set", dynamic process of generating evaluation datasets will certainly further mitigate this issue.
On the other hand, one can also test the robustness of models by dynamically perturbing the existing datasets. For example, texture (Geirhos et al.] 2019), frequency-pertun attacks (e.g., ---norm constraint perturbations) (Szegedy al. 2013). While these tests do not require additionally collected samples, these tests typically limit the perturbations to be relatively well-defined (e.g., a rextree-perturbed cat image still depicts because the shape of the cat is preserved during the perturbation).
can test the model's robustness against rotation (Marcos al. 2016)). datasets (Wang et al., 2020b). or adversarial
While this perturbationn test strategy leads to datasets dynamically generated along the evaluation, it is usually limited by the variations of the perturbations allowed. For example, one may not be able to use some significant distortion of the images in case the object depicted may be deformed and the underlying image-label structure of the images is distorted. Generally speaking, most of the current perturbation-bast based test protocols are scoped by the tradeoff that a minor perturbation might not introduce enough variations to the existing datasets, while significant perturbation will potentially destroy the underlying image-label structures.
Assumed Desiderata of Robustness Evaluation Protocol. As reflection of the previous discus- sion, we attempt to offer a summary list of three desired properties of the datasets serving as the benchmarks for robustness evaluation:
Stableness in Image-label Structure: the most important property of the evaluation datasets is that the samples must represent the same underlying image-label structure as the training samples. Effectiveness in Generated Samples: the test samples should be effective in exposing defects for tested models.
A Dynamic Generation Process: to mitigate selection bias of the models over techniques that focus too attentively to the specification of datasets, ideally, the evaluation protocol should consist of dynamic set of samples, preferably generated with the tested model in consideration.
Necessity of New Robustness Measurement in Dynamic Evaluation Protocol. In previous experiments, two settings are commonly used: a "standard" test set and perturbed test set. Previous approaches rank models based on accuracy under perturbed test set (Geirhos et al. 2019 Hendrycks
al., 2021a| Orhan 2019 Xie et al.] 20200 Zhang] 2019) or other metrics such as inception score (Salimans et al., 2016). effective robustness (Taori et al. 2020) and relative robustness [Toori al. 2020). While useful for initial assessments, these metrics do not fully capture robustness in dynamic evaluation protocols. Here, comparing two models on different dynamic test sets cannot definitively determine superior model robustness, as differences in performance may result from varying test set difficulties.
The main challenge identified the lack of consistent robustness metric across test sets. Ideally, a robust model should mirror the behavior of the foundation model (e.g., human users). Therefore, direct measurement of model robustness relative to the foundation model is preferable over indirect model comparisons.
3

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
3 METHOD COUNTERFACTUAL GENERATION WITH SURROGATE ORACLE
3.1 METHOD OVERVIEW
We use (x, y) to denote image sample and its corresponding label, and use 0(x) denote the model we aim to evaluate, which takes an input of the image and predicts the label.
We (xx b) to denote image generation sys- tem, which takes an input of the starting image generate another image X within the compu- tation budget b. The generation process is per- formed as an optimization process to maximize a scoring function a( z) that evaluates the align- ment between the generated image and genera- tion goal guiding the perturbation process. The higher the score is, the better the alignment is. Thus, the generation process is formalized as
Algorithm 1 Perturbed Image Generation with Foundation Models
Input: (X, Y),
g. total number of iterations
Output: generated dataset (X, Y) for each (x. y) in (X, Y) do
generate Xg- g(x, bu; 0)
if h(xu) = then
X =
arg max ((g(x, b), z),
set Xo
for iteration b, < do generate &, g(x by; 0)
8=g
where B denotes the allowed computation budget for one sample. This budget will constrain the generated image not far from the starting image so triv-
if ((x()) y then
set X,
else
that the generated one does not converge ial solution that maximizes the scoring function.
set x = Re-1 exit FOR loop
end if
In addition, we choose the model classification loss ((((X), y) as z. Therefore, the scoring func- tion essentially maximizes the loss of given im- age in the direction a different class.
end for
else
set x = end if
use (X.y) to construct (X, Y)
Finally, to maintain the unknown image-label structure the images, we leverage the power of the pretrained giant models to scope the genera- tion process: the generated images must be considered within the same class by the pretrained model, denoted (8)), which takes in the input of the image and makes prediction.
end for
Connecting all the components above, the generation process will aim to optimize the following:
arg max b= subject to h(x)= y.
b),
Our method is generic and agnostic to the choices of the three major components, namely 0, g, and h. For example, the g component can vary from something simple basic transformations adding noises or rotating images to a sophisticated method to transfer the style of the images; on the other hand, the h component can vary from an approach with high reliability and low efficiency such as actually outsourcing the annotation process to human labors to the other polarity of simply assuming a large-scale pretrained model can function plausibly as a human.
In the next part, we will introduce our concrete choices of g and h leading to the later empirical results, which build upon the recent advances of vision research.
3.2 ENGINEERING SPECIFICATION
We use VQGAN Esser al. 1021 the image generation system g(x, b), and the g(x, b) is boosted by the evaluated model ((x) serving as the a(x, z) to guide the generation process, where I(0(x).) y) is the model classification loss current perturbed images.
The generation is an iterative process guided by the scoring function: at each iteration, the system adds more style-wise transformations the result of the previous iteration. Therefore, the total number of iterations allowed is denoted as the budget B (see Section 4.3 for details of finding the best perturbation). In practice, the value of budget B is set based the resource concerns.
4

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
To guarantee the image-label structure of images, consider using foundation models, e.g., the CLIP (Radford et al. 2021) model, to serve as h, and design the text fragment input of CLIP to be "an image /class/" However, given the CLIP model's ' asss-than-pe than-perfect zero-shot accuracy on most of the base datasets (Radford et al. 2021). there exists a potential risk of introducing a label noise to the generated test set. Therefore, in order to reduce the dependency of robustness evaluation on the robustness of the specific foundation model, we employ a majority voting mechanism across an ensemble of multiple foundation models to validate the correctness of labels assigned to the generated images. In our experiments, we assemble a zoo of foundation models, including the CLIP model, Con-NeXt---CvStt (Singh et al. 2023) from the RobustBench Leaderboard (Croce al. 2020), and CoCa (Yu al. 2022) from the robust foundation models leaderboard The ensemble of these foundation models is drawn from diverse sources and exhibits variability, thus enhancing the credibility of label validation for the generated images through a collective majority vote. Afterwards, we directly optimize VQGAN encoder space which is guided by our scoring function. We show the
algorithm in Algorithm
3.3 MEASURING ROBUSTNESS
Foundation Model-oriented Robustness (FMR). By design, the image-label structures of perturbed images will maintained by the foundation model. Thus, a smaller accuracy drop on the perturbed images indicates more similar predictions to foundation models. To precisely define FMR, we introduce perturbed accuracy (PA), the accuracy on the perturbed images that our generative model successfully produces. As the standard accuracy clean test set (SA) may influence PA to some extent, to disentangle PA from SA, we normalize PA with SA as FMR:
PA FMR = SA
X 100%
In settings where the foundation model is human labors, FMR measures the robustness difference between the evaluated model and human perception. In experiment setting, FMR measures the robustness difference between models trained on fixed datasets (the tested model) and the models trained on unfiltered, highly varied, and highly noisy data (the zoo of foundation models).
At last, we devote a short paragraph to kindly remind some readers that, despite the alluring idea of designing systems that forgo the usages of underlying image-label structure or foundation model, has been proved or argued multiple times that it is impossible to create that knowledge with nothing
but data, in either context of machine learning (Locatello et al., 2019 Mahajan al. 2019 Wang
et al.
2021 Zhao et al. 2022 2023) or causality (Barcinboim et al. 2020 Xia et al.
2021). Pearl
2009
Sec. 1.4).
4 EXPERIMENTS EVALUATION AND UNDERSTANDING OF MODELS
4.1 EXPERIMENT SETUP
We consider four different scenarios, ranging from the basic benchmark MNIST (LeCun et al. 1998), through CIFARIO (Krizhevsky et al. 2009). 9-class ImageNet (Santurkar et al. 2019). to full flegged 1000-class ImageNet (Deng et al. 2009). For ImageNet, we resize all images to 224 x 224 px. We also center and re-scale the color values with PAGB [0.485. 0.456, 0.406] and [0.229, 0.224, 0.225). The perturbation step size for each iteration is 0.001. The total number of iterations allowed (computation budget B) is 50.
For each of the experiment, we report set of three results:
Standard Accuracy (SA): the clean test accuracy of the evaluated model.
Perturbed Accuracy (PA): accuracy on the images that our generation process successfully produces
a perturbed image.
Foundation Model-oriented Robustness (FMR): robustness of the model compared with the founda- tion model.
5

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
4.2 ROBUSTNESS EVALUATION FOR STANDARD VISION MODELS
We consider a large range of models (Appedix M) and evaluate pre-trained variants of a LeNet architecture (LeCun al. 1998) for the MNIST experiment and ResNet architecture (He al. 2016a) for the remaining experiments. For ImageNet experiment, we also consider pretrained transformer variants of ViT (Dosovitskiy al. (020), Swin (Liu al. (022). Twins (Chu al. 202T, Visformer (Chen et al. 2021) and DeiT (Touvron et al. (021) from the fimm library (Wightman
2019). We evaluate the recent ConvNeXt (Liu et al.] (022) as well. All models are trained on the
ILSVRC2012 subset of IN comprised of 1.2 million images in the training and total of 1000
classes (Deng et al. 2009; Russakovsky et al. 2015).
We report our results in Table [ As expected, these models can barely maintain their performances when tested on data from different distri- butions, as shown by many previ- ous works (e.g., Geirroos et al. 2019 Hendrycks & Dietterich| 2019).
Table 1: The robustness test of standard models. We note 1) there exists a performance gap between standard models and the foundation model, and 2) transformer-vartii perform the vanilla ResNet in terms of FMR
out-
Data
Model
SA
PA
FMR
MNIST
LeNet
99.09
24.76
24.99
Interestingly, on ImageNet, though both transformer-variants models and vanilla CNN-architecture model, i.e. ResNet, attain similar clean image ac- curacy, cersfoormr-variing substan- tially outperform ResNet50 in terms of FMR under our dynamic evaluation protocol. We conjecture such a perfor- mance gap partly originated from the differences training setups; more specifically, it may be resulted by the fact transformer-varian variants by default use strong data augmentation strategies while ResNet50 use none of them. The augmentation strategies (e.g., Mixup (Zhang et al., (017). Cutmix YYun al. 2019) and Random Erasing (Zhong et al. 2020), etc.) already naively introduce OOD samples during training, therefore are potentially helpful for securing model robustness towards data shifts. When equiping with the similar data augmentation strategies, CNN-architecture model, i.e., ConvNext, has achieved comparable performance in terms of FMR. This hypothesis has also been verified in recent works (Bai et al. [2021 Wang et al. 2022ch. We will offer more discussions on the robustness enhancing methods in Section 4.3
CIFARTO
ResNet18
95.38
49.30
51.69
9-class IN
ResNet18
92.30
24.89
26.97
ResNet50
76.26
30.59
40.11
ViT
82.40
39.57
48.02
DeiT
78.57
41.18
52.41
ImageNet
Twins
80.53
46.47
57.71
Visformer
79.88
45.71
57.22
Swin 81.67
54.93
67.26
ConvNeXt
82.05
45.44
55.38
Surprisingly, we notice large difference within the transformer family in the proposed FMR metric. Despite Swin Transformer's suboptimal accuracy on the standard dataset, it achieves the best FMR. One possible reason for this phenomenon may due to their internal architecture that are related to the self-attention mechanism. Therefore, we conduct in-depth analysis on the effects of head numbers. The results reveal that increased head numbers enhance expressivity and robustness, albeit at the expense of clean accuracy. More details can be found in Appendix C
Besides comparing performance between different standard models, FMR brings the chance to directly compare models with the foundation model. Across all of our experiments, the FMR shows the significant gap between models and the foundation model, which is trained on the unfiltered and highly varied data, seemingly suggesting that training with a more diverse dataset would help with robustness. This overarching trend has also been identified in (Taori al. (020)). However, quantifying when and why training with more data helps is still an interesting open question.
4.3 ROBUSTNESS EVALUATION FOR ROBUST VISION MODELS
Recently, some techniques have been introduced to cope with corruptions or style shifts. To investi- gate whether those OOD robust models can maintain the performance in our dynamic evaluation, we evaluate the pretrained ResNet50 models combining with the five leading methods from the ImageNet- C leaderboard, namely, Stylized ImageNet training (SIN; (Geirhos et al. 2019)), adversarial noise training (ANT; (Rusak et al.)), a combination of ANT and SIN (ANT+SIN; (Rusak et al.)), opti- mized data augmentation using Augmix (AugMix; (Hendrycks al. 2019)), DeepAugment (Deep-

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
(a) gold fish
(b) snoek
anemone
(d) snoek
(e) coralreef
(f) hen
(g) snoek
(h) badger
(i gold fish
i nautilus
(k) lamp
(I) nautilus
(m) lamp
(n) anemone (0) coralreef (p) milk can
Figure 2:
Visualization of the images generated by our system in evaluating the common corruption robust model, with the original image shown (left image of each row). The caption for each image is either the original label or the predicted label by the corresponding model. The evaluated models are SIN, ANT, ANT+SIN, Augmix, DeepAug, DeepAug+AM and DAT from left to right.
Aug; (Hendrycks et al. 202 ). combination of Augmix and DeepAug (DeepAugtAM: (Hendrycks ct al. 2021a)) and Discrete Adversarial Training (DAT; (Mao et al. 2022b)).
The results are displayed in Table 2 Surprisingly, we find that some com- corruption robust models, i.e., SIN, ANT, ANT+SIN, fail to main- tain their power under our dynamic evaluation protocol. We take the SIN method an example. The FMR of SIN method is 40.07, which is even lower than that of vanilla ResNet50. These methods are well fitted in the benchmark ImageNet-C, verifying the weakness of relying marks to rank methods. The selected best method may not be true reflec- tion of the real world, but a model well fits certain datasets, which in turn proves the necessity of our dynamic evaluation protocol.
Table 2:
The robustness test of generated countefactual im- ages for OOD robust models. SA* represents the model's top-1 accuracy on ImageNet-C: dataset. We note that apply- ing DAT yields the best FMR under dynamic evaluation protocol.
Model SA
SA
FMR
ResNet50
76.26
39.20
30.59
40.11
ANT
76.26
50.41
30.61
40.14
SIN
76.24
45.19
30.55
40.07
fixed bench-
ANT+SIN
76.26
$2.60
31.09
40.77
DeepAug
76.26
52.60
33.24
43,59
Augmix
76.73
48.31
38,89
50.68
DeepAug+AM
76.68
58.10
42.60
55.56
DAT
76.57
68.00
52.57
68.66
DeepAug, Augmix, DeepAug+AM perform better than SIN and ANT methods in terms of FMR they dynamically perturb the datasets, alleviating the hazards of "model selection with test set" to some extent. DAT outperform others in terms of FMR, which validates the effectiveness of perturbation in the meaningful symbolic space rather than the continuous pixel space. However, their performance is limited by the variations of the perturbations allowed, resulting in marginal improvements compared with ResNet50.
Besides, we visualize the perturbed images generated according to the evaluated style-shift robust models Figure More results and the discussion on the realism of the generated images are shown
in Appendix O and P We have the following observations:
Preservation of Local Textual Details. Recent studies highlight that CNNs often prioritize object textures shapes for learning (Gatys et al. 2015 Ballester & Araujo 2016 Gatys et al. 2017]] Geirhos et al. 2019 Wang et al., (020b). Our perturbed images retain mislcading textures, making evaluation more challenging textures become nuisance rather than predictive. For instance, in Figure 21 we generated images with skin textures resembling chicken skin, which misleads the ResNet with DeepAug method.
Generalization Shape Perturbations. Our attack dynamically adjusts intensity using the model's gradient, affecting both texture and shape while preserving image-label structures. This results in
7

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
Table 3: Study of different image generator choices on ImageNet dataset. The numbers of PA and FMR are reported. The results are consistent under different image generator configurations.
ADM
Improved DDPM
EfficiantVDDVE
StyleGAN-XL FMR
VQGAN
Model
FMR
FMR
ResNer50
32.36
42.43
31.43
41.21
30.28
39.71
31.65
41.50
32.09
42.08
ANT
31.88
41.80
32.54
42.67
31.29
41.03
31.94
41.88
31.65
41.50
SIN
32.17
42.20
32.05
42.04
31.15
40.86
31.39
41.17
31.64
41.50
ANT+SIN
32.47
42.58
33.50
43.93
32.68
42.85
33.01
43.29
32.15
42.16
DeepAng
33.32
43.69
34.39
45.10
33.83
44.36
34,46
45.19
34,30
44.98
Augmix
39.47
51.44
40.16
52.34
39.95
52.07
39.30
51.22
40.01
52.14
DeepAugtAM
43,43
56.64
43.17
56.30
41.32
53.89
41.71
54,39
43.65
56.92
successful model attacks with shape- images, as demonstrated in the SIN (Figure 2b and and ANT+SIN (Figure 2d and Figure 21) examples.
Figure 2
Recognition of Model Properties. By integrating various methods, we generate more complex perturbed images, such those combining DeepAug's chicken-like head with Augmix's skin patterns (Figure 2h), demonstrating our method's ability to adapt to model properties for challenging evaluations. This shows our protocol dynamically tailors attacks to model specifics, producing perturbed images that reveal weaknesses beyond static benchmarks, i.e., ImageNet-C.
4.4 UNDERSTANDING THE PROPERTIES OF OUR EVALUATION SYSTEM
We continue to investigate several properties of the models in the next couple sections. To save space, we will mainly present the results CIFARIO experiment here and save the details to the appendix:
Appendix D we explore the transferability of the generated images and validate the reliability of the FMR metric. The results of reasonable transferability suggest that our method of generating images is not -pocc--peciificity and can be potentially used in a broader scope: we can leverage the method generate static set of images and set benchmark to help the development of robustness methods.
In Appendix [F] we compare the vanilla model model trained by PGD (Madry al. 2017). We find that these two models process the data differently. However, their robustness weaknesses are exposed similar degree by our system.
In Appendix G we investigate enhancing evaluated robustness by training the model with images generated by our evaluation system. Due to computational constraints, we use a static image set for training, which indeed improves model robustness in our system.
We also notice that the generated images tend to shift the color of the original images, so we tested the robustness of grayscale models in Appendix H the results suggest removing the color channel will not improve robustness performances.
4.5 EXPERIMENTS REGARDING METHOD CONFIGURATION
Generator Configuration. We conduct ablation study on the generator choice agree the performance ranking in Table [ and Table 2 We consider several image generator architechi- tures, namely, variational autoencoder (VAE) (Kingma & Welling 2013; Rezende al. 2014) like Efficient-VDVAR (Hazami et al. 2022), diffusion models (Sohl-Dickstein et al. (015) like Improved DDPM (Nichol & Dhariwal 2021) and ADM (Dhariwal Nichol, 2021). and GAN like StyleGAN- XL (Sauer et al. (022). As shown in Table we find that the conclusion is consistent under different generator choices, which validates the correctness of our conclusions in Section 4.2 and Section 4.3
Sparse VQGAN. In resource-constrained scenarios, we enhance efficiency by sparsifying VQGAN, discovering that only 0.69% of dimensions significantly impact style. By masking the remaining 99.31% %. we create sparse VQGAN submodel, reducing runtime by 12.7% on 9-class ImageNet and 28.5% on ImageNet, making our protocol viable even with limited computing resources. Further details are in the Appendix
Step Size.
The optimal step size varies with the computation budget (B). Under limited budgets, a larger step size is necessary but may highlight model limitations, affecting evaluation outcomes. With ample B, a smaller step size can alleviate these issues, proving sufficient for practical applications, as detailed in Appendix(]
8

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
5 DISCUSSION
5.1 DISCUSSION ON THE BIAS ISSUES
Selection Bias. In previous sections, we have mentioned that ranking models based on the fixed datasets will potentially lead to the selection bias of methods (Duda et al.) 1973] Friedman et al. 200T]. While our dynamic evaluation protocol help mitigates this issue, inviiablle to introduce other biases when we select specific generators and foundation models. Here, we provide more analyses and discussions.
Bias towards Generators. As our evaluation protocol requires an image generator, the quality or diversity of the generated images may bounded by the choice of generator. However, Table 3 shows the consistent conclusions made in the paper, which verifies that the proposed method is robust to the choice of generator.
Bias towards the Foundation Models. In Appendix we take the CLIP model as example and explore the category unbalance issue. We observe its performance is affected by imbalanced online sample distributions, leading to perturbed images of varied difficulty. Fortunately, model configurations significantly mitigate this issue (see Appendix D). proving effective for real-world use. Additionally, using ensemble of foundation models enhances this mitigation.
Bias of the Metric. As the generated samples are biased to the zoo of foundation models' zero-shot performance, "PA" and "FMR" scores will also be biased. In Appendix [ we conduct theoretical analysis to guarantee the correctness of the proposed method. Our theoretical analysis confirms that while both traditional datasets and foundation model zoos can approximate true distributions, the latter achieves this with less variance. Hence, we advocate that bias towards foundation model zoos preferable conventional datasets.
Potential Negative Impacts. Although the bias incurred by the zoo of foundation models is less detrimental than the biases arisen from fixed benchmark datasets, a more detailed discussion on the potential negative impacts is necessary. Therefore, we discuss the potential negative impacts as well as the societal bias of relying on large models in Appendix K
5.2 DISCUSSION ON THE EFFECTS OF FOUNDATION MODEL'S S ZERO-SHOT PERFORMANCE
Domain Gap Concerns. Despite the zero-shot strengths of our foundation model ZoO, it may underperform in niche areas, e.g., medical imaging, where general knowledge falls short. However, framework's adaptable design allows for the easy inclusion of domain-specific pre-trained models, providing versatile solution for a wide range of applications.
Zero-shot Adversarial Robustness Concerns.
Foundation models like CLIP are vulnerable to adversarial attacks (Mao et al. 2022a potentially un- dermining evaluation effectiveness if attackers access and manipulate CLIP's weights. In Appendix L our study into CLIP's zero-shot adversarial robustness reveals it outperforms XCiT-LI2 (Debenedetti al. 2022 resilience against FGSM attacks, despite susceptibility classification changes. How- ever, gradient masking and other simple techniques can safeguard CLIP in production, significantly reducing white-box attack risks. For black-bor attacks, CLIP demonstrates strong resilience (See Appendix [.. By integrating robust model ensemble and employing a majority vote for image-label validation, our approach enhances security. Therefore, CLIP, particularly when safeguarded by weight and gradient protection techniques and supported by robust model ensemble, stands strong candidate for the ideal foundation model to preserve image-label integrity currently.
CONCLUSION
In this paper, we first summarized the common practices of model evaluation strategies for robust vision machine learning. We then discussed three desiderata of the robustness evaluation protocol. Further, we offered a simple method that can fulfill these three desiderata at the same time, serving the purpose of evaluating vision models' robustness across generic i.i.d benchmarks, without requirement on the prior knowledge the underlying image-label structure depicted by the images, although relying on zoo of foundation models.

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
REFERENCES
Yutong Bai, Jieru Mei, Alan I. Yuille, and Cihang Xie. Are transformers more robust than cnns? Advances in Neural Information Processing Systems, 34:26831 -26843, 2021.
Pedro Ballester and Ricardo Matsumura Araujo. On the performance of googlenet and alexnet applied to sketches. In Thirtieth AAAI Conference on Artificial Intelligence, 2016.
Elias Bareinboim, Juan D Correa, Duligur Ibeling, and Thomas Icard. On pearl's hierarchy and the foundations of causal inference. ACM Special Volume in Honor of Judea Pearl (provisional title), 2(3):4, 2020.
Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Percira. Analysis of representations for domain adaptation. In Advances in neural information processing systems, pp. 137- 2007.
Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine learning, 79 (9112):151-111 2010.
Digbalay Bose, Rajat Hebbar, Krishna Somandepalli, Haoyang Zhang, Yin Cui, Kree Cole- McLaughlin, Huisheng Wang, and Shrikanth Narayanan. Movieclip: Visual scene recognition in movies. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp.
2023.
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness. arXiv preprint arXiv: 2019.
Zhengsu Chen, Lingxi Xie, Jianwei Niu, Xuefeng Liu, Longhui Wei, and Qi Tian. Visformer: The vision-friendly transformer. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 589-598, 2021.
Xiangxiang Chu, Zhi Tian, Yuqing Wang, Bo Zhang, Haibing Ren, Xiaolin Wei, Huaxia Xia, and Chunhua Shen. Twins: Revisiting the design of spatial attention in vision transformers. Advances in Neural Information Processing Systems, 34, 2021.
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flam- marion, Mung Chiang. Prateck Mittal, and Matthias Hein. Robustbench: standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09670. 2020.
Alexander D' Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew Hoffman, et al. Underspecification presents challenges for credibility in modern machine learning. arXiv preprint a:Xiv:2011.03395. 2020.
Edoardo Debenedetti, Vikash Sehwag, and Prateek Mittal. A light recipe train robust vision transformers. arXiv preprint arXiv:2209.07399, 2022.
Aaron Defazio, Francis Bach, and Simon Lacoste-Julien. Saga: A fast incremental gradient method with support for non-strongly convex composite objectives. Advances in neural information processing systems, 27, 2014.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Ieee, 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 34:8780-8794, 2021.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn. Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani. Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
10

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
Richard O Duda, Peter E Hart, and David G Stork. Pattern classification and scene analysis, volume 3. Wiley New York, 1973.
Chris Edwards. Malevolent machine learning. Commun. ACM, 62(12):13-15, nov 2019. ISSN 0001-0782.
Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Brandon Tran, and Alek- sander Madry. Adversarial robustness a prior for learned representations. arXiv preprint arXiv:/906. 00945, 2019.
Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution im- age synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12873-12883, 2021.
Jerome Friedman, Trevor Hastie, Robert Tibshirani, et al. The elements of statistical learning, volume 1. Springer series in statistics New York, 2001.
Leon Gatys, Alexander S Ecker, and Matthias Bethge. Texture synthesis using convolutional neural networks. Advances in neural information processing systems, 28, 2015.
Leon A Gatys, Alexander S Ecker, and Matthias Bethge. Texture and art with deep neural networks. Current opinion in neurobiology, 16:1778-86, 2017.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2019.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples (2014). International Conference on Learning Representations, 2015.
Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. arXiv preprint arXiv:2007.01434, 2020.
Louay Hazami, Rayhane Mama, and Ragavan Thurairatnam. Efficient-vdvac: Less more. arXiv preprint arXiv:2203. 2022.
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016a.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European conference on computer vision, pp. 630-645. Springer. 2016b.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint 1903: 2019.
Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint 19X2::912200788 2019.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8340-8349, 2021a.
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adver- sarial examples. In Proceedings of the IEEE/CVF Conference Computer Vision and Patlern Recognition, pp. 15262-15271, 202
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv: 1704.04861, 2017.

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700-4708, 2017.
Diederik P Kingma and Max Welling, arXiv:1312.6114 2013.
Auto-encoding variational bayes. arXiv preprint
Pang Wei Koh, Shiori Sagawa, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Trena Gao, Tony Lee, al. Wilds: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning, pp. 5637-5664.
PMLR, 2021.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEFE, 86(11): 86(11):2278-2322 1998.
Chaozhuo Li, Senzhang Wang, Yukun Wang, Philip Yu, Yanbo Liang, Yun Liu, and Zhoujun Li. Adversarial learning for weakly-supervised social network alignment. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 996-1003, 2019.
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference on computer vision, pp. 5542-5550, 2017.
Xihui Liu, Dong Huk Park, Samanch Azadi, Gong Zhang, Arman Chopikyan, Yuxiao Hu, Humphrey Shi, Anna Rohrbach, and Trevor Darrell. More control for free! image synthesis with semantic diffusion guidance. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 289-299, 2023.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. (0012-10022, 2021.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. convnet for the 2020s. arXiv preprint arXiv:2201.03545. 2022.
Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentan- gled representations. In international conference machine learning, pp. 41 4114-4124, PMLR, 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint 1706iv:1006.6660 2017.
Divyat Mahajan, Chenhao Tan, and Amit Sharma. Preserving causal constraints counterfactual explanations for machine learning classifiers. arXiv preprint arXiv:1912.03277 2019.
Chengzhi Mao, Scott Geng, Junfeng Yang, Xin Wang, and Carl Vondrick. Understanding zero-shot adversarial robustness for large-scale models. arXiv preprint arXiv:2212.07011 20222.
Xiaofeng Mao, Yuefeng Chen, Ranjie Duan, Yao Zhu, Gege Qi, Xiaodan Li, Rong Zhang, Hui Xue, et al. Enhance the visual representation via discrete adversarial training. Advances in Neural Information Processing Systems, 35:7520-7533, 2022b.
Diego Marcos, Michele Volpi, and Devis Tuia. Learning rotation invariant convolutional filters for texture classification. In 20/6 23rd International Conference on Pattern Recognition (ICPR), pp. 2012-2017. IEEE, 2016.
Sachit Menon, Ishaan Preetam Chandratreya, and Carl Vondrick. Task bias in vision- -ssiun-language models. arXiv preprint 04244222/2.04412, 2022.
12

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via invariant
feature representation. In International Conference on Machine Learning, pp. 10-18, 2013.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pp. 8162-8171, PMLR, 2021.
A Emin Orhan. Robustness properties of facebook's resnext wsl models. arXiv preprint
17Xiv:1907.07640. 2019.
Judea Pearl. Causality. Cambridge university press, 2009.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103. 000000000 2021.
Benjamin Recht, Rehecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize imagenet? In International Conference on Machine Learning, pp. 5389-5400. PMLR, 2019.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In International conference machine learning, pp. 1278-1286. PMLR, 2014.
Andras Rozsa, Manuel Günther, and Terrance E Boult. Are accuracy and robustness correlated. In 2016 15th IEEE international conference on machine learning and applications (ICMLA), pp. 227-232. IEEE, 2016.
Evgenia Rusak, Lukas Schott, Roland Zimmermann, Julian Bitterwolfb, Oliver Bringmann, Matthias Bethge, and Wieland Brendel. Increasing the robustness of dnns against im-age corruptions by playing the game of noise.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):211-25 2015.
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
Shibani Santurkar, Dimitris Tsipras, Brandon Tran, Andrew Ilyas, Logan Engstrom, and Aleksander Madry. Image synthesis with single (robust) classifier. arXiv preprint arXiv: /oXXiv:990000 096.09453, 2019.
Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets. In ACM SIGGRAPH 2022 Conference Proceedings, pp. 1-10, 2022.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1409. 1556, 2014.
Naman D Singh, Francesco Croce, and Matthias Hein. Revisiting adversarial training for imagenet: Architectures, training and generalization across threat models. arXiv preprint arXiv:2303.01870 2023.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. International Conference on Machine Learning, pp. 2256-2265. PMLR, 2015.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties neural networks. arXiv preprint arXiv: 2013.
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanct, Scott Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1 1-9, 2015.
Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning, pp. 6105-61 14, PMLR, 2019.
13

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems, 33:18583-1888 18599, 2020.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International Conference on Machine Learning, pp. 10347-10357. PMLR, 2021.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. arXiv preprint arXiv: / 2018.
Haohan Wang, songwei Ge, Eric P Xing, and Zachary C Lipton, Learning robust global representa- tions by penalizing local predictive power. arXiv preprint arXiv: 2019.
Haohan Wang, Zeyi Huang, Xindi Wu, and Eric P Xing. Squared f2 norm as consistency loss for leveraging augmented data learn robust and invariant representations. arXiv preprini arXiv:201 1. 13002,, 2020a.
Haohan Wang, Xindi Wu, Zeyi Huang, and Eric P Xing. High-frequency component helps explain the generalization of convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8684-8694 2020b.
Haohan Wang. Zeyi Huang, Hanlin Zhang, and Eric Xing. Toward learning human-aligned cross- domain robust models by countering misaligned features. arXiv preprint arXiv:2111.03740, 2021.
Junyang Wang, Yi Zhang, and Jitao Sang. Fairelip: Social bias elimination based on attribute prototype learning and representation neutralization. arXiv preprint arXiv:2210. 14562, 2022a.
Yiqi Wang, Chaozhuo Li, Zheng Liu, Mingzheng Li, Jiliang Tang, Xing Xie, Lei Chen, and Philip S Yu. An adaptive graph pre-training framework for localized collaborative filtering, ACM Transac- tions on Information Systems, 41(2):1-27 2022b.
Zeyu Wang, Yutong Bai, Yuyin Zhou, and Cihang Xie. Can cnns be more robust than transformers? arXiv preprint arXiv:2206.034: 2022c
Ross Wightman. Pytorch image models.
pytorch- Sisihh-mmmge-ee
2019.
Shirley Wu, Mert Yuksekgonul, Linjun Zhang, and James Zou. Discover and cure: Concept-aware mitigation of spurious correlation. arXiv preprini urXiv:2305.00651 2023.
Kevin Xia, Kai-Zhan Lee, Yoshua Bengio, and Elias Bareinboim. The causal-neural connection: Expressiveness, learnability, and inference. Advances in Neural Information Processing Systems, 34, 2021.
Yao Xiao, Ziyi Tang, Pengxu Wei, Cong Liu, and Liang Lin. Masked images are counterfactual samples for robust fine-tuning. arXiv preprint arXiv:2303. 030522, 2023.
Qizhe Xie, Minh-Thang
Thang Luong, Eduard Hovy, and Quoe V Le. Self-training with noisy student improves imagenet classification. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 10687-10698, 2020.
Hao Yan, Chaozhuo Li, Ruosong Long, Chao Yan, Jianan Zhao, Wenwen Zhuang, Jun Yin, Peiyan Zhang, Weihao Han, Hao Sun, et al. comprehensive study on text-attributed graphs: Benchmark- ing and rethinking. Advances in Neural Information Processing Systems, 36, 2024.
Nanyang Ye, Kaican Li, Lanqing Hong, Haoyue Bai, Yiting Chen, Fengwei Zhou, and Zhenguo Li. Ood-bench: Benchmarking and understanding -ut-o--dist generalization datasets and algorithms. arXiv preprint arXiv:2106.0372. 03721, 2021.
Fisher Yu, Dequan Wang, Evan Shelhamer, and Trevor Darrell. Deep layer aggregation. In Pro- ceedings of the IEEE conference on computer vision and pattern recognition, pp. 2412- 2018.
14

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. 2022.
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 6023-6032, 2019.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv: 2017.
Qiming Zhang, Yufei Xu, Jing Zhang, and Dacheng Tao. Vitaev2: Vision transformer advanced by exploring inductive bias for image recognition and beyond. International Journal of Computer Vision, pp. 1-22, 2023.
Richard Zhang. Making convolutional networks iiftt-invariant again. In International conference on machine learning. pp. 7324-7334. PMLR, 2019.
Tianyuan Zhang and Zhanxing Zhu. Interpreting adversarially trained convolutional neural networks. In International Conference on Machine Learning, pp. 7502-7511, PMLR, 2019.
Jianan Zhao, Meng Qu, Chaozhuo Li, Hao Yan, Qian Liu, Rui Li, Xing Xie, and Jian Tang. Learning on large-scale text-attributed graphs via variational inference. arXiv preprint arXiv:2210. 14.009 2022.
Yi Zhao, Chaozhuo Li, Jiquan Peng, Xiaohan Fang, Feiran Huang, Senzhang Wang, Xing Xie, and Jibing Gong. Beyond the overlapping users: Cross-domain recommendation via adaptive anchor link learning. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1488-1492 2023.
Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmenta- tion. Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 13001-13008, 2020.
Daquan Zhou, Zhiding Yu, Enze Xie, Chaowei Xiao, Animashree Anandkumar, Jiashi Feng, and Jose M Alvarez. Understanding the robustness in vision transformers. In International Conference on Machine Learning, pp. 27378-27394, PMLR, 2022a.
Kankan Zhou, Yibin LAI, and Jing Jiang. Vistereoset: study of stereotypical bias in pre-trained vision-language models. Association for Computational Linguistics, 2022b.
15

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
A DISTRIBUTION ESTIMATION
A. PROBLEM FORMULATION
Given unknown ground truth distribution:
P Unknown(je, Σ)
(I)
where j E R and Σ E RPxP.
the samples our study are sampled from this distribution.
We use X to denote the 1.th dataset, with ny samples, and we use X,,; to denote the hth sample in it.
We aim consider the estimation of y from two different models. The conventional smaller model which operates only one dataset, and WLOG, assume the smaller model works on Xo: and the bigger, Zoo of CLIP-style models, which operates on a collection datasets, we say it works on m datasets, i.e., (X X2, X..... X. we will compare E( 14] and El 1u). VAR(Te) and
VAR(FCLIP), E Σ 2) and E| > and VAR (o) and VAR(ECLPP..
Assumption Due to dataset collection bias, assume that, while all the data sampled with the fixed distribution above, the bias of dataset collection will introduce bias in the estimation of the true parameter jr, therefore
p = μ + E
where
P4 1 TE
Xi.j
(3)
11 3
and
6, N(0
Assumption II Due to dataset collection bias, assume that, while all the data are sampled with the fixed distribution above, the bias of dataset collection will introduce bias in the estimation of the true parameter M, therefore
Σ +
where
Σ =
1 74
Xi.j Txx,5 [.)]
(6)
and
6, Exp(1),
(7)
Proposition A.1.
Under Assumptions I and II, we have estimators
ElJCLIP [4] Bioo - pl.
E| - - E[E0 Σ
VAR(FeLi) < (A))
VAR(ECLIP) S VAR(Eg)
where < holds element-wise.
Proof. Estimation of p. Under Assumptions and II, we have
No μ 60
(8)
We can obtain Elpo- 44] and VAR( by marginalizing out the randomness introduced by e:
Eljoo 14] EEE++++ 21] E(col 0.
16

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
VAR(A))
Bl E [
Ei(p co) E² 1(1 co]
Blu²² 2110 + +g] - -u R(eo))
E[cğ] - [ooool
VAR((o)
I
(10)
For El p1] and (AR(P))))), we have:
Σ
C,74
Σ
Elcinn
EliCLIP n4] 18|
(11)
14
n
and
VAR(FCLIP) ElJCCLPP 9 F22 [UCLIP)
El(H + ECLIP) E²² 2 ECLIP)
Ely2 + 2/ECCUP + caup (u + B((cLP))2
Elecupl - E² [Ecup]
(12)
Since B(CcLP] ElpCii pa] = 0, we have:
VAR(FCLLP) = B(CCUP = TE[( Σx Σ
E: )a)
(13)
When we expand the square of sum, we will get the many squared terms (which are left finally) and many more that involves at least one E[e;]z, where z can be any arbitrary stuff, and then since Bles] = 0, won't 't matter. Therefore, we have:
VAR(FCLIP) = IE[( Σ G., )21
Σ n E(E
(14)
(En n2)
Since n, for i 1, ..., m. have Σ n < (E th n,)
Therefore,
Σx BEB(EE (2 n2))
EE m n2) 3
VAR(HCLIP)
[E =1
(15)
n2)
Estimation of Σ. We can obtain E[Eo- Σ] and VAR(E)) by marginalizing out the randomness introduced by
E EJ - E((EO 1) Ele 1E -
(16)
VAR(En) = EEE5 E2 [
= E[((EE) E
E[(E2> L 1
-R[<'"][[>>] E E
-2E[E>] - B
-2-- 2
E²
(17)
17

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
For E[ECLIP Σ] and VAR EcLIP) we have:
E E 2 Dl لا 2 = E[((ELIP 1)2] Elecup [|
(18)
VAR(ECLPP) E[ECLIP 2
] 1 (EC.PP)
= E(((ccuP)))) E [ECLIP
EleEi - [ [
= Eicai E| - E²
(19)
Consider that
1 D. (Xi,)
PCLIP) 2
D CLpp" ECLIP
ECLIP
(20)
27
We will have:
CLIP
CLIP Σ
(21)
CUP 5)2
Thus, we have E
Next, we will compute Elca.p as follows:
E(C'uP = E
CLIP Σ
E CLLP
(22)
Σ
By definition, we have:
D Th (xc.. - p3:)
ECLIP
(23)
Therefore,
(E Dnn (Xi. - 12)
CLIP
(24)
As the value of wij fus can be either positive or negative, we have:
En (Xi_
p2) 1
CLU I
(25)
D
E
Since both (xn. ) and n, are positive values, we further have:
Σ" (Xij
2))2)
E
E, E
CLI S ym
(26)
n4
Dmn n
Thus, we can obtain
(Em )
6CLIP
1,,
m2
(27)
Therefore, we have:
ED (I 2
E E 2
E 4 < E
(28)
(Smm
(5mm);)²
By Assumption A.1
6 ~ Exp(1) 1). we have E(e]] 1 and VAR(E) 1.
Since e' are independent with each other, we have:
( VAR(E,T ()) R22 [S (
VAR(E) (2 E()
m + m²
(29)
18

---

Foundation -odel-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
Substituting Eq. 29 into Eq. 28 have:
E| Ei S m+ + m² E²
(30)
(2,"n,))
Since rı 2 for = 1,,, m, we have Minn; 2 m and n;) 2 mi.
Since 2 have: m²
Therefore,
m + m²
2m
E|CCLP]] S
52
Σ² = 25
(31)
m
m²
Substituting Eq.
31 into Eq.
we have:
25
E|
=
(32)
Substituting Eq.
32
into Eq. 19
we have:
VAR EcUi)
Ei
2E[E3]
-E E
(33)
We summarize the above results as follows: For conventional fixed dataset estimators, we have:
Bljoo 14] 0
VAR (AR())) = I
E[C 2 0
VAR(Eø) = E
For CLIP-style estimators, we have:
E ELPCLIP 11
VAR <
EE 2
VAR S E,
where < holds element-wise.
The results show that, both conventional estimator and zoo of CLIP-style estimator can recover the true μu, E of the unknown distribution, but zoo of CLIP-style estimator will have a lower variance, which is more stable to accomplish the task. This conclusion holds for any distributions.
With these theoretical evidence, we kindly argue that biased towards the zoo of CLIP-style models better than biased on conventional fixed datasets. In addition, recent advances incorporating the foundation model into various tasks (Liu et al. 20233] ZZhan et al. [223]] Bose et al.) (023) also reveals that the community has utilized the foundation model on a large scale and pays little attention on these biases.
B NOTES ON THE EXPERIMENTAL SETUP
B. NOTES ON MODELS
Note that we only re-evaluate existing model checkpoints, and hence do not perform any hyperpa- rameter tuning for evaluated models. Our model evaluations are done on 8 NVIDIA V100 GPUs. With our Sparsified VQGAN model, method is also feasible to work with small amount of GPU resources. As shown in Appendix ! the proposed protocol can work on a single NVIDIA V100 GPU efficiently.
19

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
B.2 HYPERPARAMETER TUNING
Our method is generally parameter-free except for the computation budget and perturbation step size. In our experiments, the computation budget is the maximum iteration number of Sparse VQGAN. We consider the predefined value to be 50, as it guarantees the degree of perturbation with acceptable time costs. We provide the experiment for step size configuration in Section 4.5
C IN-DEPTH ANALYSIS ON THE TRANSFORMER FAMILY
In Table [ we notice large difference between the methods in the proposed FMR metric, even within the transformer family. After checking the distribution of misclassified perturbed images of different models, we find that these images are rather random and do not reveal any obvious "weak classes". One possible reason for this phenomenon may due to their internal architecture that related to the self-attention (Self-Att) mechanism. Many current Vision Transformer architectures adopt multi-head self-attention (MHSA) design where each head tends to focus different object components. In some sense, MHSA A can be interpreted as a mixture information bottlenecks (IB) where the stacked Stlf-Att modules in Vision Transformers repeat of the IB process which promotes grouping and noise filtering. More details of the connection between the Self-Att and IB can be found in ((Zhou al. 2022a), Sec.2.3). As revealed in (Zhou et al. 2022a), having more heads leads to improved expressivity and robustness. But the reduced channel number per head also causes decreased clean accuracy. The best trade-off achieved with 32 channels per head.
be broadly regarded
iterative
Table 4 illustrates the head number configurations of various models employed in our experiment.
Table 4: Details of head numbers configurations.)
Model
Head Number
VIT
12
DeiT
12
Twins
(3,6,12,24)
Visformer
6
Swin
14,8,16,32)
Swin Transformer exhibits the highest number of heads among them. Despite its suboptimal accuracy on the standard dataset, it achieves the best FMR. This corroborates the finding in (Zhou al.) (022a) that increased head numbers enhance expressivity and robustness, albeit at the expense of clean accuracy.
To further verify the impact head numbers, we trained Swin Transformer with varying head configurations and obtained the following results in Table
Table The performance of Swin Transformer with different head number configurations. We find that increased heads enhance expressivity and robustness.)
#Params
Head Number
SA
FMR
88M
12,4,8,66)
80.82
64.85
88M
(3,6,12,24)
81.98
67.48
88M
(4,8,16,32)
81.67
69.73
88M
(5,10,20,40)
81.05
69,97
With comparable numbers of parameters, observe that their accuracies the standard dataset are relatively similar. With the augmentation of head numbers, the FMR value also escalates, which validates our hypothesis that increased heads enhance expressivity and robustness.
20

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
D TRANSFERABILITY OF GENERATED IMAGES
We first study whether our generated images are model-specific, since the generation of the images involves the gradient of the original model. We train several architectures, namely EfficientNet (Tan
& Le 2019). MobileNet (Howard et al. 2017). SimpleDLA (Yu et al. 2018), VGG19 (Simonyan Zisserman, 2014), PreActResNet (He al., 2016b), GoogLeNet (Szegedy al.] 2015). and DenseNet121 (Huang et al. (017) and test these models with the images that generated when testing ResNet. We also train another ResNet following the same procedure to check the transferability across different runs in one architecture.
Transferability of the generated images. Tsowss reasonable transferability of the generated images the FMR are all lower than the SA, although we can also the FMR when tested in
Table 6: Performances of transfcrability.
observe improvement the new models. These results suggest that our method of generating images be potentially used in broader scope: we can also leverage the method to generate a static set of images and set benchmark dataset to help the development of robustness methods.
Model
SA
PA
FMR
ResNet
95.38
51.67
54.17
ResNet
94.67
56.09
59.25
DenseNet
94.26
60.48
64.17
SimpleDLA
92.25
61.03
66.16
GoogLeNet
92.06
61.10
66.38
PreActResNet
90.91
61.14
67.25
Reliability of the FMR metric. Moreover, these results contribute to the validation of the reliability of the FMR metric: given that each model's FMR gets computed using a different test set, it is not clear why FMR would be a reliable metric that can be used to compare two models. In this experiment, however, the models are tested using the same fixed test set that was initially generated during the evaluation on ResNet. Remarkably, the strong correlation observed between FMR and PA at the fixed test sets lends credence to the reliability of the FMR metric.
EfficientNet
91.37
62.57
68.48
MobileNet
91.63
62.97
68.72
VGG
93.54
66.01
70.57
New findings. In addition, our results might potentially help mitigate a debate on whether more accurate architectures are naturally more robust: accurate architectures indeed lead to better empirical performances on certain (usually fixed) robust- ness benchmarks (Rozsa et al. 2016 Hendryeks & Dietterich| 2019); while the other hand, some counterpoints suggest the higher robustness numerical performances are only because these models capture more non-robust features that also happen exist in the fixed benchmarks (Tippras et al. 2018 Wang et al. 2020b Taori et al. 22200). Table show some examples to support the latter argument: in particular, we notice that VGG, while ranked in the middle of the accuracy ladder, interestingly stands out when tested with generated images. These results continue to support our argument that dynamic robustness test scenario can help reveal more properties of the model.
hand, have results showing that more
E INITIATING WITH ADVERSARIAL ATTACKED IMAGES
Since our method using the gradient of the evaluated model reminds readers about the gradient-based attack methods in adversarial robustness literature, we test whether initi- ating the perturbation process with adversarial example will further degrade the accuracy.
Table 7: Results on whether initiating with adversarial images (e = 0.003).
Data SA
FMR
regular
95.38
57.80
w. FGSM
95.30
65.79
We first generate the images with FGSM attack (Goodfel) low et al. 2015). Table 7 shows that initiating with the FGSM adversarial examples barely affect the FMR, which
is probably because the major style-wise perturbation will erase the imperceptible perturbations the adversarial examples introduce.
F ADVERSARIALLY ROBUST MODELS
With evidence suggesting the adversarially robust models are considered more human perceptually
aligned (Engstrom et al. 2019 Zhang & Zhu 2019 Wang et al.) (020b), we compare the vanilla model to a model trained by PGD (Madry et al. 2017 (Coo norm smaller than 0.03).
21

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
As shown in Table 8 adversarially trained model and vanillaly trained model indeed process the data differently: the transferability of the generated images between these two regimes can barely hold. In particular, the PGD model almost maintain its performances when tested with the images generated by the vanilla model.
Table 8: Performances comparison with vanilla model and PGD trained model.
Data
Model
SA
FMR
Van.
95.38
57.79
Van.
PGD
85.70
95.96
Van.
95.38
62.41
However, despite the differences, the PGD model's robust- ness weak spots are exposed to similar degree with the vanilla model by our test system: the FMR of the vanilla model and the PGD model are only 57.79 and 66.18, respectively. We believe this result can further help advocate our belief that the robustness test needs to be a dynamic process generating images conditioning on the model to test, and thus further help validate the importance of our contribution.
PGD
PGD
85.70
66.18
G AUGMENTATION THROUGH STATIC ADVERSARIAL. TRAINING
Intuitively, inspired by the success of adversarial training (aadry et al. 2017) in defending models against adversarial attacks, natural method to improve the empirical performances under new test protocol is to augment the training data with perturbed training images generated by the same process. We aim to validate the effectiveness of this method here.
However, the computational load of generation process is not ideal serve the standard adversarial training strategy, and we can only have one copy of the perturbed training samples. Fortunately, we notice that some recent advances in training with data augmentation can help learn robust representations with limited scope of augmented samples (Wang et al. 2020a which we use here.
Table 9: Test performances of the model trained vanilla manner (denoted as Van.) with augmentation data offered through our approach (marked by the second column). We report sets of performances, split by whether the per- turbed images are generated according to the vanilla model or the augmented model (marked by the first column).
We report our results in Table 9 The first thing we ob- serve is that the model trained with the augmentation data offered through our approach could preserve relatively higher performance (FMR 89.13) when testing with the perturbed images generated according to the vanilla model. Since have shown the perturbed samples have rea- sonable transfcrability in the main manuscript, this result indicates the robustness we brought when training with the perturbed images generated by our approach.
Data
Model
SA
FMR
Van.
95.38
57.82
Van.
Aug
87.41
89.13
Van.
95.38
58.03
Aug.
Aug
87.41
78.61
In addition, when tested with the perturbed images gener- ated according to the augmented model, the augmented model displays a marked resilience (FMR
78.61) in the face of these perturbations compared with the model trained in vanilla manner (FMR 58.03). Nevertheless, it is noteworthy that the augmented model's performance does exhibit a discemiile decline under these circumstances, which once more underscores the efficacy of our approach.
H GRAYSCALE MODELS
Our previous visualization suggests that a shorteut the perturbed generation system can take is to significantly shift the color of the images, for which grey-scale model should easily maintain the performance. Thus, we train a grayscale model by changing the ResNet input channel to be I and transforming the input images to be grayscale upon feeding into the model. We report the results in Table 10
Interestingly, notice that the grayscale model cannot defend against the shift introduced by system by ignoring the color information. On the contrary, it seems to encourage our system to generate more perturbed images that lower the performances.
In addition, we visualize some perturbed images generated according to each model and show them in Figure 3 We some evidence that the graycale model forces the generation system to
22

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
Table 10: Test performances of the model trained in vanilla manner (denoted as Van.) or with grayscale model. We report two sets of performances, split by whether the perturbed images are generated according the vanilla model or the grayscale one (marked by the first column).
Data
Model
SA
FMR
Van.
95.38
57.79
Van.
Gray
93.52
66.06
Van.
95.38
67.48
Gray
Gray
93.52
44.76
focus more on the shape of the object and less of the color of the images. We find particularly interesting that system sometimes generates different images differently for different models while the resulting images deceive the respective model to make the same prediction.
(a) dog
(b) horse
(c) horse
(d) plane
(e) ship
(l) bird
(g) ship
(h) car
(i) plane
(j) bird
(k) ship
(I) ship
Figure 3: Visualization of the perturbed images generated by our system in evaluating the vanilla model (middle image of each group) and the grayscale model (third image of each group), with the original image shown. The caption for each image is either the original label or the predicted label by the corresponding model.
I SPARSE SUBMODEL OF VQGAN FOR EFFICIENT PERTURBATION
While our method will function properly as described above, we notice that the generation process still has a potential limitation: the bound-free perturbation of VQGAN will sometimes perturb the semantics the images, generating results that will be rejected by the foundation model later and thus leading waste of computational efforts.
To counter this challenge, we use sparse variable selection method to analyze the embedding dimensions of VQGAN to identify a subset of dimensions that is mainly responsible for the non- semantic variations.
In particular, with dataset (X, Y) of n samples, first use VQGAN generate style-transferred dataset (X', Y). During the generation process, we preserve the latent representations of input sam- ples after the VQGAN encoder in the original dataset. We also preserve the final latent representations before the VQGAN decoder that are quantized after the iterations in the style-transferred dataset. Then, we create new dataset (E, L) of 2n samples, for each sample (c. 1) E (E. L), the latent representation for the sample (from either the original dataset the style-transferred one), and / is labelled as the sample is from the original dataset and if the style-transferred dataset.
Then, we train é regularized logistic regression model to classify the samples of (E, L). With w denoting the weights the model, we solve the following problem
arg min
Σ
I(ew, ()
(e.))
23

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
Table 11: Classification results between vanilla and perturbed images with LASSO.
Data
Sparsity
Test score
MNIST 97.99
78.50
CIFAR-10
98.45
78.00
9-class ImageNet
99.31
72.00
ImageNet
99.32
69.00
and the sparse pattern (zeros or not) of w will inform us about which dimensions are for the style.
We generate the flattened latent representations of input images after the VQGAN Encoder with negative labels. Following Algorithm| [ we generate the flattened final latent representations before the VQGAN decoder with positive labels. Altogether, we form binary classification dataset where the number of positive and negative samples is balanced. The positive samples are the latent representations of perturbed images while the negative samples are the latent representations of input images. We set the split ratio of train and test set to be 0.8 0.2. We perform the explorations on various datasets, i.e. MNIST, CIFAR- 10, 9-class ImageNet and ImageNet.
The classification model we consider is LASSO as it enables automatically feature selection with strong interpretabilltyy We set the regularization strength to be 36.36. We adopt saga (Defazio et al. 2014) as the solver to use in the optimization process. The classification results are shown in Table
We observe that the coefficient matrix of features be far sparser than expect. We take the result of 9-class ImageNet as an example. Surprisingly, we find that almost 99.31 % dimensions in average could be discarded when making judgements. We argue the preserved 0.69% dimensions are highly correlated to VQGAN perturbation. Therefore, we keep the corresponding 99.3 % dimensions unchanged and only let the rest 0.69% dimensions participate in computation. Our computation loads could be significantly reduced while still maintain the competitive performance compared with the unmasked versior
We conduct the run-time experiments on single NVIDIA V100 GPU. Following our experiment setting, we evaluate a vanilla R on 9-class ImageNet and a vanilla ResNet-50 on ImageNet. As shown in Table T2 the run-time on ImageNet can be reduced by 28.5% with our sparse VQGAN. Compared with large-scale masked dimensions (i.e., 99.31% %)., we attribute the relatively incremental run-time improvement (i.e., 12.7% on 9-class ImageNet, 28.5% on ImageNet) to the fact that we have to perform mask and unmask operations each time when calculating the model gradient, which offsets the calculation efficiency brought by the sparse VQGAN to certain extent.
Table 12: Run-time Comparision between VQGAN and Sparse VQGAN.
Time
Method
9-class ImageNet
ImageNet
VQGAN
521.5 + 1.2s
52602.4 + 2.7s
Sparse VQGAN
455.4 + 1.2s
40946.1 +: 2.7s
Improv.
12.7%
28.5%
J ANALYSIS OF SAMPLES THAT ARE MISCLASSIFIED BY THE MODEL
We notice that, the CLIP model has been influenced by the imbalance sample distributions across the Internet.
In this experiment, we choose a larger step size so that the foundation model may not be able to maintain the lmbee structure the first perturbation. We report the Validation Rate (VR) which
Athhughh LASSO is originally regression model, we prohabilize the regression values get the final classification results.
We note that the overlapping degree of the preserved dimensions for each dataset is not high, which means that need to specify these dimensions when facing new datasets.
24

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
the percentage of images validated by the foundation model that maintains the image-label structure. (In our official configurations, the step size value is small enough that the VR on each dataset is always 1. Therefore, omit this value in the main experiments.) We present the results on 9-class ImageNet experiment to show the details for each category.
Table 13: Details of test on 9-class ImageNet for vanilla ResNet- 18 (step size is 0.1, computation budget B 50)
Type
SA
VR
FMR
Dog
93.33
95.33
17.98
Cat
96.67
94.00
31.55
Frog
85.33
80.67
20.34
Turtle
84.67
78.67
29.03
Bird
91.33
96.00
28.13
Primate
96.00
48.00
62.21
Fish 94.00
76.67
45.33
Crab 96.00
87.33
19.87
Insect
93.33
78.00
33.88
Total 92.30
81.63
30.28
Table 13 shows that the VR values for most categories are still higher than 80%, some even reach 95%, which means produce sufficient number of perturbed images. However, notice that the VR value for primate images is quite lower compared with other categories, indicating around 52% perturbed primate images blocked by the orcle.
As shown in Table 13 the FMR value for each category significantly drops compared with the SA value, indicating the weakness of trained models. An interesting finding that the FMR value for Primate images are quite higher than other categories, given the fact that more perturbed Primate images are blocked by the foundation model. We attribute to the limitation of foundation models. As the CLIP model has been influenced by the imbalance sample distributions across the Internet, it could only handle easy perturbed samples well. Therefore, the perturbed images preserved would be those that can be easily classified by the models.
Table 14: Details of test 9-class ImageNet for vanilla ResNet- (step size is 0.001, computation budget B is 50)
Type
SA
VR
FMR
Dog
93.33
100.00
18.09
Cat
96.67
100.00
28.60
Frog
85.33
100.00
20.72
Turtle
84.67
100.00
24.80
Bird
91.33
100.00
27.68
Primate
96.00
100.00
27.11
Fish 94.00
100.00
25.13
Crab 96.00
100.00
19.15
Insect
93.33
100.00
23.16
Total
92.30
100.00
23.94
In our official configuration, we set a relatively smaller step size to perturb the image and obtain enough more perturbed images. shown in Table [4] using smaller step size value and enough computation budget barely affect the overall results. In addition, with smiler step size, manage to perturb the image little by little and can get enough more perturbed images (VR becomes 100 on every category, indicating that all the images are perturbed and maintained their image-label structure). Admittedly, the foundation model's bias still exists here, e.g., the Primate images (FMR- 28.11) still easier than Dog images (FMR 18.09). However, considering the huge performance gap between the foundation model and the evaluated models, images that are easy for the foundation model are hard enough for the evaluated models (The FMR of Dogs and Primate images are closer and
25

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
smaller compared with those in Table 13 which is sufficiently efficacious for real- world applications. Additionally, the employment of an ensemble of multiple foundation models in our methodology serves to provide a further layer of alleviation for the aforementioned issue.
K DISCUSSIONS ON THE SOCIETAL BIAS OF RELYING ON LARGE MODELS
K.1 POTENTIAL NEGATIVE IMPACTS OF FOUNDATION MODELS
Although the bias incurred by foundation models is less detrimental than the biases arisen from fixed benchmark datasets, more detailed discussion on the potential negative impacts is necessary. One potential bias of making vision models behave more like the foundation models is that the vision model may inherit the limitations and assumptions of foundation models' training data and objective function. For example, foundation models' training data may not cover all possible visual concepts or scenarios that are relevant to a given task; foundation models' objective function may not align with the desired outcome or evaluation metric of given task; foundation models' natural language supervision may introduce ambiguities or inconsistencies that affect the model's performance or interpretation. These limitations and assumptions may affect the generalization and robustness of vision models that rely on foundation models. Moreover, we add recent works that especially investigate the bias of foundation models, and guide the readers to it for further warning, e.g., (Menon et al. 2022) and (Zhou et al. 2022b).
K.2 SOCIETAL BIAS OF RELYING ON LARGE MODELS
Moreover, our method relies on large models, where their societal bias is still unclear, therefore a related discussion would be beneficial.
Large-scale models could leverage the rich knowledge and generalization ability encoded in the training stage. However, one potential societal bias of relying on large models' supervision on preserving the perturbed image could be that would privilege certain groups or perspectives over others based on social or cultural norms. As the data used to train the pre-trained models may be imbalanced, incomplete, or inaccurate, leading to biased representations of certain groups or concepts, the perturbed images preserved by the pre-trained models may reflect stereotypes, or discrimination against certain groups of people based on their race, gender, age, religion, etc., which may be harmful, offensive, or deceptive to the users. Bridging the gap between the pre-trained model and the evaluated vision models will make the vision models inherit the limitations of pre-trained models, which have adverse consequences for people who are affected by them, such reinforcing stereotypes, discrimination, or exclusion.
We add recent works that investigate the societal bias of large models, and guide the readers to it for further warning, e.g., (Wang et al. 2022a).
L EXPERIMENTS ON THE ZERO-SHOT ADVERSARIAL ROBUSTNESS OF CLIP
We conduct the following experiment to compare the adversarial vulnerability between CLII and robust ViT-like model pre-trained checkpoints of XCiT-LI2 (Debenedetti et 2022) from the RobustBench Leaderboard (Croce et al. 2020). The results are shown in Table 15 We find that the vanilla CLIP shows better robustness performance under our quick experiments through FGSM attack. However, if we continue the attack process, we will eventually obtain the adversary that changes the CLIP's classification decision the targeted class.
Fortunately, in production, one can use simpler techniques such as gradient masking to protect CLIP's weights from malicious users, thus, the opportunities the CLIP being attacked from a white-boo box manner are quite low. In terms of black-box attacks, CLIP actually shows a strong resilience toward the adversarial samples generated for other models, for which we also have some supporting evidence: In Appendix we generate the images with the FGSM attack by the tested model. Table shows that initiating with the FGSM adversarial examples barely affects the FMR, which implies that CLIP succeeds in defending these black-box adversarial images and preserving the hard ones such that the FMR does not change significantly (Otherwise, CLIP will discard heavily perturbed images and preserve easy ones with minor perturbation, leading to high FMR values). Furthermore, our approach
26

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
Table 15: Comparison of the zero-shot adversarial robustness of CLIP with pretrained robust model. We find that CLIP shows better robustness performance compared with XCTT-L12. We note that the CLIP's classification decision can be changed the targeted class as attack continues.
Target loss
p[true=0]
p[target=1]
Step
CLIP
XCIT-L12
CLIP
XCIT-L12
CLIP
XCIT-L12
8.621
4.712
0.6749
0.7437
0.0052
0.0728
20
2.715
1.605
0.5083
0.4074
0.0986
0.2009
40
2.316
0.8877
0,4007
0.2562
0.1357
0.3116
60
1.684
0.7420
0.2177
0.1407
0.2144
0,4760
80
1.540
0.6520
0.1813
0.1338
0.3335
0.5210
incorporates an ensemble of foundation models, including robust models such as ConyNext-T--CC from the RobustBench Leaderboard, and employs a majority vote mechanism to validate the fidelity of the image-label relationships.
Thus, CLIP, especially when equipped with techniques to protect its weights and gradients, and coupled with an ensemble of robust foundation models, might the closest one serve as the ideal foundation models to maintain the image-label structure at this moment.
M LIST OF EVALUATED MODELS
The following lists contains all models we evaluated on various datasets with references and links to the corresponding source code.
M.1 PRETRAINED VQGAN MODEL
checkpoint of vggan imagenet f16 16384 from Setps://heibor uni mni-her uni-heidde otat-aaeeeeeeere...e////00000000
We use the
M.2 PRETRAINED FOUNDATION MODELS
1. Model weights of ViT-B/32 and usage code are taken from openai/CLIP
https://githbb.com
2. CoCa (Yu al. 2022 https::/ggttaai.
3. ConvNeXt-T-CvSt revisiting-at
(Singh et al.
2023
https://ggtthub..oommmmdded
M.3 TIMM MODELS TRAINED ON IMAGENET (WIGHTMAN 2019
Weights taken from http://www////got/to/ooo//////ta
rree aase/master//e htt.m//mmmmmmmmmmmmmmmmmmmmt models
1. ResNet50 (He
2016a
2. ViT (Dosovitskiy et al.
2020
3. DeiT (Touvron et al.
2021
4. Twins (Chu et al. 2021
5. Visformer (Chen et al.
202
6. Swin (Liu el al. 2021
7. Co (Lu et al.
2022
27

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
M.4 ROBUST RESNET5O MODELS
1. ResNet50 SIN+IN (Geirhos et al. 2019 https://gi.huub..om//gggnn.n
Lexture-vs--ee shape
2. ResNet50 ANT
game-of-no (Rusak et al.
http:://gstaubb.com//etttt..
3. ResNet50 ANT+SIN
fame-of-noise Rusak al.)
ittps://g1tlub..com//om
4. ResNe150 Augmix (endryykk et al.
2019
https://github.com
spogle-tessarch/auumme
5. ResNet50 DeepAugment
(Hendrycks et al. 2021a
https://github.com
(snddyycks//mmgeneet-r
6. ResNeetoODeepAAumenn-AAuuug (Hendrycks et al. Teedryycks/mmagenet=
202
https://itthb..om
7. ResNet50 Discrete Adversarial Training (DAT) (Mao et al.
2022b)
https://githu:
ccon/allbabaaeesyyrroo
M.5 ADDITIONAL IMAGE GENERATORS
1.
Efficient- Hazami et al. 2022
Efficient VDVAE
2. Improved DDPM Nichol & Dhariwal 2021
https:://gittub.com//eennne
mmgenerat inn http:/eee/mmmttt/ooommo/
3. ADM (Dhariwal & Nichol
2021)
https://gitiubb.commperaal
guided di ffusion
4. StyleGAN (Sauer et al.] 2022)
stylegan x1
M.6 PRETRAINED XCIT-LI2 MODEL
Model weights of XCIT-L12 Debenedetti et al. 2022 are taken from https:://ithub.. com/
dedeswi m/vi vits row.stneesssss...............s.e torch
N
LEADERBOARDS FOR ROBUST IMAGE MODEL
We launch leaderboards for robust image models. The goal of these leaderboards follows:
To keep on track of sfatc-of-the-art on each adversarial vision task and new model architectures with our dynamic evaluation process.
To the comparison of robust vision models at glance (e.g., performance, speed, size, etc.).
To access their research papers and implementations on different frameworks.
We offer a sample of the robust ImageNet classification leaderboard in supplementary materials.
ADDITIONAL PERTURBED IMAGE SAMPLES
In Figure 4 we provide additional perturbed images generated according to each model. We have similar observations to Section 4.3 First, the generated perturbed images exhibit diversity that many other superficial factors of the data would be covered, i.e., texture, shape and styles. Second, our method could recognize the model properties, and automatically generate those hard perturbed images to complete the evaluation.
In addition, the generated images show reasonable transferability in Table 6 indicating tha method can be potentially used in a broader scope: we can also leverage the method to generate static set of images and set benchmark dataset to help the development of robustness methods.
28

---

Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
airliner
wing
shuttle
warplane
wing
warplane
warplane
tram
prison
crib
vending
chest
school bus
tram
ostrich
dromedary
Ilama
poodle
Ilama
dromedary
llama
tree frog
mamba
greenlizard
greenlizard
greenlizard
chamelcon
anole
washer
speaker
iPod
microwave
remote
iPod
speaker
Figure 4: Visualization of the images generated by our system in evaluating the common corruption robust model, with the original image shown (left image of each row). The caption for each image is either the original label or the predicted label by the corresponding model. The evaluated models are SIN, ANT, ANT+SIN, Augmix, DeepAug and DeepAug+AM from left right.
Therefore, also offer two static benchmarks in supplementary materials that are generated based on CNN architecture, i.e., ConvNext and transformer variant, i.e., ViT, respectively.
P DISCUSSION ON THE REALISM OF THE GENERATED IMAGES
We notice that some generated images look unnatural, as the generated images being realistic is not part of the optimization function. We acknowledge that making the generated images appear more natural will be a further desideratum, as this contributes to enhancing the human-perces perceppprceetible interpretability.
Nonetheless, the current research agenda of the robustness evaluation community is still to encourage the evaluation expose the model's 's weakness, such expose and eliminate the model's learning of spurious correlation in rare cases.
Similar evidence can be found in (Xiao al., 2023). where the authors utilize masked images as counterfactual samples for robust fine- tine-tuning. In this paper, the authors argue that masked images can break the spurious correlation between features and labels that may degrade OOD robustness, and that canture-based distillation with the pre-trained model on these counterfactual samples can achieve better trade-off between IID and OOD performance. According to our second desideratum, our generated counterfactual images might also look unnatural. However, although it appears unnatural, it beneficial uncovering and eliminating spurious correlations for enhancing the model robustness.
29


Reviews:
[{"review_id": "uPz1JPzxTr", "reviewer": "Reviewer_LU8n", "summary": "The paper proposes a method to generate perturbed images which have to take the \"to-be\" evaluated models into account and uses the prediction of the pre-trained large foundation models as \"oracle/ground-truth/constraints\". These perturbed images could serve as \"dynamic\" evaluation benchmarks for robustness evaluation compared to previously fixed benchmarks.", "questions": "see [weaknesses].", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) The paper is well-organized and the writing is clear and easy to follow;\n2) The problem is clearly stated and the experiments have included enough details.\n3) The authors have provided a nice discussion, which really benefits readers to better understand this proposed method.", "weaknesses": "1) \\theta is an input for Algorithm-1, but do not see where it has been used in Algorithm-1.\n\n2) It is not clear to me why this method has any advantages over other benchmarking datasets. The big difference is this proposed method will take the \"to-be\" evaluated model into account when generating the \"adversarial examples\". It looks like this proposed method could generate \"personalized\" \"adversarial\" examples for the \"to-be\" evaluated models. So each \"to-be\" evaluated model may be evaluated on totally different datasets. But in order to fairly compare across different \"to-be\" evaluated models, the authors normalize their performance by the pre-trained large foundation model? However, I do not see any benefits as to why this evaluation pipeline could better reflect real-world scenarios.\n\n3) It is not clear what \"SA\" means. Is SA the clean test accuracy of the foundation model? Also, there is an abusive use of \"SA\" to present two different terms in this paper---- SA for \"standard accuracy\" and SA for \"self-attention\".\n\n4) If the perturbed generated images are the images that cause the maximum classification loss of the \"to-be\" evaluated model, then it will definitely result in a very bad classification performance of the \"to-be\" evaluated model on the perturbed images. It is not clear to me how the authors do this kind of evaluation. From my current understanding, the generated images have already had the knowledge of the \"to-be\" evaluated model. Then I doubt how \"PA/SA\" could be a fair comparison for different \"to-be\" evaluated models.\n\n5) The overall idea looks very similar to \"adversarial training\". The big difference is instead of using \"norm-bouned\" perturbation, here the authors use the prediction of large foundation models as a constraint to limit the perturbation to not go too far. So from this perspective, I only see limited novelty of this proposed method. And even, the perturbed images may be eventually biased towards the foundation models.\n\n6) Could we also report the performance of the \"to-be\" evaluated models on some of fixed benchmarks as a robustness reference?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ION8ohZaG3", "reviewer": "Reviewer_eMG1", "summary": "This paper introduces a novel approach to evaluate machine learning models' robustness. Rather than relying solely on fixed benchmarks, the authors propose a method that directly measures image classification models' performance in comparison to a surrogate oracle, a set of foundational models. They extend image datasets with perturbed samples that maintain the original image-label structure but are distinct from the original set. This approach allows for a more comprehensive evaluation of models' robustness, overcoming limitations associated with fixed benchmarks and constrained perturbations. The paper not only presents evaluation results but also provides insights into model behaviors and the new evaluation strategies employed.", "questions": "See above.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper discusses the evaluation of model performance, which is an important topic.", "weaknesses": "1. The text in Image 1 is too small to discern clearly.\n2. I am skeptical about the significance of the paper. I really don't understand why the performance needs to be compared with the foundation model. If there are already foundation model for the given task, is it really necessary to train a new model with similar architecture from scratch?\n3. If users need to train different generative models for various downstream tasks, I think this evaluation approach is not user-friendly.\n4. Is there a possibility that because the evaluation method proposed in this paper uses VQGAN, and DAT also utilizes VQGAN during training, DAT performs best under this specific evaluation scenario? If that's the case, the evaluation method proposed in this paper might not provide enough insight to the readers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z8hcp8EGHf", "reviewer": "Reviewer_FCU2", "summary": "This paper proposed foundation model-oriented robustness, where foundation models are used as the oracle when testing vision models’ robustness. A pipeline is developed to maximize model’s misalignment while preserving semantics with an ensemble of foundation models, and a limiting computation budget. Lastly the authors presented a systematic study of Foundation Model-oriented Robustness (FMR) on ImageNet and ImageNet-C.", "questions": "1. As discussed in the introduction, two main limitations of existing evaluations are the limited type of perturbations, and testing data in a different distribution from training set. How well did this work improve these issues over previous evaluations? Are the perturbations more diverse? For the data distribution, there is also a gap between ImageNet training and foundation model training left unaddressed.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The proposed pipeline made use of existing foundation models as the oracle, addressing several issues of existing testing benchmarks, such as limited to specific curated perturbations or a distribution different from the training set.\n2. The flexibility of the proposed method allows robustness evaluation on a wide range of image datasets and vision models.\n3. The authors presented a thorough study of vision models (CNN or transformer based), and different augmentation methods.", "weaknesses": "1. The perturbations produced by the model seem limited to those barely visible to human beings. Is this desirable? For now all perturbations seem weak, they are small perturbations around the image, barely visible to humans. There are many strong generation models, such as edge-conditioned ControlNet, or inpating based diffusion models that can generate very diverse outputs in terms of colors, and styles. I think these strong perturbations would be very interesting and naturally suitable in this framework.\n2. There’s limited understanding of what the perturbed images are. This is important to support the “diversity” argument in the introduction. How similar are the images to ImageNet-C perturbations?\n3. It’s challenging to read the texts in Figure 1…", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)