Generalizability of Adversarial Robustness Under Distribution Shifts
Authors: Kumail Alhamoud
Hasan Abed Al Kader Hammoud
Motasem Alfarra
Bernard Ghanem
Venue: Accepted by TMLR ()

Abstract: Recent progress in empirical and certified robustness promises to deliver reliable and deployable Deep Neural Networks (DNNs). Despite that success, most existing evaluations of DNN robustness have been done on images sampled from the same distribution on which the model was trained on. However, in the real world, DNNs may be deployed in dynamic environments that exhibit significant distribution shifts. In this work, we take a first step towards thoroughly investigating the interplay between empirical and certified adversarial robustness on one hand and domain generalization on another. To do so, we train robust models on multiple domains and evaluate their accuracy and robustness on an unseen domain. We observe that: (1) both empirical and certified robustness generalize to unseen domains, and (2) the level of generalizability does not correlate well with input visual similarity, measured by the FID between source and target domains. We also extend our study to cover a real-world medical application, in which adversarial augmentation significantly boosts the generalization of robustness with minimal effect on clean data accuracy.

Published in Transactions on Machine Learning Research (05/2023)
Generalizability of Adversarial Robustness Under Distribu- tion Shifts
Kumail Alhamoud King Abdullah University of Science and Technology (KAUST)
bumailhamoouu@kuut.tcom
Hasan Abed Al Kader Hammoud"
King Abdullah University of Science and Technology (KAUST)
Motasem Alfarra
motasemm.eljrrmn@aassteeh
King Abdullah University of Science and Technology (KAUST)
Bernard Ghanem
bermarigghaneem@@kaustteeee
King Abdullah University of Science and Technology (KAUST)
Reviewed on OpenReview: https: // openreview. net/ forum? id= XNFo34Q4CJ
Abstract
Recent progress in empirical and certified robustness promises to deliver reliable and de- ployable Deep Neural Networks (DNNs). Despite that success, most existing evaluations of DNN robustness have been done on images sampled from the same distribution on which the model was trained. However, the real world, DNNs may be deployed in dynamic environments that exhibit significant distribution shifts. In this work, we take a first step towards thoroughly investigating the interplay between empirical and certified adversarial robustness on hand and domain generalization another. To do so, train robust models on multiple domains and evaluate their accuracy and robustness main. We observe that: (1) both empirical and certified robustness generalize to unseen domains, and (2) the level of generalizability does not correlate well with input visual simi- larity, measured by the FID between source and target domains. We also extend our study to cover a real-world medical application, which adversarial augmentation significantly boosts the generalization of robustness with minimal effect on clean data accuracy.
unseen do-
1 Introduction
Deep Neural Networks (DNNs) are vulnerable to small and carefully designed perturbations, known adversarial attacks (Szegedy et al., 2014; Goodfellow et al., 2015). That is, DNN fo Rd P()') produce two different predictions for the inputs E gd and x 6, although both æ and x are perceptually indistinguishable. Furthermore, DNNs are found to be brittle against simple semantic transformations such as rotation, translation, and scaling (Engstrom et al., 2019).
These observations raise concerns regarding the deployability of DNNs in security-critical applications, such as self-driving and medical diagnosis (Papernot et al., 2016; Finlayson al., 2019; Ma et al., 2021). This brittleness motivated several efforts to build models that are not only accurate but also robust (Gu & Rigazio, 2015). Building robust models is usually achicved either (i) empirically, where the DNN training routine is changed to include such malicious adversarial examples in the training set (Madry et al., 2018), (ii) certifiably, where theoretical guarantees are given about the robustness of DNN in a region around a given input x (Lécuyer et al., 2019). While recent work on adversarial robustness has made significant strides in developing accurate and robust models, most methods are only evaluated on in-distribution data. This means
"The first authors contributed equally: each author has the right their name first on their CV.

---

Published in Transactions on Machine Learning Research (05/2023)
Robust Training on Source Domain
Robust Evaluation on Target Domain
Adversary
Adversary
Source
Thrrge
Giraffe
?
M
M
Figure/ 1: Does robust model trained in a (source) domain maintain its robustness when deployed in another (target) domain? We investigate the generalizability of empirical and certified robustness to various unseen domains.
that the training and testing datasets are assumed to be independently and identically distributed (ID). However, this IID assumption rarely holds in practice, as data in the real world can be sampled from various distributions with significant domain shifts. For example, deep-learning based medical image classifier may be trained on data collected from one hospital, but later deployed a different hospital (Bándi et al., 2019). Unfortunaeely, DNNs struggle to generalize to out-of-domain data (Geirhos et al., 2020; 2021), even in the absence of adversarial examples. This lack of generalization has led the research community to invest in the problem of Domain Generalization (DG). The aim of DG is to learn invariant representations from diverse distributions of data, denoted as source domains, such that these representations generalize to unseen distribution, known as the target domain (Wang et al., 2021; Guirajani & Lopez-Paz, 2021). This sctup mimics the unexpected nature of real-world distribution shifts, where models can be regularly exposed lo novel domains, and fine-tuning on all these domains becomes impractical. While there has been considerable effort in improving the gencralizability of DNNs (Tzeng et al., 2014; Sun & Sacnko, 2016; Motiian et al., 2017: Zhang et al., 2021; Shen al., 2021; Wang al., 2021; Zhou al., 2022), the generalizability of adversarial robustness unseen domains remains unexplored.
Our work examines the interplay between domain generalization and adversarial robustness through com- prehensive experiments on five standard DG benchmarks provided by DomainBed (Gulrajani & Lopez-Paz, 2021) and WILDS (Koh et al., 2021). We investigate empirical and certified robustness against input per- turbations and spatial deformations. We first investigate the generalizability of empirical robustness, which DNN obtains by employing the popular adversarial training method (Madry al., 2018) on the source data. We then inspect the generalizability of certified robustness against input perturbations and parametric deformations by employing Randomized Smoothing (RS) (Cohen et al., 2019) and DeformRS (Alfarra al., 2022a). To the best of our knowledge, we provide the first large-scale experimental analysis of the general- izability of adversarial robustness to unseen domains. Our analysis leads to the following contributions:
We contrast the behavior of robustness under both transfer learning and domain generalization. Unlike transfer learning, domain generalization does not necessarily improve through robust training.
2. We empirically show that visual similarity between the source and target domains does not correlate well with the level of generalizability to the target domain.
3. We show that empirical and certified robustness generalize to unseen domains in different setups, including in a medical setting with a real-world distribution shift.
2
Related Work
Domain Genoralization. Domain generalization (DG) studies the ability to learn representations that can be readily applied to data from unseen domains (Wang et al., 2021; Zhou el. al., 2022). In the DG setup, a model is trained on multiple source domains and then evaluated on an unseen target domain, which exhibits a significant shift from the training domains. DG methods can be categorized into different groups. (i) Data augmentation techniques learn generalizable models by increasing the diversity of the source data (Gong et al., 2019; Zhou et al., 2020; 2021). (ii) Representation learning methods rxtract domain-invariant
2

---

Published in Transactions on Machine Learning Research (05/2023)
Table I: Comparison between Domain Generalization (DG) and Transfer Learning. DG differs from transfer learning in two ways. 1) The model in DG never sees the target data during training, so fine-tuning on the target is not allowed. 2) Target labels are fixed in DG; however, the target samples are drawn from a domain that is distinct from the source domains.
Problem Setup
Training Data
Target Data
Problem Condition
Access to Target
Transfer learning
Ssource, Starget S (Siji I,....N. N}
Starget Sx+1
Ysource # Viarget (Xy(Sk) # Pxy for k &
Domain generalization
x
representations that scamlessly apply in any unseen domain (Blanchard et al., 2011; Nguyen et al., 2021; Lu et al., 2022) (iii) Learning-stralegy approaches may achieve generalization through meta-learning, self- supervised learning, or optimization procedures that seek flat minima (Li al., 2018; Carlucci et al., 2019; Cha et al., 2021). In this work, we study DG from adversarial robustness lens. In particular, we analyze both the generalization accuracy and robustness of adversarially trained classifiers in unseen domains.
Adversarial Robustness. Adversarial attacks are imperceptible, scmantic- prreatti-preserving perturbations that can fool DNNs (Goodfellow et al., 2015; Szegedy et al., 2014). Given the security concerns that adversarial attacks induced, several works proposed changing the training routine to enhance model robustness (Zhang et al., 2019). For example, adversarial training (Madry et al., 2018) encourages the model to classify ad- versarial examples correctly. While empirical defenses like adversarial training are effective in enhancing the robustness of the underlying model, such approaches do not guarantee robustness. Subsequently, many empirical defenses were broken when more powerful attacks were designed (Carlini & Wagner, 2017; Athalye et al., 2018). As a result, there has been growing interest in certifiably robust. classifiers, for which no adversary can exist in a specified region around data point (Raghunathan et al., 2018; Mohapatra et al., 2020; Lee et al., 2021). A scalable approach to achieving cortified robustness is Randomized Smoothing (RS) (Cohen al., 2019). RS constructs a smooth classifier from any arbitrary base classifier by outputting the most probable class when the input is subjected to Gaussian noise. Recently, DeformRS extended RS to provide certified robustness against parameterized geometric deformations (Alfarra et al., 2022a; S, et al., 2022). In this work, we aim to study the interplay between (empirical and certified) robustness and domain generalization by deploying adversarial training, RS, and DeformRS.
Adversarial Training in Dynamic Environments. To learn generalizable knowledge, machine learning researchers have proposed several problems, such as transfer learning, continual learning, domain adaptation, and domain gencralization (Zhuang et al., 2020; Delange et al., 2021; Wang & Deng, 2018; Wang et al., 2021). Among these problemss, only transfer learning, where a model pre-trains on tasks with large datasets and then adapts to downstream tasks with limited data, has been thoroughly studied under the lens of adversarial robustness. Salman et al. (2020) showed that, in terms of downstream task accuracy, adversarially trained representations outperform nominally trained representations. Utrera et al. (2021) further explained that adversarial training in the source domain increases shape bias, resulting in better transferability. Finally, Deng et al. (2021) provided a theoretical justification to support these empirical findings. Besides downstream task accuracy, Shafahi et al. (2020) studied the transferability of robustness itself. Although useful, these transfer learning results presume fine-tuning on the target domain, which is not possible in many real-life scenarios. Table illustrates the differences between transfer learning and domain generalization, which is the setup we adopt. In this paper, we take first step to empirically investigate whether adversarial training leads to robust representations that generalize well without requiring prior knowledge of the target domain.
3 Background on Domain Generalization
Domain Generalization Setup. Given an input space X' and a label space V, one can define joint distribution Pxy over X' and y. A domain, or distribution, is collection of samples drawn from Pxy. In multi-source domain generalization, there are N source domains of varying sizes (D,)N= where for each n, the domain is defined by D. ((rg,1))) (Pal XY- We define the training set S by the union of
3

---

Published in Transactions on Machine Learning Research (05/2023)
the N source domains S = U Dn. and we assume the existence of some unseen target domain Dx+1 {1y -5)) ~ (P(++1) We enforce that p() * Poa for k + k, E {I,..., N + 1}, which means that the target domain is distinct from the source domains, which are, in turn, distinct from each other. The aim of DG to use the source domains S learn a mapping f - y that minimizes the error on the unseen target domain. More formally, we seek a parameterized model fo- such that:
0 = arg min E (an))=())))) [(()()).)),
(1)
where C is the cross-cntropy loss for the classification task. However, the model is not allowed to sample the larget domain during training. Therefore, most methods use the empirical risk of the source datasets as a proxy for the true target risk. The supervised average risk (E) is given by:
N N TL
s.al
3
Σ [C(fo())
(2)
SR
with (I, ~ S. In practice, we define fixed held-out validation set S S. The average risk on this source validation set is used to select the best model, which is evaluated on the target domain without any fine-tuning steps. In what follows, Section 4 (and Section 5) investigates the generalizability of empirical (and certified) robustness to diverse target domains. In both sections, we follow the advice of DomainBed (Gulrajani & Lopez-Paz, 2021) in using empirical risk minimization (ERM) as a baseline method, and we focus on the effect of robustifying ERM.
4 Empirical Robustness and Domain Generalization
In this section, we study the generalizability of empirical robustness methods that enhance the adversarial robustness of DNNs. We begin with brief introduction of Adversarial Training (AT) (Madry et al., 2018), after which we study the effect of deploying AT in domain generalization setup.
4.1 Background and Setup
Adversarial Attacks. Adversarial attacks are imperceptible perturbations that, once added to a "clean" input sample, cause the classifier fo to misclassify the perturbed sample. Formally, let (x, y) be an input label pair where fe correctly classifies z (i.e. argmax, fi(x) y). An attacker crafts a small perturbation 5 such that arg max, f/(( (xr + 6) # y, which is usually obtained by solving the following optimization problem:
max [(fo( + +8), y) 8
s.t. 101lp S e,
where p E 00), e >> 0 is small constant that enforces the imperceptibility of the added perturbation, and L is suitable loss function (e.g. Cross Entropy). Let 8 be the solution to the problem in Eq.. 3, then the adversarial example is denoted by Fade +
Adversarial Training as Augmentation. Adversarial Training (AT) (Madry et al., 2018) trains the clas- sifier on adversarial examples rather than clean samples. In particular, AT obtains the network parameters 8 by solving the following optimization problem:
min BERyND
max (.116ll.Se
((fo(x+0),,))
+
(4)
where D is a data distribution. In general, the inner maximization problem is solved through K steps of Projected Gradient Descent (PGD) (Madry al., 2018). While conducting adversarial training enhances the model's robustness against adversarial attacks, this usually comes at the cost of losing some clean accuracy (performance unperturbed samples). To alleviate the drop in performance, we follow the method by
4

---

Published in Transactions on Machine Learning Research (05/2023)
Zhang et al. (2019) and deploy adversarial training data augmentation scheme. In particular, we obtain
network parameters 0* that minimize the following objective:
arg min R (Z.))) (AC(O)())))))))))))))))))))))))) + (1 X) (xFad)) v)},
where À E [0 controls the robustness-accurary trade-off. Furthermore, we experiment with the more powerful method TRADES (Zhang et al., 2019). In particular, TRADES minimizes the following loss:
0" = arg arg min E ((ag))a [C(fo(r), + max fC(f(()).f((z+)))) 1e
(6)
0
Adversarial Training Setup. In our experiments, we focus on image classification and adopt the frame- work of DomainBed (Gulrajani & Lopez-Paz, 2021), which is the standard benchmark in the image domain generalization literature. All models are initialized with ResNet-50 backbone pre-trained on ImageNet- 1K (Deng et al., 2009). We train PGD models with adversarial angmentation to minimize the objective in Eq. 5 on the source domains, where À = 0.5 and Todv is computed with a Projected Gradient Descent (PGD) attack (Madry et al., 2018) using 5 PGD steps. The TRADES models are trained to minimize the objective in Eq. 6, where 3 = 3. For all the models, the target domain remains unseen until test time. Specifically, we follow the moining-domaiin validation strategy described in DomainBed for model selection. We study robustness under variety of datasets: PACS, OfficeHome, VLCSS, and TerraIncognita (Gulrajani & Lopez- Paz, 2021). In the main paper, we report lo results using 2/255. The appendix includes additional results for 4/255 and = 8/255 robustness evaluation, leading to similar trends and conclusions.
Evaluation Setup. For each considered dataset, we select a subset of N - 1 domains to be the source (training) domains and keep the Nohh domain as the target (evaluation) domain. For example, we have four splits in PACS: Photo, Art, Cartoon, and Sketch. Hence, we have four combinations of source vs. target domain splits. We follow DomainBed in reporting the average result across all N different source vs. target splits Furthermore, we run each experiment with 3 different seeds and report the standard deviation across our runs. Note that we split the source domains (training set) into two subsets: a training subset (80%) and validation subset (20%). It's important to note that the results we present in Table 2 are based on the validation subset.
To evaluate the robustness of our models, we assessed their performance on the same norm budget they were trained on. For AutoAttack, we used the default number of steps provided in its implementation. For PGD, we conducted the evaluation with 20 steps.
4.2 Generalization of Empirical Robustness
In this section, we investigate the generalizaiility of empirical robustness to unseen domains. More precisely, we are interested in understanding the interplay between standard accuracy and robust accuracy in the scope of source vs. target domains. In Table 2, we report the standard accuracy, fox AutoAttack robust accuracy (Croce & Hein, 2020), and f PGD robust accuracy for nominally- and addversarially-trannee models using both TRADES Eq. equation 6 and PGD augmentation Fq. equation 5 techniques. The reported results show the average and standard deviation of four runs with different seeds and with e fixed to 2/255. Results for c c= 1/255 are in the appendix. Next, we analyze these results to answer the following questions.
Q1: Does adversarial training result in a better generalization to clean samples in the target domain? No, which is evident from comparing the clean target accuracy of the baseline with the other two adversarial training methods in Table 2. We further observe that the more robust TRADES generalizes worse than the simpler training with PGD augmentation. training in the source domain is favorable, robust training does not improve the clean data accuracy in the target domain if no fine-tuning is allowed. This result contrasts with findings from the transfer learning literature, where models trained robustly in the source domain outperform standard- trained models across a varioty of downstream tasks (Salman et al., 2020). It is especially surprising given that previous works suggest that robust training encourages shape bias over texture bias, hinting at better generalization (Geirhos et al., 2019; Utrera et al., 2021). Moreover, Deng et al. (2021) showed that adversarial
Unlike transfer learning, where robust
5

---

Published in Transactions on Machine Learning Research (05/2023)
Table 2: Evaluation of &o Robustness. We train models in the source domain and then evaluate their clean accuracy and robust accuracy in the source and target domains. The models are trained in one of three ways: nominal training (baseline), PGD adversarial training, or TRADES adversarial training. The robust accuracy is measured against AutoAttack and PGD adversarial attacks. For adversarial training and evaluation, the perturbation norm is set to 2/255. Overall, the robustness of PGD-trained and TRADES-trained models transfers to the target distribution.
Source
Target
Method
Dataset
Clean Acc.
Acc. (AA)
Acc.(PGD)
Clean Acc.
Acc. (AA)
Acc. (PGD)
PACS
94.69 0.23
1.56 +: 0.33
3.96 +: 0.56
80.24 1.72
0.34 0.16
1.16 :: 0.38
OfficeHame:
77.08 0.30
0.08 + 0.02
0.40 + 0.05
58.88
0.85
0.56 01.11
Baseline
VLCS
84.34 = 0.12
0.00 + 0.00
0.00 + 0.00
74.35
1.04
0.00 0.00
0.02 : 0.04
Terratncognita
86.72 0.22
0.00 + 0.00
0.00 + 0.00
44.10
2.52
0.00 + 0.00
0.00 + 0.00
PACS
92.83 0.22
76.00 + 0.33
78.12 + 0.37
75.29 +: 0.56
56.69 + 0.91
55.82 +: 1.74
OfficeHome
72.04 0.40
52.32 + 0.69
52.81 + 0.98
52.19
0.73
34.03 0.26
34.51 + 0.21
PGD
VLCS
79.70 = 0.22
38.76 + 0.41
60.02 + 0.75
69.15
0.74
47.12 0.69
46.73 # 1.64
Terrainc
71.62 0.74
52.85 2.25
56.05 + 2.47
27.53
3.96
5.59 0187
PACS
91.16 0.08
79.89 +: 0.22
79.70 + 0.95
72:32 + 0.77
57.96 + 1.56
57.63 4: 1.45
OfficHHame
69.12 0.15
54.52 + 0.74
56.14 + 0.59
48.47 + 0.45
35.79 + 1.14
36.11 + 1.63
TRADES
VLCS
78.58 0.17
63.01 + 0.79
63.30 + 0.63
09.36 + 0.68
52.78 + 1.66
53.27 + 0.97
TerraIncogniss
69.81 0.43
58.64 + 0.79
59.38 + 0.99
25.27 3.16
5.49 0.59
7.84 + 0.65
training in the source domain results in provably better representations for fine-tuning on the target domain. Such seemingly contradictory findings can be reconciled by considering the key differences between transfer learning and domain generalization summarized in Table 1. Specifically, previous works in transfer learning assume that the model can sample the target domain at some point to perform fine-tuning. Since domain generalization does not allow access to the target domain, such benefits are not guaranteed. We encourage future works to investigate under what conditions adversarial training helps the generalization accuracy with fine-turing the target.
Q2: Does a higher source domain robustness correspond to a higher target domain robustness? As expected, DNNs lose some robustness when evaluated on a target domain that is distinct from the training domains. This is evident by comparing the Robust Acc. (AA) and Robust Acc. (PGD) of the source domain to those of the larget domain experiences a drop of 19.31% in terms of AutoAttack robust accuracy and 22.30% in terms of PGD robust accuracy. However, it is consistently obscrved that corresponds to higher robustness in the target domain. Our results suggest that one way to increase the out-of-distributioo robustness of a deployed model is to improve its robustness in the source validation set, which supports the applicability of ongoing efforts in adversarial robustness research (Zhang et al., 2019; Wang et al., 2020; Wu et al., 2020). This is also validated by comparing the TRADES results with those of PGD Table 2. TRADES, a stronger adversarial training method compared to PGD augmentation, achieves higher source domain robust accuracics and, in turn, higher target domain robust accuracics.
Table 2. For example, the PACS model trained with PGD augmentation
higher robustness in the source domain
Q3: Does the robustness-accuracy trade-off generalize to unseen domains? As observed in Table 2, 3 a more robust model comes at the cost of standard accuracy not only in the source domain, but also in the target domain. In most cases, both PGD and AutoAttack robust accuracies of standard-trained (baseline) models are around 0%. Those models, on the other hand, are the models that achieve the best in terms of generalizability, í.e. in terms of clean accuracy. However, the most robust models, i.c. TRADES, tend to have the lowest target domain clean data accuracy. Therefore, consistent with the robustness literature (Tsipras et al., 2019), robustness comes the cost of standard accuracy even in the unseen larget domains.
6

---

Published in Transactions on Machine Learning Research (05/2023)
5 Certified Robustness and Domain Generalization
In Section 4, we analyzed the interplay between empirical robustness (obtained by adversarial training) and domain generalization. While empirical robustness studies give hints about the reliability of a given model under adversarial altacks, they give no guaranlees against. the existence of such adversaries. To deploy DNNs in dynamic environments (Koh al., 2021), we need robustness guarantees to carry over into unseen domains. To that end, study the generalizability of the certified robustncss of DNNs. We also study whether the visual similarity between the source and largel domains influences this robust generalizability. We deploy Randomized Smoothing (RS) and DeformRS to certify DNNs against input perturbations and deformations, and we utilize the FID and R-FID metrics to compute cross-distribution similarity. We start by giving a brief overview of RS and DeformRS, followed by some background on FID and R-FID.
5.1 Background and Setup
Certifying Against Additive Perturbations and Input Deformations. Randomized smooth- ing (RS) (Cohen et al., 2019) is a method for constructing a "smooth" classifier from a given classifier fe. The smooth classifier returns the average prediction of fo when the input æ subjected to additive Gaussian noise:
go (x) = 1E [o(x +)]-
Let 9o predict the label ca for input a with some confidence, i.e. [/ + +]] PA 2 pB = maxeşea [e[f + e)], then, as shown by Zhai et al. (2020), ge's prediction is certifiably robust at with certification radius:
R 2 ( (PA) - & (pB)),
(8)
where --1 is the inverse CDF of the standard Gaussian distribution. As a result of Eq. 8, arg max, 9 (x+ arg max, (x), V|[ó|12 < R.
=
While Eq. provides theoretical guarantees for robustness against additive perturbations, DNNs are also brittle against simple input transformations such as rotation. Alfarra et al. (2022a) extended randomized smoothing to certify parametric input deformations through DeformRS, which defined the parametric smooth classifier for given input with pixel coordinates p follows:
90(x, p) = E.~D [z.
(9)
where Iy is an intorpolation function (c.g. bilinear interpolation) and Vo is a paramctric deformation function with parameters e (e.g. is a rotation function and Φ the rotation angle). Analogous to the RS formulation in Eq. 7, 9 outputs the average prediction of fa over deformed versions of the input x. Alfarra al. (2022a) showed that parametrie-domain smooth classifiers are certifiably robust against perturbations to the parameters of the deformation function. In particular, g's prediction is constant with certification radius:
R = a (PA PB)
for D = u[-o,,))
(10)
R ( (pA( - + (px))
for D = o2)
2
Simply put, as long as the perturbatious to the deformation function parameters (e.g. rotation angle) are within R, the prediction of g remains constant. In this work, we leverage RS and DeformRS to study the generalizability of certified robustness to unseen target domains.
Measuring Perceptual Similarity between Source and Target Distributions. The Fréchet Incep- tion Distance (FTD) is a widely used metrie for evaluating generative models (Heusel et al., 2017). Tt provides a measure of similarity between the generated images and a reference dataset based on extracted feature representations. These representations are extracted using pre-trained neural network, typically the In- ception network (Szegedy et al., 2016). FID assumes that the Inception features of an image distribution D follow Gaussian distribution with mean μD and covariance Ep, and it measures the &g Wasscrstoin distance between the two Gaussian distributions. Hence, FID(D,, D2) can be calculated as:
FID(PI, D2) = |||l - -22 + Tr [E + Σ2 2(E,E2)
(11)
7

---

Published in Transactions on Machine Learning Research (05/2023)
Pixel Perturbation
Translation
Affine
1.0
Source ACK = 0.857
Source ACR 0.353
Target 389
ACR 0.171
0.8
Source ACR (VIT) 6969
Source ACR IVIT) 0.454
Certifred Acc.
Target ACR [V 0.415
Target ACR (VIT) 0.223
0.6
0.4
Source ACR 1.175
0.2
1.304
Source ACR (VIT] 1.425
Target ACR (MIT)
1.259
0.0
0.0
0.5
1.0 Radius
1.5
2.0
0.0
0.5
1.0 Radius
1.5
2.0
0.0
0.5
1.0 Radius
1.5
2.0
DCT
Scaling
Rotation
1.0
Source ACX = 207
Target
0.8
Source ACR (VTT) m 0.285
Certified Acc.
Target
Source ACR 0.513
Source ACR 1.883
0.2
Target ACR
Target
Source ACR TV 0.587
Source ACR (ViT) = 2.097
Target ACR (VTT)
Target 1.764
0.0
0.0
0.5
1.0
1.5
2.0
0.0
0.2
0.4
0.6
0.0
1.0 Radius
1.5
Radius
Radius
Figure 2: Generalizability of certified robustness. We certify ResNet-50 and ViT-Base against pixel perturbations and input deformations in the source and target domains of PACS. We observe that 1) certified robustness generalizes to unseon domains, and that 2) a stronger architecture (ViT-Basc) leads to a better source and larget certified accuracy.
where Tr(-) the trace operator. Note that the statistics of both distributions are empirically estimated from their corresponding image samples.
Recently, Alfarra et al. (2022b) demonstrated that quality measures for generative models, such as FID, can be vulnerable to adversarial attacks, leading to unreliable assessments of perceptual similarity between two distributions (Kynkäänniemi et al., 2023). In light of these findings, the authors proposed a robust variant of FID, called R-FID, that addresses this issue by replacing the standard Inception feature extractor with an adversarially trained Inception network, resulting in more reliable metric for evaluating image quality.
Our experiments aim at capturing the interplay between visual perceptual similarity between source and target domains and the generalization of robustness. The notion of perceplual similarity, i.e. the degree to which two image distributions appear similar to humans, has been explored in evaluating generative models (Zhang et al., 2018; Shmelkov et al., 2018). Since the FID has been demonstrated to correlate with human judgment of image quality (Heusel et al., 2017; Lucic et al., 2018), we use it to compare different image domains and assess their perceptual similarity. Unlike the generative models literature that measures FID/R-FID between real and generated distributions of images, we measure the FID source and target distributions which are two real image distributions. Our goal is to whether there is correlation between the distributions' visual similarity and the models' ability to generalize to the unseen target distribution.
FID/R-FID RR-FID between the
Experimental Sctup. To split the data into source and target domains, we use the Photo, Art, Cartoon, and Skelch distributions from PACS (Li et al., 2017). We use RS to certify pixel perturbations and DeformRS to certify five input deformations: rotation, translation, scaling, affine, and a deformation characterized by Discrete Cosine Transform (DCT) basis. Following (Gulrajani
Lopez-Paz, 2021), we employ data augmentation during training and train solely on the source domains. To evaluate the certified robustness of the trained classifier, plot the certified accuracy curves for both the source and target domains for each
8

---

Published in Transactions on Machine Learning Research (05/2023)
Target: Art
Target: Cartoon
Target: Photo
Target: Sketch
0.0
Scaling
0.6
0.4
FIG 18.3
FID 18.5
FID = 34,3
FID - 27.0
0.2
APID 20.2
RFD2 - 40.9
PFID - 67-7
ArD 65.6
ACR 0.12
0.16
-0.045
0.0
0.0
0.2
0.4
0.6
0.0
0.2
0.4
0.6
0.0
0.2
0.4
0.6
0.0
0.2
0.4
0.6
Radius
Radius
Radius
Radius
Translation
0.8
acc
0.6
0.4
0.2
0.0
0.0
0.5
1.0
1.5
2.0
0.0
0.5
1.0
1.5
2.0
0.0
0.5
1.0
1.5
2.0
0.0
0.5
1.0
1.5
2.0
Radius
Radiuus
Radius
Radius
Source
Target
Figure 3: Does visual similarity correlate with robustness generalizability? We vary the target domain and plot the certified accuracy curves for two deformations: scaling and translation. sample from each domain is shown in the second row. The FID/R-FID distances between the source domains and each target are reported in the first row. Visual similarity, measured by FID and R-FID, does not correlate with the level of robustness generalization to the target domain.
considered deformation. The certified accuracy at radius R is the percentage of the test set that is both classified correctly and has a certified radius of at least R. We calculate the certified radius for a given input through either Eq. 8 for pixel perturbations or Eq. 10 for input deformations. Here, we report the envelope plots, which illustrate the best certified accuracy per radius over all values of the smoothing deformation parameter a. We leave the detailed results for each choice of a the appendix. We employ Monte Carlo sampling with 100k samples and probability of failure of 10-3 estimate pa and bound pg = PA: following the standard practice (Zhai et al., 2020; Cohen et al., 2019; Alfarra et al., 2022a). Finally, we follow (Zhai et al., 2020) in reporting the Average Certified Radius (ACR) of correctly classified samples.
Regarding the architecture, we follow DomainBed (Gulrajani & Lopez-Paz, 2021) in selecting ResNet-50 as a backbone. To assess the effect of deploying a more powerful architecture on the generalizability to unseen domains, we also include experiments with the recent transformer model ViT-Base (Dosovitskiy et al., 2021).
5.2 Generalizability of Certified Robustness to Unseen Target Domains
We investigate when certified robustness generalizes to unseen domains. We first show how much Certified Accuracy (CA) is maintained when the target domain exhibits a distribution shift. Then, we study whether a stronger backbone architecture boosts the CA generalizability. Finally, we evaluate whether perceptual similarity, as measured by FID and R-FID (Heusel et al., 2017: Alfarra et al., 2022b), predicts the general- ization of certified robustness. Q4: Can certified robustness, obtained via randomized smoothing, generalize to unseen domains? We train smooth classifiers on a collection of source domains and certify the models both the source and target domains. The target domains are unseen before certification. We plot the source CA curve with dashed black lines and the target CA curve with solid textcolorblack in Figure 2, along with the corresponding ACR. Our results show that certified robustness, acquired by randomized smoothing, is maintained in the unseen domain. When certified against pixel perturbations in the unseen domain, the average certified radius of ResNet-50 drops by around 6% only. Utilizing DeformRS, we extend this result from pixel perturbations to geometric deformations, such as rotation. To explain these findings, we show in Table in the appendix that invariance to different deformations, learned through data augmentation, generalizes to unseen distríbutions. Overall,
considcrable portion of the

---

Published in Transactions on Machine Learning Research (05/2023)
our results illustrate the importance of research efforts that improve on randomized smoothing (Zhai et al., 2020; Eiras et al., 2022). To address real-world securily challenges, we encourage, fulure certified robustness works to conduct experiments on domain generalization datasets.
Q5: Does the target certified accuracy improve when the feature extractor is improved? To investigate the influence of the backbone architecture on the certified robustness of a deployed model, we change the architecture from ResNet-50 to ViT-Base and plot the target CA curve in solid textcolorblack in Figure 2. We observe that the target ACR obtained by ViT-Base PACS is higher than the target ACR obtained by ResNet-50 across deformations.
A significant improvement of the target certified robustness is achieved by using a stronger backbone. This result is consistent with the robustness literature (Gowal et al., 2020), where stronger backbones exhibit better robustness, and the domain gener- alization literature (Gulrajani & Lopez-Paz, 2021), where stronger backbones exhibit better generalization accuracy. We believe that research on better generulization can lead to better certified robustness in unseen domains.
Q6: Does the generalizability of certified robustness correlate with the perceptual similarity between the source and target domains? In all previous experiments, considered the average certified accuracy over all possible target domains. We now conduct a more fine-grained study to these target domains individually. We measure the drop in the average certified radius (AACR) R) between the source and target domains with the perceptual similarity between both domains captured by FID (Heusel et al., 2017) and the more robust R-FID (Alfarra et al., 2022b). To that end, we conduct experiments on PACS where we select one domain as the unseen target and treat the rest as source domains. We train a classifier on the source data and plot the certified accuracy curves against scaling and translation deformations on both the source and target domains in Figure 3 accompanied by AACR. We also report the FID and R-FID between the source and target domains. Note that higher FID/R-FID RR-FID indicates less similarity of distributions. ② Perceptual similarity, as captured by FID and R-FID, is not predictive of performance and robustness generalizability. Surprisingly, the photo domain, which has the highest FID (34.3) and R-FID (87.7) scores, exhibits the largest certified accuracy generalization. In this case, the ACR for the target domain is higher than the source domain resulting in a negative AACR (-0.1 when certifying against translation). The appendix includes experiments with other deformations where we observe similar behavior, along with potential reasons why the FID FID/R-FII RIDFID are poor predictors of robustness generalization. We regard the development of suitable distribution similarity metrie, which better correlates with the level of generulizability, as an important research direction.
Table 3: Evaluation on the Medical Dataset CAMELYONI7 shows robust generalization.
Baseline
PGD
TRADES
Target Source
Clean Acc.
97.48 + 0.10
95.31 4 0.16
93.59 1 0.18
Rob Acc. (A.)
0.44 + 0.40
83.33 + 1.80
85.86 + 1.11
Rob Acc. (PGD)
1.10 + 0.26
84.23 + 1.38
85.89 + 0.99
Clean Acc,
93.56 + 2.51
91.41 + 1.78
90.49 + 1.75
Rob Acc. (AA)
0.10 + 0.17
61.04 + 4.53
67.58 + 3.97
Rob Acc. (PGD)
0.88 + 0.75
62.5 + 2.66
66.31 + 7.69
6 Real-world Application: Medical Images
To demonstrate the applicability of the DG setup to real-world settings, we investigate the generalization of robustness in medical diagnostics. Data collected by medical imaging techniques, like computed tomography (CT) and magnelic resonance imaging (MRI), is susceptible to noise. This noise includes intensity variations caused by subject movement (Shaw et al., 2019), respiratory motion (Axel et al., 1986), quantum noise associated with X-rays (Hsich, 1998), and inhomogeneity in the MRI magnetic field (Leemput et al., 1999). Moreover, due to privacy concerns, a model trained in one medical institution should be deployed in another with limited data sharing and retraining (Kaissis et al., 2020; Ziller et al., 2021; Liu et al., 2021). To test
10

---

Published in Transactions on Machine Learning Research (05/2023)
1.0
1.0
A ACR 0.44
0.8
0.8
Certified Acc.
Certified Acc.
0.6
0.6
0.4
0.4
0.2
0.2
ACR 0.064
0.0
0.0
0.5
1.0
1.5
2.0
0.0
0.2
0.4
0.6
Radius
Radius
(a) Pixel Perturbations
(b) Scaling
Figure 4: Certified robustness in the medical domain. The generalization of robustness against pixel variations in medical images is critical. Yet, there is gap in certified robustness when the DNN is deployed in an unscen hospital.
the gencralization of robustness in this practical setup, use the DG dataset WILDS CAMELYON17 (Bándi et al., 2019; Koh et al., 2021) to train models on tissue images from four hospitals and evaluate them images from an unseen hospital. For the first time, in addition to domain generalization, we explore robustness in CAMELYON17. The task in WILDS CAMELYONI7 is to predict whether a tissue image contains a cancerous tumor or not.
Adversarial Augmentation for Better Pixel Variation Generalizability. In Table 3, we test the generalization accuracy of a standard-trained (e 0) and robust (c = 2/255) models by following the setup from Section 4. Unlike the domains studied in Table 2, adversarial training maintains a high clean accuracy in the unseen target hospital. While the clean accuracy the standard model 93.5%, the clean accuracy of the PGD-trained model is 91.4%. This clean accuracy generalization may be attributed to the similarity between pixel perturbations and the underlying domain shift in the medical images. More importantly, the AutoAttack robust accuracy improves from 1% to 83% in the source hospitals, and from 0% to 61% in the unseen hospital. The results are similar for the TRADES-trained model, with even higher robustness generalization. We encourage future works to study different adversarial training methods that go beyond pizel perturbations, and to propose application-specijp augmentations for different distribution shifts.
Certified Robustness. Next, we investigate the generalizability of certified robust to the unseen hos- pital. We follow the setup in Section and measure the certified accuracy the source and target domains. We observe from Figure 4 that some of the certified robustness generalizes to the unseen hospital when eval- uated with pixel perturbations and scaling deformations. We include the results for other deformations in the appendix. We note that the drop in certified accuracy to the unseen hospital (given pixel perturbations) is 4 times what we saw in the PAGS dataset in Section This concerning, as many sources of noise affect medical imaging data, so robust medical diagnostics is important for real-world adoption of AI for Health. We encourage fulure research to develop better methods to close the targel-source gap in certified robustness.
7 Conclusion
We conducted an extensive empirical analysis of the interplay between adversarial robust and domain generalization. We found that empirical and certified robustness generalizes to unseen domains, including in real-world setting. We also showed that visual similarity not predictive of the level of generalizability. Based on our findings, we encourage more research on: (i) methods that improve certified accuracy in unseen domains, and (ii) distribution similarity metrics that correlate with the generalization accuracy.
11

---

Published in Transactions on Machine Learning Research (05/2023)
Acknowledgments
This work supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research (OSR) under Award No. OSR-CRG2021-4644 and the SDAIA-KAUST Center of Excellence in Data Science and Artificial Intelligence (SDATA-KAUST AT).
References
M Alfarra, A Bibi, N Khan, P Torr, and B Ghanem. Deformrs: Certifying input deformations with random- ized smoothing. In Proc. of AAAI Conference on Artificial Intelligence, 2022a.
Motasem Alfarra, Juan C. Pércz, Anna Frühstücck Philip HS Torr, Peter Wonka, and Bernard Ghanem, On the robustness of quality measures for gans. Xiv preprinl arXiv:2201.13019 2022b.
Anish Athalye, Nicholas Carlini, and David A. Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In JCML, pp. 274-283, 2018.
L. Axel, R. M. Summers, H. Y. Kressel, and C. Charles. Respiratory effects in two-dimensionall fourier transform mr imaging. Radiology, 160, 1986. ISSN 00338419. doi:
Gilles Blanchard, Gyemin Lee, and Clayton Scott. Generalizing from several related classification tasks to a new unlabeled sample. In J. Shawe-TTaylor, R. Zemel, P. Bartlett, F. Pereira, and K.Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 24. Curran Associates, Inc., 2011.
Péter Bándi, Oscar Geessink, Quirine Manson, Marcory Van Dijk, Maschenka Balkenhol, Mcyke Hermsen, Babak Ehteshami Bejnordi, Byungjae Lee, Kyunghyun Paeng, Aoxiao Zhong, Quanzheng Li, Farhad Ghazvinian Zanjani, Svitlana Zinger, Keisuke Fukuta, Daisuke Komura, Vlado Ovtcharov, Shenghua Cheng, Shaoqun Zeng, Jeppe Thagaard, Anders Dahl, Huangjing Lin, Hao Chen, Ludwig Jacobsson, Martin Hedlund, Melih Çetin, Eren Halici, Hunter Jackson, Richard Chen, Fabian Both, Jörg Franke, Heidi Kusters-Vandevell Vandevelde, Willem Vreuls, Peter Bult, Bram Van Ginneken, Jeroen Van Der Laak, and Geert Litjens. From detection of individual metastases to classification of lymph node status at the pa- tient level: The camelyon17 7 challenge. IEEE Transactions Medical Imaging, 38, 2019. ISSN 1558254X. doi: 10.1 1w.11199TTI..10 255.775550011287735050
Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, AlSec '17, pp. 3-14, 2017.
Fabio M. Carlucci, Antonio D'Innocente, Silvia Bucci, Barbara Caputo, and Tatiana Tommasi. Domain generalization by solving jigsaw puzzles. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pattern Recognition (CVPR), June 2019.
Junbum Cha, Sanghyuk Chun, Kyungjae Lee, Han-Cheol Cho, Seunghyun Park, Yunsung Tee, and Sungrae Park. Swad: Domain generalization by seeking flat minima. Advances in Neural Information Processing Systems, 34, 2021.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning, pp. 1310-1320. PMLR, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pp. 2206 2216. PMLR, 2020.
Matthias Delange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytclaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pallern Analysis and Machine Intellggnce, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei, Imagenet: A large-scale hierarchical image database. In IEFE Conference on Computer Vision and Pattern Recognation (CVPR), 2009.
12

---

Published in Transactions on Machine Learning Research (05/2023)
Zhun Deng, Linjun Zhang, Kailas Vodrahalli, Kenji Kawaguchi, and James Zou. Adversarial training helps transfer learning via better representations. Tn A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostala Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby, An image is worth 16x16 words: Transformers for image recognition scale. In International Conference Learning Representations, 2021.
Francisco Eiras, Motasem Alfarra, Philip Torr, M. Pawan Kumar, Puncet K. Dokania, Bernard Ghanem, and Adel Bibi. ANCER: Anisotropic certification via sample-wise volume maximization. Transactions on Machine Learning Research, 2022.
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. Exploring the landscape of spatial robustness. In ICML, 2019.
Samuel G. Finlayson, John D. Bowers, Joichi Ito, Jonathan L. Zittrain, Andrew L. Beam, and Isaac S.
Kohane. Adversarial allacks on medical machine learning. Science, 363(3433):1287 1289, 2019.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2019.
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shorteut learning in deep neural networks. Nature Machine Intelli- gence, 2(11):665 673, 2020.
Robert Geirhos, Kantharaju Narayanappa, Benjamin Mitzkus, Tizian Thieringer, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Partial success in closing the gap between human and machine vision. A. Beygekimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021.
Rui Gong, Wen Li, Yulua Chen, and Tucc Van Gool. Dlow: Domain flow for adaptation and generalization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), June 2019.
lan J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. CoRR, abs/ 1412.6572, 2015.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples, 2020.
Shixiang Shane Gu and Luca Rigazio. Towards deep neural network architectures robust to adversarial examples. CoRR, abs/1412.500 2015.
Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. In International Conference on Learning Representations, 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing Systems, 2017.
Jiang Hsieh. Adaptive streak artifact reduction in computed tomography resulting from excessive x-ray photon noise. Medical Physics, 25, 1998. ISSN 00942405. doi: 10.1118/1.59841
Georgios A. Kaissis, Marcus R. Makowski, Daniel Rückert, and Rickmer F. Braren. Secure, privacy- preserving and federated machine learning in medical imaging. Nature Machine Intelligence, 2, 2020. ISSN 25225839. doi: 10.0038//42556-0000-0066
13

---

Published in Transactions on Machine Learning Research (05/2023)
Pang Wei Koh, Shiori Sagawa, Henrik Markhun, Sang Michael Xie, Marvin Zhang, Akshay Balsubra- mani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Trena Gao, Tony Tee, Etienne David, lan Stavness, Wei Guo, Berton Earnshaw, Imran Haque, Sara M Beery, Jure Leskovee, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelseaa Finn, and Percy Liang. Wilds: A benchmark of tn-the-wild distri- bution shifts. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 5637-5664. PMLR, 18 24 Jul 2021.
Tuomas Kynkäänniemi, Tero Karras, Mika Aittala, Timo Aila, and Jaakko Lehtinen. The role of imagenet classes in fréchet inception distance. The Eleventh International Conference on Leurning Representa- tions, 2023.
Mathias Lécuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel J. Hsu, and Suman Sekhar Jana. Certified robustness to adversarial examples with differential privacy. 2019 IEEF Symposium on Security and Privacy (SP), pp. 656-672, 2019.
Sungyoon Lee, Woojin Lee, Jinseong Park, and Jaewook Lee. Towards better understanding of training certifiably robust models against adversarial examples. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Newral Information Processing Systems, 2021.
Koen Van Leemput, Frederik Maes, Dirk Vandermeulen, and Paul Suetens. Automated model-based tissue classification of mr images of the brain. IEEE Trensactions Medical Imaging, 18, 1999. ISSN 02780062.
doi: 10.1109/42.81127
Da Li, Yongxin Yang, Yi Zhe Song, and Timothy M. Hospedales. Deeper, broader and artier domain generalization. volume 2017-October, 2017. doi: 10.1109/ICCV 20017.991.
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Learning to generalize: Meta-lcarning for domain generalization. In Phirty-Secona AAAI Conference on Artificial Intelligence, 2018.
Quande Liu, Cheng Chen, Jing Qin, Qi Dou, and Pheng-Ann Heng. Feddg: Federated domain generalization on medical image segmentation via episodic learning in continuous frequency space. In Proceedings of the IEEE/CVP Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1013-1023, June 2021.
Wang Lu, Jindong Wang, Haoliang Lí, Yiqiang Chen, and Xing Xie. Domain-invariant feature exploration for domain generalization. Transactions Machine Learning Research, 2022.
Mario Lucie, Karol Kurach, Marcin Michalski, Sylvain Gelly, and Olivier Bousquet. Are gans created equal? a large-scale study. Advances in neural information processing systems, 31, 2018.
Xingjun Ma, Yuhao Niu, Lin Gu, Yisen Wang, Yitian Zhao, James Bailey, and Feng Lu. Understanding adversarial attacks on deep learning based medical image analysis systems. Pattern Recognition, 110: 107332, 2021.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
Ject Mohapatra, Tsui- Wei Weng, Pin-Yu Chen, Sijia Liu, and Luca Danicl. Towards verifying robustness of neural networks against a family of semantic perturbations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 244 252, 2020.
Saeid Motiian, Marco Piccirilli, Donald A. Adjeroh, and Gianfranco Doretto. Unified deep supervised domain adaptation and generalization. In Proceedings of the IEFE International Conference Computer Vision (ICCV), Oct 2017.
A. Tuan Nguyen, Toan Tran, Yarin Gal, and Atilim Gunes Baydin. Domain invariant representation Icarning with domain density transformations. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vanghan (eds.), Advances in Neural Information Processing Systems, 2021.
14

---

Published in Transactions on Machine Learning Research (05/2023)
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In 2016 IEEE European Symposium on Securily and Privacy (BuroS8P), pp.
2016.
Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Certified defenses against adversarial examples. In International Conference on Leurning Represenlations, 2018.
Gabriel Pérez S., Juan C. Pérez, Motasem Alfarra, Silvio Giancola, and Bernard Ghanem. 3deformrs: Certifying spatial deformations point clouds. In Proceedings of the JEEE/CVE Conference Computer Vision and Pattern Recognation (CVPR), pp. 15169-15179, June 2022.
Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adversarially robust imagenet models transfer better? In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 3533 3545. Curran Associates, Inc., 2020.
Ali Shafahí, Parsa Saadatpanah, Chen Zhu, Amin Ghiasi, Christoph Studer, David Jacobs, and Tom Gold- stein. Adversarially robust transfer learning. In International Conference Learning Represenlations, 2020.
Richard Shaw, Carole Sudre, Sebastien Ourselin, and M. Jorge Cardoso. Mri k-space motion artefact augmentation: Model robustness and task-specific uncertainty. In M. Jorge Cardoso, Aasa Feragen, Ben Glocker, Ender Konukoglu, Ipek Oguz, Gozde Unal, and Tom Vercauteren (eds.), Proceedings of The 2nd International Conference on Medical Imaging with Deep Learning, volume 102 of Proceedings of Machine Learning Research, pp. 427 436. PMLR, 08 10 Jul 2019.
Zheyan Shen, Jiashuo Liu, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. Towards out-of- distribution generalization: survey, 2021.
Konstantin Shmelkov, Cordelia Schmid, and Karteek Alahari. How good my gan? In Proceedings of the European conference on computer vision (ECCV), pp. 213 229, 2018.
Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation, 2016.
Christian Szegedy, Wojciech Zaremba, Πya Sutskever, Joan Bruna, D. Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing propertics of neural networks. CoRR, abs/1312 1312.6199, 2014.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2016.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. In International Conference Learning Representations, 2019.
Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maxi- mizing for domain invariance, 2014.
Francisco Utrera, Fvan Kravitz, N. Benjamin Frichson, Rajiv Khanna, and Michael W. Mahoney. Adversarially-trained deep nets transfer better: Illustration on image classification. In International Con- ference Learning Representations, 2021.
Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, and Tao Qin. Generalizing to unseen domains: A survey on domain generalization. In Zhi-Hua Zhou (ed.), Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pp. 4627-4635. International Joint Conferences on Artificial Intelligence Organization, 2021. Survey Track.
Mei Wang and Weihong Deng. Deep visual domain adaptation: A survey. Neurocomputing, 312:135-133 1533, 2018.
15

---

Published in Transactions on Machine Learning Research (05/2023)
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjum Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International Conference on Learning Represen- tations, 2020.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (ods.), Advances in Neural Information Processing Systems, volune 33, pp. 2958 2969. Curran Associales, Inc., 2020.
Runtian Zhai, Chen Dan, Di He, Huan Zhang, Boqing Gong, Pradcep Ravikumar, Cho-Jui Hsich, and Liwei Wang. Macer: Attack-free and scalable robust training via maximizing certified radius. In International Conference Learning Representations (ICLR), 2020.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. The- orctically principled tradc-off between robustness and accuracy. volume 97, pp. 7472-7482. PMLR, 2 2019.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pottern recognition, pp. 580-595, 2018.
Xingxuan Zhang, Peng Cui, Renzhe Xu, Linjum Zhou, Yue He, and Zheyan Shen. Deep stable learning for out-of-distribution generalization. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5372-5382, 5382, June 2021.
Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang, Learning to generate novel domains for domain generalization. In European conference on computer vision, pp. 561-578. Springer, 2020.
Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. Domain generalization with mixstyle. In Interna- lional Conference Learning Representations, 2021.
Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. IEFE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-20, 2022.
Fushen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He. A comprehensive survey on transfer learning. Proceedings of the IEEE, 109(1):43 76, 2020.
Alexander Ziller, Dmitrii Usynin, Rickmer Braren, Marcus Makowski, Daniel Rueckert, and Georgios Kaissis. Medical imaging deep learning with differential privacy. Scientific Reports, 11, 2021. ISSN 20452322. doi: 10.
16

---

Published in Transactions on Machine Learning Research (05/2023)
Appendix
A Empirical Robustness and Domain Generalization
A.1 Testing Different Budget Norms
Table 4: Evaluation of Lo Robustness. We train models in the source domain and then evaluate their clean accuracy and robust accuracy in the source and target domains. The models are trained using either PGD adversarial training, or TRADES adversarial training. The robust accuracy is measured against AutoAttack and PGD adversarial attacks. For adversarial training and evaluation, the perturbation norm is set to é = 4/255. Overall, the robustness of PGD-trained and TRADES-Irained models transfers to the target distribution.
Source
Target
Robust Acc.
Robust Acc.
Robust Acc.
Robust Acc.
Method
Dataset
Clean Ace.
Clean Acc.
(AA)
(PGD)
(AA)
(PGD)
PACS
90.68 0.27
66.43 0.85
67.95 + 0.76
70.97 0.58
45.48 + 1.85
45.51
OfficHomme
68.23 +: 0.12
41.75 0.19
43.13 + 0.65
47.77
0.63
25.22 + 1.15
26.5 = 1.19
PGD
VLCS
77.07 +: 0.18
48.58 0.52
50.14 + 1.15
66.54 + 0.85
36.69 #: 2.20
36.22 = 2.52
Terralncognita
66.29 2.91
45.89 6.38
51.84 + 1.26
26.15
3:26
3.56 + 1.44
5.83 2.77
PACS
66.71 = 0.36
53.10 + 0.88
53.32 + 1.01
50.96 + 1.27
35.23 = 1,28
35.65 1.80
OfficeHome
56.94 8.76
39.01 + 5.87
40.93 + 6.97
40.54 +: 4.21
26.44 - 0.92
26.33 3.33
TRADES
VLCS
75.69 0.30
51.31 + 0.40
55.52 + 0.89
65.20 + 1.27
42.48 - 0.98
43.07 1.30
TerraInc
64.20 0.08
52.07 0.81
53.02 + 0.77
27.97 + 1.22
10.45 = 3.97
12.04 2.77
Table 5: Evaluation of fo Robustness. We train models in the source domain and then evaluate their clean accuracy and robust accuracy in the source and target domains. The models are trained using either PGD adversarial training, or TRADES adversarial training. The robust accuracy is measured against AutoAttack and PGD adversarial attacks. For adversarial training and evaluation, the perturbation norm is set to 8/255. Overall, the robustness of PGD-trained and TRADES-trained models transfers to the target distribution.
Source
Target
Method
Dataset
Clean Acc.
Acc. (AA)
Acc. (PGD)
Clean Ace.
Acc. (AA)
Acc. (PGD)
PACS
86.36
48.24
52.83
64.06
29.00
32.92
PGD
TerraIncognita
78.36
6.35
34.67
34.64
0.39
8.98
PACS
85.
58.43
59.85
60.45
32.81
36.60
TRADES
Terralncognita
60.78
44.30
44.01
28.29
15.43
16.31
Q1: Do adversarially robust models generalize better than their standard-trained counter- parts? Again, the answer is no. Adversarially trained models tend to experience a drop in generalizability when compared to their standard-trained counterparts.
Q2: Does a higher source-domain robustness correspond to a higher target-domain robustness? As expected, the answer is still yes. As observed in Table 4, when we have a higher robustness in the source domain, we consistently observe a higher robustness the target domain even for higher budget norm
Q3: Does the robustness-accuracy tradc-off generalize unscen domains? Yes, similar what was observed in e = 3/255 experiments, the robustness-accuracy trade-off exists in unseen domains even for higher budget norms. Robustness in the target domain comes at the cost of clean accuracy.
17

---

Published in Transactions on Machine Learning Research (05/2023)
A.2 Empirical Robustness Ablations
In this section, conduct various ablations to assess the impact of different hyperparameters in adversarial training on our observations.
First, we investigate the effect of the number of adversarial training steps. Tables 6 and illustrate the results of decreasing the number of PGD and TRADES steps to 1, as well as increasing them to for PACS and Terraincognita datasets respectively. As expected, increasing the number of steps leads to higher robust accuracy, at the expense of clean accuracy.
Table 6: Effect of Number of Adversarial Training Steps (PACS). Increasing the number of adver- sarial traiming steps improves the model's ability handle adversarial examples in both source and target domains, i.e. incrcases model robustness. However, this improvement comes at the cost of reduced accuracy clean examples in both domains.
Source
Target
Method
Steps
Clean Acc.
Acc. (AA)
Acc. (PGD)
Clean Acc.
Acc. (AA)
Acc. (PGD)
1
94.55
59.90
84.20
78.51
42.29
63.12
PGD
92.30
78.48
87.06
73.52
55.47
66.13
1
93.95
66.99
84.72
77.38
47.17
65.02
TRADES
10
01.29
79.92
86.93
73.24
60.45
68.38
Table 7: Effect of Number of Adversarial Training Steps (Terralncognita). Increasing the number of adversarial training steps improves the model's ability to handle adversarial examples in both source and target domains, í.e. increases model robustness. However, this improvement comes at the cost of reducod accuracy on clean examples in both domains.
Source
Target
Method
Steps
Clean Ace.
Acc. (AA)
Acc. (PGD)
Clean Acc.
Acc. (AA)
Acc. (PGD)
1
84.40
8.53
59.11
36.30
0.49
6.25
PGD
10
69.66
58.33
65.59
22.78
5.96
14.75
1
84.34
17.15
66.6
33.96
1.17
8.98
TRADES
10
68.81
58.63
64.42
26.08
12.79
18.07
Second, we examine the effect of changing the value of / on TRADES. By definition, increasing the value of places a greater emphasis on robustness. Therefore, it is expected that increasing 3 to 6.0 would result in an increase in robust accuracy, but a docrease clean accuracy. In contrast, a decrease B to 1.0 is expected to have the opposite effect, with a decrease in robust accuracy and an increase clean accuracy. The results are summarized in Table 8.
Table 8: Effect of Changing 3 on TRADES. By definition, increasing the value of 3 means putting higher weight on robustness. Therefore, as expected when we B increase to 6.0 the robust accuracy increases, whereas the clean accuracy drops and vice versa for B = 1.0.
Source
Target
Dataset
Clean Acc.
Acc. (AA)
Acc. (PGD)
Clean Acc.
Acc. (AA)
Acc. (PGD)
1.0
92.33
78.91
77.90
75.73
57.23
59.12
PACS
6.0
90.02
80.50
79.65
71.09
58.11
60.75
1.0
78.21
51.82
53.58
27.70
3.42
3.12
Terralncoguita
6.0
65.55
59.02
57.29
26.11
13.38
15.14
18

---

Published in Transactions on Machine Learning Research (05/2023)
B Certified Robustness and Domain Generalization
B.1 How is it possible for randomized smoothing to to certify models in unseen domains?
To understand why randomized smoothing is able to cortify models in unseen domains, we invostigate the hypothesis that the learned invariance lo noise and geomel ieformalions generalizes to unseen domains. To that end, we train models in the source domain and then evaluate their accuracy on deformed samples from the source and target domains. During evaluation, the images are transformed with some c-parameterized deformation. Models that never the corresponding deformation before evaluation time are denoled by 'Without' in Table 9. Models trained deformed images denoted by 'With'. All the experiments are repeated four times with different seeds, and the average performance is reported. We observe that augmenting the source dataset equips the models with invariance against various deformations, and that this deformation invariance generalizes to the unseen target distribution. This invariance generalization is consistent with Figure 2, which illustrates the Monaann-ennealliabiiee of the certified robustness obtained through randomized smoothing.
Table 9: Training with augmentations leads to domaan-generrllizablee invariance to various defor- mations. During evaluation, the images are transformed with some e-parameterized deformation. Models denoted by 'With' are trained on augmented images transformed by the same dcformation, and models denoted by 'Without" do not deformed images at training time. We note that the learned invariance against various deformations generalizes to the unseen target distribution.
Without
With
Deformation
Source
Target
Source
Target
0.1
87.98
75.13
93.36
79.60
0.25
60.30
52.74
91.37
75.05
Gaussian Noise
0.37
38.99
36.90
89.06
72.49
0.5
23.59
21.17
86,38
68.70
0.1
92.32
79.48
93,86
79.59
0.3
75.07
61.70
87.85
68.44
Affine
0.5
50.53
40.91
74.65
53.51
0.7
35.99
28.37
63.51
42.14
0.1
81.06
66.53
91.48
72.52
0.3
36.89
28.78
79.97
53.47
DCT
0.5
23.69
19.42
70.02
41.51
0.7
19.40
16.94
61.74
37.10
0.1
92.71
77.96
94.03
79.02
0.3
84.06
70.58
92.14
74.92
Rotation
0.5
75.25
61.27
90.36
70.19
0.7
68.98
57.76
89.32
62.10
0.1
93.96
79.68
94.25
80.63
0.3
92.77
78.80
94.01
77.97
Scaling
0.5
90.26
76.52
92.91
78.34
0.7
85.74
72.27
90.75
73.95
0.1
93.91
81.06
94.53
81.15
0.3
91.98
78.32
93.37
78.76
Translation
0.5
86.72
73.18
90.09
73.70
0.7
77.87
65.39
84.94
66.65
19

---

Published in Transactions on Machine Learning Research (05/2023)
B.2 The effect of Φ on the generalization of certified robustness
To complement Figure 2, investigale the behavior when the deformation parameter e varies. Following Section 5.2, we certify ResNet-50 and ViT-Base against pixel perturbations and imput deformations in the souree and target domains of PACS. We break down each envelope curve in Figure 2 into multiple curves, each representing one choice of ó in Eq. 9. We label each curve with the corresponding & value in Figure 5. We observe that the effect of á largely depends on the type of perturbation. On the one hand, for Scaling and Pixel Perturbations, higher & values corresponds to larger Average Certified Radius (ACR); on the other hand, for Affine, DCT, and Translation, a higher values might correspond to a smaller ACR. This is because for the latter group of deformations, higher & results in completely deformed image, which hinders the certification ability of the model, even at small radius. We visualize images from these deformations in Section D. Note that the trends in Figure 2 still stand. Specifically, (1) for e values where there's a reasonable certified accuracy in the source, that certified accuracy generalizes to the target. Moreover, (2) A stronger architecture (ViT-Base) generally leads to a better source and target certified accuracy.
B.3 Does visual similarity correlate with robustness generalizability?
We repeat the experiments in Section 5.2, which aim to evaluate the ability of FID/R-FID to predict the generalization of robustness, on pixel perturbations and the following deformations: rotation, affine, and DCT. We observe from Figure 6 that the FID/R- FR-FID values do not predict the level of generalizability of certified robustness, which matches our paper findings.
Why does visual similarity not correlate with the level of robustness generalizability" The FID and R-FID metries rely on two fundamental assumptions. First, they assume that the deep features extracted from ImageNet pre-trained networks follow a multivariate Gaussian distribution (Heusel et al., 2017). Second, they assume that the extracted features correlate with human perceptual judgment (Zhang et al., 2018). Visual similarity, measured by these metrics, and robustness generalizability are not always correlated in our experiments. While surprising, this discrepancy has several possible reasons. Firstly, the approximation of Inception features as multivariate Gaussians is oversimplified assumption that may not capture the complexities of each image distribution, as noted in previous work (Shmelkov et al., 2018). Secondly, The use of an ImageNet pretrained Inception module to extract features for each image distribution may not be representative of the considered image distributions in the domain generalization literature. Moreover, the use of ImageNet, pre-trained Inception might introduce its own biases, as observed in (Kynkäänniemi et al., 2023). Lastly, deep neural networks tend to be biased towards texture, while humans are biased towards shape (Geirhos et al., 2019), which suggests that perceived visual similarity may not always align with DNN performance on classification tasks, as we also find in our experiments.
While our work shods some light the interplay between visual similarity and robustness generalization, more research should be done to disentangle the correlation between different components of the distribution similarity metric and the generalization. For instance, it is unclear how different choices for the feature ex- tractor or distance metric used to compute the FID or R-FID may impact the correlation with generalization performance. Moreover, other metries that better capture the image distribution's 's subtleties, possibly based on both shape and texture, may provide more consistent correlation with generalization.
20

---

Published in Transactions on Machine Learning Research (05/2023)
C weal-world Application: Medical Images
We repeat the certified robustness experiments in Section 6 on the following deformations: affine, DCT, translation, and rotation. We observe from Figure 8 that the source-target certification gap is similar for affine, DCT, and translation. However, the certified accuracy curves for rotation are different. This makes sense when we consider the way the Camelyon17 dataset constructed (Bándi et al., 2019; Koh et al., 2021). The dataset includes cropped histopathological images, each of which may contain a tumor tissue in the central 32x32 region. Due the nature of this construction, rotated versions of the image look similar, which explains why the source and target certified radii remain almost constant. Samples from the Camelyon17 dataset are visualized in Figure 7.
D Visualizing the domain generalization datasets
In Figure 12, we visualize few samples from each of the domain generalization datasets used in the paper. We note that the datasets are diverse in terms of the nature of domain shifts and real-world applicability. Along with the clean samples, we visualize deformed versions of the samples under various values of a for the studied deformations.
21

---

Published in Transactions on Machine Learning Research (05/2023)
(a) (Pizel) o= = 0.1
(b) Φ = 0.25
(c) Ø 0.37
(d) 0 0.5
1.0
1.0
1.0
1.0
0.8
0.5
Cortileed Acc.
Gortilee Acc.
Certified MEC.
0.6
0.6
Cartifee
0.6
0.6
0.4
0.4
Soarca AGR 6.313
Sourca ACA 0,666
Source AGI 8:53
Sourte ACA 1.14
0.2
Source ADX 04 - 0.55
Torget AC3, 0.37
022
Source 6C1 2611) 0.:22
Trrge AC3, 0 676
0.2
Source ADR (1
be ACI 0.457
0.2
Source PCR (M11 - 2
TargetAC- - 7.043
Target 0610)
Target
Terget 1N87 2.23
0.0
0.8
0.0
0.0
1.5
1.5
2.0
Radius
Radius
Radius
Radius
(e)
(Scaling) Φ = 0.1
(f) Φ = 0.3
+ = 0.5
(h) Φ = 0.7
1.0
2.0
2.0
1.0
0.8
Cortifedd Acc
Cartified AcC.
Certifeed Acc.
0.6
0.6
ffrtifed
0.6
0.6
0.4
0.4
0.4
0.4
Sarree AEl (V - 6.044
0.575
02
Dager
Stuine delk 2411 0.278
0.2
Soumee aCk (417) 0.44%
0.273
0.2
Seurce ACk (MT) - 3.567
3.243
Targl 4CI (61) 0.555
Targll ACI, (4I)) - 9:5
TargstAck (NT) - 0.353
Target (NI)) 2.324
0.0
0,000
0.025
0,050 0.075 0.100
0.0
0.0
0.1
0.2
0.3
0.0
0.0
0.1
0.2
0.3
0.4
0.5
0.0
0.0
0.4
0.6
Radius
Radius
Radius
Radius
(i) (Scaling) o= = 0.1
(i) Φ = 0.3
(k) Φ = 0.5
(I) Ø = 0.7
1.0
1.0
0.8
0.8
0.8
0.8
Cortffed ACC.
Cortifieed Art.
0.6
Cartified
0.6
0.6
0.4
0.4
0.4
Sourch AGR 6.37
Targut 0:389
0.2
Searco ACB
Scarco-AGh ACh
Tarpet ACA-- 0.5
0:312
0.53
TergstaCs
1.264
Sourte ACA. 1.732
Tergst.ACH 3.343
0.0
0.0
Targst 4C% (611) 259
0.1
Radius
0,2
0.3
0.0
0.0
0.2
0.4
Radius
0.6
0.8
1.0
0.0
0.0
0.5
Redius
1.0
1.5
0.0
0.0
Terpet ACN (VT))
0.5
Radius
1.0
1.5
2.0
(m)
(Afffine s= 0.1
0.3
3
0.5
(p)
0.7
1.0
Seuro ACK 0.328
Source ACR 3191
Source Torpet ACB 6.300
0.8
0.8
Bapet
0.8
Target ACR 1.044
Source 244
0.8
Soorce AGR 8:33
Cortrfied Acc.
Aic (17)
5.032
0.6
Curtifee
0.8
Certified
0.6
Cortifiied
0.6
0.4
0.4
0.4
Saarce AOR = 0.30
0.2
Twope 6C4
ACI 11:11
0.2
0.2
0.2
Radius
0.3
0.00
0.25
0.50
Radius
0.75
1.00
1.25
0.0
0.0
0.5
Radius
1.0
1.5
2.0
0.0
Radius
(q) (DCT) 6- = 0.1
(r) Φ 0.3
(s) Φ
0.5
(t)
Φ
0.7
1.0
1.0
1.0
1.0
Sourts ack (Iff- 0227
ACA 3.09
0.8
Trrge
de'4 cXT) 2020
0.8
Source ACR (TT) 3172
XER
Seennce XCk (IT) 6.07
XIR 6.2
Cortiled Acc.
Terge ACK (NT) 032:
Taget ACI (W7) - 6.02
Certiffed Aec
Sapad AOI 017) 0.224
0.6
Cortified
0.6
0.6
0.4
0.4
0.2
0.2
0.2
0.2
Radius
0.0
0.25
Radius
0.75
1.00
1.25
0.0
0.0
0.5
Redius
1.0
1.5
2.0
Radius
(u)
(Translation) á 0=00 0.1
(v)
0.3
(w) Ø = 0.5
(x)
Ø
0.7
10
1.0
1.0
Source 150
Source
Target 0.323
0.8
Source (CC i 6.572
0.8
0.8
Cortried Acc.
Target 3-42
Cortified Acc.
ACN (TT) 6.237
Certified Acc.
lirect AIH (Vi1 0.125
0.6
Carcifieel
0.6
0.6
0.6
0.4
0.4
0.4
0.4
AQI 19
0.2
0.0
0.0
Tarall 4C1 (617) 00
0.1
0.2
0.3
0.4
0.0
0.00
0.25
0.50
0.75 Radius
1.00
1.25
0.0
0.0
0.5
1.0 Radius
1.5
2.0
0.0
Radius
Radius
Figure 5: Effect of Varying the Deformation Parameter Φ. We observe that 1) for Pixel Perturbations, Scaling, and Rotation, the higher & gets, the larger the Average Certified Radius (ACR) becomes; and (2) for Affine, DCT, and Translation, high ø values can Bluul in low ACRs.

---

Published in Transactions on Machine Learning Research (05/2023)
(a)
(Affine) Target: Art
(b) Target: Cartoon
(c)
Target: Photo
(d) Target: Sketch
1.0
1.0
1.0
1.0
Contired Acc
Carlifed Acc.
Ceriifeed Acc.
0.6
0.6
Cortifed
0.6
0.6
F10 18.3
FID 18.5
FID
Fi0 27.0
0.
AFID - 29.2
02
RPD - 40.5
02
RFD - B7.7
0.2
REI 65.6
ACR
AAC 0.35
A
0.0
0.5
1.0
1.5
0.0
0.5
1.0
2.0
0.0
0.0
0.5
1.0
1.5
2.0
0.0
0.5
1.0
1.5
2.0
Radius
Radius
Radius
(e)
(A ) Target: Art.
(f) Target: Cartoon
(g)
Target: Photo
(h) Target: Sketch
1.0
1.0
1.0
1.0
AACK
0.066
AACR 0.38
0.8
0.8
0.0
Coriited Acc.
Cartifed Acc.
Certified Acc.
Certifeed
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.0
0.0
0.5
1.0
1.5
2.0
0.0
0.0
0.5
3.0
1.5
2.0
0.0
0.0
0.5
1.0
1.5
2.0
0.0
0.0
0.5
1.0
1.5
2.0
Radius
Radius
Radius
Radius
(i)
(DCT) Target: Art
(i)
Target: Cartoon
(k) Target: Photo
(1) Target: Sketch
0.075
AAC
0.8
0.8
0.8
0.8
Coniieed Acc
Contified Acc.
Contified Acc.
0.6
Coniifed
0.4
0.4
0.4
0.4
02
1.5
2.0
0.0
3.0
2.0
0.0
0.0
1.0
1.5
0.0
Radius
Radius
Radius
Radius
(m)
(Pizel) Targetz Art.
(n)
Target: Cartoon
(o) Target: Photo
(p) Target: Sketch
1.0
2.0
1.0
1.0
ACR 0.62
AAC 0.38
à ACR - 0.32
A ACR - 0.071
0.8
Cortited BCC.
Cartifed Acc.
Certifeed ACC.
0.6
0.6
Cartifee
0.6
0.6
0.4
0.4
0.2
02
0.2
0.2
Radius
Radius
Figure 6: Does visual similarity correlates with robustness generalizability? We vary the target distribution and plot the certified accuracy curves for different deformations. The FID/R-FID distances between the source and target distributions are shown in the first row. Visual similarity (FID and R-FID) does not correlate with the level of robustness generalization to the target domain.
Clean Samples
(a) Hospital
(b) Hospital
(c) Hospital
(d) Hospital
(e) Hospital
Figure 7: A visualization of the images taken from the 5 hospitals in Camelyon17.
23

---

Published in Transactions on Machine Learning Research (05/2023)
(a) (Affine)
(b) (DCT)
(c) (Translation)
(d) (Rotation)
1.0
ACR 0.29
À ACR 0.28
A ACR 0.26
0.8
0.8
0.8
0.8
Coriited Acc.
Cartifed Acc.
Certified Acc.
Certifeed
0.6
0.6
0.4
0.4
0.4
0.4
4 ACR 0.04
0.0
0.0
0.0
2
0.0
0.0
0.5
1.0
15
2.0
Radius
Radius
Radius
Radius
Figure 8: Does visual similarity correlates with robustness generalizability? We vary the target distribution and plot the certified accuracy curves for two deformations: scaling and translation. A sample from each distribution is shown in the second row. The FID/R-FID distances between the source distributions and each target are inset in the first row. Visual similarity, measured by FID and R-FID, does not correlate with the level of robustness generalization to the target domain.
24

---

Published in Transactions on Machine Learning Research (05/2023)
Clean Samples
Affine, 0.1, 0.3, 0.5
Rotation, 0 = 0.1, 0.3, 0.5
25

---

Published in Transactions on Machine Learning Research (05/2023)
Translation, 0.3, 0.5
DCT, 0 0.1,0.3,0.3 0.5
26

---

Published in Transactions on Machine Learning Research (05/2023)
Pixel Perturbation, o 0.3, 0.5
Scaling, 0.1,0.3, 0.5
27

---

Published in Transactions on Machine Learning Research (05/2023)
Art
Cartoon
Photo
Sketch
PACS
Art
Clipart
Product
Real- World
OfficeHome
Caltech
LabelMe
SUN09
VOC2007
VLCS
Location 38
Location 43
Location 46
Location 100
TerraIncognita
Hospital 0
Hospital 1
Hospital 2
Hospital
Hospital 4
Camelyon17
Figure 12: Domains of Studied Datasets. In our work considered five datasets each consisting of multiple domains.
28


Reviews:
[{"review_id": "0I4GXZpsTP", "reviewer": "Reviewer_1R7v", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The research problem, adversarial robustness under distribution shift, is very practical and important for deploying many real-world applications.\n2. The work contains lots of empirical studies and findings, which would be interesting for the community.\n3. The paper is well-written and easy to understand, the illustrations of the problem setting is clear, and the empirical analysis is also well presented. \n\nWeaknesses:\n1. Why use FID and R-FID score as the evaluation of the visual similarity? It would be better to explain the reasons, and also briefly present what is FID and R-FID in the paper. \n2. Why visual similarity captured by FID and R-FID does not correlate with the level of robustness generalization \nto the target domain? Any possible interpretations about this? Would this point still be true if using other scores to measure the similarity? More discussions about this point would be interesting.\n3. There is a clear watermark on the elephant image in Figure 3 (i.e., in the 2nd row and 2nd column). Is there also a copyright issue about using this image? Are the shown images the real images used in the target domain? ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this paper, the authors studied the generalization ability of adversarial robustness from multiple source domains to one unseen target domain under the distribution shift. Specifically, the work shows the generalization ability of both adversarial training (called empirical robustness in the paper) and certified robustness from the source to the target domain without any target domain information. The work also includes lots of empirical evaluations and interesting empirical findings. ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Briefly present more information about the FID and R-FID scores and the reasons why use them as the evaluation of the visual similarity.\n2. Add more discussions about the point 2 mentioned in the Weaknesses.\n3. Check if the use of the illustration images in Figure 3 is proper or not, and remove the watermark if it is permitted to be used in this paper.  ", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "P2rMkeMb9A", "reviewer": "Reviewer_fkDt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n1) This paper systematically investigates the robustness generalization under distribution shift scenarios. The authors discuss both empirical and certified robustness, which is comprehensive.\n2) The writing of the paper is friendly to readers. The authors' description of the background and problem setting is clear, including the statement of the difference between domain generalization and transfer learning.\n3) The experimental design and results of the paper validate most of the arguments. The experiments in real-world application is interesting.\n\nWeaknesses:\n1) The main argument of the paper, \"robustness can generalize to unseen domains,\" is not surprising to me because adding adversarial perturbations is already a form of a specific distribution shift. It is easy to imagine that a model that is robust to adversarial perturbations can partially handle other types of distribution shifts (domain generalization).\n2) I am confused by some of the conclusions. For example, in Q1 of section 4.2, how does the author define \"generalize better\"? And why does the author say that the TRADES model generalizes worse than the PGD model when it looks like they both have similar performance loss in the target domain? In Q3, how is \"TRADES model tends to be the least generalizable\" defined?\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper investigates the generalizability of empirical and certified adversarial robustness to unseen domains in the context of domain generalization. The authors observe that both types of robustness generalize to unseen domains, and that the level of generalizability does not correlate well with input visual similarity. They also extend their study to cover a real-world medical application, in which adversarial augmentation significantly boosts the generalization of robustness with minimal effect on clean data accuracy.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1) It may be necessary to provide clearer descriptions of the evaluation criteria for empirical robustness generalization. Some of the conclusions mentioned above require more reasonable explanations.\n2) It would be helpful to have a clearer description of some of the experimental setups, such as the number of attack steps and perturbation radius used by AA and PGD in the empirical robustness experiments.\n3) The paper presents validation experiments for perturbation radii of epsilon=2/255 and 4/255, but I am curious about how the robustness generalization would perform under larger perturbation radii.\n", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vmufUHq5gT", "reviewer": "Reviewer_RYco", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n\n+ This paper is well-written and well-organized. The motivation is clear. Previous studies focus on the interplay between adversarial robustness and transfer learning. This paper seems to for the first time study the interplay between adversarial robustness and domain generalization.\n+ The authors conducted comprehensive experiments, which clearly demonstrate the interplay between adversarial robustness and domain generalization. The authors clearly state the key findings from their experiments, which could be interesting and meaningful for the community.\n\nWeaknesses:\n\n- Although comprehensive empirical studies have been done, the paper lacks some theoretical analyses for interpreting their empirical findings.  \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper empirically evaluates the generalization and adversarial robustness on target domains achieved by the models that are adversarially trained on source domains where the target domain is distinct from the source domain. By deploying both empirical and certified defence, the authors show that the adversarially-trained models can lead to higher robustness on unseen domains compared with standardly-trained models. Besides, without finetuning, the adversarially-trained models cannot result in higher standard test accuracy on unseen domains. Finally, the authors provide two interesting research directions based on their empirical findings.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Could the authors provide more implementational details in Table 2? Actually, I am confused about the results in the column of “Source”. What is the source domain here? And what is the meaning of the results of “Source”?\n\n2. Should the term $P_{XY}^{(T)}$ in Eq. (1) be $P_{XY}^{(|T|)}$?\n\n3. I found that you use PGD-5 for adversarial raining and $\\beta=3$ for TRADES. Normally, we use PGD-10 for adversarial raining and $\\beta=6$ for TRADES, which follows the setting of the original paper.  Could you discuss the reason for your choice of these hyper-parameters, since this paper is an empirical study and the hyper-parameters could have a large impact on the final results? Possibly, you can provide the ablation studies on these critical hyper-parameters.\n", "broader_impact_concerns": "I have no broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper studied (empirical and certified) adversarial robustness under distribution shifts. This is an important yet underexplored research problem. The authors raised interesting questions and found inspiring answers with supportive evaluations, as the first step towards the goal of understanding how adversarial robustness transfers under distribution shifts. The paper received quite positive reviews from both adversarial-training and distribution-shift experts. Thus, we should definitely accept it for publication, and I think it deserves a Feature Certification.
Decision: Accept (poster)