Optimal transport based adversarial patch to leverage large scale attack transferability
Authors: Pol Labarbarie
Adrien CHAN-HON-TONG
Stéphane Herbin
Milad Leyli-abadi
Venue: ICLR 2024 poster (2024)
Keywords: adversarial patch attack
transferability
Primary area: societal considerations including fairness, safety, privacy

Abstract: Adversarial patch attacks, where a small patch is placed in the scene to fool neural networks, have been studied for numerous applications. Focusing on image classification, we consider the setting of a black-box transfer attack where an attacker does not know the target model. Instead of forcing corrupted image representations to cross the nearest decision boundaries or converge to a particular point, we propose a distribution-oriented approach. We rely on optimal transport to push the feature distribution of attacked images towards an already modeled distribution. We show that this new distribution-oriented approach leads to better transferable patches. Through digital experiments conducted on ImageNet-1K, we provide evidence that our new patches are the only ones that can simultaneously influence multiple Transformer models and Convolutional Neural Networks. Physical world experiments demonstrate that our patch can affect systems in deployment without explicit knowledge.

Published as a conference paper at ICLR 2024
OPTIMAL TRANSPORT BASED ADVERSARIAL PATCH TO LEVERAGE LARGE SCALE ATTACK TRANSFERABILITY
Pol Labarbarie Adrien Chan-Hon- Tongn Stéphane Herbin Milad Leyli-Abadi 'RT--SysteMX, ?ONERA/DTIS, University Paris-Saclay Palaiseau, France
[firstname. name feir-ssystemx fr [firstname name @onera. fr
ABSTRACT
Adversarial patch attacks, where a small patch is placed in the scene to fool neural networks, have been studied for numerous applications. Focusing on image classi- fication, we consider the setting of a black-box transfer attack where an attacker does not know the target model. Instead of foreing corrupted image representa- tions to cross the nearest decision boundaries or converge to particular point, we propose distribution-orientes approach. We rely on optimal transport to push the feature distribution of attacked images towards an already modeled distribution. We show that this new Distributionn-orentter approach leads to better transferable patches. Through digital experiments conducted on ImageNet- K, we provide evidence that our new patches are the only ones that can simultaneously influence multiple Transformer models and Convolutional Neural Networks. Physical world experiments demonstrate that our patch can affect systems in deployment without explicit knowledge.
INTRODUCTION
Deep neural networks have shown vulnerability to adversarial examples, i.e., norm- boun--dunded perturba- tions of their inputs designed fool them (Szegedy et [013 Biggio [iggio/[2013) Such vulnerability has motivated researchers to develop empirical robustification methods (Madry et al., 2017) to provide some theoretical robustness guarantees (Cohen 2019). Other research is dedicated to designing more powerful attacks (Kurakin et al., 2016). These attacks are invisible patterns added to the whole image a pixel array - which, therefore, has to be accessible: a strong practical limitation.
Adversarial patch attacks (APA) are more realistic type of attack expected to be realizable in the physical world. They rely on adding a small textured patch to the scene. Since such patch can be easily printed and localized
object in the environment, it poses serious threat in various contexts and related tasks. For example, Brown et al. (2017) produce a patch capable of fooling multiple ImageNet-11 K classification models. Patch attacks can also threaten other visual tasks (Thys et al. 2019 Lee & Kolter 2019 Saha et al., 2020 Hu et al. 2022 Nesti et al.) (022). Saha et al. (2020) design patch which, when placed on a stop sign the roadway, may result in the missed detection of pedestrian crossing the road. Another APA for a semantic segmentation task is proposed in (Nesti al. 2022). which reduces the baseline model accuracy. Despite the good attacking performance of current APA in whitebox configuration (applied on the same model that they have been learned), their effectiveness mitigated when transferring to unseen models (blackbox configuration). Prior works focus either on studying the whitebox performance of their patch against Transformer architecture (Fu al., 2022 Lovisotto al. 2022) or on studying the whitebox and/or the blackbox performance of their patch against classical CNNs (Brown et al. 2017 Karmon et al.) 2018 Liu et al. 2019 Doan et al.) 2022; Casper et al. 2022). Focusing on image classification, we propose
attack perspective to improve the transfer of patch attacks to unseen models.
Most previous work on APA for classification influence the network to output a target class with high confidence (Brown et al. 2017 Karmon et al. 2018 Liu al. 2019: Doan et al.) 2022 Casper ct al. 2002).. This strategy involves pushing the deep representation of images to cross the nearest decision boundary of the source model. The strategy has two drawbacks: it is highly dependent

---

Published as a conference paper at ICLR 2024
Decision boundaries based
Feature point bused
Distribution based (our upprowch)
Class 1
Class
Target point
source distribution
targeted distribution
corrupted distribution
Three different strategies for designing patch attacks. Left: The attack pushes multiple samples to the other side of decision boundary defined for a particular model. Middle: The attack matches a given point the feature space that is expected class. Right: Our strategy narrows the distribution gap between the samples corrupted by the patch and another misleading distribution in feature space. It does not depend on the decision boundaries, nor on the choice of specific target point in the feature space.
Figure 1:
represent sample from different
the model on which the attack is based, and the patch may push the corrupted image representations into unknown regions of the representation space. Instead of blindly maximizing the probability of
a target class, research on invisible adversarial examples suggests considering the feature space of
deep networks (Zhou et al. 2018; Rozsa et al. 2017 Inkawhich et al.) [019 2020). For example, Inkawhich et al. (219)) propose optimizing the adversarial examples to match the deep feature
representations of existing target image.
To overcome the dependence of the attack on a single decision boundary of the source model and to citiiibutionoorientte relax the specificity of the selection of the target feature point, we propose distribution of set approach. The learning principle of our patch attack is to globally alter the feature of images from particular class match another known distribution be taken from another class. To do so, we propose to optimize with respect to the Wasserstein loss which has several practical and theoretical advantages (Peyré al.) 2019 Frogner et al.) 2015 Arjovsky et al. 2017). The role of the patch, when placed in the scene, is to push the feature distribution towards this known misleading class distribution (Fig []. Such global strategy in feature space is expected to allow better transferability capability. as it is independent of the decision boundary constructed by the classifier and the choice of the target point (as proposed by Inkawhich et al. (2019). By conducting extensive experiments on ImageNet-IK (Deng et al. (209)), we indeed show that our APA is more model transferable and is more physically feasible than previous APAs for a large ensemble of network architectures, including classical CNNs (Simonyan & Zisserman 2014 Szegedy et al. 2016
He et al. 2016: Huang et al. 2017), recent CNNs (Tan & Le 2019 Liu et al. 2022) and Vision
Transformers ("Tourron al.) 2021 Liu al. 2021)
The main contributions this work are to:
introduce a new framework based on optimal transport for creating patch attacks that are highly transferable to unknown networks. This framework is based on the idea of attacking feature distributions, which is independent to the classifier decision boundaries and more robust to optimization artifacts than the feature point method;
show that our attack works for the most extensive spectrum of deep networks considered in the patch attack literature: we deal with various versions Convolutional Neural Networks, Transformers, and adversarially trained models and show transferability superiority through extensive experiments on ImageNet-IK:
provide digital and physical experiments demonstrating that our patch is potentially harmful in the physical world (including against state of the defense).
2

---

Published as a conference paper at ICLR 2024
2
RELATED WORK
2.1 TRANSFERABLE INVISIBLE ADVERSARIAL EXAMPLES
After discovering adversarial examples, some works studied whether these invisible adversarial attacks were transferable from one model to another. To enhance the adversarial examples transferability, Inkawhich et al. (2019) consider the feature space of deep networks rather than their decision space. The rationale for such a strategy is that the feature space is expected to capture the useful information in an image more universally, allowing better attack transferability between models. They propose to build the attack to push the corrupted image features close to single specific feature point (feature representation of chosen sample of pre-chosen class). To strengthen the transferability [Inkahhcch et al. 2020) train offline multiple specific auxiliary classifiers. Rather than independently train each adversarial examples by itcrative methods, Poursaced et al. (2018); Naseer et al. (2019a) 2021);
Zhao al. (2023) train Generative Adversarial Network (GAN) to transform clean images to adversarial examples. Naseer et al. (202T) (called TTP method) learn their GAN by minimizing the Kullback-Leibler divergence between the probability-class distribution of adversarial examples and the probability distribution of target class images. From a generalization error bound for black-box targeted attacks, (Zhao et al. (2023) derive an algorithm train their GAN minimize the maximum model discrepancy (M3D method) between two models. However, all these previously presented works study only the transferability of invisible adversarial examples, and, is not clear that they work for APA.
2.2 ADVERSARIAL PATCH ATTACK
APAs were first introduced for image classification by Brown al. (2017). Rather than finding a small additive adversarial noise, they instead constrain the optimization procedure to small part of the image while also allowing the optimization to unconstrained in magnitude. They design a patch (called GAP) by maximizing, under patch transformations, the log Softmax of selected target class. The resulting patch was capable of fooling five ImageNet-IK classifica- tion models. To increase the fooling ef- fectiveness of the patch,|Karmon Karmon al. (2018) (LaVAN method) add a new term the loss criterion initially pro- posed by Brown et al.) (2017). By minimizing the log Softmax of image ground truth, this new term ensures the misclassification of the attacked image. Other methods propose to use generative-based approaches: [iu et al. (2019) (PS-GAN method) train a GAN
Table 1: Typology of Adversarial Patch Attacks (APA) pa- pers and transferable invisible adversarial examples works depending on the requirement to illustrate a transferable physical APA. Each column represents a essential character- istic demonstrate the real-world criticality of an APA. The symbols v, z and ( represent "measured", "ambiguous" and "not evaluated", respectively.
Transler
Defended
APA
Narrow
Broad
networks
GAP (Brown et al. 2017
LaVAN (Karnon el al. 2018
PS-GAN (Lu
2019
TnT (Dean 2022
to generate a background harmmooooooo patch that enhances both the visual fi-
Casper all
Inkawhich et al. 2019
delity and attacking ability of the patch. Instead of training GAN, Doan al.
ITP al. 2021
M3D2hho ct al. (2023)
/
Ours
(002) and Casper et al. (2022) use
pre-trained GAN directly. To change the generated flower an adversarial flower, Doan et al. (022)) (TnT method) modify the latent representation of the generator, [Casper et al. (2022) perturb the latent representation some chosen generator layer. All the above works optimize their patch to maximize the log Softmax of some classifiers.
2.3 TRANSFERABILIT EVALUATION
The previously presented APAs Brown et al. (017)); Karmon et al. (2018): Liu et al. (2019); Doan
et al. (2022): Casper et al. (2022) use different evaluation settings in their experimentation. Some
of these works measure the transferability of their patch to unknown models. Brown et al. (0117) and Doan et al. (2022 measure their patch transferability but only consider dated CNNs like ResNet trained with a less generalizing learning policy than recent ones. The GAP patch is also applied at random locations in these images, giving rise to possible occlusion of the main object. The PS-GAN
3

---

Published as a conference paper at ICLR 2024
generated patch (Liu et al. 2019) shows transferability among unseen models. However, the evaluated models are not tfate-of-the-art and the experiments are conducted on the small GTRSB dataset (Stallkamp et al. 201T). Focused on the interpretability of Decision-making of deep networks, Casper et al. (022) do not provide quantitative results on network transferability. Table contextualizes our approach with respect to others.
Our work bridges the gap between the transferability studies of invisible adversarial examples and adversarial patch attacks. We follow the principle that the attack should be designed with an optimization metric defined in the feature space rather than in the decision space (Inkawhich et al. 2019p. Moreover, we generalize the point-wise strategy proposed by [Inaa presents several drawbacks. First, optimizing when the objective is to push multiple points to unique point is likely to fail. When the optimization succeeds, the power of the attack depends highly on the choice of the target point (see Appendix [B. Furthermore, single feature point can be well-classified by one network but misclassified by another, thus limiting the transferability of the attack to multiple networks. We propose to avoid target point selection and to broaden the range of features under attack by considering target point distributions instead. This is done through optimal transport for the loss to be optimized.
al. (2019). which
3 METHODOLOGGY
3.1 BACKGROUND
The theory of optimal transport (Peyré et al. 2019 Villani et al. 2009) provides several techniques for efficient computation distances between distributions. has been shown that optimizing with respect to the Wasserstein loss has various practical benefits over the KL-divergence loss (Peyré) et al,, 2012: Frogner et al., 2013; Arjovsky et al., 2017 Gulrajani et al., 2017). Unlike the KL- divergence and its related dissimilarity measures (e.g. Sensen-Shannan divergence), the Wasserstein distance can provide meaningful notion of closeness (i.e. distance) for distributions supported on non-overlapping low dimensional manifolds.
Let (R = {u E P(RR): fg [==[)dd((x) < oo} be the set of probability measures on Rd with finite moment of order p. with p [1, +00). The p- Wasserstein distance is defined as
inf [
Wg(,, =Ell(u.)) =x"R ||x yl|" "ml (x,
(I)
where n. u E Pp(R). ||-I1 is the Euclidean norm and IT(,, 1/) is the of probability measures on Rd Rd whose marginals with respect to the first and second variables are given by y and
respectively. The quantity W (n, v)) in not analytically avaible in general. To solve Eq | the standard
methods are linear programs and have a worst-case computational complexity in (n log(n)) where is the number of samples (Peyré al. 2019).
To leverage the computational efficiency of Eq Rabin et al. 2 Bonneel et al. (2015) define new metric named Wasserstenn distance. This new metric based the fact that for one-dimensional probability measure the p- Wassersstii distance (I has the following closed-form
WF(,, ()( v) = / Qu((s) (s)
where Q, and Qu are the quantile functions of μ and respectively. Let Sd-1 be the d-dimensional unit sphere and a the uniform distribution on gd-1, For 0 E 3d-1, we define the linear form for all x E Rdd by (xx) = (0, æ). The Sliced- Wasserstein distance is then defined by
SWE(uv)=
Wp (F(0G,,O))))r (o))d(())
-a-1
where 14, E (Rd), E [1, +ox) and 0j and 0j are the push-forward by 0* of and respectively. In practise, Bs approximated with a standard Monte Carlo method. We denote by SWE(R, p) its numerical approximation where K is the number of random projections. Since 0 μ and 0Jv are univariate distributions, the resulting complexity of the approximation is general more | The corresponding computational complexity is Ol O(K (Kdn + Kn log(n)). We show also in section of Appendix that the empirical computation time of our approach remains similar to other methods.
efficient than resolving Eg
4

---

Published as a conference paper at ICLR 2024
3.2 OPTIMAL TRANSPORT BASED LOSS
We consider the standard notation where (Ii, Wi) E X' x V. pei==,,..; are samples drawn from joint distribution of random variables X and Y. We are considering an image classification problem, where the input is sampled from X' = 1x/x where h x w are the image dimensions and is the number of channels and where the output is sampled from a set of M labels {y/,.... yM Let F X + ) be a given pre-trained neural network that we wish to attack. We assume that the functional architecture of follows classical encoder-decoder schema. We denote by f the encoder part of F, where f is composed by a set of J layers; L = (T,..., Except for the last layer which usually directly result from an average pooling layer, we apply an average pooling layer to obtain feature vector. For all E C, f(() maps I E X' to the feature space ((t) R where et is the number of channels.
(1 For given target class y. we denote by vy the multivariate target distribution f (X) when the class of X is y. The principle of our proposed method is to design patch by moving the corrupted image distribution towards the target vy and solve:
8 arg min Ex
LIE
where μxx, is the estimated feature distribution of the corrupted source images and OT could be WP or SWP.
In practice, we solve regularized version of Eq 4 using Expectation over Transformations (EoT)
from Athalye al. (2018). This regularization makes patches more physically realizable. Let T be distribution over transformation (e.g., rotations, scaling, blur, ...) and E distribution over locations. Following Brown et al. (017)); Casper et al. (2022) we denote by A(8, x, e, () the patch applicator operator in image where is the patch, are patch transformations and is the patch location in the image æ. Our patch is therefore trained to optimize the following objective:
8 arg min Ex,tTTeee E
Σ (o (9) TV
(5)
where TV is the total variation loss discouraging high fehh-frequeccy patterns. We will denote by (WP)(N) and (SWP)(M) when we attack layers by solving the standard or the sliced version of the Wasserstein distance respectively. We choose by convention that for N = 1, Ly, i.e. attacking the last layer of f.
EXPERIMENTS
This section evaluates our APA through digital, hybrid and physical world experiments. all experiments, the objective is to craft APA with a high targeted success rate (tSuc).
We consider the single-source model setting and test attacking transferability to other models. Transferability is tested between ImageNet-1K IK (Deng et al. 2009) models; ResNet 18/34/50- V1/50-V2 (He et al.] 2016). DenseNet 121/161/169/201 (Huang et al. 2017). EfficientNet BO/B1/B2/B3/BB4 (1 (Tan& & Le 2.19) ConvNext Tiny/Small (Liu al. 2022). VGG19 (Deng et al. 2009), Inception- V3 (Szeedyy et al. 2016) and Swin Tiny/Small/Bant (T/S/B) (Liu et al. 2021 from Pytorch Model Zoo, DeiT T/S/B from Timm Model Zoo, ResNet50 ReLU Adv and DeiT S Adv, adversarially trained models (trained against invisible adversarial examples) from Bai al. (202T) and finally ResNet50 self-supervised learned from Caron al. (2020). We regroup these models into the following categories depending on their architecture and their training recipes: CNNs-VI (ResNet 18/34/50-VI V1. DenseNet 121/161/169/201, VGG19, Inception-Vis (EfficientNet B0/B CO0/B//B//33/ 1/B2/B3/B4), (DeiT T/S/B). Swin
CNNs-V2 = (ResNet 50- (2/50-)--s)ff),
ENet
{Swin T/S/B) and AT (ResNet50 ReLU Adv and DeiT Adv), where AT stands for Adversarially trained.
CNext [ConvNext T/S), DeiT
Evaluated methods. We consider GAP (Brown et al. 2017). LaVAN (Karmon et al. 2018), TnT DDoan al. 2022). Casper et al. (2022) and Logit (Zhao et al. 2021 decision boundary-basee
5

---

Published as a conference paper at ICLR 2024
baselines. Because of its case of computation compared to Inkawhich et al. (2020), which requires off-line training of multiple specific auxiliary models, we choose to adapt the proposed method by baseline (we name it L2) to craft an APA based on attacking the feature space. We also adapt the recent state-of-the-art works on transferable invisible adversarial examples (Naseer al. 202T Zhao et al. (023) craft an APA. To do so, convert generative methods
Inkawhich et al. (0199)
(asser et al.
2021: Zhao et al. 2023 to iterative ones: the objective is to create one universal APA
and not one for each image.
Experimental setup. For the sake of comparison, baselines and our method are crafted using the same training recipes. To control the balance between the adversarial loss and the total variation loss, the gradient of each loss is computed individually, normalized, and combined using a weighted sum. Patch values are clipped into the image range each iteration. Following prior works Brown et al.) (2017); & Kolter (2019); Casper et al. (2022), we choose and fix the sampling distributions from HoT (Eykholt et al. 2018) for all the methods. During training and evaluation, patches are randomly placed to the side of images (to avoid occluding the object of interest), and transformations and noises are applied to the patch to mimic real-works situations. Appendix [ evaluates the robustness of models according to the patch position in the image. We randomly choose nine targeted classes (see Appendix A for details) and design a patch to fool the network targeting each of these classes. We split the ImageNet- 1K validation set into training set of 40000 images patches and a test set of 10000 images on which we evaluate their impact. The patch optimization is performed using 100 epochs (1 epoch equals 1000 iterations) with a batch size of 50 images and for three different learning rates (0.1, 0.5, 1). We choose for our method are explained in Appendix [T.. We evaluate the APA with the best loss among the three learning rates, leading to one patch per method and per class. Finally, reported tSuc are the average over the classes and patch sizes (from 70 70 to 90 90 which is the standard setting consider in the literature
which train
2 and K = 500 (reasons
(Brown et al. [017]] Poursaeed et al. 2018 Doan et al. [0222] Casper et al. (2022).
4.1 DIGITAL EXPERIMENTS
4.1.1 TRANSFERABILITY AMONG NETWORKS
We select from the previously defined families the following models: ResNet34, ResNet50- V1, ResNet50-V2, ResNet50-self, EfficientNer-BO, ConvNext-S, DeiT-S, Swin-T, Swin-S, Swin-
B. We design patches to attack one of these source models. Then we measure the attacking transferability when the resulting patch is used to fool the remaining models (target models). For example, patches trained and tested on the Swin family provide three patches. Each one is trained individually on Swin- T, Swin-S or Swin-B, and is evaluated against the two other Swin models. Ta- ble summarizze, for each method, the best transferring attack performance. We select and report the results of the source family producing the highest mean tar- geted success rate (tSuc, rate which
Table 2: Best transfer results from single model to all others obtained for each method (tSuc (%) higher is better for an attack).
Method
min
mean
max
GAP (Brown et al. 2017)
2.22
15.46
37.33
LAVAN (Karmon et al.] 2018)
2.26
8.67
31.4
L2 (Inkawhich et al. 2019))
4.44
13.6
32.78
TnI (Doan et 2022)
0.67
2.11
5.84
Casper et al. 222
0.33
3.81
14.85
the attacked images are classified as the patch target label) according the method. Our method shows the best transferability capacity: highest mean, min and max tSuc.
Logit Zhao et al.
(2021)
2.22
7.55
26.55
TTP Nascer et (2021)
2.33
13.77
31.87
M3D Zhao et al.
2023)
0.84
5.19
17.11
Ours (SW?) 2)300
8.93
22.56
45.31
Ours (W3)()
8.09
21.14
49.1
Table 3 details transferability results for all source families. From this table, we can make several conclusions: Networks trained with older training recipes (CNNs-vI) seem more vulnerable to attacks regardless of the attacking procedures. These networks more sensitive to salient patches present in the image. As presented in Bai et al. (2021), new training recipes (scheduler, augmenting training data like RandAug and Mixup, ...) appear to robustify models for convolutional networks and transformers. Baseline methods were not able to create a patch that can strongly transfer to CNext and Swin models even when these patches learned using the same model category. This indicates that rather than catch the useful common

---

Published as a conference paper at ICLR 2024
information in deep networks, baseline methods produce a patch that tends to overfit the specific
weights of the model. This is particularly the case for decision boundaries- based APAs (GAP (Brown
et al. 2017). La (Karmon et al. 2018). TnT (Doan et al., 2022) and Casper et al. (2022)). Naseer et al. (202T) method seems to create a patch that better captures the overall source model decision boundaries. This method leads to better results than decision boundaries-
APAs the DeiT
family of networks (networks with more complex representations than CNNs-VI networks). Creating a patch that minimizes the maximum discrepancy between two models (Zhao et 2023) is unstable and generally results in patch that is not transferable. A possible explanation is that APA induces a higher shift in the feature space than invisible adversarial examples.
A patch resulting from an optimization defined in the feature space reduces the patch overfitting and increases the transferability to other networks. This suggests that the patch has learned more about the common information to model the different classes rather than trying to cross the decision boundaries. However, the 1.2 methodology (Inkawhich et al. 2019) unstable and is highly dependent on the choice of the target point, resulting in lower performance than our method (see Appendix B Our two methods (exact and sliced version) outperform the previous methods on transferability. We remark that patches learned using Swin CNext seem more universal as they transfer to multiple models. When crafted on Swin models, we produce patch capable of transferring uniformly well to all the models. We show in Appendix that ensemble of CNNs-v1 models can not reach the level of transferability obtained by our method when targeting Swin models. These results indicate that our method allows the patch to learn more about the common information shared across networks. The following experiments are performed on Swin patches as they lead to more uniform transferability across networks.
4.1.2 EFFECTIVENESS AGAINST ROBUSTIFLED NETWORKS
We now consider a more realistic scenario in which the attacked system uses a de- fense mechanism. We propose to use Local Gradients Smoothing (LGS) (Naseer) al. 20195). as it is one of the strongest defense mechanism against patch attacks. LGS smooths salient regions in images before feeding them to the network. We reproduce the previous experiments for three different smoothing factors A E 1.9, 2.3} for LGS while fix- ing other parameters as in the article (we report here results for x = 1.5, see K for other results). For each method, we evaluate their Swin patches against net- works robustified by LGS. Our method achieves the best transfer results demon- strating the criticality of our attack even for robustified networks (Table 4 We now suppose the target network has heen adversarially trained (AT) against invis- ible adversarial examples. The patch at- tacks which are not learned on AT models could reduce their accuracy when transferred to these models. However, the AT models do not get fooled by the patch to predict the targeted class. (clean accuracy: 65.44, attacked accuracy: 57.65, tsuc: 0.72). AT models seem to have different class representations and are hard to force to predict chosen class. When designed one AT model and transferred to another model, our patches and GAP patches produce the best transfer performances (see Appendix G
Table 5: Transfer results of digital, scanned and scanned defended patches (mean tSue (%)). Patches are designed on Swin models and for class bird house.
1.9,
Digital
Scan
Scan
Defended
Clean
0.1
0.1
0.1
GAP Brown et al. 2017
1.3
1.1
0.92
La Karmon 2019
2018
1.53
1.05
0.88
L2 Inkawhich
8.65
4.54
4.26
(Doan el al. 2022
0.78
0.43
0.37
Casper et al. 2022
1.13
0.49
0.39
TIP Naseer el al.
2021
1.06
0.73
0.52
M3D
Zhao et al. 2023
1.84
1.08
0.68
Ours SW:) 20500
19
12.41
12.11
Ours W3)
20.04
12.59
12.41
4.2 HYBRID EXPERIMENTS
In this section, we propose to measure the physicality of patches through hybrid experiment and to simulate the potential effect of patches in the real world. We consider the following steps: printing and digitalization. Scanned patches in the previous section (physical transformations are applied to them). We use patches designed on Swin-T and the results for three different settings (i.e., digital, scan and scan with defense) are
placed numerically in images using the same procedure as
7

---

Published as conference paper at ICLR 2024
Table 3: Transfer results (tSuc (%), higher is better attack) between categories of models. Results averaged over classes, patch sizes and over networks within category. Patches are placed randomly in the image without object overlapping. Physical transformations (e.g., noise, rotations) applied patches. Control stands for inserting real object of the corresponding class patch.
Target
CNNs-v1
CNNs-v2
ENet
CNext
DeiT
Swin
mean std
Clean
Method
0.1
0.1
0.1
0.1
0.1
0.1
0.1 /0
Source
Control
CNNs-v1
36.61
9.64
5.54
2.43
4.03
10.22
/12.04
CNNs-v2
15.6
9.57
3,66
2.74
3.51
2,49
6.26
/.882
GAP Brown al. 2017
ENet
37:33
10.11
29.38
2.22
8.91
4.3
15.46
113.28
CNext
0.33
0.77
0.23
0.43
00.27
DeTT
1.43
1.97
0.46
1.25
11.54
3.58
3.37
1.778
Swin
1.46
1.54
0.66
1.58
2.12
1 11.83
CNNs-v1
31.4
8.56
4,32
2.26
2.49
3.01
8.67
/10.8
CNNsvv2
9.68
2.33
ENet
8.74
4.76
11.31
1.08
3.33
2.58
5.3 3.59
LaVAN Karmon 2018
0.45
0.63
0.26
0.47
0.52
10.19
DeiT
2.1
1.53
0.93
0.61
5.84
2.45
2.24
1.73
Swin
1.45
1.41
0.57
1,31
1.29
9.44
2,58
13.08
CNN
14.76
5.06
2.69
0.88
1.78
1.08
4.37 14,85
CNNs-v2
3.25
3.5
0.64
1.44
0.57
1.74
1.119
ENet
14.33
4.12
13.35
0.79
3.02
1.88
6.25
15.47
Inkawhich
CNext
2,46
9.66
0.92
20.2
1.73
10.67
7.6 76.81
DeTT
17.88
10.23
8.15
4.44
32.78
13.6 79.5
Swin
8.2
8.54
3.22
7.24
5.38
23.24
9.3 6.49
CNNs-vI
5.84
1.5
2.12
0.67
1.43
1.08
2.11 11.33
CNNs-v2
1.82
0.69
0.59
0.37
0.52
0.77 10.48
Doan et al. 2022
ENet
2.13
0,92
1.4
0.71
10.57
CNext
0.48
0.49
0.24
0.32
0.3
0.4
0.37
10.09
DeTT
Swin
1.41
1.03
0.55
0.81
1.61
1.68
/0.42 1.18
CNNs-vl
12.87
1.62
1,2
0.28
0.33
2.75 14.56
CNNs-12
7.8
7.44
1,26
0.83
0.95
0.78
3.17
13.15
0.85
15.23
Casper et al. 2022
CNext
0.42
0.28
0.22
0.45
0,15
0.22
0.29
10.11
DerT
2,86
1.38
0.98
10.19
3.09
13.25
Swin
0.56
0.4
0.35
0.52
0.32
1.87
0.67
10,54
CNNs
5.43
3.46
71188
CNNswv2
17.55
9.67
3.3
3.87
3.66
3.87
6.99 152.1
ENet
31.87
8.88
27.16
8.75
13.77
/11.77
TTP Naseer et al. 2021
CNext
0.49
3.39
0.22
7.87
0.48
2.71
2.53
/2.67
DeTT
3.87
3.24
1.51
1.64
13:75
3.85
4.64
/4.18
Swin
1.53
1.22
0.53
1.18
1.69
CNNs-vi
17.11
6.18
3.59
0.84
1.98
1.43
5.19
75.61
CNNs-v2
7.45
11.77
1.77
2.13
1.51
2.33
4.49
13.84
Zhao al. 2023
ENet
11.21
1.97
3.34
0.54
1.0
0.88
3.16
13.72
CNext
0.34
0.36
0.16
1.78
10.57
DeiT
2.39
1.59
0.81
1.07
7.7
2.9
2.74
2,33
10.95
CNNs-vl
25.25
6.15
4.73
1.7
5.15
2.61
7.6
78.04
CNNs-v2
16.93
8.67
4.02
5.77
/4.69
Ours (SW 2
ENet
22.53
5.83
18.8
2.07
8.49
3.03
10.13 77.8
27000
CNext
DeiT
23:65
12.16
7.27
5.21
32.39
9.35
15.01
/9.77
Swin
25.2
20.21
8.93
19.54
16.16
45.31
22.56
/11.3
v1
39.65
13.01
8.27
2.44
4.89
3.16
11.9
/1.91
CNNsv2
1.669
Ours (W))
ENet
35.12
10.45
32.0
2.27
7.8
3.79
15.24
133.25
CNext
3.47
12.2
0.92
25.14
2.04
15.12
9.82
8.664
DeiT
22.26
11.43
10.18
5.29
39.51
9.25
16.32
11.59
Swin
20.55
17.89
8.09
17.7
13.55
49.1
21.14
/1312
reported. Our patch obtains the best transfer results and performs well in a complex setting: sean with defense (see Table 5) This result confirms the potentially harmful behavior of our patch in the real-world.
4.3 QUALITATIVE PHYSICAL EXPERIMENTS
In this section, we give some qualitative results concerning the physicality of our attack. We select three objects present in ImageNet-1K (banana, cup, keyboard) and record videos of them when
8

---

Published as a conference paper at ICLR 2024
Table 4: Transfer results on robustified models by LGS defense Naseer al. Patches are designed on Swin models.
2019b) (tSuc (%)).
Target ENet
CNNs-v1
CNNs-v2
CNext
DeiT
Swin
mean
Clean
0.1
0.1
0.1
0.1
0.1
0.1
0.170
GAP Brown al. 2017
0.72
0.87
0.35
0.78
1.13
2.34
1.03 0.63
LAN Karmon
2018
0.56
0.69
0.3
0.69
0.82
2.65
0.95 0.78
L2 Inkawhich et at. 2019
4.79
6,44
1.72
7.79
4.79
13.85
6,56 3.75
ThT Doan el al 202
0.84
0.59
0.52
0.53
0.7
0.85
0.67 / 0,13
Casper ei al 122
0.37
0.4
0,2
0.32
0.25
0.59
0.36/ 0.13
TIP (Nsseer el dl 2021
0.68
0.77
0.28
0.68
0.76
1.98
0.86 0.53
M3D Zhao et al. 2025
0.83
0.81
0.36
0.77
1.17
1.17
0.85 0.27
Ours (SW 27000 Ours (W3)
10.56
11.86
3.81
18.9
11.67
31.68
14.75 8.75
13.23
13.4
4.37
21.42
13.84
32.08
16.39 / 8.58
patch is placed object. Figure[ishows examples our patch near objects. In conducted experimentations, all the patches were not able transfer (tSuc lower than 2%), except for L2 and our patches. The transfer results for the L.2 method, our first ((SW2) and second (( ((W2)s00 methods are 9.3%, 23.4% and 29.3% respectively. These results confirm that real-world classifiers be swayed without explicit knowledge of their architecture or their weights.
the object. During the video, patches moved around the
Figure 2:
Examples of frame of our APA close to different objects. Our patch is designed sway networks to output the class bird house.
4.4 ABLATION STUDIES
We study the impact of the choice of the targeted layers on the patch transferabilityy We apply our attack ( ((22) version) on different layers for Swin models and report results Table The last layer of the encoder (1 = Iy) seems essential to model and close the gap between the corrupted image distribution and the target distribution.
coherent since this layer expected to model and separate classes before linear classification. We observe that the multi-layer objective leads to better results as it helps the optimization to converge to a better local minimum, leading to stronger patch. However, most layers fail model the targeted distribution correctly. In Appendix propose study the effect of the power and the number of slices K.
Table 6: Transfer results of digital patches when varying the choice the targeted layers (tSuc (%)). Patches are designed Swin models. See Appendix
[ for details concerning layers.
L
(T, 8, (,2 (1,-2.1)
(1--2)
(T,
mean
14.66
23.47
17.31
21.14
DISCUSSION AND CONCLUSION
This paper presents citriibttionorriented method based on optimal transport for designing APAs. This new method reduces patch overfitting to the source architecture and strengthens transferability to Convolutional Neural Networks and Transformer architectures. When designed on Swin models, our patch is the only one capable of strongly fooling multiple architectures from different model families, even when the model robustness has been enhanced by defense mechanism. Hybrid and physical experiments illustrate that our attack can disturb real-world classifiers without any knowledge of the system.

---

Published as a conference paper at ICLR 2024
ETHICS STATEMENT
This paper presents how to make attacks potentially harmful to real-life deep networks. Ignoring the existence of attacks like the one presented in this work leaves systems with false sense of security and may be disastrous to numerous applications like autonomous vehicles. Research promotes transparency and fosters proactive approach to addressing potential vulnerabilities. hope that our work empowers individuals and organizations to take necessary precautions, ultimately leading to safer and more secure AI landscape.
Although this paper is just one snapshot of the attack/defense racc, want to summarize some recommendations based on our experiments.
Despite Swin being a state-of-the-art model, we advise against relying on for critical computer vision functions. Indeed, an attacker using method would design its patch on Swin models to maximize transferability across the different network families. However, show, patches designed Swin models are the most critical for Swin models.
Conversely, using either of ConvNext AT model with defense are quite resilient to patch not designed on them with the drawback a moderate initial performance, and, ConvNext are the best trade-off today (good initial performance and moderate loss of performance against an attack even when designed on ConvNext).
good shot: AT models combined
ACKNOWLEDGMENTS
This work has been supported by the French government under the "France 2030" program, as part of the SystemX Technological Research Institute within the Confiance.ai project. This work was granted access to the HPC resources of IDRIS under the allocation 20AD-011014245 made by GENCI.
REFERENCES
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pp. 214-22% PMLR, 2017.
Anish Athalye et al. Synthesizing robust adversarial examples. In ICML. PMLR, 2018.
Yutong Bai, Jieru Mei, Alan L Yuille, and Cihang Xie. Are transformers more robust than cnns? Advances in Neural Information Processing Systems, 34:26831 -66843, 2021.
Battista et al. Biggio. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery in databases, pp. 387-402, Springer, 2013.
Nicolas Bonneel, Julien Rabin, Gabriel Peyré, and Hanspeter Pfister. Sliced and radon wasserstein barycenters of measures. Journal of Mathemutical Imaging and Vision, 51:22-45, 2015.
Tom B Brown et al. Adversarial patch. arXiv preprint arXiv: arXiv: 2017.
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems, 33:9912-9922 2020.
Stephen Casper, Max Nadeau, Dylan Boddield-Menell and Gabriel Kreiman. Robust feature-level adversaries are interpretability tools. Advances Neural Information Processing Systems, 35: 33093-33106 2022.
Jeremy others Cohen. Certified adversarial robustness via randomized smoothing. ICML, 2019.
Jia Deng et al. Imagenet: A large-scale hierarchical image database. In 2009 IEEE CVPR, 2009.
10

---

Published as a conference paper at ICLR 2024
Bao Gia Doan, Minhui Xue, Shiqing Ma, Ehsan Abbasnejad, and Damith C Ranasinghe. Tnt attacks! universal naturalistic adversarial patches against deep neural network systems. IEEE Transactions on Information Forensics and Security, 17:3816-3833 -0000, 2022.
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical- attacks on deep learning visual classification. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1625-1634, 2018.
Charlie Frogner, Chiyuan Zhang, Hossein Mobahi, Mauricio Araya, and Tomaso A Poggio. Learning with wasserstein loss. Advances in neural information processing systems, 28, 2015.
Yonggan Fu, Shunyao Zhang, Shang Wu, Cheng Wan, and Yingyan Lin. Patch-fool: Are vision transformers always robust against adversarial perturbations? arXiv preprint arXiv:2203.08399, 2022.
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training of wasserstein gans. Advances in neural information processing systems, 30, 2017.
Kaiming He et al. Deep residual learning for image recognition. In JEEE CVPR, 2016.
Zhanhao Hu al. Adversarial texture for fooling person detectors in the physical world. In Proceedings of the IEEE/CVF CVPR, pp. 13307-13316, 2022.
Gao Huang et al. Densely connected convolutional networks. In JEEE CVPR, 2017.
Nathan Inkawhich, Kevin Liang Binghui Wang, Matthew Inkawhich, Lawrence Carin, and Yiran Chen. Perturbing across the feature hierarchy improve standard and strict blackbox attack transferabilitty. Advances in Neural Information Processing Systems, 13:20791-208011 2020.
Nathan Inkawhich et al. Feature space perturbations yield more transferable adversarial examples. In Proceedings of the IEEE/CVF CVPR, 2019.
Danny Karmon, Daniel Zoran, and Yoav Goldberg. Lavan: Localized and visible adversarial noise. In International Conference on Machine Learning, pp. 2507-2515. PMLR, 2018
Alexey Kurakin al. Adversarial machine learning at seale. arXiv preprint arXiv:1611.01236 2016.
Mark Lee and Zico Kolter. On physical adversarial patches for object detection. arXiv preprint 1.Xi:1996..1897,, 2019.
Aishan Liu, Xianglong Liu, Jiaxin Fan, Yuqing Ma, Anlan Zhang, Huiyuan Xie, and Dacheng Tao. Perceptual-sensitive gan for generating adversarial patches. In Proceedings of the AAAI conference artificial intelligence, volume 33, pp. 1028-10355 2019.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference computer vision, pp. 10012-10022, 2021.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11976-10 11986, 2022.
Giulio Lovisotto, Nicole Finnie, Mauricio Munoz, Chaithanya Kumar Mummadi, and Jan Hen- drik Metzen. Give me your attention: Dot-product attention considered harmful for adversarial patch robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5523--15243 2022.
Aleksander Madry et al. Towards deep learning models resistant to adversarial attacks. arXiv preprins wrXiv:/706.06088 2017.
Muhammad Muzammal Naseer, Salman H Khan, Muhammad Haris Khan, Fahad Shahbaz Khan, and Fatih Porikli. Cross-domait transferability of adversarial perturbations. Advances in Neural Information Processing Systems, 32, 2019a.
11

---

Published as a conference paper at ICLR 2024
Muzammal Naseer, Salman Khan, and Fatih Porikli. Local gradients smoothing: Defense against localized adversarial attacks. In 2019 JEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1300-1307. IEEE, 2019b.
Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli. On generating transferable targeted perturbations. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp-
2021.
Federico Nesti et al. Evaluating the robustness of semantic segmentation for autonomous driving against real-world adversarial patch attacks. In Proceedings of the IEEE/CVF WACV, pp. 2280- 2289, 2022.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lever, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, nigh-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Gabriel Peyré, Jalal Fadili, and Julien Rabin. Wasserstein active contours. In 2012 19th IEEE International Conference Image Processing, pp. 2541-2544. IEFF, 2012.
Gabriel Peyré, Marco Cuturi, et al. Computational optimal transport: With applications data science. Foundaaions and Trends@ in Machine Learning, 115-6))335-667, 2019.
Omid Poursaeed, Isay Katsman, Bicheng Gao, and Serge Belongie. Generative adversarial pertur- bations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4422-4431, 2018.
Julien Rabin, Gabriel Peyré, Julie Delon, and Mare Bernot. Wasserstein barycenter and its application to texture mixing. In Scale Space and Variational Methods in Computer Vision: Third International Conference, SSVM 2011, Ein-Gedi, Israel, May 29-June 2, 2011, Revised Selected Papers 3, pp. 435-446. Springer, 2012.
Andras Rozsa al. Lots about attacking deep features. In IJCB. IEEE, 2017.
Aniruddha Saha al. Role of spatial context in adversarial robustness for object detection. In Proceedings of the IEEE/CVF CVPR Workshops, pp. 784-785, 2020.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1555 2014.
Johannes Stallkamp, Marc Schlipsing, Jan Salmen, and Christian Igel. The german traffic sign recognition benchmark: a multi-class classification competition. In The 201 international joins conference on neural networks, pp. 1453-1460. IEEE, 2011.
Christian Szegedy et al. Intriguing properties of neural networks. arXiv preprint a1X6v:1312.6199. 2013.
Christian Szegedy et al. Rethinking the inception architecture for computer vision. In IEEE CVPR, 2016.
Mingxing Tan and Quoe Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pp. 6105-61 14. PMLR, 2019.
Simen Thys et al. Fooling automated surveillance cameras: adversarial patches to attack person detection. In IEEE/CVF CVPR workshops, 2019.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, pp. 10347-10357. PMLR, 2021.
Cédric Villani al. Optimal transport: old and new, volume 338. Springer, 2009.
Anqi Zhao, Tong Chu, Yahao Liu, Wen Li. Jingjing Li, and Lixin Duan. Minimizing maximum model discrepancy for transferable black-box targeted attacks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1162-8162, 2023.
12

---

Published as a conference paper at ICLR 2024
Zhengyu Zhao, Zhuoran Liu, and Martha Larson. On success and simplicity: A second look at transferable targeted attacks. Advances in Neural Information Processing Systems, 34:61 15-6128, 2021.
Wen Zhou et al. Transferable adversarial perturbations. In ECCV, 2018.
13

---

Published as a conference paper at ICLR 2024
Appendices
A Implementation details
14
Feature point method instability
15
Feature point method generalization
19
D Benefits of Optimal Transport
19
Model robustness and patch position
20
Ensemble methods
21
Transferability adversarially trained models
21
H
Robustness according to physical transformations
22
Ablation studies
23
Decision boundary-based methods overfitting
24
K Complementary tables
27
Printable patches
28
A IMPLEMENTATION DETAILS
Our method training routine uses the PyTorch library (Paszke et al. 2019). For the training of each patch on medium and large models consider single NVIDIA V100-32G single NVIDIA A100 respectively. To train patches on smaller NVIDIA cards we should reduce the batch size. When not specified, patches are designed target one the following classes: salamander, starfish, bird house, bullfrog, pinwheel, mongoose, brown bear, accordion and common iguana.
We use Expectation over Transformations (EoT (Eykholt al., (018)) to obtain a more physically realizable patch, similarly to prior work APAs (Brown et al. 2017 Lee & Kolter, 2019 Casper al. 2022). For all the methods (GAP (Brown et al. 2017). ) LaVAN (Karmon et al. 2018), L.2 (Inkawhich et al. 2019) and ours), during training, randomly rotate the patch up to five degrees for the x and y-axis and up to 10 degrees for the Z-axis. We also randomly scale the patch between 70 x 70 to 110 x 110 pixels, adjust patch brightness between [-0.1, 0.1] and patch blur between [0.8, 1.2], and apply normal noise of magnitude 0.1 on the patch. Patches are randomly translated in the image but not in the center.
To control the balance between the adversarial loss and the total variation loss, the gradient of each loss computed individually, normalized, and combined using weighted sum. Following Nesti et al. (2022) we choose Wode = 1 and wrv = 0.1 where wade is the weight for the adversarial loss and WTV is the weight for the TV loss.
Computation time. We measure and report the computation time of each method Table 7 This Table reports the averaged computational time for the different methods. Our method has similar computational time as other methods. This result may be counterintuitive as OT losses are known to be slow, but in setting, the number of samples is low. The M3D method ZZhao et al. 2023) is
14

---

Published as a conference paper at ICLR 2024
much slower than other methods. It is coherent, this method trains alternatively the patch and two models in a min-max game.
Table 7: Computational time of the different methods to obtain a fully optimized patch (minutes).
Times are averaged over ten optimization runs. Each run is launched on the same setup composed by a single NVIDIA A A100.
Method
Time
GAP (Brown et al 2017
20
LaVAN Karmon al. 2018
30
L2 (Inkawhich el al. 2019
20
TnT (Doan et al 2022
30
Casper et (2022)
35
TTP Naseer et al.
202
30
M3D
Zhao al.
2023
66
Ours
SW (1 2/500 (W3)
19
Ours
20
B FEATURE POINT METHOD INSTABILITY
To measure the stability of the L2 method (Inkawhich et al.) 2019), we launch the optimization three randomly selected target points. Patches are designed to sway ResNet50-vl or Swin-T to output the class Australian terrier. Figure plots the learning curves and the resulting patches for our distribution based approach for Resnet50-vl I and Swin-T, respectively. Figure and plot the learning curves and the resulted patches of the L2 method for Resnet50-v and Swin-T. respectively. These four graphs show that our method the easiest to optimize and is more robust to optimization artifacts. For the Swin-T model, the optimization for the L. method becomes noisy. Table reports the transfer results of the obtained patches from previous figures. Although the optimization has converged for the first target of the L2 method for ResNe150-v1, the obtained patch is harmless. Even if the APA works, its attacking capacity depends on the considered target point. For example, the mean transferability on Swin-T can decreased by factor four. general, our instribttoo-ooriented approach outperforms the L2 method.
Table 8: Transfer results between categories of models (tSuc (%)) for the L2 method and for our distribution-oriented method. Three different target points are evaluated for the I.2 method. Results are for the source model ResNet50-V1 and Swin-T, for the class Australian terrier and for patches of size 60 60. Patches are placed randomly in the image but not at the center images.
Target
menn / std
Source
Method
CNNs-11
CNNs
ENet
CNea
DeiT
Swin
AT
Target 0.1
0.1
0.1
0.1
0.1
0.1
0.1
0.170
ResNet50-vl
Inkawbich
2019
Target 36.64
2.52
9.35
0.52
3.59
0.5
3.71
8.12/
Target
43.83
4.18
8.82
0.75
4.98
0.58
6.09
9.89
Ours
43.34
4.76
8.75
0.92
6.46
0.63
4,68
9.94 13.9
Target
4.12
1.18
2.41
0.23
1.83
1.9
0.39
1.72/78
Swin-T
Inkawhich
Target
26.97
Target
0.17
0.11
0.12
0.1
0.1
0.07
0.1
0.11 0.00
15

---

Published as a conference paper at ICLR 2024
Ours / ResNet50-v1
Lr=
Loss
26
Epoch
Ours Swin-T
Lr 0.1
La 0,5
LOss
50-
Epoch
Figure 3: Learning curves and resulted patches of our distribution-oriennedd method. The optimization run for three different learning rate. The source model ResNet50-vl or Swin-T and the targeted class is Australian terrier.
16

---

Published as a conference paper at ICLR 2024
Target
Lr-0,1 0,1
L,5
Lr-1
Losss
100
20
36
Epoch
Target 2
1,1
Lr- 0,5
Lr-1
70
LOSs
18-
17
Epoch
Target 3
Lr-0,1
Lr 0.5
Loss
Epoch
Figure 4: Learning curves and resulted patches of the L2 method for different targeted points. For each targeted point the optimization is run for three different learning rate. The source model is ResNet50-vl and the targeted class is Australian terrier.
17

---

Published as a conference paper at ICLR 2024
Target
Lr-0.1 0,1
Lr 0,5
70
5.5
L,OSS
10
www. M
29
189
Epoch
Target 2
0,1
0,5
6.2
60
33.
L,88S
3.50
3.23
435
Am00
M
16
Epoch
Target 3
Lr- 0,1
0,5
225
5.00
235
1.2
3.50
1.50
L,888
235
7.00
7.09
3.55
432
Epoch
Figure 5: Learning curves and resulted patches of the L2 method for different targeted points. For each targeted point the optimization is run for three different learning rate. The source model is Swin-T and the targeted class is Australian terrier.
18

---

Published as a conference paper at ICLR 2024
C FEATURE POINT METHOD GENERALIZATION
We provide in this section proof that the exact 2- Wassersteei distance coincide with the Las-baed method Inkawhich et al. (2019) when the source distribution uniformly distributed and the targeted distribution is supported by unique point. We recall that
inf [ nel((w.)) J
(2
||
y112da (2,, v),
defines the 2- 2-Wasserstein distance. This distance can be interpreted through probabilistic point of view. If we name (X, Y) couple of random variables over Rd Rd with X ~ μ.. Y and (X,Y) I( V), we can write
(2, min Exxyy [||X - Y|1 (X,Y)
we suppose that the target distribution is composed by a unique point, 2.c., v/ = 6y, then we have
W v) mmmn min Ex [[|X y/ y/
(8)
X
our problem we have empirical distribution based on samples, we name A. and im the empirical distributions of based each sample from each distribution is uniformly distributed, i.c., An E dz, and im =
samples and based
samples respectively, We suppose that
1 >> m
j=1 6g., where ó is the Kronecker symbol. The estimated 2- 2-Wasserstein distance is
m
W2 Pm)
min (Ell(w)))
Tyy Tell=; v.ll?.
If we suppose that the target distribution is composed by unique point, i.e., & = 6y, then we have
1 n n1
W D)
Σ x w112.
(10)
which is equal to the L2-based criterion. Minimizing with respect to the 2- 2-Wasserstein is equivalent consider the L.2-based criterion (Inkawhich et al. 2019). As result, our method includes and generalizes the L2-based method.
D BENEFITS OF OPTIMAL TRANSPORT
Optimal transport-based losses (both exact and sliced) has the following advantages:
OT losses take into account the underlying metric space (through the cost matrix) on which the probability distributions defined,
for non-overlapping distributions such ours, the Kullback-Leible divergence is infinite.
To illustrate the first point, we consider the toy example shown in Figure 6 We define four different one-dimensional distributions supported here by five points. We compute the 1- I-Wasserstein distance and the KL divergence between the red and the blue distributions for each column (results are shown between graphs). The blue mass has been moved near the first point from right left. The 1- Wasserstein distance captures this mass shift, while the KL divergence does not and remains constant. This toy example highlights that OT losses capture the underlying geometry which distributions are defined. More details concerning the advantages of OT over other methods can found in (Arjovsky al. 2017) (Part 2: Different Distances).
19

---

Published as a conference paper at ICLR 2024
0.5
Pooabiiity
0.4
0.4
0.3
0.1
0.1
0.0
0.0
1-Wasserstein distance 2.0
1-Wasserstein distance 0.5 divergence 0.8959
KL divergence 0.8959
0.6
0.6
0.5
0.5
Probabiity
0.2
0.2
0.1
0.1
Figure 6: Example of distributions defined on five points with different mass values. The 1- Wasserstein distance and the KL divergence computed between the red and the blue distribution for each column.
MODEL ROBUSTNESS AND PATCH POSITION
In this section, we evaluate the robustness of models according to the patch position in images. We consider the same families of models as before. We define nine patch positions and measure the patch transferability when the patch is fixed at of these positions. Figure represents the nine patch positions. We regroup these positions into three categories: Corner, Cross, and Center. We measure the patch transferability for patch of size 40 X 40 (== 3% image size). Results averaged over methods (GAP (Brown et al. 2017). LaVAN (Karmon et al. 2018), 1.2 (Inkawhich et al., 2019) and ours), classes, and categories of patch position. Table 9 reports the patch transferability according to its position. CNNs-vl models are much more biased by the center of images than other network families. The accuracy of CNNs-vI drops by factor 14 % when the patch is moved corners the center of images. This effect is not entirely due to the occluding of the object of interest since the patch very small. Very recent families of networks (CNext and Swin models) are the more balanced networks in using context in images. For these models, the accuracy is nearly the same when the patch is placed in either corners or the center. measure actual efficiency patches and to not occlude the object of interest in the case of large patches, its patches may not be placed in the center of images.
Table Transfer results according to the categories of patch position (Accuracy (%)). Results are averaged over methods, over classes, over patch positions and are for patches of size 40 x 40.
Target
CNNs-v1
CNNs-v2
ENet
CNext
DeiT
Swin
AT
Clean
74.90
77.63
80.15
82.42
77.81
82.43
65.44
Position
Corner
71.07
76.57
78.85
81.97
76.33
81.43
64.24
Cross
67.65
75.36
77.71
81.66
75.66
81.44
62.44
Center
61.52
72.01
74.23
80.72
73.94
80.71
57.06
20

---

Published as a conference paper at ICLR 2024
Corner
Cross
Corner
Cross
Center
Cross
Corner
Cross
Corner
Figure 7: Illustration of the categories of patch positions.
E-CNN-v1 -> CNN-v1
E-CNN-v1 >> CNN-v2
E-CNN-v1 -> ENet
E-CNN-v1 >> CNext
tSuc (96)
12.5
40
10
10.0
LWWN
20
122
# Models
E-CNN-v1 -> DeiT
E-CNN-V1 -> 5win
E-CNN-V1 AT
Whitebox
7.5
5.0
50
0
2
3
Figure 8: Transfer and whitebox results for patches built on an ensemble of models (tSuc (%)). Results are averaged over classes and over patch sizes. Patches are placed randomly in the image but the center of images.
F ENSEMBLE METHODS
Ensemble methods train a single patch across an ensemble of models simultaneously. We determine an attacker building his attack on an ensemble CNN-v1 models can significantly increase its attacking performance on CNext or Swin models. We consider the following ordered list of models E-CNN-v1 = [ResNet50/34/185 vl DenseNet121] in which networks are added to the ensemble in this order. Figure [ plots the targeted success rate (tSuc) ensemble. Even with the largest ensemble of four models, patches failed to significantly increase CNext and Swin models. This result confirms that attacker
function of the number of models in the
their transferability performances expecting to sway all the models uniformly should design his attack on Swin models using our methodology. E shows that the feature point method becomes unstable with the increased number of models in the ensemble.
G TRANSFERABILITY ON ADVERSARIALLY TRAINED MODELS
In this section, we study the robustness of Adversarially Trained (AT) models. We consider two scenarios: when the patch is learned on AT models and when not. To strongly transfer on an AT model, the patch must be designed on an AT model (Table I0.. None of the other source models can show good transferability results when applied AT models. These results suggest that AT models learn different representations than other networks. From Table [] we see that the GAP method (Brown et al.! 2017) and method the best procedures to design a patch target an AT' model.
21

---

Published as a conference paper at ICLR 2024
Table 10: Transfer results between categories of models (tSuc (%)). Results averaged over classes and over patch sizes. Patches designed using method (W)).)
Target
CNNs-v1
CNNs-v2
ENet
CNext
DeiT
Swin
AT
mean / std
Clean
0.1
0.1
0.1
0.1
0.1
0.1
0.1
0.1/0
Source
CNNs-vl
39.65
13.01
8.27
2.44
4.89
3.16
0.82
10.32 12.56
CNNs-v2
19.0
11.35
3.82
4.51
3.74
4.19
0.45
6.72 / 5.86
ENet
35.12
10.45
32.0
2.27
7.8
3.79
3.49
13.56 / 12.94
CNext
3,47
12.2
0.92
25.14
2.04
15.12
0.16
8.44 8.69
DeiT
22.26
11.43
10.18
5.29
39.51
9.25
5.08
14.72 11.43
Swin
20.55
17.89
8.09
17.7
13.55
49.1
0.72
18.23 / 14.09
AT
39.75
10.69
17.35
3.51
19.87
5.31
38.95
19.35 / 13.77
Table 11: Transfer results between categories of models (tSuc (%)). Results are averaged over classes and over patch sizes. Patches are placed randomly in the image
the center of images.
Target
mean / std
CNNs-v1
CNNs-v2
ENet
CNext
DeiT
Swin
AT
Clean
0.1
0.1
0.1
0.1
Method
GAP Brown
2017
43.05
11.67
16.7
3.35
20.09
5.23
39.17
19.98 14.51
LaVAN (Karmnn
2018
37.27
10.94
14.08
3.43
18.18
5.21
29.96
17.018
11.64
L2
(nnkwhhch et a 2019
6.78
1.86
2.23
0.59
4,39
1.1
8.35
3.618 2.77
Tn1 Doan ct al 2022
3.71
1.33
1.41
0.8
2.61
0.85
8.03
2.688 2.39
asper ct 2020
1.38
2.74
0.54
7.55
0.97
13.91
4.78
'1P [Nasoer
35.25
13.75
2.91
17.92
35.17
17.028
12.43
M.11> [Zhap
6.24
5.61
3.45
0.82
1.82
1.12
2.53
3.088 1,98
Ours (SW37
22.52
5.22
8.05
2.17
11.51
3.22
21.57
00.11877799
(W2)
10.69
3.51
38.95
H ROBUSTNESS ACCORDING TO PHYSICAL TRANSFORMATIONS
In this section, we measure the robustness of patches according to physical transformations. We evaluate the L.2 (Inkawhich et al. 2019), our exact Wasserstein (W3) and Sliced-Wasserstenn (SW2)
27500 patches as they are the only to transfer in the easiest scenario, i.e., without patch rota- tion, medium brightness and small distance patch-camera (Section 4.3 of the main article). Patch transferability is measured according to z-axis rotations (rotations in the image plane), variation of light (low and high) and distance between camera and the object (the patch placed near the object). Results are reported in Table 12 and Figure 9 Our patches transfer even in the worst-case scenario (far from the camera or when rotated), while other patches do not. This indicates that our patches may be critical in real-world scenarios. Globally, our method produces patches with better transferability than other methods.
Table 12: Transfer results according to rotations and variation light (tSuc %). Patches are designed to sway networks output the class bird house. Patches are printed and placed in the real-world near a cup. Results are averaged over video frames and over all the networks.
z-axis rotations
Variation of light
Method
-45
0"
45"
Low
High
I.2 (Inkawhich et al. 2019)
0.8
5.7
0.23
4.4
5.7
Ours (SW?) 50
6.1
11.5
6.53
12
11.5
Ours (W?)
7.1
14.8
7.05
12.6
14.8
22

---

Published as a conference paper at ICLR 2024
(W
20
(sW; 1500
mean tSuc (%)
5
0-
30 cm
50 cm
Distance camera-object
Figure 9: Transfer results as a function of the distance camera-object. Patches are designed to sway networks output the class bird house. Patches are printed and placed in the real-world near a cup. Results are averaged over video frames and over all the networks.
ABLATION STUDIES I
In this section, we study the effect of our method hyper-parameters. We solve the exact and the sliced Wasserstein distance for p E {1,2} and report the results Table This Table shows that both values of p lead to the same transferability. To penalize higher feature values, we set the value of p p=2.
We launch the Wasseddt-ssrrteiin distance (SW) for the following number of projections: K (500, 1000, 5000, 10000, 50000). There is no clear advantage to considering many projections (Table The best transferability results are obtained with K 500.
16
We now study the effect of the number of attacked layers (N). Table we report the transferability results according to different numbers of targeted layers. We obtain better results for the exact Wasserstein distance when considering multiple layers. We observe that it helps the optimization to converge to better local minimum, leading stronger patches. For the Sliced-Waad Wasserstein distance, targeting multiple layers seems counterproductive. Table details the result presented in the article the choice of the essential layer target. The last layer of the encoder = (.)) seems essential to model and close the gap between the two distributions and, particularly, for the Sliced- Wasserstein distance.
To evaluate the data dependency of our method, we create different targeted distributions by changing the number of points which compose it (m 1, 2, 10, 100, 300, 600, 900). We launch the optimiza- tion of patches for five different sampling seeds and three different classes. We consider the Swin-T model as the source model. We evaluate patches using the same procedure explained in the main article (Section 4. We report the results of the three runners-up basclines (GAP, LaVAN and TTP). As these methods Figure [T we see that the average targeted success rate (tSuc) increases with respect to the number of target samples. When considering multiple points, our method leads to better transfer results and more stable than the L2-based method (see Bb. Our method performs better than decision-boundary- based methods (GAP, LaVAN and TTP). However, we would like to emphasize that our method requires multiple images of the target class to overcome the limitations of the basdd approach (see Appendix B). This data dependency is a practical limitation of our method. This practical limitation may be simply leveraged by considering the training data of the source model when available.
not consider distributions, they correspond to straight lines in the figure. From
23

---

Published as a conference paper at ICLR 2024
Table 13: Transfer results according the power p (tSuc (%)). Results averaged over classes and
over patch sizes. Patches are designed on Swin-T.
Target CNext
CNNs-v1
CNNs-v2
ENet
DeiT
Swin
mean std
Clean
0.1
0.1
0.1
0.1
0.1
0.1
0.1/0
P
(W2)
25.62
19.88
10.96
18.84
13.28
55.67
24.04 / 14.91
(W))
26.37
20.99
10.86
19.56
13.3
56.98
24.68 / 15.31
(SW 2 2/10000
(1)
27.82
20.22
11.29
18.6
16.66
41,43
22.67 / 9972
(SW) 10000
(1)
28.74
22.72
11.24
19.89
16.07
43.13
23.63 / 10.26
Table 14: Transfer results according to the number of targeted layers (N) (tSuc (%)). Results are averaged over classes and over patch sizes. Patches are designed on the Swin family. Layers , and 2 correspond to the second and third block of Swin models (which are composed by four blocks in total).
Target CNext
CNNs-v1
CNNs-v2
ENet
DeiT
Swin
mean std
Clean
0.1
0.1
0.1
0.1
0.1
0.1
0.1 /
(N)
(1r)
20.55
17.89
8.09
17.7
13.55
49.1
21.14 13.12
(W3)(M)
(1)
24.87
20.38
9,59
19.77
17.77
48.42
23.47 / 12,06
19.14
12.45
7.52
10.56
13.55
24.75
14.66 / 14.66
(1))
25.26
18.7
9.19
17.27
15.32
44.11
21.647 11.11
(M)
(17-2.13
24.22
18.26
8.25
15.27
17.47
19.66 8.14
15.94
10.23
6.35
8.6
12.43
17.35
11.82 / 3,89
Table 15: Transfer results according to targeted layer in the single targeted layer setting (tSuc (%)). Results are averaged over classes and over patch sizes. Patches are designed on the Swin family, Layers ر and 2 correspond to the second and third block of Swin models (which are composed by four blocks in total).
Target CNext
CNNs-v1
CNNs-v2
ENet
DeiT
Swin
mean / std
Clean
0.1
0.1
0.1
0.1
0.1
0.1
0.1 /
(W3)
1--2
17.02
15.03
6.59
14.32
12.55
38.35
17.31 /999
20.55
17.89
8.09
17.7
13.55
49.1
21.14 / 13.12
[J-8
0.3
0.19
0.19
0.14
0.17
0.2
0,2
(2)
1--2
15.39
11.2
5.2
9.08
13.37
20.44
12.45 / /4.81
L 25.26
18.7
9.19
17.27
15.32
44.11
21.64 11.11
J DECISION BOUNDARY-BASED METHODS OVERFITTING
In this section, we conduct an additional experiment to support that decision boundary-based methods learn patch that tends to overfit on the source model classifier. For this purpose, we consider the transfer not between 2 different models but between 2 models sharing the same encoder but different classifiers. We select from the different methods patches trained attack the source model Swin-T (Liu et al.) 2. On top of this Swin-TT encoder, train a new linear classifier from scratch on the ImageNet train set (Deng et al. 2009). This new linear classifier reaches the same level clean accuracy the previous classifier (from Pytorch (Paszke et al. 2019)) while being different. We measured the patch performance when targeting this new network (same encoder, different lincar classifier). As expected, the transferability of decision boundary based patches drops drastically (nearly by half) while our patches transferability remains almost the same.
24

---

Published as a conference paper at ICLR 2024
1-8
1-2
l
x x48
H. W XC
x
# x2C
# W
x x8C
Stage
Stage
Stage
Stage
Lincar Embedding
HxWx3
Patch Partition
Patch Merging
Patch Merrinn
Patch Merging
Swin
Swin
Swin
Images
Transformer Block
Transformer
Transformer Block
Transformer
X2
(a) Architecture
Figure 10: Figure from (Liu et article.
2021 In red are displayed the targeted layers consider in the
25
mean tSuc (%)
(W3) (2)
15
(SW)
GAP
LavAN
10
TTP
2
10
100
300
600
900
targets
Figure 11: Transfer results as a function of the number of targets points supported the target distribution (mean tSuc (%)). Each dotted line correspond to different sampling of points to create the target distribution. The solid line the average of the five dotted lines. Patches are designed on the Swin-T source model. Results are averaged three classes, over patch sizes and over all the targeted networks.
25

---

Published as a conference paper at ICLR 2024
Table 16: Transfer results (tSuc (%), higher is better attack) between categories of models. Results averaged over classes and over patch sizes. Patches are placed randomly the image without object overlapping. Physical transformations (e.g., noise, rotations) are applied to patches. Control stands for inserting real object of the corresponding class as a patch.
Target
mean std
CNNs-v1
CNNs-v2
ENet
CNext
DeiT
Swin
Clean
Method
0.1
0.1
0.1
0.1
0.1
0.1
0.1 /
Control
2.85
0.86
0.54
1.57
0.93
1.39
000.5
CNNs-v1
25.25
6.15
4.73
1.7
5.15
2.61
7.6
8.04
CNNs-v2
16.93
8.67
4.02
4,08
5.77
3.56
7.17
/4.69
(SW2)))
ENet
22.53
5.83
18.8
2.07
8.49
3.03
10.13 /7.8
27500
CNext
3.97
11.62
1.1
29.97
3.14
14.75
10.76
/9.86
DeiT
23.65
12.16
7.27
5.21
32.39
9.35
15.01
/9.77
Swin
25.2
20.21
8.93
19.54
16.16
45.31
22.56
/1..3
CNNs-v1
26.38
6.13
5,59
1,96
5.85
2.8
8.12
8.32
CNNs-v2
16.88
8.82
3.97
3.89
5.8
3.53
7.15
/4.71
(SW2)
ENet
23.56
19.18
2.25
8.55
3.01
10.5
8.00
2/0000
CNext
4.44
12.07
33.22
3.24
15.3
11.57
10.89
DefT
22.77
11.97
5,36
35.25
15.37
100.48
Swin
24.2
17.73
44.53
21.72
/1.16
CNNs-vI
26.4
6.11
5.37
1.83
5.2
2.65
7.93
7.4
CNNs-v2
14.49
8.35
3.73
3.64
5.89
3.24
6.55 /3.96
(1)
ENet
27.44
7.04
19.85
2.16
8.88
3.12
11,42 79.2
2/5000
CNext
4.52
13.79
1.18
31.54
3.18
16.4
11.77
10.45
DeiT
24.14
12.89
8.37
5,02
36.29
9.17
15.98
10,9
Swin
24.02
19.69
9.53
17.97
15.06
44.74
21.83
/11.15
CNNs-vl
25.73
6.25
5.51
1,86
5.75
2.67
7.96
88.11
CNNs-v2
18.38
10.46
4.
4.73
6.15
4.01
7.99
1..14
(SW2)
ENet
24.49
20.26
2.14
8.64
2.98
10.85
/8.52
200000
CNext
2.92
23.33
12.18
8.6 /7.68
DefT
23.87
12.22
4,89
36.3
11.01
Swin
23.68
18.08
8.92
17.95
15.42
44.61
21.44
11..24
CNNs-vi
26.16
6.16
5.4
1.89
5.32
2.7
7.94
7.229
CNNs-v2
13.67
8.71
3.09
4.0
4.67
3.4
6.26
/3.8
SSW2 (1)
ENet
25.66
6.06
20.4
2.11
8.73
2.99
10.99
88.91
200000
CNext
3,06
10.97
0.95
27.34
3.34
16.73
10.4
/9.31
DeiT
23.95
11.84
8.65
4.6
35.72
8,58
15.56
/ 10.86
Swin
25.26
18.7
9.19
17.27
15.32
44.11
21.64
/11.11
CNNs-v1
39.65
13.01
8.27
2.44
4.89
3.16
11.9
12.91
CNNs-v2
19.0
11.35
4.51
3.74
4.19
7.77
5.69
(W3)
ENet
35.12
10.45
32.0
2.27
7.8
3.79
15.24
/13.25
CNext
3.47
12.2
0.92
2.04
15.12
9.82
8.66
DeiT
22.26
11.43
5,29
39.51
9.25
/
Swin
20.55
17.89
8.09
17.7
13.55
49.1
21.14
13.12
Table 17: Transfer results when changing the linear classifier while the encoder remains fixed
(variation of tSuc (%)). Patches are designed to fool the Swin-T model (Pytorch version, encoder and linear classifier). The transferability measured when targeting new network (same encoder, different linear classifier). Results are averaged over classes and over patch sizes. Method
Variation of tSuc (%)
GAP (Brown et al. 2017)
61.4
LaVAN (Karmon et al.
2018
42.6
TTP Naseer al.
2021
-1.8
Ours (SW 2
0.27
Ours W (1)
5.6
26

---

Published as a conference paper at ICLR 2024
K COMPLEMENTARY TABLES
present In this section, we provide additional tables. Table i8 the same as Table pressent in the main paper but results presented for different values of smoothing factors
Table 18: Transfer results robustified models by LGS defense Naseer al. Patches are designed on Swin models.
2019b)
(tSuc (%)
Target ENet
CNNs-v1
CNNs-v2
CNext
DeiT
Swin
mean
Clean
0.1
0.1
0.1
0.1
0..
0.1
0.170
GAP Brown al. 2017
0.72
0,87
0.35
0.78
1.13
2.34
1.03 //03/00 0.63
LaVAN Karmon al 2018
0.56
0.69
0.3
0.69
0.82
2.65
0,95 / 0.78
L2 Inkawhich el a.. 2019
4.79
6,44
1.72
7.79
4.79
13.85
6,56/ 3.75
TD' Dnan ef al 2022
0.84
0.59
0.52
0.53
0.7
0.85
0.67 / 0.13
Casper et al
20z2
0.37
0.4
0.2
0.32
0.25
0.59
0.36/ 0.36/0.13
TIP
Nascer ot al. 2021
0.68
0.77
0.28
0.68
0.76
1.98
0.86 0.53
M3D ZZhuo Zhhoo et al. 2025
0.83
0.81
0.36
0.77
1.17
1.17
0.85 0.27
Ours (SWA)) 2/500
10.56
11.86
3.81
18.9
11.67
31.68
14.75 / 8.75
Ours (W3)
13.23
4.37
21.42
13.84
32.08
16.39
Clean
0.1
0.1
0.1
0.1
0.1
0.1
0.1 /
GAP Brown al. 2017
0,81
0.32
0.68
1.76
0.86 0.45
LaVAN (Karmon at.
2018
0.45
0.61
0.26
0.55
0.72
1.72
0.72 0,47
(Inkawhich al. 2019
5.72
11.26
5.57
ThT Doan al == 1.9
0.82
0.61
0,51
0.52
0.65
Casper el
0.32
0.33
0.19
0.25
0.3
TIP
2021
0.56
0.73
0.24
0.59
0.66
1.34
0.69 0.33
M3D 7Zhao 2026)
0.68
0.7
0.31
0.7
1.03
1.01
0.74 0.24
Ours (SW 27600
8.56
10.49
3.27
15.93
10.39
25.96
12,43 7.1
Ours (W3)
10.95
11.98
3.78
18.37
12.35
27.07
14.08 7.19
Clean
0.1
0.1
0.1
0.1
0.1
0.1
0.170
GAP Brown al.
2017
0.52
0.74
0.29
0.58
0.87
1.34
0
LaVAN Karmon
2011
0.38
0.55
0.24
0.47
0.64
1.19
0.58 /
L2 Ink inkawhich et al. 2019
3,35
4,95
1.35
5.46
3.74
8.93
4.63 2.32
Th Doan
20M
0.8
0.64
0.52
0.52
0.58
0.8
0.64 0.12
Casper el 20z
0.47
0.69
0.22
0.53
0.57
0.92
0.26 0,08
TTP Naseer of al 202
0.47
0.69
0.22
0.53
0.57
0.92
0.57 0.21
M3D Zhao et 2023
0.55
0.59
0.27
0.64
0.9
0.9
0.64 0.22
(SW2
6.76
2.81
13.1
20.69
10.28 5.59
Ours (W3)
8.85
10.61
3.27
15.28
10.86
22.28
11.86 5.85
27

---

Published as a conference paper at ICLR 2024
L
PRINTABLE PATCHES
Swin-T
Swin-S Australian terrier
Swin-B
Ranana
Golden
Toaster
Volcano
Figure 12: Printable patches designed on Swin models with our distribution-oriented method.
28


Reviews:
[{"review_id": "tmBTO1eGIb", "reviewer": "Reviewer_edp6", "summary": "The authors introduce a framework based on optimal transport for crafting patch attacks that are highly transferable to unknown networks. This framework is based on the idea of attacking feature distributions, which is claimed to be less model-dependent than relying on decision boundaries and more robust to optimization artifacts than the feature point method.", "questions": "1. Compare with other distribution-matching methods, such as algorithms other than OT (Optimal Transport).\n2. Revise the description concerning model dependency.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "I believe the main strength of this article is that it provides an implementation based on optimal transport for distribution matching-based transfer methods.", "weaknesses": "I believe the main drawback of this article is that it does not discuss other methods based on distribution matching.\nThe article's main claim is somewhat ambiguous; is it advocating that distribution-matching methods are superior, or that methods based on optimal transport are superior? A comparison with other distribution-matching methods would make the paper more compelling.\nAnother issue is that the article's statements are not sufficiently rigorous. The statements seem to suggest that the method presented can relax dependency on specific models. However, in reality, the method still relies on a particular model, even though it exhibits better transfer performance in experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kVmwUXZDde", "reviewer": "Reviewer_Ctvh", "summary": "This paper presents a distribution based approach for adversarial patch attacks. Instead of optimizing the patch to cross decision boundaries or converge to a specific point, the proposed method uses optimal transport to push the feature distribution of attacked images towards a known distribution. The paper demonstrates that this distribution-oriented approach leads to better transferable patches that can influence multiple models and can be effective in physical world experiments. The paper provides comprehensive digital, hybrid, and physical experiments to validate the effectiveness and transferability of the proposed method.", "questions": "see the weakness", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper introduces a novel approach for designing adversarial patch attacks based on optimal transport and distribution, which is a unique and innovative idea.\n\nThe author conducted comprehensive experiments, including in virtual, hybrid, and physical environments. The results demonstrate that the method outperforms the baseline in terms of transferability.\n\nThe paper is well-structured and provides a thorough review of APA, giving readers a clear understanding of the background and related work.", "weaknesses": "Limited technical contribution: While I acknowledge that optimizing patches based on optimal transport theory has some novelty, the technical contribution of this paper seems rather minimal. I believe that the method's use of Wasserstein and its variants as a loss metric is a straightforward application of existing techniques. Additionally, the usage of EOT and TV Loss is common. Thus, I'm concerned that the technical contribution of this paper might be too weak for ICLR.\n\nThe experimental results lack discussions and explanations: For instance, why is it that in the experiments, the Sliced-Wasserstein distance (SW) always underperforms compared to the Wasserstein ones (W)? What causes this? I'm concerned about whether the settings for the two proposed methods are fair. Moreover, I believe there needs to be a more comprehensive ablation study and explanation regarding the choice of target layers. The authors only investigated the impact of the last three feature layers. Why did adding J-2 significantly affect performance?\n\nPhysical experiments are somewhat weak: I believe this paper fails to demonstrate the superiority of its method in physical experiments, especially since the author claims that the method provides better transferability but only compares it to a weaker baseline (L2). Additionally, could you investigate the robustness of your method when facing physical transformations? For instance, under variations in camera angles and distances.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HWV5jkhuQI", "reviewer": "Reviewer_dpUD", "summary": "This paper considers the setting of a black-box transfer attack on image classification where an attacker does not know the target model. Instead of forcing corrupted image representations to cross the nearest decision boundaries or converge to a particular point, this paper proposes a distribution-oriented approach and relies on optimal transport to push the feature distribution of attacked images towards an already modeled distribution. This work shows that the proposed new distribution-oriented approach can lead to better transferable patches.", "questions": "This work designs a new patch attack with optimal transport to narrow the distribution gap in the generation of adversarial patch and achieve the superior performance with empirical evidence. Yet, it would be better for this work to provide more details and explanations.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This work introduces a new framework based on optimal transport for creating patch attacks that are highly transferable to unknown networks. This framework is based on the idea of attacking feature distributions, which is less model-dependent than relying on decision boundaries and more robust to optimization artifacts than the feature point method.\n2. This work shows that the proposed attack works for the most extensive spectrum of deep networks considered in the patch attack literature, such as various versions of Convolutional Neural Networks, Transformers, and adversarially trained models. The proposed method also shows transferability superiority through extensive experiments.", "weaknesses": "1. For qualitative experiments, this paper gives some results by selecting three objects present in ImageNet-1K (banana, cup, keyboard) and recording videos of them when one patch is placed or not next to the object. Yet, the performance of model in the physical world may be affected by the different angles or intensities of light. Thus, it would be more meaningful if this paper could provide the performances of the proposed approach with the change of angles or lights for the qualitative evaluation.\n2. For digital experiments, the authors select from the previously defined families the following models and measure the attacking transferability when the resulting patch is used to fool the remaining models. The results show that the proposed approach can generate the stronger patch attack than its counterparts. Yet, the visual perception to human vision should also be considered. It would be more convincing if the image with the generated adversarial patch with the proposed method and its counterparts can be evaluated with the PSNR or SSIM as the evaluation metrics for the degree of recognition for human vision.\n3. According to the table of computation time in the Appendix of this work, the proposed approach with sliced version SW and the normal W can achieve comparable efficiency with the previous methods. Note that the computation of optimal transport may need more time than those methods without OT, the authors may explain why the approach with OT to generate the adversarial patch can achieve the efficiency without the drop of performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ycKcOss1Ri", "reviewer": "Reviewer_MKWR", "summary": "This paper leverages the p-Wasserstein distance and the sliced-Wasserstein distance between the corrupted image distribution and the target distribution to generate transferable adversarial patches for attacking ViTs and CNNs. In this way, the proposed method pushes the corrupted feature distribution towards a target feature distribution. The authors attack a diverse set of victim models, including CNNs, ViTs, and adversarially trained models.", "questions": "How to obtain the target distribution $\\mathcal{v}_y^{(l)}$.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper is well-written.   \nThe authors conduct extensive experiments to demonstrate the effectiveness of the proposed method.", "weaknesses": "The authors utilize the Wasserstein distance between the corrupted and the target distributions to optimize adversarial patches. This method is too simple, and there is no theoretical contribution in this paper. Besides, this paper should discuss why are distribution-based methods better than decision boundaries-based methods, feature point-based methods, and generation-based methods (e.g.TTP). Why is the Wasserstein distance suitable in optimizing adversarial patches compared to KL?\nAdditionally, the authors convert the generative methods (TTP) to iterative ones. Thus, the iterative targeted attacks should be also compared [1][2].    \nIn summary, I think the contribution of this paper is under the acceptance threshold.\n\n\n[1] Zhengyu Zhao, et al. On success and simplicity: A second look at transferable targeted attacks. Advances in Neural Information Processing Systems, 34:6115–6128, 2021.  \n[2] Wei, Zhipeng, et al. Enhancing the Self-Universality for Transferable Targeted Attacks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wBeGSM9dmC", "reviewer": "Reviewer_bKEx", "summary": "The paper is interested in generating an adversarial patch, i.e., a small patch placed in a scene that changes the prediction of a neural network. The paper introduces an approach based on optimal transport to generate a patch, such that the estimated feature distribution of corrupted source images becomes close to the distribution of features from the target class, using either the Wasserstein and Sliced-Wasserstein distance. The black-box transferability of the attack is compared to state-of-the-art attacks for the Image Classification task on ImageNet-1K, on a broad set of networks. Qualitative results with a printed patch are also presented to demonstrate the real-world applicability of the attack.", "questions": "I would like to see a study on the strength of the proposed attack, depending on the number of target images used to approximate the target distribution. How many target images are used to generate the patches in the experiments ? The paper mentions 40000 images to train the patches, but I assume that they are also split into source and target images.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- **Broad Model Evaluation:** The paper offers a comprehensive evaluation of attacked models, ranging from classical Convolutional Neural Networks (CNNs) to the more contemporary Vision Transformers. This large coverage is important, as it shows the adaptability and effectiveness of the proposed attack across diverse neural network architectures. Furthermore, as mentioned in the paper, recent networks and training recipes are both naturally more robust than older CNNs, it is then fundamental to evaluate against them as well. From the results, we can see that attacks optimized for a specific category of models does not transfer as efficiently to all other categories.\n- **High Transferability Results:** The proposed approach demonstrates strong transferability results, outperforming in general both state-of-the-art patch and non-patch attacks. Notably, the approach accomplishes this while maintaining a similar level of computational efficiency as other existing methods.\n- **Clarity and Reproducibility:** The paper is well written and easy to follow, with a clear description of the proposed approach and of the experiments. Additionally, the code is available as supplementary material which helps reproducibility.", "weaknesses": "- **Data Dependency:** One limitation currently not discussed in the paper is the target data requirements. The proposed attack method relies on a substantial number of target examples to generate the patch to accurately approximate the target class distribution. This means that the attack has access to a lot more information compared to attacks that matches a single feature point. This data dependency could be an important limitation in practice.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)