Content-based Unrestricted Adversarial Attack
Authors: Zhaoyu Chen
Bo Li
Shuang Wu
Kaixun Jiang
Shouhong Ding
Wenqiang Zhang
Venue: NeurIPS 2023 poster (2023)
Keywords: unrestricted attack
adversarial example
diffusion model
black-box attack
adversarial transferability
TL;DR: We propose a attack framework called Content-based Unrestricted Adversarial Attack, which utilizes a low-dimensional manifold to optimize images along adversarial direction, resulting in natural and highly transferable adversarial examples.

Abstract: Unrestricted adversarial attacks typically manipulate the semantic content of an image (e.g., color or texture) to create adversarial examples that are both effective and photorealistic, demonstrating their ability to deceive human perception and deep neural networks with stealth and success. However, current works usually sacrifice unrestricted degrees and subjectively select some image content to guarantee the photorealism of unrestricted adversarial examples, which limits its attack performance. To ensure the photorealism of adversarial examples and boost attack performance, we propose a novel unrestricted attack framework called Content-based Unrestricted Adversarial Attack. By leveraging a low-dimensional manifold that represents natural images, we map the images onto the manifold and optimize them along its adversarial direction. Therefore, within this framework, we implement Adversarial Content Attack (ACA) based on Stable Diffusion and can generate high transferable unrestricted adversarial examples with various adversarial contents. Extensive experimentation and visualization demonstrate the efficacy of ACA, particularly in surpassing state-of-the-art attacks by an average of 13.3-50.4\% and 16.8-48.0\% in normally trained models and defense methods, respectively.

Content-based Unrestricted Adversarial Attack
Zhaoyu Chen 1 Bo Li Shuang Wu", Kaixun Jiang', Shouhong Ding", Wenqiang Zhang! 13..
'Academy for Engineering and Technology, Fudan University
Youtu Lab, Tencent School of Computer Science, Fudan University (zhaoyuchen20, wqzhang) denfuudnndd.. edu. edu.cn, cn,
ajumagiclibo@@mmi.. com, {calvinwu, ericshdingi@teenent.comm
Abstract
Unrestricted adversarial attacks typically manipulate the semantic content of an image (e.g., color or texture) create adversarial examples that are both effec- tive and photorealistic, demonstrating their ability to deceive human perception and deep neural networks with stealth and success. However, current works usu- ally sacrifice unrestricted degrees and subjectively select some image content guarantee the photorealism of unrestricted adversarial examples, which limits its attack performance. To ensure the photorealism of adversarial examples and boost attack performance, we propose novel unrestricted attack framework called Content-basse Unrestricted Adversarial Attack. By leveraging low-diment low-dmensional manifold that represents natural images, we map the images onto the manifold and optimize them along its adversarial direction. Therefore, within this framework, we implement Adversarial Content Attack (ACA) based on Stable Diffusion and generate high transferable unrestricted adversarial examples with various ad- versarial contents. Extensive experimentation and visualization demonstrate the efficacy of ACA, particularly in surpassing state-of-the-arr attacks by average of 13.3-50.4% and 16.8-48.0% in normally trained models and defense methods, respectively.
Introduction
Deep neural networks (DNNs) have significantly progressed in many tasks [18, 7J. However, with the rise of adversarial examples, the robustness of DNNs has been dramatically challenged [16]. Adversarial examples show the vulnerability of DNNs and expose security vulnerabilities in many security-sense senssttvsee applications. To avoid potential risks and further research the robustness of DNNs, it is of great value expose many "blind spots" of DNNs possible at the current research stage.
Nowadays, various methods are proposed generate adversarial examples [4, 5, 6]. To maintain human visual imperceptibilityy and images' photorealism, adversarial perturbations within the con- straint of lp norm are generated by these adversarial attacks. However, it is well known that the adversarial examples generated under 4, norm have obvious limitations: firstly, they are not ideal in terms of perceptual similarity and are still easily perceptible by humans [24, 23, 62]; secondly, these adversarial perturbations in the natural world, resulting in the adversarial examples being different from the hard examples that appear in the real world [64]. In addition, current defense methods against Lp norm adversarial examples overestimate their abilities, known as the Dunning-Kruun effect [28]. can effectively defend against I norm adversarial examples but robust enough when facing new and unknown attacks [25]. Therefore, unrestricted adversarial attacks are beginning to emerge, using unrestricted practically meaningful.
not natural enough and have inevitable domain shift with the noise
but natural changes replace small i, norm perturbations, which
This work done when Zhaoyu Chen intern Youtu Lab, Tencent. indicates corresponding authors.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Existing unrestricted adversarial attacks generate adversarial examples based on image content such as shape, texture, and color. Shape-based unrestricted attacks [56, 1] iteratively apply small deformations to the image through a gradient descent step. Then, texture-based unrestricted attacks [2, 40] are introduced, which manipulate an image's general attributes (texture or style) to generate adversarial examples. However, texture-based -aaeed attacks result in unnatural results and have low adversarial transferability. Researchers then discover that manipulating pixel values along dimensions generates more natural adversarial examples, leading to the rise of color-based unrestricted attacks [20, 30, 2, 61, 47, 60J. Nonetheless, color-based unrestricted attacks tend to compromise flexibility in unconstrained settings to guarantee the photorealism of adversarial examples. They achieved either through reliance on subjective intuition and objective metrics or by implementing minor modifications, thereby constraining their potential for adversarial transferability.
Considering the aforementioned reasons, we argue that an ideal unrestricted attack should meet three criteria: i) it needs to maintain human visual imperceptibility and the photorealism of the images; ii) the attack content should be diverse, allowing for unrestricted modifications of image contents such as texture and color, while ensuring semantic consistency; iii) the adversarial examples should have high attack performance so that they can transfer between different models. However, there still a substantial disparity between the current and ideal attacks.
To address this gap. we propose a novel unrestricted attack framework called Content-based Unre- stricted Adversarial Attack. Firstly, we consider mapping images onto a www-dimessional manifold. This cow-dimensional manifold is represented by generative model and expressed as a latent space. This generative model is trained on millions of natural images, possessing two characteristics: i) sufficient capacity to ensure the photorealism of generated images; ii) well-alignment of image contents with latent space ensures a diversity of content. Subsequently, more generalized images can be generated by walking along the low-dimensional manifold. Optimizing the adversarial objective on this latent space allows us to achieve more diverse adversarial contents. In this paper, we propose Adversarial Content Attack (ACA) utilizing the diffusion model as a low-dimensional manifold. Specifically, we employ Image Latent Mapping (ILM) to map images onto the latent space, and utilize Adversarial Latent Optimization (ALO) to optimize the latents, thereby generating unrestricted adversarial examples with high transferability. In conclusion, our main contributions are:
We propose novel attack framework called Content-based Unrestricted Adversarial Attack, which utilizes high-capacity and well-aligned low-dimensional manifolds to generate adversarial examples that are more diverse and natural in content.
We achieve an unrestricted content attack, known as the Adversarial Content Attack. By utilizing Image Latent Mapping and Adversarial Latent Optimization techniques, we optimize latents in diffusion model, generating high transferable unrestricted adversarial examples.
The effectiveness of our attack has been validated through experimentation and visualization. Notably, we have achieved a significant improvement of 13.3~550.4% over state-of-the-at attacks in terms of adversarial transferability.
2 Background and Preliminary
Problem Definition. For a deep learning classifier F-(-) with parameters 0, we denote the clean image as x and the corresponding true label as y. Formally, unrestricted adversarial attacks aim to create imperceptible adversarial perturbations (such as image distortions, texture or color modifications, etc.) for a given input to generate adversarial example Fadv that can mislead the classifier F-):):
max L(Fo (Fadv), w), Radv
s.t. Fade is natural,
(1)
where L(-) is the loss function. Because existing unrestricted attacks are limited by their attack contents, it prevents them from generating sufficiently natural adversarial examples and restricts their attack performance on different models. We hope that unrestricted adversarial examples are more natural and possess higher transferability. Therefore, we consider a more challenging and practical black-box setting to evaluate the attack performance of unrestricted adversarial examples. In contrast to the white-box setting, the black-box setting has no access to any information about the target model (i.e., architectures, network weights, and gradients). It can only generate adversarial examples by using a substitute model F+)+) and exploiting their transferability to fool the target model Fo(-).

---

Content-based Unrestricted Adversarial Attack. Existing unconstrained attacks tend to modify fixed content in images, such as textures colors, compromising flexibility in unconstrained settings to ensure the photorealism of adversarial examples. For instance, ColorFool [47] manually selects the parts of the image that are sensitive to human perception in order to modify the colors, and its effectiveness is greatly influenced by human intuition. Natural Color Fool [60], utilizes ADE20K [63] to construct the distribution of color distributions, resulting in its performance being restricted by the selected dataset. These methods subjectively choose the content to be modified for minor modifications, but this sacrifices the flexibility under unrestricted settings and limits the emergence of more "unrestricted" attacks. Taking this into consideration, contemplate whether it is possible to achieve an unrestricted adversarial attack that can adaptively modify the content of an image while ensuring the semantic consistency of the image.
ideal unrestricted attack should ensure the
photorealism of adversarial examples, pos- sess diverse adversarial contents, and exhibit potential attack performance. To address this gap, we propose a novel unrestricted at- tack framework called Content-based Un- restricted Adversarial Attack. Within this framework, assume that natural images can be mapped onto a low-dimensional man- ifold by generative model. As this low- dimensional manifold well-trained on nat- ural images, it naturally ensures the photo- realism of the images and possesses the rich content present natural images. Once we map an image onto -ww-dimesiional mani- fold, moving it along the adversarial direction on the manifold yields an unrestricted adver- sarial example. We argue that such frame- work is closer to the ideal of unrestricted ad- inherently guarantees
the Manriolé of Natural Images
Advensnaal Garteer
Projectod Grallint
Shaps Sub-manifrid
Texture Sah-maniiold
Coler Sab-manifold
Adversarial examples are generated along the adversarial direction the low-dimensional man- ifold of natural images. This manifold represents many contents of natural images, so the generated unrestricted adversarial examples combine multiple adversarial contents (shape, texture and color).
Figure 1:
versarial attacks, the photorealism of adversarial examples and rich image content. Moreover, since the clas- sifier itself fits the distribution of this low-dimensional manifold, adversarial examples generated along the manifold have more adversarial contents and the potential for strong attack performance, as shown in Figure 1.
Naturally, selecting a low- diwdeemnsiioall manifold represented by a generative model necessitates careful consideration. There are two characteristics taken into account: i) sufficient capacity to ensure photorealism in the generated images; and ii) well-alignment ensures that image attributes are aligned with the latent space, thereby promoting diversity in content generation. Recently, diffusion models have emerged as leading approach for generating high-quality images across varied datasets, frequently outperforming GANs [9]. However, several large-scale text-to-image diffusion models, including Imagen [44], DALL-E2 [41], and Stable Diffusion [42], have only recently come to the fore, exhibiting unparalleled semantic generation capabilities. Considering the trade-off between computational cost and high-fidelity image generation, we select Stable Diffusion as the fow-dimensional manifold in this paper. is based on prompt input and is capable of generating highly realistic natural images that conform to the semantics the prompts.
3 Adversarial Content Attack
Based on the aforementioned framework and the full utilization of the diffusion model's 's capability, we achieve the unrestricted content-based attack known as Adversarial Content Attack (ACA). as shown in Figure 2. Specifically, we first employ Image Latent Mapping (ILM) to map images onto the latent space represented by this low-dimensional manifold. Subsequently, we introduce an Adversarial Latent Optimization (ALO) technique that moves the latent representations of images along the adversarial direction on the manifold. Finally, based on iterative optimization, ACA can generate highly transferable unrestricted adversarial examples that appear quite natural. The
3

---

Clean image 4)
Latent Space
Condition
Fire Engine
Image Latent Mapping
Nuil lest embollding [
Inversion sampling
Pronupé: lege five track with in american flg on
Classifier
QKV
QKV
OKY
At
baraiven
Creponia and piocrr asage
Grod
DDIM sampling
Truck
Tow
Adversorial
Grad
Grad
Ship
Adversarial Latent Optimization
Grad
vF--)
ecample
Text embedding
Figure 2: Pipeline of Adversarial Content Attack. First, we use Image Latent Mapping to map images into latent space. Next, Adversarial Latent Optimization is used generate adversarial examples. Eventually, the generated adversarial examples can fool the target classifier.
algorithm for ACA is presented in Algorithm 1, and we further combine the diffusion model to design the corresponding mapping and optimization methods.
3.1 Image Latent Mapping
For the diffusion model, the easiest image mapping is the inverse process of DDIM sampling [9, 48] with the condition embedding C = (P(P) of prompts P. based on the assumption that the ordinary differential equation (ODE) process be reversed in the limit of small steps:
2++1
at+1 az
24 + vautily
1
1
V
I
-1 1) c0(
(2)
ax+1
at
where 20 is the given real image, a schedule (.....,,TI E (0, 1) and a, = (1 3:). In general, this process is the reverse direction of the denoising process (#0 + zT instead of 27 - zo), which can map the image 2u to 2r in the latent space. Image prompts are automatically generated using image caption models (e.g., BLIP v2 [31]). For simplicity, the encoding of the VAE is ignored.
Text-to-iman image synthesis usually emphasizes the effect of the prompt. Therefore, classifier-frre guidance technique [19] is proposed. Its prediction is also performed unconditionally, which is then extrapolated with the conditioned prediction. Given
the guidance scale parameter and Ø = (""") as the embedding of a null text, the classifier-free guidance prediction is expressed by:
(z C,C, Ø) = c0 C, + (1 w) eg c0 Ø),
where to = 7.5 is the default value for Stable Diffusion. However, since the noise is predicted by the model eg in the inverse process of DDIM sampling, slight error is incorporated in every step. Due to the existence of a large guidance scale parameter w in the classifier-free guidance technique, slight errors are amplified and lead to cumulative errors. Consequently, executing the inverse process of DDIM sampling with classifier-free guidance not only disrupts the Gaussian distribution of noises but also induces visual artifacts of unreality [37].
To mitigate cumulative errors, we follow [37] and optimize a null text embedding 2, for each timestamp First, the inverse process of DDIM sampling with w outputs series of consecutive latents { ) where = = zn- Then, we conduct the following optimization with w = 7.5 (the default value for Stable Diffusion) and žΤ 2, during N iterations for the timestamps (T }}:
min ||اا_1 24- 2 C. 2,
(4)
2イ-1 21-1(E,,,,,0))
10-- at
+ Var-1
-
- -
1
V 04 -1)
-
(0( (5)
( 14-1
At the end of each step, we update _;- to given image the low-dimensional manifold, consisting of the noise 27, the null text embedding Ø, and the text embedding C = (P)). Note that compared to other strategies [12, 43, 26, 54]. the current strategy is simple and effective and does not require fine-tuning to obtain high-quality image reconstruction. Next, exploit this latent to generate unrestricted adversarial examples.
Finally, we obtain the latent of the

---

3.2 Adversarial Latent Optimization
In this section, we propose an optimization method for latents to maximize the attack performance on unrestricted adversarial examples. In the latent space of given image after ILM, the null text embedding 2, ensures the quality of the reconstructed image, while the text embedding C ensures the semantic information of the image. Therefore, optimizing both embeddings may not be ideal. Considering that the noise 2T largely represents the image's s information in the latent space, we choose to optimize it instead. However, this optimization is still challenged by complex gradient calculations and the overflow of the value range.
Based on the latents generated by ILM, we define the denoising process of diffusion models as S2(-) through Equation 5, and it involves T iterations:
0(2r, T,C, {2, ) = 20 (.
,,C,)))),..,
(6)
Therefore, the reconstructed image is denoted as 20 = Q(= T, T,C. (m.)). The computational process of VAE is disregarded herein, as is differentiable. Combining Equation 7, our adversarial objective optimization is expressed by:
max C (Fe (30), 8) s.t. ||0||= kk Q22 + 8, 6,T,C, (2,)) and 2o is natural, (7)
where 5 is the adversarial perturbation on the latent space. Our loss function consists of two parts: i) cross-entropy loss Lee, which mainly guides adversarial examples toward misclassification. fi) mean square error loss Lmoe mainly guides the generated adversarial examples to be as close as possible to clean images on la distance. Therefore, the total loss function L expressed as:
[(Fo(Zo), y. zu) Lee (F(())).)) y)) - 3 -mmee 2u).
(8)
where is 0.1 in this paper. The loss function C aims to maximize the cross-entropy loss and minimize the l2 distance between the adversarial example žg and the clean image 2o-
To ensure the consistency of 2g and E0, we assume that ō does not change the consistency when d is extremely small, i.e., | S K. The crux pertains to determining the optimal 5 that yields the maximum classification loss. Analogous to conventional adversarial attacks, we employ gradient- based techniques to estimate 8 through: 5 = v/V 2y L (Fo (Eo), v), where denotes the magnitude of perturbations that occur the direction of the gradient. To expand V (Fo(z)), y) by the chain rule, we can have these derivative terms as follows:
Oc
0%
dzi
02T-1
Ve, Le (Fo
Ozn
Ozi
8z2
02T
Skip Gradient. After observing the items, we find that although each item is differentiable, it is not feasible derive the entire calculation graph. First, we analyze the term ac which represents the 3% derivative of the classifier with respect to the reconstructed image 20 and provides the adversarial gradient direction. Then, for 02
ås each calculation of the derivative represents the calculation of backpropagation. Furthermore, a complete denoising process accumulates T calculation graphs. resulting in memory overflow (similar phenomena are also found in [45]). Therefore, the gradient of the denoising process cannot be directly calculated.
Fortunately, we propose skip gradient to approximate dzr Uz the diffusion process, the denoising process aims to eliminate the Gaussian noise added in DDIM sampling [48, 9, 92.. DDIM samples 2, at any arbitrary time step in a closed form using reparame- terization trick:
021
Bar-1
Recalling
d2y
Varza + v1 are, N00,
(10)
Consequently, we perform a manipulation by rearranging Equation 10 to obtain 20 Rt
V /Logg g. Hence, we further obtain
dze = In Stable Diffusion, timestep is at most 1000, so 0z, var
0sa lim (-+0000 dz. 1-1000 Vau 2u 14.58. In summary, Dec can be re-expressed as V,y C (Fo( v) o In summary, skip gradients approximate the
lim
8sc can be regarded as a constant p and Equation 9
gradients of the denoising process while reducing the computation and memory usage.
Differentiable Boundary Processing. Since the diffusion model does not explicitly constrain the value range of %o the modification of 2T may cause the value range to be exceeded. So we introduce
5

---

Algorithm Adversarial Content Attack
Input:
imput image 50 with the label y, a text embedding C (P)), classifier Fo()). DDIM steps T. image mapping iteration N,, attack iterations Na. and momentum factor
1: Calculate latents {=0 ..: 273 using Equation over Fu with
2: Initialize = 7.5. 2T + 27, Ø t "(""),"" + 0.90 t 0
// Image Latent Mapping
4: for = T.T ..... do
5: for j = ,...,, N; do
6:
23 t- Ø3 <V 2, ||=211111111 - - -
7: end for
8: 2t-1 2 t=,, D.), Φt-1 t 2t
9: end for
10: // Advversarial Latent Optimization
11: for = =I....,M Na do
12: 2 + Ωt (2 5g T, (R,))))
13: 9k p - V 2 ((Fooo(oa))))
14:
Ōz + II, (6x
+ (m))
end for
16: 20 + e (22 (oT + 6N,, T, (2))71)
Output: The unrestricted adversarial example T0-
differentiable boundary processing e(-) to solve this problem. e(-) constrains the values outside [0, 1] to the range of [0, 1]. The mathematical expression of DPB is follows:
lanh(1000:))/10000
0,
②(x) =
0 < x < 1,
(11)
tanh(1000(2: 1)) 100001
1.
Next, define II the projection of the adversarial perturbation ó onto k-ball. We introduce momentum and express the optimization adversarial latents as:
V L (Fo ((e (o)), (o).))))))))) y)) Ile - μ. 91-1 + | C (Fo (el 50). v))
(12)
6. 11, (8 + 7) sign(gx)).
(13)
In general, Adversarial Latent Optimization (ALO) employs skip gradient determine the gradient of the denoising process, and integrates differentiable boundary processing to regulate the value range of adversarial examples. and finally performs iterative optimization according to the gradient.
Combined with Image Latent Mapping, Adversarial Content Attack is illustrated in Algorithm I.
Experiments
4.1 Experimental Setup
Datasets. Our experiments are conducted on the ImageNet-compatible Dataset [29]. The dataset consists of 1,000 images from ImageNet's validation set [8], and widely used in [10, 13, 58, 60].
Attack Evaluation. We choose SAE [20], ADer [1], ReColorAdy [30], cAdv [2], tAdv [2], ACE [61], ColorFool [47], NCF [60] comparison methods of Adversarial Content Attack (ACA). The parameters for these unrestricted attacks follow the corresponding default settings. Our attack evaluation metric is the attack success rate (ASR, %), which is the percentage of misclassified images.
Models. To evaluate the adversarial robustness of network architectures, we select convolutional neural networks (CNNs) and vision transformers (ViTs) as the attacked models, respectively. For CNNs, we choose normally trained MoblieNet-V2 (MN-v2) [46], Inception-v3 (Ine-v3) [50], ResNet- 50 (RN-50) and ResNet- 152 (RN-152) [18), Densenet- 161 (Dense- [22], and EfficientNerbb7 (EF. b7) [52]. For ViTs, we consider normally trained MoblieViT (MTDViT-s) [35], Vision Transformer (ViT-B) [1]], Swin Transformer (Swin-B) [34], and Pyramid Vision Transformer (PVT-v2) [55].

---

Table 1: Performance comparison of adversarial transferability on normally trained CNNs and ViTs.
We report attack success rates (%) of each method ("*" means white-bot fhite-box attack results).
Models
Surrogure Model
Avg.
Attack
CNNs
Transformers
ASR (%)
MN-v2
Inc-v3
RN-50
Dense-161
RN-152
EF-b7
MobViT-s
VTT-B
Swin-B
PVT-v2
Cleun
12.1
4.8
7.0
6.3
5.6
8.7
8.9
3.5
3.6
6.83
ILM
13.5
5.5
8.0
6.3
5.9
8.3
8.3
9.0
4.8
4.0
7.36
SAE
60.2
21.2
54.6
42.7
44,9
30.2
82.5
38.6
21.1
20.2
37.08
ADef
14.5
9.0
8.0
7.1
9.8
80.8
9.7
4.6
8.27
ReColorAdy
37.4
14.7
26.7
22.4
21.0
20.8
96.1
21.5
16.3
16.7
21.94
13.2
28.2
22.0
30.21
MobViT-s
tAdv
33.6
18.8
22.1
18.7
18.7
15.8
97.4
15.3
11.2
13.7
18.66
ACE
30.7
9.7
20.3
16.3
14.4
13.8
99.24
16.5
5.8
14.92
ColorFool
47.1
12.0
40.0
28.1
30.7
19.3
81.7*
24.3
10.0
24.58
NCF
67.7
31.2
60.3
41.8
52.2
32.2
74.5
39.1
20.8
23.1
40.93
(Ours)
66.2
58.1
55.9
55.1
56.90
90.8*
53.2
41.9
26.9
18.3
ADe
56.6*
7.6
8.4
7.7
7.1
10.9
11.7
9.5
4.5
4,5
7.99
ReColorAdv
97.7*
18.6
33.7
24.7
26.4
20.7
31.8
17.7
12.2
12.6
22.04
cAdv
96.6*
26.8
39.6
33.9
29.9
32.7
41.9
33.1
20.6
19.7
30,91
MN-v2
tAdv
99,9*
27.2
31.5
24.3
24.5
22.4
40.5
16.1
15.9
15.1
24.17
ACE
99.1*
17.9
12.6
11.7
5.6
11.50
ColorFool
93.3*
9.5
25.7
15.3
15.4
13.4
15.7
14.2
5.9
6.4
13.50
93.2
56.3
20.6
40.28
ACA (Ours)
93.1*
56.8
62.6
55.7
56.0
51.0
59.6
48.7
48.6
50.4
54.38
SAE
63.2
25.9
88.0
41.9
46.5
28.8
45.9
35.3
20.3
19.6
36.38
ADer
13.5
7.7
55,7*
8.4
7.8
11.4
12.3
9.2
4.6
4.9
9.09
ReColorAdy
40.6
28.3
33.3
13.4
23.72
cAdv
44.2
25.3
97.2*
36.8
37.0
34,9
40.1
30.6
19.3
20.2
32.04
RN-50
43.4
99.0*
28.8
30.2
21.6
16.5
15.1
25.97
ACE
32.8
9.4
99.1*
16.1
15.2
12.7
20.5
13.1
6.1
5.3
14.58
ColorFool
41.6
90.1*
18.6
21.0
15.4
20.4
15.4
5,9
6.8
17.21
NCF
71,2
33.6
91.4*
48.5
60.5
32.4
52.6
36.8
19.8
21.7
41,90
ACA (Ours)
69.3
61.6
88,3*
61.9
61.7
60.3
62.6
$2.9
$1.9
$3,2
59.49
SAF
54.5
26.9
49.7
33..4
41.4
30.4
46.1
78.4*
19.9
18.1
36. 16
ADer
15.3
9.9
8.4
7.6
12.0
12.4
81.5*
5.5
9.41
ReColorAdy
25.5
12.1
17.5
13.9
14.4
15.4
22.9
97.7*
10.9
8.6
15.69
cAdv
31.4
27.0
26.1
22.5
19.9
26.1
32.9
96.5%
18.4
16.9
24.58
VIT-B
LAdv
39.5
22.8
25.8
23.2
22.3
20.8
34.1
93.5*
16.3
15.3
24.46
ACE
30.9
11.4
22.0
15.5
15.2
13.0
17.0
98.6*
6.5
6.3
15.31
ColorFool
45.3
13.9
35.7
24.3
28.8
83,1"
9.3
23.67
NCF
55.9
25.3
50.6
34.8
42.3
29.9
40.6
81.0%
20.0
19.1
35.39
ACA (Ours)
64.6
58.8
60.2
S8.1
58.1
57.1
60.8
87.7*
$5.5
$4.9
58.68
Implementation Details. Our experiments are run on an NVIDIA Tesla A100 with Pytorch. DDIM steps T = 50, image mapping iteration N; = 10, attack iterations No 10, B 0.1. ( 0.01, n = 0.04, i = 0, and μ = The version of Stable Diffusion [42] is v1.4. Prompts for images are automatically generated using BLIP v2 [31].
4.2 Attacks on Normally Trained Models
In this section, we assess the adversarial transferability of normally trained convolutional neural networks (CNNs) and vision transformers (ViTs), including methods such as SAE [20], ADer []], ReColorAdv [30], cAdv [2], tAdv [2], ACE [61], ColorFool [47], NCF [60], and our ACA. Adver- sarial examples are crafted via MobViT-s, MN-v2, RN-50, and VIT-B, respectively. Avg. ASR (%)
refers to the average attack success rate on non substitut models.
Table illustrates the performance comparison of adversarial transferability on normally trained CNNs and ViTs. It can be observed that adversarial examples by ours generally exhibit superior transferability compared to those generated by state-of-the-an competitors and the impact of ILM on ASR is exceedingly marginal. When CNNs (RN-50 and MN-v2) are used as surrogate models. our ACA exhibits minimal differences with state-of tf-e----tthtt----- art NCF in MN-v2, RN-50, and RN-152. However, Inc-v3, Dense- 161, and EF-b7, such as when RN-50 used the surrogate model, we significantly outperform NCF by 28.0%. %. 13.4% and 27.9%. respectively. This indicates that our ACA has higher transferability in heterogeneous CNNs. Furthermore, our ACA demonstrates state-of- the-art transferability in current unconstrained attacks under the more challenging cross-archs architcccee

---

Table 2: Performance comparison of adversarial transferability on adversarial defense methods.
Attack
HGD R&P
NIPS-r3 JPEG
Bit-Red DiffPure Inc-V3gnsa Inc-V3 RncRes-V2ens
Res-De Shupe-Re Avg. ASR (%)
Clean
1.2
3.2
17.6
15.4
6.8
8,9
2.6
6.7
6.77
ILM
1.5
1.9
3.5
7.1
18.5
16.1
6.8
9.8
3.0
5.1
8.1
7.40
SAE
21.4
19.0
25.2
25.7
43.5
39.8
25.7
29.6
20.0
35.1
49.6
30.42
ADer
6.9
27.5
10.1
12.1
ReColorAdy
5.1
7.0
10.0
20.0
24.3
20.0
11.1
15.5
7.4
11.6
18.4
13.67
cAdv
12.2
14.0
17.7
11.1
33.9
32.9
19.9
23.2
14.6
16.2
25.3
20.09
tAdv
10.9
12.4
14.4
17.8
29.6
21.2
17.7
12.5
16.4
25.4
17.94
ACR
4.9
5.9
11.1
12.6
28.1
24.9
12.4
15.4
7.6
11.6
21.0
14.14
ColorFool
15.3
18.0
37.9
33.8
17.8
21.3
10.5
20,3
20.78
NCF
22.8
21.1
25.8
26.8
43.9
39.6
27.4
31.9
21.8
34.4
47.5
31.18
ACA (Ours) 52.2 53.6
53,9
59.7
63.4
63.7
59.8
62.2
53.6
55.6
60.8
58.05
setting. Specifically, when the surrogate model is RN-50, surpass NCF by significant margins of 10.0%, 16.1 %, 32.1 %, and 32.5% % in MobViT-s, ViT-B, Swin-B, and PVT-v2, respectively. There primary reasons for this phenomenon: i) our ACA utilizes low-dimennt wwwddieesioonl manifold search of natural images for adversarial examples, with the manifold itself determining the transferability of the adversarial examples, independent of the model's architecture; ii) the diffusion model incorporates the self-attention structure, exhibiting certain degree of architectural similarity.
Overall, the reformatioon-asser attack (ADer) exhibits lower attack performance in both white-box and black-box settings. Texture-based attacks (tAdv) show better white-box attack performance, but are less transferable than existing color-based attacks (NCF and SAE). Our ACA leverages the low-dimensional manifold natural images to adaptively combine image attributes and generate unrestricted adversarial examples, resulting in significant outperformance of tfateoof-the-at methods by 13.3 550.4% % on average. These results convincingly demonstrate the effectiveness of our method in fooling normally trained models.
4.3 Attacks on Adversarial Defense Methods
The situation in that adversarial defense methods can effectively protect against current adversarial attacks exhibits the Dunning- Kruger effect [28]. Actually, such defense methods demonstrate efficacy in defending against adversarial examples within the lp norm, yet their robustness falters in the face of novel and unseen attacks [25]. Therefore, we investigate whether unrestricted attacks can break through existing defenses. Here, choose input pre-process defenses (HGD [33], R&P [57], NIPS-r3', JPEG [17], 1 Rit-Red [59], and DiffPure [39]) and adversarial training models (Inc-v3 Inc-v3ens4: and Inc-v2ens [53]). Considering that unrestricted attacks are carried out from the perspective of shape and texture, we also choose shape-texture debaised models (ResNet50-Debaisee (Res-De) [32] and Shape-ResNet (Shape-Res) [14]).
The results of black-box transferability in adversarial defense methods are demonstrated Table the surrogate model is VIT-B, and the target model is Inc-v3 for input pre-process defenses. Our method displays persistent superiority over other advanced attacks by a significant margin. Our ACA surpasses NCF, SAE, ColorFool by 9..13%.8 27.63 %, and 37.27% average ASR. In robust models, based on I, adversarial training and shape-texture debiased models are not particularly effective and can still be easily broken by unrestricted adversarial examples. Our approach adaptively generate various combinations of adversarial examples based on the manifold, thus exhibiting high transferability different defense methods. Additionally, Bit-Red and Diffpure reduce the ground- truth class's i confidence and increase the adversarial examples' transferability. These findings further reveal the incompleteness and vulnerability of existing adversarial defense methods.
existii
4.4 Visualization
Quantitative Comparison. Following [47] and [60], quantitatively assess the image quality
using the non-reference perceptual image quality measure. Therefore, we choose NIMA [51]. HyperlQA [49], MUSIQ [27], and TReS [15]. NIMA-AVA and MUSIQ-AVA
trained on AVA [38],
and MUSIQ-KonIQ is trained on KonIQ-10K [21], following PyIQA [3]. As illustrated in Table 3,
ILM maintains the same image quality clean images, and ACA achieves the best results in
httpp: com/ an womnthhmsniipss http:////0///ee///aarrr/ooo

---

Clean
ILM
SAE
cAdy
IAdv
ColorFool
NCF
ACA (Ous)
(a) Visualization of state-of ttth--of unrestricted attacks
(b) Adversarial examples of Adversarial Content Attack (ACA)
(c) Case Study
Figure 3: (a) Compared with other attacks, ACA generates the most natural adversarial examples; (b) ACA can generate images with various adversarial content, which can combine shape, texture, and color changes; (c) In some cases, ACA may slightly modify the semantic subject.
image quality assessment. ColorFool obtains equal or higher image quality than the clean images because it requires manually selecting several -enaiivee semantic classes and adds uncontrolled perturbations on numan-insensitivee semantic classes. In other words, ColorFool is bound by human intuition, so do deteriorate the perceived image quality (our results are similar to [47]). ACA even surpasses ColorFool, mainly because: i) Our adversarial examples are generated based on the www-dimeniional manifold of natural images, which can adaptively combine the adversarial content and ensure photorealism; ii) Stable Diffusion itself is an extremely powerful generation model, which produces images with very high image quality; iii) These no-reference image metrics are often trained on aesthetic datasets, such as AVA [38] or KonIQ
[21]. Some of the images in these datasets are post-processed (such as Photoshop). which is more line with human aesthetics. Because ACA adaptively generates adversarial examples
low-dimensional manifold, this kind of minor image editing is similar to proc-pso,,,,, which more in line with human aesthetic perception and better image quality.
Qualitative Comparison. We visualize unrestricted attacks of Top-5 black-box transferability. including SAE, cAdv, tAdv, ColorFool, and NCF. In Figure 3(a). we visualize adversarial examples generated by dif- ferent unrestricted attacks. night scenes and food, color and texture changes are easily perceptible, while our method still keeps image photorealism. Next, we give more adversarial examples generated by ACA. clearly observed that our method can adaptively combine content to generate adversarial examples, shown in Figure 3(b). For example, the hot air balloon in the lower left corner modifies both the color of the sky and the texture the hot air balloon. The straw- berry in the lower right corner has some changes in shape and color while keeping the semantics unchanged. However, in some cases, the body of semantics changes, as shown in Figure 3(c). It may be because the prompts generated by BLIP v2 cannot describe the content of the image well.
Table 3: Image quality assessment.
NIMA -AVAF HyperIQAt
MUSIQ MUSIQ -AVA1 -KonQ TReSt
Clean I.M
5,15
0.667
4.07
$2.66
82.01
5.15
0.672
4.08
52.55
81.80
SAE
5.05
0.597
3.79
47.24
ADe
4.89
0.608
3.89
47.39
70.10
ReColorAdv
5.07
0.668
3.97
51.08
80.32
cAdy
4.97
0.623
3.87
48.32
75.12
ACE
5.12
0.648
3.96
50.49
77.25
ColorFool
5.24
4.05
52.27
78.54
NCF
4.96
0.634
3.87
50.33
74.10
ACA (Ours)
5.54
0.691
4.37
56.08
85.11
4.5 Time Analysis
In this section, we illustrate the attack speed of various unrestricted attacks. We choose MN-v2 [46] as the surrogate model and evaluate the inference time NVIDIA Tesla A100. Table 4 shows the

---

Table 4: Attack speed of unrestricted attacks, We choose MN-v2 as the surrogate model and evaluate the inference time on an NVIDIA Tesla A100.
Attack
SAE
ADer
ReColorAdv
cAdv
IAdv
ACE
ColorFool
NCF
ACA (Ours)
Time (sec) 8.80
0.41
3.86
18.67
4.88
6.64
12.18
10.45
60.0+65.33=125533
average time (in seconds) required to generate an adversarial example per image. ACA does have significant time cost compared to other attacks. Further, we analyze the time cost and find that Image Latent Mapping (ILM) and Adversarial Latent Optimization (ALO) each accounted for 50% of the time cost. However, most of the time cost of ILM and ALO lies in the sampling process of the diffusion model. In this paper, our main contribution to propose a new unrestricted attack paradigm. Therefore, we focus on the improvement of the attack framework, rather than the optimization of time cost. Since the time cost is mainly focused on the sampling of the diffusion model, we have noticed that many recent works have accelerated distilled the diffusion model, which can greatly reduce the time of the sampling process. For example. [36] can reduce the total number of sampling steps by least 20 times. If these acceleration technologies are applied to our ACA, ACA can theoretically achieve an attack speed of close to seconds and we think this is a valuable optimization direction.
4.6 Ablation Study
The ablation studies of momentum (MO) and differentiable boundary processing (DBP) are shown in Figure 4 and the surrogate model is
Atlack Success Rate (%)
Origa
MO-
ViT-B [1]]. Origin stands for ACA without
MO and DBP. MO Origin with momentum, and it can be observed that the adversarial transferability is significantly improved after the introduction of momentum. MO+
Origin with momentum and DBP. Since DBP further optimizes the effectiveness of the adversarial examples and constrains the image within the range of values, it can still improve the adversarial transferability. Although the above strategies are not the main contribution of this paper, the above experiments illustrate that they can boost adversarial transferability.
Dewen
Rhh PD
Vitpp
Seded
Figure 4: Ablation studies of momentum (MO) and differentiable boundary processing (DBP).
5 Conclusions
In this paper, we propose a novel unrestricted attack framework called Content-based Unrestricted Adversarial Attack. We map the image onto low-dimensional manifold of natural images. When images are moved along the adversarial gradient on the manifold, unrestricted adversarial examples with diverse adversarial content and photorealistic can be adaptively generated. Based the diffusion model, we implement Adversarial Content Attack. Experiments show that the existing defense methods form of unrestricted attack, hoping to draw attention to the threat posed by unrestricted attacks.
very effective against unrestricted content-based attacks. We propose new
Limitations. Due to the inherent limitations of diffusion models, considerable number of sampling steps are required in the inference process, resulting in a relatively longer runtime, taking ~ 2.5 minutes using a single A 100 GPU. Additionally, the current adaptive generation of unrestricted adversarial examples does not allow for fine -rae-ge image editing. These issues are expected to be resolved in the future with further advancements in diffusion models.
Negative Social Impacts. Adversarial examples from our method exhibit a high photorealism and strong black-box transferability, which raises concerns about the robustness of DNNs, as malicious adversaries may exploit this technique to mount attacks against security-sensinn
applications.
Acknowledgements. This work was supported by National Natural Science Foundation of China (No.62072112) Scientific and Technological innovation action plan of Shanghai Science and Tech- nology Committee (No.2251 105511102200 102202).
10

---

References
[1] Rima Alaifari, Giovanni S. Alberti, and Tandri Gauksson. Adef: an iterative algorithm to con- struct adversarial deformations. 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. 2, 6,
[2] Anand Bhattad, Min Jin Chong, Kaizhao Liang, B. Li, and David A. Forsyth. Unrestricted adversarial examples via semantic manipulation. In International Conference on Learning Representations, 2019. 2, 7
[3] Chaofeng Chen and Jiadi Mo. IQA-PyTorch: Pytorch toolbox for image quality assessment. [Online]. Available: https: thub. wtt.c/oooc/ccccceeeeeee chwofenge/IQA.e.c SAtese/IIAPPPPTooche 2022.
[4] Zhaoyu Chen, Bo Li, Shuang Wu, Shouhong Ding, and Wenqiang Zhang. Query-efficient decision-based black-box patch attack. IEEE Transactions on Information Forensics and Security, 18:5522-5536. 2023. 1
[5] Zhaoyu Chen, Bo Li, Shuang Wu, Jianghe Xu, Shouhong Ding, and Wenqiang Zhang. Shape matters: deformable patch attack. In European Conference on Computer Vision, pages 529-548. Springer, 2022.
[6] Zhaoyu Chen, Bo Li, Jianghe Xu, Shuang Wu, Shouhong Ding, and Wenqiang Zhang. Towards practical certifiable patch defense with vision transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15148-15158, 2022. 1
[7] Yiting Cheng, Fangyun Wei, Jianmin Bao, Dong Chen, and Wenqiang Zhang. Adpl: Adaptive dual path learning for domain adaptation of semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. 1
[8] Jia Deng, Alex Berg, Sanjeev Satheesh, H Su, Aditya Khosla, and Li Fei-Fei. Imagenet large scale visual recognition competition 2012 (ilsvre2012). See net. org/challenges/ISVRC, 41,
2012. 6
[9] Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat gans image syn- thesis. In Mare' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 8780-8794, 2021. 3, 5,5
[10] Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading defenses to transferable adversarial examples by translation-invant invariant attacks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4312-43211 2019. 6
[11] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, 2021. 6, 10
[12] Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, and Daniel Cohen Or. image is worth one word: Personalizing text-to-image generation using textual inversion. CoRR, abs/2208 0is/2208.061888 2022. 4
[13] Lianli Gao, Qilong Zhang, Jingkuan Song, Xianglong Liu, and Heng Tao Shen. Patch-wise attack for fooling deep neural network. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXVIIII 16, pages 307-322. Springer, 2020. 6
[14] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019,
11

---

[15] S Alireza Golestanch, Saba Dadsetan, and Kris M Kitani. No-reference image quality assessment via transformers, relative ranking, and self consistency. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pages 1220-1230, 2022. 8
[16] lan J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adver- sarial examples. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA. USA, May 7-9, 2015. Conference Track Proceedings, 2015.
[17] Chuan Guo, Mayank Rana, Moustapha Cissé, and Laurens van der Maaten. Countering adversarial images using input transformations. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. 8
[18] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 770-778, 2016. 1,6
[19] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. CoRR, abs/2207.12598
2022.
[20] Hossein Hosseini and Radha Poovendran. Semantic adversarial examples. In 2018 IEEE Conference on Computer Vision and Puttern Recognition Workshops, CVPR Workshops 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 1614-1619. Computer Vision Foundation / IEEE Computer Society, 2018. 2, 6,
[21] V. Hosu, H. Lin, T. Sziranyi, and D. Saupe. Koniq- An ecologically valid database for deep learning of blind image quality assessment. IEEE Trunsactions on Image Processing, 29:4041-4056, 2020. 8, 9
[22] Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. Densely connected convolutional networks. In 2017 IEEE Conference Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 2261-22616 IEEE Computer Society, 2017.
[23] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. Image-to-image translation with conditional adversarial networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 5967-5976.
IEEE Computer Society, 2017. 1
[24] Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision ECCV 2016 14th European Conference, Amsterdam, The Netherlands, October 1-14, 2016, Proceedings, Part II, volume 9906 of Lecture Notes in Computer Science, pages 694-711. Springer, 2016.
[25] Daniel Kang. Yi Sun, Dan Hendrycks, Tom Brown, and Jacob Steinhardt. Testing robustness against unforeseen adversaries. arXiv preprint arXiv: 1998008006 2019. 1,
[26] Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. CoRR, abs/2210.09276 2022. 4
[27] Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5148-5157 2021. 8
[28] Justin Kruger and David Dunning. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of personality and social psychology, 77(6): 121, 1999. 1,
[29] Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. Artificial intelligence safety and security, pages 99-112 12, Chapman and Hall/CRC, 2018. 6
12

---

[30] Cassidy Laidlaw and Soheil Feizi. Functional adversarial attacks. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché-Buee Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8- 14, 2019, Vancouver, BC, Canada, pages (0408-1048,, 2019. 2, 7.
[31] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2lip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301. 12597, 2023. 4,7
[32] Yingwei Li, Qihang Yu, Mingxing Tan, Jieru Mei, Peng Tang, Wei Shen, Alan L. Yuille, and Cihang Xie. Shape-texture debiased neural network training. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. 8
[33] Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang. Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In 2018 IEEE Conference on Computer Vision and Puttern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 1778-1787. Computer Vision Foundation / IEEE Computer Society, 2018.
[34] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In 202 I IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 9992-10002, 2021. 6
[35] Sachin Mchta and Mohammad Rastegari. Mobilevit: Light- general- purpose, and mobile- friendly vision transformer. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 2022.
[36] Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik P. Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation guided diffusion models. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 14297-14306, 2023. 10
[37] Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In /EEE/CVF Conference on Computer Vision and Patern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 6038-6047, 2023. 4
[38] Naila Murray, Luca Marchesotti, and Florent Perronnin. Ava: large-scale database for aesthetic visual analysis. In 2012 IEEE conference on computer vision and pattern recognition, pages 1408-2415. IEEE, 2012. 8.9
[39] Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Animashree Anand- kumar. Diffusion models for adversarial purification. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 16805-16827. PMLR, 2022. 8
[40] Haonan Qiu, Chaowei Xiao, Lei Yang, Xinchen Yan, Honglak Lee, and Bo Li. Semanticadv: Generating adversarial examples via conditiooe image editing. In Computer Vision ECCV 2020- 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIV, volume 12359 of Lecture Notes in Computer Science, pages 19-37. Springer, 2020. 2
[41] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with CLIP latents. CoRR. abs/2204.06 06155, 2022. 3
[42] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In IEEE/CVF Conference Computer Vision and Pottern Recognition, CVPR 2022, New Orleans, IA, USA, June 18-24, 2022, pages 10674-10685, 2022. 3,5,
13

---

[43] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. CoRR. abs/2208. 12242, 2022. 4
[44] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding, Advances in Neural Information Processing Systems, 35:36479-36499 2022. 3
[45] Hadi Salman, Alaa Khaddaj. Guillaume Leclere, Andrew Ilyas, and Aleksander Madry. Raising the cost of malicious ai ai-powered image editing. CoRR, abs/2302.06588, 2023. 5
[46] Mark Sandler, Andrew G. Howard, Menglong Zhu, Andrey Zhmoginov, and Liang- Laan-Chichh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 45100-520,, 2018. 6, 9
[47] Ali Shahin Shamsabadi, Ricardo Sánchez-Matilla, and Andrea Cavallaro. Colorfool: Semantic adversarial colorization. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 1148-1157. Computer Vision Foundation IEEE, 2020. 2, 3, 6, 7, 8,
[48] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffiusion implicit models. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. 4,
[49] Shaolin Su, Qingsen Yan, Yu Zhu, Cheng Zhang, Xin Ge, Jinqiu Sun, and Yanning Zhang. Blindly assess image quality in the wild guided by self-adaptive hyper network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3667-3676, 2020. 8
[50] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016,
pages 2818 2826,2266222 2016. 6
[51] Hossein Talebi and Peyman Milanfar. Nima: Neural image assessment. IEEE transactions on image processing, 27(3)3999 4011, 2018. 8
[52] Mingxing Tan and Quoc V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 6105-6114 14. PMLR, 2019. 6
[53] Florian Tramer, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick D. McDaniel. Ensemble adversarial training: Attacks and defenses. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
[54] Dani Valevski, Matan Kalman, Yossi Matias, and Yaniv Leviathan. Unitune: Text-driven image editing by fine tuning an image generation model single image. CoRR, abs/2210.09477. 2022.
[55] Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. PVT v2: Improved baselines with pyramid vision transformer. Comput. Vis. Media, 6(3):4154224 2022. 6
[56] Chaowei Xiao, Jun Yan Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song. Spatially transformed adversarial examples. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30- May 3, 2018, Conference Truck Proceedings. OpenReview.net, 2018. 2
14

---

[57] Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan L. Yuille. Mitigating adversarial effects through randomization. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
[58] Cihang Xie, Zhishuai Zhang. Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan Yuille. Improving transferability of adversarial examples with input diversity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2730-2739, 2019. 6
[59] Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. In 25th Annual Network and Distributed System Security Symposium, NDSS 2018, San Diego California, USA, February 18-21, 2018. The Internet Society, 2018.
[60] Shengming Yuan, Qilong Zhang, Lianli Gao, Yaya Cheng, and Jingkuan Song. Natural color fool: Towards boosting black-box unrestricted attacks. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 7546-7560. Curran Associates, Inc., 2022. 2, 3, 6, 8
[61] Zhengyu Zhao, Zhuoran Liu, and Martha A. Larson. Adversarial color enhancement: Generating unrestricted adversarial images by optimizing a color filter. In 31 British Machine Vision Conference 2020, BMVC 2020, Virtual Event, UK, September 7-10, 2020. BMVA Press, 2020.
2, 6.
[62] Zhengyu Zhao, Zhuoran Liu, and Martha A. Larson. Towards large yet imperceptible adversarial image perturbations with perceptual color distance. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 1036- 1045. Computer Vision Foundation / IEEE, 2020. 1
[63] Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Semantic understanding of scenes through the ADE20K dataset. Int. J. Comput. Vis., 127(3):30--321, 2019.
[64] Yao Zhu, Yuefeng Chen, Xiaodan Li, Kejiang Chen, Yuan He, Xiang Tian, Bolun Zheng, Yaowu Chen, and Qingming Huang. Toward understanding and boosting adversarial transferability from a distribution perspective. IEEE Trans. Image Process., 31:6487-6501, 2022.
15


Reviews:
[{"review_id": "pRpNJDnDbO", "reviewer": "Reviewer_oMF6", "summary": "This paper proposes the Adversarial Content Attack based on the diffusion model. The proposed attack method first maps the image onto a low-dimensional manifold of natural images and then moves images along the adversarial gradient on the manifold to generate photorealistic adversarial examples. The authors conduct extensive experimentation and demonstrate the efficacy of the Adversarial Content Attack in both normally trained models and defense methods.", "questions": "This paper mainly focuses on the untargeted attack. I wonder whether the proposed attack method can also be applied to the targeted attack case, that is, to specify the misclassified category.", "limitations": "The paper has discussed the limitations and negative social impacts of the proposed attack method.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper presents a host of visualization and experiments to support the intuitions and conclusions.\n2. The investigated topic is important and useful.", "weaknesses": "1. This work seems like an application of the diffusion model in model debugging research, finding some hard samples of deep models. However, it lacks mentions or comparisons with related work, including [1],[2], etc.\n\n2. In Table 3, I can hardly understand why the generated adversarial image can achieve better photorealism than the real image.\n\n3. In Table 2, Inc-v3$_{ens4}$ is used as the target model, which is a defense method. But confusingly, the attack success rate (**62.2**) surpasses the case when using normal Inc-v3 (**58.8**) as the target.\n\nI will be pleased to raise my score if these questions can be properly answered.\n\n[1] Xiaodan Li, Yuefeng Chen, Yao Zhu, Shuhui Wang, Rong Zhang, Hui Xue. ImageNet-E: Benchmarking Neural Network Robustness via Attribute Editing.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.\n\n[2] Maximilian Augustin, Valentyn Boreiko, Francesco Croce, Matthias Hein. Diffusion Visual Counterfactual Explanations. Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for valuable advice. We will follow your advice to complement the discussion of related work and the explanation of image quality. We would appreciate it very much if you can champion us.\n\n**[Q1: Related works about model debugging.]** Thanks for your help in improving the quality of our paper. ImageNet-E [1] is released after the NIPS submission is over, so we do not discuss it in time. We will add the following discussion to the final version:\n\n- The image editing of ImageNe-E [1] focuses on the single element that explicitly controls the image. It evaluates the robustness of background, size, pose, and direction, and does not emphasize the resulting adversarial robustness. However, our method can adaptively and implicitly generate adversarial examples with shapes, colors, and textures, with a variety of adversarial content, emphasizing the adversarial robustness. Further, it is found through experiments that it also has strong adversarial transferability. The focus is different on control methods, elements and robustness.\n\n- DVCE [2] is similar to ours in terms of generation effect, but in terms of technical route, optimization strategy and parameter update are different. The proposed Cone Projection is only suitable for robust classifiers, while our method does not have any requirements on the model. In addition, model debugging focuses on the bias of the classifier, such as some spurious features. While ACA pays more attention to the adversarial vulnerability of the model, expecting to find small modified error examples.\n\n**[Q2: Explanation on image quality.]** Thanks to your serious review, we have also noticed that the adversarial examples generated by ACA are more photorealism than real images, and explained the possible reasons in Lines 296-304:\n\n- Our adversarial examples are generated based on the low-dimensional manifold of natural images, which can adaptively combine the adversarial content and ensure photorealism;\n- Stable Diffusion itself is an extremely powerful generation model, which produces images with very high image quality.\n\nIt should be noted that this is not the first time a similar situation has appeared in this work. It also appeared in ColorFool (CVPR2020), where the unrestricted adversarial examples have better quality than the original image. In addition to the above 2 points, we further analyze the possible reasons. These no-reference image metrics are often trained on aesthetic datasets, such as ACA or KonIQ-10K. Some of the images in these datasets are post-processed (such as Photoshop), which is more in line with human aesthetics. Because ACA adaptively generates adversarial examples on a low-dimensional manifold, this kind of minor image editing is similar to post-processing, which is more in line with human aesthetic perception and better image quality. We will update the explanation of this part in the final version.\n\n**[Q3: ASR on defense models.]** Inc-v3$\\_{ens4}$ is a robust model based on adversarial training and adversarial training generally reduces the clean accuracy of the model (compared to the normal training model). Furthermore, Inc-v3$\\_{ens4}$ is based on $l_p$ perturbation for adversarial training, which does not have a good defense effect on unrestricted adversarial examples [55]. This experiment also verifies that the existing adversarial examples provide false adversarial robustness. \n\n**[Q4: Targeted attacks.]** Please refer to **Author Rebuttal #Q2**."}, {"review_id": "X6Tv6UiexC", "reviewer": "Reviewer_Rhy1", "summary": "In this paper, authors propose an unrestricted untargeted attack based on optimising the latent space of stable diffusion model. The generated adversarial samples are empirically shown to be more transferable than the existing semantic attacks. Moreover, authors validate the effectiveness of adversarial samples when attacking different representative adversarial trained models and show consistent performance book.", "questions": "Overall, I do not have major questions as the proposed method is properly motivated and leverages the powerful generative model to aid the attack generation. However, I have few minor questions mostly around implementation and design choices:\n\n- In section 3.1, authors propose to optimise the null text embedding $∅_t$ at every timestamp to offset the error. Is there any difference to your implementation as compared to the method of [32]. Can you please clarify your contribution in the section 3.1 ie. mapping image $z_0$ to latent space $z_t$.  \n\n- In particular, authors show the benefit of momentum factor $\\mu$  and boundary function $ϱ(·)$ to improve the ASR in Appendix D.  Did you perform ablation study to set the value of  $\\mu$ to 1 and can authors provide more insights on the design of boundary function in particular for inputs outside the valid range [0, 1]?  \n\n- How do you ensure the perturbation value of $k = 0.1$ in the latent space is not drastically modifying the original image.  Can you present an ablation of this key parameter vs attack performance vs image quality ?  In Table 3, there is no discussion about the non-reference metrics mentioned in the paper. Why is the method not evaluated with metrics such as LPIPS that captures perceptual similarity? Please also include FID to the metrics.  \n\n- How is the performance of the attacks in targeted attack setting? Do you require larger shift in latent space to craft example? \n\n- Please also benchmark the baseline attacks in terms of attack speed for completeness.\n\n- Can you constrain the perturbation to a local region in latent space to generate a kind of patch attacks in image space? \n\nFinally, I request the authors to discuss their plans in releasing the codebase to reproduce the experiments as I believe this will be invaluable to the community.", "limitations": "- The inference time of the attack is much higher taking 2.5 minutes for image due to many many levels of optimisation such as for null text embedding, perturbation in latent space.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The method is a replacement of popular GANs in the semantic attacks with the powerful stable diffusion models, pushing the state-of-the-art by many steps emperically.\n- Experiments are thorough in attacking both normal and adversarial trained models\n- Compared against many baseline unrestricted attacks and proposed attacks outperforms all the methods.", "weaknesses": "-  The paper  lacks technical novelty as the latent space optimisation for generating adversarial attacks as been popular from many years [17].\n\n- Since this is a unrestricted attack, the generated image quality is difficult to assess with metrics. Nonetheless, authors have computed 5 metrics to understand the image quality\n\n- Authors did not discuss about the code release to reproduce the experiments. For this particular paper, the implementation of this proposed method is not simple as the section 3.1 and 3.2 mostly discusses about the difficulties in optimising the latent space and how they overcome it through skip gradient, momentum and boundary function.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable comments and we address your concerns as follows.\n\n**[Q1: Technical novelty of latent space optimization.]** Techniques for latent space optimization using Generative Adversarial Networks (GAN) are common, but latent optimization using diffusion models has not been widely explored.\n\n- First, the latent of GAN and diffusion model is different. The latent of GAN is usually decoupled and limited to certain attributes, while the latent of the diffusion model is aligned with the text. We choose the null text embedding for optimization, propose a corresponding attack algorithm, and illustrate through experiments that it can generate high-transferability adversarial examples.\n  \n- Then, we find that the sampling process of the diffusion model is unable to construct the calculation graph of the gradient chain, because the GPU memory will overflow (Lines 192-198), so we propose the skip gradient to solve the unique problem of this diffusion model.\n\nTherefore, our work is designed for the latent of the diffusion model, which is the biggest difference between us and the previous work on latent space optimization.\n\n**[Q2: Choice of image quality assessment.]** Please refer to **Author Rebuttal #Q1**.\n\n**[Q3: Comparison on [32] and contributions.]** The biggest contribution of this paper is to propose a novel unrestricted attack framework called Content-based Unrestricted Adversarial Attack. Under this framework, we first employ Image Latent Mapping (ILM) to map images onto the latent space and utilize Adversarial Latent Optimization (ALO) to optimize the latent. We emphasize that our contribution is to introduce the ILM module to realize ACA so that images can be mapped to latent space to complete subsequent attacks. The existence of this model is very necessary, and [32] is just one implementation of ILM. This implementation can be replaced, or even superseded by better methods. Compared to other strategies [9, 39, 23, 49], the current strategy [32] is simple and effective and does not require fine-tuning to obtain high-quality image reconstruction. Our framework is not limited to this implementation, and the proposed framework can still be applied if there are better implementations in the future.\n\n**[Q4: Abaltion study on momentum factor.]** Please refer to **Author Rebuttal #Q3**.\n\n**[Q5: Insight about boundary function.]** The motivation of Differentiable Boundary Processing is to ensure that the value range of adversarial examples is between [0,1]. Because when the diffusion model generates an image, the value range may not be in [0,1]. When saving adversarial examples, it is usually directly cropped to [0,1], and the part of the perturbation that is not in [0,1] is ignored, which may cause the attack to fail. To reduce the ASR reduction caused by the error during storage, we use DBP to constrain the value range of the adversarial examples to [0,1] as much as possible.\n\n**[Q6: Perturbation value.]** Please refer to **Author Rebuttal #Q3**.\n\n**[Q7: Targeted attacks.]** Please refer to **Author Rebuttal #Q2**.\n\n**[Q8: Attack speed.]** Here, we illustrate the attack speed of various unrestricted attacks. We choose MN-v2 as the surrogate model and evaluate the inference time on an NVIDIA Tesla A100. The table shows the average time (in seconds) required to generate an adversarial example per image. ACA does have a significant time cost compared to other attacks. Further, we analyze the time cost and find that Image Latent Mapping and Adversarial Latent Optimization each accounted for ~50% of the time cost. However, most of the time cost of ILM and ALO lies in the sampling process of the diffusion model. We also take the initiative to explain this issue in the limitation.\n\n Attack      | SAE    | ADer   | ReColorAdv | cAdv    | tAdv   | ACE    | ColorFool | NCF     | ACA (Ours) \n:-----------:|:------:|:------:|:----------:|:-------:|:------:|:------:|:---------:|:-------:|:----------:\n Time (sec)  | 8.80   | 0.41   | 3.86       | 18.67   | 4.88   | 6.64   | 12.18     | 10.45   | 60.0+65.33=125.33     \n\n**In this paper, our main contribution is to propose a new unrestricted attack paradigm. Therefore, we focus on the improvement of the attack framework, rather than the optimization of time cost.** Since the time cost is mainly focused on the sampling of the diffusion model, we have noticed that many recent works have accelerated or distilled the diffusion model, which can greatly reduce the time of the sampling process. For example, [A2] can reduce the total number of sampling steps by at least 20 times. If these acceleration technologies are applied to our ACA, ACA can theoretically achieve an attack speed of close to 6 seconds. Thank you for your valuable suggestions, we think this is a valuable optimization direction.\n\n[A2] On Distillation of Guided Diffusion Models, CVPR 2023\n\n**[Q9: Local edit.]** Local editing is currently not supported in this paper. But with the rapid development of diffusion models and the emergence of more control and editing techniques, we believe that it is possible to incorporate local editing into our proposed unrestricted attack framework in the future. In addition, this issue has been displayed in the main body as a limitation.\n\n**[Q10: Release codes.]** Thank you for recognizing the value of our work. During the rebuttal, we have submitted an anonymous link with the codes to AC. To promote the research on unrestricted attacks, we promise to release the codes after accepting this work."}, {"review_id": "VlHy1bsNOU", "reviewer": "Reviewer_frod", "summary": "The paper introduces a novel attack framework called Content-based Unrestricted Adversarial Attack, which aims to generate diverse and natural adversarial examples with high transferability. The authors argue that existing methods, such as lp norm-based attacks, have limitations in terms of perceptual similarity, naturalness, and robustness. To address these issues, they propose mapping images onto a low-dimensional manifold represented by a generative model trained on natural images. This manifold ensures both photorealism and content diversity. By optimizing the adversarial objective on this latent space, they generate unrestricted adversarial examples. The proposed method, called Adversarial Content Attack (ACA), utilizes Image Latent Mapping (ILM) and Adversarial Latent Optimization (ALO) techniques to optimize the latent in a diffusion model. The effectiveness of ACA is validated through experiments and visualization, demonstrating significant improvements of 13.3~50.4% in terms of adversarial transferability compared to state-of-the-art attacks. \n\nOverall, the main contributions of the paper are the introduction of the Content-based Unrestricted Adversarial Attack framework, the development of the Adversarial Content Attack method, and the experiments demonstrating improvements in generating diverse and transferable adversarial examples.", "questions": "Please refer to the weakness part. In brief, the reviewer would like to understand more about the following points:\n\n- Could the authors provide more attack and defense baselines as mentioned in the weakness?\n- Could the authors provide some experiments on other datasets?\n- Please address the mentioned unclear claim.\n- Why the authors did not provide the code for review but answered the Reproducibility as \"yes\"?\n", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The authors effectively communicated the motivation, problem statement, and methodology of the proposed framework. By addressing the limitations (imperceptibility/photorealism/effectiveness) of existing methods, they proposed a novel attack framework that leverages a low-dimensional manifold represented by a generative model. By combining image mapping onto a latent space, optimizing adversarial objectives, and utilizing a diffusion model, the authors introduce a novel approach to generating diverse and natural adversarial examples. This paper might be the first to explore unrestricted adversarial examples through such a framework.\n\nThis paper offers a thorough explanation of the proposed attack framework, detailing the underlying techniques of ILM and ALO. The authors further support their claims through experimentation and visualization, providing evidence of the effectiveness of their approach and demonstrating improvements in adversarial transferability compared to state-of-the-art attacks. The improvements in adversarial transferability also shed light on the potential impact of this method in uncovering vulnerabilities in security-sensitive applications and advancing our understanding of robustness in DNNs.\n\nOverall, this paper's strengths encompass originality in proposing a novel attack framework, quality in terms of methodology and experimental evaluation, clarity in explaining the concepts and techniques, and significance in addressing limitations and raising awareness of unrestricted but realistic adversarial examples.\n", "weaknesses": "While the paper has several strengths, there are some weaknesses that could be addressed to further improve the work:\n\n**Comparison with State-of-the-Art Attacks:** While the paper mentions that the proposed method achieves significant improvements in terms of adversarial transferability compared to state-of-the-art attacks, a more comprehensive comparison would strengthen the evaluation. It would be valuable to include a thorough analysis and comparison with a wider range of existing unrestricted attack methods, such as [1]. In particular, Laidlaw et al. proposed an efficient way to generate imperceptible adversarial examples. The reviewer also suggests evaluating the proposed attack on other adversarially trained models, for example, the defense method that could be generalized to unforeseen perturbations [1], or having used synthetic data during adversarial training [2].\n\n**Defense Method:** This paper does not discuss how to defend the proposed attacks but primarily on the efficacy of the proposed method. It would be great if the authors provide potential solutions or mitigation strategies for the threats.\n\n**Generalization to Different Datasets:** The authors only evaluate their method on a subset of the ImageNet validation set and do not say how the results generalize to other datasets. It would be beneficial to investigate the generalization of the proposed approach to different datasets.\n\n**Unclear Claim:** The authors mentioned the Dunning-Kruger effect to emphasize that current defense methods against lp norm adversarial examples overestimate their abilities. However, this work does not provide further details and arguments to support this. Although similar arguments have also been proposed in [3], in which the authors argue that lp-based robustness evaluation might be biased, the reviewer thinks that using the Dunning-Kruger effect here is not rigorous. The reviewer suggests that the authors rethink this argument, and even consider removing it if it is not an important contribution of the paper.\n\n**Typos:** Although this paper is well-written and easy to follow, the reviewer found some typos and grammar errors. For example, in line 161, *follw*. It would be great if the authors have proofread the paper before submitting it.\n\n    [1] Laidlaw et al. Perceptual adversarial robustness: Defense against unseen threat models. (ICLR 2021)\n    [2] Croce et al. Robustbench: a standardized adversarial robustness benchmark. (NeurIPS 2021)\n    [3] Hsiung et al. CARBEN: Composite Adversarial Robustness Benchmark. (IJCAI 2022)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and effort in our work. We supplement the experiments with the attack and defense, and release the code. Hope you can further support our work.\n\n**[Q1: Comparison with state-of-the-art attacks.]** Thanks for your valuable suggestions, we supplement the attack experiments of Perceptual Projected Gradient Descent (PPGD) and Lagrangian Perceptual Attack (LPA) in [1]. PPGD exhibits poor attack performance, while LPA has better resistance to migration. However, compared with the state-of-the-art NCF and ACA, there is still a large gap (ACA exceeds 34.47% on Avg. ASR).\n\n Attack     | MN-v2 | Inc-v3 | RN-50 | Dense-161 | RN-152 | EF-b7 | MobViT-s | ViT-B | Swin-B | PVT-v2 | Avg. ASR (%)  \n:----------:|:-----:|:------:|:-----:|:---------:|:------:|:-----:|:--------:|:-----:|:------:|:------:|:-------------:\n PPGD       | 23.1  | 12.3   | 99.7* | 16.6      | 18     | 13.3  | 14.9     | 10.6  | 6.3    | 6.9    | 13.56         \n LPA        | 37.6  | 24     | **100***  | 34.4      | 38     | 22    | 29.2     | 13.5  | 12.2   | 14.3   | 25.02         \n NCF        | **71.2**  | 33.6   | 91.4* | 48.5      | 60.5   | 32.4  | 52.6     | 36.8  | 19.8   | 21.7   | 41.90         \n ACA (Ours) | 69.3  | **61.6**   | 88.3* | **61.9**      | **61.7**   | **60.3**  | **62.6**    | **52.9**  | **51.9**   | **53.2**   | **59.49**         \n\nFurthermore, we complement the defense experiments on adversarially trained models. Since the pre-trained robust model weights are not released in [1], we cannot reproduce the adversarial training model on ImageNet quickly due to time and computing power. Therefore, we choose ViT -B-CvSt and ConvNext-L-CvSt in state-of-the-art on the leaderboard in [2] (the model comes from [A3] and the input size is 224). Our ACA still outperforms other unrestricted attacks by a significant margin.\n\n Attack         | Clean | ILM  | SAE   | ADer | ReColorAdv | cAdv  | tAdv  | ACE   | ColorFool | NCF  | ACA (Ours) \n:--------------:|:-----:|:----:|:-----:|:----:|:----------:|:-----:|:-----:|:-----:|:---------:|:----:|:----------:\n ViT-B-CvSt      | 8.4   | 8.7  | 38.9  | 11.2 | 10.5       | 20.6  | 11.3  | 16.9  | 31.2      | 35.8 | **51.1**       \n ConvNext-L-CvSt  | 7.4   | 7.9  | 34.5  | 11.0   | 9.5        | 17.3  | 10.7  | 15.4  | 26.9      | 33.0   | **49.7**       \n\nThanks again for your suggestion, we will add this part of the experiment to the final version.\n\n[A3] Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Models, arXiv preprint arXiv:2303.01870.\n\n**[Q2: Defense method.]** We believe that improving the adversarial robustness of the model itself is a potential defense strategy. First, recent work on LLM [A4, A5] shows that larger models are more robust. Therefore, the construction of the foundation model may be a potential direction. Secondly, we can consider designing specific training strategies, such as the one you mentioned [1]. It may be an idea to add unrestricted adversarial examples to adversarial training. In the end, we thought that maybe a better way of visual encoding might also be a solution. We will incorporate these ideas into the final version.\n\n[A4] Nicholas Carlini et. al., Are aligned neural networks adversarially aligned?\n[A5] Jindong Wang et. al., On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective\n\n**[Q3: Generalization to different datasets.]** We choose ImageNet mainly to set up alignment with other methods, which is conducive to a fair comparison. In terms of models, ImageNet has the most image classification models, which is convenient for the experiment of transfer attack; in terms of difficulty, ImageNet has 1000 categories, rich in content and semantics, which are more in line with images in real scenes, so experiments on this dataset can make the results generalizable. In addition, in the field of transfer attacks, ImageNet-compatible Dataset is widely used. We admit that our method will have limitations on small-sized images, such as CIFAR because it requires a larger-sized input, but the results on ImageNet have been able to illustrate the effectiveness and generalization of ACA. If time permits, we will add experiments on more datasets in the future.\n\n**[Q4: Unclear claim.]** A recent paper [A6] describes the Dunning-Kruger effect as a cognitive bias in that humans tend to overestimate their abilities. We argue that the adversarial robustness of current adversarial defenses is currently overestimated. In fact, the current adversarial defense can only better defend against the $l_p$ adversarial example, but cannot effectively defend against unrestricted adversarial examples. We detail the experimental results of attacking defenses in Section 4.3. ACA achieves a transfer attack success rate of more than 50% in all defense methods. We think this greatly illustrates the insufficiency of current adversarial defenses in the face of unrestricted adversarial examples, so we interpret this overestimation as a Dunning-Kruger effect. Regarding this statement, we are happy to continue to communicate with you in the discussion session.\n\n[A6] Reliability in Semantic Segmentation: Are We on the Right Track? CVPR 2023\n\n**[Q5: Typos.]** Thank you for your detailed review of our work, which is of great help to us in improving the quality of the paper. We will carefully check all typos and revise them in the final version.\n\n**[Q6: Reproducibility as \"yes\".]** Following the NeurIPS Paper Checklist Guidelines, our paper provides enough information to reproduce the experiment, so we choose \"yes\". Also during the rebuttal, we submitted the anonymous link containing the code to AC. In order to promote the research on unrestricted attacks, we promise to release the code after accepting this work."}, {"review_id": "DxH5fWb5T8", "reviewer": "Reviewer_QZtm", "summary": "This paper proposes a novel unrestricted attack framework called Content-based Unrestricted Adversarial Attack (ACA). The author argues that current unrestricted attacks have limitations in terms of maintaining human visual imperceptibility, generating natural adversarial examples, and achieving high attack performance. This paper is well-written and easy to follow.", "questions": "/Na", "limitations": "/Na", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) The paper introduces a novel attack framework that addresses the limitations of current unrestricted attacks. The use of a low-dimensional manifold and optimization along its adversarial direction allows for the generation of diverse and natural adversarial examples.\n2) The paper provides a clear motivation and problem definition, outlining the challenges and goals for unrestricted adversarial attacks.\n3) The paper includes extensive experimentation and visualization to validate the effectiveness of ACA. The results show significant improvements over state-of-the-art attacks in terms of adversarial transferability.", "weaknesses": "The paper does not include sufficient evaluation and ablation studies for the proposed method. Since ACA used skip gradient, I think I should compare Skip connections matter [1] in the experiment.\n\nThe paper could benefit from a more thorough review of relevant literature. While the authors mention existing unrestricted attacks, there is limited discussion on related work and the novelty  of ACA compared to previous approaches.\n\n\n\n[1] Skip connections matter: On the transferability of adversarial examples generated with resnets, ICLR 2019", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable comments. What excites us is not the high rating you have given us, but the questions you have proposed. This assures us that you have a deep knowledge of the field and have recognized the significance of our work to the community.  **Therefore, we hope you can champion our work in the discussion.** We are convinced that our work is worthy of being presented to a wider audience of researchers and can assist ML systems in mitigating the threat of unrestricted attacks in practice. We address your concerns as follows:\n\n**[Q1: Insufficient evaluation and ablation studies.]** Based on your valuable suggestions, we add an experimental comparison with SGM in Q2. For more evaluation, we add more attack and defense experiments (please refer to **Reviewer frod #Q1**). In the supplementary material, we provide ablation studies with momentum, differentiable boundary processing, and $\\beta$. Further, we also supplement the ablation studies of momentum factor $\\mu$ and perturbation value $\\kappa$ (please refer to **Author Rebuttal #Q3**).\n\n**[Q2: Comparison on Skip connections matter (SGM).]** Thank you for pointing out that this is actually a duplication of terms, that is, our skip gradient (SG) is similar in name to the Skip connections matter (SGM) you mentioned, but they are technically different. \n- In terms of purpose, our SG is to solve the problem of memory overflow caused by gradient backpropagation during the sampling process of the diffusion model, but SGM is using more gradients from the skip connections rather than the residual modules to enhance transferability.\n- In terms of implementation, SG uses the derivative in the denoising process to approximate the gradient, while SGM uses a single decay factor to enhance the gradient on skip connections. \n- In terms of scope of application, SG is suitable for diffusion models, while SGM is suitable for neural networks with skip connections.\n\nOverall, both SG and SGM directly or indirectly improve adversarial transferability. We will add a corresponding discussion to illustrate the above issues in the final version. Furthermore, we supplement related experiments following your suggestion. Because SGM is only effective for gradient optimization-based methods and is a plug-and-play module, we choose ADer, ReColorAdv, cAdv, ACE and ACA for experiments (ResNet50 is the surrogate model). Experiments show that SGM has almost no effect on ADer and cAdv, but can significantly improve transferability on other methods (ACA improves Avg. ASR by 5.87%).\n\n Attack          | MN-v2 | Inc-v3 | RN-50 | Dense-161 | RN-152 | EF-b7 | MobViT-s | ViT-B | Swin-B | PVT-v2 | Avg. ASR (%)  \n:---------------:|:-----:|:------:|:-----:|:---------:|:------:|:-----:|:--------:|:-----:|:------:|:------:|:-------------:\n ADer            | 15.5  | 7.7    | 55.7* | 8.4       | 7.8    | 11.4  | 12.3     | 9.2   | 4.6    | 4.9    | 9.09          \n ADer+SGM        | 14.8  | 7.3    | 10.5* | 7.7       | 7.4    | 11.1  | 11.8     | 9.5   | 4.5    | 4.7    | 8.76          \n ReColorAdv      | 40.6  | 17.7   | 96.4* | 28.3      | 33.3   | 19.2  | 29.3     | 18.8  | 12.9   | 13.4   | 23.72         \n ReColorAdv+SGM  | 58.3  | 26.7   | 98.6* | 48.4      | 53.0   | 25.8  | 43.5     | 23.8  | 21.1   | 20.9   | 35.72         \n cAdv            | 44.2  | 25.3   | 97.2* | 36.8      | 37.0   | 34.9  | 40.1     | 30.6  | 19.3   | 20.2   | 32.04         \n cAdv+SGM        | 43.4  | 25.5   | 57.8* | 32.2      | 30.3   | 31.9  | 39.0     | 30.9  | 19.3   | 19.5   | 30.22         \n ACE             | 32.8  | 9.4    | 99.1* | 16.1      | 15.2   | 12.7  | 20.5     | 13.1  | 6.1    | 5.3    | 14.58         \n ACE+SGM         | 61.4  | 20.9   | 99.2* | 33.8      | 37.8   | 20.8  | 36.3     | 26.0  | 11.7   | 10.0   | 28.74         \n ACA (Ours)      | 69.3  | 61.6   | 88.3* | 61.9      | 61.7   | 60.3  | 62.6     | 52.9  | 51.9   | 53.2   | 59.49         \n ACA (Ours) +SGM | 76.7  | 67.0   | 91.7* | 68.8      | 69.0   | 59.9  | 68.5     | 58.5  | 58.2   | 61.6   | 65.36         \n\n\n**[Q3: Thorough review of relevant literature.]** Thanks for pointing out related work and novelty issues discussed in the current version. **For related work**, we briefly summarize the shortcomings of current shape-, texture-, and color-based unrestricted attacks in Lines 35-45. Then we further analyze the shortcomings of ColorFool and Natural Color Fool in Lines 86-96. **For the novelty of ACA**, we summarize the innovations in the attack form in Lines 97-100: ACA can adaptively combine multiple contents (shape, texture and color), guarantee the photorealism of the image, and exhibit potential attack performance. In Section 3 , we demonstrate the technical innovations of ACA, including Image Latent Mapping and Adversarial Latent Optimization. Following your comments, we will add detailed descriptions of each unrestricted attack, systematically summarize the shortcomings, and discuss the differences with ACA in more detail to highlight the novelty of ACA. Thank you again for your help in improving the quality of the paper."}]

Author rebuttal:
We sincerely appreciate all reviewers' valuable feedback and the efforts of the program chair and area chair. So, we are committed to addressing the issues you raised and improving our manuscript accordingly. Next, we provide point-to-point responses to each reviewer and address all details.

**[Q1: Choice of image quality assessment.]** The quality assessment of generated images has always been a concern of the academic community, and numerous endeavors have been undertaken to tackle the issue. We acknowledge that objective metrics alone cannot perfectly reflect the image quality of unrestricted adversarial examples, so we provide qualitative and quantitative analysis in Section 4.4 to analyze image quality. The reason why we did not choose common perceptual image quality metrics such as LPIPS or FID is that the backbone (AlexNet/VGG/InceptionV3) they use is pre-trained on ImageNet and vulnerable to adversarial attacks, which may lead to biased image quality estimation. So we don't provide relevant values. Further, the unrestricted attacks recently published at the top tier conference like ColorFool (CVPR2020) and NCF (NIPS2022), both choose NIMA, a non-reference perceptual image quality measure, as the evaluation standard. As this metric has been rigorously peer-reviewed and verified to be feasible, we supplemented the NIMA metric with four additional metrics to achieve a more comprehensive evaluation of the visual quality. The proposed evaluation scheme may not be the optimal solution, but we believe that it is a relatively appropriate strategy at the moment.

**[Q2: Targeted Attack.]** Yes, our ACA can achieve targeted attacks by modifying the loss function, just like PGD. Considering that previous unrestricted attacks pay more attention to untargeted adversarial transferability, our experiments are also mainly aligned with previous work. Further, we implement this targeted attack, integrate this attack into codes and submit it to AC. Therefore, we hope that reviewers will support our work to advance the field's attention to the threat of unrestricted attacks.

**[Q3: More abaltion study.]** Following **Reviewer Rhy1**'s suggestion, we supplement the ablation study of momentum factor $\mu$ and perturbation value $\kappa$ (MN-v2 as the surrogate model). 

- **Momentum factor $\mu$**: As $\mu$ becomes greater, the black-box average ASR will rise first. It peaks at $\mu = 1$, greater $\mu$ leads to a slight drop in performance. It is worth noting that **even without momentum** ($\mu=0$), our ACA also outperforms the next highest attack (NCF) by 12.04%.

 $\mu$  | MN-v2 | Inc-v3 | RN-50 | Dense-161 | RN-152 | EF-b7 | MobViT-s | ViT-B | Swin-B | PVT-v2 | Avg. ASR (%)  
:---:|:-----:|:------:|:-----:|:---------:|:------:|:-----:|:--------:|:-----:|:------:|:------:|:-------------:
 0   | 91.8*  | 53.1   | 58.9  | 55.2      | 54.7   | 53.4  | 56.2     | 47.8  | 45.1   | 46.5   | 52.32         
 0.2 | 93.1* | 55.1   | 60.4  | 54.0        | 53.6   | 50.5  | 55.1     | 45.9  | 43.6   | 46.8   | 51.67         
 0.4 | 93.5*  | 53.7   | 60.3  | 53.6      | 53.9   | 52.2  | 57.5     | 47.9  | 46.1   | 46.4   | 52.40         
 0.6 | 92.9*  | 54.5   | 59    | 56.2      | 56.1   | 52.7  | 58.9     | 48.6  | 47.0     | 47.7   | 53.41         
 0.8 | 92.7*  | 55.6   | 59.4  | 56.5      | 56.3   | 52.7  | 57.6     | 50.5  | 47.9   | 47.6   | 53.79         
 1   | 93.1* | 56.8   | 62.6  | 55.7      | 56.0   | 51.0  | 59.6     | 48.7  | 48.6   | 50.4   | **54.38**         
 2   | 87.8*  | 58.2   | 60.2  | 56.1      | 55.7   | 52.6  | 59.0       | 49.3  | 47.9   | 49.9   | 54.32         

- **Perturbation value $\kappa$**: In the table below, we can find that the attack success rate will increase with the increase of $\kappa$, because the increase of $\kappa$ will lead to an increase in the degree of image content change. But in terms of image quality, HyperIQA, MUSIQ-Koniq, and TReS degrade image quality as $\kappa$ increases. Since NIMA-AVA and MUSIQ-AVA are trained on AVA, AVA has some post-processed images. When $\kappa$ is small, the effect of ACA can be similar to the post-processing of the image, so these two metrics will increase slightly (For more explanation please refer to **Reviewer oMF6 #Q2**). But as $\kappa$ becomes larger, the degree of image content change becomes larger, and these two metrics also begin to decrease. In summary, we find that $\kappa=0.1$ can achieve a better image quality.
  
$\kappa$    | Avg. ASR (%) | NIMA-AVA | HyperIQA | MUSIQ-AVA | MUSIQ-Koniq | TReS    
:----:|:------------:|:--------:|:--------:|:---------:|:-----------:|:-------:
 0.01 | 28.09        | 5.46     | **0.718**    | 4.31      | **57.85**       | **87.82**   
 0.05 | 30.70        | 5.51     | 0.710    | 4.36      | 57.60       | 86.97   
 0.1  | 32.10        | **5.54**     | 0.695    | **4.37**      | 56.18       | 85.11   
 0.15 | 32.62        | 5.46     | 0.675    | 4.32      | 54.23       | 82.34   
 0.2  | **33.39**        | 5.45     | 0.652    | 4.29      | 52.85       | 79.93 

Meta-review: In this paper, motivated by the goals of ensuring the photorealism of adversarial examples and boost attack performance, the authors propose a novel unrestricted attack framework called Content-based Unrestricted Adversarial Attack. After the rebuttal process, the responses from the authors mostly satisfy the reviewers and the reviewers reach a consistent consensus of raising the scores and accepting this manuscript. Taking all positive comments from the reviewers into consideration, this paper is recommended to be accepted!
Decision: Accept (poster)