Title: Content-based Unrestricted Adversarial Attack
Venue: NeurIPS 2023
Accepted: yes
Keywords: unrestricted attack

adversarial example

diffusion model

black-box attack

adversarial transferability
Abstract: Unrestricted adversarial attacks typically manipulate the semantic content of an image (e.g., color or texture) to create adversarial examples that are both effective and photorealistic, demonstrating their ability to deceive human perception and deep neural networks with stealth and success. However, current works usually sacrifice unrestricted degrees and subjectively select some image content to guarantee the photorealism of unrestricted adversarial examples, which limits its attack performance. To ensure the photorealism of adversarial examples and boost attack performance, we propose a novel unrestricted attack framework called Content-based Unrestricted Adversarial Attack. By leveraging a low-dimensional manifold that represents natural images, we map the images onto the manifold and optimize them along its adversarial direction. Therefore, within this framework, we implement Adversarial Content Attack (ACA) based on Stable Diffusion and can generate high transferable unrestricted adversarial examples with various adversarial contents. Extensive experimentation and visualization demonstrate the efficacy of ACA, particularly in surpassing state-of-the-art attacks by an average of 13.3-50.4\% and 16.8-48.0\% in normally trained models and defense methods, respectively.

Content-based Unrestricted Adversarial Attack

Abstract

Unrestricted adversarial attacks typically manipulate the semantic content of an image ( e.g. , color or texture) to create adversarial examples that are both effective and photorealistic, demonstrating their ability to deceive human perception and deep neural networks with stealth and success. However, current works usually sacrifice unrestricted degrees and subjectively select some image content to guarantee the photorealism of unrestricted adversarial examples, which limits its attack performance. To ensure the photorealism of adversarial examples and boost attack performance, we propose a novel unrestricted attack framework called Content-based Unrestricted Adversarial Attack. By leveraging a low-dimensional manifold that represents natural images, we map the images onto the manifold and optimize them along its adversarial direction. Therefore, within this framework, we implement Adversarial Content Attack (ACA) based on Stable Diffusion and can generate high transferable unrestricted adversarial examples with various adversarial contents. Extensive experimentation and visualization demonstrate the efficacy of ACA, particularly in surpassing state-of-the-art attacks by an average of 13.3-50.4% and 16.8-48.0% in normally trained models and defense methods, respectively.

1 Introduction

Deep neural networks (DNNs) have significantly progressed in many tasks [18, 7]. However, with the rise of adversarial examples, the robustness of DNNs has been dramatically challenged [16]. Adversarial examples show the vulnerability of DNNs and expose security vulnerabilities in many security-sensitive applications. To avoid potential risks and further research the robustness of DNNs, it is of great value to expose as many 'blind spots' of DNNs as possible at the current research stage.

Nowadays, various methods are proposed to generate adversarial examples [4, 5, 6]. To maintain human visual imperceptibility and images' photorealism, adversarial perturbations within the constraint of l p norm are generated by these adversarial attacks. However, it is well known that the adversarial examples generated under l p norm have obvious limitations: firstly, they are not ideal in terms of perceptual similarity and are still easily perceptible by humans [24, 23, 62]; secondly, these adversarial perturbations are not natural enough and have an inevitable domain shift with the noise in the natural world, resulting in the adversarial examples being different from the hard examples that appear in the real world [64]. In addition, current defense methods against l p norm adversarial examples overestimate their abilities, known as the Dunning-Kruger effect [28]. It can effectively defend against l p norm adversarial examples but is not robust enough when facing new and unknown attacks [25]. Therefore, unrestricted adversarial attacks are beginning to emerge, using unrestricted but natural changes to replace small l p norm perturbations, which are more practically meaningful.

†

Existing unrestricted adversarial attacks generate adversarial examples based on image content such as shape, texture, and color. Shape-based unrestricted attacks [56, 1] iteratively apply small deformations to the image through a gradient descent step. Then, texture-based unrestricted attacks [2, 40] are introduced, which manipulate an image's general attributes (texture or style) to generate adversarial examples. However, texture-based attacks result in unnatural results and have low adversarial transferability. Researchers then discover that manipulating pixel values along dimensions generates more natural adversarial examples, leading to the rise of color-based unrestricted attacks [20, 30, 2, 61, 47, 60]. Nonetheless, color-based unrestricted attacks tend to compromise flexibility in unconstrained settings to guarantee the photorealism of adversarial examples. They are achieved either through reliance on subjective intuition and objective metrics or by implementing minor modifications, thereby constraining their potential for adversarial transferability.

Considering the aforementioned reasons, we argue that an ideal unrestricted attack should meet three criteria: i) it needs to maintain human visual imperceptibility and the photorealism of the images; ii) the attack content should be diverse, allowing for unrestricted modifications of image contents such as texture and color, while ensuring semantic consistency; iii) the adversarial examples should have a high attack performance so that they can transfer between different models. However, there is still a substantial disparity between the current and ideal attacks.

To address this gap, we propose a novel unrestricted attack framework called Content-based Unrestricted Adversarial Attack . Firstly, we consider mapping images onto a low-dimensional manifold. This low-dimensional manifold is represented by a generative model and expressed as a latent space. This generative model is trained on millions of natural images, possessing two characteristics: i) sufficient capacity to ensure the photorealism of generated images; ii) well-alignment of image contents with latent space ensures a diversity of content. Subsequently, more generalized images can be generated by walking along the low-dimensional manifold. Optimizing the adversarial objective on this latent space allows us to achieve more diverse adversarial contents. In this paper, we propose Adversarial Content Attack (ACA) utilizing the diffusion model as a low-dimensional manifold. Specifically, we employ Image Latent Mapping (ILM) to map images onto the latent space, and utilize Adversarial Latent Optimization (ALO) to optimize the latents, thereby generating unrestricted adversarial examples with high transferability. In conclusion, our main contributions are:

· We propose a novel attack framework called Content-based Unrestricted Adversarial Attack, which utilizes high-capacity and well-aligned low-dimensional manifolds to generate adversarial examples that are more diverse and natural in content.

· We achieve an unrestricted content attack, known as the Adversarial Content Attack. By utilizing Image Latent Mapping and Adversarial Latent Optimization techniques, we optimize latents in a diffusion model, generating high transferable unrestricted adversarial examples.

· The effectiveness of our attack has been validated through experimentation and visualization. Notably, we have achieved a significant improvement of 13.3 ∼ 50.4% over state-of-the-art attacks in terms of adversarial transferability.

2 Background and Preliminary

Problem Definition. For a deep learning classifier F θ ( · ) with parameters θ , we denote the clean image as x and the corresponding true label as y . Formally, unrestricted adversarial attacks aim to create imperceptible adversarial perturbations (such as image distortions, texture or color modifications, etc.) for a given input x to generate an adversarial example x adv that can mislead the classifier F θ ( · ) :

where L ( · ) is the loss function. Because existing unrestricted attacks are limited by their attack contents, it prevents them from generating sufficiently natural adversarial examples and restricts their attack performance on different models. We hope that unrestricted adversarial examples are more natural and possess higher transferability. Therefore, we consider a more challenging and practical black-box setting to evaluate the attack performance of unrestricted adversarial examples. In contrast to the white-box setting, the black-box setting has no access to any information about the target model (i.e., architectures, network weights, and gradients). It can only generate adversarial examples by using a substitute model F ϕ ( · ) and exploiting their transferability to fool the target model F θ ( · ) .

Content-based Unrestricted Adversarial Attack. Existing unconstrained attacks tend to modify fixed content in images, such as textures or colors, compromising flexibility in unconstrained settings to ensure the photorealism of adversarial examples. For instance, ColorFool [47] manually selects the parts of the image that are sensitive to human perception in order to modify the colors, and its effectiveness is greatly influenced by human intuition. Natural Color Fool [60], utilizes ADE20K [63] to construct the distribution of color distributions, resulting in its performance being restricted by the selected dataset. These methods subjectively choose the content to be modified for minor modifications, but this sacrifices the flexibility under unrestricted settings and limits the emergence of more "unrestricted" attacks. Taking this into consideration, we contemplate whether it is possible to achieve an unrestricted adversarial attack that can adaptively modify the content of an image while ensuring the semantic consistency of the image.

An ideal unrestricted attack should ensure the photorealism of adversarial examples, possess diverse adversarial contents, and exhibit potential attack performance. To address this gap, we propose a novel unrestricted attack framework called Content-based Unrestricted Adversarial Attack . Within this framework, we assume that natural images can be mapped onto a low-dimensional manifold by a generative model. As this lowdimensional manifold is well-trained on natural images, it naturally ensures the photorealism of the images and possesses the rich content present in natural images. Once we map an image onto a low-dimensional manifold, moving it along the adversarial direction on the manifold yields an unrestricted adversarial example. We argue that such a framework is closer to the ideal of unrestricted adversarial attacks, as it inherently guarantees the photorealism of adversarial examples and rich image content. Moreover, since the clas-

the Manifold of

Natural Images

Adversarial

Gradient

Projected

Gradient

Texture Sub-manifold

Color Sub-manifold

sifier itself fits the distribution of this low-dimensional manifold, adversarial examples generated along the manifold have more adversarial contents and the potential for strong attack performance, as shown in Figure 1.

Naturally, selecting a low-dimensional manifold represented by a generative model necessitates careful consideration. There are two characteristics taken into account: i) sufficient capacity to ensure photorealism in the generated images; and ii) well-alignment ensures that image attributes are aligned with the latent space, thereby promoting diversity in content generation. Recently, diffusion models have emerged as a leading approach for generating high-quality images across varied datasets, frequently outperforming GANs [9]. However, several large-scale text-to-image diffusion models, including Imagen [44], DALL-E2 [41], and Stable Diffusion [42], have only recently come to the fore, exhibiting unparalleled semantic generation capabilities. Considering the trade-off between computational cost and high-fidelity image generation, we select Stable Diffusion as the low-dimensional manifold in this paper. It is based on prompt input and is capable of generating highly realistic natural images that conform to the semantics of the prompts.

3 Adversarial Content Attack

Based on the aforementioned framework and the full utilization of the diffusion model's capability, we achieve the unrestricted content-based attack known as Adversarial Content Attack (ACA) , as shown in Figure 2. Specifically, we first employ Image Latent Mapping (ILM) to map images onto the latent space represented by this low-dimensional manifold. Subsequently, we introduce an Adversarial Latent Optimization (ALO) technique that moves the latent representations of images along the adversarial direction on the manifold. Finally, based on iterative optimization, ACA can generate highly transferable unrestricted adversarial examples that appear quite natural. The

QKV

QKV

QKV

QKV

Text Encoder

Fire

Engine

Truck

Prompt:

A lego fire

truck with an american

flag on it

Text embedding

Image Latent Mapping

Adversarial Latent Optimization

T

Iterations

Grad

Skip

Grad

Constrain and

preserve image

content

Null text embedding

Condition

Classifier

Adversarial

example

Clean image

Skip

Grad

Grad

T

Iterations

Inversion of DDIM sampling

DDIM sampling

algorithm for ACA is presented in Algorithm 1, and we further combine the diffusion model to design the corresponding mapping and optimization methods.

3.1 Image Latent Mapping

For the diffusion model, the easiest image mapping is the inverse process of DDIM sampling [9, 48] with the condition embedding C = ψ ( P ) of prompts P , based on the assumption that the ordinary differential equation (ODE) process can be reversed in the limit of small steps:

where z 0 is the given real image, a schedule { β 0 , ..., β T } ∈ (0 , 1) and α t = ∏ t 1 (1 -β i ) . In general, this process is the reverse direction of the denoising process ( z 0 → z T instead of z T → z 0 ), which can map the image z 0 to z T in the latent space. Image prompts are automatically generated using image caption models (e.g., BLIP v2 [31]). For simplicity, the encoding of the V AE is ignored.

Text-to-image synthesis usually emphasizes the effect of the prompt. Therefore, a classifier-free guidance technique [19] is proposed. Its prediction is also performed unconditionally, which is then extrapolated with the conditioned prediction. Given w as the guidance scale parameter and ∅ = ψ ('') as the embedding of a null text, the classifier-free guidance prediction is expressed by:

where w = 7 . 5 is the default value for Stable Diffusion. However, since the noise is predicted by the model ϵ θ in the inverse process of DDIM sampling, a slight error is incorporated in every step. Due to the existence of a large guidance scale parameter w in the classifier-free guidance technique, slight errors are amplified and lead to cumulative errors. Consequently, executing the inverse process of DDIM sampling with classifier-free guidance not only disrupts the Gaussian distribution of noises but also induces visual artifacts of unreality [37].

To mitigate cumulative errors, we follow [37] and optimize a null text embedding ∅ t for each timestamp t . First, the inverse process of DDIM sampling with w = 1 outputs a series of consecutive latents { z ∗ 0 , ..., z ∗ T } where z ∗ 0 = z 0 . Then, we conduct the following optimization with w = 7 . 5 (the default value for Stable Diffusion) and ¯ z T = z t during N iterations for the timestamps t = { T, ..., 1 } :

At the end of each step, we update ¯ z t -1 to z t -1 ( ¯ z t , t, C , ∅ t ) . Finally, we obtain the latent of the given image in the low-dimensional manifold, consisting of the noise ¯ z T , the null text embedding ∅ t , and the text embedding C = ψ ( P ) . Note that compared to other strategies [12, 43, 26, 54], the current strategy is simple and effective and does not require fine-tuning to obtain high-quality image reconstruction. Next, we exploit this latent to generate unrestricted adversarial examples.

3.2 Adversarial Latent Optimization

In this section, we propose an optimization method for latents to maximize the attack performance on unrestricted adversarial examples. In the latent space of a given image after ILM, the null text embedding ∅ t ensures the quality of the reconstructed image, while the text embedding C ensures the semantic information of the image. Therefore, optimizing both embeddings may not be ideal. Considering that the noise ¯ z T largely represents the image's information in the latent space, we choose to optimize it instead. However, this optimization is still challenged by complex gradient calculations and the overflow of the value range.

Based on the latents generated by ILM, we define the denoising process of diffusion models as Ω( · ) through Equation 5, and it involves T iterations:

Therefore, the reconstructed image is denoted as ¯ z 0 = Ω( z T , T, C , { ∅ t } ) . The computational process of VAE is disregarded herein, as it is differentiable. Combining Equation 7, our adversarial objective optimization is expressed by:

where δ is the adversarial perturbation on the latent space. Our loss function consists of two parts: i) cross-entropy loss L ce , which mainly guides adversarial examples toward misclassification. ii) mean square error loss L mse mainly guides the generated adversarial examples to be as close as possible to clean images on l 2 distance. Therefore, the total loss function L is expressed as:

where β is 0 . 1 in this paper. The loss function L aims to maximize the cross-entropy loss and minimize the l 2 distance between the adversarial example ¯ z 0 and the clean image z 0 .

To ensure the consistency of z 0 and ¯ z 0 , we assume that δ does not change the consistency when δ is extremely small, i.e. , || δ || ∞ ≤ κ . The crux pertains to determining the optimal δ that yields the maximum classification loss. Analogous to conventional adversarial attacks, we employ gradientbased techniques to estimate δ through: δ ≃ η ∇ z T L ( F θ ( ¯ z 0 ) , y ) , where η denotes the magnitude of perturbations that occur in the direction of the gradient. To expand ∇ z T L ( F θ ( ¯ z 0 ) , y ) by the chain rule, we can have these derivative terms as follows:

Skip Gradient. After observing the items, we find that although each item is differentiable, it is not feasible to derive the entire calculation graph. First, we analyze the term ∂ L ∂ ¯ z 0 , which represents the derivative of the classifier with respect to the reconstructed image ¯ z 0 and provides the adversarial gradient direction. Then, for ∂z t ∂z t +1 , each calculation of the derivative represents the calculation of a backpropagation. Furthermore, a complete denoising process accumulates T calculation graphs, resulting in memory overflow (similar phenomena are also found in [45]). Therefore, the gradient of the denoising process cannot be directly calculated.

Fortunately, we propose a skip gradient to approximate ∂z 0 ∂z T = ∂ ¯ z 0 ∂z 1 · ∂z 1 ∂z 2 · · · ∂z T -1 ∂z T . Recalling the diffusion process, the denoising process aims to eliminate the Gaussian noise added in DDIM sampling [48, 9, 42]. DDIM samples z t at any arbitrary time step t in a closed form using reparameterization trick: √

Consequently, we perform a manipulation by rearranging Equation 10 to obtain z 0 = 1 √ α t z t -

√ 1 -α t α t ε . Hence, we further obtain ∂z 0 ∂z t = 1 √ α t . In Stable Diffusion, timestep t is at most 1000, so lim t → 1000 ∂z 0 ∂z t = lim t → 1000 1 √ α t ≈ 14 . 58 . In summary, ∂z 0 ∂z t can be regarded as a constant ρ and Equation 9 can be re-expressed as ∇ z T L ( F θ ( ¯ z 0 ) , y ) = ρ ∂ L ∂ ¯ z 0 . In summary, skip gradients approximate the gradients of the denoising process while reducing the computation and memory usage.

Differentiable Boundary Processing. Since the diffusion model does not explicitly constrain the value range of ¯ z 0 , the modification of z T may cause the value range to be exceeded. So we introduce

Algorithm 1 Adversarial Content Attack

differentiable boundary processing ϱ ( · ) to solve this problem. ϱ ( · ) constrains the values outside [0 , 1] to the range of [0 , 1] . The mathematical expression of DPB is as follows:

Next, we define Π κ as the projection of the adversarial perturbation δ onto κ -ball. We introduce momentum g and express the optimization adversarial latents as:

In general, Adversarial Latent Optimization (ALO) employs skip gradient to determine the gradient of the denoising process, and integrates differentiable boundary processing to regulate the value range of adversarial examples, and finally performs iterative optimization according to the gradient. Combined with Image Latent Mapping, Adversarial Content Attack is illustrated in Algorithm 1.

4 Experiments

4.1 Experimental Setup

Datasets. Our experiments are conducted on the ImageNet-compatible Dataset [29]. The dataset consists of 1,000 images from ImageNet's validation set [8], and is widely used in [10, 13, 58, 60].

Attack Evaluation. We choose SAE [20], ADer [1], ReColorAdv [30], cAdv [2], tAdv [2], ACE [61], ColorFool [47], NCF [60] as comparison methods of Adversarial Content Attack (ACA). The parameters for these unrestricted attacks follow the corresponding default settings. Our attack evaluation metric is the attack success rate (ASR, %), which is the percentage of misclassified images.

Models. To evaluate the adversarial robustness of network architectures, we select convolutional neural networks (CNNs) and vision transformers (ViTs) as the attacked models, respectively. For CNNs, we choose normally trained MoblieNet-V2 (MN-v2) [46], Inception-v3 (Inc-v3) [50], ResNet50 (RN-50) and ResNet-152 (RN-152) [18], Densenet-161 (Dense-161) [22], and EfficientNet-b7 (EFb7) [52]. For ViTs, we consider normally trained MoblieViT (MobViT-s) [35], Vision Transformer (ViT-B) [11], Swin Transformer (Swin-B) [34], and Pyramid Vision Transformer (PVT-v2) [55].

Implementation Details. Our experiments are run on an NVIDIA Tesla A100 with Pytorch. DDIM steps T = 50 , image mapping iteration N i = 10 , attack iterations N a = 10 , β = 0 . 1 , ζ = 0 . 01 , η = 0 . 04 , κ = 0 . 1 , and µ = 1 . The version of Stable Diffusion [42] is v1.4. Prompts for images are automatically generated using BLIP v2 [31].

4.2 Attacks on Normally Trained Models

In this section, we assess the adversarial transferability of normally trained convolutional neural networks (CNNs) and vision transformers (ViTs), including methods such as SAE [20], ADer [1], ReColorAdv [30], cAdv [2], tAdv [2], ACE [61], ColorFool [47], NCF [60], and our ACA. Adversarial examples are crafted via MobViT-s, MN-v2, RN-50, and ViT-B, respectively. Avg. ASR (%) refers to the average attack success rate on non-substitute models.

Table 1 illustrates the performance comparison of adversarial transferability on normally trained CNNs and ViTs. It can be observed that adversarial examples by ours generally exhibit superior transferability compared to those generated by state-of-the-art competitors and the impact of ILM on ASR is exceedingly marginal. When CNNs (RN-50 and MN-v2) are used as surrogate models, our ACA exhibits minimal differences with state-of-the-art NCF in MN-v2, RN-50, and RN-152. However, in Inc-v3, Dense-161, and EF-b7, such as when RN-50 is used as the surrogate model, we significantly outperform NCF by 28.0% , 13.4% and 27.9% , respectively. This indicates that our ACA has higher transferability in heterogeneous CNNs. Furthermore, our ACA demonstrates state-ofthe-art transferability in current unconstrained attacks under the more challenging cross-architecture

setting. Specifically, when the surrogate model is RN-50, we surpass NCF by significant margins of 10.0% , 16.1% , 32.1% , and 32.5% in MobViT-s, ViT-B, Swin-B, and PVT-v2, respectively. There are two primary reasons for this phenomenon: i) our ACA utilizes a low-dimensional manifold search of natural images for adversarial examples, with the manifold itself determining the transferability of the adversarial examples, independent of the model's architecture; ii) the diffusion model incorporates the self-attention structure, exhibiting a certain degree of architectural similarity.

Overall, the deformation-based attack (ADer) exhibits lower attack performance in both white-box and black-box settings. Texture-based attacks (tAdv) show better white-box attack performance, but are less transferable than existing color-based attacks (NCF and SAE). Our ACA leverages the low-dimensional manifold of natural images to adaptively combine image attributes and generate unrestricted adversarial examples, resulting in a significant outperformance of state-of-the-art methods by 13.3% ∼ 50.4% on average. These results convincingly demonstrate the effectiveness of our method in fooling normally trained models.

4.3 Attacks on Adversarial Defense Methods

The situation in that adversarial defense methods can effectively protect against current adversarial attacks exhibits the Dunning-Kruger effect [28]. Actually, such defense methods demonstrate efficacy in defending against adversarial examples within the l p norm, yet their robustness falters in the face of novel and unseen attacks [25]. Therefore, we investigate whether unrestricted attacks can break through existing defenses. Here, we choose input pre-process defenses (HGD [33], R&P [57], NIPS-r3 3 , JPEG [17], Bit-Red [59], and DiffPure [39]) and adversarial training models (Inc-v3 ens 3 , Inc-v3 ens 4 , and Inc-v2 ens [53]). Considering that some unrestricted attacks are carried out from the perspective of shape and texture, we also choose shape-texture debaised models (ResNet50-Debaised (Res-De) [32] and Shape-ResNet (Shape-Res) [14]).

The results of black-box transferability in adversarial defense methods are demonstrated in Table 2. the surrogate model is ViT-B, and the target model is Inc-v3 ens 3 for input pre-process defenses. Our method displays persistent superiority over other advanced attacks by a significant margin. Our ACA surpasses NCF, SAE, ColorFool by 27.13% , 27.63% , and 37.27% on average ASR. In robust models, based on l p adversarial training and shape-texture debiased models are not particularly effective and can still be easily broken by unrestricted adversarial examples. Our approach can adaptively generate various combinations of adversarial examples based on the manifold, thus exhibiting high transferability to different defense methods. Additionally, Bit-Red and Diffpure reduce the groundtruth class's confidence and increase the adversarial examples' transferability. These findings further reveal the incompleteness and vulnerability of existing adversarial defense methods.

4.4 Visualization

Quantitative Comparison. Following [47] and [60], we quantitatively assess the image quality using the non-reference perceptual image quality measure. Therefore, we choose NIMA [51], HyperIQA [49], MUSIQ [27], and TReS [15]. NIMA-AVA and MUSIQ-AVA are trained on AVA [38], and MUSIQ-KonIQ is trained on KonIQ-10K [21], following PyIQA [3]. As illustrated in Table 3, our ILM maintains the same image quality as clean images, and ACA achieves the best results in

Clean

ILM

SAE

cAdv

tAdv

ColorFool

NCF

ACA (Ours)

(a) Visualization of state-of-the-art unrestricted attacks

(b) Adversarial examples of Adversarial Content Attack (ACA)

(c) Case Study

image quality assessment. ColorFool obtains equal or higher image quality than the clean images because it requires manually selecting several human-sensitive semantic classes and adds uncontrolled perturbations on human-insensitive semantic classes. In other words, ColorFool is bound by human intuition, so do not deteriorate the perceived image quality (our results are similar to [47]). ACA even surpasses ColorFool, mainly because: i) Our adversarial examples are generated based on the low-dimensional manifold of natural images, which can adaptively combine the adversarial content and ensure photorealism; ii) Stable Diffusion itself is an extremely powerful generation model, which produces images with very high image quality; iii) These no-reference image metrics are often trained on aesthetic datasets, such as A V A [38] or KonIQ-10K [21]. Some of the images in these datasets are post-processed (such as Photoshop), which is more in line with human aesthetics. Because ACA adaptively generates adversarial examples on a low-dimensional manifold, this kind of minor image editing is similar to post-processing, which is more in line with human aesthetic perception and better image quality.

Qualitative Comparison. We visualize unrestricted attacks of Top-5 black-box transferability, including SAE, cAdv, tAdv, ColorFool, and NCF. In Figure 3(a), we visualize adversarial examples generated by different unrestricted attacks. In night scenes and food, color and texture changes are easily perceptible, while our method still keeps image photorealism. Next, we give more adversarial examples generated by ACA. It is clearly observed that our method can adaptively combine content to generate adversarial examples, as shown in Figure 3(b). For example, the hot air balloon in the lower left corner modifies both the color of the sky and the texture of the hot air balloon. The straw-

berry in the lower right corner has some changes in shape and color while keeping the semantics unchanged. However, in some cases, the body of semantics changes, as shown in Figure 3(c). It may be because the prompts generated by BLIP v2 cannot describe the content of the image well.

4.5 Time Analysis

In this section, we illustrate the attack speed of various unrestricted attacks. We choose MN-v2 [46] as the surrogate model and evaluate the inference time on an NVIDIA Tesla A100. Table 4 shows the

average time (in seconds) required to generate an adversarial example per image. ACA does have a significant time cost compared to other attacks. Further, we analyze the time cost and find that Image Latent Mapping (ILM) and Adversarial Latent Optimization (ALO) each accounted for 50% of the time cost. However, most of the time cost of ILM and ALO lies in the sampling process of the diffusion model. In this paper, our main contribution is to propose a new unrestricted attack paradigm. Therefore, we focus on the improvement of the attack framework, rather than the optimization of time cost. Since the time cost is mainly focused on the sampling of the diffusion model, we have noticed that many recent works have accelerated or distilled the diffusion model, which can greatly reduce the time of the sampling process. For example, [36] can reduce the total number of sampling steps by at least 20 times. If these acceleration technologies are applied to our ACA, ACA can theoretically achieve an attack speed of close to 6 seconds and we think this is a valuable optimization direction.

4.6 Ablation Study

The ablation studies of momentum (MO) and differentiable boundary processing (DBP) are shown in Figure 4 and the surrogate model is ViT-B [11]. Origin stands for ACA without MOand DBP. MO is Origin with momentum, and it can be observed that the adversarial transferability is significantly improved after the introduction of momentum. MO+DBP is Origin with momentum and DBP. Since DBP further optimizes the effectiveness of the adversarial examples and constrains the image within the range of values, it can still

Origin

80

MO

Rate

MO+DBP

70

Success

60

Attack

50

40

30

MN-v2

Ine-v3

RN50

RN-152

EF-b7

VIT-B*

Swin-B

PVT-v2

Dense-161

MobViT-s

improve the adversarial transferability. Although the above strategies are not the main contribution of this paper, the above experiments illustrate that they can boost adversarial transferability.

5 Conclusions

In this paper, we propose a novel unrestricted attack framework called Content-based Unrestricted Adversarial Attack. We map the image onto a low-dimensional manifold of natural images. When images are moved along the adversarial gradient on the manifold, unrestricted adversarial examples with diverse adversarial content and photorealistic can be adaptively generated. Based on the diffusion model, we implement Adversarial Content Attack. Experiments show that the existing defense methods are not very effective against unrestricted content-based attacks. We propose a new form of unrestricted attack, hoping to draw attention to the threat posed by unrestricted attacks.

Limitations. Due to the inherent limitations of diffusion models, a considerable number of sampling steps are required in the inference process, resulting in a relatively longer runtime, taking ∼ 2 . 5 minutes using a single A100 GPU. Additionally, the current adaptive generation of unrestricted adversarial examples does not allow for fine-grained image editing. These issues are expected to be resolved in the future with further advancements in diffusion models.

Negative Social Impacts. Adversarial examples from our method exhibit a high photorealism and strong black-box transferability, which raises concerns about the robustness of DNNs, as malicious adversaries may exploit this technique to mount attacks against security-sensitive applications.

Acknowledgements. This work was supported by National Natural Science Foundation of China (No.62072112), Scientific and Technological innovation action plan of Shanghai Science and Technology Committee (No.22511102202).

References

Rima Alaifari, Giovanni S. Alberti, and Tandri Gauksson. Adef: an iterative algorithm to construct adversarial deformations. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net, 2019. 2, 6, 7

Anand Bhattad, Min Jin Chong, Kaizhao Liang, B. Li, and David A. Forsyth. Unrestricted adversarial examples via semantic manipulation. In International Conference on Learning Representations , 2019. 2, 6, 7

Chaofeng Chen and Jiadi Mo. IQA-PyTorch: Pytorch toolbox for image quality assessment. [Online]. Available: https://github.com/chaofengc/IQA-PyTorch , 2022. 8

Zhaoyu Chen, Bo Li, Shuang Wu, Shouhong Ding, and Wenqiang Zhang. Query-efficient decision-based black-box patch attack. IEEE Transactions on Information Forensics and Security , 18:5522-5536, 2023. 1

Zhaoyu Chen, Bo Li, Shuang Wu, Jianghe Xu, Shouhong Ding, and Wenqiang Zhang. Shape matters: deformable patch attack. In European Conference on Computer Vision , pages 529-548. Springer, 2022. 1

Zhaoyu Chen, Bo Li, Jianghe Xu, Shuang Wu, Shouhong Ding, and Wenqiang Zhang. Towards practical certifiable patch defense with vision transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15148-15158, 2022. 1

Yiting Cheng, Fangyun Wei, Jianmin Bao, Dong Chen, and Wenqiang Zhang. Adpl: Adaptive dual path learning for domain adaptation of semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023. 1

Jia Deng, Alex Berg, Sanjeev Satheesh, H Su, Aditya Khosla, and Li Fei-Fei. Imagenet large scale visual recognition competition 2012 (ilsvrc2012). See net. org/challenges/LSVRC , 41, 2012. 6

Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat gans on image synthesis. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual , pages 8780-8794, 2021. 3, 4, 5

Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading defenses to transferable adversarial examples by translation-invariant attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4312-4321, 2019. 6

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 , 2021. 6, 10

Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. CoRR , abs/2208.01618, 2022. 4

Lianli Gao, Qilong Zhang, Jingkuan Song, Xianglong Liu, and Heng Tao Shen. Patch-wise attack for fooling deep neural network. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXVIII 16 , pages 307-322. Springer, 2020. 6

Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net, 2019. 8

S Alireza Golestaneh, Saba Dadsetan, and Kris M Kitani. No-reference image quality assessment via transformers, relative ranking, and self-consistency. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 1220-1230, 2022. 8

Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , 2015. 1

Chuan Guo, Mayank Rana, Moustapha Cissé, and Laurens van der Maaten. Countering adversarial images using input transformations. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . OpenReview.net, 2018. 8

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016 , pages 770-778, 2016. 1, 6

Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. CoRR , abs/2207.12598, 2022. 4

Hossein Hosseini and Radha Poovendran. Semantic adversarial examples. In 2018 IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2018, Salt Lake City, UT, USA, June 18-22, 2018 , pages 1614-1619. Computer Vision Foundation / IEEE Computer Society, 2018. 2, 6, 7

V. Hosu, H. Lin, T. Sziranyi, and D. Saupe. Koniq-10k: An ecologically valid database for deep learning of blind image quality assessment. IEEE Transactions on Image Processing , 29:4041-4056, 2020. 8, 9

Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. Densely connected convolutional networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017 , pages 2261-2269. IEEE Computer Society, 2017. 6

Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. Image-to-image translation with conditional adversarial networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017 , pages 5967-5976. IEEE Computer Society, 2017. 1

Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II , volume 9906 of Lecture Notes in Computer Science , pages 694-711. Springer, 2016. 1

Daniel Kang, Yi Sun, Dan Hendrycks, Tom Brown, and Jacob Steinhardt. Testing robustness against unforeseen adversaries. arXiv preprint arXiv:1908.08016 , 2019. 1, 8

Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. CoRR , abs/2210.09276, 2022. 4

Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 5148-5157, 2021. 8

Justin Kruger and David Dunning. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of personality and social psychology , 77(6):1121, 1999. 1, 8

Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Artificial intelligence safety and security , pages 99-112. Chapman and Hall/CRC, 2018. 6

Cassidy Laidlaw and Soheil Feizi. Functional adversarial attacks. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d'Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada , pages 10408-10418, 2019. 2, 6, 7

Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping languageimage pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597 , 2023. 4, 7

Yingwei Li, Qihang Yu, Mingxing Tan, Jieru Mei, Peng Tang, Wei Shen, Alan L. Yuille, and Cihang Xie. Shape-texture debiased neural network training. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net, 2021. 8

Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018 , pages 1778-1787. Computer Vision Foundation / IEEE Computer Society, 2018. 8

Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021 , pages 9992-10002, 2021. 6

Sachin Mehta and Mohammad Rastegari. Mobilevit: Light-weight, general-purpose, and mobilefriendly vision transformer. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 , 2022. 6

Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik P. Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023 , pages 14297-14306, 2023. 10

Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023 , pages 6038-6047, 2023. 4

Naila Murray, Luca Marchesotti, and Florent Perronnin. Ava: A large-scale database for aesthetic visual analysis. In 2012 IEEE conference on computer vision and pattern recognition , pages 2408-2415. IEEE, 2012. 8, 9

Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Animashree Anandkumar. Diffusion models for adversarial purification. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA , volume 162 of Proceedings of Machine Learning Research , pages 16805-16827. PMLR, 2022. 8

Haonan Qiu, Chaowei Xiao, Lei Yang, Xinchen Yan, Honglak Lee, and Bo Li. Semanticadv: Generating adversarial examples via attribute-conditioned image editing. In Computer Vision ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIV , volume 12359 of Lecture Notes in Computer Science , pages 19-37. Springer, 2020. 2

Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with CLIP latents. CoRR , abs/2204.06125, 2022. 3

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. Highresolution image synthesis with latent diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022 , pages 10674-10685, 2022. 3, 5, 7

Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. CoRR , abs/2208.12242, 2022. 4

Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems , 35:36479-36494, 2022. 3

Hadi Salman, Alaa Khaddaj, Guillaume Leclerc, Andrew Ilyas, and Aleksander Madry. Raising the cost of malicious ai-powered image editing. CoRR , abs/2302.06588, 2023. 5

Mark Sandler, Andrew G. Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018 , pages 4510-4520, 2018. 6, 9

Ali Shahin Shamsabadi, Ricardo Sánchez-Matilla, and Andrea Cavallaro. Colorfool: Semantic adversarial colorization. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020 , pages 1148-1157. Computer Vision Foundation / IEEE, 2020. 2, 3, 6, 7, 8, 9

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net, 2021. 4, 5

Shaolin Su, Qingsen Yan, Yu Zhu, Cheng Zhang, Xin Ge, Jinqiu Sun, and Yanning Zhang. Blindly assess image quality in the wild guided by a self-adaptive hyper network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3667-3676, 2020. 8

Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016 , pages 2818-2826, 2016. 6

Hossein Talebi and Peyman Milanfar. Nima: Neural image assessment. IEEE transactions on image processing , 27(8):3998-4011, 2018. 8

Mingxing Tan and Quoc V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA , volume 97 of Proceedings of Machine Learning Research , pages 6105-6114. PMLR, 2019. 6

Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian J. Goodfellow, Dan Boneh, and Patrick D. McDaniel. Ensemble adversarial training: Attacks and defenses. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . OpenReview.net, 2018. 8

Dani Valevski, Matan Kalman, Yossi Matias, and Yaniv Leviathan. Unitune: Text-driven image editing by fine tuning an image generation model on a single image. CoRR , abs/2210.09477, 2022. 4

Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. PVT v2: Improved baselines with pyramid vision transformer. Comput. Vis. Media , 8(3):415-424, 2022. 6

Chaowei Xiao, Jun-Yan Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song. Spatially transformed adversarial examples. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . OpenReview.net, 2018. 2

Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan L. Yuille. Mitigating adversarial effects through randomization. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . OpenReview.net, 2018. 8

Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L Yuille. Improving transferability of adversarial examples with input diversity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2730-2739, 2019. 6

Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. In 25th Annual Network and Distributed System Security Symposium, NDSS 2018, San Diego, California, USA, February 18-21, 2018 . The Internet Society, 2018. 8

Shengming Yuan, Qilong Zhang, Lianli Gao, Yaya Cheng, and Jingkuan Song. Natural color fool: Towards boosting black-box unrestricted attacks. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 7546-7560. Curran Associates, Inc., 2022. 2, 3, 6, 7, 8

Zhengyu Zhao, Zhuoran Liu, and Martha A. Larson. Adversarial color enhancement: Generating unrestricted adversarial images by optimizing a color filter. In 31st British Machine Vision Conference 2020, BMVC 2020, Virtual Event, UK, September 7-10, 2020 . BMVA Press, 2020. 2, 6, 7

Zhengyu Zhao, Zhuoran Liu, and Martha A. Larson. Towards large yet imperceptible adversarial image perturbations with perceptual color distance. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020 , pages 10361045. Computer Vision Foundation / IEEE, 2020. 1

Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Semantic understanding of scenes through the ADE20K dataset. Int. J. Comput. Vis. , 127(3):302-321, 2019. 3

Yao Zhu, Yuefeng Chen, Xiaodan Li, Kejiang Chen, Yuan He, Xiang Tian, Bolun Zheng, Yaowu Chen, and Qingming Huang. Toward understanding and boosting adversarial transferability from a distribution perspective. IEEE Trans. Image Process. , 31:6487-6501, 2022. 1

Peer reviews:
comments: SUMMARY:
This paper proposes a novel unrestricted attack framework called Content-based Unrestricted Adversarial Attack (ACA). The author argues that current unrestricted attacks have limitations in terms of maintaining human visual imperceptibility, generating natural adversarial examples, and achieving high attack performance. This paper is well-written and easy to follow.

STRENGTHS:
1) The paper introduces a novel attack framework that addresses the limitations of current unrestricted attacks. The use of a low-dimensional manifold and optimization along its adversarial direction allows for the generation of diverse and natural adversarial examples.
2) The paper provides a clear motivation and problem definition, outlining the challenges and goals for unrestricted adversarial attacks.
3) The paper includes extensive experimentation and visualization to validate the effectiveness of ACA. The results show significant improvements over state-of-the-art attacks in terms of adversarial transferability.

WEAKNESSES:
The paper does not include sufficient evaluation and ablation studies for the proposed method. Since ACA used skip gradient, I think I should compare Skip connections matter [1] in the experiment.

The paper could benefit from a more thorough review of relevant literature. While the authors mention existing unrestricted attacks, there is limited discussion on related work and the novelty  of ACA compared to previous approaches.



[1] Skip connections matter: On the transferability of adversarial examples generated with resnets, ICLR 2019

QUESTIONS:
/Na

LIMITATIONS:
/Na
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a novel attack framework called Content-based Unrestricted Adversarial Attack, which aims to generate diverse and natural adversarial examples with high transferability. The authors argue that existing methods, such as lp norm-based attacks, have limitations in terms of perceptual similarity, naturalness, and robustness. To address these issues, they propose mapping images onto a low-dimensional manifold represented by a generative model trained on natural images. This manifold ensures both photorealism and content diversity. By optimizing the adversarial objective on this latent space, they generate unrestricted adversarial examples. The proposed method, called Adversarial Content Attack (ACA), utilizes Image Latent Mapping (ILM) and Adversarial Latent Optimization (ALO) techniques to optimize the latent in a diffusion model. The effectiveness of ACA is validated through experiments and visualization, demonstrating significant improvements of 13.3~50.4% in terms of adversarial transferability compared to state-of-the-art attacks. \n\nOverall, the main contributions of the paper are the introduction of the Content-based Unrestricted Adversarial Attack framework, the development of the Adversarial Content Attack method, and the experiments demonstrating improvements in generating diverse and transferable adversarial examples.\n\nSTRENGTHS:\nThe authors effectively communicated the motivation, problem statement, and methodology of the proposed framework. By addressing the limitations (imperceptibility/photorealism/effectiveness) of existing methods, they proposed a novel attack framework that leverages a low-dimensional manifold represented by a generative model. By combining image mapping onto a latent space, optimizing adversarial objectives, and utilizing a diffusion model, the authors introduce a novel approach to generating diverse and natural adversarial examples. This paper might be the first to explore unrestricted adversarial examples through such a framework.\n\nThis paper offers a thorough explanation of the proposed attack framework, detailing the underlying techniques of ILM and ALO. The authors further support their claims through experimentation and visualization, providing evidence of the effectiveness of their approach and demonstrating improvements in adversarial transferability compared to state-of-the-art attacks. The improvements in adversarial transferability also shed light on the potential impact of this method in uncovering vulnerabilities in security-sensitive applications and advancing our understanding of robustness in DNNs.\n\nOverall, this paper's strengths encompass originality in proposing a novel attack framework, quality in terms of methodology and experimental evaluation, clarity in explaining the concepts and techniques, and significance in addressing limitations and raising awareness of unrestricted but realistic adversarial examples.\n\nWEAKNESSES:\nWhile the paper has several strengths, there are some weaknesses that could be addressed to further improve the work:\n\n**Comparison with State-of-the-Art Attacks:** While the paper mentions that the proposed method achieves significant improvements in terms of adversarial transferability compared to state-of-the-art attacks, a more comprehensive comparison would strengthen the evaluation. It would be valuable to include a thorough analysis and comparison with a wider range of existing unrestricted attack methods, such as [1]. In particular, Laidlaw et al. proposed an efficient way to generate imperceptible adversarial examples. The reviewer also suggests evaluating the proposed attack on other adversarially trained models, for example, the defense method that could be generalized to unforeseen perturbations [1], or having used synthetic data during adversarial training [2].\n\n**Defense Method:** This paper does not discuss how to defend the proposed attacks but primarily on the efficacy of the proposed method. It would be great if the authors provide potential solutions or mitigation strategies for the threats.\n\n**Generalization to Different Datasets:** The authors only evaluate their method on a subset of the ImageNet validation set and do not say how the results generalize to other datasets. It would be beneficial to investigate the generalization of the proposed approach to different datasets.\n\n**Unclear Claim:** The authors mentioned the Dunning-Kruger effect to emphasize that current defense methods against lp norm adversarial examples overestimate their abilities. However, this work does not provide further details and arguments to support this. Although similar arguments have also been proposed in [3], in which the authors argue that lp-based robustness evaluation might be biased, the reviewer thinks that using the Dunning-Kruger effect here is not rigorous. The reviewer suggests that the authors rethink this argument, and even consider removing it if it is not an important contribution of the paper.\n\n**Typos:** Although this paper is well-written and easy to follow, the reviewer found some typos and grammar errors. For example, in line 161, *follw*. It would be great if the authors have proofread the paper before submitting it.\n\n    [1] Laidlaw et al. Perceptual adversarial robustness: Defense against unseen threat models. (ICLR 2021)\n    [2] Croce et al. Robustbench: a standardized adversarial robustness benchmark. (NeurIPS 2021)\n    [3] Hsiung et al. CARBEN: Composite Adversarial Robustness Benchmark. (IJCAI 2022)\n\nQUESTIONS:\nPlease refer to the weakness part. In brief, the reviewer would like to understand more about the following points:\n\n- Could the authors provide more attack and defense baselines as mentioned in the weakness?\n- Could the authors provide some experiments on other datasets?\n- Please address the mentioned unclear claim.\n- Why the authors did not provide the code for review but answered the Reproducibility as \"yes\"?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, authors propose an unrestricted untargeted attack based on optimising the latent space of stable diffusion model. The generated adversarial samples are empirically shown to be more transferable than the existing semantic attacks. Moreover, authors validate the effectiveness of adversarial samples when attacking different representative adversarial trained models and show consistent performance book.\n\nSTRENGTHS:\n- The method is a replacement of popular GANs in the semantic attacks with the powerful stable diffusion models, pushing the state-of-the-art by many steps emperically.\n- Experiments are thorough in attacking both normal and adversarial trained models\n- Compared against many baseline unrestricted attacks and proposed attacks outperforms all the methods.\n\nWEAKNESSES:\n-  The paper  lacks technical novelty as the latent space optimisation for generating adversarial attacks as been popular from many years [17].\n\n- Since this is a unrestricted attack, the generated image quality is difficult to assess with metrics. Nonetheless, authors have computed 5 metrics to understand the image quality\n\n- Authors did not discuss about the code release to reproduce the experiments. For this particular paper, the implementation of this proposed method is not simple as the section 3.1 and 3.2 mostly discusses about the difficulties in optimising the latent space and how they overcome it through skip gradient, momentum and boundary function.\n\nQUESTIONS:\nOverall, I do not have major questions as the proposed method is properly motivated and leverages the powerful generative model to aid the attack generation. However, I have few minor questions mostly around implementation and design choices:\n\n- In section 3.1, authors propose to optimise the null text embedding $∅_t$ at every timestamp to offset the error. Is there any difference to your implementation as compared to the method of [32]. Can you please clarify your contribution in the section 3.1 ie. mapping image $z_0$ to latent space $z_t$.  \n\n- In particular, authors show the benefit of momentum factor $\\mu$  and boundary function $ϱ(·)$ to improve the ASR in Appendix D.  Did you perform ablation study to set the value of  $\\mu$ to 1 and can authors provide more insights on the design of boundary function in particular for inputs outside the valid range [0, 1]?  \n\n- How do you ensure the perturbation value of $k = 0.1$ in the latent space is not drastically modifying the original image.  Can you present an ablation of this key parameter vs attack performance vs image quality ?  In Table 3, there is no discussion about the non-reference metrics mentioned in the paper. Why is the method not evaluated with metrics such as LPIPS that captures perceptual similarity? Please also include FID to the metrics.  \n\n- How is the performance of the attacks in targeted attack setting? Do you require larger shift in latent space to craft example? \n\n- Please also benchmark the baseline attacks in terms of attack speed for completeness.\n\n- Can you constrain the perturbation to a local region in latent space to generate a kind of patch attacks in image space? \n\nFinally, I request the authors to discuss their plans in releasing the codebase to reproduce the experiments as I believe this will be invaluable to the community.\n\nLIMITATIONS:\n- The inference time of the attack is much higher taking 2.5 minutes for image due to many many levels of optimisation such as for null text embedding, perturbation in latent space.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes the Adversarial Content Attack based on the diffusion model. The proposed attack method first maps the image onto a low-dimensional manifold of natural images and then moves images along the adversarial gradient on the manifold to generate photorealistic adversarial examples. The authors conduct extensive experimentation and demonstrate the efficacy of the Adversarial Content Attack in both normally trained models and defense methods.\n\nSTRENGTHS:\n1. The paper presents a host of visualization and experiments to support the intuitions and conclusions.\n2. The investigated topic is important and useful.\n\nWEAKNESSES:\n1. This work seems like an application of the diffusion model in model debugging research, finding some hard samples of deep models. However, it lacks mentions or comparisons with related work, including [1],[2], etc.\n\n2. In Table 3, I can hardly understand why the generated adversarial image can achieve better photorealism than the real image.\n\n3. In Table 2, Inc-v3$_{ens4}$ is used as the target model, which is a defense method. But confusingly, the attack success rate (**62.2**) surpasses the case when using normal Inc-v3 (**58.8**) as the target.\n\nI will be pleased to raise my score if these questions can be properly answered.\n\n[1] Xiaodan Li, Yuefeng Chen, Yao Zhu, Shuhui Wang, Rong Zhang, Hui Xue. ImageNet-E: Benchmarking Neural Network Robustness via Attribute Editing.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.\n\n[2] Maximilian Augustin, Valentyn Boreiko, Francesco Croce, Matthias Hein. Diffusion Visual Counterfactual Explanations. Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021.\n\nQUESTIONS:\nThis paper mainly focuses on the untargeted attack. I wonder whether the proposed attack method can also be applied to the targeted attack case, that is, to specify the misclassified category.\n\nLIMITATIONS:\nThe paper has discussed the limitations and negative social impacts of the proposed attack method.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-14"}