Title: DarkSAM: Fooling Segment Anything Model to Segment Nothing
Venue: NeurIPS 2024
Accepted: yes
Keywords: Segment Anything Model

Adversarial Example
Abstract: Segment Anything Model (SAM) has recently gained much attention for its outstanding generalization to unseen data and tasks. Despite its promising prospect, the vulnerabilities of SAM, especially to universal adversarial perturbation (UAP) have not been thoroughly investigated yet. In this paper, we propose DarkSAM, the first prompt-free universal attack framework against SAM, including a semantic decoupling-based spatial attack and a texture distortion-based frequency attack. We first divide the output of SAM into foreground and background. Then, we design a shadow target strategy to obtain the semantic blueprint of the image as the attack target. DarkSAM is dedicated to fooling SAM by extracting and destroying crucial object features from images in both spatial and frequency domains. In the spatial domain, we disrupt the semantics of both the foreground and background in the image to confuse SAM. In the frequency domain, we further enhance the attack effectiveness by distorting the high-frequency components (i.e., texture information) of the image. Consequently, with a single UAP, DarkSAM renders SAM incapable of segmenting objects across diverse images with varying prompts. Experimental results on four datasets for SAM and its two variant models demonstrate the powerful attack capability and transferability of DarkSAM. Our codes are available at: https://github.com/CGCL-codes/DarkSAM.

DarkSAM: Fooling Segment Anything Model to Segment Nothing

Abstract

Segment Anything Model (SAM) has recently gained much attention for its outstanding generalization to unseen data and tasks. Despite its promising prospect, the vulnerabilities of SAM, especially to universal adversarial perturbation (UAP) have not been thoroughly investigated yet. In this paper, we propose DarkSAM, the first prompt-free universal attack framework against SAM, including a semantic decoupling-based spatial attack and a texture distortion-based frequency attack. We first divide the output of SAM into foreground and background. Then, we design a shadow target strategy to obtain the semantic blueprint of the image as the attack target. DarkSAM is dedicated to fooling SAM by extracting and destroying crucial object features from images in both spatial and frequency domains. In the spatial domain, we disrupt the semantics of both the foreground and background in the image to confuse SAM. In the frequency domain, we further enhance the attack effectiveness by distorting the high-frequency components ( i.e. , texture information) of the image. Consequently, with a single UAP, DarkSAM renders SAM incapable of segmenting objects across diverse images with varying prompts. Experimental results on four datasets for SAM and its two variant models demonstrate the powerful attack capability and transferability of DarkSAM. Our codes are available at: https://github.com/CGCL-codes/DarkSAM.

1 Introduction

With the advancement of deep learning, large language models, such as GPT [2], LaMDA [33], and PaLM [6], have achieved tremendous success, yet the development of large vision models lags behind. Recently, Segment Anything Model (SAM) [19] was proposed as a foundational vision model, demonstrating exceptional generalization capabilities for handling complex segmentation tasks. Unlike traditional segmentation models [24, 42] that output pixel-level labels, SAM introduces a novel prompt-guided image segmentation paradigm by directly producing label-free masks for object segmentation. Benefiting from its powerful zero-shot capability, SAM has been rapidly deployed across various downstream scenarios, such as medical images [34], videos [36], and 3D point clouds [12].

Segment Anything Model (SAM)

UAP

image embedding

Mask decoder

Prompt encoder

prompt

'Segment

Anything

'

'Segment

Nothing

'

Adversary

Deep neural networks (DNNs) are shown to be vulnerable to adversarial examples [16, 21, 41, 46], and SAM is no exception. Standard adversarial attacks are designed for classification tasks and cause misclassification by manipulating global image features through image-level perturbations. Existing attacks can be divided into crafting sample-wise adversarial perturbation [25] and universal adversarial perturbation (UAP) [27]. The former is tailored for specific inputs, while the latter seeks a single perturbation applicable across a wide range of inputs, thereby intensifying its complexity and difficulty. As a pioneering prompt-guided segmentation model, SAM relies on both input images and prompts to yield label-free masks, rendering existing adversarial attacks [1, 10, 25, 26] focusing only for images and relying on labels ineffective.

Recent efforts [17, 39] started to explore the robustness of SAM against sample-wise adversarial perturbations. Attack-SAM [39] employs classical FGSM [10] and PGD [25] to remove or manipulate the predicted mask for a given image and prompt pair. Meanwhile, another study [17] also investigates the robustness of SAM against various adversarial attacks and corrupted images. However, the more challenging universal adversarial attacks, which more closely resemble real-world scenarios, remain far less thoroughly explored. The introduction of extra and varying prompts in SAM's input, coupled with the lack of label information in its output for attack optimization, renders attacking SAM exceedingly challenging, posing an intriguing problem:

Is it feasible to fool the Segment Anything model to segment nothing through a single UAP?

In this paper, we take a substantial step towards bridging the gap between SAM and UAP. We propose DarkSAM, the first truly prompt-free universal adversarial attack on the prompt-guided image segmentation models ( i.e. , SAM and its variants), aiming to disable their segmentation ability across diverse input images using a single UAP, irrespective of prompts (see Fig. 1). Unlike classification models that focus on global features, prompt-guided segmentation models concentrate more on local critical objects within images ( e.g. , objects indicated by prompts). Therefore, our intuition is to destroy crucial object features in the image to mislead SAM into incorrectly segmenting the input images. To this end, DarkSAM is dedicated to decoupling the crucial object features of images from both spatial and frequency domains, utilizing a UAP to disrupt them. 1) In the spatial domain, we begin by dividing SAM's output into foreground ( i.e. , positive mask values) and background ( i.e. , negative mask values) via a Boolean mask. We then scramble SAM's decision by destroying the features of the foreground and background of the image, respectively. 2) In the frequency domain, inspired by the factor that SAM is biased towards image texture over shape [38], we employ a frequency filter to decompose images into high-frequency components (HFC) and low-frequency components (LFC). By increasing the dissimilarity in the HFC of adversarial and benign examples while maintaining consistency in their LHC, we further enhance the effectiveness and transferability of UAP. Experimental results on four segmentation benchmark datasets for SAM and its two variant models, HQ-SAM [18] and PerSAM [40], demonstrate that DarkSAM achieves high attack success rates and transferability.

Our main contributions are summarized as follows:

We propose DarkSAM, the first truly universal adversarial attack against SAM. We employ a single perturbation to prevent SAM from segmenting objects across a range of images under any form of prompt, which further unveils its vulnerability.

We design a brand-new prompt-free hybrid spatial-frequency universal attack framework against the prompt-guided image segmentation models to generate a UAP thus making them segment nothing, which consists of a semantic decoupling-based spatial attack and a texture distortion-based frequency attack.

We conduct extensive experiments on four datasets for SAM and its two variant models. Both the qualitative and quantitative results demonstrate that DarkSAM achieves high attack success rates and transferability.

2 Background and Related Works

2.1 Prompt-guided Image Segmentation

Segment Anything Model [19] is a cutting-edge advancement in computer vision, garnering widespread attention [3, 5, 20, 22, 34] for its powerful segmentation capabilities. Recent works have been dedicated to exploring various variants of SAM to further enhance performance, such as HQ-SAM [18], PerSAM [40], and MobileSAM [37]. Distinct from traditional semantic segmentation models [4, 24, 42] that predominantly focus on pixel-level label prediction, SAM undertakes the label-free mask prediction by generating object masks for a wide array of subjects using prompts. It consists of three components: an image encoder, a prompt encoder, and a lightweight mask decoder. The image encoder generates image representations in latent space and the prompt encoder utilizes positional embeddings for representing prompts, such as points and boxes. The mask decoder, combining outputs from both image and prompt encoders, predicts effective masks to segment targeted objects.

Given an image x ∈ R H × W × C and a corresponding prompt P to SAM, denoted as f ( x, P ) ∈ R H × W , the model returns a mask m with the predicted segmentation. The prediction process of SAM can be represented as follow:

where θ represents the parameter of f ( · ) . For an image x , each pixel located at coordinates ( i, j ) , referred to as x ij , is deemed a part of the masked region when its corresponding mask value m ij exceeds a defined threshold of zero. Recent exploratory studies [39, 17, 35] have revealed vulnerabilities of SAM to adversarial examples and common image corruptions. Different from previous works, our goal is to develop a powerful universal adversarial attack for such prompt-guided image segmentation models.

2.2 Universal Adversarial Perturbation

Deep neural networks have been shown vulnerable to adversarial examples [10, 25, 44, 45, 46], where attackers can deceive models by introducing subtle noise to images. Universal adversarial perturbation [27] (UAP) was first proposed to fool the victim model by imposing a single adversarial perturbation on a series of images. Existing works can be divided into data-dependent universal adversarial attacks [14, 27, 30] and data-free universal attacks [28, 29, 31], both designed for classification attacks. The former relies on the specific data characteristics of target dataset for UAP generation, while the latter provides a more generalized approach without relying on such data. Meanwhile, some works [15] have also explored UAPs for traditional segmentation models, but they rely on pixel-level labels, which are not applicable to emerging prompt-guided segmentation models. The concurrent works [8, 13] explore UAPs against SAM from the perspectives of direct noise optimization and perturbing the output of the image encoder of SAM, respectively. Different from them, we aim to comprehensively decouple and disrupt crucial object features in images from both spatial and frequency domains, thereby deceiving SAM into failing to segment input images.

3 Methodology

3.1 Problem Formulation

As a fundamental vision model, SAM typically operates in an online mode, allowing users to set prompts randomly. Therefore, we define the threat model as a quasi-black-box setting, where ad-

Single mask

'

Blueprint

'

...

...

Benign examples

prompt 1

prompt 2

...

prompt 1

prompt 2

...

SAM

SAM

Merge

Merge

(a)

(b)

(c)

REINZ

versaries have access to the official open-source SAM, but not to the pre-training dataset and the downstream dataset ( i.e. , those used by users). The adversaries' goal is to craft a UAP δ using a surrogate dataset D s ( i.e. , unrelated to the pre-training and downstream dataset), thereby compromising the model's performance, i.e. , rendering adversarial examples unable to be correctly segmented by SAM. Additionally, the δ should be sufficiently small, and constrained by l p -norm of ϵ . This problem can be formulated as:

̸

3.2 Intuition Behind DarkSAM

Unlike the standard deep learning paradigm that inputs a single image and outputs a one-hot label or pixel-level label, SAM requires both images and prompts as inputs, and then outputs label-free masks, indicating the shape information of critical objects. Therefore, a truly universal adversarial attack against SAM should implement a single perturbation to achieve ineffective segmentation for any combination between a series of images and different prompts. However, this task is hindered by the following challenges:

Challenge I: The dual ambiguity in attack targets arising from varying images and prompts.

Previous UAP works only need to optimize in the target images, hence the introduction of prompts could lead to invalid attacks, as different prompts for a fixed image yield distinct segmentation results. For instance, the image in the top-left corner of Fig. 2 shows a can and a spoon. For the same image, feeding different prompts will result in different masks output by SAM (see Fig. 2(b)). In conclusion, diverse variations in target images and prompts increase the uncertainty of attack targets. For varying images, existing UAP solutions ( e.g. , UAPGD [9]) can provide references, and the main challenge here is the uncertainty of the attack target brought by unknown prompts. To this end, we propose a shadow target strategy by increasing the number of prompts during the attack process to enhance the cross-prompt transferability of UAP. Specifically, for a given input image, we randomly select k prompts ( e.g. , points or boxes) to create a prompt auxiliary set. By merging their masks output by SAM, we form a semantic blueprint of the image, which serves as the target for our attack, as illustrated in Fig. 2(c). This semantic blueprint effectively encompasses the main semantic content of the original image, substantially reducing the ambiguity associated with unknown prompts.

Challenge II: Suboptimal attack efficacy due to semantic decoupling deficiency. Since promptguided segmentation models output masks that are neither one-hot nor pixel-level labels, traditional attack methods that rely on label deviation for optimization guidance become ineffective. Another approach involves directly modifying the output, such as adjusting the adversarial examples' masks to diverge from their originals, potentially yielding marginal attack success as verified in Sec. 4.4. Nonetheless, the intrinsic sensitivity of segmentation models to pixel-level details significantly constrains the potency of these attacks, underscoring a notable limitation in their applicability.

Given the focus of prompt-guided segmentation models on local, critical object features rather than global image features, we are motivated to comprehensively decouple the key semantic features

Frequency domain

Filter

pull

…

…

LHC

HFC

benign example

…

push

adv example

IDWT

…

adv example

benign example

…

benign results

adv results

optimization

random prompts (eg, point)

background region

positive

negative

foreground region

…

Spatial domain

SAM

…

…

negative

positive

optimization

'Segment Everything Mode'

'Shadow Target Strategy'

'Texture Distortion'

'Semantic

Decoupling

'

UAP

…

of an image from the perspective of both spatial and frequency domains, aiming to fool SAM by manipulating these features. We first define the main object within the image ( i.e. , the target of segmentation, typically a region rich in texture) as the foreground , with the rest being defined as the background . As the mask output by SAM indicates the foreground with positive values and the background with negative ones, we use a Boolean mask to separately extract these foreground and background masks. Subsequently, we optimize the UAP, switching adversarial examples' foreground to negative and background to positive, disrupting the image's semantics for a spatial attack. At the same time, inspired by the recent study [38] that SAM is biased towards texture of the image over shape, we investigate the alteration of the high-frequency components ( i.e. , texture information) of adversarial examples in the frequency domain, while simultaneously constraining the low-frequency components ( i.e. , shape information), in order to further enhance the effectiveness and transferability of our attack. By separately decoupling and destroying crucial features in both the spatial and frequency domains, we provide valuable optimization directions for UAP generation, thereby facilitating effective attacks on SAM.

3.3 DarkSAM: A Complete Illustration

In this section, we present DarkSAM, a novel prompt-free hybrid spatial-frequency universal adversarial attack against the prompt-guided image segmentation models ( i.e. , SAM and its variants). The pipeline of DarkSAM is depicted in Fig. 3, encompassing a semantic decoupling-based spatial attack and a texture distortion-based frequency attack. We start by randomly generating k different prompts to form an auxiliary prompt set P a , acquiring the semantic blueprints of the target images as the attack targets. By individually manipulating the semantic content of adversarial examples' foreground and background in the spatial domain, and increasing the distance between the HFC of adversarial and benign examples in the frequency domain, while also constraining the difference in their LFC, we enhance the attack performance and transferability of the UAP. We provide the detailed optimization process of DarkSAM in Algorithm 1. The overall optimization objective J total of DarkSAM is as follow:

where J sa and J fa are the spatial and frequency attack losses, and λ controls the importance.

Semantic decoupling-based spatial attack. Initially, we utilize two Boolean mask m fg and m fg to separately extract the foreground and background mask of the adversarial examples based on the positive and negative values in the mask output by SAM. As for the foreground, our intention is to render it unidentifiable and unsegmentable by SAM. Thus, we optimize its mask towards a negative fake mask ξ neg , enabling its fusion with the background to achieve segmentation evasion. The foreground evasion loss J fe can be described as:

where ξ neg is a fake mask that conforms to the shape of the image, containing threshold values of -τ in regions corresponding to the foreground, and 0 elsewhere. J d serves as the distance metric function, representing the mean squared error loss. For the background, we optimize its mask towards a positive fake mask ξ pos (opposite to ξ neg ), misleading SAM into interpreting it as a

semantically meaningful object, consequently causing further interference in the assessment of the foreground. The associated loss is

The loss of the semantic decoupling-based spatial attack can be expressed as:

Texture distortion-based frequency attack. In the frequency domain, the high-frequency components of an image denote the finer details, including noise and textures, while the low-frequency components contain the general outline and overall structural information of the image. We employ the discrete wavelet transform (DWT), utilizing a low-pass filter L and a high-pass filter H to decompose the image x into different components, constituting a low-frequency component c ll , a high-frequency component c hh , and two mid-frequency components c lh and c hl , via

Subsequently, we employ the inverse discrete wavelet transform (IDWT) to reconstruct the signal that have been decomposed through DWT into an image. We choose the LFC and HFC while dropping the other components to obtain the reconstructed images ϕ ( x ) and ψ ( x ) as

By adding a UAP to the images, we alter their HFC, disrupting the original texture information. Simultaneously, we enforce constraints on the low-frequency disparities between adversarial and benign examples to redirect a larger portion of the perturbation towards the high-frequency domain. As a result, we enhance the attack performance and cross-domain transferability of the UAP by introducing variations in the frequency domain. The loss of texture distortion-based frequency attack can be expressed as:

where µ is a pre-defined hyperparameter.

Algorithm 1 DarkSAM

Input: image x ∈ D s , SAM f ( x ) with parameter θ , hyper parameters k , τ , λ , and µ , maxperturbation constraint ϵ

Output: A universal adversarial perturbation δ

1: Initialize random prompt sets P a and noise δ o

3: Project x ∗ to [0 , 1] via clipping

2: Initialize adversarial examples: x ∗ ←-x + δ o

4:

Separating different frequency components of

c

ll

, c lh

, c hl

, c hh

←-

(

x

)

5: Restore part of the frequency components into an image using the inverse discrete wavelet transform: ϕ ( x ) , ψ ( x ) ←-IDWT ( c ll ) , IDWT ( c hh )

6: Calculate m fg by determining the sign of each value in output of f θ ( x, P a )

7: while max iterations or not converge do

8: Calculate spatial loss mentioned in Eq. 6

9: Calculate frequency loss mentioned in Eq. 10

10: Update δ through backprop

11: Clip δ to satisfy imperceptibility constraint ϵ

12: Project x ∗ to [0 , 1] via clipping

13: end while

x

using the discrete wavelet transform:

4 Experiments

4.1 Experimental Setup

Datasets and models. We evaluate our method using four public segmentation datasets: ADE20K [43], MS-COCO [23], CITYSCAPES [7], and SA-1B [19]. For each dataset, we randomly select 100 images for UAP generation and 2,000 images for testing purposes. All images are uniformly resized to 3 × 1024 × 1024. For victim models, we use the pre-trained SAM [19], HQ-SAM [18] and PerSAM [40] with the ViT-B backbone.

Parameter setting. Following [9, 27, 32], we set the upper bound of UAP to 10 / 255 . For our experiments, we adjust the hyperparameters k , τ , λ , and µ to 10 , 1 , 0 . 1 , and 0 . 01 , respectively, and set the batch size to 1 . To evaluate the cross-prompt attack capabilities of DarkSAM, we employ three distinct prompt types: point, box, and segment everything (also abbreviated as 'all') mode.

Evaluation metrics. To evaluate the effectiveness of DarkSAM, we use the mean Intersection over Union (mIoU) metric. To facilitate data presentation, we also use the attack success rate (ASR) as a metric to evaluate attack performance. ASR represents the difference between the mIoU values of benign and adversarial examples.

4.2 Attack Performance

To comprehensively evaluate DarkSAM's effectiveness, we perform experiments on three promptguided image segmentation models including SAM, HQ-SAM, and PerSAM, across four datasets. For each setup, we generate UAPs using point and box prompts, respectively, and then evaluate DarkSAM's attack performance using the corresponding single-point or single-box prompt. We first calculate the clean mIoU of different models across four datasets using point and box as prompts. Specifically, for the SA-1B dataset, we directly extract point and box prompts from the annotations, whereas for the other datasets, we obtain internal points and external boxes as prompts by calculating the object contour coordinates within their annotations.

The experiments in Tab. 1 show that DarkSAM can effectively fool these prompt-guided image segmentation models with an average mIoU reduction of more than 60% across 96 different experimental settings. The results in Tab. 1 also indicate that box prompts not only yield higher segmentation accuracy but also demonstrate greater robustness. For adversaries, the choice of surrogate datasets has a minor impact on crafting UAPs, yet they consistently facilitate excellent attack performance. Notably, DarkSAM demonstrates a distinct advantage when the SA-1B dataset, the training data for SAM, is employed as the surrogate dataset. In addition to the above quantitative experimental results, we also present qualitative findings. Specifically, we provide the visualization of SAM segmentation results for adversarial examples made by DarkSAM using point and box prompts across four different datasets in Fig. 4. These results include masks of objects in images output by SAM under point, box, and segment everything prompt modes. From Fig. 4, we can see that SAM successfully segments benign images across three types of prompt modes, but it is unable to segment adversarial examples, i.e. , the output masks are close to 'dark'. The qualitative results further corroborate the powerful attack capability of DarkSAM.

!"#$%#&'($#)

*+,-./0

12&3434

356723*',2

2*&8!

*9:&'($#)

!"#$%#&!(;

*9:&!(;

*9:&*<<

!"#$%#&*<<

4.3 Transferability Study

We study the attack transferability of DarkSAM across data domain, prompt types, and models, respectively. 1 Cross-domain. The results in Tab. 1 demonstrate DarkSAM's excellent cross-domain transferability, where UAPs generated with the surrogate dataset (ADE20K) achieve a high ASR on datasets from various different domains. We also explore the role of the frequency attack ( i.e. , J fa , denoted as FA) in enhancing cross-domain transferability. As shown in Fig. 5 (a) , frequency attack can effectively improve the attack performance based on the spatial attack ( i.e. , J sa , denoted as SA). 2 Cross-prompt. We examine the performance of DarkSAM across various types of prompts. As demonstrated in the last three columns of Fig. 4, UAPs created based on both point and box prompts perform well under the segment everything mode. Additionally, we provide results of transferability experiments between point and box prompts in Tab. 2. This includes testing UAPs created with point prompts in the box prompt setting and vice versa. Based on the observed results, it is discernible that UAPs crafted using box prompts generally demonstrate better transferability compared to those using point prompts. This increased efficacy can likely be attributed to the box prompts offering more integral and detailed prompt information. 3 Cross-model. We use UAPs created with points and boxes based on SAM to attack HQ-SAM and PER-SAM. The results in Fig. 5 (b) - (e) showcase DarkSAM's exceptional transferability across different models.

4.4 Comparison Study

To comprehensively demonstrate the superiority of our proposed method, we compare DarkSAM with popular UAP schemes, including UAP [27], UAPGD [9], and SSP [32]. We also consider the state-of-the-art adversarial attack against traditional segmentation models, SegPGD [11], and the

ADE

COCO

CITY

SA-1B

40

45

50

55

60

65

70

75

ASR

SA

SA + FA

(a) SA&FA

ADE

COCO

CITY

SA1B

ADE

COCO

CITY

SA1B

62.31

58.04

44.97

62.27

62.54

62.67

43.06

68.61

49.05

35.10

36.43

44.99

62.92

63.05

44.58

72.69

40

45

50

55

60

65

70

75

80

(b) Point-HQ

ADE

COCO

CITY

SA1B

ADE

COCO

CITY

SA1B

64.30

59.55

47.26

72.31

64.02

61.80

43.07

75.48

55.28

41.35

39.13

55.33

64.27

61.99

45.84

77.82

40

45

50

55

60

65

70

75

80

(c) Point-PER

ADE

COCO

CITY

SA1B

ADE

COCO

CITY

SA1B

71.24

68.68

47.26

68.63

71.79

77.31

53.36

74.85

63.07

49.44

52.86

64.30

60.46

55.42

45.26

58.05

40

45

50

55

60

65

70

75

80

(d) Box-HQ

ADE

COCO

CITY

SA1B

ADE

COCO

CITY

SA1B

70.83

67.77

49.22

74.27

70.97

75.70

52.70

79.25

63.93

52.96

55.97

72.22

57.87

51.36

40.43

59.73

40

45

50

55

60

65

70

75

80

(e) Box-PER

latest sample-wise attack against SAM, Attack-SAM [39]. For a fair comparison, we adapt them to a UAP optimization strategy and keep other settings consistent with DarkSAM. We select SAM as the victim model and assess the effectiveness of these UAP methods across four datasets, using the same dataset for both generating and testing the UAPs. The results in Tab. 3 indicate that DarkSAM outperforms all methods with a considerable margin. The negative experimental values ('*') indicate that the attack does not work at all. This phenomenon may stem from counterproductive perturbations that inadvertently cause the input samples to resemble the training set used by SAM, paradoxically enhancing accuracy and resulting in negative ASR values. We also provide visualizations of the segmentation results of the adversarial examples made by these methods using box prompts in Fig. 6, obtained in point, box, and segment-everything modes, respectively. The results further demonstrate the superiority of DarkSAM.

4.5 Ablation Study

In this section, we explore the effect of different modules, prompt number, attack strengths, training data size, and threshold values on DarkSAM. We conduct experiments using point prompts on SAM across the ADE20K dataset.

The effect of different modules. We investigate the effect of various modules on the attack performance of DarkSAM. For clarity and convenience, we use A, B, C, and D to denote J fe , J bm , J hfc , and J lfc , respectively. The results in Fig. 7 (a) show that no variants can compete with the complete method, implying the indispensability of each component for DarkSAM.

The effect of prompt number. We study the effect of the prompt number in proposed shadow target strategy on attack performance of DarkSAM. We conduct experiments with varying numbers of point prompts, ranging from 1 to 100 . The results in Fig. 7 (b) show a gradual increase in attack performance from 1 to 10 (default setting), followed by a downward trend. This could be attributed to an excess of random points leading to masks with redundant information, thereby impacting the attack efficacy.

The effect of perturbation budget. As shown in Fig. 7 (c), we evaluate DarkSAM's attack performance with ϵ from 4 / 255 to 32 / 255 . With the increase in ϵ , there is a corresponding enhancement in attack performance. Notably, our attack still maintains high efficacy at the 6 / 255 setting, with an average ASR exceeding 45% .

The effect of number of training samples. We explore the effect of varying the number of training images used to create UAP on DarkSAM. Utilizing a range from 10 to 1000 images to craft UAPs, the results in Fig. 7 (d) reveal that employing merely 100 images can achieve excellent attack performance, demonstrating a strong applicability advantage.

Benign

Ours

UAPGD

Attack-SAM

SSP

UAP

Point

Box

All

w/o A

w/o B w/o AB w/o C

w/o D w/o CD

Ours

0

10

20

30

40

50

60

70

ASR

57.13

55.72

18.46

55.66

60.67

61.56

64.96

(a) Module

1

5

8

10

15

20

50

100

0

10

20

30

40

50

60

70

ASR

SAM

HQ-SAM

PerSAM

(b) Prompt

4/255

6/255

8/255

10/255 12/255 16/255 32/255

0

10

20

30

40

50

60

70

ASR

SAM

HQ-SAM

PerSAM

(c) Epsilon

10

50

100

200

300

400

500

1000

0

10

20

30

40

50

60

70

ASR

SAM

HQ-SAM

PerSAM

(d) Number

1

10

20

50

100

200

500

1000

52

54

56

58

60

62

64

66

68

70

ASR

SAM

HQ-SAM

PerSAM

(e) Threshold

The effect of threshold values. Weexamine the effect of varying threshold values τ in the fake mask ξ on DarkSAM. As illustrated in Fig. 7 (e), we test a range of values from 1 to 1000 . The results indicate that these different values have a minimal overall effect on DarkSAM's performance.

5 Conclusions, Limitations, and Broader Impact

In this paper, we propose DarkSAM, the first truly universal adversarial attack against SAM. With a single perturbation, DarkSAM renders SAM incapable of segmenting objects across diverse images with varying prompts, thereby exposing its vulnerability. To tackle the challenge of dual ambiguity in attack targets, we present a shadow target strategy to obtain semantic blueprint as a attack target. We then design a novel prompt-free hybrid spatial-frequency universal attack framework, which consists of a semantic decoupling-based spatial attack and a texture distortion-based frequency attack. By disrupting the crucial object features in both the spatial and frequency domains of the images, it successfully addresses the challenge of suboptimal attack efficacy, thus deceiving SAM. Our extensive experiments on SAM, HQ-SAM, and PerSAM across four datasets, both qualitatively and quantitatively, demonstrate DarkSAM's powerful attack ability and strong attack transferability.

In terms of limitations, DarkSAM may not be suitable for traditional segmentation models because its output is not a label-free mask. This characteristic might limit its applicability in scenarios where labeled masks are essential for accurate segmentation. The adversarial examples produced by DarkSAM could potentially mislead SAM-based segmentation platforms, posing significant security risks, particularly in sensitive domains like medical image analysis.

References

Anurag Arnab, Ondrej Miksik, and Philip HS Torr. On the robustness of semantic segmentation models to adversarial attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'18) , pages 888-897, 2018.

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Proceedings of the 44th International Conference on Neural Information Processing Systems (NeurIPS'20) , pages 1877-1901, 2020.

Cheng Chen, Juzheng Miao, Dufan Wu, Aoxiao Zhong, Zhiling Yan, Sekeun Kim, Jiang Hu, Zhengliang Liu, Lichao Sun, Xiang Li, Tianming Liu, Pheng-Ann Heng, and Quanzheng Li. Ma-sam: Modality-agnostic sam adaptation for 3d medical image segmentation. arXiv preprint arXiv:2309.08842 , 2023.

Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587 , 2017.

Tianrun Chen, Lanyun Zhu, Chaotao Deng, Runlong Cao, Yan Wang, Shangzhan Zhang, Zejian Li, Lingyun Sun, Ying Zang, and Papa Mao. Sam-adapter: Adapting segment anything in underperformed scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'23) , pages 3367-3375, 2023.

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Y. Zhao, Yanping Huang, Andrew M. Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416 , 2022.

Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'16) , pages 3213-3223, 2016.

Francesco Croce and Matthias Hein. Segment (almost) nothing: Prompt-agnostic adversarial attacks on segmentation models. arXiv preprint arXiv:2311.14450 , 2023.

Yingpeng Deng and Lina J Karam. Universal adversarial attack via enhanced projected gradient descent. In Proceedings of the IEEE International Conference on Image Processing (ICIP'20) , pages 1241-1245, 2020.

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 , 2014.

Jindong Gu, Hengshuang Zhao, Volker Tresp, and Philip HS Torr. Segpgd: An effective and efficient adversarial attack for evaluating and boosting segmentation robustness. In Proceedings of the 17th European Conference on Computer Vision (ECCV'22) , pages 308-325, 2022.

Haoyu Guo, He Zhu, Sida Peng, Yuang Wang, Yujun Shen, Ruizhen Hu, and Xiaowei Zhou. Sam-guided graph cut for 3d instance segmentation. arXiv preprint arXiv:2312.08372 , 2023.

Dongshen Han, Sheng Zheng, and Chaoning Zhang. Segment anything meets universal adversarial perturbation. arXiv preprint arXiv:2310.12431 , 2023.

Jamie Hayes and George Danezis. Learning universal adversarial perturbations with generative models. In Proceedings of the IEEE Symposium on Security and Privacy Workshops (SPW'18) , pages 43-49, 2018.

Jan Hendrik Metzen, Mummadi Chaithanya Kumar, Thomas Brox, and Volker Fischer. Universal adversarial perturbations against semantic image segmentation. In Proceedings of the IEEE International Conference on Computer Vision (ICCV'17) , pages 2755-2764, 2017.

Shengshan Hu, Yechao Zhang, Xiaogeng Liu, Leo Yu Zhang, Minghui Li, and Hai Jin. Advhash: Set-to-set targeted attack on deep hashing with one single adversarial patch. In Proceedings of the 29th ACM International Conference on Multimedia (MM'21) , pages 2335-2343, 2021.

Yihao Huang, Yue Cao, Tianlin Li, Felix Juefei-Xu, Di Lin, Ivor W Tsang, Yang Liu, and Qing Guo. On the robustness of segment anything. arXiv preprint arXiv:2305.16220 , 2023.

Lei Ke, Mingqiao Ye, Martin Danelljan, Yifan Liu, Yu-Wing Tai, Chi-Keung Tang, and Fisher Yu. Segment anything in high quality. In Proceedings of the 47th International Conference on Neural Information Processing Systems (NeurIPS'23) , 2023.

Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chlo´ e Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Doll´ ar, and Ross B. Girshick. Segment anything. arXiv preprint arXiv:2304.02643 , 2023.

Feng Li, Hao Zhang, Peize Sun, Xueyan Zou, Shilong Liu, Jianwei Yang, Chunyuan Li, Lei Zhang, and Jianfeng Gao. Semantic-sam: Segment and recognize anything at any granularity. arXiv preprint arXiv:2307.04767 , 2023.

Minghui Li, Jiangxiong Wang, Hao Zhang, Ziqi Zhou, Shengshan Hu, and Xiaobing Pei. Transferable adversarial facial images for privacy protection. In Proceedings of the 32nd ACM International Conference on Multimedia (MM'24) , 2024.

Yuheng Li, Mingzhe Hu, and Xiaofeng Yang. Polyp-sam: Transfer sam for polyp segmentation. arXiv preprint arXiv:2305.00293 , 2023.

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Proceedings of the 12nd European Conference on Computer Vision (ECCV'14) , pages 740-755, 2014.

Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'15) , pages 3431-3440, 2015.

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 , 2017.

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'16) , pages 2574-2582, 2016.

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'17) , pages 1765-1773, 2017.

Konda Reddy Mopuri, Utsav Garg, and R Venkatesh Babu. Fast feature fool: A data independent approach to universal adversarial perturbations. arXiv preprint arXiv:1707.05572 , 2017.

Konda Reddy Mopuri, Aditya Ganeshan, and R. Venkatesh Babu. Generalizable data-free objective for crafting universal adversarial perturbations. IEEE Transactions on Pattern Analysis and Machine Intelligence , 41(10):2452-2465, 2018.

Konda Reddy Mopuri, Utkarsh Ojha, Utsav Garg, and R. Venkatesh Babu. Nag: Network for adversary generation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'18) , pages 742-751, 2018.

Konda Reddy Mopuri, Phani Krishna Uppala, and R. Venkatesh Babu. Ask, acquire, and attack: Data-free uap generation using class impressions. In Proceedings of the 15th European Conference on Computer Vision (ECCV'18) , pages 19-34, 2018.

Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli. A self-supervised approach for adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'20) , pages 262-271, 2020.

Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Yanqi Zhou, Chung-Ching Chang, Igor Krivokon, Will Rusch, Marc Pickett, Kathleen S. Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny Soraker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Alejandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Cohen, Rachel Bernstein, Ray Kurzweil, Blaise Ag¨ uera y Arcas, Claire Cui, Marian Croak, Ed H. Chi, and Quoc Le. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239 , 2022.

Junde Wu, Rao Fu, Huihui Fang, Yuanpei Liu, Zhaowei Wang, Yanwu Xu, Yueming Jin, and Tal Arbel. Medical sam adapter: Adapting segment anything model for medical image segmentation. arXiv preprint arXiv:2304.12620 , 2023.

Song Xia, Wenhan Yang, Yi Yu, Xun Lin, Henghui Ding, Lingyu Duan, and Xudong Jiang. Transferable adversarial attacks on sam and its downstream models. In Proceedings of the 38th International Conference on Neural Information Processing Systems (NeurIPS'24) , 2024.

Jinyu Yang, Mingqi Gao, Zhe Li, Shang Gao, Fangjing Wang, and Feng Zheng. Track anything: Segment anything meets videos. arXiv preprint arXiv:2304.11968 , 2023.

Chaoning Zhang, Dongshen Han, Yu Qiao, Jung Uk Kim, Sung-Ho Bae, Seungkyu Lee, and Choong Seon Hong. Faster segment anything: Towards lightweight sam for mobile applications. arXiv preprint arXiv:2306.14289 , 2023.

Chaoning Zhang, Yu Qiao, Shehbaz Tariq, Sheng Zheng, Chenshuang Zhang, Chenghao Li, Hyundong Shin, and Choong Seon Hong. Understanding segment anything model: Sam is biased towards texture rather than shape. arXiv preprint arXiv:2311.11465 , 2023.

Chenshuang Zhang, Chaoning Zhang, Taegoo Kang, Donghun Kim, Sung-Ho Bae, and In So Kweon. Attack-sam: Towards evaluating adversarial robustness of segment anything model. arXiv preprint arXiv:2305.00866 , 2023.

Renrui Zhang, Zhengkai Jiang, Ziyu Guo, Shilin Yan, Junting Pan, Hao Dong, Peng Gao, and Hongsheng Li. Personalize segment anything model with one shot. arXiv preprint arXiv:2305.03048 , 2023.

Yechao Zhang, Shengshan Hu, Leo Yu Zhang, Junyu Shi, Minghui Li, Xiaogeng Liu, and Hai Jin. Why does little robustness help? a further step towards understanding adversarial transferability. In Proceedings of the 45th IEEE Symposium on Security and Privacy (S&P'24) , 2024.

Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'17) , pages 2881-2890, 2017.

Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'17) , pages 633-641, 2017.

Ziqi Zhou, Shengshan Hu, Minghui Li, Hangtao Zhang, Yechao Zhang, and Hai Jin. Advclip: Downstream-agnostic adversarial examples in multimodal contrastive learning. In Proceedings of the 31st ACM International Conference on Multimedia (MM'23) , pages 6311-6320, 2023.

Ziqi Zhou, Shengshan Hu, Ruizhi Zhao, Qian Wang, Leo Yu Zhang, Junhui Hou, and Hai Jin. Downstream-agnostic adversarial examples. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV'23) , pages 4345-4355, 2023.

Ziqi Zhou, Minghui Li, Wei Liu, Shengshan Hu, Yechao Zhang, Wei Wan, Lulu Xue, Leo Yu Zhang, Dezhong Yao, and Hai Jin. Securely fine-tuning pre-trained encoders against adversarial examples. In Proceedings of the 2024 IEEE Symposium on Security and Privacy (SP'24) , 2024.

Appendix

A Datasets

ADE20K: ADE20K [43] is a dataset for scene parsing that includes images from a variety of environments. It contains more than 20,000 images, classified into 150 categories, covering both natural landscapes and indoor settings. Each image in ADE20K is pixel-wise annotated, making it suitable for scene parsing and semantic segmentation tasks.

MS-COCO: MS-COCO [23] is a large-scale dataset for image recognition, segmentation, and image captioning. It contains more than 200,000 labeled images, 150,000 validation images, and over 80,000 test images. The dataset includes 80 different object categories and over 250,000 object instances. MS-COCO is known for its detailed annotations for each image, including object segmentation, object detection, and image captioning.

CITYSCAPES: CITYSCAPES [7] is a dataset for urban street scenes, primarily used for training and testing vision systems for autonomous driving. It includes street scenes from 50 different cities, with approximately 5,000 finely annotated images. These images include various urban scenarios and a range of traffic participants.

SA-1B: SA-1B [19] contains 11 million diverse, high-resolution, privacy-protected images and 1.1 billion high-quality segmentation masks. These masks were automatically generated by SAM. The dataset aims to facilitate computer vision research and is characterized by an average of 100 masks per image.

B Platform

Experiments are conducted on a server running a 64-bit Ubuntu 20.04.1 system with an Intel(R) Xeon(R) Silver 4210R CPU @ 2.40GHz processor, 125GB memory, and two Nvidia GeForce RTX 3090 GPUs, each with 24GB memory. The experiments are performed using the Python language and PyTorch library version 2.1.0.

Benign-Point

Adv-Point

★

C Supplementary Attack Performance

C.1 Evaluation on MobileSAM

We evaluate the attack performance of DarkSAM against another SAM's variant model, MobileSAM [37], on four datasets. All experimental settings are kept consistent with Sec.4.2. The results in Tab. A1 demonstrate the effectiveness of DarkSAM against MobileSAM, further proving its strong attack capability. Notably, in line with the conclusions in Sec.4.2, the choice of surrogate datasets has a certain impact on the attack performance. SA-1B serves as a notably superior surrogate dataset, while CITYSCAPES exhibits comparatively lower performance in certain scenarios. This discrepancy may be attributed to CITYSCAPES' limited scope, which solely encompasses the urban street scene, consequently restricting the transferability of the generated UAPs.

C.2 Evaluation on SAM with ViT-L backbone

We present both quantitative and qualitative results of DarkSAM on SAM with a ViT-L backbone, denoted as SAM-L. The quantitative findings in Table A2 illustrate the effectiveness of DarkSAM in deceiving SAM-L. Notably, these results indicate that SAM-L exhibits greater robustness compared to SAM-B (SAM with a ViT-B backbone) due to its more intricate network architecture. Additionally, we offer visualization results of DarkSAM's attacks on SAM-L under point, box, and segment everything modes. Figs. A1 and A2 demonstrate that adversarial examples generated based on point prompts effectively mislead SAM-L. Similarly, adversarial examples crafted using box prompts also prove to be successful in deceiving SAM-L, as depicted in Figs. A3 and A4.

Benign-All

Adv-All

Benign-Box

Adv-Box

Benign-All

Adv-All

ADE

COCO

CITY

SA-1B

ADE

COCO

CITY

SA-1B

63.93

60.24

44.30

74.46

63.77

61.45

45.71

75.62

62.98

58.70

50.32

75.74

63.40

60.74

46.69

76.98

40

50

60

70

80

90

(a) Point-HQ2PER

ADE

COCO

CITY

SA-1B

ADE

COCO

CITY

SA-1B

64.65

60.03

50.09

73.64

64.89

61.22

50.33

74.74

64.02

58.63

50.47

74.70

64.39

60.47

50.26

76.32

40

50

60

70

80

90

(b) Point-HQ2SAM

ADE

COCO

CITY

SA-1B

ADE

COCO

CITY

SA-1B

70.57

71.16

49.08

78.06

70.74

69.07

50.78

77.06

66.86

61.08

61.15

63.79

69.03

69.96

48.56

79.49

40

50

60

70

80

90

(c) Box-HQ2PER

ADE

COCO

CITY

SA-1B

ADE

COCO

CITY

SA-1B

70.98

71.14

63.03

78.25

71.25

69.10

63.05

78.63

64.38

61.08

63.43

64.17

69.14

69.68

62.86

80.67

40

50

60

70

80

90

(d) Box-HQ2SAM

ADE

COCO

CITY

SA-1B

ADE

COCO

CITY

SA-1B

62.32

59.34

41.71

64.50

62.84

63.33

42.12

71.49

52.57

35.02

47.80

36.03

51.71

36.48

32.62

68.81

40

50

60

70

80

90

(e) Point-PER2HQ

ADE

COCO

CITY

SA-1B

ADE

COCO

CITY

SA-1B

64.68

59.63

50.47

70.71

65.30

62.48

50.59

76.90

55.64

40.59

50.17

41.48

58.30

45.01

50.59

76.04

40

50

60

70

80

90

(f) Point-PER2SAM

ADE

COCO

CITY

SA-1B

ADE

COCO

CITY

SA-1B

72.05

75.92

53.41

74.36

72.10

77.20

48.54

79.56

70.80

67.71

61.14

62.08

66.85

64.44

49.19

76.89

40

50

60

70

80

90

(g) Box-PER2HQ

ADE

COCO

CITY

SA-1B

ADE

COCO

CITY

SA-1B

73.18

75.56

63.14

81.16

72.88

76.00

63.29

84.85

70.21

67.80

64.48

67.82

62.84

62.88

61.92

82.43

40

50

60

70

80

90

(h) Box-PER2SAM

Benign-Point

Adv-Point

☆

Benign-Point

Adv-Point

10

20

30

50

100

1000 2000 3000

55

60

65

70

75

80

85

ASR

P2P

B2B

D Supplementary Transferability Study

In this section, we delve deeper into the cross-model transferability of DarkSAM, considering both different model types and diverse model backbones. We maintain uniformity with the experimental settings outlined in Sec. 4.3.

1) Cross model type transferability. We explore the transferability of UAPs crafted by DarkSAM on PerSAM and HQ-SAM when attacking other types of models. These models all share the ViT-B backbone. The 'Point-' and 'Box-'prefixes indicate that the UAPs are crafted and tested under point and box prompts, respectively. The suffixes 'HQ2PER' and 'HQ2SAM' denote the UAPs crafted on HQ-SAM against PerSAM and SAM, respectively. Similar notations carry the same implications. The results in Fig. A5 further demonstrate the robust cross-model type transferability of DarkSAM.

2) Cross model backbone transferability. We investigate the cross-model backbone transferability of DarkSAM. Specifically, we craft UAPs based on the ADE20K dataset on SAM-L and SAM-B, respectively, and test the transferability of these attacks between the two models. From Figs. A6 and A7, we can see that adversarial examples crafted on SAM-B effectively mislead SAM-L, and

Benign

Adv

Benign

Adv

Benign

Adv

Benign

Adv

★★

conversely, those crafted on SAM-L deceive SAM-B. These results demonstrate the cross-model backbone transferability of DarkSAM.

E Supplementary Ablation Study

The effect of random seeds. Considering the relationship between random seeds and the selection of images in training and testing, we investigate the effect of random seeds on DarkSAM. All our experiments default to a random seed setting of 100 . As illustrated in Fig. A8, we select eight different random seeds and conduct experiments to attack SAM on the ADE20K dataset with these seeds. 'P2P' and 'B2B' respectively denote the creation and testing of UAPs using point and box prompts. The results in Fig. A8 indicate that DarkSAM consistently exhibits stable and superior attack performance across various random seed settings.

The mixed use of point and box prompts in the shadow target strategy . We delve into the effects of employing a hybrid approach of points and boxes as prompts in the shadow target strategy for DarkSAM. In this method, we craft UAPs using a balanced mix of five random points and five boxes. The outcomes, as detailed in Tab. A3, reveal that UAPs constructed with this mixed approach maintain robust attack performance. This finding accentuates the adaptability and effectiveness of our proposed shadow target strategy.

Multipoint evaluation. We explore the effects of using multiple point prompts during the inference phase of SAM on the efficacy of DarkSAM's attacks. Compared to single-point prompts, multipoint prompts offer augmented object positional data, potentially enhancing segmentation accuracy. To evaluate this, we generate UAPs on the SA-1B dataset using point prompts and subsequently test these under multipoint prompts mode. The results, as illustrated in Fig. A9, indicate that for benign examples, the use of multipoint prompts indeed results in a more accurate segmentation of the target object relative to single-point prompts. Conversely, in the case of adversarial examples, the addition of multiple prompts does not aid SAM in achieving successful segmentation, thereby underscoring the powerful effectiveness of DarkSAM in compromising segmentation accuracy.

1

2

3

4

5

Severity

0

10

20

30

40

50

60

mIou

C-Benign

C-Adv

(a) Contrast

1

2

3

4

5

Severity

0

10

20

30

40

50

60

mIou

B-Benign

B-Adv

(b) Brightness

F Defense

SAM is renowned for its powerful zero-shot capabilities, thus we believe an appropriate defensive measure is to refrain from making additional structural and parametric modifications to the pre-trained SAM to avoid compromising its original knowledge. Therefore, we consider employing input preprocessing methods to counter adversarial examples. We select two famous image corruption methods from the Imagecorruptions repository, contrast (C) and brightness (B), to test adversarial examples. Results in Fig. A10 demonstrate that DarkSAM effectively withstands such preprocessing-based defenses.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: See Abstract and Sec. 1.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: See Sec. 5.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [N/A]

Justification: The paper does not involve theory assumptions and proofs.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: See Sec. 4.1.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We have provided the source code in the supplementary material.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: See Sec. 4.1.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We have provided the influence of random seed on the experimental results in Fig. A8.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?

Answer: [Yes]

Justification: After a thorough review of the paper, no violations of the NeurIPS Code of Ethics are found.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: See Sec. 5.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [N/A]

Justification: The paper does not involve safeguards that have been put in place for responsible release of data or models.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The data used in the paper are properly credited, and the license and terms of use are explicitly mentioned and properly respected.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [N/A]

Justification: The paper does not release new assets.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [N/A]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [N/A]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces DarkSAM, a prompt-free universal attack framework against the Segment Anything Model (SAM) in a quasi-black-box setting. The framework consists of a semantic decoupling-based spatial attack and a texture distortion-based frequency attack. While SAM uses geometric prompt inputs to guide segmentation of critical objects within images, DarkSAM disrupts these processes by decoupling the object features of images in both spatial and frequency domains using a universal adversarial perturbation (UAP). In the spatial domain, it scrambles SAM’s decisions by destroying the features of the foreground and background of the image separately. In the frequency domain, it decomposes images into high-frequency components (HFC) and low-frequency components (LFC), increasing the dissimilarity in the HFC of adversarial and benign examples while maintaining consistency in their LFC. Experiments are conducted on four public segmentation datasets (ADE20K, MS-COCO, CITYSCAPES, and SA-1B), with 100 images used for UAP generation and 2,000 images for testing for each dataset. Victim models include the pre-trained SAM, HQ-SAM, and PerSAM with the ViT-B backbone.\n\nSTRENGTHS:\n1. The paper is well-written and easy to follow.\n2. The specific design of the shadow target strategy is tailored to SAM for prompt-based segmentation, which is unique compared to adversary attacks against traditional segmentation pipelines.\n3. The attacking results are impressive. All three models show very low segmentation performance across multiple datasets.\n\nWEAKNESSES:\nSAM is a milestone work, and a series of follow-up studies have been proposed recently. However, this paper does not provide an up-to-date review in Section 2.1 and lacks comparison in Section 4, weakening its significance. For example:\n- SAM-based adversary attack:\n[1] Practical Region-level Attack against Segment Anything Models, CVPR 2024.\n- Other SAM-based models:  \n[2] From SAM to CAMs: Exploring Segment Anything Model for Weakly Supervised Semantic Segmentation, CVPR 2024.  \n[3] RobustSAM: Segment Anything Robustly on Degraded Images, CVPR 2024.  \n[4] Matching Anything by Segmenting Anything, CVPR 2024.  \n[5] FocSAM: Delving Deeply into Focused Objects in Segmenting Anything, CVPR 2024.  \n[6] ASAM: Boosting Segment Anything Model with Adversarial Tuning, CVPR 2024.  \n[7] BA-SAM: Scalable Bias-Mode Attention Mask for Segment Anything Model, CVPR 2024.  \n[8] Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactively, ECCV 2024.  \n[9] CAT-SAM: Conditional Tuning Network for Few-Shot Adaptation of Segmentation Anything Model, ECCV 2024.  \n[10] Semantic-SAM: Segment and Recognize Anything at Any Granularity, ECCV 2024.\n\nQUESTIONS:\n1. How does DarkSAM perform with variants of the SAM models?\n2. Please include a comparison with [1] \"Practical Region-level Attack against Segment Anything Models\" (CVPR 2024).\n\nLIMITATIONS:\nThe paper presents a limitation that DarkSAM is tailored to SAM and cannot operate on traditional segmentation models.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work investigates adversarial attacks against Segment Anything Models (SAMs) and presents DarkSAM, the first universal adversarial attack designed for these models. DarkSAM leverages a single perturbation to effectively undermine SAM’s object segmentation capabilities across a variety of images and prompts. The authors conduct a comprehensive evaluation of DarkSAM across four datasets and three SAM variants (SAM, HQ-SAM, and PerSAM), covering attack performance, transferability, comparative analysis, and ablation studies.\n\nSTRENGTHS:\n1.\tThe paper introduces a unique perspective on adversarial attacks for prompt-guided segmentation models, which is a relatively unexplored area in the literature. The proposed DarkSAM method is innovative in its approach to decoupling object features for attack optimization.\n2.\tThe research question is well-defined, and the authors thoroughly compare DarkSAM with a multitude of established baselines.\n3.\tThe paper presents both qualitative and quantitative results, effectively demonstrating the impact of DarkSAM. These results provide a thorough assessment of its performance across various conditions\n\nWEAKNESSES:\n1.\tIt is recommended that the authors further supplement the experimental section with relevant analyses, such as explaining why the spatial domain attack is more critical than the frequency domain attack within the proposed framework.\n2.\tIn this paper, the usage of mIoU and ASR appears to be analogous, with both metrics conveying the same information. Could the authors provide insight into the justification for employing both metrics concurrently?\n3.\tIn Figure 7, the visualization of segmentation masks is notably dark, impeding discernibility for the reader. The authors should consider increasing the brightness of these images or employing more vivid colors.\n\nQUESTIONS:\n1.\tI agree with the authors' prompt-free approach to the attack on SAM. I am curious about the types of prompts that may be more advantageous in crafting effective UAPs during the attack generation process. Additionally, could the authors provide an explanation regarding the selection of prompts?\n\nLIMITATIONS:\n1.\tThe paper relies on heuristic research and lacks a corresponding theoretical framework for analysis.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-09"}

comments: SUMMARY:
This paper introduces DarkSAM, a universal adversarial attack against the Segment Anything Model and its variants. DarkSAM aims to prevent these models from successfully segmenting objects within images. The experimental results demonstrate the effectiveness and transferability of the proposed method.


I have read the response of the authors and comments of other reviewers, I decide to keep my weak accept score.

STRENGTHS:
1.This paper introduces a new universal adversarial attack framework for prompt-guided image segmentation models.
2.The combination of spatial and frequency domain attacks is a sophisticated approach that demonstrates a good understanding of SAM.
3.This paper is well-written. Following the introduction, I can easily understand the goal of this paper.

WEAKNESSES:
1.The related work can be improved. This paper could benefit from an expanded discussion on adversarial attacks targeted at traditional segmentation models.
2.Lack of specific explanation. This method is novel and interesting but I’m curious about the reason why it works. What is the exact process for determining random prompts, and does this method ensure coverage of all potential attack targets?
3.The experimental results lack error bars. Repeating the experiments a few times and reporting the results with error bars would make the findings more convincing.

QUESTIONS:
See Weaknesses

LIMITATIONS:
The authors adequately addressed the limitations and potential negative societal impact of their work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-10