GAMA: Generative Adversarial Multi-Object Scene Attacks
Anonymous Author(s)
AffiliationAddress email
Abstract
The majority of methods for crafting adversarial attacks have focused on scenes witha single dominant object (e.g., images from ImageNet).
On the other hand, naturalscenes include multiple dominant objects that are semantically related.
Thus, it iscrucial to explore designing attack strategies that look beyond learning on single-objectscenes or attack single-object victim classifiers.
Due to their inherent property ofstrong transferability of perturbations to unknown models, this paper presents the firstapproach of using generative models for adversarial attacks on multi-object scenes.
In order to represent the relationships between different objects in the input scene, weleverage upon the open-sourced pre-trained vision-language model CLIP (Contrastive
Language-Image Pre
-training), with the motivation to exploit the encoded semantics inthe language space along with the visual space.
We call this attack approach Generative
Adversarial Multi-object scene Attacks ( GAMA ).
GAMA demonstrates the utilityof the CLIP model as an attacker’s tool to train formidable perturbation generatorsfor multi-object scenes.
Using the joint image-text features to train the generator, weshow that GAMA can craft potent transferable perturbations in order to fool victimclassifiers in various attack settings.
For example, GAMA triggers ∼ 16% moremisclassification than state-of-the-art generative approaches in black-box settings whereboth the classifier architecture and data distribution of the attacker are different fromthe victim.
Our code will be made publicly available.
Introduction
Despite attaining significant results, decision-making of deep neural network models is brittle and can besurprisingly manipulated with adversarial attacks that add highly imperceptible perturbations to the systeminputs [1, 2].
This has led to dedicated research in designing diverse types of adversarial attacks that leadto highly incorrect decisions on diverse state-of-the-art classifiers [2–13].
The majority of such adversarialattacks [2, 8–18] has focused on scenes with a single dominant object ( e.g ., images from ImageNet [19]).
However, natural scenes consist of multiple dominant objects that are semantically associated [20–25].
This calls for attack methods that are effective in such multi-object scenes.
A recent body of work in adversarial attacks [26–30] has shown the importance of exploring attackmethodologies for real-world scenes (although designed for attacking object detectors).
However, suchmethods are image-specific approaches that are known to have poor time complexity when perturbing largebatches of images, as well as poor transferability to unknown models (more details in Section 2) due to theirinherent property of perturbing images independently from one another.
Different from such approaches,our interest lies in the generative model-based approaches [10–13] which are distribution-driven and craftperturbations by learning to fool a surrogate classifier for a large number of images.
These generativeadversarial attacks show stronger transferability of perturbations to unknown victim models and can perturblarge batches of images in one forward pass through the generator demonstrating better time complexity[11, 31].
However, these generative attack methods have focused on learning from single-object scenes
( e.g ., ImageNet in [11–13], CUB-200-2011 [32] in [13]) or against single-object surrogate classifiers( e.g ., ImageNet classifiers [33] in [10–13]).
When trained against multi-object (also known as multi-label)classifiers to learn perturbations on multi-object scenes, such methods perform poorly as they do notexplicitly incorporate object semantics in the generator training (see Table 2 and Table 3).
As real-worldscenes usually consist of multi-object images, designing such attacks is of importance to victim modelusers that analyze complex scenes for making reliable decisions e.g . self-driving cars [34].
To this end,we propose the first generative attack approach, called G enerative A dversarial M ulti-object scene A ttacksor GAMA , that focuses on adversarial attacks on multi-object scenes.
Progress in recent vision-and-language (VL) models [35–39]that allow joint modelling of image and text have garneredinterest in recent times due to their versatile applicability invarious image downstream tasks like inpainting, editing, etc .[40–51].
For the first time in literature, we introduce theutility of a pre-trained open-source framework of the popular
VL model named CLIP (Contrastive Language-Image Pretraining) [36] in generating adversarial attacks.
Trained on400 million image-text pairs collected from the internet, CLIPhas been shown to provide robust joint representations of VLsemantics [40, 46] and strong zero-shot image classificationon diverse datasets [36, 44].
This allows us to access diverse
VL features cheaply without any training as end-user.
Ourproposed GAMA attack employs the CLIP model to exploit the natural language semantics encoded intext features along with the vision features (due to its joint image-text alignment property).
Different fromprior works, GAMA utilizes CLIP model’s extracted knowledge from ∼ 400 million images to maximizethe feature differences of perturbed image x p against two different types of features computed from cleanimage x c : ( 1 ) features of x c computed from surrogate models, and ( 2 ) features of x c computed from
CLIP’s image encoder.
Additionally, GAMA also guides x p to contain different features compared to x cby using features from CLIP’s text encoder via a contrastive loss function.
For example, in Figure 1 for 10random text prompts (examples extracted from CLIP’s ‘ViT-B/16’ model using Pascal-VOC’s classes),we use CLIP’s image-text aligning property to estimate that a clean image x c with closest text prompt“sofa and bottle” (text features ρ c ) is least similar to the text prompt “car and bicycle” 1 (text features ρ p )among 5 randomly chosen candidates (indicated by dotted circles).
GAMA uses ρ p to contrast and movethe perturbed x p away from x c in feature space.
Hence, the perturbed image features are comparablyrobust to data distribution changes in victim models as G θ ( · ) is optimized to create perturbations that differin features from two different image features.
This allows GAMA to launch highly transferable attacks onunseen victim models (see Section 4).
To summarize, we make the following contributions in this paper:
Multi-object scene based generative attack.
We propose the first multi-object scene based generativeattack, GAMA , that is designed to consider object semantics through vision-and-language models.2.
Pre-trained CLIP model as an attacker tool.
We propose the first generative attack on classifiers thatutilizes the open-source pre-trained CLIP model as an attacker’s tool to train perturbation generators.3.
Extensive Attack Evaluations.
Our extensive experiments on various black-box settings (where victimsare multi-label/single-label classifiers and object detectors) show GAMA ’s state-of-the-art transferability
of perturbations
(Table 2, 3, 5, 4, 6, and 7).
Additionally, we also show that GAMA outperforms its
baselines in terms of attack robustness when the victim deploys state-of-the-art defenses (Figure 9).
Related works
Adversarial attacks on classifiers.
Several state-of-the-art adversarial attacks [2, 6, 8–17, 52–60] havebeen designed to disturb the predictions of classifiers.
Broadly these approaches can be categorized intotwo strategies: instance or image-specific attacks and generative model-based attacks.
Instance (or imagespecific) attacks [2, 6, 8, 9, 14–17, 52–60] create perturbations for every image exclusively.
Specifically,these perturbations are computed by querying the victim model for multiple iterations in order to eventuallyalter the image imperceptibly (e.g. texture level changes to image [60]) to cause its misclassification.
Due
[Table]
been shown to be extremely poor [11, 13, 31].
Furthermore, learning perturbations based on single-imagegenerally restrict their success of misclassification only on the known models [11, 13]
[Table]
To alleviate these drawbacks
[Table]
a new category of attack strategies has been explored in [10– [Table] 61] wherea generative model is adversarially trained against a surrogate victim model (in other words, treated as adiscriminator ) to craft perturbations on whole data distribution.
This attack strategy particularly allows oneto perturb multiple images simultaneously once the generative model is optimized [Table] as well as enhances thetransferability of perturbations to unseen black-box models [10, 11].
For example, Generative Adversarial
Perturbations or GAP [10] and Cross-Domain Attack or CDA [11] presented a distribution-driven attackthat trains a generative model for creating adversarial examples by utilizing the cross-entropy loss andrelativistic cross-entropy loss [62] objective, respectively.
Different from these, Transferable Adversarial
Perturbations or TAP [12] and Beyond ImageNet Attack or BIA [13] presented an attack methodology tofurther enhance transferability of perturbations using feature separation loss functions ( e.g . mean squareerror loss) at mid-level layers of the surrogate model.
Most of these methods focused on creating transferableperturbations assuming that the surrogate model is trained in the same domain as the target victim model[13].
Further, a mid-level layer is manually selected for each architecture and is also sensitive to thedataset (shown later in Section 4).
Similarly, [61] proposes to change image attributes to create semanticmanipulations using their disentangled representations via generative models.
Most of these generativeattacks employed classifiers that operate under the regime that input images include single dominantobjects.
Some recent attacks [26–30] have focused on analyzing complex images which contain multipleobjects, however, they are instance-driven attacks that introduce aforesaid drawbacks of transferabilityand time complexity.
In contrast to these aforementioned works, GAMA is a generative model-basedattack designed to craft imperceptible adversarial perturbations that can strongly disrupt both multi-labeland single-label classifiers.
Moreover, GAMA uses a novel perturbation generation strategy that employsa pre-trained CLIP model [36] based framework to craft highly effectual and transferable perturbationsby leveraging multi-modal (image and text) embeddings.
We summarize these differences between priorgenerative attacks and GAMA in Table 1.
Applications of Vision-and-Language (VL) representations.
Due to their robust zero-shot performance, joint vision-and-language pre-trained models [35–39] haveallowed new language-driven solutions various downstream tasks [40–51, 63].
The differentiating attributeof using VL models [36], when compared to existing conventional image-based pre-trained models [33], isthat they provide high-quality aligned visual and textual representations learnt from large-scale image-textpairs.
In this work, we leverage one such powerful VL framework named CLIP [36] to an adversary’sadvantage and show its utility in preparing a perturbation generator for formidable attacks across multipledistributions.
Employing freely available pre-trained models for tasks other than they were trained for hasbeen common practice ( e.g . VGG [64] models in [65, 66], CLIP for domain adaptation of generators in[46]).
To the best of our knowledge, the proposed attack is the first to introduce such VL model usage tosubvert classifier decisions.
Proposed Attack Methodology: GAMA
Problem Statement.
Our goal is to train a generative model G θ ( · ) (weights θ ) from a training distributionof images with multiple-objects.
Once θ is optimized, G θ ( · ) can create perturbations on diverse types(multi-object or otherwise) of input images that can lead to misclassification on an unknown victim classifier.
Suppose we have access to a source dataset D consisting of N training samples from C number of classes, with each sample/image possibly consisting of multiple object labels, i.e., multi-label images.
Each i thsample in D is represented as x ( i ) ∈
R H × W × T (with height H , width W , and channels T ) containinglabels y ( i )
=[
y ( i ) 1 , ··· ,y ( i ) C ] ∈Y ⊆{ 0 , 1 } C .
More specifically, if sample x ( i ) associated with class c , y ( i ) c
=1indicates the existence of an object from class c in x ( i ) .
Further, we have access to a surrogate multi-labelclassifier trained on D denoted as f ( · ) which is employed to optimize the perturbation generator G θ ( · ) ’sweight θ .
For ease of exposition, we drop the superscript i in further discussion.
3.1 Adversary Equipped with Pre-Trained CLIP
We aim to train a generator G θ ( · ) that learns to create perturbations from its observations by foolinga surrogate classifier f ( · ) during its training phase.
Now, as G θ ( · ) learns to create perturbations δ inaccordance to f ( · ) , it is bounded by the features extract from f ( · ) in order to contrast x and x δ ( e.g .final-layer logits in [10, 11] or mid-level features [12, 13]).
In this work, we explore a case where wehave access to a pre-trained vision-and-language model like CLIP that can be utilized as a loss networkto train G θ ( · ) .
Our motivation for using CLIP is to exploit its joint text and image matching property andcompute two embeddings: clean image embedding extracted from the image encoder and its least similartext embedding extracted from the text encoder.
For example in Figure 1, CLIP’s joint text-image matchingproperty allows us to compute the least similar embedding from text prompt “ cat and bicycle ” from animage that’s most similar to embedding from text prompt “ sofa and bottle ”.
We next describe GAMA ’smethod and present a novel strategy to use CLIP’s model to train G θ ( · ) .
Note that we assume eachimage contains two co-occurring classes for creating text prompts, mainly restricted due to computationof co-occurrence matrices of dimension C × C available for multi-label datasets.
As we will see later,co-occurrence matrices allow us to discard pairs of classes that would not occur in real-world scenarios.
GAMA Overview.
Before training G θ ( · ) , we first compute a text embedding matrix A txt =[ ρ 1 , ρ 2 , ··· , ρ N ] ∈ R N × K with ρ n ∈ R K (explained in detail later) using the CLIP text encoder T ( · ) .
During training G θ ( · ) , we start by feeding the clean image x to the CLIP image encoder I ( · ) and computing an image embedding ρ img
= I ( x ) ∈ R K .
Next, a particular vector ρ txt ∈ R K from A txt is retrieved that isleast similar to ρ img .
Then, we feed x to G θ ( · ) and create x δ while ensuring it to be under given perturbationℓ ∞ budget ϵ using the perturbation projection operator P ( · ) .
These clean and perturbed images are then fedto the surrogate classifier f ( · ) to extract K -dimensional embeddings at specific k th layer, denoted by f k ( x )and f k ( x δ ) respectively.
Finally, the aforementioned quadruplet embeddings ( ρ txt , ρ img , f k ( x ) , f k ( x δ ) ) are used to compute a contrastive learning based CLIP text embedding-guided loss L txt ( ρ txt , f k ( x ) , f k ( x δ ))and a regression learning based CLIP image embedding-guided loss L img ( ρ img , f k ( x δ )) to compute the finalobjective L .
We also include a loss function that further maximizes the difference between f k ( x ) andf k ( x δ ) solely from the surrogate classifier’s perspective.
This loss L is minimized to update the weights ofthe generator θ .
The whole GAMA paradigm is illustrated in Figure 2 and summarized in Algorithm 1.
The details of loss objectives L img and L txt (with text embedding matrix A txt ) are discussed next.
CLIP text embedding-guided loss ( L txt ).
Let z
=
f k ( x ) and (cid:98) z
= f k ( x δ ) .
The CLIP frameworkinherently learns the text and vision embedding association via a contrastive learning regime [36, 68],constraining the feature embeddings of the input image and its counterpart language description to be assimilar as possible.
Different from CLIP’s image embedding ρ img , CLIP’s text embedding ρ txt allows usto look beyond the pixel-based features.
More specifically, CLIP’s vision-and-language aligning abilityallows us to utilize text features to craft transferable image perturbations.
Hence, we can optimize G θ ( · ) tocreate perturbed images x δ that do not follow the same text embedding alignment as their clean imagecounterpart x .
In order to cause this text misalignment, we create a triplet of embeddings where the anchor(cid:98) z is pushed away from z while pulling it closer to a text embedding ρ txt that is least associated or similarto a clean image x .
To compute this triplet, the following two steps are performed.
• Before training, compute A txt .
The goal is create a dictionary or matrix of text embeddings whichcan be utilized to retrieve ρ txt during optimization of G θ ( · ) .
Firstly, we generate language derivatives or text prompts using classes of source distribution.
This means we only need to know all the available Cclasses in D but not their specific association with x .
Secondly, with assumption that each clean image xis associated with two classes, we can generate C 2 text prompts and create a matrix A txt of size C 2 × K .
For example, if classes ‘cat’, ‘dog’, ‘person’ and ‘boat’ exist in D , then one can create text promptssuch as “ a photo depicts cat and dog ” or “ a photo depicts person and boat ” (see Figure 1 for 10random examples extracted from CLIP’s ‘ViT-B/16’ model using Pascal-VOC’s classes).
Here, the partof the text prompt underlined is a recommended ‘prefix’ common to all text prompts as suggested in[67].
However, such A txt can contain embeddings from prompts that are generated from classes that donot exist in real life.
To circumvent this, we utilize an object co-occurrence matrix O∈ R C × C (a binarymatrix) to estimate the co-occurrence relationships between classes.
Computed from the training data setcontaining C classes, O is first initialized with a matrix containing only zeros.
Then, an element O ij ( i throw and j th column of O ) is set to 1 if objects from classes
y i and y j appear together at least in one image.
Computing such co-occurrence information is a common practice in multi-object downstream problems[26, 27, 69–72].
We use O provided by [69].
Using such a co-occurrence matrix, we only create textprompts from a pair of classes that occur together according to O .
This leads to a text embedding matrixof size A txt of size
∥O∥
× K where
∥O∥ 0 denotes total non- zero elements.
• During training, compute ρ txt .
CLIP’s training objective allows it to push the embeddings of associated
image-text pairs closer compared to non-matched pairs.
We leverage this property to compute the least
similar text embedding ρ txt w.r.t.
image embedding ρ img .
During each training epoch, we randomly
sample B candidates [ ρ 1 , ρ 2 , ··· , ρ B ] from A txt and then compute ρ txt as follows:
[Equation]
Here, cs ( · ) denotes cosine similarity.
Next, we force f k ( x δ ) to align with ρ txt while misaligning withf k ( x ) .
Mathematically, this is implemented as contrastive learning [73, 74] objective as follows.
[Equation]
where α> 0 is the desired margin between clean and perturbed image embedding, and [ v ] + =max(0 ,v ) .
L txt pulls away embeddings of x and x δ by making them keep a margin α while pushing dissimilarembeddings (cid:98) z and ρ txt closer than the given margin.
CLIP image embedding-guided loss ( L img ).
Due to CLIP’s learning on ∼ 400 million internet retrievedimages from diverse categories and its consequential strong zero-shot image recognition performanceover different distributions [36, 41], we argue that its image encoder I ( · ) outputs an embedding that hascaptured attributes of input image with distinct generalized visual features.
GAMA leverages this to ouradvantage, and maximizes the difference between (cid:98) z and CLIP’s image encoder’s embedding for cleanimage ρ img .
The aim of such an objective is to increase the transferability strength of G θ ( · ) perturbationsusing the generalized features computed from I ( · ) .
This is envisioned using a regression learning basedloss described as follows:
[Equation]
Final Learning Objective ( L surr ).
Loss functions L img and L txt are finally added to a surrogate model loss
L surr that minimizes the cosine similarity of f k ( x ) and f k ( x δ ) .
Choice of layer k is dependent on featureoutputs of the CLIP model employed.
All embeddings are normalized before computing the loss functions.
[Equation]
Overall, L surr maximizes the difference between x and x δ from surrogate f ( · ) ’s perspective, while L img and
L txt enhance its transferability using CLIP’s perspective.
Attack evaluation.
We assume that the attacker has no knowledge of victim classifier g ( · ) and its datadistribution D t .
Further, there is a perturbation budget of ϵ defined by an ℓ ∞ norm.
To launch an attack, weinput a clean image x t from target dataset D t to optimized G θ ( · ) and craft imperceptible perturbations δ tin order to alter the decision space of the target victim classifier g ( · ) (pre-trained on D t ).
Mathematically,this can be represented as y t ̸
= (cid:98) y t where, y t =
g (cid:0) x t (cid:1) and (cid:98) y t
= g (cid:0) x t + δ t (cid:1) with ∥ δ t ∥ ∞ ≤ ϵ .
We can causefollowing attack scenarios after training G θ ( · ) against f ( · ) on D :
• Scenario 1 : an attack termed white-box if f ( · )=
g ( · ) and D = D t• Scenario 2 : an attack termed black-box if either f ( · ) ̸ = g ( · ) or D̸ =
D t
A real-world attack is generally modeled by Scenario 2 as an adversary would not have the knowledge ofvictim model g ( · ) ’s architecture, its training data distribution D t and the task it performs e.g . single-labelclassification, multi-label classification, or object detection, etc .
The perturbations that make an attacksuccessful in Scenario 2 should be highly transferable.
Experiments
In this section, we analyze the strength of GAMA under diverse practical attack settings.
We also performan ablation analysis of GAMA , test the attack robustness against various defenses ([78, 79], median
[Table] [Table]
[Table] [Table] [Table]
[Table] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
blurring, context-consistency check), as well performance of attacks on different architecture designs.
Notethat we provide more black-box attack results of our method in the supplementary material.
Baselines.
As there are no prior works for generative attacks that learn on multi-object scenes usingmulti-label classifiers, we define our baselines by adapting existing state-of-the-art generative attackssummarized in Table 1.
Specifically, the cross-entropy loss in GAP [10] and CDA [11] is replaced withbinary cross-entropy loss to handle the prediction of multiple labels during training.
Training Details.
We use the multi-label datasets PASCAL-VOC [80] and MS-COCO [81] to traingenerators for the baselines and our method.
Unless otherwise stated, perturbation budget is set to ℓ ∞ ≤ 10for all experiments.
We chose the following surrogate models f ( · ) (Pascal-VOC or MS-COCO pre-trainedmulti-label classifiers): ResNet152 (Res152)
[77], DenseNet169 (Den169) [82], and VGG19 [64].
For the
CLIP model, we use the ‘ViT-B/16’ framework [36].
See supplementary material for more training details.
Inference Metrics.
We measure attack performances on multi-label classifiers using hamming score (
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
4.1 Results and Analysis
[Table]
All trained perturbation generators (trained onlyon multi-label datasets) are extensively evaluatedunder following victim model settings.
• White-box and black-box (multi-label classification, different model than f ( · ) ):
We evaluate the attacks in white-box and black-box settings on six victim multi-label classifiers (VGG16,VGG19, ResNet50 (Res50), Res152, Den169, andDenseNet121 (Den121)) in Table 2 and Table 3(white-box attacks are marked with cell color).We outperform all baselines in the majority of cases, with an average absolute difference (w.r.t. closestmethod) of ∼ 13 percentage points (pp) for Pascal-VOC and ∼ 4.46pp for MS-COCO.• Black-box (single-label classification): We evaluate the attacks in a black-box setting with varioussingle-label classifiers for CIFAR10/100 [85] (coarse-grained tasks [13]), CUB-200-2011 (CUB) [32],Stanford Cars (Car) [86], and FGVC Aircrafts (Air) [87] (fine-grained tasks [13]) in Table 6, and ImageNet[88] (
50K validation set) in Table 4 and Table 5.
Following [13], the victim models of coarse-grained tasksare taken from [76], fine-grained task models (Res50, SENet154 (SeNet), and SE-ResNet101 (se-Res101)[89]) from [90], and six ImageNet models from [33].
Here, we beat our closest baseline in all cases by∼ 13.33pp for Pascal-VOC and ∼ 5.83pp for MS-COCO on six ImageNet models.
Note that the ImageNetresults also demonstrate the drop in performance of TAP [12] and BIA [13] attacks that show close to 0%top-1 accuracy when G θ ( · ) is trained on ImageNet on the attacker side [12, 13].
We hypothesize that sucha drop in performance is due to sensitivity to the dataset of the manually selected mid-level layer of f ( · )used by the attacker.
We observe a similar trend when attacking non-ImageNet distributions as suggestedby BIA [13] in coarse and fine-grained tasks in Table 6.
In this case, GAMA beats the prior attacks byaverage ∼ 13.33pp when G θ
) is trained with Pascal-VOC.
• Black-box (Object detection): We also evaluate a difficult black-box attack with state-of-the-art MSCOCO object detectors (Faster RCNN with Res50 backbone (FRCN) [91], RetinaNet with Res50 backbone (RNet) [92], DEtection TRansformer (DETR) [93], and Deformable DETR (D 2 ETR)
[94]) in Table
available from
It can be observed that
GAMA outperforms its competitors when G θ
· ) is trained with
Pascal-VOC
In Figure 3, we further study the transferability of attacks depending on the type of victimarchitecture: standard which follow the standard modules like Residual blocks [77] to build the classifier,and custom where the victim classifier doesn’t adhere to a specific pattern of network modules.
In bothcases, GAMA consistently maintains better attack rates than other generative attacks.
This demonstratesconvincing transferability of perturbations crafted from GAMA ’s G θ ( · ) under diverse black-box settings.
We provide additional results in the supplementary material.
Robustness of Attacks against Defenses.
To analyze the robustness of all the methods, we launchmisclassification attacks ( G θ ( · ) trained on MS-COCO with the surrogate model as Den169) when thevictim deploys input processing based defense such as median blur with window size as 3 × 3 , and Neural
Representation purifier (NRP) [78] on three ImageNet models (VGG16, Res152, Den169).
From Table 8(a)and Table 8(b), we can observe that the attack success of GAMA is better than prior methods even whenthe victim pre-processes the perturbed image before making decisions.
In Figure 8(c), we observe that
Projected Gradient Descent (PGD) [79] assisted Res50 is difficult to break with GAMA performing slightlybetter than other methods.
Finally, motivated by [96], we analyze an output processing defense scenariowhere the victim can check the context consistency of predicted labels on perturbed images using theco-occurrence matrix O .
In particular, if a perturbed image is misclassified showing co-occurrence oflabels not present in O , we term this as a detected attack .
Otherwise, we call it an undetected attack .
To
[Figure]
We analyze the contribution to the introduc- tion of each loss function L and L measure this performance, we first compute the co-occurrence matrix O δ by perturbing all the test setimages and estimate its precision w.r.t.
ground-truth O .
To check for attacks that have high precision valuep and high misclassification rate, we calculate a ‘context score’ (in this case, higher is better) that is aharmonic mean of p and misclassification rate (1-accuracy).
We show the attack performance against thiscontext consistency check in Figure 8(d) for both Pascal-VOC and MS-COCO averaged over all surrogatemodels under white-box attacks.
It can be seen that GAMA presents itself as the best undetected attackcompared to prior works.
Ablation Analysis.
We dissect the contribution of each loss function in our proposed loss objective of
Equation (4) in Figure 4 where G θ ( · ) is trained with Pascal-VOC with Res152 surrogate model.
We analyzethe attack transferability to different victim models (VGG16, Res50).
We observe that the introductionof each loss objective ( left to right ) increases the strength of the attack both in the same distribution(Pascal-VOC) as the attacker and in the unknown distribution (ImageNet) on both victim classifiers.
Finally,we visualize some perturbed image examples crafted by GAMA in Figure 5.
Conclusion
In this paper, we propose a new generative attack GAMA that can learn to create perturbations againstmulti-object scenes.
For the first time in generative attack literature, we show the utility of a pre-trainedvision-and-language model CLIP to optimize effectual perturbation generators.
Specifically, CLIP’sjoint text and image aligning property allow us to use natural language semantics in order to handlethe multi-object semantics in the input scene to be perturbed.
To demonstrate GAMA ’s efficiency, weperform extensive experiments that show state-of-the-art attacks across a wide range of black-box victimmodels (multi-label/single-label classifiers, and object detectors).
We also evaluate the robustness of ourattacks against various defense mechanisms.
As part of our future works, we will explore more complexmethodologies to employ vision-language models both for adversarial attacks and defense systems.
Limitations and Societal Impacts
Limitations .
The pre-trained CLIP model ‘ViT-B16’ outputs a 512-dimensional embedding that restrictsus to compare the features extracted from the surrogate model in our losses of the same size.
Anotherlimitation of our method is the use of co-occurrence matrices to extract the right pair of classes that existtogether in real-world scenes.
In this paper, we make an assumption that text prompts are created usingtwo classes that exist together according to the co-occurrence matrix of size C × C (for C classes in thedata distribution).
However, we can also use a triplet of classes that exist together in an input scene whichwould need a co-occurrence tensor of size C × C × C .
Computing such a huge tensor would be tedious tocover all the images provided in the train set (usually in the order of 1000).
Societal Impacts .
Adversarial attacks are designed with the sole goal to subvert machine decisions byany means available.
Our attack approach shows one such method where a benign open-sourced visionlanguage model can be utilized by an attacker to create potent perturbations.
This demonstrates the needfor the victim to prepare for constantly evolving attacks that may cause major harm in real-world systems.
We believe that our work can help further propagate research into designing efficient and robust modelsthat do not break down to attacks built upon multi-modal (in our case, text-image) features.