Generalizing Adversarial Training to Composite Semantic Perturbations
Abstract
Model robustness against adversarial examples has been widely studied, yet the lack of generalization to more realistic scenarios can be challenging.
Speciﬁcally, recent works using adversarial training can successfully improve model robustness, but these works primarily consider adversarial threat models limited to (cid:96) p -norm bounded perturbations and might overlook semantic perturbations and their composition.
In this paper, we ﬁrstly propose a novel method for generating composite adversarial examples.
By utilizing component-wise PGD update and automatic attack- order scheduling, our method can ﬁnd the optimal attack composition.
We then propose generalized adversarial training ( GAT ) to extend model robustness from (cid:96) p norm to composite semantic perturbations, such as Hue, Saturation,
Brightness, Contrast, and Rotation.
The results show that GAT can be robust not only on any single attack but also on combination of multiple attacks.
GAT also outperforms baseline adversarial training approaches by a signiﬁcant margin.
1. Introduction
Deep neural networks have shown remarkable success in a wide variety of real-life applications, ranging from biometric authentication (e.g., facial image recognition), medical diagnosis (e.g., CT lung cancer detection) to autonomous driving systems (trafﬁc sign classiﬁcation), etc.
While these models can achieve great performance on benign data points, recent researches have shown models can be easily fooled by malicious data points crafted intentionally with adversarial perturbations, even with high standard accuracy (Szegedy et al., 2014). .edu.tw > , Pin-Yu
Chen < pinyu.chen@ibm.com > , Tsung-Yi Ho < tyho@cs.nthu.edu.tw > .
Accepted by the ICML 2021 workshop on A Blessing in Disguise: The Prospects and Perils of Adversarial Machine Learning.
Copyright 2021 by the author(s).
Therefore, the rapid growth of defense mechanisms attempts to improve models’ robust accuracy against adversarial examples (Madry et al., 2018; Zhang et al., 2019).
Nonetheless, most existing methods typically consider single threat models bounded by speciﬁc distance (e.g., (cid:96) 2 , (cid:96) ∞ , etc.) and might overlook risk from the combination of multiple threats models.
To tackle this issue, we propose a novel defense approach, named generalized adversarial training (GAT) , which can harden against a wide range of threat models, from single (cid:96) ∞ and semantic (e.g., Hue, Saturate,
Rotation, Brightness, Contrast) to the combination of them.
Furthermore, we empirically discover adversarial training via combining each threat model sequentially in different orders has a signiﬁcant inﬂuence on the robust accuracy of classiﬁers.
As Fig.
1 shows the ﬂow of GAT, we choose several attacks from attack pools, then sequentially combining the perturbations of them with a limited number of running steps.
The order of combinations can be scheduled by the end of the combination in each round based on the order scheduling process.
We propose a more potent composite threat model that allows scheduling the order from attack pools and optimizing perturbations of each attack component to evaluate the robustness of our GAT.
Different from existing works, this paper aims to address the following questions: (a) Can we generalize adversarial training from speciﬁc single threat models to multiple?
(b)
Can we optimize the order of combination between semantic and (cid:96) p norm perturbations?
(c) Can GAT outperform other adversarial training baselines for either (cid:96) p or unseen threat models?
Our main contributions in this paper provide afﬁrmative answers to the aforementioned fundamental questions.
We propose GAT , a novel and uniﬁed approach to defense composite adversarial examples generated from multiple threat models, including (cid:96) ∞ , Hue, Saturation, Rotation, Brightness, and Contrast.
To the best of our knowledge, GAT is the ﬁrst framework that robust on both single and composite adversarial attacks.
We elucidate the design principles of our componentwise projected gradient descent (PGD) for updating parameterized perturbations in each semantic attack.
We also propose using order scheduling with the mirror descent algorithm (MDA) to optimize the scheduling matrix, which can further strengthen the composite threat models.
We evaluate robust accuracy for our GAT and compare it with other adversarial training benchmarks (Laidlaw et al., ; Madry et al.,
; Zhang
et al., 2019)
Our results show that GAT outperforms them on all composite
adversarial threats by 25% ∼ 40% in
ran
dom order
and
by 8% ∼ 25% in scheduled order
, sug-
gesting that GAT is an effective approach to defend multiple adversarial attacks.
2. Related Work 2.1. Adversarial Semantic Perturbations
Most recent studies on adversarial machine learning ML) focus on generating examples that can fool the neural network to make the wrong prediction (Biggio & Roli, 2018).
Several works have primarily focused on the vulnerability of deep neural networks against general (cid:96) p adversarial threats (Goodfellow et al., 2015; Carlini & Wagner, 2017;
Chen et al., 2018).
Some others consider the adversarial threats beyond (cid:96) p , which generally occurs in natural transformation such as geometry, color, and lightness, named semantic perturbations.
Typically, in contrast to the (cid:96) p norm perturbations, which usually have a speciﬁc bound for perturbation, semantic attacks are not suitable for this restriction and parametrized them can be a challenging task.
For generating semantic perturbations with color translation, (Hosseini &
Poovendran, 2018) randomly choose the Hue value in HSV color space.
(Laidlaw & Feizi, 2019) update 3-dimensional values in LUV space with PGD.
For geometric transformation, (Xiao et al., 2018; Engstrom et al., 2019) target rotate transformation, the former used coordinate-wise optimization in each pixel, which is computationally expensive.
The latter propose a simple way by parametrizing a set of tunable parameters for spatial transformation.
Also, (Wong et al., 2019) deﬁned Wasserstein distance for those adversarial examples with large norm bounded.
In (Mao et al., 2020), they ﬁrst propose using genetic algorithms for searching the best combination of multiple adversarial attacks that are stronger than single adversarial attacks.
However, they separately search the examples in subspace corresponding to the same norm.
2.2. Adversarial Training
The most well-known approach for learning a robust model is adversarial training (Madry et al., 2018; Zhang et al., 2019).
However, most of them only target on the single adversarial threat.
Speciﬁcally, a robust classiﬁer that can help defend against a speciﬁc (cid:96) p threat model still has low robustness to either (cid:96) q threats ( p (cid:54) =
q ) or semantic threats.
The adversarial robustness under multiple adversarial threats has been discussed in (Tram`er & Boneh, 2019; Maini et al., 2020).
They propose multiple norm adversarial training, which yield models simultaneously robust against multiple (cid:96) p (e.g., (cid:96) 1 , (cid:96) 2 , and (cid:96) ∞ ) attacks.
In contrast to their works, we shed the light on the signiﬁcant of model robustness against multiple threats not only in (cid:96) p but also semantic perturbations.
3. Generalized Adversarial Training
This section presents our proposed method, named GAT, for adversarial training with composite adversarial examples and an order scheduling algorithm for multiple attacks.
A schematic overview of GAT is illustrated in Fig.
3.1. Problem Formulation Composite adversarial attacks with order scheduling :
We propose method to generate adversarial examples with the combination of multiple attack algorithms.
Let F :
X → R d be an image classiﬁer deﬁned on X ∈ X , where d is the number of classes and F ( X ) ∈ R d .
Suppose there is an attack space
Ω A = { A 1 , A 2 , ...
A K } with corresponding epsilon space { (cid:15) 1 , (cid:15) 2 , ...(cid:15) K } , where K is the number of attacks.
An input X can be transformed to X adv via updating perturbations δ k in each attack operation A k ∈ Ω A , where k ≤ K .
By selecting one attack A k , δ k can be optimized through maximizing the classiﬁcation error (e.g., cross-entropy loss L ce ) within a boundary (cid:15) k :
[Equation]
To combine multiple attacks from Ω A in a speciﬁc order, we can deﬁne the associated scheduling matrix Z for X adv .
Suppose we wish to combine N attacks, the scheduling matrix
Z is then a N × N square matrix.
In our cases, Z ij , i, j ≤ N is binary.
Z ij = 1 means the attack A j is launched at time i .
By construction, Z is a doubly stochastic matrix, 1 (cid:62)
Z = 1 and Z 1 = 1 , 1 is all-one vector.
Relax Z to a binary matrix, Z =
[ Z 1 , ...
Z N ] .
From time step 1 to N, we deﬁne our composite adversarial images X
c − adv as Eq.
[Equation]
In every time step t ∈ [1 , N ] , we launch an attack indexed by Z t and optimize the δ t within the constraint:
[Equation]
Sinkhorn Operator : Learning an optimal order which is discrete and non-differentiable from scheduling matrix can be viewed as a problem in differentiable relaxation.
In (Mena et al., 2018), they theoretically showed how to extend Sinkhorn operator (Sinkhorn, 1966) to learn over the permutations.
Similar as them, in our case, choosing an order over doubly stochastic matrix Z can be cast as a maximization problem, where the set of Z is convex belonging to the Birkhoff polytope B .
In more speciﬁc, we formulate our problem in Eq. 3.
P denotes the doubly stochastic matrix in the set B P , (cid:104) P, Z (cid:105) [Equation]
= (cid:80)
[Equation]
(max i
[Equation]
P
[Equation]
· Z
[Equation]
[Equation]
Our goal aims to use P to approximate the optimal Z .
To update the scheduling matrix Z , we can use mirror descent algorithm (Wang & Banerjee).
The update function can be formed as a double-loop algorithm
[Equation]
where Π B k is the projection function for Z to project back onto the Birkhoff polytope which can be solved by Sinkhorn algorithm in a limit of iterations i (Sinkhorn & Knopp, 1967).
3.2. Component-wise Projected Gradient Descent
For most of the semantic perturbations, their parameters are within the continuous value.
In (Mohapatra et al., 2020), they show how semantic perturbations can be transformed from semantic space to general (cid:96) p space.
Motivated by this, we propose to update the parameters of semantic attacks by gradient descent algorithm with speciﬁc projection to each continuously semantic space.
We show how to update the parameters in correspondence to ﬁve different semantic perturbations, including (i) hue, (ii) saturation, (iii) brightness, (iv) contrast, and (v) rotation.
As Eq.
5, we extend the iterative method (Kurakin et al., 2017) to optimize our semantic perturbations, which is deﬁned as:
[Equation]
where α is a small step size, J ( · ) is the loss function (e.g., cross entropy), and we denote the clip (cid:15) ( z ) as:
[Equation]
Hue : The scale of this space is represented as a color wheel, ranging from 0 to 2 π .
We deﬁne (cid:15) h ∈ [0 , π ] as the bound of δ h , which is the variation of hue value, i.e. | δ h | ≤ (cid:15) h .
Let x h denote the hue value of the image x in HSV space.
The variance δ h is initially chosen from uniform distribution
U ( − (cid:15) h , (cid:15) h ) , and the hue value x h can be updated iteratively by Eq.
[Equation]
Saturation :
This space determines the colorfulness of the image.
If the saturation value x s of image x gets closer to zero, the color becomes more gray.
We deﬁne saturation factor (cid:15) s ∈ R + and | δ s | ≤ (cid:15) s .
Same as the Hue function, we randomly initialize δ s from U (0 , ∞ ) and optimize x s iteratively by using Eq. 8:
[Equation]
Brightness and Contrast : Different from Hue and Saturation, this space determines the lightness of images.
The transformation is directly applied on the pixel space instead of HSV space.
Let δ b , δ c denote the variation of the brightness value and contrast factor, subject to | δ b | ≤ (cid:15) b ∈ [ − 1 , 1] and (cid:107) δ c (cid:107) ≤ (cid:15)
c ∈ R + .
Let x t denote the attacked image of x in step t .
The iterative relation is deﬁned as Eq. 9:
[Equation]
Rotation : For this transformation, we wish to ﬁnd parameter θ such that rotating image around the center in an angle can be misclassiﬁed by model.
Here, we deﬁne the rotate angle (cid:15) θ ∈ [ − π, π ] and δ θ ≤ (cid:107) (cid:15) θ (cid:107) .
Given an input image x , the pixel index of X is [ u, v ] , the center of x is [ u c , v c ] , and the scale factor is γ ∈ [0 , 1] .
As Eq.
10 shows, we construct the transformed function for each index [ u, v ] .
Here,
P = γ cos [Equation]
θ t + δ θt +1 ) and R = γ sin( θ t + δ θt +1 ) .
[Equation] +
[Equation]
clip [Equation] θ
[Equation]
3.3. Generalized Adversarial Training (GAT)
To harden the classiﬁer against multiple composite examples, we wish to extend the basic adversarial training approach.
Suppose we wish to train a model F ( · ) over a [Table] general data distribution ( x, y ) ∼ D and also robustify against the adversarial data distribution ( x (cid:48) , y (cid:48) ) ∼ D (cid:48) generated from ( x, y ) .
GAT follows the min - max optimization function in (Zhang et al., 2019).
In particular, the inner maximization is solved using composite adversarial attacks proposed in Eq.
[Equation]
where λ is a balance factor between natural loss and robust loss.
The training procedure aims to minimize a regularized surrogate loss L ( · ) by optimizing F .
4. Experiments
In this section, we ﬁrst elucidate the experimental settings and then present the performance evaluation of the proposed defense method GAT against multiple composite attacks on the Cifar-10 dataset (Krizhevsky, 2009).
Attack and Training Setting : To evaluate the robustness of the classiﬁer, we generate the adversarial examples by using Cifar10 test sets, including 10000 images.
For each attack A k ∈ Ω A , we restrict (cid:15) k within speciﬁc interval (in
Appendix 6.1).
We adopt the whole training sets, including 50000 images for doing generalized adversarial training and apply WideResNet34 (Zagoruyko & Komodakis, 2016) as the model architecture in two training settings, including training from scratch ( GAT-s ) and ﬁnetuning on the pretrained model ( GAT-f ) training with TRADES (cid:96) ∞ (Zhang et al., 2019).
Here, we report robust accuracy (RA), which is the percentage of inputs for which the model against the adversarial threats.
4.1. Experimental Results
In Table 1, we show the RA (%) of composite semantic and full attacks.
The composite semantic attack consists of the combination of 5 attacks.
For instance, one can generate an example with Hue, Saturation, Rotation, Brightness and
Contrast in a speciﬁc order.
For full attacks, we generate examples with All 5 semantic attacks + (cid:96) ∞ .
We evaluate RA in two order settings for each attack: random and scheduled.
The result shows GAT-s outperforms other baselines by 13% ∼ 49.3% on semantic attacks and 21% ∼ 37.1% on full attacks.
More importantly, our GAT-f can further improve the RA by 3.2% ∼ 8.4 %, compared to GAT-s. Table ??
shows the RA of three composite attacks with different combinations and other results are shown in the Appendix 6.6.
To better understanding the improvement in our approaches, we do the analysis of loss landscape.
As Fig.
2 shows, the curve of cross entropy (Xent) loss of one random adversarial example becomes smoother and lower in our GAT.
More visualization are shown in Appendix 6.2.
5. Conclusion
In this paper, we proposed GAT, a novel approach to strengthen the classiﬁers on those composite adversarial examples via using component-wise PGD and scheduling algorithm to ﬁnd the worst-case examples from multiple semantic spaces.
Compared to the existing adversarial training method, GAT allowed the classiﬁer to defend against a variety of adversarial threats, from L p norms to semantic.
Evaluated on the Cifar-10 dataset with two training settings for GAT, including training from scratch and ﬁnetuning on
TRADES- (cid:96) ∞ pretrained model, our results demonstrated that GAT achieved high robust accuracy on most composite attacks, which provided a new perspective for defense method on multiple adversarial threats.
6. Appendix 6.1. Attack power settings
Hue, (cid:15) h
Saturate, (cid:15) s
Bright, (cid:15) b
Contrast, (cid:15) c
Rotate, (cid:15) θ (cid:96) ∞ , (cid:15) (cid:96)
Range of Attack Power
[Table]
6.2. Analysis of Loss Landscape and the Component-wise PGD Visualization
To better understanding why GAT leads to great improvement, in Fig.
3, we compare the curve of loss landscape for models trained by our GAT (from scratch and ﬁnetune) with others trained by TRADES( (cid:96) ∞ ) and without adversarial training.
We visualize the loss landscape of random example when performing different attacks in an inference time.
We empirically observe that our models have more smoother and lower curve (red/green line) than the others on all of semantic single attacks, which shows the effectiveness of using composite adversarial examples for adversarial training.
To show the effectiveness of component-wise PGD (comp-PGD), in Fig.
4, we simulate the update process of comp-PGD when performing semantic attack.
For each attack, we propose to ﬁnd the adversarial examples via sampling 20 random start points for updating (cid:15) .
We observe that in semantic attacks, comp-PGD indeed help searching the worst case by maximizing the loss during each attack.
6.3. Model Details
Here, we show the details of baseline models training with different adversarial training approaches.
The marker † means using the architecture ResNet50 for training and ∗ means using WideResNet34 for training.
Normal † : The model training with 50000 clean images on ResNet50.
Normal ∗ : The model training with 50000 clean images on WideResNet34.
PAT † self :
The pretrained model training with perceptual adversarial training, where the adversarial examples correlates with the LPIPS distance (based on ResNet50) from the natural example (Laidlaw et al., 2020).
PAT † alex : The pretrained model training with perceptual adversarial training, where the adversarial examples correlates with the LPIPS distance (based on AlexNet) from the natural example (Laidlaw et al., 2020).
Madry †∞ : The pretrained model training with L p adversarial training proposed in (Madry et al., 2018).
Trades ∗∞ : The pretrained model training with L p adversarial training proposed in (Zhang et al., 2019).
6.4. Implementation Details
For the attack part, we set the inner iteration number i as 5, which means each component has 5 steps during PGD.
Due to the non-convex feature of semantic attacks, in the process of each component doing PGD, we trigger the early-stopped condition to achieve the maximum attack success rate, once the adversarial example has been successfully generated.
Notably, in the setting using scheduled order, we have an outer loop for updating the scheduling matrix with at most 5 times.
For the adversarial training part, we apply GAT with the WideResNet34 network (Zagoruyko & Komodakis, 2016).
To make our training faster, we rather using composite examples update with random value in each semantic attack than using examples update with PGD step.
6.5. Algorithm Details
Algorithm
Input: classiﬁer f ( · ) , Attack space Ω A , input x , label y , step α , maximum number of iterations T , number of gradient update iteration K , doubly while
12: x
13: if F
X adv
= y ) then 14:
Break 15: else 16: Schedule[-Order with Loss and Z by (4) 17: Z t + 
-]
( ∂L ce ( · ) ∂Z ij k
S
← arg max i i 20: end if 21: end while
6.6. Additional Experiment Results and Adversarial Examples
In Table 3 and Table [Table] since the single attacks have no advantage in combination, the loss did not drop too much in this setting.
We can see that for normal training models, the defense against semantic attacks is generally better than other adversarial training models.
Also, in Table 9 and Table [Table] GAT and normal training models perform well for semantic attacks.
However, once the (cid:96) ∞ attack is enabled, the normal training models lost its advantages.
In Table 5 and Table [Table] we can see that although the two attacks do not outperforms, the GAT can be greatly improved in the three attacks (Table 7 and
Table 8) and the full attack (Table 9 and Table 10) once the number of enabled attacks in the combination is increased.
6.6.1. S INGLE A TTACK
[Table] [Table]
[Table] [Table] [Table]
[Table]
6.6.3. M ULTIPLE A TTACKS : 3 ATTACKS
Since the width of the page is limited [Table] here we noted the attack in the following abbreviations:
• 0 : Hue, 1 : Saturate, 2 : Rotation, 3 : Brightness, 4 : Contrast, 5 : (cid:96) ∞ .
• s : scheduled order, r : random order.
We enabled the (cid:96) ∞ attack in all three attacks to mix the norm spaces, which is semantic space, spatial space, and L p space
[Table]
Training Clean (0,1,5) (r) (0,1,5) (s) (0,2,5) (r) (0,2,5) (s) (3,4,5) (r) (3,4,5) (s) Normal † 95.
[Table]
[Table] [Table]
[Table] [Table]
6.7. Additional Adversarial Examples
Here we provide some of the adversarial examples in above experimental settings.
We arrange the images into different columns; the most left column is the original images; the following two columns are the examples and their magniﬁed differences compared with the originals.
[Table]
[Figure]
ℓ ∞ , Hue ℓ ∞ , Saturate ℓ ∞ , Rotation ℓ ∞ , Brightness ℓ ∞ , Contrast
[Figure]