S EMANTICS P RESERVING A DVERSARIAL A TTACKS
Anonymous authors
Paper under double-blind review
A BSTRACT
While progress has been made in crafting visually imperceptible adversarial examples, constructing semantically meaningful ones remains a challenge.
In this paper, we propose a framework to generate semantics preserving adversarial examples.
First, we present a manifold learning method to capture the semantics of the inputs.
The motivating principle is to learn the low-dimensional geometric summaries of the inputs via statistical inference.
Then, we perturb the elements of the learned manifold using the Gram-Schmidt process to induce the perturbed elements to remain in the manifold.
To produce adversarial examples, we propose an efﬁcient algorithm whereby we leverage the semantics of the inputs as a source of knowledge upon which we impose adversarial constraints.
We apply our approach on toy data, images and text, and show its effectiveness in producing semantics preserving adversarial examples which evade existing defenses against adversarial attacks.
I NTRODUCTION
In response to the susceptibility of deep neural networks to small adversarial perturbations (Szegedy et al., 2014), several defenses have been proposed (Liu et al., 2019; Sinha et al., 2018; Raghunathan et al., 2018; Madry et al., 2017; Kolter & Wong, 2017).
Recent attacks have, however, cast serious doubts on the robustness of these defenses (Athalye et al., 2018; Carlini & Wagner, 2016).
A standard way to increase robustness is to inject adversarial examples into the training inputs (Goodfellow et al., 2014a).
This method, known as adversarial training, is however sensitive to distributional shifts between the inputs and their adversarial examples (Ilyas et al., 2019).
Indeed, distortions, occlusions or changes of illumination in an image, to name a few, do not always preserve the nature of the image.
In text, slight changes to a sentence often alter its readability or lead to substantial differences in meaning.
Constructing semantics preserving adversarial examples would provide reliable adversarial training signals to robustify deep learning models, and make them generalize better.
However, several approaches in adversarial attacks fail to enforce the semantic relatedness that ought to exist between the inputs and their adversarial counterparts.
This is due to inadequate characterizations of the semantics of the inputs and the adversarial examples — Song et al.
(2018) and Zhao et al.
(2018b) conﬁne the distribution of the latents of the adversarial examples to a Gaussian.
Moreover, the search for adversarial examples is customarily restricted to uniformly-bounded regions or conducted along suboptimal gradient directions (Szegedy et al., 2014; Kurakin et al., 2016; Goodfellow et al., 2014b).
In this study, we introduce a method to address the limitations of previous approaches by constructing adversarial examples that explicitly preserve the semantics of the inputs.
We achieve this by characterizing and aligning the low dimensional geometric summaries of the inputs and the adversarial examples.
The summaries capture the semantics of the inputs and the adversarial examples.
The alignment ensures that the adversarial examples reﬂect the unbiased semantics of the inputs.
We decompose our attack mechanism into: (i.) manifold learning , (ii.)
perturbation invariance , and (iii.)
adversarial attack .
The motivating principle behind step (i.) is to learn the low dimensional geometric summaries of the inputs via statistical inference.
Thus, we present a variational inference technique that relaxes the rigid Gaussian prior assumption typically placed on VAEs encoder networks (Kingma & Welling, 2014) to capture faithfully such summaries.
In step (ii.), we develop an approach around the manifold invariance concept of (Roussel, 2019) to perturb the elements of the learned manifold while ensuring the perturbed elements remain within the manifold.
Finally, in step (iii.), we propose a learning algorithm whereby we leverage the rich semantics of the inputs and the perturbations as a source of knowledge upon which we impose adversarial constraints to produce adversarial examples.
Unlike (Song et al., 2018; Carlini & Wagner, 2016; Zhao et al., 2018b; Goodfellow et al., 2014b) that resort to a costly search of adversarial examples, our algorithm is efﬁcient and end-to-end.
The main contributions of our work are thus: (i.) a variational inference method for manifold learning in the presence of continuous latent variables with minimal assumptions about their distribution, (ii.)
an intuitive perturbation strategy that encourages perturbed elements of a manifold to remain within the manifold, (iii.)
an end-to-end and computationally efﬁcient algorithm that combines (i.) and (ii.) to generate adversarial examples in a black-box setting, and (iv.) illustration on toy data, images and text, as well as empirical validation against strong certiﬁed and non-certiﬁed adversarial defenses.
P RELIMINARIES & A RCHITECTURE
Notations.
Let x be a sample from the input space X , with label y from a set of possible labels
Y , and D = { x n } Nn =1 a set of N such samples x .
Also, let d be a distance measure on X capturing closeness in input space, or on Z , the embedding space of X , capturing semantics similarity.
Adversarial Examples.
Given a classiﬁer g , and its loss function (cid:96) , an adversarial example of x is produced by maximizing the objective below over an (cid:15) -radius ball around x (Athalye et al., 2017).
[Equation]
such that
[Equation]
Above, the search region for adversarial examples is conﬁned to a uniformly-bounded ball B ( x ; (cid:15) ) .
In reality, however, the shape imposed on B is quite restrictive as the optimal search region may have a different topology.
It is also common practice to produce adversarial examples in the input space
X — via an exhaustive and costly search procedure (Shaham et al., 2018; Song et al., 2018; Zhao et al., 2018b; Athalye et al., 2017; Carlini & Wagner, 2016; Goodfellow et al., 2014b).
Unlike these approaches, however, we wish to operate in Z , the lower dimensional embedding space of X , with minimal computational overhead.
Our primary intuition is that Z captures well the semantics of D .
[Equation]
[Figure]
[Equation]
[Figure]
Attack Model.
Given a sample x ∈ D and its class y ∈ Y , we want to construct an adversarial example x (cid:48) that shares the same semantics as x .
We assume the semantics of x (resp.
x (cid:48) ) is modeled by a learned latent variable model p ( z ) (resp.
p (cid:48)
( z (cid:48) ) ), where z, z
(cid:48) ∈ Z .
In this setting, observing x (resp.
x (cid:48) ) is conditioned on the observation model p ( x | z ) (resp.
p ( x (cid:48) | z (cid:48) ) ), that is: x ∼ p ( x | z ) and x (cid:48)
∼ p
( x (cid:48) | z (cid:48) ) , with z ∼ p ( z ) and z (cid:48) ∼ p (cid:48)
( z (cid:48) ) .
We learn this model in a way that d ( x, x (cid:48) ) is small and g ( x ) =
y ∧ g ( x (cid:48) )
(cid:54) =
y while ensuring also that d ( z, z (cid:48) ) is small.
Intuitively, we get the latent z ∼ p ( z ) which encodes the semantics of x .
Then, we perturb z in a way that its perturbed version z (cid:48) ∼ p (cid:48)
( z (cid:48) ) lies in the manifold that supports p ( z ) .
We deﬁne a manifold as a set of points in Z where every point is locally Euclidean (Roussel, 2019).
We devise our perturbation procedure by generalizing the manifold invariance concept of (Roussel, 2019) to Z .
For that, we consider two embedding maps h : X → Z and h (cid:48) :
X → Z , parameterized by θ and θ (cid:48) , as surrogates for p ( z ) and p ( z (cid:48) ) .
We assume θ and θ (cid:48) follow the implicit distributions p ( θ ) and p ( θ (cid:48) ) .
In the following, we consider M such embedding maps h and h (cid:48) .
1 If we let z
= h ( x ; θ ) and z (cid:48)
=
h (cid:48)
( x ; θ (cid:48) ) , we ensure that z (cid:48) is in the manifold that supports p ( z ) by constraining d ( z, z (cid:48) ) to be small.
Then, given a map dec φ : Z → X , we craft x (cid:48)
=
dec φ ( z (cid:48) ) in a way that d ( x, x (cid:48) ) is small and g ( x ) =
y ∧ g ( x (cid:48) )
(cid:54) =
y .
Then, we say that x (cid:48) is adversarial to x and preserves its semantics.
Model Architecture.
To implement our attack model, we propose as a framework the architecture illustrated in Figure 1.
Our framework is essentially a variational auto-encoder with two encoders E and E’ that learn the geometric summaries of D via statistical inference.
We present two inference mechanisms — implicit manifold learning via Stein variational gradient descent (Liu & Wang, 2016) and Gram-Schmidt basis sign method (Dukes, 2014) — to draw instances of model parameters from the implicit distributions p ( θ ) and p ( θ (cid:48) ) that we parameterize E and E’ with.
Both encoders optimize the uncertainty inherent to embedding D in Z while guaranteeing easy sampling via Bayesian ensembling.
Finally, the decoder p φ acts as a generative model for constructing adversarial examples.
Threat Model.
We consider in this paper a black-box scenario where we, as an attacker, have only access to the predictions of a classiﬁer g .
As the attacker, we want to construct adversarial examples not knowing the intricacies of g such as its loss function, nor having access to its gradient.
We focus on this scenario because it is challenging and more plausible in real-life than the white-box case.
This threat model serves to evaluate both certiﬁed defenses and non-certiﬁed ones under our attack model
I MPLICIT M ANIFOLD L EARNING
Manifold learning is based on the assumption that high dimensional data lies on or near lower dimensional manifolds in a data embedding space.
In the variational auto-encoder (VAE) (Kingma &
Welling, 2014) setting, the datapoints x n ∈ D are modeled via a decoder x n ∼ p ( x n | z n ; φ ) .
To learn the parameters φ , one typically maximizes a variational approximation to the empirical expected log-likelihood 1 /N (cid:80) Nn =1 log p ( x n ; φ ) , called evidence lower bound (ELBO), deﬁned as:
[Equation]
The expectation E z | x ; ψ can be re-expressed as a sum of a reconstruction loss, or expected negative log-likelihood of x , and a KL ( q ( z | x ; ψ ) (cid:107) p ( z )) term.
The KL term acts as a regularizer and forces the encoder q ( z | x ; ψ ) to follow a distribution similar to p ( z ) .
In VAEs, p ( z ) is deﬁned as a spherical
Gaussian .
That is, VAEs learn an encoding function that maps the data manifold to an isotropic Gaussian.
However, Jimenez Rezende & Mohamed (2015) have shown that the Gaussian form imposed on p ( z ) leads to uninformative latent codes; hence to poorly learning the semantics of D (Zhao et al., 2017).
To sidestep this issue, we minimize the divergence term KL ( q ( z | x ; ψ ) (cid:107) p ( z | x ; φ )) using Stein
Variational Gradient Descent (Liu & Wang, 2016) instead of explicitly optimizing the ELBO.
Stein Variational Gradient Descent (SVGD) is a nonparametric variational inference method that combines the advantages of MCMC sampling and variational inference.
Unlike ELBO (Kingma & Welling, 2014), SVGD does not conﬁne a target distribution p ( z ) it approximates to simple or tractable parametric distributions.
It remains yet an efﬁcient algorithm.
To approximate p ( z ) , SVGD maintains M particles z
= { z m } Mm =1 , initially sampled from a simple distribution, it iteratively transports via functional gradient descent.
At iteration t , each particle z t ∈ z t is updated as follows:
[Equation]
where α t is a step-size and k ( ., . ) is a positive-deﬁnite kernel.
In the equation above, each particle determines its update direction by consulting with other particles and asking their gradients.
The importance of the latter particles is weighted according to the distance measure k ( ., . ) .
Closer particles are given higher consideration than those lying further away.
The term ∇ z
[Equation]
k ( z m , z ) is a regularizer that acts as a repulsive force between the particles to prevent them from collapsing into one particle.
Upon convergence, the particles z m will be unbiased samples of the true implicit distribution p ( z ) .
Manifold Learning via SVGD.
To characterize the manifold of D , which we denote M , we learn the encoding function q ( . ; ψ ) .
Similar to Pu et al.
(2017), we optimize the divergence KL ( q ( z | x ; ψ ) (cid:107) p ( z | x ; φ )) using SVGD.
As an MCMC method, SVGD, however, induces inherent uncertainty we ought to capture in order to learn M efﬁciently.
To potentially capture such uncertainty, Pu et al.
(2017) use dropout.
However, according to Hron et al.
(2017), dropout is not principled.
Bayesian methods, on the contrary, provide a principled way to model uncertainty through the posterior distribution over model parameters.
Kim et al.
(2018) have shown that SVGD can be cast as a Bayesian approach for parameter estimation and uncertainty quantiﬁcation.
Since SVGD always maintains M particles, we introduce thus M instances of model parameters
Θ = { θ m } Mm =1 , where every θ m ∈ Θ is a particle that deﬁnes the weights and biases of a Bayesian neural network.
For large M , however, maintaining Θ can be computationally prohibitive because of the memory footprint.
Furthermore, the need to generate the particles during inference for each test case is undesirable.
To sidestep these issues, we maintain only one (recognition) network f η that takes as f
[Equation]
z ξ x p
[Equation]
x     ˜ z θ
Algorithm
Require: Input x ∈ D
Require: Model parameters η 1: Sample ξ ∼ N ( 0 , I ) 2:
Sample θ ∼ f η ( ξ ) 3:
Given x , sample z ∼ p ( z | x ; θ ) 4: Sample ˜ x ∼ p ( x | z, φ ) 5: Sample ˜ z ∼ p ( z | ˜ x, θ ) 6: Use x and ˜ z to compute p (˜ z | x ; θ )
[Equation]
−
[Equation]
with θ t +1 m ← θ tm + α t τ ( θ tm ) , where τ ( θ t )
= 1 M
[Equation]
We use the notation SVGD τ (Θ ) to denote an SVGD update of Θ using the operator τ ( . ) .
As the particles θ are Bayesian, upon observing D , we update the prior p ( θ tj ) to obtain the posterior p ( θ tj |D )
∝ p ( D| θ tj ) p ( θ tj ) which captures the uncertainty.
We refer the reader to Appendix A for a formulation of p ( θ tj |D ) and p ( D| θ tj ) .
The data likelihood p ( D| θ tj ) is evaluated over all pairs ( x, ˜ z ) where x ∈ D and ˜ z is a dependent variable.
However, ˜ z is not given.
Thus, we introduce the inversion process described in Figure 2 to generate such ˜ z using Algorithm 1.
For any input x ∈ D , we sample its latent code z from p ( z | x ; D ) , which we approximate by Monte Carlo over Θ ; that is:
[Equation]
θ
[Equation] [Equation]
θ m ∼ p ( θ |D ) .
[Equation]
P ERTURBATION I NVARIANCE
Here, we focus on perturbing the elements of M .
We want the perturbed elements to reside in M and exhibit the semantics of D that M captures.
Formally, we seek a linear mapping h (cid:48) :
M → M such that for any point z ∈ M , a neighborhood U of z is invariant under h (cid:48) ; that is: z (cid:48) ∈
U ⇒ h (cid:48)
( z (cid:48) )
∈ U .
In this case, we say that M is preserved under h (cid:48) .
Trivial examples of such mappings are linear combinations of the basis vectors of subspaces S of M called linear spans of S .
Rather than ﬁnding a linear span h (cid:48) directly, we introduce a new set of instances of model parameters Θ (cid:48)
= { θ (cid:48) m } Mm =1 .
Each θ (cid:48) m denotes the weights and biases of a Bayesian neural network.
Then, for any input x ∈ D and its latent code z ∼ p ( z | x ; D ) , a point in M , we set h (cid:48)
( z ) =
z (cid:48) where z (cid:48) ∼ p
( z (cid:48) | x ; D ) .
We approximate p ( z (cid:48) | x ; D ) by Monte Carlo using Θ (cid:48) , as in Equation 3.
We leverage the local smoothness of M to learn each θ (cid:48) m in a way to encourage z (cid:48) to reside in M in a close neighborhood of z using a technique called Gram-Schmidt Basis Sign Method.
Gram-Schmidt Basis Sign Method (GBSM).
Let X be a batch of samples of D , Z m a set of latent codes z m ∼ p ( z | x ; θ m ) where x ∈ X , and θ m ∈ Θ .
For any m ∈ { 1 .., M } , we learn θ (cid:48)
m to generate perturbed versions of z m ∈
Z m along the directions of an orthonormal basis U m .
As M is locally
Euclidean, we compute the dimensions of the subspace Z m by applying Gram-Schmidt (Dukes, 2014) to orthogonalize the span of representative local points.
We formalize GBSM as follows:
[Equation]
The intuition behind GBSM is to utilize the fact that topological spaces are closed under their basis vectors to render M invariant to the perturbations δ m .
To elaborate more on GBSM, we ﬁrst sample a model instance θ (cid:48) m .
Then, we generate z (cid:48)
m ∼ p
( z (cid:48) | x ; θ (cid:48) m ) for all x ∈ X .
We orthogonalize Z m and ﬁnd the perturbations δ m that minimizes (cid:37) along the directions of the basis vectors u im ∈
U m .
We want the perturbations δ m to be small.
With δ m ﬁxed, we update θ (cid:48) m by minimizing (cid:37) again.
We use the notation GBSM (Θ (cid:48) , ∆) where ∆ = { δ m } Mm =1 to denote one update of Θ (cid:48) via GBSM.
Manifold Alignment.
Although GBSM confers us latent noise imperceptibility and sampling speed, Θ (cid:48) may deviate from Θ ; in which case the manifolds they learn will mis-align.
To mitigate this issue, we regularize each θ (cid:48) m ∈ Θ (cid:48) after every GBSM update.
In essence, we apply one SVGD update on Θ (cid:48) to ensure that Θ (cid:48) follows the transform maps constructed by the particles Θ (Han & Liu, 2017).
θ (cid:48) t +1 ← θ (cid:48) t
+ α t π ( θ (cid:48) t ) where π ( θ (cid:48) t )
= 1 M
[Equation]
We use the notation SVGD π (Θ (cid:48) ) to refer to the gradient update rule in Equation 4.
In this rule, the model instances Θ (cid:48) determine their own update direction by consulting only the particles Θ instead of consulting each other.
Maintaining Θ (cid:48)
= { θ (cid:48) m } Mm =1 for large M is, however, computationally prohibitive.
Thus, as in Section 3, we keep only one (recognition) network f η (cid:48) that takes as input ξ (cid:48) m ∼ N ( 0 , I ) and outputs θ (cid:48) m ∼ f ( ξ (cid:48) m ; η (cid:48) ) .
Here too we reﬁne η (cid:48) through a small number of gradient steps to learn the trajectories that Θ (cid:48) follows as it gets updated via GBSM and SVGD π .
[Equation]
−
[Equation]
(cid:48)
[Equation]
(cid:13)(cid:13)(cid:13)where θ (cid:48)
t +1 m ← θ (cid:48) tm + α t π ( θ (cid:48) tm ) .
G ENERATING A DVERSARIAL E XAMPLES
In this paper, a black-box scenario is considered.
In this scenario, we have only access to the predictions of the classiﬁer g .
We produce adversarial examples by optimizing the loss below.
The ﬁrst term is the reconstruction loss inherent to VAEs.
This loss accounts here for the dissimilarity between any input x ∈ D and its adversarial counterpart x (cid:48) , and is constrained to be smaller than (cid:15) attack so that x (cid:48) resides within an (cid:15) attack -radius ball of x .
Unless otherwise speciﬁed, we shall use norm L 2 as reconstruction loss.
The second term is an auxiliary log-likelihood loss (for g) of a target class y (cid:48)
∈
Y \ { y } where y is the class of x .
This loss deﬁnes the cost incurred for failing to fool g .
L
[Equation]
=
[Equation]
+ min y
[Equation]
∈Y (cid:20)
[Equation]
= y
[Equation]
log (1 − P ( y (cid:48)
[Equation]
x (cid:48)
[Equation]
) (cid:21) such that (cid:107) x
[Equation]
x (cid:48) (cid:107)
[Equation]
(cid:15) attack .
In Algorithm 2, we show how we unify our manifold learning and perturbation strategy into one learning procedure to generate adversarial examples without resorting to an exhaustive search.
R ELATED W ORK
Manifold Learning.
VAEs are generally used to learn manifolds (Yu et al., 2018; Falorsi et al., 2018; Higgins et al., 2016) by maximizing the ELBO of the data log-likelihood (Alemi et al., 2017; Chen et al., 2017).
Optimizing the ELBO entails reparameterizing the encoder to a Gaussian distribution (Kingma & Welling, 2014).
This reparameterization is, however, restrictive (Jimenez
Rezende & Mohamed, 2015) as it may lead to learning poorly the manifold of the data (Zhao et al., 2017).
To alleviate this issue, we use SVGD, similar to Pu et al.
While our approach and that of Pu et al.
(2017) may look similar, ours is more principled.
As discussed in (Hron et al., 2017), dropout which Pu et al.
(2017) use is not Bayesian.
Since our model instances are Bayesian, we are better equipped to capture the uncertainty.
Capturing the uncertainty requires, however, evaluating the data likelihood.
As we are operating in latent space, this raises the interesting challenge of assigning target-dependent variables to the inputs.
We overcome this challenge using our inversion process.
Adversarial Examples.
Studies in adversarial deep learning (Athalye et al., 2018; Kurakin et al., 2016; Goodfellow et al., 2014b; Athalye et al., 2017) can be categorized into two groups.
The ﬁrst group (Carlini & Wagner, 2016; Athalye et al., 2017; Moosavi-Dezfooli et al., 2016) proposes to
[Table] [Table]
Require
[Table]
Number
[Table]
instances
[Table]
Require
[Table]
Require
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
7: for t = 1 to T do 8:
Sample Θ = { θ m } Mm =1 where θ m ∼ f η ( ξ m ) 9:
Sample Θ (cid:48)
= { θ (cid:48) m } M m =1 where θ (cid:48) m ∼ f η (cid:48) ( ξ m ) 10:
Use Θ and Θ (cid:48) in Equation 3 to sample z and z (cid:48) 11:
Sample ˜ x ∼ p ( x | z, φ ) and x (cid:48) ∼ p ( x (cid:48)
| z (cid:48) , φ ) (cid:46) clean and perturbed reconstructions 12: η , η (cid:48) , ∆ ← InnerTraining ( Θ , Θ (cid:48) , η , η (cid:48) , ∆ , ˜ x ) 13: L ˜ x := (cid:107) x − ˜ x (cid:107) 2 ;
L x (cid:48) :
= (cid:107) x − x (cid:48)
(cid:107)(cid:46) reconstruction losses on ˜ x and x (cid:48) 14: L x (cid:48) :
= (cid:40)
L x (cid:48) , if L x (cid:48) > (cid:15) attack
L x (cid:48)
+ min y (cid:48) ∈Y (cid:2)
y = y (cid:48) · log (1 − P ( y (cid:48)
| x (cid:48) )) (cid:3) , otherwise 15: η ← η − α ∇ η L ˜ x ; η (cid:48) ← η (cid:48) − α (cid:48)
∇ η (cid:48)
L x (cid:48) (cid:46)
SGD update using Adam optimizer 16: φ ← φ − (cid:15) ∇ φ ( L ˜ x
+ L x (cid:48) )
(cid:46) SGD update using Adam optimizer generate adversarial examples directly in the input space of the original data by distorting, occluding or changing illumination in images to cause changes in classiﬁcation.
The second group (Song et al., 2018; Zhao et al., 2018b), where our work belongs, uses generative models to search for adversarial examples in the dense and continuous representations of the data rather than in its input space.
Adversarial Images.
Song et al.
(2018) propose to construct unrestricted adversarial examples in the image domain by training a conditional GAN that constrains the search region for a latent code z (cid:48) in the neighborhood of a target z .
Zhao et al.
(2018b) use also a GAN to map input images to a latent space where they conduct their search for adversarial examples.
These studies are the closest to ours.
Unlike in (Song et al., 2018) and (Zhao et al., 2018b), however, our adversarial perturbations are learned, and we do not constrain the search for adversarial examples to uniformly-bounded regions.
In stark contrast to Song et al.
(2018) and Zhao et al.
(2018b) approaches also, where the search for adversarial examples is exhaustive and decoupled from the training of the GANs, our approach is efﬁcient and end-to-end.
Lastly, by capturing the uncertainty induced by embedding the data, we characterize the semantics of the data better, allowing us thus to generate sound adversarial examples.
Adversarial Text.
Previous studies on adversarial text generation (Zhao et al., 2018a; Jia & Liang, 2017; Alvarez-Melis & Jaakkola, 2017; Li et al., 2016) perform word erasures and replacements directly in the input space using domain-speciﬁc rules or heuristics, or they require manual curation.
Similar to us, Zhao et al.
(2018b) propose to search for textual adversarial examples in the latent representation of the data.
However, in addition to the differences aforementioned for images, the search for adversarial examples is handled more gracefully in our case thanks to an efﬁcient gradient-based optimization method in lieu of a computationally expensive search in the latent space.
E XPERIMENTS & R ESULTS
Before, we presented an attack model whereby we align the semantics of the inputs with their adversarial counterparts.
As a reminder, our attack model is black-box , restricted and non-targeted .
Our adversarial examples reside within an (cid:15) attack − radius ball of the inputs as our reconstruction loss, which measures the amount of changes in the inputs, is bounded by (cid:15) attack (see Equation 6).
We validate the adversarial examples we produce based on three evaluation criteria: (i.) manifold preservation , (ii.)
adversarial strength , and (iii.) soundness via manual evaluation.
We provide in
Appendix A examples of the adversarial images and sentences that we construct.
7. M ANIFOLD P RESERVATION
We experiment with a 3D non-linear Swiss Roll dataset which comprises 1600 datapoints grouped in 4 classes.
We show in Figure 3, on the left, the 2D plots of the manifold we learn.
In the middle, we plot the manifold and its elements that we perturbed and whose reconstructions are adversarial.
On the right, we show the manifold overlaid with the latent codes of the adversarial examples produced by PGD (Goodfellow et al., 2014b) with (cid:15) attack ≤ 0 .
Observe in Figure 3, in the middle, how the latent codes of our adversarial examples espouse the Swiss Roll manifold, unlike the plot on the right.
A DVERSARIAL S TRENGTH
Setup.
As argued in (Athalye et al., 2018), the strongest non-certiﬁed defense against adversarial attacks is adversarial training with Projected Gradient Descent (PGD)
(Goodfellow et al., 2014b).
Thus, we evaluate the strength of our MNIST, CelebA and SVHN adversarial examples against adversarially trained ResNets (He et al., 2015) with a 40-step PGD and noise margin (cid:15) attack ≤ 0 . 3 .
The ResNet models follow the architecture design of Song et al.
Similar to Song et al.
(2018) — whose attack model resembles ours 2 —, for MNIST, we also target the certiﬁed defenses (Raghunathan et al., 2018; Kolter & Wong, 2017) with (cid:15) attack = 0 .
1 using norm L ∞ as reconstruction loss.
For all the datasets, the accuracies of the models we target are higher than 96.3%.
Next, we present our attack success rates and give examples of our adversarial images in Figure 4.
Attack Success Rate (ASR) is the percentage of examples misclassiﬁed by the adversarially trained
Resnet models.
For (cid:15) attack = 0 . 3 , the publicly known ASR of PGD attacks on MNIST is 88.79%.
However, our ASR for MNIST is 97.2%, higher than PGD.
Also, with (cid:15) attack ≈ 1 . 2 , using norm L ∞ as reconstruction loss in Equation 6, we achieve an ASR of 97.6% against (Kolter & Wong, 2017) where PGD achieves 91.6%.
Finally, we achieve an ASR of 87.6% for SVHN, and 84.4% for CelebA.
7.2. A DVERSARIAL T EXT
Datasets.
For text, we consider the SNLI (Bowman et al., 2015) dataset.
SNLI consists of sentence pairs where each pair contains a premise ( P ) and a hypothesis ( H ), and a label indicating the relationship ( entailment, neutral, contradiction ) between the premise and hypothesis.
For instance, the following pair is assigned the label entailment to indicate that the premise entails the hypothesis.
Premise: A soccer game with multiple males playing.
Hypothesis: Some men are playing a sport.
[Table] [Table]
[Figure]
Setup.
We perturb the hypotheses while keeping the premises unchanged.
Similar to Zhao et al
(2018b), we generate adversarial text at word level using a vocabulary of 11,000 words.
We also use
ARAE (Zhao et al., 2018a) for word embedding, and a CNN for sentence embedding.
To generate perturbed hypotheses, we experiment with three decoders: (i.) p φ is a transpose CNN, (ii.)
p φ is a language model, and (iii.)
we use the decoder of a pre-trained ARAE (Zhao et al., 2018a) model.
We detail their conﬁgurations in Appendix B.
The transpose CNN generates more meaningful hypotheses than the language model and the pretrained ARAE model although we notice sometimes changes in the meaning of the original hypotheses.
We discuss these limitations more in details in Appendix A. Henceforward, we use the transpose CNN to generate perturbed hypotheses.
See Table 1 for examples of generated hypotheses.
Attack Success Rate (ASR).
We attack an SNLI classiﬁer that has a test accuracy of 89.42%.
Given a pair ( P, H ) with label l , its perturbed version ( P,
H’ ) is adversarial if the classiﬁer assigns the label l to ( P, H ), ( P, H’ ) is manually found to retain the label of ( P, H ), and such label differs from the one the classiﬁer assigns to ( P, H’ ).
To compute the
ASR, we run a pilot study which we detail next.
7. M ANUAL E VALUATION
To validate our adversarial examples and assess their soundness vs. Song et al.
(2018), Zhao et al.
(2018b) and PGD (Madry et al., 2017) adversarial examples, we carry out a pilot study whereby we ask three yes-or-no questions: (Q1) are the adversarial examples semantically sound? , (Q2) are the true inputs similar perceptually or in meaning to their adversarial counterparts?
and (Q3) are there any interpretable visual cues in the adversarial images that support their misclassiﬁcation?
Pilot Study I.
For MNIST, we pick 50 images (5 for each digit), generate their clean reconstructions, and their adversarial examples against a 40-step PGD ResNet with (cid:15) attack ≤ 0 .
We target also the certiﬁed defenses of Raghunathan et al.
and Kolter & Wong (2017) with (cid:15) attack = 0 .
For
SVHN, we carry out a similar pilot study and attack a 40-step PGD ResNet.
For CelebA, we pickimages (25 for each gender), generate adversarial examples against a 40-step PGD ResNet.
For all three datasets, we hand the images and the questionnaire to 10 human subjects for manual evaluation.
We report in Table 2 the results for MNIST and, in Table 4, the results for CelebA and SVHN.
We hand the same questionnaire to the subjects with 50 MNIST images, their clean reconstructions, and the adversarial examples we craft with our method.
We also handed the adversarial examples generated using Song et al.
, Zhao et al.
(2018b) and PGD methods.
We ask the subjects to assess the soundness of the adversarial examples based on the semantic features (e.g., shape, distortion, contours, class ) of the real MNIST images.
We report the evaluation results in Table 3.
[Table] [Table]
Pilot Study II - SNLI.
Using the transpose CNN as decoder p φ , we generate adversarial hypotheses for the SNLI sentence pairs with the premises kept unchanged.
Then, we select manually 20 pairs of clean sentences (premise, hypothesis), and adversarial hypotheses.
We also pick 20 pairs of sentences and adversarial hypotheses generated this time using Zhao et al.
(2018b)’s method against their treeLSTM classiﬁer.
We choose this classiﬁer as its accuracy (89.04%) is close to ours (89.42%).
We carry out a pilot study where we ask two yes-or-no questions: (Q1) are the adversarial samples semantically sound? and (Q2) are they similar to the true inputs?
We report the results in Table 4.
[Table]
Takeaways.
As reﬂected in the pilot study and the attack success rates, we achieve good results in the image and text classiﬁcation tasks.
In the image classiﬁcation task, we achieve better results than
PGD and Song et al.
(2018) both against the certiﬁed and non-certiﬁed defenses.
The other takeaway is: although the targeted certiﬁed defenses are resilient to adversarial examples crafted in the input space, we can achieve manual success rates higher than the certiﬁed rates when the examples are constructed in the latent space, and the search region is unrestricted.
In text classiﬁcation, we achieve better results than Zhao et al.
(2018b) when using their treeLSTM classiﬁer as target model.
C ONCLUSION
Many approaches in adversarial attacks fail to enforce the semantic relatedness that ought to exist between original inputs and their adversarial counterparts.
Motivated by this fact, we developed a method tailored to ensuring that the original inputs and their adversarial examples exhibit similar semantics by conducting the search for adversarial examples in the manifold of the inputs.
Our success rates against certiﬁed and non-certiﬁed defenses known to be resilient to traditional adversarial attacks illustrate the effectiveness of our method in generating sound and strong adversarial examples.
Although in the text classiﬁcation task we achieved good results and generated informative adversarial sentences, as the transpose CNN gets more accurate — recall that it is partly trained to minimize a reconstruction error —, generating adversarial sentences that are different from the input sentences and yet preserve their semantic meaning becomes more challenging.
In the future, we intend to build upon the recent advances in text understanding to improve our text generation process.
A PPENDIX A: D ISCUSSION & A DVERSARIAL EXAMPLES
Posterior Formulation.
Similar to (Kim et al., 2018), we formalize p ( θ |D ) for every θ ∈ Θ as: p ( θ |D )
∝ p ( D| θ ) p ( θ ) =
(cid:89) ( x, ˜ z ) p (˜ z
| x ; θ ) p ( θ ) where x ∈ D and ˜ z is generated using Algorithm= (cid:89)
[Equation]
For every θ (cid:48) ∈ Θ (cid:48) , we compute p ( θ (cid:48) |D ) the same way.
Note that θ (resp.
θ (cid:48) ) consists in fact of network parameters W ∼ f η (resp.
W (cid:48) ∼ f η (cid:48) ) and scaling parameters γ and λ .
For notational simplicity, we used before the shorthands θ ∼ f η and θ (cid:48) ∼ f η (cid:48) .
The parameters γ and λ are initially sampled from a Gamma distribution and updated as part of the learning process.
In our experiments, we set the hyper-parameters of the Gamma distributions a and b to 1 . 0 and 0 .
1 , and a (cid:48) and b (cid:48) to 1 . 0 .
Latent Noise Level.
We measure the amount of noise ∆ we inject into the latent codes of our inputs by computing the average spectral norm of the latent codes of their adversarial counterparts.
The input changes are captured by our reconstruction loss which is bounded by (cid:15) attack (see Equation 6).
For
MNIST, CelebA, and SVHN, the noise levels are 0 .
004 ± 0 . 0003 , 0 .
026 ± 0 .
005 , and 0 .
033 ± 0 .
The takeaways are: (i.) they are imperceptible, and (ii.)
the distributions that Θ and Θ (cid:48) follow are similar.
To validate (ii.), we compute the marginals of clean and perturbed latent codes randomly sampled from Θ and Θ (cid:48) .
As shown in Figure 5, the marginal distributions overlap relatively well.
[Figure]
Discussion.
We discuss the choices pertaining to the design of our approach and their limitations.
We discuss also the evaluation process of our approach against (Song et al., 2018; Zhao et al., 2018b).
Space/Time Complexity.
As noted in (Jimenez Rezende & Mohamed, 2015), the Gaussian prior assumption in VAEs might be too restrictive to generate meaningful enough latent codes (Zhao et al., 2017).
To relax this assumption and produce informative and diverse latent codes, we used
SVGD.
To generate manifold preserving adversarial examples, we proposed GBSM.
Both SVGD and
GBSM maintain a set of M model instances.
As ensemble methods, both inherit the shortcomings of ensemble models most notably in space/time complexity.
Thus, instead of maintaining 2 ∗ M model instances, we maintain only f η and f η (cid:48) from which we sample these model instances.
We experimented with M set to 2 , 5 , 10 and 15 .
As M increases, we notice some increase in sample quality at the expense of longer runtimes.
The overhead that occurs as M takes on larger values reduces, however, drastically during inference as we need only f η (cid:48) to sample the model instances θ (cid:48) m ∈ Θ (cid:48) in order to construct adversarial examples.
One way to alleviate the overhead during training is to enforce weight-sharing for θ m ∈ Θ and θ (cid:48)
m ∈ Θ (cid:48) .
However, we did not try this out.
Preserving Textual Meaning.
To construct adversarial text, we experimented with three architecture designs for the decoder p φ : (i.) a transpose CNN, (ii.)
a language model, and (iii.)
the decoder of a pre-trained ARAE model (Zhao et al., 2018a).
The transpose CNN generates more legible text than the other two designs although we notice sometimes some changes in meaning in the generated adversarial examples.
Adversarial text generation is challenging in that small perturbations in the latent codes can go unnoticed at generation whereas high noise levels can render the outputs nonsensical.
To produce adversarial sentences that faithfully preserve the meaning of the inputs, we need good sentence generators, like GPT (Radford, 2018), trained on large corpora.
Training such large language models requires however time and resources.
Furthermore, in our experiments, we considered only a vocabulary of size 10,000 words and sentences of length no more than 10 words to align our evaluation with the experimental choices of (Zhao et al., 2018b).
Measuring Perceptual Quality is desirable when the method relied upon to generate adversarial examples uses GANs or VAEs; both known to produce often samples of limited quality.
As Song et al.
(2018) perform unrestricted targeted attacks — their adversarial examples might totally differ from the true inputs — and Zhao et al.
(2018b) do not target certiﬁed defenses, a fair side-by-side comparison of our results and theirs using metrics like mutual information or frechet inception distance , seems unachievable.
Thus, to measure the quality of our adversarial examples and compare our results with (Song et al., 2018) and (Zhao et al., 2018b), we carried out the pilot study.
[Equation]
[Table]
A DVERSARIAL I MAGES : SVHN
Here, we provide few random samples of non-targeted adversarial examples we generate with our approach on the SVHN dataset as well as the clean reconstructions.
[Table]
A DVERSARIAL I MAGES : MNIST
Here, we provide few random samples of non-targeted adversarial examples we generate with our approach on the MNIST dataset as well as the clean reconstructions.
Both the reconstructed and the adversarial images look realistic although we notice some artifacts on the latter.
Basic
Iterative Methods (Kurakin et al., 2016), among others, also suffer from this.
Because the marginal distributions of the latent codes of the inputs and their perturbed versions overlap, we conclude that indeed the adversarial images preserve the manifold of the inputs (see Figure 5).
[Table] [Table]
[Table] [Table] [Table]
[Table]
A PPENDIX B: E XPERIMENTAL S ETTINGS
[Table]