Class-Disentanglement and Applications in Adversarial Detection and Defense
Authors: Kaiwen Yang
Tianyi Zhou
Yonggang Zhang
Xinmei Tian
Dacheng Tao
Venue: NeurIPS 2021 Poster (2021)
Keywords: class disentanglement
interpretable machine learning
adversarial detection

Abstract: What is the minimum necessary information required by a neural net $D(\cdot)$ from an image $x$ to accurately predict its class? Extracting such information in the input space from $x$ can allocate the areas $D(\cdot)$ mainly attending to and shed novel insights to the detection and defense of adversarial attacks. In this paper, we propose ''class-disentanglement'' that trains a variational autoencoder $G(\cdot)$ to extract this class-dependent information as $x - G(x)$ via a trade-off between reconstructing $x$ by $G(x)$ and classifying $x$ by $D(x-G(x))$, where the former competes with the latter in decomposing $x$ so the latter retains only necessary information for classification in $x-G(x)$. We apply it to both clean images and their adversarial images and discover that the perturbations generated by adversarial attacks mainly lie in the class-dependent part $x-G(x)$. The decomposition results also provide novel interpretations to classification and attack models. Inspired by these observations, we propose to conduct adversarial detection and adversarial defense respectively on $x - G(x)$ and $G(x)$, which consistently outperform the results on the original $x$. In experiments, this simple approach substantially improves the detection and defense against different types of adversarial attacks.

Class-Disentaglleennt and Applications in Adversarial Detection and Defense
Kaiwen Yang', Tianyi Zhou? Yonggang Zhang', Xinmei Tian', Dacheng Tao University of Science and Technology of China'; University of Washington, Seattle? University of Maryland, College Park"; JD Explore Academy (kwyang, yonggang @mail. uste. edu. cn, xi tianyizh@uw, edu, dacheng tao@gmaill com
edu en,
Abstract
What is the minimum necessary information required by neural net D(-) from an image x to accurately predict its class? Extracting such information in the input space from can allocate the areas D(-) mainly attending and shed novel insights to the detection and defense of adversarial attacks. In this paper, we propose "clas--iienttanllemen"" that trains variational autoencoder G(-) to extract this class-dependent information as x - G(x) via a trade-off between reconstructing x by G(xx) and classifying æ by D(x G(z)), where the former competes with the latter in decomposing x so the latter retains only necessary information for classification in G(x). We apply to both clean images and their adversarial images and discover that the perturbations generated by adversarial attacks mainly lie in the class -las-deepnddnnt part x G(z). The decomposition results also provide novel interpretations to classification and attack models. Inspired by these observations, we propose conduct adversarial detection and adversarial defense respectively on z G(x)) and G(z), which consistently outperform the results on the original æ. In experiments, this simple approach substantially improves the detection and defense against different types of adversarial attacks. Code is available:
Introduction
Deep learning provides an end-to-end solution to challenging tasks with data of complicated structures and have achieved breakthrough across different domains in recent years. However, what essential information the neural nets extract from the input and mainly rely on in producing predictions for the targeted tasks still remains mysterious. Moreover, the trained models are known to be fragile and sensitive to adversarial perturbations on the input, but how to explain this failure is still an open problem. Given entangled raw features as input, the inference neural nets tend remove rask-redundant information and compress task-essential information to abstract concepts. On the one hand, identifying the task-essential information in the input can allocate the areas that the neural nets mainly attend to and explain the model prediction. On the other hand, this information can also be the main target of adversarial attacks, while the removed task-redundant information may suffer less distortion under attacks and still preserve clues to correct the prediction. Hence, disentanglement of these two types of information in the input (termed "aakk-disentangeemeet") may shed critical insights to address aforementioned problems.
Disentangled representations [3] have been extensively studied in unsupervised learning when training a generative model, where each dimension of the representation controls a single generative factor of data variations and changing does not affect other factors. Probably the most studied disentanglement model is 3-variational autoencoder (VAE) [20], whose objective aims to reconstruct
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

the imput r from the latent representation via p(z|z) while keeping p(z[ær close to the standard Gaussian p(z) (O I), which encourages the disentanglement of latent factors in 2. However, VAE usually produces unrealistic and blurry images comparing to other generative models such as GAN [15]. And encouraging the disentanglement can further weaken the reconstruction performance. In addition, the learned latent factors in are often exceedingly abstract and their correspondence to specific tasks can be obscure. Hence, it is challenging to derive lask-disettanglement of input from the disentangled latent representation.
Information bottleneck (IB) [42, 43, 1] can potentially provide an information -ioormation-theoretii perspective of the disentaglllem problem. IB models the information flow from input to task output y through some latent representation 2. In IB, the ideal only preserves the minimum necessary information to predict y but discards all redundant information in x that cannot further contribute to predicting y. This leads to an optimization principle that maximizes the mutual information V2; and meanwhile keeps [22; z) small. From IB's s perspective, for supervised learning such as classification, maximizing [(2; y) corresponds to minimizing the classification error of using to predict the class y. In unsupervised learning models like VAE, maximizing I (z; y) corresponds to minimizing the reconstruction error (or maximizing the data likelihood). additional constraint is adopted to limit the amount of information transmitted from to z, forming an "information bottleneck" that filters out information redundant to the task. In B- V-VA,, the constraint keeps the posterior p(z|z) sufficiently close to the standard Gaussian. For classification, previous researches [45, 50] argue that the hierarchical structure of neural nets encourages more information compression as going to deeper layers and the deep learning scheme (SGD cross entropy loss) inherently solves an IB problem. However, IB has been mainly studied for latent bottleneck representations [37, 2] but underexplored to disentangle the task-relevant information in the input space. Specifically, it is non-trivial to design an IB constraint in the input space.
both cases, an
LG
Probabilistic Encoder 4 (z/x)
Probabilistic Decoder pa (x(2)
Image Classifier D(x)
D(x G(x))
Lp Classification Loss
X
G(x)
(x))
Training and inference
:Only training stage
Figure 1: Architecture and Training/Inference of class-disentangled variational auto-encoder (CD-VAE).
In this paper, we focus on image classification tasks and study dllas-ddienttanglmment" as a special case of "task- daas--disntaanlemment i.ee, how to decompose an input image into = G(z) Gzx). where G(x)) contains all the information redundant or irrelevant the class y. while - G(z) captures the essential information to predict the class y. To this end, we develop a simple architecture composed of variational autoencoder (VAE) G(-) generating G(rr) from x and classification network D--) that can be applied to both x G(ar) and æ. A class-disentanglemenn model is achieved by jointly training G(-) to reconstruct x and D(-) to classify I G(x), where each task forms G(xz) only competes the essential
an IB constraint for the other in the objective. Therefore, information for classification with G(z) and Gizz) preserves the class-redundant information for better reconstruction.
Unlike disentanglement of latent factors and IB defined in latent space [5, 10, 24]. class- disentanglement performs disentanglement of class-essential and class-redundant information in the more intricate but interpretable input space. Hence, can be applied to analysing the behaviors of neural nets and intrinsic properties of complicated data, shedding novel insights that cannot be revealed by latent factors. For example, G(x)) can highlight the areas an image Jr that neural nets mainly attend to and help explain why correct incorrect predictions are made. In this paper,

---

we mainly explore its upplications on analyzing adversarial examples. In particular, we find that the adversarial perturbations generated by mainstream attack algorithms primarily lie in x G(z),, while ((x) still preserves uncontaminated information useful for classification. Therefore, we propose to detect adversarial attacks on x - G(x)) and defend the attacks by make predictions based on G(x)). In extensive experiments on both CIFAR and ImageNet dataset, these simple strategies significantly im- prove previous approaches directly applied on a and additionally provide novel interpretations to the results, We also present an empirical study of the trade-off between reconstruction and classification when changing their weights in the objective.
2 Related Work
Disentangled Representation Different variants of VAE [5, 10, 20, 24] have been developed in recent years for learning disentangled latent representations. They all rely on unsupervised learning to automatically discover explanatory factors of variation. However, as suggested by the theoretical analysis in [33]. unsupervised disentanglement is fundamentally impossible without proper inductive biases on models and data. Hence, (implicit) supervision is necessary and might provide crucial guidance to disentanglement, as shown in class-iisetaagleement of this paper as well. Inspired by image translation, disentangling the content and style of an image is studied in [32, 4, 14, 22]. These methods leverage the supervised information, i.e., the domains or groups that every sample belongs to. Unlike previous works, distsasdinntnnlemmenn in this paper combines unsupervised learning and supervised learning in one objective to train disentanglement model. These two tasks compete with each other and form IB constraints to each other, which leads to efficient disentanglement of class-essential and class-redundant information. Class-disentanglement of latent representations has been studied in previous works [47, 18], which however fundamentally differ from our input- optimization formulations, model architectures, and/or applications. More
space disentanglement detailed comparison can be found in Sec. of the Appendix. The input-space class-disentane disentangllemet can be explained as a non-trivial extension of Robust PCA [6, 52], which decomposes data matrix into a low-rank part and a sparse part.
Ad Adversarial Detection aims to distinguish adversarial examples from natural examples. Given a neural net trained on a clean dataset for the original task, many existing methods [36, 17, 9] train a binary classification network on top of some intermediate-layer of the given network as the adver- sarial detector. However, as pointed out by [7], these methods may fail when the detector model leaked. Another strategy is motivated by the observation that adversarial examples have very different distribution from natural examples on intermediate-layee features. So a detector can be built upon some statistics of the distribution, e.g., [13] estimates the kernel density of example its proximity to the natural examples' manifold; [30] models the natural examples' distribution by a multivariate Gaussian and the Mahalanobis distance of an example to this Gaussian is used for adversarial detec- tion; [34] introduces local intrinsic dimensionality to describe the distribution of adversarial examples. Most of these methods directly apply detection the input example æ. Instead, we propose to conduct the detection on the x G(x) capturing the essential information of the class. This simple method consistently improves the detection accuracy and is complementary to existing approaches.
Adversarial Defense is usually achieved by adversarial training [35] and its variants [49] that augment the training set with adversarial examples. Another type of adversarial defense trains GAN [41] or an auto-encoder [31 [to remove adversarial perturbations (if any) from a sample before sending it to the classifier. For instance, [41] proposes an adversarial perturbation erasing GAN (APE-GAN) that can modify an adversarial example close to a natural example; [31] proposed a high-level representation guided denoiser (HGD) to remove adversarial noise. In this paper, show that classification on the class-redundan part G(x)) is robust to adversarial attacks. So our method falls into this category. However, it differs from APE-GAN and HGD do not utilize adversarial examples during training.
3 Class-Disentaglleennt
In this section, we first introduce simple VAE-classifier architecture and an associated optimization objective to train it for clas-diisentagllemen. We then conduct an empirical study that applies the glasa-disettanglementt model to both clean images and their adversarial examples. The results show that the adversarial perturbations mainly reside in the class-essential part G(z).. These
3

---

observations motivate us to conduct adversarial detection and defense on G(z)) and G(z), respectively, which are elaborated at the end of this section.
3.1 Class-Dieentaggedd Variational Auto-Encoder
We propose model architecture shown in Fig. to decompose an input image x into part retaining only essential information for classification (i.e., class-essential) and a part covering all the rest redundant information (i.e., class-redundant), Adding the two parts together exactly recovers z. In the proposed dllass-disenttngled variational auto-encoder (CD-VAE)) we employ variataional auto-encoder (VAE) G(-) to capture the class-redundant information G(x) and then send the complementary part R(x) _ x ((x)) to a classification network D(-). They are trained jointly: VAE G(.) aims for reconstructing while classifier (--) aims for predicting the class of from x G(r). Therefore, the reconstruction and classification compete with each other on the input pixels of and thus discourage sending redundant information to each other. From the information bottleneck perspective, each of them places an IB constraint for the information flowing to the other.
In CD-VAE, VAE G(-) is composed of (1) an encoder parameterized by e and producing the posterior (=(2)) of latent factors 2; and (2) an decoder parameterized by 0 and producing the data likelihood (z]z)) Classification network D..) predicts the class probabilities for x based on x G(z) and has parameters w. We jointly train G(-) and D(-) by optimizing the following objective, which combines the objective Lo( 8) of B- VAE and the cross-entrope loss (w( for classification, i.c.
min ((aws)-peacaaa..)) (a..)) [La( 0) + +ID(wW]] 6.0.w
LG(0, 0) (=))) log pp(l))) + [Dkule(()))ll(())) LD(w) = - log D(r G G()))
(2)
(3)
where p(z) = (0, I) is the prior of a that encourages its disentanglement, and DxL(-1) measures the Bellback-Leibles divergence of posterior go from prior p(z). The first term in Lo 0)
is the marginal likelihood of data in the VAB-generation process. It aims for reconstructing x under the IB constraint of the KI.-divergence. In (w), D(x G(x);; [ is the predicted probability for the ground-truth class y. There are two hyperparameters in the CD-VAE objective: 3 controls the inside the 8- 3-VAE, while controls the trade-off between
strength of reconstruction task (by G(x)) and classification task (by D(x G(z))).
Not every pixel in an image is related to the class label y and many previous works [51, 40] imply that a neural net classifier makes predictions by merely attending to sparse regions of the image. However, the rest information outside the scope of classifiers, e.g., illumination, background colors, patterns shared with other classes, objects of other classes, etc., are all critical to the reconstruction of x. Hence, the trade-off between VAE G(.) and classifier D(-) in CD-VAE is efficient for class-disentanglement while the classifier D(.) quickly picks the class-specific regions in a, the class-redundant information more general across classes can be easily captured by G(-)'s bottleneck factors & with higher priority. The optimization of Eq. (1) can converge fast in a few epochs, as shown in Sec. F of the Appendix.
3.2 Class-Disnttaglemennt on Natural Images vs. Adversarial Images: An Empirical Study
CD-VAE provides novel perspectives to understand (1) how a neural net classifier predicts the class of an image and (2) how an adversarial example attacks the classifier. In particular, x - G(x) reveals the regions in an image that classifier mainly attend to, and our empirical study shows that they also the regions that adversarial examples mainly attack. Since adversarial perturbations are optimized and generated in the input space, class disentanglement in the input space is more adapted for their analysis than eepresenttioondiiittaaneceeeee
Given a pre-trained classifier D(.) and a clean image x with label y, an adversarial image x' = x 8 is usually generated by optimizing d to maximize the classification loss within an e-ball around x i.e.
max - log D( + 5)[b,, Se
(4)
'Here D(-) differs from D(-) in CD-VAE and classifier pre- pre-trained on clean data x.
4

---

Original label: Vizsla, Macaw, Airedale, King crab, English springer, Gibbon, Weimaraner, Saluki
Predicted attack: Afghan hound, Lorikeet, Otterhaund, Rock crab, Cocker spaniel, Patas, Chesapeake bay retriever, Basenji
x
(x))
R(x)
8G
6R
Figure 2: Class- Dlass-Disentanglemmet results by CD-VAE restricted ImageNet.
where the popular choices for include 00, 1. This is called "non-targeted" attack since it does not specify target class that the prediction should change to after applying perturbation 8. A widely used algorithm to tackle the above optimization is projected gradient descent (PGD), which iterates between gradient sign ascent of step size a and projection Pllz'-ell, around r for multiple steps, i.c.,
e-ball
s' - Pl (s' + siginn (Var llg D(x)) ((x')lv))
(5)
Although there exist other types of adversarial attacks that can also be analyzed by class- disentanglement, we mainly use PGD attack here and will include more attacks in later experiments.
In our empirical study, we compare the class-disentange disentanglement results of a clean images x and an adversarial image x' generated from it, i.e., G(x), R(x)
G(æ)), G(x'), R(x') x' G(x'). We are particularly interested in (1) which parts of an image suggested by R(x) and R(x') that neural net classifier mainly attends to; and (2) the difference between x and x' on their original pixels, class-essential parts, class-redundant parts and hidden space. Specifically, we investigate
8 r r', dG A G(x) G(æ'), 8n A R(x) R(x'), 82 --zz
(6)
For each dataset from CIFAR-10 and ImageNet, we train CD-VAE model on the (clean) training sct. We then apply the CD-VAE model to both the (clean) test/validation set images and the corresponding adversarial images generated by PGD attack -bal constraint) [35] towards a pre-trained classifier (WideResNet-28 28- 10 [48] for CIFAR-10 10 and ResNet-50 [19] for ImageNet). Fig. 2 presents examples of the ImageNet experiment. Similar results for CIFAR-10 can be found in Appendix.
Consine Similarity
FID
G(z) vs. G(x') R(x) vs. R(x')
0.9997 + 0.0003
0.90
0.9460 0.0260
22.31
Table 1: Cosine similarity and FID.
It shows that the class-essential part R(x)) image but can capture critical features of the class that are sufficient for the classification task, e.g., cars and nose of dog, eyes of parrot, etc. Hence, R(z) can provide natural interpretation
G(æ) only highlight sparse regions of each
5

---

to the prediction made by the neural net classifier, which has complicated structure and is usually treated as a black box. Comparison among 6, őn and òG shows that the adversarial perturbation ó mainly lies in ön and cause slight change KG on G(a)). This indicates that the attacks to mainly reside in R(x') but do not heavily distort G(x)). This observation directly leads to two applications. First, we may better detect adversarial examples in the space of R(ar). In Fig. 2, the sparse regions captured by R(x') largely narrow the search range for the attacked regions and thus might make the detection easier. Secondly, G(x) in Fig. 2 still contains useful information for class prediction it redundant and less discriminative than R(x) but is almost undistorted in G(z'), so classification on ((x') might be more robust to adversarial attacks and provide simple defense strategy.
Our quantitative analysis to 8. òR and êg also support the observa- tions above. In Table 2, we report several statistics of them, c.g., the mean and standard deviation for their lp norms with p 21, so). To further study the similarity of R(x) and R(x') as images, we also report the statistics for their cosine similarity and Fréchet Inception Distance (FID) in Table 1. R(x')
1 10-3
L2
x
142.83 37.02
427.99 92.59
2.49 0.20 2.08 0.11
G(x) R(x)
133.68 + 37.07
92.14 397.77
36.76 11.75
127.95+137.0
2.45
x'
142.74 + 36.57
427.91 + 91.19
2.53 + 0.18
G(w") R(x')
133.52 36.96
397.26 91.85
2.08 + 0.11
38.13 = 11.06
131.20 + 35.37
2.46 + 0.45
8
13.65 + 0.88
39.09 + 1.40
0.14 + 0.00
&G
4.15 = 0.65
16.56 2.47
0.39 0.17
and R(x) are distant from each other while G(x') is close to G(z) in terms of all those distance metrics. This again supports the adversarial detection on R(x) and adversarial defense on G(r). In addition, we caleulate the (Pearson) correlation between and s' on each dimension and report the its histogram and p-value in Fig. 3. shows a strong correlation between & and z', which implies that G(x) is robust to the adversarial perturbations. An extensive quantitative analysis of 6, 8g and 5n with respect to larger is provided in Sec. E of the Appendix.
8R
13.78 + 0.92
40.97 + 1.87
0.48 + 0.16
Table 2: e, norm/distance of different parts CD-VAE (ImageNet).
Test
dataset
R(x)
G(x)
Training
96.01(99.84)
92.68(99.65 65)
18. 18.86(67.93)
CIFAR-10
R(x)
95.81(99.81)
96.20(99.82 82)
18.66.6))
G(x)
51.84(86.52)
25.67(68.98)
75.25(97.39)
X
82.53(96.88) 88)
11.17(25.61)
65.26(88.94) 94)
Restricted ImageNet
R(x)
63.33(82.14)
82.14(96.47) 44.46(69.13)
5.36(12.01)
G(x))
1.71(5.02)
74.08(93.37)
Table 3: Training on one part of CD-VAE (using large ry = 2) and test on another part: Top-1(Top-5)
In Table 3, in order to further understand how the class-
information is distributed into R(x) and G(z) and why the adversarial attacks towards x mainly affect R(x)
203
Pvvalun-O.0
instead of G(z), we train three classifiers respectively on I, G(x) and R(x)) of training images and evaluate their accuracy when applied to x, G(x) and R(xx) of test images. The results are reported in Table 3, from which we observe: (1) the classifier trained on R(x))
Frequency
150
highly effective on x and vice versa (with only one exception) so they share class-information important to both classifiers; (2) the classifier trained on G(x) has some classification accuracy, indicating that G(x) still preserves useful information about class even with large
50-
0.930
5:000
cornelation coefficient
Figure 3: Pearson correlation of and z'.
7. But the classifier trained on x performs poorly on G(z) so it ignores and does not rely on the class-information G(rr); (3) the classifier trained on R(x) achieves poor accuracy on G(xr) and vice versa, indicating that they use different information for classification. Therefore, adversarial attacks to the classifier trained on J will mainly distort the class-information shared by and important
to both z and R(z) but slightly affect the class-inforee dass-information in G(z).

---

3.3 Applying Class-Disentagglemnnt to Adversarial Detection
Inspired by the empirical study, we propose to apply existing adversarial detection methods to R(z) x x=)(x) instead of z. For example, given an image [30] builds a kemel density estimator
K(,, (xX)) for the intermediate- -ayer embedding (e.g., the t layer output D)(-)) of a classifier D(-)) of the (clean) training set X and use it compute the kernel density of given sample x. i.e.,
K(2, Dx(X)). Intuitively, adversarial examples will have smaller density than the natural samples. Hence, previous works detect them via thresholding to the density value. Instead, we propose to apply the same method to R(x)) computed by CD- CD-VAE, i.e., we detect sample x as an adversarial example if its kernel density K (R(æ), (x((()))) is below a threshold Note this simple strategy is complementary to existing techniques for adversarial detection methods and thus can be generally applied to improving them.
3.4 Applying Class-Disentanglemenn to Adversarial Defense
Defense against Grey-box attack As revealed in Table 2-3, G(z) preserves information useful for classification but redundant and orthogonal to the class information in R(x), which can be mainly distorted by adversarial attacks against the classifier pre-trained on æ. This leads to a simple defense strategy: given an adversarial example x' generated by adversarial attacks to classifier trained on I, we can use G(x') generated by CD-VAE for robust classification since the class information in G(x') is less distorted by attacks. Note this strategy holds for grey-box attacks that have access to the pre-trained classifier but not the CD- model.
Defense against White-box attack We further develop an adversarial training method of CD- DA- as a defense against white-bor box attacks [8] that can access both the classifier and the whole CD-VAE model. In Section 3.4, we propose to classify adversarial images by a classifier trained on G(z'). Hence, white-box attacks generate an adversarial image x' = x + by optimizing the perturbation towards maximizing the classification loss of G(z') within an Ep e-ball around x, i.e.,
max - log Do(G(< + d)]l.. |
(7)
where De is the classifier trained on G(x') and y is the true label. Under white-box attacks, the perturbation ő tends to mainly distort ((x) to deceive the classifier Da((), so &G could dominate 8 and R(-) is no longer the main focus oe attacks. Unlike existing adversarial training methods, which train the whole model (i.e., VAE and De in our case) to correctly classify x', class-disentanglemenn model in CD-VAE enables a novel and more effective adversarial training strategy. In particular, for samples whose predictions by De are heavily distorted, i.e., the true-class probability is similar or lower than the probability of a wrong class: (G('))))) max,' De (G((')))) My'l S c with margin c, we train DG(G(x))) to maximize its predieted probability the true class y and meanwhile train D(R(z)) )) (i.e., the classifier in CD-VAE) to maximize its predicted probability on the most probable wrong-class argmax,'*y (G((()"))
Thereby, besides encouraging DG(G(zz)) to correctly predict y, it enforces the distorted class information in x' or G(x') to move to R(x') instead of G(x'). Hence, G(z') can preserve the class-related information robust to adversarial attacks. Formally, the original CD-VAE objective is modified with new Lp, i.e.,
min 0.0,0,000
((((' (C -mx
max Pc(G(r")) (4 (o, 0) + (ww(w)))))
(8)
La(0, 0) = - (xlee) log po (1 [=) 3Dxr ll)))))
(9)
LD(w,wG) = - log Do (G(((())) [vl log D R( ())):)))))))))))) Jargmax DO DG (10)
where x' is generated from x by attacks in Eq. (7), (o, 0) is the same as Eq. (2) in the original objective, and LD wa) is composed of the two losses associated with the above strategy. By optimizing this objective, CD-VAE can disentangle the mainly distorted class-informatio class-information robust to the white-box attack within x', and assign them to R(x') and G(x') respectively. In practice, we start from CD- CD-VAE prentr on clean images and then mix the original CD-VAE training (i.e., Eq. (1)-(3) on clean images with the above adversarial training on strong adversarial examples x' within margin
and the
Here DD-) is the classifier in CD-VAE trained R(x) and adversarial example x' attacks D(-) in CD-V
7

---

4 Experiments
In this section, we evaluate the classification and reconstruction in CD-VAE under different y and apply CD-VAE to adversarial detection and defense of variety of attacks, as suggested by the empirical study above. We further compare CD-VAE with popular approaches developed for the two tasks and show consistent improvement over them brought by our strategies proposed in Sec. 3.3- 3.4. In particular. all the experiments are conducted on two datasets, CIFAR-10 [26] and restricted ImageNet [44]. CIFAR-10 10 consists of 60,000 color images belonging to 10 classes, with 50,000 training images and 10000 test images. Restricted ImageNet [44] is an ImageNet [11] subset with 203 classes including 7,,748 training images and 10, 150 test images.
CD-VAE Setting For CIFAR-IO, we use VAE G(-) with a few convolutional layers [25] and WideResNet-288 10 [48] as the image classifier D(-). For restricted ImageNet, due to the high resolution (299 x 299), we use VQ- (Q-VAE [38] as G(-) and ResNet-500 [19] as D(-). We set 8 = 0.2. During training. we use Adam [12] with a weight decay of le-6 as the optimizer to minimize Eq. (1) an end-to-end manner for 300(60) epochs on CIFAR- MaAR-1O(restritte) ImageNet).
4.1 Trade-off between Reconstruction and Classification in CD-VAE (via 7)
100
100
1:0
AccuracyyCaaaee aasiiincatiionn Detection)
ACCo Accurdyy ccnnstructionn Derencze
Aocu rac cenve Detection)
-22
Recirdtracion Detacion
Counitcstion
-00
Delane
40
20
Ciediitiion Receminrccion
25
264
2+-3
2+2
2+-3
263
2e-2
(a) CIFAR-10
(b) Restricted ImageNet
Figure 4: Trade-off between reconstruction accuracy and classification accuracy by changing 7.
In the CD-VAE objective Eq. (1), 7 controls the trade-off between classification on R(z) and reconstruction by G(x)). When applied to adversarial detection and defense, may lead to more complicated trade-off behaviors. In Fig. 4, we evaluate classification accuracy of D ((((z)),
G(x)
reconstruction accuracy on (zz) (i.e., I|x [x=-((=)]la/| ulli), adversarial detection accuracy, and the defense accuracy on adversarial examples, when using different y to train CD-VAE
In both plots, classification (reconstruction) accuracy quickly grows when increasing (decreasing) ? from small (large) values and then saturates after sufficiently large (small) y, indicating that moving more information to Rxx) (G(x)) cannot improve the classification (reconstruction). Therefore, for both datasets, a sweet spot is achieved at 7 2 for the
trade-off. Changing 7 also leads to the changes of visual patterns in R(x) and G(x), as shown in Fig. in Sec. A of the Appendix, which reveals the priority of different class information. Fig. 4 also reports how the adversarial detection and defense aceuracy changes with 7 (against PGD-attack to a pre-trained classifier the training set). They does not change monotonically with since increasing (decreasing) y may move both perturbation-spee -speciffc and true class-specific information into R(x) (G(z))) while the former can improve the detection, the latter introduces noises.
4.2
Adversarial Detection
We apply our strategy in Sec. 3.3 several state-of- oa-t-oof-the-ar art adversarial detection approaches, i.e..
kernel density (KD) [13], local intrinsic dimensionality (LID) [34] and mahalanbobis distance (MD) [30], and compare with their original versions on the tasks of detecting adversarial examples generated by several state-of-thed adversarial attacks, i.e., fast gradient sign method (FGSM) [16], basic iterative method (BIM)[27), Carlini& Wagner's method (C& and projected gradient descent method (35)) where ball and foo -bal e-constraints (with both considered, and iterative attacks run for 5 steps using step size <
1 and 8/ 8/255) are We use WideResNNet28-110 (ResNet-50) the DNN feature extractor for CIFAR-10 (restricted ImageNet), which is D(-) from
V22
8

---

FGSM
BIM
C&W
PGD-Lo
PGD-12
Method
TNR
AUC
TNR
AUC
TNR
AUC
TNR
AUC
TNR
AUC
KD
42.38
85.74
74.54
94.82
73.33
94.75
73.12
94.59
70.62
93.62
KD (R(x))
57.10
89.69
96.79
99.27
94.67
98.73
96.56
99.30
97.04
99.32
LID
69.05
93.60
77.73
95.20
74.98
94.32
71.52
93.19
72.57
93.46
LID (R(x))
92.60
98.59
86.42
97.29
76.42
95.10
87.54
97.57
87.63
97.38
MD
94,91
98.69
88.33
97.66
86,30
97.36
77.23
95.38
76.70
95.33
MD (R(x))
99.68
99.36
98.92
99.74
98.94
99.68
99.13
99.79
99.13
99.77
Table 4:
TNR and AUC (%) adversarial detection on vs. R(x) (ours) against attacks (CIFAR-10)
FGSM
BIM
C&W
PGD-13c
PGD-/1
Method
TNR
AUC
TNR
AUC
TNR
AUC
TNR
AUC
TNR
AUC
KD
8.76
70.25
99.26
99.82
99.22
99.77
99.25
99.80
44.02
82.81
KD (R(x))
99.40
99.63
100.0
100.0
100.0
100.0
100.0
100.0
92.85
98.40
LID
44.28
85,91
97.92
99.57
97.58
99.49
97.80
99.51
53.78
87.37
LID (R(z))
99.91
99.93
99.46
99.81
98.24
99.40
99.43
99.73
61.32
89.18
MD
34.69
84.96 99.99
99.33 100.0
99.78 99.99
98.72
99.61
99.18 99.99
99.70 99.95
44.16 95,42
82.61
MD (R(x))
100.0
100.0
99.98
98,86
Table 5: TNR and AUC (%) of adversarial detection on z vs. R(x) (ours) against 5 attacks (restricted ImageNet)
CD-VAE for our strategy but trained on the training set for all other baselines. We use true negative rate (TNR) at 95% true positive rate and area under the receiver operating characteristic curve (AUC) evaluate the detection performance. The results on CIFAR-10 and restricted ImageNet are reported in Table 4-5. By applying the existing detection methods on R(x) instead of x, our strategy consistently and significantly improves all these methods' detection accuracy. For example, FGSM is the most challenging attacks to detect but our method can still improve TNR of KD from 8. 8.76% to 99.40% on restricted ImageNet. White-box detection's performance on other datasets and its generalization other unseen attacks reported in Sec. B and Sec. C of the Appendix.
4.3 Adversarial Defense against Grey-Box Attacks
We evaluate our adversarial defense strategy in Sec. 3.4 against several state-of-the-art attacks, iee, gradient-based attack (BIM[27), PGD[35] and C& ((8)) and spatial transform-based attacks (StAdv[46)), with 1, 8/255, 0.05 for l2, foc, and StAdv, respectively. We the iterations of attacks to 100 and the attacks are applied against pre-trained WideResNet-28- 10 (Resnet50) on CIFAR-10 (restricted ImageNet). According the trade-off in Fig. we choose y 2e 3 for CIFAR-10 and y = 2e- - 1 for restricted ImageNet. We compare our method with adversarial training based methods and preprocessing based methods (HGD[31), APE-GAN[41])
The defense accuracy over adversarial examples are reported in Table 6. Our strategy outperforms most baselines by large margin when defending all types of attacks, while other methods only excel in defending certain attacks, e.g., adversarial training (AT) methods excel on the attacks used in AT but generalize poorly to other attacks. HGD is the most competitive based methods but our method consistently outperforms it. Comparing to other baselines relying on adversarial examples for training, CD-VAE trained only on natural examples already achieves higher defense accuracy. Note the degeneration of clean data accuracy in Table 6 is due to the widely observed trade-off between it and the robustness. In CD-VAE, evaluated in Sec 4. such trade-off can be controlled by tuning 7.
among pre-processing
4.4 Adversarial Defense against White-Box Attacks
We evaluate our adversarial defense strategy towards white-box defense under several -fate-of-the-art white-box attacks on CIFAR-10, the experiment details of which are given Sec. D in the Appendix. We report the defense accuracy in Table. 7. Our method achieves the highest "Unseen Attacks (mean)", which is the defense accuracy averaged over all the attacks that are not used for adversarial training of the defense model. It also achieves the highest defense accuracy against two out of the five attacks,

---

Attack
Dataset
Defense
Clean
PGD-6x
PGD-22
C&W-
C&W-22
SIAdv
Normal
96.01
0.0
0.0
0.0
0.0
0.0
AT PGD-
86.8
51.7
24.3
52.0
26.0
4.8
TRADES tec
84.9
55.1
28.0
53.8
28.3
9.2
AT PGD-&2
85.0
41.9
50.1
43.4
50.6
7.8
CIFARIO
AT StAdv
86.2
0.1
0.3
0.2
0.5
53.9
HGD APE-GAN
80.75
75.93
75.44
75.84
77.15
23.04
90.93
59.28
65.17
59.23
65.30
7.28
Ours
86.81
77.05
78.02
77.04
78.29
19.41
Normal
82.53
0.0
0.0
0.0
0.0
0.0
ImageNet
AT PGD-62
69.89
10.93
60.95
9.49
60.07
0.31
Ours
65.26
52.48
63.12
52.95
64.98
4.75
Table 6: Defense accuracy (%) of our strategy and baselines against various attacks, AT-adversarial training.
Unseen Attacks (mean)
Attack JPEG
Defense
Clean
80
82
ReColor
StAdv
Normal
96.0
0.1
0.0
0.0
0.0
0.4
0.0
AT PGD-L
86.8
27.2
49.0
19.2
30.2
54.5
4.8
TRADES 8oo
84.9
31.0
52.5
23.3
60.6
9.2
AT PGD-&2
85.0
39.5
47.8
60.3
53.5
7.8
AT ReColorAdy
93.4
7.9
8.5
3.9
19.2
65.0
0.0
AT StAdv
86.2
1.8
0.1
0.2
1.9
5.1
53.9
HGD
80.8
0.1
0.0
0.0
0.0
0.4
0.0
APE-GAN
90.9
0.2
0.0
0.0
0.0
1.1
0.0
SAT- (NcurIPS 2020)
32.4
53.1
23.4
35.9
66.5
3.9
(ICML 2020)
83.4
33.6
51.6
24.2
35.4
67.8
7.0
OAT-Cxx 2021)
82.4
50.3
30.2
34.9
48.7
71.0
46,4
Ours-f
81.2
51.4
40.5
43.1
62.1
73.1
27.4
Ours-f2
81.0
50.4
39.4
42.4
61.6
72.2
28.4
Table 7: Defense accuracy (%) of our strategy and baselines against white-box attacks CIFAR 10. AT- adversarial training. "Unseen Attacks (mcan)" reports the defense accuracy averaged over the attacks that are not used adversarial training of defense model. Ours-&oo and Ours- trained using adversarial examples generated by C& W attacks [8] within -bal of bzlball respectively.
iee, JPEG and ReColor. For other three attacks, our model achieves comparable accuracy the best defense model against each attack. The two preprocessing-basee method, i.e., HGD and APE-GAN, both fail under all white-box attacks and achieve nearly 0%% accuracy. The conventional adversarial training (AT) methods perform well in defending but generalize poorly against unseen attacks. In contrast, our model trained merely adversarial examples generated by C&W-L attacks not only generalizes well towards fox attack but also show much better robustness towards other types of unseen attacks. In addition, we compare with three latest state-of-the-- methods, i.c., SAT[21], OIA[39] and PAT[29]] Even when compared method still outperforms them by large margin the defense
model from specific attack it is trained for
method
to their best model checkpoints, accuracy against unseen attacks, because CD- VAE can automatically move the class information primarily distorted by adversarial attacks from G(æ') to R(x') according to the two losses in Eq. (10).
5 Conclusion
In this paper, we propose a VAE+classifier structure "class-disentangle" variational autoencoder (CD-VAE)" to decompose input image x into G(z)++(() where R(x) x z G(z) captures the essential information for classification while G(z) covers all class-redundan information. We further propose an objective to jointly train the VAE and classifier to gain such dass-disentanglememn capability. CD-VA DD- provides novel perspectives to understand (1) how neural net classifier predicts the class of an image and (2) how an adversarial example attacks the class. Inspired by these observations, propose to conduct adversarial detection and adversarial defense on R(xr) and G(z), respectively. These two simple strategies substantially improve the detection and defense against various adversarial attacks.
10

---

Acknowledgements
This work was supported in part by NSFC No. 61872329 and the Fundamental Research Funds for the Central Universitie sunder contract WK3490000005. We would like to thank NeurlPS area chairs and anonymous reviewers for their efforts in reviewing this paper and their constructive comments! We acknowledge the support of GPU cluster built by MCC Lab of Information Science and Technology Institution, USTC.
References
[1] Alexander Alemi, Ian Fischer, Joshua V Dillon, and Kevin Murphy. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410 2016.
[2] Rana Ali Amjad and Bernhard C Geiger. Learning representations for neural network-bai based classification using the information bottleneck principle. IEEE transactions on pattern analysis and machine intelligence, 42(9):2225 2239, 2019.
[3] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798- 1828, 2013.
[4] Diane Bouchacourt, Ryota Tomioka, and Sebastian Nowozin. Multi-level variational autoen- coder: Learning disentangled representations from grouped observations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
[5] Christopher P Burgess, Irina Higgins, Arka Pal, Loic Matthey, Nick Watters, Guillaume Des- jardins, and Alexander Lerchner. Understanding disentangling in f-vac. arXiv preprint 180iv:1804.0 03599, 2018.
[6] Emmanuel Candès, Xiaodong Li, Ma, and John Wright. Robust principal component analysis? Journal of ACM, 58(3), 2011.
[7] Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pages 3-14, 2017.
[8] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks, In 2017 ieee symposium on security and privacy (sp), pages 39-57. IEEE, 2017.
[9] Fabio Carrara, Rudy Becarelli, Roberto Caldelli, Fabrizio Falchi, and Giuseppe Amato. Ad- versarial examples detection in features distance spaces. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pages 0-0, 2018.
[10] Ricky TQ Chen, Xuechen Li, Roger Grosse, and David Duvenaud. Isolating sources of disentanglement variational autoencoders. arXiv preprint vrXivv/802.04900 2018
[11] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference computer vision and puttern recognition, pages
Icee, 2009.
[12] Jacob Devlin, Ming-Wor Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 110.4:1810.04805. 2018.
[13] Reuben Feinman, Ryan R Curtin, Saurabh Shintre, and Andrew B Gardner. Detecting adversarial samples from artifacts. arXiv preprint 1703::/0030041000 2017.
[14] Abel Goozalez-Garrch Joost Van De Weijer, and Yoshua Bengio. Image-to-image translation for cross-domain disentanglement. arXiv preprint | 1805.09733 2018.
[15] Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. arXiv preprint 1xXiv:406.2660 2014.

---

[16] lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. arXiv preprint arXiv:1412.6572, 2014.
[17] Shuangchi Gu, Ping Yi, Ting Zhu, Yao Yao, and Wei Wang- Detecting adversarial examples in deep neural networks using normalizing filters. UMBC Student Collection, 2019.
[18] Sina Hajimiri, Aryo Lotfi, and Mahdieh Soleymani Baghshah Semi-supervised disentanglement of class-related and baasennneeen factors in vae. arXiv preprint wrXiv:2102.00899 2021.
[19] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[20] Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. ß-vac: Learning basic visual concepts with constrained variational framework. 2016.
[21] Lang Huang, Chao Zhang, and Hongyang Zhang. Self-adaptive training: beyond empirical risk minimization. Advances in Neural Information Processing Systems, 33, 2020.
[22] Maximilian llse, Jakub M Tomezak, Christos Louizos, and Max Welling. Diva: Domain invariant variational autoencoders, In Medical Imaging with Deep Learning, pages 322-348. PMLR, 2020.
[23] Daniel Kang, Yi Sun, Tom Brown, Dan Hendrycks, and Jacob Steinhardt, Transfer of adversarial robustness between perturbation types. arXiv preprint UrXiv:1905.01034 2019.
[24] Hyunjik Kim and Andriy Mnih. Disentangling by factorising. In International Conference on Machine Learning, pages 2649-2658. PMLR, 2018.
[25] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv: 14, 2013.
[26] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[27] Alexey Kurakin, Ian Goodfellow, Samy Bengio, et al. Adversarial examples in the physical world, 2016.
[28] Cassidy Laidlaw and Soheil Feizi. Functional adversarial attacks. arXiv:1906.00001, 2019.
arXiv preprint
[29] Cassidy Laidlaw, Sahil Singla, and Soheil Feizi. Perceptual adversarial robustness: Defense against unseen threat models. arXiv preprint arXiv:2006. /2655, 2020.
[30] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting outtof-distribution samples and adversarial attacks. arXiv preprint 1807.038883388 2018.
[31] Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1778-1787, 2018.
[32] Yen-Chen Ben-Cheng Liu, Yu- Ying Yeh, Tzu-Chien Fu, Sheng-De Wang, Wei-Chen Chiu, and Yu- Chiang Frank Wang. Detach and adapt: Learning cross-domain disentangled deep representation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8867-8876, 2018
[33] Francesco Locatello, Stefan Bauer, Mario Lucie, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentangled representations. international conference on machine learning, pages 41 14- 4124. PMLR, 2019.
12

---

[34] Xingjun Ma, Bo Li, Yisen Wang, Sarah M Erfani, Sudanthi Wijewickrema, Grant Schoenebeck, Dawn Song, Michael Houle, and James Bailey. Characterizing adversarial subspaces using local intrinsic dimensionality. arXiv preprint arXiv:/801.080 1801102611, 2018.
[35] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint orXiv11706.0608. 2017.
[36] Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff. On detecting adversarial perturbations. arXiv preprint arXiv: 2017.
[37] Saeid Motiann, Marco Piccirilli, Donald A Adjeroh, and Gianfranco Doretto. Information bottleneck learning using privileged information for visual recognition. In Proceedings of the
IEEE Conference on Computer Vision and Pattern Recognition, pages 1505, 2016.
[38) Aaron van den Oord, Oriol Vinyals. and Koray Kavukcuoglu. Neural discrete representation learning. arXiv preprint 1exxiv//11.0000 2017.
[39] Leslic Rice, Eric Wong, and Zico Kolter.
Overfitting in adversarially robust deep learning. In International Conference Machine Learning, pages 8093-8 1004. PMLR, 2020.
[40] Ramprasaath Selvaraju, Michael Cogswell, Abhishek Das. Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618-626, 2017.
[41] Shiwei Shen, Guoqing Jin, Ke Gao, and Yongdong Zhang. Ape-gan: Adversarial perturbation elimination with gan. arXiv preprint arXiv:1707.0544: 2017.
[42] Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. arXiv preprint physics/0004057, 2000.
[43] Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In 2015 IEEE Information Theory Workshop (ITW), pages 1-5. IEEE, 2015.
[44] Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry, Robustness may be at odds with accuracy. arXiv preprint arXiv::805: /2/52, 2018
[45] Zifeng Wu, Chunhua Shen, and Anton Van Den Hengel. Wider or deeper: Revisiting the resnet model for visual recognition. Pattern Recognition, 90: 19:1119-3,, 2019.
[46] Chaowei Xiao, Jun-Yan Yan Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song. Spatially transformed adversarial examples. arXiv preprint arXiv:1801.022
2018.
[47] Shuo Yang. Tianyu Guo, Yunhe Wang, and Chang Xu. Adversarial robustness through dis- entangled representations. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pages 3145-3153, 2021.
[48] Sergey Zagoruyko and Nikos Komodakis. 16X.v:1605.07146. 2016.
Wide residual networks.
arXiv preprint
[49] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off off between robustness and accuracy. In International Conference on Machine Learning, pages 7472-7482, PMLR, 2019.
[50] Jingwei Zhang, Tongliang Liu, and Dacheng Tao. An information-theoretic view for deep learning, arXiv preprint
8848000000000..00001000000000 2018.
[51] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2921-2929, 2016.
[52] Tianyi Zhou and Dacheng Tao. Godec: Randomized low-rank sparse matrix decomposition in noisy case. In ICML, pages 33-40, 2011.
13


Reviews:
[{"review_id": "yP0aLHs9_AM", "reviewer": "Reviewer_FeME", "summary": "The manuscript proposed a method for the detection and defense against adversarial attacks in classifier models. Towards this goal, a method is proposed to break down inputs $x$  into two components $x=G(x) + R(x)$ where the $R(x)$ contains relevant (discriminative) class-level information and $G(x)$ contains more redundant information shared among classes.\nThe main observation behind the proposed methods is that adversarial attacks target task-essential information ($R(x)$ in this case) and have reduced effect in task-redundant information ($G(x)$).\n\nThe manuscript shows that adversarial attacks can be effectively detected by focusing on R(x). Likewise,  robustness towards these attacks, .i.e defenses, can be achieved feeding the classifier with $G(x)$ instead of $x$.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The content of the manuscript is clear and it has a good flow. \nThe proposed method is sound, interesting and the idea of dividing the input space into the relevant ($R(x)$) and redundant ($G(x)$) components is, to the best of my knowledge, novel. \n\nI find the main strengths of the proposed method to lie on its simplicity and its characteristic of being attack-agnostic (no need to have prior knowledge of possible attack methods).\n\nMy main concerns with the manuscript are the following:\n\n-C1: The idea of adversarial attacks targeting specific class-relevant information is not that novel. While the exploitation of this idea is different, to a good extent the proposed method has similarities with Granda et al., arxiv:2010.15974. Positioning the proposed method with respect to that work would strengthen the manuscript and further highlight the novel aspects of the proposed method. \n\n-C2: in L.72 it is stated that through the proposed method, of focusing on $R(x)$, it is possible to analyze the behavior of neural networks and shed novel insights that cannot be revealed by latent factors. I find this statement not properly supported by experiments in the evaluation section. Moreover, while this can be considered a difference between the input space and latent representations, through the use of visualization algorithms it is now possible to check visually these latent representations.\nIn my opinion the statement in L.72 should be toned down.\n\n-C3: In L.100 it is stated that models that train a binary classification networks (detector) on top of the internal representation of the model to be defended may fail if the detector model is leaked. Doesn't the proposed method also suffer from the same problem? what if the used VAE used to extract the component $G(x)$ is leaked? \nPerhaps this part of the text needs some rephrasing, in its current form it gives the impression that this is a weakness of these methods but not of the proposed one.\n\n\n-C4: Perhaps I missed something, it seems that the defense/robustness against adversarial attacks (Sec. 3.4) is to use as input $G(x)$ instead of $x$. However, if critical/discriminative class-related information is not in $G(x)$ (since it is in $R(x)$) wouldn't it be sacrificing, significantly, the performance of the model (classifier) to be defended. This is already suggested by the results in Table 3 where the model is trained with $x$ and tested with $G(x)$. Since this seem to be the case, the question is whether this drop in predictive performance is a good trade-off to pay for the sake of robustness.\n\n-C5: When evaluating the proposed adversarial defense (Sec.4.3) a new set of spatial transform-based attacks are considered which were not considered for the evaluation of the detection method (Sec. 4.2). I find this a bit counter-intuitive. Is there a good motivation for this? Having a consistent set of baselines and settings will strengthen the validation of the proposed method.\n\n==================\nPost Rebuttal Update\n==================\n\nI thank authors for addressing my feedback on their rebuttal. \nI went through other reviews and the provided rebuttal.\n\nOverall the provided response addressed most of my criticisms to a good extent.\n\nAs part of point (1) the rebuttal claims that the proposed method can also be used for interpretation of DNNs. I find content supporting this claim to be very weak. Likewise, the second point (2) of the response hints at this interpretation capability can be achieved through the inspection of $R(x)$. Looking at Fig.2 presented in the paper, I find the visualizations of $R(x)$ somewhat ambiguous. I struggle to see how $R(x)$ can serve as means of analyzing the behavior of neural networks. I am still of the thought that the interpretation capabilities of the proposed method (claimed in the paper) are somewhat unjustified. \nIn my opinion the claims made by the manuscript along the interpretation aspects should be be toned down.\n\nKeeping the comments from above I have decided increasing my initial rating. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "N.A.", "review_text": "", "extra_scores": {"ethical_concerns": "N.A.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "uqCOZseLF5", "reviewer": "Reviewer_qF3G", "summary": "The paper studies adversarial robustness in Deep Neural Networks. The main claim is that adversarial attacks are focusing on relatively small / sparse regions of the image which carry the essential class information. This small region can be efficiently estimated using a novel class disentanglement method based on VAEs. The paper conducts an empirical study on how attacks behaves on the class essential part R(x) and the remaining part G(x) and derive new methods both for adversarial defense and detection.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall I like the idea of the paper. It is well organized and readable. The idea of using disentanglement methods to address adversarial robustness is new to me. The empirical study is conducted carefully and the list of used baselines is complete (especially for the detection part).\n\nA few concerns / questions that I have:\n* The idea of the decomposition is nice. It would be interesting to see if the inductive bias induced by the competition with the classifier brings enough stability to this decomposition. Claiming that R(x) corresponds to the regions targeted by the attacker implies somehow that the decomposition is \"unique\".\n* By design, adversarial attacks focuses on the pixels that are important for classification so it is hard to assess how good is the proposed decomposition. Could it be compared with simpler decompositions for instance based on saliency maps / grad-cam ?\n* Some visualization are not that informative (delta G and delta R in Figure 2, the magnitude of the change is enough)\n* The benchmarks are conducted against the major white-box attacks but it could include additional black-box attacks, especially for the detection part (+ using both BIM and PGD is a bit redundant). Query-based attacks can be a good solution for this.\n* In section 3.2, the value of epsilon is not clearly stated (0.14 ?). Why did you choose this value and did you try different ones ? More generally, it would have been interesting to plot evolutions of metrics wrt epsilon for PGD / BIM / FGSM (in order to see if larger perturbation impacts more or less G(x) and study how the distortion evolves).\n* The Appendix focuses on the important case where the attacker is aware of the defense / detection. IMO this part should belong to main paper and not to a supplementary part. A solution would be to summarize the contribution and move it to a section in the main part.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, nothing to report for this section.", "review_text": "", "extra_scores": {"ethical_concerns": "Nothing to report", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6h"}, "extra_text": {}}, {"review_id": "uo44q2cmK0s", "reviewer": "Reviewer_kR37", "summary": "The paper proposes an approach to separate class-specific and irrelevant information from an input image. A VAE is used to reconstruct irrelevant information. The VAE is jointly trained with a classifier to achieve this separation. The paper concludes that the class-specific signal can be used for adversarial detection while the irrelevant information can be used for adversarial defense. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- This is an input processing defense approach where a VAE is first applied to reconstruct an input image that is sufficient for classification. Subsequently a classifier operated on the non-redundant information and classifies the reconstructed image. \n- Information bottleneck prinicpal is applied to achieve disentanglement in the input space.\n- An interesting relationship with the adversarial examples is shown i.e., the perturbations mostly relate to the information unrrelated to the image class. \n\nWeaknesses:\n- The comparisons with other detection and defense mechanisms mostly include old approaches (pre 2018) and do not compare with the state of the art defense mechanisms. \n- The main contribution of this work is to apply Information Bottleneck principal on the latent representation of a VAE. There exists previous VAE based approaches for adversarial defense, as I elaborate in a later comment. Therefore, the novelty is limited to the combination of beta-VAE with the IB loss. \n- Section A.2 in supp. material mentions the values of hyper-parameters used for training, how are these values set?\n- Paper is written in an obscured way at several important points. E.g., the abstract can be improved for clarity and by avoiding notations that are not required as such at the very beginning. \n- AutoAttack and other white box attack results are provided in the supplementary material. I believe these are the main results and should be included in the main paper. \n- The paper does not discuss and compare with relevant literature. For example, there exist a number of approaches based on VAE as an input processing step [A,B]. The proposed technique should be discussed in the context of these works and compared with the existing related methods. \n- Do the authors plan to release the code? There are several implementation details that are less clear and a codebase will help reproduce the reported results. \n\n[A] Defense-VAE: A Fast and Accurate Defense against Adversarial Attacks\n[B] PuVAE: A Variational Autoencoder to Purify Adversarial Examples\n\nPost-rebuttal Comments: The authors have done a great job of answering the raised queries. I am therefore inclined towards accepting this paper and request authors to make the promised changes. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations and societal impact are not discussed. These considerations are very much relevant to the nature of the work. ", "review_text": "", "extra_scores": {"ethical_concerns": "None. \n\n", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}, {"review_id": "tiO_uAANQ4X", "reviewer": "Reviewer_TtXL", "summary": "Inspired by information bottleneck, the paper proposed a framework that creates class-disentanglement in the input space by setting up a competition between VAE reconstruction on G(x) and accurate classification on x-G(x). Then empirical observations suggest that popular adversarial attack methods mainly affect the task-essential information x-G(x) and has less influence on task-redundant part G(x). Such an observation then motivates a new strategy towards adversarial detection and defense. Extensive empirical study on CIFAR-10 and Imagenet corroborate the hypothesis. ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Different from existing work on information bottleneck over latent space, this paper presents an interesting framework trying to learn class-disentanglement on the input space. The observations that adversarial attack mainly affects the task-essential information is quite interesting and indeed leads to practical algorithms for adversarial attack detection and defense, which empirically works better based on the provided results. Below are my detailed comments/questions:\n\n- Since this is an empirical study and few formal definitions/theorems are provided, I found some concepts/definitions are not clear. For example, in line 61, \"G(x) contains all the information redundant or irrelevant to the class y\". \n\nWhat is redundant information? I guess the redundant information in G(x) is task-relevant but redundant/same with respect to the information in x-G(x). Then if x-G(x) is negatively affected by the adversarial attack, this part of information in G(x) should also be easily affected? Why is G(x) robust then?\n\nIf the task irrelevant information means information that is not directly related to the classification task such as background color, then adversarial attack such as PGD should not change this part too much since they do not affect the class prediction too much. \n\nThus G(x), as a mixture of both, seems to be affected less but the important part in G(x) may still be affected?\n- One advantage of existing information bottleneck methods on latent space is that, since raw sample x may contain many task-irrelevant information and the useful information likely lies in a submanifold of original sample space, it is desirable to extract task-essential information in a low-dimensional latent space and learn a downstream classifier purely on the latent space (e.g. improve data efficiency). While in the proposed approach, the classification has to be performed on a space with the same dimension as the original data space (either x, G(x), or x-G(x)).\n- The PGD attack is with respect to a pre-trained classifier on clean data x, not the classifier in CD-VAE on x-G(x), right? Because this will make sure x-G(x) is vulnerable to adversarial attack not merely because we are attacking D(x-G(x)).\n- For adversarial attack restricted to lp-norm, there has been rich literatures on certified defense, i.e. defense with theoretical guarantee. Since this paper also focuses on lp-norm attack, formal theoretical discussions on the proposed method will greatly improve the paper. For example, in line 116, the paper claims \"we show that classification on the class-redundant part G(x) is robust to adversarial attacks\". Is this true for all kinds of attack? What is the assumptions needed? And if we can extract class-redundant part in the latent space using IB, is that also robust to adversarial attack? Any experimental results on this?\n- In Eq 2, why do we need disentanglement (beta-VAE) in the feature space of VAE. Such a disentanglement notion seems irrelevant to the class disentanglement discussed in this paper.\n- No error bars are provided for all the tables, which may be needed to demonstrate that the empirical findings are not from randomness (especially for a pure empirical paper).", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No potential negative societal impact was discussed.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "aKLQ9Jm6hy", "reviewer": "Reviewer_81hb", "summary": "This paper proposes a novel method called “class-disentanglement” for adversarial defense and detection. It decomposes one image into two parts: the reconstruction part, which contains the redundant information, and the residual part, which captures the essential information for prediction. Empirical results show that adversarial perturbations majorly lie in the residual part, and the reconstruction part still contains class information irrelevant to the residual part. Motivated by this observation, the authors improve existing adversarial detection methods, such as KD, LID, and MD, by implementing these methods on the residual part. Also, they train a classifier over the reconstruction part and exhibit competitive robustness.", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: This paper contributes some new ideas. But analyzing adversarial examples with class-disentanglement is not so novel, such as the work [1], which considers this problem in the latent space.\nQuality: This paper is well organized. \nClarity: The paper is easy to read. However, there still are some typos. \nSignificance: This paper reveals an interesting observation that the information for classification is redundant. Adversarial perturbation merely impacts a fraction of them. However, the experimental results are not sufficient.\n\nWeaknesses and my questions of this paper:\n1.\tI suspect the cosine similarity report in Table 1. The mean and standard deviation of the cosine similarity between G(x) and G(x’) is 1 and 0, respectively. That means, for all considered x, G(x) and G(x’) are exactly the same up to a constant coefficient. It seems abnormal. \n\n2.\tIn Table 2, the author report Lp norms of δ and δR to support their claim that “ the adversarial perturbation δ mainly lies in δR…” (lines 198-202). However, even if δ and δR have similar Lp norms, possibly we cannot conclude that they are correlated. \n\n3.\tGeneralizability is quite important to evaluate the effectiveness of adversarial detection or defense methods. However, the authors merely considered the case where the adversarial examples for training and evaluating the detector or defense model are generated by the same attack method.\nOther limitations:\n1.\tThe line 222 has a typo “ In order to”\n2.\tThe line 292 has a typo “ sImageNet”\n\n[1] Yang, Shuo et al, “Adversarial Robustness through Disentangled Representations”, Proceedings of the AAAI Conference on Artificial Intelligence, 2021.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "NA", "review_text": "", "extra_scores": {"ethical_concerns": "NA", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2 hours"}, "extra_text": {}}, {"review_id": "SqKcqGcuSR", "reviewer": "Reviewer_mKY1", "summary": "Use VAEs to extract \"class-dependent information\". Validate that adversarial perturbations are more prone to lie in the class-dependent component of the decomposition. Leverage the decomposition for detection and defense. ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "*Originality*: Existing work which utilized VAEs for adversarial defense does not appear to be cited (https://arxiv.org/abs/1805.09190, https://arxiv.org/abs/1802.06552, https://arxiv.org/abs/1806.00081, https://arxiv.org/abs/1906.00230). \n\nFurthermore, I would be surprised that the idea of decomposing an image into class-independent and class-dependent features has not been explored before in, say, the \"interpretability\" literature, however the related work solely focuses on generic methods for disentangled representation learning, adversarial detection, and adversarial defense. I would have much appreciated the related work section would focus on, or at least have a section regarding, the novelty of the task-dependent decomposition. \n\n*Quality*: I am wary that the work relies upon suboptimality in optimization, which is reflected in a significant reduction in clean accuracy visible in Table 6. Given this, I would not say this work provides a sound contribution for detection/defense where the performance bounds are well-understood for practitioners. \n\n*Clarity*: I would not say the submission is clearly written, and thereby I would not say the work is easily reproducible. More details below.  \n\nCould the authors clarify how the adversarial was generated in Sec. 3.2? Equation (1) implies D() is trained on x - G(x), I am not sure why then D() should classify x correctly, which is the presumption with which standard adversarial attacks, i.e. what was detailed in Eq. (4) rests upon. \n\nOn this point, it does not appear a D() is trained jointly with G() with regards to the experiment specified in Sec. 3.2, and thus going forward in the experiments section, it is not clear whether a) D() is pretrained on the input images and fixed/frozen when optimizing G(), or b) D() is trained simultaneously with G(), as Eq. (1) implies. Regardless, the reader should not have to make any assumptions regarding such significant details. \n\nRegarding Sec. 3.4, if again the above presumptions are correct, G() is optimized s.t. it can reconstruct the given image well, while if subtracted from the input, the pretrained classifier functions well. This trade-off should lead to G() prioritizing capturing class-independent features, as mentioned by the authors. More specifically, G() should capture as many features as possible, for improved reconstruction, thus only the minimal necessary features for the classifier to function should be discarded. If G() is optimal, then yes the output of G() intuitively should be less perturbed by an adversarial, but it is questionable whether the output of G() should be useful for classification, given what the classifier *needs* to be accurate, has been by definition, removed. Thus, why classifying the output of G() should work, seems to rely on a somewhat mysterious interplay in the learning objective trade-off, and this lack of clarity makes me question the robustness of the demonstrated empirical results in the work. \n\nIn line with the above argument, a significant reduction in the clean accuracy can be seen in Table 6, however this clear detriment is not discussed by the authors. \n\n*Significance*: a) not clear that the approach is novel given the lack of discussion in related work on class-dependent decomposition of images; b) not clear if this work can be built upon given critical missing details in the main paper, namely with regards to the optimization of D(); c) not clear, *why* the defense in particular works, and as it currently stands, using the defense leads to a significant reduction in clean accuracy. \n\n**Post-Rebuttal**: Authors have addressed my concerns in full, and I have thus raised my score. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I would not say the authors adequately addressed the limitations and potential negative societal impact of their work. While I don't see any unique potential negative societal impact of this work, a extended, clear limitations discussion focused on the reduction in clean accuracy, could have, in part, ameliorated my concerns with regards to this paper. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2-3 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper addresses adversarial detection and defense through a model that disentangles task-relevant and -irrelevant information. The model performs favorably against baselines, and the authors provided additional more recent baselines following feedback from reviewers. On the whole, the paper received a large number of reviews that were positive with engaging and informative discussion, so I believe that this paper has something positive to offer for the venue. There were some noted weaknesses to the paper; the reviewers believe the paper should still be accepted, but they ask that some of their requested changes make it to the final draft. I recommend acceptance as a poster.
Decision: Accept (Poster)