Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: Inspired by information bottleneck, the paper proposed a framework that creates class-disentanglement in the input space by setting up a competition between VAE reconstruction on G(x) and accurate classification on x-G(x). Then empirical observations suggest that popular adversarial attack methods mainly affect the task-essential information x-G(x) and has less influence on task-redundant part G(x). Such an observation then motivates a new strategy towards adversarial detection and defense. Extensive empirical study on CIFAR-10 and Imagenet corroborate the hypothesis. 
main_review: Different from existing work on information bottleneck over latent space, this paper presents an interesting framework trying to learn class-disentanglement on the input space. The observations that adversarial attack mainly affects the task-essential information is quite interesting and indeed leads to practical algorithms for adversarial attack detection and defense, which empirically works better based on the provided results. Below are my detailed comments/questions:

- Since this is an empirical study and few formal definitions/theorems are provided, I found some concepts/definitions are not clear. For example, in line 61, "G(x) contains all the information redundant or irrelevant to the class y". 

What is redundant information? I guess the redundant information in G(x) is task-relevant but redundant/same with respect to the information in x-G(x). Then if x-G(x) is negatively affected by the adversarial attack, this part of information in G(x) should also be easily affected? Why is G(x) robust then?

If the task irrelevant information means information that is not directly related to the classification task such as background color, then adversarial attack such as PGD should not change this part too much since they do not affect the class prediction too much. 

Thus G(x), as a mixture of both, seems to be affected less but the important part in G(x) may still be affected?
- One advantage of existing information bottleneck methods on latent space is that, since raw sample x may contain many task-irrelevant information and the useful information likely lies in a submanifold of original sample space, it is desirable to extract task-essential information in a low-dimensional latent space and learn a downstream classifier purely on the latent space (e.g. improve data efficiency). While in the proposed approach, the classification has to be performed on a space with the same dimension as the original data space (either x, G(x), or x-G(x)).
- The PGD attack is with respect to a pre-trained classifier on clean data x, not the classifier in CD-VAE on x-G(x), right? Because this will make sure x-G(x) is vulnerable to adversarial attack not merely because we are attacking D(x-G(x)).
- For adversarial attack restricted to lp-norm, there has been rich literatures on certified defense, i.e. defense with theoretical guarantee. Since this paper also focuses on lp-norm attack, formal theoretical discussions on the proposed method will greatly improve the paper. For example, in line 116, the paper claims "we show that classification on the class-redundant part G(x) is robust to adversarial attacks". Is this true for all kinds of attack? What is the assumptions needed? And if we can extract class-redundant part in the latent space using IB, is that also robust to adversarial attack? Any experimental results on this?
- In Eq 2, why do we need disentanglement (beta-VAE) in the feature space of VAE. Such a disentanglement notion seems irrelevant to the class disentanglement discussed in this paper.
- No error bars are provided for all the tables, which may be needed to demonstrate that the empirical findings are not from randomness (especially for a pure empirical paper).
limitations_and_societal_impact: No potential negative societal impact was discussed.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes an approach to separate class-specific and irrelevant information from an input image. A VAE is used to reconstruct irrelevant information. The VAE is jointly trained with a classifier to achieve this separation. The paper concludes that the class-specific signal can be used for adversarial detection while the irrelevant information can be used for adversarial defense. 
main_review: Strengths:
- This is an input processing defense approach where a VAE is first applied to reconstruct an input image that is sufficient for classification. Subsequently a classifier operated on the non-redundant information and classifies the reconstructed image. 
- Information bottleneck prinicpal is applied to achieve disentanglement in the input space.
- An interesting relationship with the adversarial examples is shown i.e., the perturbations mostly relate to the information unrrelated to the image class. 

Weaknesses:
- The comparisons with other detection and defense mechanisms mostly include old approaches (pre 2018) and do not compare with the state of the art defense mechanisms. 
- The main contribution of this work is to apply Information Bottleneck principal on the latent representation of a VAE. There exists previous VAE based approaches for adversarial defense, as I elaborate in a later comment. Therefore, the novelty is limited to the combination of beta-VAE with the IB loss. 
- Section A.2 in supp. material mentions the values of hyper-parameters used for training, how are these values set?
- Paper is written in an obscured way at several important points. E.g., the abstract can be improved for clarity and by avoiding notations that are not required as such at the very beginning. 
- AutoAttack and other white box attack results are provided in the supplementary material. I believe these are the main results and should be included in the main paper. 
- The paper does not discuss and compare with relevant literature. For example, there exist a number of approaches based on VAE as an input processing step [A,B]. The proposed technique should be discussed in the context of these works and compared with the existing related methods. 
- Do the authors plan to release the code? There are several implementation details that are less clear and a codebase will help reproduce the reported results. 

[A] Defense-VAE: A Fast and Accurate Defense against Adversarial Attacks
[B] PuVAE: A Variational Autoencoder to Purify Adversarial Examples

Post-rebuttal Comments: The authors have done a great job of answering the raised queries. I am therefore inclined towards accepting this paper and request authors to make the promised changes. 
limitations_and_societal_impact: Limitations and societal impact are not discussed. These considerations are very much relevant to the nature of the work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The manuscript proposed a method for the detection and defense against adversarial attacks in classifier models. Towards this goal, a method is proposed to break down inputs $x$  into two components $x=G(x) + R(x)$ where the $R(x)$ contains relevant (discriminative) class-level information and $G(x)$ contains more redundant information shared among classes.
The main observation behind the proposed methods is that adversarial attacks target task-essential information ($R(x)$ in this case) and have reduced effect in task-redundant information ($G(x)$).

The manuscript shows that adversarial attacks can be effectively detected by focusing on R(x). Likewise,  robustness towards these attacks, .i.e defenses, can be achieved feeding the classifier with $G(x)$ instead of $x$.
main_review: The content of the manuscript is clear and it has a good flow. 
The proposed method is sound, interesting and the idea of dividing the input space into the relevant ($R(x)$) and redundant ($G(x)$) components is, to the best of my knowledge, novel. 

I find the main strengths of the proposed method to lie on its simplicity and its characteristic of being attack-agnostic (no need to have prior knowledge of possible attack methods).

My main concerns with the manuscript are the following:

-C1: The idea of adversarial attacks targeting specific class-relevant information is not that novel. While the exploitation of this idea is different, to a good extent the proposed method has similarities with Granda et al., arxiv:2010.15974. Positioning the proposed method with respect to that work would strengthen the manuscript and further highlight the novel aspects of the proposed method. 

-C2: in L.72 it is stated that through the proposed method, of focusing on $R(x)$, it is possible to analyze the behavior of neural networks and shed novel insights that cannot be revealed by latent factors. I find this statement not properly supported by experiments in the evaluation section. Moreover, while this can be considered a difference between the input space and latent representations, through the use of visualization algorithms it is now possible to check visually these latent representations.
In my opinion the statement in L.72 should be toned down.

-C3: In L.100 it is stated that models that train a binary classification networks (detector) on top of the internal representation of the model to be defended may fail if the detector model is leaked. Doesn't the proposed method also suffer from the same problem? what if the used VAE used to extract the component $G(x)$ is leaked? 
Perhaps this part of the text needs some rephrasing, in its current form it gives the impression that this is a weakness of these methods but not of the proposed one.


-C4: Perhaps I missed something, it seems that the defense/robustness against adversarial attacks (Sec. 3.4) is to use as input $G(x)$ instead of $x$. However, if critical/discriminative class-related information is not in $G(x)$ (since it is in $R(x)$) wouldn't it be sacrificing, significantly, the performance of the model (classifier) to be defended. This is already suggested by the results in Table 3 where the model is trained with $x$ and tested with $G(x)$. Since this seem to be the case, the question is whether this drop in predictive performance is a good trade-off to pay for the sake of robustness.

-C5: When evaluating the proposed adversarial defense (Sec.4.3) a new set of spatial transform-based attacks are considered which were not considered for the evaluation of the detection method (Sec. 4.2). I find this a bit counter-intuitive. Is there a good motivation for this? Having a consistent set of baselines and settings will strengthen the validation of the proposed method.

==================
Post Rebuttal Update
==================

I thank authors for addressing my feedback on their rebuttal. 
I went through other reviews and the provided rebuttal.

Overall the provided response addressed most of my criticisms to a good extent.

As part of point (1) the rebuttal claims that the proposed method can also be used for interpretation of DNNs. I find content supporting this claim to be very weak. Likewise, the second point (2) of the response hints at this interpretation capability can be achieved through the inspection of $R(x)$. Looking at Fig.2 presented in the paper, I find the visualizations of $R(x)$ somewhat ambiguous. I struggle to see how $R(x)$ can serve as means of analyzing the behavior of neural networks. I am still of the thought that the interpretation capabilities of the proposed method (claimed in the paper) are somewhat unjustified. 
In my opinion the claims made by the manuscript along the interpretation aspects should be be toned down.

Keeping the comments from above I have decided increasing my initial rating. 

limitations_and_societal_impact: N.A.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper studies adversarial robustness in Deep Neural Networks. The main claim is that adversarial attacks are focusing on relatively small / sparse regions of the image which carry the essential class information. This small region can be efficiently estimated using a novel class disentanglement method based on VAEs. The paper conducts an empirical study on how attacks behaves on the class essential part R(x) and the remaining part G(x) and derive new methods both for adversarial defense and detection.
main_review: Overall I like the idea of the paper. It is well organized and readable. The idea of using disentanglement methods to address adversarial robustness is new to me. The empirical study is conducted carefully and the list of used baselines is complete (especially for the detection part).

A few concerns / questions that I have:
* The idea of the decomposition is nice. It would be interesting to see if the inductive bias induced by the competition with the classifier brings enough stability to this decomposition. Claiming that R(x) corresponds to the regions targeted by the attacker implies somehow that the decomposition is "unique".
* By design, adversarial attacks focuses on the pixels that are important for classification so it is hard to assess how good is the proposed decomposition. Could it be compared with simpler decompositions for instance based on saliency maps / grad-cam ?
* Some visualization are not that informative (delta G and delta R in Figure 2, the magnitude of the change is enough)
* The benchmarks are conducted against the major white-box attacks but it could include additional black-box attacks, especially for the detection part (+ using both BIM and PGD is a bit redundant). Query-based attacks can be a good solution for this.
* In section 3.2, the value of epsilon is not clearly stated (0.14 ?). Why did you choose this value and did you try different ones ? More generally, it would have been interesting to plot evolutions of metrics wrt epsilon for PGD / BIM / FGSM (in order to see if larger perturbation impacts more or less G(x) and study how the distortion evolves).
* The Appendix focuses on the important case where the attacker is aware of the defense / detection. IMO this part should belong to main paper and not to a supplementary part. A solution would be to summarize the contribution and move it to a section in the main part.
limitations_and_societal_impact: Yes, nothing to report for this section.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: Use VAEs to extract "class-dependent information". Validate that adversarial perturbations are more prone to lie in the class-dependent component of the decomposition. Leverage the decomposition for detection and defense. 
main_review: *Originality*: Existing work which utilized VAEs for adversarial defense does not appear to be cited (https://arxiv.org/abs/1805.09190, https://arxiv.org/abs/1802.06552, https://arxiv.org/abs/1806.00081, https://arxiv.org/abs/1906.00230). 

Furthermore, I would be surprised that the idea of decomposing an image into class-independent and class-dependent features has not been explored before in, say, the "interpretability" literature, however the related work solely focuses on generic methods for disentangled representation learning, adversarial detection, and adversarial defense. I would have much appreciated the related work section would focus on, or at least have a section regarding, the novelty of the task-dependent decomposition. 

*Quality*: I am wary that the work relies upon suboptimality in optimization, which is reflected in a significant reduction in clean accuracy visible in Table 6. Given this, I would not say this work provides a sound contribution for detection/defense where the performance bounds are well-understood for practitioners. 

*Clarity*: I would not say the submission is clearly written, and thereby I would not say the work is easily reproducible. More details below.  

Could the authors clarify how the adversarial was generated in Sec. 3.2? Equation (1) implies D() is trained on x - G(x), I am not sure why then D() should classify x correctly, which is the presumption with which standard adversarial attacks, i.e. what was detailed in Eq. (4) rests upon. 

On this point, it does not appear a D() is trained jointly with G() with regards to the experiment specified in Sec. 3.2, and thus going forward in the experiments section, it is not clear whether a) D() is pretrained on the input images and fixed/frozen when optimizing G(), or b) D() is trained simultaneously with G(), as Eq. (1) implies. Regardless, the reader should not have to make any assumptions regarding such significant details. 

Regarding Sec. 3.4, if again the above presumptions are correct, G() is optimized s.t. it can reconstruct the given image well, while if subtracted from the input, the pretrained classifier functions well. This trade-off should lead to G() prioritizing capturing class-independent features, as mentioned by the authors. More specifically, G() should capture as many features as possible, for improved reconstruction, thus only the minimal necessary features for the classifier to function should be discarded. If G() is optimal, then yes the output of G() intuitively should be less perturbed by an adversarial, but it is questionable whether the output of G() should be useful for classification, given what the classifier *needs* to be accurate, has been by definition, removed. Thus, why classifying the output of G() should work, seems to rely on a somewhat mysterious interplay in the learning objective trade-off, and this lack of clarity makes me question the robustness of the demonstrated empirical results in the work. 

In line with the above argument, a significant reduction in the clean accuracy can be seen in Table 6, however this clear detriment is not discussed by the authors. 

*Significance*: a) not clear that the approach is novel given the lack of discussion in related work on class-dependent decomposition of images; b) not clear if this work can be built upon given critical missing details in the main paper, namely with regards to the optimization of D(); c) not clear, *why* the defense in particular works, and as it currently stands, using the defense leads to a significant reduction in clean accuracy. 

**Post-Rebuttal**: Authors have addressed my concerns in full, and I have thus raised my score. 
limitations_and_societal_impact: I would not say the authors adequately addressed the limitations and potential negative societal impact of their work. While I don't see any unique potential negative societal impact of this work, a extended, clear limitations discussion focused on the reduction in clean accuracy, could have, in part, ameliorated my concerns with regards to this paper.
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: This paper proposes a novel method called “class-disentanglement” for adversarial defense and detection. It decomposes one image into two parts: the reconstruction part, which contains the redundant information, and the residual part, which captures the essential information for prediction. Empirical results show that adversarial perturbations majorly lie in the residual part, and the reconstruction part still contains class information irrelevant to the residual part. Motivated by this observation, the authors improve existing adversarial detection methods, such as KD, LID, and MD, by implementing these methods on the residual part. Also, they train a classifier over the reconstruction part and exhibit competitive robustness.
main_review: Originality: This paper contributes some new ideas. But analyzing adversarial examples with class-disentanglement is not so novel, such as the work [1], which considers this problem in the latent space.
Quality: This paper is well organized. 
Clarity: The paper is easy to read. However, there still are some typos. 
Significance: This paper reveals an interesting observation that the information for classification is redundant. Adversarial perturbation merely impacts a fraction of them. However, the experimental results are not sufficient.

Weaknesses and my questions of this paper:
1.	I suspect the cosine similarity report in Table 1. The mean and standard deviation of the cosine similarity between G(x) and G(x’) is 1 and 0, respectively. That means, for all considered x, G(x) and G(x’) are exactly the same up to a constant coefficient. It seems abnormal. 

2.	In Table 2, the author report Lp norms of δ and δR to support their claim that “ the adversarial perturbation δ mainly lies in δR…” (lines 198-202). However, even if δ and δR have similar Lp norms, possibly we cannot conclude that they are correlated. 

3.	Generalizability is quite important to evaluate the effectiveness of adversarial detection or defense methods. However, the authors merely considered the case where the adversarial examples for training and evaluating the detector or defense model are generated by the same attack method.
Other limitations:
1.	The line 222 has a typo “ In order to”
2.	The line 292 has a typo “ sImageNet”

[1] Yang, Shuo et al, “Adversarial Robustness through Disentangled Representations”, Proceedings of the AAAI Conference on Artificial Intelligence, 2021.
limitations_and_societal_impact: NA
review_title: null

Meta-review: The paper addresses adversarial detection and defense through a model that disentangles task-relevant and -irrelevant information. The model performs favorably against baselines, and the authors provided additional more recent baselines following feedback from reviewers. On the whole, the paper received a large number of reviews that were positive with engaging and informative discussion, so I believe that this paper has something positive to offer for the venue. There were some noted weaknesses to the paper; the reviewers believe the paper should still be accepted, but they ask that some of their requested changes make it to the final draft. I recommend acceptance as a poster.
Decision: Accept (Poster)