Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: In this paper, the authors propose a conjunction of Bayesian learning and regularization via information gain as a means of adversarial defense. In particular, they start from the perspective that Bayesian learning of neural network parameters ought to be more robust to adversarial examples, and then state that by jointly minimizing the risk and regularizing adversarial and clean information gain that the resulting Bayesian posterior will be even more robust. 

While this -- robustness and adversarial training of Bayesian neural networks -- is a worthwhile and important direction, the authors make several crucial mistakes and literature oversights which inhibit the potential impact of this work. 
main_review: Unfortunately, there are many claims made in this paper which are either overly strong, incorrect, or fail to appreciate prior work. 

Firstly, the authors correctly note that Bayesian neural networks are empirically and theoretically more robust to attacks. I think the authors should certainly cite [1] here as a motivating example. 

The authors highlight the work of Adv-BNN as state of the art for Bayesian neural networks against adversarial attacks, but this has been greatly surpassed in [2] (more on this below). Moreover, their "proposed adaptive PGD attack" is not their own proposal and has been used in the Bayesian robustness literature for at least 3 years. The reason this proposed attack has been used is that the one given in Adv-BNN and repeated in equations (3) and (7) of this paper is both theoretically incorrect and empirically suboptimal. As pointed out by the authors, [3] clearly states the incorrectness of the attack used in Adv-BNN and shows it is suboptimal in practice for Bayesian neural networks (further corroborated by Tables 1 and 2 of this paper). This error is the Bayesian counter-part to the expectation over transformations (EoT) attack formalized in [4] which follows up on a work the authors cite, in particular [5]. Further, in works on BNNs this "proposal" has been adopted and is widely used, see [1,2,6], so I think it is incorrect to deem this as a "proposed" attack in the evaluation/contribution section of the paper and should be moved to the related works. 

Further on this, despite the clear incorrectness of equation (3) and its corresponding formulation for this paper in equation (7) according to both deterministic principles [4,5] and Bayesian principles [2,3] the authors still use this attack during training and partially during evaluation. Thus, the methodology, as presented, is suboptimal/incorrect. In fact, the results section of this paper corroborate this claim (Table 2 versus Table 1) 

Outside of this incorrectness, the authors propose a regularization which is simply a specific form of Adversarial Logit Pairing [7]. This paper originally appeared in NeurIPS 2018, but was later retracted after it was found that this kind of robustness is very easily attacked and provides no substantial gains see [8]. I think if the authors would like to stick to this methodology then they ought to properly evaluate their method with a Bayesian adaptation of the method in [8]. I do not doubt the theoretical perspective given by the authors, only that it is practically effective/worthwhile. Moreover, I think the authors ought to be very careful in their general use of ERM approaches to likelihoods. This is perhaps acceptable for variational inference methods which replace marginalization with maximization, but in general, the method of [2] is preferable as it works for both maximization and marginalization and is thus on better grounds from a Bayesian point of view. Further on this point, I think it would benefit the paper if the authors stated their algorithm in a more general form as simply proposing a modification of a single inference method reduces the apparent contribution. 

Finally, I do have a further question about the methods evaluation. Is this method compared with AdvBNN applied to SVGD? Or is AdvBNN using Bayes by Backprop (as in its original formulation)? If so, the authors should ensure that the same inference method is used between their method and that of the network they compare against as inference method is known to have a considerably effect on the robustness of Bayesian posteriors [1,2,6]. In fact, the empirical performance noted in this paper might be solely the effect of a more faithful inference method and may have no correlation with the use of their method. Unless, the "BNN" label in Figure 2 is also SVGD. Still, the authors ought to compare with [2] prior to making the claim that they have state of the art robustness. 

[1] - https://arxiv.org/abs/2002.04359 (Appeared in NeurIPS last year)
[2] - https://arxiv.org/abs/2102.05289 (Appeared in AISTATS last year)
[3] - https://arxiv.org/abs/1907.00895 (Correction to Adv-BNN in 2019)
[4] - https://arxiv.org/pdf/1707.07397.pdf (Appeared in ICML 2018)
[5] - https://arxiv.org/pdf/1802.00420.pdf (Already cited in paper but here for completeness)
[6] - https://arxiv.org/abs/2012.12640 (Appeared in AABI 2021)
[7] - https://arxiv.org/abs/1803.06373 (Appeared, later retracted NeurIPS 2018)
[8] - https://arxiv.org/abs/1807.10272


summary_of_the_review: The paper is in a worthwhile direction, but ultimately has too many oversights and errors to be accepted as is. The attack methodology relied upon is known to be incorrect and suboptimal. The regularization introduced is a particular flavor of a defense which is known to be suboptimal and easily attacked. The claim of state-of-the-art performance is outdated for BNNs.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes leveraging "information gain" with Bayesian Neural Networks to improve the robustness of Bayesian Neural Networks. Specifically, it maintains a set of particle models and uses SVGD to update this set of particle models. Furthermore, it uses the PGD-based attack to a specific particle model to craft adversarial examples and enforces the information gain of adversarial examples to be close to that of the corresponding benign examples. 
main_review: **Strengths of the paper**
- The information gain is useful to improve the robustness of Bayesian Neural Networks.
- The proposed method outperforms the baselines.

**Weaknesses of the paper**
- Although the proposed method is intuitive, its framework is not solidly and rigorously developed.
- Information gain seems identical to the mutual information $I(y,\theta \mid x, D)$ proposed and investigated in Bayesian Active Learning [1,2]. The authors need to discuss their proposed information gain and the mutual information proposed in [1,2].  

[1] Houlsby, Neil, et al. "Bayesian active learning for classification and preference learning." arXiv preprint arXiv:1112.5745 (2011).

[2]  Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep Bayesian active learning with image data. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1183–1192. JMLR. org, 2017.
summary_of_the_review: The posterior $p(\theta \mid D_{adv})$ in Eq. (5) is not mathematically rigorous because it does not contain $D$ on the left side, but $D$ is involved on the right side. 

I cannot see the definition of the information gain and it seems to be identical to the mutual information proposed in [1,2]. Moreover, the first term on the right side of  Eq. (8) is $H(y \mid x, D)$ because it is independent with $\theta$. The second term should be $E_\theta[H(y \mid x, \theta)]$ according to your derivation in Appendix.

SVGD is not proposed to minimize the loss function directly. Although the update of the particle models in Algorithm 1 is fine, it requires more effort to define the distribution for which we only know its unnormalized version. Furthermore, Eq. (9) needs to have the absolute value for the difference between $IG(x)$ and $IG(x_{adv})$. In addition, the notion here is inconsistent with the lack of $y$.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper explores adversarial training for Bayesian Neural Networks. In contrast to prior work which used Gaussian variational posteriors to estimate BNN parameters during adversarial training, this work uses Stein Variational Gradient Descent (SVGD) on a loss potential that includes an adversarial term similar to standard adversarial training and a term encouraging similarity between the Information Gain (IG) on natural and adversarial images. They evaluate PGD and adaptive adversarial attacks and find significant performance improvement over standard adversarial training and adversarial BNN. A theoretical justification is presented.
main_review: Strengths:
* The method appears to offer significant improvements over prior adversarial BNN defense.
* The authors implement an appropriate adaptive attack to account for the randomness of Bayesian Neural Networks. It might be helpful to refer to the attack with the widely-used terminology of Expectation-Over-Transformation (EOT) attack, as in Athalye et. al (2018). However, the authors place much more prominence in the text on a non-adaptive PGD attack, which might cause misperceptions (see Weaknesses).

Weaknesses:
* While the authors perform an appropriate adaptive EOT attack, the plots and main results present the outcomes of a non-adaptive PGD attack. It is well known that deterministic attacks are not representative of the performance of defenses that have stochastic elements, and the authors indeed see a significant drop in accuracy. Since a defense is only as good as its performance against an optimal attack, the authors should also include a curve showing robustness for their method against adaptive attacks in Figure 2 to give fair comparison versus adversarial training.
* The theoretical analysis appears to be quite trivial. Unless I am mistaken, adding the term $|E_\theta [ IG(x) ] - E_\theta [ IG(x_\text{adv}) ] |$ in (11) only makes the bound looser and thus less useful. The theoretical presentation does not enhance the arguments of the paper.
* The work lacks novelty, and its novel elements are not properly evaluated. The main contributions are the inclusion of SVGD and the Information Gain term. The theoretical importance of the Information Gain term is not very convincing. The authors do not perform an ablative study using just SVGD and no IG term vs. Gaussian variational inference plus IG term to narrow down which elements are important, or if both are necessary.
* The adaptive attack is not completely explored. The authors do not mention how many EOT replicates are used. 20 PGD steps is not enough for the EOT defense performance to reach its full strength in some cases where there is a high degree of stochasticity, and more steps should be used to ensure convergence.

Other Comments:
* How are the particles $\theta_i^0$ initialized during the testing phase? I would assume they are initialized from a set of fixed parameters identified during the training phase, is that correct? More details on this point would enhance my understanding of the work.
* Source code would help to further evaluate this defense.
summary_of_the_review: I recommend to not accept this paper. The theoretical analysis of the Information Gain term appears vacuous and distracting, although I could be missing something. The contributions (SVGD and Information Gain for BNN posterior sampling) are not significantly novel, and their relative importance is not properly evaluated. The experimental results appear strong, but it's difficult to judge the quality of the defense experiments without being able to inspect the to source code (in particular, the implementation of the EOT attack).
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors propose to learn a multi-modal posterior of the Bayesian neural network to defense the adversarial attacks, which can prevent the collapse and encourage the diversity accordingly. The authors further investigate the information gain for adversarial Bayesian learning, which is used to guide the training a BNN with the information gain bound. Experimental results demonstrate tis superior performance for adversarial attacks. 
main_review: In general, the paper is clearly written by addressing an important problem, but I still have several concerns.

1.	It is interesting that the authors propose to use the Bayesian formulation for adversarial training. Nevertheless, the authors are expected to make a thorough analysis since the benign examples and the adversarial examples are not independent. The authors are expected to clarify the reasonability of Eq. 5.
2.	To some extent, the paper is highly dependent on the information gain. However, it is nontrivial to estimate the information gain especially for the adversarial scenarios. A neural network is always overconfident for the adversarial examples. The authors are expected to clarify to how to estimate the information gain reliably. 
3.	The paper is inherently a regularized version for BNN with information gain. The authors are expected to clarify the novelties and technical contributions. 
4.	In the experimental section, the performance is significantly better than the alternative methods, e.g., Adv-BNN, especially for the adaptive attacks. The authors are expected to make thorough analysis on why or for what technical aspects it can achieve the performance improvement. 
5.	The authors are expected to make more comprehensive analysis, e.g., more experiments on ImageNet, more defense/attacks methods. 
   

summary_of_the_review: The paper is easy to follow, but the authors are expected to clarify their technical contributions and make more comprehensive experiments.
review_title: null

Meta-review: In this paper, the authors leverage information gain in conjunction with Bayesian Neural Networks in order to to improve the robustness of Bayesian Neural Networks. However, as pointed out by reviwers, there are several mistakes in theier derivations and evaluations. Moreover, the authors failed to crrectly refer to the exisiting work proposing similar methods.
Decision: Reject