Venue: ICLR 2018 conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: In this manuscript, the authors empirically investigated the robustness of some different deep neural networks ensembles to two types of attacks, namely FGSM and BIM, on two popular datasets, MNIST and CIFAR10. The authors concluded that the ensembles are more accurate on both clean and adversaries samples than a single deep neural network. Therefore, the ensembles are more robust in terms of the ability to correctly classify the adversary attacks.

As the authors stated, an attack that is designed to fool one network does not necessarily fool the other networks in the same way. This is likely why ensembles appear more robust than single deep learners. However, robustness of ensembles to the white-box attacks that are generated from the ensemble is still low for FGS. Generally speaking, although FGS attacks generated from one network can fool less the whole ensembles, generating FGS adversaries from a given ensemble is still able to effectively fool it. Therefore, if the attacker has access to the ensemble or even know the classification system based on that ensemble, then the ensemble-based system is still vulnerable to the attacks generated specifically from it. Simple ensemble methods are not likely to confer significant robustness gains against adversaries.

In contrast to FGS results, surprisingly BIM-Grad1 is able to fool more the ensemble than BIM-Grad2. Therefore, it seems that if the attacker makes BIM adversaries from only a single classifier, then she can simply and yet effectively mislead the whole ensemble. In comparison to BIM-Grad2, BIM-Grad1 results show that BIM attacks from one network (BIM-Grad1) can more successfully fool the other different networks in the ensembles in a similar way! BIM-Grad2 is not that much able to fool the ensemble-based system even this attack generated from the ensemble (white-box attacks). In order to confirm the robustness of the ensembles to BIM attacks, the authors can do more experiments by generating BIM-Grad2 attacks with higher number of iterations.

Indeed, the low number of iterations might cause the lower rate of success for generating adversaries by BIM-Grad2. In fact, BIM adversaries from the ensembles might require more number of iterations to effectively fool the majority of the members in the ensembles. Therefore, increasing the number of iterations can increase the successful rate of generating BIM-Average Grad2 adversaries. Note that in this case, it is recommended to compare the amount of distortion (perturbation) with different number of iterations in order to indicate the effectiveness of the ensembles to white-box BIM attacks.

Despite to averaging the output probabilities to compute the ensemble final prediction, the authors generated the adversaries from the ensemble by computing the sum of the gradients of the classifiers loss. A proper approach would have been to average of these gradients. The fact the sum is not divided by the number of members (i.e., sum of gradients instead of average of gradients) is increasing the step size of the adversarial method proportionally to the ensemble size, raising questions on the validity of the comparison with the single-model adversarial generation.

Overall, I found the paper as having several methodological flaws in the experimental part, and rather light in terms of novel ideas. As noticed in the introduction, the idea of using ensemble for enhancing robustness as already been proposed. Making a paper only to restate it, is too light for acceptation. Moreover, experimental setup using a lot of space for comparing results on standard datasets (i.e., MNIST and CIFAR10), even with long presentation of these datasets. Several issues are raised in the current experiments and require adjustments. Experiments should also be more elaborated to make the case stronger, following at least some of indications provided. 

review_title: Simple but effective idea, light in the presentation and experimentations 

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary: This paper proposes to use ensembling as an adversarial defense mechanism. The defense is evaluated on MNIST and CIFAR10 ans shows reasonable performance against FGSM and BIM.

Clarity: The paper is clearly written and easy to follow. 

Originality: Building an ensemble of models is a well-studied strategy that was shown long ago to improve generalization. As far as I know, this paper is however the first to empirically study the robustness of ensembles against adversarial examples. 

Quality: While this paper contributes to show that ensembling works reasonably well against adversarial examples, I find the contribution limited in general.
- The method is not compared against other adversarial defenses. 
- The results illustrate that adding Gaussian noise on the training data clearly outperforms the other considered ensembling strategies. However, the authors do not go beyond this observation and do not appear to try to understand why it is the case. 
- Similarly, the Bagging strategy is shown to perform reasonably well (although it appears as a weaker strategy than Gaussian noise) but no further analysis is carried out. For instance, it is known that the reduction of variance is maximal in an ensemble when its constituents are maximally decorrelated. It would be worth studying more systematically if this correlation (or 'diversity') has an effect on the robustness against adversarial examples. 
- I didn't understand the motivation behind considering two distinct gradient estimators. Why deriving the exact gradient of an ensemble is more complicated?

Pros: 
- Simple and effective strategy.
- Clearly written paper. 
Cons:
- Not compared against other defenses.
- Limited analysis of the results. 
- Ensembling neural networks is very costly in terms of training. This should be considered.

Overall, this paper presents an interesting and promising direction of research. However, I find the current analysis (empirically and/or theoretically) to be too limited to constitutes a solid enough piece of work. For this reason, I do not recommend this paper for acceptance. 
review_title: A solid and effective idea, but a limited analysis. 

final_score: rating: 7: Good paper, accept
review_content: This paper describes the use of ensemble methods to improve the robustness of neural networks to adversarial examples. Adversarial examples are images that have been slightly modified (e.g. by adding some small perturbation) so that the neural network will predict a wrong class label.

Ensemble methods have been used by the machine learning community since long time ago to provide more robust and accurate predictions.

In this paper the authors explore their use to increase the robustness of neural networks to adversarial examples.

Different ensembles of 10 neural networks are considered. These include techniques such as bagging or injecting noise in the 
training data. 

The results obtained show that ensemble methods can sometimes significantly improve the robustness against adversarial examples. However,
the performance of the ensemble is also highly deteriorated by these examples, although not as much as the one of a single neural network.

The paper is clearly written.

I think that this is an interesting paper for the deep learning community showing the benefits of ensemble methods against adversarial
examples. My main concern with this paper is the lack of comparison with alternate techniques to increase the robustness against adversarial examples. The authors should have compared with the methods described in:

(Goodfellow et al., 2014; Papernot et al., 2016c), 
(Papernot et al., 2016d) 
(Gu & Rigazio, 2014)

Furthermore, the ensemble approach has the main disadvantage of increasing the prediction time by a lot. For example, with 10 elements in the ensemble, predictions are 10 times more expensive.
------------------------------
I have read the updated version of the paper. I think the authors have done a good job comparing with related techniques. Therefore, I have slightly increased my score.

review_title: A simple technique missing comparison with related ones.

Meta-review: The paper empirically evaluates the effectiveness of ensembles of deep networks against adversarial examples. The paper adds little to the existing literature in this area: an detailed study on "ensemble adversarial training" already exists, and the experimental evaluation in this paper is limited to MNIST and CIFAR (results on those datasets do not necessarily transfer very well to much higher-dimensional datasets such as ImageNet). Moreover, the reviewers identify several shortcomings in the experimental setup of the paper. 
Decision: Reject