Venue: ICLR 2018 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This paper propose a simple method for guarding trained models against adversarial attacks. The method is to prune the network’s activations at each layer and renormalize the outputs. It’s a simple method that can be applied post-training and seems to be effective.

The paper is well written and easily to follow. Method description is clear. The analyses are interesting and done well. I am not familiar with the recent work in this area so can not judge if they compare against SOTA methods but they do compare against various other methods.

Could you elaborate more on the findings from Fig 1.c Seems that  the DENSE model perform best against randomly perturbed images. Would be good to know if the authors have any intuition why is that the case.

There are some interesting analysis in the appendix against some other methods, it would be good to briefly refer to them in the main text.

I would be interested to know more about the intuition behind the proposed method. It will make the paper stronger if there were more content arguing analyzing the intuition and insight that lead to the proposed method.

Also would like to see some notes about computation complexity of sampling multiple times from a larger multinomial.

Again I am not familiar about different kind of existing adversarial attacks, the paper seem to be mainly focus on those from Goodfellow et al 2014. Would be good to see the performance against other forms of adversarial attacks as well if they exist.
review_title: Simple and yet effective method against adversarial attack post traning.

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper investigates a new approach to prevent a given classifier from adversarial examples. The most important contribution is that the proposed algorithm can be applied post-hoc to already trained networks. Hence, the proposed algorithm (Stochastic Activation Pruning) can be combined with algorithms which prevent from adversarial examples during the training.

The proposed algorithm is clearly described. However there are issues in the presentation.

In section 2-3, the problem setting is not suitably introduced.
In particular one sentence that can be misleading:
“Given a classifier, one common way to generate an adversarial example is to perturb the input in direction of the gradient…”
You should explain that given a classifier with stochastic output, the optimal way to generate an adversarial example is to perturb the input proportionally to the gradient. The practical way in which the adversarial examples are generated is not known to the player. An adversary could choose any policy. The only thing the player knows is the best adversarial policy.

In section 4, I do not understand why the adversary uses only the sign and not also the value of the estimated gradient. Does it come from a high variance? If it is the case, you should explain that the optimal policy of the adversary is approximated by “fast gradient sign method”. 

In comparison to dropout algorithm, SAP shows improvements of accuracy against adversarial examples. SAP does not perform as well as adversarial training, but SAP could be used with a trained network. 

Overall, this paper presents a practical method to prevent a classifier from adversarial examples, which can be applied in addition to adversarial training. The presentation could be improved.

review_title: overall, this paper presents a practical method to prevent a classifier from adversarial examples, which can be applied in addition to adversarial training. The presentation could be improved.

final_score: rating: 6: Marginally above acceptance threshold
review_content: The authors propose to improve the robustness of trained neural networks against adversarial examples by randomly zeroing out weights/activations. Empirically the authors demonstrate, on two different task domains, that one can trade off some accuracy for a little robustness -- qualitatively speaking.

On one hand, the approach is simple to implement and has minimal impact computationally on pre-trained networks. On the other hand, I find it lacking in terms of theoretical support, other than the fact that the added stochasticity induces a certain amount of robustness. For example, how does this compare to random perturbation (say, zero-mean) of the weights? This adds stochasticity as well so why and why not this work? The authors do not give any insight in this regard.

Overall, I still recommend acceptance (weakly) since the empirical results may be valuable to a general practitioner. The paper could be strengthened by addressing the issues above as well as including more empirical results (if nothing else).


review_title: Interesting heuristic but little theoretical justification

Meta-review: This is a borderline paper.  The reviewers are happy with the simplicity of the proposed method and the fact that it can be applied after training; but are concerned by the lack of theory explaining the results.  I will recommend accepting, but I would ask the authors add the additional experiments they have promised, and would also suggest experiments on imagenet.
Decision: Accept (Poster)