Paper:
Improved Network Robustness with Adversary Critic
Ideally, what confuses neural network should be confusing to humans. However, recent experiments have shown that small, imperceptible perturbations can change the network prediction. To address this gap in perception, we propose a novel approach for learning robust classifier. Our main idea is: adversarial examples for the robust classifier should be indistinguishable from the regular data of the adversarial target. We formulate a problem of learning robust classifier in the framework of Generative Adversarial Networks (GAN), where the adversarial attack on classifier acts as a generator, and the critic network learns to distinguish between regular and adversarial images. The classifier cost is augmented with the objective that its adversarial examples should confuse the adversary critic. To improve the stability of the adversarial mapping, we introduce adversarial cycleconsistency constraint which ensures that the adversarial mapping of the adversarial examples is close to the original. In the experiments, we show the effectiveness of our defense. Our method surpasses in terms of robustness networks trained with adversarial training. Additionally, we verify in the experiments with human annotators on MTurk that adversarial examples are indeed visually confusing.

Review:
Work description The paper proposes a new method for improving robustness of NNs to adversarial attacks.  The work progresses towards distinguishing two types of errors. The first type is misclassification of (adversarial) images that may be in or out of the data manifold but are unlikely to be instances of other classes. The second type is misclassification of images that are likely instances of other classes. The second type of errors comes from the inherent overlap of the data and cannot be avoided (e.g. there are digits that are “in between” the classes and are difficult to classify for humans). The first type is the harmful one, which needs to be addressed. The main idea is to achieve that the net misclassifies a perturbed image only if it looks like the target class and is robust to all other perturbations. The proposed learning method explicitly forces the distribution of misclassified images for a given adversarial class to look like the true data samples of that class. The distance between these distributions leads to the regularizer which is expressed as a GAN pair in which the generator is the algorithm that finds adversarial examples leading to the misclassification with high confidence. The experiments on MNIST compare to SOTA methods and show improved robustness results. A human study is conducted to prove that the remaining adversarial perturbations are indeed confusing to the human (and thus are of the second type).  Evaluation I thank the authors for their work. The paper is very clear. It gives an excellent overview of existing methods and how they can be viewed as (approximate) robust optimization methods. The method is still a bit “heuristic” in the sense that the regularizing objectives are appended somewhat arbitrary. However, it is much sounder than many other proposals.  The experiments are on a simple dataset but cleanly made and giving a comparison to the baseline method and 3 robust training techniques. It is not clear what is the computational complexity of the method (i.e. are second order gradients involved?). However, the positive aspect is that the test-time model is the standard NN and is thus efficient. In my opinion the paper makes a good contribution and I see no problems. Clarity (minor) L 35-36: this is confusing since after adversarial training is introduced, it suggests training with adversarial examples that should be similar to regular images. This stays confusing in Fig 1 and lines 43-44. In Fig 1. it is not explained what is what. A good formulation when I finally got the idea is in lines 133-135. The design of the high confidence attack is perhaps not well explained. I see that without the high confidence the “generator” would only produce images on the classification boundary between the classes. Then enforcing the generator distribution to be close to the data distribution would actually enforce the model to have fractured boundaries. Instead it is somehow desirable to cover the whole adversarial target class, i.e. to also generate the images in its interior. What is the purpose of cycle consistency (7) is not clear. Discussion The difference between classifier and the critique is not completely clear. Could the critique be used as a classifier? Could the critique and the classifier be the same?  It seems that works on supervised learning formulations of generative models would be highly relevant. I.e. when one learns the joint mode of data and labels p(y,x). The objective (3) looks like the pseudo-likelihood learning of such generative model: the first part is the conditional likelihood p(y|x), modeled by the classifier, and the second part is the conditional likelihood p(x|y), modeled by the generator (which is the adversarial remapping of the true distribution). In that sense the method performs generative learning – fitting the joint distribution p(y,x) to the data. Such formulations are known to improve generalization properties with scarce data but achieve lower accuracy when more data is available. Alternatively to (3), is it possible to enforce the constraint that “images that are misclassified by the network may not look like the original class”? Technical In (1) max should be under the sum Minor L78: employ utilize  Final comments I believe the rebuttal addresses well all raised questions. I think it is an excellent work both conceptually and practically.