Paper:
Deep neural networks were shown to be vulnerable to crafted adversarial perturbations, and thus bring serious safety problems. To solve this problem, we proposed AE-GAN +rs , a framework for purifying input images by searching the closest natural reconstruction with little computation. We first build a reconstruction network AE-GAN, which adapted auto-encoder by introducing adversarial loss to the objective function. In this way, we can enhance the generative ability of decoder and preserve the abstraction ability of encoder to form a self-organized latent space. In the inference time, when given an input, we will start a search process in the latent space which aims to find the closest reconstruction to the given image on the distribution of natural data. The encoder can provide a good start point for the searching process, which saves much computation cost. Experiments show that our method is robust against various attacks and can reach comparable even better performance to similar methods with much fewer computations.

Review:
Summary: This paper proposes AE-GAN+sr, an auto-encoder based GAN for equipping neural networks with better defenses against adversarial attacks. The authors evaluate their method on black-box attacks, white-box attacks, and gray-box attacks on MNIST and Fashion-MNIST, and show decent empirical results when compared to baselines.
Decision: Reject. The writing was hard to follow and the experimental evaluations could have been stronger.
Supporting Arguments/Feedback: 
- There are a lot of phrases which should be reworded in the text (e.g. “it requires attention to” in paragraph 1, “detect if sample is from a normal distribution” at the end of Section 1, etc.) for better clarity. The writing was pretty hard to follow and the paper would really benefit overall after it has been improved.
- It was hard for me to tell from the text how exactly the models were implemented and trained (see Question #1). Making this point more clear, in either the main text or the Appendix, would be really helpful.
- The performance of AE-GAN+sr as evaluated on MNIST and FashionMNIST did not provide convincing evidence that it outperformed baselines such as Defense-GAN or adversarial training, though I did appreciate the authors’ additional analysis on the tradeoffs between computational cost and performance for the Defense-GAN. The results would also have been more compelling had the authors evaluated their method on more complex datasets such as CIFAR-10.
Questions:
- It wasn’t clear to me how the encoder-assisted search process (Section 3.4) works in practice. When you use gradient descent to find the best encoding for a corrupted input, how many steps do you need to take? Does this step happen while the autoencoder and GAN are being trained, or are those pre-trained and you just take additional gradient steps in the latent space?
- Given that the BiGAN also incorporates an encoder (and they were also benchmarked against in the experiments), where do the advantages of the AE-GAN+sr come from? I think making this point in the text would also be helpful as well.