Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: This paper presents a new discriminator metric for adversarial attack's detection by deriving the different properties of l-th neuron network layer on different adv/benign samples. This method can achieve good AUC score comparing to other start-of-art detection methods and also achieve good robustness under corresponding adaptive attack. The framework is clear and the experiment is solid.

However, I have several concerns:

Major:
1. It seems that the whole process assumes that there is difference for the parameters in the environment of GGD with adv/benign samples, and the goal is to search for the major components of it and use a classifier to detect. To extract the approximation of parameters,  the authors use the "response entries" of l-th layer for several observations => this means the authors regard all the "response entries" of one layer as different samples on one certain GGD. This makes me feel a bit tricky, and it would be great if you the authors can provide some evidence or explanation here.

2. In the experiment's remark, the authors mentioned that the mean parameter of GGD is set to 0 and most of them are actually close to 0 (around 1e-2) so the assumption is right. However 1e-2 is not a value "very close to zero" and it would be great to show / explain the variance here.

3. I can't find the parameter of your evaluated attack method (like confidence, eps, etc.) Please also provide experimental details for reproducibility. 

Minor:

Here are some reference error (e.g. P6, "For each database, as described in Section ??"). Please fix that.

Overall, this paper is a interesting based on the performance of detection. But the  assumptions made by the paper are a bit confusing and it would be good to clarify and provide clarification for them. Authors should explain the assumptions and give some extra experiment results if needed.

review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This paper proposes an approach to adversarial detection.  The approach first computes a representation of the activation layers using the Benford-Fourier coefficients.  One then generates a range of noisy instances, and trains an SVM using those noisy instances as supervised labels (e.g., noisy instances are adversarial).  The SVM uses the Benford-Fourier coefficients of the activation layer as the input features.  The results show good performance against some baselines such as LID.

I'm not really an expert in this area, but I'm a bit surprised that LID is considered the baseline to beat.  I imagine that most adversarial defense approaches are for robust prediction, rather than detection.  It also seems the authors chose to compare with defenses that are computationally cheaper (so not RCE or Defense-GAN), but a study of computational trade-offs is absent in the paper.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper proposes an adversarial detection method via Fourier coefficients. The proposed method seems promising, and empirical evaluations are reasonable.

However, I find that the proposed MBF detection metric is much more complicated to calculate than any of its baselines, e.g., LID or K-density. So I wonder if the good performance of MBF mainly comes from its ’complexity‘. I mean, if we use the feature vectors of different layers in CNNs and combine them with a different non-linear function and feed into an SVM classifier,  can we still obtain a hard-to-evade detector? I think a fair complexity is particularly important when you try to evade the detector by optimization-based adaptive attacks and claim superiority over other baselines.
review_title: Official Blind Review #3

Meta-review: This paper presents a new metric for adversarial attack's detection. The reviewers find the idea interesting, but the some part has not been clearly explained, and there are questions on the reproducibility issue of the experiments. 
Decision: Reject