Venue: ICLR 2018 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper tried to analyze the subspaces of the adversarial examples neighborhood. More specifically, the authors used Local Intrinsic Dimensionality to analyze the intrinsic dimensional property of the subspaces. The characteristics and theoretical analysis of the proposed method are discussed and explained. This paper helps others to better understand the vulnerabilities of DNNs.
review_title: Not familiar with this research topic.

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The paper considers a problem of adversarial examples applied to the deep neural networks. The authors conjecture that the intrinsic dimensionality of the local neighbourhood of adversarial examples significantly differs from the one of normal (or noisy) examples. More precisely, the adversarial examples are expected to have intrinsic dimensionality much higher than the normal points (see Section 4).  Based on this observation they propose to use the intrinsic dimensionality as a way to separate adversarial examples from the normal (and noisy) ones during the test time. In other words, the paper proposes a particular approach for the adversarial defence.

It turns out that there is a well-studied concept in the literature capturing the desired intrinsic dimensionality: it is called the local intrinsic dimensionality (LID, Definition 1) . Moreover, there is a known empirical estimator of LID, based on the k-nearest neighbours. The authors propose to use this estimator in computing the intrinsic dimensionalities for the test time examples. For every test-time example X the resulting Algorithm 1 computes LID estimates of X activations computed for all intermediate layer of DNN. These values are finally used as features in classifying adversarial examples from normal and noisy ones. 

The authors empirically evaluate the proposed technique across multiple state-of-the art adversarial attacks, 3 datasets (MNIST, CIFAR10, and SVHN) and compare their novel adversarial detection technique to 2 other ones recently reported in the literature. The experiments support the conjecture mentioned above and show that the proposed technique *significantly* improves the detection accuracy compared to 2 other methods across all attacks and datasets (see Table 1).

Interestingly, the authors also test whether adversarial attacks can bypass LID-based detection methods by incorporating LID in their design. Preliminary results show that even in this case the proposed method manages to detect adversarial examples most of the time. In other words, the proposed technique is rather stable and can not be easily exploited.

I really enjoyed reading this paper. All the statements are very clear, the structure is transparent and easy to follow. The writing is excellent. I found only one typo (page 8, "We also NOTE that..."), otherwise I don't actually have any comments on the text.

Unfortunately, I am not an expert in the particular field of adversarial examples, and can not properly assess the conceptual novelty of the proposed method. However, it seems that it is indeed novel and given rather convincing empirical justifications, I would recommend to accept the paper. 

review_title: Intrinsic dimensionality around the adversarial example is very different from the one of normal or noisy data

final_score: rating: 7: Good paper, accept
review_content: The authors clearly describe the problem being addressed in the manuscript and motivate their solution very clearly. The proposed solution seems very intuitive and the empirical evaluations demonstrates its utility. My main concern is the underlying assumption (if I understand correctly) that the adversarial attack technique that the detector has to handle needs to be available at the training time of the detector. Especially since the empirical evaluations are designed in such a way where the training and test data for the detector are perturbed with the same attack technique. However, this does not invalidate the contributions of this manuscript.

Specific comments/questions:
- (Minor) Page 3, Eq 1: I think the expansion dimension cares more about the probability mass in the volume rather than the volume itself even in the Euclidean setting.
- Section 4: The different pieces of the problem (estimation, intuition for adversarial subspaces, efficiency) are very well described.
- Alg 1, L3: Is this where the normal exmaples are converted to adversarial examples using some attack technique? 
- Alg 1, L12: Is LID_norm computed using a leave-one-out estimate? Otherwise, r_1(.) for each point is 0, leading to a somewhat "under-estimate" of the true LID of the normal points in the training set. I understand that it is not an issue in the test set.
- Section 4 and Alg 1: S we do not really care about the "labels/targets" of the examples. All examples in the dataset are considered "normal" to start with. Is this assuming that the "initial training set" which is used to obtain the "pre-trained DNN" free of adversarial examples?
- Section 5, Experimental Setup: Seems like normal points in the test set would get lesser values if we are not doing the "leave-one-out" version of the estimation.
- Section 5: The authors have done a great job at evaluating every aspect of the proposed method.

review_title: Intuitive solution to important problem; well written 

Meta-review: The paper characterizes the latent space of adversarial examples and introduces the concept of local intrinsic dimenstionality (LID). LID  can be used to detect adversaries as well build better attacks as it characterizes the space in which DNNs might be vulnerable. The experiments strongly support their claim.
Decision: Accept (Oral)