Venue: ICLR 2018 conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The authors have undertaken a large scale empirical evaluation on sensitivity and generalization for DNNs within the scope of image classification. They are investigating the suitability of the F-norm of the input-output Jacobian in large scale DNNs and they evaluate sensitivity and generalization metrics across the input space, both on and of the data manifold. They convincingly present strong empirical evidence for the F-norm of the Jacobian to be predictive and informative of generalization of the DNN within the image classification domain.

The paper is well written. The problem is clearly presented and motivated. Most potential questions of a reader as well as interesting details are supplied by footnotes and the appendix.
The contributions are to my knowledge both novel and significant.
The paper seem to be technically correct. The methodology and conclusions are reasonable.
I believe that this is important work and applaud the authors for undertaking it. I hope that the interesting leads will be further investigated and that similar studies will be conducted beyond the scope of image classification. '
The research and further investigations would be strengthened if they would include a survey on the networks presented in the literature in a similar manner as the authors did with the generated networks within the presented study. For example compare networks from benchmark competitions in terms of sensitivity and generalization using the metrics presented here.

Please define "generalization gap" and show how you calculate/estimate it. The term us used differently in much of the machine learning literature(?). Given this and that the usually sought after generalization error is unobtainable due to the unknown joint distribution over data and label, it is necessary to clarify the precise meaning of "generalization gap" and how you calculated it. I intuitively understand but I am not sure that the metric I have in mind is the same as the one you use. Such clarification will also improve the accessibility for a wider audience.

Figure 4:
I find Figure 4:Center a bit confusion. Is it there to show where on the x-axis of Figure 4:Top,the three points are located? Does this mean that the points are not located at pi/3, pi, 5pi/3 as indicated in the figure and the vertical lines of the figure grid? If it is not, then is it maybe possible to make the different sub-figure in Figure 4 more distinctive, as to not visually float into each other?

Figure 5:
The figure makes me curious about what the regions look like close to the training points, which is currently hidden by the content of the inset squares. Maybe the square content can be made fully transparent so that only the border is kept? The three inset squares could be shown right below each sub-figure, aligned at the x-axis with the respective position of each of the data points.
review_title: The authors present a large scale empirical evaluation on sensitivity and generalization for DNNs within the scope of image classification. They convincingly present strong empirical evidence for the F-norm of the input-output Jacobian to be predictive/informative of generalization with large scale DNNs.

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposes an analysis of the robustness of deep neural networks with respect to data perturbations. 

*Quality*
The quality of exposition is not satisfactory. Actually, the paper is pretty difficult to evaluate at the present stage and it needs a drastic change in the writing style.

*Clarity*
The paper is not clear and highly unstructured.

*Originality* 
The originality is limited for what regards Section 3: the proposed metrics are quite standard tools from differential geometry. Also, the idea of taking into account the data manifold is not brand new since already proposed in “Universal Adversarial Perturbation” at CVPR 2017.

*Significance*
Due to some flaws in the experimental settings, the relevance of the presented results is very limited. First, the authors essentially exploit a customized architecture, which has been broadly fine-tuned regarding hyper-parameters, gating functions and optimizers. Why not using well established architectures (such as DenseNets, ResNets, VGG, AlexNet)? 
Moreover, despite having a complete portrait of the fine-tuning process is appreciable, this compromises the clarity of the figures which are pretty hard to interpret and absolutely not self-explanatory: probably it’s better to only consider the best configuration as opposed to all the possible ones.
Second, authors assume that circular interpolation is a viable way to traverse the data manifold. The reviewer believes that it is an over-simplistic assumption. In fact, it is not guaranteed a priori that such trajectories are geodesic curves so, a priori, it is not clear why this could be a sound technique to explore the data manifold.

CONS:
The paper is difficult to read and needs to be thoroughly re-organized. The problem is not stated in a clear manner, and paper’s contribution is not outlined. The proposed architectures should be explained in detail. The results of the sensitivity analysis should be discussed in detail. The authors should explain the approach of traversing the data manifold with ellipses (although the reviewer believes that such approach needs to be changed with something more principled). Figures and results are not clear.
The authors are kindly asked to shape their paper to match the suggested format of 8 pages + 1 of references (or similar). The work is definitely too long considered its quality. Additional plots and discussion can be moved to an appendix.
Despite the additional explanation in Footnote 6, the graphs are not clear. Probably authors should avoid to present the result for each possible configuration of the hyper-parameters, gatings and optimizers and just choose the best setting.
Apart from the customized architecture, authors should have considered established deep nets, such as DenseNets, ResNets, VGG, AlexNet.
The idea of considering the data manifold within the measurement of complexity is a nice claim, which unfortunately is paired with a not convincing experimental analysis. Why ellipses should be a proper way to explore the data manifold? In general, circular interpolation is not guaranteed to be geodesic curves which lie on the data manifold.

Minor Comments: 
Sentence to rephrase: “We study common in the machine learning community ways to ...”
Please, put the footnotes in the corresponding page in which it is referred.
The reference to ReLU is trivially wrong and need to be changed with [Nair & Hinton ICML 2010]

**UPDATED EVALUATION AFTER AUTHORS' REBUTTAL**
We appreciated the effort in providing specific responses and we also inspected the updated version of the paper. Unfortunately, despite the authors' effort, the reviewer deems that the conceptual issues that have been highlighted are still present in the paper which, therefore, is not ready for acceptance yet.   
review_title: The paper is poorly organized, the presented analysis is hard to read, the obtained results (and mainly all figures) are unclear. 

final_score: rating: 5: Marginally below acceptance threshold
review_content: This work investigates sensitivity and generalisation properties of neural networks with respect to a number of metrics aimed at quantifying the robustness with respect to data variability, varying parameters and representativity of training/testing data. 
The validation is based on the Jacobian of the network, and in the detection of the “transitions” associated to the data space. These measures are linked, as the former quantifies the sensitivity of the network respect to infinitesimal data variations, while the latter quantifies the complexity of the modelled data space. 
The study explores a number of experimental setting, where the behaviour of the network is analysed on synthetic paths around training data, from pure random data points, to curves interpolating different/same data classes.
The experimental results are performed on CIFAR10,CIFAR100, and MNIST. Highly-parameterised networks seem to offer a better generalisation, while lower Jacobian norm are usually associated to better generalisation and fewer transitions, and can be obtained with data augmentation.

The paper proposes an interesting analysis aimed at the empirical exploration of neural network properties, the proposed metrics provide relevant insights to understand the behaviour of a network under varying data points. 

Major remarks.

The proposed investigation is to my opinion quite controversial. Interesting data variation does not usually corresponds to linear data change. When considering the linear interpolation of training data, the authors are actually creating data instances not compatible with the original data source: for example, the pixel-wise intensity average of digits is not a digit anymore. For this reason, the conclusions drawn about the model sensitivity are to my opinion based a potentially uninteresting experimental context. Meaningful data variation can be way more complex and high-dimensional, for example by considering spatial warps of digits, or occlusions and superpositions of natural images. This kind of variability is likely to correspond to real data changes, and may lead to more reliable conclusions. For this reason, the proposed results may provide little indications of the true behaviour of the models data in case of meaningful  data variations. 

Moreover, although performed within a cross-validation setting, training and testing are still applied to the same dataset. Cross-validation doesn’t rule out validation bias, while it is also known that the classification performance significantly drops when applied to independent “unseen” data, provided for example in different cohorts. I would expect that highly parameterised models would lead to worse performance when applied to genuinely independent cohorts, and I believe that this work should extend the investigation to this experimental setting.

Minor remarks.

The authors should revise the presentation of the proposed work. The 14 figures(!) of main text are not presented in the order of appearance. The main one (figure 1) is provided in the first paragraph of the introduction and never discussed in the rest of the paper. 

review_title: Interesting experimental setting and validation metrics. Unclear presentation and modelling rationale

Meta-review: Reviewers always find problems in papers like this.

AnonReviewer1 would have preferred to have seen a study of traditional architectures, rather than fully connected ones, which are now less frequently used. They thought the paper was too long, the figures too cluttered, and were not convinced by the discussion around linear v. elliptical trajectories.

I appreciate the need for a parametrizable architecture, although it may not be justified to translate these insights to other architectures, and then the fact that fully connected architectures are less common undermines the impact of the work. I don't find the length a problem, and I don't find the figures a problem.

After the back and forth, AnonReviewer3 believes that there are data compatibility issues associated with the studied transformations and that non-linear transformations would have been more informative. I find the reviewers response to be convincing.

AnonReviewer2 is strongly in favor of acceptance, finding the work exhaustive, interesting, and of high quality. I'm inclined to agree.


Decision: Accept (Poster)