Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The contributions of the paper center on i) the introduction of diffusion-related tools for studying classifier decision boundaries; and ii) using those tools to connect decision boundary geometry, adversarial robustness, and generalization. The paper provides an analysis that provides insight into how curvature and local decision boundary geometry is influenced by adversarial defences, suggests a method for checking adversarial robustness, and then makes statements about model generalization based on geometric properties revealed by monte-carlo simulation of diffusions. 


Strengths:

The paper brings together beautiful areas of mathematics, probability, and random processes in an effort to characterize decision boundaries of DNNs and the behavior of DNNs on unseen examples, adversarial or otherwise. Curvature, heat diffusions, diffusion geometry, compression play a role. Feynman-Kac duality is leveraged to pass from the intractable analytical methods in the literature to stochastic simulations that can be undertaken by practitioners with data. The paper appears to make some novel links between generalization and decision boundary diffusion geometry, offers an apparently novel analysis of the impact of common adversarial defenses to Brownian adversaries, and at a minimum offers some new insights into how we might think about, and interpret, complex decision boundaries learned by neural nets or other nonlinear classifiers. The paper also follows up with experiments in the context of real applications and complex models. A series of Appendices provide technical details and experimental protocols.


Weaknesses:

The breadth of topics, the range of tools, steps, and quantities seems to have left the authors without enough space to get it all across. While the Appendices provide valuable details, additional definitions and discussion, the main body of the paper lacks clarity, context, and sufficient organization to allow an average reader to follow, or even appreciate, some of the arguments and key contributions. This is potentially the main weakness of the paper. If this were a longer journal submission, it might even make sense to separate the work into two papers: one exploring adversarial learning, and another exploring generalization.

The introduction and motivation sections of the paper could be improved by explicitly stating at a high level what the paper is contributing, and what insights will come out of the analyses. For example, the abstract states: “This leads to new insights concerning the "flattening-of-boundary" phenomenon.” What insights are in store, specifically? Imagine the paper were to be selected for a popular pod-cast, or a 5-minute lightning oral presentation at a conference: how would you distill it down to the essential contributions, components, and logical steps required to arrive at the key results?

Another concern is that while adversarial defense training by studying hitting times is valuable, it feels slightly misplaced/incomplete because an adversary might (often?) follow geodesics/geometry to very quickly obtain an error sample in cases where diffusion distances are high (e.g. a dumbbell). So while it’s helpful to be able to say that a random walk has a low probability of becoming an error sample from a point x, it doesn’t necessarily give us a guarantee about a “determined” adversary, who isn’t constrained at all to follow random walks (“Brownian attacks”).


Recommendation: 
Overall I recommend a borderline reject. The paper has some interesting ideas, and probably novel contributions, but needs revising for clarity and conciseness to be digestible or impactful. The paper needs to also be more specific about how the results relate to, leverage, and complement those in the literature/references. For example, the paper says in a few places that conclusions “agree with” reference [X]. Does this mean they have just rederived the same result, offering no additional insight? What is the significance of the agreement, and what does it provide to the community as a takeaway? It would be helpful to call out very concretely what is being contributed, and how it relates/differs relative to the literature.


Other suggestions:

- Perhaps clarify early on in the paper where the Brownian motion is happening. A reader might wonder: Is it in the ambient space? Is it on a graph? Is it on a manifold parameterized by some kind of intrinsic (or local) coordinates extracted from the model?

- Discuss the significance and interpretation of Lemmas 2.2 and 2.3. Why are they introduced? What are they saying intuitively, and how are they going to support your arguments later? There’s very little text around them (maybe due to a space problem and some ruthless trimming to make it all fit!)

- For the sake of completeness, define explicitly how you intend to handle ties in taking the argmax, and by extension, when defining E(y) (i.e. so that N is a subset of E(y)).

- Section 4 seems to have lost sight of the overall desired result, by omitting how L(g) controls L(f). Apologies if I’ve missed something obvious, but it seems like the strategy to control generalization of f in terms of hitting times is to pass to the compressed version, and control that. So where’s the link between generalization of f and generalization of g? Does Def 1 say it? 

- Eq (7) -- \phi_{E} should be introduced to make the definition self contained. Discuss why Def 1 is reasonable and what it means intuitively. The paper says it’s an “initial suggestion”. Why did you suggest this particular defn, over others?

- pg. 4 “isocapacitory” results: the analogy abruptly jumps from heat diffusion to “charge” accumulation. Try to link the two and provide a transition (without the reader having to refer to the lengthy appendices).

- It could be interesting to make a connection to heat diffusion classifiers (e.g. Szlam, 
Regularization on Graphs with Function-adapted Diffusion Processes, JMLR 2008), in which heat is instead diffused outward from the labels to classify new points, giving a potentially interesting characterization of the decision boundary (by construction). Maybe such methods can locally approximate more complex learning algorithms thereby providing global insight into their decision boundaries? 


UPDATE TO REVIEW FOLLOWING AUTHOR REVISIONS AND COMMENTS
---------------------------------------------------------------------------------------------------
I thank (and commend) the authors for their detailed, point-by-point responses.
The authors have made a good effort in their revisions to improve and clarify the exposition, and rectify the other comments made by the reviewers. The paper could still benefit from a deeper rewrite -- there's just so much that can be packed into a conference paper with limited real estate, and the authors are seeking to make several contributions under the umbrella of one submission (as the title suggests). So clarity suffers, and impact will suffer as a result. But, in my mind that shouldn't necessarily be a show stopper at this stage, in light of the revisions. I am therefore upgrading my recommendation.
review_title: Borderline reject; paper lacks coherence/clarity but might be improved with revisions.

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper under review introduces a number of geometric measures (isoperimetric, isocapacitory ratios that relate to Brownian motion or heat diffusion probabilities) that are applied to study neural network decision boundaries locally.  Specifically, the studies applying the measures to study adversarially trained NN empirically, and there are generalization and network compression bounds analytically proven that are derived that relate to Brownian motion probabilities.    Empirical observations on LeNet and Wide ResNet trained on MNIST and CIFAR showed adversarially trained or noise trained networks did exhibit curvature of the decision boundary, showing finer structure than previously known.

The paper is clear and the application of isoperimetric and isocapacitory measures appears novel, as well as the empirical finding of curvature (determined through the isocapacitory measure) of adversarially trained NNs provides some new insight into the shape of decision boundaries of NN for robustness.  I didn't find the empirical results very convincing: the isoperimetric / isocapacitory measures do not show a clear distinction for adversarially trained networks, which appears close to the ordinary trained networks.  Its not clear whether the training methods are not sufficient to produce a robust enough NN or the measures introduced do not adequately describe the adversarially trained nets.  All seem to exhibit curvature, but I'd assume the adversarially trained ones exhibit less curvature, which is not the case from the expt.
review_title: empirical results on geometric ratios not conclusive on adversarial trained networks

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: Title: HEATING UP DECISION BOUNDARIES: ISOCAPACITORY SATURATION, ADVERSARIAL SCENARIOS AND GENERALIZATION BOUNDS

Summary of the paper:

The idea of the paper is to introduce a new view on the geometry of the decision boundary of a classifier. Just as we may speak of the "margin" as in large margin methods and the hinge loss, the paper introduces the idea of a heat diffusion from the decision boundary - the amount of heat diffused to the data points gives a more subtle notion of stability. What a creative idea. Even cooler, the authors show that we may utilise the Feynman-Kac duality to now cast this in terms of the probability of a random walk starting at the data to hit the decision boundary. This is appealing because it differentiates between being near to an e.g. long thin decision boundary which approaches from just one side, to being completely surrounded by the decision boundary - something not accounted for by distance to the boundary.

Most of the paper is a really nice and gentle discussion of this idea. I gained some appreciation for the work even though it is extremely technical under the hood, and this is due to the nice writing in the main paper. To allow this the author's had to push their main result to the end of the paper (proposition 4.1) and leave the details to the appendix, but this is a fair trade-off in my opinion.

It is remarkable that in spite of the technicality involved, the authors manage to obtain proposition 4.1, an impressive first application of the new notions. 

The appendices are dense, but still nicely written and enjoyable even for the relatively uninitiated such as myself.

Pros:

This paper stands out as genuinely creative and novel. It is written with an enjoyable style which will surely motivate many theoreticians to delve into the details. I only wish I had the time and talent to do so myself.

While primarily theoretical and highly novel, the authors even include some thought provoking experimental results.

Cons:

One might argue whether the structure of the paper is ideal, but I would counter that the paper is extremely pleasant to read, and that it makes more sense to try to lure the reader into a curious mindset rather than bash them on the head with heavy details from the outset. Nicely done.

Recommendation:

I strongly recommend to accept this paper. Even if the details - which I have not checked - prove to have issues, the novelty of the work makes it a must-have in the portfolio of ideas included in the upcoming ICLR.

review_title: creative, novel and compelling

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper proposes an isocapacitory measure for analysing decision bound, in a way complementing the isoperimetric analysis proposed by Ford et al. 2019. The authors showed that the new measure captures different geometric properties of the decision boundary, potentially useful for adversarial training. The paper also proposed a new generalisation bound, although did not compare with other generalisation bounds.
  
The paper contains a number of interesting ideas and experiments. In general, the combination of heat diffusion and geometric analysis in the context of machine learning seems an interesting angle worth exploring.

The paper in its current form is a bit difficult to read. The key ideas are not clearly separated from the existing work. Some math notations are not clearly defined. In particular, many definitions rely on globally defined constants such as r and t, which has not been made clear to the reader. 

Below are some specific comments:

- Equations 2-5 should be clear that r and t are constants in the definitions or inequalities. The assumption r=sqrt(nt) should be made explicit. This affects the definition of c_n in Equation 4.
-  Equation 3 is incorrect. When mu is less than 0.5, the RHS is negative.





review_title: Idea interesting but writing can be improved

Meta-review: Four reviewers have reviewed this paper and after rebuttal, they were overall positive about the proposed idea. We congratulate authors on the paper.
Decision: Accept (Poster)