Heating up decision boundaries: isocapacitory saturation, adversarial scenarios and generalization bounds

1 INTRODUCTION AND BACKGROUND . The endeavor to understand certain geometric aspects of decision problems has lead to intense research in statistical learning . These range from the study of data manifolds , through landscapes of loss functions to the delicate analysis of a classifier ’ s decision boundary . In the present work we focus on the latter . So far , a wealth of studies has analyzed the geometry of decision boundaries of deep neural networks ( DNN ) , reaching profound implications in the fields of adversarial machine learning ( adversarial examples ) , robustness , margin analysis and generalization . Inspired by recent isoperimetric results and curvature estimates ( Ford et al . ( 2019 ) ; Moosavi-Dezfooli et al . ( 2019 ) ; Fawzi et al . ( 2016 ) ) , we attempt to provide some new aspects of decision boundary analysis by introducing and studying a corresponding diffusion-inspired approach . In this note the guiding idea is to place a heat source at the classifier ’ s decision boundary and estimate its size/shape in terms of the amount of heat the boundary is able to emit within a given time ( Fig . 1 ) . The goal is to extract geometric information from the behavior of heat transmission . This technique of heat content seems well-known within capacity/potential theory and has led to a variety of results in spectral analysis relating heat diffusion and geometry , Jorgenson & Lang ( 2001 ) ; Grigor ’ Yan & Saloff-Coste ( 2002 ) ; Maz ’ ya ( 2011 ) . However , working with such heat diffusion directly in terms of the corresponding differential equations is impractical . To this end , we note that , due to Feynman-Kac duality , the heat estimates are convertible to Brownian motion hitting probabilities . Thus we circumvent the need for solving intractable differential equations and instead are able to employ a straightforward Monte-Carlo sampling scheme in the ambient data space ( Section 3 ) . Background on defense training We apply the above analysis in the context of adversarial machine learning ( Section 4 ) where one studies the interaction between an adversary and a ML system . One of the goals of the subject is to design attack/defense training strategies improving the robustness of a given ML model - in the present work we are interested in how adversarial/noise defense training are reflected geometrically . Many different metrics to estimate robustness have been proposed : on one hand , there is adversarial robustness ( the probability that error samples lie very near a given data point x ) ; on the other hand , there is corruption robustness ( the probability of getting an error sample after perturbing a given data point x with some specified noise ) . In our context , heat diffusion naturally suggests a capacitory robustness metric : this metric is built upon the probability that Brownian motion started at a given data point x will hit error samples within a given time window . One can perceive this metric as a combination of adversarial and noise robustness ( Brownian motion has continuous paths and specified stopping time determined by boundary impact ) . In this perspective , our work is aligned with studies of other robustness metrics and curvature results ( cf . Fawzi et al . ( 2016 ) for a `` semi-random '' projection robustness and relations to curvature ) . We study the capacitory metric on the well-known CIFAR10 and MNIST datasets and observe that defense training techniques may either yield a certain ( although not substantial ) decrease ( noise training ) or fail to have a significant effect on continuous Brownian attacks overall . Surprisingly , in both cases the studied capacitory metric does not converge to the corresponding value as in the case of a flat decision boundary . Due to our comparison statements and curvature considerations , this means that locally around clean data points the geometry is in general flattened out but may still retain complexity and substantial areas of ( small ) non-vanishing curvature . In other words , from the point of view of our heat diffusion metrics , decision boundaries locally exhibit non-flat behaviour . Background on generalization estimates Finally , we observe that the collected heat/hittingprobability metrics can further be used to obtain generalization bounds where , in a nutshell , one evaluates the performance of a model on unseen data in terms of the performance over a given sampled data , the model ’ s expressiveness , dimension , etc . In this regard , we view decision boundary heat diffusion traits as an indicator of how noise-stable a given model is - this relates Brownian hitting bounds with recent compression-based generalization techniques in the spirit of Arora et al . ( 2018 ) ; Suzuki et al . ( 2018 ; 2020 ) . More precisely , we proceed in two steps : first , we construct a `` smaller '' compressed model that is almost equivalent to the initial one in an appropriate heat-theoretic way ; second , we obtain generalization estimates for the smaller model in terms of the decision boundary hitting probabilities ( computed on the empirical dataset ) . Furthermore , the bounds are significantly improved under additional geometric assumptions on the decision boundary of the initial model . Additional related work The interplay between heat diffusion and geometry lies at the heart of many topics in geometric analysis and spectral theory ( cf . Jorgenson & Lang ( 2001 ) ; Grigor ’ Yan ( 2001 ) for a far reaching overview ) . Some direct applications of heat diffusion techniques to zero sets of eigenfunctions are seen , for example , in Steinerberger ( 2014 ) ; Georgiev & Mukherjee ( 2018a ; b ) . The literature on adversarial ML is vast : to name a few central works in the field , we refer to Dalvi et al . ( 2004 ) ; Biggio & Roli ( 2018 ) ; Szegedy et al . ( 2014 ) . Much effort has been invested in designing and understanding strategies that will render a model robust to various attacks ( e.g . Madry et al . ( 2018 ) ; Carlini & Wagner ( 2017 ) ) . In particular , the geometry of decision boundaries has been the focus of many works in the subject leading to breakthroughs in curvature estimates , boundary flatness and robustness , schemes for detecting boundary complexity , proposing adversarial attacks/defenses and diffusion based techniques towards constructing decision boundary from partially pre-labelled data ( e.g . Ford et al . ( 2019 ) ; Fawzi et al . ( 2016 ; 2017 ; 2018 ) ; Dezfooli et al . ( 2018 ) ; Moosavi-Dezfooli et al . ( 2019 ) ; Karimi et al . ( 2019 ) ; Karimi & Tang ( 2020 ) ; He et al . ( 2018 ) ; Szlam et al . ( 2008 ) ) . The theory of generalization bounds has formed a classical main line of ML and statistical inference research ( Vapnik ( 1999 ) ) . In this direction central questions address the generalization properties of heavily over-parametrized deep neural network models . According to some classical VC-dimension results such models should overfit the data and generalize poorly . Extensive research effort has been invested in developing appropriate sharper techniques to explain generalization of DNN models : on one hand there are the methods based on norm estimation whose bounds are not explicitly using the number of the network ’ s parameters ( see Golowich et al . ( 2019 ) ; Neyshabur et al . ( 2015 ; 2018 ) ; Wei & Ma ( 2019 ) ; Bartlett et al . ( 2017 ) , etc ) . On the other hand , recent results based on compression and VC-dimension can lead to sharper bounds ( Arora et al . ( 2018 ) ; Suzuki et al . ( 2018 ; 2020 ) ) . 2 CONTRIBUTIONS , CONTEXT AND PAPER OUTLINE . An outline of our essential contributions is given as follows : 1 . We analyze decision boundary geometries in terms of novel heat diffusion and Brownian motion techniques with thorough theoretical estimates on curvature and flattening . 2 . We show , both theoretically and empirically ( in terms of adversarial scenarios on stateof-art DNN models ) , that the proposed heat diffusion metrics detect the curvature of the boundary ; they complement , and in some respects are more sensitive in comparison to previous methods of boundary analysis - intuitively , our heat driven metrics are sharper on a finer scale and can detect small-scale `` wiggles and pockets '' . As an application , we are thus able to provide evidence that adversarial defenses lead to overall flatter boundaries but , surprisingly , the heat traits do not converge to the corresponding flat-case , and hence , finer-scale non-linear characteristics ( e.g . `` wiggles and pockets '' ) are persistent . 3 . Moreover , the preservation of `` wiggles and pockets '' means that susceptibility to naive Brownian motion attacks is not significantly decreased via adversarial defense mechanisms . 4 . Finally , we introduce a novel notion of compression based on heat diffusion and prove that stability of heat signature translates to compression properties and generalization capabilities . In terms of context , the present note is well-aligned with works such as Ford et al . ( 2019 ) ; Dezfooli et al . ( 2018 ) ; Fawzi et al . ( 2016 ; 2018 ) . Among other aspects , these works provide substantial analysis of the interplay between geometry/curvature and adversarial robustness/defenses - in particular , we use some of the these tools ( e.g . isoperimetric saturation ) as benchmarks and sanity checks . However , in contrast , in our work we provide a non-equivalent technique to address decision boundary geometry for which we provide an extensive theoretical and empirical evaluation with insights on the preservation of finer-scale traits . Intuitively , previous distance-based geometric methods could be considered as a `` coarser lens '' , whereas the present heat-diffusion tools appear to be much more sensitive . As a large-scale example , Brownian particles emanating from a point are able to distinguish between a decision boundary which is a hyperplane at distance d and a decision boundary which is a cylinder of radius d wrapping around the point . Our notion of compression is inspired by Arora et al . ( 2018 ) , and establishes a connection between the Johnson-Lindenstrauss dimension reduction algorithm with diffusion techniques . Furthermore , we bridge the proposed heat-theoretic techniques with generalization bounds in the spirit of Arora et al . ( 2018 ) ; Suzuki et al . ( 2020 ) . In particular , this shows that overall lower heat quantities at sample points imply better generalization traits . A step-wise road map of the present work is given below : • ( Subsection 3.1 ) We start by discussing what heat diffusion is and how it is to be evaluated - here we discuss that , via Feynman-Kac duality , one can essentially work with Brownian motion hitting probabilities . • ( Subsections 3.2 and 3.3 ) We introduce the isocapacitory saturation τ - a heat-theoretic metric that will be used to estimate boundary flatness . Moreover , here we emphasize the properties of τ such as relations to curvature ( Proposition 3.1 ) and the novel information obtained from heat theoretic methods in comparison to previous distance-based ones . • ( Subsection 3.4 ) We compute τ for certain geometric model cases such as hyperplanes , cones , wedges and `` spiky '' sets ( Lemmas 3.2 and 3.3 ) . This allows us later to evaluate how much a given geometry resembles these model cases . • ( Section 4 ) Next , we are in a position to evaluate and compare τ for decision boundaries of DNNs . We experimentally illustrate the effect of adversarial defense mechanisms and noise robustness on τ ( PGD/FGSM on MNIST and CIFAR-10 ) . • ( Section 5 ) We prove that heat transmission relates to generalization bounds ( Propositions 5.1 and 5.2 ) - in particular , lower levels of heat at sample points yield sharper generalization bounds . Finally , we complete the discussion by informally stating our compression scheme . • ( Appendix ) Our methods leverage several tool sets extensively . For this reason our goal in the main text is to only collect and showcase the techniques and results . However , the thorough in-depth analysis is provided in the Appendix where the reader can find all relevant proofs and further background and references .

Summary: The paper under review introduces a number of geometric measures (isoperimetric, isocapacitory ratios that relate to Brownian motion or heat diffusion probabilities) that are applied to study neural network decision boundaries locally.  Specifically, the studies applying the measures to study adversarially trained NN empirically, and there are generalization and network compression bounds analytically proven that are derived that relate to Brownian motion probabilities.    Empirical observations on LeNet and Wide ResNet trained on MNIST and CIFAR showed adversarially trained or noise trained networks did exhibit curvature of the decision boundary, showing finer structure than previously known.