On the Decision Boundaries of Deep Neural Networks: A Tropical Geometry Perspective

1 INTRODUCTION . Deep Neural Networks ( DNNs ) have demonstrated outstanding performance across several research domains , including computer vision ( Krizhevsky et al. , 2012 ) , speech recognition ( Hinton et al. , 2012 ) , natural language processing ( Bahdanau et al. , 2015 ; Devlin et al. , 2018 ) , quantum chemistry ( Schütt et al. , 2017 ) , and healthcare ( Ardila et al. , 2019 ; Zhou et al. , 2019 ) to name a few ( LeCun et al. , 2015 ) . Nevertheless , a rigorous interpretation of their success remains evasive ( ShalevShwartz & Ben-David , 2014 ) . For instance , and in an attempt to uncover the expressive power of DNNs , Montufar et al . ( 2014 ) studied the complexity of functions computable by DNNs that have piecewise linear activations . They derived a lower bound on the maximum number of linear regions . Several other works have followed to improve such estimates under certain assumptions ( Arora et al. , 2018 ) . In addition , and in attempt to understand some of the subtle behaviours DNNs exhibit , e.g . the sensitive reaction of DNNs to small input perturbations , several works directly investigated the decision boundaries induced by a DNN used for classification . The work of SeyedMohsen Moosavi-Dezfooli ( 2019 ) showed that the smoothness of these decision boundaries and their curvature can play a vital role in network robustness . Moreover , He et al . ( 2018a ) studied the expressiveness of these decision boundaries at perturbed inputs and showed that these boundaries do not resemble the boundaries around benign inputs . Li et al . ( 2018 ) showed that under certain assumptions , the decision boundaries of the last fully connected layer of DNNs will converge to a linear SVM . Also , Beise et al . ( 2018 ) showed that the decision regions of DNNs with width smaller than the input dimension are unbounded . More recently , and due to the popularity of the piecewise linear ReLU as an activation function , there has been a surge in the number of works that study this class of DNNs in particular . As a result , this has incited significant interest in new mathematical tools that help analyze piecewise linear functions , such as tropical geometry . While tropical geometry has shown its potential in many applications such as dynamic programming ( Joswig & Schröter , 2019 ) , linear programming ( Allamigeon et al. , 2015 ) , multi-objective discrete optimization ( Joswig & Loho , 2019 ) , enumerative geometry ( Mikhalkin , 2004 ) , economics ( Akian et al. , 2009 ; Mai Tran & Yu , 2015 ) , it has only been recently used to analyze DNNs . For instance , Zhang et al . ( 2018 ) showed an equivalency between the family of DNNs with piecewise linear activations and integer weight matrices and the family of tropical rational maps , i.e . ratio between two multi-variate polynomials in tropical algebra . The work of Zhang et al . ( 2018 ) was mostly concerned about characterizing the complexity of a DNN and specifically counting the number of linear regions , into which the function represented by the DNN can divide the input space , by counting the number of vertices of some polytope representation . This novel approach recovered the results of Montufar et al . ( 2014 ) with a much simpler analysis . Contributions . In this paper , we take the results of Zhang et al . ( 2018 ) some steps further and present a novel perspective on the decision boundaries of DNNs using tropical geometry . To that end , our contributions are three-fold . ( i ) We derive a geometric representation ( convex hull between two zonotopes ) for a super set to the decision boundaries of a DNN in the form ( Affine , ReLU , Affine ) . ( ii ) We demonstrate support for the lottery ticket hypothesis ( Frankle & Carbin , 2019 ) using a geometric perspective . ( iii ) We leverage the geometrical representation of the decision boundaries ( the decision boundaries polytope ) in two interesting applications : network purning and adversarial attacks . In regards to tropical pruning , we provide a new geometric perspective in which one can directly compress the decision boundaries polytope efficiently resulting in only minor perturbations to the decision boundaries . We conduct extensive experiments on AlexNet ( Krizhevsky et al. , 2012 ) and VGG16 ( Simonyan & Zisserman , 2014 ) on SVHN ( Netzer et al. , 2011 ) , CIFAR10 , and CIFAR 100 ( Krizhevsky & Hinton , 2009 ) datasets , in which 90 % pruning rate can be achieved with a marginal drop in testing accuracy . As for tropical adversarial attack , we show that one can construct input adversaries that can change network predictions by perturbing the decision boundaries polytope . We conduct extensive experiments on MNIST ( LeCun , 1998 ) . 2 PRELIMINARIES TO TROPICAL GEOMETRY . We provide here some preliminaries to tropical geometry . For a thorough detailed review , we refer the reader to the work of Itenberg et al . ( 2009 ) ; Maclagan & Sturmfels ( 2015 ) . Definition 1 . ( Tropical Semiring1 ) The tropical semiring T is the triplet { R∪ { −∞ } , ⊕ , } , where ⊕ and define tropical addition and tropical multiplication , respectively . They are denoted as : x⊕ y = max { x , y } , x y = x+ y , ∀x , y ∈ T . It can be readily shown that −∞ is the additive identity and 0 is the multiplicative identity .. Given the previous definition , a tropical power can be formulated as x a = x x · · · x = a.x , for x ∈ T , a ∈ N , where a.x is standard multiplication . Moreover , the tropical quotient can be defined as : x y = x− y where x− y is the standard subtraction . For ease of notation , we write x a as xa . Now , we are in a position to define tropical polynomials , their solution sets and tropical rationals . Definition 2 . ( Tropical Polynomials ) For x ∈ Td , ci ∈ R and ai ∈ Nd , a d-variable tropical polynomial with n monomials . f : Td → Td can be expressed as : f ( x ) = ( c1 xa1 ) ⊕ ( c2 xa2 ) ⊕ · · · ⊕ ( cn xan ) , ∀ ai 6= aj when i 6= j . We use the more compact vector notation xa = xa11 x a2 2 · · · x ad d where x , a ∈ Rd . Moreover and for ease of notation , we will denote ci xai as cixai throughout the paper . Definition 3 . ( Tropical Rational Functions ) A tropical rational function is a standard difference or equivalently , a tropical quotient of two tropical polynomials : f ( x ) − g ( x ) = f ( x ) g ( x ) . Algebraic curves or hypersurfaces in algebraic geometry , which are the solution sets to polynomials , can be analogously extended to tropical polynomials too . Definition 4 . ( Tropical Hypersurfaces ) A tropical hypersurface of a tropical polynomial f ( x ) = c1x a1 ⊕ · · · ⊕ cnxan is the set of points x where f is attained by two or more monomials in f , i.e . T ( f ) : = { x ∈ Rd : cixai = cjxaj = f ( x ) , for some ai 6= aj } . Tropical hypersurfaces divide the domain of f into convex regions , where f is linear in each region . Moreover , every tropical polynomial can be associated with a Newton polytope . 1A semiring is a ring that lacks an additive inverse . Definition 5 . ( Newton Polytopes ) The Newton polytope of a tropical polynomial f ( x ) = c1xa1 ⊕ · · · ⊕ cnxan is the convex hull of the exponents ai ∈ Nd regarded as points in Rd , i.e . ∆ ( f ) : = ConvHull { ai ∈ Rd : i = 1 , 2 , . . . , n and ci 6= −∞ } . A tropical polynomial determines a dual subdivision , which can thus be constructed by projecting the collection of upper faces ( UF ) in P ( f ) : = ConvHull { ( ai , ci ) ∈ Rd×R : i = 1 , . . . , n } to Rd . That is to say , the dual subdivision determined by f is given as δ ( f ) : = { π ( p ) ⊂ Rd : p ∈ UF ( P ( f ) ) } where π : Rd × R → Rd is the projection that drops the last coordinate . It has been shown by Maclagan & Sturmfels ( 2015 ) that the tropical hypersurface T ( f ) is the ( d-1 ) -skeleton of the polyhedral complex dual to δ ( f ) . So , each vertex of δ ( f ) corresponds to one region in Rd where f is linear . Zhang et al . ( 2018 ) showed an equivalency between tropical rational maps and any neural network f : Rn → Rk with piecewise linear activations and integer weights through the following theorem . Theorem 1 . ( Tropical Characterization of Neural Networks , Zhang et al . ( 2018 ) ) . A feedforward neural network with integer weights and real biases with piecewise linear activation functions is a function f : Rn → Rk , whose coordinates are tropical rational functions of the input , i.e. , f ( x ) = H ( x ) Q ( x ) = H ( x ) −Q ( x ) , where H and Q are tropical polynomials . While this result is new in the context of tropical geometry , it is not surprising , since any piecewise linear function can be represented as a difference of two max functions over a set of hyperplanes Melzer ( 1986 ) . Mathematically , that is to say if f is a piecewise linear function , it can be written as f ( x ) = maxi∈ [ m ] { a > i x } − maxj∈ [ n ] { b > j x } , where [ m ] = { 1 , . . . , m } and [ n ] = { 1 , . . . , n } . Thus , it is clear that each of the two maxima above is a tropical polynomial recovering Theorem 1 . 3 DECISION BOUNDARIES OF DEEP NEURAL NETWORKS AS POLYTOPES . In this section , we analyze the decision boundaries of a network in the form ( Affine , ReLU , Affine ) using tropical geometry . For ease , we use ReLUs as the non-linear activation , but any other piecewise linear function can also be used . The functional form of this network is : f ( x ) = Bmax ( Ax + c1,0 ) + c2 , where max ( . ) is an element-wise operator . The outputs of the network f are the logit scores . Throughout this section , we assume2 that A ∈ Zp×n , B ∈ Z2×p , c1 ∈ Rp and c2 ∈ R2 . For ease of notation , we only consider networks with two outputs , i.e . B2×p , where the extension to a multi-class output follows naturally and it is discussed in the appendix . Now , since f is a piecewise linear function , each output can be expressed as a tropical rational as per Theorem 1 . If f1 and f2 refer to the first and second outputs respectively , we have f1 ( x ) = H1 ( x ) Q1 ( x ) and f2 ( x ) = H2 ( x ) Q2 ( x ) , where H1 , H2 , Q1 and Q2 are tropical polynomials . In what follows and for ease of presentation , we present our main results where the network f has no biases , i.e . c1 = 0 and c2 = 0 , and we leave the generalization to the appendix . Theorem 2 . For a bias-free neural network in the form of f ( x ) : Rn → R2 where A ∈ Zp×n and B ∈ Z2×p , let R ( x ) = H1 ( x ) Q2 ( x ) ⊕H2 ( x ) Q1 ( x ) be a tropical polynomial . Then : • Let B = { x ∈ Rn : f1 ( x ) = f2 ( x ) } defines the decision boundaries of f , then B ⊆ T ( R ( x ) ) . • δ ( R ( x ) ) = ConvHull ( ZG1 , ZG2 ) . ZG1 is a zonotope in Rn with line segments { ( B+ ( 1 , j ) + B− ( 2 , j ) ) [ A+ ( j , : ) , A− ( j , : ) ] } pj=1 and shift ( B− ( 1 , : ) + B+ ( 2 , : ) ) A− . ZG2 is a zonotope in Rn with line segments { ( B− ( 1 , j ) + B+ ( 2 , j ) ) [ A+ ( j , : ) , A− ( j , : ) ] } pj=1 and shift ( B+ ( 1 , : ) + B− ( 2 , : ) ) A− . Note that A+ = max ( A , 0 ) and A− = max ( −A , 0 ) . The line segment ( B+ ( 1 , j ) + B− ( 2 , j ) ) [ A+ ( j , : ) , A− ( j , : ) ] has end points A+ ( j , : ) and A− ( j , : ) in Rn and scaled by ( B+ ( 1 , j ) + B− ( 2 , j ) ) . The proof for Theorem 2 is left for the appendix . Digesting Theorem 2 . Theorem 2 can be broken into two major results . The first , which is on the algebra side , i.e . finding the solution set to tropical polynomials , states that the decision boundaries 2Without loss of generality , as one can very well approximate real weights as fractions by multiplying by least common multiple of the denominators as discussed in Zhang et al . ( 2018 ) . B is a subset of the tropical hypersurface of the tropical polynomial R ( x ) , i.e . T ( R ( x ) ) . The second result , which is on the geometry side , of Theorem 2 relates the tropical polynomial R ( x ) to the geometric representation of the solution set to R ( x ) , i.e . T ( R ( x ) ) , referred to as the dual subdivision , i.e . δ ( R ( x ) ) . In particular , Theorem 2 states that the dual subdivision for a network f is the convex hull of two zonotopes denoted as ZG1 and ZG2 . Note that this dual subdivision is a function of only the network parameters A and B. Theorem 2 bridges the gap between the behaviour of the decision boundariesB , through the super-set T ( R ( x ) ) , and the polytope δ ( R ( x ) ) , which is the convex hull of two zonotopes . It is worthwhile to mention that Zhang et al . ( 2018 ) discussed a special case of the first part of Theorem 2 for a neural network with a single output and a score function s ( x ) to classify the output . To the best of our knowledge , this work is the first to propose a tropical geometric formulation of a super-set containing the decision boundaries of a multi-class classification neural network . In particular , the first result of Theorem 2 states that one can alter the network , e.g . by pruning network parameters , while preserving the decision boundaries B , if one preserves the tropical hypersurface of R ( x ) or T ( R ( x ) ) . While preserving the tropical hypersurfaces can be equally difficult to preserving the decision boundaries directly , the second result of Theorem 2 comes in handy . For a bias free network , π becomes an identity mapping with δ ( R ( x ) ) = ∆ ( R ( x ) ) , and thus the dual subdivision δ ( R ( x ) ) , which is the Newton polytope ∆ ( R ( x ) ) in this case , becomes a well structured geometric object that can be exploited to preserve decision boundaries . Since Maclagan & Sturmfels ( 2015 ) ( Proposition 3.1.6 ) showed that the tropical hypersurface is the skeleton of the dual to δ ( R ( x ) ) , the normal lines to the edges of the polytope δ ( R ( x ) ) are in one-to-one correspondence with the tropical hypersurface T ( R ( x ) ) . Figure 1 details this intimate relation between the decision boundaries , tropical hypersurface T ( R ( x ) ) , and normals to δ ( R ( x ) ) . Before any further discussion , we recap the definition of zonotopes . Definition 6 . Let u1 , . . . , up ∈ Rn . The zonotope formed by u1 , . . . , up is defined as Z ( u1 , . . . , up ) : = { ∑p i=1 xiu i : 0 ≤ xi ≤ 1 } . Equivalently , the zonotope can be expressed with respect to the generator matrix U ∈ Rp×n , where U ( i , : ) = ui > as ZU : = { U > x : ∀x ∈ [ 0 , 1 ] p } . Another common definition for zonotopes is the Minkowski sum ( refer to appendix A for the definition of the Minkowski sum ) of a set of line segments that start from the origin with end points u1 , . . . , up ∈ Rn . It is also well known that the number of vertices of a zonotope is polynomial in the number of line segments . That is to say , |vert ( ZU ) | ≤ 2 ∑n−1 i=0 ( p−1 i ) = O ( pn−1 ) ( Gritzmann & Sturmfels , 1993 ) . While Theorem 2 presents a strong relation between a polytope ( convex hull of two zonotopes ) and the decision boundaries , it remains unclear how such a polytope can be efficiently constructed . Although the number of vertices of a zonotope is polynomial in the number of its generating line segments , fast algorithms for enumerating these vertices are still restricted to zonotopes with line segments starting at the origin ( Stinson et al. , 2016 ) . Since the line segments generating the zonotopes in Theorem 2 have arbitrary end points , we present the next result that transforms these line segments into a generator matrix of line segments starting from the origin , as prescribed in Definition 6 . This result is essential for the efficient computation of the zonotopes in Theorem 2 . Proposition 1 . Consider p line segments in Rn with two arbitrary end points as follows { [ ui1 , ui2 ] } p i=1 . The zonotope formed by these line segments is equivalent to the zonotope formed by the line segments { [ ui1 − ui2,0 ] } p i=1 with a shift of ∑p i=1 u i 2 . The proof is left for the appendix . As per Proposition 1 , the generator matrices of zonotopes ZG1 , ZG2 in Theorem 2 can be defined as G1 = Diag [ ( B+ ( 1 , : ) ) + ( B− ( 2 , : ) ) ] A and G2 = Diag [ ( B+ ( 2 , : ) ) + ( B− ( 1 , : ) ) ] A , both with shift ( B− ( 1 , : ) + B+ ( 2 , : ) + B+ ( 1 , : ) + B− ( 2 , : ) ) A− . In what follows , we show several applications for Theorem 2 . We begin by leveraging the geometric structure to help in reaffirming the behaviour of the lottery ticket hypothesis .

Summary: This paper proposes a method to characterize the decision boundary of deep neural networks. The authors take advantage of a new perspective on deep neural network i.e., tropical geometry. They present and prove a theorem connecting the decision boundary of deep neural networks to tropical hyperspheres.  Then, they use the theoretical results of the theorem for two applications i.e., network pruning and adversarial examples generation.  For the former, they show that using the introduced decision boundary characterization using the tropical geometry, one can dimish the number of parameters in a model with insignificant loss in performance. For the latter, they introduce a new method for adversarial examples generation by altering the decision boundary.