TropEx: An Algorithm for Extracting Linear Terms in Deep Neural Networks

1 INTRODUCTION . Many of the most widely used neural network architectures , including VGG ( Simonyan & Zisserman , 2015 ) , GoogLeNet ( Szegedy et al. , 2015 ) and ResNet ( He et al. , 2016 ) , make use of rectified linear activations ( ReLU , ( Hahnloser et al. , 2000 ; Glorot et al. , 2011 ) , i.e. , σ ( x ) = max { x , 0 } ) and are therefore piecewise linear functions . Despite the apparent simplicity of these functions , there is a lack of theoretical understanding of the factors that contribute to the success of such architectures . Previous attempts of understanding piecewise linear network functions have focused on estimating the number of linear terms , which are the linear pieces ( affine functions ) that constitute the network function . A linear region is being defined as a maximally connected subset of the input space on which the network function is linear . Since computing the exact number of linear regions is intractable , work has focused on obtaining upper and lower bounds for this number ( Arora et al. , 2016 ; Serra et al. , 2018 ; Pascanu et al. , 2013 ; Raghu et al. , 2017 ; Montufar et al. , 2014 ; Montúfar , 2017 ; Xiong et al. , 2020 ; Zhang et al. , 2018 ) . To our knowledge , the currently best upper and lower bounds were calculated by Serra et al . ( 2018 ) . Raghu et al . ( 2017 ) show these bounds to be asymptotically tight . All of the mentioned papers share the intuition that the number of linear regions of neural networks measures their expressivity . Since the bounds grow linearly in width and exponentially in depth , deep networks are interpreted to have greater representational power . However , these bounds are staggeringly high : the upper bound on the number of linear regions in ( Serra et al. , 2018 ) exceeds 10300 even for the smallest networks we experimented on . ( There are approximately 1080 atoms in the universe . ) For slightly larger networks , the upper bound exceeds 1017000 whereas the lower bound exceeds 1083 linear regions . The number of training samples is generally much smaller than the estimated number of linear regions ( ≤ 106 ) , so that almost none of the linear regions contains training data . This raises the question of how representative the number of linear regions is for network performance and how information extracted from training samples passes on to the many linear regions free of data for successful generalization to test data . There are indications that a high number of linear regions is not required for good network performance . Frankle & Carbin ( 2019 ) point out that smaller networks perform similarly well as large ones , when a suitable initialization of the smaller network can be found from training the larger one . Hence , ∗Denotes equal contribution . the expressivity of the large network is helpful to explore the parameter space , but the small , less expressive network is sufficient to achieve high accuracy . Lee et al . ( 2019 ) and Croce et al . ( 2018 ) modify the training loss to encourage larger linear regions with the goal of robustness to adversarial attacks . Hanin & Rolnick ( 2019b ; a ) argue that in practice there are fewer linear regions than expected from the bounds and empirically investigate this for the MNIST data set . All these observations question the explanatory power of astronomically high bounds for the number of linear regions . More recently , the focus of research on linear regions has been shifting away from pure counting towards an understanding of the linear regions themselves . Zhang & Wu ( 2020 ) study geometric properties of linear regions and notice that batch normalization and dropout , albeit leading to similar network accuracies , produce differently looking linear regions . Our approach to the understanding of linear regions differs in that it investigates the linear coefficients of linear regions . To this end , we propose TropEx , a tropical algebra-based algorithm extracting linear terms of the network function N ( Figure 1 ) using a data set X . TropEx outputs an extracted function N ( X ) containing only the linear terms corresponding to regions on which data lies . As a result , N and N ( X ) agree on neighbourhoods of all data points . This creates a tool for the study of generalization from a new viewpoint , i.e. , the perspective of linear regions and their coefficients . Our contributions are as follows : • A new computational framework representing tropical functions ( Definition B.4 ) as matrices to efficiently perform tropical calculations appearing in networks with rectified linear activations . • This framework allows us to derive TropEx , an algorithm to systematically extract linear terms from piecewise linear network functions.1 • An application of TropEx to fully-connected ( FCN ) and convolutional networks ( CNN ) reveals that ( i ) consistently all training and test samples fall into different linear regions ; ( ii ) Simple tasks ( MNIST ) can be solved with the few linear regions of training samples alone , while this does not hold for more complex data sets . ( iii ) FCNs and CNNs differ in how they use linear regions free of training data for their performance on test data : Several measures illustrate that CNNs , in contrast to FCNs , tend to learn more diverse linear terms . ( iv ) We confirm that the number of linear regions alone is not a good indicator for network performance and show that the coefficients of linear regions contain information on architecture and classification performance . 2 BACKGROUND AND OVERVIEW . It was recently shown by Charisopoulos & Maragos ( 2018 ) ; Zhang et al . ( 2018 ) that ReLU neural network functions are the same as tropical rational maps . Tropical rational maps are exactly those functions where each entry in the output vector can be written as a difference of maxima Ni ( x ) = max { a+i1 ( x ) , . . . , a + in ( x ) } −max { a − i1 ( x ) , . . . , a − im ( x ) } , ( 1 ) 1Link to open source implementation : https : //github.com/martrim/tropex where each a+ij , a − ij : Rd → R is an affine function with only positive coefficients , taking the form x 7→ ∑ j wjxj+w0 with allwj ∈ R≥0 . Since the number of terms in ( 1 ) dwarfs the number of atoms in the universe , it is impossible to obtain this expression in practice . Therefore , we only extract those terms that correspond to linear regions of data points . For a fixed data point x ∈ X , the maximum of the network outputs can be written as maxiNi ( x ) = a+x ( x ) − a−x ( x ) , where a+x , a−x are the affine functions such that a+x ( x ) ≥ a+ij ( x ) , a−x ( x ) ≥ a − ij ( x ) for all i , j. TropEx extracts a + x and a − x . The extracted terms can be used to construct a tropical map N ( X ) ( x ) = ( N ( X ) 1 ( x ) , . . . , N ( X ) s ( x ) ) with maximally enlarged linear regions , given by N ( X ) i ( x ) = max { a + xk1 ( x ) , . . . , a+xkDi ( x ) } −max { a−xk1 ( x ) , . . . , a − xkDi ( x ) } , ( 2 ) where there are Di data points xk1 , . . . , xkDi given label i by the original network . Being a tropical rational map , the functionN ( X ) is again a ReLU neural network function by Zhang et al . ( 2018 ) . The maximal entries of the two output vectors ( hence also the assigned labels ) of the extracted function N ( X ) and the original network N agree in the neighbourhood of any data point x ∈ X . We discuss the basics of tropical algebra in Appendix B.1 and refer to Maclagan & Sturmfels ( 2015 ) for a detailed introduction . The relation of tropical geometry and ReLU networks is studied in Zhang et al . ( 2018 ) ; Charisopoulos & Maragos ( 2018 ) ; Alfarra et al . ( 2021 ) . 3 METHOD . 3.1 MATRIX REPRESENTATION OF TROPICAL RATIONAL MAPS . If one were to represent tropical rational maps symbolically on a computer , computations would be too slow . Therefore , we present tropical rational maps as multi-dimensional arrays . Definition 3.1 . Given an affine function a : Rd0 → R ; x 7→ ∑ k wkxk + w0 , we will call the vector ( w0 , w1 , . . . , wd0 ) its coefficient vector , the scalar w0 its constant part and the vector ( w1 , . . . , wd0 ) its variable part . We can represent functions Ni : Rd0 → R as in equation ( 1 ) in the following way : Let the rows of the matrix A+i ∈ Rn×d0 and the vector a + i ∈ Rn×1 be the variable and the constant parts of the affine functions a+ij , respectively . ( Analogously for A − i and a − i . ) We can then define ( A+i , a + i ) ( x ) = max { A + i x + a + i } , where the maximum is taken over the rows of the resulting column vector . If we define the formal quotient2 of matrix-vector pairs by ( A+i , a + i ) / ( A − i , a − i ) ( x ) = max { A+i x+ a + i } −max { A − i x+ a − i } , thenNi ( x ) = ( A + i , a + i ) / ( A − i , a − i ) ( x ) , giving us a matrixrepresentation of the function Ni . An entire network function with s output dimensions can then be represented by a list ( ( A+i , a + i ) / ( A − i , a − i ) ) 1≤i≤s . The advantage of the proposed matrix representation of tropical rational maps are natural operations performing calculations that arise for ( concatenations of ) layers of neural networks ( see supplements ) . A dense layer ` : Rd1 → Rd2 with ReLU activation is represented as a list ( ( A+i , a + i ) / ( A − i , a − i ) ) 1≤i≤d2 . Denoting by Wpos and Wneg the positive and negative part of a matrix W , respectively , i.e . wposij = max { wij , 0 } and w neg ij = max { −wij , 0 } , the matrix representation of a single neuron ni ( x ) = max { w · x+ b , 0 } is given by A+i = ( wpos wneg ) , a+i = ( bpos bneg ) ; A−i = w neg , a−i = b neg .

Summary: The paper proposes a method for representing a computation of a neural network over a finite set of inputs by a projection onto a set of linear transformations (from the input space) called tropical rational map (TRM).  TRM is essentially a snapshot representing the internal representation of however deep network based on its response to a finite set of inputs.  Authors use this linearised approximation of the network internal to gather information about the complexity and generalisability of its function mapping.