Where is the Information in a Deep Network?

1 INTRODUCTION . At the end of training a deep neural network , all that is left of past experience is a set of values stored in its weights . So , studying what “ information ” they contain seems like a natural starting point to understand how deep networks learn . But how is the information in a deep neural network even defined ? The weights are not a random variable , and the network outputs a deterministic function of its input , with degenerate ( infinite ) Shannon Mutual Information between the two . This presents a challenge for theories of Deep Learning based on Shannon Information ( Saxe et al. , 2018 ) . Several frameworks have been developed to reason about information in fixed sets of values , for instance by Fisher and Kolmogorov , but they either do not relate directly to relevant concepts in Deep Learning , such as generalization and invariance , or can not be estimated in practice for modern deep neural networks ( DNNs ) . Beyond how they define information , existing theories of Deep Learning are limited by whose information they address : Most approaches focus on information of the activations of the network – the output of its layers – rather than their parameters , or weights , although recent information-theoretic approaches to study the weights are discussed in the next section . The weights are a representation of past data ( the training set of inputs and outputs ) , trained for predicting statistics of the training set itself ( e.g. , the output ) , relative to prior knowledge . The activations are a representation of ( possibly unseen ) future inputs ( test set ) , ideally sufficient to predict future outputs , and invariant to nuisance variability in the data that should not affect the output . We have no access to future data , and the Shannon Information their representation contains does not account for the finite training set , hence missing a link to generalization . But how are these properties of sufficiency and invariance achieved through the training process ? Sufficiency alone is trivial — any invertible function of the data is , in theory , sufficient — but it comes at the expense of complexity1 ( or minimality ) and invariance of the representation . Invariance alone is similarly trivial – any constant function is invariant . A learning criterion therefore must trade off accuracy , complexity and invariance . The best achievable complexity trade-off is what we define 1In this paper , we refer to complexity as information complexity , to be distinguished from complexity of the hypothesis space , for instance measured by the VC Dimension . as Information for the task . The challenge is that we wish to characterize sufficiency and invariance of representations of the test data , while we only have access to the training set . So , throughout this paper , we discuss four distinct concepts : ( 1 ) Sufficiency of the weights , captured by a training loss ( e.g. , empirical cross-entropy ) ; ( 2 ) complexity and minimality of the weights , captured by the information they contain ; ( 3 ) sufficiency of the activations , captured by the test loss which we can not compute , but can bound using the Information in the Weights ; ( 4 ) invariance of the activations , a property of the test data , which is not explicitly present in the formulation of the learning process when training a deep neural network . To do all that , we first need to formally define both information of the weights and of the activations . 1.1 SUMMARY OF CONTRIBUTIONS AND RELATED WORK . Our first contribution is to measure the Information in the Weights of a deep neural network as the trade-off between the amount of noise we could add to the weights ( measured by its entropy relative to a prior ) , and the performance the network would achieve in the task at hand . Informally , given an encoding algorithm , this is the number of bits needed to encode the weights in order to solve the task at some level of precision , as customary in Rate-Distortion Theory . The optimal trade-off traces a curve that depends on the task and the architecture , and solutions along the curve can be found by optimizing an Information Lagrangian . The Information Lagrangian is in the general form of an Information Bottleneck ( IB ) ( Tishby et al. , 1999 ) , but is fundamentally different from the IB used in most prior work in deep learning ( Tishby & Zaslavsky , 2015 ) , which refers to the activations , rather than the weights . Our measure of information is practical even in large-scale networks , with millions of parameters , and retains the dependency on the number of samples in the training set . Our second contribution is to derive a relation between the two informations ( Section 4 ) , where we show that the Information Lagrangian of the weights of deep networks bounds the Information Bottleneck of the activations , but not vice-versa . This is important , as the IB of the activations is degenerate when computed on the training set , hence can not be used at training time to enforce properties . On the other hand , the Information Lagrangian of the weights remains well defined , and through our bound it controls invariance at test time . Our method requires specifying a parametrized noise distribution , as well as a prior , to measure information . While this may seem undesirable , we believe it is essential and key to the flexibility of the method , as it allows us to compute concrete quantities , tailored to DNNs , that relate generalization and invariance in novel ways . Of all possible choices of noise and prior to compute the Information in the Weights , there are a few standard ones : An uninformative prior yields the Fisher Information of the weights . A prior obtained by averaging training over all relevant datasets yields the Shannon mutual information between the dataset ( now a random variable ) and the weights . A third important choice is the noise distribution induced by stochastic gradient descent ( SGD ) during the training process , which we discuss in the paper . As it turns out , all three resulting notions of information are important to understand learning in deep networks : Shannon ’ s relates closely to generalization , via the PAC-Bayes Bound ( Section 3.1 ) . Fisher ’ s relates closely to invariance in the representation of test data ( activations ) as we show in Section 4 . The noise distribution of SGD is what connects the two , and establishes the link between invariance and generalization . Although it is possible to minimize Fisher or Shannon Information independently , we show that when the weights are learned using SGD , the two are related . This is our third contribution , which is made possible by the flexibility of our framework ( Section 3.3 ) . Finally , in Section 5 we discuss open problems and further relations with prior work . There is a growing literature on information and generalization bounds for the weights of deep networks ( Xu & Raginsky , 2017 ; Pensia et al. , 2018 ) . Given a data generating distribution D ∼ µ ( x , y ) , a training algorithm w = A ( D ) is said to be ( , µ ) -information stable if I ( w ; D ) < . The generalization gap of a training algorithm can then be bounded in terms of its information stability . Indeed , the quantity I ( w ; D ) is related to our definition of information in the weights ( Section 3.2 ) , but we emphasize that our general definition of amount of information in the weights extends to the case where both the dataset D and w are assumed to be given and fixed ( as it is often common in Deep Learning ) , and not resampled every time . First , some of our main results are to prove that convergence to flat minima ( low Fisher Information ) and “ path ” stability of SGD ( Hardt et al. , 2015 ) imply “ information ” stability in the sense of Xu & Raginsky ( 2017 ) ( Proposition 3.7 ) . Unlike those works , our bound does not depend solely on the noise induced by the steps of SGD , but also the geometry of the loss lanscape , which allows to better capture some fundamental properties of Deep Learning . Second , while those work only bound the generalization performance on the training task , we connect the information stability with amount of information in the activations of a DNN and invariance to nuisances ( Section 4 ) . This this can be used to guarantee the quality of the learned representation in a transfer learning setting . Note that the fact that some weights can be perturbed at little loss has been known empirically for a while ( Hinton & Van Camp , 1993 ) . We exploit this property to define information for a particular set of weights , in a manner that is quite distinct from standard PAC-Bayes , using Fisher ’ s information instead . This paper takes Achille et al . ( 2019 ) as the starting point of investigation , attempting to measure the quantities at play more accurately , which leads us beyond Shannon ’ s formalism , to a more general setting that also includes Fisher ’ s formalism and the relation between the two , mediated by the properties of deep neural networks and SGD . All the ( information ) quantities we measure are specific to a particular weight vector , not its distribution . 2 PRELIMINARIES AND NOTATION . We denote with x ∈ X an input ( e.g. , an image ) , and with y ∈ Y a “ task variable , ” a random variable which we are trying to infer , e.g. , a label Y = { 1 , . . . , C } . A dataset is a finite collection of samples D = { ( xi , yi ) } Ni=1 that specify the task . A DNN model trained with the cross-entropy loss encodes a conditional distribution pw ( y|x ) , parametrized by the weights w , meant to approximate the posterior of the task variable y given the input x . The Kullbach-Liebler , or KL-divergence , is the relative entropy between p ( x ) and q ( x ) : KL ( p ( x ) ‖ q ( x ) ) = Ex∼p ( x ) [ log ( p ( x ) /q ( x ) ) ] . It is always non-negative , and zero if and only if p ( x ) = q ( x ) . It measures the ( asymmetric ) similarity between two distributions . Given a family of conditional distributions pw ( y|x ) parametrized by a vector w , we can ask how much perturbing the parameter w by a small amount δw will change the distribution , as measured by the KL-divergence . To second-order , this is given by Ex KL ( pw ( y|x ) ‖ pw+δw ( y|x ) ) = δwtFδw + o ( ‖δw‖2 ) where F is the Fisher Information Matrix ( or simply “ Fisher ” ) , defined by F = Ex , y∼p ( x ) pw ( y|x ) [ ∇ log pw ( y|x ) t∇ log pw ( y|x ) ] = Ex∼p ( x ) pw ( y|x ) [ −∇2w log pw ( y|x ) ] . For its relevant properties see Martens ( 2014 ) . It is important to notice that the Fisher depends on the ground-truth data distribution p ( x , y ) only through the domain variable x , not the task variable y , since y ∼ pw ( y|x ) is sampled from the model distribution when computing the Fisher . This property will be used later . Given two random variables x and z , their Shannon mutual information is defined as I ( x ; z ) = Ex∼p ( x ) [ KL ( p ( z|x ) ‖ p ( z ) ) ] that is , the expected divergence between the distribution of z after an observation of x , and the prior distribution of z . It is positive , symmetric , zero if and only if the variables are independent ( Cover & Thomas , 2012 ) , and measured in Nats when using the natural logarithm . In supervised classification one is usually interested in finding weights w that minimize the crossentropy loss LD ( w ) = E ( x , y ) ∼D [ − log pw ( y|x ) ] on the training set D. The loss LD ( w ) is usually minimized using stochastic gradient descent ( SGD ) , which updates the weights w with an estimate of the gradient computed from a small number of samples ( mini-batch ) . That is , wk+1 = wk − η∇L̂ξk ( w ) , where ξk are the indices of a randomly sampled mini-batch and L̂ξk ( w ) = 1 |ξk| ∑ i∈ξk [ − log pw ( yi|xi ) ] . Notice that Eξk [ ∇L̂ξk ( w ) ] = ∇LD ( w ) , so we can think of the mini-batch gradient∇L̂ξk ( w ) as a noisy version of the real gradient . Using this intuition we can write : wk+1 = wk − η∇LD ( wk ) + √ η Tξk ( wk ) ( 1 ) with the induced “ noise ” term T k ( w ) = √ η ( ∇L̂ξk ( w ) − ∇L ( w ) ) . Written in this form , eq . ( 1 ) is a Langevin diffusion process , with ( non-isotropic ) noise Tξk Li et al . ( 2017 ) ; Chaudhari & Soatto ( 2018 ) .

Summary: The paper deals with where the information is in a deep network and how information is propagated when new data points are observed. The authors measure information in the weights of a DNN as the trade-off between network accuracy and weight complexity. They bring out the relationships between Shannon MI and Fisher Information and the connections to PAC-Bayes bound and invariance. The main result is that models of low information generalize better and are invariance-tolerant.