Where is the Information in a Deep Network?

1 INTRODUCTION . At the end of training a deep neural network , all that is left of past experience is a set of values stored in its weights . So , studying what “ information ” they contain seems like a natural starting point to understand how deep networks learn . But how is the information in a deep neural network even defined ? The weights are not a random variable , and the network outputs a deterministic function of its input , with degenerate ( infinite ) Shannon Mutual Information between the two . This presents a challenge for theories of Deep Learning based on Shannon Information ( Saxe et al. , 2018 ) . Several frameworks have been developed to reason about information in fixed sets of values , for instance by Fisher and Kolmogorov , but they either do not relate directly to relevant concepts in Deep Learning , such as generalization and invariance , or can not be estimated in practice for modern deep neural networks ( DNNs ) . Beyond how they define information , existing theories of Deep Learning are limited by whose information they address : Most approaches focus on information of the activations of the network – the output of its layers – rather than their parameters , or weights , although recent information-theoretic approaches to study the weights are discussed in the next section . The weights are a representation of past data ( the training set of inputs and outputs ) , trained for predicting statistics of the training set itself ( e.g. , the output ) , relative to prior knowledge . The activations are a representation of ( possibly unseen ) future inputs ( test set ) , ideally sufficient to predict future outputs , and invariant to nuisance variability in the data that should not affect the output . We have no access to future data , and the Shannon Information their representation contains does not account for the finite training set , hence missing a link to generalization . But how are these properties of sufficiency and invariance achieved through the training process ? Sufficiency alone is trivial — any invertible function of the data is , in theory , sufficient — but it comes at the expense of complexity1 ( or minimality ) and invariance of the representation . Invariance alone is similarly trivial – any constant function is invariant . A learning criterion therefore must trade off accuracy , complexity and invariance . The best achievable complexity trade-off is what we define 1In this paper , we refer to complexity as information complexity , to be distinguished from complexity of the hypothesis space , for instance measured by the VC Dimension . as Information for the task . The challenge is that we wish to characterize sufficiency and invariance of representations of the test data , while we only have access to the training set . So , throughout this paper , we discuss four distinct concepts : ( 1 ) Sufficiency of the weights , captured by a training loss ( e.g. , empirical cross-entropy ) ; ( 2 ) complexity and minimality of the weights , captured by the information they contain ; ( 3 ) sufficiency of the activations , captured by the test loss which we can not compute , but can bound using the Information in the Weights ; ( 4 ) invariance of the activations , a property of the test data , which is not explicitly present in the formulation of the learning process when training a deep neural network . To do all that , we first need to formally define both information of the weights and of the activations . 1.1 SUMMARY OF CONTRIBUTIONS AND RELATED WORK . Our first contribution is to measure the Information in the Weights of a deep neural network as the trade-off between the amount of noise we could add to the weights ( measured by its entropy relative to a prior ) , and the performance the network would achieve in the task at hand . Informally , given an encoding algorithm , this is the number of bits needed to encode the weights in order to solve the task at some level of precision , as customary in Rate-Distortion Theory . The optimal trade-off traces a curve that depends on the task and the architecture , and solutions along the curve can be found by optimizing an Information Lagrangian . The Information Lagrangian is in the general form of an Information Bottleneck ( IB ) ( Tishby et al. , 1999 ) , but is fundamentally different from the IB used in most prior work in deep learning ( Tishby & Zaslavsky , 2015 ) , which refers to the activations , rather than the weights . Our measure of information is practical even in large-scale networks , with millions of parameters , and retains the dependency on the number of samples in the training set . Our second contribution is to derive a relation between the two informations ( Section 4 ) , where we show that the Information Lagrangian of the weights of deep networks bounds the Information Bottleneck of the activations , but not vice-versa . This is important , as the IB of the activations is degenerate when computed on the training set , hence can not be used at training time to enforce properties . On the other hand , the Information Lagrangian of the weights remains well defined , and through our bound it controls invariance at test time . Our method requires specifying a parametrized noise distribution , as well as a prior , to measure information . While this may seem undesirable , we believe it is essential and key to the flexibility of the method , as it allows us to compute concrete quantities , tailored to DNNs , that relate generalization and invariance in novel ways . Of all possible choices of noise and prior to compute the Information in the Weights , there are a few standard ones : An uninformative prior yields the Fisher Information of the weights . A prior obtained by averaging training over all relevant datasets yields the Shannon mutual information between the dataset ( now a random variable ) and the weights . A third important choice is the noise distribution induced by stochastic gradient descent ( SGD ) during the training process , which we discuss in the paper . As it turns out , all three resulting notions of information are important to understand learning in deep networks : Shannon ’ s relates closely to generalization , via the PAC-Bayes Bound ( Section 3.1 ) . Fisher ’ s relates closely to invariance in the representation of test data ( activations ) as we show in Section 4 . The noise distribution of SGD is what connects the two , and establishes the link between invariance and generalization . Although it is possible to minimize Fisher or Shannon Information independently , we show that when the weights are learned using SGD , the two are related . This is our third contribution , which is made possible by the flexibility of our framework ( Section 3.3 ) . Finally , in Section 5 we discuss open problems and further relations with prior work . There is a growing literature on information and generalization bounds for the weights of deep networks ( Xu & Raginsky , 2017 ; Pensia et al. , 2018 ) . Given a data generating distribution D ∼ µ ( x , y ) , a training algorithm w = A ( D ) is said to be ( , µ ) -information stable if I ( w ; D ) < . The generalization gap of a training algorithm can then be bounded in terms of its information stability . Indeed , the quantity I ( w ; D ) is related to our definition of information in the weights ( Section 3.2 ) , but we emphasize that our general definition of amount of information in the weights extends to the case where both the dataset D and w are assumed to be given and fixed ( as it is often common in Deep Learning ) , and not resampled every time . First , some of our main results are to prove that convergence to flat minima ( low Fisher Information ) and “ path ” stability of SGD ( Hardt et al. , 2015 ) imply “ information ” stability in the sense of Xu & Raginsky ( 2017 ) ( Proposition 3.7 ) . Unlike those works , our bound does not depend solely on the noise induced by the steps of SGD , but also the geometry of the loss lanscape , which allows to better capture some fundamental properties of Deep Learning . Second , while those work only bound the generalization performance on the training task , we connect the information stability with amount of information in the activations of a DNN and invariance to nuisances ( Section 4 ) . This this can be used to guarantee the quality of the learned representation in a transfer learning setting . Note that the fact that some weights can be perturbed at little loss has been known empirically for a while ( Hinton & Van Camp , 1993 ) . We exploit this property to define information for a particular set of weights , in a manner that is quite distinct from standard PAC-Bayes , using Fisher ’ s information instead . This paper takes Achille et al . ( 2019 ) as the starting point of investigation , attempting to measure the quantities at play more accurately , which leads us beyond Shannon ’ s formalism , to a more general setting that also includes Fisher ’ s formalism and the relation between the two , mediated by the properties of deep neural networks and SGD . All the ( information ) quantities we measure are specific to a particular weight vector , not its distribution . 2 PRELIMINARIES AND NOTATION . We denote with x ∈ X an input ( e.g. , an image ) , and with y ∈ Y a “ task variable , ” a random variable which we are trying to infer , e.g. , a label Y = { 1 , . . . , C } . A dataset is a finite collection of samples D = { ( xi , yi ) } Ni=1 that specify the task . A DNN model trained with the cross-entropy loss encodes a conditional distribution pw ( y|x ) , parametrized by the weights w , meant to approximate the posterior of the task variable y given the input x . The Kullbach-Liebler , or KL-divergence , is the relative entropy between p ( x ) and q ( x ) : KL ( p ( x ) ‖ q ( x ) ) = Ex∼p ( x ) [ log ( p ( x ) /q ( x ) ) ] . It is always non-negative , and zero if and only if p ( x ) = q ( x ) . It measures the ( asymmetric ) similarity between two distributions . Given a family of conditional distributions pw ( y|x ) parametrized by a vector w , we can ask how much perturbing the parameter w by a small amount δw will change the distribution , as measured by the KL-divergence . To second-order , this is given by Ex KL ( pw ( y|x ) ‖ pw+δw ( y|x ) ) = δwtFδw + o ( ‖δw‖2 ) where F is the Fisher Information Matrix ( or simply “ Fisher ” ) , defined by F = Ex , y∼p ( x ) pw ( y|x ) [ ∇ log pw ( y|x ) t∇ log pw ( y|x ) ] = Ex∼p ( x ) pw ( y|x ) [ −∇2w log pw ( y|x ) ] . For its relevant properties see Martens ( 2014 ) . It is important to notice that the Fisher depends on the ground-truth data distribution p ( x , y ) only through the domain variable x , not the task variable y , since y ∼ pw ( y|x ) is sampled from the model distribution when computing the Fisher . This property will be used later . Given two random variables x and z , their Shannon mutual information is defined as I ( x ; z ) = Ex∼p ( x ) [ KL ( p ( z|x ) ‖ p ( z ) ) ] that is , the expected divergence between the distribution of z after an observation of x , and the prior distribution of z . It is positive , symmetric , zero if and only if the variables are independent ( Cover & Thomas , 2012 ) , and measured in Nats when using the natural logarithm . In supervised classification one is usually interested in finding weights w that minimize the crossentropy loss LD ( w ) = E ( x , y ) ∼D [ − log pw ( y|x ) ] on the training set D. The loss LD ( w ) is usually minimized using stochastic gradient descent ( SGD ) , which updates the weights w with an estimate of the gradient computed from a small number of samples ( mini-batch ) . That is , wk+1 = wk − η∇L̂ξk ( w ) , where ξk are the indices of a randomly sampled mini-batch and L̂ξk ( w ) = 1 |ξk| ∑ i∈ξk [ − log pw ( yi|xi ) ] . Notice that Eξk [ ∇L̂ξk ( w ) ] = ∇LD ( w ) , so we can think of the mini-batch gradient∇L̂ξk ( w ) as a noisy version of the real gradient . Using this intuition we can write : wk+1 = wk − η∇LD ( wk ) + √ η Tξk ( wk ) ( 1 ) with the induced “ noise ” term T k ( w ) = √ η ( ∇L̂ξk ( w ) − ∇L ( w ) ) . Written in this form , eq . ( 1 ) is a Langevin diffusion process , with ( non-isotropic ) noise Tξk Li et al . ( 2017 ) ; Chaudhari & Soatto ( 2018 ) .

Summary: This paper presents a theoretical account of information encoded within deep neural networks subject to information theoretic measures. In contrast to other efforts that examine information encoded in weights, this work emphasizes the effective information in the activations. This characterization is further related to information in the weights, and a theoretical justification is made for what this means with respect to properties of generalization and invariance in the network.