On the Statistical and Information Theoretical Characteristics of DNN Representations

1 INTRODUCTION . A learned representation can significantly affect the performance of deep networks , and the representation ’ s distributed and deep natures are the essential elements for the success of deep learning ( Bengio et al. , 2013 ) . As a consequence , deep networks have a greater expressiveness compared to the other machine learning algorithms ( Hinton et al. , 1986 ) or shallow networks ( Montufar et al. , 2014 ; Telgarsky , 2015 ; Eldan & Shamir , 2016 ; Raghu et al. , 2016 ) . Besides the distributed and deep natures that have been intensively studied , a hidden layer ’ s representation characteristics are considered to be important as well . Nonetheless , a relatively smaller number of studies have been completed on the topic , and the goal of this work is to understand the representation characteristics better . Therefore , the meaning of representation in this work is restricted to the activation vector of a single hidden layer and a unit refers to a neuron of the hidden layer . A few previous studies considered manipulating statistical characteristics of representations such as reducing covariance among hidden units ( Cogswell et al. , 2015 ; Xiong et al. , 2016 ) , encouraging representational sparsity ( Glorot et al. , 2011 ) , or forcing parsimonious representations via clustering ( Liao et al. , 2016 ) . In some of the similar works , a popular argument has been that the representation regularization reduces the generalization error via altering a representation characteristic . This argument , however , has not been rigorously studied . Another popular argument has been the reduction of effective capacity via regularization . This argument has been recently disproved by Zhang et al . ( 2016 ) where they empirically show that explicit regularization methods like L2 weight decay and dropout can not sufficiently limit the effective capacity of deep networks . Since a novel information-theoretic analysis method was proposed for deep learning ( Tishby & Zaslavsky , 2015 ; Shwartz-Ziv & Tishby , 2017 ) , information theoretic characteristics of representation have become an important research topic . In their work , mutual information I ( zl ; x ) and I ( zl ; y ) are used to address the learning dynamics and generalization of deep learning , where zl is the hidden layer l ’ s representation , x is the input , and y is the output . It is further discussed that a good representation is the one that contains a minimal amount of information from the input while containing a sufficient amount of information from the output . In Achille & Soatto ( 2017 ) , the Information Bottleneck Lagrangian ( Tishby et al. , 2000 ) is decomposed into the sum of a crossentropy term and a regularization term . The regularization term turns out to be I ( zl ; x ) that needs to be minimized . Some of the recent works will be additionally addressed in Section 5 . 2 REPRESENTATION CHARACTERISTICS . In this section , we briefly address the most popular statistical characteristics and information theoretic characteristics of representations . Consider a neural network NA whose architecture A is fixed and the weights for the lth layer are given by { Wl } and { bl } after training . Notation-wise , we simply write NA = ( W , b ) to define a network and y or NA ( x ) to refer to its deterministic output for a given input x . The layer index l is omitted when the meaning is obvious . The lth layer ’ s activation vector for the given input x is noted as zl ( x ) or simply zl , and the ith element of zl is noted as zl , i . The mean , variance , and standard deviation of zl , i are defined as µzl , i , vzl , i , and σzl , i , respectively . The covariance of zl is defined as Cl . Then , the basic representation characteristics can be summarized as in Table 1 . Six of them are statistical characteristics , and the last two are information theoretic characteristics . Previous studies on statistical characteristics are often based on regularizers . Srivastava et al . ( 2014 ) addresses dropout for preventing co-adaptation among hidden units by randomly putting zeros in a layer ’ s activation vector . Ioffe & Szegedy ( 2015 ) explains batch normalization ( BN ) that reduces internal covariate shift via normalizing activations of each unit to speed up network training . Cogswell et al . ( 2015 ) suggest DeCov that utilizes a penalty loss function to reduce activation covariance among hidden units . Choi & Rhee ( 2018 ) considers extension to class-wise regularization and provides four representation regularizers . Among them , CR ( Covariance Regularizer ) is equivalent to DeCov , and we adopt the notation of CR . Glorot et al . ( 2011 ) explains L1 representation regularization , called L1R in this work , that applies L1 penalty on activations . These representation regularization methods have distinct effects on representation characteristics , and examples of the learned representations are shown in Figure 1 . The representation regularizers are described in Appendix B , and a quantitative analysis of representation characteristics are provided in Appendix C. Because the true distribution of data is not accessible , the numerical results in the following sections are evaluated using the empirical distribution of the test dataset . Then , the expectations in Table 1 are with respect to the empirical distribution . For instance , Cl is calculated as the covariance matrix of N activation vectors { z1l , ... , zNl } where znl corresponds to the activation vector for the n ’ th test data sample , xn . Rank can be calculated by examining Cl , but often there are tiny eigenvalues that hinder a proper assessment of the rank . Therefore , we evaluate stable rank instead , and it will be explained further in Section 4 . Two information-theoretical characteristics , I ( zl ; x ) and I ( zl ; y ) , are estimated using upper and lower bounds explained in Kolchinsky & Tracey ( 2017 ) ; Kolchinsky et al . ( 2017 ) . Further details are provided in Section 5 . ReLU is the only activation function that is considered in this work . When ReLU is used , ACTIVATION AMPLITUDE , COVARIANCE , and CORRELATION are calculated using only the positive activation values such that the numerical evaluations can provide meaningful insights on what is happening to the non-zero representation values . 3 SCALING , PERMUTATION , COVARIANCE , AND CORRELATION . After training is completed for a deep networkNA , the output of the network becomes a deterministic function of the input x . Without an activation function , i.e . a linear layer , zl = WTl zl−1 +bl . When ReLU is applied to layer l , the activation vector becomes zl = ReLU ( WTl zl−1 +bl ) = max ( WTl zl−1 +bl , 0 ) . In this section , we investigate the most flexible affine transformation that can be applied to a layer ’ s representation zl without influencing the output NA ( x ) for any arbitrary input vector x . While complicated transformations over multiple layers can be explored , we limit our focus to manipulating only the weights of layer l and layer l + 1 for the analytical tractability . Because scaling and permutation are well known results , covariance and correlation are the main focus of this section . 3.1 IDENTICAL OUTPUT NETWORK ( ION ) . We first consider a linear layer l. For a linear layer , it turns out that any affine transformation can be applied as long as the transformation does not cause an information loss . Theorem 1 . ( ION for a linear layer ) For a deep network NA = ( W , b ) whose layer l is linear , there exists ÑA = ( W̃ , b̃ ) that satisfy the following conditions : ∀x , NA ( x ) = ÑA ( x ) ; ( 1 ) ∀x , z̃l = Q ( zl−m ) , ( 2 ) where Q is any nonsingular square matrix of a proper size and m is any vector of a proper size . The first condition says that the two networks generate identical outputs for any input x . The second condition says that zl can be affinely transformed using any nonsingular matrix Q . The proof is straightforward and can be found in Appendix A . While simple , Theorem 1 has significant implications on the representation characteristics of zl . Let ’ s inspect covariance and correlation ( normalized version of covariance ) first . If NA is a network that is globally optimal for a task and has at least one linear layer l , then NA ’ s covariance Cl can be whitened to have C̃l = I by choosing m as the expected mean and Q as a whitening matrix . The resulting network ÑA will have zero correlation between any pair of units in layer l , but will be globally optimal , too . In fact , there are infinitely many globally optimal networks with different covariance characteristics , and one can easily construct an ION with an arbitrary covariance matrix C̃l as long as its rank is the same as Cl ’ s rank . With this result , it becomes unclear why one should pursue a lower correlation when training a deep network . Unless regularization for a low correlation somehow helps optimization to reach a better performing network , there seems to be no reason to pursue low ( or high ) correlation . For dead neurons , a similar claim can be made . If globally optimal NA has no dead neurons in layer l and Cl is not full rank , one can make an affine transformation to align the null spaces of Cl to some of the neurons . Then , the resulting network ÑA will be still globally optimal , but with some dead neurons in layer l. For higher layers of classification tasks , typically the rank of Cl is close to the number of classes . For classification tasks with only 2∼10 classes , it is possible to construct an ION that has as many dead neurons as the size of Cl ’ s null space . This can be done without negatively affecting the performance , and the wisdom of ‘ reduce the number of dead neurons ’ becomes dubious . For scaling and permutation , their influences are rather insignificant . As for the scaling that can affect activation amplitude , it often has no effect on the network ’ s performance . For instance , scaled activation amplitude can affect the probability of classification tasks when softmax is in the last layer , but the class with the highest probability remains the same anyway . When representation regularizers are used , often activation amplitude is squashed to reduce the cost of representation penalty function but the network can still perform well . As reported in Choi & Rhee ( 2018 ) , such an activation squashing can make covariance much smaller , but the effect is removed when correlation is calculated . As for the permutation , it is considered to be meaningless because the index number itself is not important . Before discussing further , a similar result is developed for ReLU layers . The resulting Q , however , is much more limited . The proof can be found in Appendix A as well . Theorem 2 . ( ION for a ReLU layer ) For a deep network NA = ( W , b ) whose activation function of layer l is ReLU , there exists ÑA = ( W̃ , b̃ ) that satisfy the following conditions : ∀x , NA ( x ) = ÑA ( x ) ; ( 3 ) ∀x , z̃l = Qzl , ( 4 ) where Q is any permuted positive diagonal matrix of a proper size . Furthermore , it can be shown that any Q that satisfy the above two conditions must be a permuted positive diagonal matrix . Using a permuted positive diagonal matrix Q , covariance can be affected by independently scaling activation amplitudes of layer l ’ s units . As explained above , such a scaling is canceled out when calculating correlation and therefore a linear transformation can not affect correlation while keeping the output identical . There are a few possibilities for overcoming the limitations of Theorem 2 , and they are discussed in the following subsection . For rank and mutual information , the invertible affine transformation has no effect . They are discussed in the following sections .

Summary: This paper makes concerted efforts to examine the existing beliefs about the significance of various statistical characteristics of hidden layer activations (or representations) in a DNN. In the past, many works have argued for encouraging the certain statistical behavior of these representations (e.g., sparsity, low correlation etc) in order to have better classification accuracy. However, this paper tries to argue that such efforts are not very useful as these statistical characteristics don't provide any systematic explanation for the performance of DNNs across different settings.