Model information as an analysis tool in deep learning

1 INTRODUCTION . The ultimate goal of many deep learning research has been improving performance on specific datasets , for example , aiming for superior classification accuracy on the ILSVRC challenge . We have witnessed super-human performance on tasks in vision and language processing , but we are still far from understanding how the learning process works and whether they resemble the learning of human . This is partially due to the metric we use providing too little information on the dynamics of learning . Besides , performance on the test set as a sole goal can sometimes even lead to undesirable outcomes or misleading conclusions ( Lipton & Steinhardt , 2019 ) . Recently , several works propose to use the description length of a model ( or surrogate estimations thereof ) to understand the behavior of learning . In this paper , we refer to such measures of the amount of information content in a model as model information . Blier & Ollivier ( 2018 ) first demonstrated efficiently encoding a deep neural network with prequential coding technique . Zhang et al . ( 2020 ) then proposed an approximation of model information and utilized model information to analyze the information content in a task . They also used model information to explain phenomenons in transfer learning and continual learning . They showed that model information provides a different perspective than performance , directly characterizing the information transfer in the learning process . Voita & Titov ( 2020 ) used model information as a probe to analyze what kind of information is present in a text representation model . They claim that model information is more informative and stable than performance metrics when used as a probe . Model information can provide an informational perspective to learning . It can potentially help to answer questions about learning dynamics , such as how much information exists in a dataset or model , or how much information is transferred in a learning step . Furthermore , we can also reformulate existing problems into problems about information , for example , similarity and capacity , as we will show in this paper . Comparing model information with model performance , model information not only accounts for how good a model can perform but also how fast it learns to perform well ( in the sense of sample efficiency , a discussion is given by Yogatama et al . ( 2019 ) ) . The latter can be interpreted as related to the quantity of information transferred in model training . In this paper , we try to illustrate that model information can provide a framework for analyzing and understanding phenomena in deep learning . In the next section , we provide a general definition of model information , independent of how model information is estimated . We then unify the analysis of fundamental elements of deep learning under the framework of model information . In the following sections , we use several common problems as examples to show how to use the model information framework as an analysis tool . 2 AN INFORMATIONAL PERSPECTIVE TO ELEMENTS OF DEEP LEARNING . To measure the amount of information in a model , Voita & Titov ( 2020 ) use the codelength of a dataset subtracting the cross-entropy of the final model : L = Lpreqθinit ( y1 : N |x1 : N ) + N∑ i log pθN ( yi|xi ) . ( 1 ) Zhang et al . ( 2020 ) propose to estimate model information by subtracting codelength of k examples ( the k examples are independent and different from the training set of size N ) with an initial model and a final model , and show that it is more reliable than ( 1 ) : L = Lpreqθinit ( y1 : k|x1 : k ) − L preq θN ( y1 : k|x1 : k ) . ( 2 ) Both methods share an idea that model information can be derived by comparing the codelength of encoding the model and data together , with the codelength of encoding the data alone . This is essentially an approximation to the Kolmogorov complexity of the “ generalizable knowledge ” in model M ( in Zhang et al . ( 2020 ) , this is denoted by K ( M ) − K ( M |T ) ) . Throughout this paper , we use D to denote a dataset including examples sampled from task T , and use MD to denote a model M that has been trained to converge on D. Assume labels y in task T have an underlying generation mechanism f : yi = f ( xi , i ) ( where i is i.i.d . noise ) , then the “ generalizable knowledge ” in T is K ( f ) , and generalizable knowledge in MD is at most K ( f ) . We denote the amount of generalizable information a model M contains about a dataset D as L ( M , D ) . Then L ( MD , D ) is the “ model information ” : the amount of generalizable information a trained model MD contains about dataset D. Ideally , if learning is “ perfectly efficient ” , then we can expect L ( MD , D ) = K ( MD ) −K ( MD|T ) = K ( f ) . In this paper , we investigate what can be achieved with a model information measure L. We found that model information is a powerful tool for analyzing and understanding many phenomenons in deep learning . This covers the fundamental elements of machine learning : task , data , model , and algorithm . In Table 1 , we list problems as examples for each domain and summarize how model information can be used to perform analysis of the corresponding problem . As we shall see , model information can enable attacking some of the above problems from a neat and theoretically-sound perspective . Table 1 is not an exhaustive list but just examples of what one can do with model information . In the following sections , we detail the application of model information to perform analysis of each problem and explain how it can lead to useful insights . For following experiments in this paper , we use ( 2 ) to estimate model information and set the encoding set size k=10000 , unless otherwise stated . 3 TASK : DIFFICULTY . The success of deep learning is marked by its superior performance at solving difficult tasks , like large-scale visual classification and question answering . For evaluating the capability of algorithms , label : 3 label : 3 label : 3 label : 3 label : 3 label : 4 Image noise Label noise missing pixels missing objects massart noise Figure 2 : Injecting three kinds of noise on MNIST . it is important to understand the difficulty of the tasks and the datasets we have . Human definitions of difficulty are mostly subjective , for example , difficulty scoring , task performance , and time needed to complete the task ( Ionescu et al. , 2016 ) . Several metrics have been proposed for measuring the difficulty of a task . Many focus on the complexity of the data . For images , difficulty is linked with image quality , objectiveness , and clutter-ness ( Ionescu et al. , 2016 ) . For text , the relevant factors include perplexity , reading ease , and diversity ( Collins et al. , 2018 ) . For general real-valued data , one can measure the distribution of feature value , class separability , and the geometry of class manifolds ( Ho & Basu , 2002 ) . There are also other factors that affect the difficulty of a task , for example , class balance and class ambiguity . As pointed out by Collins et al . ( 2018 ) , these characteristics all describe some aspect of the task , and difficulty can not be described by any one of them alone . Complexity measures of input data can sometimes have little impact on task difficulty . An important line of work defines the difficulty of a classification problem by the complexity of the optimal decision boundary ( Ho & Basu , 2002 ) . The complexity of the boundary can be characterized by its Kolmogorov complexity or its minimum description length . This coincidences with the idea of the model information . While for simple tasks , one can directly characterize the geometry of the decision boundary , for complex tasks that use a neural network to model the decision boundary , the description length can only be approximated . With description length , the difficulty of a task can be interpreted as the amount of information needed to solve the task . This amount of information is at most K ( f ) , which is the complexity of the input-output relationship of the task . We measure the model information L ( MD , D ) of a trained model , and use it as a measure of the information content in dataset D. In Figure 1 , we use five 10-class classification tasks to illustrate the idea . The results show that the datasets vary greatly in information content : For classifying digits , SVHN ( Netzer et al. , 2011 ) requires more information than MNIST ( LeCun et al. , 1998 ) or EMNIST ( Cohen et al. , 2017 ) , meaning classifying digits in street view is more difficult than in standardized MNIST images . CIFAR-10 ( Krizhevsky & Hinton , 2009 ) is even richer in information content and therefore more difficult , as natural objects are more complex than digits . There is also a general trend that the more difficult the task , the lower the performance and the confidence of the model . However , performance and confidence are not only determined by task complexity , they are also affected by noise present in the task . Next we want to disentangle the two factors : task complexity ( measured by information ) and noise . We apply three kinds of noise on MNIST to represent different forms of noise present in a dataset ( Figure 2 ) . There are two kinds of image noise : missing pixels , where a small random block of pixels is missing in the image , missing objects , where a large portion of the digit is missing , and one form of label noise : massart noise , where a small ratio of labels are replaced by random labels . To see how different kinds of noise affect model behavior , we plot model information , model accuracy , and confidence in Figure 3 . What we observe is that information complexity and noise are two independent dimensions that affect performance on a task . Pixels missing from the image will make the task more complex and require more information to correctly classify the digits . This is because one must learn more features corresponding to each digit , as any of the features could be absent at any time . However , the model can still perform well after learning enough information about the task . Missing objects and wrong labels , on the other hand , are pure noise that does not affect the information complexity of the task . They effectively make some of the examples uninformative and simply confusing , which makes the model less confident in its predictions . 4 DATA : DOMAIN SIMILARITY . Understanding the data is a fundamental element of understanding learning . Labeled data for deep learning comes from many sources and domains , and people seek to train models that can adapt well to a new domain , generalize to unseen domains , or be independent to domains ( Gulrajani & Lopez-Paz , 2020 ) . However , few attempts to understand the data before using them to train the model . Similarity is a fundamental concept for understanding the relationship between different data . In deep learning , we are more concerned about similarity on semantic level . Semantically similar images can sometimes differ very much in the RGB space . The informational similarity measure proposed by Lin ( 1998 ) measures the similarity between A and B by the ratio of the amount of information needed to describe the commonality between A and B and the amount information needed to describe A together with B : sim ( A , B ) = I ( common ( A , B ) ) I ( description ( A , B ) ) = K ( fB ) −K ( fB |fA ) K ( fA , fB ) . ( 3 ) It is the Jaccard similarity between A and B measured with information . A property of the informational similarity measure is universality : it does not depend on a particular representation ( or modeling ) of A and B . We can use model information to estimate the information terms in ( 3 ) , and turn the similarity measure into ( 4 ) : ( using approximations L ( MA , A ) ≈ K ( fA ) and L ( MA , B ) ≈ K ( fB ) −K ( fB |fA ) ) , see discussion on the symmetricity of S in Appendix A.3 ) S ( A , B ) = L ( MA , B ) L ( MA , A ) + L ( MB , B ) − L ( MA , B ) . ( 4 ) Another benefit of information similarity is that the similarity can be measured with respect to one side , resulting in a unidirectional similarity measure : Suni ( A , B ) = I ( common ( A , B ) I ( description ( B ) = L ( MA , B ) L ( MB , B ) . ( 5 ) Note that generally Suni ( A , B ) 6= Suni ( B , A ) . The unidirectional similarity measure is useful for depicting the relationship between A and B . For example , if Suni ( A , B ) < 1 and Suni ( B , A ) = 1 , then one can tell that A is a subset of B . We perform experiments on two commonly-used domain adaptation datasets : Office-31 ( Saenko et al. , 2010 ) and Office-Home ( Venkateswara et al. , 2017 ) . They each have 3 and 4 different domains for images of the same set of classes ( Figure 4 ) . We calculate information similarity S and unidirectional information similarity Suni for each pair of domains in each dataset . The baseline we use for comparison is the distance of first-order and second-order statistics in the feature space produced by ResNet-56 ( He et al. , 2016b ) pretrained on ImageNet ( Russakovsky et al. , 2015 ) ( see Appendix for details ) . This is inspired by the Maximum Mean Discrepancy ( MMD ) ( Gretton et al. , 2006 ) method for comparing data distributions . In domain adaptation literature , one often aims to minimize such distances to match domains ( Pan et al. , 2009 ) . As illustrated in Figure 5 , information similarity measure give a similarity score between 0 and 1 , which is intuitive and comparable across datasets . S and Suni largely agree with feature distances , but the latter is not comparable across datasets . For instance , dslr and webcam in Office-31 are much more similar than any other pair in the two datasets , which is reflected in S but not in feature distance . S is also more faithful within dataset for corresponding to visual similarities . Unidirectional information similarity Suni gives extra information , for instance , Suni ( amazon , dslr ) > Suni ( dslr , amazon ) shows that amazon contains more distinct information than dslr . This can be explained that each category in amazon has images for 90 different objects , while each category in dslr only has images for 5 objects .

Summary: The paper examines different use-cases of a quantity proposed in prior works which is said to capture the model information. It shows that this quantity behaves as expected overall. Quantifying the amount of information a deep neural network is a very interesting question for the community with both theoretical and practical appeal (from an analytical perspective but also from a model selection perspective, for example).