Model information as an analysis tool in deep learning

1 INTRODUCTION . The ultimate goal of many deep learning research has been improving performance on specific datasets , for example , aiming for superior classification accuracy on the ILSVRC challenge . We have witnessed super-human performance on tasks in vision and language processing , but we are still far from understanding how the learning process works and whether they resemble the learning of human . This is partially due to the metric we use providing too little information on the dynamics of learning . Besides , performance on the test set as a sole goal can sometimes even lead to undesirable outcomes or misleading conclusions ( Lipton & Steinhardt , 2019 ) . Recently , several works propose to use the description length of a model ( or surrogate estimations thereof ) to understand the behavior of learning . In this paper , we refer to such measures of the amount of information content in a model as model information . Blier & Ollivier ( 2018 ) first demonstrated efficiently encoding a deep neural network with prequential coding technique . Zhang et al . ( 2020 ) then proposed an approximation of model information and utilized model information to analyze the information content in a task . They also used model information to explain phenomenons in transfer learning and continual learning . They showed that model information provides a different perspective than performance , directly characterizing the information transfer in the learning process . Voita & Titov ( 2020 ) used model information as a probe to analyze what kind of information is present in a text representation model . They claim that model information is more informative and stable than performance metrics when used as a probe . Model information can provide an informational perspective to learning . It can potentially help to answer questions about learning dynamics , such as how much information exists in a dataset or model , or how much information is transferred in a learning step . Furthermore , we can also reformulate existing problems into problems about information , for example , similarity and capacity , as we will show in this paper . Comparing model information with model performance , model information not only accounts for how good a model can perform but also how fast it learns to perform well ( in the sense of sample efficiency , a discussion is given by Yogatama et al . ( 2019 ) ) . The latter can be interpreted as related to the quantity of information transferred in model training . In this paper , we try to illustrate that model information can provide a framework for analyzing and understanding phenomena in deep learning . In the next section , we provide a general definition of model information , independent of how model information is estimated . We then unify the analysis of fundamental elements of deep learning under the framework of model information . In the following sections , we use several common problems as examples to show how to use the model information framework as an analysis tool . 2 AN INFORMATIONAL PERSPECTIVE TO ELEMENTS OF DEEP LEARNING . To measure the amount of information in a model , Voita & Titov ( 2020 ) use the codelength of a dataset subtracting the cross-entropy of the final model : L = Lpreqθinit ( y1 : N |x1 : N ) + N∑ i log pθN ( yi|xi ) . ( 1 ) Zhang et al . ( 2020 ) propose to estimate model information by subtracting codelength of k examples ( the k examples are independent and different from the training set of size N ) with an initial model and a final model , and show that it is more reliable than ( 1 ) : L = Lpreqθinit ( y1 : k|x1 : k ) − L preq θN ( y1 : k|x1 : k ) . ( 2 ) Both methods share an idea that model information can be derived by comparing the codelength of encoding the model and data together , with the codelength of encoding the data alone . This is essentially an approximation to the Kolmogorov complexity of the “ generalizable knowledge ” in model M ( in Zhang et al . ( 2020 ) , this is denoted by K ( M ) − K ( M |T ) ) . Throughout this paper , we use D to denote a dataset including examples sampled from task T , and use MD to denote a model M that has been trained to converge on D. Assume labels y in task T have an underlying generation mechanism f : yi = f ( xi , i ) ( where i is i.i.d . noise ) , then the “ generalizable knowledge ” in T is K ( f ) , and generalizable knowledge in MD is at most K ( f ) . We denote the amount of generalizable information a model M contains about a dataset D as L ( M , D ) . Then L ( MD , D ) is the “ model information ” : the amount of generalizable information a trained model MD contains about dataset D. Ideally , if learning is “ perfectly efficient ” , then we can expect L ( MD , D ) = K ( MD ) −K ( MD|T ) = K ( f ) . In this paper , we investigate what can be achieved with a model information measure L. We found that model information is a powerful tool for analyzing and understanding many phenomenons in deep learning . This covers the fundamental elements of machine learning : task , data , model , and algorithm . In Table 1 , we list problems as examples for each domain and summarize how model information can be used to perform analysis of the corresponding problem . As we shall see , model information can enable attacking some of the above problems from a neat and theoretically-sound perspective . Table 1 is not an exhaustive list but just examples of what one can do with model information . In the following sections , we detail the application of model information to perform analysis of each problem and explain how it can lead to useful insights . For following experiments in this paper , we use ( 2 ) to estimate model information and set the encoding set size k=10000 , unless otherwise stated . 3 TASK : DIFFICULTY . The success of deep learning is marked by its superior performance at solving difficult tasks , like large-scale visual classification and question answering . For evaluating the capability of algorithms , label : 3 label : 3 label : 3 label : 3 label : 3 label : 4 Image noise Label noise missing pixels missing objects massart noise Figure 2 : Injecting three kinds of noise on MNIST . it is important to understand the difficulty of the tasks and the datasets we have . Human definitions of difficulty are mostly subjective , for example , difficulty scoring , task performance , and time needed to complete the task ( Ionescu et al. , 2016 ) . Several metrics have been proposed for measuring the difficulty of a task . Many focus on the complexity of the data . For images , difficulty is linked with image quality , objectiveness , and clutter-ness ( Ionescu et al. , 2016 ) . For text , the relevant factors include perplexity , reading ease , and diversity ( Collins et al. , 2018 ) . For general real-valued data , one can measure the distribution of feature value , class separability , and the geometry of class manifolds ( Ho & Basu , 2002 ) . There are also other factors that affect the difficulty of a task , for example , class balance and class ambiguity . As pointed out by Collins et al . ( 2018 ) , these characteristics all describe some aspect of the task , and difficulty can not be described by any one of them alone . Complexity measures of input data can sometimes have little impact on task difficulty . An important line of work defines the difficulty of a classification problem by the complexity of the optimal decision boundary ( Ho & Basu , 2002 ) . The complexity of the boundary can be characterized by its Kolmogorov complexity or its minimum description length . This coincidences with the idea of the model information . While for simple tasks , one can directly characterize the geometry of the decision boundary , for complex tasks that use a neural network to model the decision boundary , the description length can only be approximated . With description length , the difficulty of a task can be interpreted as the amount of information needed to solve the task . This amount of information is at most K ( f ) , which is the complexity of the input-output relationship of the task . We measure the model information L ( MD , D ) of a trained model , and use it as a measure of the information content in dataset D. In Figure 1 , we use five 10-class classification tasks to illustrate the idea . The results show that the datasets vary greatly in information content : For classifying digits , SVHN ( Netzer et al. , 2011 ) requires more information than MNIST ( LeCun et al. , 1998 ) or EMNIST ( Cohen et al. , 2017 ) , meaning classifying digits in street view is more difficult than in standardized MNIST images . CIFAR-10 ( Krizhevsky & Hinton , 2009 ) is even richer in information content and therefore more difficult , as natural objects are more complex than digits . There is also a general trend that the more difficult the task , the lower the performance and the confidence of the model . However , performance and confidence are not only determined by task complexity , they are also affected by noise present in the task . Next we want to disentangle the two factors : task complexity ( measured by information ) and noise . We apply three kinds of noise on MNIST to represent different forms of noise present in a dataset ( Figure 2 ) . There are two kinds of image noise : missing pixels , where a small random block of pixels is missing in the image , missing objects , where a large portion of the digit is missing , and one form of label noise : massart noise , where a small ratio of labels are replaced by random labels . To see how different kinds of noise affect model behavior , we plot model information , model accuracy , and confidence in Figure 3 . What we observe is that information complexity and noise are two independent dimensions that affect performance on a task . Pixels missing from the image will make the task more complex and require more information to correctly classify the digits . This is because one must learn more features corresponding to each digit , as any of the features could be absent at any time . However , the model can still perform well after learning enough information about the task . Missing objects and wrong labels , on the other hand , are pure noise that does not affect the information complexity of the task . They effectively make some of the examples uninformative and simply confusing , which makes the model less confident in its predictions . 4 DATA : DOMAIN SIMILARITY . Understanding the data is a fundamental element of understanding learning . Labeled data for deep learning comes from many sources and domains , and people seek to train models that can adapt well to a new domain , generalize to unseen domains , or be independent to domains ( Gulrajani & Lopez-Paz , 2020 ) . However , few attempts to understand the data before using them to train the model . Similarity is a fundamental concept for understanding the relationship between different data . In deep learning , we are more concerned about similarity on semantic level . Semantically similar images can sometimes differ very much in the RGB space . The informational similarity measure proposed by Lin ( 1998 ) measures the similarity between A and B by the ratio of the amount of information needed to describe the commonality between A and B and the amount information needed to describe A together with B : sim ( A , B ) = I ( common ( A , B ) ) I ( description ( A , B ) ) = K ( fB ) −K ( fB |fA ) K ( fA , fB ) . ( 3 ) It is the Jaccard similarity between A and B measured with information . A property of the informational similarity measure is universality : it does not depend on a particular representation ( or modeling ) of A and B . We can use model information to estimate the information terms in ( 3 ) , and turn the similarity measure into ( 4 ) : ( using approximations L ( MA , A ) ≈ K ( fA ) and L ( MA , B ) ≈ K ( fB ) −K ( fB |fA ) ) , see discussion on the symmetricity of S in Appendix A.3 ) S ( A , B ) = L ( MA , B ) L ( MA , A ) + L ( MB , B ) − L ( MA , B ) . ( 4 ) Another benefit of information similarity is that the similarity can be measured with respect to one side , resulting in a unidirectional similarity measure : Suni ( A , B ) = I ( common ( A , B ) I ( description ( B ) = L ( MA , B ) L ( MB , B ) . ( 5 ) Note that generally Suni ( A , B ) 6= Suni ( B , A ) . The unidirectional similarity measure is useful for depicting the relationship between A and B . For example , if Suni ( A , B ) < 1 and Suni ( B , A ) = 1 , then one can tell that A is a subset of B . We perform experiments on two commonly-used domain adaptation datasets : Office-31 ( Saenko et al. , 2010 ) and Office-Home ( Venkateswara et al. , 2017 ) . They each have 3 and 4 different domains for images of the same set of classes ( Figure 4 ) . We calculate information similarity S and unidirectional information similarity Suni for each pair of domains in each dataset . The baseline we use for comparison is the distance of first-order and second-order statistics in the feature space produced by ResNet-56 ( He et al. , 2016b ) pretrained on ImageNet ( Russakovsky et al. , 2015 ) ( see Appendix for details ) . This is inspired by the Maximum Mean Discrepancy ( MMD ) ( Gretton et al. , 2006 ) method for comparing data distributions . In domain adaptation literature , one often aims to minimize such distances to match domains ( Pan et al. , 2009 ) . As illustrated in Figure 5 , information similarity measure give a similarity score between 0 and 1 , which is intuitive and comparable across datasets . S and Suni largely agree with feature distances , but the latter is not comparable across datasets . For instance , dslr and webcam in Office-31 are much more similar than any other pair in the two datasets , which is reflected in S but not in feature distance . S is also more faithful within dataset for corresponding to visual similarities . Unidirectional information similarity Suni gives extra information , for instance , Suni ( amazon , dslr ) > Suni ( dslr , amazon ) shows that amazon contains more distinct information than dslr . This can be explained that each category in amazon has images for 90 different objects , while each category in dslr only has images for 5 objects .

Summary: The central concept of this paper is model information, a description length of a discriminative model. The authors advocate usage of model information for analyzing several aspects in deep learning. In particular, they show how model information can be used to judge about difficulty of supervised tasks, domain similarity, model capacity, roles of different network components, and knowledge distillation. All of these are important topics and are relevant to the ICLR community. While most of the definitions and interpretations seem intuitive and valid, there are a few concerns and questions, and in some cases, it is hard to decide whether the conclusions of the interpretations should be trusted.