Measuring Density and Similarity of Task Relevant Information in Neural Representations

1 INTRODUCTION . Neural models achieve state of the art performance on a wide range of tasks in computer vision , machine learning and natural language processing ( Goodfellow et al. , 2016 ) . The key to the success of these models is their ability to automatically create rich representations of the input ( i.e images , words or sentences ) that possess the salient information necessary to perform particular tasks . However , despite the success of these neural feature extractors , most neural representations are hard to interpret ( Shi et al. , 2016b ; Lipton , 2016 ) . Recent research has thus attempted to answer the natural question : what information about the input do neural representations capture ? For example , in natural language processing ( NLP ) tasks – the main test bed upon which we examine the methods in this paper – Shi et al . ( 2016a ; b ) reveal that syntactic phenomena are captured by neural encoderdecoder translation systems , Adi et al . ( 2016 ) demonstrate that sentence representations are aware of sentence structure like length , word-order and word-content , and Conneau et al . ( 2018 ) probe sentence representations for 10 linguistic properties . However , while previous work has given us an idea of what information is contained , it is still unclear how this information is captured . In this paper , we attempt to fill this gap in our knowledge of interpreting neural representations , specifically focusing on the following questions : • How densely is the information encoded within the representation , and across what dimensions is it distributed ? • How is the information shared , for a given input , between two similar ( or dissimilar ) tasks ? We examine these questions through a comprehensive quantitative study of how popular sentence representations encode information . We propose methods to characterize the information density – how densely information necessary to solve a downstream task is encoded ( §3 ) – and information similarity – how similarly information is encoded for different tasks ( §4 ) . For information density , we introduce the notion of “ Condensed Feature Sets ” ( CFS ) , a minimal set of features required for a classifier to achieve a certain level of accuracy on a target task . For information similarity , we take a CFS for a given task and compare it to the CFS for other tasks , which provides cues about how many common features are shared among them . We further demonstrate that these metrics , in addition to satisfying our curiosity , have immediate practical use ; specifically allowing us to choose tasks for transfer learning ( §5 ) . Transfer learning Figure 1 : Input representation from a neural extractor is investigated for task-specific information . Here , information for Task A and C is compactly encoded , whereas information for Task B is dispersed . Also , similar dimensions in Task A and C are discriminative and hence the tasks require similar information and form good candidates for transfer learning . is the act of training on one task then transferring the knowledge acquired therefrom to improve performance on another task , and is most effective when these tasks are highly related ( Pan & Yang , 2010 ) . We directly use our information similarity metric between two tasks as a proxy for transfer learning potential between them . In experiments ( §6 ) , we examine these metrics over 4 different neural sentence encoders and 13 diverse syntactic or semantic natural language tasks . Using our measures of information density , we discover that information necessary for many of these tasks is concentrated in a very few units ( §7.1 ) . Further , information similarity metrics are able to predict good candidate tasks for transfer learning that correlate highly with actual transfer learning performance ( §7.2 ) . 2 CONCEPTUAL OVERVIEW . As discussed in the introduction , this paper delineates two concepts : “ information density ” and “ information similarity ” . In this section , we give an informal and intuitive explanation of these concepts and their utility , while in the following sections ( §3 and §4 ) we provide mathematical details to instantiate these concepts explicitly . Information density is defined with respect to a particular method of encoding inputs into hidden representations , and a particular down-stream task that uses these representations . We treat the encoding of this task-specific information as “ dense ” if the necessary information to solve the task is largely encoded in only a few elements of the hidden representation , and “ dispersed ” if many neurons play a significant role in solving the tasks . This equips us to ask questions such as “ Is the information for task A encoded more compactly than the information for task B ? ” ( which can be answered in affirmative for the case in Figure 1 ) . One can also answer questions like “ Is encoder X or encoder Y able to encode the information for task A in a more compact way ? ” Information similarity is defined with respect to two tasks in the context of a common fixed encoder . Specifically , we define two tasks as similar if similar elements in the hidden representation contribute to solving the two tasks . This enables us to inquire “ Is task A more similar to task B or task C ? ” ( For the case in figure 1 , the answer to the question is Task C ) . This allows us to empirically answer questions about which tasks are most related to each other from the point of view of a particular encoded representation , which has practical applications in transfer learning ( as shown in §5 ) . 3 METRICS FOR INFORMATION DENSITY . More formally , to measure information density , let X = [ x1 , x2 , . . . , xn ] ᵀ be input representations , where xi ∈ Rd . These input representations capture ( to a varying degree ) information essential to solve a set of classification tasks T . Each task t ∈ T comprises of a set of 〈x , y〉 pairs . To quantify the information in the input representations X , about a specific classification task t , we measure the accuracy a classifier f ∈ F would obtain on the task t using X . We restrict F to be a set of linear functions over the input space X . If instead of using all the d dimensions , we require only a handful of k d dimensions to retain a large fraction of the accuracy , then we can conclude that the information density for task t is high for the input representations X . To formalize this , we first define accuracy score of the best classifier f for task t using only a subset γ of the columns of X , score ( F , t , γ ) = max f∈F ∑ 〈xi , yi〉∈t ( 1 ( f ( xiγ ) = yi ) ) n where : xiγ = [ xij | j ∈ γ ] We then define the concept of a Condensed Feature Set ( CFS ) . Let CFSα ( F , T ) ⊆ { 1 , 2 , . . . , d } be the smallest subset such that : score ( F , t , CFS ) ≥ α ( score ( F , t , { 1 , 2 , . . . , d } ) − bt ) + bt where bt is the base rate ( i.e the percentage of examples in the majority class of task t ) , and α is a hyperparameter that we dub the ‘ retention ratio ’ . Intuitively , CFS is the smallest subset of dimensions that are able to retain α times the original accuracy1 gain over the majority class baseline . It is worthwhile to note that the CFS depends on the choice of representation . Two different ways to represent input could lead to altogether different CFS for the same task . Computing CFS is a combinatorially hard problem , and we use a greedy forward selection approach to approximate it . We start with an empty set C and iteratively add dimension i ∈ { 1 , 2 , . . . , d } \C that provides the highest increase in classification accuracy until the set C satisfies the criterion for it to be the CFS set . It is important to note that forward selection provides an upper bound on the cardinality of CFS for a given task , classifier and a representation choice . However , empirically we find that even this approximate method results in quite small sets of salient features in our experiments in Section 7 . Also , notably the CFS does not imply that the other dimensions are devoid of any information , but rather that task specific information is sufficiently contained in the CFS . 4 METRICS FOR INFORMATION SIMILARITY . In this section , we tackle the problem of evaluating information similarity between two tasks by using either the previously described CFS ( §3 ) , or direct comparisons between classifier weights . 4.1 ESTIMATING INFORMATION SIMILARITY USING CFS . To assess the information similarity for two tasks t1 , t2 , we first compute their CFSs C1 and C2 , then compute a metric of similarity between them . A first intuitive attempt at a metric is set overlap ( |C1∩C2||C1∪C2| ) . While the fraction of common hidden units in the condensed feature sets is a straightforward metric , it has two key drawbacks . First , set overlap does not take into account the relative influence of each dimension on classification 1Original accuracy is the accuracy score attained using the full set of dimensions performance . While this could potentially be addressed by some sort of weighting of dimensions , a more concerning drawback is that there could be a high redundancy ( or correlation ) among different hidden units leading to the possibility of two sets with very small overlap potentially encoding very similar information . These drawbacks preclude us from using this metric . Instead , we propose a second metric , information transfer . We wish to evaluate how useful the information content in C1 is to the task t2 . One way to approximate this is to compute the accuracy of a classifier on task t2 using the features C1 . This metric is robust to the redundancy problem mentioned earlier since two redundant sets would yield very similar accuracies using a linear classifier . 4.2 ESTIMATING INFORMATION SIMILARITY USING CLASSIFIER WEIGHTS . An alternative way to measure the information similarity for two tasks is to compute the similarity of the classifier weights trained for the two tasks . Specifically , using the representation X , we train two linear classifiers and then compute the inverse distance between the classifier weights . Let w1 and w2 be the normalized absolute classifier weights , such that they are positive and sum up to 1 , then we can define similarity ( t1 , t2 ) = ‖w1 −w2‖−1 . By using the absolute values of the learned weights , our notion of information similarity does not suffer from cases where the tasks have a strong negative correlation . In the extreme case of perfect negative correlation , the learned vectors would be w and −w , and we would rightly estimate a high information similarity between two clearly related tasks . 5 APPLICATION TO TRANSFER LEARNING . We can use our formalization of condensed feature sets and information similarity to solve a common problem in transfer learning – of choosing appropriate tasks for transfer learning . More formally , we have a set T of classification tasks and a representation X , consider the following problem : Given a primary task tp ∈ T , find a candidate task tc ∈ { T \ tp } such that transfer learning from tc would lead to the highest performance gain on the task tp . This is a common problem when the primary task tp is a low resource task and there are potential high resource candidate tasks which could be used for learning from . The most common learning methodology is to train the candidate task first , and then transfer it ’ s sentence encoder weights to the main task by fine-tuning ( Mou et al. , 2016 ; Conneau et al. , 2017 ; 2018 ) . This problem of finding the best candidate task for a given main task , tp , can be trivially solved by trial and error . However , this requires training 2n neural networks ( for each task , once to train the candidate task and other to fine tune after pre-training ) , where n is the number of candidate tasks ( T \ tp ) . Training such a large number of networks can be computationally expensive , and to the best of our knowledge , there are no efficient solutions to this problem . Practitioners often rely on their intuitive understanding of the similarity of tasks as a proxy to the task ’ s transfer potential or transferability . Using our transfer similarity criterion , we can produce a ranked recommendation list of candidate tasks .

Summary: This paper proposes simple metrics for measuring the "information density" in learned representations. Overall, this is an interesting direction. However there are a few key weaknesses in my view, not least that the practical utility of these metrics is not obvious, since they require supervision in the target domain. And while there is an argument to be made for the inherent interestingness of exploring these questions, this angle would be more compelling if multiple encoder architectures were explored and compared.