Measuring Compositionality in Representation Learning

1 INTRODUCTION sp ea ke r m od el aaxxx aazzz bbmxx bbbyy lis te ne r m od el ⟨green triangle⟩ ⟨blue , triangle⟩ ⟨green , square⟩ ⟨⟨dark , blue⟩ , square⟩ ( a ) ( b ) ( c ) ( d ) Figure 1 : Representations arising from a communication game . In this game , an observation ( b ) is presented to a learned speaker model ( c ) , which encodes it as a discrete character sequence ( d ) to be consumed by a listener model for some downstream task . The space of inputs has known compositional structure ( a ) . We want to measure the extent to which this structure is reflected ( perhaps imperfectly ) in the structure of the learned codes . The success of modern representation learning techniques has been accompanied by an interest in understanding the structure of learned representations . One feature shared by many humandesigned representation systems is compositionality : the capacity to represent complex concepts ( from objects to procedures to beliefs ) by combining simple parts ( Fodor & Lepore , 2002 ) . While many machine learning approaches make use of human-designed compositional analyses for representation and prediction ( Socher et al. , 2013 ; Dong & Lapata , 2016 ) , it is also natural to ask whether ( and how ) compositionality arises in learning problems where compositional structure has not been built in from the start . Consider the example in Figure 1 , which shows a hypothetical character-based encoding scheme learned for a simple communication task ( similar to the one studied by Lazaridou et al. , 2016 ) . Is this encoding scheme compositional ? That is , to what extent can we analyze the agents ’ messages as being built from smaller pieces ( e.g . pieces xx meaning blue and bb meaning triangle ) ? A large body of work , from early experiments on language evolution to recent deep learning models ( Kirby , 1998 ; Lazaridou et al. , 2017 ) , aims to answer questions like this one . But existing solutions rely on manual ( and often subjective ) analysis of model outputs ( Mordatch & Abbeel , 2017 ) , or at best automated procedures tailored to the specifics of individual problem domains ( Brighton & Kirby , 2006 ) . They are difficult to compare and difficult to apply systematically . We are left with a need for a standard , formal , automatable and quantitative technique for evaluating claims about compositional structure in learned representations . The present work aims at first steps toward meeting that need . We focus on an oracle setting where the compositional structure of model inputs is known , and where the only question is whether this structure is reflected in model outputs . This oracle evaluation paradigm covers most of the existing representation learning problems in which compositionality has been studied . The first contribution of this paper is a simple formal framework for measuring how well a collection of representations ( discrete- or continuous-valued ) reflects an oracle compositional analysis of model inputs . We propose an evaluation metric called TRE , which provides graded judgments of compositionality for a given set of ( input , representation ) pairs . The core of our proposal is to treat a set of primitive meaning representations as hidden , and optimize over them to find an explicitly compositional model that approximates the true model as well as possible . For example , if the compositional structure that describes an object is a simple conjunction of attributes , we can search for a collection of “ attribute vectors ” that sum together to produce the observed object representations ; if it is a sparse combination of ( attribute , value ) pairs we can additionally search for “ value vectors ” and parameters of a binding operation ; and so on for more complex compositions . Having developed a tool for assessing the compositionality of representations , the second contribution of this paper is a survey of applications . We present experiments and analyses aimed at answering four questions about the relationship between compositionality and learning : • How does compositionality of representations evolve in relation to other measurable model properties over the course of the learning process ? ( Section 4 ) • How well does compositionality of representations track human judgments about the com- positionality of model inputs ? ( Section 5 ) • How does compositionality constrain distances between representations , and how does TRE relate to other methods that analyze representations based on similarity ? ( Section 6 ) • Are compositional representations necessary for generalization to out-of-distribution inputs ? ( Section 7 ) We conclude with a discussion of possible applications and generalizations of TRE-based analysis . 2 RELATED WORK . Arguments about whether distributed ( and other non-symbolic ) representations could model compositional phenomena were a staple of 1980s-era connectionist–classicist debates . Smolensky ( 1991 ) provides an overview of this discussion and its relation to learnability , as well as a concrete implementation of a compositional encoding scheme with distributed representations . Since then , numerous other approaches for compositional representation learning have been proposed , with ( Mitchell & Lapata , 2008 ; Socher et al. , 2012 ) and without ( Dircks & Stoness , 1999 ; Havrylov & Titov , 2017 ) the scaffolding of explicit composition operations built into the model . The main experimental question is thus when and how compositionality arises “ from scratch ” in the latter class of models . In order to answer this question it is first necessary to determine whether compositional structure is present at all . Most existing proposals come from linguistics and and philosophy , and offer evaluations of compositionality targeted at analysis of formal and natural languages ( Carnap , 1937 ; Lewis , 1976 ) . Techniques from this literature are specialized to the details of linguistic representations—particularly the algebraic structure of grammars ( Montague , 1970 ) . It is not straightforward to apply these techniques in more general settings , particularly those featuring non-string-valued representation spaces . We are not aware of existing work that describes a procedure suitable for answering questions about compositionality in the general case . Machine learning research has responded to this absence in several ways . One class of evaluations ( Mordatch & Abbeel , 2017 ; Choi et al. , 2018 ) derives judgments from ad-hoc manual analyses of representation spaces . These analyses provide insight into the organization of representations but are time-consuming and non-reproducible . Another class of evaluations ( Brighton , 2002 ; Andreas & Klein , 2017 ; Bogin et al. , 2018 ) exploits task-specific structure ( e.g . the ability to elicit pairs of representations known to feature particular relationships ) to give evidence of compositionality . Our work aims to provide a standard and scalable alternative to these model- and task-specific evaluations . Other authors refrain from measuring compositionality directly , and instead base analysis on measurement of related phenomena , for which more standardized evaluations exist . Examples include correlation between representation similarity and similarity of oracle compositional analyses ( Brighton & Kirby , 2006 ) and generalization to structurally novel inputs ( Kottur et al. , 2017 ) . Our approach makes it possible to examine the circumstances under which these surrogate measures in fact track stricter notions of compositionality ; similarity is discussed in Sec . 6 and generalization in Sec . 7 . A long line of work in natural language processing ( Coecke et al. , 2010 ; Baroni & Zamparelli , 2010 ; Clark , 2012 ; Fyshe et al. , 2015 ) focuses on learning composition functions to produce distributed representations of phrases and sentences—that is , for purposes of modeling rather than evaluation . We use one experiment from this literature to validate our own approach ( Section 5 ) . On the whole , we view work on compositional representation learning in NLP as complementary to the framework presented here : our approach is agnostic to the particular choice of composition function , and the aforementioned references provide well-motivated choices suitable for evaluating data from language and other sources . Indeed , one view of the present work is simply as a demonstration that we can take existing NLP techniques for compositional representation learning , fit them to representations produced by other models ( even in non-linguistic settings ) , and view the resulting training loss as a measure of the compositionality of the representation system in question . 3 EVALUATING COMPOSITIONALITY . Consider again the communication task depicted in Figure 1 . Here , a speaker model observes a target object described by a feature vector . The speaker sends a message to a listener model , which uses the message to complete a downstream task—for example , identifying the referent from a collection of distractors based on the content of the message ( Enquist & Arak , 1994 ; Lazaridou et al. , 2017 ) . Messages produced by the speaker model serve as representations of input objects ; we want to know if these representations are compositional . Crucially , we may already know something about the structure of the inputs themselves . In this example , inputs can be identified via composition of categorical shape and color attributes . How might we determine whether this oracle analysis of input structure is reflected in the structure of representations ? This section proposes an automated procedure for answering the question . Representations A representation learning problem is defined by a dataset X of observations x ( Figure 1b ) ; a space Θ of representations θ ( Figure 1d ) ; and a model f : X → Θ ( Figure 1c ) . We assume that the representations produced by f are used in a larger system to accomplish some concrete task , the details of which are not important for our analysis . Derivations The technique we propose additionally assumes we have prior knowledge about the compositional structure of inputs . In particular , we assume that inputs can be labeled with treestructured derivations d ( Figure 1a ) , defined by a finite set D0 of primitives and a binary bracketing operation 〈· , ·〉 , such that if di and dj are derivations , 〈di , dj〉 is a derivation . Derivations are produced by a derivation oracle D : X → D. Compositionality In intuitive terms , the representations computed by f are compositional if each f ( x ) is determined by the structure of D ( x ) . Most discussions of compositionality , following Montague ( 1970 ) , make this precise by defining a composition operation θa ∗ θb 7→ θ in the space of representations . Then the model f is compositional if it is a homomorphism from inputs to representations : we require that for any x with D ( x ) = 〈D ( xa ) , D ( xb ) 〉 , f ( x ) = f ( xa ) ∗ f ( xb ) . ( 1 ) In the linguistic contexts for which this definition was originally proposed , it is straightforward to apply . Inputs x are natural language strings . Their associated derivations D ( x ) are syntax trees , and composition of derivations is syntactic composition . Representations θ are logical representations of meaning ( for an overview see van Benthem & ter Meulen , 1996 ) . To argue that a particular fragment of language is compositional , it is sufficient to exhibit a lexicon D0 mapping words to their associated meaning representations , and a grammar for composing meanings where licensed by derivations . Algorithms for learning grammars and lexicons from data are a mainstay of semantic parsing approaches to language understanding problems like question answering and instruction following ( Zettlemoyer & Collins , 2005 ; Chen , 2012 ; Artzi et al. , 2014 ) . But for questions of compositionality involving more general representation spaces and more general analyses , the above definition presents two difficulties : ( 1 ) In the absence of a clearly-defined syntax of the kind available in natural language , how do we identify lexicon entries : the primitive parts from which representations are constructed ? ( 2 ) What do we do with languages like the one in Figure 1d , which seem to exhibit some kind of regular structure , but for which the homomorphism condition given in Equation 1 can not be made to hold exactly ? Consider again the example in Figure 1 . The oracle derivations tell us to identify primitive representations for dark , blue , green , square , and triangle . The derivations then suggest a process for composing these primitives ( e.g . via string concatenation ) to produce full representations . The speaker model is compositional ( in the sense of Equation 1 ) as long as there is some assignment of representations to primitives such that for each model input , composing primitive representations according to the oracle derivation reproduces the speaker ’ s prediction . In Figure 1 there is no assignment of strings to primitives that reproduces model predictions exactly . But predictions can be reproduced approximately—by taking xx to mean blue , aa to mean square , etc . The quality of the approximation serves as a measure of the compositionality of the true predictor : predictors that are mostly compositional but for a few exceptions , or compositional but for the addition of some noise , will be well-approximated on average , while arbitrary mappings from inputs to representations will not . This suggests that we should measure compositionality by searching for representations that allow an explicitly compositional model to approximate the true f as closely as possible . We define our evaluation procedure as follows : Tree Reconstruction Error ( TRE ) First choose : • a distance function δ : Θ×Θ→ [ 0 , ∞ ) satisfying δ ( θ , θ′ ) = 0⇔ θ = θ′ • a composition function ∗ : Θ×Θ→ Θ Define f̂η ( d ) , a compositional approximation to f with parameters η , as : f̂η ( di ) = ηi for di ∈ D0 f̂η ( 〈d , d′〉 ) = f̂η ( d ) ∗ f̂η ( d′ ) for all other d f̂η has one parameter vector ηi for every di in D0 ; these vectors are members of the representation space Θ . Given a dataset X of inputs xi with derivations di = D ( xi ) , compute : η∗ = arg min η ∑ i δ ( f ( xi ) , f̂η ( di ) ) ( 2 ) Then we can define datum- and dataset-level evaluation metrics : TRE ( x ) = δ ( f ( x ) , f̂η∗ ( d ) ) ( 3 ) TRE ( X ) = 1 n ∑ i TRE ( xi ) ( 4 ) TRE and compositionality How well does the evaluation metric TRE ( X ) capture the intuition behind Equation 1 ? The definition above uses parameters ηi to witness the constructability of representations from parts , in this case by explicitly optimizing over those parts rather than taking them to be given by f . Each term in Equation 2 is analogous to an instance of Equation 1 , measuring how well f̂η∗ ( xi ) , the best compositional prediction , matches the true model prediction f ( xi ) . In the case of models that are homomorphisms in the sense of Equation 1 , TRE reduces to the familiar case : Remark 1 . TRE ( x ) = 0 for all x if and only if Equation 1 holds exactly ( that is , f ( x ) = f ( xa ) ∗f ( xb ) for any x , xa , xb with D ( x ) = 〈D ( xa ) , D ( xb ) 〉 ) . Proof . One direction follows immediately from defining f̂η∗ ( x ) = f ( x ) . For the other , f ( x ) = f̂ ( D ( x ) ) = f̂ ( 〈D ( xa ) , D ( xb ) 〉 ) = f̂ ( D ( xa ) ) ∗ f̂ ( D ( xb ) ) = f ( xa ) ∗ f ( xb ) . Learnable composition operators The definition of TRE leaves the choice of δ and ∗ up to the evaluator . Indeed , if the exact form of the composition function is not known a priori , it is natural to define ∗ with free parameters ( as in e.g . Baroni & Zamparelli , 2010 ) , treat these as another learned part of f̂ , and optimize them jointly with the ηi . However , some care must be taken when choosing ∗ ( especially when learning it ) to avoid trivial solutions : Remark 2 . Suppose D is injective ; that is , every x ∈ X is assigned a unique derivation . Then there is always some ∗ that achieves TRE ( X ) = 0 : simply define f ( xa ) ∗ f ( xb ) = f ( x ) for any x , xa , xb as in the preceding definition , and set f̂ = f . In other words , some pre-commitment to a restricted composition function is essentially inevitable : if we allow the evaluation procedure to select an arbitrary composition function , the result will be trivial . This paper features experiments with ∗ in both a fixed functional form and a learned parametric one . Implementation details For models with continuous Θ and differentiable δ and ∗ , TRE ( X ) is also differentiable . Equation 2 can be solved using gradient descent . We use this strategy in Sections 4 and 5 . For discrete Θ , it may be possible to find a continuous relaxation with respect to which δ ( θ , · ) and ∗ are differentiable , and gradient descent again employed . We use this strategy in Section 7 ( discussed further there ) . An implementation of an SGD-based TRE solver is provided in the accompanying software release . For other problems , task-specific optimizers ( e.g . machine translation alignment models ; Bogin et al. , 2018 ) or general-purpose discrete optimization toolkits can be applied to Equation 2 . The remainder of the paper highlights ways of using TRE to answer questions about compositionality that arise in machine learning problems of various kinds .

Summary: The paper tackles a very interesting problem about representations, especially of the connectionist kind -- how do we know if the learned representations capture the compositional structure present in the inputs, and tries to come up with a systematic framework to answer that question. The framework assumes the presence of an oracle that can give us the true compositional structure. Then the author try to answer some refreshing questions about the dynamics of learning and compositionality while citing some interesting background reading.