Discovering the compositional structure of vector representations with Role Learning Networks

1 OVERVIEW . Certain AI tasks consist in computing a function ϕ that is governed by strict rules : e.g. , if ϕ is the function mapping a mathematical expression to its value ( e.g. , mapping ‘ 19−2∗7 ’ to 5 ) , then ϕ obeys the rule that ϕ ( x+ y ) = sum ( ϕ ( x ) , ϕ ( y ) ) for any expressions x and y . This rule is compositional : the output of a structure ( here , x + y ) is a function of the outputs of the structure ’ s constituents ( here , x and y ) . The rule can be stated with full generality once the input is assigned a symbolic structure giving its decomposition into constituents . For a fully-compositional task , completely determined by compositional rules , an AI system that can assign appropriate symbolic structures to inputs and apply appropriate compositional rules to these structures will display full systematic generalization : it will correctly process arbitrary novel combinations of familiar constituents . This is a core capability of symbolic AI systems . Other tasks , including most natural language tasks such as machine translation , are only partially characterizable by compositional rules because natural language is only partially compositional in nature . For example , if ϕ is the function that assigns meanings to English adjectives , it generally obeys the rule that ϕ ( in- + x ) = not ϕ ( x ) , ( e.g. , ϕ ( inoffensive ) = not ϕ ( offensive ) ) , yet there are exceptions : ϕ ( inflammable ) = ϕ ( flammable ) . On these “ partially-compositional ” AI tasks , this strategy of compositional analysis has demonstrated considerable , but limited , generalization capabilities . Deep learning research has shown that Neural Networks ( NNs ) can display remarkable degrees of combinatorial generalization , often surpassing symbolic AI systems for partially-compositional tasks ( Wu et al. , 2016 ) , and exhibit good generalization ( although generally falling short of symbolic AI systems ) on fully-compositional tasks ( Lake & Baroni , 2018 ; McCoy et al. , 2019a ) . Given that standard NNs have no obvious mechanisms for representing symbolic structures , parsing inputs into such structures , nor applying compositional symbolic rules to them , this success raises the question we address in this paper : How do NNs achieve such strong generalization on partially-compositional tasks , and good performance on fully-compositional tasks ? An important step towards answering this question was reported in McCoy et al . ( 2019a ) , which showed that when trained on highly compositional tasks , standard NNs learned representations that were well approximated by symbolic structures ( Sec . 2 ) . Processing in these NNs assigns such representations to inputs and generates outputs that are governed by compositional rules stated over those representations . We refer to the networks to be analyzed as target NNs , because we will propose a new type of NN ( in Sec . 4 ) — the Role Learner ( ROLE ) — which is used to analyze the target network , after first discussing related analysis methods in Sec . 3 . In contrast with the analysis model of McCoy et al . ( 2019a ) , which relies on a hand-specified hypothesis about the underlying structure , ROLE automatically learns a symbolic structure that best approximates the internal representation of the target network . Automating the discovery of structural hypotheses provides two advantages . First , ROLE achieves success at analyzing networks for which it is not clear what the underlying structure is . We show this in Sec . 6 , where ROLE successfully uncovers the symbolic structures learned by a seq2seq RNN trained on the SCAN task ( Lake & Baroni , 2018 ) . Second , removing the need for hand-specified hypotheses allows the data to speak for itself , which simplifies the burden on the user , who only needs to provide input sequences and associated embeddings . We first consider fully-compositional ( hence synthetic ) tasks : a simple string-manipulation task in Sec . 5 , and the richer SCAN task , which has been the basis of previous work on combinatorial generalization in NNs , in Sec . 6 . Discovering symbolic structure within a model enables us to perform precise alterations to the internal representations in order to produce desired combinatorial alterations in the output ( Sec . 6.3 ) . Then , in Sec . 7 , we turn briefly to partially-compositional tasks in NLP . In Sec . 8 we consider how what we have learned about standard NNs can suggest new inductive biases to strengthen compositionality in NN learning . 2 NN EMBEDDING OF SYMBOL STRUCTURES . We build on McCoy et al . ( 2019a ) , which introduced the analysis task DISCOVER ( DISsecting COmpositionality in VEctor Representations ) : take a NN and , to the extent possible , find an explicitly-compositional approximation to its internal distributed representations . McCoy et al . ( 2019a ) showed that , in GRU ( Cho et al. , 2014 ) encoder-decoder networks performing simple , fullycompositional string manipulations , the medial encoding ( between encoder and decoder ) could be extremely well approximated , up to a linear transformation , by Tensor Product Representations ( TPRs ) ( Smolensky , 1990 ) , which are explicitly-compositional vector embeddings of symbolic structures . To represent a string of symbols as a TPR , the symbols in the string 337 might be parsed into three constituents { 3 : pos1 , 7 : pos3 , 3 : pos2 } , where posn is the role of nth position from the left edge of the string ; other role schemes are also possible , such as roles denoting right-to-left position : { 3 : third-to-last , 3 : second-to-last , 7 : last } . The embedding of a constituent 7 : pos3 is e ( 7 : pos3 ) = eF ( 7 ) ⊗ eR ( pos3 ) , where eR , eF are respectively a vector embedding of the roles and a vector embedding of the fillers of those roles : the digits . The embedding of the whole string is the sum of the embeddings of its constituents . In general , for a symbol structure S with roles { rk } that are respectively filled by the symbols { fk } , eTPR ( S ) = ∑ k eF ( fk ) ⊗ eR ( rk ) . The example above used the role scheme of linear position from the left edge , but other role schemes are possible , such as linear position from the right edge or position in a tree . McCoy et al . ( 2019a ) showed that — for a given seq2seq architecture learning a given string-mapping task — there exists a highly accurate TPR approximation of the medial encoding , given an appropriate pre-defined role scheme . The main technical contribution of the present paper is the Role Learner ( ROLE ) model , a RNN network that learns its own role scheme to optimize the fit of a TPR approximation to a given set of internal representations in a pre-trained target NN . This makes the DISCOVER framework more general by removing the need for human-generated hypotheses as to the role schemes the network might be implementing . Learned role schemes , we will see in Sec . 6.1 , can enable good TPR approximation of networks for which human-generated role schemes fail . 3 RELATED WORK . This work falls within the larger paradigm of using analysis techniques to interpret NNs ( see Belinkov & Glass ( 2019 ) for a recent survey ) , often including a focus on compositional structure ( Hupkes et al. , 2019 ; 2018 ; Lake & Baroni , 2018 ; Hewitt & Manning , 2019 ) . Two of the most popular analysis techniques are the behavioral and probing approaches . In the behavioral approach , a model is evaluated on a set of examples carefully chosen to require competence in particular linguistic phenomena ( Marvin & Linzen , 2018 ; Wang et al. , 2018 ; Dasgupta et al. , 2019 ; Poliak et al. , 2018 ; Linzen et al. , 2016 ; McCoy et al. , 2019b ) . This technique can illuminate behavioral shortcomings but says little about how the internal representations are structured , treating the model as a black box . In the probing approach , an auxiliary classifier is trained to classify the model ’ s internal representations based on some linguistically-relevant distinction ( Adi et al. , 2017 ; Conneau et al. , 2018 ; Conneau & Kiela , 2018 ; Belinkov et al. , 2017 ; Blevins et al. , 2018 ; Peters et al. , 2018 ; Tenney et al. , 2019 ) . In contrast with the behavioral approach , the probing approach tests whether some particular information is present in the model ’ s encodings , but it says little about whether this information is actually used by the model . Indeed , in at least some cases models will fail despite having the necessary information to succeed in their representations , showing that the ability of a classifier to extract that information does not mean that the model is using it ( Vanmassenhove et al. , 2017 ) . DISCOVER bridges the gap between representation and behavior : It reveals not only what information is encoded in the representation , but also shows how that information is causally implicated in the model ’ s behavior . Moreover , it provides a much more comprehensive window into the representation than the probing approach does ; while probing extracts particular types of information from a representation ( e.g. , “ does this representation distinguish between active and passive sentences ? ” ) , DISCOVER exhaustively decomposes the model ’ s representational space . In this regard , DISCOVER is most closely related to the approaches of Andreas ( 2019 ) , Chrupała & Alishahi ( 2019 ) , and Abnar et al . ( 2019 ) , who also propose methods for discovering a complete symbolic characterization of a set of vector representations , and Omlin & Giles ( 1996 ) and Weiss et al . ( 2018 ) , which also seek to extract more interpretable symbolic models that approximate neural network behavior . 4 THE ROLE LEARNER ( ROLE ) MODEL . ROLE1 produces a vector-space embedding of an input string of T symbols S = s1s2 . . . sT by producing a TPR T ( S ) and then passing it through a linear transformation W. ROLE is trained to approximate a pre-trained target string-encoder E . Given a set of N training strings { S ( 1 ) , . . . , S ( N ) } , ROLE minimizes the total mean-squared error ( MSE ) between its output WT ( S ( i ) ) and E ’ s corresponding output , E ( S ( i ) ) . 1Code available at https : //github.com/iclr2020-anonymous1/role-learner ROLE is an extension of the Tensor-Product Encoder ( TPE ) introduced in McCoy et al . ( 2019a ) ( as the “ Tensor Product Decomposition Network ” ) , which produces a linearly-transformed TPR given a string of symbols and pre-assigned role labels for each symbol ( see Appendix A.1 for details ) . Crucially , ROLE is not given role labels for the input symbols , but learns to compute them . More precisely , it learns a dictionary of nR dR-dimensional role-embedding vectors , R ∈ RdR×nR , and , for each input symbol st , computes a soft-attention vector at over these role vectors : the role vector assigned to st is then the attention-weighted linear combination of role vectors , rt = Rat . ROLE simultaneously learns a dictionary of nF dF-dimensional symbol-embedding filler vectors F ∈ RdF×nF , the φth column of which is fφ , the embedding of symbol type φ ; φ ∈ 1 , . . . , nF where nF is the size of the vocabulary of symbol types . The TPR generated by ROLE is thus T ( S ) = ∑T t=1 fτ ( st ) ⊗ rt , where τ ( st ) is symbol st ’ s type . Finally , ROLE learns a linear transformation W to map this TPR into Rd , where d is the dimension of the representations of the encoder E it is learning to approximate . ROLE uses an LSTM ( Hochreiter & Schmidhuber , 1997 ) to compute the role-assigning attentionvectors at from its learned embedding F of the input symbols st : at each t , the hidden state of the LSTM passes through a linear layer and then a softmax to produce at ( depicted in Figure 1 ) .2 Since a TPR for a discrete symbol structure deploys a discrete set of roles specifying discrete structural positions , ideally a single role would be selected for each st : at would be one-hot . ROLE training therefore deploys regularization to bias learning towards one-hot at vectors ( based on the regularization proposed in Palangi et al . ( 2017 ) , developed for the same purpose ) . See Appendix A.4 for the precise regularization terms that we used . Although the regularization can yield good results , performing continuous gradient descent on a nearly discrete symbolic landscape can cause networks to get stuck in local optima where the learned symbolic structure is not ideal . For this reason , the performance at convergence can vary appreciably across multiple runs . It is important to note that , while we impose this regularization on ROLE , there is no explicit bias favoring discrete compositional representations in the target encoder E : any such structure that ROLE finds hidden in the representations learned by E must be the result of biases implicit in the vanilla RNN-architecture of E when applied to its target task .

Summary: In this paper, the authors study the problem of understanding the compositional generalization abilities of NNs. A new type of NN, called ROLE, is proposed to learns to approximate the representations of a target NN E by learning a symbolic constituent structure and an embedding of that structure intoE’s representational vector space. A number of tasks are conducted, including a simple fully-compositional task, a SCAN task, a partially-compositional NLP task. Multiple tasks. The experiment results show that the proposed approach can help to understand how NNs achieve strong generalization on partially-compositional tasks, and good performance on fully-compositional task.