Original: D ISCOVERING THE COMPOSITIONAL STRUCTURE OF VECTOR REPRESENTATIONS WITH R OLE L EARNING
Revised: D ISCOVERING THE COMPOSITIONAL STRUCTURE OF VECTOR REPRESENTATIONS WITH R OLE L EARNING

Original: N ETWORKS
Revised: N ETWORKS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Neural networks are able to perform tasks that rely on compositional structure even though they lack obvious mechanisms for representing this structure.
Revised: Neural networks are able to perform tasks that rely on compositional structure even though they lack obvious mechanisms for representing this structure.

Original: To analyze the internal representations that enable such success, we propose ROLE, a technique that detects whether these representations implicitly encode symbolic structure.
Revised: To analyze the internal representations that enable such success, we propose ROLE, a technique that detects whether these representations implicitly encode symbolic structure.

Original: ROLE learns to approximate the representations of a target encoder E by learning a symbolic constituent structure and an embedding of that structure into E ’s representational vector space.
Revised: ROLE learns to approximate the representations of a target encoder E by learning a symbolic constituent structure and an embedding of that structure into E ’s representational vector space.

Original: The constituents of the approximating symbol structure are deﬁned by structural positions — roles — that can be ﬁlled by symbols.
Revised: The constituents of the approximating symbol structure are deﬁned by structural positions — roles — that can be ﬁlled by symbols.

Original: We show that when E is constructed to explicitly embed a particular type of structure (string or tree), ROLE successfully extracts the ground-truth roles deﬁning that structure.
Revised: We show that when E is constructed to explicitly embed a particular type of structure (string or tree), ROLE successfully extracts the ground-truth roles deﬁning that structure.

Original: We then analyze a GRU seq2seq network trained to perform a more complex compositional task (SCAN), where there is no ground truth role scheme available.
Revised: We then analyze a GRU seq2seq network trained to perform a more complex compositional task (SCAN), where there is no ground truth role scheme available.

Original: For this model, ROLE successfully discovers an interpretable symbolic structure that the model implicitly uses to perform the
Revised: For this model, ROLE successfully discovers an interpretable symbolic structure that the model implicitly uses to perform the

Original: SCAN task, providing a comprehensive account of the representations that drive the behavior of a frequently-used but hard-to-interpret type of model.
Revised: SCAN task, providing a comprehensive account of the representations that drive the behavior of a frequently-used but hard-to-interpret type of model.

Original: We verify the causal importance of the discovered symbolic structure by showing that, when we systematically manipulate hidden embeddings based on this symbolic structure, the model’s resulting output is changed in the way predicted by our analysis.
Revised: We verify the causal importance of the discovered symbolic structure by showing that, when we systematically manipulate hidden embeddings based on this symbolic structure, the model’s resulting output is changed in the way predicted by our analysis.

Original: Finally, we use ROLE to explore whether popular sentence embedding models are capturing compositional structure and ﬁnd evidence that they are not; we conclude by suggesting how insights from ROLE can be used to impart new inductive biases to improve the compositional abilities of such models.
Revised: Finally, we use ROLE to explore whether popular sentence embedding models are capturing compositional structure and ﬁnd evidence that they are not; we conclude by suggesting how insights from ROLE can be used to impart new inductive biases to improve the compositional abilities of such models.

Original:  O VERVIEW
Revised:  O VERVIEW

Original: Certain AI tasks consist in computing a function ϕ that is governed by strict rules: e.g., if ϕ is the function mapping a mathematical expression to its value (e.g., mapping ‘ 19 − 2 ∗ 7 ’ to 5 ), then ϕ obeys the rule that ϕ ( x + y )
Revised: Certain AI tasks consist in computing a function ϕ that is governed by strict rules: e.g., if ϕ is the function mapping a mathematical expression to its value (e.g., mapping ‘ 19 − 2 ∗ 7 ’ to 5 ), then ϕ obeys the rule that ϕ ( x + y )

Original: =
Revised: =

Original: sum ( ϕ ( x ) , ϕ ( y )) for any expressions x and y .
Revised: sum ( ϕ ( x ) , ϕ ( y )) for any expressions x and y .

Original: This rule is compositional : the output of a structure (here, x + y ) is a function of the outputs of the structure’s constituents (here, x and y ).
Revised: This rule is compositional : the output of a structure (here, x + y ) is a function of the outputs of the structure’s constituents (here, x and y ).

Original: The rule can be stated with full generality once the input is assigned a symbolic structure giving its decomposition into constituents.
Revised: The rule can be stated with full generality once the input is assigned a symbolic structure giving its decomposition into constituents.

Original: For a fully-compositional task, completely determined by compositional rules, an AI system that can assign appropriate symbolic structures to inputs and apply appropriate compositional rules to these structures will display full systematic generalization : it will correctly process arbitrary novel combinations of familiar constituents.
Revised: For a fully-compositional task, completely determined by compositional rules, an AI system that can assign appropriate symbolic structures to inputs and apply appropriate compositional rules to these structures will display full systematic generalization : it will correctly process arbitrary novel combinations of familiar constituents.

Original: This is a core capability of symbolic AI systems.
Revised: This is a core capability of symbolic AI systems.

Original: Other tasks, including most natural language tasks such as machine translation, are only partially characterizable by compositional rules because natural language is only partially compositional in nature.
Revised: Other tasks, including most natural language tasks such as machine translation, are only partially characterizable by compositional rules because natural language is only partially compositional in nature.

Original: For example, if ϕ is the function that assigns meanings to English adjectives, it generally obeys the rule that ϕ ( in - + x ) = not ϕ ( x ) , (e.g., ϕ ( inoffensive ) = not ϕ ( offensive ) ), yet there are exceptions: ϕ ( inflammable ) =
Revised: For example, if ϕ is the function that assigns meanings to English adjectives, it generally obeys the rule that ϕ ( in - + x ) = not ϕ ( x ) , (e.g., ϕ ( inoffensive ) = not ϕ ( offensive ) ), yet there are exceptions: ϕ ( inflammable ) =

Original: ϕ ( flammable ) .
Revised: ϕ ( flammable ) .

Original: On these “ partially-compositional ” AI tasks, this strategy of compositional analysis has demonstrated considerable, but limited, generalization capabilities.
Revised: On these “ partially-compositional ” AI tasks, this strategy of compositional analysis has demonstrated considerable, but limited, generalization capabilities.

Original: Deep learning research has shown that Neural Networks (NNs) can display remarkable degrees of combinatorial generalization, often surpassing symbolic AI systems for partially-compositional tasks
Revised: Deep learning research has shown that Neural Networks (NNs) can display remarkable degrees of combinatorial generalization, often surpassing symbolic AI systems for partially-compositional tasks

Original: (Wu et al., 2016), and exhibit good generalization (although generally falling short of symbolic AI systems) on fully-compositional tasks (Lake & Baroni, 2018; McCoy et al., 2019a).
Revised: (Wu et al., 2016), and exhibit good generalization (although generally falling short of symbolic AI systems) on fully-compositional tasks (Lake & Baroni, 2018; McCoy et al., 2019a).

Original: Given that standard NNs have no obvious mechanisms for representing symbolic structures, parsing inputs into such structures, nor applying compositional symbolic rules to them, this success raises the question we address in this paper: How do NNs achieve such strong generalization on partially-compositional tasks, and good performance on fully-compositional tasks?
Revised: Given that standard NNs have no obvious mechanisms for representing symbolic structures, parsing inputs into such structures, nor applying compositional symbolic rules to them, this success raises the question we address in this paper: How do NNs achieve such strong generalization on partially-compositional tasks, and good performance on fully-compositional tasks?

Original: An important step towards answering this question was reported in McCoy et al.
Revised: An important step towards answering this question was reported in McCoy et al.

Original: (2019a), which showed that when trained on highly compositional tasks, standard NNs learned representations that were well approximated by symbolic structures (Sec. 2).
Revised: (2019a), which showed that when trained on highly compositional tasks, standard NNs learned representations that were well approximated by symbolic structures (Sec. 2).

Original: Processing in these NNs assigns such representations to inputs and generates outputs that are governed by compositional rules stated over those representations.
Revised: Processing in these NNs assigns such representations to inputs and generates outputs that are governed by compositional rules stated over those representations.

Original: We refer to the networks to be analyzed as target NNs , because we will propose a new type of NN (in Sec. 4) — the Role Learner (ROLE) — which is used to analyze the target network, after ﬁrst discussing related analysis methods in Sec. 3.
Revised: We refer to the networks to be analyzed as target NNs , because we will propose a new type of NN (in Sec. 4) — the Role Learner (ROLE) — which is used to analyze the target network, after ﬁrst discussing related analysis methods in Sec. 3.

Original: In contrast with the analysis model of McCoy et al. (2019a), which relies on a hand-speciﬁed hypothesis about the underlying structure,
Revised: In contrast with the analysis model of McCoy et al. (2019a), which relies on a hand-speciﬁed hypothesis about the underlying structure,

Original: ROLE automatically learns a symbolic structure that best approximates the internal representation of the target network.
Revised: ROLE automatically learns a symbolic structure that best approximates the internal representation of the target network.

Original: Automating the discovery of structural hypotheses provides two advantages.
Revised: Automating the discovery of structural hypotheses provides two advantages.

Original: First,
Revised: First,

Original: ROLE achieves success at analyzing networks for which it is not clear what the underlying structure is.
Revised: ROLE achieves success at analyzing networks for which it is not clear what the underlying structure is.

Original: We show this in Sec. 6, where ROLE successfully uncovers the symbolic structures learned by a seq2seq RNN trained on the SCAN task (Lake & Baroni, 2018).
Revised: We show this in Sec. 6, where ROLE successfully uncovers the symbolic structures learned by a seq2seq RNN trained on the SCAN task (Lake & Baroni, 2018).

Original: Second, removing the need for hand-speciﬁed hypotheses allows the data to speak for itself, which simpliﬁes the burden on the user, who only needs to provide input sequences and associated embeddings.
Revised: Second, removing the need for hand-speciﬁed hypotheses allows the data to speak for itself, which simpliﬁes the burden on the user, who only needs to provide input sequences and associated embeddings.

Original: We ﬁrst consider fully-compositional (hence synthetic) tasks: a simple string-manipulation task in Sec. 5, and the richer SCAN task, which has been the basis of previous work on combinatorial generalization in NNs, in Sec. 6.
Revised: We ﬁrst consider fully-compositional (hence synthetic) tasks: a simple string-manipulation task in Sec. 5, and the richer SCAN task, which has been the basis of previous work on combinatorial generalization in NNs, in Sec. 6.

Original: Discovering symbolic structure within a model enables us to perform precise alterations to the internal representations in order to produce desired combinatorial alterations in the output (Sec. 6.3).
Revised: Discovering symbolic structure within a model enables us to perform precise alterations to the internal representations in order to produce desired combinatorial alterations in the output (Sec. 6.3).

Original: Then, in Sec. 7, we turn brieﬂy to partially-compositional tasks in NLP.
Revised: Then, in Sec. 7, we turn brieﬂy to partially-compositional tasks in NLP.

Original: In
Revised: In

Original: Sec. 8 we consider how what we have learned about standard NNs can suggest new inductive biases to strengthen compositionality in NN learning.
Revised: Sec. 8 we consider how what we have learned about standard NNs can suggest new inductive biases to strengthen compositionality in NN learning.

Original:  NN EMBEDDING OF SYMBOL STRUCTURES
Revised:  NN EMBEDDING OF SYMBOL STRUCTURES

Original: We build on McCoy et al. (2019a), which introduced the analysis task DISCOVER (DISsecting
Revised: We build on McCoy et al. (2019a), which introduced the analysis task DISCOVER (DISsecting

Original: COmpositionality in VEctor Representations) : take a NN and, to the extent possible, ﬁnd an explicitly-compositional approximation to its internal distributed representations.
Revised: COmpositionality in VEctor Representations) : take a NN and, to the extent possible, ﬁnd an explicitly-compositional approximation to its internal distributed representations.

Original: McCoy et al.
Revised: McCoy et al.

Original: (2019a) showed that, in GRU (Cho et al., 2014) encoder-decoder networks performing simple, fullycompositional string manipulations, the medial encoding (between encoder and decoder) could be extremely well approximated, up to a linear transformation, by Tensor Product Representations (TPRS)
Revised: (2019a) showed that, in GRU (Cho et al., 2014) encoder-decoder networks performing simple, fullycompositional string manipulations, the medial encoding (between encoder and decoder) could be extremely well approximated, up to a linear transformation, by Tensor Product Representations (TPRS)

Original: (Smolensky, 1990), which are explicitly-compositional vector embeddings of symbolic structures.
Revised: (Smolensky, 1990), which are explicitly-compositional vector embeddings of symbolic structures.

Original: To represent a string of symbols as a TPR, the symbols in the string 337 might be parsed into three constituents { 3 : pos1 , 7 : pos3 , 3 : pos2 } , where pos n is the role of n th position from the left edge of the string; other role schemes are also possible, such as roles denoting right-to-left position: { 3 : third - to - last , 3 : second - to - last , 7 : last } .
Revised: To represent a string of symbols as a TPR, the symbols in the string 337 might be parsed into three constituents { 3 : pos1 , 7 : pos3 , 3 : pos2 } , where pos n is the role of n th position from the left edge of the string; other role schemes are also possible, such as roles denoting right-to-left position: { 3 : third - to - last , 3 : second - to - last , 7 : last } .

Original: The embedding of a constituent 7 : pos3 is e ( 7 : pos3) =
Revised: The embedding of a constituent 7 : pos3 is e ( 7 : pos3) =

Original: e F ( 7 ) ⊗ e R (pos3) , where e R , e F are respectively a vector embedding of the roles and a vector embedding of the ﬁllers of those roles: the digits.
Revised: e F ( 7 ) ⊗ e R (pos3) , where e R , e F are respectively a vector embedding of the roles and a vector embedding of the ﬁllers of those roles: the digits.

Original: The embedding of the whole string is the sum of the embeddings of its constituents.
Revised: The embedding of the whole string is the sum of the embeddings of its constituents.

Original: In general, for a symbol structure S with roles { r k } that are respectively ﬁlled by the symbols { f k } , e TPR ( S ) =
Revised: In general, for a symbol structure S with roles { r k } that are respectively ﬁlled by the symbols { f k } , e TPR ( S ) =

Original: (cid:80)
Revised: (cid:80)

Original: k e F
Revised: k e F

Original: [Equation]
Revised: [Equation]

Original: The example above used the role scheme of linear position from the left edge, but other role schemes are possible, such as linear position from the right edge or position in a tree.
Revised: The example above used the role scheme of linear position from the left edge, but other role schemes are possible, such as linear position from the right edge or position in a tree.

Original: McCoy et al. (2019a) showed that — for a given seq2seq architecture learning a given string-mapping task — there exists a highly accurate TPR approximation of the medial encoding, given an appropriate pre-deﬁned role scheme.
Revised: McCoy et al. (2019a) showed that — for a given seq2seq architecture learning a given string-mapping task — there exists a highly accurate TPR approximation of the medial encoding, given an appropriate pre-deﬁned role scheme.

Original: The main technical contribution of the present paper is the Role Learner (ROLE) model, a
Revised: The main technical contribution of the present paper is the Role Learner (ROLE) model, a

Original: RNN network that learns its own role scheme to optimize the ﬁt of a TPR approximation to a given set of internal representations in a pre-trained target NN.
Revised: RNN network that learns its own role scheme to optimize the ﬁt of a TPR approximation to a given set of internal representations in a pre-trained target NN.

Original: This makes the DISCOVER framework more general by removing the need for human-generated hypotheses as to the role schemes the network might be implementing.
Revised: This makes the DISCOVER framework more general by removing the need for human-generated hypotheses as to the role schemes the network might be implementing.

Original: Learned role schemes, we will see in Sec. 6.1, can enable good TPR approximation of networks for which human-generated role schemes fail.
Revised: Learned role schemes, we will see in Sec. 6.1, can enable good TPR approximation of networks for which human-generated role schemes fail.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: This work falls within the larger paradigm of using analysis techniques to interpret NNs (see Belinkov & Glass (2019) for a recent survey), often including a focus on compositional structure (Hupkes et al., 2019; 2018; Lake & Baroni, 2018; Hewitt & Manning, 2019).
Revised: This work falls within the larger paradigm of using analysis techniques to interpret NNs (see Belinkov & Glass (2019) for a recent survey), often including a focus on compositional structure (Hupkes et al., 2019; 2018; Lake & Baroni, 2018; Hewitt & Manning, 2019).

Original: Two of the most popular analysis techniques are the behavioral and probing approaches.
Revised: Two of the most popular analysis techniques are the behavioral and probing approaches.

Original: In the behavioral approach, a model is evaluated on a set of examples carefully chosen to require competence in particular linguistic phenomena (Marvin & Linzen, 2018;
Revised: In the behavioral approach, a model is evaluated on a set of examples carefully chosen to require competence in particular linguistic phenomena (Marvin & Linzen, 2018;

Original: Wang et al., 2018; Dasgupta et al., 2019; Poliak et al., 2018;
Revised: Wang et al., 2018; Dasgupta et al., 2019; Poliak et al., 2018;

Original: Linzen et al., 2016; McCoy et al., 2019b).
Revised: Linzen et al., 2016; McCoy et al., 2019b).

Original: This technique can illuminate behavioral shortcomings but says little about how the internal representations are structured, treating the model as a black box.
Revised: This technique can illuminate behavioral shortcomings but says little about how the internal representations are structured, treating the model as a black box.

Original: In the probing approach, an auxiliary classiﬁer is trained to classify the model’s internal representations based on some linguistically-relevant distinction (Adi et al., 2017; Conneau et al., 2018;
Revised: In the probing approach, an auxiliary classiﬁer is trained to classify the model’s internal representations based on some linguistically-relevant distinction (Adi et al., 2017; Conneau et al., 2018;

Original: Conneau & Kiela, 2018;
Revised: Conneau & Kiela, 2018;

Original: Belinkov et al., 2017; Blevins et al., 2018; Peters et al., 2018; Tenney et al., 2019).
Revised: Belinkov et al., 2017; Blevins et al., 2018; Peters et al., 2018; Tenney et al., 2019).

Original: In contrast with the behavioral approach, the probing approach tests whether some particular information is present in the model’s encodings, but it says little about whether this information is actually used by the model.
Revised: In contrast with the behavioral approach, the probing approach tests whether some particular information is present in the model’s encodings, but it says little about whether this information is actually used by the model.

Original: Indeed, in at least some cases models will fail despite having the necessary information to succeed in their representations, showing that the ability of a classiﬁer to extract that information does not mean that the model is using it (Vanmassenhove et al., 2017).
Revised: Indeed, in at least some cases models will fail despite having the necessary information to succeed in their representations, showing that the ability of a classiﬁer to extract that information does not mean that the model is using it (Vanmassenhove et al., 2017).

Original: DISCOVER bridges the gap between representation and behavior: It reveals not only what information is encoded in the representation, but also shows how that information is causally implicated in the model’s behavior.
Revised: DISCOVER bridges the gap between representation and behavior: It reveals not only what information is encoded in the representation, but also shows how that information is causally implicated in the model’s behavior.

Original: Moreover, it provides a much more comprehensive window into the representation than the probing approach does; while probing extracts particular types of information from a representation (e.g., “does this representation distinguish between active and passive sentences?”),
Revised: Moreover, it provides a much more comprehensive window into the representation than the probing approach does; while probing extracts particular types of information from a representation (e.g., “does this representation distinguish between active and passive sentences?”),

Original: DISCOVER exhaustively decomposes the model’s representational space.
Revised: DISCOVER exhaustively decomposes the model’s representational space.

Original: In this regard, DISCOVER is most closely related to the approaches of Andreas (2019), Chrupała & Alishahi (2019), and Abnar et al.
Revised: In this regard, DISCOVER is most closely related to the approaches of Andreas (2019), Chrupała & Alishahi (2019), and Abnar et al.

Original: (2019), who also propose methods for discovering a complete symbolic characterization of a set of vector representations, and Omlin & Giles (1996) and Weiss et al.
Revised: (2019), who also propose methods for discovering a complete symbolic characterization of a set of vector representations, and Omlin & Giles (1996) and Weiss et al.

Original: (2018), which also seek to extract more interpretable symbolic models that approximate neural network behavior.
Revised: (2018), which also seek to extract more interpretable symbolic models that approximate neural network behavior.

Original:  T HE R OLE L EARNER (ROLE) M ODEL
Revised:  T HE R OLE L EARNER (ROLE) M ODEL

Original: ROLE 1 produces a vector-space embedding of an input string of T symbols S =
Revised: ROLE 1 produces a vector-space embedding of an input string of T symbols S =

Original: s 1 s 2 . . .
Revised: s 1 s 2 . . .

Original: s T by producing a TPR T ( S ) and then passing it through a linear transformation W .
Revised: s T by producing a TPR T ( S ) and then passing it through a linear transformation W .

Original: ROLE is trained to approximate a pre-trained target string-encoder E .
Revised: ROLE is trained to approximate a pre-trained target string-encoder E .

Original: Given a set of N training strings { S (1) , . . .
Revised: Given a set of N training strings { S (1) , . . .

Original: , S ( N ) } ,
Revised: , S ( N ) } ,

Original: ROLE minimizes the total mean-squared error (MSE) between its output W T ( S ( i ) ) and E ’s corresponding output, E ( S ( i ) ) .
Revised: ROLE minimizes the total mean-squared error (MSE) between its output W T ( S ( i ) ) and E ’s corresponding output, E ( S ( i ) ) .

Original: ROLE is an extension of the Tensor-Product Encoder (TPE) introduced in McCoy et al.
Revised: ROLE is an extension of the Tensor-Product Encoder (TPE) introduced in McCoy et al.

Original: (2019a)
Revised: (2019a)

Original: (as the “Tensor Product Decomposition Network”), which produces a linearly-transformed TPR given a string of symbols and pre-assigned role labels for each symbol (see Appendix A.1 for details).
Revised: (as the “Tensor Product Decomposition Network”), which produces a linearly-transformed TPR given a string of symbols and pre-assigned role labels for each symbol (see Appendix A.1 for details).

Original: Crucially,
Revised: Crucially,

Original: ROLE is not given role labels for the input symbols, but learns to compute them.
Revised: ROLE is not given role labels for the input symbols, but learns to compute them.

Original: More precisely, it learns a dictionary of n R d R -dimensional role-embedding vectors,
Revised: More precisely, it learns a dictionary of n R d R -dimensional role-embedding vectors,

Original: R ∈ R d R × n R , and, for each input symbol s t , computes a soft-attention vector a t over these role vectors: the role vector assigned to s t is then the attention-weighted linear combination of role vectors, r t = R a t .
Revised: R ∈ R d R × n R , and, for each input symbol s t , computes a soft-attention vector a t over these role vectors: the role vector assigned to s t is then the attention-weighted linear combination of role vectors, r t = R a t .

Original: ROLE simultaneously learns a dictionary of n F d F -dimensional symbol-embedding ﬁller vectors F ∈ R d F × n F , the φ th column of which is f φ , the embedding of symbol type φ ; φ ∈ 1 , . . . , n F where n F is the size of the vocabulary of symbol types.
Revised: ROLE simultaneously learns a dictionary of n F d F -dimensional symbol-embedding ﬁller vectors F ∈ R d F × n F , the φ th column of which is f φ , the embedding of symbol type φ ; φ ∈ 1 , . . . , n F where n F is the size of the vocabulary of symbol types.

Original: The TPR generated by ROLE is thus T ( S ) =
Revised: The TPR generated by ROLE is thus T ( S ) =

Original: (cid:80) Tt =1 f τ ( s t ) ⊗ r t , where τ ( s t ) is symbol s t ’s type.
Revised: (cid:80) Tt =1 f τ ( s t ) ⊗ r t , where τ ( s t ) is symbol s t ’s type.

Original: Finally, ROLE learns a linear transformation W to map this TPR into R d , where d is the dimension of the representations of the encoder E it is learning to approximate.
Revised: Finally, ROLE learns a linear transformation W to map this TPR into R d , where d is the dimension of the representations of the encoder E it is learning to approximate.

Original: ROLE uses an LSTM (Hochreiter & Schmidhuber, 1997) to compute the role-assigning attentionvectors a t from its learned embedding F of the input symbols s t : at each t , the hidden state of the
Revised: ROLE uses an LSTM (Hochreiter & Schmidhuber, 1997) to compute the role-assigning attentionvectors a t from its learned embedding F of the input symbols s t : at each t , the hidden state of the

Original: LSTM passes through a linear layer and then a softmax to produce a t (depicted in Figure 1).
Revised: LSTM passes through a linear layer and then a softmax to produce a t (depicted in Figure 1).

Original: Since a TPR for a discrete symbol structure deploys a discrete set of roles specifying discrete structural positions, ideally a single role would be selected for each s t : a t would be one-hot.
Revised: Since a TPR for a discrete symbol structure deploys a discrete set of roles specifying discrete structural positions, ideally a single role would be selected for each s t : a t would be one-hot.

Original: ROLE training therefore deploys regularization to bias learning towards one-hot a t vectors (based on the regularization proposed in Palangi et al.
Revised: ROLE training therefore deploys regularization to bias learning towards one-hot a t vectors (based on the regularization proposed in Palangi et al.

Original: (2017), developed for the same purpose).
Revised: (2017), developed for the same purpose).

Original: See Appendix A. for the precise regularization terms that we used.
Revised: See Appendix A. for the precise regularization terms that we used.

Original: Although the regularization can yield good results, performing continuous gradient descent on a nearly discrete symbolic landscape can cause networks to get stuck in local optima where the learned symbolic structure is not ideal.
Revised: Although the regularization can yield good results, performing continuous gradient descent on a nearly discrete symbolic landscape can cause networks to get stuck in local optima where the learned symbolic structure is not ideal.

Original: For this reason, the performance at convergence can vary appreciably across multiple runs.
Revised: For this reason, the performance at convergence can vary appreciably across multiple runs.

Original: It is important to note that, while we impose this regularization on ROLE, there is no explicit bias favoring discrete compositional representations in the target encoder E : any such structure that ROLE ﬁnds hidden in the representations learned by E must be the result of biases implicit in the vanilla
Revised: It is important to note that, while we impose this regularization on ROLE, there is no explicit bias favoring discrete compositional representations in the target encoder E : any such structure that ROLE ﬁnds hidden in the representations learned by E must be the result of biases implicit in the vanilla

Original: RNN-architecture of E when applied to its target task.
Revised: RNN-architecture of E when applied to its target task.

Original:  A SIMPLE FULLY - COMPOSITIONAL TASK
Revised:  A SIMPLE FULLY - COMPOSITIONAL TASK

Original: We ﬁrst apply ROLE to two target Tensor Product Encoder (TPE) models which are fully compositional by design.
Revised: We ﬁrst apply ROLE to two target Tensor Product Encoder (TPE) models which are fully compositional by design.

Original: Since we know what role scheme each target model deploys, we can test how well ROLE learns these ground-truth roles.
Revised: Since we know what role scheme each target model deploys, we can test how well ROLE learns these ground-truth roles.

Original: The TPEs are trained on the fully compositional task of autoencoding sequences of digits.
Revised: The TPEs are trained on the fully compositional task of autoencoding sequences of digits.

Original: We use two types of TPEs: one that uses a simple left-to-right role scheme (e.g., ﬁrst , second , third ) and one that uses a complex tree position role scheme (e.g., left child of the root of the tree , right child of the left child of the left child of the root of the tree ), where the trees are generated from digit sequence inputs using a deterministic parsing algorithm (see Appendix A.2 for explanations and examples of the designed role schemes).
Revised: We use two types of TPEs: one that uses a simple left-to-right role scheme (e.g., ﬁrst , second , third ) and one that uses a complex tree position role scheme (e.g., left child of the root of the tree , right child of the left child of the left child of the root of the tree ), where the trees are generated from digit sequence inputs using a deterministic parsing algorithm (see Appendix A.2 for explanations and examples of the designed role schemes).

Original: The left-to-right TPE was paired with a unidirectional RNN decoder, while the tree-position TPE was paired with the tree-RNN decoder used by McCoy et al.
Revised: The left-to-right TPE was paired with a unidirectional RNN decoder, while the tree-position TPE was paired with the tree-RNN decoder used by McCoy et al.

Original: (2019a).
Revised: (2019a).

Original: Both of these target models attained near-perfect performance on the autoencoding task (Table 1).
Revised: Both of these target models attained near-perfect performance on the autoencoding task (Table 1).

Original: Once the encoders are ﬁnished training, we extract the encoding for each sequence in the dataset and use this to train ROLE.
Revised: Once the encoders are ﬁnished training, we extract the encoding for each sequence in the dataset and use this to train ROLE.

Original: See Appendices A.3 and A.5 for additional training details.
Revised: See Appendices A.3 and A.5 for additional training details.

Original: Table 1 reports the approximation performance of ROLE in two ways.
Revised: Table 1 reports the approximation performance of ROLE in two ways.

Original: Substitution Accuracy is the proportion of the items for which the decoder produces the correct output string when it is fed the
Revised: Substitution Accuracy is the proportion of the items for which the decoder produces the correct output string when it is fed the

Original: ROLE approximation.
Revised: ROLE approximation.

Original: The V-Measure (Rosenberg & Hirschberg, 2007) assesses the extent to which the clustering of the role vectors assigned by ROLE matches the ground truth role assignments.
Revised: The V-Measure (Rosenberg & Hirschberg, 2007) assesses the extent to which the clustering of the role vectors assigned by ROLE matches the ground truth role assignments.

Original: The ROLE approximation of the left-to-right TPE attained perfect performance, with a substitution accuracy of 100% and a V-Measure of 1.0, indicating that the role scheme it learned perfectly matched the ground truth.
Revised: The ROLE approximation of the left-to-right TPE attained perfect performance, with a substitution accuracy of 100% and a V-Measure of 1.0, indicating that the role scheme it learned perfectly matched the ground truth.

Original: On the signiﬁcantly more complex case of tree position roles, ROLE achieves
Revised: On the signiﬁcantly more complex case of tree position roles, ROLE achieves

Original: [Table]

Original: [Table]
Revised: [Table]

Original: essentially the same accuracy as the target encoder E and has considerable success at recovering the ground truth roles for the vectors it was analyzing.
Revised: essentially the same accuracy as the target encoder E and has considerable success at recovering the ground truth roles for the vectors it was analyzing.

Original: These results show that, when a target model has a known fully compositional structure, ROLE can successfully ﬁnd that structure.
Revised: These results show that, when a target model has a known fully compositional structure, ROLE can successfully ﬁnd that structure.

Original:  T HE SCAN TASK
Revised:  T HE SCAN TASK

Original: We have established that ROLE can uncover the compositional structure used by a model that is compositional by design.
Revised: We have established that ROLE can uncover the compositional structure used by a model that is compositional by design.

Original: But, returning to our central question from Sec. 1, how can models without explicit compositional structure (namely, standard RNNs) still be as successful at fully compositional tasks as fully compositional models?
Revised: But, returning to our central question from Sec. 1, how can models without explicit compositional structure (namely, standard RNNs) still be as successful at fully compositional tasks as fully compositional models?

Original: Our hypothesis is that, though these models have no constraint forcing them to be compositional, they still have the ability to implicitly learn compositional structure.
Revised: Our hypothesis is that, though these models have no constraint forcing them to be compositional, they still have the ability to implicitly learn compositional structure.

Original: To test this hypothesis, we apply ROLE to a standard RNN-based seq2seq (Sutskever et al., 2014) model trained on a fully compositional task.
Revised: To test this hypothesis, we apply ROLE to a standard RNN-based seq2seq (Sutskever et al., 2014) model trained on a fully compositional task.

Original: Because the RNN has no constraint forcing it to use
Revised: Because the RNN has no constraint forcing it to use

Original: TPRs, we do not know a priori whether there exists any solution that ROLE could learn; thus, if
Revised: TPRs, we do not know a priori whether there exists any solution that ROLE could learn; thus, if

Original: ROLE does learn anything it will be a signiﬁcant empirical ﬁnding about how these RNNs operate.
Revised: ROLE does learn anything it will be a signiﬁcant empirical ﬁnding about how these RNNs operate.

Original: We consider the SCAN task (Lake & Baroni, 2018), which was designed to test compositional generalization and systematicity.
Revised: We consider the SCAN task (Lake & Baroni, 2018), which was designed to test compositional generalization and systematicity.

Original: SCAN is a synthetic sequence-to-sequence mapping task, with an input sequence describing an action plan, e.g., jump opposite left , being mapped to a sequence of primitive actions,
Revised: SCAN is a synthetic sequence-to-sequence mapping task, with an input sequence describing an action plan, e.g., jump opposite left , being mapped to a sequence of primitive actions,

Original: e.g., TL TL JUMP (see Sec.
Revised: e.g., TL TL JUMP (see Sec.

Original: 6.3 for a complex example).
Revised: 6.3 for a complex example).

Original: We use TL to abbreviate
Revised: We use TL to abbreviate

Original: TURN _ LEFT , sometimes written LTURN ; similarly, we use TR for TURN _ RIGHT .
Revised: TURN _ LEFT , sometimes written LTURN ; similarly, we use TR for TURN _ RIGHT .

Original: The SCAN mapping is deﬁned by a complete set of compositional rules (Lake & Baroni, 2018, Supplementary Fig. 7).
Revised: The SCAN mapping is deﬁned by a complete set of compositional rules (Lake & Baroni, 2018, Supplementary Fig. 7).

Original: 6. T HE COMPOSITIONAL STRUCTURE OF SCAN ENCODER REPRESENTATIONS
Revised: 6. T HE COMPOSITIONAL STRUCTURE OF SCAN ENCODER REPRESENTATIONS

Original: For our target SCAN encoder E , we trained a standard GRU with one hidden layer of dimension 100 for 100,000 steps (batch-size 1) with a dropout of 0.1 on the simple train-test split.
Revised: For our target SCAN encoder E , we trained a standard GRU with one hidden layer of dimension 100 for 100,000 steps (batch-size 1) with a dropout of 0.1 on the simple train-test split.

Original: (The hidden dimension and dropout rate were determined by a limited hyper-parameter search; see Appendix A.6).
Revised: (The hidden dimension and dropout rate were determined by a limited hyper-parameter search; see Appendix A.6).

Original: E achieves 98 .
Revised: E achieves 98 .

Original: 47% (full-string) accuracy on the test set.
Revised: 47% (full-string) accuracy on the test set.

Original: Thus E provides what we want: a standard
Revised: Thus E provides what we want: a standard

Original: RNN achieving near-perfect accuracy on a non-trivial fully compositional task.
Revised: RNN achieving near-perfect accuracy on a non-trivial fully compositional task.

Original: After training, we extract the ﬁnal hidden embedding from the encoder for each example in the training and test sets.
Revised: After training, we extract the ﬁnal hidden embedding from the encoder for each example in the training and test sets.

Original: These are the encodings we attempt to approximate as explicitly compositional
Revised: These are the encodings we attempt to approximate as explicitly compositional

Original: TPRs.
Revised: TPRs.

Original: We provide ROLE with 50 roles to use as it wants (additional training information is in
Revised: We provide ROLE with 50 roles to use as it wants (additional training information is in

Original: Appendix A.7).
Revised: Appendix A.7).

Original: We evaluate the substitution accuracy that this learned role scheme provides in three ways.
Revised: We evaluate the substitution accuracy that this learned role scheme provides in three ways.

Original: The continuous method tests ROLE in the same way as it was trained, with input symbol s t assigned role vector r t = R a t .
Revised: The continuous method tests ROLE in the same way as it was trained, with input symbol s t assigned role vector r t = R a t .

Original: The continuous method does not use a discrete set of role vectors because the weighted sum that generates a t allows for continuously-valued weights.
Revised: The continuous method does not use a discrete set of role vectors because the weighted sum that generates a t allows for continuously-valued weights.

Original: The remaining two methods test the efﬁcacy of a truly discrete set of role vectors.
Revised: The remaining two methods test the efﬁcacy of a truly discrete set of role vectors.

Original: First, in the snapped method, a t is replaced at evaluation time by the one-hot vector m t singling out role m t = arg max( a t ) : r t =
Revised: First, in the snapped method, a t is replaced at evaluation time by the one-hot vector m t singling out role m t = arg max( a t ) : r t =

Original: R m t .
Revised: R m t .

Original: This method serves the goal of enforcing the discreteness of roles, but it is expected to decrease performance because it tests ROLE in a different way than it was trained.
Revised: This method serves the goal of enforcing the discreteness of roles, but it is expected to decrease performance because it tests ROLE in a different way than it was trained.

Original: Our ﬁnal evaluation method, the discrete method, uses discrete roles without having such a train/test discrepancy; in this method, we use the one-hot vector m t to output roles for every symbol in the dataset and then train a
Revised: Our ﬁnal evaluation method, the discrete method, uses discrete roles without having such a train/test discrepancy; in this method, we use the one-hot vector m t to output roles for every symbol in the dataset and then train a

Original: TPE which does not learn roles but rather uses the one-hot vector m t as input during training.
Revised: TPE which does not learn roles but rather uses the one-hot vector m t as input during training.

Original: In this case, ROLE acts as an automatic data labeler, assigning a role to every input word.
Revised: In this case, ROLE acts as an automatic data labeler, assigning a role to every input word.

Original: For comparison, we also train TPEs using a variety of discrete hand-crafted role schemes: left-to
Revised: For comparison, we also train TPEs using a variety of discrete hand-crafted role schemes: left-to-right (LTR), right-to-left (RTL), bidirectional (Bi), tree position, Wickelrole (Wickel), and bag-of-words (BOW) (additional information provided in Appendix A.2).

Original: [Table]
Revised: [Table]

Original: The substitution accuracy from these different methods is shown in Table 2.
Revised: The substitution accuracy from these different methods is shown in Table 2.

Original: All of the predeﬁned role schemes provide poor approximations, none surpassing 44 .
Revised: All of the predeﬁned role schemes provide poor approximations, none surpassing 44 .

Original: 12% accuracy.
Revised: 12% accuracy.

Original: The role scheme learned by ROLE does signiﬁcantly better than any of the predeﬁned role schemes: when tested with the basic, continuous role-attention method, the accuracy is 94 .
Revised: The role scheme learned by ROLE does signiﬁcantly better than any of the predeﬁned role schemes: when tested with the basic, continuous role-attention method, the accuracy is 94 .

Original: The success of ROLE tells us two things.
Revised: The success of ROLE tells us two things.

Original: First, it shows that the target model’s compositional behavior relies on compositional internal representations: it was by no means guaranteed to be the case that ROLE would be successful here, so the fact that it is successful tells us that the encoder has learned compositional representations.
Revised: First, it shows that the target model’s compositional behavior relies on compositional internal representations: it was by no means guaranteed to be the case that ROLE would be successful here, so the fact that it is successful tells us that the encoder has learned compositional representations.

Original: Second, it adds further validation to the efﬁcacy of ROLE, because it shows that it can be a useful analysis tool in cases of signiﬁcantly greater complexity than the autoencoding task.
Revised: Second, it adds further validation to the efﬁcacy of ROLE, because it shows that it can be a useful analysis tool in cases of signiﬁcantly greater complexity than the autoencoding task.

Original: 6. I NTERPRETING THE LEARNED ROLE SCHEME
Revised: 6. I NTERPRETING THE LEARNED ROLE SCHEME

Original: Analyzing the roles assigned by ROLE to the sequences in the SCAN training set, we created a symbolic algorithm for predicting which role will be assigned to a given ﬁller.
Revised: Analyzing the roles assigned by ROLE to the sequences in the SCAN training set, we created a symbolic algorithm for predicting which role will be assigned to a given ﬁller.

Original: This is described in
Revised: This is described in

Original: Appendix A.8.1 and discussed at some length in Appendix A.8.2.
Revised: Appendix A.8.1 and discussed at some length in Appendix A.8.2.

Original: Though the algorithm was created based only on sequences in the SCAN training set, it is equally successful at predicting which roles will be assigned to test sequences, exactly matching ROLE’s predicted roles for 98.7% of sequences.
Revised: Though the algorithm was created based only on sequences in the SCAN training set, it is equally successful at predicting which roles will be assigned to test sequences, exactly matching ROLE’s predicted roles for 98.7% of sequences.

Original: The details of this algorithm illuminate how the ﬁller-role scheme encodes information relevant to the task.
Revised: The details of this algorithm illuminate how the ﬁller-role scheme encodes information relevant to the task.

Original: First, one of the initial facts that the decoder must determine is whether the sequence is a single command, a pair of commands connected by and , or a pair of commands connected by after ; such a determination is crucial for knowing the basic structure of the output (how many actions to perform and in what order).
Revised: First, one of the initial facts that the decoder must determine is whether the sequence is a single command, a pair of commands connected by and , or a pair of commands connected by after ; such a determination is crucial for knowing the basic structure of the output (how many actions to perform and in what order).

Original: We have found that role 30 is used for, and only for, the ﬁller and , while roleis used in and only in sequences containing after (usually with after as the ﬁller bound to role 17).
Revised: We have found that role 30 is used for, and only for, the ﬁller and , while roleis used in and only in sequences containing after (usually with after as the ﬁller bound to role 17).

Original: Thus, the decoder can use these roles to tell which basic structure is in play: if role 30 is present, it is an and sequence; if role 17 is present, it is an after sequence; otherwise it is a single command.
Revised: Thus, the decoder can use these roles to tell which basic structure is in play: if role 30 is present, it is an and sequence; if role 17 is present, it is an after sequence; otherwise it is a single command.

Original: Once the decoder has established the basic syntactic structure of the output, it must then ﬁll in the particular actions.
Revised: Once the decoder has established the basic syntactic structure of the output, it must then ﬁll in the particular actions.

Original: This can be accomplished using the remaining roles, which mainly encode absolute position within a command.
Revised: This can be accomplished using the remaining roles, which mainly encode absolute position within a command.

Original: For example, the last word of a command before after (e.g., jump left after walk twice ) is always assigned role 8, while the last word of a command after after (e.g., jump left after walk twice ) is always assigned role 46.
Revised: For example, the last word of a command before after (e.g., jump left after walk twice ) is always assigned role 8, while the last word of a command after after (e.g., jump left after walk twice ) is always assigned role 46.

Original: Therefore, once the decoder knows (based on the presence of role 17) that it is dealing with an after sequence, it can check for the ﬁllers bound to roles 8 and 46 to begin to ﬁgure out what the two subcommands surrounding after look like.
Revised: Therefore, once the decoder knows (based on the presence of role 17) that it is dealing with an after sequence, it can check for the ﬁllers bound to roles 8 and 46 to begin to ﬁgure out what the two subcommands surrounding after look like.

Original: The identity of the last word in a command is informative because that is where a cardinality (i.e., twice or thrice ) appears if there is one.
Revised: The identity of the last word in a command is informative because that is where a cardinality (i.e., twice or thrice ) appears if there is one.

Original: Thus, by checking what ﬁller is at the end of a command, the model can learn whether there is a cardinality present and, if so, which one.
Revised: Thus, by checking what ﬁller is at the end of a command, the model can learn whether there is a cardinality present and, if so, which one.

Original: This description of how the decoding could take place does not necessarily match how it actually does take place; for example, it is likely that some of the steps we have described as occurring serially, for expository simplicity, actually occur in parallel.
Revised: This description of how the decoding could take place does not necessarily match how it actually does take place; for example, it is likely that some of the steps we have described as occurring serially, for expository simplicity, actually occur in parallel.

Original: We leave for future work the question of which operations are actually being performed and how those operations are instantiated in an RNN.
Revised: We leave for future work the question of which operations are actually being performed and how those operations are instantiated in an RNN.

Original: 6. P RECISION CONSTITUENT - SURGERY ON INTERNAL REPRESENTATIONS TO PRODUCE DESIRED OUTPUTS
Revised: 6. P RECISION CONSTITUENT - SURGERY ON INTERNAL REPRESENTATIONS TO PRODUCE DESIRED OUTPUTS

Original: The substitution-accuracy results above show that if the entire learned representation is replaced by
Revised: The substitution-accuracy results above show that if the entire learned representation is replaced by

Original: ROLE’s approximation, the output remains correct.
Revised: ROLE’s approximation, the output remains correct.

Original: But do the individual words in this TPR have the appropriate causal consequences when processed by the decoder?has had considerable signiﬁcance: the original compositionality challenge to neural network models of cognition by Fodor and colleagues (Fodor & Pylyshyn, 1988) insisted that constituents run : 11 left : 36 twice : 8 after : : 10 opposite : 17 right : 4 thrice : 46 →
Revised: But do the individual words in this TPR have the appropriate causal consequences when processed by the decoder?

Original: TR TR JUMP TR TR JUMP TR TR JUMP TL RUN TL RUN − run : 11 + look : 11 →
Revised: TR TR JUMP TR TR JUMP TR TR JUMP TL RUN TL RUN − run : 11 + look : 11 →

Original: TR TR JUMP TR TR JUMP TR TR JUMP TL LOOK TL LOOK − jump : 10 + walk : 10 →
Revised: TR TR JUMP TR TR JUMP TR TR JUMP TL LOOK TL LOOK − jump : 10 + walk : 10 →

Original: TR TR WALK TR TR WALK TR TR WALK TL LOOK TL LOOK −left :
Revised: TR TR WALK TR TR WALK TR TR WALK TL LOOK TL LOOK −

Original: 36 + right : 36 →
Revised: →

Original: TR TR WALK TR TR WALK TR TR WALK TR LOOK TR LOOK − twice: 8 + thrice : 8 →
Revised: TR TR WALK TR TR WALK TR TR WALK TR LOOK TR LOOK − twice 8 + thrice : 8 →

Original: TR TR WALK TR TR WALK TR TR WALK TR LOOK
Revised: TR TR WALK TR TR WALK TR TR WALK TR LOOK

Original: TR LOOK TR LOOK − opposite : 17 + around : 17 →
Revised: TR LOOK TR LOOK − opposite : 17 + around : 17 →

Original: TR WALK TR WALK TR WALK TR WALK TR WALK TR WALK

Original: TR WALK

Original: [Figure]
Revised: [Figure]

Original: WALK
Revised: WALK

Original: [Figure]
Revised: [Figure]

Original: To address this causal question (Pearl, 2000), we actively intervene on the constituent structure of the internal representations by replacing one constituent with another syntactically equivalent one 4 , and see whether this produces the expected change in the output of the decoder.
Revised: To address this causal question (Pearl, 2000), we actively intervene on the constituent structure of the internal representations by replacing one constituent with another syntactically equivalent one 4 , and see whether this produces the expected change in the output of the decoder.

Original: We take the encoding generated by the RNN encoder E for an input such as jump opposite left , subtract the vector embedding of the opposite constituent, add the embedding of the around constituent, and see whether this causes the output to change from the correct output for jump opposite left ( TL TL JUMP ) to the correct output for jump around left (
Revised: We take the encoding generated by the RNN encoder E for an input such as jump opposite left , subtract the vector embedding of the opposite constituent, add the embedding of the around constituent, and see whether this causes the output to change from the correct output for jump opposite left ( TL TL JUMP ) to the correct output for jump around left (

Original: TL JUMP TL JUMP TL JUMP TL JUMP ).
Revised: TL JUMP TL JUMP TL JUMP TL JUMP ).

Original: The roles in these constituents are determined by the algorithm of Appendix A.8.
Revised: The roles in these constituents are determined by the algorithm of Appendix A.8.

Original: If changing a word leads other roles in the sequence to change (according to the algorithm), we update the encoding with those new roles as well.
Revised: If changing a word leads other roles in the sequence to change (according to the algorithm), we update the encoding with those new roles as well.

Original: Such surgery can be viewed as based in a more general extension of the analogy approach used by Mikolov et al.
Revised: Such surgery can be viewed as based in a more general extension of the analogy approach used by Mikolov et al.

Original: for analysis of word embeddings.
Revised: for analysis of word embeddings.

Original: An example of applying a sequence of ﬁve such constituent surgeries to a sequence are shown in Figure 2 (left).
Revised: An example of applying a sequence of ﬁve such constituent surgeries to a sequence are shown in Figure 2 (left).

Original: The proportion of cases for which a random sequence of k such successive surgeries produced the correct output at each step is shown in Figure 2 (right).
Revised: The proportion of cases for which a random sequence of k such successive surgeries produced the correct output at each step is shown in Figure 2 (right).

Original: The baseline of 0 substitutions shows the encoder’s accuracy of 98 .
Revised: The baseline of 0 substitutions shows the encoder’s accuracy of 98 .

Original: The accuracy stays above 83% for any number of successive surgeries.
Revised: The accuracy stays above 83% for any number of successive surgeries.

Original:  P ARTIALLY - COMPOSITIONAL NLP TASKS
Revised:  P ARTIALLY - COMPOSITIONAL NLP TASKS

Original: The previous sections explored fully-compositional tasks where there is a strong signal for compositionality.
Revised: The previous sections explored fully-compositional tasks where there is a strong signal for compositionality.

Original: In this section, we explore whether the representations of NNs trained on tasks that are only partially-compositional also capture compositional structure.
Revised: In this section, we explore whether the representations of NNs trained on tasks that are only partially-compositional also capture compositional structure.

Original: Partially-compositional tasks are especially challenging to model because a fully-compositional model may enforce compositionality too strictly to handle the non-compositional aspects of the task, while a model without a compositional bias may not learn any sort of compositionality from the weak cues in the training set.
Revised: Partially-compositional tasks are especially challenging to model because a fully-compositional model may enforce compositionality too strictly to handle the non-compositional aspects of the task, while a model without a compositional bias may not learn any sort of compositionality from the weak cues in the training set.

Original: We test four sentence encoding models for compositionality: InferSent (Conneau et al., 2017), Skipthought (Kiros et al., 2015), Stanford Sentiment Model (SST) (Socher et al., 2013), and SPINN (Bowman et al., 2016).
Revised: We test four sentence encoding models for compositionality: InferSent (Conneau et al., 2017), Skipthought (Kiros et al., 2015), Stanford Sentiment Model (SST) (Socher et al., 2013), and SPINN (Bowman et al., 2016).

Original: For each of these models, we extract the encodings for the SNLI premise of cognitive representations must individually be causally efﬁcacious in  for those constituents  provide an explanation of the compositionality  cognition (Fodor & McLaughlin, 1990; Fodor, 1997).
Revised: For each of these models, we extract the encodings for the SNLI premise of cognitive representations must individually be causally efﬁcacious in order for those constituents to provide an explanation of the compositionality of cognition (Fodor & McLaughlin, 1990; Fodor, 1997).

Original: That TPRs meet 
Revised: That TPRs meet the challenge of explaining compositionality was argued in Smolensky (1987; 1991). We extract syntactic categories from the SCAN grammar (Lake & Baroni, 2018, Supplementary Fig. 6) by saying that two words belong to the same category if every occurrence of one could be grammatically replaced by the other. on our analysis in Appendix A.8, we do not replace occurrences of and and after since the presence of either of these words causes substantial changes in the roles assigned to the sequence.

Original: [Table]
Revised: [Table]

Original: sentences (Bowman et al., 2015).
Revised: sentences (Bowman et al., 2015).

Original: We use the extracted embeddings to train ROLE with 50 roles available (additional training information provided in Appendix A.10).
Revised: We use the extracted embeddings to train ROLE with 50 roles available (additional training information provided in Appendix A.10).

Original: As a baseline
Revised: As a baseline

Original: [Table]
Revised: [Table]

Original: we also train TPEs that use pre-deﬁned role schemes (additional training information in Appendix A.9).
Revised: we also train TPEs that use pre-deﬁned role schemes (additional training information in Appendix A.9).

Original: For all of the sentence embedding models except Skip-thought, ROLE with continuous attention provides the lowest mean squared error at approximating the encoding (Table 3)
Revised: For all of the sentence embedding models except Skip-thought, ROLE with continuous attention provides the lowest mean squared error at approximating the encoding (Table 3)

Original: [Table]
Revised: [Table]

Original: The BOW (bag-of-words) role scheme represents a TPE that does not use compositional structure by assigning the same role to every ﬁller; for each of the sentence embedding models tested except for
Revised: The BOW (bag-of-words) role scheme represents a TPE that does not use compositional structure by assigning the same role to every ﬁller; for each of the sentence embedding models tested except for

Original: SST, performance is within the same order of magnitude as structure-free BOW.
Revised: SST, performance is within the same order of magnitude as structure-free BOW.

Original: Parikh et al.
Revised: Parikh et al.

Original: (2016) found that a bag-of-words model scores extremely well on Natural Language Inference despite having no knowledge of word order, showing that structure is not necessary to perform well on the sorts of tasks commonly used to train sentence encoders.
Revised: (2016) found that a bag-of-words model scores extremely well on Natural Language Inference despite having no knowledge of word order, showing that structure is not necessary to perform well on the sorts of tasks commonly used to train sentence encoders.

Original: Although not deﬁnitive, these results provide no evidence that these sentence embedding models rely on compositional representations.
Revised: Although not deﬁnitive, these results provide no evidence that these sentence embedding models rely on compositional representations.

Original:  F UTURE WORK
Revised:  F UTURE WORK

Original: TPRs provide NNs an aspect of the systematicity of symbolic computation by disentangling ﬁllers and roles.
Revised: TPRs provide NNs an aspect of the systematicity of symbolic computation by disentangling ﬁllers and roles.

Original: The learner needs to learn to process ﬁllers — providing, essentially, what each input constituent contributes to the output — and to process roles — essentially, how these contributions are used in the output.
Revised: The learner needs to learn to process ﬁllers — providing, essentially, what each input constituent contributes to the output — and to process roles — essentially, how these contributions are used in the output.

Original: In this work, we used ROLE to interpret the workings of a target encoder E , and in future work, we plan to train ROLE in an end-to-end manner, either using it as the encoder itself, or using it to regularize a standard (e.g., RNN) encoder with a loss term that rewards learning compositional encodings that ROLE can approximate well.
Revised: In this work, we used ROLE to interpret the workings of a target encoder E , and in future work, we plan to train ROLE in an end-to-end manner, either using it as the encoder itself, or using it to regularize a standard (e.g., RNN) encoder with a loss term that rewards learning compositional encodings that ROLE can approximate well.

Original: We will test whether such an explicit bias for compositionality allows networks to train faster, or with fewer parameters, and to achieve more systematic generalization.
Revised: We will test whether such an explicit bias for compositionality allows networks to train faster, or with fewer parameters, and to achieve more systematic generalization.

Original: Recent work showed improvements in compositionality by separating out syntax and semantics with attention (Russin et al., 2019), and our results suggest that ROLE can also disentangle syntax and semantics.
Revised: Recent work showed improvements in compositionality by separating out syntax and semantics with attention (Russin et al., 2019), and our results suggest that ROLE can also disentangle syntax and semantics.

Original: The structured representations that TPRs provide allow us to structure the processing that occurs over these representations.
Revised: The structured representations that TPRs provide allow us to structure the processing that occurs over these representations.

Original: In fully-compositional tasks, the processing of ﬁllers and roles can be encoded (and hence in principle learned) independently.
Revised: In fully-compositional tasks, the processing of ﬁllers and roles can be encoded (and hence in principle learned) independently.

Original: In partially-compositional tasks, the processing of ﬁllers and roles may be approximately encoded independently, with key interactions when a task deviates from full compositionality.
Revised: In partially-compositional tasks, the processing of ﬁllers and roles may be approximately encoded independently, with key interactions when a task deviates from full compositionality.

Original: In this work, we showed how a hidden embedding can be factored into ﬁllers and roles.
Revised: In this work, we showed how a hidden embedding can be factored into ﬁllers and roles.

Original: Similarly, it is possible to factor a weight matrix into weights that process roles and weights that process ﬁllers.
Revised: Similarly, it is possible to factor a weight matrix into weights that process roles and weights that process ﬁllers.

Original: An illustration of this with a simple example from the SCAN task is provided in Appendix A.11.
Revised: An illustration of this with a simple example from the SCAN task is provided in Appendix A.11.

Original: We plan to explore whether providing networks with a bias favoring weight matrices that factor in this way can improve systematic generalization.
Revised: We plan to explore whether providing networks with a bias favoring weight matrices that factor in this way can improve systematic generalization.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We have introduced ROLE, a neural network that learns to approximate the representations of an existing target neural network E using an explicit symbolic structure.
Revised: We have introduced ROLE, a neural network that learns to approximate the representations of an existing target neural network E using an explicit symbolic structure.

Original: ROLE successfully discovers symbolic structure both in models that explicitly deﬁne this structure and in an RNN without explicit structure trained on the fully-compositional SCAN task.
Revised: ROLE successfully discovers symbolic structure both in models that explicitly deﬁne this structure and in an RNN without explicit structure trained on the fully-compositional SCAN task.

Original: When applied to sentence embedding models trained on partially-compositional tasks, ROLE performs better than hand-speciﬁed role schemes but still provides little evidence that the sentence encodings represent compositional structure.
Revised: When applied to sentence embedding models trained on partially-compositional tasks, ROLE performs better than hand-speciﬁed role schemes but still provides little evidence that the sentence encodings represent compositional structure.

Original: Uncovering the latent symbolic structure of NN representations on fully-compositional tasks is a signiﬁcant step towards explaining how they can achieve the level of compositional generalization that they do, and suggests types of inductive bias to improve such generalization for partially-compositional tasks.
Revised: Uncovering the latent symbolic structure of NN representations on fully-compositional tasks is a signiﬁcant step towards explaining how they can achieve the level of compositional generalization that they do, and suggests types of inductive bias to improve such generalization for partially-compositional tasks.

Original: A. D ESIGNED ROLE SCHEMES
Revised: A. D ESIGNED ROLE SCHEMES

Original: We use six hand-speciﬁed role schemes as a baseline to compare the learned role schemes against.
Revised: We use six hand-speciﬁed role schemes as a baseline to compare the learned role schemes against.

Original: Examples of each role scheme are shown in Table 4.
Revised: Examples of each role scheme are shown in Table 4.

Original: Left-to-right (LTR): Each ﬁller’s role is its index in the sequence, counting from left to right.
Revised: Left-to-right (LTR): Each ﬁller’s role is its index in the sequence, counting from left to right.

Original: Right-to-left (RTL): Each ﬁller’s role is its index in the sequence, counting from right to left.
Revised: Right-to-left (RTL): Each ﬁller’s role is its index in the sequence, counting from right to left.

Original: Bidirectional (Bi): Each ﬁller’s role is a pair of indices, where the ﬁrst index counts from left to right, and the second index counts from right to left.
Revised: Bidirectional (Bi): Each ﬁller’s role is a pair of indices, where the ﬁrst index counts from left to right, and the second index counts from right to left.

Original: Tree: Each ﬁller’s role is given by its position in a tree.
Revised: Tree: Each ﬁller’s role is given by its position in a tree.

Original: This depends on a tree parsing algorithm.
Revised: This depends on a tree parsing algorithm.

Original: Wickelroles (Wickel): Each ﬁller’s role is the ﬁller before it and the ﬁller after it.
Revised: Wickelroles (Wickel): Each ﬁller’s role is the ﬁller before it and the ﬁller after it.

Original: (Wickelgren, 1969) 6.
Revised: (Wickelgren, 1969) 6.

Original: Bag-of-words (BOW): Each ﬁller is assigned the same role.
Revised: Bag-of-words (BOW): Each ﬁller is assigned the same role.

Original: The position and context of the ﬁller is ignored.
Revised: The position and context of the ﬁller is ignored.

Original: [Table]
Revised: [Table]

Original: to-left
Revised: to-left

Original: Bidirectional (0, 3) (1, 2) (2, 1) (3, 0) (0, 5) (1, 4) (2, 3) (3, 2) (4, 1) (5, 0)
Revised: Bidirectional (0, 3) (1, 2) (2, 1) (3, 0) (0, 5) (1, 4) (2, 3) (3, 2) (4, 1) (5, 0)

Original: Wickelroles #_ 3_ 1_ 1_# #_ 5_ 2_ 3_ 1_ 9_#
Revised: Wickelroles #_ 3_ 1_ 1_# #_ 5_ 2_ 3_ 1_ 9_#

Original: Tree
Revised: Tree

Original: L
Revised: L

Original: RLL RLR
Revised: RLL RLR

Original: RR
Revised: RR

Original: LL
Revised: LL

Original: LRLL LRLR
Revised: LRLL LRLR

Original: LRRL LRRR
Revised: LRRL LRRR

Original: R
Revised: R

Original: Bag
Revised: Bag

Original: [Table]
Revised: [Table]

Original: words rrrrrrrrrr
Revised: words rrrrrrrrrr

Original: A. TPE S TRAINED ON DIGIT SEQUENCE TASKS
Revised: A. TPE S TRAINED ON DIGIT SEQUENCE TASKS

Original: We trained two TPEs end-to-end with an RNN decoder for our target networks on the digit sequence tasks.
Revised: We trained two TPEs end-to-end with an RNN decoder for our target networks on the digit sequence tasks.

Original: The left-to-right (LTR) TPE used a left-to-right role scheme applied to each element in the input and was connected to a unidirectional GRU decoder.
Revised: The left-to-right (LTR) TPE used a left-to-right role scheme applied to each element in the input and was connected to a unidirectional GRU decoder.

Original: The tree TPE used tree positions for each element in the input and was connected to a tree GRU decoder; here, the digit strings were parsed as a binary tree by a deterministic algorithm given in McCoy et al.
Revised: The tree TPE used tree positions for each element in the input and was connected to a tree GRU decoder; here, the digit strings were parsed as a binary tree by a deterministic algorithm given in McCoy et al.

Original: (2019a, App. C).
Revised: (2019a, App. C).

Original: The ﬁller and role dimensions were both 20 for the LTR TPE.
Revised: The ﬁller and role dimensions were both 20 for the LTR TPE.

Original: The ﬁller dimension was 20 for the Tree TPE, and the role dimension was 120.
Revised: The ﬁller dimension was 20 for the Tree TPE, and the role dimension was 120.

Original: We used a hidden size of 60 for the GRU decoders.
Revised: We used a hidden size of 60 for the GRU decoders.

Original: We used a patience offor early stopping.
Revised: We used a patience offor early stopping.

Original: The left-to-right TPE achieves 100% accuracy on the test set and the tree TPE achieves 98.62% on the test set.
Revised: The left-to-right TPE achieves 100% accuracy on the test set and the tree TPE achieves 98.62% on the test set.

Original: A. ROLE REGULARIZATION
Revised: A. ROLE REGULARIZATION

Original: Letting A = { a t } Tt =1 , the regularization term applied during ROLE training is R =
Revised: Letting A = { a t } Tt =1 , the regularization term applied during ROLE training is R =

Original: λ
Revised: λ

Original: ( R 1 + R 2 + R 3 ) , where λ is a regularization hyperparameter and:
Revised: ( R 1 + R 2 + R 3 ) , where λ is a regularization hyperparameter and:

Original: [Equation]
Revised: [Equation]

Original: Since each a t results from a softmax, its elements are positive and sum to 1.
Revised: Since each a t results from a softmax, its elements are positive and sum to 1.

Original: Thus the factors in
Revised: Thus the factors in

Original: R 1 ( A ) are all non-negative, so R 1 assumes its minimal value of 0 when each a t has binary elements; since these elements must sum to 1, such an a t must be one-hot.
Revised: R 1 ( A ) are all non-negative, so R 1 assumes its minimal value of 0 when each a t has binary elements; since these elements must sum to 1, such an a t must be one-hot.

Original: R 2 ( A ) is also minimized when each a t is one-hot because when a vector’s L 1 norm is 1, its L 2 norm is maximized when it is one-hot.
Revised: R 2 ( A ) is also minimized when each a t is one-hot because when a vector’s L 1 norm is 1, its L 2 norm is maximized when it is one-hot.

Original: Although each of these terms individually favor one-hot vectors, empirically we ﬁnd that using both terms helps the training process.
Revised: Although each of these terms individually favor one-hot vectors, empirically we ﬁnd that using both terms helps the training process.

Original: In a discrete symbolic structure, each position can hold at most one symbol, and the ﬁnal term R 3 in ROLE’s regularizer R is designed to encourage this.
Revised: In a discrete symbolic structure, each position can hold at most one symbol, and the ﬁnal term R 3 in ROLE’s regularizer R is designed to encourage this.

Original: In the vector s A = (cid:80) Tt =1 a t , the ρ th element is the total attention weight, over all symbols in the string, assigned to the ρ th role: in the discrete case, this must be 0 (if no symbol is assigned this role) or(if a single symbol is assigned this role).
Revised: In the vector s A = (cid:80) Tt =1 a t , the ρ th element is the total attention weight, over all symbols in the string, assigned to the ρ th role: in the discrete case, this must be 0 (if no symbol is assigned this role) or(if a single symbol is assigned this role).

Original: Thus R 3 is minimized when all elements of s are 0 or 1 ( Ris similar to R 1 , but with squared terms since we are no longer assured each element is at most 1).
Revised: Thus R 3 is minimized when all elements of s are 0 or 1 ( Ris similar to R 1 , but with squared terms since we are no longer assured each element is at most 1).

Original: It is important to normalize each role embedding in the role matrix R so that small attention weights have correspondingly small impacts on the weighted-sum role embedding.
Revised: It is important to normalize each role embedding in the role matrix R so that small attention weights have correspondingly small impacts on the weighted-sum role embedding.

Original: A. ROLE TRAINED ON DIGIT SEQUENCE TASKS
Revised: A. ROLE TRAINED ON DIGIT SEQUENCE TASKS

Original: Once the TPEs in Sec.
Revised: Once the TPEs in Sec.

Original: A.3 were trained, we extracted the hidden embedding for each item in the training, dev, and test sets.
Revised: A.3 were trained, we extracted the hidden embedding for each item in the training, dev, and test sets.

Original: For both ROLE models trained on the digit sequence task, we used a bidirectional 2-layer LSTM (Hochreiter & Schmidhuber, 1997) with ﬁller dimension of 20, and regularization constant λ = 1 .
Revised: For both ROLE models trained on the digit sequence task, we used a bidirectional 2-layer LSTM (Hochreiter & Schmidhuber, 1997) with ﬁller dimension of 20, and regularization constant λ = 1 .

Original: For training, we used the ADAM (Kingma & Ba, 2015) optimizer with a learning rate of .001, batch size 32, and an early stopping patience of 10.
Revised: For training, we used the ADAM (Kingma & Ba, 2015) optimizer with a learning rate of .001, batch size 32, and an early stopping patience of 10.

Original: The ROLE model trained on the LTR TPE was given 20 roles each of dimension 20.
Revised: The ROLE model trained on the LTR TPE was given 20 roles each of dimension 20.

Original: The ROLE model trained on the Tree TPE was given 120 roles each of dimension 120.
Revised: The ROLE model trained on the Tree TPE was given 120 roles each of dimension 120.

Original: A. RNN TRAINED ON SCAN
Revised: A. RNN TRAINED ON SCAN

Original: To train the standard RNN on SCAN, we ran a limited hyperparameter search similar to the procedure in Lake & Baroni (2018).
Revised: To train the standard RNN on SCAN, we ran a limited hyperparameter search similar to the procedure in Lake & Baroni (2018).

Original: Since our goal was to produce a single embedding that captured the entire input sequence, we ﬁxed the architecture to GRU with a single hidden layer.
Revised: Since our goal was to produce a single embedding that captured the entire input sequence, we ﬁxed the architecture to GRU with a single hidden layer.

Original: We did not train models with attention, since we wanted to investigate whether a standard RNN could capture compositionality.
Revised: We did not train models with attention, since we wanted to investigate whether a standard RNN could capture compositionality.

Original: The remaining hyperparameters were hidden dimension and dropout.
Revised: The remaining hyperparameters were hidden dimension and dropout.

Original: We ran a search over the hidden dimension sizes of 50, 100, 200, and 400 as well as dropout with a value of 0, .1, and .5 applied to the word embeddings and recurrent layer.
Revised: We ran a search over the hidden dimension sizes of 50, 100, 200, and 400 as well as dropout with a value of 0, .1, and .5 applied to the word embeddings and recurrent layer.

Original: Each network was trained with the ADAM optimizer (Kingma & Ba, 2015) and a learning rate of .001 for 100,000 steps with a batch-size of 1.
Revised: Each network was trained with the ADAM optimizer (Kingma & Ba, 2015) and a learning rate of .001 for 100,000 steps with a batch-size of 1.

Original: The best performing network had a hidden dimension or 100 and dropout of .1.
Revised: The best performing network had a hidden dimension or 100 and dropout of .1.

Original: A. ROLE TRAINED ON SCAN
Revised: A. ROLE TRAINED ON SCAN

Original: For the ROLE models trained to approximate the GRU encoder trained on SCAN, we used a ﬁller dimension of 100, a role dimension of 50 with 50 roles available.
Revised: For the ROLE models trained to approximate the GRU encoder trained on SCAN, we used a ﬁller dimension of 100, a role dimension of 50 with 50 roles available.

Original: For training, we used the ADAM (Kingma & Ba, 2015) optimizer with a learning rate of .001, batch size 32, and an early stopping patience of 10.
Revised: For training, we used the ADAM (Kingma & Ba, 2015) optimizer with a learning rate of .001, batch size 32, and an early stopping patience of 10.

Original: The role assignment module used a bidirectional 2-layer LSTM (Hochreiter &
Revised: The role assignment module used a bidirectional 2-layer LSTM (Hochreiter &

Original: Schmidhuber, 1997).
Revised: Schmidhuber, 1997).

Original: We performed a hyperparameter search over the regularization coefﬁcient λ using the values in the set [.1, .02, .01].
Revised: We performed a hyperparameter search over the regularization coefﬁcient λ using the values in the set [.1, .02, .01].

Original: The best performing value was .02, and we used this model in our analysis.
Revised: The best performing value was .02, and we used this model in our analysis.

Original: A. SCAN R OLE A NALYSIS
Revised: A. SCAN R OLE A NALYSIS

Original: The algorithm below characterizes our post-hoc interpretation of which roles the Role Learner will assign to elements of the input to the SCAN model.
Revised: The algorithm below characterizes our post-hoc interpretation of which roles the Role Learner will assign to elements of the input to the SCAN model.

Original: This algorithm was created by hand based on an analysis of the Role Learner’s outputs for the elements of the SCAN training set.
Revised: This algorithm was created by hand based on an analysis of the Role Learner’s outputs for the elements of the SCAN training set.

Original: The algorithm works equally well on examples in the training set and the test set; on both datasets, it exactly matches the roles chosen by the Role Learner for 98.7% of sequences (20,642 out of 20,910).
Revised: The algorithm works equally well on examples in the training set and the test set; on both datasets, it exactly matches the roles chosen by the Role Learner for 98.7% of sequences (20,642 out of 20,910).

Original: A.8. A ROLE - ASSIGNMENT ALGORITHM IMPLICITLY LEARNED BY THE SCAN SEQ 2 SEQ ENCODER
Revised: A.8. A ROLE - ASSIGNMENT ALGORITHM IMPLICITLY LEARNED BY THE SCAN SEQ 2 SEQ ENCODER

Original: The input sequences have three basic types that are relevant to determining the role assignment: sequences that contain and (e.g., jump around left and walk thrice ), sequences that contain after (e.g., jump around left after walk thrice ), and sequences without and or after (e.g., turn opposite right thrice ).
Revised: The input sequences have three basic types that are relevant to determining the role assignment: sequences that contain and (e.g., jump around left and walk thrice ), sequences that contain after (e.g., jump around left after walk thrice ), and sequences without and or after (e.g., turn opposite right thrice ).

Original: Within commands containing and or after , it is convenient to break the command down into the command before the connecting word and the command after it; for example, in the command jump around left after walk thrice , these two components would be jump around left and walk thrice .
Revised: Within commands containing and or after , it is convenient to break the command down into the command before the connecting word and the command after it; for example, in the command jump around left after walk thrice , these two components would be jump around left and walk thrice .

Original: • Sequence with and : – Elements of the command before and : ∗ Last word:∗ First word (if not also last word):∗ opposite if the command ends with thrice :∗ Direction word between opposite and thrice :∗ opposite if the command does not end with thrice :∗ Direction word after opposite but not before thrice :∗ around :∗ Direction word after around :∗ Direction word between an action word and twice or thrice :– Elements of the command before and : ∗ First word:∗
Revised: • Sequence with and : – Elements of the command before and : ∗ Last word:∗ First word (if not also last word):∗ opposite if the command ends with thrice :∗ Direction word between opposite and thrice :∗ opposite if the command does not end with thrice :∗ Direction word after opposite but not before thrice :∗ around :∗ Direction word after around :∗ Direction word between an action word and twice or thrice :– Elements of the command before and : ∗ First word:∗

Original: Last word (if not also the ﬁrst word):∗ Second-to-last word (if not also the ﬁrst word):∗ Second of four words:– and :• Sequence with after : – Elements of the command before after : ∗ Last word:∗
Revised: Last word (if not also the ﬁrst word):∗ Second-to-last word (if not also the ﬁrst word):∗ Second of four words:– and :• Sequence with after : – Elements of the command before after : ∗ Last word:∗

Original: Second-to-last word:∗ First word (if not the last or second-to-last word):∗ Second word (if not the last or second-to-last word):– Elements of the command after after : ∗ Last word:
Revised: Second-to-last word:∗ First word (if not the last or second-to-last word):∗ Second word (if not the last or second-to-last word):– Elements of the command after after : ∗ Last word:

Original: ∗ Second-to-last word
Revised: ∗ Second-to-last word

Original:  ∗ First
Revised:  ∗ First

Original: if the command
Revised: if the command

Original: right
Revised: right

Original:  ∗  ∗
Revised:  ∗  ∗

Original: the command does not end
Revised: the command does not end

Original: right
Revised: right

Original: does not contain both thrice and a rotation:∗ Second word if the command ends with thrice :∗ Second word if the command does not end with thrice :– after : 17 if no other word has role 17 or if the command after after ends with around left ; 43 otherwise • Sequence without and or after : – Action word directly before a cardinality:– Action word before, but not directly before, a cardinality:– thrice directly after an action word:– twice directly after an action word:– opposite in a sequence ending with twice :– opposite in a sequence ending with thrice :– around in a sequence ending with a cardinality:– Direction word directly before a cardinality:– Action word in a sequence without a cardinality:– opposite in a sequence without a cardinality:– Direction after opposite in a sequence without a cardinality:– around in a sequence without a cardinality:– Direction after around in a sequence without a cardinality:– Direction directly after an action in a sequence without a cardinality:
Revised: does not contain both thrice and a rotation:∗ Second word if the command ends with thrice :∗ Second word if the command does not end with thrice :– after : 17 if no other word has role 17 or if the command after after ends with around left ; 43 otherwise • Sequence without and or after : – Action word directly before a cardinality:– Action word before, but not directly before, a cardinality:– thrice directly after an action word:– twice directly after an action word:– opposite in a sequence ending with twice :– opposite in a sequence ending with thrice :– around in a sequence ending with a cardinality:– Direction word directly before a cardinality:– Action word in a sequence without a cardinality:– opposite in a sequence without a cardinality:– Direction after opposite in a sequence without a cardinality:– around in a sequence without a cardinality:– Direction after around in a sequence without a cardinality:– Direction directly after an action in a sequence without a cardinality:

Original: To show how this works with an example, consider the input jump around left after walk thrice .
Revised: To show how this works with an example, consider the input jump around left after walk thrice .

Original: The command before after is jump around left .
Revised: The command before after is jump around left .

Original: left , as the last word, is given role 8.
Revised: left , as the last word, is given role 8.

Original: around , as the second-to-last word, gets role 36.
Revised: around , as the second-to-last word, gets role 36.

Original: jump , as a ﬁrst word that is not also the last or second-to-last word gets role 11.
Revised: jump , as a ﬁrst word that is not also the last or second-to-last word gets role 11.

Original: The command after after is walk thrice .
Revised: The command after after is walk thrice .

Original: thrice , as the last word, gets role 46.
Revised: thrice , as the last word, gets role 46.

Original: walk , as the second-to-last word, gets role 4.
Revised: walk , as the second-to-last word, gets role 4.

Original: Finally, after gets role 17 because no other elements have been assigned role 17 yet.
Revised: Finally, after gets role 17 because no other elements have been assigned role 17 yet.

Original: These predicted outputs match those given by the Role Learner.
Revised: These predicted outputs match those given by the Role Learner.

Original: A.8. D ISCUSSION OF THE ALGORITHM
Revised: A.8. D ISCUSSION OF THE ALGORITHM

Original: We offer several observations about this algorithm.
Revised: We offer several observations about this algorithm.

Original: This algorithm may seem convoluted, but a few observations can illuminate how the
Revised: This algorithm may seem convoluted, but a few observations can illuminate how the

Original: roles assigned by such an algorithm support success on the SCAN task.
Revised: roles assigned by such an algorithm support success on the SCAN task.

Original: First, a sequence will contain role 30 if and only if it contains and , and it will contain role 17 if
Revised: First, a sequence will contain role 30 if and only if it contains and , and it will contain role 17 if

Original: and only if it contains after .
Revised: and only if it contains after .

Original: Thus, by implicitly checking for the presence of these two roles (regardless of the ﬁllers bound to them), the decoder can tell whether the output involves one or two basic commands, where the presence of and or after leads to two basic commands and the absence of both leads to one basic command.
Revised: Thus, by implicitly checking for the presence of these two roles (regardless of the ﬁllers bound to them), the decoder can tell whether the output involves one or two basic commands, where the presence of and or after leads to two basic commands and the absence of both leads to one basic command.

Original: Moreover, if there are two basic commands, whether it is role 17 or role 30 that is present can tell the decoder whether the input order of these commands also corresponds to their output order (when it is and in play, i.e., role 30), or if the input order is reversed (when it is after in play, i.e., role 17).
Revised: Moreover, if there are two basic commands, whether it is role 17 or role 30 that is present can tell the decoder whether the input order of these commands also corresponds to their output order (when it is and in play, i.e., role 30), or if the input order is reversed (when it is after in play, i.e., role 17).

Original: With these basic structural facts established, the decoder can begin to decode the speciﬁc commands.
Revised: With these basic structural facts established, the decoder can begin to decode the speciﬁc commands.

Original: For example, if the input is a sequence with after , it can begin with the command after after , which it can decode by checking which ﬁllers are bound to the relevant roles for that type of command.
Revised: For example, if the input is a sequence with after , it can begin with the command after after , which it can decode by checking which ﬁllers are bound to the relevant roles for that type of command.

Original: It may seem odd that so many of the roles are based on position (e.g., “ﬁrst word" and “second-to-last word"), rather than more functionally-relevant categories such as “direction word."
Revised: It may seem odd that so many of the roles are based on position (e.g., “ﬁrst word" and “second-to-last word"), rather than more functionally-relevant categories such as “direction word."

Original: However, this approach may actually be more efﬁcient: Each command consists of a single mandatory element (namely, an action word such as walk or jump ) followed by several optional modiﬁers (namely, rotation words, direction words, and cardinalities).
Revised: However, this approach may actually be more efﬁcient: Each command consists of a single mandatory element (namely, an action word such as walk or jump ) followed by several optional modiﬁers (namely, rotation words, direction words, and cardinalities).

Original: Because most of the word categories are optional, it might be inefﬁcient to check for the presence of, e.g., a cardinality, since many sequences will not have one.
Revised: Because most of the word categories are optional, it might be inefﬁcient to check for the presence of, e.g., a cardinality, since many sequences will not have one.

Original: By contrast, every sequence will have a last word, and checking the identity of the last word provides much functionally-relevant information: if that word is not a cardinality, then the decoder knows that there is no cardinality present in the command (because if there were, it would be the last word); and if it is a cardinality, then that is important to know, because the presence of twice or thrice can dramatically affect the shape of the output sequence.
Revised: By contrast, every sequence will have a last word, and checking the identity of the last word provides much functionally-relevant information: if that word is not a cardinality, then the decoder knows that there is no cardinality present in the command (because if there were, it would be the last word); and if it is a cardinality, then that is important to know, because the presence of twice or thrice can dramatically affect the shape of the output sequence.

Original: In this light, it is unsurprising that the SCAN encoder has implicitly learned several different roles that essentially mean the last element of a particular subcommand.
Revised: In this light, it is unsurprising that the SCAN encoder has implicitly learned several different roles that essentially mean the last element of a particular subcommand.

Original: The algorithm does not constitute a simple, transparent role scheme.
Revised: The algorithm does not constitute a simple, transparent role scheme.

Original: But its job is to describe the representations that the original network produces, and we have no a priori expectation about how complex that process may be.
Revised: But its job is to describe the representations that the original network produces, and we have no a priori expectation about how complex that process may be.

Original: The role-assignment algorithm implicitly learned by ROLE is interpretable locally (each line is readily expressible in simple English), but not intuitively transparent globally.
Revised: The role-assignment algorithm implicitly learned by ROLE is interpretable locally (each line is readily expressible in simple English), but not intuitively transparent globally.

Original: We see this as a positive result, in two respects.
Revised: We see this as a positive result, in two respects.

Original: First, it shows why ROLE is crucial: no human-generated role scheme would provide a good approximation to this algorithm.
Revised: First, it shows why ROLE is crucial: no human-generated role scheme would provide a good approximation to this algorithm.

Original: Such an algorithm can only be identiﬁed because
Revised: Such an algorithm can only be identiﬁed because

Original: ROLE is able to use gradient descent to ﬁnd role schemes far more complex than any we would hypothesize intuitively.
Revised: ROLE is able to use gradient descent to ﬁnd role schemes far more complex than any we would hypothesize intuitively.

Original: This enables us to analyze networks far more complex than we could analyze previously, being necessarily limited to hand-designed role schemes based on human intuitions about how to perform the task.
Revised: This enables us to analyze networks far more complex than we could analyze previously, being necessarily limited to hand-designed role schemes based on human intuitions about how to perform the task.

Original: Second, when future work illuminates the computation in the original SCAN GRU seq2seq decoder, the baroqueness of the role-assignment algorithm that ROLE has shown to be implicit in the seq2seq encoder can potentially explain certain limitations in the original model, which is known to suffer from severe failures of systematic generalization outside the training distribution (Lake and Baroni, 2018).
Revised: Second, when future work illuminates the computation in the original SCAN GRU seq2seq decoder, the baroqueness of the role-assignment algorithm that ROLE has shown to be implicit in the seq2seq encoder can potentially explain certain limitations in the original model, which is known to suffer from severe failures of systematic generalization outside the training distribution (Lake and Baroni, 2018).

Original: It is reasonable to hypothesize that systematic generalization requires that the encoder learn an implicit role scheme that is relatively simple and highly compositional.
Revised: It is reasonable to hypothesize that systematic generalization requires that the encoder learn an implicit role scheme that is relatively simple and highly compositional.

Original: Future proposals for improving the systematic generalization of models on SCAN can be examined using ROLE to test the hypothesis that greater systematicity requires greater compositional simplicity in the role scheme implicitly learned by the encoder.
Revised: Future proposals for improving the systematic generalization of models on SCAN can be examined using ROLE to test the hypothesis that greater systematicity requires greater compositional simplicity in the role scheme implicitly learned by the encoder.

Original: While the role assignment algorithm of A.8.1 may not be simple, from a certain perspective, it is quite surprising that it is not far more complex.
Revised: While the role assignment algorithm of A.8.1 may not be simple, from a certain perspective, it is quite surprising that it is not far more complex.

Original: Although ROLE is provided 50 roles to learn to deploy as it likes, it only chooses to use 16 of them (only 16 are ever selected as the arg max( a t ) ; see Sec. 6.1).
Revised: Although ROLE is provided 50 roles to learn to deploy as it likes, it only chooses to use 16 of them (only 16 are ever selected as the arg max( a t ) ; see Sec. 6.1).

Original: Furthermore, the SCAN grammar generates 20,910 input sequences, containing a total of 151,688 words (an average of 7.25 words per input).
Revised: Furthermore, the SCAN grammar generates 20,910 input sequences, containing a total of 151,688 words (an average of 7.25 words per input).

Original: This means that, if one were to generate a series of conditional statements to determine which role is assigned to each word in every context, this could in theory require up to 151, conditionals (e.g., “if the ﬁller is ‘jump’ in the context ‘walk thrice after opposite left’, then assign role 17”).
Revised: This means that, if one were to generate a series of conditional statements to determine which role is assigned to each word in every context, this could in theory require up to 151, conditionals (e.g., “if the ﬁller is ‘jump’ in the context ‘walk thrice after opposite left’, then assign role 17”).

Original: However, our algorithm involves just 47 conditionals.
Revised: However, our algorithm involves just 47 conditionals.

Original: This reduction helps explain how the model performs so well on the test set: If it used many more of the 151,688 possible conditional rules, it would completely overﬁt the training examples in a way that would be unlikely to generalize.
Revised: This reduction helps explain how the model performs so well on the test set: If it used many more of the 151,688 possible conditional rules, it would completely overﬁt the training examples in a way that would be unlikely to generalize.

Original: The 47-conditional algorithm we found is more likely to generalize by abstracting over many details of the context.
Revised: The 47-conditional algorithm we found is more likely to generalize by abstracting over many details of the context.

Original: Were
Revised: Were

Original: it not for ROLE’s ability to characterize the representations
Revised: it not for ROLE’s ability to characterize the representations

Original: generated by the original encoder in terms of implicit roles, providing an equally complete and accurate interpretation of those representations would necessarily require identifying the conditions determining the activation level of each of the 100 neurons hosting those representations.
Revised: generated by the original encoder in terms of implicit roles, providing an equally complete and accurate interpretation of those representations would necessarily require identifying the conditions determining the activation level of each of the 100 neurons hosting those representations.

Original: It seems to us grossly overly optimistic to estimate that each neuron’s activation level in the representation of a given input could be characterized by a property of the input statable in, say, two lines of roughly 20 words/symbols; yet even then, the algorithm would require 200 lines, whereas the algorithm in A.8.1 requires 47 lines of that scale.
Revised: It seems to us grossly overly optimistic to estimate that each neuron’s activation level in the representation of a given input could be characterized by a property of the input statable in, say, two lines of roughly 20 words/symbols; yet even then, the algorithm would require 200 lines, whereas the algorithm in A.8.1 requires 47 lines of that scale.

Original: Thus, by even such a crude estimate of the degree of complexity expected for an algorithm describing the representations in terms of neuron activities, the algorithm we ﬁnd, stated over roles, is 4 times simpler.
Revised: Thus, by even such a crude estimate of the degree of complexity expected for an algorithm describing the representations in terms of neuron activities, the algorithm we ﬁnd, stated over roles, is 4 times simpler.

Original: A. TPE S TRAINED ON SENTENCE EMBEDDING MODELS
Revised: A. TPE S TRAINED ON SENTENCE EMBEDDING MODELS

Original: For each sentence embedding model, we trained three randomly initialized TPEs for each role scheme and selected the best performing one as measured by the lowest MSE.
Revised: For each sentence embedding model, we trained three randomly initialized TPEs for each role scheme and selected the best performing one as measured by the lowest MSE.

Original: For each TPE, we used the original ﬁller embedding from the sentence embedding model.
Revised: For each TPE, we used the original ﬁller embedding from the sentence embedding model.

Original: This ﬁller dimensionality is 25 for
Revised: This ﬁller dimensionality is 25 for

Original: SST, 300 for SPINN and InferSent, and 620 for Skipthought.
Revised: SST, 300 for SPINN and InferSent, and 620 for Skipthought.

Original: We applied a linear transformation to the pre-trained ﬁller embedding where the input size is the dimensionality of the pre-trained embedding and the output size is also the dimensionality of the pre-trained embedding.
Revised: We applied a linear transformation to the pre-trained ﬁller embedding where the input size is the dimensionality of the pre-trained embedding and the output size is also the dimensionality of the pre-trained embedding.

Original: This linearly transformed embedding is used as the ﬁller vector in the ﬁller-role binding in the TPE.
Revised: This linearly transformed embedding is used as the ﬁller vector in the ﬁller-role binding in the TPE.

Original: For each TPE, we use a role dimension of 50.
Revised: For each TPE, we use a role dimension of 50.

Original: Training was done with a batch size of 32 using the ADAM optimizer with a learning rate of .001.
Revised: Training was done with a batch size of 32 using the ADAM optimizer with a learning rate of .001.

Original: To generate tree roles from the English sentences, we used the constituency parser released in version 3.9.1 of Stanford CoreNLP (Klein & Manning, 2003).
Revised: To generate tree roles from the English sentences, we used the constituency parser released in version 3.9.1 of Stanford CoreNLP (Klein & Manning, 2003).

Original: A. ROLE TRAINED ON SENTENCE EMBEDDING MODELS
Revised: A. ROLE TRAINED ON SENTENCE EMBEDDING MODELS

Original: For each sentence embedding model, we trained three randomly initialized ROLE models and selected the best performing one as measured by the lowest MSE.
Revised: For each sentence embedding model, we trained three randomly initialized ROLE models and selected the best performing one as measured by the lowest MSE.

Original: We used the original ﬁller embedding from the sentence embedding model (25 for SST, 300 for SPINN and InferSent, and 620 for Skipthought).
Revised: We used the original ﬁller embedding from the sentence embedding model (25 for SST, 300 for SPINN and InferSent, and 620 for Skipthought).

Original: We applied a linear transformation to the pre-trained ﬁller embedding where the input size is the dimensionality of the pre-trained embedding and the output size is also the dimensionality of the pre-trained embedding.
Revised: We applied a linear transformation to the pre-trained ﬁller embedding where the input size is the dimensionality of the pre-trained embedding and the output size is also the dimensionality of the pre-trained embedding.

Original: This linearly transformed embedding is used as the ﬁller vector in the ﬁller-role binding in the TPE.
Revised: This linearly transformed embedding is used as the ﬁller vector in the ﬁller-role binding in the TPE.

Original: We also applied a similar linear transformation to the pre-trained ﬁller embedding before input to the role learner LSTM.
Revised: We also applied a similar linear transformation to the pre-trained ﬁller embedding before input to the role learner LSTM.

Original: For each ROLE model, we provide up to 50 roles with a role dimension of 50.
Revised: For each ROLE model, we provide up to 50 roles with a role dimension of 50.

Original: Training was done with a batch size of 32 using the ADAM optimizer with a learning rate of .001.
Revised: Training was done with a batch size of 32 using the ADAM optimizer with a learning rate of .001.

Original: We performed a hyperparameter search over the regularization coefﬁcient λ using the values in the set { 1 , 0 . 1 , 0 . 01 , 0 .
Revised: We performed a hyperparameter search over the regularization coefﬁcient λ using the values in the set { 1 , 0 . 1 , 0 . 01 , 0 .

Original: For SST, SPINN, InferSent and
Revised: For SST, SPINN, InferSent and

Original: SST, respectively, the best performing network used λ = 0 .
Revised: SST, respectively, the best performing network used λ = 0 .

Original: A. F ACTORING WEIGHTS TO PROCESS FILLERS AND ROLES INDEPENDENTLY IN SCAN
Revised: A. F ACTORING WEIGHTS TO PROCESS FILLERS AND ROLES INDEPENDENTLY IN SCAN

Original: At the end of Sec. 8 we remarked: “the structured representation bias that TPRs provide allows us to also provide biases to structure the processing that occurs over these representations.
Revised: At the end of Sec. 8 we remarked: “the structured representation bias that TPRs provide allows us to also provide biases to structure the processing that occurs over these representations.

Original: In fullycompositional tasks, the processing of ﬁllers and roles can be encoded (and hence in principle learned) independently. . . .
Revised: In fullycompositional tasks, the processing of ﬁllers and roles can be encoded (and hence in principle learned) independently. . . .

Original: In this work, we showed how a hidden embedding can be factored into ﬁllers and roles.
Revised: In this work, we showed how a hidden embedding can be factored into ﬁllers and roles.

Original: Similarly, it is possible to factor a weight matrix into weights that process roles and weights that process ﬁllers.”
Revised: Similarly, it is possible to factor a weight matrix into weights that process roles and weights that process ﬁllers.”

Original: Here we exemplify this with SCAN.
Revised: Here we exemplify this with SCAN.

Original: One of the compositional rules for the mapping ϕ deﬁned by the SCAN task is: ϕ ( x twice ) =
Revised: One of the compositional rules for the mapping ϕ deﬁned by the SCAN task is: ϕ ( x twice ) =

Original: ϕ ( x ) ϕ
Revised: ϕ ( x ) ϕ

Original: ( x ) ; for example, ϕ ( walk twice ) =
Revised: ( x ) ; for example, ϕ ( walk twice ) =

Original: ϕ ( walk ) ϕ
Revised: ϕ ( walk ) ϕ

Original: ( walk ) =
Revised: ( walk ) =

Original: WALK WALK .
Revised: WALK WALK .

Original: For the purposes of illustration, suppose that, given the input string x twice , a NN encoder produces a representation that is approximated by the TPR of a single-constituent structure in which the ﬁller x is assigned the role “argument of twice ”: x : r 2 ce -arg .
Revised: For the purposes of illustration, suppose that, given the input string x twice , a NN encoder produces a representation that is approximated by the TPR of a single-constituent structure in which the ﬁller x is assigned the role “argument of twice ”: x : r 2 ce -arg .

Original: So the TPR encoding of jump twice is e ( jump twice ) =
Revised: So the TPR encoding of jump twice is e ( jump twice ) =

Original: e ( jump : r 2 ce -arg ) =
Revised: e ( jump : r 2 ce -arg ) =

Original: e F ( jump ) ⊗ e R ( r 2 ce -arg ) , where e F , e R are the embedding functions for ﬁllers and roles, respectively.
Revised: e F ( jump ) ⊗ e R ( r 2 ce -arg ) , where e F , e R are the embedding functions for ﬁllers and roles, respectively.

Original: Let us also suppose that the output string is encoded as a
Revised: Let us also suppose that the output string is encoded as a

Original: TPR with positional roles R i , so that WALK LOOK has ﬁller:role bindings { WALK : R 1 , LOOK : R 2 } and so has TPR e ( { WALK : R 1 , LOOK : R 2 } ) =
Revised: TPR with positional roles R i , so that WALK LOOK has ﬁller:role bindings { WALK : R 1 , LOOK : R 2 } and so has TPR e ( { WALK : R 1 , LOOK : R 2 } ) =

Original: e F ( WALK ) ⊗ e R ( R 1 )
Revised: e F ( WALK ) ⊗ e R ( R 1 )

Original: +
Revised: +

Original: e F ( LOOK ) ⊗ e R ( R 2 ) .
Revised: e F ( LOOK ) ⊗ e R ( R 2 ) .

Original: So to generalize correctly to the input jump twice , producing output JUMP JUMP , the system needs to learn two things: how to map the ﬁller — ϕ F ( jump ) =
Revised: So to generalize correctly to the input jump twice , producing output JUMP JUMP , the system needs to learn two things: how to map the ﬁller — ϕ F ( jump ) =

Original: JUMP — and how to map the role — ϕ R ( r 2 ce -arg )
Revised: JUMP — and how to map the role — ϕ R ( r 2 ce -arg )

Original: = { R 1 , R 2 } .
Revised: = { R 1 , R 2 } .

Original: The desired mapping is ϕ : e ( jump : r 2 ce -arg )
Revised: The desired mapping is ϕ : e ( jump : r 2 ce -arg )

Original: (cid:55)→ e ( JUMP JUMP ) , that is, ϕ : e F ( jump ) ⊗ e R ( r 2 ce -arg )
Revised: (cid:55)→ e ( JUMP JUMP ) , that is, ϕ : e F ( jump ) ⊗ e R ( r 2 ce -arg )

Original: (cid:55)→ e F ( JUMP ) ⊗ e R ( R 1 )
Revised: (cid:55)→ e F ( JUMP ) ⊗ e R ( R 1 )

Original: +
Revised: +

Original: e F ( JUMP ) ⊗ e R
Revised: e F ( JUMP ) ⊗ e R

Original: ( R 2 ) .
Revised: ( R 2 ) .

Original: We now show that this can be accomplished through the separate ﬁller- and role-mappings ϕ F : e F ( jump ) (cid:55)→ e F ( JUMP ) and ϕ R : e R ( r 2 ce -arg )
Revised: We now show that this can be accomplished through the separate ﬁller- and role-mappings ϕ F : e F ( jump ) (cid:55)→ e F ( JUMP ) and ϕ R : e R ( r 2 ce -arg )

Original: (cid:55)→ e R ( R 1 ) + e R ( R 2 ) .
Revised: (cid:55)→ e R ( R 1 ) + e R ( R 2 ) .

Original: To do this, we show that if ϕ F , ϕ R are respectively computed over embedding vectors through weight matrices W F , W R in a NN, then the weight tensor W = W F ⊗ W R will correctly map the embedding of jump twice to the embedding of JUMP JUMP .
Revised: To do this, we show that if ϕ F , ϕ R are respectively computed over embedding vectors through weight matrices W F , W R in a NN, then the weight tensor W = W F ⊗ W R will correctly map the embedding of jump twice to the embedding of JUMP JUMP .

Original: The tensor product of weight matrices is deﬁned as [ W F ⊗ W R ] jlik =
Revised: The tensor product of weight matrices is deﬁned as [ W F ⊗ W R ] jlik =

Original: [ W F ] ji [ W R ] lk which entails that ( W F ⊗ W R )
Revised: [ W F ] ji [ W R ] lk which entails that ( W F ⊗ W R )

Original: [ f ⊗ r ] =
Revised: [ f ⊗ r ] =

Original: [ W F f ] ⊗ [ W R r ] .
Revised: [ W F f ] ⊗ [ W R r ] .

Original: Therefore W [ e ( jump twice )]
Revised: Therefore W [ e ( jump twice )]

Original: =
Revised: =

Original: W [ e ( jump : r 2 ce -arg )]
Revised: W [ e ( jump : r 2 ce -arg )]

Original: =
Revised: =

Original: W [ e F ( jump ) ⊗ e R ( r 2 ce -arg )]
Revised: W [ e F ( jump ) ⊗ e R ( r 2 ce -arg )]

Original: =
Revised: =

Original: [ W F e F ( jump )] ⊗ [ W R e R ( r 2 ce -arg )]
Revised: [ W F e F ( jump )] ⊗ [ W R e R ( r 2 ce -arg )]

Original: =
Revised: =

Original: e F ( JUMP ) ⊗ [ e R ( R 1 )
Revised: e F ( JUMP ) ⊗ [ e R ( R 1 )

Original: + e R
Revised: + e R

Original: ( R 2 )]
Revised: ( R 2 )]

Original: =
Revised: =

Original: e F ( JUMP ) ⊗ e R ( R 1 )
Revised: e F ( JUMP ) ⊗ e R ( R 1 )

Original: +
Revised: +

Original: e F ( JUMP ) ⊗ e R
Revised: e F ( JUMP ) ⊗ e R

Original: ( R 2 )
Revised: ( R 2 )

Original: =
Revised: =

Original: e ( JUMP JUMP ) , as desired.
Revised: e ( JUMP JUMP ) , as desired.

Original: This suggests that greater compositional generalization might be achieved by future models that are explicitly biased to utilize TPRs as internal representations and explicitly biased to factor their processing weights into those that process ﬁllers (independently of their roles) and those that process roles (independently of their ﬁllers), as in the SCAN example with W = W F ⊗ W R above.
Revised: This suggests that greater compositional generalization might be achieved by future models that are explicitly biased to utilize TPRs as internal representations and explicitly biased to factor their processing weights into those that process ﬁllers (independently of their roles) and those that process roles (independently of their ﬁllers), as in the SCAN example with W = W F ⊗ W R above.