Original: RNN S IMPLICITLY IMPLEMENT TENSOR - PRODUCT REPRESENTATIONS
Revised: RNN S IMPLICITLY IMPLEMENT TENSOR - PRODUCT REPRESENTATIONS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Recurrent neural networks (RNNs) can learn continuous vector representations of symbolic structures such as sequences and sentences; these representations often exhibit linear regularities (analogies).
Revised: Recurrent neural networks (RNNs) can learn continuous vector representations of symbolic structures such as sequences and sentences; these representations often exhibit linear regularities (analogies).

Original: Such regularities motivate our hypothesis that RNNs  implicitly compile symbolic structures into tensor product representations (TPRs; Smolensky, 1990), which additively combine tensor products of vectors representing roles (e.g., sequence position) and vectors representing ﬁllers (e.g.,a particular word).
Revised: Such regularities motivate our hypothesis that RNNs that show such regularities implicitly compile symbolic structures into tensor product representations (TPRs; Smolensky, 1990), which additively combine tensor products of vectors representing roles (e.g., sequence position) and vectors representing ﬁllers (e.g., particular words).

Original: To test this hypothesis, we introduce Tensor Product Decomposition Networks (TPDNs), which use TPRs to approximate existing vector representations.
Revised: To test this hypothesis, we introduce Tensor Product Decomposition Networks (TPDNs), which use TPRs to approximate existing vector representations.

Original: We demonstrate using synthetic data that TPDNs can successfully approximate linear and tree-based RNN autoencoder representations, suggesting that these representations exhibithighly interpretable compositional structure; we explore the settings that lead RNNs to induce such structure-sensitive representations.
Revised: We demonstrate using synthetic data that TPDNs can successfully approximate linear and tree-based RNN autoencoder representations, suggesting that these representations exhibit interpretable compositional structure; we explore the settings that lead RNNs to induce such structuresensitive representations.

Original: By contrast with these results, TPDN experimentswith four standard sentence encoding models show that those sentence encodings can be largely approximated using bag-of-words representations, with only marginal improvements from more sophisticated structures.
Revised: By contrast, further TPDN experiments show that the representations of four models trained to encode naturally-occurring sentences can be largely approximated with a bag of words, with only marginal improvements from more sophisticated structures.

Original: We conclude that TPDNs provide a powerful method for interpreting vector representations, and that standard RNNs can induce compositional sequence representations that are remarkably well approximated by
Revised: We conclude that TPDNs provide a powerful method for interpreting vector representations, and that standard RNNs can induce compositional sequence representations that are remarkably well approximated by

Original: TPRs; at the same time, existing training tasks for sentence representation learning may not be sufﬁcient for inducing robust structural representations.
Revised: TPRs; at the same time, existing training tasks for sentence representation learning may not be sufﬁcient for inducing robust structural representations.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Neural networks construct continuous vector representations of discrete elements such as words or sentences. In some cases, the spatial relationships between such vector representations have been argued to display geometric regularities that parallel plausible symbolic structures of the elements being represented (Mikolov et al. 2013; see Figure 1).
Revised: Compositional symbolic representations are widely held to be necessary for intelligent behavior (Newell, 1980), particularly in the domain of language (Fodor & Pylyshyn, 1988). However, neural networks have shown great success in natural language processing despite using continuous vector representations rather than explicit symbolic structures.

Original: These regularities may explain why these continuous representations succeed at tasks traditionally believed to require symbolic manipulation.
Revised: How can these continuous representations yield such success in a domain traditionally believed to require symbolic manipulation?

Original: Analogical relationships  such as those of Mikolov et al. 
Revised: One possible answer is that neural network representations implicitly encode compositional structure. This hypothesis is supported by the spatial relationships between such vector representations, which have been argued to display geometric regularities that parallel plausible symbolic structures of the elements being represented (Mikolov et al. 2013; see Figure 1). [Figure]

Original: (2013) are special cases of linearity properties shared by severalgeneral methodsthat were developed in the 1990s for designing compositional vectorial embeddings of symbolic structures.
Revised: Analogical relationships such as those in Figure 1 are special cases of linearity properties shared by several methods developed in the 1990s for designing compositional vector embeddings of symbolic structures.

Original: The most general of these is tensor product representations (TPRs; Smolensky 1990).
Revised: The most general of these is tensor product representations (TPRs; Smolensky 1990).

Original: Symbolic structures are ﬁrst decomposed into ﬁller-role[Figure] bindings ; for example, to represent the sequence [5 , 2 , 4] , the ﬁller 5 may be bound to the role ofthe ﬁrst elementof the sequence , the ﬁller 2 may be bound to the role ofthe second elementof the sequence , and so on.
Revised: Symbolic structures are ﬁrst decomposed into ﬁller-role bindings ; for example, to represent the sequence [5 , 2 , 4] , the ﬁller 5 may be bound to the role of ﬁrst element , the ﬁller 2 may be bound to the role of second element , and so on.

Original: Each ﬁller f i and — crucially — each role r i has a vector embedding; these two vectors are combined using their tensor product f i ⊗ r i , and these tensor products are summed to produce the representation of the sequence: (cid:80)
Revised: Each ﬁller f i and — crucially — each role r i has a vector embedding; these two vectors are combined using their tensor product f i ⊗ r i , and these tensor products are summed to produce the representation of the sequence: (cid:80)

Original: f i ⊗ r i .
Revised: f i ⊗ r i .

Original: This linear combination can predict the linear relations between sequence representations illustrated in Figure 1.
Revised: This linear combination can predict the linear relations between sequence representations illustrated in Figure 1.

Original: In this article, we test the hypothesis thatstructure-sensitive recurrent neural network vector representations of sequences can be approximated as a linear combination of ﬁller-role bindings, as in TPRs.
Revised: In this article, we test the hypothesis that vector representations of sequences can be approximated as a sum of ﬁller-role bindings, as in TPRs.

Original: Smolensky (1990) showed how to perform tensor product ﬁller-role binding in the forward direction given known role and vector embeddings; we propose the Tensor Product Decomposition Network (TPDN) which inverts this process, taking a set of continuous vector representations to be analyzed and learning ﬁller and role embeddings that best predict those vectors, given a particular hypothesis for the relevant set of roles (e.g., sequence indexes or structural positions in a parse tree).
Revised: We introduce the Tensor Product Decomposition Network (TPDN) which takes a set of continuous vector representations to be analyzed and learns ﬁller and role embeddings that best predict those vectors, given a particular hypothesis for the relevant set of roles (e.g., sequence indexes or structural positions in a parse tree).

Original: To derive structure-sensitive representations, in Section 2 we look at a taskthat is driven by structure, not content: autoassociation of sequences of meaningless symbols, denoted by digits.
Revised: To derive structure-sensitive representations, in Section 2 we look at a task driven by structure, not content: autoencoding of sequences of meaningless symbols, denoted by digits.

Original: Theprimary focus here is on sequential structure, although we also devise a version of the task that incorporates tree structure.
Revised: The focus here is on sequential structure, although we also devise a version of the task that uses tree structure.

Original: For theintermediate representations of these autoencoders, TPDNs ﬁnd excellent approximations that are TPRs.
Revised: For the representations learned by these autoencoders, TPDNs ﬁnd excellent approximations that are TPRs.

Original: In Section 3, we turn toprominent sentence-embedding models in the contemporary literature.
Revised: In Section 3, we turn to sentence-embedding models from the contemporary literature.

Original: It is an open question how structure-sensitive these representations are; to the degree that they are structure-sensitive, our hypothesis is that they can be approximated by TPRs.
Revised: It is an open question how structure-sensitive these representations are; to the degree that they are structuresensitive, our hypothesis is that they can be approximated by TPRs.

Original: Here, TPDNs ﬁnd less accurate approximations, but they also show that a TPR equivalent to a bag-of-words already provides a reasonably good approximation; these results call into question just how structure-sensitive these sentence representations are.
Revised: Here, TPDNs ﬁnd less accurate approximations, but they also show that a TPR equivalent to a bag-of-words already provides a reasonable approximation; these results suggest that these sentence representations are not very structure-sensitive.

Original: We therefore return to synthetic data in Section 4, exploring the model-architecture and training-task settings that aremore likely to lead RNNs to induce structure-sensitive representations.
Revised: We therefore return to synthetic data in Section 4, exploring which architectures and training tasks are likely to lead RNNs to induce structure-sensitive representations.

Original: Throughout these studies, TPDNs provide a powerful method for interpretingcontinuous vector representations and show that standard RNNs can induce compositional representationsof sequences that are remarkably well approximated by TPRs.
Revised: To summarize the contributions of this work, TPDNs provide a powerful method for interpreting vector representations, shedding light on hard-to-understand neural architectures. We show that standard RNNs can induce compositional representations that are remarkably well approximated by TPRs and that the nature of these representations depends, in intrepretable ways, on the architecture and training task. Combined with our ﬁnding that standard sentence encoders do not seem to learn robust representations of structure, these ﬁndings suggest that more structured architectures or more structure-dependent training tasks could improve the compositional capabilities of existing models.

Original: 1. T HE T ENSOR P RODUCT D ECOMPOSITION N ETWORK
Revised: 1. T HE T ENSOR P RODUCT D ECOMPOSITION N ETWORK

Original: The Tensor Product Decomposition Network (TPDN), depicted in Figure 2c, learns a TPR that best approximates an existing set of vector encodings.
Revised: The Tensor Product Decomposition Network (TPDN), depicted in Figure 2c, learns a TPR that best approximates an existing set of vector encodings.

Original: While this architecture can be applied to any structured space, including embeddings of images or words, this work focuses on the application of
Revised: While this architecture can be applied to any structured space, including embeddings of images or words, this work focuses on the application of

Original: TPDNs to sequences.
Revised: TPDNs to sequences.

Original: The model is provided with a hypothesized role scheme, the dimensionality of the ﬁller embeddings, and the dimensionality of the role embeddings.
Revised: The model is provided with a hypothesized role scheme, the dimensionality of the ﬁller embeddings, and the dimensionality of the role embeddings.

Original: The elements of each sequence are assumed to be the ﬁllers in that sequence’s representation; for example, if the hypothesized roles are positional indexes counting from the end of the sequence, then the hypothesized ﬁller-role bindings for [5 , 2 , 4] would be (4: last , 2: second-to-last , 5: third-to-last ).
Revised: The elements of each sequence are assumed to be the ﬁllers in that sequence’s representation; for example, if the hypothesized roles are positional indexes counting from the end of the sequence, then the hypothesized ﬁller-role bindings for [5 , 2 , 4] would be (4: last , 2: second-to-last , 5: third-to-last ).

Original: The model then learns embeddings for these ﬁllers and roles that minimize the distance between the TPRs generated from these embeddings and the existing encodings of the sequences.
Revised: The model then learns embeddings for these ﬁllers and roles that minimize the distance between the TPRs generated from these embeddings and the existing encodings of the sequences.

Original: Before the comparison is performed, the tensor product (which is a matrix) is ﬂattened into a vector, and a linear transformation M is applied (see Appendix B for an ablation study showing that this transformation, which was not a part of the original TPR proposal, is necessary).
Revised: Before the comparison is performed, the tensor product (which is a matrix) is ﬂattened into a vector, and a linear transformation M is applied (see Appendix B for an ablation study showing that this transformation, which was not a part of the original TPR proposal, is necessary).

Original: The overall function computed by the architecture is thus M ( ﬂatten ( (cid:80) i r i ⊗ f i )) .
Revised: The overall function computed by the architecture is thus M ( ﬂatten ( (cid:80) i r i ⊗ f i )) .

Original: PyTorch code implementing the TPDN model will be made available on GitHub.
Revised: PyTorch code implementing the TPDN model will be made available on GitHub.

Original:  A PPROXIMATING RNN AUTOENCODER REPRESENTATIONS
Revised:  A PPROXIMATING RNN AUTOENCODER REPRESENTATIONS

Original: To establish the effectiveness of the TPDN at uncovering the structural representations used by
Revised: To establish the effectiveness of the TPDN at uncovering the structural representations used by

Original: RNNs, we ﬁrst apply the TPDN to sequence-to-sequence networks trained on an autoencoding objective: they are expected to encode a sequence of digits and then decode that encoding to reproduce the same sequence (Figure 2a).
Revised: RNNs, we ﬁrst apply the TPDN to sequence-to-sequence networks trained on an autoencoding ob- E      (a)  -4 -1-12 -3-20 -5(b) + 3 r7 r6 r(1) (2) (3) (4) (5) E (c) + 3 r7 r6 r E   (d) jective: they are expected to encode a sequence of digits and then decode that encoding to reproduce the same sequence (Figure 2a).

Original: In addition to testing the TPDN, this experiment also addresses a scientiﬁc question: do different architectures  induce different representations?E      (a)  -4 -1-12 -3-20 -5(b) + 3 r7 r6 r(1) (2) (3) (4) (5) E (c) + 3 r7 r6 r E   (d)
Revised: In addition to testing the TPDN, this experiment also addresses a scientiﬁc question: do different architectures (speciﬁcally, unidirectional, bidirectional, and tree-based sequence-to-sequence models) induce different representations?

Original: 2. E XPERIMENTAL SETUP
Revised: 2. E XPERIMENTAL SETUP

Original: Digit sequences: The sequences consisted of the digits from 0 to 9.
Revised: Digit sequences: The sequences consisted of the digits from 0 to 9.

Original: We randomly generated 50, unique sequences with lengths ranging from 1 to 6 inclusive and averaging 5.2; these sequences were divided into 40,000 training sequences, 5,000 development sequences, and 5,000 test sequences.
Revised: We randomly generated 50, unique sequences with lengths ranging from 1 to 6 inclusive and averaging 5.2; these sequences were divided into 40,000 training sequences, 5,000 development sequences, and 5,000 test sequences.

Original: Architectures: For all sequence-to-sequence networks, we used gated recurrent units (GRUs, Cho et al.
Revised: Architectures: For all sequence-to-sequence networks, we used gated recurrent units (GRUs, Cho et al.

Original: as the recurrent units.
Revised: as the recurrent units.

Original: We considered three types of network topologies: unidirectional, bidirectional, and tree-based, each of which can be used for the encoder or the decoder.
Revised: We considered three encoder-decoder architectures: unidirectional, bidirectional, and tree-based.

Original: The unidirectional encoders and decoders are the standard left-to-right sequence-to-sequence setup, used by Sutskever et al.
Revised: The unidirectional encoders and decoders follow the setup of Sutskever et al.

Revised: (2014): the encoder is fed the input elements one at a time, updating its hidden state after each element.

Revised: The decoder then produces the output sequence using the ﬁnal hidden state of the encoder as its input.

Original: The bidirectional encoder combines left-to-right and right-to-left unidirectional encoders (Schuster & Paliwal, 1997); for symmetry, we also create a bidirectional decoder, which has both a left-to-right and a right-to-left unidirectional decoder whose hidden states are concatenated to form bidirectional hidden states from which output predictions are made.
Revised: The bidirectional encoder combines left-to-right and right-to left unidirectional encoders (Schuster & Paliwal, 1997); for symmetry, we also create a bidirectional decoder, which has both a left-to-right and a right-to-left unidirectional decoder whose hidden states are concatenated to form bidirectional hidden states from which output predictions are made.

Original: Our ﬁnal topology is tree-based RNNs (Pollack, 1990; Socher et al., 2010), speciﬁcally the Tree-GRU encoder of Chen et al.
Revised: Our ﬁnal topology is tree-based RNNs (Pollack, 1990; Socher et al., 2010), speciﬁcally the Tree-GRU encoder of Chen et al.

Original: (2017) and the tree decoder of Chen et al.
Revised: (2017) and the tree decoder of Chen et al.

Original: These architectures require a tree structure as part of their input;since digit sequences have no natural hierarchical structure, we generated a tree for each sequence using a deterministic algorithm that clusters digits based on their values (see Appendix C).
Revised: These architectures require a tree structure as part of their input; we generated a tree for each sequence using a deterministic algorithm that clusters digits based on their values (see Appendix C).

Original: To control for initialization effects, we trained ﬁve instances of each architecture with different random initializations.
Revised: To control for initialization effects, we trained ﬁve instances of each architecture with different random initializations.

Original: Role schemes: We consider 6 possible methods that networks might use to represent the roles of speciﬁc digits within a sequence; see Figure 3a for examples of these role schemes.
Revised: Role schemes: We consider 6 possible methods that networks might use to represent the roles of speciﬁc digits within a sequence; see Figure 3a for examples of these role schemes.

Original: Left-to-right: Each digit’s role is its index in the sequence, counting from left to right.
Revised: Left-to-right: Each digit’s role is its index in the sequence, counting from left to right.

Original: Right-to-left: Each digit’s role is its index in the sequence, counting from right to left.
Revised: Right-to-left: Each digit’s role is its index in the sequence, counting from right to left.

Original: Bidirectional:
Revised: Bidirectional:

Original: Each digit’s role is an ordered pair containing its left-to-right index and its right-to-left index (cf.
Revised: Each digit’s role is an ordered pair containing its left-to-right index and its right-to-left index (cf.

Original: human representations of spelling, Fischer-Baum et al. 2010).
Revised: human representations of spelling, Fischer-Baum et al. 2010).

Original: Wickelroles:
Revised: Wickelroles:

Original: Each digit’s role is the digit before it and the digit after it (Wickelgren, 1969).
Revised: Each digit’s role is the digit before it and the digit after it (Wickelgren, 1969).

Original: The edges of the sequence are marked with the special symbol #.

Original: Such roles were used by

Original: Rumelhart et al.

Original: in

Original: modeling English past tense formation.

Original: Tree positions: Each digit’s role is its position in a tree, such as RRL (left child of right child of right child of root).
Revised: Tree positions: Each digit’s role is its position in a tree, such as RRL (left child of right child of right child of root).

Original: The tree structuresconsidered are  theoutputs of the digitparsing algorithm in Appendix C. 6.
Revised: The tree structures are given by the algorithm in Appendix C. 6.

Original: Bag-of-words: All digits have the same role.
Revised: Bag-of-words: All digits have the same role.

Original: We call this a bag-of-words because it represents which digits (“words”) are present and in what quantities, but ignores their positions.
Revised: We call this a bag-of-words because it represents which digits (“words”) are present and in what quantities, but ignores their positions.

Original: Hypothesis: We hypothesize that RNN autoencoders will learn to use role representations that parallel their architectures: left-to-right roles for a unidirectional network, bidirectional roles for a bidirectional network, and tree-position roles for a tree-based network.

Original: Left to-right
Revised: Left to right

Original: Right to left
Revised: Right to left

Original: Bidirectional (0, 3) (1, 2) (2, 1) (3, 0) (0, 5) (1, 4) (2, 3) (3, 2) (4, 1) (5, 0)
Revised: Bidirectional (0, 3) (1, 2) (2, 1) (3, 0) (0, 5) (1, 4) (2, 3) (3, 2) (4, 1) (5, 0)

Original: Wickelroles #311 # #52319 #
Revised: Wickelroles #311 # #52319 #

Original: Tree
Revised: Tree

Original: L
Revised: L

Original: RLL RLR
Revised: RLL RLR

Original: RR
Revised: RR

Original: LL
Revised: LL

Original: LRLL LRLR
Revised: LRLL LRLR

Original: LRRL LRRR
Revised: LRRL LRRR

Original: R
Revised: R

Original: Bag of words rrrrrrrrrr(a)          (b) (
Revised: Bag of words rrrrrrrrrr(a)          (b) (

Original: [Figure]
Revised: [Figure]

Revised: Hypothesis: We hypothesize that RNN autoencoders will learn to use role representations that parallel their architectures: left-to-right roles for a unidirectional network, bidirectional roles for a bidirectional network, and tree-position roles for a tree-based network.

Original: Evaluation: We evaluate how well a given sequence-to-sequence network can be approximated by a TPR with a particular role scheme as follows.
Revised: Evaluation: We evaluate how well a given sequence-to-sequence network can be approximated by a TPR with a particular role scheme as follows.

Original: First, we train a TPDN with the role scheme in question (Section 1.1).
Revised: First, we train a TPDN with the role scheme in question (Section 1.1).

Original: Then, we take the original encoder/decoder network and substitute the ﬁtted
Revised: Then, we take the original encoder/decoder network and substitute the ﬁtted

Original: TPDN for its encoder (Figure 2d).
Revised: TPDN for its encoder (Figure 2d).

Original: We do not conduct any additional training upon this substituted network; the decoder retains exactly the weights it learned in association with the original encoder, while the TPDN retains exactly the weights it learned for approximating the original encoder (including the weights on the ﬁnal linear layer).
Revised: We do not conduct any additional training upon this substituted network; the decoder retains exactly the weights it learned in association with the original encoder, while the TPDN retains exactly the weights it learned for approximating the original encoder (including the weights on the ﬁnal linear layer).

Original: We then compute the accuracy of the resulting hybrid network; we call this metric the substitution accuracy .
Revised: We then compute the accuracy of the resulting hybrid network; we call this metric the substitution accuracy .

Original: High substitution accuracy indicates that the TPDN has approximated the encoder well enough for the decoder to handle the resulting vectors.
Revised: High substitution accuracy indicates that the TPDN has approximated the encoder well enough for the decoder to handle the resulting vectors.

Original: 2. R ESULTS
Revised: 2. R ESULTS

Original: Performance of seq2seq networks: The unidirectional and tree-based architectures both performed the training task nearly perfectly, with accuracies of 0.999 and 0.989 (averaged across ﬁve runs).
Revised: Performance of seq2seq networks: The unidirectional and tree-based architectures both performed the training task nearly perfectly, with accuracies of 0.999 and 0.989 (averaged across ﬁve runs).

Original: Accuracy was lower (0.834) for the bidirectional architecture; this might mean that the hidden size of 60 becomes too small when divided into two 30-dimensional halves, one half for each direction.
Revised: Accuracy was lower (0.834) for the bidirectional architecture; this might mean that the hidden size of 60 becomes too small when divided into two 30-dimensional halves, one half for each direction.

Original: Quality of TPDN approximation: For each of the six role schemes, we ﬁtted a tensor product  network onto the vectors generated by the trained encoder, and evaluated it using the substitution accuracy metric described in Section 2.1.
Revised: Quality of TPDN approximation: For each of the six role schemes, we ﬁtted a tensor product decomposition network onto the vectors generated by the trained encoder, and evaluated it using the substitution accuracy metric described in Section 2.1.

Original: The results are shown in Figure 3c.
Revised: The results are shown in Figure 3c.

Original: The different architectures do use different representations to solve the task.
Revised: The different architectures do use different representations to solve the task.

Original: The tree-based autoencoder can be successfully approximated using tree-position roles and not using any of the other role schemes.
Revised: The tree-based autoencoder can be successfully approximated using tree-position roles and not using any of the other role schemes.

Original: By contrast, the unidirectional architecture is approximated very closely (with a substitution accuracy of over 0.99 averaged across ﬁve runs) by bidirectional roles.
Revised: By contrast, the unidirectional architecture is approximated very closely (with a substitution accuracy of over 0.99 averaged across ﬁve runs) by bidirectional roles.

Original: Left-to-right roles are also fairly successful (accuracy = 0.87), and right-to-left roles are decidedly unsuccessful (accuracy = 0.11).
Revised: Left-to-right roles are also fairly successful (accuracy = 0.87), and right-to-left roles are decidedly unsuccessful (accuracy = 0.11).

Original: This asymmetry suggests that the unidirectional network is using mildly bidirectional roles: while it is best approximated by bidirectional roles, the representation strongly favors one direction over the other.
Revised: This asymmetry suggests that the unidirectional network is using mildly bidirectional roles: while it is best approximated by bidirectional roles, the representation strongly favors one direction over the other.

Original: Though the model uses the bidirectional role scheme, then, roles with the same left-to-right position (e.g. (2,3), (2,4), and (2,5)) can be collapsed without much loss of accuracy.
Revised: Though the model uses the bidirectional role scheme, then, roles with the same left-to-right position (e.g. (2,3), (2,4), and (2,5)) can be collapsed without much loss of accuracy.

Original: Finally, the bidirectional architecture is not approximated well by any of the role schemes we investigated.
Revised: Finally, the bidirectional architecture is not approximated well by any of the role schemes we investigated.

Original: This could mean that the bidirectional architecture is implementing some other role scheme we did not consider.
Revised: This could mean that the bidirectional architecture is implementing some other role scheme we did not consider.

Original: Alternately, it might have simply not adopted a robust method for representing sequence structure; this could explain why its accuracy on the training task was relatively low (0.83).
Revised: Alternately, it might have simply not adopted a robust method for representing sequence structure; this could explain why its accuracy on the training task was relatively low (0.83).

Original:  S ENTENCE ENCODINGS
Revised:  E NCODINGS OF NATURALLY - OCCURRING SENTENCES

Original: Will the success of TPDNs in approximating representations fromRNN autoencoders of digit sequences extend to larger-scale representation?
Revised: Will the success of TPDNs in approximating representations from digit sequence autoencoders extend to representations from models trained on naturally occurring data?

Original: We explore this question using sentence representationsextracted from four existing trained models: InferSent (Conneau et al., 2017), a bidirectional LSTM trained on the Stanford Natural Language Inference (SNLI) corpus (Bowman et al., 2015); Skip-thought (Kiros et al., 2015), an LSTM trained to predict, given a sentence, what the preceding or following sentence is; the Stanford sentiment model (SST) (Socher et al., 2013), a tree-based recursive neural tensor network trained to predict the sentiment of movie reviews; and SPINN (Bowman et al., 2016), a tree-based RNN trained on SNLI.
Revised: We explore this question using sentence representations from four existing trained models: InferSent (Conneau et al., 2017), a BiLSTM trained on the Stanford Natural Language Inference (SNLI) corpus (Bowman et al., 2015); Skip-thought (Kiros et al., 2015), an LSTM trained to predict, given a sentence, what the preceding or following sentence is; the Stanford sentiment model (SST) (Socher et al., 2013), a treebased recursive neural tensor network trained to predict the sentiment of movie reviews; and SPINN (Bowman et al., 2016), a tree-based RNN trained on SNLI.

Original: More model details are in Appendix E.
Revised: More model details are in Appendix E.

Original: 3. TPDN APPROXIMATION
Revised: 3. TPDN APPROXIMATION

Original: We now ﬁt TPDNs to these four sentence encoding models.
Revised: We now ﬁt TPDNs to these four sentence encoding models.

Original: We experiment with all of the role schemes used in Section 2 except for Wickelroles; for sentence representations, the vocabulary size | V | is so large that the Wickelrole scheme, which requires | V | 2 distinct roles, becomes intractable.
Revised: We experiment with all of the role schemes used in Section 2 except for Wickelroles; for sentence representations, the vocabulary size | V | is so large that the Wickelrole scheme, which requires | V | 2 distinct roles, becomes intractable.

Original: Preliminary experiments showed that the TPDN performedextremely poorly when learning the ﬁller embeddings from scratch. Instead, we used pretrained word embeddings; for each model, we use the word embeddings used by that model.
Revised: Preliminary experiments showed that the TPDN performed poorly when learning the ﬁller embeddings from scratch, so we used pretrained word embeddings; for each model, we use the word embeddings used by that model.

Original: To partially ﬁne-tune the embeddings, we train a linear transformation on top of the word embedding layer (though the embeddings themselves remain ﬁxed).
Revised: We ﬁne-tuned the embeddings with a linear transformation on top of the word embedding layer (though the embeddings themselves remain ﬁxed).

Original: Thus,in this case, what the model has to learn are: the role embeddings, the linear transformation to apply to the ﬁxed ﬁller embeddings, and the ﬁnal linear transformation applied to the sum of the ﬁller/role bindings.
Revised: Thus, what the model has to learn are: the role embeddings, the linear transformation to apply to the ﬁxed ﬁller embeddings, and the ﬁnal linear transformation applied to the sum of the ﬁller/role bindings.

Original: We train the decomposition on the sentence embeddings that each of the four models generate for all the premise sentences in the SNLI corpus (Bowman et al., 2015).
Revised: We train the decomposition on the sentence embeddings that each of the four models generate for all the premise sentences in the SNLI corpus (Bowman et al., 2015).

Original: We also tried instead using the sentences in the WikiText-2 corpus (Merity et al., 2016) but found better performance with the SNLI sentences. This is plausibly because the shorter, simpler sentences in the SNLI corpus made it easier for the model to learn the role embeddings without distraction from the ﬁllers.
Revised: For other training details see

Original: For other training details see Appendix E. Table 1a shows the mean squared errors  for various role schemes.
Revised: Appendix E. Table 1a shows the mean squared errors (MSEs) for various role schemes.

Original: In general, the mean squared errors show only small differences betweenthe different role schemes, with perhaps the one exception being that tree-position roles do perform noticeably better than other role schemes for the SST model.
Revised: In general, the MSEs show only small differences between role schemes, with perhaps the one exception being that tree-position roles do perform noticeably better than other role schemes for SST.

Original: Notably, bag-of-words roles perform nearly as well as the other role schemes, which is in stark contrast to the poor performance of bag-of-words roles in Section 2.
Revised: Notably, bagof-words roles perform nearly as well as the other role schemes, which is in stark contrast to the poor performance of bag-of-words roles in Section 2.

Original: Because mean squared error is not very interpretable on an absolute level, in the next section we perform more interpretable tests by looking at performance on downstream tasks.
Revised: MSE is useful for comparing models but is not as useful for assessment of a model’s absolute performance since the exact value of this error is not very interpretable. In the next section, we use downstream tasks for a more interpretable evaluation.

Original: 3. P ERFORMANCE ON DOWNSTREAM TASKS
Revised: 3. P ERFORMANCE ON DOWNSTREAM TASKS

Original: Tasks:
Revised: Tasks:

Original: We assess how well the tensor product approximations perform at downstream tasks compared to the models they approximate.
Revised: We assess how well the tensor product approximations perform at downstream tasks compared to the models they approximate.

Original: We use four tasks that are widely accepted for evaluations of sentence embeddings: (1) Stanford Sentiment Treebank (SST), rating the sentiment of movie reviews (Socher et al., 2013); (2) Microsoft Research Paraphrase Corpus (MRPC), classifying whethertwo sentences are paraphrases of each other (Dolan et al., 2004); (3) Semantic Textual Similarity Benchmark (STS-B), labeling how similar two sentences are on a continuous scale from 0 to 5 (Cer (a) [Table] LTR
Revised: We use four tasks that are widely accepted for evaluation of sentence embeddings: (1) Stanford Sentiment Treebank (SST), rating the sentiment of movie reviews (Socher et al., 2013); (2) Microsoft Research Paraphrase Corpus (MRPC), classifying whether

Original: [Table]
Revised: [Table]

Original: InferSent

Original: [Table]
Revised: [Table]

Original: et al., 2017); and (4) Stanford Natural Language Inference (SNLI), determining whether one sentence entails a second sentence, contradicts the second sentence, or neither (Bowman et al., 2015).

Original: Evaluation: We used SentEval (Conneau & Kiela, 2018) to train a classiﬁer for each of the four tasks on the original model being approximated.
Revised: Evaluation: We use SentEval (Conneau & Kiela, 2018) to train a classiﬁer for each task on the original encodings produced by the sentence encoding model.

Original: We freeze this classiﬁer and use it to classify the vectors generated by the TPDN.
Revised: We freeze this classiﬁer and use it to classify the vectors generated by the TPDN.

Original: We then measure what proportion of the classiﬁer’s predictions for the approximation match its predictions for the original model.
Revised: We then measure what proportion of the classiﬁer’s predictions for the approximation match its predictions for the original sentence encodings.

Original: (We also try training SentEval classiﬁers on top of the TPDN instead of the original model; see Appendix F for these results.

Original: In general, for all models besides Skip-thought, the TPDN approximations perform nearly as well as the original models, and in some cases the approximations even outperform the originals.)

Original: Results: For all tasks besides SNLI, we found that there was not a marked difference between bagof-words roles and other role schemes. The results averaged across these tasks are in Table 2a.
Revised: Results: For all tasks besides SNLI, we found that there was not a marked difference between bag-of-words roles and other role schemes (Table 2a).

Original: For SNLI, we did see instances where other role schemes outperformed bag-of-words (Table 2b).
Revised: For SNLI, we did see instances where other role schemes outperformed bag-of-words (Table 2b).

Original: Tables 2a and 2b both show that Skip-thought’s behavior cannot be approximated very well by any of the role schemes that we considered. Setting aside Skip-thought, we now discuss the other three models.
Revised: Considering the individual models’ behavior with SNLI, both tree-based models (SST and SPINN) are best approximated with tree-based roles.

Original: If these models truly contained no representation of structure, then bag-of-words roles would perform perfectly in Tables 2a and 2b.
Revised: InferSent is better approximated with structural role schemes than with bag-of-words roles, but performance is similar across all structural role schemes.

Original: They fact that bag-of-words roles do not achieve accuracies at or very close to 1.0 indicates that there is some inﬂuence of structure on the models, and Table 2b (along with Table 1a) indicates that the types of structure that are weakly encoded appear to be tree-based for the tree-based models (SST and SPINN) and either tree-based or linearindex-based for InferSent.
Revised: Finally, Skip-thought cannot be approximated well with any of the role schemes that we considered. It is unclear why Skip-thought has lower results than the other models. Overall, even for SNLI, bag-of-words roles provide a reasonably strong approximation, with only modest improvements provided by more structured role schemes.

Original: However, we can tell that this structural information  is not represented so fully as to constitute a proper tensor product representation, because if it did we would expect the associated metrics in Tables 2a and 2b to be near 1.0 for the relevant structural role scheme, and to be near 0.0 for the bag-of-words roles, as was the case with the experiments in Section 2. Thus, overall we can conclude that  the sentence encoding models do represent some weak notion of structure but that this representation of structure is not very robust orsystematic and is largely overwhelmed by the fact that most of their behavior can be explained purely with a bag-of-words model.
Revised: Based on these results, we hypothesize that these models’ representations can be characterized as a bag-of-words representation plus some incomplete structural information that is not always encoded. This explanation is consistent with the fact that bag-of-words roles yield a strong but imperfect approximation for the sentence embedding models. However, this is simply a conjecture; it is possible that these models do use a robust, systematic structural representation that either involves a role scheme we did not test or that cannot be characterized as a tensor product representation at all.

Original: 3. A NALOGIES
Revised: 3. A NALOGIES

Original: Perhaps a more targeted approach can more clearly illuminate the presence of structure in these models.

Original: We therefore complement the TPDN analyses by examining analogies as a source of evidence for a model’s representations.
Revised: We now complement the TPDN analyses by examining analogies as a source of evidence for a model’s representations.

Original: We construct sentence-based analogies that we would expect to hold only under certain role schemes.
Revised: We construct sentence-based analogies that we would expect to hold only under certain role schemes.

Original: For example, consider the following analogy (expressed as an equation as in Mikolov et al. 2013):
Revised: For example, consider the following analogy (expressed as an equation as in Mikolov et al. 2013):

Original: I see now − I see = you know now − you know (1)
Revised: [Equation]

Original: A left-to-right role scheme makes (1) equivalent to (2) ( f : r denotes the binding of ﬁller f to role r ): (I:0 + see:1 + now:2) – (I:0 + see:1 ) =
Revised: A left-to-right role scheme makes (1) equivalent to (2) ( f : r denotes the binding of ﬁller f to role r ): (I:

Original: (you:0 + know:1 + now:2) – (you:0 + know:1) (2)
Revised: [Equation] see:1 + now:2) – (I: [Equation] see:1 )

Original: In (2), both sides reduce to now:2, so (1) holds for representations using left-to-right roles.
Revised: [Equation]

Original: However, if (2) instead used right-to-left roles, it would not reduce in any clean way, so (1) would not hold.
Revised: (you:0 + [Equation]

Original: [Table] LTR RTL [Table] BOW
Revised: :1 +

Original: [Table] Skip [Table]
Revised: [Equation]

Original: SST

Original: [Table] (b) Model
Revised: [Equation]

Original: LTR RTL

Original: [Table]
Revised: [Equation]

Original: InferSent 0.

Original: [Table] [Table] [Table] [Table]
Revised: In (2), both sides reduce to now:2, so (1) holds for representations using left-to-right roles.

Original: Skip-thought

Original: [Table] [Table] [Table] [Table]
Revised: However, if (2) instead used right-to-left roles, it would not reduce in any clean way, so (1) would not hold.

Original: We construct a dataset of role-diagnostic analogies, where each analogy should only hold for certain role schemes.
Revised: We construct a dataset of role-diagnostic analogies, where each analogy should only hold for certain role schemes.

Original: For example, (1) works for left-to-right roles or bag-of-words roles, but not the other role schemes.
Revised: For example, (1) works for left-to-right roles or bag-of-words roles, but not the other role schemes.

Original: The analogies use a vocabulary based on Ettinger et al.
Revised: The analogies use a vocabulary based on Ettinger et al.

Original: (2018) to ensure plausibility of the constructed sentences.
Revised: (2018) to ensure plausibility of the constructed sentences.

Original: For each analogy, we create 4 equations, one isolating each of the four terms (e.g. I see =
Revised: For each analogy, we create 4 equations, one isolating each of the four terms (e.g. I see =

Original: I see now – you know now + you know ).
Revised: I see now – you know now + you know ).

Original: We then compute the Euclidean distance between the two sides of each equation using each model’s encodings of the relevant sentences.
Revised: We then compute the Euclidean distance between the two sides of each equation using each model’s encodings.

Original: The results are in Table 1b.
Revised: The results are in Table 1b.

Original: InferSent, Skip-thought, and SPINN all show results most consistent with bidirectional roles, while SST shows results most consistent witheither tree-based or bidirectional roles.
Revised: InferSent, Skip-thought, and SPINN all show results most consistent with bidirectional roles, while SST shows results most consistent with tree-based or bidirectional roles.

Original: Notably, the bag-of-words column shows poor performance by all models, indicating that in controlled enough settings these models can be shown to have some more structured behavior even though evaluation on examples from applied tasks does not clearly bring out that structure.
Revised: The bag-of-words column shows poor performance by all models, indicating that in controlled enough settings these models can be shown to have some more structured behavior even though evaluation on examples from applied tasks does not clearly bring out that structure.

Original: These analogies thus provide independent evidence for the conclusions of the TPDN analysis: these models have a weak notion of structure built into them, but that structure is largely drowned out by the non-structure-sensitive, bag-of-words aspects of their representations.
Revised: These analogies thus provide independent evidence for our conclusions from the TPDN analysis: these models have a weak notion of structure, but that structure is largely drowned out by the non-structure-sensitive, bag-of-words aspects of their representations.

Revised: However, the other possible explanations mentioned above − namely, the possibilities that the models use alternate role schemes that we did not test or that they use some structural formalism other than tensor product representation − still remain.

Original:  W HEN DO RNN S LEARN COMPOSITIONAL REPRESENTATIONS ?
Revised:  W HEN DO RNN S LEARN COMPOSITIONAL REPRESENTATIONS ?

Original: The previous section found that all of thepopular sentence representation architectures surveyed did not robustly encode structure and could even be approximated relatively well with astructureless bag-of-words model.
Revised: The previous section suggested that all of the sentence representation architectures surveyed did not robustly encode structure and could even be approximated relatively well with a bag-of-words model.

Original: Motivated by this ﬁnding, we now investigate howdifferent aspects of training inﬂuence models’ learned representations to gain insight into howmodels can be inﬂuenced to learn better compositionality.
Revised: Motivated by this ﬁnding, we now investigate how aspects of training inﬂuence models’ learned representations to gain insight into how to encourage compositionality.

Original: To conduct this investigationin a clearer and more interpretable way, we return to the setting (from Section 2) of operating over sequences of digits.
Revised: To make this investigation clearer and more interpretable, we return to the setting (from Section 2) of operating over sequences of digits.

Original: We investigate two aspects of training: the architecture and the training task.
Revised: We investigate two aspects of training: the architecture and the training task.

Original:  apart the contribution of the encoder and decoder: In Section 2, we investigated autoencoders whose encoder and decoder had the same topology (whether unidirectional, bidirectional, or tree-based).
Revised: Teasing apart the contribution of the encoder and decoder: In Section 2, we investigated autoencoders whose encoder and decoder had the same topology (whether unidirectional, bidirectional, or tree-based).

Original: To understand the extent to which each of the two components of the network contributes to the type of representation that is learned, we now expand the investigation to include networks where the encoder and decoder differ.
Revised: To understand the extent to which each of the two components of the network contributes to the type of representation that is learned, we now expand the investigation to include networks where the encoder and decoder differ.

Revised: We crossed all three encoder types with all three decoder types (nine architectures in total).

Revised: The results of these experiments are shown in Tablein Appendix D.

Original: We ﬁnd that the decoder largely dictates what type of roles are learned: architectures with unidirectional decoders prefer mildly bidirectional roles, architectures with bidirectional decoders fail to be well-approximated by any role scheme, and architectures with tree-based decoders are best approximated by tree-based roles.
Revised: The decoder largely dictates what type of roles are learned: architectures with unidirectional decoders prefer mildly bidirectional roles, architectures with bidirectional decoders fail to be well-approximated by any role scheme, and architectures with tree-based decoders are best approximated by tree-based roles.

Original: However, there is still some effect of the encoder; most noticeably, in the tree/uni and tree/bi architectures, the treeposition roles perform better than they do for the other architectures with the same decoders.
Revised: However, there is still some effect of the encoder; most noticeably, in the tree/uni and tree/bi architectures, the tree-position roles perform better than they do for the other architectures with the same decoders.

Original: (For the exact results, see Appendix D.)

Original: Though work on novel architectures often focuses on proposing novel encoders, this ﬁnding suggests that it may be more important to focus on the decoder for getting neural networks to learn representations structured in a certain way.
Revised: Though work on novel architectures often focuses on proposing novel encoders, this ﬁnding suggests that it may be more important to focus on the decoder for getting neural networks to learn representations structured in a certain way.

Original: The contribution of the training task: We next explore how the training task affects the representations that are learned.
Revised: The contribution of the training task: We next explore how the training task affects the representations that are learned.

Original: We test four tasks, illustrated in Table 3a: autoencoding (returning the input sequence unchanged), reversal (reversing the input), sorting (returning the input digits in ascending order), and interleaving (alternating digits from the left and right edges of the input).
Revised: We test four tasks, illustrated in Table 3a: autoencoding (returning the input sequence unchanged), reversal (reversing the input), sorting (returning the input digits in ascending order), and interleaving (alternating digits from the left and right edges of the input).

Original: Table 3b gives the substitution accuracy for a TPDN trained to approximate a unidirectional encoder that was trained with a unidirectional decoder on each task.
Revised: Table 3b gives the substitution accuracy for a TPDN trained to approximate a unidirectional encoder that was trained with a unidirectional decoder on each task.

Original: We observe that training task does play a noticeable role in determining the learned representations.
Revised: Training task noticeably inﬂuences the learned representations.

Original: First, though the model has learned mildly bidirectional roles favoring the left-to-right direction for autoencoding, for reversal the rightto-left direction is far preferred over left-to-right (indeed, it is so strong that the reversal model is probably most properly said to be using right-to-left roles, not bidirectional ones).
Revised: First, though the model has learned mildly bidirectional roles favoring the left-to-right direction for autoencoding, for reversal the right-to-left direction is far preferred over left-to-right.

Original: For the interleaving task, the model is approximated best with strongly bidirectional roles: that is, bidirectional roles work nearly perfectly, while neither unidirectional scheme works well.
Revised: For the interleaving task, the model is approximated best with strongly bidirectional roles: that is, bidirectional roles work nearly perfectly, while neither unidirectional scheme works well.

Original: Finally, on the sorting task (in stark contrast to the other tasks), bag-of-words roles work nearly as well as all the other schemes, suggesting that the model has learned to discard most structural information given that the sorting task does not require structural information.
Revised: Finally, on the sorting task, bag-of-words roles work nearly as well as all the other schemes, suggesting that the model has learned to discard most structural information given that the sorting task does not require structural information.

Original: These experiments therefore show how the task can either encourage or discourage the  model from developing representations of structure.
Revised: These experiments suggest that compositional representations only arise in RNNs when the task requires them. This result offers a possible explanation for why the sentence embedding models may not robustly encode structure: perhaps the tasks on which these models are trained do not depend heavily on sentence structure (e.g. Parikh et al. (2016) achieved high accuracy on SNLI using a model that ignores word order), meaning that the models learn to ignore structural information, as was the case with the models trained on the sorting task.

Original: [Table]
Revised: [Table]

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: There are several approaches for interpreting the behavior of neural networks.
Revised: There are several approaches for interpreting the behavior of neural networks.

Original: One approach infers the information encoded in the representations from the behavior of the system on examples that measure speciﬁc dimensions of the generalization, for example in semantics (Pavlick, 2017; Dasgupta et al., 2018; Poliak et al., 2018) or syntax (Linzen et al., 2016).
Revised: One approach infers the information encoded in the representations from the behavior of the system on examples that measure speciﬁc dimensions of the generalization, for example in semantics (Pavlick, 2017; Dasgupta et al., 2018; Poliak et al., 2018) or syntax (Linzen et al., 2016).

Original: Another approach is based on probing tasks, which assess what information can be easily decoded from a vector representation (Shi et al. 2016;
Revised: Another approach is based on probing tasks, which assess what information can be easily decoded from a vector representation (Shi et al. 2016;

Original: Belinkov et al. 2017; K ´ ad ´ ar et al. 2017; Ettinger et al. 2018; cf.
Revised: Belinkov et al. 2017; K´ad´ar et al. 2017; Ettinger et al. ; cf.

Original: work in cognitive neuroscience, e.g. Norman et al. 2006).
Revised: work in cognitive neuroscience, e.g. Norman et al. 2006).

Original: Our method is wider-reaching than the probing task approach, or the Mikolov et al.
Revised: Our method is wider-reaching than the probing task approach, or the Mikolov et al.

Original: (2013) analogy approach: instead of decoding a single feature, we attempt to exhaustively decompose the vector space into a linear combination of ﬁller-role bindings.
Revised: (2013) analogy approach: instead of decoding a single feature, we attempt to exhaustively decompose the vector space into a linear combination of ﬁller-role bindings.

Original: The TPDN’s successful decomposition of sequence representations in our experiments shows that
Revised: The TPDN’s successful decomposition of sequence representations in our experiments shows that

Original: RNNs can sometimes be approximated with no nonlinearities or recurrence.
Revised: RNNs can sometimes be approximated with no nonlinearities or recurrence.

Original: This ﬁnding is related to Levy et al.
Revised: This ﬁnding is related to Levy et al.

Original: (2018), who argued that LSTMs dynamically compute weighted sums of their inputs;
Revised: (2018), who argued that LSTMs dynamically compute weighted sums of their inputs;

Original: TPRs replace the weights of the sum with the role vectors.
Revised: TPRs replace the weights of the sum with the role vectors.

Original: Levy et al.
Revised: Levy et al.

Original: (2018) also showed that recurrence is largely unnecessary for practical applications.
Revised: (2018) also showed that recurrence is largely unnecessary for practical applications.

Original: Vaswani et al.
Revised: Vaswani et al.

Original: (2017) report very good performance for a sequence model without recurrence; importantly, they ﬁnd it necessary to incorporate sequence position embeddings, which are similar to the left-to-right roles discussed in Section 2.
Revised: (2017) report very good performance for a sequence model without recurrence; importantly, they ﬁnd it necessary to incorporate sequence position embeddings, which are similar to the left-to-right roles discussed in Section 2.

Original: Methods for interpreting neural networks using more interpretable architectures have been proposed before based on rules and automata (Omlin & Giles, 1996;
Revised: Methods for interpreting neural networks using more interpretable architectures have been proposed before based on rules and automata (Omlin & Giles, 1996;

Original: Weiss et al., 2018).
Revised: Weiss et al., 2018).

Original: Finally, there is a large body of work on incorporating explicit symbolic representations into neural networks (for a recent review, see Battaglia et al. 2018); indeed, tree-shaped RNNs are an example of this approach.
Revised: Finally, there is a large body of work on incorporating explicit symbolic representations into neural networks (for a recent review, see Battaglia et al. 2018); indeed, tree-shaped RNNs are an example of this approach.

Original: While our work is orthogonal to this line of work, we note that TPRs can be proﬁtably used as an explicit component of neural networks (Palangi et al., 2018).
Revised: While our work is orthogonal to this line of work, we note that TPRs can be proﬁtably used as an explicit component of neural networks (Palangi et al., 2018).

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: What kind of internal representations could allow simple sequence-to-sequence models to perform the remarkable feats they do, including tasks previously thought to require compositional, symbolic representations (e.g., translation)?
Revised: What kind of internal representations could allow simple sequence-to-sequence models to perform the remarkable feats they do, including tasks previously thought to require compositional, symbolic representations (e.g., translation)?

Original: Perhaps, for structure-sensitive tasks, these modelsimplicitly learn  tensor-product representations (TPRs),which are distributed embeddings of symbol structures that enable powerful symbolic computation to be performed with neural operations.This seems a priori unlikely, since TPRs (and related methods) have simple, multi-linear structure, and highperforming sequence-to-sequence architectures have recurrence, non-linearities, and multiplicative gating. Yet the work reported here shows that in heavily structure-sensitive tasks, sequence-tosequence models do learn representations that are extremely well approximated by TPRs; this was demonstrated with networks performing pure sequence-manipulation tasks.
Revised: The work reported here shows that in heavily structure-sensitive tasks, sequence-to-sequence models learn representations that are extremely well approximated by tensor-product representations (TPRs), distributed embeddings of symbol structures that enable powerful symbolic computation to be performed with neural operations.

Original: This method of approximating learned internal representations via TPRs is carried out by the proposed tensor-product decomposition network (TPDN).
Revised: This was demonstrated by approximating learned internal representations via TPRs using the proposed tensor-product decomposition network (TPDN).

Original: TPDNs enable the study of how network architecture and task variations induce different types and degrees of structure-sensitivity in representations; they indicate, for example, that the decoder plays a greater role than the encoder in determining the structure of the learned representation.
Revised: Furthermore, we have demonstrated that network architecture and task variations induce different types and degrees of structure-sensitivity in representations, and that the decoder plays a greater role than the encoder in determining the structure of the learned representation.

Original: TPDNs applied to mainstream sentence-embedding models reveal that unstructured bag-of-words models provide a respectable approximation; nonetheless, this experiment does also provide evidence for a moderate degree of structure-sensitivity.
Revised: TPDNs applied to mainstream sentence-embedding models reveal that unstructured bag-of-words models provide a respectable approximation; nonetheless, this experiment does also provide evidence for a moderate degree of structure-sensitivity.

Original: The presence of structuresensitivity is corroborated by targeted analogy tests motivated by the linearity of TPRs.
Revised: The presence of structure-sensitivity is corroborated by targeted analogy tests motivated by the linearity of TPRs.

Original: Overall,
Revised: A clear limitation of the current research is that the TPDNs we used to decode network representations require a hypothesis about the nature of the representations to be selected in advance.

Original: TPDNs provide an illuminating tool for understanding structure in learned vector representations.
Revised: A fruitful future research direction would be to automatically explore hypotheses about the nature of the TPR encoded by a network.

Original: B A NALYSIS OF ARCHITECTURE COMPONENTS
Revised: B A NALYSIS OF ARCHITECTURE COMPONENTS

Original: Here we analyze how several aspects of the TPDN architecture contribute to our results.
Revised: Here we analyze how several aspects of the TPDN architecture contribute to our results.

Original: For all of the experiements described in this section, we used TPDNs to approximate a sequence-to-sequence network with a unidirectional encoder and unidirectional decoder that was trained to perform the reversal task (Section 4); we chose this network because it was strongly approximated by right-toleft roles, which are relatively simple (but still non-trivial).
Revised: For all of the experiements described in this section, we used TPDNs to approximate a sequence-to-sequence network with a unidirectional encoder and unidirectional decoder that was trained to perform the reversal task (Section 4); we chose this network because it was strongly approximated by right-toleft roles, which are relatively simple (but still non-trivial).

Original: B. I S THE FINAL LINEAR LAYER NECESSARY ?
Revised: B. I S THE FINAL LINEAR LAYER NECESSARY ?

Original: One area where our model diverges from traditional tensor product representations is in the presence of the ﬁnal linear layer (step 5 in Figure 2c).
Revised: One area where our model diverges from traditional tensor product representations is in the presence of the ﬁnal linear layer (step 5 in Figure 2c).

Original: This layer is necessary if one wishes to have freedom to choose the dimensionality of the ﬁller and role embeddings; without it, the dimensionality of the representations that are being approximated must factor exactly into the product of the dimensionality of the ﬁller embeddings and the dimensionality of the role embedding (see Figure 2c).
Revised: This layer is necessary if one wishes to have freedom to choose the dimensionality of the ﬁller and role embeddings; without it, the dimensionality of the representations that are being approximated must factor exactly into the product of the dimensionality of the ﬁller embeddings and the dimensionality of the role embedding (see Figure 2c).

Original: It is natural to wonder whether the only contribution of this layer is in adjusting the dimensionality or whether it serves a broader function.
Revised: It is natural to wonder whether the only contribution of this layer is in adjusting the dimensionality or whether it serves a broader function.

Original: Table 4 shows the results of approximating the reversal sequence-tosequence network with and without this layer; it indicates that this layer is highly necessary for the successful decomposition of learned representations.
Revised: Table 4 shows the results of approximating the reversal sequence-tosequence network with and without this layer; it indicates that this layer is highly necessary for the successful decomposition of learned representations.

Original: B. V ARYING THE DIMENSIONALITY OF THE FILLER AND ROLE EMBEDDINGS
Revised: B. V ARYING THE DIMENSIONALITY OF THE FILLER AND ROLE EMBEDDINGS

Original: Two of the parameters that must be provided to the TPDN are the dimensionality of the ﬁller embeddings and the dimensionality of the role embeddings.
Revised: Two of the parameters that must be provided to the TPDN are the dimensionality of the ﬁller embeddings and the dimensionality of the role embeddings.

Original: We explore the effects of these parameters in Figure 4.
Revised: We explore the effects of these parameters in Figure 4.

Original: We ﬁnd that, for the role embeddings, substitution accuracy increases noticeably with each increase in dimensionality until the dimensionality hits 6, where accuracy plateaus.
Revised: For the role embeddings, substitution accuracy increases noticeably with each increase in dimensionality until the dimensionality hits 6, where accuracy plateaus.

Original: This behavior is likely due to the fact that the reversal seq2seq network is most likely to employ right-to-left roles, which involves 6 possible roles in this setting.
Revised: This behavior is likely due to the fact that the reversal seq2seq network is most likely to employ right-to-left roles, which involves 6 possible roles in this setting.

Original: A dimensionality of 6 is therefore the minimum embedding size needed to make the role vectors linearly independent; linear independence is an important property for the ﬁdelity of a tensor product representation (Smolensky, 1990).
Revised: A dimensionality of 6 is therefore the minimum embedding size needed to make the role vectors linearly independent; linear independence is an important property for the ﬁdelity of a tensor product representation (Smolensky, 1990).

Original: The accuracy also generally increases as ﬁller dimensionality increases, but there is a less clear point where it plateaus for the ﬁllers than for the roles.
Revised: The accuracy also generally increases as ﬁller dimensionality increases, but there is a less clear point where it plateaus for the ﬁllers than for the roles.

Original: B. F ILLER - ROLE BINDING OPERATION
Revised: B. F ILLER - ROLE BINDING OPERATION

Original: The body of the paper focused on using the tensor product ( f i ⊗ r
Revised: The body of the paper focused on using the tensor product ( f i ⊗ r

Original: i
Revised: i

Original: , see Figure 2b) as the operation for binding ﬁllers to roles.
Revised: , see Figure 2b) as the operation for binding ﬁllers to roles.

Original: There are other conceivable binding operations.
Revised: There are other conceivable binding operations.

Original: Here we test two alternatives, both of which can be viewed as special cases of the tensor product or as related to it: circular convolution, which is used in holographic reduced representations (Plate, 1995), and elementwise product ( f i (cid:12) r i ).
Revised: Here we test two alternatives, both of which can be viewed as special cases of the tensor product or as related to it: circular convolution, which is used in holographic reduced representations (Plate, 1995), and elementwise product ( f i (cid:12) r i ).

Original: Both of these are restricted such that roles and ﬁllers must have the same embedding dimension ( N f = N r ).
Revised: Both of these are restricted such that roles and ﬁllers must have the same embedding dimension ( N f = N r ).

Original: We ﬁrst try setting this dimension to 20, which is what was used as both the role and ﬁller dimension in all tensor product experiments with digit sequences.
Revised: We ﬁrst try setting this dimension to 20, which is what was used as both the role and ﬁller dimension in all tensor product experiments with digit sequences.

Original: We found that while these dimensions were effective for the tensor product binding operation, they were not effective for elementwise product and circular convolution (Table 5).
Revised: We found that while these dimensions were effective for the tensor product binding operation, they were not effective for elementwise product and circular convolution (Table 5).

Original: When the dimension was increased to 60, however, the elementwise product performed roughly as well as as the tensor product; circular convolution now learned one of the two viable role schemes (right-to-left roles) but failed to learn the equally viable bidirectional role scheme.
Revised: When the dimension was increased to 60, however, the elementwise product performed roughly as well as as the tensor product; circular convolution now learned one of the two viable role schemes (right-to-left roles) but failed to learn the equally viable bidirectional role scheme.

Original: Thus, our preliminary experiments suggest that these other two binding operations do show promise, but seem to require larger embedding dimensions than tensor products do.
Revised: Thus, our preliminary experiments suggest that these other two binding operations do show promise, but seem to require larger embedding dimensions than tensor products do.

Original: At the same time, they still have fewer parameters overall compared to the tensor product because their ﬁnal linear layers (of dimensionality N ) are much smaller than those used with a tensor product (of dimensionality N 2 ).
Revised: At the same time, they still have fewer parameters overall compared to the tensor product because their ﬁnal linear layers (of dimensionality N ) are much smaller than those used with a tensor product (of dimensionality N 2 ).

Original: C T HE DIGIT PARSING ALGORITHM
Revised: C T HE DIGIT PARSING ALGORITHM

Original: When inputting digit sequences to our tree-based model, the model requires a predeﬁned tree structure for the digit sequence.
Revised: When inputting digit sequences to our tree-based model, the model requires a predeﬁned tree structure for the digit sequence.

Original: We use the following algorithm to generate this tree structure: at each timestep, combine the smallest element of the sequence (other than the last element) with its neighbor immediately to the right, and replace the pair with that neighbor.
Revised: We use the following algorithm to generate this tree structure: at each timestep, combine the smallest element of the sequence (other than the last element) with its neighbor immediately to the right, and replace the pair with that neighbor.

Original: If there is a tie for the smallest digit, choose the leftmost tied digit.
Revised: If there is a tie for the smallest digit, choose the leftmost tied digit.

Original: For example, the following shows step-by-step how the tree for the sequence 523719 would be generated:
Revised: For example, the following shows step-by-step how the tree for the sequence 523719 would be generated:

Original: D F ULL RESULTS OF SEQUENCE - TO - SEQUENCE EXPERIMENTS
Revised: D F ULL RESULTS OF SEQUENCE - TO - SEQUENCE EXPERIMENTS

Original: Section 4 summarized the results of our experiments which factorially varied the training task, the encoder and the decoder.
Revised: Section 4 summarized the results of our experiments which factorially varied the training task, the encoder and the decoder.

Original: Here we report the full results of these experiments in two tables: Tableshows the accuracies achieved by the sequence-to-sequence models at the various training tasks, and
Revised: Here we report the full results of these experiments in two tables: Tableshows the accuracies achieved by the sequence-to-sequence models at the various training tasks, and

Original: Table 7 shows the substitution accuracies of TPDNs applied to the trained sequence-to-sequence models for all architectures and tasks.
Revised: Table 7 shows the substitution accuracies of TPDNs applied to the trained sequence-to-sequence models for all architectures and tasks.

Original: E M ODEL AND TRAINING DETAILS E. S EQUENCE - TO - SEQUENCE MODELS
Revised: E M ODEL AND TRAINING DETAILS E. S EQUENCE - TO - SEQUENCE MODELS

Original: As much as possible, we standardized parameters across all sequence-to-sequence models that we trained on the digit-sequence tasks.
Revised: As much as possible, we standardized parameters across all sequence-to-sequence models that we trained on the digit-sequence tasks.

Original: For all decoders, when computing a new hidden state, the only input to the recurrent unit is the previous hidden state (or parent hidden state, for a tree-based decoder), without using any previous outputs as inputs to the hidden state update.
Revised: For all decoders, when computing a new hidden state, the only input to the recurrent unit is the previous hidden state (or parent hidden state, for a tree-based decoder), without using any previous outputs as inputs to the hidden state update.

Original: This property is necessary for using a bidirectional decoder, since it would not be possible to generate the output both before and after each bidirectional decoder hidden state.
Revised: This property is necessary for using a bidirectional decoder, since it would not be possible to generate the output both before and after each bidirectional decoder hidden state.

Original: We also inform the decoder of when to stop decoding; that is, for sequential models, the decoder stops once its output is the length of the sequence, while for tree-based models we tell the model which positions in the tree are leaves.
Revised: We also inform the decoder of when to stop decoding; that is, for sequential models, the decoder stops once its output is the length of the sequence, while for tree-based models we tell the model which positions in the tree are leaves.

Original: Stopping could alternately be determined by some action of the decoder (e.g., generating an end-of-sequence symbol); for simplicity we chose the strategy outlined above instead.
Revised: Stopping could alternately be determined by some action of the decoder (e.g., generating an end-of-sequence symbol); for simplicity we chose the strategy outlined above instead.

Original: For all architectures, we used a digit embedding dimensionality of 10 (chosen arbitrarily) and a hidden layer size of 60 (this hidden layer size was chosen because 60 has many integer factors, making it amenable to the dimensionality analyses in Appendix B.2).
Revised: For all architectures, we used a digit embedding dimensionality of 10 (chosen arbitrarily) and a hidden layer size of 60 (this hidden layer size was chosen because 60 has many integer factors, making it amenable to the dimensionality analyses in Appendix B.2).

Original: For the bidirectional architectures, the forward and backward recurrent layers each had a hidden layer size of 30, so that their concatenated hidden layer size was 60.
Revised: For the bidirectional architectures, the forward and backward recurrent layers each had a hidden layer size of 30, so that their concatenated hidden layer size was 60.

Original: For bidirectional decoders, a linear layer condensed the 60-dimensional encoding into 30 dimensions before it was passed to the forward and backward decoders.
Revised: For bidirectional decoders, a linear layer condensed the 60-dimensional encoding into 30 dimensions before it was passed to the forward and backward decoders.

Original: The networks were trained using the Adam optimizer (Kingma & Ba, 2015) with the standard initial learning rate of 0.001.
Revised: The networks were trained using the Adam optimizer (Kingma & Ba, 2015) with the standard initial learning rate of 0.001.

Original: We used negative log likelihood, computed over the softmax probability distributions for each output sequence element, as the loss function.
Revised: We used negative log likelihood, computed over the softmax probability distributions for each output sequence element, as the loss function.

Original: Training proceeded with a batch size of 32, with loss on the held out development set computed after every 1,000 training examples.
Revised: Training proceeded with a batch size of 32, with loss on the held out development set computed after every 1,000 training examples.

Original: Training was halted when the loss on the heldout development set had not improved for any of the development loss checkpoints for a full epoch of training (i.e. 40,000 training examples).
Revised: Training was halted when the loss on the heldout development set had not improved for any of the development loss checkpoints for a full epoch of training (i.e. 40,000 training examples).

Original: Once training completed, the parameters from the best-performing checkpoint were reloaded and used for evaluation of the network.
Revised: Once training completed, the parameters from the best-performing checkpoint were reloaded and used for evaluation of the network.

Original: E. TPDN S TRAINED ON DIGIT MODELS
Revised: E. TPDN S TRAINED ON DIGIT MODELS

Original: When applying TPDNs to the digit-based sequence-to-sequence models, we always used 20 as both the ﬁller embedding dimensionality and role embedding dimensionality.
Revised: When applying TPDNs to the digit-based sequence-to-sequence models, we always used 20 as both the ﬁller embedding dimensionality and role embedding dimensionality.

Original: This decision was based on the experiments in Appendix B.2; we selected ﬁller and role embedding dimensionalities that were safely above the cutoff needed to lead to successful decomposition.
Revised: This decision was based on the experiments in Appendix B.2; we selected ﬁller and role embedding dimensionalities that were safely above the cutoff needed to lead to successful decomposition.

Original: The TPDNs were trained with the same training regimen as the sequence-to-sequence models, except that, instead of using negative log likelihood as the loss function, for the TPDNs we used mean squared error between the predicted vector representation and the actual vector representation from the original sequence-to-sequence network.
Revised: The TPDNs were trained with the same training regimen as the sequence-to-sequence models, except that, instead of using negative log likelihood as the loss function, for the TPDNs we used mean squared error between the predicted vector representation and the actual vector representation from the original sequence-to-sequence network.

Revised: The TPDNs were given the sequences of ﬁllers (i.e. the digits), the roles hypothesized to go with those ﬁllers, the sequence embeddings produced by the RNN, and the dimensionalities of the ﬁller embeddings, role embeddings, and ﬁnal linear transformation.

Revised: The parameters that were updated by training were the speciﬁc values for the ﬁller embeddings, the role embeddings, and the ﬁnal linear transformation.

Original: E. S ENTENCE EMBEDDING MODELS
Revised: E. S ENTENCE EMBEDDING MODELS

Original: For all four sentence encoding models, we used publicly available and freely downloadable pretrained versions found at the following links:
Revised: For all four sentence encoding models, we used publicly available and freely downloadable pretrained versions found at the following links:

Original: • InferSent: https://github.com/facebookresearch/InferSent • Skip-thought: https://github.com/ryankiros/skip-thoughts • SST: https://nlp.stanford.edu/software/corenlp.shtml • SPINN: https://github.com/stanfordnlp/spinn
Revised: • InferSent: https://github.com/facebookresearch/InferSent • Skip-thought: https://github.com/ryankiros/skip-thoughts • SST: https://nlp.stanford.edu/software/corenlp.shtml • SPINN: https://github.com/stanfordnlp/spinn

Original: InferSent is a bidirectional LSTM with 4096-dimensional hidden states.
Revised: InferSent is a bidirectional LSTM with 4096-dimensional hidden states.

Original: For Skip-thought, we use the unidirectional variant, which is an LSTM with 2400-dimensional hidden states.
Revised: For Skip-thought, we use the unidirectional variant, which is an LSTM with 2400-dimensional hidden states.

Original: The SST model is a recurrent neural tensor network (RNTN) with 25-dimensional hidden states.
Revised: The SST model is a recurrent neural tensor network (RNTN) with 25-dimensional hidden states.

Original: Finally, for SPINN, we use the SPINN-PI-NT version, which is equivalent to a tree-LSTM (Tai et al., 2015) with 300dimensional hidden states.
Revised: Finally, for SPINN, we use the SPINN-PI-NT version, which is equivalent to a tree-LSTM (Tai et al., 2015) with 300dimensional hidden states.

Original: E. TPDN S TRAINED ON SENTENCE MODELS
Revised: E. TPDN S TRAINED ON SENTENCE MODELS

Original: For training a TPDN to approximate the sentence encoding models, the ﬁller embedding dimensions were dictated by the size of the pretrained word embeddings; these dimensions were 300 for
Revised: For training a TPDN to approximate the sentence encoding models, the ﬁller embedding dimensions were dictated by the size of the pretrained word embeddings; these dimensions were 300 for

Original: InferSent and SPINN, 620 for Skip-thought, and 25 for SST.
Revised: InferSent and SPINN, 620 for Skip-thought, and 25 for SST.

Original: The linear transformation applied to the word embeddings did not change their size.
Revised: The linear transformation applied to the word embeddings did not change their size.

Original: For role embedding dimensionality we tested all role dimensions in { 1 , 5 , 10 , 20 , 40 , 60 } .
Revised: For role embedding dimensionality we tested all role dimensions in { 1 , 5 , 10 , 20 , 40 , 60 } .

Original: The best-performing dimension was chosen based on preliminary experiments and used for all subsequent experiments; we thereby chose role dimensionalities of 10 for InferSent and Skip-thought, 20 for SST, and 5 for SPINN.
Revised: The best-performing dimension was chosen based on preliminary experiments and used for all subsequent experiments; we thereby chose role dimensionalities of 10 for InferSent and Skip-thought, 20 for SST, and 5 for SPINN.

Original: In general, role embedding dimensionalities of 5, 10, and 20 all performed noticeably better than 1, 40, and 60, but there was not much difference between 5, 10, and 20.
Revised: In general, role embedding dimensionalities of 5, 10, and 20 all performed noticeably better than 1, 40, and 60, but there was not much difference between 5, 10, and 20.

Original: The training regimen for the TPDNs on sentence models was the same as for the TPDNs trained on digit sequences.
Revised: The training regimen for the TPDNs on sentence models was the same as for the TPDNs trained on digit sequences.

Revised: The TPDNs were given the sequences of ﬁllers (i.e. the words), the roles hypothesized to go with those ﬁllers, the sequence embeddings produced by the RNN, the initial pretrained word embeddings, the dimensionalities of the lienarly-transformed ﬁller embeddings, the role embeddings, and the ﬁnal linear transformation.

Revised: The parameters that were updated by training were the speciﬁc values for the role embeddings, the linear transformation that was applied to the pretrained word embeddings, and the ﬁnal linear transformation.

Revised: The sentences whose encodings we trained the TPDNs to approximate were the premise sentences from the SNLI corpus (Bowman et al., 2015).

Original:  F D OWNSTREAM TASK PERFORMANCE FOR TPDN S APPROXIMATING SENTENCE ENCODERS
Revised: We also tried instead using the sentences in the WikiText-2 corpus (Merity et al., 2016) but found better performance with the SNLI sentences. This is plausibly because the shorter, simpler sentences in the SNLI corpus made it easier for the model to learn the role embeddings without distraction from the ﬁllers. F D OWNSTREAM TASK PERFORMANCE FOR TPDN S APPROXIMATING SENTENCE ENCODERS

Original: For each TPDN trained to approximate a sentence encoder, we evaluate it on four downstream tasks: (i) Stanford Sentiment Treebank (SST), which is labeling the sentiment of movie reviews (Socher et al., 2013); this task is further subdivided into SST2 (labeling the reviews as positive or negative ) and SST5 (labeling the reviews on a 5-point scale, where 1 means very negative and 5 means very positive ).
Revised: For each TPDN trained to approximate a sentence encoder, we evaluate it on four downstream tasks: (i) Stanford Sentiment Treebank (SST), which is labeling the sentiment of movie reviews (Socher et al., 2013); this task is further subdivided into SST2 (labeling the reviews as positive or negative ) and SST5 (labeling the reviews on a 5-point scale, where 1 means very negative and 5 means very positive ).

Original: The metric we report for both tasks is accuracy.
Revised: The metric we report for both tasks is accuracy.

Original: (ii) Microsoft Research
Revised: (ii) Microsoft Research

Original: Paraphrase Corpus (MRPC), which is labeling whether two sentences are paraphrases of each other (Dolan et al., 2004).
Revised: Paraphrase Corpus (MRPC), which is labeling whether two sentences are paraphrases of each other (Dolan et al., 2004).

Original: For this task, we report both accuracy and F1.
Revised: For this task, we report both accuracy and F1.

Original: (iii) Semantic Textual Similarity
Revised: (iii) Semantic Textual Similarity

Original: Benchmark (STS-B), which is giving a pair of sentences a score on a scale from 0 to 5 indicating how similar the two sentences are (Cer et al., 2017).
Revised: Benchmark (STS-B), which is giving a pair of sentences a score on a scale from 0 to 5 indicating how similar the two sentences are (Cer et al., 2017).

Original: For this task, we report Pearson and Spearman correlation coefﬁcients.
Revised: For this task, we report Pearson and Spearman correlation coefﬁcients.

Original: (iv) Stanford Natural Language Inference (SNLI), which involves labeling two sentences to indicate whether the ﬁrst entails the second, contradicts the second, or neither (Bowman et al., 2015).
Revised: (iv) Stanford Natural Language Inference (SNLI), which involves labeling two sentences to indicate whether the ﬁrst entails the second, contradicts the second, or neither (Bowman et al., 2015).

Original: For this task, we report accuracy as the evaluation metric.
Revised: For this task, we report accuracy as the evaluation metric.

Original: F. S UBSTITUTION PERFORMANCE
Revised: F. S UBSTITUTION PERFORMANCE

Original: The ﬁrst results we report for the TPDN approximations of sentence encoders is similar to the substitution accuracy used for digit encoders.
Revised: The ﬁrst results we report for the TPDN approximations of sentence encoders is similar to the substitution accuracy used for digit encoders.

Original: Here, we use SentEval (Conneau & Kiela, 2018) to train linear classiﬁers for all downstream tasks on the original sentence encoding model; then, we freeze the weights of these classiﬁers and use them to classify the test-set encodings generated by the TPDN approximation.
Revised: Here, we use SentEval (Conneau & Kiela, 2018) to train linear classiﬁers for all downstream tasks on the original sentence encoding model; then, we freeze the weights of these classiﬁers and use them to classify the test-set encodings generated by the TPDN approximation.

Original: We use the classiﬁer parameters recommended by the SentEval authors: using a linear classiﬁer (not a multi-layer perceptron) trained with the Adam algorithm (Kingma &
Revised: We use the classiﬁer parameters recommended by the SentEval authors: using a linear classiﬁer (not a multi-layer perceptron) trained with the Adam algorithm (Kingma &

Original: Ba, 2015) using a batch size of 64, a tenacity of 5, and an epoch size of 4.
Revised: Ba, 2015) using a batch size of 64, a tenacity of 5, and an epoch size of 4.

Original: The results are shown in
Revised: The results are shown in

Original: Table 8.
Revised: Table 8.

Original: F. A GREEMENT BETWEEN THE TPDN AND THE ORIGINAL MODEL
Revised: F. A GREEMENT BETWEEN THE TPDN AND THE ORIGINAL MODEL

Original: Next, we analyze the same results from the previous section, but instead of reporting accuracies we report the extent to which the TPDN’s predictions agree with the original model’s predictions.
Revised: Next, we analyze the same results from the previous section, but instead of reporting accuracies we report the extent to which the TPDN’s predictions agree with the original model’s predictions.

Original: For
Revised: For

Original: SST, MRPC, and SNLI, this agreement is deﬁned as the proportion of their labels that are the same.
Revised: SST, MRPC, and SNLI, this agreement is deﬁned as the proportion of their labels that are the same.

Original: For STS-B, the agreement is the Pearson correlation between the original model’s outputs and the
Revised: For STS-B, the agreement is the Pearson correlation between the original model’s outputs and the

Original: TPDN’s outputs.
Revised: TPDN’s outputs.

Original: The results are in Table 9.
Revised: The results are in Table 9.

Original: F. T RAINING A CLASSIFIER ON THE TPDN
Revised: F. T RAINING A CLASSIFIER ON THE TPDN

Original: Finally, we consider treating the TPDNs as models in their own right and use SentEval to both train and test downstream task classiﬁers on the TPDNs.
Revised: Finally, we consider treating the TPDNs as models in their own right and use SentEval to both train and test downstream task classiﬁers on the TPDNs.

Original: The results are in Table 10.
Revised: The results are in Table 10.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: to
Revised: to

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: LTR
Revised: LTR

Original: [Table]
Revised: [Table]

Original: Bi
Revised: Bi

Original: Tree
Revised: Tree

Original: BOW
Revised: BOW

Original: Original
Revised: Original

Original: [Table]
Revised: [Table]

Original: SST 0.
Revised: SST 0.

Original: [Table]
Revised: [Table]

Original: MRPC
Revised: MRPC

Original: [Table]
Revised: [Table]

Original: MRPC
Revised: MRPC

Original: [Table]
Revised: [Table]

Original: STS
Revised: STS

Original: [Table]
Revised: [Table]

Original: Pearson) 0.
Revised: Pearson) 0.

Original: STS
Revised: STS

Original: [Table]
Revised: [Table]

Original: Spearman) 0.
Revised: Spearman) 0.

Original: SNLI 0.
Revised: SNLI 0.

Original: Skip
Revised: Skip

Original: [Table] [Table]
Revised: [Table] [Table]

Original: MRPC
Revised: MRPC

Original: [Table]
Revised: [Table]

Original: accuracy) 0.
Revised: accuracy) 0.

Original: MRPC
Revised: MRPC

Original: [Table]
Revised: [Table]

Original: F1) 0.
Revised: F1) 0.

Original: STS
Revised: STS

Original: [Table]
Revised: [Table]

Original: STS
Revised: STS

Original: [Table]
Revised: [Table]

Original: SNLI 0.
Revised: SNLI 0.

Original: SST
Revised: SST

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: LTR
Revised: LTR

Original: RTL
Revised: RTL

Original: Bi
Revised: Bi

Original: Tree
Revised: Tree

Original: BOW
Revised: BOW

Original: Original
Revised: Original

Original: Infersent
Revised: Infersent

Original: SST 0. SST 0.
Revised: SST 0. SST 0.

Original: MRPC 0.
Revised: MRPC 0.

Original: STS-B 0.
Revised: STS-B 0.

Original: SNLI 0.
Revised: SNLI 0.

Original: Skip-thought
Revised: Skip-thought

Original: SST 0. SST 0.
Revised: SST 0. SST 0.

Original: MRPC 0.
Revised: MRPC 0.

Original: STS-B -0.
Revised: STS-B -0.

Original: SNLI 0.
Revised: SNLI 0.

Original: SST
Revised: SST

Original: SST 0. SST 0.
Revised: SST 0. SST 0.

Original: MRPC 0.
Revised: MRPC 0.

Original: STS-B 0.
Revised: STS-B 0.

Original: SNLI 0.
Revised: SNLI 0.

Original: SPINN
Revised: SPINN

Original: SST 0.
Revised: SST 0.

Original: SST 0.
Revised: SST 0.

Original: [Table] [Table]
Revised: [Table] [Table]