Sequence-to-Sequence Networks Learn the Meaning of Reflexive Anaphora (2020)

Abstract: Reflexive anaphora present a challenge for semantic interpretation: their meaning varies depending on context in a way that appears to require abstract variables. Past work has raised doubts about the ability of recurrent networks to meet this challenge. In this paper, we explore this question in the context of a fragment of English that incorporates the relevant sort of contextual variability. We consider sequence-to-sequence architectures with recurrent units and show that such networks are capable of learning semantic interpretations for reflexive anaphora which generalize to novel antecedents. We explore the effect of attention mechanisms and different recurrent unit types on the type of training data that is needed for success as measured in two ways: how much lexical support is needed to induce an abstract reflexive meaning (i.e., how many distinct reflexive antecedents must occur during training) and what contexts must a noun phrase occur in to support generalization of reflexive interpretation to this noun phrase?

## introduction recurrent neural network architectures have demonstrated remarkable success in natural language processing, achieving state of the art performance across an impressive range of tasks ranging from machine translation to semantic parsing to question answering @xcite @xcite . these tasks demand the use of a wide variety of computational processes and information sources (from grammatical to lexical to world knowledge), and are evaluated in coarse-grained quantitative ways. as a result, it is not an easy matter to identify the specific strengths and weaknesses in a network's solution of a task. in this paper, we take a different tack, exploring the degree to which neural networks successfully master one very specific aspect of linguistic knowledge: the interpretation of sentences containing reflexive anaphora. we address this problem in the context of the task of semantic parsing, which we instantiate as mapping a sequence of words into a predicate calculus logical form representation of the sentence's meaning. marcus (1998) argues that this kind of abstraction, which he takes to require the use of algebraic variables to assert identity, is beyond the capacity of recurrent neural networks. marcus's demonstration involves a simple recurrent network (srn, elman 1990) language model that is trained to predict the next word over a corpus of sentences of the following form: all sentences in this training set have identical subject and object nouns. marcus shows, however, that the resulting trained network does not correctly predict the subject noun when tested with a novel preamble 'a book is a . . .'. though intriguing, this demonstration is not entirely convincing: since the noun occurring in the novel preamble, book in our example, did not occur in the training data, there is no way that the network could possibly have known which (one-hot represented) output should correspond to the reflexive for a sentence containing the novel (one-hot represented) subject noun, even if the network did successfully encode an identity relation between subject and object. @xcite explore a related task in the context of srn interpretation of reflexives. in their experiments, srns were trained to map input words to corresponding semantic symbols that are output on the same time step in which a word is presented. for most words in the vocabulary, this is a simple task: the desired output is a constant function of the input (mary corresponds to mary, sees to see, etc.). for reflexives however, the target output depends on the subject that occurs earlier in the sentence. frank et al. tested the network's ability to interpret a reflexive in sentences containing a subject that had not occurred as a reflexive's antecedent during training. however, unlike marcus' task, this subject and its corresponding semantic symbol did occur in other (non-reflexive) contexts in the training data, and therefore was in the realm of possible inputs and outputs for the network. nonetheless, none of the srns that they trained succeeded at this task for even a single test example. since those experiments were conducted, substantial advances have been made on recurrent neural network architectures, some of which have been crucial in the success of practical nlp systems. these innovations open up the possibility that modern network architectures may well be able to solve the variable identity problem necessary for mapping reflexive sentences to their logical form. in the experiments we describe below, we explore whether this is the case. ## experimental setup our experiments take the form of a semantic parsing task, where sequences of words are mapped into logical form representations of meaning. following dong and lapata (2016), we do this by means of a sequence-to-sequence architecture @xcite in which the input sentence is fully processed by an encoder network before it is decoded into a sequence of symbols in the target domain (cf. @xcite @xcite . this approach removes the need to synchronize the production of output symbols with the input words, as in @xcite , allowing greater flexibility in the nature of semantic representations. the sequence-to-sequence architecture is agnostic as to the types of recurrent units for the encoding and decoding phases of the computation, and whether the decoder makes use of an attention mechanism. here, we explore the effects of using different types of recurrent units and including attention or not. specifically, we examine the performance and training characteristics of sequence-to-sequence models based on srns, grus, and lstms with and without multiplicative attention @xcite . in all experiments, we perform 5 runs with different random seeds for each combination of recurrent unit type (one layer of srn, lstm or gru units for both the encoder and decoder) and attention (with or without multiplicative attention). all models used hidden and embedding of size of 256. training was done using stochastic gradient descent with learning rate of 0.01. models were trained for a maximum of 100 epochs with early stopping when validation loss fails to decrease by 0.005 over three successive epochs. we conduct all of our experiments with synthetic datasets from a small fragment of english sentences generated using a simple context-free grammar. this fragment includes simple sentences with transitive and intransitive verbs. subjects are always proper names and objects are either proper names or a reflexive whose gender matches that of the subject. our vocabulary includes 8 intransitive verbs, 7 transitive verbs, 15 female names, and 11 male names. the grammar thus generates 5,122 distinct sentences. all sentences are generated with equal probability, subject to the restrictions imposed by each experiment. we use a unification extension to cfg to associate each sentence with a predicate calculus interpretation. the symbols corresponding to the predicates and the entities in our logical language are identical with the verbs and names used by our grammar, yielding representations like those shown in (1) and ( 2 ). the output sequences corresponding to the target semantic interpretations include parentheses and commas as separate symbols. quite clearly, this dataset does not reproduce the richness of english sentence structure or the distribution of reflexive anaphora, and we leave the exploration of syntactically richer domains for future work. however, even this simple fragment instantiate the kind of contextual variable interpretation found in all cases of reflexive interpretation and therefore it allows us to probe the ability of networks to induce a representation of such meanings. as discussed in the previous section, we are interested in whether sequence-to-sequence models can successfully generalize their knowledge of the interpretation of sentences containing reflexives to ones having novel antecedents. to do this, we employ a poverty of the stimulus paradigm that tests for systematic generalization beyond a finite (and ambiguous) set of training data @xcite . in our experiments, we remove certain classes of examples from the training data set and test the effect on the network's success in interpreting reflexive-containing sentences. each of our experiments thus defines a set of sentences that are withheld during training. the non-withheld sentences are randomly split 80%-10%-10% between training, validation, and testing sets. accuracy for each set is computed on a sentence-level basis, i.e., an accurate output requires that all symbols generated by the model be identical to the target. our experiments focus on two sorts of manipulations of the training data: (1) varying the number of lexical items that do and do not occur as the antecedents of reflexives in the training set, and (2) varying the syntactic positions in which the non-antecedent names occur. as we will see, both of these manipulations substantially impact the success of reflexive generalization in ways that vary across network types. in the first experiment, we directly test whether or not networks can generalize knowledge of how to interpret herself to a new antecedent. we withhold all examples whose input sequence includes the reflexive herself bound by the single antecedent alice, of the form shown in (4). ## experiment 2: doesn't alice know alice? while the networks in experiment 1 are not trained on sentences of the form shown in (4), they are trained on sentences that have the same target semantic form, namely sentences in which alice occur as both subject and object of a transitive verb. ## experiment 4: what if alice doesn't know anyone? the experiments we have described thus far removed from the training data input sentences and logical forms that were exactly identical to those associated with reflexive sentences. the next pair of experiments increases the difficulty of the generalization task still further, by withholding from the experiment 2 training data all sentences containing the withheld reflexive antecedent, alice, in a wider range of grammatical contexts, and testing the effect that this has on the network's ability to interpret alicereflexive sentences. experiment 4a starts by withholding sentences where alice appears as the subject of a transitive verb (including those with reflexive objects, which we already removed in earlier experiments). this manipulation tests the degree to which the presence of alice as a subject more generally is crucial to the network's generalization of reflexive sentences to a novel name. we also run a variation of this experiment (experiment 4b) in which sentences containing alice as the subject of intransitives are also removed, i.e., sentences of the following form: if subjecthood is represented in a uniform manner across transitive and intransitive sentences, the absence of such sentences from the training data might further impair the network's ability to generalize to reflexive sentences. ## experiment 5: what if nobody knows alice? in the final experiment, we restrict the grammatical context in which alice appears by removing from the training data of experiment 2 all instances of transitive sentences with alice in object position (but it is retained in subject position, apart from reflexive sentences). in a second variant (experiment 5b), we further restrict the training data to exclude all intransitive sentences with alice subjects. although english, as a language with nominative-accusative alignment, treats subjects of intransitives in a grammatically parallel fashion to subjects of transitives, other languages (with ergative-absolutive alignment) treat intransitive subjects like transitive objects. though the word order of our synthetic language suggests nominative-accusative alignment, intransitive subjects have in common with transitive objects being the final argument in the logical form, which might lead to them being treated in similar fashion. ## conclusions because of their abstract meaning, reflexive anaphora present a distinctive challenge for semantic parsing that had been thought to be beyond the capabilities of recurrent networks. the experiments described here demonstrate that this was incorrect. sequence-to-sequence networks with a range of recurrent unit types are in fact capable of learning an interpretation of reflexive pronouns that generalizes to novel antecedents. our results also show that such generalization is nonetheless contingent on the appearance of the held-out antecedent in a variety of syntactic positions as well as the diversity of antecedents providing support for the reflexive generalization. additionally successful generalization depends on the network architecture in ways that we do not fully understand. it is at present unknown whether the demands that any of these architecture impose on the learning environment for successful learning of reflexives are consistent with what children experience, but this could be explored with both corpus and experimental work. future work will also be necessary to elucidate the nature of the networks' representations of reflexive interpretation and to understand how they support lexical generalization (or not). the question we have explored here is related to, but distinct from, the issue of systematicity @xcite , according to which pieces of representations learned in distinct contexts can freely recombine. this issue has been addressed using sequence-to-sequence architectures in recent work with the synthetic scan robot command interpretation dataset (lake and baroni, 2018) and on language modeling @xcite , in both cases with limited success. one aspect of the scan domain that is particularly relevant to reflexive interpretation is commands involving adverbial modifiers such as twice. commands like jump twice must be interpreted by duplicating the meaning of the verb, i.e., as jump jump, which is similar to what we require for the interpretation of the reflexive object, though in a way that does not require sensitivity to syntactic structure that we have not explored here. recently, lake (2019), @xcite and @xcite have proposed novel architectures that increase systematic behavior, and we look forward to exploring the degree to which these impact performance on reflexive interpretation. our current work has focused exclusively on recurrent networks, ranging from srns to grus and lstms. recent work by @xcite shows that transformer networks attain superior performance on a variety of sequence-to-sequence tasks while dispensing with recurrent units altogether. examining both the performance and training characteristics of transformers will allow us to compare the effects of attention and recurrence on the anaphora interpretation task. this is especially interesting given the impact that attention had on performance in our experiments. finally, while our current experiments are revealing about the capacity of recurrent networks to learn generalizations about context-sensitive interpretation, there are nonetheless limited in a number of respects because of simplifications in the english fragment we use to create our synthetic data. reflexives famously impose a structural requirement on their antecedents (c-command). in the following example, the reflexive's antecedent must be student and cannot be teacher. we do not know whether the architectures that have succeed on our experiments would do similarly well if the relevant generalization required reference to (implicit) structure. past work has explored the sensitivity of recurrent networks to hierarchical structure, with mixed results @xcite . in ongoing work, we are exploring this question by studying more complex synthetic domains both with the kind of recurrent sequence-to-sequence network used here as well networks that explicitly encode or decode sentences in a hierarchical manner. a second simplification concerns the distribution of reflexives themselves. english reflexives can appear in a broader range of syntactic environments apart from transitive objects @xcite . it would be of considerable interest to explore the reflexive interpretation in a naturalistic setting that incorporate this broader set of distributions.