SlowGuess/ACL_2020_OCR huggingface dataset:
Figure 1: Proportion HIGH vs LOW attachment in English. Human results from the original Fernández (2003) experiment and RNN LM results from ExtENDED DATA (derived from Fernández (2003) and Carreiras and Clifton Jr (1993)).
English. The second hypothesis is perhaps weakened by the results of Section 4, where both attachment types were learnable despite any recency bias. However, we know that other syntactic attachment biases can influence RC attachment in humans (Scheepers, 2003). It could be that other kinds of attachment (such as prepositional phrase attachment) have varying proportions of attachment biases in the training data. Perhaps conflicting attachment biases across multiple constructions force the model to resort to the use of a 'default' recency bias in cases of ambiguity.
6 Syntactically blocking low attachment
6.1 Stimuli
To determine whether the behavior of the RNNs is driven by a learned attachment preference or a strong recency bias, we created stimuli using the stimulus template described in Section 3.1 (e.g., (3)). All of these stimuli had only the higher nominal syntactically available for attachment; the lower nominal was blocked by the addition of a relative clause:
(7) a. Everybody ignored the boy that the girls hated that was boring.
b. *Everybody ignored the boys that the girl hated that was boring.
In (7) only (7-a) is grammatical. This follows because  $boy(s)$  is the only nominal available for mod
<sup>11</sup>As before, some of these stimuli are infelicitous. We do not concern ourselves with this distinction in the present work, given the results in Gulordava et al. (2018).
Figure 2: Proportion HIGH vs LOW attachment with syntactically unavailable lower nominal. Human results estimated from Linzen and Leonard (2018) and RNN LM results from the EXTENDED DATA (derived from Fernandez (2003) and Carreiras and Clifton Jr (1993)) with the lower nominal blocked.
ification. In (7-a), the RC verb was agrees in number with this nominal, while in (7-b), was agrees in number with the now blocked lower nominal girl rather than with boys. For all such sentence pairs, we calculated the difference in surprisal between (7-a) and (7-b). If their behavior is driven by a legitimate syntactic attachment preference, the models should exhibit an overwhelming HIGH bias (i.e. the mean difference should be less than zero).
As before, the differences in surprisal were calculated for each pair of experimental items. If the difference was greater than zero, the attachment was coded as LOW. The results categorically coded for HIGH/LOW attachment are given in Figure 2, including the results expected for humans given the pattern in Linzen and Leonard (2018). A two-tailed t-test was conducted to see if the mean difference in surprisal differed from zero. The results were statistically significant  $(p < 10^{-5}$ , BF  $>100$ ). The mean difference in surprisal was 1.15, however, suggesting that the models still had a LOW bias when the lower nominal was syntactically unavailable for attachment. This is in stark contrast to what one would expect if these models had learned the relationship between syntactic constituents and relative clause attachment. A possible
Figure 3: Proportion HIGH vs LOW attachment in Spanish. Human results from the original Fernández (2003) experiment and RNN LM results from the EXTENDED DATA (derived from Fernández (2003) and Carreiras and Clifton Jr (1993)).
alternative to the recency bias explanation is that RNN LMs might learn that there is a general LOW attachment bias in English and overgeneralize this pattern even in cases where one of the nominals is syntactically unavailable.
7 The case of default HIGH bias: Spanish
Our English analyses suggest that RNN LMs either learn a general English LOW attachment preference that they apply in all contexts, or that they have a 'default' recency bias that prevents them from learning HIGH attachment preferences with more complex, naturalistic training data. In the case of the former, we would expect that models trained on a language whose speakers generally prefer HIGH attachment should be able to learn HIGH attachment. Spanish has a well-attested HIGH bias in humans (Carreiras and Clifton Jr, 1993; Carreiras and Clifton, 1999; Fernandez, 2003) offering a way to distinguish between competing recency bias and over-generalization accounts. That is, if the models can learn a HIGH bias when trained on Spanish data, we should be able to conclude that the general LOW bias in English is being overgeneralized by the RNNs to corner cases where HIGH bias should be preferred.
7.1 Results
As before, the differences in surprisal were calculated for each pair of experimental items. If the difference was greater than zero, the attachment was coded as LOW. Two sample t-tests were conducted to see if the mean difference in surprisal differed
$^{12}$ Linzen and Leonard (2018) conducted experiments probing the agreement errors for subject-verb agreement with intervening RCs (and prepositional phrases). Our work is concerned with agreement between an object and its modifying RC. As such, their task serves as an approximate estimate of the errors we would expect for humans.
significantly from zero for both the direct simulation of Fernández (2003) and the EXTENDED DATA that included the stimuli derived from Carreiras and Clifton Jr (1993). The results categorically coded for HIGH/LOW attachment for the extended stimulus set are given in Figure 3, alongside the human results reported in Fernández (2003).
For the direct simulation, the mean did not differ significantly from 0 (BF < 1/3). This suggests that there is no attachment bias for the Spanish models for the stimuli from Fernández (2003), contrary to the human results. For the extended set of stimuli, the results were significant  $(p < 10^{-5}$ , BF > 100) with a mean difference greater than zero  $(\mu = 0.211)$ . Thus, rather than a HIGH bias, as we would expect, the RNN LMs once again had a LOW bias.
In this work, we explored the ability of RNN LMs to prioritize multiple simultaneous valid interpretations in a human-like way (as in John met the student of the teacher that was happy). While both LOW attachment (i.e. the teacher was happy) and HIGH attachment (i.e. the student was happy) are equally semantically plausible without a disambiguating context, humans have interpretation preferences for one attachment over the other (e.g., English speakers prefer LOW attachment and Spanish speakers prefer HIGH attachment). Given the recent body of literature suggesting that RNN LMs have learned abstract syntactic representations, we tested the hypothesis that these models acquire human-like attachment preferences. We found that they do not.
We first used a synthetic language experiment to demonstrate that RNN LMs are capable of learning a HIGH bias when HIGH attachment is at least as frequent as LOW attachment in the training data. These results suggest that any recency bias in RNN LMs is weak enough to be easily overcome by sufficient evidence of HIGH attachment. In English, the RNNs exhibited a human-like LOW bias, but this preference persisted even in cases where LOW attachment was ungrammatical. To test whether the RNNs were over-learning a general LOW bias of English, we tested whether Spanish RNNs learned the general HIGH bias in that language. Once again, RNN LMs favored LOW attachment over HIGH attachment. The inability of RNN LMs to learn the Spanish HIGH attachment preference sug
gests that the Spanish data may not contain enough HIGH examples to learn human-like attachment preferences.
In post-hoc analyses of the Spanish Wikipedia training corpus and the AnCora Spanish newswire corpus (Taulé et al., 2008), we find a consistent production bias towards LOW attachment among the RCs with unambiguous attachment. In Spanish Wikipedia, LOW attachment is  $69\%$  more frequent than HIGH attachment, and in Spanish newswire data, LOW attachment is  $21\%$  more frequent than HIGH attachment.[13] This distributional bias in favor of LOW attachment does not rule out a subsequent HIGH RC bias in the models. It has been established in the psycholinguistic literature that attachment is learned by humans as a general abstract feature of language (see Scheepers, 2003). In other words, human syntactic representations of attachment overlap, with prepositional attachment influencing relative clause attachment, etc. These relationships could coalesce during training and result in an attachment preference that differs from any one structure individually. However, it is clear that whatever attachment biases exist in the data are insufficient for RNNs to learn a human-like attachment preference in Spanish. This provides compelling evidence that standard training data itself may systematically lack aspects of syntax relevant to performing linguistic comprehension tasks.
We suspect that there are deep systematic issues leading to this mismatch between the expected distribution of human attachment preferences and the actual distribution of attachment in the Spanish training corpus. Experimental findings from psycholinguistics suggest that this issue could follow from a more general mismatch between language production and language comprehension. In particular, Kehler and Rohde (2015, 2018) have provided empirical evidence that the production and comprehension of these structures are guided by different biases in humans. Production is guided by syntactic and information structural considerations (e.g., topic), while comprehension is influenced by those considerations plus pragmatic and discourse factors (e.g., coherence relations). As such, the biases in language production are a proper subset of those of language comprehension. As it stands now, RNN LMs are typically trained on production data
<sup>13</sup>https://github.com/UniversalDependencies/UD_Spanish-AnCora
(that is, the produced text in Wikipedia).<sup>14</sup> Thus, they will have access to only a subset of the biases needed to learn human-like attachment preferences. In its strongest form, this hypothesis suggests that no amount of production data (i.e. raw text) will ever be sufficient for these models to generallyably pattern like humans during comprehension tasks.
The mismatch between human interpretation biases and production biases suggested by this work invalidates the tacit assumption in much of the natural language processing literature that standard, production-based training data (e.g., web text) are representative of the linguistic biases needed for natural language understanding and generation. There are phenomena, like agreement, that seem to have robust manifestations in a production signal, but the present work demonstrates that there are others, like attachment preferences, that do not. We speculate that the difference may lie in the inherent ambiguity in attachment, while agreement explicitly disambiguates a relation between two syntactic units. This discrepancy is likely the reason that simply adding more data doesn't improve model quality (e.g., van Schijndel et al., 2019; Bisk et al., 2020). Future work needs to be done to understand more fully what biases are present in the data and learned by language models.
Although our work raises questions about mismatches between human syntactic knowledge and the linguistic representations acquired by neural language models, it also shows that researchers can fruitfully use sentences with multiple interpretations to probe the linguistic representations acquired by those models. Before now, evaluations have focused on cases of unambiguous grammaticality (i.e. ungrammatical vs. grammatical). By using stimuli with multiple simultaneous valid interpretations, we found that evaluating models on single-interpretation sentences overestimates their ability to comprehend abstract syntax.
We would like to thank members of the NLP group and the C.Psyd lab at Cornell University, and the Altmann and Yee labs at University of Connecticut, who gave feedback on an earlier form of this work. We would also like to thank the three anonymous reviewers and Yonatan Belinkov. Special thanks go
to Dorit Abusch and John Whitman for invaluable suggestions and feedback, and Laure Thompson for comments on an earlier draft.
Maria Barrett, Joachim Bingel, Nora Hollenstein, Marek Rei, and Anders Søgaard. 2018. Sequence classification with human attention. In Proceedings of the 22nd Conference on Computational Natural Language Learning, pages 302-312, Brussels, Belgium. Association for Computational Linguistics.
Jean-Philippe Bernardy and Shalom Lappin. 2017. Using deep neural networks to learn syntactic agreement. Linguistic Issues in Language Technology (LiLT), 15.
Yonatan Bisk, Ari Holtzman, Jesse Thomason, Jacob Andreas, Yoshua Bengio, Joyce Chai, Mirella Lapata, Angeliki Lazaridou, Jonathan May, Aleksandr Nisnevich, Nicolas Pinto, and Joseph Turian. 2020. Experience grounds language. arXiv preprint arXiv:2004.10151.
Marc Brysbaert and Don C Mitchell. 1996. Modifier attachment in sentence parsing: Evidence from dutch. The Quarterly Journal of Experimental Psychology Section A, 49(3):664-695.
Manuel Carreiras and Charles Clifton. 1999. Another word on parsing relative clauses: Eyetracking evidence from Spanish and English. Memory & Cognition, 27(5):826-833.
Manuel Carreiras and Charles Clifton Jr. 1993. Relative clause interpretation preferences in Spanish and English. Language and Speech, 36(4):353-372.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics.
Chris Dyer, Gábor Melis, and Phil Blunsom. 2019. A critical analysis of biased parsers in unsupervised parsing. arXiv preprint arXiv:1909.09428.
Émile Enguehard, Yoav Goldberg, and Tal Linzen. 2017. Exploring the syntactic abilities of RNNs with multi-task learning. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 3-14. Association for Computational Linguistics.
Eva M. Fernández. 2003. Bilingual sentence processing: Relative clause attachment in English and Spanish. John Benjamins Publishing, Amsteradam.
<sup>14</sup>Some limited work has explored training models with human comprehension data with positive results (Klerke et al., 2016; Barrett et al., 2018).
Stefan L Frank and John Hoeks. 2019. The interaction between structure and meaning in sentence comprehension: Recurrent neural networks and reading times. *PsyArXiv* preprint: 10.31234.
Stefan L. Frank, Leun J. Otten, Giulia Galli, and Gabriella Vigliocco. 2015. The ERP response to the amount of information conveyed by words in sentences. *Brain & Language*, 140:1-11.
Lyn Frazier and Charles Clifton. 1996. Construal. MIT Press, Cambridge, Mass.
Richard Futrell and Roger Levy. 2019. Do RNNs learn human-like abstract word order preferences? In Proceedings of the Society for Computation in Linguistics, volume 2, pages 50-59.
Richard Futrell, Ethan Wilcox, Takashi Morita, and Roger Levy. 2018. RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency. arXiv preprint arXiv:1809.01329.
Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018. Colorless green recurrent networks dream hierarchically. In Proceedings of the 2018 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics.
John Hale. 2001. A probabilistic earley parser as a psycholinguistic model. In Proceedings of the second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies, pages 1-8. Association for Computational Linguistics.
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735-1780.
Andrew Kehler and Hannah Rohde. 2015. Pronominal reference and pragmatic enrichment: A bayesian account. In CogSci.
Andrew Kehler and Hannah Rohde. 2018. Prominence and coherence in a bayesian theory of pronoun interpretation. Journal of Pragmatics.
Sigrid Klerke, Yoav Goldberg, and Anders Søgaard. 2016. Improving sentence compression by learning to predict gaze. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1528-1533, San Diego, California. Association for Computational Linguistics.
Jey Han Lau, Alexander Clark, and Shalom Lappin. 2017. Grammaticality, acceptability, and probability: A probabilistic view of linguistic knowledge. Cognitive Science, 41:1202-1241.
Roger Levy. 2008. Expectation-based syntactic comprehension. Cognition, 106(3):1126-1177.
Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the ability of LSTMs to learn syntax-sensitive dependencies. Transactions of the Association for Computational Linguistics, 4:521-535.
Tal Linzen and Brian Leonard. 2018. Distinct patterns of syntactic agreement errors in recurrent networks and humans. In Proceedings of the 2018 Annual Meeting of the Cognitive Science Society, pages 690-695. Cognitive Science Society.
R Thomas McCoy, Robert Frank, and Tal Linzen. 2018. Revisiting the poverty of the stimulus: hierarchical generalization without a hierarchical bias in recurrent neural networks. In Proceedings of the 40th Annual Conference of the Cognitive Science Society.
Richard D. Morey and Jeffrey N. Rouder. 2018. BayesFactor: Computation of Bayes Factors for Common Designs. R package version 0.9.12-4.2.
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics.
Grusha Prasad, Marten van Schijndel, and Tal Linzen. 2019. Using priming to uncover the organization of syntactic representations in neural language models. In Proceedings of the 23rd Conference on Computational Natural Language Learning.
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. Technical report, OpenAI.
Shauli Ravfogel, Yoav Goldberg, and Tal Linzen. 2019. Studying the inductive biases of RNNs with synthetic variations of natural languages. In Proceedings of NAACL-HLT.
Jeffrey N. Rouder, Paul L. Speckman, Dongchu Sun, Richard D. Morey, and Geoffrey Iverson. 2009. Bayesian t-tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16(2):225-237.
Christoph Scheepers. 2003. Syntactic priming of relative clause attachments: Persistence of structural configuration in sentence production. Cognition, 89(3):179-205.
Helmut Schmid. 1999. Improvements in part-of-speech tagging with an application to German. In Natural language processing using very large corpora, pages 13-25. Springer.
Claude Shannon. 1948. A mathematical theory of communication. Bell System Technical Journal, 27:379-423, 623-656.
Nathaniel J Smith and Roger Levy. 2013. The effect of word predictability on reading time is logarithmic. Cognition, 128(3):302-319.
Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019. Energy and policy considerations for deep learning in NLP. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.
Mariona Taule, M. Antônia Martí, and Marta Recasens. 2008. AnCora: Multilevel annotated corpora for catalan and spanish. In Proceedings of the Sixth International Conference on Language Resources and Evaluation.
Andrew Trask, Felix Hill, Scott E Reed, Jack Rae, Chris Dyer, and Phil Blunsom. 2018. Neural arithmetic logic units. In Advances in Neural Information Processing Systems, pages 8035-8044.
Marten van Schijndel and Tal Linzen. 2018. Modeling garden path effects without explicit hierarchical syntax. In Proceedings of the 40th Annual Meeting of the Cognitive Science Society.
Marten van Schijndel, Aaron Mueller, and Tal Linzen. 2019. Quantity doesn't buy quality syntax with neural language models. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019. SuperGLUE: A stickier benchmark for general-purpose language understanding systems. In Advances in Neural Information Processing Systems, pages 3261-3275.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP.
Ethan Wilcox, Roger Levy, and Richard Futrell. 2018. What Syntactic Structures block Dependencies in RNN Language Models? In Proceedings of the 41st Annual Meeting of the Cognitive Science Society.
Ethan Wilcox, Roger Levy, and Richard Futrell. 2019. Hierarchical representation in neural language models: Suppression and recovery of expectations. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP.
A Fernández (2003) Replications
A.1 English
We compute RNN surprisal for each experimental item from Fernandez (2003) as detailed in Section
Figure 4: Proportion HIGH vs LOW attachment in English. Human results from the original Fernández (2003) experiment and RNN LM results from the stimuli from Fernández (2003).
3.3 in the paper. The results coded for HIGH/LOW attachment are given in Figure 4, including the results for humans reported by Fernandez (2003). While these categorical results enable easier comparison to the human results reported in the literature, statistical robustness was determined using the original distribution of surprisal values. Specifically, a two-tailed t-test was conducted to see if the mean difference in surprisal differed from zero (i.e. the model has some attachment bias). The result is highly significant  $(p < 10^{-5}$ , Bayes Factor  $(\mathrm{BF}) > 100)$  with a mean surprisal difference of  $\mu = 0.66$ . This positive difference suggests that the RNN LMs have a LOW bias, similar to English readers.
Figure 5: Proportion HIGH vs LOW attachment in Spanish. Human results from the original Fernández (2003) experiment and RNN LM results from the stimuli from Fernández (2003).
A.2 Spanish
The results coded for HIGH/LOW attachment for the Spanish replication are given in Figure 5, including the human results reported by Fernández (2003). The mean did not differ significantly from  $0\mathrm{(BF} < 1 / 3)$ . This suggests that there is no attachment bias for the Spanish models for the stimuli from Fernández (2003), contrary to the human results.