Syntactic dependency length shaped by strategic memory allocation (2024)

Abstract: Human processing of nonlocal syntactic dependencies requires the engagement of limited working memory for encoding, maintenance, and retrieval. This process creates an evolutionary pressure for language to be structured in a way that keeps the subparts of a dependency closer to each other, an efficiency principle termed dependency locality. The current study proposes that such a dependency locality pressure can be modulated by the surprisal of the antecedent, defined as the first part of a dependency, due to strategic allocation of working memory. In particular, antecedents with novel and unpredictable information are prioritized for memory encoding, receiving more robust representation against memory interference and decay, and thus are more capable of handling longer dependency length. We examine this claim by analyzing dependency corpora of 11 languages, with word surprisal generated from GPT-3 language model. In support of our hypothesis, we find evidence for a positive correlation between dependency length and the antecedent surprisal in most of the languages in our analyses. A closer look into the dependencies with core arguments shows that this correlation consistently holds for subject relations but not for object relations.

## introduction language processing requires efficient use of bounded cognitive systems, creating evolutionary pressures that have been argued to shape the structure of human language @xcite . in the domain of syntax, one source of evidence for this idea comes from the principle of dependency locality, which holds that linguistic units connected in a syntactic dependency tend to stay close in linear order, due to the limited resources of working memory (wm) to hold the subparts of a non-local dependency in working memory @xcite @xcite @xcite @xcite @xcite . with this basic finding, a natural next step is to see how far this efficiency-based account for dependency locality can go, with a more and more realistic characterization of the nature and constraints of wm. we explore strategic memory allocation as one such constraint that may further shape the structure of syntactic dependencies. the idea is that limited wm resources are strategically allocated, subject to a trade-off between the economical investment of wm on each linguistic unit stored, and the minimization of potential cost in future processing tasks @xcite @xcite . specifically, we propose that linguistic units with novel and unpredictable information should receive prioritized wm resources for encoding and storage in memory, thus yielding more robust representation against memory interference and decay. the result of this strategic memory allocation is that when an antecedent carries novel and unpredictable information, it can tolerate longer dependency length. in the current study, we test this hypothesis in 11 languages: amharic, danish, english, german, italian, japanese, korean, mandarin, russian, spanish, and turkish. to preview the results, across all the dependency types, we find a general positive correlation between antecedent surprisal and dependency length for more than half of the languages in our analysis. a closer look into the dependencies with core arguments demonstrates that the effect consistently emerges for most of the languages in subject relations, but not in object relations. ## results figure 1 shows the average orthographic dependency length as a function of antecedent surprisal, along with linear fits. the visualization for the information-theoretic dependency length yields similar patterns (see additional figure in appendix). for subject relations, we find evidence for a positive effect of antecedent surprisal on dependency length for 7 out of 11 languages, namely amharic, danish, english, german, italian, mandarin, russian, and spanish. there is no significant effect for japanese and korean. for german and turkish, however, the data support a negative antecedent surprisal effect. for object relations, there is little evidence for a positive antecedent surprisal effect on the orthographic dependency length, with the effect being significantly negative for amharic, german, korean, mandarin, and russian. for the dependency length as surprisal, there is a positive antecedent surprisal effect in danish, english, italian, and spanish. but the effect is negative for german and korean. ## discussion in general, our results provide evidence for a positive correlation between antecedent surprisal and dependency length, indicating that dependencies whose antecedent is more surprising and informative are able to tolerate longer dependency length. this is especially true for subject relations, where 7 out of 11 languages in our analysis exhibit a positive antecedent surprisal effect on dependency length. for object relations, however, the data presents a mixed picture without clear support for an expected antecedent surprisal effect. this asymmetry between the subject and the object can be due to the possibility that object relations may be under more grammatical pressure than subject relations to put head and dependent closer to each other. for example, according to the accessibility hierarchy proposed by @xcite as a linguistic universal, the subject is more relativizable than the object crosslinguistically to form a relative clause. it is worth noting that the correlation observed in the current study is compatible with some other theories as well. for example, the uniform information density (uid) theory holds that language production should avoid abrupt fluctuation of information across linguistic units @xcite . therefore, surprising antecedents may be followed by longer sequence of units for a smoother transition to the other side of the dependency, which is supposed to bear lower surprisal due to the high mutual information with its antecedent @xcite . however, compared to uid, which is a computational-level theory @xcite , the strategic memory allocation proposed in the current study focuses on the processes more at the mechanistic level. ## conclusions in a nutshell, we find empirical support for a positive correlation between the length of a dependency and the surprisal of its antecedent. a closer look into the dependencies with core arguments shows that this relationship consistently holds for subject relations, but not for the object, possibility due to the stronger grammatical constraint between the object and the verb. at the population level, this finding indicates that although working memory constraints exert a general pressure on language structure to organize in a way that minimizes dependency length, this pressure is further modulated by informativity. this crosslinguistic pattern is consistent with our hypothesis of strategic working memory allocation as an individual-level processing strategy, where less predictable but more informative linguistic units are prioritized in working memory to maintain a more robust representation against memory interference and decay, thus are more tolerant for longer dependency length.