Effects of Filler-gap Dependencies Working Memory Requirements for Parsing

Effects of Filler-gap Dependencies on Working Memory Requirements for Parsing William Schuler (schuler@ling.osu.edu) Department of Linguistics, 1712 Neil Avenue, Columbus, OH 43210 USA S Abstract VP NP Corpus studies by Schuler, AbdelRahman, Miller, and Schwartz (2010), appear to support a model of comprehen- sion taking place in a general-purpose working memory store, by providing an existence proof that a simple probabilis- tic sequence model over stores of up to four syntactically- contiguous memory elements has the capacity to reconstruct phrase structure trees for over 99.9% of the sentences in the Penn Treebank Wall Street Journal corpus (Marcus, Santorini, & Marcinkiewicz, 1993), in line with capacity estimates for general-purpose working memory, e.g. by Cowan (2001). But capacity predictions of this simple structure-based model ig- nore non-structural dependencies, such as long-distance filler- gap dependencies, that may place additional demands on work- ing memory. Distinguishing unattached gap fillers from open attachment sites in syntactically-contiguous memory elements requires this contiguity constraint to be strengthened to a con- straint that working memory elements be semantically con- tiguous. This paper presents corpus results showing that this stricter semantic contiguity constraint still predicts working memory requirements in line with capacity estimates such as that of Cowan (2001). Keywords: DT NN The president VP VB meets NP DT the NN S/VP VP/NN Figure 1: Incomplete constituents in an incremental parse of the sentence ‘The president meets the board on Friday,’ showing non-immediate dominance between incomplete con- stituents ‘The president ...’ and ‘meets the ...’. parsing is only part of comprehension. In order to obtain valid interpretations, a comprehension model must also retain non- structural information like gap fillers in long-distance depen- dency constructions. Filler-gap dependencies are common (occurring in about 20% of sentences in the Wall Street Jour- nal corpus), and figure prominently in the psycholinguistics literature on memory bounds in parsing (Gibson, 1991; Just & Carpenter, 1992, etc.). But since the introduction of a gap filler adds no unsatisfied structural attachment sites (and thus no need to retain additional memory elements), they are ig- nored in the capacity predictions of the simple syntax-based model described above. Distinguishing unattached gap fillers from open attachment sites in syntactically-contiguous mem- ory elements requires the syntactic contiguity constraint of Schuler et al. (2010) to be strengthened to a constraint that working memory elements be semantically contiguous (that is, linked by roles, in the sense of Gibson, 1991). This would force gap fillers into separate memory elements, re- quiring additional memory resources to process sentences in which filler-gap dependencies co-occur with structurally- nested constituents. This paper presents coverage results on the same Wall Street Journal corpus showing that, despite their prevalence, long-distance dependencies do not seem to occur in deeply structurally-embedded contexts, and this stricter semantic contiguity constraint still predicts working memory require- ments in line with capacity estimates such as that of Cowan (2001). This seems to support the hypothesis that compre- hension may take place in a general-purpose working mem- ory store, using a simple notion of semantically-contiguous incomplete constituents as memory elements. Introduction It is tempting to think of sentence comprehension as learned manipulations of elements in a general-purpose working memory store. This assumption underlies many established comprehension models (e.g. Johnson-Laird, 1983; El- man, 1991; Gibson, 1991; Just & Carpenter, 1992; Gibson, 1998; Lewis & Vasishth, 2005, making various assumptions about the size and nature of this memory). Corpus stud- ies by Schuler et al. (2010), appear to support this hypoth- esis by providing an existence proof that a simple probabilis- tic sequence model over stores of up to four syntactically- contiguous memory elements — simple random variables with discrete domains over pairs of constituent categories at which other structures may attach — has the capacity to reconstruct phrase structure trees for over 99.9% of the sentences in the Penn Treebank Wall Street Journal corpus (Marcus et al., 1993). This is in line with capacity estimates for general-purpose working memory, e.g. by Cowan (2001), but is also compatible with a continuously degrading avail- ability (McElree, 2001), since the model’s use of this store degrades very rapidly after one element. To the extent that the Wall Street Journal is comprehensible (and its editors are doing their jobs) this suggests that comprehension can take place in a general-purpose working memory store, using a small set of incomplete constituent states derived from phrase structure. The structural attachment sites in the memory elements of Schuler et al. (2010) are necessary in order to accurately re- construct syntactic relations in phrase structure trees. But