A Targeted Assessment of Incremental Processing in Neural Language Models and Humans (2021)

Abstract: We present a targeted, scaled-up comparison of incremental processing in humans and neural language models by collecting by-word reaction time data for sixteen different syntactic test suites across a range of structural phenomena. Human reaction time data comes from a novel online experimental paradigm called the Interpolated Maze task. We compare human reaction times to by-word probabilities for four contemporary language models, with different architectures and trained on a range of data set sizes. We find that across many phenomena, both humans and language models show increased processing difficulty in ungrammatical sentence regions with human and model ‘accuracy’ scores a la Marvin and Linzen (2018) about equal. However, although language model outputs match humans in direction, we show that models systematically under-predict the difference in magnitude of incremental processing difficulty between grammatical and ungrammatical sentences. Specifically, when models encounter syntactic violations they fail to accurately predict the longer reading times observed in the human data. These results call into question whether contemporary language models are approaching human-like performance for sensitivity to syntactic violations.

## introduction a substantial body of work has investigated contemporary language models (lms) by assessing whether their behavior is consistent with the rules of syntax @xcite @xcite ). 1 among other structures, these studies have investigated agreement @xcite long distance dependencies @xcite , pronominal and particle licensing @xcite , and expectations for phrase-level constituents @xcite . many of the studies which report aggregate behavior across a broad number of phenomena focus on accuracy scores, or the proportion of time lms or human subjects in an online experiment prefer a grammatical variant in matching grammatical / ungrammatical sentence pairs. while these investigations provide much insight, they collapse a crucial dimension of comparison, namely the difference in magnitude between the grammatical and ungrammatical conditions. as long as the direction of their predictions are the same, an lm which finds grammatical conditions only marginally worse than their corresponding ungrammatical counterpart will receive the same score as a model that displays large differences between the two conditions. at the same time, a related line of work has investigated the quantitative relationship between incremental predictions of language models and human reaction times @xcite . smith and @xcite found that this relationship is log-linear across multiple orders of magnitude for 3-gram models, and recent investigations have shown that this holds for contemporary neural network models as well @xcite . so far, this work has largely focused on the aggregate relationship, instead of isolating individual phenomena in targeted testing environments. we combine these two approaches with a targeted assessment of incremental processing in neural language models and humans. we collect incremental processing data on a series of sixteen test suites, adapted from @xcite , each of which targets a different syntactic phenomenon. for lm incremental processing data, we collect test suite name tag example wh-cleft structures cleft what she did/spied was see the giraffe/the giraffe filler-gap dependency, subject gap fgd-subj i know who/that /my mother sent the present to taylor. filler-gap dependency, object gap fgd-obj i know who/that my mother sent /the present to taylor. filler-gap dependency, pp gap fgd-pp i know who/that my mother sent the present to /taylor last weekend. main verb/reduced rc gardenpath mvrr the ship ∅/that was sunk/steered in the storm carried treasure. npi licensing, any, subj rc modifier npl-any-src no/the senator that no/the journalist likes has gotten any votes. npi licensing, any, obj rc modifier npl-any-orc no/the senator that likes no/the journalists has gotten any votes. npi licensing, ever, subj rc modifier npl-ever-src no/the senator that no/the journalist likes has ever won. npi licensing, ever, obj rc modifier npl-ever-orc no/the senator that likes no/the journalists has ever won. subject-verb number agr., subj rc modifier svna-src the lawyer/lawyers that helped the mayor is/are organized. subject-verb number agr., obj rc modifier svna-orc the lawyer/lawyers that the mayor hired is/are very organized. subject-verb number agr., pp modifier svna-pp the lawyer/lawyers next to the mayor is/are very organized. reflexive anaphora, masc., subj rc modifier rna-m-src the dukes/duke that hunted the rabbits saw himself/themselves in the mirror. reflexive anaphora, masc., obj rc modifier rna-m-orc the dukes/duke that the knights distrust saw himself/themselves in the mirror. reflexive anaphora, fem., subj rc modifier rna-f-src the queens/queen that hunted the rabbits saw herself/themselves in the mirror. reflexive anaphora, fem., obj rc modifier rna-f-orc the queens/queen that the knights distrust saw herself/themselves in the mirror. we deploy three analysis techniques to investigate how well models capture the human incremental processing data. first, we compute accuracy metrics (for lms) and consistency scores (for humans) for each of our test suites, which correspond to the proportion of the time behavior is consistent with the relevant grammatical rules. we find that, for this analysis, humans and machine performance is about equal. next, we compare the observed reaction-time slowdown between grammatical/ungrammatical conditions within a test suite to the slowdown predicted by each of our models. for this analysis we use the methodology developed by @xcite , who use a ms/bit (milliseconds of reaction time per bit of surprisal) conversion metric derived from a fitted regression model to convert between the outputs of lms and slowdowns in human reaction times. we find that models systematically under-predict the observed human data. in our third analysis, we train a linear regression models to predict reaction times from probabilities in non-critical sentence regions, and show that these models are relatively poor at predicting reaction times in critical sentence regions. that is, in areas of the sentence where human reaction time is influenced by grammatical violations, lm probabilities routinely under-predict human processing difficulty as measured by reaction time. taken together, these results indicate that contemporary neural network languages models are systematically less sensitive to grammatical violations compared to humans. ## methods we collect incremental processing data on a series of test suites, each of which targets an individual syntactic phenomenon. composition of the test suites is described in section 2.1. methods used to collect incremental processing data are outlined in section 2.2, for human reaction times. section 2.3 describes the models tested. linear regression models used to predict reaction times from model outputs will be referred to as 'linear fits' to avoid confusion with language models. ## discussion our experiments have tackled the question of whether syntactic difficulty can be reduced to byword probabilities by providing a comparison of language model and human behavior that is both incremental and targeted. our methods build on those presented in van schijndel and @xcite and van schijndel and linzen (2020), but differ from theirs in a number of key respects, which we review briefly below to highlight to novel aspects of our own investigation. first, all of our test suites target grammatical/ungrammatical contrasts (except for the mvrr gardenpath test), whereas van schijndel and linzen test locally ambiguous sentence regions that (may) require re-analysis for proper processing. second, we assess a broad range of grammatical violations across sixteen test suites that target seven distinct structures. third, we deploy a novel measurement of processing time (interpolated maze), instead of self-paced reading. we fit our own linear models from the i-maze data, and use a ms/bit scalar term derived from lexical distractor items. finally, we provide a novel analysis that compares the residuals of linear fits between critical and non-critical regions, and we break down these residuals based on the grammaticality of the condition.