Can Language Models Be Tricked by Language Illusions? Easier with Syntax, Harder with Semantics (2023)

Abstract: Language models (LMs) have been argued to overlap substantially with human beings in grammaticality judgment tasks. But when humans systematically make errors in language processing, should we expect LMs to behave like cognitive models of language and mimic human behavior? We answer this question by investigating LMs’ more subtle judgments associated with “language illusions” – sentences that are vague in meaning, implausible, or ungrammatical but receive unexpectedly high acceptability judgments by humans. We looked at three illusions: the comparative illusion (e.g. “More people have been to Russia than I have”), the depth-charge illusion (e.g. “No head injury is too trivial to be ignored”), and the negative polarity item (NPI) illusion (e.g. “The hunter who no villager believed to be trustworthy will ever shoot a bear”). We found that probabilities represented by LMs were more likely to align with human judgments of being “tricked” by the NPI illusion which examines a structural dependency, compared to the comparative and the depth-charge illusions which require sophisticated semantic understanding. No single LM or metric yielded results that are entirely consistent with human behavior. Ultimately, we show that LMs are limited both in their construal as cognitive models of human language processing and in their capacity to recognize nuanced but critical information in complicated language materials.

## introduction linguistic evaluations of language models use human language processing data (e.g. human norming data @xcite , acceptability judgments @xcite , behavioral or neural measures of language processing @xcite ) as benchmarks to investigate whether lms possess knowledge of language. this assumes that human-produced data correctly instantiates abstract rules of a language and that humans fully utilize their linguistic knowledge in laboratories and everyday life. however, this assumption is an oversimplification. humans make consistent errors during language processing @xcite . under these circumstances, should we expect language models to behave the same as humans? or should they circumvent human limitations and achieve error-free performance? consider, for example, the well-studied case of subject-verb agreement. while we expect an lm of standard american english to prefer "the key to the cabinets is on the shelf" to "the key to the cabinets are on the shelf" (as discussed in @xcite , a wealth of psycholinguistic research has systematically documented that humans can ignore errors and accept globally ungrammatical strings (stemming from @xcite . should lms follow the ideal grammar or mimic human's (sometimes) errorful behavior? 1 we add to this discussion by investigating three language illusions. basic examples of each are given in (1): the comparative illusion (1-a), the depth-charge illusion (1-b), and the negativepolarity item (npi) illusion (1-c). all three in (1) are literally unnatural english sentences, despite the fact that humans often find them surprisingly acceptable. in this paper, we relied on minimally different strings springing out from the basic illusion sentences that are either (a) considered fully acceptable by human participants, (b) considered fully unacceptable by human participants, or (c) rated surprisingly acceptable by humans (i.e. instances of the relevant illusion). we explored whether language models capture the basic contrast between acceptable and unacceptable strings, whether they rate illusion sentences as better than their unacceptable counterparts, and finally, whether models capture nuanced linguistic manipulations that influence human judgments of the illusion material. further, we compared two ways of measuring models' preferences, one over the whole sentence (perplexity) and another of a privileged position in the sentence (surprisal). if lms pattern like human comprehension behavior that involves errors, we expect to derive measures that similarly rate illusion sentences as more acceptable than typical unacceptable sentences. if, on the other hand, lms align with ideal grammatical judgments, illusion sentences should be rated as unacceptable. our findings indicate that none of the language models we investigated consistently exhibited illusion effects or demonstrated overall human-like judgment behaviors. nor do they possess the necessary linguistic knowledge for errorfree, literal sentence processing. these findings add more insights into the discussion of lms' emulation of human behavior and their construal as cognitive models of human language processing. ## comparative illusion a canonical comparative illusion surfaces in sentences like "more people have been to russia than i have". people accept it at first glance but have trouble pinning down the exact meaning (montalbetti, 1984) one of which could be that the number of the group of people who've been to russia is greater than the number of "me". potential rational nonliteral inference could be "people have been to russia more times than i have" or "people have been to russia but i haven't" (o @xcite . psycholinguistic research has found that various factors modulate the strength of the illusion, including the repeatability of the event described by the verb phrase, the subject form of the than-clause subject (e.g. "... than the student has" vs. "...i have"), as well as the number of that subject (e.g. "i have" vs. "we have") @xcite . there is also a claim arguing that the processing mechanism follows the noisy-channel predictions under an information-theoretic account @xcite . we adapted the experimental materials with 32 items from @xcite . 6 an example is in (2) where (2-a) is the canonical comparative illusion, (2-b) is the acceptable control, and (2-c) is the unacceptable one. 7 6 see ## depth-charge illusion consider the most famous depth-charge sentence no head injury is too trivial to be ignored @xcite . people overwhelmingly interpret it as meaning "no matter how trivial head injuries are, we should not ignore them", while the literal meaning is the opposite as "we should ignore them". to understand the depth-charge sentence requires knowing meaning composition rules, multiple negation processing @xcite , adequate world knowledge reasoning @xcite , and the neighboring constructions of too...to such as so...that, so...as to and enough to... @xcite . since existing research already shows that language models are quite limited in processing negation (e.g. kassner and schütze, 2019; ettinger, 2020), we speculate that lms might encounter difficulty in the more complicated case of depth-charge sentences. the evaluation materials were adapted from @xcite with 32 items. an example is (3) where we take the surprisal of the sentence-final word for comparison. 11 more statistic model information: iterating over lms, metrics, and the subject structure (np vs. pronoun), we initiated statistical models taking both repeatability (@xmath0. ## npi illusion negative polarity items and their licensing conditions have been investigated in prior work with language models. for a canonical npi (e.g. ever, any) to be acceptable, it has to be in the scope of negation. 15 existing computational research has shown that the syntactic dependency between the licensor and the npi is captured by language models @xcite @xcite but with more difficulty as compared to subject-verb agreement or other syntactic dependencies @xcite @xcite . in this task, we expanded the suite of lms and metrics and explored sensitivities to four types of licensors. 14 the sentence no head injury is too trivial to not be ignored should be plausible because compositionally, "too trivial to not be ignored" means "too trivial to be treated" which yields a plausible sentence given the sentential negation. 15 the licensing conditions of negative polarity items are far more than in the scope of negation. we focus on the classic licensing condition and refer to @xcite for a review. our materials came from @xcite with 32 items. the essential triad is (4) where the illusion condition has the npi ever not in the scope of the negation word no. ## conclusion we tested four language models' ability to process three language illusions and asked (1) whether they judge unacceptable illusion sentences to be more acceptable as humans (termed an illusion effect) and (2) whether they are sensitive to linguistic manipulations that modulate human judgments. our results are based on whole-sentence perplexity and critical word surprisal. we show that none of the lms demonstrated consistent illusion effects or exhibited overall human-like judgment behaviors. we conclude that given the case of language illusions, language models neither behave like humans with full sets of cognitive abilities and error-prone behavior nor possess the necessary linguistic knowledge for error-free, literal sentence processing. language models cannot be viewed as cognitive models of language processing, which makes understanding them even more intriguing.