A fine-grained comparison of pragmatic language understanding in humans and language models (2023)

Abstract: Pragmatics and non-literal language understanding are essential to human communication, and present a long-standing challenge for artificial language models. We perform a fine-grained comparison of language models and humans on seven pragmatic phenomena, using zero-shot prompting on an expert-curated set of English materials. We ask whether models (1) select pragmatic interpretations of speaker utterances, (2) make similar error patterns as humans, and (3) use similar linguistic cues as humans to solve the tasks. We find that the largest models achieve high accuracy and match human error patterns: within incorrect responses, models favor literal interpretations over heuristic-based distractors. We also find preliminary evidence that models and humans are sensitive to similar linguistic cues. Our results suggest that pragmatic behaviors can emerge in models without explicitly constructed representations of mental states. However, models tend to struggle with phenomena relying on social expectation violations.

## introduction non-literal language understanding is an essential part of communication. for example, in everyday conversations, humans readily comprehend the nonliteral meanings of metaphors (my new coworker is a block of ice), polite deceits (i love the gift), indirect requests (it's a bit cold in this room), and irony (classy pajamas, dude!). these phenomena fall under the broad label of pragmatics, which encompasses the aspects of meaning that go beyond the literal semantics of what is said @xcite @xcite . a long-standing challenge for nlp is to build models that capture human pragmatic behaviors. code and data: https://github.com/jennhu/lm-pragmatics the remarkable abilities of modern language models (lms) have triggered a recent effort to investigate whether such models capture pragmatic meaning, both through philosophical arguments @xcite @xcite and empirical evaluations @xcite @xcite @xcite . however, prior empirical studies have primarily evaluated lms based on a binary distinction between pragmatic and non-pragmatic responses, providing limited insights into models' weaknesses. a model could fail to reach the target pragmatic interpretation in multiple ways -for example, by preferring a literal interpretation, or by preferring a non-literal interpretation that violates certain social norms. understanding these error patterns can suggest specific directions for improving the models, and foreshadow where pragmatics might go awry in user-facing settings (e.g., @xcite @xcite . from a cognitive perspective, understanding the pragmatic abilities of lms could also offer insights into humans. human pragmatic language comprehension involves a variety of mechanisms, such as basic language processing, knowledge of cultural and social norms @xcite , and reasoning about speakers' mental states @xcite @xcite . however, it remains an open question when language understanding relies on explicit mentalizing -which may be cognitively effortful -versus lower-cost heuristics (e.g., @xcite . because lms lack explicit, symbolic representations of mental states, they can serve as a tool for investigating whether pragmatic competence can arise without full-blown mentalizing (e.g., belief updates in the rational speech act framework; @xcite . in this paper, we perform a fine-grained comparison of humans and lms on pragmatic language understanding tasks. adopting the approach of targeted linguistic evaluation (e.g., @xcite @xcite , our analysis serves two goals: assessing the pragmatic capabilities of modern lms, and revealing whether pragmatic behaviors emerge without explicitly constructed mental representations. our test materials are a set of english multiple-choice questions curated by expert researchers @xcite , covering seven diverse pragmatic phenomena. we use zero-shot prompting to evaluate models with varying sizes and training objectives: gpt-2 @xcite , tk-instruct @xcite , flan-t5 @xcite , and instructgpt @xcite . through model analyses and human experiments, we investigate the following questions: (1) do models recover the hypothesized pragmatic interpretation of speaker utterances? (2) when models do not select the target response, what errors do they make -and how do these error patterns compare to those of humans? (3) do models and humans use similar cues to arrive at pragmatic interpretations? we find that flan-t5 (xl) and openai's text-davinci-002 achieve high accuracy and mirror the distribution of responses selected by humans. when these models are incorrect, they tend to select the incorrect literal (or straightforward) answer instead of distractors based on lowlevel heuristics. we also find preliminary evidence that models and humans are sensitive to similar linguistic cues. our results suggest that some pragmatic behaviors emerge in models without explicitly constructed representations of agents' mental states. however, models perform poorly on humor, irony, and conversational maxims, suggesting a difficulty with social conventions and expectations. ## related work prior work has evaluated lms' ability to recognize non-literal interpretations of linguistic input, such as scalar implicature @xcite @xcite or figurative language @xcite @xcite . in a broad-scale evaluation, @xcite test five types of implicatures arising from @xcite conversational maxims, and evaluate their models after training on the task. in our work, we consider gricean implica-tures as one of seven phenomena, and we evaluate pre-trained lms without fine-tuning on our tasks. similar to our work, @xcite also use prompting to evaluate lms on pragmatic interpretation tasks. they formulate implicature tests as sentences ending with "yes" or "no" (e.g., "esther asked "can you come to my party on friday?" and juan responded "i have to work", which means no."). a model is considered pragmatic if it assigns higher probability to the token that makes the sentence consistent with an implicature. in our work, models must select from multiple interpretations, enabling a detailed error analysis and comparison to humans. ruis et al.'s materials also focus on indirect question answering as an implicature trigger, whereas we consider a broader range of pragmatic phenomena and utterance types. since pragmatic language understanding often draws upon knowledge of social relations, our tasks are conceptually related to benchmarks for evaluating social commonsense (e.g., @xcite . these evaluations focus on the interpretation of actions and events, whereas we focus on the interpretation of speaker utterances. another hypothesized component of pragmatics is theory of mind (tom; @xcite , or the ability to reason about others' mental states. benchmarks for evaluating tom in models (e.g., @xcite @xcite primarily focus on false-belief tasks @xcite , which assess whether a model can represent the beliefs of another agent that are factually incorrect but consistent with that agent's observations. lms have been shown to succeed on some tom tests (kosinski, 2023) while failing on others @xcite . ## results we now return to the three questions posed in the introduction, in each of the following subsections. 9 parameter estimates come from https://blog.eleuther.ai/ gpt3-model-sizes/ . although the size of text-davinci-002 is unknown, we assume that it is larger than instructgpt-3 (ada). 10 the openai model results might not be reproducible, but ## discussion we used an expert-curated set of materials @xcite to compare lms and humans on seven pragmatic phenomena. we found that flan-t5 (xl) and text-davinci-002 achieve high accuracy and match human error patterns: within incorrect responses, these models tend to select the literal interpretation of an utterance over heuristicbased distractors. we also found preliminary evidence that lms and humans are sensitive to similar linguistic cues: model and human accuracy scores correlate at the item level for several tasks, and degrade in similar ways when context is removed. our results suggest that language models can consistently select the pragmatic interpretation of a speaker's utterance -but how? the models tested in our experiments reflect a variety of learning processes through which pragmatic knowledge could emerge. gpt-2 is trained to learn the distribution of linguistic forms; the tk-instruct and flan-t5 models are pre-trained on a denoising task and fine-tuned on thousands of instruction-based tasks; and the openai models receive signal from human feedback. our experiments are not designed to tease apart the contributions of these training procedures to models' behaviors. therefore, we do not intend to make strong claims about the mechanisms by which models learn pragmatics. a shared feature of our tested models is the lack of explicitly constructed mental state representations. in this sense, our results are potentially compatible with two hypotheses. one possibility is that the models do not have an ability that can be considered an analog of theory of mind (tom). this view is supported by evidence that language models perform poorly on social commonsense and false-belief tasks @xcite , and are remarkably brittle to small perturbations of classic tests @xcite . if models truly lack tom, then their pragmatic behaviors might be explained by inferences based on low-level linguistic cues. taken a step further, this finding could potentially suggest that certain human pragmatic behaviors arise through inferences based on language statistics, with no need for mental state representations. a second possibility is that models do have a heuristic version of tom, which is not explicitly engineered but instead emerges as a by-product of optimizing for other objectives (such as linguistic prediction). since language contains many descriptions of agents' beliefs, emotions, and desires, it may be beneficial -perhaps even necessary -to induce representations of these mental states in order to learn a generative model of linguistic forms. indeed, @xcite argues that whereas language models have no explicit representation of communicative intents, they can infer approximate representations of the mental states of the agents that produce a given linguistic context. if this hypothesis is true, however, it would still remain unclear whether tom is necessary to support the pragmatic behaviors tested in our evaluation materials. our experiments do not differentiate between these two hypotheses. however, fine-grained behavioral evaluations -such as those presented in this work -are important for revealing models' capabilities and weaknesses, and offer a first step toward understanding how pragmatic behaviors can be supported. a promising direction for future work is to test models with a wider range of training objectives, or even new architectures, such as distinct language and social reasoning modules (see @xcite . in addition, although there is evidence for the role of mentalizing in our tested pragmatic phenomena (see section 3.1), one limitation of our stimuli is that they were not specifically designed to require tom. new datasets that perform targeted manipulations of tom alongside tests of language comprehension could help reveal how linguistic experience and tom jointly support pragmatic behaviors. josh tenenbaum for their constructive feedback. we also thank quinn langford for help with coding details of the stimuli. this work was in part supported by a grant from the simons foundation to the simons center for the social brain at mit. j.