Causal interventions expose implicit situation models for commonsense language understanding (2023)

Abstract: Accounts of human language processing have long appealed to implicit “situation models” that enrich comprehension with relevant but unstated world knowledge. Here, we apply causal intervention techniques to recent transformer models to analyze performance on the Winograd Schema Challenge (WSC), where a single context cue shifts interpretation of an ambiguous pronoun. We identify a relatively small circuit of attention heads that are responsible for propagating information from the context word that guides which of the candidate noun phrases the pronoun ultimately attends to. We then compare how this circuit behaves in a closely matched “syntactic” control where the situation model is not strictly necessary. These analyses suggest a distinct pathway through which implicit situation models may be constructed to guide pronoun resolution

## introduction language understanding is deeply intertwined with world knowledge. for example, when reading a sentence like "the fish ate the worm," we can guess that the fish was probably hungrier before eating and that the worm is no longer alive, even though neither property is explicitly mentioned @xcite . classical psycholinguistic accounts have suggested that such knowledge enters into language understanding through structured schemas called situation models @xcite @xcite @xcite @xcite ) that are dynamically constructed during comprehension. put succinctly, a situation models is a representations of the web of entities and relations that are implied without being explicitly 1 code & data: https://github.com/taka-yamakoshi/ situation-models / demo: https://huggingface.co/ spaces/taka-yamakoshi/causal-intervention-demo