It’s not Rocket Science: Interpreting Figurative Language in Narratives (2022)

Abstract: Figurative language is ubiquitous in English. Yet, the vast majority of NLP research focuses on literal language. Existing text representations by design rely on compositionality, while figurative language is often non- compositional. In this paper, we study the interpretation of two non-compositional figurative languages (idioms and similes). We collected datasets of fictional narratives containing a figurative expression along with crowd-sourced plausible and implausible continuations relying on the correct interpretation of the expression. We then trained models to choose or generate the plausible continuation. Our experiments show that models based solely on pre-trained language models perform substantially worse than humans on these tasks. We additionally propose knowledge-enhanced models, adopting human strategies for interpreting figurative language types: inferring meaning from the context and relying on the constituent words’ literal meanings. The knowledge-enhanced models improve the performance on both the discriminative and generative tasks, further bridging the gap from human performance.

## introduction figurative language is a medium for making language expressive, communicating abstract ideas otherwise difficult to visualize, and provoking emotions @xcite . despite the ubiquity of figurative language across various forms of speech and writing, the vast majority of nlp research focuses primarily on literal language. figurative language is often more challenging due to its implicit nature and is seen as ''a bottleneck in automatic text understanding'' @xcite . ## data we build datasets aimed at testing the understanding of figurative language in narratives, focusing on idioms (section 3.1) and similes (section 3.2). we posit that a model that truly understands the meaning of a figurative expression, like humans do, should be able to infer or decide what happens next in the context of a narrative. thus, we construct a dataset in the form of the story-cloze test. ## discriminative task the first task we derive from our dataset is of discriminative nature in the setup of the story cloze task. given a narrative n and two candidate continuations {c 1 , c 2 }, the goal is to choose which of the continuations is more plausible. ## conclusion we introduced a narrative understanding benchmark focused on interpreting figurative language, specifically idioms and similes. following the story cloze test, we designed discriminative and generative tasks with the goal of continuing a narrative. we found that pre-trained lms irrespective of their size struggle to perform well in zero-shot and few-shot setting, and that the supervised models while competitive are still behind human performance by a significant margin. we further bridged some of this gap with knowledge-enhanced models that are inspired by the way humans interpret figurative expressions. our analysis reassessed known findings that although lms generate grammatical human-like texts, they are often inconsistent and the model's ability to distinguish characters in a story is limited. we hope this work will spark additional interest in the research community to further advance the representations and modeling of figurative language, which is too common to ignore.