LM s stand their Ground: Investigating the Effect of Embodiment in Figurative Language Interpretation by Language Models (2023)

Abstract: Figurative language is a challenge for language models since its interpretation is based on the use of words in a way that deviates from their conventional order and meaning. Yet, humans can easily understand and interpret metaphors, similes or idioms as they can be derived from embodied metaphors. Language is a proxy for embodiment and if a metaphor is conventional and lexicalised, it becomes easier for a system without a body to make sense of embodied concepts. Yet, the intricate relation between embodiment and features such as concreteness or age of acquisition has not been studied in the context of figurative language interpretation concerning language models. Hence, the presented study shows how larger language models perform better at interpreting metaphoric sentences when the action of the metaphorical sentence is more embodied. The analysis rules out multicollinearity with other features (e.g. word length or concreteness) and provides initial evidence that larger language models conceptualise embodied concepts to a degree that facilitates figurative language understanding.

## introduction infants acquire their first conceptual building blocks by observation and manipulation in the physical world. these primary building blocks enable them to make sense of their perceptions @xcite . in return, their embodiment defines the capabilities with which they can explore and understand the world. the early conceptual system is built from spatial schemas, which enables early word understanding @xcite . these so-called image schemas are recurring cognitive structures shaped by physical interaction with the environment. they emerge from bodily experience and motivate subsequent conceptual metaphor mappings @xcite . the metaphorical mapping is visible in our everyday language whenever we use figurative language. for example, if we say that she dances like a turtle, that is to say, that she dances poorly. the metaphor in this phrase is readily interpreted by humans, who would favour the interpretation of dances poorly over dances well. the turtle dance example employs a conceptual mapping in which the turtle provides the source domain for the attributes slow and rigid, which turn the dance target domain into poorly dancing. this mapping draws from the human, bodily experience of dancing and therefore enables interpretation. for a language model (lm), the understanding of figurative language is a great challenge @xcite . by nature of their digital implementation as computer algorithms, lms are nonembodied and do not ground their conceptualisation by physical interaction with the environment. instead, lms learn statistical features of language by deep learning vast amounts of data @xcite . whether these learned statistical features allow lms to mirror or copy natural language understanding (nlu) is subject to discussion @xcite . moreover, @xcite suggest an embodied language understanding paradigm for lms can benefit nlu systems through grounding by metaphoric inference. one can argue that most embodied metaphors are heavily conventional (e.g. up is good, down is bad, knowledge is light, ig-norance is darkness) and as such, they are lexicalised in a language without an inherent need to understand their bodily basis. this lexicalisation should allow lms to conceptualise and interpret them correctly and more robustly than less conventional metaphors. conventionality relates to word frequency and age of acquisition (aoa), i.e. more frequent words and words that are acquired early in life are more conventional. we argue that embodiment has a measurable effect on the interpretation of metaphors that differs from the effect of other linguistic features. moreover, we investigate whether an interpretation of figurative language with more embodied concepts is easier for lms. analogously, we investigate conflating factors such as the aoa, word frequency, concreteness and word length. the relation between embodiment and lms' ability to interpret figurative language has not yet been investigated and is the key contribution of this research. the following section (2) starts with a review of language model abilities, more specifically figurative language interpretation abilities. the review identifies a suitable data set for our experiment and describes its formation in section 3. we use a subset of the fig-qa data set @xcite , a winograd-style figurative language understanding task, and correlate the performance of various lms concerning the degree of embodiment of the metaphorical actions that the lms are tasked to interpret. in section 4, we identify that models, that can reach a certain performance on our fig-qa subset, shows a significant and positive correlation between the rating of the embodiment of the action involved in the metaphorical phrase and the model's ability to interpret the metaphor correctly. an in-depth analysis of additional features, such as the aoa, word length or frequency, does not indicate multicollinearity among those features. in section 5 we conclude that the degree of embodiment of the action within the metaphoric phrase is a predictor of the lms' ability to correctly interpret the figurative language. lastly, we discuss the limitations and broader implications of the work. ## statistical evaluation the review of related works shows that there are abilities of lms that go beyond mere language generation, e.g. logical reasoning, and action planning. it is unclear how lms conceptualise actions that humans conceptualise using interaction with the environment. figurative language acts as a test bed to assess metaphorical conceptualisations since they are grounded in embodied experience and interaction with the environment. we take @xcite 's findings as a starting point to focus on the effect of embodiment in figurative language interpretation by langauge models of various sizes.