Are You Not moved? Incorporating Sensorimotor Knowledge to Improve Metaphor Detection (2023)

Abstract: Metaphors use words from one domain of knowledge to describe another, which can make the meaning less clear and require human interpretation to understand. This makes it difficult for automated models to detect metaphorical usage. The objective of the experiments in the paper is to enhance the ability of deep learning models to detect metaphors automatically. This is achieved by using two elements of semantic richness, sensory experience, and body-object interaction, as the main lexical features, combined with the contextual information present in the metaphorical sentences. The tests were conducted using classification and sequence labeling models for metaphor detection on the three metaphorical corpora VUAMC, MOH-X, and TroFi. The sensory experience led to significant improvements in the classification and sequence labelling models across all datasets. The highest gains were seen on the VUAMC dataset: recall increased by 20.9%, F1 by 7.5% for the classification model, and Recall increased by 11.66% and F1 by 3.69% for the sequence labelling model. Body-object interaction also showed positive impact on the three datasets.

## introduction metaphors are an important and widespread form of language construction. a metaphorical sentence's meaning is not a direct, literal translation of its parts, but rather an overall collection of meanings in a specific context. for example, the phrase "weigh my options" refers to the situation in which the advantages and disadvantages of an option are examined for a decision. it is a considera-tion, not a literal weighing. this form of notation refers to the conceptual metaphor theory @xcite , where the source domain weighing provides the words used to describe the target domain consideration. an-other example that discusses love while addressing the concept of heat: "i bumped into an old flame at the library". such examples demonstrate that understanding and interpreting metaphors are complex tasks for the field of nlp. many fields, such as information extraction @xcite and sentiment analysis @xcite @xcite benefit from metaphor detection. many experiments are being undertaken to improve the detection task using machine learning and deep learning models. the term "sensorimotor knowledge" describes knowledge learned through the body's interactions with its surroundings. this knowledge could aid in comprehending metaphors, understanding how they are constructed, and consequently, enhance the performance of automated metaphor detection. by incorporating sensorimotor knowledge as a feature in neural network models, this improvement could become feasible. while some research has been conducted on sensory experience and conceptual norms for automated metaphor detection, as of the writing of this paper, no study on the impact of adding body-object interaction to neural network models as a feature for metaphor detection has been published. the paper makes the following contributions: the paper is structured as follows: section 2 provides a preview of the existing literature of related works. in section 3, the theories forming the foundation of this study are introduced. section 4 introduces the models and datasets used in the experiments, along with the steps to be followed in section 5 for both models. section 6 presents the analysis and decisions made during the experiments. finally, section 7 provides a conclusion, summarising the paper's findings. ## related work the concept of semantic richness comes from the theory of semantic representation, stating that information is stored and retrieved through an interconnected network of concepts. this network includes features and information contributing to the meaning of each concept @xcite . richer concepts have more semantic information, leading to faster activation, improved processing, and better decisionmaking in the brain @xcite . similar concepts may not evoke the same semantic information, showing varying levels of richness. semantic richness is assessed based on two categories: elements related to the network's strength and elements linked to the perceptual aspect of the network @xcite . while numerous studies have examined strength-related elements in natural language processing (nlp), like the number of features and neighborhood density @xcite @xcite , the elements associated with the perceptual part of the network have received less attention. based on shared information from the environment that senses sensory input (such as taste, sight, sound, etc.), language facilitates a common ground for communication. this idea holds true for both literal and figurative languages, as introduced by @xcite , who attempted to measure the impact of these sensorial elements on metaphor identification using a dependency-parsed corpus of adjective-noun (an) pairs. meanwhile, @xcite tested the conceptual norms as a linguistic enhancement method for metaphor detection of vuamc verbs. however, as of the date of this publication, the concept of body-object interaction has not been researched in association with automated metaphor detection. for the task of metaphor detection, in the hope of better automated detection, it is essential to understand this complex form of language, and these features could facilitate such understanding. ## theories the mind is capable of forming mental images and evoking various sensations when reading or hearing certain words. this ability to trigger sensory and/or perceptual experiences in the mind is known as a sensory experience @xcite . for instance, when the word incense is encountered, the mind may generate a mental picture, and the word fragrance may evoke the actual smell associated with incense. metaphors are a type of language that relies on describing a mental image to represent an abstract concept. they achieve this by using words from a concrete, sensed domain and applying them to another domain. as mentioned in the introduction, @xcite described the conceptual metaphor mapping where the concept of consideration is depicted as a sensed weighing experience. this theory is further developed in @xcite , which suggests that bodily interactions with the environment are projected onto the new conceptual notions of these metaphors. this developed theory aims to explain how conceptual metaphors can be understood even when the direct experiential connection between the source and target domains is lacking, leading to some mappings being vague. for instance, the metaphor "he is hungry for recognition" can be understood by mapping desire is hunger because "food is desired". this physical reaction of hunger is connected to the abstract idea of seeking recognition. based on the discussion above, metaphors create an image-scheme knowledge called the sensory interaction system, where abstract concepts paint mental images of human bodily interactions. words with a high score on the image scale could be used in metaphors where their score is noticeably higher than that of the context, further associated with some degree of human sensory input. the body-object interaction rating reflects how easy it is to interact physically with the word's referent @xcite . the high scale score indicates that the body's interaction with this concept is easier. for example, key was found to be more concrete and perceivable and linked to highlevel sensory, haptic and visual experiences, this is in contrast to the word mountains, which scores low on the scale of body-object interaction with a lesser degree of the characteristics previously mentioned. one can see mountains but cannot interact with them with everyday human physical actions, while one can see, touch, and turn to unlock with keys. within the cognitive sciences, researchers investigate the influence of body-object interaction measurement on various cognitive activities, including word recognition and information acquisition. the theory of the embodied view of cognition, as presented by @xcite , posits that conceptual knowledge is grounded in perceptual interactions with the environment. this means that learning and understanding new concepts are built upon prior knowledge acquired through interactions with the surrounding environment. this observation could be applied to the idea of metaphors, as @xcite 's theory could be extended to the concept of body-object interaction. for instance, in the example 'this movie stinks", it is clear that the statement expresses a negative remark about the movie, based on the known experience that 'stink is bad". in other words, such metaphors can be easily comprehended because the tactile and visual experiences associated with them are akin to what the metaphor is referring to. consequently, body-object interaction measures may aid in translating this knowledge into language that can be used to explain words and concepts. particularly, abstract ideas could be better understood by employing this measure. ## metaphor detection experiments setup this section provides details about the experiments, which consist of two stages. the first stage is the preprocessing stage, where the svm model trained on sen and boi lists assigns prediction scores to all tokens in the metaphorical corpora. in the second stage, the effect of these added predictions on metaphorical tokens is tested using the bistlm and rnn hg models @xcite for both sentence-level and wordlevel metaphor detection. ## conclusion this paper examined the impact of adding sensorimotor knowledge (sensory experience and bodyobject interaction) as external lexical resources to neural network models for the automatic detection of metaphors in text. sensory relies on an image scheme, where a mental image is evoked along with other sensory activations to convey the intended meaning. on the other hand, body-object is derived from concreteness and imageability, which describe how easy it is to interact with a particular entity. both concepts have been extensively studied in fields outside of nlp. the ratings from the lists, as well as the ratings obtained from trained svms, were tested on three metaphorical datasets using two types of deep learning models: one classifies sentences, and the other classifies words as literals or metaphors based on context. the models' performances demonstrated promising results, showing improvements in recall and f1 for metaphor detection across the three datasets. for future work, a more comprehensive breakdown of the variables could be used to acquire the ratings for sensory and body-object lists. this could include making the type of activated sensory more apparent. additionally, considering imageability and concreteness in these tests might help bridge the gap in some of the variations in ratings observed, as mentioned