An Expectation-Realization Model for Metaphor Detection (2024)

Abstract: We propose a new model for metaphor detection in which an expectation component estimates representations of expected word meanings in a given context, whereas a realization component computes representations of target word meanings in context. We also introduce a systematic evaluation methodology that estimates generalization performance in three settings: within distribution, a new strong out of distribution setting, and a novel out-of-pretraining setting. Across all settings, the expectation-realization model obtains results that are competitive with or better than previous metaphor detection models.

## introduction and motivation metaphors enhance the communicative aspects of language by connecting concepts from new domains, often abstract, with more familiar ones, usually concrete @xcite . metaphorical expressions have many uses, from helping frame an issue in order to emphasize some aspects of reality @xcite , to creating a strong emotional effect @xcite . the ubiquity of metaphors means their computational treatment @xcite has received significant attention in the nlp community, as surveyed by @xcite and more recently @xcite . owing to its important communicative function, metaphorical expression detection has been approached over the years using a wide variety of nlp techniques, ranging from models employing hand-engineered features @xcite , to rnns @xcite , to more recently pre-trained language models @xcite @xcite , to mention just a few. recent state of the art models for metaphor detection rely on the metaphor identification procedure (mip) @xcite , according to which metaphors happen whenever the contextual meaning of a word is different from its basic, literal meaning. implementations of mip vary mainly in how they estimate representations of the basic meaning of a word: melbert @xcite uses simply the bert embedding of the word without any context, whereas basicbert and ba-sicmip @xcite use an average of all literal uses of the word as marked in the training data. in this paper we propose a new theory of metaphor identification, the expectation-realization model, that is motivated by the observation that the metaphorical use of a word, i.e. its realization in context, leads to surprise due to a violation of a literal word expectation engendered by the same context. surprise offers a general mechanism through which stories and music trigger emotion @xcite , and correlates with creative uses of language, such as humor and metaphor (bunescu and uduehi, 2022). correspondingly, we propose an architecture that is structured around two modules: one module aims to estimate the literal meaning expectation through the use of a context where the target word is masked, whereas the other module aims to estimate the realized meaning of the target word as used in context. the new model is competitive with previous soa in terms of within distribution (wid) generalization. we further propose two new evaluation scenarios: a strong out-of-distribution (ood) setting that ensures target lexemes do not appear during training, and a novel out-of-pretraining (oop) setting that aims to ensure that the metaphorical phrase was not seen during pretraining. the large gap between oop and wid results elucidates why pretrained lms struggle with metaphor identification. ## the expectation-realization model the architecture of the expectation-realization (er) model for metaphor detection is shown in of the target word in context, a copy of the transformer encoder of a pre-trained language model (shown on the left) processes the input text s where the target word at position t is marked with a special token. to compute the expectation (e) of the literal meaning v m,t induced by the context, the same pre-trained language model (shown on the right) process the same input text m where the target word is masked. additionally, global expectation v m and realization v s representations are also computed at the sentence level using the embeddings for the special [cls] token. the concatenation of the local target word er embeddings and the sentence-level er embeddings are passed through non-linear layers f and g, respectively, to capture interactions between expectation and realization embeddings at word-level as h @xmath0,t ; v s,t ], and at sentence level as h @xmath1; v s ]. to enable a fair comparison with previous models, we instantiate the pre-trained transformer encoder using roberta base @xcite . the concatenated local and global er representations are then used as input features to a logistic regression model that estimates the probability ŷ that the target word is used metaphorically. the er model parameters together with the pretrained lm parameters are trained and fine-tuned, respectively, in order to minimize a loss function sim that contains a cross-entropy loss l i ce and a similarity loss l i sim computed as: ## experimental evaluation we run evaluations on three english metaphor datasets: the vua-18 amsterdam metaphor corpus @xcite , trofi @xcite and lcc @xcite . metaphoricity score of 3 are considered as positive and examples with metaphoricity score of 0 as negatives. the trofi dataset consists of a collection of literal and nonliteral usage of 50 verbs which occur in 3,737 sentences selected from the wsj corpus. for the evaluations on vua-18 dataset, we use the same hyperparameter settings from @xcite for training all models. for the lcc and trofi experiments, the development dataset was used for determining the best hyperparameter settings. we use the same hyperparameter settings for all the models. the batch size and max sequence length were set at 32 and 150, respectively. we train for 12 epochs without dropout, and linearly increase the learning rate from 0 to 5e-5 in the first two epochs, after which we decreased it linearly to 0 during the remaining 10 epochs. the tuned similarity weights α 1 and α 2 were 1.0. for the within-distribution experiments and 0.0 for outof-distribution experiments. results are averaged over 5 runs with different random seeds. the detailed ranges used for hyperparameters tuning are presented in appendix a. given that vua-18 is the only dataset on which all 3 metaphor-detection baselines were previously evaluated, we use it to compare their performance against er. as shown in ## conclusion and future work we introduced a new model for metaphor detection rooted in the hypothesis that non-literal uses of words trigger surprise, or violation of expectations given by the context. we further proposed two new evaluation scenarios: strong out-of-distribution and out-of-pretraining. extensive experiments show that the simple er model is competitive with, and often outperforms, state-of-the-art models. in this work, expectations of literal meaning were computed based on context words. in future work, we plan to also compute expectations of literal meanings of words by leveraging large amounts of text where words are known to be used literally, such as descriptions of physical, concrete concepts in wikipedia. furthermore, we plan to generalize the er approach from word-level metaphors to phrase-level constructions, such as idioms, which too violate expectations of literal language use.