Verb Metaphor Detection via Contextual Relation Learning (2021)

Abstract: Correct natural language understanding requires computers to distinguish the literal and metaphorical senses of a word. Recent neu- ral models achieve progress on verb metaphor detection by viewing it as sequence labeling. In this paper, we argue that it is appropriate to view this task as relation classification between a verb and its various contexts. We propose the Metaphor-relation BERT (Mr-BERT) model, which explicitly models the relation between a verb and its grammatical, sentential and semantic contexts. We evaluate our method on the VUA, MOH-X and TroFi datasets. Our method gets competitive results compared with state-of-the-art approaches.

## introduction metaphor is ubiquitous in our daily life for effective communication @xcite . metaphor processing has become an active research topic in natural language processing due to its importance in understanding implied meanings. this task is challenging, requiring contextual semantic representation and reasoning. various contexts and linguistic representation techniques have been explored in previous work. early methods focused on analyzing restricted forms of linguistic context, such as subjectverb-object type grammatical relations, based on hand-crafted features @xcite @xcite . later, word embeddings and neural networks were introduced to alleviate the burden of feature engineering for relation-level metaphor detections @xcite . however, although grammatical relations provide the most direct clues, other contexts in running text are mostly ignored. recently, token-level neural metaphor detection draws more attention. several approaches discov- * these authors contributed equally to this work. ered that wider context can lead to better performance. do @xcite considered a fixed window surrounding each target token as context. @xcite and @xcite argued that the full sentential context can provide strong clues for more accurate prediction. some recent work also attempted to design models motivated by metaphor theories @xcite . despite the progress of exploiting sentential context, there are still issues to be addressed. first of all, a word's local context, its sentential context and other contexts should be all important for detecting metaphors; however, they are not well combined in previous work. more importantly, as shown in figure 1 , most token-level metaphor detection methods formulate metaphor detection as either a single-word classification or a sequence labeling problem @xcite . the context information is mainly used for learning contextual representations of tokens, rather than modeling the interactions between the target word and its contexts @xcite . in this paper, we focus on token-level verb metaphor detection, since verb metaphors are of the most frequent type of metaphoric expressions @xcite . as shown in figure 1 , we propose to formulate verb metaphor detection as a relation extraction problem, instead of token classification or sequence labeling formulations. in analogy to identify the relations between entities, our method models the relations between a target verb and its various contexts, and determines the verb's metaphoricity based on the relation representation rather than only the verb's (contextual) representation. we present a simple yet effective model -metaphor-relation bert (mrbert), which is adapted from a bert @xcite based state-of-the-art relation learning model (bal-sentence encoder dini @xcite . our model has three highlights, as illustrated in figure 2 . first, we explicitly extract and represent context components, such as a verb's arguments as the local context, the whole sentence as the global context, and its basic meaning as a distant context. so multiple contexts can be modeled interactively and integrated together. second, mrbert enables modeling the metaphorical relation between a verb and its context components, and uses the relation representation for determining the metaphoricity of the verb. third, the model is flexible to incorporate sophisticated relation modeling methods and new types of contexts. we conduct experiments on the largest metaphor detection corpus vu amsterdam metaphor corpus (vua) @xcite . our method obtains competitive results on the large vua dataset. detail analysis demonstrates the benefits of integrating various types of contexts for relation classification. the results on relatively small datasets, such as moh-x and trofi, also show good performance and model transferability. ## formulating verb metaphor detection this section briefly summarizes the common formulations of token-level verb metaphor detection as a background, and discusses the relation between this paper and previous work. the task a given sentence contains a sequence of n tokens @xmath0, ..., x n , and a target verb in this sentence is x i . verb metaphor detection is to judge whether x i has a literal or a metaphorical sense. basic formulations most neural networks based approaches cast the task as a classification or sequence labeling problem (do @xcite . as shown in figure 1 , the classification paradigm predicts a single binary la-bel to indicate the metaphoricity of the target verb, while the sequence labeling paradigm predicts a sequence of binary labels to all tokens in a sentence. based on the basic formulations, various approaches have tried to enhance feature representations by using globally trained contextual word embeddings @xcite or incorporating wider context with powerful encoders such as bil-stm @xcite and transformers @xcite . limitations and recent trends however, the above two paradigms have some limitations. first, contextual information is mostly used to enhance the representation of the target word, but the interactions between the target word and its contexts are not explicitly modeled @xcite . to alleviate this, @xcite proposed a new paradigm by viewing metaphor detection as a reading comprehension problem, which uses the target word as a query and captures its interactions with the sentence and clause. a concurrent work to this work @xcite adopted a pre-trained contextualized model based late interaction mechanism to compare the basic meaning and the contextual meaning of a word. second, exploiting wider context will bring in more noise and may lose the focus. fully depending on data-driven models to discover useful contexts is difficult, given the scale of available datasets for metaphor detection is still limited. the grammar structures, such as verb arguments, are important for metaphor processing @xcite , but is not well incorporated into neural models. @xcite showed that data augmentation based on syntactic patterns can enhance a standard model. @xcite adopted graph convolutional networks to incorporate dependency graphs, but did this paper presents a new paradigm for verb metaphor detection to overcome these limitations, by viewing the task as a relation extraction task. we assume a target verb and its multiple contexts are entities, and metaphor detection is to determine whether a metaphorical relation holds between the verb and its contexts. we will introduce the proposed model in section 3. before diving into details, we argue that viewing metaphor as a relation is reasonable and consistent with existing metaphor theories. according to @xcite , metaphors show a violation of selectional preferences in a given context. the conceptual metaphor theory views metaphors as transferring knowledge from a familiar, or concrete domain to an unfamiliar, or more abstract domain @xcite . the metaphor identification procedure (mip) theory @xcite aims to identify metaphorically used words in discourse based on comparing their use in particular context and their basic meanings. all the theories care about a kind of relations between a target word and its contexts, which may help identify metaphors. ## metaphor-relation bert (mrbert) we propose the metaphor-relation bert (mr-bert) model to realize verb metaphor detection as a relation classification task. figure 2 shows the architecture of mrbert. we use the pre-trained language model bert as the backbone model. there are three main procedures: (1) extract and represent contexts; (2) model the contextual relations between the target verb and its contexts; (3) manipulate the contextual relations for predicting the verb's metaphoricity. ## related work metaphor detection is a key task in metaphor processing @xcite . it is typically viewed as a classification problem. the early methods were based on rules @xcite , while most recent methods are data-driven. next, we summarize data-driven methods from the perspective of context types that have been explored. ## conclusion this paper presented the metaphor-relation bert (mrbert) model for verb metaphor detection. we propose a new view to formulate the task as modeling the metaphorical relation between the target verb and its multiple context components, i.e., contextual relations. we propose and evaluate various ways to extract, model and integrate contextual relations for metaphoricity prediction. we conduct comprehensive experiments on the vua dataset. the evaluation shows that mrbert achieves superior or competitive performance compared with previous methods. we also observe that incorporating grammatical relations can help balance local and global contexts, and the basic meaning of the verb as a distant context is effective. further experiments on small datasets moh-x and trofi also show good model transferability of mrbert.