Label-Enhanced Hierarchical Contextualized Representation for Sequential Metaphor Identification (2021)

Abstract: Recent metaphor identification approaches mainly consider the contextual text features within a sentence or introduce external linguistic features to the model. But they usually ignore the extra information that the data can provide, such as the contextual metaphor information and broader discourse information. In this paper, we propose a model augmented with hierarchical contextualized representation to extract more information from both sentence-level and discourse-level. At the sentence level, we leverage the metaphor information of words that except the target word in the sentence to strengthen the reasoning ability of our model via a novel label-enhanced contextualized representation. At the discourse level, the position-aware global memory network is adopted to learn long-range dependency among the same words within a discourse. Finally, our model combines the representations obtained from these two parts. The experiment results on two tasks of the VUA dataset show that our model outperforms every other state-of-the-art method that also does not use any external knowledge except what the pre-trained language model contains.

## introduction metaphor is a type of figurative language and its essence is understanding and experiencing one kind of thing in terms of another @xcite . as a common language expression, we often use metaphors to express our thoughts vividly and concisely in daily communication. for example, in the sentence it is one of the keys for success of a commercial product, keys is used to help understand the importance of it. however, this characteristic of metaphor makes it challenging to identify metaphors in texts. but the identification * both authors contributed equally to this research † corresponding author of metaphors is meaningful and can help us to understand the meaning of the texts, from which many downstream applications such as machine translation @xcite and opinion mining @xcite can benefit. recent metaphor researches @xcite , and acl 2020 metaphor shared task @xcite regard it as a sequence labeling task. although many previous works have explored ways to enhance the contextualized representation within a sentence @xcite , or to introduce some external knowledge @xcite @xcite , most of them do not make full use of the information in the dataset, from which the metaphor identification process may benefit. firstly, when considering the metaphoricity of the target word, the metaphor information of other words in the sentence can also be helpful. e.g., in the sentence he find himself in the position of the gambler who gambled all and lost, gambler and gambled are metaphors. the word gambled is the action of the gambler, and it's reasonable for a gambler to gamble. thus, the model might prefer to classify gambled as literal. however, if we know that gambler is a metaphoric word, then it is obvious that gambled is also a metaphoric word that refers to the risky thing he did. based on this observation, we propose a novel label-enhanced contextualized representation method to introduce the contextual metaphor information. it embeds the label of each word(i.e. metaphoric or literal) in the same space as the output of the encoder first and then takes both the output of the encoder and the label embedding as the input of a transformer @xcite . we believe it could enhance the reasoning ability of the model by attending to metaphor information of other words in the sentence. besides, marking the metaphoric words in context could also help the target word understand the context better, especially in the com-figure 1 : an example that shows the two occurrences of word gambled within a discourse plicated sentence, because metaphorical words are not used as their literal meaning, increasing the difficulty of understanding the context. to the best of our knowledge, we are the first that proposes this method. secondly, some existing benchmark metaphor datasets, such as vuamc, contain sentences from long articles, and the contextual information in the articles will be very useful for metaphor identification. some previous work used paragraph embedding @xcite or neighbouring sentence representation @xcite . based on this, we use a discourse-level attention architecture that could capture both global and local features in the whole discourse for the target word. first, we introduce the work of @xcite to extract local information. then, we propose an improved method of global attention @xcite , which is called position-aware global memory network, to represent global information of the target word. it is based on the observation that a metaphor brings another domain/frame into the discourse, so it is likely that metaphors mapping to the same domain/frame reoccur throughout the discourse, especially among the same words. specifically, our model uses an attention mechanism between the target word and its other occurrences in the discourse. figure 1 shows the two occurrences of word gambled in two sentences within a discourse. based on the above sentence-level and discourselevel methods, we propose a novel hierarchical contextualized representation model for metaphor identification, as shown in figure 2 . to verify the effectiveness of our model, we conduct experiments on the all pos and verbs tasks of the vu amsterdam metaphor corpus (vua) @xcite . our model outperforms several baseline models with 1.1% (vua all pos) and 1.0% (vua verbs) improvement in f1 score. in addition, the results of our model surpass deepmet @xcite , which is the state-of-the-art model in metaphor identification, with the same experiment setup. our contributions in this paper can be summa-rized as follows. ## conclusion in this paper, we propose a hierarchical contextualized representation model to strengthen the model's ability to leverage contextual information. our model makes use of the contextual metaphor information in the sentence level and the long-range relation of the words in the discourse level. we improve the ability of the model to reason the contextual metaphoric relationships and understand the meaning of context by introducing contextual label representation for the target word. to obtain broader discourse information, we adopt a position-aware global memory network to extract relations among the occurrences of words in discourse. the results of our model on the two tasks of vua dataset surpass the state-of-the-art models which also do not use external knowledge. in future work, we will explore changing the golden label used in the label embedding stage to the iterative prediction result, which may avoid the deviation caused by the absence of golden labels during testing. meanwhile, albeit limited, the work of @xcite could provide further direction for this research, such as using words belonging to the same topic/frame/domain instead of only the same words.