F rame BERT : Conceptual Metaphor Detection with Frame Embedding Learning (2023)

Abstract: In this paper, we propose FrameBERT, a BERT-based model that can explicitly learn and incorporate FrameNet Embeddings for concept-level metaphor detection. FrameBERT not only achieves better or comparable performance to the state-of-the-art, but also is more explainable and interpretable compared to existing models, attributing to its ability of accounting for external knowledge of FrameNet.

## introduction metaphor is a pervasive linguistic device, which attracts attention from both fields of psycholinguistics and computational linguistics due to the key role it plays in the cognitive and communicative functions of language @xcite @xcite . linguistically, metaphor is defined as a figurative expression that uses one or several words to represent another concept given the context, rather than taking the literal meaning of the expression @xcite . for instance, in the sentence "this project is such a headache!", the contextual meaning of headache is "a thing or person that causes worry or trouble; a problem", different from its literal meaning, "a continuous pain in the head". metaphor detection presents a significant challenge as it necessitates comprehending the intricate associations between the abstract concepts embodied by the metaphoric expression and the related context. recent studies in this field have demonstrated its potential to positively impact various natural language processing (nlp) applications, including sentiment analysis @xcite , metaphor generation @xcite , and mental health care @xcite . different strategies have been proposed for modeling relevant context, including employing limited linguistic context such as subject-verb and verbdirect object word pairs @xcite , incorporating a wider context encompassing a fixed window surrounding the target word (do @xcite , and considering the complete sentential context @xcite . some recent efforts attempt to improve context modelling by explicitly leveraging the syntactic structure (e.g., dependency tree) of a sentence in order to capture important context words, where the parse trees are typically encoded with graph convolutional neural networks @xcite . despite the progress, we also observe the inadequacy of existing models in semantic modelling, which plays a crucial role in metaphor detection. for instance, it has been noted that bert's tendency to aggregate shallow semantics instead of precise meaning, as its objective, may limit the context modelling ability @xcite . external knowledge such as framenet has been widely used to provide extra semantic information and has been shown useful in a wide range of nlp tasks, such as question answering @xcite , machine reading comprehension @xcite , and identifying software requirements @xcite . very recently, framenet has also been employed to the task of metaphor generation via learning mappings between domains, with promising results achieved @xcite . however, such a valuable source of knowledge, surprisingly, has not been explored in the deep learning literature for metaphor detection. we hypothesise that incorporating external knowledge of concepts is essential for improving metaphor detection and model explainability. in this paper, we propose framebert, a bertbased model for conceptual metaphor detection underpinned by learning and incorporating embedding representation of semantic frames in framenet. framebert directly addresses the limitation of the existing works, which solely rely on the shallow semantics captured by hand-crafted psycholinguistics features or encoded by large pretrained language models such as bert. this is achieved by explicitly learning and incorporating framenet embeddings into the model training process. to our knowledge, this is the first attempt to apply framenet in deep learning models for metaphor detection. we also leverage metaphor identification procedure @xcite and selectional preference violation (spv) @xcite to inform our model design. extensive experiments conducted on four public benchmark datasets (i.e., vua moh-x, trofi) show that framebert can significantly improve metaphor detection for all datasets compared to our base model without exploiting framenet embeddings. our model also yields better or comparable performance to state-of-the-art models in micro f1 measure. furthermore, we show the explainable feature of framebert, attributing to its ability of extracting semantic frames from text. the code and dataset can be found at https://github. com/liyucheng09/metaphorframe . ## model we propose framebert, a novel model that can explicitly learn and incorporate framenet embeddings for concept-level metaphor detection. figure 1 illustrates the overall architecture of frame-bert, which consists of two components: a sentence encoder ( § 2.1) and a concept encoder ( § 2.2). ## experimental results overall results. our model outperforms all the baseline models on vua-20, including the state-of-the-art-model melbert (with @xmath0.05 for a two-tailed t-test). for vua-18, we outperformed all the baselines significantly including the re-produced results for melbert. rand_in_eval represents the first experiment where the shuffle process is only performed in evaluation stage and rand_in_train_&_eval represents the second experiment where the shuffle process is performed in both training and evaluation stages. in w/o frame fine-tuning experiment, we remove the frame fine-tuning process. ## conclusion we proposed framebert, the first conceptual model for metaphor detection by explicitly learning and incorporating framenet embeddings for concept-level metaphor detection. extensive experiments show that our model can yield better or comparable performance to the state-of-the-art.