Adversarial Multi-task Learning for End-to-end Metaphor Detection (2023)

Abstract: Metaphor detection (MD) suffers from limited training data. In this paper, we started with a linguistic rule called Metaphor Identification Procedure and then proposed a novel multi-task learning framework to transfer knowledge in basic sense discrimination (BSD) to MD. BSD is constructed from word sense disambiguation (WSD), which has copious amounts of data. We leverage adversarial training to align the data distributions of MD and BSD in the same feature space, so task-invariant representations can be learned. To capture fine-grained alignment patterns, we utilize the multi-mode structures of MD and BSD. Our method is totally end-to-end and can mitigate the data scarcity problem in MD. Competitive results are reported on four public datasets. Our code and datasets are available.

## introduction metaphor involves a mapping mechanism from the source domain to the target domain, as proposed in conceptual metaphor theory @xcite . smash in the above sentence means "hit hard" literally (source domain). however, it is employed in a creative way, indicating "overthrow or destroy" (target domain). the mapping from the source to the target makes the word a metaphor. understanding metaphors in human languages is essential for a machine to dig out the underlying intents of speakers. thus, metaphor detection and understanding are crucial for sentiment analysis @xcite , and machine translation @xcite , etc. metaphor detection (md) requires a model to predict whether a specific word is literal or metaphorical in its current context. linguistically, if there is a semantic conflict between the contextual meaning and a more basic meaning, the word is a metaphor @xcite @xcite . the advent of large pretrained language models has pushed the boundaries of md far ahead @xcite . however, md suffers from limited training data, due to complex and difficult expert knowledge for data annotation @xcite . recently, @xcite used self-training to expand md corpus, but error accumulation could be a problem. many researchers used various external knowledge like part of speech tags @xcite , dictionary resources @xcite , dependency parsing @xcite , etc., to promote md performance. these methods are not end-to-end, thus they impeded continuous training on new data. to address the data scarcity problem in md, we propose a novel task called basic sense discrimination (bsd) from word sense disambiguation (wsd). bsd regards the most commonly used lexical sense as a basic usage, and aims to identify whether a word is basic in a certain context. both bsd and md need to compare the semantic difference between the basic meaning and the current contextual meaning. despite the lack of md data, we can distill knowledge from bsd to alleviate data scarcity and overfitting, which leads to the usage of multi-task learning. we design the adversarial multi-task learning framework (admul) to facilitate the knowledge transfer from bsd to md. admul aligns the data distributions for md and bsd through adversarial training to force shared encoding layers (for example, bert) to learn task-invariant representations. furthermore, we leverage the internal multi-mode structures for fine-grained alignment. the literal senses in md are forcibly aligned with basic senses in bsd, which can push the literal senses away from the metaphorical. similarly, the non-basic senses in bsd are aligned with metaphors in md, which enlarges the discrepancy between basic and non-basic senses to enhance model performance. the contributions of this paper can be summarized as follows: our model is free of any external resources, totally end-to-end, and can be easily trained. metaphor detection: metaphor detection is a popular task in figurative language computing @xcite . with the progress of natural language processing, various methods have been proposed. traditional approaches used different linguistic features like word abstractness, word concreteness, part of speech tags and linguistic norms, etc., to detect metaphors @xcite @xcite . these methods are not end-to-end and rely heavily on feature engineering. the rise of deep learning boosted the advancement of metaphor detection significantly. @xcite , @xcite and @xcite used rnn and word embeddings to train md models. recently, lots of works combined the advantages of pre-trained language models and external resources to enhance the performance of metaphor detection @xcite @xcite @xcite . though great improvements have been made, these models still suffer from the lack of training data, which is well exemplified by their poorer performance on small datasets. multi-task learning: multi-task learning (mtl) can benefit a target task via related tasks. it has brought great success in computer vision and natural language processing. mtl learns universal representations for different task inputs, so all tasks share a common feature space, where knowledge transfer becomes possible. previous studies trained mtl models by deep neural networks like cnn or rnn, achieving promising results in text classification @xcite . @xcite al. (2019a) and clark et al. (2019) combined mtl framework with bert @xcite , obtaining encouraging results on multiple glue tasks. there are some other successful mtl applications in machine translation @xcite , information extraction @xcite , and sentiment analysis @xcite , etc. @xcite applied mtl to study the interplay of metaphor and emotion. @xcite combined wsd and md for better metaphor detection results. however, to the best of our knowledge, we are the first to use adversarial mtl for metaphor detection based on the linguistic nature of metaphors. ## conclusion in this paper, we proposed admul, an adversarial multi-task learning framework for end-to-end metaphor detection. admul uses a new task, basic sense discrimination to promote md, achieving promising results on several datasets. the zero-shot results even surpass the previous fine-tuned stateof-the-art method. the ablation study demonstrates that the strong ability of admul comes not only from the pre-trained language model, but also from our adversarial multi-task learning framework.