Can Yes-No Question-Answering Models be Useful for Few-Shot Metaphor Detection? (2022)

Abstract: Metaphor detection has been a challenging task in the NLP domain both before and after the emergence of transformer-based language models. The difficulty lies in subtle semantic nuances that are required to detect metaphor and in the scarcity of labeled data. We explore few-shot setups for metaphor detection, and also introduce new question answering data that can enhance classifiers that are trained on a small amount of data. We formulate the classification task as a question-answering one, and train a question-answering model. We perform extensive experiments for few shot on several architectures and report the results of several strong baselines. Thus, the answer to the question posed in the title is a definite “Yes!”

## introduction in the past year, pretrained language models established themselves as the foundation for stateof-the-art solutions for most of the common nlp tasks. usually, one should fine tune a model on a dataset specific to her task and domain so as to achieve high performance, and this requires labeled data, which is not always available in the necessary quantity. in the past few years, a large body of work has been dedicated to transfer learning between domains and models @xcite , and application of models trained on one task to another task by prompting @xcite . these techniques reduce the amount of training data needed for a specific task, and enable the sharing of semantic knowledge between models. metaphor detection is a highly challenging task in the nlp domain. it relies on word level, delicate semantics that are not trivial even for humans, and, thus, even though pretrained language models do encode some metaphoric information @xcite , the task is not considered solved. as for languages other than english -high quality language models are already often available @xcite , but metaphor detection without appropriate labeled data is very difficult @xcite , and this is why few-shot is a relevant scenario to study. as @xcite suggest, metaphor detection can be viewed as a reading-comprehension task where one needs to answer a question whether a specific word is metaphoric or literal in the context of a given sentence. they formatted metaphor detection as a classification task of the full sentence (global context), the word in question and a short sentence fragment that contains this query word (local context). the texts, along with pos tags of each word, are fed into the classifier to obtain a prediction. similar to the vast majority of classification tasks, this classifier is expected to learn how to identify metaphors based on the labels it is provided during training, but the input itself does not suggest that the task is regarding metaphor. we take the reading-comprehension approach further in two respects: first, we experiment with several phrasings of explicit natural-language questions about whether the query token is metaphoric within the context of the sentence. thus we employ the capability of large language models to understand delicate semantics (at least up to some point) by querying the models directly. second, we design our classifier with a backbone of a yesno question-answering model. given the context sentence, we ask explicitly, "is the word in question metaphoric in this context?" we evaluate our model in a few-shot scenario and compare it to several baselines. ## related work over the past few years, as in other fields in nlp, transformer-based architectures have dominated the models for metaphor detection. leong et al. (2020) report the results of the 2020 shared task, and can be referred to for prior models that are not transformer based. deepmet @xcite , the highest-scoring system in that shared task, transforms metaphor detection into a reading comprehension task, querying for the label of each token given its context in the sentence. the classification model is a siamese network that encodes two contexts for the token -the entire sentence and the sentence fragment wherein the token is located. the model is also fed with the pos tag of the token in question. melbert @xcite is a transformerbased model that applies two theoretical concepts of metaphor identification: (1) a metaphor's literal meaning is different from its metaphorical meaning in the sentence. (2) the metaphor is unusual in the context of the sentence. mrbert @xcite employs a similar architecture to melbert, adding the encoded grammatical local context of the query token. few-shot learning refers to learning from a small number of training examples. one few-shot technique for nlp is pattern exploiting training (pet) @xcite over the roberta architecture. pet, requiring task-specific unlabeled data, uses natural language patterns to represent the inputs as cloze style questions. answers are then filled in by the predictions of the language model. adapet @xcite extends pet by providing denser supervision during fine-tuning, outperforming pet without the need of unlabeled data. gpt-3 @xcite takes few-shot abilities forward and demonstrates strong performance without directly fine-tuning on task-specific data. instead, in the few-shot scenario, at inference time it is presented with a few labeled instances as a part of the query. ## metaphor detection model metaphor detection can be regarded as a tokenclassification task within a sentence. the word in question in a given sentence can be classified either as metaphoric or literal. in this work, we experiment with the formulation of metaphor detection as a yes-no question answering (qa) task with two concatenated inputs: a question and a passage, that is, a text segment to which the question refers. for each word in question, we suggest several different constructions of questions and passages. these formulations are shown in our suggested architecture for metaphor detection is presented in figure . 1. we begin by finetuning a roberta base model @xcite on qa data (see section 4.1). next, this model is fine-tuned on different sizes of metaphor data, phrased as questions. the results are compared to the roberta base model and to deepmet. since we are aiming to analyse the advantages of the qa model in a fewshot scenario, rather than to outperform the state of the art, our baseline models are ones that are similar in terms of architecture and additional resources. training on the entire vua dataset we are experimenting with, the roberta baseline achieves the f1 score of 71.4, while melbert, the current state of the art, attains an f1 of 72.3. ## conclusions and future work qa-based models were shown to be effective for metaphor detection when training data is very limited. we analyzed the contribution of the questionlike phrasing and the underlying qa model, and report strong baselines for the few-shot scenario. another contribution is the use of wordnet. transformer-based language models are pretrained on unlabeled data, thus many linguistic resources that were previously extensively used are less needed now. we have shown how the high-quality annotated data from wordnet can be utilized to train a qa model that can answer questions about semantics. we believe that similar techniques can generate high-quality datasets for training models for other nlp tasks. as future work, we suggest exploring natural language inference models as underlying models for metaphor detection. those models have been shown to be strong zero-shot models for various nlp tasks, so they can probably be of assistance in the metaphor domain. another direction we aim to explore is the combination of our qa based technique with models such as deepmet and melbert.