Neural Metaphor Detection with Visibility Embeddings (2021)

Abstract: We present new results for the problem of sequence metaphor labeling, using the recently developed Visibility Embeddings. We show that concatenating such embeddings to the input of a BiLSTM obtains consistent and significant improvements at almost no cost, and we present further improved results when visibility embeddings are combined with BERT.

## introduction when browsing through vision-language datasets, one can make the intuitive observation that their textual parts ("visual corpora") contain more physical language, mostly descriptive, which tends to be non-metaphorical by nature (see, for example, typical images from the visual genome dataset in figure 1 ). recently, this property was used to build visibility embeddings, which aim to provide a good estimation of a word's concreteness, a feature that has been long related to metaphoricity @xcite . many metaphors indeed involve noticeable differences between the abstractness of words constructing them, like "clean conscience" (vs. "clean air"). metaphors are not created in isolation, commonly do not stand alone as non-literal expressions, and are highly context-dependent in nature. even the most concrete and physical text can be considered as metaphorical when mentioned in a different context than its original one, or in proximity to another text from the target domain. for example, a single use of a verb like "push" or "leak" can have both literal and metaphorical meanings, in relation to its context (see figure 1 ). technically, the task of metaphor detection at the sentence level is commonly approached as one of the following two tasks: (1) sequence labeling, in which each token in the sentence is classified as either "metaphorical" or figure 1 : images from the visual genome @xcite along with their literal ("l") description, and a metaphorical ("m") sentence with a similar verb from the moh-x dataset @xcite (concrete words in green and abstract words in red). recently, kehat and pustejovsky (2020) presented the simply constructed visibility embeddings (ve), which use references to visual/nonvisual corpora to estimate word concreteness, and applied it to the task of verb-classification. in this paper we apply ve also to the sequence labeling task, and show how they consistently improve the result of a bilstm model with bert. we also discuss possible problems when reporting results on very small annotated datasets, and the effect on adding glove to the model input. ## model details as a base structure we use the simple bilstm architectures presented by @xcite . the sequence labeling model (see figure 2 ) consists of two layers, a bilstm and a feedforward layer, to get a label for each word in the sentence. we implemented the model in python using the allennlp package @xcite . we use a pretrained bert model provided by the allennlp package, with 24 layers and 1024 hidden states, trained on cased english text. the input vector for the model consists of the concatenation of the 1024-dimensions bert vector (using all the layers of the bert model), the glove embeddings @xcite (not in all cases, see discussion in section 4.3), and the ve of varied length (we experimented with vectors from length 50 and 300). hyperparameters are fine-tuned on each dataset. ## experiment setting and results we present results and comparison for two of the most common datasets for metaphor detection: vua @xcite and moh-x @xcite . annotated datasets for the validation and training of metaphor detection systems are not easily created, and require a level of expertise. the available datasets are therefore relatively small, hand crafted sets of several hundreds to a few thousands sentences, mostly only partially annotated for the metaphoricity of their main verb. as a result, the f1-scores vary highly, even with the slight change in parameters. in order to provide a consistent evidence to our algorithm's performance, we chose to compare not only the maximal f1-scores gained by each model, but also present a "parameterized" f1-score, over different learning rates. this would allow us to analyze the results while ignoring very highly-frequent fluctuations in the performance of the models. ## summary we have presented new and improved results for sequence metaphor labeling for the vua and moh-x datasets using visibility embeddings and bert as inputs for a simply constructed bilstm. we provided detailed comparison for the effect of adding ve to the model, and showed it to be a useful no-cost component to a metaphor detection system.