# Methods of Annotating and Identifying Metaphors in the Field of Natural Language Processing
## 4 Identifying Metaphors
### Metaphor Identification with the Use of Word Embeddings
#### 4.4.2 Metaphor Identification Using Contextual Word Embeddings
In the experimental system, VUA, MOH-X and TroFi datasets were used, while RoBERTa was used for the encoder and contextual word display. The achieved results were an F1 score of 0.790 for VUA all types of words, 0.756 for VUA verbs, 0.847 for MOH-X and 0.745 for TroFi. Metaphor-relation BERT (MrBERT) [51] model, proposed by [PERSON] et al., is focusing on token level verb metaphor detection, by approaching the task as a relation extraction problem. The whole sentence is viewed as a global context, while local context is represented by the words that have close grammatical relation to the target word. Authors, motivated by MIP, see distant context as a basic meaning of the word. These three contexts are then extracted and represented. Final transformer layer is used to get the contextual representation while output from BERT tokenizer is used to get the context independent representation. Contextual relation, relation between the target verb and one of its contexts, is utilized to determine the metaphoricity of the verb. The authors evaluate their model on VUA verb and report F1 score of 0.759, while VUA all F1 score is 0.772. An additional model based on the linguistic theories, MIP and SPV, is MelBERT, proposed by [PERSON] et al. [46]. Architecture of the MelBERT adopts RoBERTa as a contextualized backbone. On input, token, position, and segment embedding are fed to transformer encoder, on top of which MIP and SPV layers are implemented. MIP layer is fed with two embedding vectors representing contextualized embedding vector and isolated embedding vector for the target word. This approach allows for identification of the semantic gap for the target word, in context and isolation. SPV layer, on other hand, is fed only with the sentence encoder, with the assumption that there is a semantic gap between contextual embedding vector of word, and word in a given context. Using these two strategies, hidden vectors hMIP and hSVP are computed and combined to get the prediction score. Finally, the cross-entropy loss function for binary classification is used. For the evaluation, VUA-18 (data set used on first workshop on figurative language processing [81]) and VUA-20 (dataset used on second workshop on figurative language processing [82]) data sets are used, so that results can be compared to the previous research [39; 40; 41]. The authors report the F1 score of 0.757 for VUA verbs, 0.785 on VUA-18 and 0.723 on VUA-20. MIP has also inspired [PERSON] and [PERSON] [50] in proposing the model for conventional metaphor identification with the word sense disambiguation method (WSD), called Metaphorical Polysemy Detection (MPD). They state that detecting novel and conventional metaphors are fundamentally different tasks and argue that metaphoricity is best treated as a word sense in lexicon. Furthermore, they argue that step 1 in MIP is very similar to word sense disambiguation. They build a model which learns to identify metaphorical senses in WordNet and for evaluation manually annotate 250 commonly cited metaphor examples from Master Metaphor List (MML). The same authors also propose a relative metaphoricity measure, with the goal to improve usual measurement (F1, precision and recall) which only gives absolute judgment, metaphorical-literal binary, while metaphor often has edge cases. Relative metaphoricity is calculated using ROC-AUC and allows evaluation whether a model can judge which word senses are more metaphorical than others. Architecture of MPD is a multi-layer perceptron (MLP) which is given as input pair of embeddings for word sense tuple, consisting of wordform and definition. The authors experiment with two WSD models--BERT WSD as a baseline and EWISER [83] as a state-of-the-art approach. BERT WSD model achieves absolute F1 measure of 0.60 for MML, 0.41 for out of the vocabulary words of the training data (OOV), and 0.47 for verbs. Furthermore, the relative metaphoricity measure in their research was 0.78 on MML and 0.64 on OOV with MPD (EWISER WSD), while for verbs was 0.71 with MPD (BERT WSD Baseline).