# Methods of Annotating and Identifying Metaphors in the Field of Natural Language Processing
## 4 Identifying Metaphors
### Metaphor Identification with the Use of Word Embeddings
#### 4.4.2 Metaphor Identification Using Contextual Word Embeddings  
To the best of our knowledge, first research on metaphor identification using a contextual word embedding is that of [PERSON] et al. [39] in which authors use ELMo word embedding. The authors approach the identification in two ways--by sequence labeling where each word is annotated either as a metaphor or as a literal use, while the second approach is a classification annotating only the verb in the sentence as a metaphor or a literal use. In both cases, the authors use a bidirectional LSTM network and add an attention mechanism layer to the classification model. They evaluate the models on three existing datasets--TroFi, MOH-X and VUA. The use of the ELMo contextual word embeddings in ablation study, as the authors state, significantly affects the results of both models, which are better when ELMo is used. It has a particularly positive impact on the sequence labeling model, where in the evaluation of the model on the VUA data set, the F1 score is 0.617 without using ELMo and 0.704 when using ELMo, while the accuracy is 0.782 without using ELMo and 0.835 with ELMo in identifying metaphors with verbs. The authors compare the results obtained by their model with the results of [PERSON] et al. [38], whose model had at the time been the most successful, but the model of [PERSON] et al. surpassed that of [PERSON] et al. It should also be noted that [PERSON] et al. feed their model data, i.e., text, that had previously been lemmatized, and they also feed it information about the type of words, which [PERSON] et al. had demonstrated has a positive influence on the results. Furthermore, [PERSON] et al. [40] presented the hypothesis that with the use of linguistic metaphor theories, when designing the architecture of deep neural network, one can improve the results of metaphor identification. The two linguistic methods on which they base their models are MIP and selectional preferences (SPV). The MIP-based model (RNN_HG) approaches metaphor classification based on the difference between the literal meaning of the word and the contextual one. For the representation of the contextual meaning, the model uses a bidirectional LSTM neural network (BiLSTM), while for the representation of the literal meaning, GloVe and ELMo word embeddings are used. The SPV-based model (RNN_MHCA) compares the target word with the context, using BiLSTM to represent the target word, while the attention mechanism is used for the context. The models were evaluated on three datasets, VUA, MOH-X and TroFi, and the results were compared with those of [PERSON] et al. [38] and [PERSON] et al. [39]. As an additional model, they modified the approach of [PERSON] et al. by using BERT instead of ELMo. The RNN_MHCA model achieved a better F1 score on the MOH-X (0.80), TroFi (0.724) and VUA dataset with all types of words (0.743), while RNN_HG achieved better results on the VUA dataset containing only verbs (0.708). That approaching the metaphor identification as a reading comprehension problem is feasible and that this gives good results, was shown by [PERSON] et al. [41]. The authors also show that information about the global and local context, as well as POS information, contribute to a better result. The reading comprehension paradigm requires three sentences, a word that needs to be understood, and an indication of whether the word is a metaphor. The model uses BERT in the embedding layer, while the Transformer neural network in two layers is used as the basis of the model. One layer is fed the global text context, the searched word, POS and FGPOS at the input while the other layer is fed the local text context, the searched word, POS, FGPOS, and the properties are separated by a special separation token [SEP]. In the training process, the RoBERTa word embedding is used, with the aim of fine-tuning the Transformer layers.