# Methods of Annotating and Identifying Metaphors in the Field of Natural Language Processing
## 3 Annotating Metaphors
### Annotating Linguistic Metaphors
Team B was instructed to annotate metaphors, including cases when \"something abstract is represented as something physical, something as something abstract, something physical is represented as something physical, something physical is represented as something abstract.\" In the annotation process, a number of parameters were recorded, such as whether it is a novel or conventional metaphor, the level of certainty that it is a metaphor (on a scale from \(-2\) to \(2\)), and the metaphors were linked to concepts in the Master Metaphor List [20], if the concept does not exist in the list, the annotators defined a new concept. Trope Finder (TroFi) data set that contains sentences in which 50 selected verbs appear, in their literal and figurative meanings, was created by [PERSON] and [PERSON] [22]. This data set was created as a part of their research of clustering method for distinguishing the literal and figurative meaning of verbs. Data set contains 6435 sentences, out of which 2740 are metaphorical. Despite the fact that the authors emphasize that the TroFi is not a metaphor recognition system, the data set created as part of this paper is publicly available ([[https://natlang.cs.sfu.ca/software/trofi.html](https://natlang.cs.sfu.ca/software/trofi.html)]([https://natlang.cs.sfu.ca/software/trofi.html](https://natlang.cs.sfu.ca/software/trofi.html)); last accessed on 11 February 2023) and is often used in machine learning models for metaphor recognition, and we cite it here as well. The data set contains sentences in which 50 selected verbs appear, in their literal and figurative meanings. As part of their metaphor identification research, [PERSON] et al. [23] created an annotated dataset for English and Russian (TSV) for metaphors expressed by adjectives and nouns. The dataset is publicly available (www.cs.cmu.edu/~ytsvetko/metaphor/datasets.zip; last accessed on 11 February 2023) and is divided into a training set containing 884 metaphorical and 884 literal expressions/sentences and a testing set containing 111 metaphorical and 111 literal expressions. The training data set was annotated by 5 annotators and the testing set by only one annotator. MOH-X [24] data set was created as part of the research aimed at answering the question of whether metaphorical expressions create stronger emotions and this data set is also publicly available ([[https://saifmohammad.com/WebPages/metaphor.html](https://saifmohammad.com/WebPages/metaphor.html)]([https://saifmohammad.com/WebPages/metaphor.html](https://saifmohammad.com/WebPages/metaphor.html)); last accessed on 12 February 2023). [PERSON] et al. used a crowdsourcing platform and each annotated instance was annotated by ten annotators, while the final data set included those that 70% or more of the annotators annotated as a metaphor. The dataset was extracted from WordNet [25; 26] based on the selected 440 verbs, where the criterion for selecting verbs was that they have more than three and less than ten meanings in WordNet, to ensure a higher probability of the verb being used metaphorically but also to avoid words with a double meaning. In addition to annotating the metaphor, the strength of the emotion expressed in each instance was also annotated for each instance, in order to test the hypothesis that metaphorical expressions are more emotional than literal ones. To the best of our knowledge, KOMET 1.0 [27], annotated corpus of metaphors in the Slovenian language, is the only publicly available ([[https://www.clarin.si/repository/xmlui/handle/11356/1293](https://www.clarin.si/repository/xmlui/handle/11356/1293)]([https://www.clarin.si/repository/xmlui/handle/11356/1293](https://www.clarin.si/repository/xmlui/handle/11356/1293)); last accessed on 8 January 2023) annotated data set for any of the South Slavic languages. [PERSON] approaches annotation using the guidelines provided by the MIPVU method, and annotates indirect and direct metaphors, borderline cases and signals that a unit could be a metaphor.