A Report on the 2020 VUA and TOEFL Metaphor Detection Shared Task (2020)

Abstract: In this paper, we report on the shared task on metaphor identification on VU Amsterdam Metaphor Corpus and on a subset of the TOEFL Native Language Identification Corpus. The shared task was conducted as apart of the ACL 2020 Workshop on Processing Figurative Language.

## introduction metaphor use in everyday language is a way to relate our physical and familiar social experiences to a multitude of other subjects and contexts @xcite ; it is a fundamental way to structure our understanding of the world even without our conscious realization of its presence as we speak and write. it highlights the unknown using the known, explains the complex using the simple, and helps us to emphasize the relevant aspects of meaning resulting in effective communication. metaphor has been studied in the context of political communication, marketing, mental health, teaching, assessment of english proficiency, among others @xcite @xcite @xcite @xcite @xcite @xcite @xcite ; see chapter 7 in @xcite for a recent review. we report on the second shared task on automatic metaphor detection, following up on the first shared task held in 2018 @xcite . we present the shared task and provide a brief description of each of the participating systems, a comparative evaluation of the systems, and our observations about trends in designs and performance of the systems that participated in the shared task. ## related work over the last decade, automated detection of metaphor has become an popular topic, which manifests itself in both a variety of approaches and in an increasing variety of data to which the methods are applied. in terms of methods, approaches based on feature-engineering in a supervised machine learning paradigm explored features based on concreteness and imageability, semantic classification using wordnet, framenet, verbnet, sumo ontology, property norms, and distributional semantic models, syntactic dependency patterns, sensorial and vision-based features @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite ; see @xcite and @xcite for reviews of supervised as well as semi-supervised and unsupervised approaches. recently, deep learning methods have been explored for token-level metaphor detection @xcite @xcite @xcite @xcite . in terms of data, researchers used specially constructed or selected sets, such as adjective noun pairs @xcite , wordnet synsets and glosses @xcite , annotated lexical items (from a range of word classes) in sentences sampled from corpora @xcite @xcite , all the way to annotation of all words in running text for metaphoricity @xcite ; @xcite review various annotated datasets. the goal of this shared task is to detect, at the word level, all content word metaphors in a given text. we are using two datasets -vua and toefl, to be described shortly. there are two tracks for each dataset, for a total of four tracks: vua all pos, vua verbs, toefl all pos, and toefl verbs. the allpos track is concerned with the detection of all content words, i.e., nouns, verbs, adverbs and adjectives that are labeled as metaphorical while the verbs track is concerned only with verbs that are metaphorical. we excluded all forms of be, do, and have for both tracks. for each dataset, each participating individual or team can elect to compete in the all pos track, verbs track, or both. the competition is organized into two phases: training and testing. ## systems we first describe the baseline systems. next, we briefly describe the general approach taken by every team. interested readers can refer to the teams' papers for more details. ## results and discussion