SlowGuess/NAACL_2021_OCR huggingface dataset:
Xiaoyu Tong*, Ekaterina Shutova*, Martha Lewis
\*ILLC, University of Amsterdam, the Netherlands
$\diamond$ Department of Engineering Mathematics, University of Bristol, United Kingdom
xiaoyu.tong@student.uva.nl, e.shutova@uva.nl, martha. Lewis@bristol.ac.uk
Metaphor is an indispensable part of human cognition and everyday communication. Much research has been conducted elucidating metaphor processing in the mind/brain and the role it plays in communication. In recent years, metaphor processing systems have benefited greatly from these studies, as well as the rapid advances in deep learning for natural language processing (NLP). This paper provides a comprehensive review and discussion of recent developments in automated metaphor processing, in light of the findings about metaphor in the mind, language, and communication, and from the perspective of downstream NLP tasks.
Metaphor is a figurative device that allows us to portray one domain, often abstract, in terms of another that is typically more concrete (Lakoff and Johnson, 1980). In the sentences His eyes lightened at the news, or Her mood darkened, the abstract domain of emotion is described in terms of the more concrete domain of brightness. Through cross-domain associations and comparisons, metaphor enables us to communicate complex abstract concepts, to convey our affective states, to reinforce our arguments and, ultimately, to forge entirely novel word meanings. Metaphor is also ubiquitous in language, occurring on average in every three sentences, according to corpus-linguistic research (Steen et al., 2010b; Shutova, 2011). This makes accurate interpretation of metaphorical language essential for many NLP tasks and applications.
A systematic and comprehensive survey of metaphor processing systems was published five years ago (Shutova, 2015). Since then, automated metaphor processing has shifted focus towards neural approaches and there has been much activity in the area, including two shared tasks (Leong et al., 2018, 2020). The development has kept pace with the development of deep learning techniques, and
the state-of-the-art performance has been advanced very quickly, stressing the need for a new survey focusing on this recent progress.
Moreover, the design and evaluation of metaphor processing systems should be informed by both the cognition of metaphor and its role in communication, as well as the potential applications of these systems. It is common practice in the NLP community to divide metaphor processing into two sub-tasks, metaphor identification and interpretation, and the two shared tasks encourage the use of a common task definition and particular metaphor corpora for the development of metaphor identification systems. However, there has been little discussion about whether or how well the existing systems address theoretical and cognitive considerations in the use of metaphor and how insights and models produced by this work could inform and benefit the wider natural language understanding (NLU) research.
This paper aims to provide a starting point for such a discussion, by reviewing recent advances in metaphor processing in the context of contemporary theories of metaphor, its sociolinguistic aspects and the needs of NLP applications. We begin with an overview of theories and findings about metaphor in the mind/brain, language, and communication. We then review metaphor processing approaches published in the last 5 years, as well as the datasets they used. We provide an analysis of these models from the linguistic and cognitive perspective and propose some future directions for modelling social aspects of metaphor use (e.g. the intent behind metaphor use, extended metaphor and metaphoric framing), as well as the underaddressed tasks of metaphor interpretation and generation. Finally, we discuss how elements of metaphor processing can be integrated in the real-world applications, such as machine translation, opinion mining, dialogue modelling, and modelling argumentative discourse (e.g., political discourse).
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4673-4686 June 6-11, 2021. ©2021 Association for Computational Linguistics
2 Theoretical accounts of metaphor
2.1 Conceptual metaphors
According to the conceptual metaphor theory (CMT), linguistic metaphors have their roots in conceptual metaphors, cross-domain mappings in one's conceptual system (Lakoff and Johnson, 1980). Based on comparable properties and relations in the target domain and the source domain, conceptual metaphors invite one to conceptualise the former through the latter. For example, He attacked every point in her argument instantiates the conceptual metaphor ARGUMENT IS WAR, in which ARGUMENT is the target domain and WAR the source domain. It uses an event in the domain of WAR to describe an event about ARGUMENT.
The target and source domains of a metaphor usually involve abstract and concrete concepts respectively, which has to do with concept representation in the brain. According to the theory of embodied cognition (Barsalou, 1999), concepts are represented within sensorimotor circuitry: GRASPING, for instance, is represented in areas that control hand movements (Gallese and Lakoff, 2005). When using expressions such as grasp a point, the same areas are involved for processing the metaphor (Aziz-Zadeh and Damasio, 2008).
Conceptual metaphors that are entrenched in one's conceptual system are termed conventional metaphors; those that are not entrenched are termed novel metaphors. ARGUMENT IS WAR is a typical conventional metaphor; the expression attack one's argument, for example, has entered contemporary dictionaries of English and is unlikely to be considered novel by the native speakers. On the contrary, Fear coiled around his heart instantiates a novel metaphor, FEAR IS A SNAKE. The verb has a single sense in the Macmillan dictionary: a particular movement of 'something long and thin'. Fear is an abstract concept, incapable of any movement to be seen. The usage is therefore novel.
2.2 Linguistic metaphors
Informed by CMT, Steen et al. (2010b) proposed the Metaphor Identification Procedure Vrije Universiteit (MIPVU) for manually identifying metaphor-related words (MRWs) in a text. MRWs are word uses that can be attributed to underlying conceptual metaphors. What are usually called metaphorically used words are a subset of MRWs termed indirect metaphors. The source-domain word in attack one's argument is an indirect metaphor: it refers
to a target-domain act, and is therefore indirectly related to the underlying conceptual metaphor, ARGUMENT IS WAR.
Other types of MRWs are direct metaphors, implicit metaphors, and metaphor flags. Direct metaphors refer to source-domain entities or events. They often co-occur with metaphor flags, signals that a metaphor is used. In (1), the linking verb is signals a use of the BEAUTY AS FLOWER metaphor. The noun flower is a direct metaphor, referring directly to a source-domain concept.
(1) Beauty is but a flower / which wrinkles will devour; [..] (Thomas Nashe)
Implicit metaphors are substitutions or ellipses that are directly or indirectly related to underlying conceptual metaphors. In (2), for example, the pronoun that co-refers with the preceding metaphorically used antidote; the determiner this in the following sentence is also used metaphorically: it refers to what is talked about in the previous sentence, which is an abstraction of its basic, physical meaning (Steen et al., 2010b). Metaphorical usage of substitutions are important for rendering cohesive discourse (Steen et al., 2010b).
(2) Fortunately, there is a single antidote effective against both these myths; and that is to start all over again.... This antidote is effective against the romantic-individualist myth.... (BNC)
Discourse-level information is essential for identifying extended metaphors, sustained use of the same metaphors in a discourse fragment. A typical example is the 'All the World's a Stage' speech written by William Shakespeare. As is presented below, the speech begins with a WORLD AS STAGE metaphor, and proceeds with exploring various target-source pairs within the metaphor. Note that the last two lines could be mistaken as literal if presented as independent sentences.
(3) All the world's a stage, / And all the men and women merely players: / They have their exits and their entrances; / And one man in his time plays many parts, [.].
2.3 Metaphor comprehension
There has been a debate about whether metaphor comprehension is a comparison or a categorisation process. According to the comparison view (e.g.,
Ortony, 1979), when one processes (1), for example, one searches for the shared properties and/or relational structures of the two domains, such as being pleasing to the eye, and the possibility of being fragile. Scholars advocating the categorisation view (e.g., Glucksberg and Keysar, 1990), however, would argue that the object, a flower, refers to a super-ordinate category that both BEAUTY and FLOWER belong to.
The Career of Metaphor Theory (Bowdle and Gentner, 2005) suggests that both comparison and categorisation are possible paths of metaphor processing; which path is chosen depends on an interaction between the conventionality of the metaphor and its linguistic realisation. More specifically, novel metaphors are processed through comparison; as a metaphor becomes conventionalised, people tend to process it through categorisation, which is less cognitively demanding than a comparison process (Bowdle and Gentner, 2005).
An inevitable result of metaphor processing is the emergence of features not inherent in the target or the source domain (Tourangeau and Rips, 1991). This notion of emergent meaning corresponds to the connotations or inference of metaphorical language in linguistics literature. Consider the difference in meaning between attack one's argument and criticise one's argument. The first contains far richer shades of meaning than the second. It might be possible to express a similar meaning without using non-literal language, but the expression is unlikely to be as concise as the use of a single word, attack.
2.4 Metaphor use in communication
The linguistic expression of emotional states often employs metaphor (Fainsilber and Ortony, 1987; Fussell and Moss, 1998). As emotion is an abstract domain, it goes with CMT and embodied cognition that we employ more concrete domains, such as physical or bodily experience, to conceptualise it. Moreover, since metaphor gives rise to emergent meaning, metaphorical language has a stronger emotional effect than literal language, regardless of the source and target domains involved (Blanchette et al., 2001; Crawford, 2009; Mohammad et al., 2016). For instance, Citron and Goldberg (2014) found that metaphorical expressions involving taste (e.g., She looked at him sweetly), evoke a higher emotional response than their literal counterparts (e.g., She looked at him kindly).
Metaphor has also proved to be an effective persuasive device (Sopory and Dillard, 2002; van Steele, 2018). The persuasive power of metaphors is pronounced in metaphoric framing effect. Since metaphors encourage a particular way to conceptualise the target domain, repeated use of the same metaphors throughout discourses in mass media tends to affect how the public perceives and reacts to societal issues that belong to the target domain (Lakoff, 1991; Entman, 2003; Lakoff and Wehling, 2012). For instance, participants in a series of studies (Thibodeau and Boroditsky, 2011, 2013) favoured different social solutions to crime after reading articles that associate CRIME with different source domains. Moreover, while the participants could identify the implicitly advocated solutions given a metaphor, they were unaware of the influence of the metaphors on their own preference.
The Deliberate Metaphor Theory is an attempt to deal with the intention behind metaphor use systematically (Steen, 2008, 2017). The theory defines deliberate metaphor use as the intentional introduction of a topic shift or perspective change to the discourse. Deliberate metaphor use is associated with online metaphor processing, the construction of conceptual metaphors during text comprehension (Steen, 2017). Examples of deliberate metaphors include conventional metaphors instantiated as copula metaphors and all novel metaphors. A systematic procedure for the identification of potential deliberate metaphors has also been proposed (Reijnierse et al., 2018).
3 Metaphor datasets
Metaphoricity annotations Tsvetkov et al. (2014) released a dataset (henceforth: TSV) consisting of an equal number (884) of metaphorical and non-metaphorical adjective-noun (AN) phrases collected from the web. The phrases were stated to be verified by multiple annotators, but the criteria for metaphor annotation were not provided.
The dataset released by Mohammad et al. (2016) (henceforth: MOH) consists of 1639 (1230 literal and 409 metaphorical) sentences extracted from WordNet, manifesting the use of 440 verbs. Metaphoricity annotation of the verb uses was obtained through crowdsourcing. Note that the WordNet sentences are mainly instances of conventional metaphor. The specified association between word senses and metaphoricity makes it easier to determine the source and target domains involved.
The Language Computer Corporation metaphor datasets (LCC) (Mohler et al., 2016) contain linguistic metaphors extracted from web corpora for a small set of target domains in four languages: English, Spanish, Russian, and Farsi. Metaphoricity is annotated at sentence level, and the conceptual metaphors and affect information is specified.
VU Amsterdam Metaphor Corpus (VUA) (Steen et al., 2010b) is the fruit of the authors' application of MIPVU. The metaphoricity of each lexical unit (187,570 in total) in a subset of the British National Corpus (BNC Consortium, 2007) was annotated. VUA is the only metaphor corpus used in studies of automated metaphor identification that is built by cognitive linguists, and the only one that deals with the metaphoricity of function words.
Metaphor paraphrase datasets Mohammad et al. (2016) also obtained literal paraphrases of the sentences, in which the metaphorically used verbs are replaced by their synonyms (171 pairs of sentences in total), selected by the authors. Note that the literal paraphrases were considered to convey less emotion than the original metaphorical sentences in their experiment. It is therefore questionable to what extent the paraphrases capture the connotations of the metaphorical sentences.
Bizzoni and Lappin (2018) built a metaphor paraphrase dataset containing 200 sets of 5 sentences; 4 paraphrases at varying levels of aptness are provided for each metaphorical sentence. Apart from verbs, the dataset also includes metaphorical uses of adjectives, copula metaphors, and multi-word metaphors. The dataset takes into account the connotations of the metaphorical sentences to some extent. For instance, candidate paraphrases for the copula metaphor My job is a dream include I love my job and I hate my job, which indicate opposite sentiment poles. A metaphor processing system will need to infer the sentiment to select the apt paraphrase.
4 Recent metaphor processing systems
4.1 Automated metaphor identification
4.1.1 Neural architectures
Most neural models treat metaphor identification as a sequence labelling task, outputting a sequence of metaphoricity labels for a sequence of input words (usually a sentence) (Bizzoni and Ghanimifard, 2018; Chen et al., 2020; Dankers et al., 2019; Gao et al., 2018; Gong et al., 2020; Mao et al., 2019;
Mykowiecka et al., 2018; Pramanick et al., 2018; Su et al., 2020; Wu et al., 2018). The first sequence labelling systems typically represented an input sentence as a sequence of pre-trained word embeddings and produced a task- and context-specific sentence representation through bidirectional long short-term memory (BiLSTM) (Dankers et al., 2019; Gao et al., 2018; Mykowiecka et al., 2018; Pramanick et al., 2018). Bizzoni and Ghanimifard (2018) experimented with separating long sentences into smaller chunks, which led to a  $6\%$  increase in F-score when using a BiLSTM architecture. Their BiLSTM system outperformed their compositional system, which employs a sequence of fully-connected neural networks (NNs) and essentially performs bigram phrase composition to modulate the representation of input words with respect to their neighbours. BiLSTM models also outperformed bidirectional gated recurrent unit (BiGRU) models in the study of Mykowiecka et al. (2018). From Gao et al. (2018), the contextualised Embeddings from Language Models (ELMo) (Peters et al., 2018) began to be used in addition to the context-free Global Vectors (GloVe) (Pennington et al., 2014) for representing input sentences (Dankers et al., 2019; Gao et al., 2018; Mao et al., 2019). The most recent systems adopt a fine-tuning approach, employing pre-trained contextual language models such as Bidirectional Encoder Representations from Transformers (BERT) (Chen et al., 2020; Dankers et al., 2019) and RoBERTa (Gong et al., 2020). A summary of the best performing recent metaphor identification systems is presented in Table 1 (a summary of all systems can be found in Appendix A).
Several BiLSTM-based systems consider both contextualised and pre-trained representations in the classification layers (Mao et al., 2019; Swarnkar and Singh, 2018). The Di-LSTM Contrast system (Swarnkar and Singh, 2018) encodes the left- and right-side context of a target word using forward and backward LSTMs. The classification is based on a concatenation of the target-word vector and its difference with the encoded context. Mao et al. (2019) combined GloVe and BiLSTM hidden states for sequence labelling, which outperformed the best model in the 2018 VUA All POS track.
Wu et al. (2018) and Su et al. (2020) employed separate encoding of local and long-range context. Wu et al. (2018) used a convolutional neural network (CNN) and a BiLSTM to extract local and
sentence context respectively. Su et al. (2020) used separate Transformer encoder layers to encode global and local text features for each word. The two systems achieved the best performance on the VUA All POS tracks in their respective shared tasks (Leong et al., 2018, 2020).
Modelling metaphor in discourse Several recent approaches have also incorporated wider discourse properties in their models. Mu et al. (2019) focused on the metaphoricity of verbs. They used general-purpose word, sentence and document embedding methods (e.g. GloVe, ELMo, doc2vec (Le and Mikolov, 2014), skip-thought (Kiros et al., 2015)) to represent the surrounding paragraphs. Their system feeds into a gradient boosting decision tree classifier (Chen and Guestrin, 2016) a concatenation of three feature vectors, representing 1) the lemma of the target verb, 2) its subject and direct object, and 3) its surrounding paragraph. All the representations are learned from embedding methods. Representing the features with ELMo led to the highest F-score (0.668), using the VUA Verbs 2018 shared task data.
Dankers et al. (2020) fine-tuned a BERT model that receives a discourse fragment as input. Using hierarchical attention (which computes both token- and sentence-level attention) after the encoded layers achieved better performance than applying general attention to all tokens. Both Dankers et al. (2020) and Mu et al. (2019) thus demonstrated the importance of context beyond sentence for word-level metaphor identification. Their qualitative analysis shows that co-reference resolution is one of the driving factors in the performance increase.
Note that the above systems use discourse to aid in detecting linguistic metaphors whose metaphoricity is otherwise ambiguous, but do not monitor whether a metaphor is sustained throughout a fragment of discourse. Thus, they are unlikely to be directly applicable to the identification of extended metaphors or metaphoric frames.
4.1.2 Cognitively-inspired approaches
Categorical features Metaphor processing is concerned with how concepts are organised in the brain/mind, and is closely related to categorisation. It therefore makes sense to employ categorical features for metaphor identification. Tekiroglu et al. (2015) tested the use of sensorial categories (the five human senses) for identifying AN synaes
The most important and most numerous of the three categories is the thematic metaphors (e.g., sweet music, soft light). Using sensorial categories in addition to Word-Net supersenses, concreteness, and imageability led to improved performance (accuracy 0.890 vs 0.845 on TSV). Mykowiecka et al. (2018) used another general-purpose resource, the Harvard IV psychosocial dictionary<sup>1</sup>, which includes categories of emotions, people and animals, objects, places, etc. However, it did not lead to consistent improvement in model performance. Bulat et al. (2017) compared property-based and linguistic embeddings for input word representation. They obtained property-based word embeddings by mapping linguistic word embeddings onto a conceptual space, using as training data a frequency-based human property-norm dataset (McRae et al., 2005). Using property-based word embeddings led to a  $4\%$  increase in F-score on TSV.
Sensory features An important function of conceptual metaphors, according to CMT, is to use bodily experience to understand abstract concepts; concreteness features have therefore also proved useful for automated metaphor identification (Bizzoni and Ghanimifard, 2018; Turney et al., 2011). Shutova et al. (2016) tested combination of visual and linguistic embeddings on MOH and TSV. The multimodal system outperformed the monomodal systems in both tests. Gong et al. (2020) also included both categorical and sensory features. In addition to RoBERTa, the system employs concreteness features, topic distributions, WordNet classes, VerbNet classes, verb clusters, and part of speech (POS), which led to improvements in performance on VUA All POS.
Word-context incongruity The neural model of Swarnkar and Singh (2018) computes the difference between a target word and its context. This operation can be associated with the comparison view of metaphor, including CMT, and is reflected in MIPVU. Shutova et al. (2016) used cosine similarity between word or phrase embeddings to predict the metaphoricity of verb-noun (VN) or AN pairs; a word pair is marked metaphorical if the cosine similarity is below a trained threshold. The systems reached an F-score of 0.71 on MOH and 0.76 on TSV using linguistic embeddings alone. Rei et al. (2017) proposed a supervised similarity network, which learns to calculate weighted co
http://www.wjh.harvard.edu/~inquirer/homecat.htm
Study	Input	Input features	Architecture	Dataset	Performance (default F)
Tekiroğlu et al. (2015)	AN	Token, Sensorial, Word-Class, Conc, Image★	Random Forest	TSV	A 0.890
Shutova et al. (2016)	AN/VN	Token, VisualEmb★	skip-gram; CNN★	TSV/MOH	0.79/0.75
Bulat et al. (2017)	AN	AttributeEmb★	SVM	TSV	0.77
Rei et al. (2017)	AN/VN	Token, Attribute-Emb (AN-TSV)★	Supervised similarity★	TSV/MOH	0.811/0.742
Mao et al. (2018)	S	Token	Cosine sim.★	MOH	0.75
Gao et al. (2018)	S	Token (GloveElmo)	BiLSTM	VUA Verbs	0.697
Gong et al. (2020)	S	Token	RoBERTa	VUA Verbs	0.771
Bizzoni and Ghanimifard (2018)	Chunk	Token	BiLSTM	VUA All POS	0.621
Wu et al. (2018)	S	Lemma, POS, WordCluster★	CNN-BiLSTM (ensemble)	VUA All POS	0.651
Dankers et al. (2020)	Discourse	Token	BERT base	VUA All POS	0.715
Chen et al. (2020)	S	Token	BERT large	VUA All POS	0.718
Gong et al. (2020)	S	Token, POS, Conc, TopicDistr, WordClass, WordCluster★	RoBERTa (ensemble)	VUA All POS	0.730
Mao et al. (2019)	S	Token (GloveElmo)	BiLSTM + Glove	VUA All POS	0.740
Su et al. (2020)	S	Token, POS	Transformers (ensemble)	VUA All POS	0.769
Pramanick et al. (2018)	S	Token, Lemma, POS, WordLemma	BiLSTM-CRF	VUA all words	0.6740
Gao et al. (2018)	S	Token (GloveElmo)	BiLSTM	VUA all words	0.726
Dankers et al. (2019)	S	Token	BERT + emotion★	VUA all words	0.769