Contextual Modulation for Relation-Level Metaphor Identification (2020)

Abstract: Identifying metaphors in text is very challenging and requires comprehending the underlying comparison. The automation of this cognitive process has gained wide attention lately. However, the majority of existing approaches concentrate on word-level identification by treating the task as either single-word classification or sequential labelling without explicitly modelling the interaction between the metaphor components. On the other hand, while existing relation-level approaches implicitly model this interaction, they ignore the context where the metaphor occurs. In this work, we address these limitations by introducing a novel architecture for identifying relation-level metaphoric expressions of certain grammatical relations based on contextual modulation. In a methodology inspired by works in visual reasoning, our approach is based on conditioning the neural network computation on the deep contextualised features of the candidate expressions using feature-wise linear modulation. We demonstrate that the proposed architecture achieves state-of-the-art results on benchmark datasets. The proposed methodology is generic and could be applied to other textual classification problems that benefit from contextual interaction.

## introduction despite its fuzziness, metaphor is a fundamental feature of language that defines the relation between how we understand things and how we express them @xcite . a metaphor is a figurative device containing an implied mapping between two conceptual domains. these domains are represented by its two main components, namely the tenor (target domain) and the vehicle (source domain) @xcite . according to the conceptual metaphor theory (cmt) of @xcite , which we adopt in this work, a concept such as "liquids" (source domain/vehicle) can be borrowed to express another such as "emotions" (target domain/tenor) by exploiting single or common properties. therefore, the conceptual metaphor "emotions are liquids" can be manifested through the use of linguistic metaphors such as "pure love", "stir excitement" and "contain your anger". the interaction between the target and the source concepts of the expression is important to fully comprehend its metaphoricity. over the last couple of years, there has been an increasing interest towards metaphor processing and its applications, either as part of natural language processing (nlp) tasks such as machine translation @xcite ), text simplification @xcite and sentiment analysis @xcite or in more general discourse analysis use cases such as in analysing political discourse @xcite , financial reporting @xcite and health communication @xcite . metaphor processing comprises several tasks including identification, interpretation and crossdomain mappings. metaphor identification is the most studied among these tasks. it is concerned with detecting the metaphoric words or expressions in the input text and could be done either on the sentence, relation or word levels. the difference between these levels of processing is extensively studied in @xcite . identifying metaphors on the word-level could be treated as either sequence labelling by deciding the metaphoricity of each word in a sentence given the context or singleword classification by deciding the metaphoricity of a targeted word. on the other hand, relationlevel identification looks at specific grammatical relations such as the dobj or amod dependencies and checks the metaphoricity of the verb or the adjective given its association with the noun. in relation-level identification, both the source and target domain words (the tenor and vehicle) are classified either as a metaphoric or literal expression, whereas in word-level identification only the source domain words (vehicle) are labelled. these levels of analysis (paradigms) are already established in literature and adopted by previous research in this area as will be explained in section 2. the majority of existing approaches, as well as the available datasets, pertaining to metaphor processing focus on the metaphorical usage of verbs and adjectives either on the word or relation levels. this is because these syntactic types exhibit metaphoricity more frequently than others according to corpus-based analysis @xcite . although the main focus of both the relationlevel and word-level metaphor identification is discerning the metaphoricity of the vehicle (source domain words), the interaction between the metaphor components is less explicit in word-level analysis either when treating the task as sequence labelling or single-word classification. relation-level analysis could be viewed as a deeper level analysis that captures information that is not captured on the word-level through modelling the influence of the tenor (e.g.noun) on the vehicle (e.g. verb/adjective). there will be reasons that some downstream tasks would prefer to have such information (i.e. explicitly marked relations), among these tasks are metaphor interpretation and cross-domain mappings. moreover, employing the wider context around the expression is essential to improve the identification process. ## related work over the last decades, the focus of computational metaphor identification has shifted from rule-based @xcite and knowledge-based approaches @xcite to statistical and machine learning approaches including supervised @xcite @xcite @xcite @xcite @xcite @xcite @xcite , semi-supervised @xcite @xcite and unsupervised methods @xcite @xcite . these approaches employed a variety of features to design their models. with the advances in neu-ral networks, the focus started to shift towards employing more sophisticated models to identify metaphors. this section focuses on current research that employs neural models for metaphor identification on both word and relation levels. word-level processing: do dinh and gurevych (2016) were the first to utilise a neural architecture to identify metaphors. they approached the problem as sequence labelling where a traditional fully-connected feed-forward neural network is trained using pre-trained word embeddings. the authors highlighted the limitation of this approach when dealing with short and noisy conversational texts. as part of the naacl 2018 metaphor shared task @xcite , many researchers proposed neural models that mainly employ lstms @xcite with pre-trained word embeddings to identify metaphors on the word-level. the best performing systems are: thu ngn @xcite , ocota @xcite and bot.zen (stemle and onysko, 2018). @xcite were the first to employ the deep contextualised word representation elmo @xcite , combined with pre-trained glove @xcite embeddings to train bidirectional lstm-based models. the authors introduced a sequence labelling model and a single-word classification model for verbs. they showed that incorporating the context-dependent representation of elmo with context-independent word embeddings improved metaphor identification. @xcite proposed a system that utilises a gradient boosting decision tree classifier. document embeddings were employed in an attempt to exploit wider context to improve metaphor detection in addition to other word representations including glove, elmo and skip-thought @xcite . @xcite explored the idea of selectional preferences violation @xcite in a neural architecture to identify metaphoric words. mao's proposed approaches emphasised the importance of the context to identify metaphoricity by employing context-dependent and context-independent word embeddings. @xcite also proposed employing multi-head attention to compare the targeted word representation with its context. an interesting approach was introduced by @xcite to model the interplay between metaphor identification and emotion regression. the authors introduced multiple multi-task learning tech-niques that employ hard and soft parameter sharing methods to optimise lstm-based and bert-based models. relation-level processing: @xcite focused on identifying the metaphoricity of adjective/verb-noun pairs. this work employed multimodal embeddings of visual and linguistic features. their model employs the cosine similarity of the candidate expression components based on word embeddings to classify metaphors using an optimised similarity threshold. @xcite introduced a supervised similarity network to detect adjective/verb-noun metaphoric expressions. their system utilises word gating, vector representation mapping and a weighted similarity function. pre-trained word embeddings and attribute-based embeddings @xcite were employed as features. this work explicitly models the interaction between the metaphor components. gating is used to modify the vector of the verb/adjective based on the noun, however the surrounding context is ignored by feeding only the candidates as input to the neural model which might lead to loosing important contextual information. limitations: as discussed, the majority of previous works adopted the word-level paradigm to identify metaphors in text. the main distinction between the relation-level and the word-level paradigms is that the former makes the context more explicit than the latter through providing information about not only where the metaphor is in the sentence but also how its components come together through hinting at the relation between the tenor and the vehicle. @xcite showed that the type of syntactic construction a verb occurs in influences its metaphoricity. on the other hand, existing relation-level approaches @xcite @xcite ignore the context where the expression appears and only classify a given syntactic construction as metaphorical or literal. studies showed that the context surrounding a targeted expression is important to discern its metaphoricity and fully grasp its meaning @xcite . therefore, current relation-level approaches will only be able to capture commonly used conventionalised metaphors. in this work, we address these limitations by introducing a novel approach to textual classification which employs contextual information from both the targeted expression under study and the wider context surrounding it. ## proposed approach feature-wise transformation techniques such as feature-wise linear modulation (film) have been recently employed in many applications showing improved performance. they became popular in image processing applications such as image style transfer @xcite ; then they found their way into multi-modal tasks, specifically visual question-answering @xcite . they also have been shown to be effective approaches for relational problems as mentioned in section 1. the idea behind film is to condition the computation carried out by a neural model on the information extracted from an auxiliary input in order to capture the relationship between multiple sources of information @xcite . our approach adopts perez's (2018) formulation of film on visual reasoning for metaphor identification. in visual reasoning, image-related questions are answered by conditioning the image-based neural network (visual pipeline) on the question context via a linguistic pipeline. in metaphor identification, we can consider that the image in our case is the sentence that has a metaphoric candidate and the auxiliary input is the linguistic interaction between the components of the candidate itself. this will allow us to condition the computation of a sequential neural model on the contextual information of the candidate and leverage the feature-wise interactions between the conditioning representation and the conditioned network. to the best of our knowledge, we are the first to propose such contextual modulation for textual classification in general and for metaphor identification specifically. our proposed architecture consists of a contextual modulation pipeline and a metaphor identification linguistic pipeline as shown in figure 1 . the input to the contextual modulator is the deep contextualised representation of the candidate expression under study (which we will refer to as targeted expression 1 ) to capture the interaction between its components. the linguistic pipeline employs an lstm encoder which produces a contextual representation of the provided sentence where the targeted expression appeared. the model is trained 1 targeted expressions are already annotated in the dataset and initially obtained either manually or automatically using a dependency parser as will be described in section 4. embed: given a labelled dataset of sentences, the model begins by embedding the tokenised sentence s of words w 1 , w 2 , ..., w n , where n is the number of words in s, into vector representations using glove embeddings. we used the uncased 200dimensional glove embeddings pre-trained on ∼2 billion tweets and contains 1.2 million words. encode: the next step is to train a neural network with the obtained embeddings. since context is important for identifying metaphoricity, sentence encoder is a sensible choice. we use an lstm sequence model to obtain a contextual representation which summarises the syntactic and semantic features of the whole sentence. the output of the lstm is a sequence of hidden states h 1 , h 2 , ..., h n , where h i is the hidden state at the i th time-step. feature-wise transformation: in this step, an affine transformation layer, hereafter affinetrans layer, applies a feature-wise linear modulation to its inputs, which are: 1) the hidden states from the encoding step; 2) the scaling and shifting parameters from the conditioning step. by feature-wise, we mean that scaling and shifting are applied to each encoded vector for each word in the sentence. attend: recently, attention mechanisms have become useful to select the most important elements in a given representation while minimising information loss. in this work, we employ an attention layer based on the mechanism presented in @xcite . it takes the output from the affine-trans layer as an input in addition to a randomly initialised weight matrix w , a bias vector b and a learnable context vector u to produce the attended output as follows: our model is trained and evaluated with and without the attention mechanism in order to differentiate between the effect of the feature modulation and the attention on the model performance. predict: the last step is to make the final prediction using the output from the previous step (attended output in case of using attention or the affinetrans layer output in case of skipping it). we use a fully-connected feed-forward layer with a sigmoid activation that returns a single (binary) class label to identify whether the targeted expression is metaphoric or not. ## datasets the choice of annotated dataset for training the model and evaluating its performance is determined by the level of metaphor identification. given the distinction between the levels of analysis, approaches addressing the task on the word-level are not fairly comparable to relation-level approaches since each task addresses metaphor identification differently. therefore, the tradition of previous work in this area is to compare approaches addressing the task on the same level against each other on level-specific annotated benchmark datasets @xcite . following prior work in this area and in order to compare the performance of our proposed approach with other relation-level metaphor identification approaches, we utilise available annotated datasets that support this level of processing. the existing datasets are either originally prepared to directly support relation-level processing such as the tsv @xcite dataset and the tweets dataset by @xcite or adapted from other word-level benchmark datasets to suit relation-level processing such as the adaptation of the benchmark datasets trofi @xcite and vu amsterdam metaphor corpus (vuamc) @xcite by @xcite and the adaptation of the moh @xcite dataset by @xcite . due to space limitation, we include in appendix a: 1) examples of annotated instances from these datasets showing their format as: sentence, targeted expression and the provided label; 2) the statistics of these datasets including their size and percentage of metaphors. relation-level datasets: these datasets focus on expressions of certain grammatical relations. obtaining these relations could be done either automatically by employing a dependency parser or manually by highlighting targeted expressions in a specific corpus. then, these expressions are manually annotated for metaphoricity given the surrounding context. there exist two benchmark datasets of this kind, namely the tsv dataset and @xcite tweets dataset, hereafter zaytw dataset. the former focuses on discerning the metaphoricity of adjective-noun expressions in sentences collected from the web and twitter while the latter focuses on verb-direct object expressions in tweets. adapted word-level datasets: annotated datasets that support word-level metaphor identification are not suitable to support relation-level processing due to the annotation difference @xcite . to overcome the limited availability of relation-level datasets, there has been a growing effort to enrich and extend benchmark datasets annotated on the word-level to suit relation-level metaphor identification. although it is non-trivial and requires extra annotation effort, @xcite and @xcite introduced adapted versions of the moh, trofi and vuamc datasets to train and evaluate models that identify metaphors on the relation-level. since the moh dataset was originally created to identify metaphoric verbs on the word-level, its adaptation by @xcite , also referred to as moh-x in several papers, focused on extracting the verb-noun grammar relations using a dependency parser. the dataset is relatively small and contains short and simple sentences that are originally sampled from the example sentences of each verb in wordnet @xcite . the trofi dataset was designed to identify the metaphoricity of 50 selected verbs on the word-level from the 1987-1989 wall street journal (wsj) corpus. the vuamc @xcite is the largest corpus annotated for metaphors and has been employed extensively by models developed to identify metaphors on the word-level. however, models designed to support relation-level metaphor identification can not use it in its current state. therefore, previous research focusing on relation-level processing @xcite @xcite did not train, evaluate or compare their approaches using it. recently, a subset of the vuamc was adapted to suit relation-level analysis by focusing on the training and test splits provided by the naacl metaphor shared task. this corpus subset as well as the trofi dataset are adapted by @xcite to suit identifying metaphoric expressions on the relation-level, focusing on verb-direct object grammar relations (i.e dobj dependencies). the stanford dependency parser was utilised to extract these relations which were then filtered to ensure quality. ## discussion overall performance. we analysed the model performance by inspecting the classified instances. we noticed that it did a good job identifying conventionalised metaphors as well as uncommon ones. appendix a shows examples of classified instances by our system from the employed benchmark datasets. our model achieves significantly better f1-score over the state-of-the-art ssn system @xcite under the one-tailed paired ttest @xcite at p-@xmath0.01 on three of the five employed benchmark datasets. moreover, our architecture showed improved performance over the state-of-the-art approaches on the tsv and moh datasets. it is worth mentioning that the size of their test sets is relatively smaller; therefore any change in a single annotated instance drastically affects the results. moreover, the approach proposed by @xcite relies on hand-coded lexical features which justifies its high f1-score. the effect of contextual modulation. when excluding the affinetrans layer and only using the simple lstm model, we observe a significant performance drop that shows the effectiveness of leveraging linear modulation. this layer adaptively influences the output of the model by conditioning the identification process on the contextual information of the targeted expression itself which significantly improved the system performance, as observed from the results. moreover, employing the contextualised representation of the targeted expression, through elmo sentence embeddings, the effect of attention. it is worth noting that the attention mechanism did not help much in our affinetrans model because affine transformation itself could be seen as playing a similar role to attention, as discussed in section 1. in attention mechanisms important elements are given higher weight based on weight scaling whereas in linear affine transformation scaling is done in addition to shifting which gives prior importance (probability) to particular features. we are planning to perform an in-depth comparison of using affine transformation verses attention in our future work. ## conclusions in this paper, we introduced a novel architecture to identify metaphors by utilising feature-wise affine transformation and deep contextual modulation. our approach employs a contextual modulation pipeline to capture the interaction between the metaphor components. this interaction is then used as an auxiliary input to modulate a metaphor identification linguistic pipeline. we showed that such modulation allowed the model to dynamically highlight the key contextual features to identify the metaphoricity of a given expression. we applied our approach to relation-level metaphor identification to classify expressions of certain syntactic constructions for metaphoricity as they occur in context. we significantly outperform the stateof-the-art approaches for this level of analysis on benchmark datasets. our experiments also show that our contextual modulation-based model can generalise well to identify the metaphoricity of unseen instances in different text types including the noisy user-generated text of tweets. our model was able to identify both conventionalised common metaphoric expressions as well as less common ones. to the best of our knowledge, this is the first attempt to computationally identify metaphors in tweets and the first approach to study the employment of feature-wise linear modulation on metaphor identification in general. the proposed methodology is generic and can be applied to a wide variety of text classification approaches including sentiment analysis or term extraction.