Modeling Conceptual Attribute Likeness and Domain Inconsistency for Metaphor Detection
Authors: Yuan Tian
Nan Xu
Wenji Mao
Daniel Dajun Zeng
Venue: EMNLP 2023 Main (2023)
Keywords: Metaphor detection
Attribute likeness
Attribute siamese network
Conceptual metaphor theory

Abstract: Metaphor detection is an important and challenging task in natural language processing, which aims to distinguish between metaphorical and literal expressions in text. Previous studies mainly leverage the incongruity of source and target domains and contextual clues for detection, neglecting similar attributes shared between source and target concepts in metaphorical expressions. Based on conceptual metaphor theory, these similar attributes are essential to infer implicit meanings conveyed by the metaphor. Under the guidance of conceptual metaphor theory, in this paper, we model the likeness of attribute for the first time and propose a novel Attribute Likeness and Domain Inconsistency Learning framework (AIDIL) for word-pair metaphor detection. Specifically, we propose an attribute siamese network to mine similar attributes between source and target concepts. We then devise a domain contrastive learning strategy to learn the semantic inconsistency of concepts in source and target domains. Extensive experiments on four datasets verify that our method significantly outperforms the previous state-of-the-art methods, and demonstrate the generalization ability of our method.

Modeling Conceptual Attribute Likeness and Domain Inconsistency for Metaphor Detection
Yuan Tian Nan Xu Wenji Mao'?,, Daniel Dajun Zeng!
Institute of Automation, Chinese Academy of Sciences
School of Artificial Intelligence, University of Chinese Academy of Sciences Beijing Wenge Technology Co., Ltd [tianyuan2021, xunan201 15, wenj i. mao, dajun. zeng]@ia.c. ac
Abstract
Example sentences
1. My car drinks gasoline. (Metaphorical)
Metaphor detection an important and chal- lenging task in natural language processing, which aims to distinguish between metaphori- cal and literal expressions in text. Previous stud- ies mainly leverage the incongruity of source and target domains and contextual clues for de- tection, neglecting similar attributes shared be- tween souree and target concepts in metaphori- cal expressions. Based on conceptual metaphor theory, these similar attributes essential to infer implicit meanings conveyed by the metaphor. Under the guidance of conceptual metaphor theory, in this paper, we model the likeness of attribute for the first time and pro- pose novel Attribute IIkeness and Domain Inconsistency Learning framework (AIDIL) for word-pair metaphor detection. Specifically, we propose attribute siamese network to mine similar attributes between source and target concepts. then devise a domain contrastive learning strategy to learn the semantic inconsis- tency of concepts in source and target domains. Extensive experiments on four datasets verify that our method significantly outperforms the previous state-of ofte-of-the-ar the-art methods, and demon- strate the generalization ability of method.
2. Children like drink soda. (Literal)
3. Drinking water is good to your health. (Literal)
Target concept
Similar attributes
Source concept
liquid, consuming constantly
sexa water
gasoline
Figure 1: Illustration of metaphor understanding with similar attributes. The target concept gasoline and the source concept (e.g., soda and water) domain- inconsistent with their intermediate similar attributes (e.g.. liquid and consuming constantly). These simi- lar attributes help understand the implicit meaning con- veyed by the metaphor, that is, drinking gasoline implies consuming gasoline lot like consuming liquid.
machine translation (Mao al., 2018), sentiment analysis (Mao and Li, 2021), and dialogue system (Sun et al.. 2023).
Existing research on metaphor detection mainly focuses on detecting metaphors at word-pair level (Rei et al., 2017; Su et al., 2020; Ge et al., 2022) and token-level (Su et al., 2021; Choi et al., 2021; Zhang and Liu, 2022). Early studies construct lin- guistic features using manually-creater resources for metaphor detection, such abstractness (Tur- ney et al., 2011), imageability (Tsvetkov al., 2014) and visibility (Shutova al., 2016). Re- cent studies utilize metaphor identification theo- ries (Su et al., 2021; Choi et al., 2021: Zhang and Liu, 2022), including selection preference violation (SPV) (Wilks, 1975, 1978) and metaphor identifi- cation procedure (MIP) (Group, 2007), to help de- sign deep learning methods for metaphor detection. Although these methods have achieved promising performances, their underlying theories only uti- lize contextual clues relevant to the basic meaning of words and their contexts for metaphor identifi- cation, lacking the deeper level understanding of
1 Introduction
Metaphor pervasive in various forms of language expressions, such as political speech, advertising text, and literary work. Merriam- Merriam-Webster Dic- tionary defines metaphor as "a figure of speech in which a word or phrase literally denoting one kind of object or idea is used in place of another suggest likeness analogy between them"! Metaphor detection aims to distinguish between metaphorical and literal expressions in text. It has gained increasing research interest in recent years and plays vital role in various NLP applications that need to understand implicit semantics, such as
"Corresponding author

---

conceptual meanings and semantic characteristics associated with metaphor.
metaphor prediction. The main contributions of our work are as follows:
In contrast to the above metaphor identifica- tion theories, a more explainable theory, concep- tual metaphor theory (CMT) (Lakoff and John- son, 2008), argues that metaphor facilitates map- ping of set of similar attributes between the concepts in two different domains (i.e. source and target domains). Inspired by CMT, Ge al. (2022) generate the source and target concepts from different domains to benefit metaphor detec- tion. However, their work only considers domain- inconsistent information, ignoring the mapping of similar attributes shared by source and target con- cepts, which provides essential clues infer the implicit meanings that metaphors tend to convey.
Inspired by CMT, we identify that metaphor implies not only inconsistent domains but also similar attributes between source and target concepts, and make the first attempt to model attribute likeness for metaphor detection.
We propose a novel attribute siamese network to incrementally mine similar attributes be- tween source and target concepts via layer- wise attribute refinement graphs on plentiful candidate attributes.
Extensive experiments verify that our method achieves significant improvements over previ- ous SOTA methods and also demonstrates its generalization ability on unseen data.
According to CMT, metaphor understanding im- plics plausible mapping of a set of similar at- tributes between the concepts in source and target domains. Hence, attribute in CMT is broad char- acteristic obtained by activating human imagina- tion and association based commonsense knowl- edge well perceptual experience. Fig. 1 gives an example to illustrate metaphor understanding with similar attributes. In this example, attribute likeness (liquid and consuming constantly) is the in- termediate association to the implicit meaning (con- suming lor) conveyed by this metaphor. There- fore, attribute likeness functions as important asso- ciated information for metaphor detection, and the mining of this attribute similarity strongly depends on appropriate refinement process from plentiful candidate attributes.
Related Work
Metaphor detection can be roughly divided into two categories: word-pair level and token level. The former determines whether word pair is metaphor- ical literal, and the latter aims to find metaphor- ical words in sentences. As word pair is a basic type and the most commonly used linguistic form to express metaphor (Tsvetkov et al., 2014), its metaphor detection is the focus of our work.
Word-Pair Metaphor Detection Early studies exploit supervised machine learning methods by constructing word embeddings from linguistic fea- tures or external knowledge which are relevant to metaphor, such as embeddings of abstractness (Tur- ney et al., 2011), imageability (Tsvetkov et al., 2014), visibility (Shutova et al., 2016) or prop- erty norm (Bulat et al., 2017). After that, some researchers explore indicative clues of metaphor for this task. Based on the observation of cosine similarity between words in a word pair indicating its metaphoricity, Rei et al. (2017) design a neural network with a gating function to model this char- acteristic. Considering the relation between con- creteness and metaphor, Su et al. (2020) construct image representations for concrete word pairs and devise multimodal model to detect metaphor.
Based on the above considerations, in this pa- per, we propose an Attribute Ilkeness and Domain Inconsistency Learning framework, namely AIDIL, for word-pair metaphor detection. We devise an attribute siamese network with layer-wise attribute refinement graphs model attribute likeness infor- mation and incrementally filter irrelevant informa- tion from plentiful candidate attributes. Meanwhile, to exploit information of inconsistent domains, we design a domain contrastive learning strategy on the hidden representations of concepts from source and target domains so as to learn an embedding space with information of domain inconsistency. The two components jointly learn the association and dispar- ity between source and target concepts for effective metaphor detection. Additionally, our method also capable of providing explainable clues like source concepts or learned attribute information in
Different from above approaches using intuitive clues for metaphor detection, Ge al. (2022) adopt widely accepted metaphor conceyyconceppuall metaphor theory (CMT) their method via gen- crating plausible concepts in source and target do- mains to help metaphor prediction, which is the

---

previous state-of-the-art method. However, ac- cording to CMT, metaphor not only indicates domain-inconsistent information but also implies a mapping process of similar attributes shared be- tween source and target concepts, which is nec- essary for explaining the implicit meanings con- veyed by the metaphor. All the aforementioned approaches neglect this important information of similar attributes for metaphor detection. Thus based on CMT, we focus on mining the attribute likeness between source and target concepts with the disparity of domains for metaphor detection.
Ketase haliau + heliss
Extuss
Eawr
Classification
Attribute Siamese Network
Domain Contrastive Learning
Metaplorrica//ieral
puch
Classifier
Contextual representation
Contextual rqpresentation
goide
i
(easy gasolina
ConcepthNet
Token-Level Metaphor Detection Considering special clues related to metaphor, some studies uti- lize external knowledge resources to benefit this task, such definitions of words (Su et al., 2021) or multiword expressions (Rohanian et al., 2020).
Encoder
slared
gosmliee
LCT dhink conoolig
drink
(CLS) drink gazoline
deishaes [CLS] water
(curc ward)
Training set
Positive nogative samples
Other studies employ shared features learned from the multi-task framework with the task of word sense disambiguation (Le et al., 2020) or sentiment analysis (Mao and Li, 2021) to detect metaphor. Recent studies adopt metaphor identification theo- ries, including selection preference violation (SPV) (Wilks, 1975, 1978) and metaphor identification procedure (MIP) (Group, 2007), to help design net- works (Su et al., 2021; Choi et al., 2021; Zhang and Liu, 2022; Wang et al., 2023; Li et al., 2023). Although the above methods inspired by SPV and MIP have achieved the state-of-the-art results, they identify metaphors based on the basic meaning of the words and their contexts, which are surface- level clues in contrast to the conceptual meanings of attributes and domains in CMT. Therefore, in ad- dition to using these SOTA methods as strong base- lines for our experiments, we utilize the conceptual attribute and domain information, and develop the computational method to model attribute likeness and domain inconsistency for effective metaphor detection.
Figure Overall architecture our proposed AIDIL framework for word-pair metaphor detection.
Nie instances. The goal of word-pair metaphor detection is to predict the label of each word pair in Die by training a model on Drr-
Method
We propose Attribute IIkeness and Domain Inconsistency Learning framework, namely AIDIL, for word-pair metaphor detection. Fig. shows its overall architecture, which contains three pri- mary components: (1) Attribute siamese network, which obtains the attribute vector via refining simi- lar attributes and filtering irrelevant attributes grad- ually based on two attribute graph sub-networks of source concept and target concept respectively guided by the core word; (2) Domain contrastive learning, which performs contrastive learning on representations of concepts from source and tar- get domains to learn the information of domain inconsistency for better generalization of contex- tual representations; and (3) Classification, which utilizes contextual representations and the attribute vector to predict the label (metaphorical/litrra)
3 Problem Definition
a word pair, we define the verb in a verb-noun pair or the adjective in an adjective-noun pair as the core word. According to CMT, the other noun is the target concept in metaphorical word pair or the source concept in literal word pair. Formally, Di.=((pk- 1)) Ner Nn is the training dataset with Nir instances, where Pk is the k-th word pair and 4 is the label (metaphorical or literal). Every word pair contains core word Woor and a concept word Weon- Dio (ps, 4x) } ]=] is the test dataset with
4.1 Encoding
To train model from good embedding start, adopt the pre-trained language model BERT (Devlin et al., 2019) as the text encoder. Given a word pair p, following Zhong and Chen (2021), we introduce 4 marker tokens ([cor], [/cor], [con] and [/con]) to explicitly mark the boundaries of the core

---

word woor and the concept word Woon in p. The word pair with marker tokens is formulated
Eattrr
yol xil
1=1
1=L
p [cor] Weor [/cor] [con] Weon l/con]-
(1)
you Jaloo
We connect p. the part-of-speech label of core word upos (verb or adjective), the basic defini- tion of core word deor {wf, wh; wN,) with Nr words, the basic definition of concept word doon - {ws, ws, ... w with No words, and other special tokens as sequence of input? T:
1=2
1=2
1 xx
strbbste refirement
guide guide
Layaroaii altnitan hec.emaecclicnnn refnemesi
P purll ydh
drsn feove word)
Danch comeeted OCN
sunnd
T= [CLS] p [SEP] ppos [SEP] deor [SEP] deons
where [CISS] and [SEP] are the classification token and separation token respectively. We use the seg- ment embedding of BERT to distinguish among special tokens, core word and its basic definition, concept word and its basic definition, and part-of- speech token. We use WordPiece split T into tokens and feed them into BERT to obtain con- textual representations H. as well contextual representations of core word howrs concept word hcons and token (CLS] h(CLS):
Aliritai descripiioss
Adiriluie descrpptions
ConcepoNei
ConcopeNet 1
gusoline
Turget cuntugo
comcept
Figure 3: Overview architecture of proposed at- tribute siamese network.
can provide valuable clues for inference, thus we use it as the guided information in the process of refinement. Overall architecture of the attribute siamese network shown in Fig. 3.
H BERT(TT) [h1, ................................ ANn xxd heor Mcool] + Micor)] E Rdd,
(3)
(4)
4.2.1 Attribute Graph Construction
hcon M|con| + /utloon E Rd.
(5)
To obtain the candidate attributes of source and target concepts, search freely available com- monsense knowledge resource ConceptNet (Speer et al., 2017) and acquire phrases in relations with source and target concepts as candidate attribute descriptions. For a metaphorical word pair (e.g., drink gasoline). which contains a core word Wicrr (e.g., drink) and a target concept Wigt (e.g., gaso- line), we can extract set of literal word pairs from the training set (e.g., drink water, drink milk and drink soda), which share the same core word (e.g., drink) with the aforementioned metaphorical word pair. We then randomly sample one (e.g.. drink water) from this set of literal word pairs. The concept word (e.g.. water) in this sampled literal word pair serves as the source concept Wsrc in our method. We define the sequence of target concept and its candidate attribute descriptions as mm)) ",,,.. where mf = W'gtt m is the k-th candidate attribute description extracted from ConceptNet, and the number of candidate tribute descriptions is n Similarly, we define the sequence of the source concept and its candi- date attribute descriptions as MM=mmm;..... m
b[CLS) h1 E R
(6)
where he C TRd is the text embedding of the i-th token in T. is the dimension of embedding, N is the number of tokens T, and ][co]] hojoor- M and hutrcon] are contextual representations of tokens [cor]. |/cor), [con] and [/con] respectively.
4.2 Attribute Siamese Network
To model the attributes constructed by activating imagination and association based on common- sense knowledge, leverage graph neural net- works, which are representative models for captur- ing the imaginative and associative processes in the human brain (Bessadok et al., 2022). Given that similar attributes are scarce, we design a layer- wise refinement process to filter irrelevant informa- tion gradually and learn the most suitable similar propose attribute siamese network with two attribute graph sub- networks refine similar attributes layer by layer on candidate attributes of the source concept and target concept. In addition, the core word is the only context of source and target concepts, which
M'
attributes. Specifically,
where mi = Were- We convert M and M into

---

hidden representations Ex ['-lei...... [n] e
the output of DenseGCN(.), and y R is the attention vector in the 1-th layer.
and - ........ G < Raad with BERT and
liner projection:
Specifically, (efine(-) formulated as:
h BERT((CLS) mLJICL))
(7)
x (W (WCorr Neor + bborr 3 X
(11)
6 hi + beor:
(8)
yg (1) = Cos(X!)) x(!),
(12)
where * C 8), k E [], di E Rdd denotes the representation of [CLS] embedded with BERT for m.. War Rddd and bin Idd are trainable parameters, and is the dimension of node features in the graph.
" Softmax(Normi (g (1))))
(13)
x Norm(X 2 (1 (1"1TT)
(14)
where 1 is a vector of size with all the compo- nents being 1, Weor E Bddd and bi.. Bd are train- able parameters, represents the element seise ma- trix multiplication, Cos(.) calculates cosine simi- larity between the k--th row vector in x and the k-th row vector in X resulting the k-th value in Vg (1) E R., Norm(.) is the layer normalization op- cration, and y E I"" the attention vector in the I-th layer (y/0) is a vector of all the ones). We in- corporate the attention vector in the (I (1-1)-th layer to refine candidate similar attributes in Eq. (13) so that the final selected similar attributes have high attention values in all the layers. Using element- wise matrix product of x and y in Eq. (14), the information of trivial nodes is controlled. We obtain the final attribute matrix Ha Bga with linear projection on the output features of the last layer:
We construct attribute graphs of target concept G' and source concept Go with E and E as node feature matrices respectively. We use an adjacency matrix A Ra** to represent the relations of nodes in G', where n is the number nodes. (4+) = 1 and (A'))+1,1 = represent there exists a relation between target concept and its j- th candidate attribute description (j E [1, n 1D. and (A) = represents the i-th and the j-th candidate attribute descriptions have the same rela- tion with target concept otherwise O. We construct A E p for G* in the same way.
4.2.2 Layer-wise Attribute Refinement
Our attribute siamese network contains two at- tribute graph sub-networks sharing the same ar- chitecture and parameters. Each attribute graph sub-network consists of L attribute refinement lay- ers which refine the similar attribute information layer by layer using the core word guided clue. em-
(hr);: = W Wo (L) + bont,
(15)
where (ho); E RRd is the i-th row vector of Ha at (L) is the i-th row vector of X((,, Wout E Badd and bost E IRd are trainable parameters.
To obtain better graph representation, ploy densely connected GCN (Guo et al., 201 20,9b,a) as the graph encoder in our attribute refinement layers, which has advantages of strengthening fea- ture propagation, alleviating the caisshing-grddien problem and encouraging feature reuse compared to traditional GCN (Kipf and Welling, 2017). The (-th attribute refinement layer contains densely connected GCN and refinement module guided by the embedding of core word hoor:
4.2.3 Attribute Mapping
We employ the above two attribute graph sub- networks on attribute graphs of target concept G' and source concept G respectively, obtain- ing aggregated attribute vectors ha E IRd and hattr E Rd, and design an attribute mapping loss Lattr to learn space where aggregated attribute vectors of target and source concepts are close, which are computed
(!) DenseGCN(X" 1,A), g (1),, () = Reline(X)) (1-1), heor), (10)
(9)
hattr Σ
(16)
where / E [1, L] is the index of attribute refinement layer, A RRaxn is the adjacency matrix, (1-1) E Raxd is the output attribute vectors of the (I - 1)- th layer and serves as input feature matrix of the attribute graph (O)) with nodes, (enseGCN())
h Σ
(ha)i
(17)
Lattr
hattr 112,
(18)
where the superscript & denotes embeddings or val- ues the sub-network target concept, the su-
is the densely connected GCN, x(()) E n x is
perscript s denotes embeddings or values in the

---

sub-network of source concept, (ha); is the i-th row vector of attribute matrix Ha. Di (L) is the i-th scalar value in y() and |- denotes mean squared L.2 norm.
Dataset
Type
Train
Dev
Test
Total
%M
#UC
MOH
SVO
647
45.40
215
TSV
AN
1568
200
200
1968
50.00
687
GUT
AN
8592
53.60
23
SVO
3418
426
4270
50.00
646
VUA-WP
AN
2682
334
334
3350
50.00
604
4.3 Domain Contrastive Learning
Table 1: Statistics of datasets. %. denotes the percent- age of metaphorical samples in the dataset and #UC denotes the number of unique core words. SVO de- notes subject-verb/verbu
According CMT, for the same core word, the concept words in metaphorical word pairs (e.g., gasoline in Fig. 1) can be regarded as part of target domain and the concept words in literal word pairs (e.g., soda and water in Fig. 1) can be regarded as part of source domain. To learn an embedding space with knowledge about the disparity of con- cepts in source and target domains, propose the domain contrastive learning strategy, which makes the representation of concept words similar to the concept words in the same domain (target/source domain) and dissimilar to the concept words in the other domain (source/target domain) for the word pairs with the same core word.
word pair and AN de- notes adjective-noun word pair.
4.4 Classification
Finally, we feed the aggregated attribute vector of target concept hattrr the representation of core word hcor and classification embedding HICLS) into the classifier, and adopt cross-entree cross-entropy loss function to compute classification loss Calass:
s-sotmaaMMMLP(ttt o heor @H(CLS))), (20)
Leluss = (1) log
(21)
Given anchor word pair p (containing core word Woor and concept word Woon) and label 4, the positive word pair pt is the same as the anchor word pair, i.c. |+ = lp+ = p. Follow- ing the previous convention (Gao et al., 2021), we pass the (=p+) to the text encoder by ap- plying the standard dropout twice, then we can obtain two different embeddings of and p re- spectively. In contrast, sample negative word pairs P- {(p D) from the train- ing dataset, which share the same core word with the anchor word pair but have opposite labels, i.e. 4 + 1, (i = Woor Using the text encoder in Section 1.1, we can get the hidden vector heon of anchor concept word Weon, the hidden vector h of positive concept word wion in pt and the hidden vectors H. words { ((won)) ) in P- calculated by Eqs. (3) and (5). The domain contrastive learning loss is computed as:
where @ is the concatenation operation, MLP(-) is two-layer multilayer perceptron with hidden dimension d, f C IR.2 is the predicted probability for all the labels, and I IR² is the ground truth.
4.5 Optimization
We optimize method with classification loss Celasss attribute mapping loss Lattr and domain contrastive learning loss Ladi
L= Celass + >aC + XcLaet
(22)
where A and de are hyper-parameters.
Experiments
5.1 Datasets
)i )= of negative concept
We experiment three publicly available word- pair metaphor datasets, and construct new dataset VUA-WP based on VUA20 (Leong al., 2020). which is the largest benchmark dataset for token- level metaphor detection task. The process of constructing the dataset VUA-WP is illustrated in Appendix A. The three benchmark datasets are as follows: (1) MOH (Shutova et al., 2016) con- tains verbal word pairs. We conduct 10-fold cross- validation for evaluation following the previous convention (Ge al., 2022); (2) TSV (Tsvetkov et al., 2014) is balanced adjective-noun word pair dataset, which contains 1768 word pairs for train- ing and 200 word pairs for testing. We randomly
Lad-- (log
( Aecon) ) Die ef ((hrom)
af (Ircon ho
log
1/2
(hhoo. - (loronn tom)
(19)
where f(31, 92)
+| | is the cosine similar- ity function with temperature parameter T.

---

sample 200 word pairs from the original training set as the development convention (Ge et al., 2022); (3) GUT (Gutiér- rez et al., 2016) is an adjective-noun word-pair metaphor dataset with only 23 unique adjectives. As an adjective has more different noun associa- tions and 98.4% word pairs in GUT never appear in TSV, we use it for testing out-of-domain gener- alization of models trained on TSV. Table shows the statistics of these datasets.
task, which cares about kind of relations be- tween a target word and its context inspired by MIP:
following the previous
MisNet (Zhang and Liu, 2022) is the SOTA method for token-level metaphor detection, which incorporates MIP and SPV into their linguistics enhanced network.'
We also use ChatGPT with advanced prompting strategies baselines. Details about experiments ChatGPT are shown in Appendix D.
5.2 Baseline Methods
We compare our method with several representative methods for word-pair metaphor detection:
5.3 Implementation Details
We use accuracy and macro-average FI for evalua- tion. The knowledge in ConceptNet is organized in {s, r, o) tuple format, where is the phrase subject of the tuple, is the relation of the tuple, and is the phrase object of the tuple. We regard the con- cept word as the phrase subject and search phrase objects which have relations with the phrase sub- ject as candidate attribute descriptions to construct attribute graphs in attribute siamese network. In the segment embedding of the text encoder, 0, 1, 2, 3 denote the special token, the core word and its basic definition, the concept word and its basic definition, and the part-of-speech token respectively. We re- port the mean and standard deviation of runs with different random seeds our experiments. Other details are illustrated in Appendix B.
Multimodal (Shutova et al., 2016) combines visual and linguistic features for prediction;
SSN-SG (Rei et al., 2017) is the first deep learning method for this task;
Concreteness (Su et al., 2020) processes con- crete word pairs with the multimodal model and processes abstract word pairs with the unimodal model;
EMI (Ge et al., 2022) is the SOTA method, which models source and target concepts to benefit metaphor detection via statistic learn- ing and reward mechanism.
We also use two methods for text classification baselines:
5,4 Main Results
TextRCNN (Lai et al., 2015) is recurrent convolutional neural network;
From the experimental results shown in Table 2, we can see that our proposed method outperforms all the baselines, which verifies the effectiveness of modeling attribute likeness and domain incon- sistency simultancously for metaphor detection. Generic methods for text classification (TextRCNN and BERT) perform poorly metaphor detection which needs high-level semantic understanding. MclBERT, MrBERT and MisNet use special net- work designs on pre-trained models inspired by linguistic theories (SPV and MIP), achieving com- parable results with the previous state-of-the-art method EMI. However, SPV and MIP only cap- ture contextual clues of metaphor, neglecting the implicit information of attributes and domains veyed by metaphors. This is the reason why these methods perform worse than EMI and our method.
BERT (Devlin al., 2019) is a pre-trained language model.
We further establish three strong baselines adapted from the representative methods in token- level metaphor detection, by replacing the target word and its context in token-level methods with the core word and the concept word in our method respectively. The details of these methods as follows:
MelBERT (Choi al., 2021) proposes model based on the pretrained language model inspired by metaphor identification theories MIP and SPV: MrBERT (Song et al., 2021) regards metaphor detection as a relation classification
For a fair comparison, we replace marker tokens in Mr- BERT with ours. We replace the basic usage of words and POS tags in MisNet with our word definitions and POS tags.

---

MOH
TSV
VUA- WUP-WP
Method
F1
Acc.
F1
Acc.
F1
Acc.
TexIRCNN (Lai et al., 2015)
68.94 0.82
69.20
0.86
1.33 72.25
73.63 = 1.16
60.95 1.45
64.08 a 1.13
BERT* (Devlin et al., 2019)
70.57
71.22 1.59
81.70
81.90 0.58
74.29 1.09
74.42 2.94
MelBERT (Choi et al., 2021)
74.63 1.13
74,98 1.14
86.46 + 1.31
86.50 =: 1.30
75.63 0.80
75.71 2
MrBERT (Song et al., 2021)
74.03 0.52
74.35 0.550
80.84
2.02
81.00 2.02
76,02 1.37
76.05 1.37
MisNet (Zhang and Liu, 2022)
75.09 0.56
75.61 +0.58
84.39 2.52
84,50 2.43
75.72 0.97
75.79
Multimodal (Shutova al., 2016)
75.00
79.00
SSN-SG (Rei et al., 2017)
74.20
74.08
80.10
82,20
Conercteness (Su ct al., 2020)
68.00
64,00
85.00
84,00
EMI (Ge et al., 2022)
75.60
75.90
86.60
87.00
Ours (BERT)
79.52
0.55
79.83
89
90.00
1.52
76.40
76.47 0.10
Ours (RoBERTa)"
79.35 0.45
79. 2.41
90.48 0.84
90.80 = 0.77
75.67 2041
75.74 0.43
Table 2: Comparison between our method and baselines. The best results in bold font and the best results of baselines are underlined. f and denote the method using BERT and RoBERTa as text encoder respectively.
MOH
TSV
VUA-WP
Ablation
Fl
Acc.
F1
Acc.
F1
Acc.
Ours (BERT)
79.52 + 0.55
79.83
0.60
89,98 # 1.53
90.00
1.32
76.40 * 0.11
76.47 + 0.10
w/o Litt
78.90 0.80
79.19
0.84
85.78
85.83
72.78
72.95
w/o Catte
79.47 0.39
79.78 0.37
85,27
1.03
85,33 1.03
74.01 1.55
74.11 1.53
wlo attribute siamese network
79.30 0.51
79.62 0.43
84.04 = 0.82
84.10 0.86
74.23 + 1.43
74.53 1,26
wlo core-word guidance
79.07 0.80
79.47 0.80
86.06 1.69
86.
74.31 0.81
7.65
wlo DenseGCN
78.90 0.51
79.22 0.62
87.39 = 1.79
87.40 1.80
75.00 0.74
7,.10
wlo part-of-speect tag
79.4I 10.51
77.771 0.46
88.64 0.88
88.67
75.50 0.31
75.50
who marker tokens
78.88 0.66
79.19 0.68
88.63 0.82
88.67 + 0.85
75.51 1.30
75.58 1.29
wlo segment ID)
79.18 0.73
79.53 + 0.75
84.66 + 1.18
84,70 1.16
74.47 0.10
74.55 +,92
wlo definition
77.80 0.300
78.10 2.13
85.34 2.64
85.40 2.61
74.62 2.10
74.89 1.71
Table 3: Results of ablation study. The best results are in bold font.
which indicates CMT is more explainable than both SPV and MIP. Although the previous SOTA method EMI considers the incongruity of source and target concepts, its performances fall far behind ours, which shows the similar attribute information they ignore is important for metaphor detection.
[/con]) and uses the embedding of first and last tokens in core/concept word to replace the embed- ding of [cor]/[con] and [/cor/[[con] respectively; (h) w/o segment ID uses and to represent spe- cial token (LCLS) and (SEP)) and other tokens in segment embeddings; (i) w/o definition ablates the basic definitions of core and concept words in the input.
5.5 Ablation Study
We conduct the ablation study to evaluate the im- pact of components in method, using the fol- lowing variants: (a) w/o Cad removes the domain contrastive learning strategy our method; (b) w/o Lattr removes the attribute mapping loss from final training objective; (c) w/o attrihute siamese net- work slashes the whole attribute siamese network from our method; (d) w/o core-word guidance uses the attribute siamese network without core word as guidance to replace the attribute siamese network our method; (e) w/o DenseGCN replaces the DenseGCN in our method with traditional GCN; (f) w/o pfrt-of-speech tag removes the part-of-speecd information the input; (g) w/o marker tokens drops the special markers ([cor], [/cor], [con] and
Experimental results in Table 3 show that re- moving the domain contrastive learning strategy reduces the performances, which verifies the ef- information
fectiveness of fomain-inconsistie for metaphor detection. To evaluate the effective- ness of attribute siamese network, we first directly ablate attribute mapping loss from our method, re- sulting in significant performance declines. Then we directly remove the attribute siamese network, the performances also drop consistently. Above two variants show the effectiveness of attribute in- formation for metaphor detection. The removal of core-word guidance strategy attribute siamese network reduces the performances, which veri- fies that the core word can guide our method to

---

Method
Acc.
MelBFRT (Choi et al., 2021)
81,72 0.75
81,74 0.74
MrBERT (Song 2021)
76.68
76.71
MisNet (Zhang and Liu, 2022)
81.03 144
81.06 1.43
Ours (BERT)
86.32 1.29
86.35 1.31
Bati
ligtt
Table Comparison between our method and baselines trained on TSV training set and tested on GUT.
(a) MOH
(b) TSV
VUA-WP
Figure 4: Results of our proposed method with different hyper--aramettrs for losses.
learn similar attributes between source and target concepts for metaphor detection. When replacing the DenseGCN with traditional GCN, the perfor- mances on all three datasets decrease, thus verify- ing the advantage of DenseGCN. In addition, when removing the tricks in our method, including the part-of-speech tag, marker tokens and segment ID, models perform worse, as these tricks are important for our method to distinguish information about the core word and the concept word. When definitions are aborted, our method tends to overfit on rela- tively small word-pair datasets, leading to poorer performances. Besides, the performance drops of all the ablation experiments on MOH, especially when removing the attribute mapping loss, are rela- tively slight. This phenomenon could be attributed to the small dataset size of MOH, which only has 647 samples. However, the ablation experiments conducted on the other two larger datasets result in significant performance drops, verifying the effec- tiveness of the different components in our method.
80
90-
Fl (55)
70
Number layers
Number of layers
Number layers
(u) MOH
(b) TSV
(c) VUA-WP
Figure Results of our method with different number of attribute refinement layers.
our method achieves the best performance when Aa = 0.01 and A. = 0.001. MOH is relatively small than other datasets Icading to unstable perfor- mances with different hyper-parameters.
Number of Attribute Refinement Layers To analyze the impact of the number of attribute re- finement layers in attribute siamese network, we experiment on varying the number attribute re- finement layers from to 5. The results are shown in Fig. 5. Models on three datasets all achieve best performances with three attribute refinement layers, indicating that too shallow network is unable to cap- ture attribute information while too deep network with plentiful parameters may lead to overfit.
5.6 Generalization Ability Analysis
To evaluate the generalization ability of our method, we use models trained on TSV training set to test on GUT. The experimental results in Table show that our method outperforms all the comparative meth- ods, thus verifying that our method shows promis- ing ability of -otmain generalization to han- dle unseen concepts. Our method mines the im- plicit information of attributes and domains, which is more explainable and effective than comparative methods using surface-level clues of metaphor.
Conclusion
In this paper, we propose an attribute likeness and domain inconsistency learning framework for word- pair metaphor detection. Inspired by the conceptual metaphor theory, our framework models attribute likeness with an attribute siamese network via layer-wise attribute refinement graphs and learns domain inconsistency with domain contrastive learning strategy. Experimental results show that our method significantly outperforms the previous word-pair and token-level SOTA methods, verify- ing the effectiveness of our proposed framework for metaphor detection.
5.7 Hyper-parameter Analysis
Hyper-parameters for Losses We tune the hyper-paramedd for attribute mapping loss X and domain contrastive learning loss De using grid scarch (i.e. [0.0001, 0.001, 0.01, 0.1, 1, 10, 100]) on three datasets. Experimental results in Fig. show that our method achieves the best perfor- mances on TSV and VUA-WP when A₂ = 0.001 and X = 0.001, and the performances drop sharply when Xa and A, are too large. For MOH,

---

Limitations
Jacob Devlin, Ming-Wo Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the Conference of the North American Chapter of the Association for Com- putatiooaa Linguisties, pages 4 4171-4186.
Our work has some limitations. Firstly, the gen- eralization of our method on other types of word pairs needs to be further explored. Verb-noun and adjective-noun word pairs are the most frequently used and basic linguistic forms in metaphorical ex- pressions. Current studies focus above two types of word pairs, and our work follows this conven- tion. In piratical situations, metaphor also occurs in verb-adverb word pairs, which needs further study in the future. In addition, we use ConceptNet as the external knowledge resource to obtain candidate attributes, whose types of relations are limited. The effectiveness of our method using other external knowledge resources can be further explored.
Tianyu Gao, Xingcheng Yao, and Danqi Chen, 2021. SimCSE: Simple contrastive learning of sentence embeddings. In Proceedings of the Conference on Empirical Methods in Natural Lunguage Processing, pages 6894-6910.
Mengshi Ge, Rui Mao, and Erik Cambria. 2022. Ex- plainable metaphor identification inspired by concep- tual metaphor theory. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 10681 10689.
Pragglejaz Group. 2007. MIP:
A method for identifying metaphorically used words in discourse. Metaphor and symbol, 22(1):1-39
Acknowledgements
This work is supported in part by the National Nat- ural Science Foundation of China under Grants #72293575, #62206287 and #72225011. We thank the anonymous reviewers for the valuable com- ments.
Zhijiang Guo, Yan Zhang, and Wei Lu. 2019a. Atten- tion guided graph convolutional networks for relation extraction. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 241-251.
Zhijiang Guo, Yan Zhang, Zhiyang Teng, and Wei Lu. 2019b. Densely connected graph convolutional net- works for huph-to-sequence learning. Transactions of the Association for Computational Linguistics, 7:297-312.
References
Alaa Bessadok, Mohamed Ali Mahjoub, and Islem Rekik. 2022. Graph neural networks in network neu- roscience. IEEE Transactions on Puttern Analysis and Machine Intelligence, 45(5):5833-5845
F. Dario Gutiérrez, Ekaterina Shutova, Tyler Marghetis, and Benjamin Bergen. 2016. Literal and metaphor- ical senses in compositional distributional semantic models. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 183-193.
Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chai- tanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. COMET: Commonsense transformers for auto- matic knowledge graph construction. In Proceedings of the Annual Meeting of the Association for Compu- tational Linguistics. page 4762-4779.
Thomas N. Kipf and Max Welling. 2017. Semi- supervised classification with graph convolutional networks. In Proceedings of the International Con- ference on Learning Representations, pages 1-14.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In Proceedings of the Annual Conference on Neural Information Processing Systems, pages 1877-1901.
Takeshi Kojima, Shixiang (Shane) Gu, Machel Reid, Yutaka Matsuo, and Yusuke Twasawa. 2022. Large language models are zero-shot reasoners, In Proceed- ings of the Annual Conference on Neural Information Processing Systems, pages 22199-2231 22211...
Luana Bulat, Stephen Clark, and Ekaterina Shutova. 2017. Modelling metaphor with srributeebasedd se- manties. In Proceedings of the Conference of the European Chapter of the Association for Computa- tional Linguistics, pages 523-528.
Siwei Lai, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Recurrent convolutional neural networks for text clas- sification. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 2267-2273.
George Lakoff and Mark Johnson. 2008. Metaphors we live by. University of Chicago press.
Minjin Choi, Sunkyung Lee, Eunseong Choi, Heesoo Park, Junbyuk Lee, Dongwon Lec, and Jongwuk Lee. 2021. MelBERT: Metaphor detection via contextual- ized late interaction using metaphorical identification theories. In Proceedings of the Conference of the North American Chapter of the Association for Com- putational Linguistics, pages 1763-1773.
Duong Le. My Thai, and Thien Nguyen. 2020. Multi- task learning for metaphor detection with graph con- volutional neural networks and word sense disam- biguation. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 8139-81444 11466

---

Chee Wee (Ben) Leong, Beata Beigman Klebanov, Chris Hamill, Egon Stemle, Rutuja Ubale, and Xi- anyang Chen. 2020. A report on the 2020 VUA and TOEFL metaphor detection shared task. In Pro- ceedings of the Workshop on Figurative Language Processing, pages 18-29.
Chang Su, Kechun Wu, and Yijiang Chen, 2021. En hanced metaphor detection via incorporation of exter- nal knowledge based on linguistic theories. In Pro- ceedings of the Annual Meeting of the Association for Computational Linguistics, pages 1280-1287.
Weiwei Sun, Shuyu Guo, Shuo Zhang, Pengjie Ren, Zhumin Chen, Maarten de Rijke, and Zhaochun Ren. 2023. Metaphorical user simulators for evaluating task-oriented dialogue systems. ACM Transactions Information Systems, 42(1):1-29.
Yucheng Li, Shun Wang, Chenghua Lin, Frank Guerin, and Loic Barrault. 2023. FrameBERT: Conceptual metaphor detection with frame embedding learning. In Proceedings of the Conference of the European Chapter of the Association for Computational Lin- guistics, pages 1558-1563.
Yulia Tsvetkov, Leonid Boytsov, Anatole Gershman, Eric Nyberg, and Chris Dyer. 2014. Metaphor detec- tion with cross-lingual model transfer. In Proceed- ings of the Annual Meeting of the Association for Computational Linguisties. pages 248-258.
Rui Mao and Xiao Li. 2021. Bridging towers of multi- task learning with a gating mechanism for aspect- based sentiment analysis and sequential metaphor identification. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, pages 13534-13542.
Peter Turney, Yair Neuman, Dan Assaf, and Yohai Co- hen. 2011. Literal and metaphorical sense identifica- tion through concrete and abstract context. In Pro- ceedings of the Conference on Empirical Methods in Natural Language Processing, pages 680-690.
Rui Mao, Chenghua Lin, and Frank Guerin. 2018. Word embedding and WordNet based metaphor identifica- tion and interpretation. In Proceedings of the Annual Meeting of the Association for Computational Lin- guistics, pages
Shun Wang, Yucheng Li, Chenghua Lin, Loic Barrault, and Frank Guerin. 2023. Metaphor detection with ef- fective context denoising. In Proceedings of the Con- ference of the European Chapter of the Association for Computational Linguistics, pages 1404-144000 1409.
Marek Rei, Luana Bulat, Douwe Kiela, and Ekaterina Shutova. 2017. Grasping the finer point: A super- vised similarity network for metaphor detection. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1537-1546.
Yorick Wilks. 1975. A preferential, pattern-sekking semantics for natural language inference. Artificial Intelligence, 6(1):53-74
Omid Rohanian, Marek Rei, Shiva Taslimipoor, and Le An Ha. 2020. Verbal multiword expressions for identification of metaphor. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 2295.
Yorick Wilks. 1978. Making preferences more active. Artificial Intelligence, 1973)199-2233
Shenglong Zhang and Ying Liu. 2022. Metaphor de- tection via linguistics enhanced Siamese network. In Proceedings of the International Conference on Computational Linguistics, pages 4 4149-4159.
Ekaterina Shutova, Douwe Kiela, and Jean Maillard. 2016. Black holes and white rabbits: Metaphor iden- tification with visual features. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, pages 160-170.
Zexuan Zhong and Danqi Chen. 2021. A frustratingly easy approach for entity and relation extraction. In Proceedings of the Conference of the North Ameri- can Chapter of the Association for Computational Linguistics, pages 50-61.
Wei Song. Shuhui Zhou, Ruiji Fu, Ting Liu, and Lizhen Liu. 2021. Verb metaphor detection via contextual relation learning. In Proceedings of the Annual Meet- ing of the Association for Computational Linguistics, pages
Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of gen- eral knowledge. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, page 4444 4451.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: simple way prevent neural networks from overfitting. The Journal of Machine Learning Research, 1929
Chang Su, Weijie Chen, Ze Fu, and Yijiang Chen, 2020. Multimodal metaphor detection based on distinguish- ing concreteness. Neurocomputing 429: 163.

---

goy, or dep.
POS
If the adjective in an adjective-noun word pair is metaphorical in VUA20, we label this word pair as metaphorical in VUA-WP:
Type
DEP
Wcoo
Weon
Woo
Weom
dobj
gov,
dep.
v.
SVO
nsubjpass
gov.
dep.
v.
n.
xsubj
gov.
dep.
V.
agent
gov.
dep.
v.
If all the words in an adjective-noun word pair are literal in VUA20, we label this word pair as literal in VUA-WP.
AN
amod
dep.
gov.
adj.
n.
nsubj gov.
dep.
adj.
Table 5: The part-of-speech and dependency informa- tion of subject-verb/verb-obyee noun (AN) candidate word pairs. DEP is short for stanford typed dependencies, POS is short for part-of- speech, gov. is short for governor, dep. is short for dependent, Weor is the core word and Meon is the con- cept word.
(SVO) and adjective-
The number of literal word pairs is far more than the number of metaphorical word pairs. We ran- domly sample literal word pairs to get a balanced dataset VUA-WP. After that, we randomly divide all the word pairs into training, development and test sets with ratio of 8:1:1.
Relation
Description
B Implementation Details
HasProperty
A has as property
SymbolOr
A symbolically represents
CausesDesire
makes someone
We use the Stanford CoreNLP to get part-of-specch information of word pairs. The basic definition of word with its POS tag in a given word pair is extracted from the dictionary Vocabulary", using the first definition of this word under the same POS tag. To construct attribute graphs, we use Concept- Net to extract the attribute descriptions of concept words. If ConceptNet doesn't 't contain a source or target concept, we use COMET (Bosselut et al.. 2019), generative language model fine-tuned ConceptNet (Speer et al., 2017). as a substitute get attribute descriptions. The relations Concept- Net we use are shown in Table 6. We get data from ConceptNet using the REST API and we only search English terms in ConceptNet. We use bert- base-uncased or roberta-base as the text encoder. We implement our method based on Pytorch". We use gradient clipping avoid exploding gradients and the max norm of the gradients is 1. We use AdamW? as our optimizer and the coefficient of weight decay 0.01. We train our models for 10 epochs with the learning rate of text encoder as Ir Irenexter and the learning rate of other components as lTere. encoder. We use Dropout (Srivastava et al., 2014) to prevent overfitting and the dropout rate is 0.5. We train all our methods one NVIDIA GeForce RTX 3090 GPU. For each dataset, the model giving best performance of macro-average Fl FI-score in the development set is used for test
RelatedTo
related
BA
A is a subtype or specific instance of
ParOr
A a part of
belongs
UsedFor
used for
CapableOr
Something that can typically do A a typical location for 8
AtLocation
Cauee
A and B are events, and is typical for to cuuse huppens us subovent event that begins with subevent A is event that concludes with suhevent B In order for happen, needs to happen
HaSSbbvvnt
and events,
HusFinsSubevvnt
Hasd astShheenttt
HasPrerequisite
RcciivesAction MadsOf
can be done
made
Table 6: Relations in ConceptNet used for method. A denotes the phrase subject and B denotes the phrase object.
Dataset Construction
We construct a word-pair metaphor dataset VUA- WP from VUA20 (Leong et al., 2020), which is the largest benchmark dataset for token-level metaphor detection task. Each word in sentences of VUA20 is labelled as metaphorical or literal. We use Stan- ford CoreNLP' to extract all the dependency rela- tions and collect all the candidate word pairs with possible dependency relations according to Table 5, After that, we label these word pairs with four rules:
If the words in a subjert-verb/verb-ojjeett word pair are metaphorical in VUA20, we label this word pair as metaphorical in VUA- WP;
2 https:/wwwwwwccahlaayyycck www.wwlaaa.c.ooommooooo
https://githubbbo http:://hh...ooooooooooom///soss
If all the words in a subject-verb/verb-on word pair are literal in VUA20, we label this word pair as literal in VUA-WP:
hhtps://uugiingface.. htwp://www....bbbebbbbuuuuaaaaaa
2 http::/hhuuggnngaaaccccoobbbbbbt
'nttp://pyyorcch.or/
Adam AdamW.htm W.tml

---

Value
Notation
Description
MOH
TSV
VUA-WP
N
140
140
140
maximum length of text tokens
Trase oncoder
5e
Se-
5e
learning of components except the encoder
Wencoder
56
56
5e
learning rate number of layers of attribute siamese network
enender
3
n
5
5
number of negative samples in domain constrastive learning dimension of text embedding
d
768
768
768
d
768
768
768
hidden dimension of MLP
d
300
dimension node features layer-wise refinement graph
n
100
100
100
mumber of maximunn nodes in layen-wise refinement graph
2
2
number of sublayers in densely connected GCN layer
0.05
0,05
0.05
lemperature of cosine similarity function
0.01
0.001
0.001
hyper-paraneter for attribute mapping loss
0.001
0.001
0.001
hyper- paramcer for domain contrastive learning loss training time
T Times
about 40 minutes
about 30 minutes
about 45 minutes
Table 7:
values in our proposed method.
Word pair
Target concept
Relation
Attribute
thinking
Causes (causes)
new thought
fresh thinking
thinking
UsedFor (is used for)
A1
thinking
Causes (causes)
new perspectives
candidate
RelatedTo (is related
hopeful
bright candidate
candidate
IsA (is type of)
person
candidate
RelatedTo (is related to)
Clinton
effort
Related (is related
force
strong effort
effort effort
RelatedTo (is related to)
trying hard
RelatedTo (is related to)
endeavour
Table 8: Cases of metaphorical word pairs and corresponding attributes with the top three attention values in the last layer of our attribute siamese network.
set. We run experiments with extensive hyper- parameter search and provide details of the best model parameters in Table 7. We experiment with of {1, 3, 4, 5), A of (0.0001, 0.001, 0.01, 0.1, 1, 10, 100), and >, [0.0001, 0.001, 0.01, 0.1, 1,
force, trying hard and endeavour, we can infer that strong effort in Table 8 implies putting great effort into something or trying hard on something.
D Experiments ChatGPT
10, 100).
Implementation Details We used OpenAl API (gpt-3.5-turbo) for testing. A good prompt es- sential for generative large language models. We explore different prompts with some advanced prompting strategies, including the standard zero- shot prompting, the zero-shot chain-of-thouge (CoT) prompting (Kojima et al., 2022) and the stan- dard few-shot prompting (Brown et al., 2020). For answer cleansing, pick up the first "yes" or "no" encountered in the answer given by ChatGPT af- ter converting all the uppercase characters in the answer string into lowercase characters.
Case Study
To show how the attribute siamese network cap- tures similar attributes of source and target con- cepts, we provide several instances and their at- tributes with the top three attention scores in the final layer of our attribute siamese network in Ta- ble 8. We can see that our attribute siamese network can capture plausible attributes of concept words which assist in understanding metaphors. Consid- cring the attributes new thought and new perspec- lives, can infer that fresh thinking in Table implies new thoughts or perspectives. Consider- ing the attribute hopeful, we can infer that bright candidate in Table implics a candidate who may have a hopeful future. Considering the attributes
Prompt Design Table 10 summarizes a list template prompts used for the experiments with different prompting strategies on ChatGPT.

---

Scientific artifact
License
fluent answers but sometimes may make incorrect or vague judgments on word-pair metaphor identi- fication.
roherta-base
license
bert-base-uncased
Apache-2.0
COMET
Apache-2.0
gpt-3.5-turbo
API license
E Licenses of Scientific Artifacts
MOH
Unspecified
TSV
ODbL-1.0 license
The licenses of the scientific artifacts we used are shown in Table 9.
GUT
Unspecified
VUA20
CC BY-SA 3.0
ConceptNet
CC BY-SA 4.0
VUA-WP
MIT license
Stanford CoreNLP
GNU General Public Li-
cense (v2 or later)
Table 9: Licenses of the scientific artifacts in this paper.
Experimental Results Table 11 shows the com- parison between method and ChatGPT using different prompt strategies our datasets. Al- though GhatGPT baselines use multiple advanced prompting strategies, they struggle on this task and their performances are far behind ours. addi- tion, the performances of ChatGPT on word-pair metaphor detection are sensitive to the design of prompts.
Failure Analysis To further explore why Chat- GPT performs poorly on word-pair metaphor de- tection, we observed the failure cases in our exper- iments with ChatGPT using zero-shot prompting TSV dataset. Our observation shows that these cases be roughly classified into four groups: wrong conjecture, uncertain judgment, contextual ambiguity, and common collocation. Table 12 gives some failure cases in each group. In the first group, ChatGPT makes wrong conjectures about the answers to the questions with unreasonable ex- planations. In the second group, ChatGPT makes uncertain judgments by stating metaphorical and literal word pairs. In the third group, ChatGPT as general language model is also liable make mistakes when the contextual information of word pairs is unavailable. Besides, the fourth group, as some metaphorical word pairs such as "dirty word" and "heavy tax" are commonly used in everyday language, ChatGPT often misinterprets them as lit- eral expressions. Wrong conjecture is the major reason for the failure in predicting the metaphori- cal tendencies with ChatGPT, which accounts for 55.55% of all the failure cases. Common colloca- tion, uncertain judgment and contextual ambiguity comprise 17.46%, 14.29% and 12.70% respectively. general, ChatGPT is often good at generating

---

Prompting strategy
No.
Template prompt
Does the word pair "[word pair]" express metaphorical meaning?
2
Does the word pair "[word pairj" express metaphorical meaning? Answer me with "yes" or "no".
3
Does the word pair "[word par] express metophorical meaning? Give me an answer solected from "yos" or "no"
4
Is the word pair "[word pair| a imeeapphrical expression?
word pair "word pair|" mnstaphoriicll expression? Answer me with "yes" "no". Is the word pair "(word pair]" metaphorical expression? Give answer selected from "yes" or "na".
7
Does the word pair "[word pair]" use metaphor?
shot
8
Does the word pair "[word par]] use metaphor? Answer me with "yos" or 'no".
9
Does the word pair "[word pair]" use metaphor? Give me an answer selected from "yes" or "no".
10
Given the word pair "[word pair]] determine if this word pair a metaphorical expression.
Given the word pair "[word pair]". determine this word pair is metaphorical expression. Answer me with "yes' or "na".
11
Given the word pair "[word pair]", determine word pair is metaphorical expression. Give me an answer selected from "yes" or "no".
12
Does the word puir "[word pair]" express metapporical meaning? Answer me with "yes" or "no" step by step.
2
Is the word pair "woord pair a metapharical expression? Answer me with "yes" or "no" step by step.
Zero CaT
Does the word pair "[word pair]" use metaphor? Answer me "yes" or "no" step by step.
Given the word pair "[word pair]", determine this word pair is metaphorical expression. Answer me with "yes" or "no" step by step.
4
Question: Does the word pair "Iueeapharical word pait express mataphorical macaning? Answer: yes.
Question: Does the word pair "[literal word pair]" express metaphorical meaning? Answer: no.
Question: Does word pair "[word pair]" express metaphorical meaning? Answer:
Question: Does the word pair [metaphorical word pairj' express metaphorical meaning? Answer with "yes" "no". Answer: yes.
2
Question: Does the word puir "[iieral word pair]' express metaphorical meaning? Answer with "yes" "no". Answer: no.
Question:: Does the word pair wword pair]" express metaphorical meaning? Answer me with "yes" or "no", Answer:
Question: Does the word puir "Inetsphooical werd paisl' express metapprorical meaning? Give me an answer selected from "yes" "no". Answer: yes.
Question: Does the word pair "[literl word pair"" express metaphorical meaning? Give me an answer selected from "yes" or "no". Answer:
Question: Daes the word pair "[word puic!" express metaphorical meaning? ? Give me an answer selected from "yes"
"no". Auswer:
Question: Is the word puiz "Iuetaphorscal word pair]" a metaphorical expression? Answer: yes.
4
Question: Is the word pair "[litral word pair]" metaphorical expression? Answer:
Question: Is the word pair "[word pair]" a metaphorical expression? Answer:
Question: Is word pair "Imetaphorical word paiel" metaphorical expression? Answer me "yes" or "no".
Answer:
Question: Do.
Is word pair "[literll word pair]" metaphorical expression? Answer me with "yes" or "no". Answer:
Question: Is word pair '[word pair metapborical expression? Answer me with "yes" "av". Answer:
Question: Is the word pair "Inettpporiical word pain] metapherical expression? Give me answer solected from
"yes" or "no". Answer: yes.
Question: Is the word pair "[iteral word pair]" metaphorical expression? Give me an answer selected from "yes" or "mo". Answer: no.
Question:: Is the word pair "[word paiel" metaphorical expression? ? Give me an answer selected from "yes" "pa".
1-shot
Answer:
Question: Does the word pair "Imetaphorical word par] use metaphor? ? Answer: yes.
7 Question: Does the word puir "Iliteral word paiel" use metapher? Answer: no.
Question: Doss the word pair "[woed pair]" use metaphor? Answer:
Question: Does the word pair " [metaphorical word pair]' use motaphor? Answer me with "yes" or "no". Answer: yes.
Question: Does the word pair "[literal word pairl" use metaphor? Answer me with "yes" or "no". Answer: no.
Question: Does the word pair "[word pair]" use metaphor? Answer me with "yes" "no". Answer:
Question: Does the word pair "Imetaphoricall word puir)" use metaphor? Give me an answer selected from "yes" or "no". Answer: yes.
Question: Does the word pair " "[iitrrll word paic|" use mataphor? Give me an answer selected from "yes" or "no".
Answer:
Question: Does the word pair "[word pairl" use metaphar? Give me an anewer selected from "yes" or "no", Answer:
Question: Given the word pair "[metaphorical word pair]", desermine this word pair is metaphorical expression. Answer: yes.
10
Question: Given the word paic "literal word pairl" determine if this word pair is a metaphorical expression. Answer: no.
Question: Given the word pair "[word pairl', determine if this word pair metaphorical expression. Anewer:
Question: Given the word pair "Imetaphorical word pair]" desermine if this word pur is metophorical expression. Answer
"yes" or "no". Answer: yes.
Question: Given the word pair "[literl word pair]" determine if this word pair a metophorical expression. Answer me with "yes" or Answer: no.
11
Question: Given the word pair "[word pair]", determine if this word pair is metaphorical expression. Answer me
with "yes" "no". Answer:
Question: Given the word pair "[metaphorical word pairl" desermine if this word par metaphorical expression. Give me an answer selected from "yes" or "no", Answer: yes.
Question: Given the word pair "[literal word pairl". determine if this word pair is a metaphorical expressioo. Give me an answer selected
12
'yes" or "no". Answer: no.
Question: Given the word pair "[word pair]", determine if this word pair is metaphorical expression. Give me an answer selected from "yes" "oo". Answer:
Table 10: Prompt design hased on different prompting strategies in ChatGPT experimentation. [word pair] denotes the input slot of the word pair that needs to be tested. [metaphorical word pair] denotes the input slot of the metaphorical word pair which is randomly sampled from the training dataset. [literal word pair] denotes the input slot of the literal word pair which randomly sampled from the training dataset.

---

MOH
TSV
VUA-WP
Method
No. of template prompt
F1
Acc.
F1
Acc.
F1
Acc.
51.89
59.66
65.72
68.50
38.55
51.58
62.02
65.22
73.38
74.50
44.72
53.68
59,45
63.52
70.44
72.00
44.43
54.34
48.91
57.81
72.22
73.50
42.76
53.95
$2.35
59.35
73.79
74.50
42.68
53.29
50.67
58.58
71.17
72.50
41.53
53.03
ChatGPT Zero-shot
55.72
60.43
60.94
64.00
40.43
52.63
$2.28
57.65
61.39
64.00
43.00
53.55
54.05
59.66
60.17
64.00
40.60
52.63
10
43.33
54.71
63.76
66.50
41.50
53.55
11
51.60
58.89
66.66
68.50
44.11
54.08
12
56.54
61.21
74.90
75.50
46.46
54.47
63.55
63.83
68.16
69.50
55.53
55.53
56.39
56.41
72.38
73.00
53.76
54.34
ChatGPT Zero-shot CoT
55.12
56.88
61.54
65.00
56.18
56.18
$2.79
53,94
75.49
75.50
55.49
57.37
60.23
64.91
69.33
70.50
42.34
52,50
$7.26
62.75
73.49
74.50
$2.50
54,90
61.05
71.03
72.50
42.44
53.42
51.27
71,20
73.00
38.92
51.58
53.39
60.74
68.78
70.50
40.83
53.03
50.14
58.89
62.81
66.50
39.21
52.11
ChatGPT I-shot
49.76
58.58
67.38
69.50
42.11
53.42
$1.97
59.97
63.54
66.50
42.60 22.60
53.68
45.45
56.26
61.43
65.50
37.63
51.32
10
52.63
60.28
75.84
76.50
42.43
53.68
11
52.86
60.59
70.13
72.00
40.58
52.89
12
48.74
58.27
64.84
68.00
40.91
53.16
65.00
66.31
63.49
67.00
48.29
54.87
$9.80
62.13
68.62
70.50
48.82
55.00
62.27
64.30
72.20
73.00
48.26
53.68
63.51
66.00
$7.09
62.50
46.62
54.34
60.06
62.60
68.45
70,50
46.53
54.21
$7.31
60.43
65.87
68.00
48.10 10
54.61
ChatGPT 5-shot
59.44
62.44
63.06
66.50
48.07
54.87
60.31
64.14
67.72
69.50
49. 13
55.13
55,55
59.97
63.10
65.50
49.53
55.39
10
63.54
65.69
71.03
72.50
49.05
54.74
54,91
59.66
68.45
70.50
47.21
55.00
12
55.58
61.05
65.28
68.00
43.98
52.76
Ours (BERT) Ours (RoBERTa)
79.52
79.83
89,98
90.00
76.40
76.47
79.35
79.74
90.48
90.80
75.67
75.74
Table 11: Comparison between our method and GhatGPT. The best results are in bold font and the best result of different template prompts for every prompting strategy underlined.

---

Category
Word pair
Prompt question
Answer of ChatGPT
Result of AIDII
mad man
Does the word pair "mad man' express
Yes, the word pair "mad man" can have metaphorical menn- could he used describe someone who behaves inationally or impulsively. like a "mad scientist" who takes
literal (V)
metaphorrica mean-
ing?
risks in their experiments, "mad genius" who has uncon-
Wroong conjectire
ventional ideas. (X)
blunt homesty
word pair "blunt honesty" eX- press metaphorical micannng?
metaphor. description of bonesty that is direct and straightforward. without any attempt soften sugarcoat truth. (f)
pair
honesty"
metaphorical
young moon
Does the word pair "young moon" ex- press metaphorical meaning?
It is possible for "young moon' to express metaphorical meaning, depeading the context which used. For example, phmsse nings, growth, or innocence. However, it could also simply visible thin
metaphorical (V)
to represent new hegin-
refer the phase
crescent in the sky. (X)
Uncertain judgmenz
red alert
Does word pair "red alent" express metaphorical mean- ing?
As an Al language model, not have personal opinions. or foelings, but can tell you that the word pair "red alent" literal meaning related emergency situations, such as the highest level of waming readiness. However, it can situations
metaphorical (V)
metapborically to urgeney, danger, or crisis outside the context of emergencies.
white anger
Does word "white anger" ex metaphorical meaning?
understand context or the intention behind the phrase "white anger". Without furthrr context, phaase does metaphorical meaning. However, it important that the phras could have a different connetation or meaning. depending on the with it. (X)
langunge model, do
metaphorical
cultural references associoled
Contestual umbiguity
breezy disregard
Does word pair "breezy disregard" express metaphori- cal meaning?
As an Al language model, cannot definitively determine if given phrase expressis metaphorical meaning without context
metaphorical (V)
the lion of "breezy" and "disregard" may suggesi metaphorical meaning that suggests an easy thing important or scrious. (A)
dismissal heing used.
dirty word
Does the word pair "dirty word" press metaphorical meaning?
No, the word pair "dirty word" is literal expression. It metaphorical (V)
used describe word that considered vulgar indecent in society. (X)
Common collocation
heavy tax
Does the word pair "heavy tax" express metaphorieal mean-
No, the word pair "heavy tax" expresses a literal meaning, metaphorical (V) needs paid.
referring to ligh amount
ing?
Table 12: Failure cases in experiments with ChatGPT using zero-shot prompting TSV dataset.


Reviews:
[{"review_id": "Ed0l6CRJUW", "reviewer": "Reviewer_Kgcm", "summary": "", "questions": "1. As far as I know, the input context of token-level metaphor detection methods are sentences, is it appropriate to directly use token-level methods for word-pair metaphor detection？\n\n2. In Table 2, the performance is almost unchanged after removing the attribute mapping loss on MOH. Why?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks very much for your encouragement and valuable comments. Below we shall respond to them one by one. \n\n\n***Q1/R1. As far as I know, the input context of token-level metaphor detection methods are sentences, is it appropriate to directly use token-level methods for word-pair metaphor detection?***\n\n**Response:** Unfortunately, it is not appropriate to directly use token-level methods for word-pair metaphor detection. We utilized three representative methods in token-level metaphor detection as baselines and adapted them to word-pair metaphor detection. Recent studies in token-level metaphor detection tackle this task as a word classification task, which take a sentence and a target word within this sentence as input and then predict the metaphoricity of the target word. In these baseline experiments, we replaced the target word and its context in token-level methods with the core word and the concept word in our method respectively. We shall add these descriptions of token-level baselines in the revision.\n\n\n***Q2/R2. In Table 2, the performance is almost unchanged after removing the attribute mapping loss on MOH. Why?***\n\n**Response:** The performance drops of all the ablation experiments on MOH, including removing the attribute mapping loss, are relatively slight. This phenomenon can be attributed to the small dataset size of MOH, which only has 647 samples. However, the ablation experiments conducted on the other two larger datasets result in significant performance drops, verifying the effectiveness of the attribute mapping loss. We shall include this discussion in the revision.\n\n\n***R3. I doubt whether word-pair metaphor detection makes sense in the field of metaphor detection now.***\n\n**Response:** Thanks for pointing this out. Word-pair metaphor detection makes sense in the field of metaphor detection now for the following reasons:\n1. Word pair is the basic and most frequently used linguistic form to express metaphor (Tsvetkov et al., 2014) and word-pair metaphor detection has consistently been an important research topic in metaphor detection (Rei et al., 2017; Su et al., 2020; Ge et al., 2022). \n2. Word-pair metaphor detection can benefit many NLP tasks. For example, it can assist machine translation systems in achieving more accurate and refined translations (Mao et al., 2018).\n3. The word pair, which offers a structured context, can serve as a bridge between linguistics, cognitive science and computational approaches, facilitating interdisciplinary insights into the essence of metaphors (Lakoff and Johnson, 2008; Ge et al., 2022).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper puts forward a novel idea that utilizes attribute likeness to improve the performance of metaphor detection. Experimental results show that the proposed method outperforms the previous word-pair methods.\n\n2. The paper is well-written and organized. The paper compares the proposed method with various types of methods and  carries out qualitative and quantitative analysis. Besides, effective ablation study shows the effects of different parts of the proposed method.", "reasons_to_reject": "1. As far as I know, the input context of token-level metaphor detection methods are sentences, is it appropriate to directly use token-level methods for word-pair metaphor detection？\n\n2. In Table 2, the performance is almost unchanged after removing the attribute mapping loss on MOH. Why?\n\n3. I doubt whether word-pair metaphor detection makes sense in the filed of metaphor detection now.  ", "questions_to_authors": "1. As far as I know, the input context of token-level metaphor detection methods are sentences, is it appropriate to directly use token-level methods for word-pair metaphor detection？\n\n2. In Table 2, the performance is almost unchanged after removing the attribute mapping loss on MOH. Why?", "paper_topic_and_main_contributions": "This paper studies the problem of word-pair metaphor detection based on conceptual metaphor theory. And this paper proposes a novel attribute siamese network to mine similar attributes between target concepts and source concepts.  Besides, this paper utilizes contrastive learning to learn domain inconsistency between source and target domains.  Experimental results on three publicly available word-pair metaphor datasets show the effectiveness of the proposed method.  And this paper demonstrate the generalization ability on a new dataset. \n", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "73tVRleAYh", "reviewer": "Reviewer_kRQ4", "summary": "", "questions": "- The opening sentence in the abstract: \"Metaphor detection is an important and challenging task to identify metaphorical expression in natural language processing\" .. what is that supposed to mean?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks for your valuable suggestions and comments. Below we shall respond to them one by one. \n\n***Q1. The opening sentence in the abstract: \"Metaphor detection is an important and challenging task to identify metaphorical expression in natural language processing\" .. what is that supposed to mean?***\n\n**Response:** Sorry for the confusion. We have rewritten this sentence for clarity.\n\n*$~~~~$“Metaphor detection is an important and challenging task in natural language processing, which aims to identify figurative expressions in the text, in the form of word pairs or individual words within the sentence.”*\n\n\n***R1. Experiment setting is not clear: The paper needs to mention the prompts used for ChatGPT. Surprisingly, ChatGPT performed very poorly on this task.***\n\n**Response:** Thanks for your comment. We present details of prompts for ChatGPT with different prompting strategies in Appendix D.2 of our paper, which are as follows:\n\n| Prompting Strategy | No. | Template Prompt                                                                                                                              |\n|--------------------|-----|----------------------------------------------------------------------------------------------------------------------------------------------|\n| Zero-shot          | 1   | Does the word pair \"[word pair]\" express metaphorical meaning?                                                                               |\n|                    | 2   | Does the word pair \"[word pair]\" express metaphorical meaning? Answer me with \"yes\" or \"no\".                                                 |\n|                    | ...   | ...                               |\n|                    | 12  | Given the word pair \"[word pair]\", determine if this word pair is a metaphorical expression. Give me an answer selected from \"yes\" or \"no\".  |\n| Zero-shot CoT      | 1   | Does the word pair \"[word pair]\" express metaphorical meaning? Answer me with \"yes\" or \"no\" step by step.                                    |\n|                    | ...  |...                               |\n|                    | 4   | Given the word pair \"[word pair]\", determine if this word pair is a metaphorical expression. Answer me with \"yes\" or \"no\" step by step.        |\n| 1-shot | 1  | Question: Does the word pair \"[metaphorical word pair]\" express metaphorical meaning? Answer: yes. Question: Does the word pair \"[literal word pair]\" express metaphorical meaning? Answer: no. Question: Does the word pair \"[word pair]\" express metaphorical meaning? Answer:                                                                                                                                                                                                                                          |\n|        | ...  |...                                                                                       |\n|        | 12 | Question: Given the word pair \"[metaphorical word pair]\", determine if this word pair is a metaphorical expression. Give me an answer selected from \"yes\" or \"'no\". Answer: yes. Question: Given the word pair \"[literal word pair]\", determine if this word pair is a metaphorical expression. Give me an answer selected from \"yes\" or \"no\". Answer: no. Question: Given the word pair \"[word pair]\", determine if this word pair is a metaphorical expression. Give me an answer selected from \"yes\" or \"no\". Answer:  |\n\nTable 11: Template prompts used for ChatGPT baselines. [word pair] denotes the input slot of the word pair that needs to be tested. [metaphorical word pair] and [literal word pair] denote the input slots of the metaphorical and literal word pairs respectively, which are randomly sampled from the training dataset. \n\n\nIn Subsection 5.4, we have added a more detailed analysis on why ChatGPT performs relatively poor on this task.\n\n*$~~~~$“GhatGPT baselines perform relatively poor on this task for the reason that understanding metaphor is a difficult task that requires deep, complex, or abstract reasoning, while ChatGPT is a general language model, which can handle some forms of reasoning but still struggles with complex reasoning tasks. In addition, the training data of metaphor detection is limited and difficult to collect in contrast to other research fields like machine translation, which further hinders ChatGPT's ability to understand metaphor.”*\n\n\n\n\n***R2. Paper presentation is not optimal: I struggled to understand the sampling process that was followed in Section 4.2.1. I encourage the authors to show an example here.***\n\n**Response:** Thanks for the suggestion. We have carefully rewritten the sampling process and added an example in Subsection 4.2.1.\n\n*$~~~~$“For a metaphorical word pair (e.g., drink gasoline), which contains a core word $w_{cor}$ (e.g., drink) and a target concept $w_{tgt}$ (e.g., gasoline), we can extract a set of literal word pairs from the training set (e.g., drink water, drink milk and drink soda), which share the same core word (e.g., drink) with the aforementioned metaphorical word pair. We then randomly sample one (e.g., drink water) from this set of literal word pairs. The concept word (e.g., water) in this sampled literal word pair serves as the source concept $w_{src}$ in our model.”*\n\nWe present the following figure to illustrate the sampling process and shall add it in the revision.\n|Metaphorical word pair|Core word|Target concept|\n|:--|:--|:--|\n|drink gasoline|*drink*|gasoline|\n\n\n↓ Sample one literal word pair from “drink water”, “drink soda”, and “drink milk”\n\n|Sampled literal word pair|Core word|Source concept|\n|:--|:--|:--|\n|drink water|*drink*|water|\n\n\n\n\n***R3. Paper presentation is not optimal: Details on the process of constructing the dataset VUA-WP are missing***\n\n**Response:** Thanks for your comment. We present the details on the process of constructing the dataset VUA-WP in Appendix A.1 of our paper, which are as follows:\n\n$~~~~$*“We construct a word-pair metaphor dataset VUA-WP from VUA20 (Leong et al., 2020), which is the largest benchmark dataset for the token-level metaphor detection task. Each word in sentences of VUA20 is labelled as metaphorical or literal. We use the Stanford dependency parser (Chen and Manning, 2014) to extract all the dependency relations and collect all the candidate word pairs with possible dependency relations according to Table 5. After that, we label these word pairs with four rules*: \n- *If all the words in a subject-verb/verb-object word pair are metaphorical in VUA20, we label this word pair as metaphorical;*\n- *If all the words in a subject-verb/verb-object word pair are literal in VUA20, we label this word pair as literal;*\n- *If the adjective in an adjective-noun word pair is metaphorical in VUA20, we label this word pair as metaphorical;*\n- *If all the words in an adjective-noun word pair are literal in VUA20, we label this word pair as literal.*\n\n*The number of literal word pairs is far more than the number of metaphorical word pairs. We randomly sample literal word pairs to get a balanced dataset VUA-WP. After that, we randomly divide all the word pairs into training, development and test sets with the ratio of 8:1:1.”*", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- A new approach to the task with strong results", "reasons_to_reject": "- Experiment setting is not clear:\n\t- The paper needs to mention the prompts used for ChatGPT. Surprisingly, ChatGPT performed very poorly on this task.\n\n- Paper presentation is not optimal:\n\t- I struggled to understand the sampling process that was followed in Section 4.2.1. I encourage the authors to show an example here.\n\n\t- Details on the process of constructing the dataset VUA-WP are missing", "questions_to_authors": "- The opening sentence in the abstract: \"Metaphor detection is an important and challenging task to identify metaphorical expression in natural language processing\" .. what is that supposed to mean?", "paper_topic_and_main_contributions": "This paper addresses the task of word-pair metaphor detection by modeling similar attributes between source and target concepts in metaphors. It is motivated by the CMT theory (Lakoff and Johnson, 2018), which states that metaphors are about the mapping of a set of attributes between the source and target concepts). Therefore, the authors propose an approach consisting of three components, each learning a subtask. The first component is a Siamese neural network that generates an attribute vector for the given metaphor by contrasting it to other literal or metaphorical word pairs. The second component learns the domain relation between the source and target concepts via contrastive learning. Finally, the third component is the classifier that takes the attribute vector and the word pair and predicts whether it is a metaphor or not. The three components are jointly learned.\n\nIn a set of experiments, the authors compare their approach to various baselines on three datasets and demonstrate state-of-the-art results on the task. The ablation study also shows that all components implemented in this approach contribute a certain boost in performance.\n\nUPDATE: The authors clarified my question and I will up my scores", "excitement": 4, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "eojbaBAa9w", "reviewer": "Reviewer_Gckv", "summary": "", "questions": "Why make use of the DenseGCN as the graph encoder?\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks for your comprehensive and insightful review. Below we shall respond to them one by one. \n\n\n***Q1. Why make use of the DenseGCN as the graph encoder?***\n\nWe use the DenseGCN as the graph encoder for the reason that it has several advantages, such as strengthening feature propagation, alleviating the vanishing-gradient problem and encouraging feature reuse (Huang et al., 2017; Guo et al., 2019). The experimental results of the ablation study on DenseGCN in Table 1 show that when replacing the DenseGCN with traditional GCN, the performances on all three datasets decrease, thus verifying the advantage of DenseGCN. We shall add these results and the analysis in the revision. \n\n\n| Method                | MOH (10-fold) | MOH (10-fold)     |  TSV           |       TSV          |      VUA-WP          |    VUA-WP             |\n|-----------------------|---------------|--------------|---------------|--------------|--------------|---------------|\n|                       | **F1**            | **Acc.**         | **F1**            | **Acc.**         | **F1**           | **Acc.**          |\n| Our method (DenseGCN) | 79.52±0.55  | 79.83±0.60 | 89.98±1.53  | 90.00±1.52 | 76.40±0.11 | 76.47±0.10  |\n| Our method (GCN)      | 78.90±0.51  | 79.22±0.62 | 87.39±1.79  | 87.40±1.80 | 75.00±0.74 | 75.10±0.72  |\n\nTable 1: Results of the ablation study on DenseGCN.\n\n\n\n***R1. As graph construction and learning is time-consuming, I think the authors should demonstrate the computational complexity of the proposed method.***\n\n**Response:** Thanks for the suggestion. We use FLOPs to evaluate the time complexity and the number of parameters to evaluate the space complexity. From the results in Table 2, we can see that the pretrained model BERT constitutes the majority of the computational complexity of our proposed method. Although we use the graph-based network as our attribute siamese network, the computational cost of this network is just 0.14G FLOPs, which is trivial compared to that of BERT. In addition, the results in Table 3 show that the computational cost of our method is comparable to that of baselines. We shall add this analysis in the revision.\n\n\n| Module                                          | FLOPs  | #Params   |\n|-------------------------------------------------|--------|-----------|\n| Attribute Siamese Network (graph-based network) | *0.14G*  | *1,615K*    |\n| BERT                                            | 11.90G | 109,487K  |\n| Our proposed method                             | 12.05G | 116,417K  |\n\nTable 2: Computational complexity of our proposed method.\n\n| Method              | FLOPs  | #Params  |\n|---------------------|--------|----------|\n| MelBERT             | 23.81G | 112M     |\n| MrBERT              | 11.90G | 110M     |\n| MisNet              | 13.61G | 114M     |\n| EMI                 | -      | 125M+    |\n| Our proposed method | 12.05G | 116M     |\n\nTable 3: Comparison of computational complexity between baselines and our proposed method. Given that EMI did not open-source their codes, we cannot get a precise FLOPs of this method. EMI uses RoBERTa as the text encoder whose number of parameters is 125M, and thus the number of parameters of EMI is more than 125M. \n\n\n\n***R2. There are too many blank spaces in the main experimental results.***\n\n\n**Response:** Sorry for the confusion. Some baselines (Multimodal, SSN-SG, Concreteness and EMI) did not release their source codes, resulting in the absence of their experimental results on our newly constructed dataset VUA-WP. In addition, the experimental results of these baselines in our main experimental results are retrieved from their papers, which do not provide the standard deviations.\n\n\n***R3. The authors do not analyze the impact of the quality of the attribute graphs.***\n\n**Response:** Thanks. The quality of the attribute graphs mainly depends on edges and nodes, which are fundamental elements in graphs. We conduct two ablation studies on the quality of edges and nodes. The experimental results are shown in Table 4. When we randomly assign 50% of the edges in the attribute graph, the performances decrease across datasets. This is because edges represent the relationships between nodes, which are important for node representation learning. If we randomly delete 50% of the nodes when constructing the attribute graphs, the essential knowledge required to learn the final attribute vector may be lost, thus leading to consistent performance drops on three datasets. These experimental results verify that the quality of the attribute graphs has an impact on our method.\n\n\n| Method           | MOH (10-fold) | MOH (10-fold)         | TSV        |   TSV             |     VUA-WP          |    VUA-WP                 |\n|------------|---------------|--------------|---------------|--------------|--------------|---------------|\n|                  | **F1**            | **Acc.**         | **F1**           | **Acc.**         | **F1**        | **Acc.**         |\n| Our method  | 79.52±0.55  | 79.83±0.60 | 89.98±1.53  | 90.00±1.52 | 76.40±0.11 | 76.47±0.10  |\n| 50% random edges | 78.43±0.56  | 78.75±0.53 | 82.90±1.27  | 83.00±1.22 | 73.54±0.32 | 73.72±0.31  |\n| - 50% nodes| 78.21±0.52| 78.57±0.57| 82.68±0.91  | 82.80±0.98 | 74.11±0.19 | 74.16±0.18  |\n\nTable 4: Results of the ablation study on the edges and nodes in the attribute graph.\n\nWe shall add a line chart to show the trend of performance change when varying the proportions of random edges and deleted nodes in the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "(1)\tThis method designs a new loss function that considers the similarity of attributes with the inspiration of the CMT.\n(2)\tThis paper is well writing.\n(3)\tIn the experimental part, the authors make use of the powerful chatGPT for comparison. \n(4)\tThe experiments are comprehensive and clear. The authors do not only analyze the impact of layers in the GNN, but also verify the impact of weights of the two proposed loss functions.\n", "reasons_to_reject": "(1)\tAs graph construction and learning is time-consuming, I think the authors should demonstrate the computational complexity of the proposed method.\n(2)\tThere are too many blank spaces in the main experimental results.\n(3)\tThe authors do not analyze the impact of the quality of the attribute graphs.\n", "questions_to_authors": "Why make use of the DenseGCN as the graph encoder?\n", "paper_topic_and_main_contributions": "The authors follow the Conceptual Metaphor Theory and find that previous studies ignore the similarity of attributes shared by the source and target domains. To remedy this, they propose a novel method entitled AIDIL. The method involves a Siamese neural network to compute the attribute similarity between the source and the target. Moreover, they propose a domain inconsistency loss based on the contrastive learning. ", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Inspired by conceptual metaphor theory, this paper presents a new model for metaphor detection, (specifically for pairs of words in context), using a Siamese network and contrastive loss. The reviewers were impressed with the quality of writing and the quality of the experiments ("comprehensive and clear").

In terms of scores, reviewers were balanced between good and strong / ambivalent and strong. However, the authors provided extensive responses in their rebuttal, including updated language and promised changes, which largely seem to mitigate the (mostly minor) concerns. In addition, the most favorable reviewer was the most engaged with the authors during the discussion.

Although I am not convinced that the relatively poor performance of ChatGPT can be taken as conclusive (given the number of degrees of freedom in such a setup), that part is not particularly important for this paper to succeed. Rather, it can stand on its strong experimental results (relative to other methods), the quality of its writing, and its contributions to an interesting area.
Decision: Accept-Main