Chinese Metaphorical Relation Extraction
Authors: Guihua Chen
Tiantian Wu
MiaoMiao Cheng
Xu Han
Jiefu Gong
Shijin Wang
Wei Song
Venue: EMNLP 2023 Findings (2023)
Keywords: Metaphor understanding
metaphorical relation extraction
linguistic metaphor
cognitive metaphor
TL;DR: Metaphorical relation extraction is a process that detects metaphorical expressions and builds connections between target and source domains.

Abstract: Metaphors are linguistic expressions that convey non-literal meanings, as well as cognitive mappings that establish connections between distinct domains of experience or knowledge.
This paper proposes a novel formulation of metaphor identification as a relation extraction problem. 
We introduce metaphorical relations as links between two spans in text, a target span and a source-related span. 
We create a dataset for Chinese metaphorical relation extraction, with more than 4,200 sentences annotated with metaphorical relations, corresponding target/source-related spans, and fine-grained span types. 
Metaphorical relation extraction is a process that detects metaphorical expressions and builds connections between target and source domains.
We develop a span-based end-to-end model for metaphorical relation extraction and demonstrate its effectiveness.
We expect that metaphorical relation extraction can serve as a bridge between linguistic metaphor identification and conceptual metaphor identification.
Our data and code are available at https://github.com/cnunlp/CMRE.

Chinese Metaphorical Relation Extraction
Guihua Tiantian Wu!;
Miaomiao Cheng', Xu Han', Jiefu Gong2 Shijin Wang2 Wei Song' "College of Information Engineering, Capital Normal University, Beijing, China State Key Laboratory of Cognitive Intelligence, iFLYTEK Research, China iiFLYTEK AI Research (Hebei), Langfang, China (ghchen, ttwu, mi mimomia,, hanxu, wsong) @cnu. edu en, (ifgong, s sjwang3 (ele@@iillllg @eiflyeee.com
Abstract
Metaphor identification is a core task in metaphor processing, which involves recognizing and analyzing metaphorical expressions in text. Metaphorical expressions convey a meaning differ- ent from their literal interpretation, such as "knowl- edge is power" and "my car is drinking gas". Pre- vious work on metaphor identification has mainly concentrated on two types of expressions.
Metaphors are linguistic expressions that con- vey non-literal meanings, as well as concep- tual mappings that establish connections be- tween distinct domains of experience knowl- edge. This paper proposes a novel formula- tion of metaphor identification as a relation extraction problem. We introduce metaphor- ical relations as links between two spans in text, target span and a source-related span. We create a dataset for Chinese metaphorical relation extraction, with more than 4,200 sen- tences annotated with metaphorical relations, corresponding auget/source-relate spans, and fine-gras fine-grained span types. Metaphorical relation extraction is process that detects metaphor- ical expressions and builds connections be- tween target and source domains. We develop a span-based end-to-end model for metaphor- ical relation extraction and demonstrate its ef- fectiveness. We expect that metaphorical rela- tion extraction can serve as bridge between linguistic metaphor identification and concep- tual metaphor identification. Our data and code are available at https://ggthhb.... com/ cnunlp/ CCMRE. /CMRE,
The first type is syntactic constructions such as noun-is nuu-is--ounn (AisB style), verb-subject adjective- noun, and verb-direct object constructions (Levin et al., 201 4; Tsvetkov et al., 2014). The motivation is to capture the constructions that violate the selec- tional preference (Wilks, 1978). So metaphor iden- tification is formulated as syntactically-relate word-pair classification problem (Tsvetkov et al., 2014; Shutova et al., 2016; Rei et al., 2017). How- ever, this formulation usually requires parser to extract word pairs that belong to pre-defined check- able relations (Levin et al., 2014), neglecting other types of expressions and ignoring useful contexts in the whole sentence.
The second type is annotating the metaphoricity of individual words in sentence. The motivation is described by the metaphor identification procedure (MIP) (Group, 2007) that a word would be anno- tated as metaphorical if its meaning in a specific context is different from its basic meaning. Based on this scheme, metaphor identification can be for- mulated as sequence labeling task, which has become popular due to the rapid development of deep contextual representation learning (Do Dinh and Gurevych, 2016; Gao et al., 2018) and pre- trained language models (PLMs) (Leong al., 2020). However, this formulation cannot explic- itly reflect the conceptual structure of metaphors. For example, in the sentence "my car is drinking gas", annotating the verb drinking as metaphorical cannot reveal the source-targe mapping.
1 Introduction
Metaphor is pervasive linguistic phenomenon in natural language (Cameron, 2003). It involves ex- pressing non-literal meanings that are not directly derived from the words of the metaphorical ex- pressions and that pose challenges for computa- tional understanding. Metaphor is also a cognitive process (Lakoff and Johnson, 2008). metaphor consists of 3 parts: the target, the source, and an analogical mapping from the source to the tar- get (Carbonell and Minton, 1983), where the tar- get and the source are concepts or conceptual do- mains. Metaphor processing has been important research field in natural language processing.
In this paper, we propose a novel way to rep- resent metaphors through metaphorical relations.
"Co-first authors.

---

numinal rehason
atributc rebixve
2C R
x
6.
M db By the river,
AM Sp
# 14.
The world targer
a stage. sounce
therc're
tender and graceful source attribute
willows. target
(a) A nominal relation
(b) An attribute relation with a source attribute
fooninal rehtion
atrilate relation
wort roliticen
32 A6,
At
95
The city
is
US industry
's
heart.
# 86 $
A is drinking
idb.
target
source part
My car target
gas.
target
source
action
larget
(d) An attribute relation with source part and nominal relation in the same sentence
(c) A sentence with simile relation and verb relation
Figure I: Examples of metaphorical relations. A metaphorical relation shown as a link from source-related span a target span. Each span region is underlined and has a fine- fine-grained type below. The fine- ane-grained span types characterize the fine-grained metaphorical relation types labeled on the links.
A metaphorical relation connects two spans, tar- get span and a source-related span. We create a metaphorical relation dataset for Mandarin Chi- nese. The dataset consists of more than 4,200 sentences annotated with metaphorical relations, turget/sourre--rcltted spans, and fine-grat fine-grained span types. As shown in Figure 1, the fine- fine-grained span types, such as source, source action, source at- tribute and source part, can characterize metaphor- ical relations into the fine-grained types, such as nominal, verb, attribute and simile relations.
les since they share a similar underlying cognitive process. So different types of metaphors can be handled in the same framework rather than pro- cessed in separate frameworks (Liu et al., 2018; Song et al., 202 bba).
Based on the annotated dataset, explore span- based models for metaphor identification. We pro- pose an end-to-end model jointly optimizing the span proposer, fine- graingaindd span type classification, and metaphorical relation extraction. Specifically, we propose pair- pair-wise span type classification module to predict the types of a pair of spans si- multancously since they are closely associated and support metaphorical relation scoring through the span type features. Experimental results demon- strate the effectiveness of the proposed method compared with the baseline that is directly adapted from an entity relation extraction model. However, there still room for further improvement the performance. Our method can be used as a baseline for future research.
Annotating metaphorical relations encompasses the previous two annotation strategies and over- comes their shortcomings. It has the following advantages. First, metaphorical relations use spans precisely
instead of words the basic unit capture the characteristics of the target and the source. Second, metaphorical relations build sur- face connections between the target and the source, overcoming the weakness annotating metaphor- ical words only. Although the source is often im- plicit, the connections can be built based on the actions, attributes, or parts of the source. For ex- ample, my and drinking form metaphorical relation, where my car is the target and drinking is an action of the source. Third, metaphorical re- lations can cover multiple types metaphors that have been discussed in previous work, including AisB style, verb, adjective metaphors, and simi-
To the best of our knowledge, this is the first work that explores span-based relations to annotate and identify metaphors. Metaphorical relations not only reveal the metaphorical expressions but also build surface connections between the target and the source. Because the sources are often implicit in the text for metaphors involving metaphorically used verbs and adjectives, we can further infer com-

---

plete source-target mappings based on metaphor- ical relation extraction. So metaphorical relation extraction has the potential to connect linguistic metaphor identification and conceptual metaphor identification (Steen, 1999). We leave this as the future work.
Based on the created metaphorical relation dataset, we open new perspective by exploring the span-based relation extraction models for metaphor identification. Song et al. (2021b) proposed a con- textual relation learning method for verb metaphor identification but it is still based on individual word annotations and requires dependency parser. Span-based Entity Relation Extraction The pro- posed metaphorical relation extraction model mo- tivated by the model for coreference resolution (Lee et al., 2017, 2018), which is a span-based relation extraction model. However, it does not explicitly consider span types, which are important in our task to indicate the structure of metaphors. To han- dle this, we propose to jointly optimize relation extraction and span type classification.
2 Related Work
Metaphor Datasets The VU Amsterdam Metaphor Corpus (VUA) is the largest and most repre- sentative dataset (Steen et al., 2010), annotated based on the metaphor identification procedure (MIP) (Group, 2007). The other popular word-level metaphor datasets include MOH-X (Mohammad et al., 2016) and TroFi (Birke and Sarkar, 2006). VUA annotates all individual words, while MOH-X and TroFi annotate verbs and their sizes are rela- tively small.
Our model also similar to the entity relation extraction models. We do not simply adapt the pipeline approaches, e.g. (Zhong and Chen, 2021), or previous joint entity relation extraction mod- els (Dixit and Al-Onaizan, 2019; Eberts and Ulges, 2020) to our task. Instead, we focus on handling the challenge in task that the span types are hard to determine without considering specific involved metaphorical relations.
Some datasets focus on the metaphoricity of syntactically-reltted word pairs, together with other attributes such as emotion or novelty (Levin et al., 2014; Tsvetkov et al., 2014; Mohler al., 2016; Do Dinh et al., 2018).
Shutova and Teufel (2010) built a dataset for annotating source-target domain mappings for metaphorically used verbs, consisting of 761 sen- tences. The mappings constructed based on the source and target domain categories borrowed from the Master Metaphor List (Lakoff, 1994).
3 Metaphorical Relation Dataset
In this section, we introduce the created Chinese metaphorical relation dataset. We first introduce metaphorical relations, then describe the details of data annotation, and finally conduct basic data analysis.
These datasets are mainly for English. A few datasets cover Spanish, Russian, and Farsi (Mohler et al., 2016; Tsvetkov et al., 2014). Zhang et al. (2018) created Chinese dataset for analyzing the emotionality of metaphors, which is a relatively large public dataset for Chinese.
3.1 Metaphorical Relations
We define a metaphorical relation as a link be- tween two spans, a target span and a source- related span. Considering the example in Fig- ure (bb), willows is the target span, and tender and graceful is source-related span. We use a directed are from the source-related span the target span to represent the metaphorical relation. The two spans trigger a metaphor because tender and grace- ful are not usually used describe willows.
Our work differs from previous work in the anno- tation scheme. We explicitly annotate metaphorical relations between spans related to the target and the source. This way is more general and expres- sive compared to annotating individual words or syntactically related word pairs.
Metaphor Identification Methods Similar to the construction of the datasets, metaphor identifica- tion can be viewed as word-pair classification problem or a word sequence labeling problem. Most of the current models depend recurrent neural networks (Do Dinh and Gurevych, 2016; Rei et al., 2017; Gao et al., 2018) and pre-trained language models (Leong et al., 2020; Dankers et al., 2020; Su et al., 2020; Li et al., 2021; ; Ge et al., 2022; Aghazadeh et al., 2022; Li et al., 2023).
A span consists of consecutive tokens to cover precise contextual clues. We expect the span should cover necessary and compact descriptions, which can reflect the properties of the source the target and provide precise contextual clues. Consider- ing the sentence, * if #5
te 2h bh $6 te (the treacherous Venetian merchant is a cunning fox), wider spans with detailed de-

---

scriptions can provide useful clues for building the mapping between the target and the source. We define fine-grained types for source-related spans, including:
3.2 Data Annotation
Data Source Our annotation is based on the dataset built by Zhang et al. (2018), which is a Chinese dataset for analyzing the emotionality of metaphors. The genres include books, journals, movie scripts, and social media texts. The dataset has 5,494 sen- tences annotated with metaphorical labels and emo- tion categories at the sentence level. The metaphor- ical labels reveal whether a sentence has noun metaphors, verb metaphors, or no metaphors. We annotate metaphorical relations in metaphorical sentences and also include a variety of external lit- eral sentences to make the distribution of metaphor- ical and literal sentences more balanced.
Concepts that are used as the source of a metaphor. Sources are usually realized in AisB style metaphors while they are often im- plicit in metaphors involving metaphorically used verbs or adjectives..
Source:
Source action:
Typical actions or passive ac- tions of an implicit source. Consider an ample, "the car is drinking gas", where drink is an action of humans or animals, indicating mapping from animal to car, and drink is also a passive action of water, resulting in a contrast between water and gas.
Metaphorical Relation Annotation For each metaphorical sentence, we ask annotators to anno- tate metaphorical relations, spans, and fine-grained span types. The annotators are two students with a background in linguistics.
Source attribute: Typical attributes an implicit source, e.g., the span tender and graceful is commonly used to describe women which is potential source concept.
We prepare a manual. which describes: (1) the definition of metaphorical relations and the stan- dard to establish metaphorical relation, which refers to the work of Shutova and Teufel (2010) and Group (2007); (2) the criteria and examples for annotating span boundaries; (3) the definition and examples of the fine-grained span types.
Source part: Parts or related concepts of an implicit source. Figure (d) shows part relation, where heart is part of person, in- dicating that the US industry is mapped person.
The annotation process is follows. In the first stage, the annotators would try annotate part of the metaphorical sentences to understand and fol- low the manual. We emphasize the importance consistent and reliable annotation, especially on the span boundaries and the existence of metaphorical relations.
With the fine-grained span types, we can catego- rize some fine-grained metaphorical relations:
Nominal relation: A nominal metaphorical relation consists of a target and a source, The most typical nominal relation is the AisB style metaphors.
In the second stage, we assign the whole dataset to both annotators and let them annotate indepen- dently. We randomly sample about 200 metaphori- cal sentences that have been re-annotated by them. The rate of inter- annotator agreement on the span pairs is about 77%. There are only a few dis- agreements due to the span region and most of the disagreements are because some relations are missed by some annotators, especially in sentences with multiple metaphorical relations. For the spans annotated by both annotators, the rate of inter- annotator agreement on fine-grained span types is 93%, which is high, indicating that once people recognize metaphorical relation, the types of its two spans can be determined easily.
Verb relation: A verb metaphorical relation connects a target and a source action.
Attribute relation: An attribute metaphor- ical relation connects a target and source attribute or source part.
Simile relation:
We can view simile as a spe- cial metaphorical relation type, which also has target and source but there exist explicit comparators such as (like) in context.
In summary, metaphorical relations establish the connection between the source and the target on the surface of the sentence. They do not depend on syntactic patterns and utilize flexible and concise spans to characterize the source and target domains.
Finally, every sentence has second pass review to resolve any conflicts and reach an agreement. During annotation, sentences that are too long or

---

Dataset
#St
avg.
Finally, the dataset has 8,494 sentences. The percentage of metaphorical sentences is 0.5. We divide it into training, development, and test sets with a ratio close to 8: 1: The basic statistics are shown in Table 1.
Training
6,794
50
1.28
Development
850
50
1.27
Test
850
50
1.28
Total
8,494
50
1.28
Table 1: Statistics of the training set, development set, and test set, #St: number of sentences; %M: metaphor percentage; avg. #R: average number of metaphorical relations per metaphorical sentence.
Figure 2 further shows the statistics of the fine- grained span types. Although the imbalanced dis- tribution adheres to our intuition, the number of source attribute/part spans is relatively small. This limitation of this work.
The Proposed Model
4.1 Overview
4000
Metaphorical relation extraction aims to extract set of non-overlapping typed span pairs from sentences and determine the type of these relations. We propose span-based end-to-end model. The main architecture is shown in Figure 3. span proposer would enumerate, score, and select the best spans, which are used for efficiently generating candidate span pairs. Each pair would be fed into two modules: pair-wise span type classification, which is specially designed for determining span types, and metaphorical relation scoring, which estimates the possibility that the candidate pair has metaphorical relation. The two modules work together extract metaphorical relations.
2000
1000
S-part S-attr
Figure 2: Distribution the fine-grained span types.
target, S: source.
have vague meanings should be removed. Adding Literal Sentences We expand external lit- eral sentences to the dataset keep a balanced ratio between metaphorical and literal sentences. We an- alyze the initially annotated dataset and alleviate the following artificial bias:
4.2 Span Proposer
Given sentence with M tokens S W1, WM we use BERT to get the contextual representation em of each token Wmm- The span proposer enumer- ates all possible spans up to a maximum length L in sentence. For given span i with the start and end indices START(i) and END(i), its representation is computed as follows,
Avoiding word bias: Similes have explicit comparators such as (like, as) and AisB style nominal metaphors use Σ (is, are). Most sentences in the initial annotated dataset that contain such words metaphorical. So should include literal sentences that also con- tain such words.
g(i) (SSTAR(()) (zND((), e,, d(i),
(1)
where @START(() and egND() are the contextual rep- resentations of the start and end tokens; e an attention- weighted average embedding over tokens the span; Φ(i) a feature vector to rep- resent the span width feature, which assigns each length with learnable embedding vector. These vectors are concatenated to represent span i. We multi-layer perceptron (MLP) with hidden layer and ReLU for scoring each span,
Avoiding sentence length bias: We hope to keep the length distribution of literal sentences close to metaphorical sentences avoid bi- ases in sentence length.
We randomly sample sentences and retrieve sen- tences with biased words from Baidu Baike, stu- dent essays, and books, and manually select literal sentences to have a balanced and unbiased dataset.
s MLP
(2)
We rank all possible spans for a sentence de- creasing order of the span scores and keep the top

---

Pair-wise span classification
Metaphorical relation scoring
of opann
Type of span?
Spon type clossification
Linear
Linear
Span proposer -pan representation, scoring pruning
#()
Span |
span
span
Sp
(.))
+0)
Contextual token representations
Relation scoring
MLP
Deep Transformer (BERT)
span
Input
[CLS]
W1
W2
W3
W4
Ws
[SEP)
Span type features
Figure 3: The architecture for the joint model for span type classification and metaphorical relation extraction.
À spans, 0 < A < 1. We choose À with the devel- opment set in experiments. The selected N spans Cş [span 1,...,sppp span N] would participate in the following stages.
the span pair,
sp(i. gi g:
(6)
MLP 8y gj, Le. tyl),
where W is a parameter matrix; gi and gj are span representations; gi o represents the semantic interactions between two spans; t; and t, are repre- sentations of the span type features. The span type features are computed based on the predicted types of span and span provided by the pair-wise span type classification module. We will introduce the details later.
4.3 Metaphorical Relation Scoring
A span can form metaphorical relations with mul- tiple other spans. For pair of spans (i.j).j i, we name span as the anchor and span j as span i' peer. Our task is to find the correct set of peers (i) for every anchor span i.
A joint distribution P(Yi,..., YN(S) over all spans in Cs is formulated as:
4,4 Pair-wise Span Type Classification
P(Y YN|S) II P(Y)))
This module assigns each span in metaphorical relation a type e E E. The span types in metaphor- ical relations are context-sensitive. Consider the following examples, my heart is broken and the city is the heart of US industry, where heart can be used as a target or a source in different con- texts. Therefore, instead of classifying span types independently, propose pair- wise span type classification approach.
i=1
N
Σ ve exp(s(i,v) v))
II 1=1
Ejer exp(s(i.
(3)
where V(i) = {e, 1,.,,,,, 1) represents all possible assignments, and means span i does not have peer; si,,j) the overall score for every candidate pair ji. The loss function for relation scoring in sentence
We feed pair of spans into single layer feed- forward neural network (FFNN) with ReLU the activation function to get a shared hidden represen- tation and then use two independent linear classi- fiers to get the probability distributions of the two spans,
(S) = -logP VN(S).
(4)
The scoring function si, is computed as fol- lows.
ý; sof tmax tmax(minearr (FFM g,l)))
==
(7) We compute the loss function of span type classi- fication across all candidate span pairs that belong to the gold span argument pairs GSS), i.e.,
s(i,j) ((i++ +(s) sp(t,),, otherwise,
y; sof tmaxx(Linear,((FPXXX(s g,D)))
(5)
where s(i) and s(j) are the scores computed based on Equation 2 and (i. is a wise score for

---

type features to 20 and fix the embedding dimen- sion of the hidden layers in Equation 2, 6, and 1000. The span type set in span type classifica- tion consists of the fine-graine iie-ggrined span types.
Ctp(S) (L)))(C)))G(S)
(B((y:
(8)
CE)
We tune the hyper-parameters of our model, our variations, and the baseline based on the per- formance on the development set, and report the results on the test set. We run the experiments with one GeForce RTX 2080Ti GPU. We use the A optimizer with learning rate of 2e-4 for relation extraction and span type classification.
where R(Cs) = {(i,)) < i, span C Cs A span j Cs}, yc, Vj are the gold span types, CE is the cross-entropy function. We avoid introducing a none type to indicate invalid spans, since this may produce incompatibility predictions, such as a metaphorical relation with a none type span.
5.2 Comparison Settings
For computing the span type features in Equa- tion 6, we use learnable type embedding table V E Rd |l to represent span types, where the di- mension is d. To reduce the influence of invalid spans and inaccurate predictions during inference, we compute a soft span type representation for span based on the predicted probability distribution y;.
We compare with the following baseline. The base- line follows the working flow state-of-the-art relation extraction model (Eberts and Ulges, 2020). is a joint entity relation extraction model, which classifies the type of each span independently and does not consider span type representations for re- lation classification.
181
5.3 Evaluating Metrics
t;
Vi.k Vk:
(9)
We evaluate both metaphorical relation extraction and span extraction with precision (P), recall (R), and FI the main evaluation metries. A span is considered correct if its type and region are both correct. A relation is considered correct if both of its span arguments are correct. We run each model with 3 random seeds and report the average performance.
where ft, is the probability that the k-th type is assigned to span and V3 is the representation of the k-th type.
4.5 Training and Inference
Training We combine the loss functions of metaphorical relation scoring and pair-wise span type classification, i.e., Ls = (S)( + (S)(S), and train the model sentence by sentence.
5.4
Comparison Results on Coarse-grained Metaphorical Relation Extraction
5.4.1
Overall Results
Decoding Given a sentence for decoding, get all span pairs that have a relation score (Equa- tion 5) larger than 0 together with the predicted span types. If multiple pairs share the same anchor and have overlaps between peers, we only keep the one that has the highest relation score. The types of metaphor relations can be inferred by rules based on the types of its spans.
first
For comparison with the baseline, we first consider the identification of the coarse-grained span types: target span or source-related span, and binary metaphorical relation. So this evaluation reflects the ability to identify the existence of metaphorical relations. Table shows the comparison results of relation extraction and span extraction against the baseline. Compared with the joint model base- line, our method has close performance in span extraction but obtains a 1.67% F1 improvement in relation extraction.
5 Experiments
5.1 Parameter Settings
We use the et al., 2021). For the span proposer, we fix the max- imum length L 15, try 0.4, 0.6, 0.8, and find that the best performance can be obtained on the development set when À 0.8 and 92% gold spans are recalled. Concerning the model architecture, we fix the dimension of the span width and span
model (Cui
Our method adopts the pair- wair-wise span type clas- sification manner and enhances the interaction be- tween span type classification and relation extrac- tion. The results show the effectiveness of the pro- posed method in identifying metaphorical relations. Each metaphorical relation has two spans. Identify- ing both span arguments of metaphorical relation

---

Relation Extraction
Span Extraction
Model
R
F1
P
R
F1
Baseline
63.77
58.64
61.79
76.13 13
67.90
71.62
Our full model
70.71
$7.59
63.46
78.99
65.57
71.64
w/o span type features pair-wise STC -iideeeendent STC
68.25
56.42
61.67
78.55
65.04
71.08
63.50
59.82
61.34
74.13
68.99
71.29
Table 2: Experimental results of the baseline and the proposed model with different settings metaphorical relation extraction and span extraction.
correctly at the same time is crucial for capturing the meaning and structure of metaphors.
Model
P
F1
Nominal
69.03
65.55
67.24
Verb
46.81
55.46
5.4.2 Ablation Studies
Attrihute
64.29
25.71
36.73
Simile
74.42
64.00
68.82
We further perform ablation studies to analyze the contributions of the pair-wise span type classifi- cation module (STC) and the span type features. Table 2 shows the results.
Table 3: Performance of the proposed method on detect-
ing the fine-gran one-grained metaphorical relation types.
When we remove the span type features, the FI performance decreases by 1.79% in metaphorical relation extraction and 0.56% in span extraction compared with the full model. Using the span type features benefits metaphorical relation extraction.
are more complex. The attribution relation identifi- cation is difficult because of the small number of training examples in our current dataset.
5.6 Error Analysis
When replace the pair-wise span classifica- tion module with an independent span type clas- sification module, the Fl performance decreases by 2. 12% in metaphorical relation extraction and 0.35% in span extraction. The pair-wise span clas- sification achieves large gain in precision for span extraction, which has a significant impact the prediction of metaphorical relations.
We analyze the errors metaphorical relation ex- traction according to the predictions on the test set. The errors can be summarized into the following types.
(1) Fail to recall (accounting for 60.0%). This type is the most common.
When we remove both modules, our full model degenerates into the one similar to the baseline. By
Sub-type Some errors involve uncommon or abstract concepts as the source, e.g., â $ is It S it &h iy th $ t * (Inferi- ority rises like the prices in wartime), where the source prices in wartime also expresses intended meaning.
comparison, we can see that the combination of span type features and the pair-wise span type clas- sification keeps good balance between precision and recall, obtaining the best overall performance.
Some errors related to tences with multiple metaphorical relations such the example in Figure (dd and some of these relations are missed. The reasons in- clude: 1) our current model extracts relations independently and does not consider the inter- action between multiple relations; 2) in some cases, the long distance between the target and the source increases the difficulty.
Sub-type 2:
5.5 Analysis on the Identification of Fine-grained Metaphorical Relations
We further analyze the ability identify fine- grained metaphor relation types, where span considered correct if its region and the fine-grained span type are correct. Table 3 shows the results. The simile relation can be identified with the best performance, followed by nominal relation and verb relation identification, while attribute relations are hard to identify.
Sub-type Attribute metaphorical relations are often missed. This should be due the small number of examples in our training data, and our model does not incorporate or learn
The results conform our intuition. The pat- terns of simile and nominal relations are generally simple, while the patterns related to verb relations

---

concept hierarchies help capture the object attribute or the whole part relations.
perspective for metaphor analysis. revealing that metaphorical relations can assume a crucial role in bridging the processing of linguistic metaphors and conceptual metaphors.
(2) Incorrectly identified relations (34.5%). This type refers false-positive extracted relations.
Limitation
Sub-type Incorrect span region errors are the most common. Some of these errors are ac- ceptable actually. For example, gold relation is between a # ttt # (tender and graceful) and By #s (willows), while the model may tract the relation between Di f (graceful) and top it (willows).
Our current work has some limitations that plan address in the future.
First, the size of the dataset can be further ex- panded, and the distribution imbalance prob- lem among metaphorical relation types should be handled. So the performance extracting minority relation types be improved.
Sub-type 2: Some complex syntactic patterns may result in incorrect connections between spans, including the clauses involving the use of the prepositions iz (ba) and * (bei) constructions, which are two types of non- canonical word orders in Mandarin Chinese. Other error-prone syntactic patterns include appositive and ellipsis.
Second, we use metaphorical relations to build surface connections between the target and the source but have not built explicit and complete conceptual mappings, because the sources are often implicit in the text. Therefore plan to infer complete conceptual mappings based on metaphorical relation extraction in the future work.
Sub-type 3: Some predicted relations involve conventional metaphors, widely used in every- day language and not labeled as metaphorical, such as - (fetter somebody), M+ # (re- alize dream).
Third, we only use a Chinese dataset for our experiments, but the metaphorical relation schema could be generalized to other lan- guages well.
(3) Others (5.5%): A few errors are related the relations that are not annotated during data annota- tion but seem to be reasonable as well. It indicates that it would be beneficial for data annotation with machine-in in loop manner.
A Acknowledgments
This work is supported by the National Natural Sci- ence Foundation of China (61876113 13, 62376166) and the National Key Research and Development Program of China (No. 2022FFC3303504) Wei Song is the corresponding author.
The error analysis can mostly explain the results shown in Table and Table It also reveals that incorporating external knowledge, modeling inter- actions among multiple relations, and expanding resources for minority metaphorical relation types should be emphasized in the future work.
References
Ehsan Aghazadeh, Mohsen Fayyaz, and Yadollah Yaghoobzadeh. 2022. Metaphors in pre-trained language models: Probing and generalization across datasets and languages. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 2037-2050.
6 Conclusion
We have presented a novel scheme for annotating metaphors in text based on metaphorical relations, which captures the connections between the source and the target through typed spans. Metaphorical relations can reveal the structure and context of metaphors, and facilitate new task formulations for metaphor processing. We create dataset and develop an end-to-end model for metaphorical re- lation extraction. We show that our model can achieve moderate performance on metaphorical re- lation extraction. Our work has opened up a fresh
Julia Birke and Anoop Sarkar. 2006. tering approach for nearly unsupervised recogni- tion of nonliteral language. In 11th Conference of the European Chapter of the Association for Computational Linguisties, pages 329-336.
clus-
Lynne Cameron. 2003. Metaphor in educational
discourse. A&C Black.

---

Jaime G Carbonell and Steven Minton. 1983. Metaphor and common-senss reasoning. Technical report, CARNEGIE-MELLON UNIV PITTSBURGH PA DEPT OF COMPUTER SCIENCE.
Kenton Lee, Luheng He, Mike Lewis, and Luke Zettle- moyer. 2017. End-to-end neural coreference reso- lution. In Proceedings of the 2017 Conference Empirical Methods in Natural Language Processing, pages 188-197, Copenhagen, Denmark. Association for Computational Linguistics.
Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, and Ziqing Yang. 2021. Pre-training with whole word masking for chinese bert. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 09:3504-3514,
Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse- to-fine inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume 2 (Short Papers), pages 687-692, New Orleans, Louisiana. Association for Computational Linguistics.
Verna Dankers, Karan Malhotra, Gaurav Kudva, Volodymyr Medentsiy, and Ekaterina Shutova. 2020. Being neighbourly: Neural metaphor identification in discourse. In Proceedings of the Second Workshop on Figurative Language Processing, pages 227-234, Online. Association for Computational Linguistics.
Chee Wee (Ben) Leong, Beata Beigman Klebanov, Chris Hamill, Egon Stemle, Rutuja Ubale, and Xi- anyang Chen. 2020. A report on the 2020 VUA and TOEFL metaphor detection shared task. In Proceedings of the Second Workshop on Figurative Language Processing, pages 18 29, Online. Associ- ation for Computational Linguistics.
Kalpit Dixit and Yaser Al-Onaizan. 2019. Span-level model for relation extraction. In Proceedings of the 57th Amnual Meeting of the Association for Computational Linguistics, pages 5308-5314, Flo- rence, Italy. Association for Computational Linguis- tics.
Lori Levin, Teruko Mitamura, Brian MacWhinney Davida Fromm, Jaime Carbonell, Weston Feely, Robert Frederking, Anatole Gershman, and Car- los Ramirez. 2014. Resources for the detection of conventionalized metaphors in four languages. In Proceedings of Ninth International Conference on Language Resources and Evaluation (LREC" 14), pages 498-501, Reykjavik, Iceland. European Lan- guage Resources Association (ELRA).
Brik-Lån Do Dinh and Iryna Gurevych. 2016. Token- level metaphor detection using neural networks. In Proceedings of the Fourth Workshop on Metaphor in NI.P, pages 28-33.
Erik-Lån Do Dinh, Hannah Wieland, and Iryna Gurevych. 2018. Weeding out conventionalized metaphors: corpus of novel metaphor annota- tions. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1312-1424, Brussels, Belgium. Association for Computational I Linguistics.
Shuqun Li, Liang Yang, Weidong He, Shiqi Zhang, Jingjie Zeng, and Hongfei Lin. 2021. Label- enhanced hierarchical contextualized representa- tion for sequential metaphor identification. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3533-3543.
Markus Eberts and Adrian Ulges, 2020. Span-based joint entity and relation extraction with transformer pre-training. In ECAI 2020, pages 2006-2013. IOS Press.
Yucheng Li, Shun Wang. Chenghua Lin, Frank Guerin, and Loie Barrault. 2023. FrameBERT: Concep- tual metaphor detection with frame embedding learning. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguisties, pages 1558-1563, Dubrovnik, Croatia. Association for Computational Linguistics.
Ge Gao, Eunsol Choi, Yejin Choi, and Luke Zettle- moyer. 2018. Neural metaphor detection in con- text. In Proceedings of the 2018 Conference on Empirical Methods Natural Language Processing, pages 607-613, Brussels, Belgium. Association for Computational Linguistics.
Mengshi Ge. Rui Mao, and Erik Cambria. 2022. Ex- plainable metaphor identification inspired by concep- tual metaphor theory. In Proceedings of the AAAT Conference Artificial Intelligence, volume 36, pages 10681-10689.
Lizhen Liu, Xiao Hu, Wei Song, Ruiji Fu, Ting Liu, and Guoping Hu. 2018. Neural multitask learn- ing for simile recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1543-1553.
Pragglejaz Group. 2007. Mip: A method for identifying metaphorically used words in discourse. Metaphor and symbol, 22(1):1-33
Saif Mohammad, Ekaterina Shutova, and Peter Tur- ney. 2016. Metaphor a medium for emotion: An empirical study. In Proceedings the Fifth Joint Conference on Lexical and Computational Semantics, pages 23-33.
George Lakoff. 1994. Master metaphor list. University of California.
George Lakoff and Mark Johnson. 2008. Metaphors we live by. University of Chicago press.
Michael Mohler, Mary Brunson, Bryan Rink, and Marc Tomlinson. 2016. Introducing the LCC metaphor

---

datasets. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC' 16), pages 422 4221-4227, Portoro7, Slovenia. European Language Resources Association (ELRA).
detection with cross-lingual model transfer. Proceedings the 52nd Annual Meeting the Association for Computational Linguistics (Volume
1: Long Papers), pages 248-258, Baltimore, Mary- land. Association for Computational Linguistics.
Marek Rei, Luana Bulat, Douwe Kiela, and Ekate- rina Shutova. 2017. Grasping the finer point: A supervised similarity network for metaphor detec- tion. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. pages 1337-156, Copenhagen, Denmark. Associa- tion for Computational Linguistics.
Yorick Wilks. 1978. Making preferences more active. Artificial intelligence, 177):297-2233
Dongyu Zhang, Hongfei Lin, Liang Yang, Shaowu Zhang, and Bo Xu. 2018. Construction of Chinese corpus for the analysis of the emotional- ity of metaphorical expressions. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 144-150, Melbourne, Australia. As- sociation for Computational Linguisties.
Ekaterina Shutova, Douwe Kiela, and Jean Maillard. 2016. Black holes and white rabbits: Metaphor identification with visual features. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 160-170, San Diego, California. Association for Computational Linguistics.
Zexuan Zhong and Danqi Chen. 2021. A frustrat- ingly casy approach for entity and relation ex- traction. In Proceedings of the 2021 Conference of the North American Chapter the Association for Computational Linguistics: Human Language Technologies, pages 50-61 Online, Association for Computational Linguistics.
Ekaterina Shutova and Simone Teufel. 2010. Mctaphor corpus annotated for source target domain map- pings. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'I0), Valletta, Malta. European Language Re- sources Association (ELRA).
Wei Song, Jingjin Guo, Ruiji Fu, Ting Liu, and Lizhen Liu. 2021a. A knowledge graph embed- ding approach for metaphor processing. IEEB/ACM Transactions Audio, Speech, and Language Processing, 29:406-420.
Wei Song, Shuhui Zhou, Ruiji Fu, Ting Liu, and Lizhen Liu. 2021b. Verb metaphor de- tection via contextual relation learning. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 4240 4251, Online. Association for Computa- tional Linguistics.
Gerard Steen. 1999. From linguistic to concep- tual metaphor in five steps. AMSTERDAM STUDIES IN THE THEORY AND HISTORY OF LINGUISTIC SCIENCE SERIES 4, pages 57-78.
Gerard Steen, Aletta G Dorst, J Berenike Herrmann, Anna Kaal, Tina Krennmayr, Trijntje Pasma, et al. 2010. A method for linguistic metaphor identifica- tion. Amsterdam: Benjamins.
Chuandong Su, Fumiyo Fukumoto, Xiaoxi Huang, Jiyi Li, Rongbo Wang, and Zhiqun Chen. 2020. DeepMet: A reading comprehension paradigm for token-level metaphor detection. In Proceedings of the Second Workshop on Figurative Language Processing, pages 30-39, Online. Association for Computational Linguistics.
Yulia Tsvetkov, Leonid Boytsov, Anatole Gershman, Eric Nyberg, and Chris Dyer. 2014. Metaphor


Reviews:
[{"review_id": "7pJ1ipQpnC", "reviewer": "Reviewer_7vFy", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your insightful comments. We address your concerns here.\n\n> I am not sure if the identified metaphorical relations fully capture the actual rich variation among metaphors. As a non-Chinese speaker, I am not able to comment on it further but I presume that many conceptual metaphors that are not necessarily easily identified by spans of literal text are being ignored by this setting. Similarly, many colloquial usages which show a difference in domains but have become prevalent enough (especially ‘verb’ relation-based metaphors) to not be metaphors will be counted as metaphors in this scheme. A further discussion of this will strengthen the paper.\n\nMetaphors have rich variations, but the metaphorical relations defined here mainly focus on the condition that both source and target domains share in the sentence. They may not be able to capture metonymy, which requires a larger context and common sense to understand. \n\n> The dataset appears to be fairly small in size and Figure 2 shows imbalance in terms of metaphor types represented in the dataset. Is this variation natural, or an artifact of annotation guidelines?\n\nOur dataset is based on a public dataset that has an imbalance issue among the metaphorical relation types. We did not analyze the natural distribution of these types, but it is a good suggestion for future work. We are currently working on alleviating the imbalance issue by incorporating extra data in a semi-supervised way that uses the current classifier to recall more minority types for human annotation.\n\n> Span enumeration appears expensive in the model and might be difficult to scale to longer text. Moreover, span pruning might help with the cost but it now adds complication to equation 8 which seems to assume that predicted spans include the ground truth spans. Due to discontinuity of pruning, it can make training difficult when generalizing to broader settings. Moreover, it is not clear how the loss in eqn 4 depends on the ground truth. The model description should be improved.\n- Currently, our approach is focused on sentence-level processing, so the span enumeration cost is acceptable. The loss function in Eq. 8 is computed over the intersection of the predicted span pairs and the ground truth span pairs, which does not incur additional costs.\n- By optimizing the objective in Eq.3, the model naturally learns to prune spans accurately. Although the initial pruning is random, only gold span pairs can receive positive updates. So the model can quickly make use of such a signal for credit assignment to the different factors, including the span scoring function. \n- Eq. 4 is based on Eq. 3, where $Y_i$​ corresponds to the gold span pairs and $\\mathcal{Y}_i$​ denotes all possible span pair predictions. So the connection between Eq. 4 and the ground truth is built. \n\n> line 343: I am confused about the description of trigger words. Are these examples excluded or handled separately?\n\nWe use the term “trigger words” to refer to words such as “like” or “as” that are often used in similes. In the original dataset, sentences that contain such words are likely to be simile sentences. This creates a bias that makes the simile relations easy to identify. To avoid this, we add sentences that also contain such words, but are not figurative, to reduce the bias. We will use a more precise terminology and explain this procedure more clearly.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "– The proposed formalization of metaphor annotation is reasonable and provides a finer-grained perspective of metaphor characterization tasks.\n\n– The dataset production mechanism procedure is soundly described and appears reliable and useful.\n\n– The model seems simple to implement and the loss function/objective is reasonable.\n\n– The proposed approach outperforms pipeline based approaches which shows the importance of joint modeling of span prediction and metaphor relation identification. It also outperforms other approaches on metaphoricity detection task.\n\n– The limitations section identifies the appropriate weaknesses.\n", "reasons_to_reject": "– I am not sure if the identified metaphorical relations fully capture the actual rich variation among metaphors. As a non-Chinese speaker, I am not able to comment on it further but I presume that many conceptual metaphors that are not necessarily easily identified by spans of literal text are being ignored by this setting. Similarly, many colloquial usages which show a difference in domains but have become prevalent enough (especially ‘verb’ relation-based metahors) to not be metaphors will be counted as metaphors in this scheme. A further discussion of this will strengthen the paper.\n\n– The dataset appears to be fairly small in size and Figure 2 shows imbalance in terms of metaphor types represented in the dataset. Is this variation natural, or an artifact of annotation guidelines?\n\n– Span enumeration appears expensive in the model and might be difficult to scale to longer text. Moreover, span pruning might help with the cost but it now adds complication to equation 8 which seems to assume that predicted spans include the ground truth spans. Due to discontinuity of pruning, it can make training difficult when generalizing to broader settings. Moreover, it is not clear how the loss in eqn 4 depends on the ground truth. The model description should be improved.\n\n– line 343: I am confused about the description of trigger words. Are these examples excluded or handled separately?\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper focuses on chinese metaphor detection with a focus on characterizing relationships between spans in text that are salient for determining the existence of metaphor and its type. This is different from prior work that either focuses on: a) characterizing/identifying a single word for metphoricity and metaphor prediction is done by using the potentially metaphorical word and its context, b) annotating and identifying syntactic relationsships between pairs of words and using selecitonal preference-based approaches to characterize metaphoricity. This work proposes to annotate spans of contiguous tokens as either pertaining to the “source” domain or the “target” domain that the metaphor maps into. The source spans are characterized to bel;ong to 4 different kinds in the annotation scheme. Based on the source types, 4 types of metaphorical relations are annotated further bertween source spans and target spans. This formalization of metaphor analysis is then used to expand an existing Chinese metaphor dataset with more literal example and finer-grained metaphorical details. Additional filtering is done on the basis of easy-to-predict lexical metaphors and length. A joint span detection and relation type prediction approach is proposed based on BERT representations and is trsained and tested on the annotated dataset. This approach is compared against a pipeline baseline, an existing metaphor detection system, and ablative variants of the proposed approach on span extraction, and metaphor relation/type detection. Analysis on standard binary task of metaphor detection in a single-token + context setting against other baselines is also performed.\n", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "o6n5NSTeyF", "reviewer": "Reviewer_SXfz", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your comments and feedback. \n\n> The annotation scheme in 3.1 is very unclear without the examples and even with, I don't fully understand it. It seems that several of the attributes such as attribute seem to be very connected to part-of-speech, even though this is something the authors criticise in other works.\n\nWe will revise the annotation scheme in 3.1. Simply put, our annotation scheme is very similar to an entity relation scheme, where target span and source-related span represent the descriptions related to the metaphorical and literal entities, respectively, and the relation is used to describe the relationship between them. Because the literal entities are often missing in the sentence, the definition of these relations is determined by referring to how the literal entity is manifested in the text, such as the action or attribute of the literal entity. This is indeed similar to part-of-speech, but what we consider here are text spans rather than words.\n\n>  In 3.2.1, The section starting \"the annotation process is as follows\" is very unclear. How do you define \"good consistency and inter-annotator agreement\", what exactly is the procedure here?\n\nThe annotation procedure consists of three stages. In the first stage, we train the annotators to understand and follow the manual. We emphasize the importance of consistent and reliable annotation, especially on the span boundaries and the existence of the relations. In the second stage, we assign the whole dataset to both annotators and let them annotate independently. In the third stage, we ask them to resolve any conflicts and reach an agreement.\n\n> It would be interesting to see if these results also applied to metaphor corpora used by other authors.\n\nIn the future, we plan to annotate metaphorical relations in existing English corpora so that we can compare our approach with previous methods, including the one suggested by the reviewer.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This is an interesting and novel view on the metaphor identification process. \nThis paper develops a new corpus.\nThe results show a large improvement over state-of-the-art metaphor corpus, although only on the author's corpus", "reasons_to_reject": "The annotation scheme in 3.1 is very unclear without the examples and even with, I don't fully understand it. It seems that several of the attributes such as attribute seem to be very connected to part-of-speech, even though this is something the authors criticise in other works.\nIn 3.2.1, The section starting \"the annotation process is as follows\" is very unclear. How do you define \"good consistency and inter-annotator agreement\", what exactly is the procedure here?\n\nIt would be interesting to see if these results also applied to metaphor corpora used by other authors.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper presents a method for identification of metaphors by formulating it as a relation extraction process. The paper develops a new dataset that annotates relation extraction according to this paradigm. They present results based on a comparison with a relation extraction system. ", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Metaphor detection via linguistics enhanced Siamese network - MisNet seems like it is of value to compare to here."}, "extra_text": {}}, {"review_id": "N4L2OTKOlS", "reviewer": "Reviewer_oJ38", "summary": "", "questions": "A, in section 4.3, the scores are not normalized, does the score of the positive pair increase exponentially?\n\nB, in section 4.4, the relation between two spans is classified, there are two losses, one is for the first span and the other is for the second span. Why did you use the two losses for each span, instead of the single loss for relation identification between them?\n\nC, what is the span type feature in equation 9, and how to obtain y_{i,k}. \n\nD, in table 2, span type classification should be after the span extraction. How to use type to span extraction? \n\nE, the span extraction is only <0.7 F1, the results of span type identification should be more lower. It is doubtful that adding the noisy features to the relation classification increase the performance.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your comments and questions. We address your concerns here.\n\n> in section 4.3, the scores are not normalized, does the score of the positive pair increase exponentially?\n\nAs shown in Eq. 3, we use a product of multinomial distributions for each span. Each term is the probability of a gold pair given the span. Therefore, the scores are actually normalized.\n\n>   in section 4.4, the relation between two spans is classified, there are two losses, one is for the first span and the other is for the second span. Why did you use the two losses for each span, instead of the single loss for relation identification between them?\n\nAs we illustrate with an example in Section 4.4, one span can have different types depending on the metaphorical relation it is involved in. This means that we cannot determine the type of each span without considering the pair of spans that form the relation. Moreover, the type of the relation between two spans does not necessarily indicate the type of each span. Our objective is to infer both the span types and the relation types simultaneously. Therefore, we use two losses for each span to optimize their type prediction, and another loss for the relation to optimize its type prediction. In this way, the span type classifier can still receive positive updates even if the relation type is wrongly predicted.\n\n> what is the span type feature in equation 9, and how to obtain $y_{i,k}$.\n\nThe span type feature is described in Section 4.3 and is used to calculate the score of each span pair in Eq. 6. The term $y_{i,k​}$ represents the probability of the $k$-th type for the $i$-th span in the sentence, which is obtained from the span type classifier in Eq. 7.\n\n>  in table 2, span type classification should be after the span extraction. How to use type to span extraction?\n\n> the span extraction is only <0.7 F1, the results of span type identification should be more lower. It is doubtful that adding the noisy features to the relation classification increase the performance.\n\nThe above concerns relate to the defination of the span extraction task. We define the span extraction task as finding the boundary and type of each span. As we stated in Section 5.2, “a span is considered correct if its type and region are both correct.\" (match the gold annotation). Therefore, span type identification is included in span extraction. We will make this clear in the revised paper.\n\n> The baseline model is not strong enough.\n\nMelBERT and MrBERT are among the competitive state-of-the-art models for token-level metaphor detection [1][2]. However, our task is different, as we aim to detect metaphorical relations at the span level. To compare our method with them fairly, we evaluate them on the task of metaphorical sentence detection, which is a binary classification problem. In this way, we can measure their performance in identifying sentences that contain metaphorical relations or metaphorical tokens.\n\n[1] Metaphor detection via linguistics enhanced Siamese network, ACL 2022.\n[2] FrameBERT: Conceptual metaphor detection with frame embedding learning, EACL 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1, they created a dataset for Chinese metaphorical relation extraction.", "reasons_to_reject": "1, the experiments are not solid and some experimental settings is not reasonable, and see question 3 and 4. \n2, the experiment results are not convincing, and see question 5.\n3, the baseline model is not strong enough.", "questions_to_authors": "A, in section 4.3, the scores are not normalized, does the score of the positive pair increase exponentially?\n\nB, in section 4.4, the relation between two spans is classified, there are two losses, one is for the first span and the other is for the second span. Why did you use the two losses for each span, instead of the single loss for relation identification between them?\n\nC, what is the span type feature in equation 9, and how to obtain y_{i,k}. \n\nD, in table 2, span type classification should be after the span extraction. How to use type to span extraction? \n\nE, the span extraction is only <0.7 F1, the results of span type identification should be more lower. It is doubtful that adding the noisy features to the relation classification increase the performance.", "paper_topic_and_main_contributions": "The paper aims to reformulate the metaphor identification as relation extraction problem, and they created a dataset for Chinese metaphorical relation extraction. Also they propose novel models to this end. The experimental results show that the proposed model achieves the state-of-the-art results in their constructed benchmark, compared to the baseline models.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors introduce a new resource for Chinese metaphor relation extraction with almost 10K annotated sentences, consisting of the addition of a new annotation layer to the metaphor dataset by Zhang et al. (2018). They annotate spans of text to identify the source and the target of the metaphor, and they add more fine-grained types for both the source spans (object, action, attribute etc.) and the relations.
Finally, they propose a span-based end-to-end model to extract span pairs from sentences and detect if they are linked by a metaphor relation. Using the newly-annotated dataset, the system is compared with the relation extraction model by Eberts and Ulges (2020) on the span/relation extraction task and with MelBERT/MrBERT on token-level metaphor detection, showing improvements over the baselines in both scenarios.

Among the positives of this paper, the introduction of a new dataset for Chinese metaphor relation extraction and a fine-grained characterization of metaphors in the annotation scheme. On the downside, there is need of some more clarification about the annotation procedure (see discussion with R2) and about the experimental settings and the baseline choice (see discussion with R3).
Decision: Accept-Findings