SlowGuess/ACL_2020_OCR huggingface dataset:
Chuang Fan $^{1}$ , Chaofa Yuan $^{1}$ , Jiachen Du $^{1}$ , Lin Gui $^{2*}$ , Min Yang $^{3}$ , Ruifeng Xu $^{1,4,5*}$
$^{1}$ Harbin Institute of Technology (Shenzhen), China  $^{2}$ University of Warwick, UK
<sup>3</sup>Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences
$^{4}$ Joint Lab of Harbin Institute of Technology and RICOH,  $^{5}$ Peng Cheng Laboratory
{fanchuanghit, bruceyuan123, jacobvan199165}  $@$  gmail.com
Lin.Gui@warwick.ac.uk, min.yang@siat.ac.cn, xuruifeng@hit.edu.cn
Emotion-cause pair extraction aims to extract all potential pairs of emotions and corresponding causes from unannotated emotion text. Most existing methods are pipelined framework, which identifies emotions and extracts causes separately, leading to a drawback of error propagation. Towards this issue, we propose a transition-based model to transform the task into a procedure of parsing-like directed graph construction. The proposed model incrementally generates the directed graph with labeled edges based on a sequence of actions, from which we can recognize emotions with the corresponding causes simultaneously, thereby optimizing separate subtasks jointly and maximizing mutual benefits of tasks interdependently. Experimental results show that our approach achieves the best performance, outperforming the state-of-the-art methods by  $6.71\%$ $(p < 0.01)$  in F1 measure.
Emotion-cause pair extraction (ECPE) is a new task to identify emotions and the corresponding causes from unannotated emotion text (Xia and Ding, 2019). This involves several subtasks, including 1) Extracting pair components from input text, e.g., emotion detection and cause detection; 2) Combining all the elements of the two sets into emotion-cause pairs and eliminating the pairs that do not exist a causal relationship. For the former subtask, a clause can be categorized into "emotion", which usually contains an emotion keyword to express specific sentiment polarity, or "cause", which contains the reason or stimuli of an observed emotion. Then, the set of all possible emotion-cause pairs will be fed into the second subtask to determine the relationship. In general, it is an essential issue in emotion analysis since it provides
Figure 1: An example of emotion-cause pair extraction.
a new perspective to investigate how emotions are provoked, expressed, and perceived.
Figure 1 shows an example of ECPE, and the text is segmented into three clauses. In this instance, only the second clause and the third clause hold an emotion causality, where "I lost my phone while shopping" is the cause of emotion "I feel sad now". Thus, the extracted results of this sample should be  $\{I$  lost my phone while shopping, I feel sad now\} The goal of ECPE is to identify all the pairs that have emotion causality in an emotion text.
However, from both theoretical and computational perspectives, due to the inherent ambiguity and subtlety of emotions, it is hard for machines to build a mechanism for understanding emotion causality like human beings. Previous approaches mostly focused on detecting the causes towards the given annotation of emotions, which was followed by most of the recent studies in this field (Lee et al., 2010; Gui et al., 2014; Gao et al., 2015; Gui et al., 2016, 2017; Li et al., 2018; Xu et al., 2019; Fan et al., 2019). Nevertheless, it suffers that emotions must be annotated before extracting the causes, which limits the applications in real-world scenarios. Towards this issue, Xia and Ding (2019) presented a new task to extract emotion-cause pairs from the unannotated text. However, they followed a pipelined framework, which models emotions and causes separately, rather than joint decoding. Hence, to overcome the drawback of error propagation may occur in existing methods. Ideally,
* Co-Corresponding Authors
3707
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3707-3717 July 5 - 10, 2020. ©2020 Association for Computational Linguistics
the emotion-cause structure should be considered as an integral framework, including representation learning, emotion-cause extraction, and reasoning.
To this end, we transform the ECPE problem into a procedure of directed graph construction, from which emotions and the corresponding causes can be extracted simultaneously based on the labeled edges. The directed graph is constructed by designing a novel transition-based parsing model, which incrementally creates the labeled edges according to the causal relationship between the connected nodes, through a sequence of defined actions. In this process, the emotion detection, cause detection, and their causality association can be jointly learned through joint decoding, without differentiating subtask structures, so that the maximum potential of information interaction between emotions and causes can be exploited. Besides, the proposed model processes the input sequence in a psycholinguistically motivated left to right order, consequently, reducing the number of potential pairs needed to be parsed and leading to speed up (if all clauses are connected by Cartesian products, the time complexity will be  $O(n^{2})$ ).
Regarding feature representation, BERT (Devlin et al., 2019) is used to produce the deep and contextualized representation for each clause, and LSTMs (Hochreiter and Schmidhuber, 1997) are performed to capture long-term dependencies among input sequences. In addition, action history and relative distance information between the emotion-cause pairs are also encoded to benefit the task.
To summarize, our contribution includes:
- Learning with a transition-based framework, so that end-to-end emotion-cause pair extraction can be easily transformed into a parsing-like directed graph construction task.
- With the proposed joint learning framework, our model can extract emotions with the corresponding causes simultaneously, often with linear time complexity.
- Performance evaluation shows that our model statistically significant improvements over the state-of-the-art methods on all the tasks<sup>1</sup>.
The formal definition of emotion-cause pair extraction is given in (Xia and Ding, 2019). Briefly,
Figure 2: The architecture of our model. Dashed lines denote the components only work in training stage.
given a piece of emotion text  $d_1^n = (c_1, c_2, \ldots, c_n)$ , which consists of several manually segmented clauses. The goal of ECPE is to output all potential pairs where exist emotion causality:
$$
P = \{\dots , (c ^ {e}, c ^ {c}), \dots \} \tag {1}
$$
where  $c^e$  is an emotion clause, and  $c^c$  is the corresponding cause clause.
Note that, the previous emotion cause extraction (ECE) task aims to extract  $c^c$  given the annotation of  $c^e$ :  $\{c^c \to c^e\}$ . In comparison, the ECPE is a new and challenging task since there is no annotation provided in the emotion text. Similar as the traditional ECE task, the ECPE is also defined at the clause level, because it is difficult to describe emotion causes at the word or phrase level. That is, in this paper, the "emotion" and "cause" are refer to "emotion clause" and "cause clause", respectively.
We present a new framework aimed at integrating the emotion-cause pair extraction into a procedure of parsing-like directed graph construction. The proposed framework incrementally constructs and labels the graph from input sequences, scoring partially segmented results using rich non-local features. Figure 2 shows the overall architecture of the proposed framework. In the following, we first introduce how to construct the directed graph based on a novel transition-based system, then the details of feature representation will be described.
3.1 Directed Graph Construction
Let  $G = (V, R)$  be an edge-labeled directed graph where:  $V = \{1, 2, \dots, n\}$  is the set of nodes that correspond to clauses in the input text and
1The code and dataset are available at: https:// github.com/HLT-HITSZ/TransECPE
Action	Change of state
SH	(σ|σ1|σ0,β0|β,E,C,R)/(σ|σ0|β0,β',E,C,R)
RA lt	(σ|σ1|σ0,β0|β,E,C,R)
(σ|σ0|,β0|β,E∪{σ0},C∪{σ1},R∪{σ1lnt→σ0})
LA lt	(σ|σ1|σ0,β0|β,E,C,R)
(σ|σ1|,β0|β,E∪{σ1},C∪{σ0},R∪{σ1lnt→σ0})
RA ln	(σ|σ1|σ0,β0|β,E,C,R)
(σ|σ0,β0|β,E∪{σ0},C,R∪{σ1lnt→σ0})
LA ln	(σ|σ1|σ0,β0|β,E,C,R)
(σ|σ0|β0,β',E∪{σ1},C,R∪{σ1lnt→σ0})
CA	(σ|σ0,β0|β,E,C,R)
(σ|σ0,β0|β,E∪{σ0},C∪{σ0},R∪{σ0lnt→σ0})
$R = V \xrightarrow{R} V$  is the set of labeled edges. We will denote a connection between a head node  $i \in V$  and a modifier node  $j \in V$  as  $i \xrightarrow{l} j$ , where  $l \in \{l_t, l_n\}$  is the causality label connecting them.  $l_t$  indicates the node  $i$  is the cause of the emotion node  $j$  while  $l_n$  indicates node  $j$  is an emotion but node  $i$  is not the corresponding cause. Besides, other nodes irrelevant to the final result have no edges. Note that, in this task, a node can be emotion and the corresponding cause simultaneously. Furthermore, an emotion node can also be associated with multiple causes. Thus, the acyclicity and single-head constraints are not necessary for our model, as arbitrary graphs are allowed.
We build the directed graph by designing a novel transition-based parser. Formally, each state of our parser is represented by a tuple:  $S = (\sigma, \beta, E, C, R)$ , where  $\sigma$  and  $\beta$  are disjoint lists called stack and buffer, which store the indices of nodes that have been processed and to be processed, respectively.  $E$  is the set of emotions, and  $C$  is the set of causes.  $R$  is used to store the edges generated so far. Besides, action history is stored to a list  $A$ .
The definition of action set plays a crucial role in the transition-based system, and it relies on the type of task. As shown in Table 1, we define 6 types of actions based on our empirical observation, and their logics are summarized as follows:
- SHIFT (SH). Pops  $\beta_0$  and puts it on the top of  $\sigma$ . It is legal only when the  $\beta$  is not empty.
Table 1: Defined transition actions in our parser. For ease of illustration, we use the subscript  $i \in \{0,1,\ldots\}$  to denote the item index in the stack (starting from right), buffer and action (starting from left). That is, the top two items in the stack can be marked as  $\sigma |\sigma_1|\sigma_0$  (similar to buffer and action).
Stack	Buffer	Action	Emotion	Cause	Edge
[]	[1,2,3,$]	SH	∅	∅	∅
[1]	[2,3,$]	SH	∅	∅	∅
[1,2]	[3,$]	SH	∅	∅	∅
[1,2,3]	[$]	RAlt	∅ ∪ {3}	∅ ∪{2}	2 lt→3
[1,3]	[$]	RAIₙ	{3} ∪ {3}	-	1 Iₙ→3
[3]	[$]	SH	-	-	-
[3,$]	[]	-	-	-	-
Table 2: Transition sequence for the text in Figure 1.
- RIGHT-  $\mathrm{ARC}_{l_t}$  ( $\mathrm{RA}_{l_t}$ ). It assigns an edge from  $\sigma_1$  to  $\sigma_0$  with label  $l_t$ :  $\sigma_1 \xrightarrow{l_t} \sigma_0$ , then copies  $\sigma_0$  to  $E$  and pops  $\sigma_1$  from  $\sigma$  to  $C$ .
- LEFT-  $\mathrm{ARCl}_t$  ( $\mathrm{LA}_{l_t}$ ). It assigns an edge from  $\sigma_0$  to  $\sigma_1$  with label  $l_t$ :  $\sigma_1 \xleftarrow{l_t} \sigma_0$ . Then copies  $\sigma_1$  to  $E$  and pops  $\sigma_0$  from  $\sigma$  to  $C$
- RIGHT-  $\mathrm{ARC}_{l_n}$  ( $\mathrm{RA}_{l_n}$ ). Adds a relation from  $\sigma_1$  to  $\sigma_0$  with label  $l_n$ :  $\sigma_1 \xrightarrow{l_n} \sigma_0$ . Then pops  $\sigma_1$  out of  $\sigma$  and only copies  $\sigma_0$  to  $E$ .
- LEFT-  $\mathrm{ARCl}_{l_n}$  ( $\mathrm{LA}_{l_n}$ ). It denotes a relation from  $\sigma_0$  to  $\sigma_1$ :  $\sigma_1 \xleftarrow{l_n} \sigma_0$  and copies  $\sigma_1$  to  $E$ . Note that, we move  $\beta_0$  to the top of  $\sigma$  to improve coverage rather than pops  $\sigma_0$ , because  $\sigma_0$  may be the cause of incoming nodes in the  $\beta$ .
CYCLE-ARC (CA). It assigns a loop edge on the node  $\sigma_0$  with label  $l_{t}$  and then copies  $\sigma_0$  to both  $E$  and  $C$ .
Action Constraints. To ensure that each parser state is valid, we need to specify some constraints on the action. For example, RIGHT-* and LEFT-* can only be conducted when there are at least two elements in the  $\sigma$ . We also empirically set a constraint that RIGHT-ARCLn will be performed when  $\sigma |\sigma_1|\sigma_0$  are both emotions but has no emotion causality. Additionally, in practical, CYCLE-ARC may conflict with other actions, e.g.,  $\sigma_0$  is the cause of itself but is also the cause of  $\sigma_{1}$ , which conflicts with the LEFT-ARCLt. For simplicity and efficiency, we separate it from other actions and distinguish it by training a binary classifier only depends on the representation of  $\sigma_0$ .
Table 2 illustrates the gold-standard sequence of transitions for the text in Figure 1. The parser state is initialized to ([], [1, 2, 3],  $\emptyset$ ,  $\emptyset$ ,  $\emptyset$ ) and the terminal state is ([...,$], [],  $E$ ,  $C$ ,  $R$ ), where $ indicates the termination of transitions.
Search Algorithm. For the ECPE task, we transform it into a procedure of directed graph construction by a sequence of actions. The input is an emotion text  $d_1^n = (c_1, c_2, \ldots, c_n)$  and the output is the corresponding sequence of actions  $A_1^m = (a_1, a_2, \ldots, a_m)$ . Hence, the task can be regarded as searching for an optimal action sequence  $A^*$  given the stream of clauses  $d_1^n$ :
$$
A ^ {*} = \operatorname {a r g m a x} _ {A} p \left(A _ {1} ^ {m} \mid d _ {1} ^ {n}\right) \tag {2}
$$
Formally, at step  $t$ , our model predicts the next action based on the current system state  $S_{t}$  and the action history  $A_1^{t - 1}$ . Thus, the task is modeled as:
$$
\left(A ^ {*}, S ^ {*}\right) = \operatorname {a r g m a x} _ {A, S} \prod_ {t} p \left(a _ {t}, S _ {t + 1} \mid A _ {1} ^ {t - 1}, S _ {t}\right) \tag {3}
$$
where  $a_{t}$  is the generated action at step  $t$ , and  $S_{t + 1}$  is the updated system state according to  $a_{t}$ .
Let  $r_t$  to denote the representation for computing the probability of the action  $a_t$  at step  $t$ , this yields:
$$
p \left(a _ {t} \mid r _ {t}\right) = \frac {\exp \left(w _ {a _ {t}} ^ {\top} r _ {t} + b _ {a _ {t}}\right)}{\sum_ {a ^ {\prime} \in \mathcal {A} (S)} \exp \left(w _ {a ^ {\prime}} ^ {\top} r _ {t} + b _ {a ^ {\prime}}\right)} \tag {4}
$$
where  $w_{a}$  denotes a learnable parameter vector and  $b_{a}$  is a bias term. The set  $\mathcal{A}(S)$  represents the legal actions that can be taken given the current parser state. Finally, the overall optimization function is:
$$
\begin{array}{l} \left(A ^ {*}, S ^ {*}\right) = \operatorname {a r g m a x} _ {A, S} \prod_ {t} p \left(a _ {t}, S _ {t + 1} \mid A _ {1} ^ {t - 1}, S _ {t}\right) \\ = \operatorname {a r g m a x} _ {A, S} \prod_ {t} p \left(a _ {t} \mid r _ {t}\right) \tag {5} \\ \end{array}
$$
where the ECPE is merged into a transition-based action prediction task. For efficient decoding, the maximum probability action is chosen greedily until the parsing procedure is termination.
3.2 Neural Transition-based Model
We apply BERT to produce the representation for each clause and use LSTMs to capture long-term dependencies of each parser state.
Representation of Clause. Given an emotion text  $d_1^n = (c_1, c_2, \ldots, c_n)$  consisting of  $n$  clauses and each clause  $c_i = (w_{i1}, w_{i2}, \ldots, w_{il})$  contains  $l$  words. We formulate each clause as a sequence  $x_i = ([\mathrm{CLS}], w_{i1}, \ldots, w_{il}, [\mathrm{SEP}])$ , where [CLS] is a special classification token that the final hidden state is used as the aggregate sequence features and [SEP] is a dummy token not used in our
model. Thus, we obtain the hidden representation as  $h_{c_i} = \mathrm{BERT}(x_i) \in \mathbb{R}^{d_b * |x_i|}$  where  $d_b$  is the size of hidden dimension and  $|x_i|$  is the length of sequence  $x_i$ . Then, the text  $d_1^n$  can be represented as  $h_d = [h_{c_1}, h_{c_2}, \ldots, h_{c_n}]$ .
Representation of Parser State. When the parsing starts, the parser state will be initialized to  $\left([1,2,\dots,n],\emptyset ,\emptyset ,\emptyset\right)$  and a series of actions will consume the clauses in the buffer to incrementally build an output until reaches the terminal state  $\left[\ldots ,\$ \right],\left[\right],E,C,R)$ , as shown in Table 2.
Specifically, at step  $t$ , considering the triple  $(\sigma_t, \beta_t, A_t)$ , where  $\sigma_t = (\ldots, \sigma_1, \sigma_0)$ ,  $\beta_t = (\beta_0, \beta_1, \ldots)$  and  $A_t = (\ldots, a_{t-2}, a_{t-1})$ . For the stack, to summarize the information from both directions, we use bidirectional LSTM to exploit two parallel passes, thus, the feature representation of  $\sigma_t$  is denoted as:
$$
s _ {t} = \mathrm {L S T M} _ {\mathrm {s}} ([ \dots , \overrightarrow {\sigma_ {1}}, \overrightarrow {\sigma_ {0}} ], [ \dots , \overleftarrow {\delta_ {1}}, \overleftarrow {\delta_ {0}} ]) \tag {6}
$$
where  $s_t = [\overrightarrow{s_t}, \overleftarrow{s_t}]$  that both  $\overrightarrow{s_t}$  and  $\overleftarrow{s_t} \in \mathbb{R}^{d_l * |\sigma_t|}$ ,  $d_l$  is the size of hidden dimension of LSTM and  $|\sigma_t|$  is the size of  $\sigma_t$ . Similarly, we can get the representation for  $\beta_t$  by:
$$
b _ {t} = \operatorname {L S T M} _ {\mathrm {b}} \left(\left[ \overrightarrow {\beta_ {0}}, \overrightarrow {\beta_ {1}}, \dots \right], \left[ \overleftarrow {\beta_ {0}}, \overleftarrow {\beta_ {1}}, \dots \right]\right) \tag {7}
$$
where  $b_{t} = [\overrightarrow{b_{t}},\overleftarrow{b_{t}} ]$  that  $\vec{b_t}$  and  $\vec{b_t}\in \mathbb{R}^{d_l*|\beta_t|}$  where  $\beta_{t}$  is the size of  $\beta_{t}$ . For action sequence, we map each action  $a$  to a distributed representation  $e_a$  through a looking-up table  $E_{a}$ , and apply an unidirectional LSTM to obtain the complete history of actions from left-to-right:
$$
\alpha_ {t} = \mathrm {L S T M} _ {\mathrm {a}} (\dots , a _ {t - 2}, a _ {t - 1}) \tag {8}
$$
Once a new action  $a_{t}$  is generated, the embedding  $e_{a_t}$  will be added into the rightmost position of the  $\mathrm{LSTM}_a$ . To enhance the position relation between the pair  $(\sigma_1,\sigma_0)$ , we also represent their relative distance  $d$  as an embedding  $e_d$  from a looking-up table  $E_{d}$ . The final representation of parser state at step  $t$  is the combination of these features.
Action Reversal. Let us visit the example in Figure 1 again. Reading it from left-to-right, as shown in the top of Figure 3, we see the clause "I lost my phone while shopping" trigger the emotion "I feel sad now", so the predicted action would be RIGHT-ARC $_{l_t}$ . However, from a different perspective, we read it from right-to-left, as shown in the bottom
Figure 3: Illustration of action reversal.
of Figure 3, the cause "I lost my phone while shopping" behind the emotion "I feel sad now", so the predicted action should be reversed to LEFT-ARCLt. That is,  $\overrightarrow{s_t}$  and  $\overleftarrow{s_t}$  should be regarded as different features to produce different action. Based on this observation, we apply  $r_t$  and  $\hat{r}_t$  to predict the original action and reversed action, respectively, which can be used to mine the deep directional information for this task:
$$
r _ {t} = \mathrm {R e L U} ([ \overrightarrow {s _ {t}} ^ {1}; \overrightarrow {s _ {t}} ^ {0}; b _ {t} ^ {0}; \alpha_ {t} ^ {- 1}; e _ {d} ]) \qquad (9)
$$
$$
\hat {r} _ {t} = \operatorname {R e L U} \left(\left[ \overleftarrow {s} _ {t} ^ {- 1}; \overleftarrow {s} _ {t} ^ {- 0}; b _ {t} ^ {0}; \alpha_ {t} ^ {- 1}; e _ {d} \right]\right) \tag {10}
$$
where  $\mathrm{ReLU}$  is an activation function for nonlinearity. Index 0 and 1 indicate the first and second representation of  $\sigma$  and  $\beta$ ,  $-1$  indicates the last representation of action history.
Training. By learning with the transition-based framework, we convert the gold output structure in a set of training data into a gold sequence of defined actions. For each parser state at step  $t$ , we maximize the log-likelihood of the classifier in formula (5), which can be revised as:
$$
\begin{array}{l} \mathcal {J} (\theta) = \sum_ {t} \log p \left(a _ {t} \mid r _ {t}\right) + \log p \left(\hat {a} _ {t} \mid \hat {r} _ {t}\right) \tag {11} \\ + \log p (c _ {t} | s _ {t} ^ {0}) + \frac {\lambda}{2} | | \theta | | ^ {2} \\ \end{array}
$$
where  $\hat{a}_t$  is the reversed action, and  $p(c_t|s_t^0)$  is the predictive distribution of CYCLE-ARC which is separated from the other actions due to the action constraints.  $\lambda$  is the coefficient of  $L_{2}$ -norm regularization, and  $\theta$  denotes all the parameters in this model. Note that, during the test decoding, only  $r_t$  and  $s_t^0$  are used to predict the next action.
4 Dataset and Implementation Details
To be consistent with previous approaches, we adopt the only benchmark (Gui et al., 2016) to evaluate our model by following (Xia and Ding,
Item	Num.	Item	Num.	Item	Num.
Emo1	1816	Cau1	1769	ECP1	1746
Emo2	118	Cau2	156	ECP2	177
Other	11	Other	20	Other	22
Table 3: Statistical information about the dataset.  $\mathrm{Emo}_1$ $(\mathrm{Cau}_1 / \mathrm{ECP}_1)$ ,  $\mathrm{Emo}_2$ $(\mathrm{Cau}_2 / \mathrm{ECP}_2)$  and other represent the texts with 1, 2 or more than 2 emotions (causes/emotion-cause-pairs).
2019). The corpus collected from SINA city news and the details are summarized in Table 3.
In this paper, we stochastically divide the corpus into a training/development/test set in a ratio of 8:1:1. In order to obtain statistically credible results, we evaluate our method 20 times with different data splits by following (Xia and Ding, 2019) and then perform one sample  $t$ -test on the experimental results. The average results of Precision  $(P)$ , Recall  $(R)$  and F-measure  $(F1)$  are employed to measure the performance. Note that when we extract the emotion-cause pairs, we obtain the emotions and causes for each text simultaneously. Thus, we also evaluate the performance of emotion extraction and cause extraction in our model.
We adopt  $\mathrm{BERT}_{\mathrm{Chinese}}$  as the basis in this work<sup>3</sup>. Adam optimizer is used for online learning (Kingma and Ba, 2015), and initial learning rates for the BERT layer and top MLP layer are set to 1e-5 and 1e-3, respectively. The hidden size of MLP layer is set to 256, and the hidden size of all LSTMs is set to 128 with 1 layer. The embeddings of position and action are initialized randomly with dimension 128 and keep unchanged during the training stage. The dropout rate is 0.5, the batch size is 3, and the coefficient of  $L_{2}$  term is 1e-5. We train the model 10 epochs in total and adopt early stopping strategy based on the performance of development set. Then, the highest F-measure model on the development set is used to evaluate the test set.
We first compare our transition-based model with the method proposed by (Xia and Ding, 2019),
<sup>2</sup>http://news.sina.com.cn/society/
$^{3}$ Our BERT model is adapted from this implementation: https://github.com/huggingface/pytorch-pretrained-BERT
3711
Method	Emotion extraction	Cause extraction	Emotion-cause pair extraction
P(%)	R(%)	F1(%)	P(%)	R(%)	F1(%)	P(%)	R(%)	F1(%)
Indep	83.75	80.71	82.10	69.02	56.73	62.05	68.32	50.82	58.18
Inter-CE	84.94	81.22	83.00	68.09	56.34	61.51	69.02	51.35	59.01
Inter-EC	83.64	81.07	82.30	70.41	60.83	65.07	67.21	57.05	61.28
SL-BERT†	77.24	67.75	72.18	70.60	60.75	65.30	67.63	58.04	62.47
MT-BERT†	82.89	72.12	77.13	72.20	61.54	66.44	70.35	59.83	64.66
Ours†	87.16	82.44	84.74	75.62	64.71	69.74	73.74	63.07	67.99
LSTMbased	80.80	84.39	82.56	67.42	65.34	66.36	65.15	63.54	64.34
-transition†	80.66	71.99	76.08	66.34	62.68	64.31	58.93	61.37	60.12
Table 4: Comparison with competitive baselines. † denotes the results are implemented in this paper. The results are average score over 20 runs, and the best scores are in bold.
which contains three models: 1) Indep: Emotion extraction and cause extraction are independently trained, then filtering the pairs that have no emotion causality; 2) Inter-CE: The difference is that the predictions of cause extraction are used to improve emotion extraction; 3) Inter-EC: Contrary to the Inter-CE, the predictions of emotion extraction are used to improve cause extraction. It is the current state-of-the-art model for this task.