SlowGuess/ACL_2020_OCR huggingface dataset:
Comparison between the last two lines' results in Table 2 demonstrates the effectiveness of lexicon-based extraction. We can see that adding the lexicon-based extraction scheme can improve the recall  $R$ , indicating that it indeed obtains more correct emotion-cause pairs. Although the precision  $P$  slightly decreases, the F-score  $F_{1}$  still performs better than only extracting the top-1 pair in a document. Thus, lexicon-based extraction is an effective
Table 2: Experimental results on emotion-cause pair extraction. Moreover, results on emotion clause extraction and cause clause extraction are also reported. “RANKCP (top-1)” denotes the model that does not use the lexicon-based extraction at test time, and directly chooses the clause pair having the highest ranking score as the unique emotion-cause pair for a document.
# Pairs	Approach	F1	P	R
One per doc.	INTER-EC	0.6288	0.6734	0.5939
RANKCP	0.6780	0.6625	0.6966
Two or more per doc.	INTER-EC	0.4206	0.5912	0.3302
RANKCP	0.5531	0.7508	0.4390
Table 3: Comparative results for documents with only one and more than one emotion-cause pair.
way to improve the pair extraction performance of our one-step approach.
Comparison on Extracting Multiple Pairs
We further compare the results on extracting multiple pairs in one document. We divide each fold's test set into two subsets: one subset contains documents having only one emotion-cause pair, and the other subset contains documents having two or more emotion-cause pairs.
Table 3 reports the comparative results on two subsets respectively. It can be seen that our model consistently outperforms INTER-EC on both subsets. Our one-step approach is relatively more effective for documents with more than one emotion-cause pair (over  $13\%$ $F_{1}$  improvement).
Results on Emotion Cause Extraction
We also provide the comparative results with recently-proposed methods for emotion cause extraction task: a rule-based method RB (Lee et al., 2010a), a traditional machine learning based method MULTI-Kernel (Gui et al., 2016), and three neural methods CONVMS-MEMNET (Gui et al., 2017), CANN (Li et al., 2018), and RTHN (Xia et al., 2019). Note that all of them utilize known emotion clauses as model input. The top half of Table 4 reports their performance.
The bottom half of Table 4 shows the comparative results of methods without using known emotion clauses as model input. It clearly demonstrates
4https://academiasinicanlplab.github.io
5 Appendix A.2 reports the results with BERT encoder.
Emotion Cause Extraction	F1	P	R
RB	0.5243	0.6747	0.4287
MULTI-Kernel	0.6752	0.6588	0.6927
CONVMS-MEMNET	0.6955	0.7076	0.6838
CANN	0.7266	0.7721	0.6891
RTHN	0.7677	0.7697	0.7662
Cause Clause Extraction	F1	P	R
CANN-E	0.3797	0.4826	0.3160
RTHN-APE	0.5694	0.5800	0.5618
INTER-EC	0.6507	0.7041	0.6083
RANKCP	0.6824	0.6927	0.6743
Table 4: Results on emotion cause extraction task. CANN - E and RTHN-APE denote the variant models of CANN and RTHN respectively, which do not utilize known emotion clauses as model input.
Loss Function	F1	P	R
Lpair	0.6241	0.6412	0.6090
Lpair + (Lemo + Lcau)	0.6610	0.6698	0.6546
Table 5: Comparison of different supervised signals for RANKCP.
that our proposed RANKCP performs much better than other methods. Besides, although RANKCP does not utilize known emotions of test documents as model input, it still outperforms RB and MULTI-Kernel, and is comparable to CONVMS-MEMNET. Thus, our approach benefits from interclause modeling and shows its effectiveness on cause clause extraction.
4.3 Further Discussions
We conduct ablation studies to analyze the effects of different components in our approach.
Effect of Two-level Supervision
Our model is trained with a mixture of two supervised signals: a low-level signal  $\mathcal{L}_{\mathrm{emo}} + \mathcal{L}_{\mathrm{cau}}$  on clause representation learning at the output of graph attention network (see Eq. 5), and a high-level signal  $\mathcal{L}_{\mathrm{pair}}$  on clause pair representation learning and ranking (see Eq. 10). To verify the effect of low-level supervision, we train our model with  $\mathcal{L}_{\mathrm{pair}}$  only, and the results compared with those of our full model are given in Table 5. It shows that training with two-level supervision boosts the extraction performance. This indicates that incorporating a low-level supervision helps learn better clause representations, and eventually facilitates the clause pair representation learning and ranking process.
Figure 3: Results of RANKCP with various graph attention layers.
Figure 4: Comparative results of our variant model that removes the clause pair representation learning and ranking component (denoted as "RANKCP w/o Rank") and our full model RANKCP.
Effect of Graph Attention Layers
Graph attention network for modeling inter-clause latent relationships is the key component of our approach. We vary the number of graph attention layers (ranging from 0 to 3) to test its effect, and the results on emotion-cause pair extraction and cause clause extraction are shown in Fig. 3.
Obviously, the model without graph attention layer can not obtain good performance. Our approach achieves the best performance with two-layer graph attention network, indicating that interclause relationships can be modeled sufficiently without stacking a lot of layers in this task.
Effect of Clause Pair Representation Learning
We further investigate if we can obtain ideal performance by directly using clause representations to predict emotion clauses and cause clauses. In other words, we remove the clause pair representation learning and ranking component, and utilize the graph attention network's predictions (i.e., Eq. 5) to produce emotion-cause pairs. After predicting emotion clauses and cause clauses in a document, we consider all combinations of the predicted emotions and causes as the extracted emotion-cause pairs, and the comparative results of this variant model and our full model are shown in Fig. 4.
RANKCP performs much better than the variant one (especially on Recall), demonstrating that
Relative Position Scheme	F1	P	R
No	(top-1 ext.)	0.6267	0.6600	0.5973
No	(lexicon-based ext.)	0.6260	0.6378	0.6160
Vanilla	(top-1 ext.)	0.6468	0.6810	0.6164
Vanilla	(lexicon-based ext.)	0.6582	0.6669	0.6510
Kernel	(top-1 ext.)	0.6562	0.6910	0.6254
Kernel	(lexicon-based ext.)	0.6610	0.6698	0.6546
Table 6: Comparison on relative position embedding schemes. "ext." is the abbreviation for "extraction".
only offering clause-level predictions is not suitable for emotion-cause pair extraction task. Thus, combining clause-level and clause pair representation learning in a unified one-step model is indeed effective for extracting emotion-cause pairs.
Effect of Relative Position Embedding
We remove the relative position embedding part in RANKCP to verify its effect. We also compare vanilla and kernel-based relative position embedding schemes. The results are given in Table 6.
Removing relative position embedding results in performance degradation, indicating that relative position between a clause pair is indeed useful for prediction. Another observation from the first two lines is that lexicon-based extraction can not outperform top-1 extraction, which further verifies that the model without relative position embedding can not offer ideal ranking list. Kernel-based embedding achieves better performance than vanilla one on both top-1 and lexicon-based extractions, thus considering the mutual impact among relative positions helps obtain more powerful clause pair representations and further improves the performance of emotion-cause pair extraction.
4.4 Case Analysis
We illustrate a document that our approach RANKCP correctly extracts its emotion-cause pair  $(c_{5}, c_{4})$  while INTER-EC fails:
4月11日  $(c_{1})$  ，长沙网友洛丽塔在网上发帖吐槽  $(c_{2})$  ，她有一个极品男友  $(c_{3})$  ，如果要去的餐馆没有团购就要求换地方  $(c_{4})$  ，这让她感觉很不爽  $(c_{5})$  ，也很没面子  $(c_{6})$  。
Translation: On April 11th  $(c_{1})$ , a netizen posted her complains on the Internet  $(c_{2})$ , she has a wacko boyfriend  $(c_{3})$ , he never goes to a restaurant without discounts  $(c_{4})$ , this makes her feel bad  $(c_{5})$ , and very embarrassed  $(c_{6})$ .
We visualize the attention weights for two clauses  $c_4$  and  $c_5$  in Fig. 5. The emotion clause  $c_5$  attends the corresponding cause  $c_4$  with the highest
Figure 5: Attention weights for two clauses  $c_4$  and  $c_5$ .
weight, indicating that graph attention effectively captures the relationship between the two clauses.
Emotion Cause Extraction Lee et al. (2010a,b) first studied emotion cause extraction and designed a linguistic rule-based system to detect cause events. Early work attempted rule-based (Chen et al., 2010; Neviarouskaya and Aono, 2013; Gao et al., 2015), commonsense-based (Russo et al., 2011), and traditional machine learning based (Ghazi et al., 2015) approaches to extract causes for certain emotion expressions.
Gui et al. (2016) proposed an event-driven multikernel SVM method and released a benchmark corpus. Both feature based (Xu et al., 2019) and neural approaches (Gui et al., 2017; Li et al., 2018; Ding et al., 2019; Yu et al., 2019) have been proposed recently. Xia et al. (2019) adopted Transformer encoder augmented with position information and integrated global prediction embedding to improve performance. Fan et al. (2019) incorporated sentiment and position regularizers to restrain parameter learning. Hu et al. (2019) exploited external sentiment classification corpus to pretrain the model.
In other research lines, some work (Cheng et al., 2017) extracted emotion causes in the context of microblog with multi-user structure. Besides, Kim and Klinger (2018) and Bostan et al. (2020) addressed emotions as structured phenomena, and studied the semantic roles of emotions including trigger phrases, experiencers, targets and causes, as well as the reader's perception.
Emotion-Cause Pair Extraction All previous studies on emotion cause analysis need to take known emotion clauses as model input. The pioneer work (Xia and Ding, 2019) first put forward emotion-cause pair extraction task. They proposed a two-step approach to extract emotion and cause clauses separately, and then train a classifier to filter out negative pairs. Unlike their work, our work is a one-step solution for end-to-end emotion-cause pair extraction via effective inter-clause modeling, achieving significantly better performance.
In this paper, we propose the first one-step neural approach RANKCP to tackle the problem of emotion-cause pair extraction, which emphasizes inter-clause modeling from a ranking perspective. Our approach effectively models inter-clause relationships to learn clause representations, and integrates relative position enhanced clause pair ranking into a unified neural network to extract emotion-cause pairs in an end-to-end fashion. Experimental results on the benchmark dataset demonstrate that RANKCP significantly outperforms previous systems, and further analysis verifies the effectiveness of each component in our model.
In future work, we shall explore the following directions. First, current studies on emotion cause analysis mainly focus on clause-level extraction which is relatively coarse-grained, and it is desirable to further design fine-grained methods that can extract span-level or phrase-level emotion expressions and causes. Second, designing effective methods to inject appropriate linguistic knowledge into neural models is valuable to emotion analysis tasks (Ke et al., 2019; Zhong et al., 2019). Finally, it would be interesting to study the semantic roles of emotion (Bostan et al., 2020), which considers the full structure of an emotion expression and is more challenging.
This work was supported in part by the Ministry of Science and Technology of China under Grants #2016QY02D0305 and #2018ZX10201001, and NSFC under Grants #71621002, #61671450 and #11832001. We thank the anonymous reviewers for their valuable comments.
Laura Bostan, Evgeny Kim, and Roman Klinger. 2020. GoodNewsEveryone: A corpus of news headlines annotated with emotions, semantic roles, and reader perception. In LREC.
Ying Chen, Sophia Yat Mei Lee, Shoushan Li, and ChuRen Huang. 2010. Emotion cause detection with linguistic constructions. In COLING.
Xiyao Cheng, Ying Chen, Bixiao Cheng, Shoushan Li, and Guodong Zhou. 2017. An emotion cause corpus for Chinese microblogs with multiple-user structures. ACM Transactions on Asian and Low-Resource Language Information Processing (TALIP), 17(1).
Robert De Beaugrande and Wolfgang U Dressler. 1981. Introduction to text linguistics. Routledge.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In *NAACL-HLT*.
Zixiang Ding, Huihui He, Mengran Zhang, and Rui Xia. 2019. From independent prediction to reordered prediction: Integrating relative position and global label information to emotion cause identification. In AAAI.
Chuang Fan, Hongyu Yan, Jiachen Du, Lin Gui, Li-dong Bing, Min Yang, Ruifeng Xu, and Ruibin Mao. 2019. A knowledge regularized hierarchical approach for emotion cause analysis. In EMNLP-IJCNLP.
Kai Gao, Hua Xu, and Jiushuo Wang. 2015. Emotion cause detection for Chinese micro-blogs based on ECOCC model. In PAKDD.
Diman Ghazi, Diana Inkpen, and Stan Szpakowicz. 2015. Detecting emotion stimuli in emotion-bearing sentences. In CICling.
Lin Gui, Jiannan Hu, Yulan He, Ruifeng Xu, Qin Lu, and Jiachen Du. 2017. A question answering approach for emotion cause extraction. In EMNLP.
Lin Gui, Dongyin Wu, Ruifeng Xu, Qin Lu, and Yu Zhou. 2016. Event-driven emotion cause extraction with corpus construction. In EMNLP.
Jiaxing Hu, Shumin Shi, and Heyan Huang. 2019. Combining external sentiment knowledge for emotion cause detection. In NLPCC.
Pei Ke, Haozhe Ji, Siyang Liu, Xiaoyan Zhu, and Minlie Huang. 2019. SentiLR: Linguistic knowledge enhanced language representation for sentiment analysis. arXiv preprint arXiv:1911.02493.
Evgeny Kim and Roman Klinger. 2018. Who feels what and why? Annotation of a literature corpus with semantic roles of emotions. In COLING.
Terry Koo, Amir Globerson, Xavier Carreras, and Michael Collins. 2007. Structured prediction models via the matrix-tree theorem. In EMNLP-CoNLL.
Sophia Yat Mei Lee, Ying Chen, and Chu-Ren Huang. 2010a. A text-driven rule-based system for emotion cause detection. In the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text.
Sophia Yat Mei Lee, Ying Chen, Shoushan Li, and Chu Ren Huang. 2010b. Emotion cause events: Corpus construction and analysis. In LREC.
Xiangju Li, Kaisong Song, Shi Feng, Daling Wang, and Yifei Zhang. 2018. A co-attention neural network model for emotion cause analysis with emotional context awareness. In EMNLP.
Yang Liu and Mirella Lapata. 2018. Learning structured text representations. Transactions of the Association for Computational Linguistics (TACL), 6.
Yang Liu and Mirella Lapata. 2019. Text summarization with pretrained encoders. In EMNLP-IJCNLP.
Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In ICLR.
William C Mann and Sandra A Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text-interdisciplinary Journal for the Study of Discourse, 8(3).
Daniel Marcu. 2000. The theory and practice of discourse parsing and summarization. MIT press.
Alena Neviarouskaya and Masaki Aono. 2013. Extracting causes of emotions from text. In IJCNLP.
Jiezhong Qiu, Jian Tang, Hao Ma, Yuxiao Dong, Kuansan Wang, and Jie Tang. 2018. DeepInf: Social influence prediction with deep learning. In KDD.
Irene Russo, Tommaso Caselli, Francesco Rubino, Ester Boldrini, and Patricio Martínez-Barco. 2011. EMOCause: An easy-adaptable approach to extract emotion cause contexts. In WASSA@ACL-HLT.
Rupesh K Srivastava, Klaus Greff, and Jürgen Schmidhuber. 2015. Training very deep networks. In NIPS.
W.T. Tutte. 1984. Graph theory. Encyclopedia of mathematics and its applications. Addison-Wesley Pub. Co., Advanced Book Program.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS.
Petar Velicković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. In ICLR.
Shih-Ming Wang and Lun-Wei Ku. 2016. ANTUSD: A large Chinese sentiment dictionary. In LREC.
Rui Xia and Zixiang Ding. 2019. Emotion-cause pair extraction: A new task to emotion analysis in texts. In ACL.
Rui Xia, Mengran Zhang, and Zixiang Ding. 2019. RTHN: A RNN-Transformer hierarchical network for emotion cause extraction. In IJCAI.
Bo Xu, Hongfei Lin, Yuan Lin, Yufeng Diao, Liang Yang, and Kan Xu. 2019. Extracting emotion causes using learning to rank methods from an information retrieval perspective. IEEE Access, 7.
Xinyi Yu, Wenge Rong, Zhuo Zhang, Yuanxin Ouyang, and Zhang Xiong. 2019. Multiple level hierarchical network-based clause selection for emotion cause extraction. IEEE Access, 7.
Peixiang Zhong, Di Wang, and Chunyan Miao. 2019. Knowledge-enriched Transformer for emotion detection in textual conversations. In EMNLP-IJCNLP.
A Experimental Results with BERT
We employ pretrained BERT (Devlin et al., 2019) to replace the original encoder (Hierarchical RNN) in RANKCP, and report the experimental results.
A.1 RANKCP with BERT Encoder
Given a document  $D = (c_{1}, c_{2}, \ldots, c_{|D|})$  where the  $i$ -th clause  $c_{i} = (w_{1}^{i}, w_{2}^{i}, \ldots, w_{|c_{i}|}^{i})$ , to feed  $D$  into pretrained BERT, for each clause we insert a [CLS] token before it and append a [SEP] token to it, obtaining  $c_{i} = ([CLS], w_{1}^{i}, w_{2}^{i}, \ldots, w_{|c_{i}|}^{i}, [SEP])$ . Following (Liu and Lapata, 2019), we use "interval" segment embeddings  $(E_{A}, E_{B}, E_{A}, \ldots)$  to distinguish clauses in a document, i.e.,  $E_{A}$  for clauses at odd positions and  $E_{B}$  for those at even positions. For each token in the document, its input representation is the sum of the corresponding token, segment, and position embeddings. The clause representation of clause  $c_{i}$  is the corresponding [CLS] token's output representation.
We implement our model based on PyTorch and Transformers, and the BERT encoder is initialized using BERT-Base, Chinese. The model is optimized by Eq. 13 for 20 epochs with early stopping, using AdamW optimizer (Loshchilov and Hutter, 2019) and 1e-5 learning rate. We schedule the learning rate that the first  $10\%$  of all training steps is a linear warm-up phrase and then a linear decay phrase is used.
Table 7 shows the results on emotion-cause pair extraction and two sub-tasks. With the pretrained BERT encoder, the results of RANKCP significantly perform better than those with hierarchical RNN, especially on Recall, which indicates the effectiveness of contextualized embeddings as external knowledge, and thus pretrained BERT is a suitable backbone network for emotion-cause pair extraction task.
Table 8 shows the comparative results on extracting one and more than one pair, and we can observe that pretrained BERT encoder further improves the performance of RANKCP for extracting multiple pairs in one document.
$^{6}$ https://github.com/huggingface/ transformers  
 $^{7}$ https://github.com/google-research/bert
Approach	Emotion-Cause Pair Extraction	Emotion Clause Extraction	Cause Clause Extraction
F1	P	R	F1	P	R	F1	P	R
RANKCP	0.6610	0.6698	0.6546	0.8548	0.8703	0.8406	0.6824	0.6927	0.6743
RANKCP w/ BERT	0.7360	0.7119	0.7630	0.9057	0.9123	0.8999	0.7615	0.7461	0.7788
Table 7: Experimental results with pretrained BERT encoder.
# Pairs	Approach	F1	P	R
One per doc.	RANKCP	0.6790	0.6625	0.6966
w/ BERT	0.7633	0.7203	0.8123
Two or more per doc.	RANKCP	0.5531	0.7508	0.4390
w/ BERT	0.5802	0.6772	0.5146
Table 8: Comparative results for documents with only one and more than one emotion-cause pair.
B More Discussions on Modeling Inter-Clause Relationships
B.1 Multi-Root Discourse Tree Induction
In previous experiments, we let RANKCP induce a discourse dependency tree (each discourse unit is a clause) while extracting emotion-cause pairs in a document. We expect that a document can be structurally represented as a multi-root dependency tree, where each root node is an emotion clause, and its child nodes plus the root itself are potential causes. To this end, we extended the original graph attention to a structured graph attention mechanism, inspired by (Koo et al., 2007; Liu and Lapata, 2018). See the next sub-section for details.
However, the structured graph attention does not lead to improvement for RANKCP. The main reason might be that dependencies in a discourse tree cannot handle a common situation well, i.e., an emotion clause and its corresponding cause clause is the same one. We leave the exploration of effective tree induction methods with the help of clause pair representation learning for future work.
B.2 Structured Graph Attention
At the  $t$ -th layer, let  $\{h_1^{(t - 1)}, h_2^{(t - 1)}, \ldots, h_{|D|}^{(t - 1)}\}$  denote the input clause representations. The structured graph attention mechanism operates on each clause  $c_i$  via the following aggregation scheme:
$$
\boldsymbol {p} _ {i} ^ {(t)} = \alpha_ {i} ^ {(t)} \boldsymbol {e} _ {\text {r o o t}} + \sum_ {j \in \mathcal {N} (i)} \alpha_ {j i} ^ {(t)} \boldsymbol {h} _ {j} ^ {(t - 1)},
$$
$$
\boldsymbol {c} _ {i} ^ {(t)} = \sum_ {j \in \mathcal {N} (i)} \alpha_ {i j} ^ {(t)} \boldsymbol {h} _ {j} ^ {(t - 1)}, \tag {15}
$$
$$
\pmb {h} _ {i} ^ {(t)} = \mathrm {R e L U} \left(\pmb {W} _ {g} ^ {(t)} \left[ \pmb {p} _ {i} ^ {(t)}; \pmb {c} _ {i} ^ {(t)}; \pmb {h} _ {i} ^ {(t - 1)} \right] + \pmb {b} _ {g} ^ {(t)}\right),
$$
where  $\pmb{p}_i^{(t)}$  and  $c_{i}^{(t)}$  are the context information aggregated from parent clauses and child clauses respectively.  $\alpha_{ij}^{(t)}$  reflects the marginal probability of a dependency between two clauses  $c_{i}$  and  $c_{j}$ .  $\alpha_{i}^{(t)}$  denotes the probability of  $c_{i}$  being a root, and  $\pmb{e}_{\mathrm{root}}$  is a special root embedding. Specifically, two MLPs compute unnormalized values  $e_{ij}^{(t)}$  and  $e_i^{(t)}$ :
$$
\begin{array}{l} e _ {i j} ^ {(t)} = \boldsymbol {w} _ {d} ^ {(t) ^ {\top}} \tanh  \left(\left[ \boldsymbol {W} ^ {(t)} \boldsymbol {h} _ {i} ^ {(t - 1)}; \boldsymbol {W} ^ {(t)} \boldsymbol {h} _ {j} ^ {(t - 1)} \right]\right), \tag {16} \\ e _ {i} ^ {(t)} = \pmb {w} _ {r} ^ {(t) ^ {\top}} \tanh \left(\pmb {W} ^ {(t)} \pmb {h} _ {i} ^ {(t - 1)}\right), \\ \end{array}
$$
Then, the normalized weights  $\alpha_{ij}^{(t)}$  and  $\alpha_{i}^{(t)}$  can be regarded as constrained attention weights to induce a non-projective discourse dependency tree based on Kirchhoff's Matrix-Tree Theorem (Tutte, 1984; Koo et al., 2007), where  $\mathbf{A}^{(t)}$  and  $\mathbf{L}^{(t)}$  denote adjacency matrix and Laplacian matrix respectively:
$$
\begin{array}{l} [ \boldsymbol {A} ^ {(t)} ] _ {i j} = \left\{ \begin{array}{l l} 0, & \text {i f} i = j  , \\ \exp (\mathsf {L e a k y R e L U} (e _ {i j} ^ {(t)})), & \text {o t h e r w i s e}  . \end{array} \right. \\ r _ {i} ^ {(t)} = \exp \left(e _ {i} ^ {(t)}\right), \tag {17} \\ \end{array}
$$
$$
\left[ \boldsymbol {L} ^ {(t)} \right] _ {i j} = \left\{ \begin{array}{l l} \sum_ {k = 1} ^ {| D |} \left[ \boldsymbol {A} ^ {(t)} \right] _ {k j} & \text {i f} i = j, \\ - \left[ \boldsymbol {A} ^ {(t)} \right] _ {i j}, & \text {o t h e r w i s e .} \end{array} \right. \tag {18}
$$
$$
\hat {\boldsymbol {L}} ^ {(t)} = \boldsymbol {L} ^ {(t)} + \operatorname {d i a g} \left(r _ {1} ^ {(t)}, \dots , r _ {| D |} ^ {(t)}\right) \tag {19}
$$
The normalized weights are:
$$
\begin{array}{l} \alpha_ {i j} ^ {(t)} = (1 - \delta_ {1, j}) [ \boldsymbol {A} ^ {(t)} ] _ {i j} [ \hat {\boldsymbol {L}} ^ {(t) ^ {- 1}} ] _ {j j} \\ - \left(1 - \delta_ {i, 1}\right) \left[ \boldsymbol {A} ^ {(t)} \right] _ {i j} \left[ \hat {\boldsymbol {L}} ^ {(t) ^ {- 1}} \right] _ {j i}, \tag {20} \\ \end{array}
$$
$$
\alpha_ {i} ^ {(t)} = r _ {i} ^ {(t)} [ \hat {\boldsymbol {L}} ^ {(t) ^ {- 1}} ] _ {i 1},
$$